{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "Tce3stUlHN0L" }, "source": [ "##### Copyright 2020 The TensorFlow Authors." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "cellView": "form", "execution": { "iopub.execute_input": "2022-12-14T21:28:21.030408Z", "iopub.status.busy": "2022-12-14T21:28:21.029863Z", "iopub.status.idle": "2022-12-14T21:28:21.033659Z", "shell.execute_reply": "2022-12-14T21:28:21.033045Z" }, "id": "tuOe1ymfHZPu" }, "outputs": [], "source": [ "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ] }, { "cell_type": "markdown", "metadata": { "id": "qFdPvlXBOdUN" }, "source": [ "# 고급 자동 미분" ] }, { "cell_type": "markdown", "metadata": { "id": "MfBg1C5NB3X0" }, "source": [ "\n", " \n", " \n", " \n", " \n", "
TensorFlow.org에서 보기Google Colab에서 실행 GitHub에서 소스 보기노트북 다운로드
" ] }, { "cell_type": "markdown", "metadata": { "id": "8a859404ce7e" }, "source": [ "[그래디언트 및 자동 미분 소개](autodiff.ipynb) 가이드는 TensorFlow에서 그래디언트를 계산하는 데 필요한 모든 내용을 포함하고 있습니다. 이 가이드는 `tf.GradientTape` API의 심층적이고 다소 일반적이지 않은 기능에 중점을 둡니다." ] }, { "cell_type": "markdown", "metadata": { "id": "MUXex9ctTuDB" }, "source": [ "## 설정" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:21.037304Z", "iopub.status.busy": "2022-12-14T21:28:21.036623Z", "iopub.status.idle": "2022-12-14T21:28:23.285707Z", "shell.execute_reply": "2022-12-14T21:28:23.285031Z" }, "id": "IqR2PQG4ZaZ0" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2022-12-14 21:28:21.979357: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory\n", "2022-12-14 21:28:21.979447: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory\n", "2022-12-14 21:28:21.979456: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.\n" ] } ], "source": [ "import tensorflow as tf\n", "\n", "import matplotlib as mpl\n", "import matplotlib.pyplot as plt\n", "\n", "mpl.rcParams['figure.figsize'] = (8, 6)" ] }, { "cell_type": "markdown", "metadata": { "id": "uGRJJRi8TCkJ" }, "source": [ "## 그래디언트 기록 제어하기\n", "\n", "[자동 미분 가이드](autodiff.ipynb)에서는 그래디언트 계산을 빌드하는 동안 테이프에서 감시할 변수 및 텐서를 제어하는 방법을 살펴보았습니다.\n", "\n", "테이프에는 기록을 조작하는 방법도 있습니다." ] }, { "cell_type": "markdown", "metadata": { "id": "gB_i0VnhQKt2" }, "source": [ "### 기록 중지하기\n", "\n", "그래디언트 기록을 중지하려면 `tf.GradientTape.stop_recording`을 사용하여 기록을 일시적으로 중지할 수 있습니다.\n", "\n", "모델 중간에서 복잡한 연산을 구별하지 않으려면, 일시 중단이 오버헤드를 줄이는 데 유용할 수 있습니다. 여기에는 메트릭 또는 중간 결과 계산이 포함될 수 있습니다." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:23.290156Z", "iopub.status.busy": "2022-12-14T21:28:23.289379Z", "iopub.status.idle": "2022-12-14T21:28:26.670905Z", "shell.execute_reply": "2022-12-14T21:28:26.669981Z" }, "id": "mhFSYf7uQWxR" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "dz/dx: tf.Tensor(4.0, shape=(), dtype=float32)\n", "dz/dy: None\n" ] } ], "source": [ "x = tf.Variable(2.0)\n", "y = tf.Variable(3.0)\n", "\n", "with tf.GradientTape() as t:\n", " x_sq = x * x\n", " with t.stop_recording():\n", " y_sq = y * y\n", " z = x_sq + y_sq\n", "\n", "grad = t.gradient(z, {'x': x, 'y': y})\n", "\n", "print('dz/dx:', grad['x']) # 2*x => 4\n", "print('dz/dy:', grad['y'])" ] }, { "cell_type": "markdown", "metadata": { "id": "DEHbEZ1h4p8A" }, "source": [ "### 처음부터 기록 재설정/시작하기\n", "\n", "완전히 다시 시작하려면 `tf.GradientTape.reset`을 사용합니다. 그래디언트 테이프 블록을 종료하고 다시 시작하는 것이 일반적으로 읽기 쉽지만 테이프 블록을 종료하는 것이 어렵거나 불가능한 경우 `reset` 메서드를 사용할 수 있습니다." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:26.675000Z", "iopub.status.busy": "2022-12-14T21:28:26.674283Z", "iopub.status.idle": "2022-12-14T21:28:26.682136Z", "shell.execute_reply": "2022-12-14T21:28:26.681347Z" }, "id": "lsMHsmrh4pqM" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "dz/dx: tf.Tensor(4.0, shape=(), dtype=float32)\n", "dz/dy: None\n" ] } ], "source": [ "x = tf.Variable(2.0)\n", "y = tf.Variable(3.0)\n", "reset = True\n", "\n", "with tf.GradientTape() as t:\n", " y_sq = y * y\n", " if reset:\n", " # Throw out all the tape recorded so far.\n", " t.reset()\n", " z = x * x + y_sq\n", "\n", "grad = t.gradient(z, {'x': x, 'y': y})\n", "\n", "print('dz/dx:', grad['x']) # 2*x => 4\n", "print('dz/dy:', grad['y'])" ] }, { "cell_type": "markdown", "metadata": { "id": "6zS7cLmS6zMf" }, "source": [ "## 정확한 그래디언트 플로 중지하기\n", "\n", "위의 전역 테이프 컨트롤과 달리, `tf.stop_gradient` 함수는 훨씬 더 정확합니다. 테이프 자체에 액세스할 필요 없이 특정 경로를 따라 그래디언트가 흐르는 것을 막는 데 사용할 수 있습니다." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:26.685828Z", "iopub.status.busy": "2022-12-14T21:28:26.685186Z", "iopub.status.idle": "2022-12-14T21:28:26.696121Z", "shell.execute_reply": "2022-12-14T21:28:26.695397Z" }, "id": "30qnZMe48BkB" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "dz/dx: tf.Tensor(4.0, shape=(), dtype=float32)\n", "dz/dy: None\n" ] } ], "source": [ "x = tf.Variable(2.0)\n", "y = tf.Variable(3.0)\n", "\n", "with tf.GradientTape() as t:\n", " y_sq = y**2\n", " z = x**2 + tf.stop_gradient(y_sq)\n", "\n", "grad = t.gradient(z, {'x': x, 'y': y})\n", "\n", "print('dz/dx:', grad['x']) # 2*x => 4\n", "print('dz/dy:', grad['y'])" ] }, { "cell_type": "markdown", "metadata": { "id": "mbb-9lnGVngH" }, "source": [ "## 사용자 정의 그래디언트\n", "\n", "경우에 따라 기본값을 사용하지 않고 그래디언트를 계산하는 방법을 정확하게 제어할 수 있습니다. 이러한 상황에는 다음이 포함됩니다.\n", "\n", "1. 작성 중인 새 op에 대해 정의된 그래디언트가 없습니다.\n", "2. 기본 계산이 수치적으로 불안정합니다.\n", "3. 정방향 패스에서 값비싼 계산을 캐시하려고 합니다.\n", "4. 그래디언트를 수정하지 않고 값(예: `tf.clip_by_value` 또는 `tf.math.round` 사용)을 수정하려고 합니다.\n", "\n", "첫 번째의 경우 새 op를 작성하기 위해 `tf.RegisterGradient`를 사용하여 직접 설정할 수 있습니다. 자세한 내용은 해당 페이지를 참조하세요(그래디언트 레지스트리는 전역이므로 주의해서 변경하세요).\n", "\n", "후자의 세 가지 경우에는 `tf.custom_gradient`를 사용할 수 있습니다." ] }, { "cell_type": "markdown", "metadata": { "id": "oHr31kc_irF_" }, "source": [ "다음은 `tf.clip_by_norm`을 중간 그래디언트에 적용하는 예입니다." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:26.699733Z", "iopub.status.busy": "2022-12-14T21:28:26.699114Z", "iopub.status.idle": "2022-12-14T21:28:26.718798Z", "shell.execute_reply": "2022-12-14T21:28:26.718102Z" }, "id": "Mjj01w4NYtwd" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tf.Tensor(2.0, shape=(), dtype=float32)\n" ] } ], "source": [ "# Establish an identity operation, but clip during the gradient pass.\n", "@tf.custom_gradient\n", "def clip_gradients(y):\n", " def backward(dy):\n", " return tf.clip_by_norm(dy, 0.5)\n", " return y, backward\n", "\n", "v = tf.Variable(2.0)\n", "with tf.GradientTape() as t:\n", " output = clip_gradients(v * v)\n", "print(t.gradient(output, v)) # calls \"backward\", which clips 4 to 2" ] }, { "cell_type": "markdown", "metadata": { "id": "n4t7S0scYrD3" }, "source": [ "자세한 내용은 `tf.custom_gradient` 데코레이터 API 문서를 참조하세요." ] }, { "cell_type": "markdown", "metadata": { "id": "v0ODp4Oi--I0" }, "source": [ "### SavedModel의 사용자 정의 그래디언트\n", "\n", "참고: 이 기능은 TensorFlow 2.6부터 사용할 수 있습니다.\n", "\n", "사용자 정의 그래디언트는 `tf.saved_model.SaveOptions(experimental_custom_gradients=True)` 옵션을 사용하여 SavedModel에 저장할 수 있습니다.\n", "\n", "SavedModel에 저장하려면 그래디언트 함수를 추적할 수 있어야 합니다(자세한 내용은 [tf.function으로 성능 향상하기](function.ipynb) 가이드 참조)." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:26.722250Z", "iopub.status.busy": "2022-12-14T21:28:26.721639Z", "iopub.status.idle": "2022-12-14T21:28:26.726235Z", "shell.execute_reply": "2022-12-14T21:28:26.725468Z" }, "id": "Q5JBgIBYjN1I" }, "outputs": [], "source": [ "class MyModule(tf.Module):\n", "\n", " @tf.function(input_signature=[tf.TensorSpec(None)])\n", " def call_custom_grad(self, x):\n", " return clip_gradients(x)\n", "\n", "model = MyModule()" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:26.729416Z", "iopub.status.busy": "2022-12-14T21:28:26.728849Z", "iopub.status.idle": "2022-12-14T21:28:26.922598Z", "shell.execute_reply": "2022-12-14T21:28:26.921668Z" }, "id": "xZTrgy2q-9pq" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Assets written to: saved_model/assets\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "tf.Tensor(2.0, shape=(), dtype=float32)\n" ] } ], "source": [ "tf.saved_model.save(\n", " model,\n", " 'saved_model',\n", " options=tf.saved_model.SaveOptions(experimental_custom_gradients=True))\n", "\n", "# The loaded gradients will be the same as the above example.\n", "v = tf.Variable(2.0)\n", "loaded = tf.saved_model.load('saved_model')\n", "with tf.GradientTape() as t:\n", " output = loaded.call_custom_grad(v * v)\n", "print(t.gradient(output, v))" ] }, { "cell_type": "markdown", "metadata": { "id": "d-LfRs5FbJCk" }, "source": [ "위의 예제에 대한 참고 사항: 위의 코드를 `tf.saved_model.SaveOptions(experimental_custom_gradients=False)`로 바꾸려고 하여도 그래디언트가 로드할 때 여전히 동일한 결과를 생성합니다. 그 이유는 그래디언트 레지스트리에 `call_custom_op` 함수에서 사용한 사용자 정의 그래디언트가 여전히 포함되어 있기 때문입니다. 그러나 사용자 정의 그래디언트 없이 저장한 후 런타임을 다시 시작하면 `tf.GradientTape`에서 로드된 모델을 실행하면 다음 오류(`LookupError: No gradient defined for operation 'IdentityN' (op type: IdentityN)`).가 발생합니다." ] }, { "cell_type": "markdown", "metadata": { "id": "8aENEt6Veryb" }, "source": [ "## 여러 테이프\n", "\n", "여러 테이프가 원활하게 상호 작용합니다.\n", "\n", "예를 들어, 각 테이프는 서로 다른 텐서 세트를 감시합니다." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:26.926522Z", "iopub.status.busy": "2022-12-14T21:28:26.925900Z", "iopub.status.idle": "2022-12-14T21:28:26.933744Z", "shell.execute_reply": "2022-12-14T21:28:26.933112Z" }, "id": "BJ0HdMvte0VZ" }, "outputs": [], "source": [ "x0 = tf.constant(0.0)\n", "x1 = tf.constant(0.0)\n", "\n", "with tf.GradientTape() as tape0, tf.GradientTape() as tape1:\n", " tape0.watch(x0)\n", " tape1.watch(x1)\n", "\n", " y0 = tf.math.sin(x0)\n", " y1 = tf.nn.sigmoid(x1)\n", "\n", " y = y0 + y1\n", "\n", " ys = tf.reduce_sum(y)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:26.937192Z", "iopub.status.busy": "2022-12-14T21:28:26.936619Z", "iopub.status.idle": "2022-12-14T21:28:26.946768Z", "shell.execute_reply": "2022-12-14T21:28:26.946007Z" }, "id": "6ApAoMNFfNz6" }, "outputs": [ { "data": { "text/plain": [ "1.0" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tape0.gradient(ys, x0).numpy() # cos(x) => 1.0" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:26.950080Z", "iopub.status.busy": "2022-12-14T21:28:26.949514Z", "iopub.status.idle": "2022-12-14T21:28:26.955627Z", "shell.execute_reply": "2022-12-14T21:28:26.954929Z" }, "id": "rF1jrAJsfYW_" }, "outputs": [ { "data": { "text/plain": [ "0.25" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tape1.gradient(ys, x1).numpy() # sigmoid(x1)*(1-sigmoid(x1)) => 0.25" ] }, { "cell_type": "markdown", "metadata": { "id": "DK05KXrAAld3" }, "source": [ "### 고계도 그래디언트\n", "\n", "`tf.GradientTape` 컨텍스트 관리자 내부의 연산은 자동 미분을 위해 기록됩니다. 해당 컨텍스트에서 그래디언트가 계산되면, 그래디언트 계산도 기록됩니다. 결과적으로, 정확히 같은 API가 고계도 그래디언트에도 작동합니다.\n", "\n", "예를 들어 다음과 같습니다." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:26.959126Z", "iopub.status.busy": "2022-12-14T21:28:26.958561Z", "iopub.status.idle": "2022-12-14T21:28:26.967402Z", "shell.execute_reply": "2022-12-14T21:28:26.966602Z" }, "id": "cPQgthZ7ugRJ" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "dy_dx: 3.0\n", "d2y_dx2: 6.0\n" ] } ], "source": [ "x = tf.Variable(1.0) # Create a Tensorflow variable initialized to 1.0\n", "\n", "with tf.GradientTape() as t2:\n", " with tf.GradientTape() as t1:\n", " y = x * x * x\n", "\n", " # Compute the gradient inside the outer `t2` context manager\n", " # which means the gradient computation is differentiable as well.\n", " dy_dx = t1.gradient(y, x)\n", "d2y_dx2 = t2.gradient(dy_dx, x)\n", "\n", "print('dy_dx:', dy_dx.numpy()) # 3 * x**2 => 3.0\n", "print('d2y_dx2:', d2y_dx2.numpy()) # 6 * x => 6.0" ] }, { "cell_type": "markdown", "metadata": { "id": "k0HV-Ah4_76i" }, "source": [ "이것은 *스칼라* 함수의 2차 미분을 제공하지만 `tf.GradientTape.gradient`는 스칼라의 그래디언트만 계산하므로 이 패턴은 Hessian 행렬을 생성하도록 일반화되지 않습니다. [Hessian 행렬](https://en.wikipedia.org/wiki/Hessian_matrix)을 구성하려면 [Jacobian 섹션](#jacobians)의 [Hessian 예제](#hessian)로 이동하세요.\n", "\n", "그래디언트에서 스칼라를 계산할 때 \"`tf.GradientTape.gradient`에 대한 중첩된 호출\"은 좋은 패턴이며, 결과 스칼라는 다음 예제와 같이 두 번째 그래디언트 계산의 소스로 작동합니다.\n" ] }, { "cell_type": "markdown", "metadata": { "id": "t7LRlcpVKHv1" }, "source": [ "#### 예: 입력 그래디언트 정규화\n", "\n", "많은 모델이 \"적대적인 예\"에 취약합니다. 이 기술 컬렉션은 모델의 출력을 혼동하기 위해 모델의 입력을 수정합니다. [FGSM을 이용한 적대적 샘플 생성](https://www.tensorflow.org/tutorials/generative/adversarial_fgsm)과 같은 가장 간단한 구현은 \"입력 그래디언트\"와 같은 입력에 대한 출력의 그래디언트를 따라 단일 단계를 수행합니다.\n", "\n", "적대적인 예에 ​​대한 견고성을 높이는 한 가지 기술은 [입력 그래디언트 정규화](https://arxiv.org/abs/1905.11468)(Finlay & Oberman, 2019)로 이는 입력 그래디언트의 크기를 최소화하려고 시도합니다. 입력 그래디언트가 작으면 출력의 변화도 작아야 합니다.\n", "\n", "아래는 입력 그래디언트 정규화의 네이티브 구현입니다. 구현은 다음과 같습니다.\n", "\n", "1. 내부 테이프를 사용하여 입력에 대한 출력 그래디언트를 계산합니다.\n", "2. 해당 입력 그래디언트의 크기를 계산합니다.\n", "3. 모델에 대한 해당 크기의 그래디언트를 계산합니다." ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:26.970874Z", "iopub.status.busy": "2022-12-14T21:28:26.970323Z", "iopub.status.idle": "2022-12-14T21:28:26.978570Z", "shell.execute_reply": "2022-12-14T21:28:26.978001Z" }, "id": "tH3ZFuUfDLrR" }, "outputs": [], "source": [ "x = tf.random.normal([7, 5])\n", "\n", "layer = tf.keras.layers.Dense(10, activation=tf.nn.relu)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:26.981794Z", "iopub.status.busy": "2022-12-14T21:28:26.981233Z", "iopub.status.idle": "2022-12-14T21:28:27.325736Z", "shell.execute_reply": "2022-12-14T21:28:27.325009Z" }, "id": "E6yOFsjEDR9u" }, "outputs": [], "source": [ "with tf.GradientTape() as t2:\n", " # The inner tape only takes the gradient with respect to the input,\n", " # not the variables.\n", " with tf.GradientTape(watch_accessed_variables=False) as t1:\n", " t1.watch(x)\n", " y = layer(x)\n", " out = tf.reduce_sum(layer(x)**2)\n", " # 1. Calculate the input gradient.\n", " g1 = t1.gradient(out, x)\n", " # 2. Calculate the magnitude of the input gradient.\n", " g1_mag = tf.norm(g1)\n", "\n", "# 3. Calculate the gradient of the magnitude with respect to the model.\n", "dg1_mag = t2.gradient(g1_mag, layer.trainable_variables)" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:27.330016Z", "iopub.status.busy": "2022-12-14T21:28:27.329385Z", "iopub.status.idle": "2022-12-14T21:28:27.334348Z", "shell.execute_reply": "2022-12-14T21:28:27.333521Z" }, "id": "123QMq6PqK_d" }, "outputs": [ { "data": { "text/plain": [ "[TensorShape([5, 10]), TensorShape([10])]" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "[var.shape for var in dg1_mag]" ] }, { "cell_type": "markdown", "metadata": { "id": "E4xiYigexMtQ" }, "source": [ "## 야고비안\n" ] }, { "cell_type": "markdown", "metadata": { "id": "4-hVHVIeExkI" }, "source": [ "이전의 모든 예제는 일부 소스 텐서와 관련하여 스칼라 대상의 그래디언트를 나타냅니다.\n", "\n", "[야고비 행렬식](https://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant)은 벡터값 함수의 그래디언트를 나타냅니다. 각 행에는 벡터 요소 중 하나의 그래디언트가 포함됩니다.\n", "\n", "`tf.GradientTape.jacobian` 메서드를 사용하면 야고비 행렬식을 효율적으로 계산할 수 있습니다." ] }, { "cell_type": "markdown", "metadata": { "id": "KzNyIM0QBYIH" }, "source": [ "참고:\n", "\n", "- `gradient`처럼: `sources` 인수는 텐서 또는 텐서의 컨테이너가 될 수 있습니다.\n", "- `gradient`와 달리: `target` 텐서는 단일 텐서여야 합니다." ] }, { "cell_type": "markdown", "metadata": { "id": "O74K3hlxBC8a" }, "source": [ "### 스칼라 소스" ] }, { "cell_type": "markdown", "metadata": { "id": "B08OKn1Orkuc" }, "source": [ "첫 번째 예는, 스칼라 소스에 대한 벡터 대상의 야코비안입니다." ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:27.337958Z", "iopub.status.busy": "2022-12-14T21:28:27.337740Z", "iopub.status.idle": "2022-12-14T21:28:27.494008Z", "shell.execute_reply": "2022-12-14T21:28:27.493326Z" }, "id": "bAFeIE8EuVIq" }, "outputs": [], "source": [ "x = tf.linspace(-10.0, 10.0, 200+1)\n", "delta = tf.Variable(0.0)\n", "\n", "with tf.GradientTape() as tape:\n", " y = tf.nn.sigmoid(x+delta)\n", "\n", "dy_dx = tape.jacobian(y, delta)" ] }, { "cell_type": "markdown", "metadata": { "id": "BgHbUk3zr-WU" }, "source": [ "스칼라에 대한 야고비안을 취하면, 결과는 **대상**의 형상을 가지며 소스에 대한 각 요소의 그래디언트를 제공합니다." ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:27.497926Z", "iopub.status.busy": "2022-12-14T21:28:27.497325Z", "iopub.status.idle": "2022-12-14T21:28:27.501109Z", "shell.execute_reply": "2022-12-14T21:28:27.500354Z" }, "id": "iZ6awnDzr_BA" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(201,)\n", "(201,)\n" ] } ], "source": [ "print(y.shape)\n", "print(dy_dx.shape)" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:27.504919Z", "iopub.status.busy": "2022-12-14T21:28:27.504284Z", "iopub.status.idle": "2022-12-14T21:28:27.677156Z", "shell.execute_reply": "2022-12-14T21:28:27.676358Z" }, "id": "siNZaklc0_-e" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAp8AAAINCAYAAAB4RhRAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABdXElEQVR4nO3dd3hUZf7+8XtmMpNegJAChN6lSonBgiWK6KKsZbH8BMEua1lWV7Gxul/FtsquDXUVdV0V3RUbCGIUFEFQigLSCS2QkFDSk0lmzu+PSQKBBBJI5kx5v65rrpk55zlnPjOOw53znPM8FsMwDAEAAABeYDW7AAAAAAQPwicAAAC8hvAJAAAAryF8AgAAwGsInwAAAPAawicAAAC8hvAJAAAAryF8AgAAwGtCzC6gIdxut3bv3q3o6GhZLBazywEAAMARDMNQYWGh2rRpI6u1/uObfhE+d+/erZSUFLPLAAAAwHHs3LlT7dq1q3e9X4TP6OhoSZ43ExMTY3I1AAAAOFJBQYFSUlJqclt9/CJ8Vne1x8TEED4BAAB82PFOkeSCIwAAAHgN4RMAAABeQ/gEAACA1/jFOZ8N4XK5VFFRYXYZAcdmsykkJIQhrgAAQJMIiPBZVFSkXbt2yTAMs0sJSBEREUpOTpbD4TC7FAAA4Of8Pny6XC7t2rVLERERat26NUfompBhGHI6ncrNzVVmZqa6det2zEFjAQAAjsfvw2dFRYUMw1Dr1q0VHh5udjkBJzw8XHa7Xdu3b5fT6VRYWJjZJQEAAD8WMIexOOLZfDjaCQAAmgqpAgAAAF5D+AQAAIDXED4BAADgNYRPAAAAeA3h0yTvvPOOWrVqpfLy8lrLR48ereuuu86kqgAAAJpXo8Pnd999p1GjRqlNmzayWCz65JNPjrvNggULdOqppyo0NFRdu3bVW2+9dQKlNoxhGCpxVppya8wg91deeaVcLpc+++yzmmV79+7V7NmzNWHChOb4aAAAAEzX6HE+i4uL1b9/f02YMEGXXXbZcdtnZmbq4osv1q233qr//Oc/ysjI0I033qjk5GSNGDHihIo+ltIKl3o/Mq/J99sQvz02QhGOhn2k4eHhuuaaazRjxgxdeeWVkqR3331X7du319lnn92MVQIAAJin0eFz5MiRGjlyZIPbT58+XZ06ddLf//53SVKvXr20aNEiPf/8880SPv3JTTfdpCFDhigrK0tt27bVW2+9peuvv54xSwEAQMBq9hmOlixZovT09FrLRowYobvvvrtZXi/cbtNvj5kTasPttka1HzhwoPr376933nlHF1xwgdauXavZs2c3U3UAAPgXwzDkchtyVd1Xug0Zhme5YUhuw5DbkAxVL/csMyS53Z5T4dxHtJWqtjFqr6tu667af11tPTUdVp+M6geH39XZxjiiTfX7q7XsWNvVWlf39rXP/vM8SWkZoVPaxMqXNHv4zM7OVmJiYq1liYmJKigoUGlpaZ1TYpaXl9e6EKegoKDBr2exWBrc9e0LbrzxRk2bNk1ZWVlKT09XSkqK2SUBAIKEYRgqr3SrrMKlEqdLpRUulR52X+J0qazC87y8wiWny60Kl2cbZ9WtwlX1+Mj7qnUVbkPuquDocrs9YbLmuVHrufvw5VWBEyfnutM66G+jgyx8noipU6fq0UcfNbsMr7jmmmt0zz336PXXX9c777xjdjkAAD9T6nQpr6hc+4udOlDiVEFZpQpKK1RYVqmCsgoVllWooLTSc3/YusKyCpVWuBQI+c5q8Rx8slokiyyyWCSLRbJaLLKo6v7wNlX3UvVzTxtr1WlvVqtnP9VtLRbJokNTeR9+clz1mXKWqqV1nTl35HaHtznW9pYjGtX9unW/RnWbDq0iji7IZM0ePpOSkpSTk1NrWU5OjmJiYuo86ilJkydP1qRJk2qeFxQUBOwRwdjYWF1++eWaPXu2Ro8ebXY5AAAfYBiGCsoqtSe/VLsPlmr3wTLtyS9VXqFT+4rLlVfkud9X5FSJ09Ukr2m3WRRutyncYVOEI0RhdpsiHDaF220Ks9sUarfKYau6hVhlr7p3hFjlsFmq7q1yhNhkP+x5iM2qEKtFtsNuRz+3HrW8rjZWa92BEv6l2cNnWlqa5syZU2vZ/PnzlZaWVu82oaGhCg0Nbe7SfEZWVpauvfbaoHrPABDsyipc2r6vRJl5RcrMK9GO/SXafbC0KnCWqai8ssH7coRYFR/pUFyEQzHhIYoJsys6zK6Y8BDPfZhn2eHrosJCFOmwKawqYNptDP0N72h0+CwqKtLmzZtrnmdmZmrVqlVq2bKl2rdvr8mTJysrK6umC/nWW2/Viy++qL/85S+aMGGCvvnmG3344YdcWCPpwIEDWrBggRYsWKCXX37Z7HIAAM2grMKlTTlFWpddoPV7CrUxp1CZecXanV+q4w0P3SLCruTYcLWJC1ebuDC1jgpVq6hQtYpyKD7KoVaRnsdRoSEcAYTfaHT4/Pnnn3XOOefUPK/uHh83bpzeeust7dmzRzt27KhZ36lTJ82ePVt/+tOf9I9//EPt2rXTv/71r6AfZknyXO1+4MABPfXUU+rRo4fZ5QAATlJ5pUtrdxdo5Y6DWrXzoH7bna/MvOJ6z6uMDgtR5/hIdYqPVPtWkWobF1YVNMOVHBvmVxfQAg1lMRozLY9JCgoKFBsbq/z8fMXExNRaV1ZWpszMTHXq1ElhYWEmVRjY+IwBoG4Hip1asnWflmXu18qqsFnhOvqf1RYRdvVKjlGPpGj1TIpWl9ZR6hQfqZaRDo5YImAcK68djj+pAABooFKnSz9t268ftuTph815Wru74Kiu81aRDg1sH6eB7VuoT9tY9UqKVuvoUEImUIXwCQDAMewvdurrdTmatyZb32/Ok7PSXWt998QoDesSr4Ht43Rq+xZq1yKcoAkcA+ETAIAj5BSUad7abM1dk62lmftrDXaeHBum07vG64yu8RrWpZUSYjgdCWgMwicAAJKclW59sz5HM3/aqYUbc2tdJNQ7OUYjTknSiD6J6pEYzZFN4CQQPgEAQW3z3kLN/GmnPl6RpX3Fzprlp7aP08g+yRpxSpLa++AsMYC/InwCAIKOYRj6flOepi/cosVb9tUsbx0dqisGtdOVg9qpc+soEysEAhfh08ecffbZGjBggKZNm9Zk+7RYLJo1axbTdwIIepUut75ck63pC7do7e4CSZLNatG5PRM0ZnCKzu7RWiHM9AM0K8JnAHj00Ue1adMmvfvuu2aXAgA+yVnp1oc/79Rr323Vjv0lkqRwu01XDU3RjWd2Vtu4cJMrBIIH4TMAfPrpp7r//vvNLgMAfI7bbejzX3fr719trAmdLSLsGjeso8aldVSLSIfJFQLBh74FExUXF2vs2LGKiopScnKy/v73v9ese+yxx9SnT5+jthkwYIAefvjhmuc7d+7U2rVrdeGFF0qSNm3apLPOOkthYWHq3bu35s+fX2v7d955R1FRUdq0aVPNsttvv109e/ZUSUlJU79FADDNT9v2a9SLi3TXB6u0Y3+J4qNC9cjveuuH+8/V3endCZ6ASQLvyKdhSBUmhSh7hNSI4TfuvfdeLVy4UJ9++qkSEhL0wAMPaMWKFRowYIAmTJigRx99VD/99JOGDBkiSVq5cqV+/fVXffzxxzX7+Oyzz3T22WcrJiZGbrdbl112mRITE7V06VLl5+fr7rvvrvWaY8eO1RdffKFrr71Wixcv1rx58/Svf/1LS5YsUUQEV3MC8H/Z+WWa+uU6fbpqtyQpOjREtwzvrAlndGKudMAHBN7/hRUl0hNtzHntB3ZLjsgGNS0qKtIbb7yhd999V+edd54k6e2331a7du0kSe3atdOIESM0Y8aMmvA5Y8YMDR8+XJ07d67Zz6effqpLL71UkvT1119r/fr1mjdvntq08XwGTzzxhEaOHFnrtV999VX169dPd955pz7++GP99a9/1aBBg07uvQOAydxuQ/9Zul1Pzd2govJKWSzSVUPa654LuqtVVKjZ5QGoQre7SbZs2SKn06nU1NSaZS1btlSPHj1qnt900016//33VVZWJqfTqffee08TJkyoWV9QUKCFCxfqkksukSStW7dOKSkpNcFTktLS0o567RYtWuiNN97QK6+8oi5dunC+KAC/tyW3SH94dYke/nStisorNbB9nD7/4xmaellfgifgYwLvyKc9wnME0qzXbkKjRo1SaGioZs2aJYfDoYqKCl1xxRU167/88kv17t1bKSkpjd73d999J5vNpj179qi4uFjR0dFNWToAeIVhGHp36Q49Pvs3lVW4FeGw6S8jeui6tI6yWZmFCPBFgRc+LZYGd32bqUuXLrLb7Vq6dKnat28vSTpw4IA2btyo4cOHS5JCQkI0btw4zZgxQw6HQ1dddZXCww8NB3J4l7sk9erVSzt37tSePXuUnJwsSfrxxx+Peu3Fixfrqaee0ueff6777rtPf/zjH/X2228359sFgCaXV1Suv/z3V32zfq8k6Yyu8Xry8r5q14Lz1wFfFnjh009ERUXphhtu0L333qtWrVopISFBDz74oKzW2mdC3HjjjerVq5ck6YcffqhZXllZqS+//FL33HNPzbL09HR1795d48aN0zPPPKOCggI9+OCDtfZXWFio6667TnfeeadGjhypdu3aaciQIRo1alSto6oA4Mt+3rZfE99boZyCcjlCrLrvwp4aP6yjrBztBHwe4dNEzzzzjIqKijRq1ChFR0frz3/+s/Lz82u16datm4YNG6b9+/fXOj904cKFioqK0qmnnlqzzGq1atasWbrhhhs0dOhQdezYUf/85z9rhmGSpLvuukuRkZF64oknJEl9+/bVE088oVtuuUVpaWlq27ZtM79rADhxhmHozR+2aeqcdap0G+rSOlIvXXuqeibFmF0agAayGIZhmF3E8RQUFCg2Nlb5+fmKian9A1NWVqbMzEx16tRJYWFhJlXYfAzDULdu3XT77bdr0qRJNcvvvPNOVVZW6uWXX272GgL9MwbgH8orXZr8v9X6eGWWJOl3/ZL15OX9FBXKcRTAFxwrrx2O/2N9WG5urj744ANlZ2dr/Pjxtdb16dOnzivZASAQ7Ssq1y3/Xq6ftx+QzWrRQxf30vXDOsrSiLGVAfgGwqcPS0hIUHx8vF577TW1aNGi1rqbb77ZpKoAwLu25hZp3Ixl2rm/VNFhIXr52lN1ZrfWZpcF4AQRPn2YH5wRAQDNak1Wvsa9uUz7ip1q3zJCb14/WF0TGBoO8GeETwCAT1qyZZ9ueudnFZVXqk/bGL01fqjiGTAe8HuETwCAz/luY65ueudnlVe6dVrnlnp97GBFh9nNLgtAEwiY8EkXdfPhswXgTd9vOhQ803sl6MVrTlWY3WZ2WQCaiN/P7W6zeX6QnE6nyZUErpKSEkmS3c5RBwDNa9GmPN34dnXwTNTL1w4ieAIBxu+PfIaEhCgiIkK5ubmy2+1HzRCEE2cYhkpKSrR3717FxcXVBH0AaA7Ltx847Ihnol6+9lQ5QvhNBwKN34dPi8Wi5ORkZWZmavv27WaXE5Di4uKUlJRkdhkAAtjGnEJNeOsnlVa4NLx7a4InEMD8PnxKksPhULdu3eh6bwZ2u50jngCa1a4DJRr7xjLll1bo1PZxeuX/ETyBQBYQ4VPyzGvO1I8A4F8Kyio0fsZPyi4oU/fEKL15/RBFOALmnyYAdeBPSwCAKSpdbv3xvZXatLdIiTGhenvCUMVFOMwuC0AzI3wCAEzxty9+03cbcxVut+mNcUOUHBtudkkAvIDwCQDwund/3K63l2yXxSJNu2qA+rSNNbskAF5C+AQAeNXKHQf06OdrJUn3juihEacwmgYQTAifAACv2VdUrtv/s0IVLkMj+yTptuFdzC4JgJcRPgEAXuFyG7rzg5Xak1+mzvGRevqKfrJYLGaXBcDLCJ8AAK948ZvN+mHzPoXbbZp+3SBFhzFlLxCMCJ8AgGa3fPt+/fObTZKkx3/fR90To02uCIBZCJ8AgGZVUFahuz5YJZfb0OgBbXTZqe3MLgmAiQifAIBm9cgna7TrQKnatQjXY6P7mF0OAJMRPgEAzWb2r3v0yardslkt+sdVAxXDeZ5A0CN8AgCaxb6icj3y6RpJ0u1nd9GgDi1MrgiALyB8AgCaxZTP1mpfsVM9k6J1x7ndzC4HgI8gfAIAmtyXq/foi1/3yGa16Jkr+ssRwj83ADz4NQAANKn8kgo9XNXdfuvwzurbjnnbARxC+AQANKmn561XXpFTXVpH6s7z6G4HUBvhEwDQZFbuOKD3lu2QJP3f6L4KDbGZXBEAX0P4BAA0iUqXWw/OWiPDkC4b2FZpXVqZXRIAH0T4BAA0iXeWbNdvewoUExaiBy7uZXY5AHwU4RMAcNL2FZXr+fkbJUn3jeyp+KhQkysC4KsInwCAk/bc/I0qLK/UKW1idNWQ9maXA8CHET4BACdlfXaB3q+6yOiR3/WWzWoxuSIAvozwCQA4YYZh6G9f/Ca3IV3UN0mpnbnICMCxET4BACcsY91e/bB5nxw2qyaP5CIjAMdH+AQAnJBKl1tPzl0vSZpwRieltIwwuSIA/oDwCQA4IR+vzNLmvUWKi7Dr9nO6mF0OAD9B+AQANFpZhUvTqoZWuv3sLooJs5tcEQB/QfgEADTauz9u1+78MiXHhmlsWkezywHgRwifAIBGKSir0EvfbpYk3Z3eTWF25m8H0HCETwBAo8xYtE0HSirUuXWkLj+1ndnlAPAzhE8AQIMVlFXojUVbJUl/Su+uEBv/jABoHH41AAAN9vYP21RQVqmuCVG6qG+y2eUA8EOETwBAgxSWVehfizIlSXec25VpNAGcEMInAKBB3lmyXfmlFerSOlK/69fG7HIA+CnCJwDguIrKK/X6955zPe84txtHPQGcMMInAOC43l+6QwdLKtQ5PlKj+nPUE8CJI3wCAI7JWenWG1Xnet4yvDNHPQGcFMInAOCYPl2VpeyCMiXGhGr0wLZmlwPAzxE+AQD1crsNvfqd51zPCad3UmgIsxkBODmETwBAvTLW79XmvUWKDg3RNantzS4HQAAgfAIA6jV94RZJ0rWndVB0mN3kagAEAsInAKBOK3Yc0PLtB+SwWTXh9I5mlwMgQBA+AQB1mvHDNknSJQPaKCEmzNxiAAQMwicA4Ch78ks1Z/UeSdJ4jnoCaEInFD5feukldezYUWFhYUpNTdWyZcuO2X7atGnq0aOHwsPDlZKSoj/96U8qKys7oYIBAM3v30u2y+U2lNqppU5pE2t2OQACSKPD58yZMzVp0iRNmTJFK1asUP/+/TVixAjt3bu3zvbvvfee7r//fk2ZMkXr1q3TG2+8oZkzZ+qBBx446eIBAE2v1OnSe8t2SJImnNHJ5GoABJpGh8/nnntON910k8aPH6/evXtr+vTpioiI0Jtvvlln+8WLF+v000/XNddco44dO+qCCy7Q1VdffdyjpQAAc8xamaWDJRVKaRmu9F6JZpcDIMA0Knw6nU4tX75c6enph3ZgtSo9PV1Lliypc5thw4Zp+fLlNWFz69atmjNnji666KKTKBsA0BwMw9Bbiz1TaY5L68hUmgCaXEhjGufl5cnlcikxsfZfwomJiVq/fn2d21xzzTXKy8vTGWecIcMwVFlZqVtvvfWY3e7l5eUqLy+veV5QUNCYMgEAJ2hZ5n5tzClSuN2mPwxJMbscAAGo2a92X7BggZ544gm9/PLLWrFihT7++GPNnj1bf/vb3+rdZurUqYqNja25paTwAwgA3vDuUs+5nqMHtlEMg8oDaAaNOvIZHx8vm82mnJycWstzcnKUlJRU5zYPP/ywrrvuOt14442SpL59+6q4uFg333yzHnzwQVmtR+ffyZMna9KkSTXPCwoKCKAA0MxyC8s1d41neKVrUzuYXA2AQNWoI58Oh0ODBg1SRkZGzTK3262MjAylpaXVuU1JSclRAdNms0nynFtUl9DQUMXExNS6AQCa14c/71SFy9CAlDj1acvwSgCaR6OOfErSpEmTNG7cOA0ePFhDhw7VtGnTVFxcrPHjx0uSxo4dq7Zt22rq1KmSpFGjRum5557TwIEDlZqaqs2bN+vhhx/WqFGjakIoAMBcLreh96q63P/faRz1BNB8Gh0+x4wZo9zcXD3yyCPKzs7WgAEDNHfu3JqLkHbs2FHrSOdDDz0ki8Wihx56SFlZWWrdurVGjRqlxx9/vOneBQDgpCzYsFdZB0sVG27X7/olm10OgABmMerr+/YhBQUFio2NVX5+Pl3wANAMbnjrJ2Ws36ubzuykBy/ubXY5APxQQ/Mac7sDQJDLzi/Ttxs8s9RdNbS9ydUACHSETwAIcv9bsUtuQxrSsYW6tI4yuxwAAY7wCQBBzO029OHPOyVJfxjMkHYAmh/hEwCC2NLM/dq+r0RRoSG6mAuNAHgB4RMAglj1Uc9R/ZMV4Wj0ACgA0GiETwAIUvmlFZqz2jOj0ZghXGgEwDsInwAQpD77ZbfKK93qkRit/u2Y0QiAdxA+ASBI/W/5LknSlYPbyWKxmFwNgGBB+ASAILQlt0irdh6UzWrRJQPamF0OgCBC+ASAIDRrRZYk6axu8UqIDjO5GgDBhPAJAEHG7TY0a6UnfF52ajuTqwEQbAifABBklmbuV9bBUkWHhej83olmlwMgyBA+ASDIfLzCc6HR7/olK8xuM7kaAMGG8AkAQaTU6aoZ25MudwBmIHwCQBD56rdsFTtdSmkZrsEdWphdDoAgRPgEgCDy+S+7JUmjB7RlbE8ApiB8AkCQOFji1MKNuZKkSxnbE4BJCJ8AECS+XJOtCpehXskx6poQbXY5AIIU4RMAgsSnqzxje17Sn6OeAMxD+ASAIJCdX6almfslSaP6J5tcDYBgRvgEgCDwxa+7ZRjS4A4t1K5FhNnlAAhihE8ACAKfVV3lzoVGAMxG+ASAALctr1i/7sqXzWrRRX3pcgdgLsInAAS4L9dkS5LSOrdSq6hQk6sBEOwInwAQ4L5c45lOc2TfJJMrAQDCJwAEtF0HSvTrrnxZLdIFvQmfAMxH+ASAADa3qst9aKeWah1NlzsA8xE+ASCAzVld1eXehwuNAPgGwicABKjs/DKt2HFQknRhH7rcAfgGwicABKi5VRcaDerQQokxYSZXAwAehE8ACFDVQyyN5KgnAB9C+ASAAJRbWK5l2zxzuY9kYHkAPoTwCQABaN7abBmG1L9drNrGhZtdDgDUIHwCQACqHmKJo54AfA3hEwACzP5ip5Zs3SeJ8z0B+B7CJwAEmPm/ZcvlNtQ7OUYdWkWaXQ4A1EL4BIAAU32V+0XM5Q7ABxE+ASCA5JdU6IfNeZKkC5nVCIAPInwCQAD5el2OKlyGuidGqWtClNnlAMBRCJ8AEEAODSzPUU8AvonwCQABotTp0qLNuZKkEadwvicA30T4BIAA8cPmPJVVuNU2Lly9kqPNLgcA6kT4BIAAkbE+R5KU3itBFovF5GoAoG6ETwAIAG63oa/X7ZUkndcr0eRqAKB+hE8ACACrs/KVW1iuqNAQpXZuaXY5AFAvwicABICv13m63M/qHq/QEJvJ1QBA/QifABAAqrvc0+lyB+DjCJ8A4Od2HSjRuj0Fslqkc3okmF0OABwT4RMA/Nw36z1HPQd3aKkWkQ6TqwGAYyN8AoCfm/+b53zP83px1BOA7yN8AoAfKyyr0I9b90mS0ntzvicA30f4BAA/9v2mPFW4DHWKj1SX1lFmlwMAx0X4BAA/Vj3EUjpd7gD8BOETAPyUy23o2/UMsQTAvxA+AcBPrdhxQAdKKhQbbtegDi3MLgcAGoTwCQB+6uuqq9zP7ZmgEBs/5wD8A79WAOCnqs/3ZIglAP6E8AkAfigzr1hbcotlt1l0VvfWZpcDAA1G+AQAP5RRddQztVMrxYTZTa4GABqO8AkAfmjBhlxJ0jk96XIH4F8InwDgZ4rLK7Usc78k6ewedLkD8C+ETwDwM0u27JPT5VZKy3B1jo80uxwAaBTCJwD4mQUbPQPLn909QRaLxeRqAKBxCJ8A4EcMw6g535MudwD+iPAJAH5kS26xdh0olcNmVVqXVmaXAwCNRvgEAD+yYIOnyz21c0tFOEJMrgYAGo/wCQB+pLrLfTgDywPwU4RPAPATtYdYYnxPAP6J8AkAfqJ6iKV2LcLVpTVDLAHwT4RPAPAT1UMsndODIZYA+C/CJwD4AYZYAhAoCJ8A4AcYYglAoCB8AoAfYIglAIHihMLnSy+9pI4dOyosLEypqalatmzZMdsfPHhQEydOVHJyskJDQ9W9e3fNmTPnhAoGgGDEEEsAAkWj/3yeOXOmJk2apOnTpys1NVXTpk3TiBEjtGHDBiUkHD30h9Pp1Pnnn6+EhAT997//Vdu2bbV9+3bFxcU1Rf0AEPAYYglAIGl0+Hzuued00003afz48ZKk6dOna/bs2XrzzTd1//33H9X+zTff1P79+7V48WLZ7XZJUseOHU+uagAIIgyxBCCQNKrb3el0avny5UpPTz+0A6tV6enpWrJkSZ3bfPbZZ0pLS9PEiROVmJioPn366IknnpDL5Tq5ygEgSFQPsXR2j9YMsQTA7zXqyGdeXp5cLpcSExNrLU9MTNT69evr3Gbr1q365ptvdO2112rOnDnavHmzbr/9dlVUVGjKlCl1blNeXq7y8vKa5wUFBY0pEwACRq0hlrrT5Q7A/zX71e5ut1sJCQl67bXXNGjQII0ZM0YPPvigpk+fXu82U6dOVWxsbM0tJSWlucsEAJ+UmccQSwACS6PCZ3x8vGw2m3Jycmotz8nJUVJSUp3bJCcnq3v37rLZbDXLevXqpezsbDmdzjq3mTx5svLz82tuO3fubEyZABAwvt+UJ0ka3LGFIkMZYgmA/2tU+HQ4HBo0aJAyMjJqlrndbmVkZCgtLa3ObU4//XRt3rxZbre7ZtnGjRuVnJwsh8NR5zahoaGKiYmpdQOAYPT9Jk+X+5ndGGIJQGBodLf7pEmT9Prrr+vtt9/WunXrdNttt6m4uLjm6vexY8dq8uTJNe1vu+027d+/X3fddZc2btyo2bNn64knntDEiROb7l0AQAByVrq1ZMs+SdKZ3eJNrgYAmkaj+3DGjBmj3NxcPfLII8rOztaAAQM0d+7cmouQduzYIav1UKZNSUnRvHnz9Kc//Un9+vVT27Ztddddd+m+++5runcBAAFo5Y4DKna61CrSod7J9AABCAwWwzAMs4s4noKCAsXGxio/P58ueABB49l5G/Tit5t1Sf82+ufVA80uBwCOqaF5jbndAcBHHTrfky53AIGD8AkAPuhAsVO/ZuVL4mIjAIGF8AkAPuiHLXkyDKl7YpSSYsPMLgcAmgzhEwB80PcbPeN7ctQTQKAhfAKAjzEMQ4s2V4dPzvcEEFgInwDgY7bmFSvroGdKzdROTKkJILAQPgHAx3y/0XOV+5BOLRTusB2nNQD4F8InAPiY6vncOd8TQCAifAKAD3FWurVkK1NqAghchE8A8CErdhxQidOl+CiHeiUxoxuAwEP4BAAfUj2r0Rld42W1WkyuBgCaHuETAHxI9fmeZ3C+J4AARfgEAB+xv9ip1TVTanK+J4DARPgEAB/xw2bPlJo9EqOVGMOUmgACE+ETAHxE9fmeHPUEEMgInwDgAwzD0KLq8T27c74ngMBF+AQAH7Alt1i788vkCLFqaMeWZpcDAM2G8AkAPqC6y31ox5ZMqQkgoBE+AcAHHJpSk/M9AQQ2wicAmKy80qUlW6qn1OR8TwCBjfAJACZbsf2gSitcio8KVc+kaLPLAYBmRfgEAJMdPsQSU2oCCHSETwAwWc2Uml053xNA4CN8AoCJ9hWVa81uptQEEDwInwBgoh+27JNhSD2TopXAlJoAggDhEwBMtKjqfE+63AEEC8InAJjEMIxD43sypSaAIEH4BACTbMkt1p78MjlsTKkJIHgQPgHAJNVd7oM7tmBKTQBBg/AJACZZtLlqiCWucgcQRAifAGCCCpdbP27dL0k6syvnewIIHoRPADDBqp0HVVReqRYRdp3SJsbscgDAawifAGCC6qvcT+/KlJoAggvhEwBMsOiw+dwBIJgQPgHAy/JLK/TLLs+Ummd043xPAMGF8AkAXrZkyz653IY6x0eqbVy42eUAgFcRPgHAyxZtrppSky53AEGI8AkAXrao6mIj5nMHEIwInwDgRTv3l2jbvhLZrBad1qWV2eUAgNcRPgHAi6pnNRqQEqeYMLvJ1QCA9xE+AcCL6HIHEOwInwDgJS63oR+2eMIn43sCCFaETwDwkrW783WwpELRoSHqnxJndjkAYArCJwB4SfWUmqd1aSW7jZ9fAMGJXz8A8JLvmVITAAifAOANJc5KLd9+QBIXGwEIboRPAPCCpZn7VeEy1DYuXJ3iI80uBwBMQ/gEAC84fIgli8VicjUAYB7CJwB4QU345HxPAEGO8AkAzWxvQZk25BTKYpFO53xPAEGO8AkAzax6Ss1T2sSoZaTD5GoAwFyETwBoZtVd7md2a21yJQBgPsInADQjwzD0fdWRzzPpcgcAwicANKcNOYXKLSxXmN2qQR1bmF0OAJiO8AkAzai6y31op1YKDbGZXA0AmI/wCQDNqHo+d7rcAcCD8AkAzaS80qWlmfskMb4nAFQjfAJAM1m+/YDKKtyKjwpVz6Ros8sBAJ9A+ASAZnJoSs1WTKkJAFUInwDQTKoHlz+D8T0BoAbhEwCawYFip1Zn5UuSzuR8TwCoQfgEgGbww5Y8GYbUPTFKiTFhZpcDAD6D8AkAzeDQ+Z50uQPA4QifANDEDMM4NL4nXe4AUAvhEwCa2LZ9Jco6WCq7zaLUzi3NLgcAfArhEwCa2KJNuZKkU9u3UIQjxORqAMC3ED4BoInR5Q4A9SN8AkATqnC5tXiLZ0rNMxnfEwCOQvgEgCa0csdBFZVXqmWkQ33bxppdDgD4HMInADShhRv3SpLO6Bovq5UpNQHgSIRPAGhCCzd6LjYa3p0udwCoC+ETAJpIXlG51mQVSJLO7M7FRgBQF8InADSR76uGWOqdHKOEaKbUBIC6ED4BoIl8t9EzxNJZdLkDQL1OKHy+9NJL6tixo8LCwpSamqply5Y1aLsPPvhAFotFo0ePPpGXBQCf5XYb+o7zPQHguBodPmfOnKlJkyZpypQpWrFihfr3768RI0Zo7969x9xu27Ztuueee3TmmWeecLEA4Kt+21OgfcVORTpsGtShhdnlAIDPanT4fO6553TTTTdp/Pjx6t27t6ZPn66IiAi9+eab9W7jcrl07bXX6tFHH1Xnzp1PqmAA8EXVV7mndYmXI4QzmgCgPo36hXQ6nVq+fLnS09MP7cBqVXp6upYsWVLvdo899pgSEhJ0ww03NOh1ysvLVVBQUOsGAL6sZoilHnS5A8CxNCp85uXlyeVyKTExsdbyxMREZWdn17nNokWL9MYbb+j1119v8OtMnTpVsbGxNbeUlJTGlAkAXlVYVqEV2w9IkoYzpSYAHFOz9g0VFhbquuuu0+uvv674+IaPeTd58mTl5+fX3Hbu3NmMVQLAyVm8ZZ8q3YY6xUeqfasIs8sBAJ8W0pjG8fHxstlsysnJqbU8JydHSUlJR7XfsmWLtm3bplGjRtUsc7vdnhcOCdGGDRvUpUuXo7YLDQ1VaGhoY0oDANNUd7mf1Y2B5QHgeBp15NPhcGjQoEHKyMioWeZ2u5WRkaG0tLSj2vfs2VOrV6/WqlWram6XXHKJzjnnHK1atYrudAB+zzAOG2KJ8z0B4LgadeRTkiZNmqRx48Zp8ODBGjp0qKZNm6bi4mKNHz9ekjR27Fi1bdtWU6dOVVhYmPr06VNr+7i4OEk6ajkA+KOtecXadaBUDptVp3VuZXY5AODzGh0+x4wZo9zcXD3yyCPKzs7WgAEDNHfu3JqLkHbs2CGrlWFGAASHhRs8Rz2HdGqhCEejf1IBIOhYDMMwzC7ieAoKChQbG6v8/HzFxMSYXQ4A1Lh+xjIt2JCrBy7qqZvPOvocdgAIFg3NaxyiBIATVFbh0o9b90liPncAaCjCJwCcoJ+27VdZhVuJMaHqkRhtdjkA4BcInwBwgr5dXz3EUmtZLBaTqwEA/0D4BIAT9O2GvZKkc3smmFwJAPgPwicAnIDMvGJl5hXLbrPoDAaXB4AGI3wCwAn4Zr3nqOeQji0VHWY3uRoA8B+ETwA4Ad+up8sdAE4E4RMAGqmovFJLMz1DLBE+AaBxCJ8A0EiLNuWqwmWoY6sIdW4dZXY5AOBXCJ8A0EjV53uew1FPAGg0wicANILbbejbqvnc6XIHgMYjfAJAI6zdXaDcwnJFOGwa2qml2eUAgN8hfAJAI1R3uZ/RNV6hITaTqwEA/0P4BIBG+IZZjQDgpBA+AaCB8orK9euug5K42AgAThThEwAaaMGGXBmG1KdtjBJjwswuBwD8EuETABqoZlajHhz1BIATRfgEgAaocLn13UbPEEt0uQPAiSN8AkAD/LztgArLK9Uq0qH+7eLMLgcA/BbhEwAa4Nuqq9yH92gtq9VicjUA4L8InwDQANXjezLEEgCcHMInABzHtrxibd5bpBCrRWd2a212OQDg1wifAHAcX/2WLUk6rXMrxYbbTa4GAPwb4RMAjuOrtTmSpAtOSTS5EgDwf4RPADiG3MJyLd9xQJJ0fm/CJwCcLMInABzD1+tyZBhS/3axSo4NN7scAPB7hE8AOIav1nrO97zglCSTKwGAwED4BIB6FJZV6IfN+yRJF9DlDgBNgvAJAPVYuDFXTpdbneMj1TUhyuxyACAgED4BoB7zqq5yP/+URFkszGoEAE2B8AkAdSivdOnbqlmNLujN+Z4A0FQInwBQhx+37ldReaVaR4dqYEqc2eUAQMAgfAJAHeZVXeV+fu9EWa10uQNAUyF8AsAR3G5D83+rmtWIq9wBoEkRPgHgCKt2HVRuYbmiQ0M0rEu82eUAQEAhfALAEaq73M/umSBHCD+TANCU+FUFgMMYhqGv1tLlDgDNhfAJAIfZklukzLxiOWxWnd2jtdnlAEDAIXwCwGGqB5Yf1rWVosPsJlcDAIGH8AkAh/lyzR5JDCwPAM2F8AkAVbbvK9aarALZrBaNOIXzPQGgORA+AaDK7NWeo56ndW6pVlGhJlcDAIGJ8AkAVeZUhc+L+7YxuRIACFyETwAQXe4A4C2ETwDQoS73tM6t6HIHgGZE+AQAHepyv6hvssmVAEBgI3wCCHp0uQOA9xA+AQS9L36lyx0AvIXwCSDoff7LbknSxf3ocgeA5kb4BBDUNmQXan12oew2iy7qQ/gEgOZG+AQQ1D77JUuSNLx7gmIjmMsdAJob4RNA0DIMQ59VdblfMoCB5QHAGwifAILWyp0HtXN/qSIcNqX3SjC7HAAICoRPAEHrs1Weo57n905UhCPE5GoAIDjwawsgKLncRs0QS5d6q8vdVSkV5XhuxXmSI1KKSpCiEqWwGO/UAAAmI3wCCEqLt+Qpr6hccRF2ndG1dfO90MGd0oY50uYMadv3UkVJ3e1a95K6nid1O1/qeJZkpWMKQGAifAIISrNWeK5yv7hvshwhzRD0dq+SFv9TWvuJZLgOLbeGSJEJUmS8J4gW7ZXKC6TcdZ7bkhelVt2kYX+U+l0l2cOavjYAMBHhE0DQKS6v1JdrsiVJlw9q17Q7P7hT+vI+acPsQ8vap0ndR0hdzpMS+xx9VLN4n7T1W2nLN9K6L6R9m6TP75K+nSqNeFzqc7lksTRtnQBgEsIngKAzd022Sitc6hQfqYEpcU2zU1eltOw16Zv/kyqKJYtN6nOZNOwOKbn/sbeNbCX1vcJzG/mUtOIdacnLUsEu6X83SL+8L138d6lFx6apFQBMxElFAILOxyt3SZIuG9hWlqY4oliUK/17tDRvsid4ppwm3faDdPm/jh88jxQaLaVNlO5cIZ39gGRzSJu/ll45Xfrt05OvFQBMRvgEEFR2HyzV4i37JEmjB7Y9+R3u/El69SzPxUSOKGnUP6TxX0oJvU5uvyGh0tn3SbctkdoPk5xF0odjpa8e9hxlBQA/RfgEEFQ+WZUlw5BSO7VUSsuIk9vZrx9JM0ZKhbul+O7STd9Kg65v2ivV47tK4z73dN9LnouY3rtSKi9qutcAAC8ifAIIGoZh6OOqq9wvP/UkLzRa+pr08Y2Su0LqNUq66RupdfcmqLIOthDpgv+Trnxbskd6Lkx651KpZH/zvB4ANCPCJ4Cg8cuufG3eW6TQEKtG9k06sZ0YhrTwaenLez3Ph94iXfmO51zN5nbKaGncZ1J4CynrZ2nGRVLBnuZ/XQBoQoRPAEFj5k87JUkX9U1WdJj9xHby3TPSt497Hp892XN1ujcHhG832HNOaXSyZ1zQt0d5LngCAD9B+AQQFEqclfr8F89c7n8YnHJiO1ny8qHgecHj0tn3mzP+ZkIvacI8KTbFMybov38vlR7wfh0AcAIInwCCwuxf96iovFIdWkXotM4tG7+D5W97hlKSpHMe9MxAZKYWHaSxn3pmS8pZLb17hVReaG5NANAAhE8AQeHDnz1d7n8YnNL4sT03zJW+uNvzeNid0ln3Nm1xJ6pVF08ArT4H9KPxDMMEwOcRPgEEvC25Rfpp2wFZLSdwlfvuVdJ/J0iGWxp4nXT+Y7411WVib+na/0kh4dLm+dKXf/FcFAUAPorwCSDgfVh1odE5PRKUFBvW8A3zd0nvjfHMWtT5bOl3z/tW8KzWbpB0+euSLNLPb0hLXjK7IgCoF+ETQEBzVrr1vxWe6TT/MKQRFxo5S6T3r5KKsqXWPaU/vCPZTvAKeW/oNcozFqgkffWQ51QBAPBBhE8AAW3e2mzlFTmVEB2qc3smNGwjw5A+v1PKXi1FxEvXfCiFxTZvoU0hbaI0eIIkQ/r4Jilvs9kVAcBRCJ8AAtq7P26XJF01JEV2WwN/8n58WVr9kWQN8RzxbNGhGStsQhaLdOFTUvs0qbxAmnktV8AD8DknFD5feukldezYUWFhYUpNTdWyZcvqbfv666/rzDPPVIsWLdSiRQulp6cfsz0ANJVNOYVamrlfVot01dD2Ddso8zvpq4c9j0c8IXU8vfkKbA4hDs80nNHJUu566ZPbuAAJgE9pdPicOXOmJk2apClTpmjFihXq37+/RowYob1799bZfsGCBbr66qv17bffasmSJUpJSdEFF1ygrKysky4eAI7lP0t3SJLO65WoNnHhx9+gMLvqynaX1P9qaejNzVxhM4lOlMa8K9kc0rrPpaXTza4IAGpYDKNxfxKnpqZqyJAhevHFFyVJbrdbKSkpuuOOO3T//fcfd3uXy6UWLVroxRdf1NixYxv0mgUFBYqNjVV+fr5iYmIaUy6AIFXirFTq4xkqLK/U2xOGanj31sfewFUpvXOptH2RlNhHuvFryd6AwOrLlr0uzblHstqlG76S2p5qdkUAAlhD81qjjnw6nU4tX75c6enph3ZgtSo9PV1Llixp0D5KSkpUUVGhli3rn2GkvLxcBQUFtW4A0BifrdqtwqoZjc7sGn/8DRZM9QRPR5Sn29rfg6ckDblR6nWJ5K6Q/jteKss3uyIAaFz4zMvLk8vlUmJiYq3liYmJys7ObtA+7rvvPrVp06ZWgD3S1KlTFRsbW3NLSTnBeZgBBCXDMPTvqguNrhnaXlbrccbm3Jwhff+s5/El/5TiuzZzhV5isUiXvCDFtZcObJM+v4vzPwGYzqtXuz/55JP64IMPNGvWLIWF1T/Q8+TJk5Wfn19z27lzpxerBODvlmXu19rdBQqzW/WHwcf547U4z3NRjuQZpqjP5c1foDeFx0lXzPBcub92lrT8LbMrAhDkGhU+4+PjZbPZlJOTU2t5Tk6OkpKSjrnts88+qyeffFJfffWV+vXrd8y2oaGhiomJqXUDgIaa8cM2SdLvB7ZTi0hH/Q0NQ/rsTqkoxzOQ/IgnvFOgt7UbLJ03xfN47v1Szlpz6wEQ1BoVPh0OhwYNGqSMjIyaZW63WxkZGUpLS6t3u6efflp/+9vfNHfuXA0ePPjEqwWA49i5v0Rf/eY5DWjC6R2P3XjF29KG2Z4Lci57PTDO86xP2h+lrudLlWXSR9dLzmKzKwIQpBrd7T5p0iS9/vrrevvtt7Vu3TrddtttKi4u1vjx4yVJY8eO1eTJk2vaP/XUU3r44Yf15ptvqmPHjsrOzlZ2draKioqa7l0AQJW3F2+T25DO7BavbonR9TfM2yzNrfqtOu8RKfnYPTJ+z2qVfj/dM/5n3kZpzl/MrghAkGp0+BwzZoyeffZZPfLIIxowYIBWrVqluXPn1lyEtGPHDu3Zs6em/SuvvCKn06krrrhCycnJNbdnn3226d4FAEgqKq/UzJ8954hPOL1T/Q1dFZ7pJytKpE5neY4KBoPIeOnyf0mySKveldZ+YnZFAIJQo8f5NAPjfAJoiLcXb9OUz9aqc3ykvp40vP6r3DP+5rm6PSxOum2xFNvWq3WaLuMx6fu/B+/7B9AsmmWcTwDwVZUut17/fqskafzpHesPntuXSIue8zweNS04g9fZk6U2A6Wyg9Int0put9kVAQgihE8AAWH26j3adaBULSMdumJQPcMrleVLH98sGW6p/zXSKb/3bpG+wmaXLvuXZI/wzGW/5EWzKwIQRAifAPyeYRiavtBz1PP6YR0V7rDV3XDOvVL+DimugzTyKS9W6IPiu0oXTvU8znhM2vOLufUACBqETwB+77tNeVq3p0DhdpvGpnWou9Hq/0q/zpQsVs+wSmGcP65Tx0k9f+eZfvN/N0nOErMrAhAECJ8A/N70BVskSVcNTVFcRB2Dyh/cKX0xyfP4rHul9qlerM6HWSzSqH9KUUlS3gZp/sNmVwQgCBA+Afi1X3Ye1JKt+xRitejGMzsf3cDtkmbdKpXnS20He8InDolsJf3+Fc/jn/4lbZhrbj0AAh7hE4Bfe+GbTZKkS/q3Udu4OmYoWvyCtH2RZI+ULnvNc7ENautyrnTaRM/jTydKRXvNrQdAQCN8AvBba7Ly9fW6vbJapInndj26we5V0jf/53k88kmpVRev1udXzntESjhFKsnzBFDfHwIagJ8ifALwW//I8Bz1HNW/jbq0jqq90lnimcXIXeG5qGbgdSZU6EfsYZ7Zj2yh0qavPF3wANAMCJ8A/NLa3fma/1uOLBbpjnO7Hd1g/sOeOcyjkqRLXvBcXINjS+wtnf+Y5/FXD0l715tbD4CARPgE4Jf+WX3Us18bdU044qjnxnmHjtz9/hUpoqWXq/NjqbdIXdOlyjLpfzdKleVmVwQgwBA+AfidtbvzNW+t56jnnecdca5nUa7nnEVJOu12z8U0aDiLRbr0ZSmilZSz2jMAPQA0IcInAL/zzLwNkqqPekYfWmEYnuBZnOu5eOa8KSZV6OeiE6VLqqbcXPKitOVbc+sBEFAInwD8yo9b92nBhlyFWC2adH732it/fkPaNM9z0czlr3suosGJ6XmRNGi85/Ent0kl+82tB0DAIHwC8BuGYejpuZ6LYK4amqKO8ZGHVub8Js170PM4/a9S4ineLzDQjHhcatVNKtwjfX4Xwy8BaBKETwB+4+t1e7Vix0GF2a268/Ar3CtKpf9O8Fwk0zVdSr3VvCIDiSPScwTZGiKt+0xa+a7ZFQEIAIRPAH6h0uXWM/M8Rz0nnN5JCTGHdal/9ZCUu06KTJBGvyJZ+WlrMm0GSuc+5Hn85X3Svi3m1gPA7/ELDcAvzPx5pzbmFCk23K5bhh82U9H62YcNqzRdikowp8BANuxOqeOZUkWxZ+B+V4XZFQHwY4RPAD6voKxCf/9qoyTpT+ndFBteNT97ftahYZXS/ih1Pc+kCgOc1eYJ9mGxUtZyaeHTZlcEwI8RPgH4vBcyNml/sVNdWkfq2tM6eBa6XdKsW6TSA1LyAIZVam6x7aTfTfM8/v5ZafsSU8sB4L8InwB8WmZesd5avE2S9NDvestuq/rZWvS8tO17yR4pXfGmFOIwr8hg0ecyqf/VkuGWZt0sleWbXREAP0T4BODTHp+9ThUuQ2f3aK1zelSdz7l9ifTtE57HFz0jtepS/w7QtEY+LcV1kA7ukD6/m+GXADQa4ROAz5r/W46+XpejEKtFD13cy7OwaK/00fWS4ZL6XCENuMbUGoNOWIx0+b88wy+t/Vha9rrZFQHwM4RPAD6pxFmpv362VpJ045mdPdNouio943kWZUvxPaRR//DMRQ7vShkqnV815/u8B6SdP5lbDwC/QvgE4JP+mbFZWQdL1TYuXHee19Wz8NvHD53nOebfUmiUuUUGs9Nul3pfKrkrPEeii/eZXREAP0H4BOBzNmQX6l/fb5Uk/fWSUxThCJE2fCktes7T4JJ/Sq17mFghZLFIl7woteoqFeySPr7RMwIBABwH4ROAT3G5DU3++FdVug2d3ztR5/dOlPZneoZVkqSht0h9rzC3SHiExUh/+Ldkj5C2fMP4nwAahPAJwKfM+CFTK3YcVFRoiP56ySlSRZn04VjPsD7thkgX/J/ZJeJwib0Pjf+58Clp09emlgPA9xE+AfiMrblFembeBknSgxf3UtvYMGnOPVL2r1JEK+nKtxjP0xf1HyMNniDJ8HS/H9hmdkUAfBjhE4BPcLkN/eW/v6q80q0zusbrqiEp0o8vSyv/LckiXfa6Z5Yd+KYLn5TaDPTMOPXeVVJZgdkVAfBRhE8APuHNRZn6efsBRTpsevLyvrJs+kqa96Bn5QX/x7ztvi4kVBrzHykqScpdJ/3vBi5AAlAnwicA063JytfT89ZLkh68uLfaObd5xvOUIZ06VkqbaGp9aKDYttLV70khYdKmr6SvHja7IgA+iPAJwFQlzkrd+cFKVbgMXdA7UVefEia9N0ZyFkkdzpAu+jsDyfuTtoOk30/3PP7xJWn5W6aWA8D3ED4BmOpvX6zT1txiJcaE6qlLu8sy8/9J+Tuklp09A8lzgZH/OeX30jlVp0zM/rOU+Z259QDwKYRPAKb54tfden/ZDlks0nNX9leLjHuknUul0Fjp6plSREuzS8SJOuteqc8VkrtSmnmdlLfZ7IoA+AjCJwBTbMop1F/++6sk6dbhXXR61hvSrzMli036w9tS6+4mV4iTYrFIl74otR0slR2U3rtSKs4zuyoAPoDwCcDrisordeu7y1XidCmtcyvd0/IHacFUz8qLnpG6nGNugWga9nDpqvek2PbS/q3Sf66QygvNrgqAyQifALzKMAz95b+/aEtusZJiwvTq4CzZvrzHs/Ksv0hDbjC3QDSt6ETpuo+l8JbS7pWeLvhKp9lVATAR4ROAV/0zY7PmrM6W3WbRO+eUKWb2bZLhlk4dJ53zgNnloTnEd5Ou/a9kj5S2fivNupkxQIEgRvgE4DWf/7Jbz3+9UZL08pkV6v7NjZKrXOr5O+ni5xhSKZC1GySNeUey2qW1s6TP7pDcbrOrAmACwicAr1i544Du+egXSdIjA0t1/oqJUkWJ1OVc6Yo3JVuIyRWi2XVNl654w3NR2ar/SHPukQzD7KoAeBnhE0Czy8wr1k3v/KzySrdu6HxQ4zP/LDkLPYPIj/mPZ2pGBIfel1YNQm+Rfn5D+vI+AigQZAifAJpVTkGZrntjqfKKnLosYY8e2ne/LGUHpXZDpWs+kBwRZpcIb+v3B+mSf3oeL3vVMxA9XfBA0CB8Amg2+SUVGvvGMu06UKqL43bo2bIpspQXSO3TpP/3Pyk02uwSYZZTx0qXvKiaI6Bf3EUABYIE4RNAsygsq9D1by3ThpxCXRL5m16ofFRWZ5HU8UzPlc9hMWaXCLOdep2nC95ilVa8I318I8MwAUGA8AmgyRWWVWjsm8u0csdBjQlbpn8YT8laWeq5uOiaD6XQKLNLhK/of5V02euSNURa8z/p/askZ7HZVQFoRoRPAE3q8OB5a9jXelL/kMVdIfW53DNfO+d44kh9r/B8N+wR0pYM6e1LmIoTCGCETwBNZl9Rua55fal+2bFfj4f9W/frTVlkSENvli77lxTiMLtE+Kpu6dLYz6TwFlLWz9K/zpNyN5pdFYBmQPgE0CR2HSjRldOXaEtWjt4Ke17X6kvPivS/SiOflqz83OA4UoZIE76S4jpIB7ZJb6RLWxeaXRWAJsa/BgBO2vrsAl3xyhK59m3RF+FTdJaWSyFh0pVvS2f8iZmL0HCtu0s3feMZiqssX3r3Mmnpq4wFCgQQwieAk/LN+hxd/vJi9Sz6UV+EPqzOxk4pKlEa94V0ymizy4M/ioyXxn0u9b1ScldKX/5F+uQ2qaLU7MoANAHCJ4ATYhiG3liUqVveXqqb3R/oTcczilax54jVzQs9XajAibKHea6CH/GEZzrOX96X3jhf2rfF7MoAnCTCJ4BGK3FW6u6Zq/TaF4v0rv1x3RUyS1YZ0qDx0vWzpZhks0tEILBYpLSJ0nWzpIhWUvZq6dWzpF8/MrsyACeB8AmgUTbvLdKlL/6g8l8/0Zeh9yvVul6GI0q6/A1p1DSuaEfT6zxcunWR1OEMyVnkGYx+1m2ec0IB+B3CJ4AGMQxDHyzboWtfnKfbDjyt6Y5pamkpkpL6yXLLd56xGoHmEtNGGveZNPx+z4xIv7wnvTyMq+EBPxRidgEAfN/+Yqfu/9+v0vov9In9bSVb98uwWGU540+eMMDRTniD1SadM1nqco4061bpQKb0ziXS4AmeIb3CYs2uEEADWAzD98evKCgoUGxsrPLz8xUTw3zQgLcYhqHPf92jlz/9TndX/EsX2n7yLG/ZWZbfvyqlDDW5QgSt8iJp/sPSz296nkclSRc9LfW6hKG9AJM0NK8RPgHUKetgqR6btUKdNr+jO0JmKdJSLsMSIsvpd0rD/yLZw80uEZAyv5e+uFvat9nzvMt50oVPesYLBeBVhE8AJ6SswqXXF27RhoXva5LlPXW2ZkuS3G2HyDpqmpTUx9wCgSNVlEnfPyv98A/J5ZSsIZ4pXc+8R4psZXZ1QNAgfAJoFLfb0JzVu/XtnPc1rvRd9bNmSpIqIxIUcsFjUr8xTJEJ37ZvizTvQWlj1dSujijptNuktD9K4XGmlgYEA8IngAYxDEOLNudpzuf/1e8PztBQ6wZJUqUtXLZhE2U5/S4pjP/v4Ec2fy1lPCbt+cXzPCxWGnanlHqrFBplbm1AACN8AjgmwzC0cMNefTf3Q527732dYVsrSaq0hsoYfKPswyd5pjkE/JFhSOs+l759XMpd71kWEe8ZtH7Q9VJES1PLAwIR4RNAncorXZq7Yqt2LXxLFxTOUjdrliTJZQlRRb//p7Dz7vOMqQgEArdLWvM/6dsnPEMzSVJIuNT/Ks+R0ISe5tYHBBDCJ4Ba9uSX6rOFy+RYOUOj3fPVwlIkSSq3RsjV/xpFnHWH1KKjuUUCzcVVIa3+SPrxZc80ndW6nCul3iZ1Pc8zjiiAE0b4BCCX29DS9du0ZeH76rrnc6Va1slq8fwvnx/WViFptyoydRyDcyN4GIa0fbEnhK6fLanqn8DoZKnvlVL/q6XE3qaWCPgrwicQpAzD0KrtuVr7/edqtXWWznYvVbjFWbM+r/VpanHOHbL1HMmRHgS3A9ukpa9Jq/4jlR08tDypryeE9rlcik4yqzrA7xA+gSDidhtam7lT25d+ptAt8zS08mfFWkpq1ueGtpfR7yolnH6dFNfexEoBH1RZLm36SvrlA2njPMldcWhd28FSz4ukHhdJrXsyexJwDIRPIMDlFzu1csViHVjztZJzFmqQsVZ2i6tmfaEtTgc7/05JZ46XPWUQ/2gCDVGy33OB0q8zpV0/1V7XoqMnhHY5T2p/GsM2AUcgfAIBJr/EqXVrVujgb98oZs9i9Sj7Ra0sBbXa7LG3V1HH85Vy2uUK63Qa3erAySjYI22cK22YI21dKLnKD62zhkhtTpU6nSl1PFNKSZUcEebVCvgAwifgx1xuQ9t37lD2usUq3/6TInN/UZeKDWplKazVrkwO7Y4ZIGvXc9Qm9Qo5EpnPGmgW5UXS1m89YTTzO+ngjtrrrSFS4ilS20GHbvHd+QMQQYXwCfiJwpJS7dy8Rge3rVLF7jUKO7BBbcq2KMWy96i2Ttm1I7KPytqertb90pXY83QpxGFC1UCQO7Bd2va9lPm9574g6+g2jijPxUsJvaWEXp5wmtBLCm/h/XoBLyB8Aj6k3FmunB2bdGDXBpXlbJFr/1Y5CrYrrixLKe4shVoq69wuy9ZOebF9ZEsZrDa9T1fLLoOkkFAvVw/gmAxDyt8pZa2QspZ77nevlCqK624f3UZq1cVza3nYfYuOkj3Mq6UDTYnwCXhJRUWF8vZmKT9nu4pzd6n8QJaMgt2yFecovDRbrZy7lWTsVYjFXe8+ShSmLHtHFcZ2V0jSKWrVeYCSeqbKFsEREsAvuV1S7gYpZ620d62U85u09zdPSD2WyNZSTFsptl3Vfdvaz6MS+AMUPqtZw+dLL72kZ555RtnZ2erfv79eeOEFDR06tN72H330kR5++GFt27ZN3bp101NPPaWLLrqowa9H+IS3lJeXqbjggEoLD6q06IBK83NVlp+ryqI8GcX7ZC3bL1vZAYU6Dyq88qBiXAfUyjh4zGBZs2/Drmxbkg6EtVN5dHvZ47souk03JXbuq5jELpLV6oV3CMBUZfmeULpvi7R/y2H3WyVn4fG3l6TQGCky3hNUI+KrHlc9D2/pmTQiLMZzH1p174jiNwbNrqF5LaSxO545c6YmTZqk6dOnKzU1VdOmTdOIESO0YcMGJSQkHNV+8eLFuvrqqzV16lT97ne/03vvvafRo0drxYoV6tOnT2NfHkHKcLtVWVmhygqnKiqcqigvlbOsWM6yUlWUFauyvMRzc5bKVV4qt7NE7opSGVU3S0WZVFkmVZbI5ixWSGWhHJXFCnUVK8xdonCjRJFGicIsFWr0MQWL5DIsOmCJ08GQeBWHtpYzIlFGVLJCYpMVmdxN8Sk91TKpvTpYberQHB8QAP8QFiulDPXcDmcYUukBKX+X5/zRmvusw57v9oxBWl7gue3f2ogXtngCaWjsoXAaGuO5Qt8eLtkjDt3qWmYPl0LCJJvdc+TV5vDcjnzMBVZogEYf+UxNTdWQIUP04osvSpLcbrdSUlJ0xx136P777z+q/ZgxY1RcXKwvvviiZtlpp52mAQMGaPr06Q16TW8e+SwuPKiNiz9VzcdiGDr0CblV/cSQUfO49n3VeuOw9TIO7a9qvcU4Yplx2HY6Yr8y6nzcmLa12tW8B0mGu2a9xZAMuSW327PccHnu3W5Zqh8bdT+21HpuyGK4ZNHhy901zy2GS1ajUjZ3pWyqlNWoVIhRKVv1TS7Z5VkXYrgUoko5Dhu/0htKDYdKLBEqskarJCRO5fY4VYTGyR3eSpaIlrJFxcsR01oRLZLUIqmD4uLbyBpi92qNAIKMYXhmYireJxXnHrqVHPa8ZL8nmJYVeI6yluXXHjS/uVlsVYHULtlCDwVSa4jnZrHVfl7z+HhtbIfWWayecYstVs9NlqrnlsOeW+t4bj3U7rht6ttv9XjJh42bfOSyRrVp6HZ1Pa9nuyPbtOgoJfeTNzTLkU+n06nly5dr8uTJNcusVqvS09O1ZMmSOrdZsmSJJk2aVGvZiBEj9Mknn9T7OuXl5SovPzSeWkFBQb1tm9qBnF0auOROr70ejuMY46KXG3aVWxwql0NOi0MVllBVWD03V/XNFia3LUzukDAZIWFSSLgUFiNrWLRCwmMUEhErR2SswiLjFBYVp4joFoqMjlO43aFwSa289kYB4DgsFs+V8uEtpPiuDdvGMDy9PtVhtLzAE2DLqo6eVpRKFSWSs6TqcXEdy0o8t8pyyeX03CqdnnFPXc4jXs9V1b7J3z1O1JAbpYv/bnYVtTQqfObl5cnlcikxMbHW8sTERK1fv77ObbKzs+tsn52dXe/rTJ06VY8++mhjSmsy9rBwrbOfIkkyDvsr4tDh4aq/mCQZh/11UdNWlsOWH9a2aj+HlldtY7EcsX3V+qOWWw4LYpZatdXaX63Xqaue+vavmr/uDIvV89elLDKs1X9l2jz7qf6rs2Z59WObLFZrTVtZrLJYa9/LapOl6rk1xCFLiEPWkBBZbQ5ZQxyyhdhltYfKarMrxO55brOHymZ3KMTukD0kVDa7XSEOh+z2MIXabI3vIgeAYGKxVHWhh0vRicdv31iGIbkqqoJoRVVAPeKx21V1q6y6uTwhteZ5pafH7fDnRh3bVD+XUdX7ZtT0wtVeZhzdRoe1rbXMaECbw/cr6bBEUGfvYoPbnMx2jdh3i05Hb2eyRp/z6Q2TJ0+udbS0oKBAKSkpXnntxHZdlPjgYq+8FgAAfs1i8Yw1zHjDaIRGhc/4+HjZbDbl5OTUWp6Tk6OkpKQ6t0lKSmpUe0kKDQ1VaCjHtAAAAAJNo8ZdcDgcGjRokDIyMmqWud1uZWRkKC0trc5t0tLSarWXpPnz59fbHgAAAIGr0d3ukyZN0rhx4zR48GANHTpU06ZNU3FxscaPHy9JGjt2rNq2baupU6dKku666y4NHz5cf//733XxxRfrgw8+0M8//6zXXnutad8JAAAAfF6jw+eYMWOUm5urRx55RNnZ2RowYIDmzp1bc1HRjh07ZD1sINthw4bpvffe00MPPaQHHnhA3bp10yeffMIYnwAAAEGI6TUBAABw0hqa15hrCwAAAF5D+AQAAIDXED4BAADgNYRPAAAAeA3hEwAAAF5D+AQAAIDXED4BAADgNYRPAAAAeA3hEwAAAF5D+AQAAIDXED4BAADgNYRPAAAAeA3hEwAAAF4TYnYBDWEYhiSpoKDA5EoAAABQl+qcVp3b6uMX4bOwsFCSlJKSYnIlAAAAOJbCwkLFxsbWu95iHC+e+gC3263du3crOjpaFoul2V+voKBAKSkp2rlzp2JiYpr99fwFn0v9+GzqxudSPz6buvG51I/Ppm58LvXz9mdjGIYKCwvVpk0bWa31n9npF0c+rVar2rVr5/XXjYmJ4YtcBz6X+vHZ1I3PpX58NnXjc6kfn03d+Fzq583P5lhHPKtxwREAAAC8hvAJAAAAryF81iE0NFRTpkxRaGio2aX4FD6X+vHZ1I3PpX58NnXjc6kfn03d+Fzq56ufjV9ccAQAAIDAwJFPAAAAeA3hEwAAAF5D+AQAAIDXED4BAADgNUEZPh9//HENGzZMERERiouLq7PNjh07dPHFFysiIkIJCQm69957VVlZecz97t+/X9dee61iYmIUFxenG264QUVFRc3wDrxjwYIFslgsdd5++umnerc7++yzj2p/6623erFy7+jYseNR7/PJJ5885jZlZWWaOHGiWrVqpaioKF1++eXKycnxUsXNb9u2bbrhhhvUqVMnhYeHq0uXLpoyZYqcTucxtwvU78xLL72kjh07KiwsTKmpqVq2bNkx23/00Ufq2bOnwsLC1LdvX82ZM8dLlXrH1KlTNWTIEEVHRyshIUGjR4/Whg0bjrnNW2+9ddR3IywszEsVe89f//rXo95nz549j7lNoH9fpLp/Zy0WiyZOnFhn+0D+vnz33XcaNWqU2rRpI4vFok8++aTWesMw9Mgjjyg5OVnh4eFKT0/Xpk2bjrvfxv5ONYWgDJ9Op1NXXnmlbrvttjrXu1wuXXzxxXI6nVq8eLHefvttvfXWW3rkkUeOud9rr71Wa9eu1fz58/XFF1/ou+++080339wcb8Erhg0bpj179tS63XjjjerUqZMGDx58zG1vuummWts9/fTTXqraux577LFa7/OOO+44Zvs//elP+vzzz/XRRx9p4cKF2r17ty677DIvVdv81q9fL7fbrVdffVVr167V888/r+nTp+uBBx447raB9p2ZOXOmJk2apClTpmjFihXq37+/RowYob1799bZfvHixbr66qt1ww03aOXKlRo9erRGjx6tNWvWeLny5rNw4UJNnDhRP/74o+bPn6+KigpdcMEFKi4uPuZ2MTExtb4b27dv91LF3nXKKafUep+LFi2qt20wfF8k6aeffqr1mcyfP1+SdOWVV9a7TaB+X4qLi9W/f3+99NJLda5/+umn9c9//lPTp0/X0qVLFRkZqREjRqisrKzefTb2d6rJGEFsxowZRmxs7FHL58yZY1itViM7O7tm2SuvvGLExMQY5eXlde7rt99+MyQZP/30U82yL7/80rBYLEZWVlaT124Gp9NptG7d2njssceO2W748OHGXXfd5Z2iTNShQwfj+eefb3D7gwcPGna73fjoo49qlq1bt86QZCxZsqQZKvQNTz/9tNGpU6djtgnE78zQoUONiRMn1jx3uVxGmzZtjKlTp9bZ/g9/+INx8cUX11qWmppq3HLLLc1ap5n27t1rSDIWLlxYb5v6fqcDzZQpU4z+/fs3uH0wfl8MwzDuuusuo0uXLobb7a5zfbB8XyQZs2bNqnnudruNpKQk45lnnqlZdvDgQSM0NNR4//33691PY3+nmkpQHvk8niVLlqhv375KTEysWTZixAgVFBRo7dq19W4TFxdX64hgenq6rFarli5d2uw1e8Nnn32mffv2afz48cdt+5///Efx8fHq06ePJk+erJKSEi9U6H1PPvmkWrVqpYEDB+qZZ5455qkZy5cvV0VFhdLT02uW9ezZU+3bt9eSJUu8Ua4p8vPz1bJly+O2C6TvjNPp1PLly2v9t7ZarUpPT6/3v/WSJUtqtZc8vzuB/t2QdNzvR1FRkTp06KCUlBRdeuml9f4O+7tNmzapTZs26ty5s6699lrt2LGj3rbB+H1xOp169913NWHCBFkslnrbBcv35XCZmZnKzs6u9Z2IjY1Vampqvd+JE/mdaiohzbp3P5WdnV0reEqqeZ6dnV3vNgkJCbWWhYSEqGXLlvVu42/eeOMNjRgxQu3atTtmu2uuuUYdOnRQmzZt9Ouvv+q+++7Thg0b9PHHH3upUu+48847deqpp6ply5ZavHixJk+erD179ui5556rs312drYcDsdR5xknJiYGzHfkSJs3b9YLL7ygZ5999pjtAu07k5eXJ5fLVefvyPr16+vcpr7fnUD9brjdbt199906/fTT1adPn3rb9ejRQ2+++ab69eun/Px8Pfvssxo2bJjWrl173N8if5Kamqq33npLPXr00J49e/Too4/qzDPP1Jo1axQdHX1U+2D7vkjSJ598ooMHD+r666+vt02wfF+OVP3fvTHfiRP5nWoqARM+77//fj311FPHbLNu3brjnsAdDE7ks9q1a5fmzZunDz/88Lj7P/w81759+yo5OVnnnXeetmzZoi5dupx44V7QmM9m0qRJNcv69esnh8OhW265RVOnTvW5qcxO1ol8Z7KysnThhRfqyiuv1E033XTMbf35O4MTM3HiRK1Zs+aY5zVKUlpamtLS0mqeDxs2TL169dKrr76qv/3tb81dpteMHDmy5nG/fv2UmpqqDh066MMPP9QNN9xgYmW+44033tDIkSPVpk2betsEy/fF3wVM+Pzzn/98zL+GJKlz584N2ldSUtJRV3tVX5GclJRU7zZHnqBbWVmp/fv317uNWU7ks5oxY4ZatWqlSy65pNGvl5qaKslzFMzXg8TJfI9SU1NVWVmpbdu2qUePHketT0pKktPp1MGDB2sd/czJyfG578iRGvu57N69W+ecc46GDRum1157rdGv50/fmbrEx8fLZrMdNZLBsf5bJyUlNaq9P/vjH/9Yc1FmY49G2e12DRw4UJs3b26m6nxDXFycunfvXu/7DKbviyRt375dX3/9daN7Q4Ll+1L93z0nJ0fJyck1y3NycjRgwIA6tzmR36mmEjDhs3Xr1mrdunWT7CstLU2PP/649u7dW9OVPn/+fMXExKh37971bnPw4EEtX75cgwYNkiR98803crvdNf+Q+orGflaGYWjGjBkaO3as7HZ7o19v1apVklTrfwhfdTLfo1WrVslqtR51+kW1QYMGyW63KyMjQ5dffrkkacOGDdqxY0etv9R9UWM+l6ysLJ1zzjkaNGiQZsyYIau18aeW+9N3pi4Oh0ODBg1SRkaGRo8eLcnTzZyRkaE//vGPdW6TlpamjIwM3X333TXL5s+f7/PfjcYwDEN33HGHZs2apQULFqhTp06N3ofL5dLq1at10UUXNUOFvqOoqEhbtmzRddddV+f6YPi+HG7GjBlKSEjQxRdf3KjtguX70qlTJyUlJSkjI6MmbBYUFGjp0qX1juxzIr9TTaZZL2fyUdu3bzdWrlxpPProo0ZUVJSxcuVKY+XKlUZhYaFhGIZRWVlp9OnTx7jggguMVatWGXPnzjVat25tTJ48uWYfS5cuNXr06GHs2rWrZtmFF15oDBw40Fi6dKmxaNEio1u3bsbVV1/t9ffX1L7++mtDkrFu3bqj1u3atcvo0aOHsXTpUsMwDGPz5s3GY489Zvz8889GZmam8emnnxqdO3c2zjrrLG+X3awWL15sPP/888aqVauMLVu2GO+++67RunVrY+zYsTVtjvxsDMMwbr31VqN9+/bGN998Y/z8889GWlqakZaWZsZbaBa7du0yunbtapx33nnGrl27jD179tTcDm8TDN+ZDz74wAgNDTXeeust47fffjNuvvlmIy4urmYUjeuuu864//77a9r/8MMPRkhIiPHss88a69atM6ZMmWLY7XZj9erVZr2FJnfbbbcZsbGxxoIFC2p9N0pKSmraHPm5PProo8a8efOMLVu2GMuXLzeuuuoqIywszFi7dq0Zb6HZ/PnPfzYWLFhgZGZmGj/88IORnp5uxMfHG3v37jUMIzi/L9VcLpfRvn1747777jtqXTB9XwoLC2vyiiTjueeeM1auXGls377dMAzDePLJJ424uDjj008/NX799Vfj0ksvNTp16mSUlpbW7OPcc881XnjhhZrnx/udai5BGT7HjRtnSDrq9u2339a02bZtmzFy5EgjPDzciI+PN/785z8bFRUVNeu//fZbQ5KRmZlZs2zfvn3G1VdfbURFRRkxMTHG+PHjawKtP7v66quNYcOG1bkuMzOz1me3Y8cO46yzzjJatmxphIaGGl27djXuvfdeIz8/34sVN7/ly5cbqampRmxsrBEWFmb06tXLeOKJJ4yysrKaNkd+NoZhGKWlpcbtt99utGjRwoiIiDB+//vf1wpm/m7GjBl1/r91+N+5wfSdeeGFF4z27dsbDofDGDp0qPHjjz/WrBs+fLgxbty4Wu0//PBDo3v37obD4TBOOeUUY/bs2V6uuHnV992YMWNGTZsjP5e777675jNMTEw0LrroImPFihXeL76ZjRkzxkhOTjYcDofRtm1bY8yYMcbmzZtr1gfj96XavHnzDEnGhg0bjloXTN+X6txx5K36/bvdbuPhhx82EhMTjdDQUOO888476jPr0KGDMWXKlFrLjvU71VwshmEYzXtsFQAAAPBgnE8AAAB4DeETAAAAXkP4BAAAgNcQPgEAAOA1hE8AAAB4DeETAAAAXkP4BAAAgNcQPgEAAOA1hE8AAAB4DeETAAAAXkP4BAAvyc3NVVJSkp544omaZYsXL5bD4VBGRoaJlQGA9zC3OwB40Zw5czR69GgtXrxYPXr00IABA3TppZfqueeeM7s0APAKwicAeNnEiRP19ddfa/DgwVq9erV++uknhYaGml0WAHgF4RMAvKy0tFR9+vTRzp07tXz5cvXt29fskgDAazjnEwC8bMuWLdq9e7fcbre2bdtmdjkA4FUc+QQAL3I6nRo6dKgGDBigHj16aNq0aVq9erUSEhLMLg0AvILwCQBedO+99+q///2vfvnlF0VFRWn48OGKjY3VF198YXZpAOAVdLsDgJcsWLBA06ZN07///W/FxMTIarXq3//+t77//nu98sorZpcHAF7BkU8AAAB4DUc+AQAA4DWETwAAAHgN4RMAAABeQ/gEAACA1xA+AQAA4DWETwAAAHgN4RMAAABeQ/gEAACA1xA+AQAA4DWETwAAAHgN4RMAAABeQ/gEAACA1/x/SWhogTu+XXQAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.plot(x.numpy(), y, label='y')\n", "plt.plot(x.numpy(), dy_dx, label='dy/dx')\n", "plt.legend()\n", "_ = plt.xlabel('x')" ] }, { "cell_type": "markdown", "metadata": { "id": "DsOMSD_1BGkD" }, "source": [ "### 텐서 소스" ] }, { "cell_type": "markdown", "metadata": { "id": "g3iXKN7KF-st" }, "source": [ "입력이 스칼라이든 텐서이든 `tf.GradientTape.jacobian`은 대상(들)의 각 요소에 대한 소스의 각 요소의 그래디언트를 효율적으로 계산합니다.\n", "\n", "예를 들어, 이 레이어의 출력은 형상 `(10, 7)`입니다." ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:27.680910Z", "iopub.status.busy": "2022-12-14T21:28:27.680308Z", "iopub.status.idle": "2022-12-14T21:28:27.691108Z", "shell.execute_reply": "2022-12-14T21:28:27.690430Z" }, "id": "39YXItgLxMBk" }, "outputs": [ { "data": { "text/plain": [ "TensorShape([7, 10])" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x = tf.random.normal([7, 5])\n", "layer = tf.keras.layers.Dense(10, activation=tf.nn.relu)\n", "\n", "with tf.GradientTape(persistent=True) as tape:\n", " y = layer(x)\n", "\n", "y.shape" ] }, { "cell_type": "markdown", "metadata": { "id": "tshNRtfKuVP_" }, "source": [ "레이어의 커널 형상은 `(5, 10)`입니다." ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:27.694525Z", "iopub.status.busy": "2022-12-14T21:28:27.693913Z", "iopub.status.idle": "2022-12-14T21:28:27.698416Z", "shell.execute_reply": "2022-12-14T21:28:27.697619Z" }, "id": "CigTWyfPvPuv" }, "outputs": [ { "data": { "text/plain": [ "TensorShape([5, 10])" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "layer.kernel.shape" ] }, { "cell_type": "markdown", "metadata": { "id": "mN96JRpnAjpx" }, "source": [ "커널에 대한 출력의 야고비안 형상은 서로 연결된 두 가지 형상입니다." ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:27.701592Z", "iopub.status.busy": "2022-12-14T21:28:27.701027Z", "iopub.status.idle": "2022-12-14T21:28:27.814635Z", "shell.execute_reply": "2022-12-14T21:28:27.813763Z" }, "id": "pRLzTTbvEimH" }, "outputs": [ { "data": { "text/plain": [ "TensorShape([7, 10, 5, 10])" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "j = tape.jacobian(y, layer.kernel)\n", "j.shape" ] }, { "cell_type": "markdown", "metadata": { "id": "2Lrv7miMvTll" }, "source": [ "대상의 차원을 합하면 `tf.GradientTape.gradient`에서 계산한 합계의 그래디언트가 남습니다." ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:27.818196Z", "iopub.status.busy": "2022-12-14T21:28:27.817568Z", "iopub.status.idle": "2022-12-14T21:28:27.826607Z", "shell.execute_reply": "2022-12-14T21:28:27.825902Z" }, "id": "FJjZpYRnDjVa" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "g.shape: (5, 10)\n", "delta: 2.3841858e-07\n" ] } ], "source": [ "g = tape.gradient(y, layer.kernel)\n", "print('g.shape:', g.shape)\n", "\n", "j_sum = tf.reduce_sum(j, axis=[0, 1])\n", "delta = tf.reduce_max(abs(g - j_sum)).numpy()\n", "assert delta < 1e-3\n", "print('delta:', delta)" ] }, { "cell_type": "markdown", "metadata": { "id": "ZKajuGlk_krs" }, "source": [ " \n", "\n", "#### 예: Hessian" ] }, { "cell_type": "markdown", "metadata": { "id": "NYcsXeo8TDLi" }, "source": [ "`tf.GradientTape`는 [Hessian 행렬](https://en.wikipedia.org/wiki/Hessian_matrix)을 구성하기 위한 명시적인 방법을 제공하지 않지만 `tf.GradientTape.jacobian` 메서드를 사용하여 빌드할 수 있습니다.\n", "\n", "참고: Hessian 행렬은 `N **2` 매개변수를 포함합니다. 그 외 여러 이유로 인해 대부분의 모델에는 실용적이지 않습니다. 이 예제는 `GradientTape.jacobian` 메서드를 사용하는 방법에 대한 설명으로 포함되어 있으며 직접적인 Hessian 기반 최적화를 보증하는 것은 아닙니다. Hessian-vector 곱은 [중첩 테이프를 사용하여 효율적으로 계산](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/eager/benchmarks/resnet50/hvp_test.py)할 수 있으며 2차 최적화에 대한 훨씬 효율적인 접근 방식입니다." ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:27.829936Z", "iopub.status.busy": "2022-12-14T21:28:27.829366Z", "iopub.status.idle": "2022-12-14T21:28:28.103270Z", "shell.execute_reply": "2022-12-14T21:28:28.102584Z" }, "id": "ELGTaell_j81" }, "outputs": [], "source": [ "x = tf.random.normal([7, 5])\n", "layer1 = tf.keras.layers.Dense(8, activation=tf.nn.relu)\n", "layer2 = tf.keras.layers.Dense(6, activation=tf.nn.relu)\n", "\n", "with tf.GradientTape() as t2:\n", " with tf.GradientTape() as t1:\n", " x = layer1(x)\n", " x = layer2(x)\n", " loss = tf.reduce_mean(x**2)\n", "\n", " g = t1.gradient(loss, layer1.kernel)\n", "\n", "h = t2.jacobian(g, layer1.kernel)" ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:28.107308Z", "iopub.status.busy": "2022-12-14T21:28:28.106686Z", "iopub.status.idle": "2022-12-14T21:28:28.110741Z", "shell.execute_reply": "2022-12-14T21:28:28.109973Z" }, "id": "FVqQuZj4XGjm" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "layer.kernel.shape: (5, 8)\n", "h.shape: (5, 8, 5, 8)\n" ] } ], "source": [ "print(f'layer.kernel.shape: {layer1.kernel.shape}')\n", "print(f'h.shape: {h.shape}')" ] }, { "cell_type": "markdown", "metadata": { "id": "_M7XElgaiMeP" }, "source": [ "이 Hessian을 뉴턴의 방법 단계에 사용하려면, 먼저 축을 행렬로 평면화하고 그래디언트를 벡터로 평면화합니다." ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:28.114392Z", "iopub.status.busy": "2022-12-14T21:28:28.113758Z", "iopub.status.idle": "2022-12-14T21:28:28.121400Z", "shell.execute_reply": "2022-12-14T21:28:28.120828Z" }, "id": "6te7N6wVXwXX" }, "outputs": [], "source": [ "n_params = tf.reduce_prod(layer1.kernel.shape)\n", "\n", "g_vec = tf.reshape(g, [n_params, 1])\n", "h_mat = tf.reshape(h, [n_params, n_params])" ] }, { "cell_type": "markdown", "metadata": { "id": "L9rO8b-0mgOH" }, "source": [ "Hessian 행렬은 대칭이어야 합니다." ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:28.124395Z", "iopub.status.busy": "2022-12-14T21:28:28.124156Z", "iopub.status.idle": "2022-12-14T21:28:28.128134Z", "shell.execute_reply": "2022-12-14T21:28:28.127463Z" }, "id": "8TCHc7Vrf52S" }, "outputs": [], "source": [ "def imshow_zero_center(image, **kwargs):\n", " lim = tf.reduce_max(abs(image))\n", " plt.imshow(image, vmin=-lim, vmax=lim, cmap='seismic', **kwargs)\n", " plt.colorbar()" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:28.131206Z", "iopub.status.busy": "2022-12-14T21:28:28.130770Z", "iopub.status.idle": "2022-12-14T21:28:28.360798Z", "shell.execute_reply": "2022-12-14T21:28:28.359922Z" }, "id": "DExOxd7Ok2H0" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmcAAAH5CAYAAADENpm/AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABcTUlEQVR4nO3de3hU5bn//0+GMAkhJCEGEtIEQ0SNKQYEJMZSpJIKaN3ylfpVq1ukFr5a0Gp6ErcFiu2OVTdFWwqXrYd6bSlWf9VWd4tboYFtDVCCbEQERaGAYQIRkyGJSUhmfn9YolEOz82sMUPyfl3XupTJPXeetdYcntzrcMeFw+GwAAAAEBN8XT0AAAAAfIzJGQAAQAxhcgYAABBDmJwBAADEECZnAAAAMYTJGQAAQAxhcgYAABBD4rt6AJ8WCoVUXV2tfv36KS4urquHAwBATAuHwzp06JCys7Pl833+NZfm5ma1trZGJbff71diYmJUcseymJucVVdXKzc3t6uHAQDAKWXPnj3Kycn5XH9nc3OzBvTpo4Yo5c/KytLOnTt73AQt5iZn/fr1kyTdISnB8Tlj/lDvnH/CV0K2AV11lXPojp/9f6bU773nHntR84um3Lf910Tn2Idu22HKHcof6hzb3GxKraTGA7YnvP++c2hrfoEptV/ufwk2tflNuS3ruem9AabchYXusfffb0qtb37TFm/Z//HGT6Pdu91jv3TtYFPu4Bb35CnJxs+U//kf99j8fFPqLfXuf9gOy28y5f71k0mm+Bk3uW+X1jZbtcfy3lz3mu29WXy++7j319rG3d7uHtvY6B7b0BDURRfldnx/fp5aW1vVINt3tqsWST8PBNTa2srkzCuLFy/W/fffr0AgoOHDh+sXv/iFxowZc8LnHTmUmSD3Hd23b4rzuFJSjB+kvXs7hyYnu49Dkvr2dY9N8dk+GP1+wzZJTjblDqW45/bbPheV5DPO5lpanENbDeOWbF8A8dbJmWE9ra8ry2omGD9NrZ/9hrePeXJmev9YT5EwbETz5MwycOMGT263fBbaNnifPrbPIMtnbTQnZ3372t6blnF/2BK9ydnJnNXTlacC9ZHk9fSpJ58UH5V1f+qpp1RWVqZ58+Zp48aNGj58uCZOnKj9+/dH49cBAAB0G1GZnC1cuFAzZszQ9OnTVVhYqKVLlyopKUmPPvpoNH4dAADoQr4oLT2V5+ve2tqqqqoqlZaWfvxLfD6VlpaqsrLyM/EtLS0KBoOdFgAAgJ7K88lZbW2t2tvblZmZ2enxzMxMBQKBz8SXl5crNTW1Y+FKTQAATi1UzrzV5es+Z84c1dfXdyx79uzp6iEBAACDWJqcLV68WHl5eUpMTFRxcbHWr19/zNhf//rX+vKXv6z+/furf//+Ki0t/Ux8OBzW3LlzNWjQIPXp00elpaV6++23T3J0bjyfnGVkZKhXr16qqanp9HhNTY2ysrI+E5+QkKCUlJROCwAAgJX1gsSKigpde+21+utf/6rKykrl5ubqkksu0XufuNfVfffdp4ceekhLly7VunXr1LdvX02cOFHN1ntFGXg+OfP7/Ro1apRWrlzZ8VgoFNLKlStVUlLi9a8DAABdLFYqZ9YLEp988kl9+9vf1ogRI1RQUKDf/OY3HXMW6aOq2aJFi3T33XfriiuuUFFRkZ544glVV1frueeeO4kRuonKYc2ysjL9+te/1m9/+1u9+eabuuWWW9TY2Kjp06dH49cBAIBu6tMXDbYc4/6W1gsSj6apqUmHDx9Wenq6JGnnzp0KBAKdcqampqq4uNg558mIyk1or776ah04cEBz585VIBDQiBEjtGLFis9cJAAAAE590TiB/0i+T18oOG/ePM2fP/8z8ce7IHHbtm1Ov/OHP/yhsrOzOyZjRy5kdL3I0StR6xAwe/ZszZ49+6SfP+YP9c53/j9rouGuyNu3m8YReuHPzrFn7XjLlDtjxFnOse/WXWbK/fCcIufY0NDNpty+3z3pHJv01a+acremDTTFW+797Y+33cn9YJ179nQdNOXeuNd9PUf+Z5kp99ZvLXSO/d73TKnN0tLcY2trbbnHvTzXOTb4jw9MuVO2vOocuzn5QlPunBEXO8c2GBsWDnXvrCYZ2+F8O/kJU3xr2w3OsdZ936uX+3vzggtsuWsOuE8x+ve35fY3G24V1VbnHBo8fMg2kFPMnj17Op2PnmBtbeLo3nvv1fLly1VRUdHl7aJirrcmAAA4tUSzcuZ6saD1gsRPeuCBB3Tvvffq5ZdfVlHRx8WNI8+rqanRoEGDOuUcMWKE24qchC6/lQYAAECkTvaCxPvuu0/33HOPVqxYodGjR3f62ZAhQ5SVldUpZzAY1Lp166J6kSOVMwAAEJFoVs4sysrKNG3aNI0ePVpjxozRokWLOl2QeMMNN+gLX/iCysvLJUk/+9nPNHfuXC1btkx5eXkd55ElJycrOTlZcXFxuv322/WTn/xEZ555poYMGaIf/ehHys7O1pQpUzxa089icgYAALqFE12QuHv3bvl8H0/7lixZotbWVn3961/vlOeTFx384Ac/UGNjo2bOnKm6ujqNHTtWK1asiOp5aUzOAABAROLkfeXMcKlfJ8e7ILGioqLTv3ft2nXiccTFacGCBVqwYMFJjsiOyRkAAIhInE5+MnW8nD0VFwQAAADEECpnAAAgIr3+uXids6eicgYAABBDqJwBAICIxMqtNLqLuHA4HO7qQXxSMBhUamqq6j/4wOmOwJKkHTvcf8Hbb9sGdN55zqEHE7NNqdPT3NsJhYwvU1+g2jm2Kc027qSG/c6xNWFbO6YBA0zh8tW6j0UZGbbkht45TfGOr9V/ijf8WWR5eUvSpy5GOq5Jk2y58+N3256Qk+Me29xsSv3W3iTn2LOG2lp3bd7i/n778ENTauXlucdmDrCN26K1zfaZ8oGtA5Yy+7c6x9Z8YGnEJvXp4x5rvduB5b3pk23/WLe5q2AwqAEDUlVfX+/+venh705NTdXPJRl2i5MPJd0hdcl6dTUqZwAAICJUzrzVk9cdAAAg5lA5AwAAEaFy5i0mZwAAICJMzrzVk9cdAAAg5lA5AwAAEaFy5q2evO4AAAAxh8oZAACICJUzb/XkdQcAAIg5VM4AAEBE4v65eJ2zp6JyBgAAEENit3J21VVS795OoaEX/uyc1pecbBvHiBHOoRv+09DnUdLYse5z46SlC0259a1vuec29MqUpNY0936ZA4yvMN8LfzLFvzvsX5xjs4y7Pqm21j022dYXcu4v3bfh975nSm3ql/ncc7bcZdfbGhVaesJu2ubeK1OSCgoMwZaGo5JyRlzsHHv4sCm1Ml9+0jn24OTrTLktH2/+TetNudtzxpjiX93g3i/zggtMqdXW5h77wgu23FeOjV6/3l273GOzstxjjW1po8InqVcUcvZUsTs5AwAApwQuCPBWT153AACAmEPlDAAARITKmbd68roDAADEHCpnAAAgIlTOvNWT1x0AACDmUDkDAAARoXLmrZ687gAAADGHyhkAAIgIlTNvMTkDAAARYXLmrbhwOBzu6kF8UjAYVGpqqqqq6pWcnOL0nLP0lnP+gxlnmcazYYN77LiJtjat/nb3Te/b4b6OkvSHLe7reWXBVlNua8sSi3cb3NsaSVL+pj84x4amXGnK7Wtrdc8d796qRpICAfdYa8exlLrd7sGJtnZM1sGEEt1bMvn2GsYt2frbWDa4pN0a7Bw7OCdkyn2wzv0r56WXTKl11VXusZYWSJLk/89HbU+48Ubn0JoDtq/hAQPcY321tvZ0Dz/n/hk081u2fR9scF/PlID7Z36woUGpo0apvr5eKSlu35teOfKdvVySrQHbiTVJukbqkvXqalTOAABAROL+uXids6fqyVVDAACAmEPlDAAARKTXPxevc/ZUVM4AAABiCJUzAAAQkTh5X+3hnDMAAADEBCpnAAAgItznzFtMzgAAQESYnHmrJ687AABAzKFyBgAAIkLlzFs9ed0BAABiTsxWzt57T+rb1y02Y4R7H8n0NFsvtLFj3eevll6ZkuS7/hvOsa2PLzPlvvIV9z54rUO/acrtj7dtQ4ssYx9JS79M35bNptytBUXOsf7mJlPu5mb3LnRpaabUUk6Oc2goyn+f+V74k3Ns6Gv/Ysu9aaNz7Ob4kabcQ4eawk0s7UktvTIlyffAfe7Bt//AlrygwBRu6ZfZv79tKD4ZPoOMvYBN/TKNPVtTLP1gLW/8Xl1/u1YqZ97qyesOAAAQczyfnM2fP19xcXGdlgLjX1wAAODU4YvS0lNF5bDmF7/4Rb388ssf/5L4mD16CgAAEFOiMjGNj49XVlZWx5JhPOYPAABOHXFRWk7G4sWLlZeXp8TERBUXF2v9+vXHjH3jjTc0depU5eXlKS4uTosWLfpMTFccEYzK5Oztt99Wdna28vPzdd1112n37t3HjG1paVEwGOy0AACAU0evKC1WTz31lMrKyjRv3jxt3LhRw4cP18SJE7V///6jxjc1NSk/P1/33nuvso5zwcYXv/hF7du3r2N55ZVXTmJ07jyfnBUXF+vxxx/XihUrtGTJEu3cuVNf/vKXdejQoaPGl5eXKzU1tWPJzc31ekgAAKAHWLhwoWbMmKHp06ersLBQS5cuVVJSkh599Oh3MDj//PN1//3365prrlFCQsIx837eRwQ9n5xNnjxZV111lYqKijRx4kT9+c9/Vl1dnX7/+98fNX7OnDmqr6/vWPbs2eP1kAAAQBRF84KATx9da2lpOeoYWltbVVVVpdLS0o/H5fOptLRUlZWVEa2f5YigF6J+MURaWprOOuss7dix46g/T0hIUEpKSqcFAABAknJzczsdYSsvLz9qXG1trdrb25WZmdnp8czMTAWM96T7JOsRQS9E/TLKhoYGvfPOO/rXf/3XaP8qAADQBeLkfbXnyAUBe/bs6VS4Od7hx2iYPHlyx/8XFRWpuLhYp59+un7/+9/rpptuisrv9Hxy9r3vfU+XX365Tj/9dFVXV2vevHnq1auXrr32Wq9/FQAA6OZcj6plZGSoV69eqqmp6fR4TU3NcU/2tzrREUEveD4527t3r6699lq9//77GjBggMaOHau1a9dqwIABpjwXNb+oFJ9bi5t36y5zzpuWZpvbJy1d6B78ta+ZcltaMlUk2C4qHh5wbyWVqVZT7oN1fufY9Hjb1bdJtbWmeEurIks7JknyL3/CPXjSJFPu+Hj39k3WTTI4o9k5dtM293FI0sgM23kWlpZMFRWm1Boxwr0lU1Girb2WEhOdQ1vbbJ8p/k3Hvqz/M7lHjDHltrRk8j939POAj+kT5/G4aHd/GcrfbPucaE10P/1l1y5Talm+w03tmCSVfc/9tbLwAcMJ5373z+RoiYX2TX6/X6NGjdLKlSs1ZcoUSVIoFNLKlSs1e/Zsz8b1eRwR9Hxytnz5cq9TAgAAnFBZWZmmTZum0aNHa8yYMVq0aJEaGxs1ffp0SdINN9ygL3zhCx3nrbW2tmrr1q0d///ee+9p06ZNSk5O1tB/NtntiiOC3LofAABEJBYqZ5J09dVX68CBA5o7d64CgYBGjBihFStWdFwksHv3bvl8H2eurq7Weeed1/HvBx54QA888IAuuugiVfyznO/VEUELJmcAACAisTI5k6TZs2cf8zBmxafOn8jLy1M4fPzTgLriiGBP7isKAAAQc6icAQCAiMRS5aw76MnrDgAAEHOonAEAgIjE6eObxnqZs6eicgYAABBDqJwBAICI9Prn4nXOnorKGQAAQAyhcgYAACLC1Zreiguf6O5rn7NgMKjU1FRNm1Yvv9+tf9rDaw09E1essA0oOdk59A8vu/d7k6Qr6x51jq257Jum3P+b5X4q5dhG20sgqfmgc2xTYrotd8N+U3woY6BzrK/Z2F+xocE9Ni3NlHrNWvdeeONenmvK/db1C5xjDa1JJUlJ8bY+rNqyxTn0YJ57r0xJSr/9BufYX11g6JMq6dvJ7vE1E93HIUnt7e6x2SvcPyMkSQUF0YmVpCVLTOFvXfVvzrFnJdp6trZmDXaObTb0+JSklMBb7sHG970yDP0yDe+dYEODUr/0JdXX1zs1CPfSke/sjZLcvy3dNEgaKXXJenW1njwxBQAAiDkc1gQAABGJk/fVHm6lAQAAgJhA5QwAAESECwK81ZPXHQAAIOZQOQMAABGhcuatnrzuAAAAMYfKGQAAiAiVM2/15HUHAACIOVTOAABAROLk/X3JevJ9zmK2fVN9VZVSHFsnhYae5Zzf2srD1E6ottaUu3VooXOsX7a2OU1t7u2Bkm62tZ8JPe7e2sa3baspt5591hQ+t8W9Rczdd9uG4t/hPvZ3E933pSTlB151D77gAlNu7d3rHltXZ0r9p12GVmmSvvY191hfQ9CU+91a93YuxtU0dTZqa7Pl/vBD99jMxndtybOynENb45NMqf3xIVP8Q790PzAze7YptamzmqELkiTpwuTN7sFDh5pyH2x23+bpie7t5oLBoFIHDerS9k3bJfXzOPchSWeL9k0AAADoYhzWBAAAEeGCAG/15HUHAACIOVTOAABARKiceasnrzsAAEDMoXIGAAAiQuXMWz153QEAAGIOlTMAABCROHlf7enJN6FlcgYAACLCYU1v9eR1BwAAiDlUzgAAQESonHkrZntrfvCBey8t3++edP8FX/2qaTytaQOdY/11hj6ckpSR4Rx6sM72Mk3XQefYUFq6KbfvL//lHvyVr5hym/v9Nbivp9LSTLlrDrhv88wBtr6DFq1ttn1vafFq3CTmPpIpye7bJWT8KPa1ufebtfSalaSkbRudY6uzRppyO7YMliQlJppSm/pfWl7fkpQZZ/t8C2W4f3YGAqbUpm1oeQ1KUlOz+3ax7h/La9YiGAwqdcCALu2tuVeS1785KClHPbO3JpUzAAAQESpn3urJ6w4AABBzqJwBAICIUDnzVk9edwAAgJhD5QwAAESEypm3mJwBAICIMDnzVk9edwAAgJhD5QwAAESEypm3evK6AwAAxBwqZwAAICJUzrwVs5Oz5mbJ79hxJcnQkqkm7N5SRJIGxMgWSo8PmuKb4t1bMiVt22objKUl0xlnmFL7//hHU/zDm8Y4x878lq2Ny3vvucdm/u05U26NHu0cWhs/2JQ6e/lC9+DrrzflXrPN9v4ZN9Y9dtcuU2rlN+9wjm3LKTTltrRkWr7clFpf/7p77OAG43uzoMA5NLO3ofWZpK0B277PMbQ2srRjkqRnnnGPnTTJ9hWfXWfY5obtLUlqaHAOfavW/TO8ocHWngyxL0amHgAA4FQVJykuLs7bnLHV+vtz1ZOrhgAAoJtZvHix8vLylJiYqOLiYq1fv/6YsW+88YamTp2qvLw8xcXFadGiRRHn9IJ5crZmzRpdfvnlys7OVlxcnJ577rlOPw+Hw5o7d64GDRqkPn36qLS0VG+//bZX4wUAALEmPj46i9FTTz2lsrIyzZs3Txs3btTw4cM1ceJE7d+//6jxTU1Nys/P17333qusrCxPcnrBPDlrbGzU8OHDtXjx4qP+/L777tNDDz2kpUuXat26derbt68mTpyo5ubmiAcLAABiUIxMzhYuXKgZM2Zo+vTpKiws1NKlS5WUlKRHH330qPHnn3++7r//fl1zzTVKSEjwJKcXzGs+efJkTZ48+ag/C4fDWrRoke6++25dccUVkqQnnnhCmZmZeu6553TNNddENloAANCjBIOdL4hLSEg46kSqtbVVVVVVmjNnTsdjPp9PpaWlqqysPKnfHY2cLjw952znzp0KBAIqLS3teCw1NVXFxcXHXImWlhYFg8FOCwAAOIVEsXKWm5ur1NTUjqW8vPyoQ6itrVV7e7syMzM7PZ6ZmalAIHBSqxWNnC48vVrzyEAtK1FeXq4f//jHXg4DAAB0E3v27FFKSkrHv491+LE76fJbacyZM0dlZWUd/w4Gg8rNze3CEQEAAJP4eMnjW2non7fSSElJ6TQ5O5aMjAz16tVLNTU1nR6vqak55sn+XZHThaeHNY8M1LISCQkJHRvedQcAAAB8kt/v16hRo7Ry5cqOx0KhkFauXKmSkpKYyenC08rZkCFDlJWVpZUrV2rEiBGSPqqErVu3TrfccouXvwoAAMSKXr0kn8e3Tg3ZurpIUllZmaZNm6bRo0drzJgxWrRokRobGzV9+nRJ0g033KAvfOELHeettba2auvWrR3//95772nTpk1KTk7W0KFDnXJGg3ly1tDQoB07Pm6bsnPnTm3atEnp6ekaPHiwbr/9dv3kJz/RmWeeqSFDhuhHP/qRsrOzNWXKFC/HDQAA0MnVV1+tAwcOaO7cuQoEAhoxYoRWrFjRcS787t275fvEJLK6ulrnnXdex78feOABPfDAA7roootUUVHhlDMa4sJhW3+EiooKfeUovRWnTZumxx9/XOFwWPPmzdPDDz+suro6jR07Vr/61a901llnOeUPBoNKTU1V/Y4dSunXz+k5rWnuPd+st03xvfAn59h3h/2LKbflcHVS4F1TblOzul//2pS69fv/5hzr32S8i/Ix7s58LD/IWeYcO3++bShJK/7gHLu14EpT7sK2zc6xTUOLTLmTGgw3Rtzh3p9SklY1X2iKHz/ePda3d7cp9+Y6956j1s/QPn3cY+vqbLkHpxmuSDd+YLXGJznH+tuaTLmtY3l1g3u/xwsvsFVIqgPuFZoNG0yp9S8Zr7oHX3CBKfdbO9zHfVaW++skGAwqNTdX9fX1n/upQR3f2ZmZSvG4chYMhZRaU9Ml69XVzJWz8ePH63jzubi4OC1YsEALFiyIaGAAAOAUER8fE4c1uwt6awIAAMSQLr+VBgAAOMVROfMUlTMAAIAYQuUMAABEplevjxYvtbd7m+8UQuUMAAAghlA5AwAAkYmP975y5nU7qFMIlTMAAIAYQuUMAABEhsqZp5icAQCAyDA58xSHNQEAAGJI7FbO3n9famlxCnXv4GZn6ZeZv8m9F6MkhaYY+jHm5Nhyx7tvlfkt7r0yJWlBw0Hn2Ic3jTHl3mHolSlJ37jf/S+rxHtNbWRNzU+NLSq1Q+79Msfn2XKv2ebea7atzT1Wki7e8pApPjT+NvdgY+/GomGGG1Tu2mXK3do/3zl2cMNWU25l5DmH1hxy75UpSQMGGILbTKnN2/DC5Gbn2KZmW//Y7Dr3bf4vGXWm3GXPuPePXWjsCZqRYQiurXWPPXTINI6ooHLmKSpnAAAAMSR2K2cAAODU0KuXufqNY6NyBgAAEEOY5gIAgMjEx1M58xCVMwAAgBjCNBcAAESGypmn2JIAACAyTM48xWFNAACAGMI0FwAARCYat9IIG28c3o1QOQMAAIghceFwbE1Ng8GgUlNTdeBAvVJSUpye44+3tdCwaGp2n78mJtpy+7Zsdo5tLbC1N/G3Nbnnjre1iImV7S3Ztrlvm63NTuvQQudY6zYJGf4u8il629syjpPhq93vHBvKsLWS8rW1uge3GXsVWd/MBq1t7tvcWojwNUfxfS/D9pZM2zyUaBtLNN8TFu/usr1/8vOiM+5gMKjU/v1VX+/+venp705NVf2kSUrp3dvb3IcPK3XFii5Zr65G5QwAACCGcM4ZAACITDSu1oytA3ufKypnAAAAMYTKGQAAiAyVM08xOQMAAJFhcuYpDmsCAADEECpnAAAgMtG4CW0oNm6Z0hWonAEAAMQQKmcAACAy0TjnjMoZAAAAYgGVMwAAEBkqZ56K2cmZX63OvdwO1vmd86bHB03jSKqtdQ/OyTHltvTL9C9/wpRbkya5567dZUpdc5p7z8n33jOl1shdf7A9ISvLObR19IWm1P6EOOfYNattl3yPG+H+Ogwl23rK7drlHpsfv9uU2/rhG8rKdo71/eW/TLnfPecy59icHPfPCEn64IB7bGbvg6bcfkvfTmNLUEu/TP+WjbbkQ4eawg+2ub9u0+ONfTsbGpxD36pNN6XOyHCPtfbKfOiX7gerbrvGvS+tDh0yjQOxL2YnZwAA4BRB5cxTnHMGAAAQQ6icAQCAyFA58xSTMwAAEJlo3IS2vd3bfKcQDmsCAADEECpnAAAgMtE4rEnlDAAAALGAyhkAAIgMlTNPUTkDAACIIUzOAABAZI5UzrxeTsLixYuVl5enxMREFRcXa/369ceNf/rpp1VQUKDExESde+65+vOf/9zp5zfeeKPi4uI6LZMMXXhORswe1mxq8yu+za3lSrrc26c0xdtaeSQlNzvHhuJtLWL8zU3uwdYXQlqac+i7DQNNqfMHuN97JvNvz5lyby240hS/Y4d77L/E2+6ZY2nJNO7xb5pyv3v3o86x+Ym21jb5ze4bZXObeysuSSoaZtuGvjb3sVvaMUlS/obfuwfnfd2UOzPOvW3b1oDt/VM41LA/Lb24JPnz8tyDje2Y3q21tRHLzzJ8vhm/iiwtmc7KsrXsk6VlX1qeKbWlJdPGve6vq4YGQ0uwbu6pp55SWVmZli5dquLiYi1atEgTJ07U9u3bNXDgZ7fpq6++qmuvvVbl5eX62te+pmXLlmnKlCnauHGjhg0b1hE3adIkPfbYYx3/TkhIiOp6UDkDAACROXKfMy+XXr3Mw1i4cKFmzJih6dOnq7CwUEuXLlVSUpIeffTofxA/+OCDmjRpkr7//e/rnHPO0T333KORI0fql7/8Zae4hIQEZWVldSz9+/c/qc3kiskZAACITBQPawaDwU5LS0vLUYfQ2tqqqqoqlZaWdjzm8/lUWlqqysrKoz6nsrKyU7wkTZw48TPxFRUVGjhwoM4++2zdcsstev/99yPZWidknpytWbNGl19+ubKzsxUXF6fnnnuu08+74tgsAADonnJzc5WamtqxlJeXHzWutrZW7e3tyszM7PR4ZmamAoHAUZ8TCAROGD9p0iQ98cQTWrlypX72s59p9erVmjx5stqjeDWp+ZyzxsZGDR8+XN/85jd15ZVHPz/o8z42CwAAulA0bqXxz3x79uxRSsrH5zx+3nOKa665puP/zz33XBUVFemMM85QRUWFJkyYEJXfad6SkydP1uTJk48bc+TYrIuWlpZOJcpg0HjyJgAA6LZSUlI6Tc6OJSMjQ7169VJNTU2nx2tqao45J8nKyjLFS1J+fr4yMjK0Y8eOqE3OonLOmeXYbHl5eadyZW5ubjSGBAAAoiUGbqXh9/s1atQorVy5suOxUCiklStXqqSk5KjPKSkp6RQvSS+99NIx4yVp7969ev/99zVo0CDT+Cw8n5xZj83OmTNH9fX1HcuePXu8HhIAAOgBysrK9Otf/1q//e1v9eabb+qWW25RY2Ojpk+fLkm64YYbNGfOnI7473znO1qxYoX+4z/+Q9u2bdP8+fO1YcMGzZ49W5LU0NCg73//+1q7dq127dqllStX6oorrtDQoUM1ceLEqK2H5/c5sx6bTUhI4Jw0AABOZVE858zi6quv1oEDBzR37lwFAgGNGDFCK1as6Djpf/fu3fL5Pq5LXXjhhVq2bJnuvvtu3XXXXTrzzDP13HPPddzjrFevXtq8ebN++9vfqq6uTtnZ2brkkkt0zz33RHXuEvWb0H4ex2YBAAAkafbs2R2Vr0+rqKj4zGNXXXWVrrrqqqPG9+nTRy+++KKXw3MS9cnZ53FsFgAAdKEjN6H1OmcPZd6SDQ0N2vGJnjk7d+7Upk2blJ6ervT0dP34xz/W1KlTlZWVpXfeeUc/+MEPon5sFgAAdKEYOazZXZjXfMOGDfrKV77S8e+ysjJJ0rRp07RkyRLPjs0mNR5Qks+tr6WlB9knWmU5mftL99w332zL3dyc5BwbH+8eK0m71rrHjot/1ZRbeRe4x44ebUpdWLfZFL9DRc6xIeP1L+NGuN/WxdIrU5Iyzohzjl231r3HpyQVD8lwji369U9NuW9Z8m+m+J/+1L3fbP6uVabcb434v86xDZtMqTVyqHuvQks7S0las9Z9m4zbVmHKXXPFTOfY99+39QI+dMgUrvzkBvdgQy9gScrJMQS3mVJLx7hZ6dG05uSbUsdnuH+fGF6C4g5U3Y95cjZ+/HiFw8f+suiKY7MAAKALUTnzFL01AQAAYkjPnZYCAABvUDnzFJUzAACAGNJzp6UAAMAb3ErDU1TOAAAAYgiVMwAAEBnOOfNUz11zAADgDSZnnuKwJgAAQAzpudNSAADgDSpnnooLH+92/10gGAwqNTVVq1fXKzk5xek5I/+zzDn/1m8tNI3H1CbEyPK6q6215R78m7nuwfPnm3K3trkXXK3jNnZxUZuhNUtKcsiU29LuydfWasq97jX31jnF/3qWKbeWL3ePLSiw5d62zRTeOmykc+wnWvY6KVxraJn19a/bklteuNYX+YgRUcvdmpHtHOuvrTblVnKyLT7R0H/I+iVsaLGkDPd2ZpJMHyqhRFtbPV+Doc+SYXsHg0Gl9u+v+vp6paS4fW965ch3dv3vfqeUJNv2OGHupialXnttl6xXV+u501IAAOANKmee4pwzAACAGNJzp6UAAMAb3ITWU1TOAAAAYgiVMwAAEBnOOfNUz11zAADgDSZnnuKwJgAAQAzpudNSAADgDSpnnqJyBgAAEEN67rQUAAB4g1tpeIrKGQAAQAyJ2cpZYaHk2krL0i+zosI2jkmT3GPz43fbkhsadw7OaDalfuv6Bc6xZ+21jbs2frBzbPZyWy9TXX+9KXzNtoHOsePG2oaya5d7bH6zrTFk8RBDvz9Lr0xJuvNO99if/MSUek3zGFP8uHj3fqZpaca/FQ3vn6Z4W1++pFpDD1Fr78aGBvdYY7NZv2F7m3tlGjW1ufePTTL2+Qwmu/cQTbFsE8m0f3yW/qFGlh7Gltio4ZwzT8XAHgUAAMARPXdaCgAAvEHlzFM9d80BAIA3mJx5isOaAAAAMaTnTksBAIA3qJx5isoZAABADOm501IAAOANbkLrKSpnAAAAMYTKGQAAiAznnHmKyhkAAEAMidlp6f33SwkJbrHf+557Xks7Jkl67jn32LLrba08Qoa58aZtSabcBQWG4B11ptxpQ93bN1nbMWmHrQ1SW5t7+ybL9pZs7bg2txWachf9+qfuwXfcYcptasn0zDOm1HVjbe2bLNs8e8caW+7SS5xjN7xiSq1xF4xwD7a0Y5JU3ZzuHNvrsCm1Bhg+gg58aGtplVn/lik+KcsQbGyBZWrJZNw/Ua3WGFpm+QPuLa38hw6dzGi8ReXMUz13zQEAgDeYnHmKw5oAAAAxpOdOSwEAgDe4lYanqJwBAADEECpnAAAgMpxz5ikqZwAAADGk505LAQCAN6iceYrKGQAA6DYWL16svLw8JSYmqri4WOvXrz9u/NNPP62CggIlJibq3HPP1Z///OdOPw+Hw5o7d64GDRqkPn36qLS0VG+//XY0V4HJGQAAiNCRypnXi9FTTz2lsrIyzZs3Txs3btTw4cM1ceJE7d+//6jxr776qq699lrddNNNeu211zRlyhRNmTJFW7Zs6Yi577779NBDD2np0qVat26d+vbtq4kTJ6q5ufmkN9eJMDkDAACRiZHJ2cKFCzVjxgxNnz5dhYWFWrp0qZKSkvToo48eNf7BBx/UpEmT9P3vf1/nnHOO7rnnHo0cOVK//OUvJX1UNVu0aJHuvvtuXXHFFSoqKtITTzyh6upqPWdpIWTE5AwAAMSsYDDYaWlpaTlqXGtrq6qqqlRaWtrxmM/nU2lpqSorK4/6nMrKyk7xkjRx4sSO+J07dyoQCHSKSU1NVXFx8TFzeiFmz7b75jelfv28z2vplygZ+2Ua+qZZjcywjVvx7o3t/rSryJR6fJ577Jpt7r0vJVuvTEm6eMtDzrGh8beZclv+aisaZuj1J+mWJf/mHLtk20ZT7jXN7v0vzb0yr4gzxas97B5rPETwqc/T41p1zcOm3Br6NffYtDRTaku/zJoaU2oNGOAe+/77ttyZybbewfrEYaETGjHCltvSL9NYfVm1wb3n6MXjbe/76oCh16zl+yRkG0c0hOQz9y92ySlJubm5nR6fN2+e5s+f/5n42tpatbe3KzMzs9PjmZmZ2rZt21F/RyAQOGp8IBDo+PmRx44VEw0xOzkDAADYs2ePUlI+njQnJCR04Wg+H6Zpbnl5uc4//3z169dPAwcO1JQpU7R9+/ZOMc3NzZo1a5ZOO+00JScna+rUqaqx/vkHAABOGW1t0VkkKSUlpdNyrMlZRkaGevXq9Zk5R01NjbKyjn40KSsr67jxR/5ryekF0+Rs9erVmjVrltauXauXXnpJhw8f1iWXXKLGxsaOmDvuuEPPP/+8nn76aa1evVrV1dW68sorPR84AADAEX6/X6NGjdLKlSs7HguFQlq5cqVKSkqO+pySkpJO8ZL00ksvdcQPGTJEWVlZnWKCwaDWrVt3zJxeMB3WXLFiRad/P/744xo4cKCqqqo0btw41dfX65FHHtGyZct08cUXS5Iee+wxnXPOOVq7dq0uuOAC70YOAABiwicrXV7mtCorK9O0adM0evRojRkzRosWLVJjY6OmT58uSbrhhhv0hS98QeXl5ZKk73znO7rooov0H//xH7rsssu0fPlybdiwQQ8//NF5qnFxcbr99tv1k5/8RGeeeaaGDBmiH/3oR8rOztaUKVO8WtXPiOics/r6eklSenq6JKmqqkqHDx/udFVDQUGBBg8erMrKyqNOzlpaWjpdeREMBiMZEgAA6KGuvvpqHThwQHPnzlUgENCIESO0YsWKjhP6d+/eLZ/v44OGF154oZYtW6a7775bd911l84880w999xzGjZsWEfMD37wAzU2NmrmzJmqq6vT2LFjtWLFCiUmGi+SMTjpyVkoFNLtt9+uL33pSx0rEQgE5Pf7lfapq5eOd1VDeXm5fvzjH5/sMAAAQBeLlcqZJM2ePVuzZ88+6s8qKio+89hVV12lq6666pj54uLitGDBAi1YsODkBnQSTvq611mzZmnLli1avnx5RAOYM2eO6uvrO5Y9e/ZElA8AAHy+onlBQE90UpWz2bNn64UXXtCaNWuUk5PT8XhWVpZaW1tVV1fXqXp2vKsaEhISesRlsQAAAC5MlbNwOKzZs2fr2Wef1apVqzRkyJBOPx81apR69+7d6aqG7du3a/fu3VG9qgEAAHSd9nbvq2bt7V29Vl3HVDmbNWuWli1bpj/+8Y/q169fx3lkqamp6tOnj1JTU3XTTTeprKxM6enpSklJ0a233qqSkhKu1AQAAHBgmpwtWbJEkjR+/PhOjz/22GO68cYbJUk///nP5fP5NHXqVLW0tGjixIn61a9+5clgAQBA7ImlCwK6g7hwOGxofhd9wWBQqamp2rSpXv36ufU4s9ykNynR1oPM615hn+R74U/u4/jav9hyb3LvxxgaMdKWW9Hr4xbV7V273xQfynDv8+lrazXlPtjgd461tmz1x7vvn2hub0ny7XjLOTY09Cxb7kC1c2xNr2xT7v793WMt21uK8ms8mu/7vbb+vq1Zg51jje0vo/oZZGKdPVhW1NBrNhgMKnXQINXX13dqc/R5OPKdvW2b+3e2q0OHgiooSO2S9epq9NYEAAARoXLmrej+2QwAAAATKmcAACAiVM68ReUMAAAghlA5AwAAETlynzOvc/ZUTM4AAEBEOKzpLQ5rAgAAxBAqZwAAICJUzrxF5QwAACCGUDkDAAARoXLmrZidnMXHu3e6qK11zzs4w70lhiRt2pbkHDsyw9bexNKSqaLClFojDK1Z0huCptyhZPc2Grt2mVIrP962DS3tUEJZthY+vr/8l3Psu+dcZsqdv2uVc+zWrItNudPS3Avi2TvWmHJbWspIUqj0EufYt3rFmXK/9Uf3znNDh5pSK7PdvTWUtb/WgQ/d3z/vv29KrQLD+7601JZ71U/2muLjc9zbN/mMn0Emxv1THTC8f7JsX6Fv7XDPfVai4Yvt0CHTOBD7YnZyBgAATg1UzrzFOWcAAAAxhMoZAACICDeh9RaTMwAAEBEOa3qLw5oAAAAxhMoZAACICJUzb1E5AwAAiCFUzgAAQESonHmLyhkAAEAMoXIGAAAiwq00vEXlDAAAIIbEbOVs926pb1+32HEvz3XO+9b1C0zjKCgwBMdnmXL7Nm10jrX0ypSk9NtvcI59d/4Tptz5ia3usc07TLk3txWa4ouGhZxjfW3u45Zs/TLzN/zelPutEf/XObbwlUdNuZWT4xxq6X0pnUQ/xmHuPSotvTIlaewV7r04315ry60cWz9Gi8z6t9xjkxNtyQ3tL629MpWXZwpvaHCPTTH2v2xtc68r+AOGPqmSsi1jabZ9hVr6ZQbT3HuTBn1R7E3qiHPOvBWzkzMAAHBqYHLmLQ5rAgAAxBAqZwAAICJUzrxF5QwAACCGUDkDAAARoXLmLSpnAAAAMYTKGQAAiAg3ofUWlTMAAIAYQuUMAABEhHPOvMXkDAAARITJmbdidnL2pWsHKyXOrT1L8B8fOOc9K9m93Y8kqaLCPXboUFPqzfHuLZmKEptMuX91gXtLpgvqTKnVlOV3jm3LsbVjyvzQNhbt2uUem2Vrr5WT476eyvu6KXfDJkPw1225m+JTnGM3vGJKrVXXPGyKr+k10znW+PYxtWQ6fIF7qydJ2vy/7rmLCmxtwZTo3pLpz1vcW/hIUrzhE/0SS2s6SRsD2ab4kVOHOMcerNppyp2e7L7Nm9Js495r6Gr1gftXjySp+Hz31mptde55e/IkpruK2ckZAAA4NVA58xYXBAAAAMQQKmcAACAi3ErDW1TOAAAAYgiVMwAAEBHOOfMWlTMAANDjHDx4UNddd51SUlKUlpamm266SQ0NDcd9TnNzs2bNmqXTTjtNycnJmjp1qmpqajrFxMXFfWZZvny5aWxMzgAAQESOVM68XqLpuuuu0xtvvKGXXnpJL7zwgtasWaOZM49/+5877rhDzz//vJ5++mmtXr1a1dXVuvLKKz8T99hjj2nfvn0dy5QpU0xj47AmAACISDQPawaDwU6PJyQkKCEhIaLcb775plasWKG///3vGj16tCTpF7/4hS699FI98MADys7+7P3x6uvr9cgjj2jZsmW6+OKLJX00CTvnnHO0du1aXXDBBR2xaWlpyjLeW/OTqJwBAICYlZubq9TU1I6lvLw84pyVlZVKS0vrmJhJUmlpqXw+n9atW3fU51RVVenw4cMqLS3teKygoECDBw9WZWVlp9hZs2YpIyNDY8aM0aOPPqpw2P3G1hKVMwAAEKFoVs727NmjlJSPO59EWjWTpEAgoIEDB3Z6LD4+Xunp6QoEAsd8jt/vV1paWqfHMzMzOz1nwYIFuvjii5WUlKT//u//1re//W01NDTotttucx4fkzMAABCzUlJSOk3OjufOO+/Uz372s+PGvPnmm14M65h+9KMfdfz/eeedp8bGRt1///3dY3IW3LJbctwZKVtedc67OflC0zhyRlzsHHuCizw+w9RL0NCPT5K+nezeW7Op4AZT7qRtG51jq7Pc+4dKUnKyKVyt/fOdY/3xtr6qHxxwj82MqzXlHjnUsD9rbbmTarc5x467YIQpt4Z+zRTev797bGZ7tW0sOe4vFkuvTEkqusgw8P/6L1NufeIwyolcGlhvSt06Yoxz7Fu7bH07R26w9VW19MtM37LGlFt5ec6hSRkZptRnZTQ7x4aGpptyhwxnEqXXvescG3/okGkc0RArN6H97ne/qxtvvPG4Mfn5+crKytL+/fs7Pd7W1qaDBw8e81yxrKwstba2qq6urlP1rKam5rjnlxUXF+uee+5RS0uLc9UvZidnAAAAFgMGDNCAAQNOGFdSUqK6ujpVVVVp1KhRkqRVq1YpFAqpuLj4qM8ZNWqUevfurZUrV2rq1KmSpO3bt2v37t0qKSk55u/atGmT+vfvbzoca7ogoLy8XOeff7769eungQMHasqUKdq+fXunmPHjx3/m/h4333yz5dcAAIBTyKl2K41zzjlHkyZN0owZM7R+/Xr97W9/0+zZs3XNNdd0XKn53nvvqaCgQOvXf1TBTk1N1U033aSysjL99a9/VVVVlaZPn66SkpKOKzWff/55/eY3v9GWLVu0Y8cOLVmyRP/+7/+uW2+91TQ+U+Vs9erVmjVrls4//3y1tbXprrvu0iWXXKKtW7eqb9++HXEzZszQggULOv6dlJRkGhQAAEA0Pfnkk5o9e7YmTJggn8+nqVOn6qGHHur4+eHDh7V9+3Y1NTV1PPbzn/+8I7alpUUTJ07Ur371q46f9+7dW4sXL9Ydd9yhcDisoUOHauHChZoxY4ZpbKbJ2YoVKzr9+/HHH9fAgQNVVVWlcePGdTyelJQU0f09AADAqaOtTerVy/uc0ZSenq5ly5Yd8+d5eXmfuQVGYmKiFi9erMWLFx/1OZMmTdKkSZMiHltE9zmrr6+X9NEKftKTTz6pjIwMDRs2THPmzOk06/y0lpYWBYPBTgsAADh1nGqHNWPdSV8QEAqFdPvtt+tLX/qShg0b1vH4N77xDZ1++unKzs7W5s2b9cMf/lDbt2/XH/7wh6PmKS8v149//OOTHQYAAEC3ctKTs1mzZmnLli165ZVXOj3+yb5U5557rgYNGqQJEybonXfe0RlnnPGZPHPmzFFZWVnHv4PBoHJzc092WAAA4HMWK7fS6C5OanI2e/bsjiahOTk5x409cknqjh07jjo586JHFgAAQHdhmpyFw2HdeuutevbZZ1VRUaEhQ4ac8DmbNm2SJA0aNOikBggAAGJbW5vk87hbN+ecOZo1a5aWLVumP/7xj+rXr19HL6nU1FT16dNH77zzjpYtW6ZLL71Up512mjZv3qw77rhD48aNU1FRUVRWAAAAoDsxTc6WLFki6aMbzX7SY489phtvvFF+v18vv/yyFi1apMbGRuXm5mrq1Km6++67zQNLSQ4pJdmt3Y6lJdOHH9rGcfiwe+zgHFt7IIvWNtufJB9MdG/J1Mf414mlJdPy5bbcX/+6LX5ww1b34IICU+7M3gedY7cGBp446BMM3WdM7ZgkSZZ2NdaeY59q+HsippZZ1t5dBkUFrbYnWFoyfeq82xMy7J/qHPd2TJKUbdjeeXnGMsfYsabw9GTDNre8ISTp9dfdY88/35S6Juz+Xs6U7TPf8jnut7Tss3xRRQmVM2+ZD2seT25urlavXh3RgAAAAHoyemsCAICIUDnzFpMzAAAQESZn3vJ4UwIAACASVM4AAEBEuAmtt6icAQAAxBAqZwAAICJtbVJcnPc5eyoqZwAAADGEyhkAAIgIlTNvUTkDAACIIVTOAABARKiceSt2J2f/8z9S375OoTkjLnZOa21Blvnyk86xBydfZ8ptaSXo37TelLvd0JPP2m/UMm5zr8y0oO0JGXnOodb+pJbedoVDbb0b16z1O8eOu2CEKbelX2Z1c7opdS/j+2eAoT3ggQ9TTLkz699yD7b0KZSk0aPdYy29TCVpm3uv1C2JZ5lSZ2QYejcGdptyW9ezqc39NZ5k3YaGfpnBRFvf2z62kZg0N7vH+i3bxO++raOFyZm3OKwJAAAQQ2K3cgYAAE4J7e3eV864CS0AAABiApUzAAAQkWicH8Y5ZwAAAIgJVM4AAEBEqJx5i8oZAABADKFyBgAAIkLlzFtUzgAAAGIIlTMAABCRaNyTrCff5yx2J2f5+VK/fk6hhm41GpwTMg3D0pLppZdMqXXVVe6xrSPc2zFJUvZ/PuoePH68KXdr/3zn2MENW025Le2YJKnmUJJz7IABtqHIUlLftcuUety2CvfgoV8z5VZamnOotR1TTY0t3rLN33/fljsz2b0l05+3DDblvjTg3i6t2tAqTbK1ZMqaaLurZ3x72D24ttaU+7bHR5riZ892jz0rw9DXSFJN2L0lk7UdU8ryh92Dv/UtW+74JufYVa+4f7Y1NsZG+6aw4eXnoidPzjisCQAAEENit3IGAABOCVTOvEXlDAAAIIZQOQMAABGhcuYtKmcAAAAxhMoZAACICJUzb1E5AwAAiCFUzgAAQETa272vnIVstyXtVpicAQCAiLS1ST6Pj8X15MkZhzUBAABiCJUzAAAQESpn3ooLh70+ShyZYDCo1NRU/e1v9UpOTnF6ztCh7vmTEm17u7XN/dUWb5zq+h64z30ct//AlNu/4VX34BEjTLmV6N7T0MqyvSXbNvc1u/e1k6TWePfedn61mnLXfODeC69/f1Nq+ePdX+OhKBfPfZs2OseGRth6N/r27naO/e9ttt6alnazlu0tRfkz5VvfdB/HUkP/XUn+ve+a4tcdcO/Be/75ptTyKUa+tX/3O1v8tde6xxqaRgeDQaXm5qq+vl4pKW7fm1458p2dmVkvn8/b3x0KBVVTk9ol69XVqJwBAICIUDnzFuecAQAAxBAmZwAAICLt7R9Vz7xcon0T2oMHD+q6665TSkqK0tLSdNNNN6nhBIeTH374YY0fP14pKSmKi4tTXV2dJ3k/jckZAADoca677jq98cYbeumll/TCCy9ozZo1mjlz5nGf09TUpEmTJumuu+7yNO+ncc4ZAACISFubFBfnbc4jlysGg8FOjyckJCghISGi3G+++aZWrFihv//97xo9erQk6Re/+IUuvfRSPfDAA8rOzj7q826//XZJUkVFhad5P43KGQAAiIjXhzSPLJKUm5ur1NTUjqW8vDzi8VZWViotLa1jAiVJpaWl8vl8WrduXZfnpXIGAABi1p49ezrdSiPSqpkkBQIBDRw4sNNj8fHxSk9PVyAQ6PK8VM4AAEBEolk5S0lJ6bQcb3J25513Ki4u7rjLtm3bPqetcvKonAEAgG7hu9/9rm688cbjxuTn5ysrK0v79+/v9HhbW5sOHjyorKysk/79XuVlcgYAACISzQsCLAYMGKABAwacMK6kpER1dXWqqqrSqFGjJEmrVq1SKBRScXGx/Rd7nDdm2zfV79vn3q7B0E7I2h7Iv2m9e+4RY0y5TeN47ve2J5SWOoe2JqebUn/wgXtsZu+DptxRbQ1laMckSf4t7q2HTD3EJG3d696KpDCt2pRbycnOoTUf2lqivP++bSgFBe6xhpesJGnVTwwtynJyTLnfanZv95SXZ0otf8C97ZRqa025W4e5t8DyX1Riyr31kUpTfGFB9NqIHTnc5aK52ZRaKfGGNm9R/LyqOeC+TQ4dCurMM7umzdGR72ypXnFx3v7ucDgoKXrrNXnyZNXU1Gjp0qU6fPiwpk+frtGjR2vZsmWSpPfee08TJkzQE088oTFjPvp+DwQCCgQC2rBhg2bMmKE1a9aoX79+Gjx4sNLT053yuuCcMwAAEKGQwmFvF0W5h+qTTz6pgoICTZgwQZdeeqnGjh2rhx9+uOPnhw8f1vbt29XU9PGEfenSpTrvvPM0Y8YMSdK4ceN03nnn6U9/+pNzXhcc1gQAAD1Oenr6catZeXl5+vTBxfnz52v+/PkR5XVhqpwtWbJERUVFHVdMlJSU6C9/+UvHz5ubmzVr1iyddtppSk5O1tSpU1VTUxPRAAEAQKxrj9LSM5kmZzk5Obr33ntVVVWlDRs26OKLL9YVV1yhN954Q5J0xx136Pnnn9fTTz+t1atXq7q6WldeeWVUBg4AAGIFkzMvmQ5rXn755Z3+/dOf/lRLlizR2rVrlZOTo0ceeUTLli3TxRdfLEl67LHHdM4552jt2rW64IILvBs1AABAN3XS55y1t7fr6aefVmNjo0pKSlRVVaXDhw+r9BOXXBUUFGjw4MGqrKw85uSspaVFLS0tHf/+dA8tAAAQ66JR6eq5lTPz1Zqvv/66kpOTlZCQoJtvvlnPPvusCgsLFQgE5Pf7lZaW1ik+MzPzuC0LysvLO/XMys3NNa8EAABAd2GenJ199tnatGmT1q1bp1tuuUXTpk3T1q1bT3oAc+bMUX19fceyZ8+ek84FAAC6QihKS89kPqzp9/s19J833Bw1apT+/ve/68EHH9TVV1+t1tZW1dXVdaqe1dTUHLdlQUJCgidNTAEAALqDiG9CGwqF1NLSolGjRql3795auXJlx8+2b9+u3bt3q6TEdidqAABwKuFqTS+ZKmdz5szR5MmTNXjwYB06dEjLli1TRUWFXnzxRaWmpuqmm25SWVmZ0tPTlZKSoltvvVUlJSVcqQkAAODINDnbv3+/brjhBu3bt0+pqakqKirSiy++qK9+9auSpJ///Ofy+XyaOnWqWlpaNHHiRP3qV786qYH9+skk9enj1gvx28lPOOf9YOINpnG057j3y8z+z0dNuaPaeHDJEudQ/5w5ptSZce79/rYGBppyFw5tNcVr1y7nUL+1CaKhX+a7tba+b4cOGYJz3HtlWmXWv2WLTzb2EtzrHrrqJ4ZgydTUcmMg25R65AZDq5WxY025lZHhHHrb4+69MiXpodvfdY4198oc1ccUf/C9D51j0+vcxy1JfkNPS79he0vSqlfce/BePNp2dwFLL9vMZxY7x/b50H1bR09I3le6OOfMySOPPHLcnycmJmrx4sVavNj9RQUAAE513ErDSzQ+BwAAiCE0PgcAABGicuYlKmcAAAAxhMoZAACIUDRuGttzLwigcgYAABBDqJwBAIAIcc6Zl6icAQAAxBAqZwAAIEJUzrzE5AwAAESIyZmXOKwJAAAQQ+LC4XC4qwfxScFgUKmpqar/4AOlpLj1IWttc59j+mXr3fjqBr9z7IUX2C77rTngPu524x8QDQ3usStW2HLPnh2dcUjSli22+AuTN7sHG3plStLBZvcee+mJTabcpg2TlmZK3dTm/ppNarP1BrTuoNbRFzrHxhvr+JZNmDJ8iCn3waqdzrHpybbPFMv+2WtsN/rBB+6xxefbPq8O1tn+lk8/o79zbGuNYeAyfo7v2GHKXZ1W6Bxb695mWJJUlHPQObYpMd05NhgMatCgVNXX1zt/b3rlyHe2tEGS132AGySN7pL16mpUzgAAAGII55wBAIAIcRNaL1E5AwAAiCFUzgAAQIS4WtNLVM4AAABiCJUzAAAQISpnXmJyBgAAIsTkzEsc1gQAAIghVM4AAECEqJx5icoZAABADInZ9k0HDri3a7C00OjVyzaeAQPcYw8csOXu797dRP5mY5udujrn0FDOYFPqQMA9NtnYzSMl2XbTwaZm978vEhNtY/G12dryRI21r5FlB2Vk2HK3tZnCQ4nuLbB8DcbXuOHFZW49tGWNe3Benim3aZs3N5tSh9LcW/5Y+Xa9a4pvzcl3jvXvess2mKws91jrh5Bhm1te35Lt7WP5zA8Gg0rNze3i9k0rJfX1OHujpAm0bwIAAEDX4pwzAAAQIc458xKVMwAAgBhC5QwAAESIypmXmJwBAIAIMTnzEoc1AQAAYgiVMwAAECEqZ16icgYAABBDqJwBAIAIhf65eJ2zZ6JyBgAAEEOonAEAgAhxzpmXYnZy5ler/HLrbdirl985b58+tnFYeqFZ+nBKks9Qsm1NNPYVy3KPb26wpba0qnvmGVvuSZNsxdzsuq3uwQUFtsE0uG+Yt2ptPQ1zctxjk2qrTbmDydnOsSnxxsMGhm0iST5rQ1OD1jb310p6srFPqqVf5uuv23Kff75zaE14oCl1puUzxbD9JMlv3Jeun9+SbL0yJWnHDvfYoUNNqQ+2uX92pidaD7sZtrnly8fY8xYfOXjwoG699VY9//zz8vl8mjp1qh588EElH+dL7uGHH9ayZcu0ceNGHTp0SB988IHS0tI6xeTl5ekf//hHp8fKy8t15513Oo+Nw5oAACBCIX1cPfNqie45Z9ddd53eeOMNvfTSS3rhhRe0Zs0azZw587jPaWpq0qRJk3TXXXcdN27BggXat29fx3LrrbeaxhazlTMAAIBgMNjp3wkJCUpISIgo55tvvqkVK1bo73//u0aPHi1J+sUvfqFLL71UDzzwgLKzj34E4vbbb5ckVVRUHDd/v379lGWtCH8ClTMAABAhr6tmH5/Dlpubq9TU1I6lvLw84tFWVlYqLS2tY2ImSaWlpfL5fFq3bl3E+e+9916ddtppOu+883T//ferzXjomcoZAACIUPRupbFnzx6lpHx8LmCkVTNJCgQCGjiw8zmd8fHxSk9PVyAQiCj3bbfdppEjRyo9PV2vvvqq5syZo3379mnhwoXOOZicAQCAmJWSktJpcnY8d955p372s58dN+bNN9/0YljHVFZW1vH/RUVF8vv9+n//7/+pvLzceWLJ5AwAAEQoNm6l8d3vflc33njjcWPy8/OVlZWl/fv3d3q8ra1NBw8ejOhcsaMpLi5WW1ubdu3apbPPPtvpOUzOAABAtzBgwAANcLivVUlJierq6lRVVaVRo0ZJklatWqVQKKTi4mJPx7Rp0yb5fL7PHEY9HiZnAAAgQrFROXN1zjnnaNKkSZoxY4aWLl2qw4cPa/bs2brmmms6rtR87733NGHCBD3xxBMaM2aMpI/OVQsEAtrxz3vtvf766+rXr58GDx6s9PR0VVZWat26dfrKV76ifv36qbKyUnfccYeuv/569e/f33l8XK0JAAB6nCeffFIFBQWaMGGCLr30Uo0dO1YPP/xwx88PHz6s7du3q6mpqeOxpUuX6rzzztOMGTMkSePGjdN5552nP/3pT5I+ulhh+fLluuiii/TFL35RP/3pT3XHHXd0yuuCyhkAAIjQqVU5k6T09HQtW7bsmD/Py8tTOBzu9Nj8+fM1f/78Yz5n5MiRWrt2bcRji9nJ2brX/Orb160t0wUXuOe1drl44QX32CvH7j9x0CdlZDiH7tplS205n3HLFlvuCy9wv1za2o5pwwbbWP4lo872BANLS6azsoInDvoky+vQ8DqRjC2ZjO2YFB/FjwxLXzBJ/oB7W6umNPeWVpKUZNnmhnZMkhRMdD/vxNhtzqS52RbvN74OTS2WrK3VLC2ZLOOQ1Jw10jk2ZDz45K9z/44IZbi/TkK+mP0qx0lijwIAgAgdad/kdc6eickZAACIUPRuQtsTmWqyS5YsUVFRUccN4UpKSvSXv/yl4+fjx49XXFxcp+Xmm2/2fNAAAADdlalylpOTo3vvvVdnnnmmwuGwfvvb3+qKK67Qa6+9pi9+8YuSpBkzZmjBggUdz0lKSvJ2xAAAIMacehcExDLT5Ozyyy/v9O+f/vSnWrJkidauXdsxOUtKSjLdXbelpUUtLS0d//5093kAAICe5KTvc9be3q7ly5ersbFRJSUlHY8/+eSTysjI0LBhwzRnzpxO9wc5mvLy8k7d5nNzc092SAAAoEu0R2npmcwXBLz++usqKSlRc3OzkpOT9eyzz6qwsFCS9I1vfEOnn366srOztXnzZv3whz/U9u3b9Yc//OGY+ebMmdOpSWgwGGSCBgAAeizz5Ozss8/Wpk2bVF9fr2eeeUbTpk3T6tWrVVhYqJkzZ3bEnXvuuRo0aJAmTJigd955R2ecccZR8yUkJDh3aQcAALGIc868ZD6s6ff7NXToUI0aNUrl5eUaPny4HnzwwaPGHmkeusN4E0AAAICeKuL7nIVCoU4n9H/Spk2bJEmDBg2K9NcAAICYReXMS6bJ2Zw5czR58mQNHjxYhw4d0rJly1RRUaEXX3xR77zzjpYtW6ZLL71Up512mjZv3qw77rhD48aNU1FRUbTGDwAAuhw3ofWSaXK2f/9+3XDDDdq3b59SU1NVVFSkF198UV/96le1Z88evfzyy1q0aJEaGxuVm5urqVOn6u677z6pgRWfH1JKituOqTngfnR2wADbOCz9Mh9+zr0XmiTN/Jb7C8/SK1OSUgJvOcdemGxrstfU7D7Zzq7baspt7ZVZ9syFzrELDT1BJWNLy9paU24FAu6xI0bYclv6ZRp7Za7akGKKv3i8+zavDtjOssg29OLcu9eUWmdluL8nasK2972lX2bK8odNufWtb7nnjj/+lfSftuoV2z0rCwoKnWOzm21jOdjm/jq09MqUpOyXn3CODV1/gym3EhOdQy39lA8dsg0Dsc/0yfzII48c82e5ublavXp1xAMCAACnGnpreumk73MGAAAA79H4HAAARIgLArxE5QwAACCGUDkDAAARonLmJSpnAAAAMYTKGQAAiBCVMy8xOQMAABFicuYlDmsCAADEECpnAAAgQrRv8lJcOBwOd/UgPikYDCo1NVVvv12vfv3cWnT07++e3x8fQzvb0sLH2r/J0k7I0AZHkkKJ7m1cfDH05np3l61QnJ8XvbG3trmPxdhhKaa2udra3GOtK9rs3mJp3eu21kPnn+8eG1Pb+3e/c4+99lpbbktbMEmbd7m3WBo2zDYUyzYPRfEAke8/3Vs9SbZ2T762VufYYDCo1AEDVF9fr5QUW4u1SB35zpbukuTenspNs6R/75L16mpUzgAAQIRo3+QlzjkDAACIIVTOAABAhLha00tUzgAAAGIIlTMAABAhKmdeYnIGAAAixK00vMRhTQAAgBhC5QwAAESIw5peonIGAAAQQ6icAQCACFE58xKVMwAAgBgSs5Wz9vaPFhf+5qBz3tZEW3+uXbvcY63tL1MMTyj7nm0evfCBDOfYg3W23Onx7j3frP343qpNN8VnuK+muVfmQ7903y63XbPflDs+Y6BzrK/B/fVtZuyrWh2wvVays9w/Yt7aYct9VqJ7/9ji83NMuS39GC19UiVTS1ClxDeZcpv7ZRrUfGj77CzKOegc29pme99b6gr+Ott7U4nu/SEtvTIlyffM751jg5P+r3tss980juigcuYlKmcAAAAxJGYrZwAA4FRB43MvMTkDAAAR4ia0XuKwJgAAQAyhcgYAACLEBQFeonIGAAAQQ6icAQCACFE58xKVMwAAgBjC5AwAAESoPUpL9Bw8eFDXXXedUlJSlJaWpptuukkNx7lx+sGDB3Xrrbfq7LPPVp8+fTR48GDddtttqq+v7xS3e/duXXbZZUpKStLAgQP1/e9/X21tbaaxcVgTAAD0ONddd5327dunl156SYcPH9b06dM1c+ZMLVu27Kjx1dXVqq6u1gMPPKDCwkL94x//0M0336zq6mo988wzkqT29nZddtllysrK0quvvqp9+/bphhtuUO/evfXv//7vzmOLC4fDYU/W0iPBYFCpqamqqqpXcrJbu5CzEnc752/NGmwaj6nVSuAtU26lpbnHWvoUSdKWLe6xQ4facscbWvLssrUVOSvL2Kqo1r2Fj/LyopZ74173dkySbZOnJNvu9WNpJ+SvrTbltrZ7srxWTPtSUjDN/b1s/KNV6XXvugcb2v1IMr2XV71ie/9cPNr9/WNtx5T5zGJTfNP0Wc6xSW3G971hh4bSbK2hLC378nMMrexka7OUsm29e96GBqVOmKD6+nqlpNj2a6SOfGdLV0vyuo1Uq6SntGfPnk7rlZCQoISEhIgyv/nmmyosLNTf//53jR49WpK0YsUKXXrppdq7d6+ys7Od8jz99NO6/vrr1djYqPj4eP3lL3/R1772NVVXVyszM1OStHTpUv3whz/UgQMH5Pe7bSMOawIAgAgd6RDg5fLRH6a5ublKTU3tWMrLyyMebWVlpdLS0jomZpJUWloqn8+ndevWOec5MiGO/+cfopWVlTr33HM7JmaSNHHiRAWDQb3xxhvOeTmsCQAAYtbRKmeRCgQCGjiw8xGP+Ph4paenKxAIOOWora3VPffco5kzZ3bK+8mJmaSOf7vmlaicAQCAiIWitEgpKSmdluNNzu68807FxcUdd9m2bVvEaxsMBnXZZZepsLBQ8+fPjzjfp1E5AwAA3cJ3v/td3XjjjceNyc/PV1ZWlvbv39/p8ba2Nh08eFBZWVnHff6hQ4c0adIk9evXT88++6x69+7d8bOsrCytX9/5fMGampqOn7licgYAACLULu8PxtlvpTFgwAANGDDghHElJSWqq6tTVVWVRo0aJUlatWqVQqGQiouLj/m8YDCoiRMnKiEhQX/605+U+KkLgkpKSvTTn/5U+/fv7zhs+tJLLyklJUWFhYXO68FhTQAA0KOcc845mjRpkmbMmKH169frb3/7m2bPnq1rrrmm40rN9957TwUFBR2VsGAwqEsuuUSNjY165JFHFAwGFQgEFAgE1N7+0UTykksuUWFhof71X/9V//u//6sXX3xRd999t2bNmmU6V47KGQAAiFBsVM4snnzySc2ePVsTJkyQz+fT1KlT9dBDD3X8/PDhw9q+fbuampokSRs3buy4knPop+6HtHPnTuXl5alXr1564YUXdMstt6ikpER9+/bVtGnTtGDBAtPYmJwBAIAeJz09/Zg3nJWkvLw8ffJWsOPHj5fLrWFPP/10/fnPf45obEzOAABAhE69ylksY3IGAAAi9PGtL7zN2TPF3OTsSMmwocG9nUfw8CHn2NYkW5sQS/smHadh6lH16uUe69jy4aTGEjS2TjG05GlosI07aB3LIfd9b15PQ+6GBlsLH9NQQlFs32TZficxFlP7JuNYgj73jWht3xRvGcvhw7bkhvdyY2P03j+HLJ9tkvp8+KEpvskwlrZotm/y2b7mbB8p0WvfZPkMDzY2SpLTIbfoMb7JuiznqSHmemvu3btXubm5XT0MAABOKXv27FFOTs7n+jubm5s1ZMgQ093vLbKysrRz587P3LKiu4u5yVkoFFJ1dbX69eunuLi4jseDwaByc3M/08ahu2E9u4+esI4S69nd9IT17G7rGA6HdejQIWVnZ8vn+/zvkNXc3KzWVlsV0ZXf7+9xEzMpBg9r+ny+4878j7Rv6O5Yz+6jJ6yjxHp2Nz1hPbvTOqampnbZ705MTOyRE6ho4ia0AAAAMYTJGQAAQAw5ZSZnCQkJmjdvnqn9wamI9ew+esI6Sqxnd9MT1rMnrCNObTF3QQAAAEBPdspUzgAAAHoCJmcAAAAxhMkZAABADGFyBgAAEEOYnAEAAMSQU2ZytnjxYuXl5SkxMVHFxcVav359Vw/JU/Pnz1dcXFynpaCgoKuHFZE1a9bo8ssvV3Z2tuLi4vTcc891+nk4HNbcuXM1aNAg9enTR6WlpXr77be7ZrARONF63njjjZ/Zt5MmTeqawZ6k8vJynX/++erXr58GDhyoKVOmaPv27Z1impubNWvWLJ122mlKTk7W1KlTVVNT00UjPjku6zl+/PjP7M+bb765i0Z8cpYsWaKioqKOO+SXlJToL3/5S8fPu8O+lE68nt1hX6J7OiUmZ0899ZTKyso0b948bdy4UcOHD9fEiRO1f//+rh6ap774xS9q3759Hcsrr7zS1UOKSGNjo4YPH67Fixcf9ef33XefHnroIS1dulTr1q1T3759NXHiRDU3N3/OI43MidZTkiZNmtRp3/7ud7/7HEcYudWrV2vWrFlau3atXnrpJR0+fFiXXHKJGhsbO2LuuOMOPf/883r66ae1evVqVVdX68orr+zCUdu5rKckzZgxo9P+vO+++7poxCcnJydH9957r6qqqrRhwwZdfPHFuuKKK/TGG29I6h77Ujrxekqn/r5ENxU+BYwZMyY8a9asjn+3t7eHs7Ozw+Xl5V04Km/NmzcvPHz48K4eRtRICj/77LMd/w6FQuGsrKzw/fff3/FYXV1dOCEhIfy73/2uC0bojU+vZzgcDk+bNi18xRVXdMl4omX//v1hSeHVq1eHw+GP9l3v3r3DTz/9dEfMm2++GZYUrqys7KphRuzT6xkOh8MXXXRR+Dvf+U7XDSpK+vfvH/7Nb37TbfflEUfWMxzuvvsSp76Yr5y1traqqqpKpaWlHY/5fD6VlpaqsrKyC0fmvbffflvZ2dnKz8/Xddddp927d3f1kKJm586dCgQCnfZramqqiouLu91+laSKigoNHDhQZ599tm655Ra9//77XT2kiNTX10uS0tPTJUlVVVU6fPhwp/1ZUFCgwYMHn9L789PrecSTTz6pjIwMDRs2THPmzFFTU1NXDM8T7e3tWr58uRobG1VSUtJt9+Wn1/OI7rQv0X3Ed/UATqS2tlbt7e3KzMzs9HhmZqa2bdvWRaPyXnFxsR5//HGdffbZ2rdvn3784x/ry1/+srZs2aJ+/fp19fA8FwgEJOmo+/XIz7qLSZMm6corr9SQIUP0zjvv6K677tLkyZNVWVmpXr16dfXwzEKhkG6//XZ96Utf0rBhwyR9tD/9fr/S0tI6xZ7K+/No6ylJ3/jGN3T66acrOztbmzdv1g9/+ENt375df/jDH7pwtHavv/66SkpK1NzcrOTkZD377LMqLCzUpk2butW+PNZ6St1nX6L7ifnJWU8xefLkjv8vKipScXGxTj/9dP3+97/XTTfd1IUjQ6Suueaajv8/99xzVVRUpDPOOEMVFRWaMGFCF47s5MyaNUtbtmw55c+JPJFjrefMmTM7/v/cc8/VoEGDNGHCBL3zzjs644wzPu9hnrSzzz5bmzZtUn19vZ555hlNmzZNq1ev7uphee5Y61lYWNht9iW6n5g/rJmRkaFevXp95kqhmpoaZWVlddGooi8tLU1nnXWWduzY0dVDiYoj+66n7VdJys/PV0ZGxim5b2fPnq0XXnhBf/3rX5WTk9PxeFZWllpbW1VXV9cp/lTdn8daz6MpLi6WpFNuf/r9fg0dOlSjRo1SeXm5hg8frgcffLDb7ctjrefRnKr7Et1PzE/O/H6/Ro0apZUrV3Y8FgqFtHLlyk7nDXQ3DQ0NeueddzRo0KCuHkpUDBkyRFlZWZ32azAY1Lp167r1fpWkvXv36v333z+l9m04HNbs2bP17LPPatWqVRoyZEinn48aNUq9e/futD+3b9+u3bt3n1L780TreTSbNm2SpFNqfx5NKBRSS0tLt9mXx3JkPY+mu+xLdANdfUWCi+XLl4cTEhLCjz/+eHjr1q3hmTNnhtPS0sKBQKCrh+aZ7373u+GKiorwzp07w3/729/CpaWl4YyMjPD+/fu7emgn7dChQ+HXXnst/Nprr4UlhRcuXBh+7bXXwv/4xz/C4XA4fO+994bT0tLCf/zjH8ObN28OX3HFFeEhQ4aEP/zwwy4euc3x1vPQoUPh733ve+HKysrwzp07wy+//HJ45MiR4TPPPDPc3Nzc1UN3dsstt4RTU1PDFRUV4X379nUsTU1NHTE333xzePDgweFVq1aFN2zYEC4pKQmXlJR04ajtTrSeO3bsCC9YsCC8YcOG8M6dO8N//OMfw/n5+eFx48Z18cht7rzzzvDq1avDO3fuDG/evDl85513huPi4sL//d//HQ6Hu8e+DIePv57dZV+iezolJmfhcDj8i1/8Ijx48OCw3+8PjxkzJrx27dquHpKnrr766vCgQYPCfr8//IUvfCF89dVXh3fs2NHVw4rIX//617CkzyzTpk0Lh8Mf3U7jRz/6UTgzMzOckJAQnjBhQnj79u1dO+iTcLz1bGpqCl9yySXhAQMGhHv37h0+/fTTwzNmzDjl/rA42vpJCj/22GMdMR9++GH429/+drh///7hpKSk8P/5P/8nvG/fvq4b9Ek40Xru3r07PG7cuHB6eno4ISEhPHTo0PD3v//9cH19fdcO3Oib3/xm+PTTTw/7/f7wgAEDwhMmTOiYmIXD3WNfhsPHX8/usi/RPcWFw+Hw51enAwAAwPHE/DlnAAAAPQmTMwAAgBjC5AwAACCGMDkDAACIIUzOAAAAYgiTMwAAgBjC5AwAACCGMDkDAACIIUzOAAAAYgiTMwAAgBjC5AwAACCG/P+KQFUSNAFncwAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "imshow_zero_center(h_mat)" ] }, { "cell_type": "markdown", "metadata": { "id": "13fBswmtQes4" }, "source": [ "뉴턴의 방법 업데이트 단계는 다음과 같습니다." ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:28.364507Z", "iopub.status.busy": "2022-12-14T21:28:28.364232Z", "iopub.status.idle": "2022-12-14T21:28:28.370975Z", "shell.execute_reply": "2022-12-14T21:28:28.370406Z" }, "id": "3DdnbynBdSor" }, "outputs": [], "source": [ "eps = 1e-3\n", "eye_eps = tf.eye(h_mat.shape[0])*eps" ] }, { "cell_type": "markdown", "metadata": { "id": "-zPdtyoWeUeV" }, "source": [ "참고: [실제로 행렬을 반전하지 마세요](https://www.johndcook.com/blog/2010/01/19/dont-invert-that-matrix/)." ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:28.374367Z", "iopub.status.busy": "2022-12-14T21:28:28.373696Z", "iopub.status.idle": "2022-12-14T21:28:29.610197Z", "shell.execute_reply": "2022-12-14T21:28:29.609458Z" }, "id": "k1LYftgmswOO" }, "outputs": [], "source": [ "# X(k+1) = X(k) - (∇²f(X(k)))^-1 @ ∇f(X(k))\n", "# h_mat = ∇²f(X(k))\n", "# g_vec = ∇f(X(k))\n", "update = tf.linalg.solve(h_mat + eye_eps, g_vec)\n", "\n", "# Reshape the update and apply it to the variable.\n", "_ = layer1.kernel.assign_sub(tf.reshape(update, layer1.kernel.shape))" ] }, { "cell_type": "markdown", "metadata": { "id": "pF6qjlHKWxF4" }, "source": [ "단일 `tf.Variable` 경우에는 비교적 간단하지만, 사소하지 않은 모델에 적용하려면 여러 변수에 걸쳐 완전한 Hessian을 생성하기 위해 신중하게 연결하고 슬라이스해야 합니다." ] }, { "cell_type": "markdown", "metadata": { "id": "PQWM0uN-GO5t" }, "source": [ "### 배치 야고비안" ] }, { "cell_type": "markdown", "metadata": { "id": "hKtB3rY6EySJ" }, "source": [ "어떤 경우에는 소스 스택과 관련하여 각 대상 스택의 야고비안을 가져오려 합니다. 여기에서 각 대상-소스 쌍의 야고비안은 독립적입니다.\n", "\n", "예를 들어, 여기에서 입력 `x`는 `(batch, ins)` 형상이 되고, 출력 `y`는 `(batch, outs)` 형상이 됩니다.\n" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:29.614759Z", "iopub.status.busy": "2022-12-14T21:28:29.614095Z", "iopub.status.idle": "2022-12-14T21:28:29.630280Z", "shell.execute_reply": "2022-12-14T21:28:29.629490Z" }, "id": "tQMndhIUHMes" }, "outputs": [ { "data": { "text/plain": [ "TensorShape([7, 6])" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x = tf.random.normal([7, 5])\n", "\n", "layer1 = tf.keras.layers.Dense(8, activation=tf.nn.elu)\n", "layer2 = tf.keras.layers.Dense(6, activation=tf.nn.elu)\n", "\n", "with tf.GradientTape(persistent=True, watch_accessed_variables=False) as tape:\n", " tape.watch(x)\n", " y = layer1(x)\n", " y = layer2(y)\n", "\n", "y.shape" ] }, { "cell_type": "markdown", "metadata": { "id": "Ff2spRHEJXBU" }, "source": [ "`x`에 대한 `y`의 전체 야고비안은 `(batch, ins, outs)`만 원하는 경우에도 `(batch, ins, batch, outs)`의 형상을 가집니다." ] }, { "cell_type": "code", "execution_count": 31, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:29.633643Z", "iopub.status.busy": "2022-12-14T21:28:29.633066Z", "iopub.status.idle": "2022-12-14T21:28:29.746757Z", "shell.execute_reply": "2022-12-14T21:28:29.745921Z" }, "id": "1zSl2A5-HhMH" }, "outputs": [ { "data": { "text/plain": [ "TensorShape([7, 6, 7, 5])" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "j = tape.jacobian(y, x)\n", "j.shape" ] }, { "cell_type": "markdown", "metadata": { "id": "UibJijPLJrpQ" }, "source": [ "스택에 있는 각 항목의 그래디언트가 독립적이면, 이 텐서의 모든 `(batch, batch)` 슬라이스는 대각선 행렬입니다." ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:29.750299Z", "iopub.status.busy": "2022-12-14T21:28:29.749676Z", "iopub.status.idle": "2022-12-14T21:28:29.979687Z", "shell.execute_reply": "2022-12-14T21:28:29.978875Z" }, "id": "ZFl9uj3ueVSH" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAIQCAYAAABQc2CaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA6RklEQVR4nO3de3gU9b3H8c8mkE3AZAmQC0ggICogcjFAjAoo5AhUURQFPPQYLgdbmyCI2iNWAYs1WLygFUF4DqinIFZbEK2iNAqcyj00FfAGioeIJgGBLERZIDvnD2TrmjDJwuxMkn2/nmeeurOzM9/f0of98pnf/tZlGIYhAAAAhF2U0wUAAABEChovAAAAm9B4AQAA2ITGCwAAwCY0XgAAADah8QIAALAJjRcAAIBNaLwAAABsQuMFAABgExovRCS/36+uXbvqd7/7XWDfjBkz5HK5dODAAQcr+5c1a9bI5XLptddes+ycX375pVwulx5//HHLznmu0tPTdf3119d43KpVq3Teeedp//79ll376quv1tVXXx14fPr9eeGFFyy7BgD8GI0X6q3nnntOLpdLmZmZIb/25ZdfVnFxsfLy8sJQ2b8899xzDf5DfP369ZoxY4YOHz4c1usMHjxYHTt2VH5+flivAwDhROOFemvJkiVKT0/X5s2btXv37pBeO3v2bI0aNUoejydM1Z0SKY3Xww8/HPbGS5J+8Ytf6Pnnn9eRI0fCcv527drp+++/13/8x3+E5fwAQOOFemnPnj1av369nnzySSUlJWnJkiW1fu0//vEP/fOf/9SIESPCWCHCYfjw4fL5fHr11VfDcn6Xy6XY2FhFR0eH5fwAQOOFemnJkiVKTEzUddddp1tuuSWkxmvFihWKiYlRv379qn3+wIEDGjFihBISEtSiRQtNmjRJx44dCzpm8eLFGjBggJKTk+V2u9WlSxfNmzcv6Jj09HTt3LlTa9eulcvlksvlCppPdPjwYd19991KT0+X2+1WmzZtdPvtt1eZY+b3+/W73/1Obdq0UWxsrAYOHBhywledp556Su3atVNcXJz69++vHTt2BD3/4YcfasyYMerQoYNiY2OVmpqqcePG6dtvvw0cM2PGDN13332SpPbt2wfG+eWXXwaO+eMf/6g+ffqoSZMmSkxMVL9+/fTuu+9Wqefvf/+7+vTpo9jYWHXo0EEvvfRSlWOSk5PVrVs3vf766zWOr6SkRGPHjlWbNm3kdrvVqlUr3XjjjUG1/dSZ5nh98sknGjFihJKSkhQXF6eLL75Yv/nNb4KO2bdvn8aNG6eUlBS53W5dcsklWrRoUY11AogsjZwuADgbS5Ys0c0336yYmBjddtttmjdvnrZs2aLevXvX+Nr169era9euaty4cbXPjxgxQunp6crPz9fGjRv1zDPP6NChQ0GNwLx583TJJZfohhtuUKNGjfTGG2/oV7/6lfx+v3JzcyVJc+bM0cSJE3XeeecFPqRTUlIkSUePHlXfvn318ccfa9y4cbrssst04MABrVy5Ul999ZVatmwZuNasWbMUFRWle++9V+Xl5fr973+v0aNHa9OmTWf9/r300ks6cuSIcnNzdezYMT399NMaMGCAtm/fHqhx9erV+uKLLzR27FilpqZq586dWrBggXbu3KmNGzfK5XLp5ptv1meffaaXX35ZTz31VKDupKQkSdLDDz+sGTNm6IorrtBvf/tbxcTEaNOmTXrvvfd07bXXBurZvXu3brnlFo0fP145OTlatGiRxowZo4yMDF1yySVBtWdkZGjFihU1jnH48OHauXOnJk6cqPT0dJWVlWn16tXau3ev0tPTa/1effjhh+rbt68aN26sO+64Q+np6fr888/1xhtvBL6cUVpaqssvv1wul0t5eXlKSkrS22+/rfHjx8vr9Wry5Mm1vh6ABs4A6pmtW7cakozVq1cbhmEYfr/faNOmjTFp0qRavb5NmzbG8OHDq+yfPn26Icm44YYbgvb/6le/MiQZ//znPwP7vvvuuyqvHzRokNGhQ4egfZdcconRv3//KsdOmzbNkGT85S9/qfKc3+83DMMw3n//fUOS0blzZ8Pn8wWef/rppw1Jxvbt280HWo09e/YYkoy4uDjjq6++CuzftGmTIcm4++67Tcf48ssvG5KMdevWBfbNnj3bkGTs2bMn6Nhdu3YZUVFRxk033WRUVlZWO0bDMIx27dpVOWdZWZnhdruNe+65p0oNjz76qCHJKC0tPeM4Dx06ZEgyZs+efcZjDMMw+vfvH/Tnc/r9Wbx4cWBfv379jPj4eOP//u//zjiG8ePHG61atTIOHDgQdMyoUaMMj8dT7XsJIDJxqxH1zpIlS5SSkqJrrrlG0ql5OSNHjtSyZctUWVlZ4+u//fZbJSYmnvH504nVaRMnTpQkvfXWW4F9cXFxgf8uLy/XgQMH1L9/f33xxRcqLy+vsYY///nP6t69u2666aYqz7lcrqDHY8eOVUxMTOBx3759JUlffPFFjdc5k2HDhun8888PPO7Tp48yMzPPOMZjx47pwIEDuvzyyyVJ27Ztq/EaK1askN/v17Rp0xQVFfxXzU/H2KVLl8C4pFOJ2cUXX1ztGE//2Zkt+xEXF6eYmBitWbNGhw4dqrHWM9m/f7/WrVuncePGqW3bttWOwTAM/fnPf9bQoUNlGIYOHDgQ2AYNGqTy8vJavV8AIgONF+qVyspKLVu2TNdcc4327Nmj3bt3a/fu3crMzFRpaakKCgpqdR7DMM743IUXXhj0+IILLlBUVFTQ3KAPPvhA2dnZatq0qZo1a6akpCQ98MADklSrxuvzzz9X165da1XrTz/wTzce59JQ/HSMknTRRRcFjfHgwYOaNGmSUlJSFBcXp6SkJLVv315S7ccYFRWlLl261HjsT8conRpndWM8/Wf30+btx9xutx577DG9/fbbSklJUb9+/fT73/9eJSUlNdbyY6cbP7M/q/379+vw4cNasGCBkpKSgraxY8dKksrKykK6LoCGizleqFfee+89ffPNN1q2bJmWLVtW5fklS5YEzR2qTosWLUJqWn76Af/5559r4MCB6tSpk5588kmlpaUpJiZGb731lp566in5/f5an7s2zvQNO7Pm0QojRozQ+vXrdd9996lHjx4677zz5Pf7NXjwYEfHePrP7sfz4KozefJkDR06VCtWrNA777yjhx56SPn5+XrvvffUs2fPcy/6B6ffi5///OfKycmp9phu3bpZdj0A9RuNF+qVJUuWKDk5WXPnzq3y3F/+8hctX75c8+fPD7pN9lOdOnXSnj17zvj8rl27AsmOdGrit9/vD0zIfuONN+Tz+bRy5cqgpOb999+vcq4zpTIXXHBBlW8R2mnXrl1V9n322WeBMR46dEgFBQV6+OGHNW3aNNPXmY3R7/fro48+Uo8ePSypWzq1lEjLli0DE/jNXHDBBbrnnnt0zz33aNeuXerRo4eeeOIJ/fGPf6zVtTp06CBJpn9WSUlJio+PV2VlpbKzs2s3CAARi1uNqDe+//57/eUvf9H111+vW265pcqWl5enI0eOaOXKlabnycrK0o4dO+Tz+ap9/qdN3R/+8AdJ0pAhQyT9K535cRpTXl6uxYsXVzlX06ZNq11YdPjw4frnP/+p5cuXV3ku3EmWdGr+1b59+wKPN2/erE2bNpmOUTr1Tc2fatq0qSRVGeewYcMUFRWl3/72t1USsnMZY2FhobKyskyP+e6776osAXLBBRcoPj7+jH/u1UlKSlK/fv20aNEi7d27N+i502OIjo7W8OHD9ec//7naBs3KnzgCUP+ReKHeWLlypY4cOaIbbrih2ucvv/zywGKqI0eOPON5brzxRs2cOVNr166t9rbknj17dMMNN2jw4MHasGGD/vjHP+rf//3f1b17d0nStddeq5iYGA0dOlS/+MUvdPToUS1cuFDJycn65ptvgs6VkZGhefPm6ZFHHlHHjh2VnJysAQMG6L777tNrr72mW2+9VePGjVNGRoYOHjyolStXav78+YFr1daaNWt0zTXXaPr06ZoxY0aNx3fs2FFXXXWV7rzzTvl8Ps2ZM0ctWrTQr3/9a0lSQkJCYF7UiRMndP755+vdd9+tNinMyMiQJP3mN7/RqFGj1LhxYw0dOlQdO3bUb37zG82cOVN9+/bVzTffLLfbrS1btqh169Zn9dM/ZWVl+vDDD6t8AeKnPvvsMw0cOFAjRoxQly5d1KhRIy1fvlylpaUaNWpUSNd85plndNVVV+myyy7THXfcofbt2+vLL7/UX//6VxUVFUk6teTH+++/r8zMTE2YMEFdunTRwYMHtW3bNv3tb3/TwYMHQx4rgAbKoW9TAiEbOnSoERsba1RUVJzxmDFjxhiNGzeu8rX+n+rWrZsxfvz4oH2nl5P46KOPjFtuucWIj483EhMTjby8POP7778POnblypVGt27djNjYWCM9Pd147LHHjEWLFlVZVqGkpMS47rrrjPj4eENS0NIF3377rZGXl2ecf/75RkxMjNGmTRsjJycnUPvp5SReffXVoGtXt+TBG2+8YUgy5s+fbzru06+dPXu28cQTTxhpaWmG2+02+vbtG7RchmEYxldffWXcdNNNRrNmzQyPx2Pceuutxtdff21IMqZPnx507MyZM43zzz/fiIqKqvIeLFq0yOjZs6fhdruNxMREo3///oGlQAzj1HIS1113XZVaf7rUg2EYxrx584wmTZoYXq/XdJwHDhwwcnNzjU6dOhlNmzY1PB6PkZmZafzpT38yvUZ1761hGMaOHTsC70VsbKxx8cUXGw899FDQMaWlpUZubq6RlpZmNG7c2EhNTTUGDhxoLFiwwLRWAJHFZRg23NcA6pj/+Z//UW5urvbu3atmzZo5Xc45+/Wvf62XX35Zu3fvltvtdrqcsOnZs6euvvpqPfXUU06XAgBnhTleiEijR49W27Ztq52kXx+9//77euihhxp007Vq1Srt2rVLU6dOdboUADhrJF4AAAA2IfECAACwCY0XAACATWi8AAAAbELjBQAAYBPbF1D1+/36+uuvFR8fb/ojtwAA4NSvJBw5ckStW7dWVJT9ecmxY8d0/PjxsJw7JiZGsbGxYTl3XWV74/X1118rLS3N7ssCAFCvFRcXq02bNrZe89ixY0qKi9PRMJ0/NTVVe/bsiajmy/bGKz4+XpJUPGuWEiLoja5WTo7TFQAA6jiv16u0du0Cn592On78uI5KuluS1asE+iQ9VVKi48eP03iF0+nbiwmxsUqIi7P78nVLQoLTFQAA6gknp+fESbK6NYrUSeaROm4AAADb2Z54AQCA+iVK1ic1kZr8ROq4AQAAbEfiBQAATJF4WYfGCwAAmKLxsk6kjhsAAMB2JF4AAMAUiZd1InXcAAAAtiPxAgAApki8rBOp4wYAALAdiRcAADBF4mWdSB03AACA7Ui8AACAKRIv60TquAEAAGxH4gUAAEy5ZH1S47L4fPUFjRcAADDlkvWNUqQ2XtxqBAAAsAmJFwAAMBX9w2b1OSMRiRcAAIBNaLwAAICpqDBtoZo7d67S09MVGxurzMxMbd68+YzH7ty5U8OHD1d6erpcLpfmzJlzzue0Ao0XAACo81555RVNmTJF06dP17Zt29S9e3cNGjRIZWVl1R7/3XffqUOHDpo1a5ZSU1MtOacVaLwAAICpupB4Pfnkk5owYYLGjh2rLl26aP78+WrSpIkWLVpU7fG9e/fW7NmzNWrUKLndbkvOaQUaLwAAUKcdP35chYWFys7ODuyLiopSdna2NmzYUGfOWRt8qxEAAJgK508Geb3eoP1ut7tKQnXgwAFVVlYqJSUlaH9KSoo++eSTs7p+OM5ZGyReAADAVDhvNaalpcnj8QS2/Px8u4blCBIvAADgmOLiYiUkJAQeVzcfq2XLloqOjlZpaWnQ/tLS0jNOnK9JOM5ZG2eVeNn91UsAAOCccCZeCQkJQVt1jVdMTIwyMjJUUFAQ2Of3+1VQUKCsrKyzGlM4zlkbITdeTnz1EgAARLYpU6Zo4cKFevHFF/Xxxx/rzjvvVEVFhcaOHStJuv322zV16tTA8cePH1dRUZGKiop0/Phx7du3T0VFRdq9e3etzxkOId9q/PFXLyVp/vz5+utf/6pFixbp/vvvt7xAAADgrHBOrq+tkSNHav/+/Zo2bZpKSkrUo0cPrVq1KjA5fu/evYqK+tdZv/76a/Xs2TPw+PHHH9fjjz+u/v37a82aNbU6Zzi4DMMwanvw8ePH1aRJE7322msaNmxYYH9OTo4OHz6s119/vcprfD6ffD5f4LHX61VaWprK58xRQlzcuVVf3/3nfzpdAQCgjvN6vfIkJqq8vDxoLpRt1/Z49Lwkqz+xv5f0C8mRcTkppIbT7KuXJSUl1b4mPz8/6NsKaWlpZ18tAACwnStMWyQK+3ISU6dOVXl5eWArLi4O9yUBAADqpJDmeJ3NVy+rWwgNAADUH1GSosNwzkgU0rid+uolAABwTl34rcaGIuRvNU6ZMkU5OTnq1auX+vTpozlz5oT9q5cAAAANQciNlxNfvQQAAM6pC8tJNBRn9ZNBeXl5ysvLs7oWAACABo3fagQAAKZIvKwTqeMGAACwHYkXAAAwReJlnUgdNwAAgO1IvAAAgCkSL+vQeAEAAFM0XtaJ1HEDAADYjsQLAACYcv2wWX3OSETiBQAAYBMSLwAAYCr6h83qc0YiEi8AAACbkHgBAABTLlmf1DDHCwAAAGFF4gUAAEyxjpd1aLwAAIApGi/rROq4AQAAbEfiBQAATJF4WSdSxw0AAGA7Ei8AAGCKxMs6kTpuAAAA25F4AQAAUyRe1onUcQMAANiOxAsAAJhyyfqf+InUnwyi8QIAAKaif9isPmck4lYjAACATUi8AACAKSbXW8e5xisnR0pIcOzydcIjjzhdQd3w4INOVwAAgC1IvAAAgCmXrE+oInVyfaQmfQAAALYj8QIAAKaY42WdSB03AACA7Ui8AACAKRIv69B4AQAAUzRe1onUcQMAANiOxAsAAJgi8bJOpI4bAADAdiReAADAlEvWL3jKAqoAAAAIKxIvAABgKvqHzepzRiISLwAAAJuQeAEAAFN8q9E6NF4AAMAUjZd1InXcAAAAtiPxAgAAplyyPqlhOQkAAACEFYkXAAAwxRwv60TquAEAAGxH4gUAAEyReFknUscNAABgOxIvAABgisTLOpE6bgAAANuReAEAAFMuWb/uVqSu40XjBQAATEX/sFl9zkjErUYAAACbkHgBAABTTK63TsjjXrdunYYOHarWrVvL5XJpxYoVYSgLAACg4Qm58aqoqFD37t01d+7ccNQDAADqmKgwbZEo5FuNQ4YM0ZAhQ8JRCwAAQIMW9jlePp9PPp8v8Njr9Yb7kgAAwELM8bJO2Medn58vj8cT2NLS0sJ9SQAAgDop7I3X1KlTVV5eHtiKi4vDfUkAAGAhl6yf38UCqmHidrvldrvDfRkAABAm3Gq0TqSOGwAAwHYhJ15Hjx7V7t27A4/37NmjoqIiNW/eXG3btrW0OAAA4DwSL+uE3Hht3bpV11xzTeDxlClTJEk5OTl64YUXLCsMAACgoQm58br66qtlGEY4agEAAHUQiZd1InXcAAAAtuNHsgEAgCkSL+tE6rgBAABsR+IFAABMkXhZh8YLAACYovGyTqSOGwAAwHYkXgAAwBSJl3UiddwAAAC2I/ECAACmSLysE6njBgAAsB2JFwAAMOWS5HK5rD1nhP78IIkXAACATUi8AACAuUaNJIsTLxmGdPKkteesB2i8AACAORovy3CrEQAAwCYkXgAAwFy4Eq8IROIFAABgExIvAABgjsTLMiReAAAANqHxAgAA5qKjT6VeVm7R0SGXMXfuXKWnpys2NlaZmZnavHmz6fGvvvqqOnXqpNjYWF166aV66623gp4fM2aMXC5X0DZ48OCQ6woFjRcAAKjzXnnlFU2ZMkXTp0/Xtm3b1L17dw0aNEhlZWXVHr9+/XrddtttGj9+vP7xj39o2LBhGjZsmHbs2BF03ODBg/XNN98Etpdffjms46DxAgAA5qxOu05vIXjyySc1YcIEjR07Vl26dNH8+fPVpEkTLVq0qNrjn376aQ0ePFj33XefOnfurJkzZ+qyyy7Ts88+G3Sc2+1WampqYEtMTDzrt6k2aLwAAIC5MDZeXq83aPP5fFUuf/z4cRUWFio7OzuwLyoqStnZ2dqwYUO1JW/YsCHoeEkaNGhQlePXrFmj5ORkXXzxxbrzzjv17bffnuu7ZYrGCwAAOCYtLU0ejyew5efnVznmwIEDqqysVEpKStD+lJQUlZSUVHvekpKSGo8fPHiwXnrpJRUUFOixxx7T2rVrNWTIEFVWVlowsuqxnAQAADDXqJEUZXFW4/dLkoqLi5WQkBDY7Xa7rb2OiVGjRgX++9JLL1W3bt10wQUXaM2aNRo4cGBYrkniBQAAHJOQkBC0Vdd4tWzZUtHR0SotLQ3aX1paqtTU1GrPm5qaGtLxktShQwe1bNlSu3fvPouR1A6Jl5MefNDpCuqEjz6h/5ekLp38TpcAANWLjj6r5R9MhXA7LyYmRhkZGSooKNCwYcMkSX6/XwUFBcrLy6v2NVlZWSooKNDkyZMD+1avXq2srKwzXuerr77St99+q1atWtW6tlDxiQcAAOq8KVOmaOHChXrxxRf18ccf684771RFRYXGjh0rSbr99ts1derUwPGTJk3SqlWr9MQTT+iTTz7RjBkztHXr1kCjdvToUd13333auHGjvvzySxUUFOjGG29Ux44dNWjQoLCNg8QLAACYO8sFT02F+BNEI0eO1P79+zVt2jSVlJSoR48eWrVqVWAC/d69exX1o3loV1xxhZYuXaoHH3xQDzzwgC688EKtWLFCXbt2lSRFR0frww8/1IsvvqjDhw+rdevWuvbaazVz5sywzjNzGYa9P5bk9Xrl8XhUfuhQ0GQ6RC5uNZ7CrUYA1fF6vfIkJqq8vNz2z83AZ/ZFFynB4sbLW1kpz2efOTIuJ5F4AQAAc3Ug8WooaLwAAIA5Gi/LcI8HAADAJiReAADAHImXZUi8AAAAbELiBQAAzEVHB37UGueGxAsAAMAmtK8AAMBco0YkXhYh8QIAALAJ7SsAADBH4mUZ3kUAAGCOxssy3GoEAACwCe0rAAAwF47lJAzD2vPVEyReAAAANiHxAgAA5sIxx4vECwAAAOFE4gUAAMyReFmGxAsAAMAmJF4AAMAciZdlaLwAAIA5Gi/LcKsRAADAJiReAADAXDgWUPX7rT1fPUHiBQAAYBMSLwAAYC4cc7xIvAAAABBOITVe+fn56t27t+Lj45WcnKxhw4bp008/DVdtAACgLjideFm9RaCQGq+1a9cqNzdXGzdu1OrVq3XixAlde+21qqioCFd9AAAADUZI7eaqVauCHr/wwgtKTk5WYWGh+vXrZ2lhAACgjmCOl2XO6V0sLy+XJDVv3vyMx/h8Pvl8vsBjr9d7LpcEAACot856cr3f79fkyZN15ZVXqmvXrmc8Lj8/Xx6PJ7ClpaWd7SUBAIATmONlmbNuvHJzc7Vjxw4tW7bM9LipU6eqvLw8sBUXF5/tJQEAgBNOL6Bq5RYd7fSoHHFW7WZeXp7efPNNrVu3Tm3atDE91u12y+12n1VxAAAADUlIjZdhGJo4caKWL1+uNWvWqH379uGqCwAA1BXhuDVYWWnt+eqJkN7F3NxcLV26VK+//rri4+NVUlIiSfJ4PIqLiwtLgQAAAA1FSI3XvHnzJElXX3110P7FixdrzJgxVtUEAADqEhIvy4R8qxEAAABnJzK/ywkAAGqPxMsy/Eg2AACATUi8AACAudPreFnp5Elrz1dP0HgBAABz4bjVyMr1AAAACKfIbDcBAEDtkXhZhsQLAADAJpHZbgIAgNoj8bIMiRcAAIBNIrPdBAAAtUfiZRkSLwAAAJtEZrsJAABqLxwLqEZHW3u+eoLGCwAAmONWo2W41QgAAGCTyGw3AQBA7ZF4WYbECwAAwCaR2W4CAIDaI/GyDIkXAACATSKz3QQAALXHchKWIfECAACwCYkXAAAwxxwvy0TmqAEAQO3ReFmGW40AAAA2icx2EwAA1B6Jl2VIvAAAAGwSme0mAACoPRIvy0TmqFGndOnkd7qEOuGz3QTQknRRR/7/AKDhovECAADmWEDVMvwTGwAAwCYkXgAAwBxzvCwTmaMGAAC1R+NlGW41AgAA2CQy200AAFB7JF6WIfECAACwSWS2mwAAoPZYTsIyJF4AAAA2IfECAADmmONlGRIvAAAAm0RmuwkAAGqPxMsykTlqAABQezReluFWIwAAgE0is90EAAC1R+JlGRIvAAAAm0RmuwkAAGqPBVQtQ+IFAABgExIvAABgjjleliHxAgAAsElktpsAAKD2SLwsE5mjBgAAtUfjZRluNQIAANgkMttNAABQeywnYRkSLwAAAJuQeAEAAHPM8bIMiRcAAIBNIrPdBAAAtUfiZZmQEq958+apW7duSkhIUEJCgrKysvT222+HqzYAAICAuXPnKj09XbGxscrMzNTmzZtNj3/11VfVqVMnxcbG6tJLL9Vbb70V9LxhGJo2bZpatWqluLg4ZWdna9euXeEcQmiNV5s2bTRr1iwVFhZq69atGjBggG688Ubt3LkzXPUBAACnnU68rN5C8Morr2jKlCmaPn26tm3bpu7du2vQoEEqKyur9vj169frtttu0/jx4/WPf/xDw4YN07Bhw7Rjx47AMb///e/1zDPPaP78+dq0aZOaNm2qQYMG6dixY+f0dplxGYZhnMsJmjdvrtmzZ2v8+PG1Ot7r9crj8aj80CElJCScy6WBBuWz3Uy5lKSLOvqdLgGoU7xerzyJiSovL7f9czOcn9mhjiszM1O9e/fWs88+K0ny+/1KS0vTxIkTdf/991c5fuTIkaqoqNCbb74Z2Hf55ZerR48emj9/vgzDUOvWrXXPPffo3nvvlSSVl5crJSVFL7zwgkaNGmXRSIOd9d/0lZWVWrZsmSoqKpSVlXXG43w+n7xeb9AGAAAgqUqP4PP5qhxz/PhxFRYWKjs7O7AvKipK2dnZ2rBhQ7Xn3bBhQ9DxkjRo0KDA8Xv27FFJSUnQMR6PR5mZmWc8pxVCbry2b9+u8847T263W7/85S+1fPlydenS5YzH5+fny+PxBLa0tLRzKhgAANjLr6iwbJKUlpYW1Cfk5+dXuf6BAwdUWVmplJSUoP0pKSkqKSmptuaSkhLT40//byjntELIXym4+OKLVVRUpPLycr322mvKycnR2rVrz9h8TZ06VVOmTAk89nq9NF8AAECSVFxcHHSr0e12O1hN+IXceMXExKhjx46SpIyMDG3ZskVPP/20nn/++WqPd7vdDf5NBACgITt58tRm9TklBVZKMNOyZUtFR0ertLQ0aH9paalSU1OrfU1qaqrp8af/t7S0VK1atQo6pkePHqEMJSTnPJvX7/dXez8WAADACjExMcrIyFBBQUFgn9/vV0FBwRnnmWdlZQUdL0mrV68OHN++fXulpqYGHeP1erVp0ybTuevnKqTEa+rUqRoyZIjatm2rI0eOaOnSpVqzZo3eeeedcNUHAAAcFs7Eq7amTJminJwc9erVS3369NGcOXNUUVGhsWPHSpJuv/12nX/++YE5YpMmTVL//v31xBNP6LrrrtOyZcu0detWLViwQJLkcrk0efJkPfLII7rwwgvVvn17PfTQQ2rdurWGDRtm5VCDhNR4lZWV6fbbb9c333wjj8ejbt266Z133tG//du/has+AAAAjRw5Uvv379e0adNUUlKiHj16aNWqVYHJ8Xv37lVU1L9u5F1xxRVaunSpHnzwQT3wwAO68MILtWLFCnXt2jVwzK9//WtVVFTojjvu0OHDh3XVVVdp1apVio2NDds4znkdr1CxjhdQPdbxOoV1vIBgdWEdr+Ji66996st2HkfG5aTI/KEkAABQa3XhVmNDwT+xAQAAbELiBQAATFVWWp9QVVZae776gsQLAADAJiReAADAFHO8rEPiBQAAYBMSLwAAYIrEyzokXgAAADYh8QIAAKZIvKxD4gUAAGATEi8AAGCKdbysQ+MFAABMcavROtxqBAAAsAmJFwAAMEXiZR0SLwAAAJuQeAEAAFMkXtYh8QIAALAJiRcAADBF4mUdEi8AAACbkHgBAABTLKBqHRovAABgiluN1uFWIwAAgE1IvAAAgCkSL+uQeAEAANiExAsAAJgi8bIOiRcAAIBNSLyAOuKijn6nS6gTvjvGvwclqUks/39A3cFyEtbhbzgAAACbkHgBAABTzPGyDo0XAAAwReNlHW41AgAA2ITECwAAmCLxsg6JFwAAgE1IvAAAgCkSL+uQeAEAANiExAsAAJhiAVXrkHgBAADYhMQLAACYYo6XdWi8AACAKRov63CrEQAAwCYkXgAAwBSJl3VIvAAAAGxC4gUAAEyxnIR1SLwAAABsQuIFAABMMcfLOiReAAAANiHxAgAApki8rEPjBQAATNF4WYdbjQAAADYh8QIAAKZIvKxD4gUAAGATEi8AAGCKBVStQ+IFAABgExIvAABg6uRJKTra+nNGIhIvAAAAm5xT4zVr1iy5XC5NnjzZonIAAEBdc/pbjVZvkeisbzVu2bJFzz//vLp162ZlPQAAoI7hVqN1zirxOnr0qEaPHq2FCxcqMTHR6poAAAAapLNqvHJzc3XdddcpOzu7xmN9Pp+8Xm/QBgAA6o/Ty0lYuUXqchIh32pctmyZtm3bpi1bttTq+Pz8fD388MMhFwYAANDQhJR4FRcXa9KkSVqyZIliY2Nr9ZqpU6eqvLw8sBUXF59VoQAAwBlMrrdOSIlXYWGhysrKdNlllwX2VVZWat26dXr22Wfl8/kU/ZPZd263W26325pqAQAA6rGQGq+BAwdq+/btQfvGjh2rTp066b/+67+qNF0AAKD+O3lSirJ45U8Sr1qIj49X165dg/Y1bdpULVq0qLIfAAAAwfjJIAAAYIrEyzrn3HitWbPGgjIAAEBdReNlHX6rEQAAwCbcagQAAKZOL6Bq9TkjEYkXAACATUi8AACAqZMnJZfL+nNGIhIvAAAAm5B4AQAAUyRe1iHxAgAAsAmJFwAAMEXiZR0aLwAAYIrGyzrcagQAALAJiRcAADBVWWl94sUCqgAAAAgrEi8AAGAqHPOxmOMFAACAsCLxAgAApki8rEPiBQAAYBMSLwAAYIrEyzokXgAAADYh8QIAAKbCseZWpK7jReMFAABMnTwpGYa154zUxotbjQAAADYh8QIAAKZIvKxD4gUAAGATEi8AAGCKxMs6JF4AAAA2ofECAACmTp4MzxYuBw8e1OjRo5WQkKBmzZpp/PjxOnr0qOlrjh07ptzcXLVo0ULnnXeehg8frtLS0qBjXC5XlW3ZsmUh1catRgB1SpNYv9Ml1AmLoqOdLqFOGBep96NwTkaPHq1vvvlGq1ev1okTJzR27FjdcccdWrp06Rlfc/fdd+uvf/2rXn31VXk8HuXl5enmm2/WBx98EHTc4sWLNXjw4MDjZs2ahVQbjRcAADBVWWn9HC9/mP6N9fHHH2vVqlXasmWLevXqJUn6wx/+oJ/97Gd6/PHH1bp16yqvKS8v13//939r6dKlGjBggKRTDVbnzp21ceNGXX755YFjmzVrptTU1LOuj1uNAADAVH261bhhwwY1a9Ys0HRJUnZ2tqKiorRp06ZqX1NYWKgTJ04oOzs7sK9Tp05q27atNmzYEHRsbm6uWrZsqT59+mjRokUyQuxISbwAAIBjvF5v0GO32y23233W5yspKVFycnLQvkaNGql58+YqKSk542tiYmKq3DZMSUkJes1vf/tbDRgwQE2aNNG7776rX/3qVzp69KjuuuuuWtdH4wUAAEydPClFWXyP7PStxrS0tKD906dP14wZM6ocf//99+uxxx4zPefHH39sVXnVeuihhwL/3bNnT1VUVGj27Nk0XgAAoH4oLi5WQkJC4PGZ0q577rlHY8aMMT1Xhw4dlJqaqrKysqD9J0+e1MGDB884Nys1NVXHjx/X4cOHg1Kv0tJS0/lcmZmZmjlzpnw+X61TOhovAABgKpyJV0JCQlDjdSZJSUlKSkqq8bisrCwdPnxYhYWFysjIkCS999578vv9yszMrPY1GRkZaty4sQoKCjR8+HBJ0qeffqq9e/cqKyvrjNcqKipSYmJiSLdGabwAAECD0blzZw0ePFgTJkzQ/PnzdeLECeXl5WnUqFGBbzTu27dPAwcO1EsvvaQ+ffrI4/Fo/PjxmjJlipo3b66EhARNnDhRWVlZgW80vvHGGyotLdXll1+u2NhYrV69Wo8++qjuvffekOqj8QIAAKYqK61f/sHq5Sl+bMmSJcrLy9PAgQMVFRWl4cOH65lnngk8f+LECX366af67rvvAvueeuqpwLE+n0+DBg3Sc889F3i+cePGmjt3ru6++24ZhqGOHTvqySef1IQJE0KqzWWE+j3Ic+T1euXxeFR+6FCtokUAiEQsoHoKC6j+8LmZmKjy8nLbPzdPf2Y3a1Yul8vaaxuGV4cPexwZl5NIvAAAgKmTJyWXy9pz2hv71B00XgAAwBSNl3VYuR4AAMAmJF4AAMAUiZd1SLwAAABsQuIFAABMkXhZh8QLAADAJiReAACgBn4ZhsUrqMrq89UPJF4AAAA2IfECAAA1qPxhs/qckYfGCwAA1IDGyyrcagQAALAJiRcAAKgBiZdVSLwAAABsQuIFAABq4Jf1yz+wnAQAAADCiMQLAADUgDleViHxAgAAsAmJFwAAqIFf1idUzPGq0YwZM+RyuYK2Tp06has2AABQJ1SGaYs8ISdel1xyif72t7/96wSNCM0AAABqI+SuqVGjRkpNTQ1HLQAAoE5icr1VQp5cv2vXLrVu3VodOnTQ6NGjtXfvXtPjfT6fvF5v0AYAABCJQmq8MjMz9cILL2jVqlWaN2+e9uzZo759++rIkSNnfE1+fr48Hk9gS0tLO+eiAQCAnfxh2iKPyzAM42xffPjwYbVr105PPvmkxo8fX+0xPp9PPp8v8Njr9SotLU3lhw4pISHhbC8NAA3aouhop0uoE8ZVRubtqB/zer3yJCaqvLzc9s9Nr9crj8cj6WNJ8Raf/Yikzo6My0nnNDO+WbNmuuiii7R79+4zHuN2u+V2u8/lMgAAwFHM8bLKOS2gevToUX3++edq1aqVVfUAAAA0WCE1Xvfee6/Wrl2rL7/8UuvXr9dNN92k6Oho3XbbbeGqDwAAOI51vKwS0q3Gr776Srfddpu+/fZbJSUl6aqrrtLGjRuVlJQUrvoAAIDjuNVolZAar2XLloWrDgAAgAaPZecBAEAN+K1Gq5zT5HoAAADUHokXAACoQTgWPCXxAgAAQBiReAEAgBrwrUarkHgBAADYhMQLAADUgMTLKjReAACgBjReVuFWIwAAgE1IvAAAQA1IvKxC4gUAAGATEi8AAFADQ9YveGpYfL76gcQLAADAJiReAACgBszxsgqJFwAAgE1IvAAAQA1IvKxC4wUAAGpA42UVbjUCAADYhMQLAADUgMTLKiReAAAANiHxAgAANfDL+gVUrT5f/UDiBQAAYBMSLwAAUAPmeFmFxAsAAMAmJF4AAKAGflmfUDHHCwAAAGFE4gUAddC4ysic//JTrugbnS6hDjjhdAFijpd1aLwAAEANWE7CKtxqBAAAsAmJFwAAqAG3Gq1C4gUAAGATEi8AAFADEi+rkHgBAADYhMQLAADUgMTLKiReAAAANiHxAgAANeAng6xC4wUAAGrAAqpW4VYjAACATUi8AABADZhcbxUSLwAAAJuQeAEAgBqQeFmFxAsAAMAmJF4AAKAGJF5WIfECAACwCYkXAACoAYmXVWi8AABADVhA1SrcagQAALAJiRcAAKgBv9VoFRIvAAAAm5B4AQCAGjC53iokXgAAADYh8QIAADUg8bIKiRcAAIBNSLwAAEANSLysQuMFAABqQONllZBvNe7bt08///nP1aJFC8XFxenSSy/V1q1bw1EbAABAgxJS4nXo0CFdeeWVuuaaa/T2228rKSlJu3btUmJiYrjqAwAAjuMng6wSUuP12GOPKS0tTYsXLw7sa9++veVFAQAANEQh3WpcuXKlevXqpVtvvVXJycnq2bOnFi5caPoan88nr9cbtAEAgPrk9E8GWblFZuIVUuP1xRdfaN68ebrwwgv1zjvv6M4779Rdd92lF1988Yyvyc/Pl8fjCWxpaWnnXDQAAEB95DIMw6jtwTExMerVq5fWr18f2HfXXXdpy5Yt2rBhQ7Wv8fl88vl8gcder1dpaWkqP3RICQkJ51A6AKChc0Xf6HQJdcAJSe+ovLzc9s9Nr9crj8cj6T5JbovP7pM025FxOSmkxKtVq1bq0qVL0L7OnTtr7969Z3yN2+1WQkJC0AYAABAuBw8e1OjRo5WQkKBmzZpp/PjxOnr0qOlrFixYoKuvvloJCQlyuVw6fPiwJef9qZAaryuvvFKffvpp0L7PPvtM7dq1C+miAACgPrF6flc41gX7l9GjR2vnzp1avXq13nzzTa1bt0533HGH6Wu+++47DR48WA888ICl5/2pkG41btmyRVdccYUefvhhjRgxQps3b9aECRO0YMECjR49ulbnOB1bcqsRAFATbjVKdeNW42SF51bjHMvH9fHHH6tLly7asmWLevXqJUlatWqVfvazn+mrr75S69atTV+/Zs0aXXPNNTp06JCaNWtm2XlPCynx6t27t5YvX66XX35ZXbt21cyZMzVnzpxaN10AAADhtGHDBjVr1izQHElSdna2oqKitGnTJsfPG/JPBl1//fW6/vrrQ30ZAACot8L3k0E/XWbK7XbL7T77dK2kpETJyclB+xo1aqTmzZurpKTE8fOG/JNBAAAAVklLSwtadio/P7/a4+6//365XC7T7ZNPPrG5+tDxI9kAAKAG4Uu8iouLg+Z4nSntuueeezRmzBjTM3bo0EGpqakqKysL2n/y5EkdPHhQqampZ12tVeel8QIAAI6p7VJTSUlJSkpKqvG4rKwsHT58WIWFhcrIyJAkvffee/L7/crMzDzrOq06L7caAQBADerPchKdO3fW4MGDNWHCBG3evFkffPCB8vLyNGrUqMA3D/ft26dOnTpp8+bNgdeVlJSoqKhIu3fvliRt375dRUVFOnjwYK3PWxs0XgAAoEFZsmSJOnXqpIEDB+pnP/uZrrrqKi1YsCDw/IkTJ/Tpp5/qu+++C+ybP3++evbsqQkTJkiS+vXrp549e2rlypW1Pm9thLSOlxVYxwsAUFus4yXVjXW8JkiKsfjsxyUtjLifDGKOFwAAqIH/h83qc0YebjUCAADYhMQLAADUIHzLSUQaEi8AAACbkHgBAIAakHhZhcQLAADAJiReAACgBiReViHxAgAAsAmJFwAAqAGJl1VovAAAQA38sr5RYgFVAAAAhBGJFwAAqAE/GWQVEi8AAACbkHgBAIAaVMr6rCYyJ9eTeAEAANiExAsAANSAxMsqJF4AAAA2IfECAAA1IPGyCo0XAACoActJWMX2xsswDEmS1+u1+9IAgHrnhNMF1AEnJf3r89PJGur+Oes+2xuvI0eOSJLS2rWz+9IAANRbR44ckcfjsfWaMTExSk1NVUnJ38Jy/tTUVMXExITl3HWVy7C5hfb7/fr6668VHx8vl8tl56UDvF6v0tLSVFxcrISEBEdqqAt4H07hfTiF9+EU3odTeB9OqQvvg2EYOnLkiFq3bq2oKPu/E3fs2DEdP348LOeOiYlRbGxsWM5dV9meeEVFRalNmzZ2X7ZaCQkJEf0Xymm8D6fwPpzC+3AK78MpvA+nOP0+2J10/VhsbGzENUfhxHISAAAANqHxAgAAsElENl5ut1vTp0+X2+12uhRH8T6cwvtwCu/DKbwPp/A+nML7AKvZPrkeAAAgUkVk4gUAAOAEGi8AAACb0HgBAADYhMYLAADAJhHXeM2dO1fp6emKjY1VZmamNm/e7HRJtlu3bp2GDh2q1q1by+VyacWKFU6XZLv8/Hz17t1b8fHxSk5O1rBhw/Tpp586XZbt5s2bp27dugUWh8zKytLbb7/tdFmOmzVrllwulyZPnux0KbaaMWOGXC5X0NapUyeny3LEvn379POf/1wtWrRQXFycLr30Um3dutXpstAARFTj9corr2jKlCmaPn26tm3bpu7du2vQoEEqKytzujRbVVRUqHv37po7d67TpThm7dq1ys3N1caNG7V69WqdOHFC1157rSoqKpwuzVZt2rTRrFmzVFhYqK1bt2rAgAG68cYbtXPnTqdLc8yWLVv0/PPPq1u3bk6X4ohLLrlE33zzTWD7+9//7nRJtjt06JCuvPJKNW7cWG+//bY++ugjPfHEE0pMTHS6NDQAEbWcRGZmpnr37q1nn31W0qnfjUxLS9PEiRN1//33O1ydM1wul5YvX65hw4Y5XYqj9u/fr+TkZK1du1b9+vVzuhxHNW/eXLNnz9b48eOdLsV2R48e1WWXXabnnntOjzzyiHr06KE5c+Y4XZZtZsyYoRUrVqioqMjpUhx1//3364MPPtD//u//Ol0KGqCISbyOHz+uwsJCZWdnB/ZFRUUpOztbGzZscLAy1AXl5eWSTjUdkaqyslLLli1TRUWFsrKynC7HEbm5ubruuuuC/p6INLt27VLr1q3VoUMHjR49Wnv37nW6JNutXLlSvXr10q233qrk5GT17NlTCxcudLosNBAR03gdOHBAlZWVSklJCdqfkpKikpISh6pCXeD3+zV58mRdeeWV6tq1q9Pl2G779u0677zz5Ha79ctf/lLLly9Xly5dnC7LdsuWLdO2bduUn5/vdCmOyczM1AsvvKBVq1Zp3rx52rNnj/r27asjR444XZqtvvjiC82bN08XXnih3nnnHd15552666679OKLLzpdGhqARk4XADgtNzdXO3bsiMi5LJJ08cUXq6ioSOXl5XrttdeUk5OjtWvXRlTzVVxcrEmTJmn16tWKjY11uhzHDBkyJPDf3bp1U2Zmptq1a6c//elPEXXr2e/3q1evXnr00UclST179tSOHTs0f/585eTkOFwd6ruISbxatmyp6OholZaWBu0vLS1VamqqQ1XBaXl5eXrzzTf1/vvvq02bNk6X44iYmBh17NhRGRkZys/PV/fu3fX00087XZatCgsLVVZWpssuu0yNGjVSo0aNtHbtWj3zzDNq1KiRKisrnS7REc2aNdNFF12k3bt3O12KrVq1alXlHx6dO3eOyNuusF7ENF4xMTHKyMhQQUFBYJ/f71dBQUHEzmeJZIZhKC8vT8uXL9d7772n9u3bO11SneH3++Xz+Zwuw1YDBw7U9u3bVVRUFNh69eql0aNHq6ioSNHR0U6X6IijR4/q888/V6tWrZwuxVZXXnllleVlPvvsM7Vr186hitCQRNStxilTpignJ0e9evVSnz59NGfOHFVUVGjs2LFOl2aro0ePBv0Lds+ePSoqKlLz5s3Vtm1bByuzT25urpYuXarXX39d8fHxgXl+Ho9HcXFxDldnn6lTp2rIkCFq27atjhw5oqVLl2rNmjV65513nC7NVvHx8VXm9zVt2lQtWrSIqHl/9957r4YOHap27drp66+/1vTp0xUdHa3bbrvN6dJsdffdd+uKK67Qo48+qhEjRmjz5s1asGCBFixY4HRpaAiMCPOHP/zBaNu2rRETE2P06dPH2Lhxo9Ml2e799983JFXZcnJynC7NNtWNX5KxePFip0uz1bhx44x27doZMTExRlJSkjFw4EDj3XffdbqsOqF///7GpEmTnC7DViNHjjRatWplxMTEGOeff74xcuRIY/fu3U6X5Yg33njD6Nq1q+F2u41OnToZCxYscLokNBARtY4XAACAkyJmjhcAAIDTaLwAAABsQuMFAABgExovAAAAm9B4AQAA2ITGCwAAwCY0XgAAADah8QIAALAJjRcAAIBNaLwAAABsQuMFAABgExovAAAAm/w/P6WRuNs+u68AAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "imshow_zero_center(j[:, 0, :, 0])\n", "_ = plt.title('A (batch, batch) slice')" ] }, { "cell_type": "code", "execution_count": 33, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:29.983183Z", "iopub.status.busy": "2022-12-14T21:28:29.982667Z", "iopub.status.idle": "2022-12-14T21:28:30.209426Z", "shell.execute_reply": "2022-12-14T21:28:30.208617Z" }, "id": "g4ZoRJcJNmy5" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhEAAAIQCAYAAAA/wjy3AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABU/klEQVR4nO3de1xU1fo/8M+AMiDKTe6JomIiKlKYhFlqkOAt7ViJXzsCeTmaVIRW0im8JpqmZHqkMjVP+sM0NesYiih6POINo9TUojTxMngLEFQuM/v3hzE5AbNnhj0ww/68X6/9qtmzZu21lzPw8Oy1n1EIgiCAiIiIyEg2TT0AIiIisk4MIoiIiMgkDCKIiIjIJAwiiIiIyCQMIoiIiMgkDCKIiIjIJAwiiIiIyCQMIoiIiMgkDCKIiIjIJAwiJKRQKDBr1izt47Vr10KhUOD8+fMGvf6ll17CU089pX2ck5MDhUKBzZs3SzxS0ykUCiQkJEjap7+/P4YNGyZpnw0RFxeH1q1bi7a7ceMGHB0dsWPHDsmOPWvWLCgUCp19/v7+iIuLk+wYzd1f56vmc5STk9NkYzKn5n5+ZNkYRBjoX//6FxQKBcLCwszS/7lz57Bq1Sq89dZbZum/xo4dO3QCnebo8uXLmDVrFvLz8816nLZt22LChAl45513zHocIiJLxSDCQOvXr4e/vz+OHDmCgoICyfv/4IMP0LFjRwwcOFDyvu+3Y8cOzJ4926zHaGqXL1/G7NmzzR5EAMDkyZNx/Phx7Nmzx2zHOHv2LD755BOz9d/cPfHEE7hz5w6eeOKJph4KUbPDIMIA586dw8GDB7FkyRJ4eHhg/fr1kvZfVVWF9evX4/nnn5e0XzK/bt26oUePHli7dq3ZjqFUKtGyZUuz9d/YBEHAnTt3Gu14NjY2sLe3h40Nf9wRSY2fKgOsX78erq6uGDp0KJ599lnJg4gDBw7g+vXriIyMrPN5tVqNt956C97e3nB0dMTTTz+NwsJCnTb//e9/8dxzz6F9+/ZQKpXw8/PDa6+9pvPDOi4uDitWrABwb21DzVZDo9Hggw8+QM+ePWFvbw8PDw9ER0fj2LFjtca0bds29OjRA0qlEt27d0dmZmaD52HXrl0ICQmBvb09goKCsGXLFp3nb968ienTp6Nnz55o3bo1nJycMHjwYHz//ffaNjk5OXjkkUcAAPHx8dpzvP+X/OHDhzFkyBC4urrC0dERwcHB+OCDD2qN59KlSxg5ciRat24NDw8PTJ8+HWq1ula7p556Cl9//TXEvhC3qqoKs2fPRpcuXWBvb4+2bduiX79+yMrK0vu6utZEFBcX47XXXoO/vz+USiXatWuHcePG4fr169o2FRUVmDlzJgICArTviTfeeAMVFRU6fWVlZaFfv35wcXFB69at0bVrV4Muq61ZswZPPvkkPD09oVQqERQUhJUrV9Y5/mHDhmHnzp3o3bs3HBwc8NFHH2nPIzExEX5+flAqlQgICMDChQuh0WhEjy8IAubNm4d27dqhVatWGDhwIE6dOlWrXV1rBgz5vNTYtGkTgoKCYG9vjx49emDr1q2Ii4uDv7+/Trvy8nJMmzZNey5du3bF4sWLa70vatYViX2GfvvtN7z00kvo2rUrHBwc0LZtWzz33HMGr7EiagwtmnoA1mD9+vX429/+Bjs7O4wZMwYrV67E0aNHtb+sGurgwYNQKBR46KGH6nz+3XffhUKhwJtvvomrV68iLS0NkZGRyM/Ph4ODA4B7P+hu376NKVOmoG3btjhy5Ag+/PBDXLx4EZs2bQIA/OMf/8Dly5eRlZWFf//737WOM378eKxduxaDBw/GhAkTUF1djf/+9784dOgQevfurW134MABbNmyBS+99BLatGmDZcuWYdSoUbhw4QLatm1r0hz8/PPPGD16NCZPnozY2FisWbMGzz33HDIzM7WLTX/99Vds27YNzz33HDp27IiioiJ89NFH6N+/P3788Uf4+vqiW7dumDNnDlJSUjBp0iQ8/vjjAIC+ffsCuPcLc9iwYfDx8cGrr74Kb29vnD59Gt988w1effVV7XjUajWioqIQFhaGxYsXY/fu3Xj//ffRuXNnTJkyRWfsoaGhWLp0KU6dOoUePXrUe46zZs1CamoqJkyYgD59+qC0tBTHjh3D8ePHdRbUiikrK8Pjjz+O06dP48UXX8TDDz+M69evY/v27bh48SLc3d2h0Wjw9NNP48CBA5g0aRK6deuGEydOYOnSpfjpp5+wbds2AMCpU6cwbNgwBAcHY86cOVAqlSgoKMD//vc/0XGsXLkS3bt3x9NPP40WLVrg66+/xksvvQSNRoOpU6fqtD179izGjBmDf/zjH5g4cSK6du2K27dvo3///rh06RL+8Y9/oH379jh48CCSk5Nx5coVpKWl6T1+SkoK5s2bhyFDhmDIkCE4fvw4Bg0ahMrKStGxG/J5AYD//Oc/GD16NHr27InU1FT8/vvvGD9+PB544AGd/gRBwNNPP429e/di/PjxCAkJwc6dO/H666/j0qVLWLp0qU57Qz5DR48excGDBxETE4N27drh/PnzWLlyJQYMGIAff/wRrVq1Ej1PIrMTSK9jx44JAISsrCxBEARBo9EI7dq1E1599dVabQEIM2fO1D5es2aNAEA4d+6c3mO88MILQtu2bWvt37t3rwBAeOCBB4TS0lLt/i+++EIAIHzwwQfafbdv3671+tTUVEGhUAi//fabdt/UqVOFuv7Z9+zZIwAQXnnllVrPaTQanXO0s7MTCgoKtPu+//57AYDw4Ycf6j3P+nTo0EEAIHz55ZfafSUlJYKPj4/w0EMPaffdvXtXUKvVOq89d+6coFQqhTlz5mj3HT16VAAgrFmzRqdtdXW10LFjR6FDhw7C77//Xu85xsbGCgB0+hQEQXjooYeE0NDQWuM/ePCgAEDYuHGj3vPs1auXMHToUL1tZs6cWevfp0OHDkJsbKz2cUpKigBA2LJlS63X15zHv//9b8HGxkb473//q/N8enq6AED43//+JwiCICxdulQAIFy7dk3vuOpS13suKipK6NSpU63xAxAyMzN19s+dO1dwdHQUfvrpJ539M2bMEGxtbYULFy7Ue+yrV68KdnZ2wtChQ3X+7d566y0BgM581XyO9u7dq3fsdX1eevbsKbRr1064deuWdl9OTo4AQOjQoYN237Zt2wQAwrx583T6fPbZZwWFQqHzeTH0M1TXGHNzcwUAwrp16/SeH1Fj4eUMEevXr4eXl5d2waNCocDo0aORkZFRZ2rbFDdu3ICrq2u9z48bNw5t2rTRPn722Wfh4+Ojc2thTUYCuJdWvX79Ovr27QtBEPDdd9+JjuHLL7+EQqHAzJkzaz3311sOIyMj0blzZ+3j4OBgODk54ddffxU9Tn18fX3xzDPPaB87OTlh3Lhx+O6776BSqQDcWxtQc11brVbjxo0b2vT78ePHRY/x3Xff4dy5c0hMTISLi4vOc389R+Deosn7Pf7443WeY82/3f2XEuri4uKCU6dO4eeffxYdqz5ffvklevXqpTNfNWrOY9OmTejWrRsCAwNx/fp17fbkk08CAPbu3asdEwB89dVXBl1CuN/977mSkhJcv34d/fv3x6+//oqSkhKdth07dkRUVJTOvk2bNuHxxx+Hq6urzhgjIyOhVquxf//+eo+9e/duVFZW4uWXX9b5t0tMTDR67PV9Xi5fvowTJ05g3LhxOrf89u/fHz179tTpb8eOHbC1tcUrr7yis3/atGkQBAHffvutzn5DPkP3j7Gqqgo3btxAQEAAXFxcDHq/EzUGBhF6qNVqZGRkYODAgTh37hwKCgpQUFCAsLAwFBUVITs7W7JjCXqup3fp0kXnsUKhQEBAgM610QsXLiAuLg5ubm7aa/j9+/cHgFo/0Ovyyy+/wNfXF25ubqJt27dvX2ufq6srfv/9d9HX1icgIKDWL/IHH3wQALTnqdFosHTpUnTp0gVKpRLu7u7w8PDADz/8YPA5AtB7yaFGzZqQ+9V3jjX/dnUFIvebM2cOiouL8eCDD6Jnz554/fXX8cMPP4iO5a9++eUX0XP4+eefcerUKXh4eOhsNXN69epVAMDo0aPx2GOPYcKECfDy8kJMTAy++OILgwKK//3vf4iMjISjoyNcXFzg4eGhXUtRVxBR1xgzMzNrjbFmbVDNGOvy22+/Aaj92fDw8NAbkNcw5PNSc4yAgIBar//rvt9++w2+vr46wT5wb+Ht/X3VMOQzdOfOHaSkpGjXWNS834uLiw16vxM1Bq6J0GPPnj24cuUKMjIykJGRUev59evXY9CgQQ0+Ttu2bRv0C1itVuOpp57CzZs38eabbyIwMBCOjo64dOkS4uLijP4LU4ytrW2d+/UFQlKYP38+3nnnHbz44ouYO3cu3NzcYGNjg8TExEY7x7rU/Nu5u7vrbffEE0/gl19+wVdffYVdu3Zh1apVWLp0KdLT0zFhwoQGjfevNBoNevbsiSVLltT5vJ+fH4B7f+3u378fe/fuxX/+8x9kZmZi48aNePLJJ7Fr16565+GXX35BREQEAgMDsWTJEvj5+cHOzg47duzA0qVLa/173P9X9f1jfOqpp/DGG2/UeYyagEdqjf15qYshn6GXX34Za9asQWJiIsLDw+Hs7AyFQoGYmJhGGSORIRhE6LF+/Xp4enpq72i435YtW7B161akp6fX+QPSGIGBgVi/fj1KSkrg7Oxc6/m/pr8FQUBBQQGCg4MBACdOnMBPP/2Ezz77DOPGjdO2q2vVf31/LXfu3Bk7d+7EzZs3DcpGSK2goACCIOiM76effgIA7Sr4zZs3Y+DAgfj00091XltcXKzzC1zfOQLAyZMn670TxhTnzp0D8Odfnfq4ubkhPj4e8fHxKCsrwxNPPIFZs2YZFUR07twZJ0+eFG3z/fffIyIiQjRDYmNjg4iICERERGDJkiWYP38+/vnPf2Lv3r31ztPXX3+NiooKbN++Xeev6prLJIaeR1lZmUn/Fh06dABw77PRqVMn7f5r166JBuSGfl5qjlFXXZi/7uvQoQN2796NW7du6WQjzpw5o9OXMTZv3ozY2Fi8//772n13795FcXGx0X0RmQsvZ9Tjzp072LJlC4YNG4Znn3221paQkIBbt25h+/btDT5WeHg4BEFAXl5enc+vW7cOt27d0j7evHkzrly5gsGDBwP486+a+/+KEQShztsWHR0dAaDWD6JRo0ZBEIQ6C1GZO8MA3Lv+vHXrVu3j0tJSrFu3DiEhIfD29gZw7zz/OpZNmzbh0qVLOvvqO8eHH34YHTt2RFpaWq3nGnKOeXl5cHZ2Rvfu3fW2u3Hjhs7j1q1bIyAgoNYtl2JGjRqF77//Xme+atScx/PPP49Lly7VWaTqzp07KC8vB3Dvttm/CgkJAQC946rrPVdSUoI1a9YYfB7PP/88cnNzsXPnzlrPFRcXo7q6ut7XRkZGomXLlvjwww91xiB2R0d9Y6/r8+Lr64sePXpg3bp1KCsr0+7ft28fTpw4odN2yJAhUKvVWL58uc7+pUuXQqFQaD+rxqjr/f7hhx9KthaLSArMRNRj+/btuHXrFp5++uk6n3/00Ue1hadGjx7doGP169cPbdu2xe7du7UL3+7n5uaGfv36IT4+HkVFRUhLS0NAQAAmTpwI4F4mo3Pnzpg+fTouXboEJycnfPnll3X+RRYaGgoAeOWVVxAVFQVbW1vExMRg4MCB+Pvf/45ly5bh559/RnR0NDQaDf773/9i4MCBJn1fhkKhQP/+/Q2q6f/ggw9i/PjxOHr0KLy8vLB69WoUFRXp/FIaNmwY5syZg/j4ePTt2xcnTpzA+vXrdf4SBe79hevi4oL09HS0adMGjo6OCAsLQ8eOHbFy5UoMHz4cISEhiI+Ph4+PD86cOYNTp07V+cvMEFlZWRg+fLjoX/xBQUEYMGAAQkND4ebmhmPHjmHz5s1Gz+3rr7+OzZs347nnnsOLL76I0NBQ3Lx5E9u3b0d6ejp69eqFv//97/jiiy8wefJk7N27F4899hjUajXOnDmDL774QluzYc6cOdi/fz+GDh2KDh064OrVq/jXv/6Fdu3aoV+/fvWOYdCgQbCzs8Pw4cPxj3/8A2VlZfjkk0/g6emJK1euGHwe27dvx7BhwxAXF4fQ0FCUl5fjxIkT2Lx5M86fP1/vJaKauh2pqakYNmwYhgwZgu+++w7ffvut6GUlYz4v8+fPx4gRI/DYY48hPj4ev//+O5YvX44ePXroBBbDhw/HwIED8c9//hPnz59Hr169sGvXLnz11VdITEzUWURpqGHDhuHf//43nJ2dERQUhNzcXOzevdvk26iJzKIxbwWxJsOHDxfs7e2F8vLyetvExcUJLVu2FK5fvy4Igum3eAqCILzyyitCQECAzr6aW7f+3//7f0JycrLg6ekpODg4CEOHDtW5DU0QBOHHH38UIiMjhdatWwvu7u7CxIkTtbeN3X+rY3V1tfDyyy8LHh4egkKh0LmdsLq6Wli0aJEQGBgo2NnZCR4eHsLgwYOFvLw8bRsAwtSpU2uN/6+3Id66dUsAIMTExIiee4cOHYShQ4cKO3fuFIKDgwWlUikEBgYKmzZt0ml39+5dYdq0aYKPj4/g4OAgPPbYY0Jubq7Qv39/oX///jptv/rqKyEoKEho0aJFrTk4cOCA8NRTTwlt2rQRHB0dheDgYJ1b62JjYwVHR8da46zr9svTp08LAITdu3eLnue8efOEPn36CC4uLoKDg4MQGBgovPvuu0JlZaXeY/x1bgVBEG7cuCEkJCQIDzzwgGBnZye0a9dOiI2N1b4XBUEQKisrhYULFwrdu3cXlEql4OrqKoSGhgqzZ88WSkpKBEEQhOzsbGHEiBGCr6+vYGdnJ/j6+gpjxoypddtlXbZv3y4EBwcL9vb2gr+/v7Bw4UJh9erVtd7zNf++dbl165aQnJwsBAQECHZ2doK7u7vQt29fYfHixTrzUhe1Wi3Mnj1b+34YMGCAcPLkyVrzVdctkIZ+XgRBEDIyMoTAwEBBqVQKPXr0ELZv3y6MGjVKCAwMrHUur732muDr6yu0bNlS6NKli7Bo0SKdW1AFwfDP0O+//y7Ex8cL7u7uQuvWrYWoqCjhzJkzBp0fUWNRCEIj5KpJ1K+//orAwEB8++23iIiIaOrhNNiOHTswbNgwfP/997Vuh2tOEhMTsX//fuTl5YlmIqj5CAkJgYeHh2i1UaLmjmsiLESnTp0wfvx4LFiwoKmHIom9e/ciJiamWQcQN27cwKpVqzBv3jwGEM1UVVVVrbUZOTk5+P777zFgwICmGRSRBWEmgoioHufPn0dkZCReeOEF+Pr64syZM0hPT4ezszNOnjzJ9Qkke1xYSURUD1dXV4SGhmLVqlW4du0aHB0dMXToUCxYsIABBBGYiSAiIiITcU0EERERmYRBBBERUSPbv38/hg8fDl9fXygUCmzbtk30NTk5OXj44YehVCoREBCAtWvX1mqzYsUK+Pv7w97eHmFhYThy5Ij0g79Po6+J0Gg0uHz5Mtq0acMV7UREMiMIAm7dugVfX1/tt/I2prt376KystIsfdvZ2cHe3t6gtuXl5ejVqxdefPFF/O1vfxNtf+7cOQwdOhSTJ0/G+vXrkZ2djQkTJsDHx0f7DbkbN25EUlIS0tPTERYWhrS0NERFReHs2bPw9PRs0LnVq7ELUxQWFgoAuHHjxo2bjLfCwsLG/vUj3LlzR2htxnPy9vYW7ty5Y/S4AAhbt27V2+aNN94QunfvrrNv9OjRQlRUlPZxnz59dAqZqdVqwdfXV0hNTTV6TIZq9EzEn19OEwHeHEJEJDfVALJrfW16Y6isrEQZgNcAKCXuuwLAUpUK169fh5OTk3a/UqmEUtnwo+Xm5tb6srqoqCgkJiYCuHdueXl5SE5O1j5vY2ODyMhI5ObmNvj49Wn03+J/XsJoAaBlYx+eiIgsQFNeznYAYNhFB8PVXJjx8/PT2T9z5kzMmjWrwf2rVCp4eXnp7PPy8kJpaSnu3LmD33//HWq1us42Nd8maw5MBRAREUmksLCwViaiOWMQQUREsmID6W9NrOnPyclJJ4iQire3N4qKinT2FRUVwcnJCQ4ODrC1tYWtrW2dbby9vSUfTw3e4klERGThwsPDkZ2drbMvKysL4eHhAO7dGRIaGqrTRqPRIDs7W9vGHJiJICIiWTFnJsJQZWVlKCgo0D4+d+4c8vPz4ebmhvbt2yM5ORmXLl3CunXrAACTJ0/G8uXL8cYbb+DFF1/Enj178MUXX+A///mPto+kpCTExsaid+/e6NOnD9LS0lBeXo74+HgpTrFODCKIiEhWLCGIOHbsGAYOHKh9nJSUBACIjY3F2rVrceXKFVy4cEH7fMeOHfGf//wHr732Gj744AO0a9cOq1at0taIAIDRo0fj2rVrSElJgUqlQkhICDIzM2sttpRSo393RmlpKZydnQFEgXdnEBHJTRWAnSgpKTHL2gF9an7/zIL0d2fcBTALaJLzakoWm4kQyjfqb2BgVTBroLAdIUk/77//td7nkxI1khzHEkg1Z4L6K0n6sQZSzZlKpf995uXB99lfvfyy/jlblsY5a0yWkIloLuR63kRERNRAFpuJICIiMgdmIqQj1/MmIiKiBmImgoiIZIWZCOnI9byJiIiogZiJICIiWWEmQjpyPW8iIiJqIMvNRIjVgSgrE++jdWtpxmIlxOpAPDFAPGbcn9N87leXxMmT4m169DD/OCyIWB2I0jLx95lTa3m9z8TqQKTMEp+zObPkNWfmpID0f0E33RebNy3LDSKIiIjMQAHpf+nLNYjg5QwiIiIyCTMRREQkK7Z/bFL3KUfMRBAREZFJmIkgIiJZ4S2e0pHreRMREVEDMRNBRESywkyEdOR63kRERNRA1puJMKCQVGW1/hjJroW8ircYUkhqQ4b+Ofu/GHnNmSGFpA4f1T9nYY/Ia84MKSTFOdNlSCEp33b65+zyRXnNWUMwEyEd6w0iiIiITMAgQjpyPW8iIiJqIGYiiIhIVpiJkI5cz5uIiIgaiJkIIiKSFWYipCPX8yYiIqIGYiaCiIhkhV8FLh2TgogVK1Zg0aJFUKlU6NWrFz788EP06dNH6rE1mFgdCLE6Eob00dyI1YHYf0B8zp7oJ685E61pcPeueCf29tIMxkrIrQ6EFMTqQOzJEf9sPjmA807SMvpyxsaNG5GUlISZM2fi+PHj6NWrF6KionD16lVzjI+IiEhSNvjz68Cl2uS6NsDo816yZAkmTpyI+Ph4BAUFIT09Ha1atcLq1avNMT4iIiJJ2ZhpkyOjzruyshJ5eXmIjIz8swMbG0RGRiI3N1fywREREZHlMmpNxPXr16FWq+Hl5aWz38vLC2fOnKnzNRUVFaioqNA+Li0tNWGYRERE0uAtntIx+3mnpqbC2dlZu/n5+Zn7kERERNQIjAoi3N3dYWtri6KiIp39RUVF8Pb2rvM1ycnJKCkp0W6FhYWmj5aIiKiBuCZCOkadt52dHUJDQ5Gdna3dp9FokJ2djfDw8Dpfo1Qq4eTkpLMRERGR9TO6TkRSUhJiY2PRu3dv9OnTB2lpaSgvL0d8fLw5xkdERCQpromQjtFBxOjRo3Ht2jWkpKRApVIhJCQEmZmZtRZbWgNDCklpRN4aNpBX8RZDCkl1665/zk6fktecGVRI6uRJ/c/36CHNWJqRCxf1v8/at5PX+8yQQlJvva1/zubPk9ecUcOZVLEyISEBCQkJUo+FiIjI7JiJkA6/O4OIiGSFQYR05HreRERE1EDMRBARkazwWzylw0wEERERmYSZCCIikpWab96Uuk85YiaCiIiITMJMhAjROhD5+eKdhIRIMRSrIVYHos+j4rHrkUMyu19drA6ESiXeRz2l55srsToQ278Rf589PUxe7zOxOhDt/cXn7MJ5658zBaT/C5prIoiIiIiMwEwEERHJCutESIdBBBERyQqDCOnI9byJiIia1IoVK+Dv7w97e3uEhYXhyJEj9bYdMGAAFApFrW3o0KHaNnFxcbWej46ONus5MBNBRESyYgmZiI0bNyIpKQnp6ekICwtDWloaoqKicPbsWXh6etZqv2XLFlRWVmof37hxA7169cJzzz2n0y46Ohpr1qzRPlYqlUaOzDjMRBARETWyJUuWYOLEiYiPj0dQUBDS09PRqlUrrF69us72bm5u8Pb21m5ZWVlo1apVrSBCqVTqtHN1dTXreTCIICIiWbEx0wYApaWlOltFRUWt41dWViIvLw+RkZF/jsnGBpGRkcjNzTXoHD799FPExMTA0dFRZ39OTg48PT3RtWtXTJkyBTdu3DCoP1MxiCAiIpKIn58fnJ2dtVtqamqtNtevX4darYaXl5fOfi8vL6gMqAlz5MgRnDx5EhMmTNDZHx0djXXr1iE7OxsLFy7Evn37MHjwYKjV6oadlB5cE9FQBhSSOniIsdr9DCkkNXgo50yHIYWk7t41/zisiCGFpNZ9zvfZ/QwpJLV6rfXPmTnXRBQWFsLJyUm73xxrEj799FP07NkTffr00dkfExOj/f+ePXsiODgYnTt3Rk5ODiIiIiQfB8BMBBERkWScnJx0trqCCHd3d9ja2qKoqEhnf1FREbxF/mAoLy9HRkYGxo8fLzqWTp06wd3dHQUFBcadhBEYRBARkawozLQZys7ODqGhocjOztbu02g0yM7ORnh4uN7Xbtq0CRUVFXjhhRdEj3Px4kXcuHEDPj4+RozOOAwiiIhIVmzNtBkjKSkJn3zyCT777DOcPn0aU6ZMQXl5OeLj4wEA48aNQ3Jycq3Xffrppxg5ciTatm2rs7+srAyvv/46Dh06hPPnzyM7OxsjRoxAQEAAoqKijByd4bgmgoiIqJGNHj0a165dQ0pKClQqFUJCQpCZmaldbHnhwgXY2Oj+nX/27FkcOHAAu3btqtWfra0tfvjhB3z22WcoLi6Gr68vBg0ahLlz55q1VgSDCCIikhVLKDYFAAkJCUhISKjzuZycnFr7unbtCkEQ6mzv4OCAnTt3mjCKhuHlDCIiIjIJMxFERCQrCkj/F7QxCyubE4sNIhS2I5p6CFaHc2Y8zpnxOGfG45xRc2WxQQQREZE5WMqaiOZArudNREREDcRMBBERyQozEdJhEEFERLLCIEI6cj1vIiIiaiBmIoiISFaYiZCOXM+biIiIGoiZCCIikhVjv3XT0D7liJkIIiIiMgkzEUREJCumfHW3IX3KETMRREREZBJmIoiISFZ4d4Z0GEQQEZGsMIiQjlzPm4iIiBqImQgiIpIVBaT/C5q3eBIREREZgZkIIiKSFa6JkI7FBhFF+Ebv855qdSONxPwUtiMk6cfV9Wu9z9+8rpHkOJZAqjkTTi3U3yAwUJLjWAKp5gx4VO+zgjpZouM0PanmbK7Iz7O3+fOMrJTFBhFERETmwEyEdOR63kRERNRAzEQQEZGsMBMhHbmeNxERETUQMxFERCQr/Cpw6TCIICIiWeG3eEqHlzOIiIjIJBabiRCtA/Hss+KdbN4szWCshFgdiId7i8eMx481n1oSBhGrA3HsmHgfvXtLMxYrIVYHQmG7wYA+/k+q4VgFsToQH9uK/x07qRnVkmhqXFgpHbmeNxERETWQxWYiiIiIzIGZCOnI9byJiIiogZiJICIiWWEmQjpyPW8iIiJqIGYiiIhIVhSQ/i9oFpsiIiKSAV7OkI5cz5uIiIgayHozEYYUklKp9D/v7S3NWKyEIYWkbhbrjyvdXGRWjEpmhaSkYEghqaJr+t9nXh7yep8ZVEiKP88kw0yEdOR63kRERNRA1puJICIiMgEzEdKR63kTERFRAzETQUREssJMhHTket5ERETUQMxEEBGRrDATIR0GEUREJCsMIqRjdBCxf/9+LFq0CHl5ebhy5Qq2bt2KkSNHmmFoEhC7bzoxUbyPtDQpRmI1xOpAtPcX/6hcOC+ve/xF5eeLtwkJMfcoLIpYHQiF7VrRPgR1nDSDsRYiP8+W29qKdpFgSD0KIiMYHTyVl5ejV69eWLFihTnGQ0REZFY2ZtqMtWLFCvj7+8Pe3h5hYWE4cuRIvW3Xrl0LhUKhs9nb2+u0EQQBKSkp8PHxgYODAyIjI/Hzzz+bMDLDGX3egwcPxrx58/DMM8+YYzxERETN3saNG5GUlISZM2fi+PHj6NWrF6KionD16tV6X+Pk5IQrV65ot99++03n+ffeew/Lli1Deno6Dh8+DEdHR0RFReHu3btmOw+5XsYhIiKZsoRMxJIlSzBx4kTEx8cjKCgI6enpaNWqFVavXl3vaxQKBby9vbWbl5eX9jlBEJCWloa3334bI0aMQHBwMNatW4fLly9j27ZtRo7OcGYPIioqKlBaWqqzERERyVVlZSXy8vIQGRmp3WdjY4PIyEjk5ubW+7qysjJ06NABfn5+GDFiBE6dOqV97ty5c1CpVDp9Ojs7IywsTG+fDWX2ICI1NRXOzs7azc/Pz9yHJCIiqpcCqLW+oMHbH33/9Y/mioqKWse/fv061Gq1TiYBALy8vKCq54vWunbtitWrV+Orr77C559/Do1Gg759++LixYsAoH2dMX1KwexBRHJyMkpKSrRbYWGhuQ9JRETUJPz8/HT+cE5NTZWk3/DwcIwbNw4hISHo378/tmzZAg8PD3z00UeS9G8qs9eJUCqVUCqV5j4MERGRYVq0ABQK8XbGEASguhqFhYVwcnLS7q7r95+7uztsbW1RVFSks7+oqAjeBn6le8uWLfHQQw+hoKAAALSvKyoqgo+Pj06fIWa8hdzoTERZWRny8/OR/8e97+fOnUN+fj4uXLgg9diIiIik16KFeTbcu4Pi/q2uIMLOzg6hoaHIzs7W7tNoNMjOzkZ4eLhBp6BWq3HixAltwNCxY0d4e3vr9FlaWorDhw8b3KcpjM5EHDt2DAMHDtQ+TkpKAgDExsZi7dq1kg2sURhSSComRv/zGRmSDMVaGFJIytNbf2x6VSWzYlSG/BVw6JD+5x99VJKhWAtDCkkpbPWniQV1skSjsQ6GFJKaJVKQahaLUTWapKQkxMbGonfv3ujTpw/S0tJQXl6O+Ph4AMC4cePwwAMPaC+HzJkzB48++igCAgJQXFyMRYsW4bfffsOECRMA3FvjkZiYiHnz5qFLly7o2LEj3nnnHfj6+pq1IKTRQcSAAQMgCII5xkJERGR+5rqcYYTRo0fj2rVrSElJgUqlQkhICDIzM7ULIy9cuAAbmz//IPv9998xceJEqFQquLq6IjQ0FAcPHkRQUJC2zRtvvIHy8nJMmjQJxcXF6NevHzIzM2sVpZISvzuDiIioCSQkJCAhIaHO53JycnQeL126FEuXLtXbn0KhwJw5czBnzhyphiiKQQQREcmLBWQimgtWrCQiIiKTMBNBRETyYmsL2Ej8N7RGZgvG/8BMBBEREZmEmQgiIpKXFi2YiZAIgwgxInUgqkXuuwaAFjK791qsDoSDo/iH9065zD6QYnUgqqvF+2ghr4+zWB0Ihe1OA/qIkmo4VkGsDsQOA36eDWkOP88YREiGlzOIiIjIJPL604WIiIiZCMkwE0FEREQmYSaCiIjkxdb23ial5rBWxATMRBAREZFJmIkgIiJ5adFC+kyE1GW0rQQzEURERGQSZiKIiEhemImQDIOIBjKokNSxY+YfiBUxpJDU4aNMkukwpJBUWZn5x2FFDCkktSeH77P7GVRIats2s4/D7BhESIafICIiIjIJMxFERCQvzERIhpkIIiIiMgkzEUREJC+2trL7wjpzYSaCiIiITMJQjIiI5KVFC2YiJMJMBBEREZnEYkMxhe2Iph6C1eGcGY9zZjzOmfE4ZxaGmQjJcBaJiEheGERIhpcziIiIyCQMxYiISF7McYunIEjbn5VgJoKIiIhMwkwEERHJiznWRDATQURERGQ4ZiKIiEhemImQDDMRREREZBJmIoiISF6YiZAMgwgiIpIXBhGS4eUMIiIiMgkzEUREJC/mKDal0Ujbn5VgJoKIiIhMwkwEERHJiznWRDATQURERGQ4ZiKIiEhemImQDDMRREREZBKLzUQIE330N0hPb5yBNAKF7QhJ+vntt6/1Pt++XfOJlKWas5IS/XPm1Jpz9lfl5frnrJU95+yvDh3SP2dhj3DOGhUzEZJhJoKIiIhMYrGZCCIiIrNgJkIyDCKIiEhezFFsSq2Wtj8rwcsZREREZBJmIoiISF7McTmDmQgiIiIiwzGIICIieanJREi9GWnFihXw9/eHvb09wsLCcOTIkXrbfvLJJ3j88cfh6uoKV1dXREZG1mofFxcHhUKhs0VHRxs9LmNY7uUMsToQMTHifWRkSDMWKyFWB+Jf6eIx40uT5bXCuDnVgWgsonUgysrEO2ndWprBWAmxOhAfrxL/bE6awPdqc7Jx40YkJSUhPT0dYWFhSEtLQ1RUFM6ePQtPT89a7XNycjBmzBj07dsX9vb2WLhwIQYNGoRTp07hgQce0LaLjo7GmjVrtI+VSqVZz4OZCCIikhcLyEQsWbIEEydORHx8PIKCgpCeno5WrVph9erVdbZfv349XnrpJYSEhCAwMBCrVq2CRqNBdna2TjulUglvb2/t5urqavI0GYJBBBERkURKS0t1toqKilptKisrkZeXh8jISO0+GxsbREZGIjc316Dj3L59G1VVVXBzc9PZn5OTA09PT3Tt2hVTpkzBjRs3GnZCIhhEEBGRvNTUiZBys7UFAPj5+cHZ2Vm7paam1jr89evXoVar4eXlpbPfy8sLKpXKoFN488034evrqxOIREdHY926dcjOzsbChQuxb98+DB48GGoz3jliuWsiiIiIzMEct3j+0V9hYSGcnJy0u82xJmHBggXIyMhATk4O7O3ttftj7lsr2LNnTwQHB6Nz587IyclBRESE5OMAmIkgIiKSjJOTk85WVxDh7u4OW1tbFBUV6ewvKiqCt7e33v4XL16MBQsWYNeuXQgODtbbtlOnTnB3d0dBQYHxJ2IgBhFERCQvTbyw0s7ODqGhoTqLImsWSYaHh9f7uvfeew9z585FZmYmevfuLXqcixcv4saNG/DxEflW7AZgEEFERNTIkpKS8Mknn+Czzz7D6dOnMWXKFJSXlyM+Ph4AMG7cOCQnJ2vbL1y4EO+88w5Wr14Nf39/qFQqqFQqlP1xS3VZWRlef/11HDp0COfPn0d2djZGjBiBgIAAREVFme08uCaCiIjkxYxrIgw1evRoXLt2DSkpKVCpVAgJCUFmZqZ2seWFCxdgY/Pn3/krV65EZWUlnn32WZ1+Zs6ciVmzZsHW1hY//PADPvvsMxQXF8PX1xeDBg3C3LlzzVorwnqDCAMKSVX/sVq2Pi1kVuvcoEJSd+/qf/6+RTx0z49n9Cf0ggJlViTIgEJSGpEkqA3kNWeGFJJavVb/nL0YJ685aw4SEhKQkJBQ53M5OTk6j8+fP6+3LwcHB+zcuVOikRnOeoMIIiIiU1hAJqK54JoIIiIiMok8QyciIpKvmmJTUvcpQwwiiIhIXng5QzK8nEFEREQmkWfoRERE8sVMhGSYiSAiIiKTGBU6paamYsuWLThz5gwcHBzQt29fLFy4EF27djXX+BpEtA7EY4+Jd/K//0kzGGshUgfi1/PicWcnf3ndry5WB+KySnzOfL3lNWeidSA+/1y8kxdekGYwVkKsDkS37uLvs9On5PU+qxczEZIxKhOxb98+TJ06FYcOHUJWVhaqqqowaNAglJeXm2t8REREZKGMCp0yMzN1Hq9duxaenp7Iy8vDE088IenAiIiIzIK3eEqmQWsiSkpKAABubm6SDIaIiIish8mhmEajQWJiIh577DH06NGj3nYVFRWoqKjQPi4tLTX1kERERA3HNRGSMfmsp06dipMnT+LAgQN626WmpmL27NmmHoaIiEhaDCIkY9LljISEBHzzzTfYu3cv2rVrp7dtcnIySkpKtFthYaFJAyUiIiLLYlToJAgCXn75ZWzduhU5OTno2LGj6GuUSqVZv8uciIjIKMxESMaos546dSo2bNiAr776Cm3atIFKpQIAODs7w8HBwSwDJCIiIstkVBCxcuVKAMCAAQN09q9ZswZxcXFSjanxGFJIqnt3/c+fOiXNWKyEIYWk9h/Qf5XsiX7yKnhjSCGp0jL9c+bUWl5zZlAhqYIC/c8HBEgzFithSCGpN2bof5+9t0Am7zNmIiRj9OUMIiIiIoBfwEVERHLDYlOS4RdwERERkUmYiSAiInnhmgjJyPOsiYhIvhhESIaXM4iIiMgk8gydiIhIvpiJkIw8z9oYYnUg/P3F+zh/XoqRWA2xOhA/nBRPgAX3kMn96n8QqwNx+674nLWyl9ecidWB+PW8+JwZUvekORGrA3E8X3zOHg6R15yRfgwiiIhIXniLp2S4JoKIiIhMwkwEERHJC9dESIaZCCIiIjKJPEMnIiKSL2YiJCPPsyYiIvliECEZXs4gIiIik8gzdCIiIvliJkIy8jxrKRlSSMrbW6RBmBQjsRqGFJIqLWOS7H6GFJLinOkypJCUhslYHYYUkvqpgHNGf2IQQURE8sJiU5JhSElEREQmYSaCiIjkhWsiJMNMBBEREZlEnqETERHJFzMRkpHnWRMRkXwxiJAML2cQERGRSSw2dFLYjmjqIUiocepANK85axycM+NxzozHObMwvMVTMsxEEBERkUksNhNBRERkFlwTIRlmIoiIiMgk8gydiIhIvpiJkAwzEURERE1gxYoV8Pf3h729PcLCwnDkyBG97Tdt2oTAwEDY29ujZ8+e2LFjh87zgiAgJSUFPj4+cHBwQGRkJH7++WdzngKDCCIikpmaTITUmxE2btyIpKQkzJw5E8ePH0evXr0QFRWFq1ev1tn+4MGDGDNmDMaPH4/vvvsOI0eOxMiRI3Hy5Eltm/feew/Lli1Deno6Dh8+DEdHR0RFReHu3bsNmi59FIIgCGbrvQ6lpaVwdnYGEAWgZWMemoiImlwVgJ0oKSmBk5NTox655vdPye+/S37s0tJSOLu6GnxeYWFheOSRR7B8+XIAgEajgZ+fH15++WXMmDGjVvvRo0ejvLwc33zzjXbfo48+ipCQEKSnp0MQBPj6+mLatGmYPn06AKCkpAReXl5Yu3YtYmJiJDpTXcxEEBERSaS0tFRnq6ioqNWmsrISeXl5iIyM1O6zsbFBZGQkcnNz6+w3NzdXpz0AREVFadufO3cOKpVKp42zszPCwsLq7VMKDCKIiEhWNLAxywYAfn5+cHZ21m6pqam1jn/9+nWo1Wp4eXnp7Pfy8oJKpapzzCqVSm/7mv8a06cU5LmclIiIyAwKCwt1LmcolcomHI35MYggIiJZqa6+t0ndJwA4OTmJrolwd3eHra0tioqKdPYXFRXB29u7ztd4e3vrbV/z36KiIvj4+Oi0CQkJMeZUjMLLGURERI3Izs4OoaGhyM7O1u7TaDTIzs5GeHh4na8JDw/XaQ8AWVlZ2vYdO3aEt7e3TpvS0lIcPny43j6lwEwEERHJijkzEYZKSkpCbGwsevfujT59+iAtLQ3l5eWIj48HAIwbNw4PPPCAdk3Fq6++iv79++P999/H0KFDkZGRgWPHjuHjjz8GACgUCiQmJmLevHno0qULOnbsiHfeeQe+vr4YOXKklKeqg0EEERFRIxs9ejSuXbuGlJQUqFQqhISEIDMzU7sw8sKFC7Cx+fNiQd++fbFhwwa8/fbbeOutt9ClSxds27YNPXr00LZ54403UF5ejkmTJqG4uBj9+vVDZmYm7O3tzXYerBNBRESNqOnrRBQWSn/s0tJS+Pk5N8l5NSVmIoiISFYs4XJGc8GFlURERGQSi81EuLp+rff5m9c1jTQS81PYjpCknw8+0D9nryRwzv5KiPfQ32DVKkmOYwmkmrP339f/PktK5Pvsr4TAAv0NTp2S5DiWQKo5Mye1WvrMgVotbX/WgpkIIiIiMonFZiKIiIjMgWsipMNMBBEREZmEmQgiIpIVZiKkw0wEERERmYSZCCIikhVmIqTDTAQRERGZxGIzEWJ1IFavFY9/XoxrPverG0KsDsSy5eJz1pxqSRhErA7EH19+o1dysjRjsRJidSA2bhJ/n41+TmbvM7E6EMuXi/eRkCDNWIh1IiRksUEEERGROfByhnR4OYOIiIhMwkwEERHJCjMR0mEmgoiIiEzCTAQREckKMxHSYSaCiIiITMJMBBERyQozEdJhJoKIiIhMYrWZCEMKSdnZ64+RKu/Kq+CNIYWkKqv1z5ldC3nNmUGFpA4c0P98v37SjMVKGFJISqxYnNwKxRlUSOrRR/U/f+iQNGORARabko7VBhFERESm4OUM6fByBhEREZmEmQgiIpIVZiKkw0wEERERmYSZCCIikhVmIqTDTAQRERGZhJkIIiKSFd7iKR2jgoiVK1di5cqVOH/+PACge/fuSElJweDBg80xtgYTqwOxa7d4ImZQpLzuVxerA3FZJT5nvt7ymjPROhAnT4r30aOHNGOxEmJ1IH44Kf4+C+4hs/eZWB2Ib78V78NCf1aT9TIqiGjXrh0WLFiALl26QBAEfPbZZxgxYgS+++47dO/e3VxjJCIikgzXREjHqCBi+PDhOo/fffddrFy5EocOHWIQQUREVoFBhHRMXhOhVquxadMmlJeXIzw8XMoxERERkRUwOog4ceIEwsPDcffuXbRu3Rpbt25FUFBQve0rKipQUVGhfVxaWmraSImIiCTATIR0jL7Fs2vXrsjPz8fhw4cxZcoUxMbG4scff6y3fWpqKpydnbWbn59fgwZMRERElsHoIMLOzg4BAQEIDQ1FamoqevXqhQ8++KDe9snJySgpKdFuhYWFDRowERFRQ9RkIqTe5KjBdSI0Go3O5Yq/UiqVUCqVDT0MERERWRijgojk5GQMHjwY7du3x61bt7Bhwwbk5ORg586d5hofERGRpFhsSjpGBRFXr17FuHHjcOXKFTg7OyM4OBg7d+7EU089Za7xmZUhhaS2f6P/is/Tw+RV8MaQQlJvzNA/Z+8tkNecGVRIKi1N//OJiVKMxGoYUkjq41X632eTJsjsfWZIIakZM/Q/v2CBNGMh2TAqiPj000/NNQ4iIqJGwbszpMPvziAiIllhECEdfosnERERmYSZCCIikhVmIqTDTAQRERGZhJkIIiKSFd7iKR1mIoiIiMgkzESIEKsD0SlAPA77tUBe96uL1YFY97n4nI17QV5zJloHIi5OvI+1ayUYiPUQqwOxJE38fZaUKLP3mVgdiO7dxfs4dUqasTQhromQDjMRREREZBIGEUREJCvW9gVcN2/exNixY+Hk5AQXFxeMHz8eZWVletu//PLL6Nq1KxwcHNC+fXu88sorKCkp0WmnUChqbRkZGUaNjZcziIhIVqztcsbYsWNx5coVZGVloaqqCvHx8Zg0aRI2bNhQZ/vLly/j8uXLWLx4MYKCgvDbb79h8uTJuHz5MjZv3qzTds2aNYiOjtY+dnFxMWpsDCKIiIgs1OnTp5GZmYmjR4+id+/eAIAPP/wQQ4YMweLFi+Hr61vrNT169MCXX36pfdy5c2e8++67eOGFF1BdXY0WLf781e/i4gJvb2+Tx8fLGUREJCvWdDkjNzcXLi4u2gACACIjI2FjY4PDhw8b3E9JSQmcnJx0AggAmDp1Ktzd3dGnTx+sXr0agiAYNT5mIoiIiCRSWlqq81ipVEKpVJrcn0qlgqenp86+Fi1awM3NDSqVyqA+rl+/jrlz52LSpEk6++fMmYMnn3wSrVq1wq5du/DSSy+hrKwMr7zyisHjYxBBRESyYs5iU35+fjr7Z86ciVmzZtVqP2PGDCxcuFBvn6dPn27wuEpLSzF06FAEBQXVGsc777yj/f+HHnoI5eXlWLRoEYMIIiKiplBYWAgnJyft4/qyENOmTUOcSP2XTp06wdvbG1evXtXZX11djZs3b4quZbh16xaio6PRpk0bbN26FS1bttTbPiwsDHPnzkVFRYXB2RMGEQ1kSCEp33ZcenI/QwpJGVIoSFYMKST1+edmH4Y1MaSQ1MO9+T7TYUAhqbu2tiIthkkzFjOqrgZET8OEPgHAyclJJ4ioj4eHBzw8PETbhYeHo7i4GHl5eQgNDQUA7NmzBxqNBmFhYfW+rrS0FFFRUVAqldi+fTvs7e1Fj5Wfnw9XV1ejLr8wiCAiIrJQ3bp1Q3R0NCZOnIj09HRUVVUhISEBMTEx2jszLl26hIiICKxbtw59+vRBaWkpBg0ahNu3b+Pzzz9HaWmpdq2Gh4cHbG1t8fXXX6OoqAiPPvoo7O3tkZWVhfnz52P69OlGjY9BBBERyYo5MxHmsH79eiQkJCAiIgI2NjYYNWoUli1bpn2+qqoKZ8+exe3btwEAx48f1965ERAQoNPXuXPn4O/vj5YtW2LFihV47bXXIAgCAgICsGTJEkycONGosTGIICIiWbG2IMLNza3ewlIA4O/vr3Nr5oABA0Rv1YyOjtYpMmUqXhAkIiIikzATQUREsmLOWzzlhpkIIiIiMgkzEUREJCvV1YCNxH9Cm3NNhCWz2CBCYTuiqYdgdThnxuOcGY9zZrzmNWeWXweCGo/FBhFERETmwEyEdLgmgoiIiEzCTAQREckKMxHSYRBBRESywiBCOrycQURERCZhJoKIiGSFxaakw0wEERERmYSZCCIikpXqakChkL5POWImgoiIiEzCTAQREckKMxHSYSaCiIiITMJMBBERyQozEdJhEEFERLLCIEI6vJxBREREJmEmgoiIZEWtlj4TwWJTREREREZgJoKIiGTFHOsX5LomwmKDiPLyr/U+38pe00gjMT+F7QhJ+hE+j9HfYMwYSY5jCaSas++/1/8+C+7B99lfJSfrn7P58zhnf1VRoX/O7Fpwzsg6WWwQQUREZA7MREiHayKIiIjIJMxEEBGRrDATIR1mIoiIiMgkzEQQEZGsmKOmg1zrRDCIICIiWamuBgRB2j7lGkTwcgYRERGZxGIzEaJ1IK5fF+/E3V2awVgLsToQqanifSQnSzMWKyFWB+KySjzO9vVuPvf4G0KsDkTffuJzdvCAvOZMtA7ECy+Id/L559IMhpiJkBAzEURERGQSi81EEBERmQMzEdJhJoKIiIhMwkwEERHJCjMR0mEmgoiIiEzCTAQREcmKWi19JkIjrxuOtBhEEBGRrFRXAzYS5+HlGkTwcgYRERGZxHozEQYUkvqpQH+M9GCAzEJHQwpJpaXpfz4xUYqRWA1DCkntyNT/PhsSLa/3mSGFpH49r3/OOvnLa84MKSTVKUD/nP1aILM5awBmIqTDTAQRERGZxHozEURERCZgJkI6zEQQERGRSZiJICIiWVGrpc8cSH3LqLVgJoKIiIhMwkwEERHJSnU1oFBI2yczEURERDJQXW2ezVxu3ryJsWPHwsnJCS4uLhg/fjzKysr0vmbAgAFQKBQ62+TJk3XaXLhwAUOHDkWrVq3g6emJ119/HdVGnkiDMhELFixAcnIyXn31VaSJ1RdoAqJ1II4dE++kd29pBmMtxOpAjBwp3se2bRIMxHqI1YH44aR4rB7cQ15Lu8XqQDwfIz5nX2TIa85E60C8/bZ4J/PmSTMYalRjx47FlStXkJWVhaqqKsTHx2PSpEnYsGGD3tdNnDgRc+bM0T5u1aqV9v/VajWGDh0Kb29vHDx4EFeuXMG4cePQsmVLzJ8/3+CxmRxEHD16FB999BGCg4NN7YKIiKjRWdPljNOnTyMzMxNHjx5F7z/+qP3www8xZMgQLF68GL6+vvW+tlWrVvD29q7zuV27duHHH3/E7t274eXlhZCQEMydOxdvvvkmZs2aBTs7O4PGZ9LljLKyMowdOxaffPIJXF1dTemCiIiIROTm5sLFxUUbQABAZGQkbGxscPjwYb2vXb9+Pdzd3dGjRw8kJyfj9u3bOv327NkTXl5e2n1RUVEoLS3FqVOnDB6fSZmIqVOnYujQoYiMjMQ8pseIiMiKmDMTUVpaqrNfqVRCqVSa3K9KpYKnp6fOvhYtWsDNzQ0qlare1/3f//0fOnToAF9fX/zwww948803cfbsWWzZskXb7/0BBADtY339/pXRQURGRgaOHz+Oo0ePGtS+oqICFRUV2sd/nWAiIqLmws/PT+fxzJkzMWvWrFrtZsyYgYULF+rt6/Tp0yaPY9KkSdr/79mzJ3x8fBAREYFffvkFnTt3NrnfvzIqiCgsLMSrr76KrKws2NvbG/Sa1NRUzJ4926TBERERSU8DQZB6Ye69/goLC+Hk5KTdW18WYtq0aYiLi9PbY6dOneDt7Y2rV6/q7K+ursbNmzfrXe9Ql7CwMABAQUEBOnfuDG9vbxw5ckSnTVFREQAY1a9RQUReXh6uXr2Khx9+WLtPrVZj//79WL58OSoqKmBra6vzmuTkZCQlJWkfl5aW1orUiIiImgMnJyedIKI+Hh4e8PDwEG0XHh6O4uJi5OXlITQ0FACwZ88eaDQabWBgiPz8fACAj4+Ptt93330XV69e1V4uycrKgpOTE4KCggzu16ggIiIiAidOnNDZFx8fj8DAQLz55pu1Agig4deDiIiIpKX+Y5O6T+l169YN0dHRmDhxItLT01FVVYWEhATExMRo78y4dOkSIiIisG7dOvTp0we//PILNmzYgCFDhqBt27b44Ycf8Nprr+GJJ57Q3lE5aNAgBAUF4e9//zvee+89qFQqvP3225g6dapRv7ONCiLatGmDHj166OxzdHRE27Zta+0nIiKyTNYTRAD37rJISEhAREQEbGxsMGrUKCxbtkz7fFVVFc6ePau9+8LOzg67d+9GWloaysvL4efnh1GjRuHt+2qJ2Nra4ptvvsGUKVMQHh4OR0dHxMbG6tSVMIS8y14bUkjqzBn9zwcGSjMWa2FIIakZM/Q/v2CBJEOxFoYUkvr1vP67rcWKMzU3hhSSEitIJbdiVAYVkhIrCihWbI6ahJubm97CUv7+/hDuK1Th5+eHffv2ifbboUMH7Nixo0Fja3AQkZOT09AuiIiIGpF1ZSIsGb87g4iIiEwi78sZREQkQxrU3JIpbZ/yw0wEERERmYSZCCIikhmuiZAKMxFERERkEmYiiIhIZjSQPnMgzzURDCLEiNSB2JMjnsx5coDM3lxidSBWrRLvY8IEacZiJcTqQPwrXfx99tJkeb3PxOpAzJknPmcpb8trzsTqQHy8SnzOJk1oDnPGyxlS4eUMIiIiMgkzEUREJDPMREiFmQgiIiIyCTMRREQkMyw2JRVmIoiIiMgkzEQQEZHMcE2EVJiJICIiIpMwE0FERDLDTIRUGEQ0kCGFpDZkMOGjw5BCUtOnm38cVsSQQlIHD/F9dj9DCkmVlnHO7mdIIannY5rDnDGIkEpzeDcQERFRE2AmgoiIZIbfnSEVZiKIiIjIJMxEEBGRzLDYlFSYiSAiIiKTMBNBREQyw7szpMJMBBEREZnEYjMRCtsRTT0Eq8M5Mx7nzHicM+NxziwNMxFSsdgggoiIyDwYREiFlzOIiIjIJMxEEBGRzDATIRVmIoiIiMgkzEQQEZHMCJC+OJQgcX/WgZkIIiIiMgkzEUREJDNcEyEVZiKIiIjIJMxEEBGRzDATIRUGEUREJDMMIqTCyxlERERkEmYiiIhIZpiJkAozEURERGQSZiKIiEhmNJC+2JTU/VkHZiKIiIjIJMxEEBGRzHBNhFSYiSAiIiKTWGwmQlj4uP4G06c3zkAagcJ2hCT9vPvu13qff2tG87lmJ9WcjRmjf842fM45q+1jvc8Kai+JjtP0pJqzr/CN3uefVjefv2Kle5+ZkwbSZw6az88KYzATQURERCax2EwEERGReXBNhFQYRBARkczwFk+p8HIGERERmYRBBBERyYzaTJt53Lx5E2PHjoWTkxNcXFwwfvx4lJWV1dv+/PnzUCgUdW6bNm3Stqvr+YyMDKPGxssZREREFmzs2LG4cuUKsrKyUFVVhfj4eEyaNAkbNmyos72fnx+uXLmis+/jjz/GokWLMHjwYJ39a9asQXR0tPaxi4uLUWNjEEFERDJjPQsrT58+jczMTBw9ehS9e/cGAHz44YcYMmQIFi9eDF9f31qvsbW1hbe3t86+rVu34vnnn0fr1q119ru4uNRqawzLDSJE6kDsyRG/EvPkAHktdBGrA/G3Z8XnbMtmec2ZWB2ISZPF5+zjdHnNmVgdiPb+4nN24by85kysDoTG1la0D5tmVEuiOSstLdV5rFQqoVQqTe4vNzcXLi4u2gACACIjI2FjY4PDhw/jmWeeEe0jLy8P+fn5WLFiRa3npk6digkTJqBTp06YPHky4uPjoVAoDB6f5QYRREREZmG+TISfn5/O3pkzZ2LWrFkm96pSqeDp6amzr0WLFnBzc4NKpTKoj08//RTdunVD3759dfbPmTMHTz75JFq1aoVdu3bhpZdeQllZGV555RWDx8cggoiISCKFhYVwcnLSPq4vCzFjxgwsXLhQb1+nT59u8Hju3LmDDRs24J133qn13P37HnroIZSXl2PRokUMIoiIiOpnvrLXTk5OOkFEfaZNm4a4uDi9bTp16gRvb29cvXpVZ391dTVu3rxp0FqGzZs34/bt2xg3bpxo27CwMMydOxcVFRUGX4JhEEFERDLT9MWmPDw84OHhIdouPDwcxcXFyMvLQ2hoKABgz5490Gg0CAsLE339p59+iqefftqgY+Xn58PV1dWoNRwMIoiIiCxUt27dEB0djYkTJyI9PR1VVVVISEhATEyM9s6MS5cuISIiAuvWrUOfPn20ry0oKMD+/fuxY8eOWv1+/fXXKCoqwqOPPgp7e3tkZWVh/vz5mG7kl1syiCAiIpmxnls8AWD9+vVISEhAREQEbGxsMGrUKCxbtkz7fFVVFc6ePYvbt2/rvG716tVo164dBg0aVKvPli1bYsWKFXjttdcgCAICAgKwZMkSTJw40aixMYggIiKyYG5ubvUWlgIAf39/CIJQa//8+fMxf/78Ol8THR2tU2TKVAwiiIhIZqwrE2HJrDaIMKiQ1Pnz+p/395diKFbDoEJSd+/qf97eXprBWAlDCkmVlukvruTUWl6FlQwpJHXhov45a99OXnNmUCGpY8f0P39fMSKixmK1QQQREZFpmImQCr/Fk4iIiEzCTAQREckMMxFSYRBBREQy0/TFppoLXs4gIiIikzATQUREMmO+786QG2YiiIiIyCRGZSJmzZqF2bNn6+zr2rUrzpw5I+mgJCNSByJpungMtWSxzKJLkToQCxeJz9mbr8trzsTqQDw9UnzOtm+T15yJ1YHo1l18zk6fktecidaBaNtWvI8bN6QZi9XjwkqpGH05o3v37ti9e/efHbTgFREiIiI5MjoCaNGihUHfYU5ERGSZmImQitFrIn7++Wf4+vqiU6dOGDt2LC5cuGCOcREREZGFMyoTERYWhrVr16Jr1664cuUKZs+ejccffxwnT55EmzZt6nxNRUUFKioqtI9LS0sbNmIiIqIGYSZCKkYFEYMHD9b+f3BwMMLCwtChQwd88cUXGD9+fJ2vSU1NrbUYk4iIqOkwiJBKg27xdHFxwYMPPoiCgoJ62yQnJ6OkpES7FRYWNuSQREREZCEaFESUlZXhl19+gY+PT71tlEolnJycdDYiIqKmozHTJj9GBRHTp0/Hvn37cP78eRw8eBDPPPMMbG1tMWbMGHONj4iIiCyUUWsiLl68iDFjxuDGjRvw8PBAv379cOjQIXh4eJhrfGZlSCGpX8/rj7M6+csr+jSkkNTDvfXP2fFj8pozQwpJiRU+k1vRM0MKSdnZ65+zyrvymjNDCkmpbG31Pu+tlst1fZa9lopRQURGRoa5xkFERERWhuUmiYhIZnh3hlT4BVxERERkEmYiiIhIZpiJkAqDCCIikhlz3JIpz4WVvJxBREREJmEmgoiIZIaXM6TCIEKEaB2IuDjxTtaulWIoVkOsDsTqteIJsBfj5JUaFKsD8dbb4nM2f5685kysDoTC9n+ifQjqx6QajlUQqwOxRKSOBAAkyaaWBBmCQQQREckMMxFS4ZoIIiIiMgkzEUREJDPMREiFmQgiIiIyCTMRREQkM/wCLqkwiCAiIplhsSmp8HIGERERmYSZCCIikhkurJQKg4iGMqSQ1MWLZh+GNTGkkNTtu0yS3c+QQlKcM12GFJL68Qzn7H4GFZLKyTH7OMh6MIggIiKZYSZCKgzDiYiIyCTMRBARkcwwEyEVZiKIiIjIJMxEEBGRzDATIRUGEUREJDOsWCkVXs4gIiIik1hsJkJhO6Kph2B1OGfG45wZj3NmPM6ZpWHZa6kwE0FEREQmsdhMBBERkXmoIf3f0PJcWMlMBBERkQV799130bdvX7Rq1QouLi4GvUYQBKSkpMDHxwcODg6IjIzEzz//rNPm5s2bGDt2LJycnODi4oLx48ejrKzMqLExiCAiIplRm2kzj8rKSjz33HOYMmWKwa957733sGzZMqSnp+Pw4cNwdHREVFQU7t69q20zduxYnDp1CllZWfjmm2+wf/9+TJo0yaixKQRBEIx6RQOVlpbC2dkZQBSAlo15aCIianJVAHaipKQETk5OjXrkP3//DIf0v3+qAHxt1vNau3YtEhMTUVxcrLedIAjw9fXFtGnTMH36dABASUkJvLy8sHbtWsTExOD06dMICgrC0aNH0bt3bwBAZmYmhgwZgosXL8LX19egMTETQUREMmO+TERpaanOVlFR0Xin9Ydz585BpVIhMjJSu8/Z2RlhYWHIzc0FAOTm5sLFxUUbQABAZGQkbGxscPjwYYOPxSCCiIhkRmOmDfDz84Ozs7N2S01NbbzT+oNKpQIAeHl56ez38vLSPqdSqeDp6anzfIsWLeDm5qZtY4hGvzvjz6sn1Y19aCIianL3fvY38pX0Osdgjj4LCwt1Lmcolco6W8+YMQMLFy7U2+Pp06cRGBgo3RDNoNGDiFu3bv3xf9mNfWgiIrIQt27d+mN9QuOxs7ODt7c3VKrdZunf29sb7u7usLe3F207bdo0xMXF6W3TqVMnk8cBAEVFRfDx8dHuLyoqQkhIiLbN1atXdV5XXV2Nmzdval9viEYPInx9fVFYWIg2bdpAoVA09uHrVVpaCj8/v1pRJNWPc2Y8zpnxOGfGs+Q5EwQBt27dMnjhnpTs7e1x7tw5VFZWmqV/Ozs7gwIIAPDw8ICHh4dZxtGxY0d4e3sjOztbGzSUlpbi8OHD2js8wsPDUVxcjLy8PISGhgIA9uzZA41Gg7CwMIOP1ehBhI2NDdq1a9fYhzWYk5OTxX3oLB3nzHicM+NxzoxnqXPW2BmI+9nb2xv8i95SXLhwATdv3sSFCxegVquRn58PAAgICEDr1q0BAIGBgUhNTcUzzzwDhUKBxMREzJs3D126dEHHjh3xzjvvwNfXFyNHjgQAdOvWDdHR0Zg4cSLS09NRVVWFhIQExMTEGBXgsWIlERGRBUtJScFnn32mffzQQw8BAPbu3YsBAwYAAM6ePYuSkhJtmzfeeAPl5eWYNGkSiouL0a9fP2RmZuoEUOvXr0dCQgIiIiJgY2ODUaNGYdmyZUaNrdHrRFiqmvuHm+LeZWvFOTMe58x4nDPjcc6osfAWzz8olUrMnDmz3pW0VBvnzHicM+NxzozHOaPGwkwEERERmYSZCCIiIjIJgwgiIiIyCYMIIiIiMgmDCCIiIjIJg4g/rFixAv7+/rC3t0dYWBiOHDnS1EOyWPv378fw4cPh6+sLhUKBbdu2NfWQLF5qaioeeeQRtGnTBp6enhg5ciTOnj3b1MOyaCtXrkRwcLC2YFJ4eDi+/fbbph6W1ViwYIG26BCRuTCIALBx40YkJSVh5syZOH78OHr16oWoqKhadcXpnvLycvTq1QsrVqxo6qFYjX379mHq1Kk4dOgQsrKyUFVVhUGDBqG8vLyph2ax2rVrhwULFiAvLw/Hjh3Dk08+iREjRuDUqVNNPTSLd/ToUXz00UcIDg5u6qFQM8dbPAGEhYXhkUcewfLlywEAGo0Gfn5+ePnllzFjxowmHp1lUygU2Lp1q7aUKhnm2rVr8PT0xL59+/DEE0809XCshpubGxYtWoTx48c39VAsVllZGR5++GH861//wrx58xASEoK0tLSmHhY1U7LPRFRWViIvLw+RkZHafTY2NoiMjERubm4Tjoyas5rytG5ubk08EuugVquRkZGB8vJyhIeHN/VwLNrUqVMxdOhQnZ9pROYi++/OuH79OtRqNby8vHT2e3l54cyZM000KmrONBoNEhMT8dhjj6FHjx5NPRyLduLECYSHh+Pu3bto3bo1tm7diqCgoKYelsXKyMjA8ePHcfTo0aYeCsmE7IMIosY2depUnDx5EgcOHGjqoVi8rl27Ij8/HyUlJdi8eTNiY2Oxb98+BhJ1KCwsxKuvvoqsrCyr+5ZKsl6yDyLc3d1ha2uLoqIinf1FRUXw9vZuolFRc5WQkIBvvvkG+/fvR7t27Zp6OBbPzs4OAQEBAIDQ0FAcPXoUH3zwAT766KMmHpnlycvLw9WrV/Hwww9r96nVauzfvx/Lly9HRUUFbG1tm3CE1BzJfk2EnZ0dQkNDkZ2drd2n0WiQnZ3Na68kGUEQkJCQgK1bt2LPnj3o2LFjUw/JKmk0GlRUVDT1MCxSREQETpw4gfz8fO3Wu3dvjB07Fvn5+QwgyCxkn4kAgKSkJMTGxqJ3797o06cP0tLSUF5ejvj4+KYemkUqKytDQUGB9vG5c+eQn58PNzc3tG/fvglHZrmmTp2KDRs24KuvvkKbNm2gUqkAAM7OznBwcGji0Vmm5ORkDB48GO3bt8etW7ewYcMG5OTkYOfOnU09NIvUpk2bWmtsHB0d0bZtW669IbNhEAFg9OjRuHbtGlJSUqBSqRASEoLMzMxaiy3pnmPHjmHgwIHax0lJSQCA2NhYrF27tolGZdlWrlwJABgwYIDO/jVr1iAuLq7xB2QFrl69inHjxuHKlStwdnZGcHAwdu7ciaeeeqqph0ZEf2CdCCIiIjKJ7NdEEBERkWkYRBAREZFJGEQQERGRSRhEEBERkUkYRBAREZFJGEQQERGRSRhEEBERkUkYRBAREZFJGEQQERGRSRhEEBERkUkYRBAREZFJGEQQERGRSf4/faGl+gtMXzAAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def plot_as_patches(j):\n", " # Reorder axes so the diagonals will each form a contiguous patch.\n", " j = tf.transpose(j, [1, 0, 3, 2])\n", " # Pad in between each patch.\n", " lim = tf.reduce_max(abs(j))\n", " j = tf.pad(j, [[0, 0], [1, 1], [0, 0], [1, 1]],\n", " constant_values=-lim)\n", " # Reshape to form a single image.\n", " s = j.shape\n", " j = tf.reshape(j, [s[0]*s[1], s[2]*s[3]])\n", " imshow_zero_center(j, extent=[-0.5, s[2]-0.5, s[0]-0.5, -0.5])\n", "\n", "plot_as_patches(j)\n", "_ = plt.title('All (batch, batch) slices are diagonal')" ] }, { "cell_type": "markdown", "metadata": { "id": "OXpTBKyeK84z" }, "source": [ "원하는 결과를 얻으려면 중복 `batch` 차원를 합산하거나 `tf.einsum`을 사용하여 대각선을 선택할 수 있습니다." ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:30.213121Z", "iopub.status.busy": "2022-12-14T21:28:30.212515Z", "iopub.status.idle": "2022-12-14T21:28:30.218592Z", "shell.execute_reply": "2022-12-14T21:28:30.217909Z" }, "id": "v65OAjEgLQwl" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(7, 6, 5)\n", "(7, 6, 5)\n" ] } ], "source": [ "j_sum = tf.reduce_sum(j, axis=2)\n", "print(j_sum.shape)\n", "j_select = tf.einsum('bxby->bxy', j)\n", "print(j_select.shape)" ] }, { "cell_type": "markdown", "metadata": { "id": "zT_VfR6lcwxD" }, "source": [ "처음부터 추가 차원 없이 계산을 수행하는 것이 훨씬 더 효율적입니다. `tf.GradientTape.batch_jacobian` 메서드는 정확히 그렇게 작업을 수행합니다." ] }, { "cell_type": "code", "execution_count": 35, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:30.221909Z", "iopub.status.busy": "2022-12-14T21:28:30.221316Z", "iopub.status.idle": "2022-12-14T21:28:30.367051Z", "shell.execute_reply": "2022-12-14T21:28:30.366358Z" }, "id": "YJLIl9WpHqYq" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "WARNING:tensorflow:5 out of the last 5 calls to .f at 0x7fc6b06325e0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.\n" ] }, { "data": { "text/plain": [ "TensorShape([7, 6, 5])" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "jb = tape.batch_jacobian(y, x)\n", "jb.shape" ] }, { "cell_type": "code", "execution_count": 36, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:30.370348Z", "iopub.status.busy": "2022-12-14T21:28:30.369707Z", "iopub.status.idle": "2022-12-14T21:28:30.375550Z", "shell.execute_reply": "2022-12-14T21:28:30.374866Z" }, "id": "-5t_q5SfHw7T" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0.0\n" ] } ], "source": [ "error = tf.reduce_max(abs(jb - j_sum))\n", "assert error < 1e-3\n", "print(error.numpy())" ] }, { "cell_type": "markdown", "metadata": { "id": "IUeY2ZCiL31I" }, "source": [ "주의: `tf.GradientTape.batch_jacobian`은 소스 및 대상의 첫 번째 차원이 일치하는지만 확인합니다. 그래디언트가 실제로 독립적인지는 확인하지 않습니다. 적합한 경우에 `batch_jacobian`만을 사용하도록 하는 것은 여러분에게 달려 있습니다. 예를 들어, `tf.keras.layers.BatchNormalization`를 추가하면 `batch` 차원에서 정규화되므로 독립성이 없어집니다." ] }, { "cell_type": "code", "execution_count": 37, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:30.378772Z", "iopub.status.busy": "2022-12-14T21:28:30.378200Z", "iopub.status.idle": "2022-12-14T21:28:30.716784Z", "shell.execute_reply": "2022-12-14T21:28:30.716030Z" }, "id": "tnDugVc-L4fj" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "WARNING:tensorflow:6 out of the last 6 calls to .f at 0x7fc71c0c2af0> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "j.shape: (7, 6, 7, 5)\n" ] } ], "source": [ "x = tf.random.normal([7, 5])\n", "\n", "layer1 = tf.keras.layers.Dense(8, activation=tf.nn.elu)\n", "bn = tf.keras.layers.BatchNormalization()\n", "layer2 = tf.keras.layers.Dense(6, activation=tf.nn.elu)\n", "\n", "with tf.GradientTape(persistent=True, watch_accessed_variables=False) as tape:\n", " tape.watch(x)\n", " y = layer1(x)\n", " y = bn(y, training=True)\n", " y = layer2(y)\n", "\n", "j = tape.jacobian(y, x)\n", "print(f'j.shape: {j.shape}')" ] }, { "cell_type": "code", "execution_count": 38, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:30.719908Z", "iopub.status.busy": "2022-12-14T21:28:30.719676Z", "iopub.status.idle": "2022-12-14T21:28:30.945861Z", "shell.execute_reply": "2022-12-14T21:28:30.944986Z" }, "id": "SNyZ1WhJMVLm" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAhEAAAIkCAYAAACgKXkMAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABogElEQVR4nO3deVxU9f4/8NcAsokDIpsIikqpqEjiFXG/SYKaactNy65Kpi2aem1R+5WaVmqalxbTNrfUr15LK71d3NEyRcXIXBOTXEEBYQRinfP7o5yagM9nZjgDo+f1fDzmoZzPZ97nc86cGT68zznv0SmKooCIiIjISk71PQAiIiK6NXESQURERDbhJIKIiIhswkkEERER2YSTCCIiIrIJJxFERERkE04iiIiIyCacRBAREZFNOIkgIiIim3AScQtLSUmBTqfDZ599Vt9DUdWKFSug0+mQmZlpWta3b1/07du33sZE1tHia3i7bx9RdTiJcDA6nc6iR0pKSn0PlW5DxcXFmDVrFo8vIrKIS30PgMx9+umnZj+vWrUK27dvr7K8Xbt2OHnyZF0OrV5t27atvoegCcXFxXj11VcBQPW/qvkaEt1+OIlwMI899pjZzwcOHMD27durLAegqUmEq6trfQ9BdcXFxfD09KzvYdSZ2/E1JNI6ns64DRiNRrz++usICQmBu7s7+vXrh4yMjCr9UlNTkZCQAG9vb3h6eqJPnz7Yt2+fWZ8bN25g8uTJCAsLg5ubGwICAnDPPffgyJEjVseqybvvvov27dvD09MTjRs3RpcuXbB27Vrhc6o731xSUoJZs2bhzjvvhLu7O5o2bYoHHngAZ8+eNds3SUlJaN++Pdzd3REYGIgnn3wS169fN4t1+PBhxMfHw8/PDx4eHmjZsiUef/xx6bZ8+eWXGDRoEIKDg+Hm5obWrVtjzpw5qKysrDL+Dh06IC0tDb1794anpydeeuklAEBpaSlmzpyJ8PBwuLm5ITQ0FC+++CJKS0ul678Z98SJE/j73/8OT09PNGvWDG+++WaVvlevXsWYMWMQGBgId3d3dOrUCStXrjS1Z2Zmwt/fHwDw6quvmk6dzZo1SziG48eP4+6774aHhwdCQkLw2muvwWg0VjvWP7+GZWVlmDFjBqKjo+Ht7Y2GDRuiV69e2L17d5Xn5ubm4p///Cf0ej18fHwwatQo/PDDD9DpdFixYoVZ3127dqFXr15o2LAhfHx8MGTIkCoT7lmzZkGn0yEjIwOjR4+Gj48PvL29kZiYiOLiYrO+y5cvx913342AgAC4ubkhIiICS5YsEe4TIq1gJuI2MG/ePDg5OeH5559HQUEB3nzzTYwYMQKpqammPrt27cKAAQMQHR2NmTNnwsnJyfTh+M0336Br164AgKeeegqfffYZJkyYgIiICOTm5uLbb7/FyZMn0blzZ6tiVeejjz7CxIkT8dBDD2HSpEkoKSnB0aNHkZqaikcffdTiba6srMS9996LnTt3Yvjw4Zg0aRJu3LiB7du349ixY2jdujUA4Mknn8SKFSuQmJiIiRMn4ty5c3jvvffw/fffY9++fWjQoAGuXr2K/v37w9/fH9OmTYOPjw8yMzOxceNG6ThWrFgBLy8vTJkyBV5eXti1axdmzJgBg8GABQsWmPXNzc3FgAEDMHz4cDz22GMIDAyE0WjEfffdh2+//Rbjxo1Du3bt8OOPP+Lf//43fvrpJ3zxxRfSMVy/fh0JCQl44IEH8PDDD+Ozzz7D1KlT0bFjRwwYMAAA8Ouvv6Jv377IyMjAhAkT0LJlS2zYsAGjR49Gfn4+Jk2aBH9/fyxZsgRPP/007r//fjzwwAMAgMjIyBrXnZWVhb///e+oqKjAtGnT0LBhQ3z44Yfw8PCQjttgMODjjz/GI488grFjx+LGjRv45JNPEB8fj4MHDyIqKgrAbxPBwYMH4+DBg3j66afRtm1bfPnllxg1alSVmDt27MCAAQPQqlUrzJo1C7/++iveffdd9OjRA0eOHEFYWJhZ/4cffhgtW7bE3LlzceTIEXz88ccICAjA/PnzTX2WLFmC9u3b47777oOLiws2b96MZ555BkajEePHj5duJ9FtTSGHNn78eKWml2n37t0KAKVdu3ZKaWmpafnbb7+tAFB+/PFHRVEUxWg0KnfccYcSHx+vGI1GU7/i4mKlZcuWyj333GNa5u3trYwfP77G8VgTqzpDhgxR2rdvL+yzfPlyBYBy7tw507I+ffooffr0Mf28bNkyBYCyaNGiaseoKIryzTffKACUNWvWmLUnJyebLd+0aZMCQDl06JBwXNUpLi6usuzJJ59UPD09lZKSErPxA1CWLl1q1vfTTz9VnJyclG+++cZs+dKlSxUAyr59+4Trvxl31apVpmWlpaVKUFCQ8uCDD5qWJSUlKQCU1atXm5aVlZUpsbGxipeXl2IwGBRFUZRr164pAJSZM2fKN15RlMmTJysAlNTUVNOyq1evKt7e3tLXsKKiwuy4VRRFuX79uhIYGKg8/vjjpmWff/65AkBJSkoyLausrFTuvvtuBYCyfPly0/KoqCglICBAyc3NNS374YcfFCcnJ2XkyJGmZTNnzlQAmK1HURTl/vvvV5o0aWK2rLrXOD4+XmnVqpXZsr9uH5EW8HTGbSAxMdHsfHOvXr0AAD///DMAID09HWfOnMGjjz6K3Nxc5OTkICcnB0VFRejXrx/27t1rSj/7+PggNTUVly9frnZd1sSqjo+PDy5evIhDhw7Vaps///xz+Pn54dlnn63SptPpAAAbNmyAt7c37rnnHtM4c3JyEB0dDS8vL1Pa3MfHBwCwZcsWlJeXWzWOP//FfePGDeTk5KBXr14oLi7GqVOnzPq6ubkhMTHRbNmGDRvQrl07tG3b1myMd999NwBUm9r/Ky8vL7NrZlxdXdG1a1fT6w8AX3/9NYKCgvDII4+YljVo0AATJ05EYWEh9uzZY9V2/zlut27dzLJP/v7+GDFihPS5zs7OpuPWaDQiLy8PFRUV6NKli9nps+TkZDRo0ABjx441LXNycqqSBbhy5QrS09MxevRo+Pr6mpZHRkbinnvuwddff11lDE899ZTZz7169UJubi4MBoNp2Z9f44KCAuTk5KBPnz74+eefUVBQIN1OotsZT2fcBpo3b272c+PGjQHAdN7/zJkzAFBt+vemgoICNG7cGG+++SZGjRqF0NBQREdHY+DAgRg5ciRatWpldazqTJ06FTt27EDXrl0RHh6O/v3749FHH0WPHj0s3NrfnD17Fm3atIGLS82H8JkzZ1BQUICAgIBq269evQoA6NOnDx588EG8+uqr+Pe//42+ffti6NChePTRR+Hm5iYcx/Hjx/Hyyy9j165dZr94AFT5BdOsWbMqFxeeOXMGJ0+eNF2LUNMYRUJCQkwTp5saN26Mo0ePmn7+5ZdfcMcdd8DJyfzvhnbt2pnabfHLL78gJiamyvI2bdpY9PyVK1firbfewqlTp8wmcC1btjRbR9OmTatchBoeHl5lLDWtu127dti6dSuKiorQsGFD03LRe0ev1wMA9u3bh5kzZ2L//v1VrpcoKCiAt7e3RdtKdDviJOI24OzsXO1yRVEAwJQZWLBggek88195eXkB+O0cca9evbBp0yZs27YNCxYswPz587Fx40YMGDDAqljVadeuHU6fPo0tW7YgOTkZn3/+Od5//33MmDHDdGuhWoxGIwICArBmzZpq22/+4r5ZsOvAgQPYvHkztm7discffxxvvfUWDhw4UOP25Ofno0+fPtDr9Zg9ezZat24Nd3d3HDlyBFOnTq2SkanuOgGj0YiOHTti0aJF1a4jNDRUup2y199RrV69GqNHj8bQoUPxwgsvICAgAM7Ozpg7d67ZxbH2JNt3Z8+eRb9+/dC2bVssWrQIoaGhcHV1xddff41///vfwqwbkRZwEqEBNy8y1Ov1iIuLk/Zv2rQpnnnmGTzzzDO4evUqOnfujNdffx0DBgywOlZ1GjZsiGHDhmHYsGEoKyvDAw88gNdffx3Tp0+Hu7u7xduUmpqK8vJyNGjQoMY+O3bsQI8ePSy60K9bt27o1q0bXn/9daxduxYjRozAunXr8MQTT1TbPyUlBbm5udi4cSN69+5tWn7u3DmLtuHmGH/44Qf069evSjZBTS1atMDRo0dhNBrNshE3T7m0aNECAKweQ4sWLUzZqT87ffq09LmfffYZWrVqhY0bN5qtd+bMmVXWsXv37iq3xP71DqSb21Dduk+dOgU/Pz+zLIQlNm/ejNLSUnz11VdmWQtLTjMRaQGvidCA6OhotG7dGgsXLkRhYWGV9mvXrgH47Y6Hv6bgAwICEBwcbLrd0NJYNcnNzTX72dXVFREREVAUxarrER588EHk5OTgvffeq9J286/Ihx9+GJWVlZgzZ06VPhUVFcjPzwfwW+r6r3+138yyiG6zvPlX7J+fW1ZWhvfff9/i7Xj44Ydx6dIlfPTRR1Xafv31VxQVFVkcS2TgwIHIysrC+vXrTcsqKirw7rvvwsvLC3369AEA0y/pm/vGkrgHDhzAwYMHTcuuXbtWY/bnz6rbf6mpqdi/f79Zv/j4eJSXl5vtI6PRiMWLF5v1a9q0KaKiorBy5Uqz8R87dgzbtm3DwIEDLdom2RgLCgqwfPlyq2MR3Y6YidAAJycnfPzxxxgwYADat2+PxMRENGvWDJcuXcLu3buh1+uxefNm3LhxAyEhIXjooYfQqVMneHl5YceOHTh06BDeeustq2LVpH///ggKCkKPHj0QGBiIkydP4r333sOgQYPQqFEji7dp5MiRWLVqFaZMmYKDBw+iV69eKCoqwo4dO/DMM89gyJAh6NOnD5588knMnTsX6enp6N+/Pxo0aIAzZ85gw4YNePvtt/HQQw9h5cqVeP/993H//fejdevWuHHjBj766CPo9XrhL57u3bujcePGGDVqFCZOnAidTodPP/3UqtMI//znP/Gf//wHTz31FHbv3o0ePXqgsrISp06dwn/+8x9s3boVXbp0sTheTcaNG4cPPvgAo0ePRlpaGsLCwvDZZ59h3759SEpKMu17Dw8PREREYP369bjzzjvh6+uLDh06oEOHDtXGffHFF/Hpp58iISEBkyZNMt3ieTPzIXLvvfdi48aNuP/++zFo0CCcO3cOS5cuRUREhNkEdejQoejatSuee+45ZGRkoG3btvjqq6+Ql5cHwDx7smDBAgwYMACxsbEYM2aM6RZPb29vab2L6vTv3x+urq4YPHgwnnzySRQWFuKjjz5CQEAArly5YnU8ottOfd0WQpax5BbPDRs2mC0/d+5clVvfFEVRvv/+e+WBBx5QmjRpori5uSktWrRQHn74YWXnzp2Kovx2a+ALL7ygdOrUSWnUqJHSsGFDpVOnTsr7779fZd2yWDX54IMPlN69e5ue17p1a+WFF15QCgoKTH0sucVTUX679e7//b//p7Rs2VJp0KCBEhQUpDz00EPK2bNnzfp9+OGHSnR0tOLh4aE0atRI6dixo/Liiy8qly9fVhRFUY4cOaI88sgjSvPmzRU3NzclICBAuffee5XDhw8Lt0VRFGXfvn1Kt27dFA8PDyU4OFh58cUXla1btyoAlN27d5uNv6ZbW8vKypT58+cr7du3V9zc3JTGjRsr0dHRyquvvmq2X6pTU9xRo0YpLVq0MFuWnZ2tJCYmKn5+foqrq6vSsWPHKseIoijKd999p0RHRyuurq4W3e559OhRpU+fPoq7u7vSrFkzZc6cOconn3wifQ2NRqPyxhtvKC1atFDc3NyUu+66S9myZUu1Y7927Zry6KOPKo0aNVK8vb2V0aNHK/v27VMAKOvWrTPru2PHDqVHjx6Kh4eHotfrlcGDBysnTpww63PzFs9r166ZLa/u2Pvqq6+UyMhIxd3dXQkLC1Pmz59vusVYdowS3e50iuLgV18REVXjiy++wP33349vv/3W6rt7iEgdnEQQkcP79ddfzS6OraysRP/+/XH48GFkZWVZdOEsEamP10QQkcN79tln8euvvyI2NhalpaXYuHEjvvvuO7zxxhucQBDVI2YiiMjhrV27Fm+99RYyMjJQUlKC8PBwPP3005gwYUJ9D41I0ziJICIiIpuwTgQRERHZhJMIIiIiskmdX1hpNBpx+fJlNGrUyK5lfomIyPEoioIbN24gODi4yhfC1YWSkhKUlZXZJbarq6vFpftvF3U+ibh8+bJFXypERES3rwsXLiAkJKRO11lSUgJ/Dw9ULdivjqCgIJw7d05TE4k6n0T8Udq4X32snoiI6lUFgJ1WlblXS1lZGQoB/AuAm8qxSwH8OysLZWVlnETY0x+nMFwAVP/ti0REdHurz9PZHgDU/jWv1QsMtbrdREREVEs8n0BERJriBPX/gtbqX+ScRBARkaZwEqEerW43ERER1RIzEUREpCnMRKhHq9tNREREteSwmQjl7NviDmFh0hiGQvEcyZJbeV0ke6iiQh6jUFLZpEmTwfIgFli3brOw/R//kMdwyrla63GU+QQI211LDPIgJSXCZl3gGGuGVCPlk/vFHYYPl8Yohqew3ZJjJCtL3C7ZHQCA8HBxe8OG6hxnyr7pwnZjt+7SGPn54nbZ+w4A9O7iqoN5ha7SGLL92qyZSvtspeTN99hj0hjFJeLPM0uOs19/FbcXFMhj+PmJ29X6PLMnZiLUo9XtJiIiolpy2EwEERGRPTAToR6tbjcRERHVEjMRRESkKcxEqEer201ERES1xEwEERFpCjMR6uEkgoiINEUH9X/p1993ktYvx51ESOpAvDFPfgi8NPSEsP1n9whpjFZhRmG768VMaQxf2U3xKhnW7qiwfVFSpDTGlAk+4g7HjkljuH77rbD956gHpDFaeVlQGEENPXuK29etk4b4sf3jwva0NPkw7r1X3H6nj7x+x9EMcX0O1XTrJmy2pKaFr5e4xsPlHHmNh2PHxH26B/0sH0hIiLyPCsqGjxS2uw69Txojc95XwvaMDPk4oqLE7bJaIwCQni7vQ9rhuJMIIiIiO9BB/cyBVjMRWj2NQ0RERLXETAQREWmK8+8PtWNqETMRREREZBNmIoiISFN4i6d6tLrdREREVEvMRBARkaYwE6EeTiKIiEhTOIlQj8NOIi5niV+Sl4ZbUEimsELY7OMjD2GUHBrXGraSxmhcN/VsABfxyzlldJ40hM7NU9heWtpZGsNVUpCqVf4RaQxcLJT3UUGe353C9vJB4nYAiA4S3yFeukexakzVOZEjLyRVRzXNcP6i+D3R3McgDzLrNWGz+7Q3pSFkdaKy3eTvzQZ1c5hJC3BlvScuJAUAEZ0aC9tdUq9LY8iOEUsKhZWXy/uQdjjsJIKIiMgemIlQj1a3m4iIiGqJmQgiItIUZiLUo9XtJiIiolpiJoKIiDSFmQj1aHW7iYiIqJaYiSAiIk3hV4Grx6ZJxOLFi7FgwQJkZWWhU6dOePfdd9G1a1dVB+blJW7PdpbfA+7hJ273nfy4NMb5WcuE7c2DyqQxsq+7SvuoQlKfAeHh0hCVleKxTpsmH8Zzz40UtgfmnpAHkRUBUElWlrg9J0ceo2SruA7ExT7yjxe3A+IYjRpZMA4L7vFXg6QcCYpd9NIYWU+J60C0Gt5fGiN/6TZhuyV1YH79Vd5HDbLjqNCCehUX/yuuA3FnG/lxdvW4+DiTvbYA0FhcruKW4AT1v3VTq2l9q7d7/fr1mDJlCmbOnIkjR46gU6dOiI+Px9WrV+0xPiIiInJQVk8iFi1ahLFjxyIxMRERERFYunQpPD09sWyZ+C92IiIiR+Bkp4cWWbXdZWVlSEtLQ1xc3B8BnJwQFxeH/fv3qz44IiIiclxWXRORk5ODyspKBAYGmi0PDAzEqVOnqn1OaWkpSktLTT8bDBbU1SciIrIT3uKpHrtv99y5c+Ht7W16hIaG2nuVREREVAesmkT4+fnB2dkZ2dnZZsuzs7MRFBRU7XOmT5+OgoIC0+PChQu2j5aIiKiWeE2EeqzabldXV0RHR2Pnzp2mZUajETt37kRsbGy1z3Fzc4Nerzd7EBER0a3P6joRU6ZMwahRo9ClSxd07doVSUlJKCoqQmJioj3GR0REpCpeE6EeqycRw4YNw7Vr1zBjxgxkZWUhKioKycnJVS62rC29u7iIk95dHsPoIi6cZEiS35bavERc/8LoEiCN4e8v7aKKsqEPC9tdM+RFnpzy84Xt8+Z1l8Z46ilx+4cLLSgkJSucpZKKCnF7ly7yGDduiNvb/iIu8AMAzac9Kmz/adZaaQxLxqqGYC/xxdHZN+TZxoYNxe1568SFpACgVfoucYeQntIYFRV1UwhOVsTJktpqkrcmnHLlx1nErInC9vPPvyONIXvtbgWcRKjHpoqVEyZMwIQJE9QeCxEREd1C+N0ZRESkKcxEqEer201ERES1xEwEERFpCr/FUz3MRBAREZFNmIkgIiJNcYb6XwWudrxbBTMRREREZBOHzUQUS+7f9kSxNEZhiTiGuwW1JsrcxXUgXPPzpDHy4CtfkQpcK8T7pDgsQhrDM+e8sN2pUP4Fah8miQ+rbd/K6wj07xAm7aMGWZ0ITxdxvRIAuAHxceblJR/H+XniOhB3fvq6NEb2E/9PviIVZP8qfv0CG8v3mUHy3pTVVQAAY9+7he1OGT9JY5R73ylfkQpKSsTtltSJkLFkn2X/P3EdiOYr35TGuPzYi5YOyWHpoP5f0LZcE7F48WIsWLAAWVlZ6NSpE95991107dq12r59+/bFnj17qiwfOHAg/vvf/wIARo8ejZUrV5q1x8fHIzk52YbRWcZhJxFERET24Ai3eK5fvx5TpkzB0qVLERMTg6SkJMTHx+P06dMICKj6x+vGjRtRVvbHBD03NxedOnXCP/7xD7N+CQkJWL58uelnNzc3K0dmHZ7OICIiqmOLFi3C2LFjkZiYiIiICCxduhSenp5Ytqz6Ssq+vr4ICgoyPbZv3w5PT88qkwg3Nzezfo0bN7brdnASQUREmmLPb/E0GAxmj9LS0irrLysrQ1paGuLi4v4Yk5MT4uLisH//fou24ZNPPsHw4cPR8C91yFNSUhAQEIA2bdrg6aefRm5urkXxbMVJBBERkUpCQ0Ph7e1tesydO7dKn5ycHFRWVlb5zqnAwEBkZWVJ13Hw4EEcO3YMTzzxhNnyhIQErFq1Cjt37sT8+fOxZ88eDBgwAJWVlbXbKAFeE0FERJpiz2siLly4AL3+j4uP7XFNwieffIKOHTtWuQhz+PDhpv937NgRkZGRaN26NVJSUtCvXz/VxwEwE0FERKQavV5v9qhuEuHn5wdnZ2dkZ2ebLc/OzkZQUJAwflFREdatW4cxY8ZIx9KqVSv4+fkhIyPDuo2wAicRRESkKfa8JsISrq6uiI6Oxs6dO03LjEYjdu7cidjYWOFzN2zYgNLSUjz22GPS9Vy8eBG5ublo2rSpFaOzDicRREREdWzKlCn46KOPsHLlSpw8eRJPP/00ioqKkJiYCAAYOXIkpk+fXuV5n3zyCYYOHYomTZqYLS8sLMQLL7yAAwcOIDMzEzt37sSQIUMQHh6O+Ph4u22Hw14TkZMjbm9eIb/4xCuslbC9sFA+Dr2XUdieXS4vJBWYfVS+IjVILsjxlKTJAEh3vDGkuTSE0//+K2zv366dNEZAlPi1U0t4uLg9r1BcFAkAMjNrP4477hC3//yIvJBUq8HVF6n5Q6Ck3TKyi70DPSSVlQBAUqBLVgQMAPLzxe2F7vJCUs0Xz5CvSAWygmMXL8pjyLbXx0ceQ1aQ6sS98kJSEQ+K/1IG/OQDqWeOUCdi2LBhuHbtGmbMmIGsrCxERUUhOTnZdLHl+fPn4eRkHvX06dP49ttvsW3btirxnJ2dcfToUaxcuRL5+fkIDg5G//79MWfOHLvWinDYSQQREZE9OMq3eE6YMAETJkyoti0lJaXKsjZt2kBRlGr7e3h4YOvWrTaMonZ4OoOIiIhswkwEERFpCr/FUz3MRBAREZFNmIkgIiJNcYQLK28XWt1uIiIiqiVmIoiISFN0UP8vaLXv9rhV6JSa7hexE4PBAG9vbwDxABrU5aqJiKjelQPYioKCArPvmKgLN3//7AIgKd1htUIAdwP1sl31iZkIIiLSFF4ToR5OIoiISFM4iVCPVrebiIiIaomZCCIi0hRmItSj1e0mIiKiWmImgoiINIWZCPVodbuJiIiolpiJICIiTXGUrwK/HTATQURERDZhJoKIiDSFXwWuHk4iiIhIU3hhpXq0ut1ERERUS8xEEBGRpjAToR6tbjcRERHVEjMRRESkKTqo/xc0b/EkIiIisgIzEUREpCm8JkI9DjuJ+O9/NwvbB/Y0yIN4eQmbjSq87E7Hjso7hYQIm3VNRtV6HACgdMsRtmd/sV8aw8ND3H7xonwcYWHi9lOn5DEkLx3atBksD2KBnTvFx9ndPctqvY6fL7pK+7TykxzPlux42XHmPUIewwJK6ivCdkPbrtIYstfXqUK+3/MKxfvVxYJPt/x8cXuLFuocZ0rRemH7Txc9pTHuDCkWdygpkQ+ksFDcLjmGAOBIuvhzMzpanX1GtwaHnUQQERHZAzMR6tHqdhMREVEtMRNBRESawkyEejiJICIiTeG3eKpHq5MnIiIiqiVmIoiISFP4LZ7qYSaCiIiIbOKwmYiEBHG7EXppDKcS8X3VTrJ7pgEUewUI2yvCIqUx9LCgpoUKzq8X14FonvqVNIbx3vuE7T4+8nG4u4vb27aVx7Dklnc13N3XKGw3Ql7jQXactSqUF8bI8xEfR+VNIqQxAuvoOMtuIa4DEbjuQ2mMstHjhO2uFhR5kB1nshoQgLxehVpOZIrrQER8NlseZNo0cbsFG1wc1ErYnnFMPgxL3r+OjhdWqker201ERES15LCZCCIiIntgJkI9Wt1uIiIiqiVmIoiISFOYiVAPJxFERKQpOqj/S5/FpoiIiIiswEwEERFpCk9nqEer201ERES15LCZCKf8PGF7sbuvNMa6deICL48/VCGN4VkiHgcq5DEcRV5PcSEpAHB1Fp/ZCy4tlcZ4f6m4QFNYmDSEtJCQanJyxO1+4mJjAPBGkvg4e+kxH2kMXx9x0SvpOAEgv24qdAU2LhO2/xwnLiQFAK3eWyRsvzx8ilVjqk5QkLxPXb19I4LEnyOXn5ghjZElKQTVuUOINIa75BM/MuiqNAYysuR9HBwzEerR6nYTERFRLTlsJoKIiMgemIlQj1a3m4iIiGqJmQgiItIUZiLUw0kEERFpCicR6tHqdhMREVEtcRJBRESa4mSnh7UWL16MsLAwuLu7IyYmBgcPHqyx74oVK6DT6cwe7n+5F15RFMyYMQNNmzaFh4cH4uLicObMGRtGZjmrT2fs3bsXCxYsQFpaGq5cuYJNmzZh6NCh6o/Mx0fY7AnJffUAHh8uvm/+u3S9NEb3LuJ74uFiwS4sqZv79zMzxe0dOshjOBUp4g4rPpTGeOwxcZ2AwkL5OIL9JPtdJT8XiutAtHKR1AkB8NIT4mIDe081l8boKbnFv9BdXq9CX3Je2kcVWeI6AT4+8u09kSCuAxHhIq9XcCJHvE+CfYqlMcQVTVSUkSFsDurSVRpC9jHyxkL51jz0kLj9Ti8LCmdY8plHUuvXr8eUKVOwdOlSxMTEICkpCfHx8Th9+jQCAqo/tvV6PU6fPm36Waczr+vz5ptv4p133sHKlSvRsmVLvPLKK4iPj8eJEyeqTDjUYvXkqaioCJ06dcLixYvtMR4iIiK7coRMxKJFizB27FgkJiYiIiICS5cuhaenJ5YtW1bjc3Q6HYKCgkyPwMBAU5uiKEhKSsLLL7+MIUOGIDIyEqtWrcLly5fxxRdfWDk6y1k9iRgwYABee+013H///fYYDxER0S3LYDCYPUqrqfJbVlaGtLQ0xMXFmZY5OTkhLi4O+/fvrzF2YWEhWrRogdDQUAwZMgTHjx83tZ07dw5ZWVlmMb29vRETEyOMWVu8JoKIiDTFnpmI0NBQeHt7mx5z586tsv6cnBxUVlaaZRIAIDAwEFk1nC5s06YNli1bhi+//BKrV6+G0WhE9+7dcfHiRQAwPc+amGqw+8mt0tJSs5mYwWCw9yqJiIjqxYULF6DX/3G9nZubmypxY2NjERsba/q5e/fuaNeuHT744APMmTNHlXXYwu6ZiLlz55rNykJDQ+29SiIiohrpgCp3OtT68XtsvV5v9qhuEuHn5wdnZ2dkZ2ebLc/OzkaQJd8cB6BBgwa46667kPH7Rbs3n1ebmLaw+yRi+vTpKCgoMD0uXLhg71USERHVzMXFPg8Lubq6Ijo6Gjt37jQtMxqN2Llzp1m2QaSyshI//vgjmjZtCgBo2bIlgoKCzGIaDAakpqZaHNMWdj+d4ebmplo6h4iI6HYwZcoUjBo1Cl26dEHXrl2RlJSEoqIiJCYmAgBGjhyJZs2ama6pmD17Nrp164bw8HDk5+djwYIF+OWXX/DEE08A+C2zMnnyZLz22mu44447TLd4BgcH26cMw++snkQUFhaa0ifAb1eEpqenw9fXF82by+8PJyIiqlcuLsBfaizUmqIAFRbU2fjdsGHDcO3aNcyYMQNZWVmIiopCcnKy6cLI8+fPw8npj5MF169fx9ixY5GVlYXGjRsjOjoa3333HSIiIkx9XnzxRRQVFWHcuHHIz89Hz549kZycbLcaEQCgUxRFUl3IXEpKCv7+979XWT5q1CisWLFC+nyDwQBvb28A8QAa1NivtHSzMM7vF6QKtcLP4g5+ftIYBogLUuld5AVtkJ8vbNY1e1IewwK5ueJ9ZslxJHsJR4+WxyhpKH5zpu+UH3KywjqDBg2WD8QCyqUPhO0/FQZLY9zpdVncQVI4DQCM7p7Cdqd8edEr2QeYLnCMPIYFLl0SH2eWXAgeFlb7cfhOeFTYfnnhWmmMYC/xhd467xFWjakmSuorwvaj7vJiU23b1n4chw+L27v7nKj1OnTtp0p6lAPYioKCArMLEOvCzd8/Be7u0Ks8iTAoCrxLSuplu+qT1ZmIvn37wsp5BxERkeOwVyZCg1gngoiIiGzCIuhERKQtzESohpkIIiIisgkzEUREpC3OzoCTyn9DG+XfLH074iSCiIi0xcWFkwiV8HQGERER2cRhMxGuyV8J21v17CkP4hUibC6DqzSEHmXC9vn/Ft/fDwBT/5kv7aOG7dvF7ZZUPpXdiy4peQEAyNgjvsDItY/8gqb8z+voIqVjx4TNd/b0kceo8BI2n8+RHyPNvcR1IL761lca474uknoVKpHV07GkTL+sZoklNU1kdSCCl78ujfHTP/6ffEVqiIoSNucfkIdwzb9a62F095IU8bCgVtJ3hZG1Hke9YyZCNcxEEBERkU0cNhNBRERkF8xEqIaZCCIiIrIJMxFERKQtzs6/PdRUWaluvFsEMxFERERkE2YiiIhIW1xc1M9EqF1G+xbBSQQREWkLJxGq4ekMIiIisonjZiK6dRM2G33kxXecSoqF7a6WVFZxEe+iqYnyAjDrdwfL16OCu+4St7u5yWPICgUF+4j3KQBkeYmLK5UfkBeSeuCF3pIe3tIYFpEUAbKo6lFhobC5uYu8CFSZl/gYua+nuBgVALy/rm6OM8lbQlqMSq0YwUHiW+rynpYXkrozfZd8RSpI/V5c2C4szIIgJSXidtlOBaRv8J8LA6QhuocZ5OtxdMxEqIaZCCIiIrKJ42YiiIiI7IGZCNUwE0FEREQ2YSaCiIi0xdnZsmtISIqZCCIiIrIJp2JERKQtLi7MRKiEe5GIiLSFkwjVOOxe1AWOqe8h3HLatBlc30NQkUp1ICR4nFmvWbPb6TirG926cZ/R7clhJxFERER2wUyEanhhJREREdmEUzEiItIWe9ziqcjL+d+OmIkgIiIimzATQURE2mKPayKYiSAiIiKyHDMRRESkLcxEqIaTCCIi0hZOIlTD0xlERERkE2YiiIhIW5iJUA0zEURERGQTZiKIiEhb7FFsymhUN94tgpkIIiIisgkzEUREpC32uCaCmQgiIiIiyzETQURE2sJMhGqYiSAiIiKbOGwmorJys7C9sFAeQ1+RJ+5QUSGNka0ECNsDPQzygeTnC5t1LcbLY1hknbBVOX1JGsEYfqew3SnnqnwYKSnidh8feYwOHYTNumZPymNYIDRUfJwlJ8tjRISXiTusWCEPEhYmbl+9Wh4jKUnYrGsySh7DImuErcolC96ckveEJX8lZnuLj9XAV8bJxxEeLmzWTf1GHsMCvXqJj7O9H/8kjVEWJt5e14pi+UBk+1VyDAGQvn91T4q31SEwE6Eah51EEBER2QUnEarh6QwiIiKyCTMRRESkLfYoNlVZqW68WwQzEURERGQTTiKIiEhbbl4TofbDSosXL0ZYWBjc3d0RExODgwcP1tj3o48+Qq9evdC4cWM0btwYcXFxVfqPHj0aOp3O7JGQkGD1uKzBSQQREVEdW79+PaZMmYKZM2fiyJEj6NSpE+Lj43H1avV3wKWkpOCRRx7B7t27sX//foSGhqJ///64dMn8rruEhARcuXLF9Pi///s/u24HJxFERKQtDpCJWLRoEcaOHYvExERERERg6dKl8PT0xLJly6rtv2bNGjzzzDOIiopC27Zt8fHHH8NoNGLnzp1m/dzc3BAUFGR6NG7c2ObdZAmHvbDy2jVxe+C+jdIYhrgHhO1ePvJxBFaIawDkFeqlMUpc5H3UoGQXiTv4hEljSPd7bo40RmqLh4XtMe3ktTXK3Otmn23ZokKQjAxx+0MPyWPkiPdr2cerpCFcs87L16OC0lLxa5P6vfy1i2kp+eix4AM5MOuEsN2w8ENpDH3hZXEHlepE7E0W13AY+ZS4BgQArHpN/PoaQ5pLYzhdlBwjlhyrUrdAnYh6VlZWhrS0NEyfPt20zMnJCXFxcdi/f79FMYqLi1FeXg5fX1+z5SkpKQgICEDjxo1x991347XXXkOTJk1UHf+fOewkgoiIyC7sUSfi97szDAbzP5Lc3Nzg5uZmtiwnJweVlZUIDAw0Wx4YGIhTp05ZtLqpU6ciODgYcXFxpmUJCQl44IEH0LJlS5w9exYvvfQSBgwYgP3798PZ2dmWrZLiJIKIiLTFHrd4/l4BOTQ01GzxzJkzMWvWLFVXNW/ePKxbtw4pKSlwd3c3LR8+fLjp/x07dkRkZCRat26NlJQU9OvXT9Ux3MRJBBERkUouXLgAvf6PU3p/zUIAgJ+fH5ydnZGdnW22PDs7G0FBQcL4CxcuxLx587Bjxw5ERkYK+7Zq1Qp+fn7IyMiw2ySCF1YSEZG22PHCSr1eb/aobhLh6uqK6Ohos4sib14kGRsbW+Ow33zzTcyZMwfJycno0qWLdDMvXryI3NxcNG3a1IadZBlOIoiIiOrYlClT8NFHH2HlypU4efIknn76aRQVFSExMREAMHLkSLMLL+fPn49XXnkFy5YtQ1hYGLKyspCVlYXC37+NsrCwEC+88AIOHDiAzMxM7Ny5E0OGDEF4eDji4+Ptth08nUFERNpijwsrrYw3bNgwXLt2DTNmzEBWVhaioqKQnJxsutjy/PnzcHL64+/8JUuWoKysDA/95Q6am9dcODs74+jRo1i5ciXy8/MRHByM/v37Y86cOdVmQ9TCSQQREVE9mDBhAiZMmFBtW0pKitnPmZmZwlgeHh7YunWrSiOzHCcRRESkLQ6QibhdOOxWBzYSF2fJ7iEuJAUAgSXVlw81cfGSxsgr8RS2+/rIv0O+uKSOLj3x8RE2G0pcpSFKSyUdwsOlMWIqxK8dcvKlMVz96ubQjOwgef3y86UxZr8XIWyfMfwn+UBCQoTNriXyAl119SHmWpgnbI/5m480xovTAoTtb06WFIECAD8/YbMeFuwzL/lngBrKXMSfI7JCUgCwKkVcTGpkguTzDpB+RuBPtwvW6PdbGYkAB55EEBER2QUzEarR5lYTEZF22aPYlJ0qQjo63uJJRERENmEmgoiItIWnM1TDTAQRERHZRJtTJyIi0i5mIlTDTAQRERHZxKqp09y5c7Fx40acOnUKHh4e6N69O+bPn482bdrYYWTioQX6y+szoMJH2Lz3gLxuQu8o2b3m8vvMPV3KpH1UUVIibNbnZ0lD6CX33n+9Q3y/OwAM7JIvbC8LEt/vDtThpP7UKXG7pH4DAMwYLb7Hv1XfO6Uxfk6R1AmQvLYA6qzmgXQ9FtQRePOJTGH7i0nyffbmUz+LO1iyP+qo5oHs5XOV1W8AMLLLCWH7tnRxvRIA6B8u2WeSb5AEAKO7/DPA4TEToRqrMhF79uzB+PHjceDAAWzfvh3l5eXo378/ioqK7DU+IiIiclBWTZ2Sk5PNfl6xYgUCAgKQlpaG3r17qzowIiIiu2CdCNXUai8WFBQAAHx9fVUZDBERkd3xdIZqbN5qo9GIyZMno0ePHujQoUON/UpLS1H6py9kMBgsqGdPREREDs/mScT48eNx7NgxfPvtt8J+c+fOxauvvmrraoiIiNTFTIRqbLrFc8KECdiyZQt2796NEMnV69OnT0dBQYHpceHCBZsGSkRERI7FqqmToih49tlnsWnTJqSkpKBly5bS57i5ucHNzc3mARIREamKmQjVWLXV48ePx9q1a/Hll1+iUaNGyMr6re6At7c3PDw87DJAIiIickxWTSKWLFkCAOjbt6/Z8uXLl2P06NFqjQkAkFcoLgTl7i6PUVEhjtG77VV5kIyLwubsZp2lITw85EWtVHFRPNbUG/JiNDF+xcL2gT3lF8ZuOxAgbO/vd0Qao6yDfL+qQo0CTZJCQT8n/yQNMUVSXGnRLAsuSK6jv4Rk703fksvyIJKxSgtJAfhwRyth+2OPyYfhWZEn76QC6WF2MV8eRFIIrr+f/PNsW7p4n/WV11a7Pb4rgZkI1Vh9OoOIiIgIuE0mlURERBZjsSnVcBJBRETawtMZquG3eBIREZFNtDl1IiIi7WImQjXMRBAREZFNtDl1IiIi7WImQjUOu9W+FeJ7ni/ni2sRAECwX5m4Q2GFNMZ5P3G9guZF8vvZUWlBUQs1nDsnbI5pZ8E4MgqFzWVtI6Uh+l9cJmzXxUdJYyipB6V9VPH888Jmw8f/kYbQ558Xd5B8vwwALHpK3K7zzpbGUH7wlvZRg+/zj4s7LF0qDyKpaYKSEmkIWR0ISUV+AEBe0hZ5JxU4LXxT3CEuTh5E8EWHAIDfi/+JdOsmbndzuyKNoazeJe1D2uGwkwgiIiK74C2equE1EURERGQTZiKIiEhbeE2EarS51UREpF2cRKiGpzOIiIjIJtqcOhERkXYxE6EaZiKIiIjIJtqcOhERkXYxE6Eax93qQnHRo4yL8mJTFRWuwvbm7vLNLxEPA/DxksZAhbyolSo6dhQ2F/s1l4bwzBEXTnKtKJbGMI4WFyNSAv8rjfFV1iBpH1WsXi1s1mf+JI+RmSluDwyUxwgKEjYrabIDEdB1ai9fjxrathW3W3K8yz5wLYjhWSguSJf3sbzIl+5BH2kfVUheX2RkyGOEh4vb3Wtf1M6SQlK6xwZKeqyr9Tjo1uG4kwgiIiJ7YLEp1fCaCCIiIrIJMxFERKQtvCZCNdrcaiIi0i5OIlTD0xlERERkE21OnYiISLuYiVANMxFERERkE52iKEpdrtBgMMDb2xtAPIAGdblqIiKqd+UAtqKgoAB6vb5O13zz90/BuXOqr9tgMMC7Zct62a76xEwEERER2USbJ3GIiEi7eE2EapiJICIiIptoc+pERETaxUyEapiJICIibbk5iVD7YaXFixcjLCwM7u7uiImJwcGDB4X9N2zYgLZt28Ld3R0dO3bE119/bdauKApmzJiBpk2bwsPDA3FxcThz5ozV47IGJxFERER1bP369ZgyZQpmzpyJI0eOoFOnToiPj8fVq9V/O+13332HRx55BGPGjMH333+PoUOHYujQoTh27Jipz5tvvol33nkHS5cuRWpqKho2bIj4+HiUlJTYbTt4iycREdUhB7jF8/p1+9zi2bixxdsVExODv/3tb3jvvfcAAEajEaGhoXj22Wcxbdq0Kv2HDRuGoqIibNmyxbSsW7duiIqKwtKlS6EoCoKDg/Hcc8/h+eefBwAUFBQgMDAQK1aswPDhw1XaUnPMRBAREanEYDCYPUpLS6v0KSsrQ1paGuLi4kzLnJycEBcXh/3791cbd//+/Wb9ASA+Pt7U/9y5c8jKyjLr4+3tjZiYmBpjqoGTCCIi0hQjnOzyAIDQ0FB4e3ubHnPnzq2y/pycHFRWViIwMNBseWBgILKysqodc1ZWlrD/zX+tiakGbV5OSkREZAcXLlwwO53h5uZWj6OxP04iiIhIUyoqfnuoHRMA9Hq99JoIPz8/ODs7Izs722x5dnY2goKCqn1OUFCQsP/Nf7Ozs9G0aVOzPlFRUdZsilV4OoOIiKgOubq6Ijo6Gjt37jQtMxqN2LlzJ2JjY6t9TmxsrFl/ANi+fbupf8uWLREUFGTWx2AwIDU1tcaYamAmgoiINMWemQhLTZkyBaNGjUKXLl3QtWtXJCUloaioCImJiQCAkSNHolmzZqZrKiZNmoQ+ffrgrbfewqBBg7Bu3TocPnwYH374IQBAp9Nh8uTJeO2113DHHXegZcuWeOWVVxAcHIyhQ4equalmOIkgIiJNcYRJxLBhw3Dt2jXMmDEDWVlZiIqKQnJysunCyPPnz8PJ6Y+TBd27d8fatWvx8ssv46WXXsIdd9yBL774Ah06dDD1efHFF1FUVIRx48YhPz8fPXv2RHJyMtzd3VXZxuqwTgQREdWh+q8TceGC+us2GAwIDfXW3FeBMxNBRESa4giZiNsFL6wkIiIimzhsJiIra7Ow3d9fHsOpoqz2A1Hjm9kkU1Sd24O1XwcAZbW4rKnxkRHSGLIS65bMtgsL5X1k/PzE7W5ug2u/EgDKpQ9qNxALFFe4SvvcuCFur6boXRXN/YqF7bqGw+RBLHD8uPi9GR4uj+EKyXvTkgNNjYNV8vrqnIfIY1hAKVgj7uDlJQ8i2R6ji/w4k703LXnvBvuJXzu1Ps/sqbJS/cxBZaW68W4VzEQQERGRTRw2E0FERGQPvCZCPcxEEBERkU2YiSAiIk1hJkI9zEQQERGRTZiJICIiTWEmQj2cRBARkabwFk/1OOwkItDDIGz/OlleVnRggmTz8vPlA5EdaT4+8hhq1JqwgGGwuA6EPuMnaQyXsDuF7atXy8cx7t7L4g4W3BN/OaeOysZmZIjbv/9eGmJj6SBhe7du8mHIdokldVG+TvaUd1KB7JA/fFgeo7uPZL/LakAAQA1fmXxTnnuwNITvqRPy9aghM7N27QC+drlP2G7JcVZeLm6X1YAAgF3fyutRkHY47CSCiIjIHng6Qz28sJKIiIhswkwEERFpCjMR6mEmgoiIiGzCTAQREWkKMxHqYSaCiIiIbMJMBBERaQozEerhJIKIiDSFxabUc8tOIgZ2OC/t8/gTzYXty95zl69IVgTGkqI4lvRRwcWL4va2bcWFpABgr7NO2N4lTZHGWJsiLvJTWCgNUXfCwoTNqVfExxAAPDD/AWH7Xr+N0hiyQkFOMEpjhIXVzdnJYPc8cXtQvjSGrrWHsF1JteAgycoSNvv6WPC+k71p1CIpjHUUkdIQA5c+I2z/zud9aYzu3STHUb58v4eH+0r7kHbcspMIIiIiW/B0hnp4YSURERHZhJkIIiLSFGYi1MNMBBEREdmEmQgiItIUZiLUw0wEERER2YSZCCIi0hTWiVCPVZOIJUuWYMmSJcj8vXZC+/btMWPGDAwYMED9kclqK3h5SUMse69Y2L5+s6c0xj/+ESFsd8oX3zMPAPDxkfdRwY8/itstuSU+5Li4DsT59uI6EgDQVlJLwpLdUVe1JI7mi+tA3HGHPMaRl8V1INpGy/fZqR/E++zXX+VJw8aNpV3UIanPYMl7U/khX9iu6+Qtj3G2gbiDJb8l/PzkfdQgWU+4fJfhu8fEdSC6PyWvNXFi3VFhu4+PvAbE7ZC25+kM9Vh1OiMkJATz5s1DWloaDh8+jLvvvhtDhgzB8ePH7TU+IiIiclBWZSIGDx5s9vPrr7+OJUuW4MCBA2jfvr2qAyMiIrIHZiLUY/M1EZWVldiwYQOKiooQGxur5piIiIjoFmD1JOLHH39EbGwsSkpK4OXlhU2bNiEioubrBkpLS1FaWmr62WAw2DZSIiIiFTAToR6rb/Fs06YN0tPTkZqaiqeffhqjRo3CiRMnauw/d+5ceHt7mx6hoaG1GjARERE5BqsnEa6urggPD0d0dDTmzp2LTp064e23366x//Tp01FQUGB6XLhwoVYDJiIiqo2bmQi1H1pU6zoRRqPR7HTFX7m5ucHNza22qyEiIiIHY9UkYvr06RgwYACaN2+OGzduYO3atUhJScHWrVvtNT4iIiJVsdiUeqyaRFy9ehUjR47ElStX4O3tjcjISGzduhX33HOP+iOTFayRFbwBAHd3YfOwf4jbAWDZCvEZn8eHy2MgP1/eRwUNJLV3+neRF8b67pS42EzEL+KiSADg2kJcXGnbSnmMLl2kXVQhO4wiw8UFywDAN0p8DJy3YJ9FvjxS2H70+VXSGHeGlUn7qEJWCcySAk6SimPKJflHU7t+wcL2k+vFhZUAAEFB8j4qOHpM/DnSoYM8RvduRmH75WT59kasWySOMXyKNEarkDo6zuyIF1aqx6pJxCeffGKvcRAREdEtht+dQUREmsJMhHr4LZ5ERERkE2YiiIhIU5iJUA8zEURERGQTZiKIiEhTeIunepiJICIiIps4bibCRTy0spBW0hCuOZfFHWT3uwN4/CFx+1c79NIY90XlSPuoQbo5knvzAaCkpPbj+PoTcV2EvqPEdSQA4NoBeW0FNVy8KG4vhqc0hmeJuJZERYU8hqwOROTL90lj4OOP5X3UcO2asNnQtqs0hL7kvLiDBX8mnlxzRNj+6LzO0hhr35PXTlFDRoa4PTxcHsOzQvwGd3eXfxbJ6kAEJy+TD2ToUHkfB8drItTjuJMIIiIiO+AkQj08nUFEREQ24SSCiIg05Vb7Fs+8vDyMGDECer0ePj4+GDNmDAoF56/z8vLw7LPPok2bNvDw8EDz5s0xceJEFBQUmPXT6XRVHuvWrbNqbDydQURE5MBGjBiBK1euYPv27SgvL0diYiLGjRuHtWvXVtv/8uXLuHz5MhYuXIiIiAj88ssveOqpp3D58mV89tlnZn2XL1+OhIQE088+Flw792ecRBARkabcStdEnDx5EsnJyTh06BC6/P7NhO+++y4GDhyIhQsXIji46hfRdejQAZ9//rnp59atW+P111/HY489hoqKCrj86cYFHx8fBNXii+h4OoOIiMhB7d+/Hz4+PqYJBADExcXByckJqampFscpKCiAXq83m0AAwPjx4+Hn54euXbti2bJlUBTr7oxjJoKIiDTFnsWmDAaD2XI3Nze4ubnZHDcrKwsBAQFmy1xcXODr64usrCyLYuTk5GDOnDkYN26c2fLZs2fj7rvvhqenJ7Zt24ZnnnkGhYWFmDhxosXjYyaCiIhIJaGhofD29jY95s6dW22/adOmVXth458fp06dqvV4DAYDBg0ahIiICMyaNcus7ZVXXkGPHj1w1113YerUqXjxxRexYMECq+I7biYiP1/Y7GrJNFJWfcnLSx4jR1wo6r4O8kJSDz8vL4ylhr59xe2GQvmcMT1d3O7uLh+H7PRa+ufydFlIN1lBqnvlA7HA4+F7he17D/e2IIq4mJSkbhoAoFs3SYcVK+RB/vc/eR8VGAcMErbrL0oKSQHA4cPidj8/eQzJBWBrk65KQzj5BUj7qOGBLuJ9cuRUc2kMLy9xManr1+XjiLmrTNzBkkJSX3wh7+PgKioAZ2f1YwLAhQsXoNf/8VrVlIV47rnnMHr0aGHMVq1aISgoCFevmh/LFRUVyMvLk17LcOPGDSQkJKBRo0bYtGkTGjRoIOwfExODOXPmoLS01OLsieNOIoiIiOzAnpMIvV5vNomoib+/P/z9/aX9YmNjkZ+fj7S0NERHRwMAdu3aBaPRiJiYmBqfZzAYEB8fDzc3N3z11Vdwt+AvwPT0dDRu3Niq0y+cRBARETmodu3aISEhAWPHjsXSpUtRXl6OCRMmYPjw4aY7My5duoR+/fph1apV6Nq1KwwGA/r374/i4mKsXr0aBoPBdK2Gv78/nJ2dsXnzZmRnZ6Nbt25wd3fH9u3b8cYbb+D555+3anycRBARkabYMxNhD2vWrMGECRPQr18/ODk54cEHH8Q777xjai8vL8fp06dRXPzb9/gcOXLEdOdG+F++mOXcuXMICwtDgwYNsHjxYvzrX/+CoigIDw/HokWLMHbsWKvGxkkEERGRA/P19a2xsBQAhIWFmd2a2bdvX+mtmgkJCWZFpmzFSQQREWmKPW/x1Bre4klEREQ2YSaCiIg0paICcFL5T2itfhW4w04idIFj6nsIt5wWLQbX9xBUpE4dCBldH1lhFesKr2iBs/PtdJzVDV2L8fU9BCK7cNhJBBERkT0wE6EeTiKIiEhTOIlQDy+sJCIiIpswE0FERJrCTIR6mIkgIiIimzATQUREmsJiU+phJoKIiIhswkwEERFpSkUFoNOpH1OLmIkgIiIimzATQUREmsJMhHo4iSAiIk3hJEI9PJ1BRERENmEmgoiINIWZCPUwE0FEREQ2YSaCiIg0pbJS/UwEi00RERERWYGZCCIi0hR7XL+g1WsiHHYScdddm4Xthw/LYzjlXBV3yMqSB/HzE7d/9pk8xr33Cpt1rSfJY1hAGSDOp5V98bU0xqlT4vbIIMk+BZDnEiBsd7HgqMvIELdHRw+WB7HIZGFrUVE/aQTPnPPiDklJ8mG8/LK4ffRoeYx584TNuvZT5TEscM894vfmtmSjPEhOjri9pEQeQ/beXL5cHiM2Vtisi54pj2GBli3F++znY8XyIGrss5AQcfumTfIY7doJm9XaZ3RrcNhJBBERkT0wE6EeXhNBRERENmEmgoiINIWZCPVwEkFERJpij9sxeYsnERERkRWYiSAiIk2pqAAURd2YzEQQERERWUGnKGrPx8QMBgO8vb0BxANoUGO/0lLxfdWuFRbcV+3uLm634L5qo7unsN2Si2lcXcT3zeuch8iDWKCoSLzPPEvy5EF8fITNxSXyeadstztBXkfAKJnfOjurUydCts9k2wIATiWSY9GSwhiSA6nMRXwcAvL3hK7hMPk4LFBZKd5nlnCqKKv9QCT7taxCfqzW1XtTqfxS2K7GWC36MJLsM9n7DpC/djq3ByURygFsRUFBAfR6vXR9arr5+yciogDOzuquu7LSgBMnvOtlu+oTMxFERERkE14TQUREmsJrItTDTAQRERHZhJkIIiLSFGYi1MNJBBERaUplpfqTCKMF3zt3O+LpDCIiIrIJMxFERKQpFRWAk8p/QjMTQURERGQFh81EuOZcFrZnOwdLY7zwlLh91bRMaQynkBBhu2tWljSGRcWGVJCZKW738vKVxvBtohO2e1bKTyR+8YW4vVkz+dw1N1faRRWeWT+LO0hefwDQNfxe2K4cbywfSEaGsNlVUgQMAHDjhryPCpyyxO9NBAVJY3Tt6SpsP7jihHwgkn3iakExuTr76sWLF4XNrhbsM18/8T7LO5UvH4fkOHOy5DizZL86OGYi1MNMBBEREdnEYTMRRERE9sBMhHqYiSAiIiKbMBNBRESaUlmpfuagbr/K0nFwEkFERJpSUQHoxNeQW02rkwieziAiIiKbMBNBRESawkyEemo1iZg3bx6mT5+OSZMmISkpSaUhWSbQwyDtsypJfA/4yMkR8hgLr4o7WHB/d13dVx3RVnySz1AoTzy5lorfCdIaAQAe6Osu7mDBveh5+XWUJDt8WNxuQZ0IJbedsN2jmbw+x69nfcQdJHUGAABt20o6LJXHsISXl7jdguP94JZCYftLSfL35hvTJJ8BsnECdVfzwF3ynrCgXkVehnifvbM6QBpj4hOSfVJXdTPotmHzJOLQoUP44IMPEBkZqeZ4iIiI7IqZCPXY9OdeYWEhRowYgY8++giNG1tQjY+IiIhskpeXhxEjRkCv18PHxwdjxoxBYaE4M9W3b1/odDqzx1NPmZdxPn/+PAYNGgRPT08EBATghRdeQIWV2SibMhHjx4/HoEGDEBcXh9dee82WEERERPXiVstEjBgxAleuXMH27dtRXl6OxMREjBs3DmvXrhU+b+zYsZg9e7bpZ09PT9P/KysrMWjQIAQFBeG7777DlStXMHLkSDRo0ABvvPGGxWOzehKxbt06HDlyBIcOHbKof2lpKUpLS00/GwzyaxmIiIgIOHnyJJKTk3Ho0CF06dIFAPDuu+9i4MCBWLhwIYKDa/4eKU9PTwTVcN3etm3bcOLECezYsQOBgYGIiorCnDlzMHXqVMyaNQuuruLvarnJqtMZFy5cwKRJk7BmzRq4yy4U+t3cuXPh7e1teoSGhlqzSiIiIpUZoSjqPoDfLmw3GAxmjz//EW2L/fv3w8fHxzSBAIC4uDg4OTkhNTVV+Nw1a9bAz88PHTp0wPTp01FcXGwWt2PHjggMDDQti4+Ph8FgwPHjxy0en1WTiLS0NFy9ehWdO3eGi4sLXFxcsGfPHrzzzjtwcXFBZWVlledMnz4dBQUFpseFCxesWSUREZHKKu30AEJDQ83+cJ47d26tRpqVlYWAAPM7b1xcXODr64sswbdIP/roo1i9ejV2796N6dOn49NPP8Vjjz1mFvfPEwgApp9Fcf/KqtMZ/fr1w48//mi2LDExEW3btsXUqVPh7Oxc5Tlubm5wc3OzZjVERES3pAsXLkCv15t+run337Rp0zB//nxhrJMnT9o8jnHjxpn+37FjRzRt2hT9+vXD2bNn0bp1a5vj/pVVk4hGjRqhQ4cOZssaNmyIJk2aVFlORETkmP7IHKgbE9Dr9WaTiJo899xzGD16tLBPq1atEBQUhKtXzesVVVRUIC8vr8brHaoTExMDAMjIyEDr1q0RFBSEgwcPmvXJzs4GAKviOm7FSsk1Fz/nyF+kVu7iwkirJh+Rxpi/vLOwfeo/5cWX4FI3uzn7mvjsVGamPMbZs+L2R+Ms2JahQ4XNP328Vxri++/lq1FFQoK43ZLbnVJShM2//tdHGkLXzFPYrhyXH+84dkzeRw2WFHGSkRyMb4zOl4ZY9PGdwvYpE8rk48jJkfdRg6zAmiVFr9LThc0TH5IVGwP6D635IjwA2LbCgs8zK1LdVDN/f3/4+/tL+8XGxiI/Px9paWmIjo4GAOzatQtGo9E0MbBE+u/HT9OmTU1xX3/9dVy9etV0umT79u3Q6/WIiJAXe7up1r/dUiQfoERERI7FfpkItbVr1w4JCQkYO3Ysli5divLyckyYMAHDhw833Zlx6dIl9OvXD6tWrULXrl1x9uxZrF27FgMHDkSTJk1w9OhR/Otf/0Lv3r1NBSL79++PiIgI/POf/8Sbb76JrKwsvPzyyxg/frxVlyDwC7iIiIgc2Jo1a9C2bVv069cPAwcORM+ePfHhhx+a2svLy3H69GnT3Reurq7YsWMH+vfvj7Zt2+K5557Dgw8+iM2bN5ue4+zsjC1btsDZ2RmxsbF47LHHMHLkSLO6EpZw3NMZREREdvHHLZnqxrQPX19fYWGpsLAwKH+qdhUaGoo9e/ZI47Zo0QJff/11rcbGTAQRERHZhJkIIiLSmFvnmghHx0kEERFpjBHq/9K33+kMR8bTGURERGQTx81EfPyxsDlk8ovyGFmSe/xl924DmPrIeWF753ubS2McWX1C2kcNjRqJ22Payb/8rKhIUo/Agu9MkdWBuHP1DGmMrDjrrhC22fDhwuayL+QXHbmGhYk7WFC/QTneRdiua79PHmNfe2kfVSxZImw2Pj1eGsJJhVoTU54qFrb3v1dcewMAtr1WR3UiNmwQNhsfGSEN4WRFAaCayOpARCaI60gAwNGPL9Z6HPWPpzPUwkwEERER2cRxMxFERER2wUyEWpiJICIiIpswE0FERBpzaxWbcmTMRBAREZFNmIkgIiKN4TURauEkgoiINIaTCLXwdAYRERHZxHEzEUOHCpuvX5eHCKyQFJsqKZEHkRQSOvLxEWmIAc91lq9HBadOidv9/CSFpCyQ/as8RmamuL3kIXkhqeaddJIe90pjWOS994TNrhXigkYAgIuS4js9e8pjnDwpbFa2tpCG0PVQpH1UERsrbHay5AIz2XtT1m5Bn22vHZSG8E3oKl+PGtq1EzY7lVhwnMn2iYsFH+dZWcJmSwpJ6WLukq/H4TEToRZmIoiIiMgmjpuJICIisgt+AZdamIkgIiIimzATQUREGsNiU2phJoKIiIhswkwEERFpDO/OUAsnEUREpDGcRKhFpyhKHd1c/huDwQBvb28A8QAa1OWqiYio3pUD2IqCggLo9bWvXWONP37/fAvAS+XohQB61st21SdmIoiISGOYiVALL6wkIiIimzATQUREGsNMhFqYiSAiIiKbMBNBREQao0D94lB1eo+Cw2AmgoiIiGzCTAQREWkMr4lQCycRRESkMZxEqIWnM4iIiMgmzEQQEZHGMBOhFmYiiIiIyCbMRBARkcYwE6EWZiKIiIjIJsxEEBGRxhihfrEptePdGpiJICIiIpswE0FERBrDayLUwkwEERER2cRhMxGlpZuF7S4WjNxJco7KaMEcShZDDTrnIarEUY7PF3do21YaQ7ZPCgvl45C9NiUl8hg+PuJ2Z+fB8iAWULI/EXfw86v1OopL5MdZRUXt2gHA16tM2K5ze1AexAK5ueL3puy1A1R6X6mx09zdhc2qvTdLPxd3sOQDTcKSzzM1dpmnu/i1U2uf2ZcR6mcOtHlNhMNOIoiIiOyDpzPUwtMZREREZBNmIoiISGN4i6damIkgIiIimzATQUREGsNrItTCTAQRERHZhJkIIiLSGGYi1OKwkwjXimJhexk8pTFKJPfn6/PPS2PkeTUXtvv6yC+mMRTWUcInJETcHhcnDbFl8i5h+8WL8mH07StujwgTv7YAsGq1/PVVRXKyuH31ammIKR22CdsnTJAPQ7ZfO3SQx5jxmqu8kwpycsTtBw7IYwz0+lbcwZJiIlFRwuZsJUAaIjD9O/l61JCeLm4/dUoaYqPXSGG7ZHcAkNfwsOTz7KstTGDTH3g0EBGRxlTa6WEfeXl5GDFiBPR6PXx8fDBmzBgUCir/ZWZmQqfTVfvYsGGDqV917evWrbNqbA6biSAiIrKPW6ti5YgRI3DlyhVs374d5eXlSExMxLhx47B27dpq+4eGhuLKlStmyz788EMsWLAAAwYMMFu+fPlyJCQkmH72saTk7J9wEkFEROSgTp48ieTkZBw6dAhdunQBALz77rsYOHAgFi5ciODg4CrPcXZ2RlBQkNmyTZs24eGHH4aXl5fZch8fnyp9rcHTGUREpDFGOz0Ag8Fg9igtLa3VSPfv3w8fHx/TBAIA4uLi4OTkhNTUVItipKWlIT09HWPGjKnSNn78ePj5+aFr165YtmwZFEWxanycRBAREakkNDQU3t7epsfcuXNrFS8rKwsBAeYXCbu4uMDX1xdZWVkWxfjkk0/Qrl07dO/e3Wz57Nmz8Z///Afbt2/Hgw8+iGeeeQbvvvuuVePj6QwiItIY+93ieeHCBej1etNSNze3antPmzYN8+eLv3n55MmTtR7Vr7/+irVr1+KVV16p0vbnZXfddReKioqwYMECTJw40eL4nEQQERGpRK/Xm00iavLcc89h9OjRwj6tWrVCUFAQrl69ara8oqICeXl5Fl3L8Nlnn6G4uBgjR4pvEQaAmJgYzJkzB6WlpTVOfv6KkwgiItKY+i825e/vD39/f2m/2NhY5OfnIy0tDdHR0QCAXbt2wWg0IiYmRvr8Tz75BPfdd59F60pPT0fjxo0tnkAAt/AkwsWCkcuK4uiD/KQxfN0lt+3k50tj6C0ZrAouF4pnv5mviQtJAcCxHjphe0Ka/KKbL74Qt6eHyQtJZWZKu6hDUhlLVuAHAKY8KN5nJxLk+6wWF0eb/Om6K7u6M0RcLOxOd8kbD4CuRQ9hu5KaJh+I5L0X6G5BwaqKCnkfNYSHC5tPeHWVhnjg4ynC9tRmi6QxWoVJPs8sKPIVFVVHheAIANCuXTskJCRg7NixWLp0KcrLyzFhwgQMHz7cdGfGpUuX0K9fP6xatQpdu/5xLGVkZGDv3r34+uuvq8TdvHkzsrOz0a1bN7i7u2P79u1444038Pzzz1s1vlt2EkFERGSb+s9EWGPNmjWYMGEC+vXrBycnJzz44IN45513TO3l5eU4ffo0iovNJ/jLli1DSEgI+vfvXyVmgwYNsHjxYvzrX/+CoigIDw/HokWLMHbsWKvGxkkEERFpzK01ifD19a2xsBQAhIWFVXtr5htvvIE33nij2uckJCSYFZmyFW/xJCIiIpswE0FERBrzR3EodWNqDzMRREREZBNmIoiISGNurS/gcmTMRBAREZFNrMpEzJo1C6+++qrZsjZt2uDUqVOqDgoAjO7ie5GdSsT3qgNAqzB3YXv2Nfn9zoGSOhFGH19pDKeKMmkfNchehvbt5TFaZ4lrGgT+rbk0RrND54XtubnycfTrJ26fOVMewxKpV8TbY8nFyzm/iPdZ/3sjpTHObzkqbL94UT4OC+rOqENWS8CCrxJWssUHwZR58roJi17Ok/aRatu29jEsIPucCPcSNgMALj8vrgMRs+VDaYy8O8YJ211c5J+JXhaM1fHdWndnODKrT2e0b98eO3bs+CNAHRVSIiIiIsdi9QzAxcWlVt89TkREVL+YiVCL1ddEnDlzBsHBwWjVqhVGjBiB8+fFqWsiIiLHUmmnh/ZYlYmIiYnBihUr0KZNG1y5cgWvvvoqevXqhWPHjqFRo0bVPqe0tBSlpaWmnw0GQ+1GTERERA7BqknEgAEDTP+PjIxETEwMWrRogf/85z8YM2ZMtc+ZO3dulYsxiYiI6g9PZ6ilVrd4+vj44M4770RGRkaNfaZPn46CggLT48KFC7VZJRERETmIWk0iCgsLcfbsWTRt2rTGPm5ubtDr9WYPIiKi+mO000N7rJpEPP/889izZw8yMzPx3Xff4f7774ezszMeeeQRe42PiIiIHJRV10RcvHgRjzzyCHJzc+Hv74+ePXviwIED8Pf3V31gToXiCzDL3OUZDVdJkadAf/nmGwrF8yy9i7zoFdzFRa/UIis2ZUldnS1bxO09k+V34/wnSCdsd31dXJwJqLMaQNi5U9x+xx3yGLJSKakfiQtJAcD1FuJ9dnWlfJ8NHSrtoo7Dh8XtffvKY+TkCJsXTSiUhvANbyVsz9vynXwcYWHyPipITha3W1LULDhI/Jfu+QRxISkAaD55pLD9xLRV0hgRYRZ85jk8lr1Wi1WTiHXr1tlrHERERHSLYblJIiLSGN6doRZOIoiISGM4iVALv8WTiIiIbMJMBBERaYw9bsnU5oWVzEQQERGRTZiJICIijeE1EWpx3EmEl5ew2aKBV4ibjRYkYvTu4loT2dc9pTECUTf3VYeHi9tl95kDQFxc7ZNT96WJaxp0Hn6nNMbllJ9qPQ5LREWJ2yWHIQDAx0fcXigveYDGp8X7LGHWo/Igw1fI+6ghKEjYbHRxlYZw8vOr9TDyjl0Wti9L7i6N8XhU3bw3VSlHUSH+QPPzk+/3vCRxHYiIUxbU1kCUBX1IKxx3EkFERGQXzESohddEEBERkU2YiSAiIo1hJkItnEQQEZHG8Lsz1MLTGURERGQTZiKIiEhjWGxKLcxEEBERkU2YiSAiIo3hhZVqcdxJhKxCj5deHkNSnMWppEQaosxdvJ7ASnHBGwBI/TFY2kcNffuK27ftkCeekpPF7cOGyceRmipuz0qSF5LKaqaT9LhXPhALDHTZJmxf+1l/aQzZodq2rXwcsoJVWL1aHqRHD0mH2hd4AgBjh0hhu1Pmz/IgGRnidkuqfIWECJsfv/eqNESrDgHy9aggwkV8zB89Ji/A5iIp4nXjhnwcMX+TpNwtOVhlHxKkKY47iSAiIrILZiLUwmsiiIiIyCbMRBARkcYwE6EWTiKIiEhjOIlQC09nEBERkU2YiSAiIo1h2Wu1MBNBRERENtEpiqLU5QoNBgO8vb0BxANoUJerJiKielcOYCsKCgqg11tQ70dFf/z+eRDq//4pB/B5vWxXfWImgoiIiGzCayKIiEhjKqH+39C8O4OIiIjIYsxEEBGRxjAToRZOIoiISGM4iVALT2cQERGRTZiJICIijTFC/eJQ2iw2VeeTiD/KUlTU9aqJiKje/fbZX8cliqodg+PHdHx1Pom4cePG7//bWderJiIiB3Hjxo3fCz/VHVdXVwQFBSEra4dd4gcFBcHV1dUusR1VnVesNBqNuHz5Mho1agSdTleXqxYyGAwIDQ3FhQsXNFVtrDa4z6zHfWY97jPrOfI+UxQFN27cQHBwMJyc6v6yvJKSEpSVldkltqurK9zd3e0S21HVeSbCyckJISEhdb1ai+n1eod70zk67jPrcZ9Zj/vMeo66z+o6A/Fn7u7umvtFb0+8O4OIiIhswkkEERER2YSTiN+5ublh5syZcHNzq++h3DK4z6zHfWY97jPrcZ9RXanzCyuJiIjo9sBMBBEREdmEkwgiIiKyCScRREREZBNOIoiIiMgmnET8bvHixQgLC4O7uztiYmJw8ODB+h6Sw9q7dy8GDx6M4OBg6HQ6fPHFF/U9JIc3d+5c/O1vf0OjRo0QEBCAoUOH4vTp0/U9LIe2ZMkSREZGmgomxcbG4n//+199D+uWMW/ePOh0OkyePLm+h0K3MU4iAKxfvx5TpkzBzJkzceTIEXTq1Anx8fG4evVqfQ/NIRUVFaFTp05YvHhxfQ/llrFnzx6MHz8eBw4cwPbt21FeXo7+/fujqKiovofmsEJCQjBv3jykpaXh8OHDuPvuuzFkyBAcP368vofm8A4dOoQPPvgAkZGR9T0Uus3xFk8AMTEx+Nvf/ob33nsPwG/f7xEaGopnn30W06ZNq+fROTadTodNmzZh6NCh9T2UW8q1a9cQEBCAPXv2oHfv3vU9nFuGr68vFixYgDFjxtT3UBxWYWEhOnfujPfffx+vvfYaoqKikJSUVN/DotuU5jMRZWVlSEtLQ1xcnGmZk5MT4uLisH///nocGd3OCgoKAPz2S5HkKisrsW7dOhQVFSE2Nra+h+PQxo8fj0GDBpl9phHZS51/AZejycnJQWVlJQIDA82WBwYG4tSpU/U0KrqdGY1GTJ48GT169ECHDh3qezgO7ccff0RsbCxKSkrg5eWFTZs2ISIior6H5bDWrVuHI0eO4NChQ/U9FNIIzU8iiOra+PHjcezYMXz77bf1PRSH16ZNG6Snp6OgoACfffYZRo0ahT179nAiUY0LFy5g0qRJ2L59O7+lkuqM5icRfn5+cHZ2RnZ2ttny7OxsBAUF1dOo6HY1YcIEbNmyBXv37kVISEh9D8fhubq6Ijw8HAAQHR2NQ4cO4e2338YHH3xQzyNzPGlpabh69So6d+5sWlZZWYm9e/fivffeQ2lpKZydnetxhHQ70vw1Ea6uroiOjsbOnTtNy4xGI3bu3Mlzr6QaRVEwYcIEbNq0Cbt27ULLli3re0i3JKPRiNLS0voehkPq168ffvzxR6Snp5seXbp0wYgRI5Cens4JBNmF5jMRADBlyhSMGjUKXbp0QdeuXZGUlISioiIkJibW99AcUmFhITIyMkw/nzt3Dunp6fD19UXz5s3rcWSOa/z48Vi7di2+/PJLNGrUCFlZWQAAb29veHh41PPoHNP06dMxYMAANG/eHDdu3MDatWuRkpKCrVu31vfQHFKjRo2qXGPTsGFDNGnShNfekN1wEgFg2LBhuHbtGmbMmIGsrCxERUUhOTm5ysWW9JvDhw/j73//u+nnKVOmAABGjRqFFStW1NOoHNuSJUsAAH379jVbvnz5cowePbruB3QLuHr1KkaOHIkrV67A29sbkZGR2Lp1K+655576HhoR/Y51IoiIiMgmmr8mgoiIiGzDSQQRERHZhJMIIiIisgknEURERGQTTiKIiIjIJpxEEBERkU04iSAiIiKbcBJB9BfXr1/H7NmzcfXq1foeCtlAURS88847OHjwYH0Phei2x0kE0V80btwYbm5uGDFiBIxGo1lbSkoKdDod8vPz63xco0ePxtChQ21+ft++fTF58mTVxiOTmZkJnU6H9PT0GvvYY3/qdDpERUVh2LBhVb5Yj4jUxUmEho0ePRo6nQ46nQ4NGjRAYGAg7rnnHixbtqzKL0+11jdr1izTz2FhYUhKSlJ9PWqYOnUq7rzzTrPxqqmuf6EDwMaNGzFnzpw6XadM9+7dTWWt1dS7d2/MmzcPI0aMQGVlpaqxiegP/O4MjUtISMDy5ctRWVmJ7OxsJCcnY9KkSfjss8/w1VdfwcVFu4fI4sWL63sIqvL19a3vIVTh6uqKoKAgu8QeNmwYhg0bZpfYRPQbZiI0zs3NDUFBQWjWrBk6d+6Ml156CV9++SX+97//mX2Z1vnz5zFkyBB4eXlBr9fj4YcfNksVz5o1C1FRUfj0008RFhYGb29vDB8+HDdu3Kh2vX379sUvv/yCf/3rX6ZsSHWqS4nn5+dDp9MhJSUFwG/XMIwYMQL+/v7w8PDAHXfcgeXLl5v6X7hwAQ8//DB8fHzg6+uLIUOGIDMz0+Z9BgD79u1DZGQk3N3d0a1bNxw7dszUlpubi0ceeQTNmjWDp6cnOnbsiP/7v/8ztY8ePRp79uzB22+/bdr2m+M5fvw47r33Xuj1ejRq1Ai9evXC2bNnzda9cOFCNG3aFE2aNMH48eNRXl5u0Zj/mv349NNP0aVLFzRq1AhBQUF49NFHq1wHIhqP0WjE7NmzERISAjc3N9MX1/3VqVOn0L17d7i7u6NDhw7Ys2ePqe2vpzNk++7mdkycOBEvvvgifH19ERQUZLeMERGJcRJBVdx9993o1KkTNm7cCOC3XxZDhgxBXl4e9uzZg+3bt+Pnn3+u8lfe2bNn8cUXX2DLli3YsmUL9uzZg3nz5lW7jo0bNyIkJASzZ8/GlStXcOXKFZvH+8orr+DEiRP43//+h5MnT2LJkiXw8/MDAJSXlyM+Ph6NGjXCN998g3379sHLywsJCQkoKyuzeZ0vvPAC3nrrLRw6dAj+/v4YPHiw6Zd5SUkJoqOj8d///hfHjh3DuHHj8M9//tN0od/bb7+N2NhYjB071rTtoaGhuHTpEnr37g03Nzfs2rULaWlpePzxx1FRUWFa7+7du3H27Fns3r0bK1euxIoVK2z+5tTy8nLMmTMHP/zwA7744gtkZmaafaOobDxvv/023nrrLSxcuBBHjx5FfHw87rvvPpw5c6bKvnruuefw/fffIzY2FoMHD0Zubm61Y5Ltu5tWrlyJhg0bIjU1FW+++SZmz56N7du327QfiKgWFNKsUaNGKUOGDKm2bdiwYUq7du0URVGUbdu2Kc7Ozsr58+dN7cePH1cAKAcPHlQURVFmzpypeHp6KgaDwdTnhRdeUGJiYmpcf4sWLZR///vfwjGeO3dOAaB8//33pmXXr19XACi7d+9WFEVRBg8erCQmJlb7/E8//VRp06aNYjQaTctKS0sVDw8PZevWrcJ1V2f37t0KAGXdunWmZbm5uYqHh4eyfv36Gp83aNAg5bnnnjP93KdPH2XSpElmfaZPn660bNlSKSsrqzbGqFGjlBYtWigVFRWmZf/4xz+UYcOGWTT26tb5Z4cOHVIAKDdu3LBoPMHBwcrrr79utuxvf/ub8swzzyiK8sdrN2/ePFN7eXm5EhISosyfP19RlD/25/Xr12scV3X7rmfPnlXWO3Xq1BpjEJF9MBNB1VIUxXSK4eTJkwgNDUVoaKipPSIiAj4+Pjh58qRpWVhYGBo1amT6uWnTpnVym+TTTz+NdevWISoqCi+++CK+++47U9sPP/yAjIwMNGrUCF5eXvDy8oKvry9KSkqqnCawRmxsrOn/vr6+aNOmjWlfVFZWYs6cOejYsSN8fX3h5eWFrVu34vz588KY6enp6NWrFxo0aFBjn/bt28PZ2dn0c232cVpaGgYPHozmzZujUaNG6NOnDwCYxikaj8FgwOXLl9GjRw+z5T169DA7JgDzfeXi4oIuXbpU6XOTpfsuMjLS7Oe6OtaIyJx2r5ojoZMnT6Jly5ZWPeevv2x0Ol2t7/JwcvptnqsoimnZX68BGDBgAH755Rd8/fXX2L59O/r164fx48dj4cKFKCwsRHR0NNasWVMltr+/f63GVpMFCxbg7bffRlJSEjp27IiGDRti8uTJ0tMnHh4e0thq7eOioiLEx8cjPj4ea9asgb+/P86fP4/4+HjTOC0Zj9os3Xf2ONaIyHrMRFAVu3btwo8//ogHH3wQANCuXTtcuHABFy5cMPU5ceIE8vPzERERYfN6XF1dpbff3fxF/+drJqqrO+Dv749Ro0Zh9erVSEpKwocffggA6Ny5M86cOYOAgACEh4ebPWpzW+GBAwdM/79+/Tp++ukntGvXDsBvF10OGTIEjz32GDp16oRWrVrhp59+Mnt+ddseGRmJb775xuILJWvj1KlTyM3Nxbx589CrVy+0bdu2yl/yovHo9XoEBwdj3759Zsv37dtX5Zj4876qqKhAWlqaaV/9lSX7jogcBycRGldaWoqsrCxcunQJR44cwRtvvIEhQ4bg3nvvxciRIwEAcXFx6NixI0aMGIEjR47g4MGDGDlyJPr06YMuXbrYvO6wsDDs3bsXly5dQk5OTrV9PDw80K1bN8ybNw8nT57Enj178PLLL5v1mTFjBr788ktkZGTg+PHj2LJli+mX1IgRI+Dn54chQ4bgm2++wblz55CSkoKJEyfi4sWLNo999uzZ2LlzJ44dO4bRo0fDz8/PVAjqjjvuwPbt2/Hdd9/h5MmTePLJJ6sUPQoLC0NqaioyMzORk5MDo9GICRMmwGAwYPjw4Th8+DDOnDmDTz/9FKdPn7Z5nDVp3rw5XF1d8e677+Lnn3/GV199VaWGhGw8L7zwAubPn4/169fj9OnTmDZtGtLT0zFp0iSzOIsXL8amTZtw6tQpjB8/HtevX8fjjz9e7bgs2XdE5Dg4idC45ORkNG3aFGFhYUhISMDu3bvxzjvv4MsvvzSde9fpdPjyyy/RuHFj9O7dG3FxcWjVqhXWr19fq3XPnj0bmZmZaN26tfDUwrJly1BRUYHo6GhMnjwZr732mlm7q6srpk+fjsjISPTu3RvOzs5Yt24dAMDT0xN79+5F8+bN8cADD6Bdu3YYM2YMSkpKoNfrbR77vHnzMGnSJERHRyMrKwubN2+Gq6srAODll19G586dER8fj759+yIoKKhKpcnnn38ezs7OiIiIMJ1KaNKkCXbt2oXCwkL06dMH0dHR+Oijj4TXSNjK398fK1aswIYNGxAREYF58+Zh4cKFZn1k45k4cSKmTJmC5557Dh07dkRycjK++uor3HHHHVX21bx589CpUyd8++23+Oqrr0x3z/yVJfuOiByHTvnzyWYium3FxsaiX79+VSZhRES2YiaC6DZXWlqKw4cP4/jx42jfvn19D4eIbiO8O4PoNvDNN99gwIABNbY7OTnhvvvuw0MPPVSHoyKi2x1PZxDdBn799VdcunSpxvbw8PA6HA0RaQUnEURERGQTXhNBRERENuEkgoiIiGzCSQQRERHZhJMIIiIisgknEURERGQTTiKIiIjIJpxEEBERkU04iSAiIiKb/H+86Yn/m0i7IgAAAABJRU5ErkJggg==\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_as_patches(j)\n", "\n", "_ = plt.title('These slices are not diagonal')\n", "_ = plt.xlabel(\"Don't use `batch_jacobian`\")" ] }, { "cell_type": "markdown", "metadata": { "id": "M_x7ih5sarvG" }, "source": [ "이 경우 `batch_jacobian`가 여전히 실행되어 예상 형상을 가진 *어떤 것*을 반환하지만 그 내용은 명확하지 않습니다." ] }, { "cell_type": "code", "execution_count": 39, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T21:28:30.949634Z", "iopub.status.busy": "2022-12-14T21:28:30.949133Z", "iopub.status.idle": "2022-12-14T21:28:31.299464Z", "shell.execute_reply": "2022-12-14T21:28:31.298542Z" }, "id": "k8_mICHoasCi" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "jb.shape: (7, 6, 5)\n" ] } ], "source": [ "jb = tape.batch_jacobian(y, x)\n", "print(f'jb.shape: {jb.shape}')" ] } ], "metadata": { "colab": { "collapsed_sections": [], "name": "advanced_autodiff.ipynb", "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.16" } }, "nbformat": 4, "nbformat_minor": 0 }