{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "Tce3stUlHN0L" }, "source": [ "##### Copyright 2020 The TensorFlow Authors." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "cellView": "form", "execution": { "iopub.execute_input": "2022-12-14T20:58:18.457972Z", "iopub.status.busy": "2022-12-14T20:58:18.457383Z", "iopub.status.idle": "2022-12-14T20:58:18.461026Z", "shell.execute_reply": "2022-12-14T20:58:18.460496Z" }, "id": "tuOe1ymfHZPu" }, "outputs": [], "source": [ "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ] }, { "cell_type": "markdown", "metadata": { "id": "qFdPvlXBOdUN" }, "source": [ "# 高度な自動微分" ] }, { "cell_type": "markdown", "metadata": { "id": "MfBg1C5NB3X0" }, "source": [ "\n", " \n", " \n", " \n", " \n", "
TensorFlow.org で表示 Google Colab で実行\n", "GitHub でソースを表示ノートブックをダウンロード
" ] }, { "cell_type": "markdown", "metadata": { "id": "8a859404ce7e" }, "source": [ "[勾配と自動微分の基礎](autodiff.ipynb)には、TensorFlow での勾配の計算に必要なものがすべて含まれています。このガイドでは、`tf.GradientTape` API のさらに深い、あまり一般的ではない機能に焦点を当てていきます。" ] }, { "cell_type": "markdown", "metadata": { "id": "MUXex9ctTuDB" }, "source": [ "## セットアップ" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:18.464425Z", "iopub.status.busy": "2022-12-14T20:58:18.464214Z", "iopub.status.idle": "2022-12-14T20:58:20.719034Z", "shell.execute_reply": "2022-12-14T20:58:20.718348Z" }, "id": "IqR2PQG4ZaZ0" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2022-12-14 20:58:19.405350: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory\n", "2022-12-14 20:58:19.405454: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory\n", "2022-12-14 20:58:19.405464: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.\n" ] } ], "source": [ "import tensorflow as tf\n", "\n", "import matplotlib as mpl\n", "import matplotlib.pyplot as plt\n", "\n", "mpl.rcParams['figure.figsize'] = (8, 6)" ] }, { "cell_type": "markdown", "metadata": { "id": "uGRJJRi8TCkJ" }, "source": [ "## 勾配の記録を制御する\n", "\n", "[自動微分ガイド](autodiff.ipynb)では、勾配計算の作成中にテープが監視する対象の変数とテンソルを制御する方法について説明しました。\n", "\n", "テープには、記録を操作するメソッドもあります。" ] }, { "cell_type": "markdown", "metadata": { "id": "gB_i0VnhQKt2" }, "source": [ "### 記録を停止する\n", "\n", "勾配の記録を停止したい場合には、`tf.GradientTape.stop_recording`を使用して一時的に記録を停止することができます。\n", "\n", "これは、モデルの途中で複雑な演算を微分を行いたくない場合、オーバーヘッドの削減に有用です。これにはメトリックの計算や、中間結果の計算が含まれます。" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:20.723281Z", "iopub.status.busy": "2022-12-14T20:58:20.722636Z", "iopub.status.idle": "2022-12-14T20:58:24.066863Z", "shell.execute_reply": "2022-12-14T20:58:24.066068Z" }, "id": "mhFSYf7uQWxR" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "dz/dx: tf.Tensor(4.0, shape=(), dtype=float32)\n", "dz/dy: None\n" ] } ], "source": [ "x = tf.Variable(2.0)\n", "y = tf.Variable(3.0)\n", "\n", "with tf.GradientTape() as t:\n", " x_sq = x * x\n", " with t.stop_recording():\n", " y_sq = y * y\n", " z = x_sq + y_sq\n", "\n", "grad = t.gradient(z, {'x': x, 'y': y})\n", "\n", "print('dz/dx:', grad['x']) # 2*x => 4\n", "print('dz/dy:', grad['y'])" ] }, { "cell_type": "markdown", "metadata": { "id": "DEHbEZ1h4p8A" }, "source": [ "### 記録をリセットまたは最初から録画し直す\n", "\n", "最初からやり直す場合は、`tf.GradientTape.reset` を使用してください。通常は勾配テープブロックを終了して再起動した方が読み取りやすくなりますが、テープブロックの終了が困難または不可能な場合は、`reset` メソッドを使用できます。" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:24.070523Z", "iopub.status.busy": "2022-12-14T20:58:24.069982Z", "iopub.status.idle": "2022-12-14T20:58:24.076917Z", "shell.execute_reply": "2022-12-14T20:58:24.076296Z" }, "id": "lsMHsmrh4pqM" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "dz/dx: tf.Tensor(4.0, shape=(), dtype=float32)\n", "dz/dy: None\n" ] } ], "source": [ "x = tf.Variable(2.0)\n", "y = tf.Variable(3.0)\n", "reset = True\n", "\n", "with tf.GradientTape() as t:\n", " y_sq = y * y\n", " if reset:\n", " # Throw out all the tape recorded so far.\n", " t.reset()\n", " z = x * x + y_sq\n", "\n", "grad = t.gradient(z, {'x': x, 'y': y})\n", "\n", "print('dz/dx:', grad['x']) # 2*x => 4\n", "print('dz/dy:', grad['y'])" ] }, { "cell_type": "markdown", "metadata": { "id": "6zS7cLmS6zMf" }, "source": [ "## 勾配フローを正確に停止する\n", "\n", "上記のグローバルなテープ制御とは対照的に、`tf.stop_gradient`関数ははるかに正確です。これはテープ自体にアクセスする必要がなく、勾配が特定のパスに沿って流れることを防ぐために使用できます。" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:24.080179Z", "iopub.status.busy": "2022-12-14T20:58:24.079666Z", "iopub.status.idle": "2022-12-14T20:58:24.089620Z", "shell.execute_reply": "2022-12-14T20:58:24.089040Z" }, "id": "30qnZMe48BkB" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "dz/dx: tf.Tensor(4.0, shape=(), dtype=float32)\n", "dz/dy: None\n" ] } ], "source": [ "x = tf.Variable(2.0)\n", "y = tf.Variable(3.0)\n", "\n", "with tf.GradientTape() as t:\n", " y_sq = y**2\n", " z = x**2 + tf.stop_gradient(y_sq)\n", "\n", "grad = t.gradient(z, {'x': x, 'y': y})\n", "\n", "print('dz/dx:', grad['x']) # 2*x => 4\n", "print('dz/dy:', grad['y'])" ] }, { "cell_type": "markdown", "metadata": { "id": "mbb-9lnGVngH" }, "source": [ "## カスタム勾配\n", "\n", "場合によっては、デフォルトを使用しないで勾配の計算方法を正確に制御する必要があります。次のような状況がそれに該当します。\n", "\n", "1. 記述中の新しい演算に勾配が定義されていない。\n", "2. デフォルトの計算が数値的に不安定。\n", "3. フォワードパスから高価な計算をキャッシュしたい。\n", "4. 勾配を変更せずに値を変更したい（`tf.clip_by_value` や `tf.math.round` などを使用）。\n", "\n", "最初のケースでは、新しい演算子を記述するために、`tf.RegisterGradient` を使用して独自の勾配を設定できます（詳細は API ドキュメントを参照）。（勾配レジストリはグローバルなので、注意して変更してください。）\n", "\n", "後者の 3 つのケースには、`tf.custom_gradient`を使用できます。" ] }, { "cell_type": "markdown", "metadata": { "id": "oHr31kc_irF_" }, "source": [ "以下は、中間勾配に `tf.clip_by_norm` を適用した例です。" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:24.092823Z", "iopub.status.busy": "2022-12-14T20:58:24.092310Z", "iopub.status.idle": "2022-12-14T20:58:24.111196Z", "shell.execute_reply": "2022-12-14T20:58:24.110628Z" }, "id": "Mjj01w4NYtwd" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tf.Tensor(2.0, shape=(), dtype=float32)\n" ] } ], "source": [ "# Establish an identity operation, but clip during the gradient pass.\n", "@tf.custom_gradient\n", "def clip_gradients(y):\n", " def backward(dy):\n", " return tf.clip_by_norm(dy, 0.5)\n", " return y, backward\n", "\n", "v = tf.Variable(2.0)\n", "with tf.GradientTape() as t:\n", " output = clip_gradients(v * v)\n", "print(t.gradient(output, v)) # calls \"backward\", which clips 4 to 2" ] }, { "cell_type": "markdown", "metadata": { "id": "n4t7S0scYrD3" }, "source": [ "詳細については、`tf.custom_gradient` デコレータの API ドキュメントをご覧ください。" ] }, { "cell_type": "markdown", "metadata": { "id": "v0ODp4Oi--I0" }, "source": [ "### SavedModel のカスタム勾配\n", "\n", "注意: この機能は TensorFlow 2.6 以降で利用できます。\n", "\n", "カスタム勾配は、`tf.saved_model.SaveOptions(experimental_custom_gradients=True)` オプションを使用して、SavedModel に保存することができます。\n", "\n", "SavedModel に保存できるようにするには、勾配関数はトレース可能である必要があります（詳細は、「[tf.function によるパフォーマンスの改善](function.ipynb)」ガイドをご覧ください）。" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:24.114307Z", "iopub.status.busy": "2022-12-14T20:58:24.113800Z", "iopub.status.idle": "2022-12-14T20:58:24.117769Z", "shell.execute_reply": "2022-12-14T20:58:24.117192Z" }, "id": "Q5JBgIBYjN1I" }, "outputs": [], "source": [ "class MyModule(tf.Module):\n", "\n", " @tf.function(input_signature=[tf.TensorSpec(None)])\n", " def call_custom_grad(self, x):\n", " return clip_gradients(x)\n", "\n", "model = MyModule()" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:24.120687Z", "iopub.status.busy": "2022-12-14T20:58:24.120144Z", "iopub.status.idle": "2022-12-14T20:58:24.314048Z", "shell.execute_reply": "2022-12-14T20:58:24.313427Z" }, "id": "xZTrgy2q-9pq" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Assets written to: saved_model/assets\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "tf.Tensor(2.0, shape=(), dtype=float32)\n" ] } ], "source": [ "tf.saved_model.save(\n", " model,\n", " 'saved_model',\n", " options=tf.saved_model.SaveOptions(experimental_custom_gradients=True))\n", "\n", "# The loaded gradients will be the same as the above example.\n", "v = tf.Variable(2.0)\n", "loaded = tf.saved_model.load('saved_model')\n", "with tf.GradientTape() as t:\n", " output = loaded.call_custom_grad(v * v)\n", "print(t.gradient(output, v))" ] }, { "cell_type": "markdown", "metadata": { "id": "d-LfRs5FbJCk" }, "source": [ "上記の例に関する注意事項: 上記のコードを `tf.saved_model.SaveOptions(experimental_custom_gradients=False)` に置き換えても、勾配は読み込み時に同じ結果を生成します。これは、勾配のレジストリに `call_custom_op` 関数で使用されたカスタム勾配がまだ含まれているためです。ただし、カスタム勾配なしで保存した後にランタイムを再起動すると、読み込まれたモデルを `tf.GradientTape` で実行したときに `LookupError: No gradient defined for operation 'IdentityN' (op type: IdentityN)` というエラーがスローされます。" ] }, { "cell_type": "markdown", "metadata": { "id": "8aENEt6Veryb" }, "source": [ "## 複数のテープ\n", "\n", "複数のテープはシームレスに連携します。\n", "\n", "たとえば、ここでは各テープが異なるテンソルのセットをウォッチしています。" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:24.317476Z", "iopub.status.busy": "2022-12-14T20:58:24.317215Z", "iopub.status.idle": "2022-12-14T20:58:24.324735Z", "shell.execute_reply": "2022-12-14T20:58:24.324180Z" }, "id": "BJ0HdMvte0VZ" }, "outputs": [], "source": [ "x0 = tf.constant(0.0)\n", "x1 = tf.constant(0.0)\n", "\n", "with tf.GradientTape() as tape0, tf.GradientTape() as tape1:\n", " tape0.watch(x0)\n", " tape1.watch(x1)\n", "\n", " y0 = tf.math.sin(x0)\n", " y1 = tf.nn.sigmoid(x1)\n", "\n", " y = y0 + y1\n", "\n", " ys = tf.reduce_sum(y)" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:24.328079Z", "iopub.status.busy": "2022-12-14T20:58:24.327475Z", "iopub.status.idle": "2022-12-14T20:58:24.337318Z", "shell.execute_reply": "2022-12-14T20:58:24.336718Z" }, "id": "6ApAoMNFfNz6" }, "outputs": [ { "data": { "text/plain": [ "1.0" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tape0.gradient(ys, x0).numpy() # cos(x) => 1.0" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:24.340342Z", "iopub.status.busy": "2022-12-14T20:58:24.339883Z", "iopub.status.idle": "2022-12-14T20:58:24.345538Z", "shell.execute_reply": "2022-12-14T20:58:24.344995Z" }, "id": "rF1jrAJsfYW_" }, "outputs": [ { "data": { "text/plain": [ "0.25" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "tape1.gradient(ys, x1).numpy() # sigmoid(x1)*(1-sigmoid(x1)) => 0.25" ] }, { "cell_type": "markdown", "metadata": { "id": "DK05KXrAAld3" }, "source": [ "### 高次勾配\n", "\n", "`tf.GradientTape` コンテキストマネージャー内の演算は、自動微分のために記録されます。そのコンテキストで勾配が計算される場合は、勾配の計算も記録されます。その結果、まったく同じ API が高次勾配にも機能します。\n", "\n", "以下に例を示します。" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:24.348944Z", "iopub.status.busy": "2022-12-14T20:58:24.348484Z", "iopub.status.idle": "2022-12-14T20:58:24.356727Z", "shell.execute_reply": "2022-12-14T20:58:24.356151Z" }, "id": "cPQgthZ7ugRJ" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "dy_dx: 3.0\n", "d2y_dx2: 6.0\n" ] } ], "source": [ "x = tf.Variable(1.0) # Create a Tensorflow variable initialized to 1.0\n", "\n", "with tf.GradientTape() as t2:\n", " with tf.GradientTape() as t1:\n", " y = x * x * x\n", "\n", " # Compute the gradient inside the outer `t2` context manager\n", " # which means the gradient computation is differentiable as well.\n", " dy_dx = t1.gradient(y, x)\n", "d2y_dx2 = t2.gradient(dy_dx, x)\n", "\n", "print('dy_dx:', dy_dx.numpy()) # 3 * x**2 => 3.0\n", "print('d2y_dx2:', d2y_dx2.numpy()) # 6 * x => 6.0" ] }, { "cell_type": "markdown", "metadata": { "id": "k0HV-Ah4_76i" }, "source": [ "これにより、*スカラー*関数の 2 次導関数が得られますが、`tf.GradientTape.gradient` はスカラーの勾配のみを計算するため、このパターンはヘッセ行列を生成するための一般化を行いません。[ヘッセ行列](https://en.wikipedia.org/wiki/Hessian_matrix)を作成するには、[ヘッセ行列の例](#hessian)（[ヤコビアンのセクション](#jacobians)）をご覧ください。\n", "\n", "「`tf.GradientTape.gradient`のネストされた呼び出し」は、勾配からスカラーを計算する場合に適したパターンです。次の例のように、結果のスカラーは 2 番目の勾配計算のソースとして機能します。\n" ] }, { "cell_type": "markdown", "metadata": { "id": "t7LRlcpVKHv1" }, "source": [ "#### 例: 入力勾配の正則化\n", "\n", "多くのモデルは「敵対的な例」の影響を受けやすいものです。そのような手法の数々は、モデルの入力を変更してモデルの出力を混乱させます。[Fast Gradient Signed Method 攻撃を使った敵対的な例](https://www.tensorflow.org/tutorials/generative/adversarial_fgsm)といった最も単純な実装は、入力に対する出力の勾配に沿って 1 つのステップを踏む「入力勾配」です。\n", "\n", "敵対的な例に対する堅牢性を高める 1 つの手法は、入力勾配のマグニチュードの最小化を試みる、[入力勾配の正則化](https://arxiv.org/abs/1905.11468)（Finlay & Oberman、2019 年）です。入力勾配が小さければ、出力の変化も小さくなるはずです。\n", "\n", "下記は、入力勾配の正則化の単純な実装です。実装は次のとおりです。\n", "\n", "1. 内部テープを使用して、入力に対する出力の勾配を計算する。\n", "2. その入力勾配の大きさを計算する。\n", "3. モデルに対する大きさの勾配を計算する。" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:24.359999Z", "iopub.status.busy": "2022-12-14T20:58:24.359540Z", "iopub.status.idle": "2022-12-14T20:58:24.368054Z", "shell.execute_reply": "2022-12-14T20:58:24.367429Z" }, "id": "tH3ZFuUfDLrR" }, "outputs": [], "source": [ "x = tf.random.normal([7, 5])\n", "\n", "layer = tf.keras.layers.Dense(10, activation=tf.nn.relu)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:24.371383Z", "iopub.status.busy": "2022-12-14T20:58:24.370821Z", "iopub.status.idle": "2022-12-14T20:58:24.710150Z", "shell.execute_reply": "2022-12-14T20:58:24.709452Z" }, "id": "E6yOFsjEDR9u" }, "outputs": [], "source": [ "with tf.GradientTape() as t2:\n", " # The inner tape only takes the gradient with respect to the input,\n", " # not the variables.\n", " with tf.GradientTape(watch_accessed_variables=False) as t1:\n", " t1.watch(x)\n", " y = layer(x)\n", " out = tf.reduce_sum(layer(x)**2)\n", " # 1. Calculate the input gradient.\n", " g1 = t1.gradient(out, x)\n", " # 2. Calculate the magnitude of the input gradient.\n", " g1_mag = tf.norm(g1)\n", "\n", "# 3. Calculate the gradient of the magnitude with respect to the model.\n", "dg1_mag = t2.gradient(g1_mag, layer.trainable_variables)" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:24.714309Z", "iopub.status.busy": "2022-12-14T20:58:24.713781Z", "iopub.status.idle": "2022-12-14T20:58:24.718352Z", "shell.execute_reply": "2022-12-14T20:58:24.717782Z" }, "id": "123QMq6PqK_d" }, "outputs": [ { "data": { "text/plain": [ "[TensorShape([5, 10]), TensorShape([10])]" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "[var.shape for var in dg1_mag]" ] }, { "cell_type": "markdown", "metadata": { "id": "E4xiYigexMtQ" }, "source": [ "## ヤコビアン\n" ] }, { "cell_type": "markdown", "metadata": { "id": "4-hVHVIeExkI" }, "source": [ "前述したすべての例では、複数または単一のソーステンソルに対し、単一のスカラーターゲットの勾配を取得しました。\n", "\n", "[ヤコビ行列](https://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant)は、ベクトル値関数の勾配を表します。各行には、それぞれベクトルの一要素の勾配が示されます。\n", "\n", "`tf.GradientTape.jacobian`メソッドを使用すると、ヤコビ行列を効率的に計算することができます。" ] }, { "cell_type": "markdown", "metadata": { "id": "KzNyIM0QBYIH" }, "source": [ "注意点:\n", "\n", "- `gradient`と同様、`sources`引数は単一のテンソル、またはテンソルのコンテナである可能性があります。\n", "- `gradient`と異なり、`target`テンソルは単一のテンソルでなければなりません。" ] }, { "cell_type": "markdown", "metadata": { "id": "O74K3hlxBC8a" }, "source": [ "### スカラーのソース" ] }, { "cell_type": "markdown", "metadata": { "id": "B08OKn1Orkuc" }, "source": [ "最初の例は、ここにスカラーソースに対するベクトルターゲットのヤコビアンです。" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:24.721968Z", "iopub.status.busy": "2022-12-14T20:58:24.721503Z", "iopub.status.idle": "2022-12-14T20:58:24.878659Z", "shell.execute_reply": "2022-12-14T20:58:24.877998Z" }, "id": "bAFeIE8EuVIq" }, "outputs": [], "source": [ "x = tf.linspace(-10.0, 10.0, 200+1)\n", "delta = tf.Variable(0.0)\n", "\n", "with tf.GradientTape() as tape:\n", " y = tf.nn.sigmoid(x+delta)\n", "\n", "dy_dx = tape.jacobian(y, delta)" ] }, { "cell_type": "markdown", "metadata": { "id": "BgHbUk3zr-WU" }, "source": [ "スカラーに対してヤコビアンをとると、結果は**ターゲット**の形状を持ち、ソースに対する各要素の勾配を表します。" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:24.882663Z", "iopub.status.busy": "2022-12-14T20:58:24.882013Z", "iopub.status.idle": "2022-12-14T20:58:24.885714Z", "shell.execute_reply": "2022-12-14T20:58:24.885139Z" }, "id": "iZ6awnDzr_BA" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(201,)\n", "(201,)\n" ] } ], "source": [ "print(y.shape)\n", "print(dy_dx.shape)" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:24.889119Z", "iopub.status.busy": "2022-12-14T20:58:24.888613Z", "iopub.status.idle": "2022-12-14T20:58:25.058855Z", "shell.execute_reply": "2022-12-14T20:58:25.058242Z" }, "id": "siNZaklc0_-e" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAp8AAAINCAYAAAB4RhRAAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABdXElEQVR4nO3dd3hUZf7+8XtmMpNegJAChN6lSonBgiWK6KKsZbH8BMEua1lWV7Gxul/FtsquDXUVdV0V3RUbCGIUFEFQigLSCS2QkFDSk0lmzu+PSQKBBBJI5kx5v65rrpk55zlnPjOOw53znPM8FsMwDAEAAABeYDW7AAAAAAQPwicAAAC8hvAJAAAAryF8AgAAwGsInwAAAPAawicAAAC8hvAJAAAAryF8AgAAwGtCzC6gIdxut3bv3q3o6GhZLBazywEAAMARDMNQYWGh2rRpI6u1/uObfhE+d+/erZSUFLPLAAAAwHHs3LlT7dq1q3e9X4TP6OhoSZ43ExMTY3I1AAAAOFJBQYFSUlJqclt9/CJ8Vne1x8TEED4BAAB82PFOkeSCIwAAAHgN4RMAAABeQ/gEAACA1/jFOZ8N4XK5VFFRYXYZAcdmsykkJIQhrgAAQJMIiPBZVFSkXbt2yTAMs0sJSBEREUpOTpbD4TC7FAAA4Of8Pny6XC7t2rVLERERat26NUfompBhGHI6ncrNzVVmZqa6det2zEFjAQAAjsfvw2dFRYUMw1Dr1q0VHh5udjkBJzw8XHa7Xdu3b5fT6VRYWJjZJQEAAD8WMIexOOLZfDjaCQAAmgqpAgAAAF5D+AQAAIDXED4BAADgNYRPAAAAeA3h0yTvvPOOWrVqpfLy8lrLR48ereuuu86kqgAAAJpXo8Pnd999p1GjRqlNmzayWCz65JNPjrvNggULdOqppyo0NFRdu3bVW2+9dQKlNoxhGCpxVppya8wg91deeaVcLpc+++yzmmV79+7V7NmzNWHChOb4aAAAAEzX6HE+i4uL1b9/f02YMEGXXXbZcdtnZmbq4osv1q233qr//Oc/ysjI0I033qjk5GSNGDHihIo+ltIKl3o/Mq/J99sQvz02QhGOhn2k4eHhuuaaazRjxgxdeeWVkqR3331X7du319lnn92MVQIAAJin0eFz5MiRGjlyZIPbT58+XZ06ddLf//53SVKvXr20aNEiPf/8880SPv3JTTfdpCFDhigrK0tt27bVW2+9peuvv54xSwEAQMBq9hmOlixZovT09FrLRowYobvvvrtZXi/cbtNvj5kTasPttka1HzhwoPr376933nlHF1xwgdauXavZs2c3U3UAAPgXwzDkchtyVd1Xug0Zhme5YUhuw5DbkAxVL/csMyS53Z5T4dxHtJWqtjFqr6tu667af11tPTUdVp+M6geH39XZxjiiTfX7q7XsWNvVWlf39rXP/vM8SWkZoVPaxMqXNHv4zM7OVmJiYq1liYmJKigoUGlpaZ1TYpaXl9e6EKegoKDBr2exWBrc9e0LbrzxRk2bNk1ZWVlKT09XSkqK2SUBAIKEYRgqr3SrrMKlEqdLpRUulR52X+J0qazC87y8wiWny60Kl2cbZ9WtwlX1+Mj7qnUVbkPuquDocrs9YbLmuVHrufvw5VWBEyfnutM66G+jgyx8noipU6fq0UcfNbsMr7jmmmt0zz336PXXX9c777xjdjkAAD9T6nQpr6hc+4udOlDiVEFZpQpKK1RYVqmCsgoVllWooLTSc3/YusKyCpVWuBQI+c5q8Rx8slokiyyyWCSLRbJaLLKo6v7wNlX3UvVzTxtr1WlvVqtnP9VtLRbJokNTeR9+clz1mXKWqqV1nTl35HaHtznW9pYjGtX9unW/RnWbDq0iji7IZM0ePpOSkpSTk1NrWU5OjmJiYuo86ilJkydP1qRJk2qeFxQUBOwRwdjYWF1++eWaPXu2Ro8ebXY5AAAfYBiGCsoqtSe/VLsPlmr3wTLtyS9VXqFT+4rLlVfkud9X5FSJ09Ukr2m3WRRutyncYVOEI0RhdpsiHDaF220Ks9sUarfKYau6hVhlr7p3hFjlsFmq7q1yhNhkP+x5iM2qEKtFtsNuRz+3HrW8rjZWa92BEv6l2cNnWlqa5syZU2vZ/PnzlZaWVu82oaGhCg0Nbe7SfEZWVpauvfbaoHrPABDsyipc2r6vRJl5RcrMK9GO/SXafbC0KnCWqai8ssH7coRYFR/pUFyEQzHhIYoJsys6zK6Y8BDPfZhn2eHrosJCFOmwKawqYNptDP0N72h0+CwqKtLmzZtrnmdmZmrVqlVq2bKl2rdvr8mTJysrK6umC/nWW2/Viy++qL/85S+aMGGCvvnmG3344YdcWCPpwIEDWrBggRYsWKCXX37Z7HIAAM2grMKlTTlFWpddoPV7CrUxp1CZecXanV+q4w0P3SLCruTYcLWJC1ebuDC1jgpVq6hQtYpyKD7KoVaRnsdRoSEcAYTfaHT4/Pnnn3XOOefUPK/uHh83bpzeeust7dmzRzt27KhZ36lTJ82ePVt/+tOf9I9//EPt2rXTv/71r6AfZknyXO1+4MABPfXUU+rRo4fZ5QAATlJ5pUtrdxdo5Y6DWrXzoH7bna/MvOJ6z6uMDgtR5/hIdYqPVPtWkWobF1YVNMOVHBvmVxfQAg1lMRozLY9JCgoKFBsbq/z8fMXExNRaV1ZWpszMTHXq1ElhYWEmVRjY+IwBoG4Hip1asnWflmXu18qqsFnhOvqf1RYRdvVKjlGPpGj1TIpWl9ZR6hQfqZaRDo5YImAcK68djj+pAABooFKnSz9t268ftuTph815Wru74Kiu81aRDg1sH6eB7VuoT9tY9UqKVuvoUEImUIXwCQDAMewvdurrdTmatyZb32/Ok7PSXWt998QoDesSr4Ht43Rq+xZq1yKcoAkcA+ETAIAj5BSUad7abM1dk62lmftrDXaeHBum07vG64yu8RrWpZUSYjgdCWgMwicAAJKclW59sz5HM3/aqYUbc2tdJNQ7OUYjTknSiD6J6pEYzZFN4CQQPgEAQW3z3kLN/GmnPl6RpX3Fzprlp7aP08g+yRpxSpLa++AsMYC/InwCAIKOYRj6flOepi/cosVb9tUsbx0dqisGtdOVg9qpc+soEysEAhfh08ecffbZGjBggKZNm9Zk+7RYLJo1axbTdwIIepUut75ck63pC7do7e4CSZLNatG5PRM0ZnCKzu7RWiHM9AM0K8JnAHj00Ue1adMmvfvuu2aXAgA+yVnp1oc/79Rr323Vjv0lkqRwu01XDU3RjWd2Vtu4cJMrBIIH4TMAfPrpp7r//vvNLgMAfI7bbejzX3fr719trAmdLSLsGjeso8aldVSLSIfJFQLBh74FExUXF2vs2LGKiopScnKy/v73v9ese+yxx9SnT5+jthkwYIAefvjhmuc7d+7U2rVrdeGFF0qSNm3apLPOOkthYWHq3bu35s+fX2v7d955R1FRUdq0aVPNsttvv109e/ZUSUlJU79FADDNT9v2a9SLi3TXB6u0Y3+J4qNC9cjveuuH+8/V3endCZ6ASQLvyKdhSBUmhSh7hNSI4TfuvfdeLVy4UJ9++qkSEhL0wAMPaMWKFRowYIAmTJigRx99VD/99JOGDBkiSVq5cqV+/fVXffzxxzX7+Oyzz3T22WcrJiZGbrdbl112mRITE7V06VLl5+fr7rvvrvWaY8eO1RdffKFrr71Wixcv1rx58/Svf/1LS5YsUUQEV3MC8H/Z+WWa+uU6fbpqtyQpOjREtwzvrAlndGKudMAHBN7/hRUl0hNtzHntB3ZLjsgGNS0qKtIbb7yhd999V+edd54k6e2331a7du0kSe3atdOIESM0Y8aMmvA5Y8YMDR8+XJ07d67Zz6effqpLL71UkvT1119r/fr1mjdvntq08XwGTzzxhEaOHFnrtV999VX169dPd955pz7++GP99a9/1aBBg07uvQOAydxuQ/9Zul1Pzd2govJKWSzSVUPa654LuqtVVKjZ5QGoQre7SbZs2SKn06nU1NSaZS1btlSPHj1qnt900016//33VVZWJqfTqffee08TJkyoWV9QUKCFCxfqkksukSStW7dOKSkpNcFTktLS0o567RYtWuiNN97QK6+8oi5dunC+KAC/tyW3SH94dYke/nStisorNbB9nD7/4xmaellfgifgYwLvyKc9wnME0qzXbkKjRo1SaGioZs2aJYfDoYqKCl1xxRU167/88kv17t1bKSkpjd73d999J5vNpj179qi4uFjR0dFNWToAeIVhGHp36Q49Pvs3lVW4FeGw6S8jeui6tI6yWZmFCPBFgRc+LZYGd32bqUuXLrLb7Vq6dKnat28vSTpw4IA2btyo4cOHS5JCQkI0btw4zZgxQw6HQ1dddZXCww8NB3J4l7sk9erVSzt37tSePXuUnJwsSfrxxx+Peu3Fixfrqaee0ueff6777rtPf/zjH/X2228359sFgCaXV1Suv/z3V32zfq8k6Yyu8Xry8r5q14Lz1wFfFnjh009ERUXphhtu0L333qtWrVopISFBDz74oKzW2mdC3HjjjerVq5ck6YcffqhZXllZqS+//FL33HNPzbL09HR1795d48aN0zPPPKOCggI9+OCDtfZXWFio6667TnfeeadGjhypdu3aaciQIRo1alSto6oA4Mt+3rZfE99boZyCcjlCrLrvwp4aP6yjrBztBHwe4dNEzzzzjIqKijRq1ChFR0frz3/+s/Lz82u16datm4YNG6b9+/fXOj904cKFioqK0qmnnlqzzGq1atasWbrhhhs0dOhQdezYUf/85z9rhmGSpLvuukuRkZF64oknJEl9+/bVE088oVtuuUVpaWlq27ZtM79rADhxhmHozR+2aeqcdap0G+rSOlIvXXuqeibFmF0agAayGIZhmF3E8RQUFCg2Nlb5+fmKian9A1NWVqbMzEx16tRJYWFhJlXYfAzDULdu3XT77bdr0qRJNcvvvPNOVVZW6uWXX272GgL9MwbgH8orXZr8v9X6eGWWJOl3/ZL15OX9FBXKcRTAFxwrrx2O/2N9WG5urj744ANlZ2dr/Pjxtdb16dOnzivZASAQ7Ssq1y3/Xq6ftx+QzWrRQxf30vXDOsrSiLGVAfgGwqcPS0hIUHx8vF577TW1aNGi1rqbb77ZpKoAwLu25hZp3Ixl2rm/VNFhIXr52lN1ZrfWZpcF4AQRPn2YH5wRAQDNak1Wvsa9uUz7ip1q3zJCb14/WF0TGBoO8GeETwCAT1qyZZ9ueudnFZVXqk/bGL01fqjiGTAe8HuETwCAz/luY65ueudnlVe6dVrnlnp97GBFh9nNLgtAEwiY8EkXdfPhswXgTd9vOhQ803sl6MVrTlWY3WZ2WQCaiN/P7W6zeX6QnE6nyZUErpKSEkmS3c5RBwDNa9GmPN34dnXwTNTL1w4ieAIBxu+PfIaEhCgiIkK5ubmy2+1HzRCEE2cYhkpKSrR3717FxcXVBH0AaA7Ltx847Ihnol6+9lQ5QvhNBwKN34dPi8Wi5ORkZWZmavv27WaXE5Di4uKUlJRkdhkAAtjGnEJNeOsnlVa4NLx7a4InEMD8PnxKksPhULdu3eh6bwZ2u50jngCa1a4DJRr7xjLll1bo1PZxeuX/ETyBQBYQ4VPyzGvO1I8A4F8Kyio0fsZPyi4oU/fEKL15/RBFOALmnyYAdeBPSwCAKSpdbv3xvZXatLdIiTGhenvCUMVFOMwuC0AzI3wCAEzxty9+03cbcxVut+mNcUOUHBtudkkAvIDwCQDwund/3K63l2yXxSJNu2qA+rSNNbskAF5C+AQAeNXKHQf06OdrJUn3juihEacwmgYQTAifAACv2VdUrtv/s0IVLkMj+yTptuFdzC4JgJcRPgEAXuFyG7rzg5Xak1+mzvGRevqKfrJYLGaXBcDLCJ8AAK948ZvN+mHzPoXbbZp+3SBFhzFlLxCMCJ8AgGa3fPt+/fObTZKkx3/fR90To02uCIBZCJ8AgGZVUFahuz5YJZfb0OgBbXTZqe3MLgmAiQifAIBm9cgna7TrQKnatQjXY6P7mF0OAJMRPgEAzWb2r3v0yardslkt+sdVAxXDeZ5A0CN8AgCaxb6icj3y6RpJ0u1nd9GgDi1MrgiALyB8AgCaxZTP1mpfsVM9k6J1x7ndzC4HgI8gfAIAmtyXq/foi1/3yGa16Jkr+ssRwj83ADz4NQAANKn8kgo9XNXdfuvwzurbjnnbARxC+AQANKmn561XXpFTXVpH6s7z6G4HUBvhEwDQZFbuOKD3lu2QJP3f6L4KDbGZXBEAX0P4BAA0iUqXWw/OWiPDkC4b2FZpXVqZXRIAH0T4BAA0iXeWbNdvewoUExaiBy7uZXY5AHwU4RMAcNL2FZXr+fkbJUn3jeyp+KhQkysC4KsInwCAk/bc/I0qLK/UKW1idNWQ9maXA8CHET4BACdlfXaB3q+6yOiR3/WWzWoxuSIAvozwCQA4YYZh6G9f/Ca3IV3UN0mpnbnICMCxET4BACcsY91e/bB5nxw2qyaP5CIjAMdH+AQAnJBKl1tPzl0vSZpwRieltIwwuSIA/oDwCQA4IR+vzNLmvUWKi7Dr9nO6mF0OAD9B+AQANFpZhUvTqoZWuv3sLooJs5tcEQB/QfgEADTauz9u1+78MiXHhmlsWkezywHgRwifAIBGKSir0EvfbpYk3Z3eTWF25m8H0HCETwBAo8xYtE0HSirUuXWkLj+1ndnlAPAzhE8AQIMVlFXojUVbJUl/Su+uEBv/jABoHH41AAAN9vYP21RQVqmuCVG6qG+y2eUA8EOETwBAgxSWVehfizIlSXec25VpNAGcEMInAKBB3lmyXfmlFerSOlK/69fG7HIA+CnCJwDguIrKK/X6955zPe84txtHPQGcMMInAOC43l+6QwdLKtQ5PlKj+nPUE8CJI3wCAI7JWenWG1Xnet4yvDNHPQGcFMInAOCYPl2VpeyCMiXGhGr0wLZmlwPAzxE+AQD1crsNvfqd51zPCad3UmgIsxkBODmETwBAvTLW79XmvUWKDg3RNantzS4HQAAgfAIA6jV94RZJ0rWndVB0mN3kagAEAsInAKBOK3Yc0PLtB+SwWTXh9I5mlwMgQBA+AQB1mvHDNknSJQPaKCEmzNxiAAQMwicA4Ch78ks1Z/UeSdJ4jnoCaEInFD5feukldezYUWFhYUpNTdWyZcuO2X7atGnq0aOHwsPDlZKSoj/96U8qKys7oYIBAM3v30u2y+U2lNqppU5pE2t2OQACSKPD58yZMzVp0iRNmTJFK1asUP/+/TVixAjt3bu3zvbvvfee7r//fk2ZMkXr1q3TG2+8oZkzZ+qBBx446eIBAE2v1OnSe8t2SJImnNHJ5GoABJpGh8/nnntON910k8aPH6/evXtr+vTpioiI0Jtvvlln+8WLF+v000/XNddco44dO+qCCy7Q1VdffdyjpQAAc8xamaWDJRVKaRmu9F6JZpcDIMA0Knw6nU4tX75c6enph3ZgtSo9PV1Lliypc5thw4Zp+fLlNWFz69atmjNnji666KKTKBsA0BwMw9Bbiz1TaY5L68hUmgCaXEhjGufl5cnlcikxsfZfwomJiVq/fn2d21xzzTXKy8vTGWecIcMwVFlZqVtvvfWY3e7l5eUqLy+veV5QUNCYMgEAJ2hZ5n5tzClSuN2mPwxJMbscAAGo2a92X7BggZ544gm9/PLLWrFihT7++GPNnj1bf/vb3+rdZurUqYqNja25paTwAwgA3vDuUs+5nqMHtlEMg8oDaAaNOvIZHx8vm82mnJycWstzcnKUlJRU5zYPP/ywrrvuOt14442SpL59+6q4uFg333yzHnzwQVmtR+ffyZMna9KkSTXPCwoKCKAA0MxyC8s1d41neKVrUzuYXA2AQNWoI58Oh0ODBg1SRkZGzTK3262MjAylpaXVuU1JSclRAdNms0nynFtUl9DQUMXExNS6AQCa14c/71SFy9CAlDj1acvwSgCaR6OOfErSpEmTNG7cOA0ePFhDhw7VtGnTVFxcrPHjx0uSxo4dq7Zt22rq1KmSpFGjRum5557TwIEDlZqaqs2bN+vhhx/WqFGjakIoAMBcLreh96q63P/faRz1BNB8Gh0+x4wZo9zcXD3yyCPKzs7WgAEDNHfu3JqLkHbs2FHrSOdDDz0ki8Wihx56SFlZWWrdurVGjRqlxx9/vOneBQDgpCzYsFdZB0sVG27X7/olm10OgABmMerr+/YhBQUFio2NVX5+Pl3wANAMbnjrJ2Ws36ubzuykBy/ubXY5APxQQ/Mac7sDQJDLzi/Ttxs8s9RdNbS9ydUACHSETwAIcv9bsUtuQxrSsYW6tI4yuxwAAY7wCQBBzO029OHPOyVJfxjMkHYAmh/hEwCC2NLM/dq+r0RRoSG6mAuNAHgB4RMAglj1Uc9R/ZMV4Wj0ACgA0GiETwAIUvmlFZqz2jOj0ZghXGgEwDsInwAQpD77ZbfKK93qkRit/u2Y0QiAdxA+ASBI/W/5LknSlYPbyWKxmFwNgGBB+ASAILQlt0irdh6UzWrRJQPamF0OgCBC+ASAIDRrRZYk6axu8UqIDjO5GgDBhPAJAEHG7TY0a6UnfF52ajuTqwEQbAifABBklmbuV9bBUkWHhej83olmlwMgyBA+ASDIfLzCc6HR7/olK8xuM7kaAMGG8AkAQaTU6aoZ25MudwBmIHwCQBD56rdsFTtdSmkZrsEdWphdDoAgRPgEgCDy+S+7JUmjB7RlbE8ApiB8AkCQOFji1MKNuZKkSxnbE4BJCJ8AECS+XJOtCpehXskx6poQbXY5AIIU4RMAgsSnqzxje17Sn6OeAMxD+ASAIJCdX6almfslSaP6J5tcDYBgRvgEgCDwxa+7ZRjS4A4t1K5FhNnlAAhihE8ACAKfVV3lzoVGAMxG+ASAALctr1i/7sqXzWrRRX3pcgdgLsInAAS4L9dkS5LSOrdSq6hQk6sBEOwInwAQ4L5c45lOc2TfJJMrAQDCJwAEtF0HSvTrrnxZLdIFvQmfAMxH+ASAADa3qst9aKeWah1NlzsA8xE+ASCAzVld1eXehwuNAPgGwicABKjs/DKt2HFQknRhH7rcAfgGwicABKi5VRcaDerQQokxYSZXAwAehE8ACFDVQyyN5KgnAB9C+ASAAJRbWK5l2zxzuY9kYHkAPoTwCQABaN7abBmG1L9drNrGhZtdDgDUIHwCQACqHmKJo54AfA3hEwACzP5ip5Zs3SeJ8z0B+B7CJwAEmPm/ZcvlNtQ7OUYdWkWaXQ4A1EL4BIAAU32V+0XM5Q7ABxE+ASCA5JdU6IfNeZKkC5nVCIAPInwCQAD5el2OKlyGuidGqWtClNnlAMBRCJ8AEEAODSzPUU8AvonwCQABotTp0qLNuZKkEadwvicA30T4BIAA8cPmPJVVuNU2Lly9kqPNLgcA6kT4BIAAkbE+R5KU3itBFovF5GoAoG6ETwAIAG63oa/X7ZUkndcr0eRqAKB+hE8ACACrs/KVW1iuqNAQpXZuaXY5AFAvwicABICv13m63M/qHq/QEJvJ1QBA/QifABAAqrvc0+lyB+DjCJ8A4Od2HSjRuj0Fslqkc3okmF0OABwT4RMA/Nw36z1HPQd3aKkWkQ6TqwGAYyN8AoCfm/+b53zP83px1BOA7yN8AoAfKyyr0I9b90mS0ntzvicA30f4BAA/9v2mPFW4DHWKj1SX1lFmlwMAx0X4BAA/Vj3EUjpd7gD8BOETAPyUy23o2/UMsQTAvxA+AcBPrdhxQAdKKhQbbtegDi3MLgcAGoTwCQB+6uuqq9zP7ZmgEBs/5wD8A79WAOCnqs/3ZIglAP6E8AkAfigzr1hbcotlt1l0VvfWZpcDAA1G+AQAP5RRddQztVMrxYTZTa4GABqO8AkAfmjBhlxJ0jk96XIH4F8InwDgZ4rLK7Usc78k6ewedLkD8C+ETwDwM0u27JPT5VZKy3B1jo80uxwAaBTCJwD4mQUbPQPLn909QRaLxeRqAKBxCJ8A4EcMw6g535MudwD+iPAJAH5kS26xdh0olcNmVVqXVmaXAwCNRvgEAD+yYIOnyz21c0tFOEJMrgYAGo/wCQB+pLrLfTgDywPwU4RPAPATtYdYYnxPAP6J8AkAfqJ6iKV2LcLVpTVDLAHwT4RPAPAT1UMsndODIZYA+C/CJwD4AYZYAhAoCJ8A4AcYYglAoCB8AoAfYIglAIHihMLnSy+9pI4dOyosLEypqalatmzZMdsfPHhQEydOVHJyskJDQ9W9e3fNmTPnhAoGgGDEEEsAAkWj/3yeOXOmJk2apOnTpys1NVXTpk3TiBEjtGHDBiUkHD30h9Pp1Pnnn6+EhAT997//Vdu2bbV9+3bFxcU1Rf0AEPAYYglAIGl0+Hzuued00003afz48ZKk6dOna/bs2XrzzTd1//33H9X+zTff1P79+7V48WLZ7XZJUseOHU+uagAIIgyxBCCQNKrb3el0avny5UpPTz+0A6tV6enpWrJkSZ3bfPbZZ0pLS9PEiROVmJioPn366IknnpDL5Tq5ygEgSFQPsXR2j9YMsQTA7zXqyGdeXp5cLpcSExNrLU9MTNT69evr3Gbr1q365ptvdO2112rOnDnavHmzbr/9dlVUVGjKlCl1blNeXq7y8vKa5wUFBY0pEwACRq0hlrrT5Q7A/zX71e5ut1sJCQl67bXXNGjQII0ZM0YPPvigpk+fXu82U6dOVWxsbM0tJSWlucsEAJ+UmccQSwACS6PCZ3x8vGw2m3Jycmotz8nJUVJSUp3bJCcnq3v37rLZbDXLevXqpezsbDmdzjq3mTx5svLz82tuO3fubEyZABAwvt+UJ0ka3LGFIkMZYgmA/2tU+HQ4HBo0aJAyMjJqlrndbmVkZCgtLa3ObU4//XRt3rxZbre7ZtnGjRuVnJwsh8NR5zahoaGKiYmpdQOAYPT9Jk+X+5ndGGIJQGBodLf7pEmT9Prrr+vtt9/WunXrdNttt6m4uLjm6vexY8dq8uTJNe1vu+027d+/X3fddZc2btyo2bNn64knntDEiROb7l0AQAByVrq1ZMs+SdKZ3eJNrgYAmkaj+3DGjBmj3NxcPfLII8rOztaAAQM0d+7cmouQduzYIav1UKZNSUnRvHnz9Kc//Un9+vVT27Ztddddd+m+++5runcBAAFo5Y4DKna61CrSod7J9AABCAwWwzAMs4s4noKCAsXGxio/P58ueABB49l5G/Tit5t1Sf82+ufVA80uBwCOqaF5jbndAcBHHTrfky53AIGD8AkAPuhAsVO/ZuVL4mIjAIGF8AkAPuiHLXkyDKl7YpSSYsPMLgcAmgzhEwB80PcbPeN7ctQTQKAhfAKAjzEMQ4s2V4dPzvcEEFgInwDgY7bmFSvroGdKzdROTKkJILAQPgHAx3y/0XOV+5BOLRTusB2nNQD4F8InAPiY6vncOd8TQCAifAKAD3FWurVkK1NqAghchE8A8CErdhxQidOl+CiHeiUxoxuAwEP4BAAfUj2r0Rld42W1WkyuBgCaHuETAHxI9fmeZ3C+J4AARfgEAB+xv9ip1TVTanK+J4DARPgEAB/xw2bPlJo9EqOVGMOUmgACE+ETAHxE9fmeHPUEEMgInwDgAwzD0KLq8T27c74ngMBF+AQAH7Alt1i788vkCLFqaMeWZpcDAM2G8AkAPqC6y31ox5ZMqQkgoBE+AcAHHJpSk/M9AQQ2wicAmKy80qUlW6qn1OR8TwCBjfAJACZbsf2gSitcio8KVc+kaLPLAYBmRfgEAJMdPsQSU2oCCHSETwAwWc2Uml053xNA4CN8AoCJ9hWVa81uptQEEDwInwBgoh+27JNhSD2TopXAlJoAggDhEwBMtKjqfE+63AEEC8InAJjEMIxD43sypSaAIEH4BACTbMkt1p78MjlsTKkJIHgQPgHAJNVd7oM7tmBKTQBBg/AJACZZtLlqiCWucgcQRAifAGCCCpdbP27dL0k6syvnewIIHoRPADDBqp0HVVReqRYRdp3SJsbscgDAawifAGCC6qvcT+/KlJoAggvhEwBMsOiw+dwBIJgQPgHAy/JLK/TLLs+Ummd043xPAMGF8AkAXrZkyz653IY6x0eqbVy42eUAgFcRPgHAyxZtrppSky53AEGI8AkAXrao6mIj5nMHEIwInwDgRTv3l2jbvhLZrBad1qWV2eUAgNcRPgHAi6pnNRqQEqeYMLvJ1QCA9xE+AcCL6HIHEOwInwDgJS63oR+2eMIn43sCCFaETwDwkrW783WwpELRoSHqnxJndjkAYArCJwB4SfWUmqd1aSW7jZ9fAMGJXz8A8JLvmVITAAifAOANJc5KLd9+QBIXGwEIboRPAPCCpZn7VeEy1DYuXJ3iI80uBwBMQ/gEAC84fIgli8VicjUAYB7CJwB4QU345HxPAEGO8AkAzWxvQZk25BTKYpFO53xPAEGO8AkAzax6Ss1T2sSoZaTD5GoAwFyETwBoZtVd7md2a21yJQBgPsInADQjwzD0fdWRzzPpcgcAwicANKcNOYXKLSxXmN2qQR1bmF0OAJiO8AkAzai6y31op1YKDbGZXA0AmI/wCQDNqHo+d7rcAcCD8AkAzaS80qWlmfskMb4nAFQjfAJAM1m+/YDKKtyKjwpVz6Ros8sBAJ9A+ASAZnJoSs1WTKkJAFUInwDQTKoHlz+D8T0BoAbhEwCawYFip1Zn5UuSzuR8TwCoQfgEgGbww5Y8GYbUPTFKiTFhZpcDAD6D8AkAzeDQ+Z50uQPA4QifANDEDMM4NL4nXe4AUAvhEwCa2LZ9Jco6WCq7zaLUzi3NLgcAfArhEwCa2KJNuZKkU9u3UIQjxORqAMC3ED4BoInR5Q4A9SN8AkATqnC5tXiLZ0rNMxnfEwCOQvgEgCa0csdBFZVXqmWkQ33bxppdDgD4HMInADShhRv3SpLO6Bovq5UpNQHgSIRPAGhCCzd6LjYa3p0udwCoC+ETAJpIXlG51mQVSJLO7M7FRgBQF8InADSR76uGWOqdHKOEaKbUBIC6ED4BoIl8t9EzxNJZdLkDQL1OKHy+9NJL6tixo8LCwpSamqply5Y1aLsPPvhAFotFo0ePPpGXBQCf5XYb+o7zPQHguBodPmfOnKlJkyZpypQpWrFihfr3768RI0Zo7969x9xu27Ztuueee3TmmWeecLEA4Kt+21OgfcVORTpsGtShhdnlAIDPanT4fO6553TTTTdp/Pjx6t27t6ZPn66IiAi9+eab9W7jcrl07bXX6tFHH1Xnzp1PqmAA8EXVV7mndYmXI4QzmgCgPo36hXQ6nVq+fLnS09MP7cBqVXp6upYsWVLvdo899pgSEhJ0ww03NOh1ysvLVVBQUOsGAL6sZoilHnS5A8CxNCp85uXlyeVyKTExsdbyxMREZWdn17nNokWL9MYbb+j1119v8OtMnTpVsbGxNbeUlJTGlAkAXlVYVqEV2w9IkoYzpSYAHFOz9g0VFhbquuuu0+uvv674+IaPeTd58mTl5+fX3Hbu3NmMVQLAyVm8ZZ8q3YY6xUeqfasIs8sBAJ8W0pjG8fHxstlsysnJqbU8JydHSUlJR7XfsmWLtm3bplGjRtUsc7vdnhcOCdGGDRvUpUuXo7YLDQ1VaGhoY0oDANNUd7mf1Y2B5QHgeBp15NPhcGjQoEHKyMioWeZ2u5WRkaG0tLSj2vfs2VOrV6/WqlWram6XXHKJzjnnHK1atYrudAB+zzAOG2KJ8z0B4LgadeRTkiZNmqRx48Zp8ODBGjp0qKZNm6bi4mKNHz9ekjR27Fi1bdtWU6dOVVhYmPr06VNr+7i4OEk6ajkA+KOtecXadaBUDptVp3VuZXY5AODzGh0+x4wZo9zcXD3yyCPKzs7WgAEDNHfu3JqLkHbs2CGrlWFGAASHhRs8Rz2HdGqhCEejf1IBIOhYDMMwzC7ieAoKChQbG6v8/HzFxMSYXQ4A1Lh+xjIt2JCrBy7qqZvPOvocdgAIFg3NaxyiBIATVFbh0o9b90liPncAaCjCJwCcoJ+27VdZhVuJMaHqkRhtdjkA4BcInwBwgr5dXz3EUmtZLBaTqwEA/0D4BIAT9O2GvZKkc3smmFwJAPgPwicAnIDMvGJl5hXLbrPoDAaXB4AGI3wCwAn4Zr3nqOeQji0VHWY3uRoA8B+ETwA4Ad+up8sdAE4E4RMAGqmovFJLMz1DLBE+AaBxCJ8A0EiLNuWqwmWoY6sIdW4dZXY5AOBXCJ8A0EjV53uew1FPAGg0wicANILbbejbqvnc6XIHgMYjfAJAI6zdXaDcwnJFOGwa2qml2eUAgN8hfAJAI1R3uZ/RNV6hITaTqwEA/0P4BIBG+IZZjQDgpBA+AaCB8orK9euug5K42AgAThThEwAaaMGGXBmG1KdtjBJjwswuBwD8EuETABqoZlajHhz1BIATRfgEgAaocLn13UbPEEt0uQPAiSN8AkAD/LztgArLK9Uq0qH+7eLMLgcA/BbhEwAa4Nuqq9yH92gtq9VicjUA4L8InwDQANXjezLEEgCcHMInABzHtrxibd5bpBCrRWd2a212OQDg1wifAHAcX/2WLUk6rXMrxYbbTa4GAPwb4RMAjuOrtTmSpAtOSTS5EgDwf4RPADiG3MJyLd9xQJJ0fm/CJwCcLMInABzD1+tyZBhS/3axSo4NN7scAPB7hE8AOIav1nrO97zglCSTKwGAwED4BIB6FJZV6IfN+yRJF9DlDgBNgvAJAPVYuDFXTpdbneMj1TUhyuxyACAgED4BoB7zqq5yP/+URFkszGoEAE2B8AkAdSivdOnbqlmNLujN+Z4A0FQInwBQhx+37ldReaVaR4dqYEqc2eUAQMAgfAJAHeZVXeV+fu9EWa10uQNAUyF8AsAR3G5D83+rmtWIq9wBoEkRPgHgCKt2HVRuYbmiQ0M0rEu82eUAQEAhfALAEaq73M/umSBHCD+TANCU+FUFgMMYhqGv1tLlDgDNhfAJAIfZklukzLxiOWxWnd2jtdnlAEDAIXwCwGGqB5Yf1rWVosPsJlcDAIGH8AkAh/lyzR5JDCwPAM2F8AkAVbbvK9aarALZrBaNOIXzPQGgORA+AaDK7NWeo56ndW6pVlGhJlcDAIGJ8AkAVeZUhc+L+7YxuRIACFyETwAQXe4A4C2ETwDQoS73tM6t6HIHgGZE+AQAHepyv6hvssmVAEBgI3wCCHp0uQOA9xA+AQS9L36lyx0AvIXwCSDoff7LbknSxf3ocgeA5kb4BBDUNmQXan12oew2iy7qQ/gEgOZG+AQQ1D77JUuSNLx7gmIjmMsdAJob4RNA0DIMQ59VdblfMoCB5QHAGwifAILWyp0HtXN/qSIcNqX3SjC7HAAICoRPAEHrs1Weo57n905UhCPE5GoAIDjwawsgKLncRs0QS5d6q8vdVSkV5XhuxXmSI1KKSpCiEqWwGO/UAAAmI3wCCEqLt+Qpr6hccRF2ndG1dfO90MGd0oY50uYMadv3UkVJ3e1a95K6nid1O1/qeJZkpWMKQGAifAIISrNWeK5yv7hvshwhzRD0dq+SFv9TWvuJZLgOLbeGSJEJUmS8J4gW7ZXKC6TcdZ7bkhelVt2kYX+U+l0l2cOavjYAMBHhE0DQKS6v1JdrsiVJlw9q17Q7P7hT+vI+acPsQ8vap0ndR0hdzpMS+xx9VLN4n7T1W2nLN9K6L6R9m6TP75K+nSqNeFzqc7lksTRtnQBgEsIngKAzd022Sitc6hQfqYEpcU2zU1eltOw16Zv/kyqKJYtN6nOZNOwOKbn/sbeNbCX1vcJzG/mUtOIdacnLUsEu6X83SL+8L138d6lFx6apFQBMxElFAILOxyt3SZIuG9hWlqY4oliUK/17tDRvsid4ppwm3faDdPm/jh88jxQaLaVNlO5cIZ39gGRzSJu/ll45Xfrt05OvFQBMRvgEEFR2HyzV4i37JEmjB7Y9+R3u/El69SzPxUSOKGnUP6TxX0oJvU5uvyGh0tn3SbctkdoPk5xF0odjpa8e9hxlBQA/RfgEEFQ+WZUlw5BSO7VUSsuIk9vZrx9JM0ZKhbul+O7STd9Kg65v2ivV47tK4z73dN9LnouY3rtSKi9qutcAAC8ifAIIGoZh6OOqq9wvP/UkLzRa+pr08Y2Su0LqNUq66RupdfcmqLIOthDpgv+Trnxbskd6Lkx651KpZH/zvB4ANCPCJ4Cg8cuufG3eW6TQEKtG9k06sZ0YhrTwaenLez3Ph94iXfmO51zN5nbKaGncZ1J4CynrZ2nGRVLBnuZ/XQBoQoRPAEFj5k87JUkX9U1WdJj9xHby3TPSt497Hp892XN1ujcHhG832HNOaXSyZ1zQt0d5LngCAD9B+AQQFEqclfr8F89c7n8YnHJiO1ny8qHgecHj0tn3mzP+ZkIvacI8KTbFMybov38vlR7wfh0AcAIInwCCwuxf96iovFIdWkXotM4tG7+D5W97hlKSpHMe9MxAZKYWHaSxn3pmS8pZLb17hVReaG5NANAAhE8AQeHDnz1d7n8YnNL4sT03zJW+uNvzeNid0ln3Nm1xJ6pVF08ArT4H9KPxDMMEwOcRPgEEvC25Rfpp2wFZLSdwlfvuVdJ/J0iGWxp4nXT+Y7411WVib+na/0kh4dLm+dKXf/FcFAUAPorwCSDgfVh1odE5PRKUFBvW8A3zd0nvjfHMWtT5bOl3z/tW8KzWbpB0+euSLNLPb0hLXjK7IgCoF+ETQEBzVrr1vxWe6TT/MKQRFxo5S6T3r5KKsqXWPaU/vCPZTvAKeW/oNcozFqgkffWQ51QBAPBBhE8AAW3e2mzlFTmVEB2qc3smNGwjw5A+v1PKXi1FxEvXfCiFxTZvoU0hbaI0eIIkQ/r4Jilvs9kVAcBRCJ8AAtq7P26XJF01JEV2WwN/8n58WVr9kWQN8RzxbNGhGStsQhaLdOFTUvs0qbxAmnktV8AD8DknFD5feukldezYUWFhYUpNTdWyZcvqbfv666/rzDPPVIsWLdSiRQulp6cfsz0ANJVNOYVamrlfVot01dD2Ddso8zvpq4c9j0c8IXU8vfkKbA4hDs80nNHJUu566ZPbuAAJgE9pdPicOXOmJk2apClTpmjFihXq37+/RowYob1799bZfsGCBbr66qv17bffasmSJUpJSdEFF1ygrKysky4eAI7lP0t3SJLO65WoNnHhx9+gMLvqynaX1P9qaejNzVxhM4lOlMa8K9kc0rrPpaXTza4IAGpYDKNxfxKnpqZqyJAhevHFFyVJbrdbKSkpuuOOO3T//fcfd3uXy6UWLVroxRdf1NixYxv0mgUFBYqNjVV+fr5iYmIaUy6AIFXirFTq4xkqLK/U2xOGanj31sfewFUpvXOptH2RlNhHuvFryd6AwOrLlr0uzblHstqlG76S2p5qdkUAAlhD81qjjnw6nU4tX75c6enph3ZgtSo9PV1Llixp0D5KSkpUUVGhli3rn2GkvLxcBQUFtW4A0BifrdqtwqoZjc7sGn/8DRZM9QRPR5Sn29rfg6ckDblR6nWJ5K6Q/jteKss3uyIAaFz4zMvLk8vlUmJiYq3liYmJys7ObtA+7rvvPrVp06ZWgD3S1KlTFRsbW3NLSTnBeZgBBCXDMPTvqguNrhnaXlbrccbm3Jwhff+s5/El/5TiuzZzhV5isUiXvCDFtZcObJM+v4vzPwGYzqtXuz/55JP64IMPNGvWLIWF1T/Q8+TJk5Wfn19z27lzpxerBODvlmXu19rdBQqzW/WHwcf547U4z3NRjuQZpqjP5c1foDeFx0lXzPBcub92lrT8LbMrAhDkGhU+4+PjZbPZlJOTU2t5Tk6OkpKSjrnts88+qyeffFJfffWV+vXrd8y2oaGhiomJqXUDgIaa8cM2SdLvB7ZTi0hH/Q0NQ/rsTqkoxzOQ/IgnvFOgt7UbLJ03xfN47v1Szlpz6wEQ1BoVPh0OhwYNGqSMjIyaZW63WxkZGUpLS6t3u6efflp/+9vfNHfuXA0ePPjEqwWA49i5v0Rf/eY5DWjC6R2P3XjF29KG2Z4Lci57PTDO86xP2h+lrudLlWXSR9dLzmKzKwIQpBrd7T5p0iS9/vrrevvtt7Vu3TrddtttKi4u1vjx4yVJY8eO1eTJk2vaP/XUU3r44Yf15ptvqmPHjsrOzlZ2draKioqa7l0AQJW3F2+T25DO7BavbonR9TfM2yzNrfqtOu8RKfnYPTJ+z2qVfj/dM/5n3kZpzl/MrghAkGp0+BwzZoyeffZZPfLIIxowYIBWrVqluXPn1lyEtGPHDu3Zs6em/SuvvCKn06krrrhCycnJNbdnn3226d4FAEgqKq/UzJ8954hPOL1T/Q1dFZ7pJytKpE5neY4KBoPIeOnyf0mySKveldZ+YnZFAIJQo8f5NAPjfAJoiLcXb9OUz9aqc3ykvp40vP6r3DP+5rm6PSxOum2xFNvWq3WaLuMx6fu/B+/7B9AsmmWcTwDwVZUut17/fqskafzpHesPntuXSIue8zweNS04g9fZk6U2A6Wyg9Int0put9kVAQgihE8AAWH26j3adaBULSMdumJQPcMrleVLH98sGW6p/zXSKb/3bpG+wmaXLvuXZI/wzGW/5EWzKwIQRAifAPyeYRiavtBz1PP6YR0V7rDV3XDOvVL+DimugzTyKS9W6IPiu0oXTvU8znhM2vOLufUACBqETwB+77tNeVq3p0DhdpvGpnWou9Hq/0q/zpQsVs+wSmGcP65Tx0k9f+eZfvN/N0nOErMrAhAECJ8A/N70BVskSVcNTVFcRB2Dyh/cKX0xyfP4rHul9qlerM6HWSzSqH9KUUlS3gZp/sNmVwQgCBA+Afi1X3Ye1JKt+xRitejGMzsf3cDtkmbdKpXnS20He8InDolsJf3+Fc/jn/4lbZhrbj0AAh7hE4Bfe+GbTZKkS/q3Udu4OmYoWvyCtH2RZI+ULnvNc7ENautyrnTaRM/jTydKRXvNrQdAQCN8AvBba7Ly9fW6vbJapInndj26we5V0jf/53k88kmpVRev1udXzntESjhFKsnzBFDfHwIagJ8ifALwW//I8Bz1HNW/jbq0jqq90lnimcXIXeG5qGbgdSZU6EfsYZ7Zj2yh0qavPF3wANAMCJ8A/NLa3fma/1uOLBbpjnO7Hd1g/sOeOcyjkqRLXvBcXINjS+wtnf+Y5/FXD0l715tbD4CARPgE4Jf+WX3Us18bdU044qjnxnmHjtz9/hUpoqWXq/NjqbdIXdOlyjLpfzdKleVmVwQgwBA+AfidtbvzNW+t56jnnecdca5nUa7nnEVJOu12z8U0aDiLRbr0ZSmilZSz2jMAPQA0IcInAL/zzLwNkqqPekYfWmEYnuBZnOu5eOa8KSZV6OeiE6VLqqbcXPKitOVbc+sBEFAInwD8yo9b92nBhlyFWC2adH732it/fkPaNM9z0czlr3suosGJ6XmRNGi85/Ent0kl+82tB0DAIHwC8BuGYejpuZ6LYK4amqKO8ZGHVub8Js170PM4/a9S4ineLzDQjHhcatVNKtwjfX4Xwy8BaBKETwB+4+t1e7Vix0GF2a268/Ar3CtKpf9O8Fwk0zVdSr3VvCIDiSPScwTZGiKt+0xa+a7ZFQEIAIRPAH6h0uXWM/M8Rz0nnN5JCTGHdal/9ZCUu06KTJBGvyJZ+WlrMm0GSuc+5Hn85X3Svi3m1gPA7/ELDcAvzPx5pzbmFCk23K5bhh82U9H62YcNqzRdikowp8BANuxOqeOZUkWxZ+B+V4XZFQHwY4RPAD6voKxCf/9qoyTpT+ndFBteNT97ftahYZXS/ih1Pc+kCgOc1eYJ9mGxUtZyaeHTZlcEwI8RPgH4vBcyNml/sVNdWkfq2tM6eBa6XdKsW6TSA1LyAIZVam6x7aTfTfM8/v5ZafsSU8sB4L8InwB8WmZesd5avE2S9NDvestuq/rZWvS8tO17yR4pXfGmFOIwr8hg0ecyqf/VkuGWZt0sleWbXREAP0T4BODTHp+9ThUuQ2f3aK1zelSdz7l9ifTtE57HFz0jtepS/w7QtEY+LcV1kA7ukD6/m+GXADQa4ROAz5r/W46+XpejEKtFD13cy7OwaK/00fWS4ZL6XCENuMbUGoNOWIx0+b88wy+t/Vha9rrZFQHwM4RPAD6pxFmpv362VpJ045mdPdNouio943kWZUvxPaRR//DMRQ7vShkqnV815/u8B6SdP5lbDwC/QvgE4JP+mbFZWQdL1TYuXHee19Wz8NvHD53nOebfUmiUuUUGs9Nul3pfKrkrPEeii/eZXREAP0H4BOBzNmQX6l/fb5Uk/fWSUxThCJE2fCktes7T4JJ/Sq17mFghZLFIl7woteoqFeySPr7RMwIBABwH4ROAT3G5DU3++FdVug2d3ztR5/dOlPZneoZVkqSht0h9rzC3SHiExUh/+Ldkj5C2fMP4nwAahPAJwKfM+CFTK3YcVFRoiP56ySlSRZn04VjPsD7thkgX/J/ZJeJwib0Pjf+58Clp09emlgPA9xE+AfiMrblFembeBknSgxf3UtvYMGnOPVL2r1JEK+nKtxjP0xf1HyMNniDJ8HS/H9hmdkUAfBjhE4BPcLkN/eW/v6q80q0zusbrqiEp0o8vSyv/LckiXfa6Z5Yd+KYLn5TaDPTMOPXeVVJZgdkVAfBRhE8APuHNRZn6efsBRTpsevLyvrJs+kqa96Bn5QX/x7ztvi4kVBrzHykqScpdJ/3vBi5AAlAnwicA063JytfT89ZLkh68uLfaObd5xvOUIZ06VkqbaGp9aKDYttLV70khYdKmr6SvHja7IgA+iPAJwFQlzkrd+cFKVbgMXdA7UVefEia9N0ZyFkkdzpAu+jsDyfuTtoOk30/3PP7xJWn5W6aWA8D3ED4BmOpvX6zT1txiJcaE6qlLu8sy8/9J+Tuklp09A8lzgZH/OeX30jlVp0zM/rOU+Z259QDwKYRPAKb54tfden/ZDlks0nNX9leLjHuknUul0Fjp6plSREuzS8SJOuteqc8VkrtSmnmdlLfZ7IoA+AjCJwBTbMop1F/++6sk6dbhXXR61hvSrzMli036w9tS6+4mV4iTYrFIl74otR0slR2U3rtSKs4zuyoAPoDwCcDrisordeu7y1XidCmtcyvd0/IHacFUz8qLnpG6nGNugWga9nDpqvek2PbS/q3Sf66QygvNrgqAyQifALzKMAz95b+/aEtusZJiwvTq4CzZvrzHs/Ksv0hDbjC3QDSt6ETpuo+l8JbS7pWeLvhKp9lVATAR4ROAV/0zY7PmrM6W3WbRO+eUKWb2bZLhlk4dJ53zgNnloTnEd5Ou/a9kj5S2fivNupkxQIEgRvgE4DWf/7Jbz3+9UZL08pkV6v7NjZKrXOr5O+ni5xhSKZC1GySNeUey2qW1s6TP7pDcbrOrAmACwicAr1i544Du+egXSdIjA0t1/oqJUkWJ1OVc6Yo3JVuIyRWi2XVNl654w3NR2ar/SHPukQzD7KoAeBnhE0Czy8wr1k3v/KzySrdu6HxQ4zP/LDkLPYPIj/mPZ2pGBIfel1YNQm+Rfn5D+vI+AigQZAifAJpVTkGZrntjqfKKnLosYY8e2ne/LGUHpXZDpWs+kBwRZpcIb+v3B+mSf3oeL3vVMxA9XfBA0CB8Amg2+SUVGvvGMu06UKqL43bo2bIpspQXSO3TpP/3Pyk02uwSYZZTx0qXvKiaI6Bf3EUABYIE4RNAsygsq9D1by3ThpxCXRL5m16ofFRWZ5HU8UzPlc9hMWaXCLOdep2nC95ilVa8I318I8MwAUGA8AmgyRWWVWjsm8u0csdBjQlbpn8YT8laWeq5uOiaD6XQKLNLhK/of5V02euSNURa8z/p/askZ7HZVQFoRoRPAE3q8OB5a9jXelL/kMVdIfW53DNfO+d44kh9r/B8N+wR0pYM6e1LmIoTCGCETwBNZl9Rua55fal+2bFfj4f9W/frTVlkSENvli77lxTiMLtE+Kpu6dLYz6TwFlLWz9K/zpNyN5pdFYBmQPgE0CR2HSjRldOXaEtWjt4Ke17X6kvPivS/SiOflqz83OA4UoZIE76S4jpIB7ZJb6RLWxeaXRWAJsa/BgBO2vrsAl3xyhK59m3RF+FTdJaWSyFh0pVvS2f8iZmL0HCtu0s3feMZiqssX3r3Mmnpq4wFCgQQwieAk/LN+hxd/vJi9Sz6UV+EPqzOxk4pKlEa94V0ymizy4M/ioyXxn0u9b1ScldKX/5F+uQ2qaLU7MoANAHCJ4ATYhiG3liUqVveXqqb3R/oTcczilax54jVzQs9XajAibKHea6CH/GEZzrOX96X3jhf2rfF7MoAnCTCJ4BGK3FW6u6Zq/TaF4v0rv1x3RUyS1YZ0qDx0vWzpZhks0tEILBYpLSJ0nWzpIhWUvZq6dWzpF8/MrsyACeB8AmgUTbvLdKlL/6g8l8/0Zeh9yvVul6GI0q6/A1p1DSuaEfT6zxcunWR1OEMyVnkGYx+1m2ec0IB+B3CJ4AGMQxDHyzboWtfnKfbDjyt6Y5pamkpkpL6yXLLd56xGoHmEtNGGveZNPx+z4xIv7wnvTyMq+EBPxRidgEAfN/+Yqfu/9+v0vov9In9bSVb98uwWGU540+eMMDRTniD1SadM1nqco4061bpQKb0ziXS4AmeIb3CYs2uEEADWAzD98evKCgoUGxsrPLz8xUTw3zQgLcYhqHPf92jlz/9TndX/EsX2n7yLG/ZWZbfvyqlDDW5QgSt8iJp/sPSz296nkclSRc9LfW6hKG9AJM0NK8RPgHUKetgqR6btUKdNr+jO0JmKdJSLsMSIsvpd0rD/yLZw80uEZAyv5e+uFvat9nzvMt50oVPesYLBeBVhE8AJ6SswqXXF27RhoXva5LlPXW2ZkuS3G2HyDpqmpTUx9wCgSNVlEnfPyv98A/J5ZSsIZ4pXc+8R4psZXZ1QNAgfAJoFLfb0JzVu/XtnPc1rvRd9bNmSpIqIxIUcsFjUr8xTJEJ37ZvizTvQWlj1dSujijptNuktD9K4XGmlgYEA8IngAYxDEOLNudpzuf/1e8PztBQ6wZJUqUtXLZhE2U5/S4pjP/v4Ec2fy1lPCbt+cXzPCxWGnanlHqrFBplbm1AACN8AjgmwzC0cMNefTf3Q527732dYVsrSaq0hsoYfKPswyd5pjkE/JFhSOs+l759XMpd71kWEe8ZtH7Q9VJES1PLAwIR4RNAncorXZq7Yqt2LXxLFxTOUjdrliTJZQlRRb//p7Dz7vOMqQgEArdLWvM/6dsnPEMzSVJIuNT/Ks+R0ISe5tYHBBDCJ4Ba9uSX6rOFy+RYOUOj3fPVwlIkSSq3RsjV/xpFnHWH1KKjuUUCzcVVIa3+SPrxZc80ndW6nCul3iZ1Pc8zjiiAE0b4BCCX29DS9du0ZeH76rrnc6Va1slq8fwvnx/WViFptyoydRyDcyN4GIa0fbEnhK6fLanqn8DoZKnvlVL/q6XE3qaWCPgrwicQpAzD0KrtuVr7/edqtXWWznYvVbjFWbM+r/VpanHOHbL1HMmRHgS3A9ukpa9Jq/4jlR08tDypryeE9rlcik4yqzrA7xA+gSDidhtam7lT25d+ptAt8zS08mfFWkpq1ueGtpfR7yolnH6dFNfexEoBH1RZLm36SvrlA2njPMldcWhd28FSz4ukHhdJrXsyexJwDIRPIMDlFzu1csViHVjztZJzFmqQsVZ2i6tmfaEtTgc7/05JZ46XPWUQ/2gCDVGy33OB0q8zpV0/1V7XoqMnhHY5T2p/GsM2AUcgfAIBJr/EqXVrVujgb98oZs9i9Sj7Ra0sBbXa7LG3V1HH85Vy2uUK63Qa3erAySjYI22cK22YI21dKLnKD62zhkhtTpU6nSl1PFNKSZUcEebVCvgAwifgx1xuQ9t37lD2usUq3/6TInN/UZeKDWplKazVrkwO7Y4ZIGvXc9Qm9Qo5EpnPGmgW5UXS1m89YTTzO+ngjtrrrSFS4ilS20GHbvHd+QMQQYXwCfiJwpJS7dy8Rge3rVLF7jUKO7BBbcq2KMWy96i2Ttm1I7KPytqertb90pXY83QpxGFC1UCQO7Bd2va9lPm9574g6+g2jijPxUsJvaWEXp5wmtBLCm/h/XoBLyB8Aj6k3FmunB2bdGDXBpXlbJFr/1Y5CrYrrixLKe4shVoq69wuy9ZOebF9ZEsZrDa9T1fLLoOkkFAvVw/gmAxDyt8pZa2QspZ77nevlCqK624f3UZq1cVza3nYfYuOkj3Mq6UDTYnwCXhJRUWF8vZmKT9nu4pzd6n8QJaMgt2yFecovDRbrZy7lWTsVYjFXe8+ShSmLHtHFcZ2V0jSKWrVeYCSeqbKFsEREsAvuV1S7gYpZ620d62U85u09zdPSD2WyNZSTFsptl3Vfdvaz6MS+AMUPqtZw+dLL72kZ555RtnZ2erfv79eeOEFDR06tN72H330kR5++GFt27ZN3bp101NPPaWLLrqowa9H+IS3lJeXqbjggEoLD6q06IBK83NVlp+ryqI8GcX7ZC3bL1vZAYU6Dyq88qBiXAfUyjh4zGBZs2/Drmxbkg6EtVN5dHvZ47souk03JXbuq5jELpLV6oV3CMBUZfmeULpvi7R/y2H3WyVn4fG3l6TQGCky3hNUI+KrHlc9D2/pmTQiLMZzH1p174jiNwbNrqF5LaSxO545c6YmTZqk6dOnKzU1VdOmTdOIESO0YcMGJSQkHNV+8eLFuvrqqzV16lT97ne/03vvvafRo0drxYoV6tOnT2NfHkHKcLtVWVmhygqnKiqcqigvlbOsWM6yUlWUFauyvMRzc5bKVV4qt7NE7opSGVU3S0WZVFkmVZbI5ixWSGWhHJXFCnUVK8xdonCjRJFGicIsFWr0MQWL5DIsOmCJ08GQeBWHtpYzIlFGVLJCYpMVmdxN8Sk91TKpvTpYberQHB8QAP8QFiulDPXcDmcYUukBKX+X5/zRmvusw57v9oxBWl7gue3f2ogXtngCaWjsoXAaGuO5Qt8eLtkjDt3qWmYPl0LCJJvdc+TV5vDcjnzMBVZogEYf+UxNTdWQIUP04osvSpLcbrdSUlJ0xx136P777z+q/ZgxY1RcXKwvvviiZtlpp52mAQMGaPr06Q16TW8e+SwuPKiNiz9VzcdiGDr0CblV/cSQUfO49n3VeuOw9TIO7a9qvcU4Yplx2HY6Yr8y6nzcmLa12tW8B0mGu2a9xZAMuSW327PccHnu3W5Zqh8bdT+21HpuyGK4ZNHhy901zy2GS1ajUjZ3pWyqlNWoVIhRKVv1TS7Z5VkXYrgUoko5Dhu/0htKDYdKLBEqskarJCRO5fY4VYTGyR3eSpaIlrJFxcsR01oRLZLUIqmD4uLbyBpi92qNAIKMYXhmYireJxXnHrqVHPa8ZL8nmJYVeI6yluXXHjS/uVlsVYHULtlCDwVSa4jnZrHVfl7z+HhtbIfWWayecYstVs9NlqrnlsOeW+t4bj3U7rht6ttv9XjJh42bfOSyRrVp6HZ1Pa9nuyPbtOgoJfeTNzTLkU+n06nly5dr8uTJNcusVqvS09O1ZMmSOrdZsmSJJk2aVGvZiBEj9Mknn9T7OuXl5SovPzSeWkFBQb1tm9qBnF0auOROr70ejuMY46KXG3aVWxwql0NOi0MVllBVWD03V/XNFia3LUzukDAZIWFSSLgUFiNrWLRCwmMUEhErR2SswiLjFBYVp4joFoqMjlO43aFwSa289kYB4DgsFs+V8uEtpPiuDdvGMDy9PtVhtLzAE2DLqo6eVpRKFSWSs6TqcXEdy0o8t8pyyeX03CqdnnFPXc4jXs9V1b7J3z1O1JAbpYv/bnYVtTQqfObl5cnlcikxMbHW8sTERK1fv77ObbKzs+tsn52dXe/rTJ06VY8++mhjSmsy9rBwrbOfIkkyDvsr4tDh4aq/mCQZh/11UdNWlsOWH9a2aj+HlldtY7EcsX3V+qOWWw4LYpZatdXaX63Xqaue+vavmr/uDIvV89elLDKs1X9l2jz7qf6rs2Z59WObLFZrTVtZrLJYa9/LapOl6rk1xCFLiEPWkBBZbQ5ZQxyyhdhltYfKarMrxO55brOHymZ3KMTukD0kVDa7XSEOh+z2MIXabI3vIgeAYGKxVHWhh0vRicdv31iGIbkqqoJoRVVAPeKx21V1q6y6uTwhteZ5pafH7fDnRh3bVD+XUdX7ZtT0wtVeZhzdRoe1rbXMaECbw/cr6bBEUGfvYoPbnMx2jdh3i05Hb2eyRp/z6Q2TJ0+udbS0oKBAKSkpXnntxHZdlPjgYq+8FgAAfs1i8Yw1zHjDaIRGhc/4+HjZbDbl5OTUWp6Tk6OkpKQ6t0lKSmpUe0kKDQ1VaCjHtAAAAAJNo8ZdcDgcGjRokDIyMmqWud1uZWRkKC0trc5t0tLSarWXpPnz59fbHgAAAIGr0d3ukyZN0rhx4zR48GANHTpU06ZNU3FxscaPHy9JGjt2rNq2baupU6dKku666y4NHz5cf//733XxxRfrgw8+0M8//6zXXnutad8JAAAAfF6jw+eYMWOUm5urRx55RNnZ2RowYIDmzp1bc1HRjh07ZD1sINthw4bpvffe00MPPaQHHnhA3bp10yeffMIYnwAAAEGI6TUBAABw0hqa15hrCwAAAF5D+AQAAIDXED4BAADgNYRPAAAAeA3hEwAAAF5D+AQAAIDXED4BAADgNYRPAAAAeA3hEwAAAF5D+AQAAIDXED4BAADgNYRPAAAAeA3hEwAAAF4TYnYBDWEYhiSpoKDA5EoAAABQl+qcVp3b6uMX4bOwsFCSlJKSYnIlAAAAOJbCwkLFxsbWu95iHC+e+gC3263du3crOjpaFoul2V+voKBAKSkp2rlzp2JiYpr99fwFn0v9+GzqxudSPz6buvG51I/Ppm58LvXz9mdjGIYKCwvVpk0bWa31n9npF0c+rVar2rVr5/XXjYmJ4YtcBz6X+vHZ1I3PpX58NnXjc6kfn03d+Fzq583P5lhHPKtxwREAAAC8hvAJAAAAryF81iE0NFRTpkxRaGio2aX4FD6X+vHZ1I3PpX58NnXjc6kfn03d+Fzq56ufjV9ccAQAAIDAwJFPAAAAeA3hEwAAAF5D+AQAAIDXED4BAADgNUEZPh9//HENGzZMERERiouLq7PNjh07dPHFFysiIkIJCQm69957VVlZecz97t+/X9dee61iYmIUFxenG264QUVFRc3wDrxjwYIFslgsdd5++umnerc7++yzj2p/6623erFy7+jYseNR7/PJJ5885jZlZWWaOHGiWrVqpaioKF1++eXKycnxUsXNb9u2bbrhhhvUqVMnhYeHq0uXLpoyZYqcTucxtwvU78xLL72kjh07KiwsTKmpqVq2bNkx23/00Ufq2bOnwsLC1LdvX82ZM8dLlXrH1KlTNWTIEEVHRyshIUGjR4/Whg0bjrnNW2+9ddR3IywszEsVe89f//rXo95nz549j7lNoH9fpLp/Zy0WiyZOnFhn+0D+vnz33XcaNWqU2rRpI4vFok8++aTWesMw9Mgjjyg5OVnh4eFKT0/Xpk2bjrvfxv5ONYWgDJ9Op1NXXnmlbrvttjrXu1wuXXzxxXI6nVq8eLHefvttvfXWW3rkkUeOud9rr71Wa9eu1fz58/XFF1/ou+++080339wcb8Erhg0bpj179tS63XjjjerUqZMGDx58zG1vuummWts9/fTTXqraux577LFa7/OOO+44Zvs//elP+vzzz/XRRx9p4cKF2r17ty677DIvVdv81q9fL7fbrVdffVVr167V888/r+nTp+uBBx447raB9p2ZOXOmJk2apClTpmjFihXq37+/RowYob1799bZfvHixbr66qt1ww03aOXKlRo9erRGjx6tNWvWeLny5rNw4UJNnDhRP/74o+bPn6+KigpdcMEFKi4uPuZ2MTExtb4b27dv91LF3nXKKafUep+LFi2qt20wfF8k6aeffqr1mcyfP1+SdOWVV9a7TaB+X4qLi9W/f3+99NJLda5/+umn9c9//lPTp0/X0qVLFRkZqREjRqisrKzefTb2d6rJGEFsxowZRmxs7FHL58yZY1itViM7O7tm2SuvvGLExMQY5eXlde7rt99+MyQZP/30U82yL7/80rBYLEZWVlaT124Gp9NptG7d2njssceO2W748OHGXXfd5Z2iTNShQwfj+eefb3D7gwcPGna73fjoo49qlq1bt86QZCxZsqQZKvQNTz/9tNGpU6djtgnE78zQoUONiRMn1jx3uVxGmzZtjKlTp9bZ/g9/+INx8cUX11qWmppq3HLLLc1ap5n27t1rSDIWLlxYb5v6fqcDzZQpU4z+/fs3uH0wfl8MwzDuuusuo0uXLobb7a5zfbB8XyQZs2bNqnnudruNpKQk45lnnqlZdvDgQSM0NNR4//33691PY3+nmkpQHvk8niVLlqhv375KTEysWTZixAgVFBRo7dq19W4TFxdX64hgenq6rFarli5d2uw1e8Nnn32mffv2afz48cdt+5///Efx8fHq06ePJk+erJKSEi9U6H1PPvmkWrVqpYEDB+qZZ5455qkZy5cvV0VFhdLT02uW9ezZU+3bt9eSJUu8Ua4p8vPz1bJly+O2C6TvjNPp1PLly2v9t7ZarUpPT6/3v/WSJUtqtZc8vzuB/t2QdNzvR1FRkTp06KCUlBRdeuml9f4O+7tNmzapTZs26ty5s6699lrt2LGj3rbB+H1xOp169913NWHCBFkslnrbBcv35XCZmZnKzs6u9Z2IjY1Vampqvd+JE/mdaiohzbp3P5WdnV0reEqqeZ6dnV3vNgkJCbWWhYSEqGXLlvVu42/eeOMNjRgxQu3atTtmu2uuuUYdOnRQmzZt9Ouvv+q+++7Thg0b9PHHH3upUu+48847deqpp6ply5ZavHixJk+erD179ui5556rs312drYcDsdR5xknJiYGzHfkSJs3b9YLL7ygZ5999pjtAu07k5eXJ5fLVefvyPr16+vcpr7fnUD9brjdbt199906/fTT1adPn3rb9ejRQ2+++ab69eun/Px8Pfvssxo2bJjWrl173N8if5Kamqq33npLPXr00J49e/Too4/qzDPP1Jo1axQdHX1U+2D7vkjSJ598ooMHD+r666+vt02wfF+OVP3fvTHfiRP5nWoqARM+77//fj311FPHbLNu3brjnsAdDE7ks9q1a5fmzZunDz/88Lj7P/w81759+yo5OVnnnXeetmzZoi5dupx44V7QmM9m0qRJNcv69esnh8OhW265RVOnTvW5qcxO1ol8Z7KysnThhRfqyiuv1E033XTMbf35O4MTM3HiRK1Zs+aY5zVKUlpamtLS0mqeDxs2TL169dKrr76qv/3tb81dpteMHDmy5nG/fv2UmpqqDh066MMPP9QNN9xgYmW+44033tDIkSPVpk2betsEy/fF3wVM+Pzzn/98zL+GJKlz584N2ldSUtJRV3tVX5GclJRU7zZHnqBbWVmp/fv317uNWU7ks5oxY4ZatWqlSy65pNGvl5qaKslzFMzXg8TJfI9SU1NVWVmpbdu2qUePHketT0pKktPp1MGDB2sd/czJyfG578iRGvu57N69W+ecc46GDRum1157rdGv50/fmbrEx8fLZrMdNZLBsf5bJyUlNaq9P/vjH/9Yc1FmY49G2e12DRw4UJs3b26m6nxDXFycunfvXu/7DKbviyRt375dX3/9daN7Q4Ll+1L93z0nJ0fJyck1y3NycjRgwIA6tzmR36mmEjDhs3Xr1mrdunWT7CstLU2PP/649u7dW9OVPn/+fMXExKh37971bnPw4EEtX75cgwYNkiR98803crvdNf+Q+orGflaGYWjGjBkaO3as7HZ7o19v1apVklTrfwhfdTLfo1WrVslqtR51+kW1QYMGyW63KyMjQ5dffrkkacOGDdqxY0etv9R9UWM+l6ysLJ1zzjkaNGiQZsyYIau18aeW+9N3pi4Oh0ODBg1SRkaGRo8eLcnTzZyRkaE//vGPdW6TlpamjIwM3X333TXL5s+f7/PfjcYwDEN33HGHZs2apQULFqhTp06N3ofL5dLq1at10UUXNUOFvqOoqEhbtmzRddddV+f6YPi+HG7GjBlKSEjQxRdf3KjtguX70qlTJyUlJSkjI6MmbBYUFGjp0qX1juxzIr9TTaZZL2fyUdu3bzdWrlxpPProo0ZUVJSxcuVKY+XKlUZhYaFhGIZRWVlp9OnTx7jggguMVatWGXPnzjVat25tTJ48uWYfS5cuNXr06GHs2rWrZtmFF15oDBw40Fi6dKmxaNEio1u3bsbVV1/t9ffX1L7++mtDkrFu3bqj1u3atcvo0aOHsXTpUsMwDGPz5s3GY489Zvz8889GZmam8emnnxqdO3c2zjrrLG+X3awWL15sPP/888aqVauMLVu2GO+++67RunVrY+zYsTVtjvxsDMMwbr31VqN9+/bGN998Y/z8889GWlqakZaWZsZbaBa7du0yunbtapx33nnGrl27jD179tTcDm8TDN+ZDz74wAgNDTXeeust47fffjNuvvlmIy4urmYUjeuuu864//77a9r/8MMPRkhIiPHss88a69atM6ZMmWLY7XZj9erVZr2FJnfbbbcZsbGxxoIFC2p9N0pKSmraHPm5PProo8a8efOMLVu2GMuXLzeuuuoqIywszFi7dq0Zb6HZ/PnPfzYWLFhgZGZmGj/88IORnp5uxMfHG3v37jUMIzi/L9VcLpfRvn1747777jtqXTB9XwoLC2vyiiTjueeeM1auXGls377dMAzDePLJJ424uDjj008/NX799Vfj0ksvNTp16mSUlpbW7OPcc881XnjhhZrnx/udai5BGT7HjRtnSDrq9u2339a02bZtmzFy5EgjPDzciI+PN/785z8bFRUVNeu//fZbQ5KRmZlZs2zfvn3G1VdfbURFRRkxMTHG+PHjawKtP7v66quNYcOG1bkuMzOz1me3Y8cO46yzzjJatmxphIaGGl27djXuvfdeIz8/34sVN7/ly5cbqampRmxsrBEWFmb06tXLeOKJJ4yysrKaNkd+NoZhGKWlpcbtt99utGjRwoiIiDB+//vf1wpm/m7GjBl1/r91+N+5wfSdeeGFF4z27dsbDofDGDp0qPHjjz/WrBs+fLgxbty4Wu0//PBDo3v37obD4TBOOeUUY/bs2V6uuHnV992YMWNGTZsjP5e777675jNMTEw0LrroImPFihXeL76ZjRkzxkhOTjYcDofRtm1bY8yYMcbmzZtr1gfj96XavHnzDEnGhg0bjloXTN+X6txx5K36/bvdbuPhhx82EhMTjdDQUOO888476jPr0KGDMWXKlFrLjvU71VwshmEYzXtsFQAAAPBgnE8AAAB4DeETAAAAXkP4BAAAgNcQPgEAAOA1hE8AAAB4DeETAAAAXkP4BAAAgNcQPgEAAOA1hE8AAAB4DeETAAAAXkP4BAAvyc3NVVJSkp544omaZYsXL5bD4VBGRoaJlQGA9zC3OwB40Zw5czR69GgtXrxYPXr00IABA3TppZfqueeeM7s0APAKwicAeNnEiRP19ddfa/DgwVq9erV++uknhYaGml0WAHgF4RMAvKy0tFR9+vTRzp07tXz5cvXt29fskgDAazjnEwC8bMuWLdq9e7fcbre2bdtmdjkA4FUc+QQAL3I6nRo6dKgGDBigHj16aNq0aVq9erUSEhLMLg0AvILwCQBedO+99+q///2vfvnlF0VFRWn48OGKjY3VF198YXZpAOAVdLsDgJcsWLBA06ZN07///W/FxMTIarXq3//+t77//nu98sorZpcHAF7BkU8AAAB4DUc+AQAA4DWETwAAAHgN4RMAAABeQ/gEAACA1xA+AQAA4DWETwAAAHgN4RMAAABeQ/gEAACA1xA+AQAA4DWETwAAAHgN4RMAAABeQ/gEAACA1/x/SWhogTu+XXQAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plt.plot(x.numpy(), y, label='y')\n", "plt.plot(x.numpy(), dy_dx, label='dy/dx')\n", "plt.legend()\n", "_ = plt.xlabel('x')" ] }, { "cell_type": "markdown", "metadata": { "id": "DsOMSD_1BGkD" }, "source": [ "### テンソルのソース" ] }, { "cell_type": "markdown", "metadata": { "id": "g3iXKN7KF-st" }, "source": [ "入力がスカラーとテンソルのどちらであっても、`tf.GradientTape.jacobian` は、単一または複数のターゲットの各要素に対するソースの各要素の勾配を効果的に計算します。\n", "\n", "たとえば、このレイヤーの出力の形状は `(10, 7)` です。" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:25.062541Z", "iopub.status.busy": "2022-12-14T20:58:25.062021Z", "iopub.status.idle": "2022-12-14T20:58:25.072052Z", "shell.execute_reply": "2022-12-14T20:58:25.071505Z" }, "id": "39YXItgLxMBk" }, "outputs": [ { "data": { "text/plain": [ "TensorShape([7, 10])" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x = tf.random.normal([7, 5])\n", "layer = tf.keras.layers.Dense(10, activation=tf.nn.relu)\n", "\n", "with tf.GradientTape(persistent=True) as tape:\n", " y = layer(x)\n", "\n", "y.shape" ] }, { "cell_type": "markdown", "metadata": { "id": "tshNRtfKuVP_" }, "source": [ "そしてレイヤーのカーネルの形状は`(5, 10)`です。" ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:25.075241Z", "iopub.status.busy": "2022-12-14T20:58:25.074838Z", "iopub.status.idle": "2022-12-14T20:58:25.078897Z", "shell.execute_reply": "2022-12-14T20:58:25.078271Z" }, "id": "CigTWyfPvPuv" }, "outputs": [ { "data": { "text/plain": [ "TensorShape([5, 10])" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "layer.kernel.shape" ] }, { "cell_type": "markdown", "metadata": { "id": "mN96JRpnAjpx" }, "source": [ "カーネルの出力のヤコビアンの形状は、次の 2 つの形状を連結したものです。" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:25.082193Z", "iopub.status.busy": "2022-12-14T20:58:25.081790Z", "iopub.status.idle": "2022-12-14T20:58:25.191648Z", "shell.execute_reply": "2022-12-14T20:58:25.191069Z" }, "id": "pRLzTTbvEimH" }, "outputs": [ { "data": { "text/plain": [ "TensorShape([7, 10, 5, 10])" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "j = tape.jacobian(y, layer.kernel)\n", "j.shape" ] }, { "cell_type": "markdown", "metadata": { "id": "2Lrv7miMvTll" }, "source": [ "ターゲットの次元を合計すると、`tf.GradientTape.gradient` によって計算された合計の勾配が残ります。" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:25.195027Z", "iopub.status.busy": "2022-12-14T20:58:25.194549Z", "iopub.status.idle": "2022-12-14T20:58:25.203138Z", "shell.execute_reply": "2022-12-14T20:58:25.202531Z" }, "id": "FJjZpYRnDjVa" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "g.shape: (5, 10)\n", "delta: 4.7683716e-07\n" ] } ], "source": [ "g = tape.gradient(y, layer.kernel)\n", "print('g.shape:', g.shape)\n", "\n", "j_sum = tf.reduce_sum(j, axis=[0, 1])\n", "delta = tf.reduce_max(abs(g - j_sum)).numpy()\n", "assert delta < 1e-3\n", "print('delta:', delta)" ] }, { "cell_type": "markdown", "metadata": { "id": "ZKajuGlk_krs" }, "source": [ " \n", "\n", "#### 例: ヘッセ行列（Hessian）" ] }, { "cell_type": "markdown", "metadata": { "id": "NYcsXeo8TDLi" }, "source": [ "`tf.GradientTape` には[ヘッセ行列](https://en.wikipedia.org/wiki/Hessian_matrix)を作成するための明示的なメソッドがありませんが、`tf.GradientTape.jacobian` を使用してこれを作成することは可能です。\n", "\n", "注意: ヘッセ行列には `N**2` 個のパラメータが含まれています。これやその他の理由により、ほとんどのモデルには非実用的です。あくまでも `GradientTape.jacobian` メソッドの使い方のデモンストレーションとしてこの例を含めましたが、直接のヘッセ行列ベースの最適化を推奨するものではありません。ヘッセ行列のベクトル積は[ネストされたテープで効率的に計算できる](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/eager/benchmarks/resnet50/hvp_test.py)ため、2 次最適化の方法としては、はるかに効率的です。" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:25.206828Z", "iopub.status.busy": "2022-12-14T20:58:25.206158Z", "iopub.status.idle": "2022-12-14T20:58:25.472210Z", "shell.execute_reply": "2022-12-14T20:58:25.471567Z" }, "id": "ELGTaell_j81" }, "outputs": [], "source": [ "x = tf.random.normal([7, 5])\n", "layer1 = tf.keras.layers.Dense(8, activation=tf.nn.relu)\n", "layer2 = tf.keras.layers.Dense(6, activation=tf.nn.relu)\n", "\n", "with tf.GradientTape() as t2:\n", " with tf.GradientTape() as t1:\n", " x = layer1(x)\n", " x = layer2(x)\n", " loss = tf.reduce_mean(x**2)\n", "\n", " g = t1.gradient(loss, layer1.kernel)\n", "\n", "h = t2.jacobian(g, layer1.kernel)" ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:25.475967Z", "iopub.status.busy": "2022-12-14T20:58:25.475394Z", "iopub.status.idle": "2022-12-14T20:58:25.479016Z", "shell.execute_reply": "2022-12-14T20:58:25.478452Z" }, "id": "FVqQuZj4XGjm" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "layer.kernel.shape: (5, 8)\n", "h.shape: (5, 8, 5, 8)\n" ] } ], "source": [ "print(f'layer.kernel.shape: {layer1.kernel.shape}')\n", "print(f'h.shape: {h.shape}')" ] }, { "cell_type": "markdown", "metadata": { "id": "_M7XElgaiMeP" }, "source": [ "このヘッセ行列をニュートン法のステップに使用するには、まず軸を平坦化して行列にし、勾配を平坦化してベクトルにします。" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:25.482069Z", "iopub.status.busy": "2022-12-14T20:58:25.481521Z", "iopub.status.idle": "2022-12-14T20:58:25.489017Z", "shell.execute_reply": "2022-12-14T20:58:25.488403Z" }, "id": "6te7N6wVXwXX" }, "outputs": [], "source": [ "n_params = tf.reduce_prod(layer1.kernel.shape)\n", "\n", "g_vec = tf.reshape(g, [n_params, 1])\n", "h_mat = tf.reshape(h, [n_params, n_params])" ] }, { "cell_type": "markdown", "metadata": { "id": "L9rO8b-0mgOH" }, "source": [ "ヘッセ行列は必ず対称行列になります。" ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:25.492736Z", "iopub.status.busy": "2022-12-14T20:58:25.492062Z", "iopub.status.idle": "2022-12-14T20:58:25.495725Z", "shell.execute_reply": "2022-12-14T20:58:25.495163Z" }, "id": "8TCHc7Vrf52S" }, "outputs": [], "source": [ "def imshow_zero_center(image, **kwargs):\n", " lim = tf.reduce_max(abs(image))\n", " plt.imshow(image, vmin=-lim, vmax=lim, cmap='seismic', **kwargs)\n", " plt.colorbar()" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:25.498788Z", "iopub.status.busy": "2022-12-14T20:58:25.498267Z", "iopub.status.idle": "2022-12-14T20:58:25.704515Z", "shell.execute_reply": "2022-12-14T20:58:25.703909Z" }, "id": "DExOxd7Ok2H0" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmcAAAH5CAYAAADENpm/AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABKo0lEQVR4nO3de3RU9b3//1eGOAkxzMQYyBAJAqIiRdCiYFpLKaQG7PIrlZ9fULsEy8GlBU812lb6VfHSnnirRS3C11Ov64hYXFWPnhaLWOJpC1hRFuKFIxS/YCFBxGRy6SSE2b8/lKmRgPud2dvZyTwfa+2lTN755LMvM/Oe9+y93zmO4zgCAABAIIQyPQEAAAD8E8kZAABAgJCcAQAABAjJGQAAQICQnAEAAAQIyRkAAECAkJwBAAAESG6mJ/B5yWRSu3btUr9+/ZSTk5Pp6QAAEGiO46ipqUllZWUKhb78mksikVB7e7svY4fDYeXn5/sydpAFLjnbtWuXysvLMz0NAAB6lJ07d2rQoEFf6t9MJBLq37evmn0aPxaLafv27VmXoAUuOevXr58kaeeZZyqS6256P//GStfj/5/Ejab5JG+5zXXspk2moTV6tPvYUIftU0kyN+w69umnTUPrf0dfdB37h5wq09iVlba5hJrjrmOThRHb2Eq6H9t4hsADD7iPnX/sk6axn9RFrmNnzDANrVD9blN8snSg+7F9PMZDiVbT2PGOAtexkVzb2DK8ybR32I4rly+Z3WJ5PkhS7X+7n/s3T3f/PJaknY3un8vlx9nmbWF93jcbMphIvvvnQ7ypSeXDhqXeP79M7e3tapZ0jaQ8j8duk/TLujq1t7eTnHll8eLFuuuuu1RXV6cxY8bo/vvv17hx477w9w5+lRnJzXWdnOXluX+iRhzb4ZOMuB+7sNA0tAxD+/rGVeD+fUiSFDH8QkGOLSGybBNJslTwg5ScWV5nIn37msbuK8Pzwbq9W22fjy3PH1+Ts7Dxpc6UnBnHzpLk7Oij3c/dehz2S1qO8eAkZ5bXK0tydlAmTwXqK8nr9CmbT4r3Zd2feuopVVdXa+HChXr99dc1ZswYVVVVac+ePX78OQAAgF7Dl+Tsnnvu0dy5c3XZZZdp5MiRWrp0qQoKCvTwww/78ecAAEAGhXxaspXn697e3q4NGzao8jMnD4VCIVVWVmrt2rWHxLe1tSkej3daAAAAspXnydnevXt14MABlZaWdnq8tLRUdXV1h8TX1NQoGo2mFq7UBACgZ6Fy5q2Mr/uCBQvU2NiYWnbu3JnpKQEAAAOSM295fm1PSUmJ+vTpo/r6+k6P19fXKxaLHRKfl5envDyvL8AFAADomTxPTMPhsMaOHavVq1enHksmk1q9erUqKiq8/nMAACDDqJx5y5e74lRXV2vWrFk644wzNG7cOC1atEgtLS267LLL/PhzAAAAvYYvydmMGTP04Ycf6qabblJdXZ1OO+00rVy58pCLBAAAQM/nR6WLypkP5s+fr/nz53f793/+jZWu7/x/694fuB739X8x9M2RdJoh1s/uEpa7oVtdHP0vU/yyxu+4H3viLuNsDj0vMVOsd/+2qC553HXsU3mXmsa+9PT/MUQPN43t5y3o/TzGLe2YJCmScH/D7GTJAOt0XOvosMX72SHA+nyYdIb72yLt67C1CBg8yL+7/iuRcB+bbzuuLF1kknL/fPDzuYPMCFxvTQAA0LNQOfNWNq87AABA4FA5AwAAaaFy5q1sXncAAIDAoXIGAADSkiPvqz05Ho/Xk5CcAQCAtOTI+2Qqm5MzvtYEAAAIECpnAAAgLX0+XbweM1tROQMAAAgQKmcAACAt3ErDW4FNzv5P4kZFnDxXsZaWTF+94VzTPFqf/p3r2P/4D9PQ+tnP3MeGOtpNY+9rdt/OY12O+3ZMkq0l01P/XWYa+8ILTeEK1dW5Dx5uaxETkvsWMa0J28vI03LfksnWjkl64KWTXMf+yxDT0Apv3mz7hYmTXIdatrdk2+aR3FbT2JaWTKFm922KJJl6+DQ12Y4rP1vIWV+DdjS4f75Z2zG98if322XCqH2msf3sgdXc7D42svdvrmNDTU3dmA2CLLDJGQAA6BmonHkrm9cdAAAgcKicAQCAtFA58xbJGQAASAvJmbeyed0BAAACh8oZAABIC5Uzb2XzugMAAAQOlTMAAJAWKmfeyuZ1BwAACBwqZwAAIC05ny5ej5mtqJwBAAAESI7jOE6mJ/FZ8Xhc0WhUH3/cqEjE1gvRjUTCFl/QvMd9cEmJaeykITe29rUzzSPXfR9OydYD0bKOkn09rXO3sPZ6tDDte+M82jvcjx1O2PpCJgu9f04eZD7GfeyB6Of+CcrYfvPz+aOGBtehmz4oNg09apRxLgaW3pqWwzsej2vgwKgaG/153/yivx2NRvVrSQUej90q6V+kjKxXpvG1JgAASAsXBHgrm9cdAAAgcKicAQCAtFA581Y2rzsAAEDgUDkDAABpoXLmrWxedwAAgMChcgYAANJC5cxb2bzuAAAAgUPlDAAApIXKmbdIzgAAQFpIzrwV2ORs0yapsNBdbH6++3H/4z9s8/i3nxlaMp1wgmnsji3b3cd22NoUWbZJqNnWwsciVFdnik8OP8k2vqFFzN/etz7V3ccPGWIcuW6X+2Bjm6Lw5s2uY5MTJ5nGXrrUFK4fXGFo4WNcT0uroo4O09Cm+KYm23FV2t+wTQxtiiQpZHniGyXzvW7O8xnWvnqGY8Xajun9993HDou1msaOGA5xy/Zu96+7HzIksMkZAADoGXI+XbweM1tlc9UQAAAgcKicAQCAtPT5dPF6zGxF5QwAACBAqJwBAIC05Mj7ag/nnAEAACAQqJwBAIC0cJ8zb5GcAQCAtJCceSub1x0AACBwqJwBAIC0UDnzVjavOwAAQODkOI7jZHoSnxWPxxWNRvXxx42KRCKZno6JtX9feOvb7oOHDzeNncy19eLsqUx98H5VbRv8uutchyZjZbaxeyhLL1PJ1v8ShwolbL0bLU112zts+8bY+rTHsmzzV16z9Rs96yz3sZbtHY/HdcwxUTU2fvnvmwffs/9L0tEej90i6TtSRtYr03jlBAAACBDPk7Obb75ZOTk5nZYRI0Z4/WcAAEBAhHxaspUvheqvfOUreumll/75R7KlHg4AAJAmX7Km3NxcxWIxP4YGAAABkyPv2y3Rvslj7733nsrKyjRs2DBdcskl2rFjx2Fj29raFI/HOy0AAKDn6OPTkq08T87Gjx+vRx99VCtXrtSSJUu0fft2feMb31BTU1OX8TU1NYpGo6mlvLzc6ykBAAD0GJ4nZ1OnTtWFF16o0aNHq6qqSr/73e/U0NCg3/zmN13GL1iwQI2Njall586dXk8JAAD4iAsCvOX7mfpFRUU66aSTtHXr1i5/npeXp7y8PL+nAQAA0CP4npg2Nzdr27ZtGjhwoN9/CgAAZECOvK+acUGAh6677jrV1tbq/fff11/+8hd997vfVZ8+fXTRRRd5/acAAAB6Hc+/1vzggw900UUX6aOPPlL//v119tlna926derfv79pnFBHu0Id7a5iLa2K3I7ZHR0dtpZJYUtLpuZm09iJ/GLXsQX5/rXksbb7sTN8vjC0Y5Jk6sdlbt0l98ehtRWXaZtbJ+7jPQutz812ud8u2XKrRUtLJvMxm2t7Lseb3c+lsNA2F+PLoUnEcKxY2jFJ0t697mPL8ve5jg0F4C4HND73lucvWcuXL/d6SAAAgKyRJZ8nAQCAX6iceYvkDAAApIXkzFvZvO4AAKCXWbx4sYYMGaL8/HyNHz9er7766mFj33rrLU2fPl1DhgxRTk6OFi1alPaYXiA5AwAAaQnKTWifeuopVVdXa+HChXr99dc1ZswYVVVVac+ePV3Gt7a2atiwYbr99tsP2xPcOqYXSM4AAECvcM8992ju3Lm67LLLNHLkSC1dulQFBQV6+OGHu4w/88wzddddd2nmzJmHvSG+dUwvkJwBAIC05Pi0SFI8Hu+0tLW1dTmH9vZ2bdiwQZWVlanHQqGQKisrtXbt2m6tlx9jukFyBgAAAqu8vFzRaDS11NTUdBm3d+9eHThwQKWlpZ0eLy0tVV1dXbf+th9jusHVmgAAIC19Pl28HlOSdu7cqUgkkno8G/pxk5wBAIDAikQinZKzwykpKVGfPn1UX1/f6fH6+vrDnuyfiTHd4GtNAACQliBcrRkOhzV27FitXr069VgymdTq1atVUVHRrfXyY0w3Als5S+aGzT0F3djXbBuzuNB9v7/8fNtckobegJZemZJU0OG+11pSX/yJpLtaE7anl7XP55Ah7mOTKjONbek9GO5oNY2dzC8wxVtYtnlBvu0lwNJX1crSK1MybvNc25PT1D/W2BPU8rpmPU4se9PaK9PSt1OSIvmG/rHGfR/Z+zfXsa2xYaaxLdvc+gZq6Ze5o9n9a35TS+bfyoNyE9rq6mrNmjVLZ5xxhsaNG6dFixappaVFl112mSTp0ksv1XHHHZc6b629vV1vv/126v///ve/a+PGjSosLNTwT/tff9GYfsj8HgUAAPDAjBkz9OGHH+qmm25SXV2dTjvtNK1cuTJ1Qv+OHTsUCv0z7du1a5dOP/301L/vvvtu3X333frmN7+pNWvWuBrTDzmO4zi+jd4N8Xhc0WhUH3/c6Oo7ZquGBlu8pXLmR6XvoETCFm+qnBX6Vzkzz9tYOfOzktNTK2eWbd5Tt7dk3ObGsnZQKmd+CsnfyllY/r12ht73r3Jm/QbEItTgU+WsKa5Ro6JqbPTnffNIDr5nvyGpn8djN0k6XcrIemUa55wBAAAECF9rAgCAtATlnLPeIpvXHQAAIHConAEAgLRQOfNWNq87AABA4FA5AwAAaaFy5q1sXncAAIDAoXIGAADSkvPp4vWY2SqwydnTT0sFLu/TeXH0v1yPuy7nO6Z5TJni/uaIoWb3N36VbDd/td8s1P3YoZtvMo29bMStrmMv7njcNHbye5ea4v1kaW+TzLXdVNayzS3bW5IuTjzsOjY5+/umsa03is01vMJY2wlZtrn5BremfW+8garx5q8WlpvnWm8obN0/lrcX883BDQ2ng3Sj5fZC9zeWHVzkft7xuH/HlFt9Pl28HjNb8bUmAABAgAS2cgYAAHoGLgjwVjavOwAAQOBQOQMAAGmhcuatbF53AACAwKFyBgAA0kLlzFvZvO4AAACBQ+UMAACkJUfeV3u4CS0AAEA38bWmt7J53QEAAAKHyhkAAEgLlTNvBTY5+9/RFxVx2VxzWaP7fpkXT9xlnIn7Hm5+8rPfm7l348eLXcf+beo809hDTNG2nomWPo9ScLb5xc4TprH/52z3/TKHm0b2l5/b29oXsr3D/Vysx1VPZd0/ln6ZxR17bHPJH2CKt/DzNcUS72efVARflrysAAAAv1A581Y2rzsAAEDgUDkDAABpoXLmrWxedwAAgMChcgYAANJC5cxbJGcAACAtJGfeyuZ1BwAACBwqZwAAIC1UzryVzesOAAAQOFTOAABAWqiceSuwydkfcqpUkBNxFWtpyfTUf5eZ5nHhhe5jQ3V1prE13N36SVJItvYzrQn3h/XFHY+bxra0ZBr29J2msZPX/dgU72erFcs2t7T7kaSLEw+7jrW0Y5Kkk551v82t27u52RSuoiL3sdZj3NZiybZ/LMeK5RiUpLDlOLQOnhu2xRtY909xfsJ1rLUdU6ij3RRv4982tOzOcPM+17GheLwbs0GQBTY5AwAAPUOOpJycHG/HdBxPx+tJsrlqCAAAEDjm5OyVV17Reeedp7KyMuXk5OjZZ5/t9HPHcXTTTTdp4MCB6tu3ryorK/Xee+95NV8AABA0ubn+LFnKnJy1tLRozJgxWrx4cZc/v/POO3Xfffdp6dKlWr9+vY4++mhVVVUpkXB//gEAAOhBSM48ZV7zqVOnaurUqV3+zHEcLVq0SDfccIPOP/98SdLjjz+u0tJSPfvss5o5c2Z6swUAAOjlPE1Lt2/frrq6OlVWVqYei0ajGj9+vNauXdtlctbW1qa2trbUv+NcdQIAQM+Smyt5fEGAuCDAG3Wf3kqitLS00+OlpaWpn31eTU2NotFoaikvL/dySgAAAD1Kxq/WXLBggRobG1PLzp07Mz0lAABgwTlnnvI0OYvFYpKk+vr6To/X19enfvZ5eXl5ikQinRYAAIBs5WlyNnToUMViMa1evTr1WDwe1/r161VRUeHlnwIAAEHRp4/3VbM+fTK9Vhljrhk2Nzdr69atqX9v375dGzduVHFxsQYPHqyrr75aP/vZz3TiiSdq6NChuvHGG1VWVqZp06Z5OW8AAIBeKcdxbJdDrFmzRt/61rcOeXzWrFl69NFH5TiOFi5cqAcffFANDQ06++yz9cADD+ikk05yNX48Hlc0GtXHHze6/orT0vMtaSwWWnq4JQPU187Cuk38ZO6ZZzgnwbzvs2Cb+7m9Jdt6+rm9g8TWE9THifgsKK/L+5ptr8vFRf7N29oq1a14PK7+/aNqbHT/vunl345Go2osLVUk5O3rWjyZVLS+PiPrlWnm5MxvJGeHmUcWJAoSydmXjeTsy0dydiiSs/QEIjk77jh/krO//z0rk7NgvEMAAABAksc3oQUAAFkoN1fyuHKmZHZU07tC5QwAACBAqJwBAID09Onj/a0vDhzwdrwehMoZAABAgFA5AwAA6fHjprFeN1LvQaicAQAABAiVMwAAkB4qZ54iOQMAAOkhOfMUX2sCAAAESGArZ6HmuOf3s5OkUF2dKT453F1PUMnefuZv71tW0LYxhgwxhZtYWpBY25UU5BvbAxk+qd2wwNaprLnZ/TZftMg0tG9tXCSpudl9bHGR8SUgkTCF+/npL5lfYJiHjzezNO7MXEObtxdesE3lfw1/232wcd7JUaNt8Ya9b38+uN+GlnZMkvSXde7n/bXY30xjWxpJJYcMcx0biDZfVM48ReUMAAAgQIKQbwMAgJ6sT5+AlPB6BypnAAAAAUKaCwAA0pObS+XMQ1TOAAAAAoQ0FwAApIfKmafYkgAAID0kZ57ia00AAIAAIc0FAADp8eNWGo7txuG9CZUzAACAAAls5SxZGFGyMOL9wMN9GPNTtnZM0rBfVbsPvu4609hJlZniLSwfjqwfpCwtXyRbS6Zwja0VyH3/9/+6jk3qctPYfp6aUVTkPta6va2f5vYl3LdYKiy0jW3ZhNb1NDG0Y7IytWOStKljpOtY6/YeYgs38fP5YN33lpZME2a7b7EkScuXu4+NmUYOAD/OOetm5Wzx4sW66667VFdXpzFjxuj+++/XuHHjDhu/YsUK3XjjjXr//fd14okn6o477tC5556b+vns2bP12GOPdfqdqqoqrVy5slvzc4PKGQAA6BWeeuopVVdXa+HChXr99dc1ZswYVVVVac+ePV3G/+Uvf9FFF12kOXPm6I033tC0adM0bdo0bd68uVPclClTtHv37tTy5JNP+roeJGcAACA9BytnXi9G99xzj+bOnavLLrtMI0eO1NKlS1VQUKCHH364y/h7771XU6ZM0Y9+9COdcsopuu222/TVr35Vv/rVrzrF5eXlKRaLpZZjjjmmW5vJLZIzAAAQWPF4vNPS1tbWZVx7e7s2bNigysrK1GOhUEiVlZVau3Ztl7+zdu3aTvHSJ19Zfj5+zZo1GjBggE4++WRdeeWV+uijj9JcqyMjOQMAAOnxsXJWXl6uaDSaWmpqarqcwt69e3XgwAGVlpZ2ery0tFR1dXVd/k5dXd0Xxk+ZMkWPP/64Vq9erTvuuEO1tbWaOnWqDhw4kM4WO6LAXhAAAAB6CB8vCNi5c6cikX9ezJeXl+ft3/kCM2fOTP3/qaeeqtGjR+uEE07QmjVrNHnyZF/+JpUzAAAQWJFIpNNyuOSspKREffr0UX19fafH6+vrFYt1ff1rLBYzxUvSsGHDVFJSoq1btxrXxD2SMwAAkJ6DN6H1cunTxzSFcDissWPHavXq1anHksmkVq9erYqKii5/p6KiolO8JK1ateqw8ZL0wQcf6KOPPtLAgQNN87MgOQMAAL1CdXW1/v3f/12PPfaY3nnnHV155ZVqaWnRZZddJkm69NJLtWDBglT8D3/4Q61cuVK/+MUv9O677+rmm2/Wa6+9pvnz50uSmpub9aMf/Ujr1q3T+++/r9WrV+v888/X8OHDVVVV5dt6cM4ZAABIjx/nnCWT5l+ZMWOGPvzwQ910002qq6vTaaedppUrV6ZO+t+xY4dCoX/Wpb72ta9p2bJluuGGG/TTn/5UJ554op599lmNGjVKktSnTx9t2rRJjz32mBoaGlRWVqZzzjlHt912m6/nvpGcAQCAXmP+/PmpytfnrVmz5pDHLrzwQl144YVdxvft21cvvviil9NzheQMAACkJyCVs94isMlZSEmF5G7HWHqnuR2ze4yn8Fn6ZXZ0mIa2hIdzbdskONtbam52PxdLr0xJUiJhnI17lu1i738ZnBc0S/9GP/sr9ljG5721X6aF9bhq73B/3Fr3vXGzmFg6pVp6ZUpSc7P72FCi1ZdY9Ay8HAIAgPRQOfMUV2sCAAAECJUzAACQHipnniI5AwAA6Tl4E1ov+di7Muj4WhMAACBAqJwBAID0+PG1JpUzAAAABAGVMwAAkB4qZ56icgYAABAgVM4AAEB6qJx5KrDJWfLTBk5ea03YxizId3+flSFDbHNJqsx1rLVdSbjDfTuPZG6BbXADSwsXyd5KatEi97FJXW4a2yLU0W6KT+ZamsTYWLa5uXVXvu1Y8fMFxrTNjW8afrYos4ydHDXaNPYQQ6yf85Zsx5Z57OZ9rmPbC4tNYyeHDHMdGzONbGuztC/h/rkWT/jYzwoZEdjkDAAA9BB+3OfMzyaqAUdyBgAA0uPH15pej9eDmL83fOWVV3TeeeeprKxMOTk5evbZZzv9fPbs2crJyem0TJkyxav5AgAA9GrmtLSlpUVjxozR97//fV1wwQVdxkyZMkWPPPJI6t95eXndnyEAAAg2KmeeMq/51KlTNXXq1CPG5OXlKRZzd6pkW1ub2traUv+Ox+PWKQEAAPQavtznbM2aNRowYIBOPvlkXXnllfroo48OG1tTU6NoNJpaysvL/ZgSAADwy8HKmddLlvI8OZsyZYoef/xxrV69WnfccYdqa2s1depUHTjM/UoWLFigxsbG1LJz506vpwQAANBjeJ6Wzpw5M/X/p556qkaPHq0TTjhBa9as0eTJkw+Jz8vL45w0AAB6Ms4585Tv7ZuGDRumkpISbd261e8/BQAA0OP5npZ+8MEH+uijjzRw4EC//xQAAMgEP25C26ePt+P1IOYt2dzc3KkKtn37dm3cuFHFxcUqLi7WLbfcounTpysWi2nbtm368Y9/rOHDh6uqqsrTiQMAgIDga01Pmdf8tdde07e+9a3Uv6urqyVJs2bN0pIlS7Rp0yY99thjamhoUFlZmc455xzddttt5vPKHnhAys93F1td8rjrcZ/WpaZ5fO97hh57dbtMYydj7ntrhmXs3WjogRi6+SbT2MtG3Oo69uLEw6axk7O/b4q3dPewPs8tvQetvTJD1//YdezDI+40jf39jgddxyb/xdZvtLnZFK7CQvex5l6Phm2eSJiGVkGu++dbu2z73trP1MLUt9N4VoufvTgbGkxDq9jtm4O60T/Wx7N9WuX+dbm40P0xmJu0vT8g+MzJ2cSJE+U4zmF//uKLL6Y1IQAA0MNQOfOU7xcEAAAAwL3sTUsBAIA3qJx5isoZAABAgGRvWgoAALzBrTQ8ReUMAAAgQKicAQCA9HDOmaeyd80BAIA3SM48xdeaAAAAAZK9aSkAAPAGlTNPBXbN5x/7pCJ9+7qKfSrPfUumS0//H+NMhrsP9fFAsrYHsrC0Y5Kki50nXMf+z9m2dkyGre07P9u4WFoyff/RCaax/+fXr7iOtW5vP18r/dzelnZMkkx9wcL5gX0Z9ZR1/1haMhV/sMk0dvuI0a5jg7R3DF2nlDS0BfPz/QGZEaTjFgAA9ERUzjzFOWcAAAABkr1pKQAA8AY3ofUUlTMAAIAAoXIGAADSwzlnnsreNQcAAN4gOfMUX2sCAAAESPampQAAwBtUzjxF5QwAACBAsjctBQAA3uBWGp6icgYAABAgga2cPamL1FcRV7GWfpkPvHSSaR7/MsR9bHjzZtPYmjjJdWhISdPQrQn3effFiYdNY1v6ZZ70rPsekpKUvO7HpvjmZvexRUWmoU3bvL3D9jnn+x0Puo619MqUpJOWu++VmrzhJtPYe/eawjVokPtY6zFu2+a23oOWfpmW55okFeQb1jORMI2t/AJbvIF1/xTnu5+7pVemZCvQhBKtprHbc91vQ2uhyNCyVeGGPa5jQ01Nton4gXPOPEXlDAAAIECyNy0FAADeoHLmqexdcwAA4A2SM0/xtSYAAECAZG9aCgAAvEHlzFNUzgAAAAIke9NSAADgDW5C6ykqZwAAAAFC5QwAAKSHc848ReUMAAAgQHIcx3EyPYnPisfjikaj+vjjRkUi7to3+dlmJ5yIu45NFrqbb3eEOtptv2D4xJEMUI4epPW0tquxCMo293N7S7b19HN7B0lQ9r3fLPvT/Nw0tGTa1WBraVUW82/elm5clqdaPB5X//5RNTa6f9/0ysH37MZ331WkXz9vx25qUnTEiIysV6Zlb80QAAB4g681PZUdH+EAAAB6iOxNSwEAgDe4lYanqJwBAAAECJUzAACQHs458xSVMwAAgADJ3rQUAAB4g8qZp6icAQCAXmPx4sUaMmSI8vPzNX78eL366qtHjF+xYoVGjBih/Px8nXrqqfrd737X6eeO4+imm27SwIED1bdvX1VWVuq9997zcxVIzgAAQJoOVs68XoyeeuopVVdXa+HChXr99dc1ZswYVVVVac+ePV3G/+Uvf9FFF12kOXPm6I033tC0adM0bdo0bd68ORVz55136r777tPSpUu1fv16HX300aqqqlLCcldhIzoEfAE6BHy5grSedAjoAh0C0haUfe83OgQcqld3CPj4Y8//djweV/SYY0zrNX78eJ155pn61a9+JUlKJpMqLy/XVVddpeuvv/6Q+BkzZqilpUUvvPBC6rGzzjpLp512mpYuXSrHcVRWVqZrr71W1113nSSpsbFRpaWlevTRRzVz5kwP1vRQ2fEqAQAAeqR4PN5paWtr6zKuvb1dGzZsUGVlZeqxUCikyspKrV27tsvfWbt2bad4SaqqqkrFb9++XXV1dZ1iotGoxo8ff9gxvRDYs+1C9bsVam12F2z4iBH+TKnSjeTESa5jly41Da0fXGGoFPhYsejoMA1t0uxyFx5UXGQ8JA0fRf38JJLMt306t2wXa2V/7173sYMHGQffuNEWf9pXXYf6WVHytSpn/WrDcKyYh863xfvJz/3Znut+G1oqYZJsG934vPdr/wThvPmkQp7v84PjlZeXd3p84cKFuvnmmw+J37t3rw4cOKDS0tJOj5eWlurdd9/t8m/U1dV1GV9XV5f6+cHHDhfjhwDsUgAAgK7t3Lmz09eaeXl5GZzNl8OU5tbU1OjMM89Uv379NGDAAE2bNk1btmzpFJNIJDRv3jwde+yxKiws1PTp01VfX+/ppAEAQHB0dPizSFIkEum0HC45KykpUZ8+fQ7JOerr6xWLxbr8nVgsdsT4g/+1jOkFU3JWW1urefPmad26dVq1apX279+vc845Ry0tLamYa665Rs8//7xWrFih2tpa7dq1SxdccIHnEwcAADgoHA5r7NixWr16deqxZDKp1atXq6Kiosvfqaio6BQvSatWrUrFDx06VLFYrFNMPB7X+vXrDzumF0xfa65cubLTvx999FENGDBAGzZs0IQJE9TY2KiHHnpIy5Yt06RJn5yr9cgjj+iUU07RunXrdNZZZ3k3cwAAEAifrXR5OaZVdXW1Zs2apTPOOEPjxo3TokWL1NLSossuu0ySdOmll+q4445TTU2NJOmHP/yhvvnNb+oXv/iFvvOd72j58uV67bXX9OCDD0qScnJydPXVV+tnP/uZTjzxRA0dOlQ33nijysrKNG3aNK9W9RBpnXPW2NgoSSouLpYkbdiwQfv37+90VcOIESM0ePBgrV27tsvkrK2trdOVF/G4+1tXAAAAHDRjxgx9+OGHuummm1RXV6fTTjtNK1euTJ3Qv2PHDoVC//zS8Gtf+5qWLVumG264QT/96U914okn6tlnn9WoUaNSMT/+8Y/V0tKiyy+/XA0NDTr77LO1cuVK5ft4BU63k7NkMqmrr75aX//611MrUVdXp3A4rKKiok6xR7qqoaamRrfcckt3pwEAADIsKJUzSZo/f77mz5/f5c/WrFlzyGMXXnihLrzwwsOOl5OTo1tvvVW33npr9ybUDd2+7nXevHnavHmzli9fntYEFixYoMbGxtSyc+fOtMYDAABfLj8vCMhG3aqczZ8/Xy+88IJeeeUVDRo0KPV4LBZTe3u7GhoaOlXPjnRVQ15eXlZcFgsAAOCGqXLmOI7mz5+vZ555Ri+//LKGDh3a6edjx47VUUcd1emqhi1btmjHjh2+XtUAAAAy58AB76tmBw5keq0yx1Q5mzdvnpYtW6bnnntO/fr1S51HFo1G1bdvX0WjUc2ZM0fV1dUqLi5WJBLRVVddpYqKCq7UBAAAcMGUnC1ZskSSNHHixE6PP/LII5o9e7Yk6Ze//KVCoZCmT5+utrY2VVVV6YEHHvBksgAAIHiCdEFAb5DjOI6T6Ul81sEO9x9/7L4LfVBY+/f52XsuW4QSra5j9yVsffAKC93HBqG3XRD52dOS58+hQs2GWxFZDnCxvb1geT5Ytnc8Htcxx0TV2Pjlv28efM9+991G9evn7d9uaoprxIjMrFem8ZYCAADSQuXMW3wUAgAACBAqZwAAIC1UzrxF5QwAACBAqJwBAIC0HLzPmddjZiuSMwAAkBa+1vQWX2sCAAAECJUzAACQFipn3qJyBgAAECBUzgAAQFqonHkrsMlZqKNdoY52V7HJ3LD7ca3tZCxHh489fNxui4Pa5X6bhHP9azvlZ/seK2O3Gl9bMvnVxsU6tpWvLXysr8SG533WsB7kBtbjqjXh/ljJz7fNxXKoWA8ry1z83CYFct+aztLGDj1DYJMzAADQM1A58xbnnAEAAAQIlTMAAJAWbkLrLZIzAACQFr7W9BZfawIAAAQIlTMAAJAWKmfeonIGAAAQIFTOAABAWqiceYvKGQAAQIBQOQMAAGnhVhreonIGAAAQIIGtnCVzw6aemW5ZeptJUkG++03kZ99BS69MSQp3uO+1lswtsE7HtfYO2zYx9/nMdz93Pw92a+9TP47tgyzb3Lq9rUzPCeM2MW1zY6NUP/vH+vk6EaR5F+S63z9J6+tbwx73wUUDTGNbmLeJoV9mfZP717am5syfnMU5Z94KbHIGAAB6BpIzb/G1JgAAQIBQOQMAAGmhcuYtKmcAAAABQuUMAACkhcqZt6icAQAABAiVMwAAkBZuQustKmcAAAABQuUMAACkhXPOvEVyBgAA0kJy5q3AJmehRKtCYXfTi3e4b3MRyXXfPuMT+a4jrQeSpaOMsfuMlOvfvC0tf3Jz/f3m3NKCxs+2OdYdlEi4j7W0wfmEf62hgtSqyLTNzU9O/7Zh1jAc5KFC4wtcSYnrUOtLp6+vKYZjtn9/98Pm5dmmgeALbHIGAAB6Bipn3uKCAAAAgAChcgYAANLCrTS8ReUMAAAgQKicAQCAtHDOmbeonAEAAAQIlTMAAJAWKmfeIjkDAABpITnzFl9rAgAABAiVMwAAkBYqZ96icgYAABAgga2cxTsKJJc9MyOJPa7HTZYM6O6UvpCfvTWtLD3fLL0yJam9w/3Yfq5jkFh77Jn6ZRoPrHB+dmx00zY39soMJQw9ePPd97ENEl/7nkpSYaHrUOtcLE8J6+ubaXDjcZXsxT1buQmtt6icAQAABIgpOaupqdGZZ56pfv36acCAAZo2bZq2bNnSKWbixInKycnptFxxxRWeThoAAATHwXPOvF6ylSk5q62t1bx587Ru3TqtWrVK+/fv1znnnKOWlpZOcXPnztXu3btTy5133unppAEAAHor08kpK1eu7PTvRx99VAMGDNCGDRs0YcKE1OMFBQWKxWLezBAAAARaR4fUp4/3Y2artM45a2xslCQVFxd3evyJJ55QSUmJRo0apQULFqi19fAn17a1tSkej3daAABAz8HXmt7q9mVdyWRSV199tb7+9a9r1KhRqccvvvhiHX/88SorK9OmTZv0k5/8RFu2bNFvf/vbLsepqanRLbfc0t1pAAAA9CrdTs7mzZunzZs3609/+lOnxy+//PLU/5966qkaOHCgJk+erG3btumEE044ZJwFCxaouro69e94PK7y8vLuTgsAAHzJuJWGt7qVnM2fP18vvPCCXnnlFQ0aNOiIsePHj5ckbd26tcvkLC8vT3l5ed2ZBgAAQK9jSs4cx9FVV12lZ555RmvWrNHQoUO/8Hc2btwoSRo4cGC3JggAAIKto0MKeXznVM45c2nevHlatmyZnnvuOfXr1091dXWSpGg0qr59+2rbtm1atmyZzj33XB177LHatGmTrrnmGk2YMEGjR4/2ZQUAAAB6E1NytmTJEkmf3Gj2sx555BHNnj1b4XBYL730khYtWqSWlhaVl5dr+vTpuuGGG8wTi+S2KuKy94+lJVOo2Xg1qKEFSVOT7WODn11fQh3u2wNZW4pYWjJZP/mErV+0+9hqxSIkYwssuZ+LtR1Ta8L9cViQb2xtk0jY4vPdtWDrDus2N7E8Oc1929zvez9bwlm3n5/tnsyvEzK8vhmea5K/bXMs29zyPLY+Lf1A5cxb5q81j6S8vFy1tbVpTQgAACCbZUeHZAAA4BsqZ94iOQMAAGkhOfOWn1+vAwAAwIjkDAAApOXgTWi9XPy+Ce2+fft0ySWXKBKJqKioSHPmzFFzc/MRfyeRSGjevHk69thjVVhYqOnTp6u+vr5TTE5OziHL8uXLTXMjOQMAAFnnkksu0VtvvaVVq1albqz/2S5HXbnmmmv0/PPPa8WKFaqtrdWuXbt0wQUXHBL3yCOPaPfu3all2rRpprlxzhkAAEhLR4eUk+P9mNInbR0/y4vOQu+8845Wrlypv/71rzrjjDMkSffff7/OPfdc3X333SorKzvkdxobG/XQQw9p2bJlmjRpkqRPkrBTTjlF69at01lnnZWKLSoqUiwW6/b8qJwBAIDAKi8vVzQaTS01NTVpj7l27VoVFRWlEjNJqqysVCgU0vr167v8nQ0bNmj//v2qrKxMPTZixAgNHjxYa9eu7RQ7b948lZSUaNy4cXr44Ye/8FZkn0flDAAApMXPytnOnTsViURSj3vRj7uurk4DBnS+gX1ubq6Ki4tT3Y+6+p1wOKyioqJOj5eWlnb6nVtvvVWTJk1SQUGB/vCHP+gHP/iBmpub9a//+q+u50dyBgAAAisSiXRKzo7k+uuv1x133HHEmHfeeceLaR3WjTfemPr/008/XS0tLbrrrrtIzgAAwJfHz8qZxbXXXqvZs2cfMWbYsGGKxWLas2fP5/5eh/bt23fYc8VisZja29vV0NDQqXpWX19/xPPLxo8fr9tuu01tbW2uq37BTc7y8/1pPmnolSnZ+smV9g9OrzpLv0w/exRae2W2d9i2Sa6P/TItrPsynOvfNrf0yzQfgz72yrTy8/ljYjwGLX1vw5ZmmfJ3m/jZi9P6fLD0y7Rsb8nea9jEkG0UGHrqdrT72GfW7RwCkpz1799f/fv3/8K4iooKNTQ0aMOGDRo7dqwk6eWXX1YymdT48eO7/J2xY8fqqKOO0urVqzV9+nRJ0pYtW7Rjxw5VVFQc9m9t3LhRxxxzjOnr2OAmZwAAAD445ZRTNGXKFM2dO1dLly7V/v37NX/+fM2cOTN1pebf//53TZ48WY8//rjGjRunaDSqOXPmqLq6WsXFxYpEIrrqqqtUUVGRulLz+eefV319vc466yzl5+dr1apV+rd/+zddd911pvmRnAEAgLQcOOB95czvm9A+8cQTmj9/viZPnqxQKKTp06frvvvuS/18//792rJli1pbW1OP/fKXv0zFtrW1qaqqSg888EDq50cddZQWL16sa665Ro7jaPjw4brnnns0d+5c09xyHOv1nT6Lx+OKRqNq/Phj1ycAWsrnfpbm/RzbT35+rWll/1rTp4n4LCjbPCjHYDYxfc3m49eafh+DQXntDNLXmn7t+3g8rugxx6ixsdH1+6ZXDr5nf+tbjcrN9fZvd3TE9cc/RjOyXpnWQ9/aAABAUPjRpJzG5wAAAAgEKmcAACAtVM68ReUMAAAgQKicAQCAtFA58xaVMwAAgAChcgYAANLixz3J/L7PWZAFNjlr7wi5vueVpfTZ1GQrFppaMjU0mMYO+dGe6lPJoLTZMdalre2YXnjBfez/Gv62aWzL3JOjRtvG9lMi4T7WeJxYhpb86cAWNNavXkwtmd5/3zR2c8kwQ7TttdDY+c527zLjRrTM3HrfMsu9yFo7rPdEcx9v6N4UiPsVdnRIXt81NZuTs8zvUQAAAKQEtnIGAAB6Bipn3qJyBgAAECBUzgAAQFqonHmLyhkAAECAUDkDAABpoXLmLSpnAAAAAULlDAAApOXAAe8rZ0nDrfJ6G5IzAACQlo4OKeTxd3HZnJzxtSYAAECAUDkDAABpoXLmrcAmZ7m5nyxuY92y9vqz9Cwz98o0xLvtM3qQZcf62pfN2NfOytIvc1PHSNPYll6CQ0wj+7zNfeyraj3EQ81x98HG5o1B6Cco2V5/JNu8bb0ypcjyB90Hf+97prGTsh1Xpv3j8+uEhaVfprH1qUYOd9+3M2now4neJ7DJGQAA6BmonHkrGB89AQAAIInKGQAASNOBA95Xury+NUdPQuUMAAAgQKicAQCAtHR0SDk53o6ZzZUzkjMAAJAWkjNv8bUmAABAgFA5AwAAaaFy5i0qZwAAAAFC5QwAAKSFypm3SM4yyNKSqaPDNnY41/0NZ4LSBqdbDBvG2B3IJCTbDX569Da38HGjW7e5haltm4/zMH+5YWzJ5KdQh6FVkbF9k2mbW188DW2TLO2YJEmJhOvQjnz38zCvIgKP5AwAAKQpKcfx+oNK9vZvypKP7wAAAD2DKTlbsmSJRo8erUgkokgkooqKCv3+979P/TyRSGjevHk69thjVVhYqOnTp6u+vt7zSQMAgCA54NOSnUzJ2aBBg3T77bdrw4YNeu211zRp0iSdf/75euuttyRJ11xzjZ5//nmtWLFCtbW12rVrly644AJfJg4AAIKC5MxLpnPOzjvvvE7//vnPf64lS5Zo3bp1GjRokB566CEtW7ZMkyZNkiQ98sgjOuWUU7Ru3TqdddZZ3s0aAACgl+r2BQEHDhzQihUr1NLSooqKCm3YsEH79+9XZWVlKmbEiBEaPHiw1q5de9jkrK2tTW1tbal/x+Px7k4JAABkhB+VruytnJkvCHjzzTdVWFiovLw8XXHFFXrmmWc0cuRI1dXVKRwOq6ioqFN8aWmp6urqDjteTU2NotFoaikvLzevBAAAQG9hTs5OPvlkbdy4UevXr9eVV16pWbNm6e233+72BBYsWKDGxsbUsnPnzm6PBQAAMiHp05KdzF9rhsNhDR8+XJI0duxY/fWvf9W9996rGTNmqL29XQ0NDZ2qZ/X19YrFYocdLy8vT3l5efaZAwAA9EJp3+csmUyqra1NY8eO1VFHHaXVq1enfrZlyxbt2LFDFRUV6f4ZAAAQWFyt6SVT5WzBggWaOnWqBg8erKamJi1btkxr1qzRiy++qGg0qjlz5qi6ulrFxcWKRCK66qqrVFFRwZWaAAAALpmSsz179ujSSy/V7t27FY1GNXr0aL344ov69re/LUn65S9/qVAopOnTp6utrU1VVVV64IEHujWxkJKu+6eZ+uAZ+r1Jtp5vyfwC09iWjW/plSnZ+nZax/az76C152Ry1GjXsUNMI9vmbp13Tx3bKkjHil/83D/W1qRJ2V6DLMz7x9gv06I14X6bF+Tbzt6xhCcNfTglW7/McEerL7H+Scr7Slf2nnOW4zjB6vsej8cVjUbV+PHHikQirn4nKMmZn6wvjNmSnPmppyZQQUrOLHrysWLB/kmfoX+4CvKDM29Lg3JLwhWPxxUdOFCNjY2u3ze9cvA9W9oqqZ/HozdJGp6R9cq04DzzAQAA0P2b0AIAAHyCm9B6icoZAABAgFA5AwAAafLjprHZe0EAlTMAAIAAoXIGAADSxDlnXqJyBgAAECBUzgAAQJqonHmJ5AwAAKSJ5MxLfK0JAAAQIIGtnNX+d0hHH+0ud5x0Rtz1uDsabC0gBg/y71JeS5uQeLMtj47kW9pU2Q6Dhgb3scX5hj4rkkL5+aZ4P1utBKUFlmV7S7ZtHqTtbemXKEkFuYZj3NLvR7I3tTQISvszv1vZhf7jcfdjf+9S09gFe3e4H3vQYNPYfgrXuZ93vMj9vOOWvlC+obeml6icAQAABEhgK2cAAKCn4Ca0XqJyBgAAECBUzgAAQJq4WtNLVM4AAAAChMoZAABIE5UzL5GcAQCANJGceYmvNQEAAAKEyhkAAEgTlTMvUTkDAAAIkMBWzr55elwRl52W9nW4b8nkZzsmc4uY/ALXodZuMkm5b7Vibg/Uscf9PPIH2AY3snQtyTUe7ZZ4a1sjUwusDzaZxm4fMdp1bJBeAIydpEzHeKjQtqaW/WntnGNq9WUd3NBiydqOycrSkilUt8s2eEmJcTbuWV7GzcesoZWU5SU/GYh7tTry/qaxjsfj9RxUzgAAQNbZt2+fLrnkEkUiERUVFWnOnDlqbm4+4u88+OCDmjhxoiKRiHJyctTQxSft7oz7eSRnAAAgTQd8WvxzySWX6K233tKqVav0wgsv6JVXXtHll19+xN9pbW3VlClT9NOf/tTTcT8vSN9qAAAAdBKPxzv9Oy8vT3l5eWmN+c4772jlypX661//qjPOOEOSdP/99+vcc8/V3XffrbKysi5/7+qrr5YkrVmzxtNxP4/KGQAASJN/lbPy8nJFo9HUUlNTk/Zs165dq6KiolQCJUmVlZUKhUJav359xselcgYAANLk3600du7cqchnrhBMt2omSXV1dRowoPMFa7m5uSouLlZdXV3Gx6VyBgAAAisSiXRajpScXX/99crJyTni8u67736Js+8eKmcAACBNwbgJ7bXXXqvZs2cfMWbYsGGKxWLas6fzbaE6Ojq0b98+xWIx8989yKtxSc4AAECv0L9/f/Xv3/8L4yoqKtTQ0KANGzZo7NixkqSXX35ZyWRS48eP7/bf92pcvtYEAABpSvq0+OOUU07RlClTNHfuXL366qv685//rPnz52vmzJmpKyr//ve/a8SIEXr11VdTv1dXV6eNGzdq69atkqQ333xTGzdu1L59+1yP6wbJGQAAyDpPPPGERowYocmTJ+vcc8/V2WefrQcffDD18/3792vLli1qbW1NPbZ06VKdfvrpmjt3riRpwoQJOv300/Wf//mfrsd1I8dxnED1R4jH44pGo2r83NUZR2Jp31RcFJz2TUlD+yY/+dq+qaTntm/yU2DaNwVom/gpZPwE3lPbN/ndkskv5vZNRUWuQ62vs362b/JLPB7XMcdE1djY6Pp908u/HY1GJT0t6WiPR2+R9P9lZL0yLbAvzTsbI+qXdLczLP0yX/mTrVg4YdQ+98E+vtMZOz8osvdvrmOLjSc/WvplhjraTWPb+fdmZHlfDDcbjhNJxYZXdUuyJdkOw1Ci9YuDPjuXXNsbnWUu5iSnwf2HBGsvRtO+l+0YN/UENY1sY01YWxO22RTs3eE+2Nor8+mnXYduPct9j09JGjTINhULSxL68rvuvwJraenObBBkgU3OAABAT5GU91drBqKje0ZwzhkAAECAUDkDAABpCsZ9znoLkjMAAJAmP259wdeaAAAACAAqZwAAIE18reklKmcAAAABQuUMAACkicqZl6icAQAABAiVMwAAkCYqZ14KbHJWflxSkYj3l9Ga2jFJ2vRBsevYUaOss/FPa2yY69iC/OBcrryv2daOydIr1dIv0aq90P1xItn6K1qfpJaWTLsabO2YymLB6VGpIvdtxKzb0LJ/LO2YJFtLM197ZRo3eEG+bSsmBw02xVtYWjKd9NcnbIMPv8h1qJ+vKZNOc/9eFY/HfZsHMiOwyRkAAOgpaN/kJZIzAACQJm5C6yVTTXbJkiUaPXq0IpGIIpGIKioq9Pvf/z7184kTJyonJ6fTcsUVV3g+aQAAgN7KVDkbNGiQbr/9dp144olyHEePPfaYzj//fL3xxhv6yle+IkmaO3eubr311tTvFBTYzmkBAAA9DRcEeMmUnJ133nmd/v3zn/9cS5Ys0bp161LJWUFBgWKxmOsx29ra1NbWlvo3JzYCAIBs1u1LTQ4cOKDly5erpaVFFRUVqcefeOIJlZSUaNSoUVqwYIFaW4985VhNTY2i0WhqKS8v7+6UAABARhzwaclO5gsC3nzzTVVUVCiRSKiwsFDPPPOMRo4cKUm6+OKLdfzxx6usrEybNm3ST37yE23ZskW//e1vDzveggULVF1dnfp3PB4nQQMAAFnLnJydfPLJ2rhxoxobG/X0009r1qxZqq2t1ciRI3X55Zen4k499VQNHDhQkydP1rZt23TCCSd0OV5eXp7y8vK6vwYAACDDOOfMS+avNcPhsIYPH66xY8eqpqZGY8aM0b333ttl7Pjx4yVJW7duTW+WAAAAWSLt+5wlk8lOJ/R/1saNGyVJAwcOTPfPAACAwKJy5iVTcrZgwQJNnTpVgwcPVlNTk5YtW6Y1a9boxRdf1LZt27Rs2TKde+65OvbYY7Vp0yZdc801mjBhgkaPHu3X/AEAQMZxE1ovmZKzPXv26NJLL9Xu3bsVjUY1evRovfjii/r2t7+tnTt36qWXXtKiRYvU0tKi8vJyTZ8+XTfccINfc/+nRMJ9bK6tWGjpl/n++6ahNSzmvgdixFjjTOa7v7+ctT+crSWff70yJekv69zP/Wuxv5nGtsw8OcR9L1PJ5z6fue73vbVXpum5JkmG4zA/3za0Rcj6Im84yK170tIv09KHU5JaOyxHre25aWytaWI9rAYNMgQbemVKkj74wHVoosTYP7SozHWo5fmQDNHsp7cx7dGHHnrosD8rLy9XbW1t2hMCAAA9Db01veTfx3cAAACYUQsFAABp4oIAL1E5AwAACBAqZwAAIE1UzrxE5QwAACBAqJwBAIA0UTnzEskZAABIE8mZl/haEwAAIEConAEAgDTRvslLgU3Okgq5b3FjaBHjJ0s7Jkl65TX38z7rLNtc/Nyxxg5YJta2RpaWTBNm21osLV/uPjZmGtlflv1jbiPl43PN2mLJMnfzehpaLPnJ1o7J1kJu5HBba6iksd2ThZ+tu6z73tKSqeDX99kmc8UVrkP93N4IvsAmZwAAoKegfZOXOOcMAAAgQKicAQCANHG1ppeonAEAAAQIlTMAAJAmKmdeIjkDAABp4lYaXuJrTQAAgAChcgYAANLE15peonIGAAAQIFTOAABAmqiceYnKGQAAQIAEtnLW3CyFXKaOhYW2cf0SMW5NS7/MvXttY5fl73Md215YbBrb0ruxo8M0tJml+5ylV6ZkO1ZCCVtf1Va571Fp7Tto2ebW/WOdi6VfZmvC9lmxQIZtbmwImzT01rT2BLVtdFt/RVO/zETCNHZHvm0u4bodrmOTg9z3s5SkUN0uU7xJUZn7WEOvTEnS5s2uQ98v+qrr2KYm2zT8QeXMS1TOAAAAAiSwlTMAANBT0PjcSyRnAAAgTdyE1kt8rQkAABAgVM4AAECauCDAS1TOAAAAAoTKGQAASBOVMy9ROQMAAAgQKmcAACBNVM68ROUMAAAgQAJbOYvktyuS764VSdLQ4iSy92+mebTGhrmOTea7b8kj2Ta+pR2TJO1odt+SaXCR7V4ySUNOH262zdvaSio5xP3+iZlGtrVk2pew7fviQvdtdizHtySFG/a4Dy4aYBrbynKsmNoxSapvcr/N+/c3DW1ibjuV7/6ZbwiVZDtWzO2YOmz7J17kviWToQOfJOnld923WJp0mu01yNKizPrctLRkGvb+y65j4y0tpnn4g8qZlwKbnAEAgJ6CDgFe4mtNAACAAKFyBgAA0kT7Ji9ROQMAAAgQKmcAACBNB+R9vSd7LwigcgYAABAgVM4AAECaqJx5icoZAABAgFA5AwAAaaJy5iWSMwAAkCZupeGlwCVnjuNIkuJNTa5/J5nrvoVGyDCuJLUWxF3HtrvvyGMWirufhyQ1tbjftfG4f+2brPNuT9oOyVwfj2BL+6Z4osM0dm7S0L7JcHxLtmO8PWToVaPgbG9Jamp2v83z8qyzcS+RsMV3tLt/vlmea1YdtkPW3r7J8AeSxvdgS7eiuPE1KBny7yC3vP1YWjLFWz/ZNwffPzPDeEBlbMyeIXDJWdOnR2/5MPc9EwEAyHZNTU2KRqNf6t8Mh8OKxWKqq3vJl/FjsZjCYdsH1N4gx8lsqn2IZDKpXbt2qV+/fsrJyUk9Ho/HVV5erp07dyoSiWRwhv5iPXuPbFhHifXsbbJhPXvbOjqOo6amJpWVlSkU+vKv80skEmr36aujcDisfEs3+l4icJWzUCikQYMGHfbnkUikVzyZvgjr2XtkwzpKrGdvkw3r2ZvW8cuumH1Wfn5+ViZQfuJWGgAAAAFCcgYAABAgPSY5y8vL08KFC5Xn52VXAcB69h7ZsI4S69nbZMN6ZsM6omcL3AUBAAAA2azHVM4AAACyAckZAABAgJCcAQAABAjJGQAAQICQnAEAAARIj0nOFi9erCFDhig/P1/jx4/Xq6++mukpeermm29WTk5Op2XEiBGZnlZaXnnlFZ133nkqKytTTk6Onn322U4/dxxHN910kwYOHKi+ffuqsrJS7733XmYmm4YvWs/Zs2cfsm+nTJmSmcl2U01Njc4880z169dPAwYM0LRp07Rly5ZOMYlEQvPmzdOxxx6rwsJCTZ8+XfX19Rmacfe4Wc+JEycesj+vuOKKDM24e5YsWaLRo0en7pBfUVGh3//+96mf94Z9KX3xevaGfYneqUckZ0899ZSqq6u1cOFCvf766xozZoyqqqq0Z8+eTE/NU1/5yle0e/fu1PKnP/0p01NKS0tLi8aMGaPFixd3+fM777xT9913n5YuXar169fr6KOPVlVVlRKJxJc80/R80XpK0pQpUzrt2yeffPJLnGH6amtrNW/ePK1bt06rVq3S/v37dc4556ilpSUVc8011+j555/XihUrVFtbq127dumCCy7I4Kzt3KynJM2dO7fT/rzzzjszNOPuGTRokG6//XZt2LBBr732miZNmqTzzz9fb731lqTesS+lL15PqefvS/RSTg8wbtw4Z968eal/HzhwwCkrK3NqamoyOCtvLVy40BkzZkymp+EbSc4zzzyT+ncymXRisZhz1113pR5raGhw8vLynCeffDIDM/TG59fTcRxn1qxZzvnnn5+R+fhlz549jiSntrbWcZxP9t1RRx3lrFixIhXzzjvvOJKctWvXZmqaafv8ejqO43zzm990fvjDH2ZuUj455phjnF//+te9dl8edHA9Haf37kv0fIGvnLW3t2vDhg2qrKxMPRYKhVRZWam1a9dmcGbee++991RWVqZhw4bpkksu0Y4dOzI9Jd9s375ddXV1nfZrNBrV+PHje91+laQ1a9ZowIABOvnkk3XllVfqo48+yvSU0tLY2ChJKi4uliRt2LBB+/fv77Q/R4wYocGDB/fo/fn59TzoiSeeUElJiUaNGqUFCxaotbU1E9PzxIEDB7R8+XK1tLSooqKi1+7Lz6/nQb1pX6L3yM30BL7I3r17deDAAZWWlnZ6vLS0VO+++26GZuW98ePH69FHH9XJJ5+s3bt365ZbbtE3vvENbd68Wf369cv09DxXV1cnSV3u14M/6y2mTJmiCy64QEOHDtW2bdv005/+VFOnTtXatWvVp0+fTE/PLJlM6uqrr9bXv/51jRo1StIn+zMcDquoqKhTbE/en12tpyRdfPHFOv7441VWVqZNmzbpJz/5ibZs2aLf/va3GZyt3ZtvvqmKigolEgkVFhbqmWee0ciRI7Vx48ZetS8Pt55S79mX6H0Cn5xli6lTp6b+f/To0Ro/fryOP/54/eY3v9GcOXMyODOka+bMman/P/XUUzV69GidcMIJWrNmjSZPnpzBmXXPvHnztHnz5h5/TuQXOdx6Xn755an/P/XUUzVw4EBNnjxZ27Zt0wknnPBlT7PbTj75ZG3cuFGNjY16+umnNWvWLNXW1mZ6Wp473HqOHDmy1+xL9D6B/1qzpKREffr0OeRKofr6esVisQzNyn9FRUU66aSTtHXr1kxPxRcH91227VdJGjZsmEpKSnrkvp0/f75eeOEF/fGPf9SgQYNSj8diMbW3t6uhoaFTfE/dn4dbz66MHz9eknrc/gyHwxo+fLjGjh2rmpoajRkzRvfee2+v25eHW8+u9NR9id4n8MlZOBzW2LFjtXr16tRjyWRSq1ev7nTeQG/T3Nysbdu2aeDAgZmeii+GDh2qWCzWab/G43GtX7++V+9XSfrggw/00Ucf9ah96ziO5s+fr2eeeUYvv/yyhg4d2unnY8eO1VFHHdVpf27ZskU7duzoUfvzi9azKxs3bpSkHrU/u5JMJtXW1tZr9uXhHFzPrvSWfYleINNXJLixfPlyJy8vz3n00Uedt99+27n88sudoqIip66uLtNT88y1117rrFmzxtm+fbvz5z//2amsrHRKSkqcPXv2ZHpq3dbU1OS88cYbzhtvvOFIcu655x7njTfecP7f//t/juM4zu233+4UFRU5zz33nLNp0ybn/PPPd4YOHer84x//yPDMbY60nk1NTc51113nrF271tm+fbvz0ksvOV/96ledE0880UkkEpmeumtXXnmlE41GnTVr1ji7d+9OLa2tramYK664whk8eLDz8ssvO6+99ppTUVHhVFRUZHDWdl+0nlu3bnVuvfVW57XXXnO2b9/uPPfcc86wYcOcCRMmZHjmNtdff71TW1vrbN++3dm0aZNz/fXXOzk5Oc4f/vAHx3F6x750nCOvZ2/Zl+idekRy5jiOc//99zuDBw92wuGwM27cOGfdunWZnpKnZsyY4QwcONAJh8POcccd58yYMcPZunVrpqeVlj/+8Y+OpEOWWbNmOY7zye00brzxRqe0tNTJy8tzJk+e7GzZsiWzk+6GI61na2urc8455zj9+/d3jjrqKOf444935s6d2+M+WHS1fpKcRx55JBXzj3/8w/nBD37gHHPMMU5BQYHz3e9+19m9e3fmJt0NX7SeO3bscCZMmOAUFxc7eXl5zvDhw50f/ehHTmNjY2YnbvT973/fOf74451wOOz079/fmTx5cioxc5zesS8d58jr2Vv2JXqnHMdxnC+vTgcAAIAjCfw5ZwAAANmE5AwAACBASM4AAAAChOQMAAAgQEjOAAAAAoTkDAAAIEBIzgAAAAKE5AwAACBASM4AAAAChOQMAAAgQEjOAAAAAuT/B4QzKYwPRTNFAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "imshow_zero_center(h_mat)" ] }, { "cell_type": "markdown", "metadata": { "id": "13fBswmtQes4" }, "source": [ "ニュートン法の更新ステップを以下に示します。" ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:25.707998Z", "iopub.status.busy": "2022-12-14T20:58:25.707457Z", "iopub.status.idle": "2022-12-14T20:58:25.713538Z", "shell.execute_reply": "2022-12-14T20:58:25.712986Z" }, "id": "3DdnbynBdSor" }, "outputs": [], "source": [ "eps = 1e-3\n", "eye_eps = tf.eye(h_mat.shape[0])*eps" ] }, { "cell_type": "markdown", "metadata": { "id": "-zPdtyoWeUeV" }, "source": [ "注意: [実際に行列を反転させないでください](https://www.johndcook.com/blog/2010/01/19/dont-invert-that-matrix/)。" ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:25.716695Z", "iopub.status.busy": "2022-12-14T20:58:25.716203Z", "iopub.status.idle": "2022-12-14T20:58:25.815562Z", "shell.execute_reply": "2022-12-14T20:58:25.814844Z" }, "id": "k1LYftgmswOO" }, "outputs": [], "source": [ "# X(k+1) = X(k) - (∇²f(X(k)))^-1 @ ∇f(X(k))\n", "# h_mat = ∇²f(X(k))\n", "# g_vec = ∇f(X(k))\n", "update = tf.linalg.solve(h_mat + eye_eps, g_vec)\n", "\n", "# Reshape the update and apply it to the variable.\n", "_ = layer1.kernel.assign_sub(tf.reshape(update, layer1.kernel.shape))" ] }, { "cell_type": "markdown", "metadata": { "id": "pF6qjlHKWxF4" }, "source": [ "これは、単一の`tf.Variable`の場合には比較的簡単ですが、これを複雑なモデルに適用するには、完全なヘッセ行列を複数の変数にわたって生成するために、注意深い連結とスライスが必要です。" ] }, { "cell_type": "markdown", "metadata": { "id": "PQWM0uN-GO5t" }, "source": [ "### ヤコビアンをバッチ処理する" ] }, { "cell_type": "markdown", "metadata": { "id": "hKtB3rY6EySJ" }, "source": [ "場合により、複数のソースの単一スタックに対して、複数のターゲットの各スタックのヤコビアンを取得して、ターゲットとソースの各ペアのヤコビアンを独立させる必要があることがあります。\n", "\n", "たとえば、ここでは入力 `x` は `(batch, ins)` の形状をしており、出力 `y` は `(batch, outs)` の形状をしています。\n" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:25.819670Z", "iopub.status.busy": "2022-12-14T20:58:25.819367Z", "iopub.status.idle": "2022-12-14T20:58:25.835172Z", "shell.execute_reply": "2022-12-14T20:58:25.834605Z" }, "id": "tQMndhIUHMes" }, "outputs": [ { "data": { "text/plain": [ "TensorShape([7, 6])" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x = tf.random.normal([7, 5])\n", "\n", "layer1 = tf.keras.layers.Dense(8, activation=tf.nn.elu)\n", "layer2 = tf.keras.layers.Dense(6, activation=tf.nn.elu)\n", "\n", "with tf.GradientTape(persistent=True, watch_accessed_variables=False) as tape:\n", " tape.watch(x)\n", " y = layer1(x)\n", " y = layer2(y)\n", "\n", "y.shape" ] }, { "cell_type": "markdown", "metadata": { "id": "Ff2spRHEJXBU" }, "source": [ "`x` に対する `y` の完全なヤコビアンは、たとえ `(batch, ins, outs)` だけが必要な場合でも、`(batch, ins, batch, outs)` の形状をしています。" ] }, { "cell_type": "code", "execution_count": 31, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:25.838541Z", "iopub.status.busy": "2022-12-14T20:58:25.838039Z", "iopub.status.idle": "2022-12-14T20:58:25.948214Z", "shell.execute_reply": "2022-12-14T20:58:25.947610Z" }, "id": "1zSl2A5-HhMH" }, "outputs": [ { "data": { "text/plain": [ "TensorShape([7, 6, 7, 5])" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "j = tape.jacobian(y, x)\n", "j.shape" ] }, { "cell_type": "markdown", "metadata": { "id": "UibJijPLJrpQ" }, "source": [ "スタック内の各アイテムの勾配が独立している場合は、このテンソルの`(batch, batch)`スライスはすべて対角行列になります。" ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:25.951430Z", "iopub.status.busy": "2022-12-14T20:58:25.950956Z", "iopub.status.idle": "2022-12-14T20:58:26.184264Z", "shell.execute_reply": "2022-12-14T20:58:26.183569Z" }, "id": "ZFl9uj3ueVSH" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAl4AAAIQCAYAAABQc2CaAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA/tElEQVR4nO3deXxU9b3/8fckkAlCFpYsLIGAKGGRxQAxKiCSK6CiKMpy8YqRB7Y2USAuP7GV4FKDRREXBPFetL2CWG2h4FWUhu1WwhZKBaQKiJewJAGBDAkSIHN+f1BGhyQnGThzJmFez8fjPErOnPmezxms+fg+3/keh2EYhgAAAOB3IYEuAAAAIFjQeAEAANiExgsAAMAmNF4AAAA2ofECAACwCY0XAACATWi8AAAAbELjBQAAYBMaLwAAAJvQeCEoud1udevWTb/97W89+6ZNmyaHw6EjR44EsLKfrF69Wg6HQx9//LFlY37//fdyOBx6+eWXLRvzUiUmJur222+v8bjly5erSZMmOnz4sGXnvummm3TTTTd5fj7/+bz33nuWnQMAfo7GC/XWW2+9JYfDoZSUFJ/f+8EHH6igoECZmZl+qOwnb7311mX/S3zdunWaNm2ajh8/7tfzDBkyRB07dlROTo5fzwMA/kTjhXprwYIFSkxM1MaNG7V7926f3jtjxgyNHj1aUVFRfqrunGBpvJ599lm/N16S9Itf/EJvv/22Tpw44Zfx27Vrpx9//FH/8R//4ZfxAYDGC/XS3r17tW7dOs2cOVMxMTFasGBBrd/797//Xf/4xz80cuRIP1YIfxgxYoTKy8v10Ucf+WV8h8Oh8PBwhYaG+mV8AKDxQr20YMECNW3aVLfddpvuuecenxqvJUuWKCwsTP3796/y9SNHjmjkyJGKjIxU8+bNNXHiRJ06dcrrmHfffVc333yzYmNj5XQ61aVLF82ZM8frmMTERO3YsUNr1qyRw+GQw+Hwmk90/PhxTZ48WYmJiXI6nWrTpo3uv//+SnPM3G63fvvb36pNmzYKDw/XoEGDfE74qvLqq6+qXbt2atSokQYMGKDt27d7vf7VV1/pgQceUIcOHRQeHq74+Hg9+OCD+uGHHzzHTJs2TU888YQkqX379p7r/P777z3HvP/+++rbt6+uuOIKNW3aVP3799cXX3xRqZ6//e1v6tu3r8LDw9WhQwf94Q9/qHRMbGysunfvrr/85S81Xl9hYaHS09PVpk0bOZ1OtWzZUnfeeadXbReqbo7XP//5T40cOVIxMTFq1KiROnXqpF//+tdexxw4cEAPPvig4uLi5HQ61bVrV82fP7/GOgEElwaBLgC4GAsWLNDdd9+tsLAwjRkzRnPmzNGmTZvUp0+fGt+7bt06devWTQ0bNqzy9ZEjRyoxMVE5OTlav369Xn/9dR07dsyrEZgzZ466du2qO+64Qw0aNNCyZcv0q1/9Sm63WxkZGZKkWbNm6ZFHHlGTJk08v6Tj4uIkSaWlperXr5927typBx98UNdee62OHDmipUuXav/+/WrRooXnXNOnT1dISIgef/xxlZSU6He/+53Gjh2rDRs2XPTn94c//EEnTpxQRkaGTp06pddee00333yztm3b5qlxxYoV+u6775Senq74+Hjt2LFD8+bN044dO7R+/Xo5HA7dfffd+vbbb/XBBx/o1Vdf9dQdExMjSXr22Wc1bdo0XX/99XruuecUFhamDRs2aOXKlbrllls89ezevVv33HOPxo8fr3Hjxmn+/Pl64IEHlJycrK5du3rVnpycrCVLltR4jSNGjNCOHTv0yCOPKDExUcXFxVqxYoX27dunxMTEWn9WX331lfr166eGDRvqoYceUmJiovbs2aNly5Z5vpxRVFSk6667Tg6HQ5mZmYqJidFnn32m8ePHy+VyadKkSbU+H4DLnAHUM5s3bzYkGStWrDAMwzDcbrfRpk0bY+LEibV6f5s2bYwRI0ZU2p+dnW1IMu644w6v/b/61a8MScY//vEPz76TJ09Wev/gwYONDh06eO3r2rWrMWDAgErHTp061ZBk/PnPf670mtvtNgzDMFatWmVIMjp37myUl5d7Xn/ttdcMSca2bdvML7QKe/fuNSQZjRo1Mvbv3+/Zv2HDBkOSMXnyZNNr/OCDDwxJxtq1az37ZsyYYUgy9u7d63Xsrl27jJCQEOOuu+4yKioqqrxGwzCMdu3aVRqzuLjYcDqdxmOPPVaphhdffNGQZBQVFVV7nceOHTMkGTNmzKj2GMMwjAEDBnj9/Zz/fN59913Pvv79+xsRERHG//3f/1V7DePHjzdatmxpHDlyxOuY0aNHG1FRUVV+lgCCE7caUe8sWLBAcXFxGjhwoKRz83JGjRqlRYsWqaKiosb3//DDD2ratGm1r59PrM575JFHJEmffvqpZ1+jRo08fy4pKdGRI0c0YMAAfffddyopKamxhj/96U/q0aOH7rrrrkqvORwOr5/T09MVFhbm+blfv36SpO+++67G81Rn+PDhat26tefnvn37KiUlpdprPHXqlI4cOaLrrrtOkrRly5Yaz7FkyRK53W5NnTpVISHe/6q58Bq7dOniuS7pXGLWqVOnKq/x/N+d2bIfjRo1UlhYmFavXq1jx47VWGt1Dh8+rLVr1+rBBx9U27Ztq7wGwzD0pz/9ScOGDZNhGDpy5IhnGzx4sEpKSmr1eQEIDjReqFcqKiq0aNEiDRw4UHv37tXu3bu1e/dupaSkqKioSLm5ubUaxzCMal+76qqrvH6+8sorFRIS4jU36Msvv1RaWpoaN26s6OhoxcTE6Omnn5akWjVee/bsUbdu3WpV64W/8M83HpfSUFx4jZJ09dVXe13j0aNHNXHiRMXFxalRo0aKiYlR+/btJdX+GkNCQtSlS5caj73wGqVz11nVNZ7/u7uwefs5p9Opl156SZ999pni4uLUv39//e53v1NhYWGNtfzc+cbP7O/q8OHDOn78uObNm6eYmBivLT09XZJUXFzs03kBXL6Y44V6ZeXKlTp06JAWLVqkRYsWVXp9wYIFXnOHqtK8eXOfmpYLf8Hv2bNHgwYNUlJSkmbOnKmEhASFhYXp008/1auvviq3213rsWujum/YmTWPVhg5cqTWrVunJ554Qj179lSTJk3kdrs1ZMiQgF7j+b+7n8+Dq8qkSZM0bNgwLVmyRJ9//rmeeeYZ5eTkaOXKlerVq9elF/0v5z+L++67T+PGjavymO7du1t2PgD1G40X6pUFCxYoNjZWs2fPrvTan//8Zy1evFhz5871uk12oaSkJO3du7fa13ft2uVJdqRzE7/dbrdnQvayZctUXl6upUuXeiU1q1atqjRWdanMlVdeWelbhHbatWtXpX3ffvut5xqPHTum3NxcPfvss5o6darp+8yu0e126+uvv1bPnj0tqVs6t5RIixYtPBP4zVx55ZV67LHH9Nhjj2nXrl3q2bOnXnnlFb3//vu1OleHDh0kyfTvKiYmRhEREaqoqFBaWlrtLgJA0OJWI+qNH3/8UX/+8591++2365577qm0ZWZm6sSJE1q6dKnpOKmpqdq+fbvKy8urfP3Cpu6NN96QJA0dOlTST+nMz9OYkpISvfvuu5XGaty4cZULi44YMUL/+Mc/tHjx4kqv+TvJks7Nvzpw4IDn540bN2rDhg2m1yid+6bmhRo3bixJla5z+PDhCgkJ0XPPPVcpIbuUa8zPz1dqaqrpMSdPnqy0BMiVV16piIiIav/eqxITE6P+/ftr/vz52rdvn9dr568hNDRUI0aM0J/+9KcqGzQrH3EEoP4j8UK9sXTpUp04cUJ33HFHla9fd911nsVUR40aVe04d955p55//nmtWbOmytuSe/fu1R133KEhQ4YoLy9P77//vv793/9dPXr0kCTdcsstCgsL07Bhw/SLX/xCpaWleueddxQbG6tDhw55jZWcnKw5c+bohRdeUMeOHRUbG6ubb75ZTzzxhD7++GPde++9evDBB5WcnKyjR49q6dKlmjt3rudctbV69WoNHDhQ2dnZmjZtWo3Hd+zYUTfeeKMefvhhlZeXa9asWWrevLmefPJJSVJkZKRnXtSZM2fUunVrffHFF1UmhcnJyZKkX//61xo9erQaNmyoYcOGqWPHjvr1r3+t559/Xv369dPdd98tp9OpTZs2qVWrVhf16J/i4mJ99dVXlb4AcaFvv/1WgwYN0siRI9WlSxc1aNBAixcvVlFRkUaPHu3TOV9//XXdeOONuvbaa/XQQw+pffv2+v777/U///M/2rp1q6RzS36sWrVKKSkpmjBhgrp06aKjR49qy5Yt+utf/6qjR4/6fK0ALlMB+jYl4LNhw4YZ4eHhRllZWbXHPPDAA0bDhg0rfa3/Qt27dzfGjx/vte/8chJff/21cc899xgRERFG06ZNjczMTOPHH3/0Onbp0qVG9+7djfDwcCMxMdF46aWXjPnz51daVqGwsNC47bbbjIiICEOS19IFP/zwg5GZmWm0bt3aCAsLM9q0aWOMGzfOU/v55SQ++ugjr3NXteTBsmXLDEnG3LlzTa/7/HtnzJhhvPLKK0ZCQoLhdDqNfv36eS2XYRiGsX//fuOuu+4yoqOjjaioKOPee+81Dh48aEgysrOzvY59/vnnjdatWxshISGVPoP58+cbvXr1MpxOp9G0aVNjwIABnqVADOPcchK33XZbpVovXOrBMAxjzpw5xhVXXGG4XC7T6zxy5IiRkZFhJCUlGY0bNzaioqKMlJQU449//KPpOar6bA3DMLZv3+75LMLDw41OnToZzzzzjNcxRUVFRkZGhpGQkGA0bNjQiI+PNwYNGmTMmzfPtFYAwcVhGDbc1wDqmP/+7/9WRkaG9u3bp+jo6ECXc8mefPJJffDBB9q9e7ecTmegy/GbXr166aabbtKrr74a6FIA4KIwxwtBaezYsWrbtm2Vk/Tro1WrVumZZ565rJuu5cuXa9euXZoyZUqgSwGAi0biBQAAYBMSLwAAAJvQeAEAANiExgsAAMAmNF4AAAA2sX0BVbfbrYMHDyoiIsL0IbcAAODcUxJOnDihVq1aKSTE/rzk1KlTOn36tF/GDgsLU3h4uF/Grqtsb7wOHjyohIQEu08LAEC9VlBQoDZt2th6zlOnTimmUSOV+mn8+Ph47d27N6iaL9sbr4iICElSwZ13KrJhQ7tPX7e8806gKwAA1HEul0sJ7dp5fn/a6fTp0yqVNFmS1asElkt6tbBQp0+fpvHyp/O3FyMbNqTxiowMdAUAgHoikNNzGkmyujUK1knmwXrdAAAAtrM98QIAAPVLiKxPaoI1+QnW6wYAALAdiRcAADBF4mUdGi8AAGCKxss6wXrdAAAAtiPxAgAApki8rBOs1w0AAGA7Ei8AAGCKxMs6wXrdAAAAtiPxAgAApki8rBOs1w0AAGA7Ei8AAGCKxMs6wXrdAAAAtiPxAgAAphyyPqlxWDxefUHjBQAATDlkfaMUrI0XtxoBAABsQuIFAABMhf5rs3rMYETiBQAAYBMSLwAAYIrlJKwTrNcNAABgOxIvAABgisTLOsF63QAAALYj8QIAAKZIvKxD4wUAAEzReFknWK8bAADAdhfVeM2ePVuJiYkKDw9XSkqKNm7caHVdAACgjgjx0xaMfL7uDz/8UFlZWcrOztaWLVvUo0cPDR48WMXFxf6oDwAA4LLhc+M1c+ZMTZgwQenp6erSpYvmzp2rK664QvPnz/dHfQAAIMBIvKzj03WfPn1a+fn5SktL+2mAkBClpaUpLy+vyveUl5fL5XJ5bQAAAMHIp8bryJEjqqioUFxcnNf+uLg4FRYWVvmenJwcRUVFebaEhISLrxYAANjO4actGPk96ZsyZYpKSko8W0FBgb9PCQAAUCf5tI5XixYtFBoaqqKiIq/9RUVFio+Pr/I9TqdTTqfz4isEAAABFSIp1A9jBiOfrjssLEzJycnKzc317HO73crNzVVqaqrlxQEAgMBjcr11fF65PisrS+PGjVPv3r3Vt29fzZo1S2VlZUpPT/dHfQAAAJcNnxuvUaNG6fDhw5o6daoKCwvVs2dPLV++vNKEewAAcHngkUHWuahnNWZmZiozM9PqWgAAAC5rPCQbAACYIvGyTrBeNwAAgO1IvAAAgCkSL+sE63UDAADYjsQLAACYIvGyDo0XAAAwReNlnWC9bgAAANvReAEAAFMOP22+mj17thITExUeHq6UlBRt3Lix2mN37NihESNGKDExUQ6HQ7Nmzap0zLRp0+RwOLy2pKSki6is9mi8AABAnffhhx8qKytL2dnZ2rJli3r06KHBgweruLi4yuNPnjypDh06aPr06YqPj6923K5du+rQoUOe7W9/+5u/LkESjRcAAKhBqJ82X8ycOVMTJkxQenq6unTporlz5+qKK67Q/Pnzqzy+T58+mjFjhkaPHi2n01ntuA0aNFB8fLxna9GihY+V+YbGCwAA1GmnT59Wfn6+0tLSPPtCQkKUlpamvLy8Sxp7165datWqlTp06KCxY8dq3759l1quKb7VCAAATDlkfVJzfo6Xy+Xy2u90OislVEeOHFFFRYXi4uK89sfFxemf//znRdeQkpKi9957T506ddKhQ4f07LPPql+/ftq+fbsiIiIuelwzJF4AACBgEhISFBUV5dlycnJsO/fQoUN17733qnv37ho8eLA+/fRTHT9+XH/84x/9dk4SLwAAYMqf63gVFBQoMjLSs7+q+VgtWrRQaGioioqKvPYXFRWZTpz3VXR0tK6++mrt3r3bsjEvROIFAABMhfhpk6TIyEivrarGKywsTMnJycrNzfXsc7vdys3NVWpqqmXXWVpaqj179qhly5aWjXkhEi8AAFDnZWVlady4cerdu7f69u2rWbNmqaysTOnp6ZKk+++/X61bt/bcqjx9+rS+/vprz58PHDigrVu3qkmTJurYsaMk6fHHH9ewYcPUrl07HTx4UNnZ2QoNDdWYMWP8dh00XgAAwFRdeGTQqFGjdPjwYU2dOlWFhYXq2bOnli9f7plwv2/fPoWE/DTqwYMH1atXL8/PL7/8sl5++WUNGDBAq1evliTt379fY8aM0Q8//KCYmBjdeOONWr9+vWJiYi718qrlMAzD8NvoVXC5XIqKilLJPfcosmFDO09d97z/fqArAADUcS6XS1FNm6qkpMRrLpRt546K0jJJjS0eu0zSMCkg1xVIJF4AAMBUXUi8LhfBet0AAAC2I/ECAACmSLysE6zXDQAAYDsSLwAAYMqhnx7xY+WYwYjGCwAAmAr912b1mMGIW40AAAA2IfECAACmmFxvncA1Xu+8IwXRgmlVmRYarEGrt2kVFYEuAQAAW5B4AQAAUw5Zn1AF6+T6YE36AAAAbEfiBQAATDHHyzrBet0AAAC2I/ECAACmSLysQ+MFAABM0XhZJ1ivGwAAwHYkXgAAwBSJl3WC9boBAABsR+IFAABMOWT9gqcsoAoAAAC/IvECAACmQv+1WT1mMCLxAgAAsAmJFwAAMMW3Gq1D4wUAAEzReFknWK8bAADAdiReAADAlEPWJzUsJwEAAAC/IvECAACmmONlnWC9bgAAANuReAEAAFMkXtYJ1usGAACwHYkXAAAwReJlnWC9bgAAANuReAEAAFMOWb/uVrCu40XjBQAATIX+a7N6zGDErUYAAACbkHgBAABTTK63js/XvXbtWg0bNkytWrWSw+HQkiVL/FAWAADA5cfnxqusrEw9evTQ7Nmz/VEPAACoY0L8tAUjn281Dh06VEOHDvVHLQAAAJc1v8/xKi8vV3l5uednl8vl71MCAAALMcfLOn6/7pycHEVFRXm2hIQEf58SAACgTvJ74zVlyhSVlJR4toKCAn+fEgAAWMgh6+d3sYCqnzidTjmdTn+fBgAA+Am3Gq0TrNcNAABgO58Tr9LSUu3evdvz8969e7V161Y1a9ZMbdu2tbQ4AAAQeCRe1vG58dq8ebMGDhzo+TkrK0uSNG7cOL333nuWFQYAAHC58bnxuummm2QYhj9qAQAAdRCJl3WC9boBAABsR+MFAABM1ZVHBs2ePVuJiYkKDw9XSkqKNm7cWO2xO3bs0IgRI5SYmCiHw6FZs2Zd8phWoPECAAB13ocffqisrCxlZ2dry5Yt6tGjhwYPHqzi4uIqjz958qQ6dOig6dOnKz4+3pIxrUDjBQAATNWFxGvmzJmaMGGC0tPT1aVLF82dO1dXXHGF5s+fX+Xxffr00YwZMzR69Ohq1xP1dUwr0HgBAABT/my8XC6X1/bz5zufd/r0aeXn5ystLe2nmkJClJaWpry8vIu6Jn+MWRs0XgAAIGASEhK8numck5NT6ZgjR46ooqJCcXFxXvvj4uJUWFh4Uef1x5i14fdHBgEAgPrNn8tJFBQUKDIy0rP/cn/MII0XAAAImMjISK/GqyotWrRQaGioioqKvPYXFRVVO3G+Jv4Ysza41QgAAEwFenJ9WFiYkpOTlZub69nndruVm5ur1NTUi7omf4xZGyReAACgzsvKytK4cePUu3dv9e3bV7NmzVJZWZnS09MlSffff79at27tmSN2+vRpff31154/HzhwQFu3blWTJk3UsWPHWo3pDzReAADAlEOSw+GwdkwfHz84atQoHT58WFOnTlVhYaF69uyp5cuXeybH79u3TyEhP+VoBw8eVK9evTw/v/zyy3r55Zc1YMAArV69ulZj+oPDsPnBiy6XS1FRUSo5dqzGe7qXu2mhoYEuoU6YVlER6BIAoM5yuVyKatpUJSUltv/e9PzOlhRpcePlMgxFSQG5rkAi8QIAAOYaNJAsbrxkGNLZs9aOWQ/QeAEAAHM0XpbhW40AAAA2IfECAADm/JV4BSESLwAAAJuQeAEAAHMkXpYh8QIAALAJiRcAADAXGiqFWJzVuN3WjldPkHgBAADYhMQLAACYa9CAxMsiNF4AAMAcjZdluNUIAABgExIvAABgjsTLMiReAAAANiHxCqBpFRWBLqFO+DY0NNAl1AlX888DgLoqNPTcZqUg/XceiRcAAIBNSLwAAIC5Bg2sT7ysfgRRPUHiBQAAYBMSLwAAYI7EyzI0XgAAwByNl2W41QgAAGATEi8AAGCOxMsyJF4AAAA2IfECAADmQkPPpV64ZCReAAAANqF9BQAA5ho0IPGyCIkXAACATWhfAQCAORIvy/ApAgAAczReluFWIwAAgE1oXwEAgDl/LCdhGNaOV0+QeAEAANiExAsAAJjzxxwvEi8AAAD4E4kXAAAwR+JlGRIvAAAAm5B4AQAAcyRelqHxAgAA5mi8LMOtRgAAAJuQeAEAAHP+WEDV7bZ2vHqCxAsAAMAmJF4AAMCcP+Z4kXgBAADAn3xqvHJyctSnTx9FREQoNjZWw4cP1zfffOOv2gAAQF1wPvGyegtCPjVea9asUUZGhtavX68VK1bozJkzuuWWW1RWVuav+gAAAC4bPrWby5cv9/r5vffeU2xsrPLz89W/f39LCwMAAHUEc7wsc0mfYklJiSSpWbNm1R5TXl6u8vJyz88ul+tSTgkAAFBvXfTkerfbrUmTJumGG25Qt27dqj0uJydHUVFRni0hIeFiTwkAAAKBOV6WuejGKyMjQ9u3b9eiRYtMj5syZYpKSko8W0FBwcWeEgAABML5BVSt3EJDA31VAXFRjVdmZqY++eQTrVq1Sm3atDE91ul0KjIy0msDAADw1ezZs5WYmKjw8HClpKRo48aNpsd/9NFHSkpKUnh4uK655hp9+umnXq8/8MADcjgcXtuQIUP8eQm+NV6GYSgzM1OLFy/WypUr1b59e3/VBQAA6oo6cKvxww8/VFZWlrKzs7Vlyxb16NFDgwcPVnFxcZXHr1u3TmPGjNH48eP197//XcOHD9fw4cO1fft2r+OGDBmiQ4cOebYPPvjgoj+m2vCp8crIyND777+vhQsXKiIiQoWFhSosLNSPP/7or/oAAAA0c+ZMTZgwQenp6erSpYvmzp2rK664QvPnz6/y+Ndee01DhgzRE088oc6dO+v555/XtddeqzfffNPrOKfTqfj4eM/WtGlTv16HT43XnDlzVFJSoptuukktW7b0bB9++KG/6gMAAIHmx8TL5XJ5bT9fCeG806dPKz8/X2lpaZ59ISEhSktLU15eXpUl5+XleR0vSYMHD650/OrVqxUbG6tOnTrp4Ycf1g8//HCpn5Ypn3I+wzD8VQcAAAhCF652kJ2drWnTpnntO3LkiCoqKhQXF+e1Py4uTv/85z+rHLewsLDK4wsLCz0/DxkyRHfffbfat2+vPXv26Omnn9bQoUOVl5enUD9N/g/O73ICAIDa88fyDxUVkqSCggKvL945nU5rz2Ni9OjRnj9fc8016t69u6688kqtXr1agwYN8ss5eUg2AAAImAtXPqiq8WrRooVCQ0NVVFTktb+oqEjx8fFVjhsfH+/T8ZLUoUMHtWjRQrt3776IK6kdGi8AAGAuwOt4hYWFKTk5Wbm5uZ59brdbubm5Sk1NrfI9qampXsdL0ooVK6o9XpL279+vH374QS1btqx1bb7iViMAADDnj1uNPo6XlZWlcePGqXfv3urbt69mzZqlsrIypaenS5Luv/9+tW7dWjk5OZKkiRMnasCAAXrllVd02223adGiRdq8ebPmzZsnSSotLdWzzz6rESNGKD4+Xnv27NGTTz6pjh07avDgwdZe68/QeAEAgDpv1KhROnz4sKZOnarCwkL17NlTy5cv90yg37dvn0JCfrqRd/3112vhwoX6zW9+o6efflpXXXWVlixZ4nnMYWhoqL766iv9/ve/1/Hjx9WqVSvdcsstev755/06z8xh2PxVRZfLpaioKJUcO8Yq9pAkfRukj4240NX/mmgKAD/ncrkU1bSpSkpKbP+96fmd/eyzigwPt3bsU6cUlZ0dkOsKJOZ4AQAA2IRbjQAAwFwdmON1uSDxAgAAsElwtpsAAKD2SLwsQ+IFAABgk+BsNwEAQO2dX0DV6jGDEI0XAAAwx61Gy3CrEQAAwCbB2W4CAIDaI/GyDIkXAACATYKz3QQAALVH4mUZEi8AAACbBGe7CQAAao/lJCxD4gUAAGATEi8AAGCOOV6WCc6rBgAAtUfjZRluNQIAANgkONtNAABQeyReliHxAgAAsElwtpsAAKD2SLwsE5xXjTrl6oqKQJdQN+zfH+gK6oY2bQJdAQD4DY0XAAAwxwKqlmGOFwAAgE1IvAAAgDnmeFkmOK8aAADUHo2XZbjVCAAAYJPgbDcBAEDtkXhZhsQLAADAJsHZbgIAgNpjOQnLkHgBAADYhMQLAACYY46XZUi8AAAAbBKc7SYAAKg9Ei/LBOdVAwCA2qPxsgy3GgEAAGwSnO0mAACoPRIvy5B4AQAA2CQ4200AAFB7LKBqGRIvAAAAm5B4AQAAc8zxsgyJFwAAgE2Cs90EAAC1R+JlmeC8agAAUHs0XpbhViMAAIBNgrPdBAAAtcdyEpYh8QIAALAJiRcAADDHHC/LkHgBAADYJDjbTQAAUHskXpbxKfGaM2eOunfvrsjISEVGRio1NVWfffaZv2oDAADwmD17thITExUeHq6UlBRt3LjR9PiPPvpISUlJCg8P1zXXXKNPP/3U63XDMDR16lS1bNlSjRo1Ulpamnbt2uXPS/Ct8WrTpo2mT5+u/Px8bd68WTfffLPuvPNO7dixw1/1AQCAQDufeFm9+eDDDz9UVlaWsrOztWXLFvXo0UODBw9WcXFxlcevW7dOY8aM0fjx4/X3v/9dw4cP1/Dhw7V9+3bPMb/73e/0+uuva+7cudqwYYMaN26swYMH69SpU5f0cZlxGIZhXMoAzZo104wZMzR+/PhaHe9yuRQVFaWSY8cUGRl5KacGLi/79we6grqhTZtAVwDUKS6XS1FNm6qkpMT235v+/J3t63WlpKSoT58+evPNNyVJbrdbCQkJeuSRR/TUU09VOn7UqFEqKyvTJ5984tl33XXXqWfPnpo7d64Mw1CrVq302GOP6fHHH5cklZSUKC4uTu+9955Gjx5t0ZV6u+jJ9RUVFVq0aJHKysqUmppa7XHl5eVyuVxeGwAAgKRKPUJ5eXmlY06fPq38/HylpaV59oWEhCgtLU15eXlVjpuXl+d1vCQNHjzYc/zevXtVWFjodUxUVJRSUlKqHdMKPjde27ZtU5MmTeR0OvXLX/5SixcvVpcuXao9PicnR1FRUZ4tISHhkgoGAAD2civEL5skJSQkePUJOTk5lc5/5MgRVVRUKC4uzmt/XFycCgsLq6y5sLDQ9Pjz/+vLmFbw+SsFnTp10tatW1VSUqKPP/5Y48aN05o1a6ptvqZMmaKsrCzPzy6Xi+YLAABIkgoKCrxuNTqdzgBW438+N15hYWHq2LGjJCk5OVmbNm3Sa6+9prfffrvK451O52X/IQIAcDk7e/bcZvWYkjwrJZhp0aKFQkNDVVRU5LW/qKhI8fHxVb4nPj7e9Pjz/1tUVKSWLVt6HdOzZ09fLsUnl7yAqtvtrvJ+LAAAgBXCwsKUnJys3Nxczz63263c3Nxq55mnpqZ6HS9JK1as8Bzfvn17xcfHex3jcrm0YcMG07nrl8qnxGvKlCkaOnSo2rZtqxMnTmjhwoVavXq1Pv/8c3/VBwAAAsyfiVdtZWVlady4cerdu7f69u2rWbNmqaysTOnp6ZKk+++/X61bt/bMEZs4caIGDBigV155RbfddpsWLVqkzZs3a968eZIkh8OhSZMm6YUXXtBVV12l9u3b65lnnlGrVq00fPhwKy/Vi0+NV3Fxse6//34dOnRIUVFR6t69uz7//HP927/9m7/qAwAA0KhRo3T48GFNnTpVhYWF6tmzp5YvX+6ZHL9v3z6FhPx0I+/666/XwoUL9Zvf/EZPP/20rrrqKi1ZskTdunXzHPPkk0+qrKxMDz30kI4fP64bb7xRy5cvV3h4uN+u45LX8fIV63gB1WAdr3NYxwvwUhfW8SoosP7c575sFxWQ6wqk4HxQEgAAqLW6cKvxcnHJk+sBAABQOyReAADAVEWF9QlVRYW149UXJF4AAAA2IfECAACmmONlHRIvAAAAm5B4AQAAUyRe1iHxAgAAsAmJFwAAMEXiZR0SLwAAAJuQeAEAAFOs42UdGi8AAGCKW43W4VYjAACATUi8AACAKRIv65B4AQAA2ITECwAAmCLxsg6JFwAAgE1IvAAAgCkSL+uQeAEAANiExAsAAJhiAVXr0HgBAABT3Gq0DrcaAQAAbELiBQAATJF4WYfECwAAwCYkXgAAwBSJl3VIvAAAAGxC4gXUFW3aBLqCumH9+kBXUDdcd12gKwA8WE7COiReAAAANiHxAgAAppjjZR0aLwAAYIrGyzrcagQAALAJiRcAADBF4mUdEi8AAACbkHgBAABTJF7WIfECAACwCYkXAAAwxQKq1iHxAgAAsAmJFwAAMMUcL+vQeAEAAFM0XtbhViMAAIBNSLwAAIApEi/rkHgBAADYhMQLAACYYjkJ65B4AQAA2ITECwAAmGKOl3VIvAAAAGxC4gUAAEyReFmHxgsAAJii8bIOtxoBAABsQuMFAABMnU+8rN785ejRoxo7dqwiIyMVHR2t8ePHq7S01PQ9p06dUkZGhpo3b64mTZpoxIgRKioq8jrG4XBU2hYtWuRTbTReAADgsjJ27Fjt2LFDK1as0CeffKK1a9fqoYceMn3P5MmTtWzZMn300Udas2aNDh48qLvvvrvSce+++64OHTrk2YYPH+5TbczxAgAApurTAqo7d+7U8uXLtWnTJvXu3VuS9MYbb+jWW2/Vyy+/rFatWlV6T0lJif7rv/5LCxcu1M033yzpXIPVuXNnrV+/Xtddd53n2OjoaMXHx190fSReAAAgYFwul9dWXl5+SePl5eUpOjra03RJUlpamkJCQrRhw4Yq35Ofn68zZ84oLS3Nsy8pKUlt27ZVXl6e17EZGRlq0aKF+vbtq/nz58swDJ/qI/ECAACmzp6VQkOtH1OSEhISvPZnZ2dr2rRpFz1uYWGhYmNjvfY1aNBAzZo1U2FhYbXvCQsLU3R0tNf+uLg4r/c899xzuvnmm3XFFVfoiy++0K9+9SuVlpbq0UcfrXV9NF4AACBgCgoKFBkZ6fnZ6XRWedxTTz2ll156yXSsnTt3WlrbhZ555hnPn3v16qWysjLNmDHDp8brkm41Tp8+XQ6HQ5MmTbqUYQAAQB3mz281RkZGem3VNV6PPfaYdu7cabp16NBB8fHxKi4uvqD+szp69Gi1c7Pi4+N1+vRpHT9+3Gt/UVGR6XyulJQU7d+/36fboxedeG3atElvv/22unfvfrFDAACAesCftxprKyYmRjExMTUel5qaquPHjys/P1/JycmSpJUrV8rtdislJaXK9yQnJ6thw4bKzc3ViBEjJEnffPON9u3bp9TU1GrPtXXrVjVt2rTaZrEqF5V4lZaWauzYsXrnnXfUtGnTixkCAADAcp07d9aQIUM0YcIEbdy4UV9++aUyMzM1evRozzcaDxw4oKSkJG3cuFGSFBUVpfHjxysrK0urVq1Sfn6+0tPTlZqa6vlG47Jly/Sf//mf2r59u3bv3q05c+boxRdf1COPPOJTfReVeGVkZOi2225TWlqaXnjhBdNjy8vLvSI4l8t1MacEAAABUp+Wk5CkBQsWKDMzU4MGDVJISIhGjBih119/3fP6mTNn9M033+jkyZOefa+++qrn2PLycg0ePFhvvfWW5/WGDRtq9uzZmjx5sgzDUMeOHTVz5kxNmDDBp9p8brwWLVqkLVu2aNOmTbU6PicnR88++6yvpwEAALgozZo108KFC6t9PTExsdIyEOHh4Zo9e7Zmz55d5XuGDBmiIUOGXHJtPt1qLCgo0MSJE7VgwQKFh4fX6j1TpkxRSUmJZysoKLioQgEAQGDUt0cG1WU+JV75+fkqLi7Wtdde69lXUVGhtWvX6s0331R5eblCL5h953Q6fZp0BgAAcLnyqfEaNGiQtm3b5rUvPT1dSUlJ+n//7/9VaroAAED9d/asFGLxs25IvGohIiJC3bp189rXuHFjNW/evNJ+AAAAeGPlegAAYIrEyzqX3HitXr3agjIAAEBdReNlHYs/RgAAAFSHW40AAMBUfVtAtS4j8QIAALAJiRcAADB19qzkcFg/ZjAi8QIAALAJiRcAADBF4mUdEi8AAACbkHgBAABTJF7WofECAACmaLysw61GAAAAm5B4AQAAUxUV1ideLKAKAAAAvyLxAgAApvwxH4s5XgAAAPArEi8AAGCKxMs6JF4AAAA2IfECAACmSLysQ+IFAABgExIvAABgyh9rbgXrOl40XgAAwNTZs5JhWDtmsDZe3GoEAACwCYkXAAAwReJlHRIvAAAAm5B4AQAAUyRe1iHxAgAAsAmJFwAAMEXiZR0aLwB1y3XXBbqCOqFzV25ISNLOHe5AlwBYisYLAACYqqiwPvFyB2lPTeMFAABMnT0rhVgcwgZr40WWDQAAYBMSLwAAYIrEyzokXgAAADYh8QIAAKZIvKxD4gUAAGATEi8AAGCqosL6hMrq5SnqCxIvAAAAm5B4AQAAU2fPSg6HtWMGa+JF4wUAAEzReFmHW40AAAA2IfECAACmSLysQ+IFAABgExIvAABgisTLOiReAAAANqHxAgAANXDLMKzdJP89M+jo0aMaO3asIiMjFR0drfHjx6u0tNT0PfPmzdNNN92kyMhIORwOHT9+3JJxL0TjBQAALitjx47Vjh07tGLFCn3yySdau3atHnroIdP3nDx5UkOGDNHTTz9t6bgXchiGvXdZXS6XoqKiVHLsmCIjI+08NQDUG5278t/FkrRzR5A+SflnXC6Xopo2VUlJie2/N8//zpaOSLL63C5JLSy/rp07d6pLly7atGmTevfuLUlavny5br31Vu3fv1+tWrUyff/q1as1cOBAHTt2TNHR0ZaNex7/zwYAADWo8NN2rrn7+VZeXn5Jlebl5Sk6OtrTHElSWlqaQkJCtGHDhoCPS+MFAAACJiEhQVFRUZ4tJyfnksYrLCxUbGys174GDRqoWbNmKiwsDPi4LCcBAABq8FNCZe2YUkFBgdetRqfTWeXRTz31lF566SXTEXfu3GldeX5C4wUAAAImMjKyVnO8HnvsMT3wwAOmx3To0EHx8fEqLi722n/27FkdPXpU8fHxF12nVePSeAEAgBr4Y/kH38aLiYlRTExMjcelpqbq+PHjys/PV3JysiRp5cqVcrvdSklJuahKrRyXOV4AAOCy0blzZw0ZMkQTJkzQxo0b9eWXXyozM1OjR4/2fPPwwIEDSkpK0saNGz3vKyws1NatW7V7925J0rZt27R161YdPXq01uPWBo0XAACogf++1egPCxYsUFJSkgYNGqRbb71VN954o+bNm+d5/cyZM/rmm2908uRJz765c+eqV69emjBhgiSpf//+6tWrl5YuXVrrcWuDdbwAoA5iHa9zWMerrqzj9X/yzzpe7QJyXYHEHC8AAFADt6xPqIKzqfbpP6mmTZsmh8PhtSUlJfmrNgAAUCfUr1uNdZnPiVfXrl3117/+9acBGhCaAQAA1IbPXVODBg0uaR0MAABQ3/hvAdVg4/PszV27dqlVq1bq0KGDxo4dq3379pkeX15eXuk5TAAAAMHIp8YrJSVF7733npYvX645c+Zo79696tevn06cOFHte3JycryewZSQkHDJRQMAADu5/bQFn0taTuL48eNq166dZs6cqfHjx1d5THl5udeTxl0ulxISElhOAgBMsJzEOSwnUVeWk9gpKcLi0U9I6sxyEr6Ijo7W1Vdf7VnltSpOp7PaB14CAID6gDleVrmk/6QqLS3Vnj171LJlS6vqAQAAuGz51Hg9/vjjWrNmjb7//nutW7dOd911l0JDQzVmzBh/1QcAAAKOdbys4tOtxv3792vMmDH64YcfFBMToxtvvFHr16+v1dPCAQBAfcWtRqv41HgtWrTIX3UAAABc9lh2HgAA1IBnNVqF7ysDAADYhMQLAADUwB8LnpJ4AQAAwI9IvAAAQA34VqNVSLwAAABsQuIFAABqQOJlFRovAABQAxovq3CrEQAAwCYkXgAAoAYkXlYh8QIAALAJiRcAAKiBIesXPDUsHq9+IPECAACwCYkXAACoAXO8rELiBQAAYBMSLwAAUAMSL6vQeAEAgBrQeFmFW40AAAA2IfECAAA1IPGyCokXAACATUi8AABADdyyfgFVq8erH0i8AAAAbELiBQAAasAcL6uQeAEAANiExAsAANTALesTKuZ4AQAAwI9IvACgDtq5IzjTgAudPEU+UDc+A+Z4WYXGCwAA1IDlJKxSF9poAACAoEDiBQAAasCtRquQeAEAANiExAsAANSAxMsqJF4AAAA2IfECAAA1IPGyCokXAACATUi8AABADXhkkFVovAAAQA1YQNUq3GoEAACXlaNHj2rs2LGKjIxUdHS0xo8fr9LSUtP3zJs3TzfddJMiIyPlcDh0/PjxSsckJibK4XB4bdOnT/epNhovAABQgwo/bf4xduxY7dixQytWrNAnn3yitWvX6qGHHjJ9z8mTJzVkyBA9/fTTpsc999xzOnTokGd75JFHfKqNW40AAOCysXPnTi1fvlybNm1S7969JUlvvPGGbr31Vr388stq1apVle+bNGmSJGn16tWm40dERCg+Pv6i6yPxAgAANfBf4uVyuby28vLyS6o0Ly9P0dHRnqZLktLS0hQSEqINGzZc0tiSNH36dDVv3ly9evXSjBkzdPbsWZ/eT+IFAAACJiEhwevn7OxsTZs27aLHKywsVGxsrNe+Bg0aqFmzZiosLLzocSXp0Ucf1bXXXqtmzZpp3bp1mjJlig4dOqSZM2fWegwaLwAAUAP/LaBaUFCgyMhIz16n01nl0U899ZReeukl0xF37txpXXlVyMrK8vy5e/fuCgsL0y9+8Qvl5ORUW/eFaLwAAEDAREZGejVe1Xnsscf0wAMPmB7ToUMHxcfHq7i42Gv/2bNndfTo0Uuam1WVlJQUnT17Vt9//706depUq/fQeAEAgBoE/pFBMTExiomJqfG41NRUHT9+XPn5+UpOTpYkrVy5Um63WykpKRdVaXW2bt2qkJCQSrc2zdB4AQCAGtSfBVQ7d+6sIUOGaMKECZo7d67OnDmjzMxMjR492vONxgMHDmjQoEH6wx/+oL59+0o6NzessLBQu3fvliRt27ZNERERatu2rZo1a6a8vDxt2LBBAwcOVEREhPLy8jR58mTdd999atq0aa3r41uNAADgsrJgwQIlJSVp0KBBuvXWW3XjjTdq3rx5ntfPnDmjb775RidPnvTsmzt3rnr16qUJEyZIkvr3769evXpp6dKlks7NPVu0aJEGDBigrl276re//a0mT57sNW5tOAzDMCy4xlpzuVyKiopSybFjtbqnCwAIXidPkQ+4XC61bBmlkpIS239vnv+dLU2XFG7x6KckPRWQ6wok/okGAACwCXO8AABADQI/uf5yQeIFAABgExIvAABQAxIvq5B4AQAA2ITECwAA1IDEyyo0XgAAoAY0Xlbx+VbjgQMHdN9996l58+Zq1KiRrrnmGm3evNkftQEAAFxWfEq8jh07phtuuEEDBw7UZ599ppiYGO3atcunpfIBAEB9U38eGVTX+dR4vfTSS0pISNC7777r2de+fXvLiwIAALgc+XSrcenSperdu7fuvfdexcbGqlevXnrnnXdM31NeXi6Xy+W1AQCA+sStn+Z5WbUFZ+LlU+P13Xffac6cObrqqqv0+eef6+GHH9ajjz6q3//+99W+JycnR1FRUZ4tISHhkosGAACoj3x6SHZYWJh69+6tdevWefY9+uij2rRpk/Ly8qp8T3l5ucrLyz0/u1wuJSQk8JBsAECNeEh2XXlI9hOSnBaPXi5pBg/JNtOyZUt16dLFa1/nzp21b9++at/jdDoVGRnptQEAAAQjnybX33DDDfrmm2+89n377bdq166dpUUBAIC6hHW8rOJT4zV58mRdf/31evHFFzVy5Eht3LhR8+bN07x58/xVHwAACDiWk7CKT7ca+/Tpo8WLF+uDDz5Qt27d9Pzzz2vWrFkaO3asv+oDAAC4bPj8yKDbb79dt99+uz9qAQAAdRK3Gq3C10UAAABswkOyAQBADUi8rELiBQAAYBMSLwAAUAMSL6uQeAEAANiExAsAANTg/EOyrR4z+NB4AQCAGrCAqlW41QgAAGATEi8AAFADJtdbhcQLAADAJiReAACgBiReViHxAgAAsAmJFwAAqAGJl1VIvAAAAGxC4gUAAGpA4mUVGi8AAFADVq63CrcaAQAAbELiBQAAasAjg6xC4gUAAGATEi8AAFCDClmf1QTn5HoSLwAAAJuQeAEAgBqQeFmFxAsAAMAmJF4AAKAGJF5WofECAAA1YDkJq9jeeBmGIUlyuVx2nxoAUM+cPMWMmBMnzv2+PP/7MzDO1pMx6z7bG68TJ05IkhLatbP71AAA1FsnTpxQVFSUrecMCwtTfHy8Cgv/6pfx4+PjFRYW5pex6yqHYXML7Xa7dfDgQUVERMjhcNh5ag+Xy6WEhAQVFBQoMjIyIDXUBXwO5/A5nMPncA6fwzl8DufUhc/BMAydOHFCrVq1UkiI/QngqVOndPr0ab+MHRYWpvDwcL+MXVfZnniFhISoTZs2dp+2SpGRkUH9L5Tz+BzO4XM4h8/hHD6Hc/gczgn052B30vVz4eHhQdcc+RM3zwEAAGxC4wUAAGCToGy8nE6nsrOz5XQ6A11KQPE5nMPncA6fwzl8DufwOZzD5wCr2T65HgAAIFgFZeIFAAAQCDReAAAANqHxAgAAsAmNFwAAgE2CrvGaPXu2EhMTFR4erpSUFG3cuDHQJdlu7dq1GjZsmFq1aiWHw6ElS5YEuiTb5eTkqE+fPoqIiFBsbKyGDx+ub775JtBl2W7OnDnq3r27Z3HI1NRUffbZZ4EuK+CmT58uh8OhSZMmBboUW02bNk0Oh8NrS0pKCnRZAXHgwAHdd999at68uRo1aqRrrrlGmzdvDnRZuAwEVeP14YcfKisrS9nZ2dqyZYt69OihwYMHq7i4ONCl2aqsrEw9evTQ7NmzA11KwKxZs0YZGRlav369VqxYoTNnzuiWW25RWVlZoEuzVZs2bTR9+nTl5+dr8+bNuvnmm3XnnXdqx44dgS4tYDZt2qS3335b3bt3D3QpAdG1a1cdOnTIs/3tb38LdEm2O3bsmG644QY1bNhQn332mb7++mu98soratq0aaBLw2UgqJaTSElJUZ8+ffTmm29KOvfcyISEBD3yyCN66qmnAlxdYDgcDi1evFjDhw8PdCkBdfjwYcXGxmrNmjXq379/oMsJqGbNmmnGjBkaP358oEuxXWlpqa699lq99dZbeuGFF9SzZ0/NmjUr0GXZZtq0aVqyZIm2bt0a6FIC6qmnntKXX36p//3f/w10KbgMBU3idfr0aeXn5ystLc2zLyQkRGlpacrLywtgZagLSkpKJJ1rOoJVRUWFFi1apLKyMqWmpga6nIDIyMjQbbfd5vXviWCza9cutWrVSh06dNDYsWO1b9++QJdku6VLl6p379669957FRsbq169eumdd94JdFm4TARN43XkyBFVVFQoLi7Oa39cXJwKCwsDVBXqArfbrUmTJumGG25Qt27dAl2O7bZt26YmTZrI6XTql7/8pRYvXqwuXboEuizbLVq0SFu2bFFOTk6gSwmYlJQUvffee1q+fLnmzJmjvXv3ql+/fjpx4kSgS7PVd999pzlz5uiqq67S559/rocffliPPvqofv/73we6NFwGGgS6ACDQMjIytH379qCcyyJJnTp10tatW1VSUqKPP/5Y48aN05o1a4Kq+SooKNDEiRO1YsUKhYeHB7qcgBk6dKjnz927d1dKSoratWunP/7xj0F169ntdqt379568cUXJUm9evXS9u3bNXfuXI0bNy7A1aG+C5rEq0WLFgoNDVVRUZHX/qKiIsXHxweoKgRaZmamPvnkE61atUpt2rQJdDkBERYWpo4dOyo5OVk5OTnq0aOHXnvttUCXZav8/HwVFxfr2muvVYMGDdSgQQOtWbNGr7/+uho0aKCKiopAlxgQ0dHRuvrqq7V79+5Al2Krli1bVvoPj86dOwflbVdYL2gar7CwMCUnJys3N9ezz+12Kzc3N2jnswQzwzCUmZmpxYsXa+XKlWrfvn2gS6oz3G63ysvLA12GrQYNGqRt27Zp69atnq13794aO3astm7dqtDQ0ECXGBClpaXas2ePWrZsGehSbHXDDTdUWl7m22+/Vbt27QJUES4nQXWrMSsrS+PGjVPv3r3Vt29fzZo1S2VlZUpPTw90abYqLS31+i/YvXv3auvWrWrWrJnatm0bwMrsk5GRoYULF+ovf/mLIiIiPPP8oqKi1KhRowBXZ58pU6Zo6NChatu2rU6cOKGFCxdq9erV+vzzzwNdmq0iIiIqze9r3LixmjdvHlTz/h5//HENGzZM7dq108GDB5Wdna3Q0FCNGTMm0KXZavLkybr++uv14osvauTIkdq4caPmzZunefPmBbo0XA6MIPPGG28Ybdu2NcLCwoy+ffsa69evD3RJtlu1apUhqdI2bty4QJdmm6quX5Lx7rvvBro0Wz344INGu3btjLCwMCMmJsYYNGiQ8cUXXwS6rDphwIABxsSJEwNdhq1GjRpltGzZ0ggLCzNat25tjBo1yti9e3egywqIZcuWGd26dTOcTqeRlJRkzJs3L9Al4TIRVOt4AQAABFLQzPECAAAINBovAAAAm9B4AQAA2ITGCwAAwCY0XgAAADah8QIAALAJjRcAAIBNaLwAAABsQuMFAABgExovAAAAm9B4AQAA2ITGCwAAwCb/H0eDrSmqpM5mAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "imshow_zero_center(j[:, 0, :, 0])\n", "_ = plt.title('A (batch, batch) slice')" ] }, { "cell_type": "code", "execution_count": 33, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:26.187736Z", "iopub.status.busy": "2022-12-14T20:58:26.187195Z", "iopub.status.idle": "2022-12-14T20:58:26.402236Z", "shell.execute_reply": "2022-12-14T20:58:26.401640Z" }, "id": "g4ZoRJcJNmy5" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgkAAAIQCAYAAAALu/2rAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABKzUlEQVR4nO3de3wU9b3/8fcmmA0CSQgEQiQQFCUgtxokgiggUeSm9qgFi+VSRKtEQbRHsD1cLBo9UkGRgtUq2gM/FBREpKHcIlJAIIgFFBAFSYFwEUm4SIDN/P6gbFmz2Zkks8lm5/V8PPbxcGdmZz7zZdd89jPf+azLMAxDAAAAPxFR1QEAAIDQRJIAAAD8IkkAAAB+kSQAAAC/SBIAAIBfJAkAAMAvkgQAAOAXSQIAAPCLJAEAAPhFkmAjl8ulCRMmeJ/PmjVLLpdLe/futfT6Rx55RLfeeqv3eU5Ojlwul+bPn29zpOXncrmUmZlp6z5TUlLUt29fW/dZEUOGDFHt2rVNt/v+++9Vq1YtLVmyxLZjT5gwQS6Xy2dZSkqKhgwZYtsxwt1Px+vi5ygnJ6fKYgqmcD8/VC2SBIv+9Kc/yeVyKT09PSj737Nnj9544w09/fTTQdn/RUuWLPFJZMLRgQMHNGHCBG3ZsiWox6lXr54eeOAB/c///E9QjwMAVYUkwaLZs2crJSVFGzZs0O7du23f/8svv6xmzZqpe/futu/7UkuWLNHEiRODeoyqduDAAU2cODHoSYIk/eY3v9HmzZu1cuXKoB1j586dev3114O2/3B3880368cff9TNN99c1aEA1Q5JggV79uzR2rVr9dJLLykhIUGzZ8+2df/nzp3T7Nmz9Ytf/MLW/SL4WrZsqdatW2vWrFlBO4bb7dZll10WtP1XNsMw9OOPP1ba8SIiIhQdHa2ICP53B5QVnxoLZs+erbp166pPnz665557bE8S1qxZo6NHjyojI8Pveo/Ho6efflqJiYmqVauW7rjjDuXl5fls8+mnn+ree+9VkyZN5Ha7lZycrMcff9znf8ZDhgzR9OnTJV2YW3DxcVFxcbFefvlltWnTRtHR0UpISNDtt9+uTZs2lYhp4cKFat26tdxut6699lplZ2dXeBz+/ve/q3379oqOjlarVq30wQcf+Kw/duyYnnzySbVp00a1a9dWTEyMevXqpS+++MK7TU5Ojq6//npJ0tChQ73neOkf8c8++0y9e/dW3bp1VatWLbVt21Yvv/xyiXj279+vu+66S7Vr11ZCQoKefPJJeTyeEtvdeuut+uijj2T2g6rnzp3TxIkTdfXVVys6Olr16tVTly5dtGzZsoCv8zcn4fjx43r88ceVkpIit9utxo0ba9CgQTp69Kh3m6KiIo0fP17Nmzf3vif++7//W0VFRT77WrZsmbp06aK4uDjVrl1bLVq0sHTZ66233tItt9yiBg0ayO12q1WrVpoxY4bf+Pv27aulS5eqQ4cOqlmzpl577TXveYwaNUrJyclyu91q3ry5XnjhBRUXF5se3zAMTZo0SY0bN9bll1+u7t27a/v27SW283fN3srn5aJ58+apVatWio6OVuvWrbVgwQINGTJEKSkpPtudOnVKTzzxhPdcWrRoocmTJ5d4X1yc12P2Gfruu+/0yCOPqEWLFqpZs6bq1aune++91/IcJ8AONao6gOpg9uzZ+q//+i9FRUXpvvvu04wZM7Rx40bvH6OKWrt2rVwul372s5/5Xf/ss8/K5XLpqaee0uHDhzV16lRlZGRoy5YtqlmzpqQL/yM7ffq0Hn74YdWrV08bNmzQtGnT9K9//Uvz5s2TJD300EM6cOCAli1bpr/+9a8ljjNs2DDNmjVLvXr10gMPPKDz58/r008/1fr169WhQwfvdmvWrNEHH3ygRx55RHXq1NErr7yiu+++W/v27VO9evXKNQZff/21+vfvr9/85jcaPHiw3nrrLd17773Kzs72Tub89ttvtXDhQt17771q1qyZDh06pNdee01du3bVl19+qaSkJLVs2VLPPPOMxo0bpwcffFA33XSTJKlz586SLvxB7Nu3rxo1aqSRI0cqMTFRX331lRYvXqyRI0d64/F4POrZs6fS09M1efJkLV++XH/84x911VVX6eGHH/aJPS0tTVOmTNH27dvVunXrUs9xwoQJysrK0gMPPKCOHTuqsLBQmzZt0ubNm30mrJo5efKkbrrpJn311Vf69a9/reuuu05Hjx7VokWL9K9//Uv169dXcXGx7rjjDq1Zs0YPPvigWrZsqa1bt2rKlCnatWuXFi5cKEnavn27+vbtq7Zt2+qZZ56R2+3W7t279Y9//MM0jhkzZujaa6/VHXfcoRo1auijjz7SI488ouLiYo0YMcJn2507d+q+++7TQw89pOHDh6tFixY6ffq0unbtqv379+uhhx5SkyZNtHbtWo0dO1YHDx7U1KlTAx5/3LhxmjRpknr37q3evXtr8+bNuu2223T27FnT2K18XiTp448/Vv/+/dWmTRtlZWXphx9+0LBhw3TFFVf47M8wDN1xxx1atWqVhg0bpvbt22vp0qX67W9/q/3792vKlCk+21v5DG3cuFFr167VgAED1LhxY+3du1czZsxQt27d9OWXX+ryyy83PU+gwgwEtGnTJkOSsWzZMsMwDKO4uNho3LixMXLkyBLbSjLGjx/vff7WW28Zkow9e/YEPMb9999v1KtXr8TyVatWGZKMK664wigsLPQuf++99wxJxssvv+xddvr06RKvz8rKMlwul/Hdd995l40YMcLw98++cuVKQ5Lx2GOPlVhXXFzsc45RUVHG7t27vcu++OILQ5Ixbdq0gOdZmqZNmxqSjPfff9+7rKCgwGjUqJHxs5/9zLvszJkzhsfj8Xntnj17DLfbbTzzzDPeZRs3bjQkGW+99ZbPtufPnzeaNWtmNG3a1Pjhhx9KPcfBgwcbknz2aRiG8bOf/cxIS0srEf/atWsNSca7774b8DzbtWtn9OnTJ+A248ePL/Hv07RpU2Pw4MHe5+PGjTMkGR988EGJ1188j7/+9a9GRESE8emnn/qsnzlzpiHJ+Mc//mEYhmFMmTLFkGQcOXIkYFz++HvP9ezZ07jyyitLxC/JyM7O9ln+hz/8wahVq5axa9cun+VjxowxIiMjjX379pV67MOHDxtRUVFGnz59fP7tnn76aUOSz3hd/BytWrUqYOz+Pi9t2rQxGjdubJw4ccK7LCcnx5BkNG3a1Lts4cKFhiRj0qRJPvu85557DJfL5fN5sfoZ8hfjunXrDEnGO++8E/D8ALtwucHE7Nmz1bBhQ++EQpfLpf79+2vu3Ll+S8/l8f3336tu3bqlrh80aJDq1KnjfX7PPfeoUaNGPrfeXawoSBfKnkePHlXnzp1lGIY+//xz0xjef/99uVwujR8/vsS6n96Sl5GRoauuusr7vG3btoqJidG3335repzSJCUl6ec//7n3eUxMjAYNGqTPP/9c+fn5ki5cm794Xdnj8ej777/3lsc3b95seozPP/9ce/bs0ahRoxQXF+ez7qfnKF2YlHipm266ye85Xvy3u7TU709cXJy2b9+ur7/+2jTWQN5//321a9fOZ7wuunge8+bNU8uWLZWamqqjR496H7fccoskadWqVd6YJOnDDz+0VOK/1KXvuYKCAh09elRdu3bVt99+q4KCAp9tmzVrpp49e/osmzdvnm666SbVrVvXJ8aMjAx5PB6tXr261GMvX75cZ8+e1aOPPurzbzdq1Kgyx17a5+XAgQPaunWrBg0a5HNLbNeuXdWmTRuf/S1ZskSRkZF67LHHfJY/8cQTMgxDf/vb33yWW/kMXRrjuXPn9P3336t58+aKi4uz9H4H7ECSEIDH49HcuXPVvXt37dmzR7t379bu3buVnp6uQ4cOacWKFbYdywhwPfvqq6/2ee5yudS8eXOfa5P79u3TkCFDFB8f772G3rVrV0kq8T9sf7755hslJSUpPj7edNsmTZqUWFa3bl398MMPpq8tTfPmzUv8ob7mmmskyXuexcXFmjJliq6++mq53W7Vr19fCQkJ+uc//2n5HCUFvCRw0cU5GZcq7Rwv/tv5SzQu9cwzz+j48eO65ppr1KZNG/32t7/VP//5T9NYfuqbb74xPYevv/5a27dvV0JCgs/j4pgePnxYktS/f3/deOONeuCBB9SwYUMNGDBA7733nqWE4R//+IcyMjJUq1YtxcXFKSEhwTuXwV+S4C/G7OzsEjFenJtzMUZ/vvvuO0klPxsJCQkBE+6LrHxeLh6jefPmJV7/02XfffedkpKSfJJ56cLE1kv3dZGVz9CPP/6ocePGeec4XHy/Hz9+3NL7HbADcxICWLlypQ4ePKi5c+dq7ty5JdbPnj1bt912W4WPU69evQr9gfV4PLr11lt17NgxPfXUU0pNTVWtWrW0f/9+DRkypMzfEM1ERkb6XR4o0bHDc889p//5n//Rr3/9a/3hD39QfHy8IiIiNGrUqEo7R38u/tvVr18/4HY333yzvvnmG3344Yf6+9//rjfeeENTpkzRzJkz9cADD1Qo3p8qLi5WmzZt9NJLL/ldn5ycLOnCt9XVq1dr1apV+vjjj5Wdna13331Xt9xyi/7+97+XOg7ffPONevToodTUVL300ktKTk5WVFSUlixZoilTppT497j0W/GlMd5666367//+b7/HuJjQ2K2yPy/+WPkMPfroo3rrrbc0atQoderUSbGxsXK5XBowYEClxAhIJAkBzZ49Ww0aNPDeEXCpDz74QAsWLNDMmTP9/g+wLFJTUzV79mwVFBQoNja2xPqflqcNw9Du3bvVtm1bSdLWrVu1a9cuvf322xo0aJB3O3+z5kv7tnvVVVdp6dKlOnbsmKVqgt12794twzB84tu1a5ckeWeRz58/X927d9df/vIXn9ceP37c5w90oHOUpG3btpV6J0l57NmzR9J/vjUGEh8fr6FDh2ro0KE6efKkbr75Zk2YMKFMScJVV12lbdu2mW7zxRdfqEePHqYVjoiICPXo0UM9evTQSy+9pOeee06/+93vtGrVqlLH6aOPPlJRUZEWLVrk86344mUMq+dx8uTJcv1bNG3aVNKFz8aVV17pXX7kyBHThNvq5+XiMfz1RfnpsqZNm2r58uU6ceKETzVhx44dPvsqi/nz52vw4MH64x//6F125swZHT9+vMz7AsqLyw2l+PHHH/XBBx+ob9++uueee0o8MjMzdeLECS1atKjCx+rUqZMMw1Bubq7f9e+8845OnDjhfT5//nwdPHhQvXr1kvSfbyWXfgsxDMPvbX21atWSpBL/o7n77rtlGIbfRkvBrhBIF67/LliwwPu8sLBQ77zzjtq3b6/ExERJF87zp7HMmzdP+/fv91lW2jled911atasmaZOnVpiXUXOMTc3V7Gxsbr22msDbvf999/7PK9du7aaN29e4pZEM3fffbe++OILn/G66OJ5/OIXv9D+/fv9NmH68ccfderUKUkXbiv9qfbt20tSwLj8vecKCgr01ltvWT6PX/ziF1q3bp2WLl1aYt3x48d1/vz5Ul+bkZGhyy67TNOmTfOJweyOiNJi9/d5SUpKUuvWrfXOO+/o5MmT3uWffPKJtm7d6rNt79695fF49Oqrr/osnzJlilwul/ezWhb+3u/Tpk2zbS4UYAWVhFIsWrRIJ06c0B133OF3/Q033OBtrNS/f/8KHatLly6qV6+eli9f7p1Ydqn4+Hh16dJFQ4cO1aFDhzR16lQ1b95cw4cPl3ShEnHVVVfpySef1P79+xUTE6P333/f7zeqtLQ0SdJjjz2mnj17KjIyUgMGDFD37t31q1/9Sq+88oq+/vpr3X777SouLtann36q7t27l+v3Glwul7p27Wqpp/w111yjYcOGaePGjWrYsKHefPNNHTp0yOePTt++ffXMM89o6NCh6ty5s7Zu3arZs2f7fJOULnxDjYuL08yZM1WnTh3VqlVL6enpatasmWbMmKF+/fqpffv2Gjp0qBo1aqQdO3Zo+/btfv9YWbFs2TL169fP9Bt7q1at1K1bN6WlpSk+Pl6bNm3S/Pnzyzy2v/3tbzV//nzde++9+vWvf620tDQdO3ZMixYt0syZM9WuXTv96le/0nvvvaff/OY3WrVqlW688UZ5PB7t2LFD7733nrdnwTPPPKPVq1erT58+atq0qQ4fPqw//elPaty4sbp06VJqDLfddpuioqLUr18/PfTQQzp58qRef/11NWjQQAcPHrR8HosWLVLfvn01ZMgQpaWl6dSpU9q6davmz5+vvXv3lnoJ52LfiqysLPXt21e9e/fW559/rr/97W+ml33K8nl57rnndOedd+rGG2/U0KFD9cMPP+jVV19V69atfRKHfv36qXv37vrd736nvXv3ql27dvr73/+uDz/8UKNGjfKZpGhV37599de//lWxsbFq1aqV1q1bp+XLl5f7NmOgXCrzVorqpF+/fkZ0dLRx6tSpUrcZMmSIcdlllxlHjx41DKP8t0AahmE89thjRvPmzX2WXby16f/9v/9njB071mjQoIFRs2ZNo0+fPj63aRmGYXz55ZdGRkaGUbt2baN+/frG8OHDvbdVXXor4Pnz541HH33USEhIMFwul8/tdufPnzdefPFFIzU11YiKijISEhKMXr16Gbm5ud5tJBkjRowoEf9Pb9M7ceKEIckYMGCA6bk3bdrU6NOnj7F06VKjbdu2htvtNlJTU4158+b5bHfmzBnjiSeeMBo1amTUrFnTuPHGG41169YZXbt2Nbp27eqz7Ycffmi0atXKqFGjRokxWLNmjXHrrbcaderUMWrVqmW0bdvW59azwYMHG7Vq1SoRp7/bE7/66itDkrF8+XLT85w0aZLRsWNHIy4uzqhZs6aRmppqPPvss8bZs2cDHuOnY2sYhvH9998bmZmZxhVXXGFERUUZjRs3NgYPHux9LxqGYZw9e9Z44YUXjGuvvdZwu91G3bp1jbS0NGPixIlGQUGBYRiGsWLFCuPOO+80kpKSjKioKCMpKcm47777StyW6M+iRYuMtm3bGtHR0UZKSorxwgsvGG+++WaJ9/zFf19/Tpw4YYwdO9Zo3ry5ERUVZdSvX9/o3LmzMXnyZJ9x8cfj8RgTJ070vh+6detmbNu2rcR4+btF0OrnxTAMY+7cuUZqaqrhdruN1q1bG4sWLTLuvvtuIzU1tcS5PP7440ZSUpJx2WWXGVdffbXx4osv+tyiaRjWP0M//PCDMXToUKN+/fpG7dq1jZ49exo7duywdH6AXVyGUQm1ZJj69ttvlZqaqr/97W/q0aNHVYdTYUuWLFHfvn31xRdflLhdLJyMGjVKq1evVm5urmklAeGjffv2SkhIMO2WCVR3zEkIEVdeeaWGDRum559/vqpDscWqVas0YMCAsE4Qvv/+e73xxhuaNGkSCUKYOnfuXIm5ETk5Ofriiy/UrVu3qgkKqERUEgCgFHv37lVGRobuv/9+JSUlaceOHZo5c6ZiY2O1bds25gcg7DFxEQBKUbduXaWlpemNN97QkSNHVKtWLfXp00fPP/88CQIcgUoCAADwizkJAADAL5IEAADgV6XPSSguLtaBAwdUp04dZoQDgMMYhqETJ04oKSnJ+6uulenMmTM6e/ZsUPYdFRWl6OjooOy7qlR6knDgwAHvj8sAAJwpLy9PjRs3rtRjnjlzRgk1a+qk+ablkpiYqD179oRVolDpScJ/fvykR1UcHgBQpc5LWlHiZ7Urw9mzZ3VS0uOS3Dbvu0jSlPx8nT17liShIv5ziaGGpMsq+/AAgBBQlZeba0qy+894uE7wC9fzAgAAFUS9HwDgKBGy/xtyuH7jDtfzAgAAFUQlAQDgKFQSrCNJAAA4CkmCdeF6XgAAoIJCtpJgfPFs4A1at66cQCqBK/JOW/bz2msfBVz/4APFthwnFNg1ZsZr/QJv8MADthwnFNg1Zqu0OOD6bh6PLccJBXaN2a9+Ffiz+c4sPpuViUqCdeF6XgAAoIJCtpIAAEAwUEmwLlzPCwAAVBCVBACAo1BJsC5czwsAAFQQlQQAgKNQSbAuXM8LAABUUOhWEsz6IDz5pPk+Jk+2J5ZqwrQPwuLA97dLkvr2tSeY6sKsD8Lvf2++j0mT7ImlmjDrg7AyMtJ0H7eEUS8FK8z6IHy5w/z7WqvU8OmlUNVcsv8bctX98HVwhW6SAABAELhk/x/1cE0SuNwAAAD8opIAAHCUyH8/7N5nOKKSAAAA/KKSAABwFG6BtC5czwsAAFQQlQQAgKNQSbAuXM8LAICQNn36dKWkpCg6Olrp6enasGFDwO2PHz+uESNGqFGjRnK73brmmmu0ZMmSoMZYfSsJVhol/d//BV5///32xFJdWGmUNHVq4PWjRtkRSfVhpVHSmjWB13fpYk8s1YSVRkkbTBoudXRYsyUrjZJOnwn8ne7yaJotWRUKlYR3331Xo0eP1syZM5Wenq6pU6eqZ8+e2rlzpxo0aFBi+7Nnz+rWW29VgwYNNH/+fF1xxRX67rvvFBcXZ0v8pam+SQIAAOUQCknCSy+9pOHDh2vo0KGSpJkzZ+rjjz/Wm2++qTFjxpTY/s0339SxY8e0du1aXXbZZZKklJSUCkZtjssNAABUorNnzyo3N1cZGRneZREREcrIyNC6dev8vmbRokXq1KmTRowYoYYNG6p169Z67rnn5Aly1Y1KAgDAUYJZSSgsLPRZ7na75Xa7fZYdPXpUHo9HDRs29FnesGFD7dixw+/+v/32W61cuVIDBw7UkiVLtHv3bj3yyCM6d+6cxo8fb9t5/BSVBAAAbJKcnKzY2FjvIysry5b9FhcXq0GDBvrzn/+stLQ09e/fX7/73e80c+ZMW/ZfGioJAABHCWYlIS8vTzExMd7lP60iSFL9+vUVGRmpQ4cO+Sw/dOiQEhMT/e6/UaNGuuyyyxR5yaTfli1bKj8/X2fPnlVUVFTFT8IPKgkAANgkJibG5+EvSYiKilJaWppWrFjhXVZcXKwVK1aoU6dOfvd74403avfu3Sou/s9dLLt27VKjRo2CliBIJAkAAIdxBelRFqNHj9brr7+ut99+W1999ZUefvhhnTp1ynu3w6BBgzR27Fjv9g8//LCOHTumkSNHateuXfr444/13HPPacSIEeUbBIvKdblh+vTpevHFF5Wfn6927dpp2rRp6tixo92xVZxZH4QzZ8z3ER1tTyzVhUkfhNVrzPPKm7s47H5tsz4IJj0BJEkO6wtg1gdhn4Uxa+KwMTPrg/DlDvPPppV+DKgc/fv315EjRzRu3Djl5+erffv2ys7O9k5m3LdvnyIi/vNvmpycrKVLl+rxxx9X27ZtdcUVV2jkyJF66qmnghpnmZOEsjaAAAAglETI/p92Lk9ZPjMzU5mZmX7X5eTklFjWqVMnrV+/vhxHKr8yn9elDSBatWqlmTNn6vLLL9ebb74ZjPgAALBVRJAe4ahM51WeBhAAAKB6KtPlhvI0gCgqKlJRUZH3+U8bTQAAUJlCoS1zdRH088rKyvJpLJGcnBzsQwIAABuUKUkoTwOIsWPHqqCgwPvIy8srf7QAAFQQcxKsK9N5lacBhNvtLtFcAgAAhL4y3wI5evRoDR48WB06dFDHjh01depUnwYQAACEMuYkWFfmJMGsAUS1YqVR0vHjgdfHxdkRSbVhpVHSM5MCf1zG/d5hDV2sNP159dXA60u5lzpcWWmUlGPScKmbw5otWWmU9KeZgT+bj/zGYZ9NmCpXx8VADSAAAAhlVBKs41cgAQCOQpJgXbieFwAAqCAqCQAARynPrzZa2Wc4opIAAAD8opIAAHCUSNn/K5B27y9UUEkAAAB+UUkwY9IHYe168zyr8w3OuvfYrA9C4UnzMYup7awxM+uDcOiI+Zg1THDWmJn1QThu0kdBkuIc1kvBrA/Crt3m77Nrmlf/95lL9n9DZk4CAABwFCoJAABHoU+CdSQJAABHIUmwLlzPCwAAVBCVBACAo1BJsC5czwsAAFQQlQQAgKNQSbAuXM8LAABUEJWECrLSKKmYXMyHlUZJ785jzC5lqVHSk08GP5BqxEqjpLWmDZf62hNMNWGlUZKVBnKhjkqCdeF6XgAAoIKoJAAAHIWfiraOJAEA4Cj8CqR1XG4AAAB+UUkAADgKExetC9fzAgAAFUQlAQDgKC7Z/w2ZiYuVzBV5Z1WHUO0wZmXHmJVdeI1Z5fRBCK8xg5OEbJIAAEAwMCfBunA9LwAAUEFUEgAAjkIlwTqSBACAo5AkWBeu5wUAACqISgIAwFGoJFgXrucFAAAqiEoCAMBR+BVI66gkAAAAv6gkAAAchZ+Kto5KAgAA8ItKAgDAUbi7wTqSBACAo5AkWBeu5wUAACqISgIAwFFcsv8bMrdAAgAAR6GSAABwFOYkWBfCScJvAq41PL0qKY7gc0Xeact+3n77o4DrB91fbMtxQoFdYyaNC7jW8KTZdJyqZ9eYGSnbAm/wzTe2HCcU2DZm/UyK0QsX2nKcUGDfZxOhIISTBAAA7EclwbpwPS8AAFBBVBIAAI5CJcG6cD0vAABQQVQSAACOwk9FW0clAQDgKJFBepTV9OnTlZKSoujoaKWnp2vDhg2WXjd37ly5XC7ddddd5Thq2ZAkAABQyd59912NHj1a48eP1+bNm9WuXTv17NlThw8fDvi6vXv36sknn9RNN91UKXGG7OUGsz4I+/5lnt80aRw+fQGsCKc+CJXFrA/Cyhzz99kt3Rw27mZ9ENasMd9Hly72xFJdmPVBmDXLfB9DhtgQCKTQmLj40ksvafjw4Ro6dKgkaebMmfr444/15ptvasyYMX5f4/F4NHDgQE2cOFGffvqpjh8/XrGgLaCSAABAJTp79qxyc3OVkZHhXRYREaGMjAytW7eu1Nc988wzatCggYYNG1YZYUoK4UoCAADBEMxKQmFhoc9yt9stt9vts+zo0aPyeDxq2LChz/KGDRtqx44dfve/Zs0a/eUvf9GWLVvsCtkSKgkAANgkOTlZsbGx3kdWVlaF93nixAn96le/0uuvv6769evbEKV1VBIAAI4SzEpCXl6eYmJivMt/WkWQpPr16ysyMlKHDh3yWX7o0CElJiaW2P6bb77R3r171a9fP++y4uILc6Fq1KihnTt36qqrrrLhLEqikgAAgE1iYmJ8Hv6ShKioKKWlpWnFihXeZcXFxVqxYoU6depUYvvU1FRt3bpVW7Zs8T7uuOMOde/eXVu2bFFycnLQzodKAgDAUVyy/xtyWZspjR49WoMHD1aHDh3UsWNHTZ06VadOnfLe7TBo0CBdccUVysrKUnR0tFq3bu3z+ri4OEkqsdxuJAkAAEcJhVsg+/fvryNHjmjcuHHKz89X+/btlZ2d7Z3MuG/fPkVEVH2xnyQBAIAqkJmZqczMTL/rcnJyAr52lpXeGjaotkmClUZJn20MnIWlX++wJjgWnD4TeMwuj3bWmFlplPTt3sBjdmWKs8bMUqMks+ZCldBuNqRYaZRk9keBZkuWhUIloboI1/MCAAAVVG0rCQAAlAeVBOvC9bwAAEAFUUkAADgKlQTrwvW8AABABVFJAAA4CpUE60gSAACOQpJgXZmThNWrV+vFF19Ubm6uDh48qAULFuiuEL2n2awPQnx983/WY0eddY+7WR8Es54AkvP6Apidb1S0+ZidPeOsMTPtgzBhgvk+rGwTTsz6IAwYYL6PuXNtCQXOUebk59SpU2rXrp2mT58ejHgAAAiqiCA9wlGZKwm9evVSr169ghELAAAIIcxJAAA4CnMSrAt6klBUVKSioiLv88LCwmAfEgAA2CDoyU9WVpZiY2O9j+Tk5GAfEgCAUrkkuVwuex9VfVJBEvQkYezYsSooKPA+8vLygn1IAABgg6BfbnC73XK73cE+DAAA1tSoIbls/u5vGNL58/buMwSUOUk4efKkdu/e7X2+Z88ebdmyRfHx8WrSpImtwQEAYDuSBMvKnCRs2rRJ3bt39z4fPXq0JGnw4MGaNWuWbYFVBiuNklyRuQHXG540u8KpFqw0SlqSHfgqVu/bndU4yEqjJFfkuoDrDU8nu8KpHiw0StodGRlwfXOPx6ZgqgkrjZLMGi7RbAk/UeYkoVu3bjIMIxixAAAQfMGqJIShcL21EwAAVBDNlAAAzkIlwTIqCQAAwC8qCQAAZ4mMlCJs/o5cHJ4TsqkkAAAAv6gkAACcpUYNKgkWkSSYMOuD8Mwk8zfauN+H55unNGZ9EHbtNh+za5o7a8zM+iDMmWs+Zr8c4KwxM+2DkJlpvpNXX7UnmOrCrA/Cpk3m++jQwZ5YqhJJgmVcbgAAAH5RSQAAOAuVBMuoJAAAAL+oJAAAnCUy8sLDTmH6WyFUEgAAgF9UEgAAzlKjhv2VBLvbPIcIKgkAAMAvKgkAAGehkmAZSUIFWWmU5IrMrYRIqg8rjZJW5lDkupSVRkmuyO8qIZJqxEKjpC2mfyj62hNLdWGlUdKoUUEPI+hIEizj/8QAAMAvKgkAAGehkmAZlQQAAOAXlQQAgLNERl6oJsAUlQQAAOAXqRQAwFlq1KCSYBGVBAAA4FfIplKuyDurOoRqhzErO8as7MJrzCqnD0J4jVkYoJJgGaMEAHAWkgTLuNwAAAD8IpUCADhLMG6BNAx79xciqCQAAAC/qCQAAJwlGHMSqCQAAAAnoZIAAHAWKgmWUUkAAAB+UUkAADgLlQTLSBIAAM5CkmAZlxsAAIBfVBIAAM4SjGZKxcX27i9EUEkAAAB+UUkAADhLMOYkUEkAAAB2mT59ulJSUhQdHa309HRt2LCh1G1ff/113XTTTapbt67q1q2rjIyMgNvbhSQBAOAsFysJdj/K4N1339Xo0aM1fvx4bd68We3atVPPnj11+PBhv9vn5OTovvvu06pVq7Ru3TolJyfrtttu0/79++0YkVKRJAAAUMleeuklDR8+XEOHDlWrVq00c+ZMXX755XrzzTf9bj979mw98sgjat++vVJTU/XGG2+ouLhYK1asCGqcITsnwcidGHiD9u0rJY7K4Iq805b9PP/8RwHXP/Xb8LlmZteYmTE8H1bKcSqDXWN2/fWB32cb1vM++6mEhMBjdjifMatUQZyTUFhY6LPY7XbL7Xb7LDt79qxyc3M1duxY77KIiAhlZGRo3bp1lg53+vRpnTt3TvHx8RUMPDAqCQAA2CQ5OVmxsbHeR1ZWVoltjh49Ko/Ho4YNG/osb9iwofLz8y0d56mnnlJSUpIyMjJsibs0IVtJAAAgKIJYScjLy1NMTIx38U+rCHZ4/vnnNXfuXOXk5Cg6Otr2/V+KJAEA4CzBaKbk8UiSYmJifJIEf+rXr6/IyEgdOnTIZ/mhQ4eUmJgY8LWTJ0/W888/r+XLl6tt27YVi9kCLjcAAFCJoqKilJaW5jPp8OIkxE6dOpX6uv/93//VH/7wB2VnZ6tDhw6VESqVBACAwwTjcsO/KwlWjR49WoMHD1aHDh3UsWNHTZ06VadOndLQoUMlSYMGDdIVV1zhndPwwgsvaNy4cZozZ45SUlK8cxdq166t2rVr23sulyBJAACgkvXv319HjhzRuHHjlJ+fr/bt2ys7O9s7mXHfvn2KiPhPsX/GjBk6e/as7rnnHp/9jB8/XhMmTAhanCQJAABnCYFKgiRlZmYqMzPT77qcnByf53v37i1HUBUXukmCWR+E5cvN9xHkW0NCjVkfhD+/YT4F5cEHwud+bSvM+iC4Is0/+IYn0q5wqgWzPgi/vN/8fTbn/5z1PjPrg/DMJPMxG/d7Z40ZQkPoJgkAAARDiFQSqgPubgAAAH5RSQAAOEsw+iScP2/v/kIESQIAwFmCcbnB7v2FCC43AAAAv8Iz9QEAoDRUEiyjkgAAAPwKz9QHAIDSUEmwrPqelZVGSWa/y23ya1vhxkqjpF27AxeXrmnurIYuVholfbAw8Jj9113OGjMrjZLWrg88Zp1vcNaYWWmU9M9tgcesbWtnjRkqR/VNEgAAKA8qCZYxJwEAAPgVnqkPAAClCUYzpcjw/A0XkgQAgLNwucEyLjcAAAC/wjP1AQCgNFQSLKOSAAAA/CpT6pOVlaUPPvhAO3bsUM2aNdW5c2e98MILatGiRbDiqxizPggnT5rvo3Zte2KpJsz6IBw6Yp5XNkxw1v3aZn0QPttoPmbp1ztrzMz6IHy5w3zMWqU6a8zM+iDw2SwDKgmWlamS8Mknn2jEiBFav369li1bpnPnzum2227TqVOnghUfAACoImVKfbKzs32ez5o1Sw0aNFBubq5uvvlmWwMDACAouAXSsgrNSSgoKJAkxcfH2xIMAAAIHeVOpYqLizVq1CjdeOONat26danbFRUVqaioyPu8sLCwvIcEAKDimJNgWbnPasSIEdq2bZvWrFkTcLusrCxNnDixvIcBAMBeJAmWletyQ2ZmphYvXqxVq1apcePGAbcdO3asCgoKvI+8vLxyBQoAACpXmVIfwzD06KOPasGCBcrJyVGzZs1MX+N2u+V2u8sdIAAAtqKSYFmZzmrEiBGaM2eOPvzwQ9WpU0f5+fmSpNjYWNWsWTMoAQIAgKpRpiRhxowZkqRu3br5LH/rrbc0ZMgQu2KqPFYaJR0/Hnh9XJwdkVQbVpqx/PmNwFexHnzAWQ1drDRKckX+JuB6wzPTrnCqBSuNku64K/D7bNFCZ73PrHw24+sHHrNjRx0yZlQSLCvz5QYAAOAM4Zn6AABQGpopWcYPPAEAAL+oJAAAnIU5CZaF51kBAFAakgTLuNwAAAD8Cs/UBwCA0lBJsCw8z8pOZn0Qtm0z30eAH8AKR2Z9EEY/aV7AemmyQ+7X/jezPgiuyL9Y2Mcwu8KpFsz6IPxigPn77L25znqfmfVBcEWa3+ZueFx2hYNqgCQBAOAs3AJpGXMSAACAX1QSAADOwpwEy6gkAAAAv8Iz9QEAoDRUEiwLz7MCAKA0JAmWcbkBAAD4FZ6pDwAApaGSYFl4nlVlstIo6eTJ4MdRjVhplPTgbyhyXcpKoyRXZOCGTE5jpVHSlc15n13KSqOkjjcwZk5CkgAAcBaaKVlGSggAAPyikgAAcBbmJFhGJQEAAPgVnqkPAACloZJgWXieFQAApSFJsIzLDQAAwK+QTX1ckXdWdQjVDmNWdoxZ2TFmZceYhRhugbSMSgIAAPArZCsJAAAEBXMSLKOSAAAA/ArP1AcAgNJQSbCMSgIAAFVg+vTpSklJUXR0tNLT07Vhw4aA28+bN0+pqamKjo5WmzZttGTJkqDHSJIAAHCWi5UEux9l8O6772r06NEaP368Nm/erHbt2qlnz546fPiw3+3Xrl2r++67T8OGDdPnn3+uu+66S3fddZe2bdtmx4iUymUYhhHUI/xEYWGhYmNjJfWUdFllHhoAUOXOSVqqgoICxcTEVOqRL/79KfjhB9uPXVhYqNi6dS2fV3p6uq6//nq9+uqrkqTi4mIlJyfr0Ucf1ZgxY0ps379/f506dUqLFy/2LrvhhhvUvn17zZwZvJ+Jp5IAAIBNCgsLfR5FRUUltjl79qxyc3OVkZHhXRYREaGMjAytW7fO737XrVvns70k9ezZs9Tt7UKSAABwlGJFBOUhScnJyYqNjfU+srKyShz/6NGj8ng8atiwoc/yhg0bKj8/32/M+fn5ZdreLuE5HRMAgCqQl5fnc7nB7XZXYTQVR5IAAHCU8+cvPOzepyTFxMSYzkmoX7++IiMjdejQIZ/lhw4dUmJiot/XJCYmlml7u3C5AQCAShQVFaW0tDStWLHCu6y4uFgrVqxQp06d/L6mU6dOPttL0rJly0rd3i5UEgAAjhLMSoJVo0eP1uDBg9WhQwd17NhRU6dO1alTpzR06FBJ0qBBg3TFFVd45zSMHDlSXbt21R//+Ef16dNHc+fO1aZNm/TnP//Z3hP5CZIEAAAqWf/+/XXkyBGNGzdO+fn5at++vbKzs72TE/ft26eIiP8U+zt37qw5c+bo97//vZ5++mldffXVWrhwoVq3bh3UOOmTAACoRFXfJyEvz/5jFxYWKjk5tkrOK5ioJAAAHCUULjdUF0xcBAAAfoVsJeGmmz4KuH51TnElRRJ8rsg7bdmP8XOTyzfz59tynFBg15hJ9wdca3jutek4Vc+uMZs7N/Bns/+9fDZ/6sMPA4/ZHX0Zs8rk8dj/zd/jsXd/oYJKAgAA8CtkKwkAAAQDcxKso5IAAAD8opIAAHAUKgnWUUkAAAB+UUkAADgKlQTrqCQAAAC/QraSYNYHYddu8/zmmubhc++xJWZ9ENasMd9Hly72xFJNmPVBePr35u+z5yY5631m1gfhnf8zH7NB9ztrzMz6IKzMMR+zW7o5a8yCiT4J1oVskgAAQDBwucE6LjcAAAC/qCQAAByFSoJ1VBIAAIBfVBIAAI5CJcE6KgkAAMAvKgkAAEehkmAdlQQAAOBXta0kWGmUVGySA0XIYc1JrDRKOn488Pq4ODsiqTasNEravCXw++y69s56n1lplMSY+bLSKOmf2wKPWdvWzhqziqCZknXVNkkAAKA8uNxgHZcbAACAX1QSAACOQiXBOioJAADALyoJAABHoZJgHZUEAADgF5UEAICjcAukdWVKEmbMmKEZM2Zo7969kqRrr71W48aNU69evYIRW4WZ9UFYkm1eSOl9u8PuPTbrg7B4sfk++va1JZTqwuye/sdGmb/PXpnqrPeZ2Zjt2m0+ZlZ6pYQTsz4Ip8+Yj9nl0c4aM1RcmZKExo0b6/nnn9fVV18twzD09ttv684779Tnn3+ua6+9NlgxAgBgG+YkWFemJKFfv34+z5999lnNmDFD69evJ0kAAFQLJAnWlXtOgsfj0bx583Tq1Cl16tTJzpgAAEAIKHOSsHXrVnXq1ElnzpxR7dq1tWDBArVq1arU7YuKilRUVOR9XlhYWL5IAQCwAZUE68p8C2SLFi20ZcsWffbZZ3r44Yc1ePBgffnll6Vun5WVpdjYWO8jOTm5QgEDAIDKUeYkISoqSs2bN1daWpqysrLUrl07vfzyy6VuP3bsWBUUFHgfeXl5FQoYAICKuFhJsPsRjircJ6G4uNjncsJPud1uud3uih4GAABUsjIlCWPHjlWvXr3UpEkTnThxQnPmzFFOTo6WLl0arPgAALAVzZSsK1OScPjwYQ0aNEgHDx5UbGys2rZtq6VLl+rWW28NVnxBZaVRUstrA1+R+Wq7w5qTWGmU9MADgde/8YY9sVQTVholuSL/FnC94QnNhmXBYqVR0tO/D/zZfG6Ssz6bVholfbAw8Jj9113OGjOYK1OS8Je//CVYcQAAUCm4u8E6frsBAOAoJAnW8SuQAADALyoJAABHoZJgHZUEAADgF5UEAICjcAukdVQSAACAX1QSTJj1QVi73jzP6nyDw+49NuuDcOaM+T6io+2JpZow64Pwyqvm77PHMp31PjPrg7BosfmY3dHXWWNm1gdhzlzzMfvlgOo/ZsxJsI5KAgAA8ItKAgDAUagkWEeSAABwFJIE67jcAAAA/KKSAABwFCoJ1lFJAAAAflFJAAA4Cs2UrKOSAAAA/KKSUEFWGiVd2ZxczIeVRkmZmcGPoxqx0ijJFbm4EiKpPqw0SnpsFJ/NS1lplPTuvOo/ZufPS5GR9u8zHFX/f20AABAUVBIAAI5CJcE6KgkAAEe5eAuk3Y9gOXbsmAYOHKiYmBjFxcVp2LBhOnnyZMDtH330UbVo0UI1a9ZUkyZN9Nhjj6mgoKDMxyZJAAAghA0cOFDbt2/XsmXLtHjxYq1evVoPPvhgqdsfOHBABw4c0OTJk7Vt2zbNmjVL2dnZGjZsWJmPzeUGAICjVKdbIL/66itlZ2dr48aN6tChgyRp2rRp6t27tyZPnqykpKQSr2ndurXef/997/OrrrpKzz77rO6//36dP39eNWpY/9NPJQEAAJsUFhb6PIqKiiq0v3Xr1ikuLs6bIEhSRkaGIiIi9Nlnn1neT0FBgWJiYsqUIEgkCQAAhwnmnITk5GTFxsZ6H1lZWRWKNT8/Xw0aNPBZVqNGDcXHxys/P9/SPo4ePao//OEPAS9RlCZkLze4Iu+s6hCqHcas7BizsmPMyo4xc468vDzFxMR4n7vdbr/bjRkzRi+88ELAfX311VcVjqewsFB9+vRRq1atNGHChDK/PmSTBAAAguH8eSnC5jr6xUpCTEyMT5JQmieeeEJDhgwJuM2VV16pxMREHT58+CfHOq9jx44pMTEx4OtPnDih22+/XXXq1NGCBQt02WWXmcb1UyQJAABUsoSEBCUkJJhu16lTJx0/fly5ublKS0uTJK1cuVLFxcVKT08v9XWFhYXq2bOn3G63Fi1apGgrnW79YE4CAMBRqlOfhJYtW+r222/X8OHDtWHDBv3jH/9QZmamBgwY4L2zYf/+/UpNTdWGDRskXUgQbrvtNp06dUp/+ctfVFhYqPz8fOXn58tTxtswqCQAABwlmJcbgmH27NnKzMxUjx49FBERobvvvluvvPKKd/25c+e0c+dOnT59WpK0efNm750PzZs399nXnj17lJKSYvnYJAkAAISw+Ph4zZkzp9T1KSkpMgzD+7xbt24+zyuCJAEA4CjVqZlSVWNOAgAA8ItKAgDAUc6fl1wu+/cZjqgkAAAAv6gkAAAchUqCdVQSAACAX1QSAACOQiXBOpIEAICjkCRYx+UGAADgF5UEAICjeDz2VxJopgQAAByFSgIAwFGCMX8gXOckhHCS8ELAtYYntZLiCD5X5J227Mcoej/wBjVC+J+7jOwaM2lWwLWGp65Nx6l6do1ZkRYHXB8VRnVXu8bsiy8+Cri+betiW44TCuz7bCIUhM9fDQAALKCSYB1zEgAAgF9UEgAAjkIlwToqCQAAwC8qCQAARwnG3Nowmq/rgyQBAOAo589LhmHvPsM1SeByAwAA8CtkKwlmfRAKT5rnNzG1w+feY0vCqA9CZTHrg/DKq+bvs8cynfU+M+2DsH69+U5uuMGeYKoJsz4I/9xm/j4Lp14KVY1KgnVUEgAAgF989QQAOAqVBOuoJAAAAL+oJAAAHIVKgnVUEgAAgF9UEgAAjuLx2F9JKA7Tm09IEgAAjnL+vBRhcx09XJMELjcAAAC/qm0lwUqjpNNnAudAl0eHaepXAWfPBx6zqBrOGjMrjZJW5gQes1u6OWvMLDVK2rEj8PrUwM3Uwo2VRkm7dgd+n13T3GHvswqgkmAdlQQAAOBXta0kAABQHlQSrKOSAAAA/KKSAABwFI/H/m/+dt9SGSqoJAAAAL+oJAAAHOX8ecnlsnef4VpJIEkAADgKSYJ1FUoSnn/+eY0dO1YjR47U1KlTbQrJPmZ9EP78hvnVlgcfCNMpq6Uw64NQeNJ8zKz0sAgnZn0Qfnm/+ZjN+T9njZlpH4Q1a8z30aWLPbFUE2Z9ED5YaP4++6+7HPY+Q4WVO0nYuHGjXnvtNbVt29bOeAAACCoqCdaVa+LiyZMnNXDgQL3++uuqW7eu3TEBAIAQUK4kYcSIEerTp48yMjLsjgcAgKA6fz44j3BU5ssNc+fO1ebNm7Vx40ZL2xcVFamoqMj7vLCwsKyHBAAAVaBMlYS8vDyNHDlSs2fPVnR0tKXXZGVlKTY21vtITk4uV6AAANijWIZh70MKz0mhZUoScnNzdfjwYV133XWqUaOGatSooU8++USvvPKKatSoIY/HU+I1Y8eOVUFBgfeRl5dnW/AAACB4ynS5oUePHtq6davPsqFDhyo1NVVPPfWUIiMjS7zG7XbL7XZXLEoAAGzj+ffD7n2GnzIlCXXq1FHr1q19ltWqVUv16tUrsRwAgNBEkmCVozsuWmmU5Io8FHC94WloVzjVgpVGSd/uDXwV68qU8Lx2VxorjZJckasDrjc8N9sVTvVgoVHSWT+Vy0tF+bn8Gc6sNEpauz7wZ7PzDc76bMJchZOEnJwcG8IAAKCyUEmwil+BBAAAfjn6cgMAwImCcctieF6qoZIAAAD8opIAAHAY5iRYRSUBAAD4RSUBAOAwxbL/m394zkkgSTBh1gfhpanmxZjRo8LzzVMasz4In200H7P06501ZmZ9EAYNMR+zd2Y5a8xM+yD8/vfmO5k0yZ5gqgmzPghmfRSs7KN64HKDVVxuAAAAfpEkAAAcxhOkR3AcO3ZMAwcOVExMjOLi4jRs2DCdPHnS0msNw1CvXr3kcrm0cOHCMh+bJAEAgBA2cOBAbd++XcuWLdPixYu1evVqPfjgg5ZeO3XqVLlcrnIfmzkJAACHqT7NlL766itlZ2dr48aN6tChgyRp2rRp6t27tyZPnqykpKRSX7tlyxb98Y9/1KZNm9SoUaNyHZ9KAgAAIWrdunWKi4vzJgiSlJGRoYiICH322Welvu706dP65S9/qenTpysxMbHcx6eSAABwmODd3VBYWOiz1O12y+12l3uv+fn5atCggc+yGjVqKD4+Xvn5+aW+7vHHH1fnzp115513lvvYEpUEAABsk5ycrNjYWO8jKyvL73ZjxoyRy+UK+NixY0e5Yli0aJFWrlypqVOnVuBMLqCSAABwmOBVEvLy8hQTE+NdWloV4YknntCQIUMC7vHKK69UYmKiDh8+7LP8/PnzOnbsWKmXEVauXKlvvvlGcXFxPsvvvvtu3XTTTcrJyQl8KpcgSaggK42SXJFnKyGS6sNKo6R9/6LIdSkrjZJckf6/sTiWhUZJJyMjTbboa08s1YSVRklWGnuFvuAlCTExMT5JQmkSEhKUkJBgul2nTp10/Phx5ebmKi0tTdKFJKC4uFjp6el+XzNmzBg98MADPsvatGmjKVOmqF+/fqbHvBRJAgAAIaply5a6/fbbNXz4cM2cOVPnzp1TZmamBgwY4L2zYf/+/erRo4feeecddezYUYmJiX6rDE2aNFGzZs3KdPxwSAkBACiDi7/dYOcjeO2qZ8+erdTUVPXo0UO9e/dWly5d9Oc//9m7/ty5c9q5c6dOnz5t+7GpJAAAEMLi4+M1Z86cUtenpKTIMIyA+zBbXxqSBACAw1SfZkpVjcsNAADALyoJAACH4aeiraKSAAAA/ArZSoIrsmKtJJ2IMSs7xqzswmvMKqcPQniNWTigkmBVyCYJAAAEB0mCVVxuAAAAflFJAAA4DJUEq6gkAAAAv6gkAAAcxpD9zY/K19Ew1FFJAAAAflFJAAA4DHMSrKKSAAAA/KKSAABwGCoJVpEkAAAchiTBKi43AAAAv6gkAAAchkqCVVQSAACAX1QSAAAOUyz7mynZvb/QQCUBAAD4RSUBAOAwzEmwikoCAADwK2QrCbm5HwVcf1378Ln+44q80579uAKPWfF5xqykXwdca3jsOk7Vs2vMjmhxwPX1PeHzjcq+99lzAdcanmttOk7Vs2/MgqlY9n/zD5//v16KSgIAAPArZCsJAAAEB3MSrCJJAAA4DLdAWsXlBgAA4BeVBACAw3C5wSoqCQAAwC8qCQAAh6GSYFXIJgmmfRBycsx30q2bHaFUG2Z9ENauNy8cdb4hPCfflMasD8LoJ83H7KXJzhoz0z4Iy5eb7yQjw55gqgmzPgiLFpu/z+7o66z3GUJDyCYJAAAEB5UEq5iTAAAA/KKSAABwGNoyW0WSAABwGJopWcXlBgAA4BeVBACAwzBx0SoqCQAAwC8qCQAAh6GSYFX1TRKsNErauzfw+pQUGwKpPqw0Str3r8DFpSaNw3NyTmmsNEqaMzfwmP1ygLPGzFKjJD6bPqw0SvpyR+D3WatUh73PUCmqb5IAAEC5UEmwijkJAADALyoJAACHoZJgFUkCAMBhaKZkFZcbAACAX1QSAAAOw283WEUlAQAA+FWmSsKECRM0ceJEn2UtWrTQjh07bA3KNmb3Wp85Y76P6GhbQqkuzPogfLbRPK9Mvz48M+rSmPVBWL3GfMxu7uKsMTP7bBZb+P4SEabf3Epj1gdh7XrzMbPSK8UZmLhoVZkvN1x77bVavnz5f3ZQgysWAACEozL/ha9Ro4YSExODEQsAAJWASoJVZZ6T8PXXXyspKUlXXnmlBg4cqH379gUjLgAAUMXKVElIT0/XrFmz1KJFCx08eFATJ07UTTfdpG3btqlOnTp+X1NUVKSioiLv88LCwopFDABAhVBJsKpMSUKvXr28/922bVulp6eradOmeu+99zRs2DC/r8nKyiox2REAgKpDkmBVhW6BjIuL0zXXXKPdu3eXus3YsWNVUFDgfeTl5VXkkAAAoJJUKEk4efKkvvnmGzVq1KjUbdxut2JiYnweAABUneIgPcJPmZKEJ598Up988on27t2rtWvX6uc//7kiIyN13333BSs+AABQRco0J+Ff//qX7rvvPn3//fdKSEhQly5dtH79eiUkJAQrvuCy0CjJrBGO05rgWGmUFFEj8JgVn3fWmFl5j7gi5wVcb3jutSucasFKo6TVkZEB19/sCc9rxKWx0ijJFVkQcL3hibUrnBBXvdoyHzt2TI8++qg++ugjRURE6O6779bLL7+s2rVrB3zdunXr9Lvf/U6fffaZIiMj1b59ey1dulQ1a9a0fOwyJQlz584ty+YAAKCCBg4cqIMHD2rZsmU6d+6chg4dqgcffFBz5swp9TXr1q3T7bffrrFjx2ratGmqUaOGvvjiC0VElG2WAe0SAQAOU33ubvjqq6+UnZ2tjRs3qkOHDpKkadOmqXfv3po8ebKSkpL8vu7xxx/XY489pjFjxniXtWjRoszH5weeAAAIUevWrVNcXJw3QZCkjIwMRURE6LPPPvP7msOHD+uzzz5TgwYN1LlzZzVs2FBdu3bVmjVrynx8kgQAgMN4gvS40DDw0selzQTLIz8/Xw0aNPBZVqNGDcXHxys/P9/va7799ltJF36Ucfjw4crOztZ1112nHj166Ouvvy7T8UkSAAAOE7xbIJOTkxUbG+t9ZGVl+Y1gzJgxcrlcAR/l/YXl4uILsTz00EMaOnSofvazn2nKlClq0aKF3nzzzTLtizkJAADYJC8vz6cfkNvt9rvdE088oSFDhgTc15VXXqnExEQdPnzYZ/n58+d17NixUn9s8WLvolatWvksb9myZZl/b4kkAQDgMMGbuGi1aWBCQoKl9gGdOnXS8ePHlZubq7S0NEnSypUrVVxcrPT0dL+vSUlJUVJSknbu3OmzfNeuXT4/r2AFSYIJs3vcV+aYX7G5pZuz+gKY9UEYN8F8zJ6Z4KwxM+uD4Io0LzsanlS7wqkWTPsgXHWV+U6++caeYKoJsz4Ivxhg/tl8b66zPptVrWXLlrr99ts1fPhwzZw5U+fOnVNmZqYGDBjgvbNh//796tGjh9555x117NhRLpdLv/3tbzV+/Hi1a9dO7du319tvv60dO3Zo/vz5ZTo+SQIAwGGqzy2QkjR79mxlZmaqR48e3mZKr7zyinf9uXPntHPnTp0+fdq7bNSoUTpz5owef/xxHTt2TO3atdOyZct0lZXk+RIkCQAAhLD4+PiAjZNSUlJkGEaJ5WPGjPHpk1AeJAkAAIepXpWEqsQtkAAAwC8qCQAAh6leP/BUlUgSAAAO85/mR/buM/xwuQEAAPhFJQEA4DBMXLSKJKGCrDRK+nYvBZtLWWmUdHM3xuxSVholuSKPVkIk1YiVRkn33x/8OKoRK42SOt7AZ9NJSBIAAA5DJcEqUkIAAOAXlQQAgMNQSbCKSgIAAPCLSgIAwGGoJFhFkgAAcBg6LlrF5QYAAOBXyFYSXJF3VnUI1Q5jVnaMWdkxZmXHmIUa2jJbRSUBAAD4FbKVBAAAgsMj+78jh+fERSoJAADALyoJAACHoZJgFZUEAADgF5UEAIDDUEmwiiQBAOAw3AJpVaUnCYZh/Pu/zlf2oQEAVe7C//v/87eg6mII/X1WvUpPEk6cOPHv/1pR2YcGAISIEydOKDY2tlKPGRUVpcTEROXnLw/K/hMTExUVFRWUfVcVl1HJ6VxxcbEOHDigOnXqyOVyVeahAyosLFRycrLy8vIUExNT1eFUC4xZ2TFmZceYlV0oj5lhGDpx4oSSkpIUEVH5c+fPnDmjs2fPBmXfUVFRio6ODsq+q0qlVxIiIiLUuHHjyj6sZTExMSH3oQp1jFnZMWZlx5iVXaiOWWVXEC4VHR0ddn/Ig4lbIAEAgF8kCQAAwC+ShH9zu90aP3683G53VYdSbTBmZceYlR1jVnaMGexS6RMXAQBA9UAlAQAA+EWSAAAA/CJJAAAAfpEkAAAAv0gS/m369OlKSUlRdHS00tPTtWHDhqoOKWStXr1a/fr1U1JSklwulxYuXFjVIYW8rKwsXX/99apTp44aNGigu+66Szt37qzqsELajBkz1LZtW29DoE6dOulvf/tbVYdVbTz//PNyuVwaNWpUVYeCaowkQdK7776r0aNHa/z48dq8ebPatWunnj176vDhw1UdWkg6deqU2rVrp+nTp1d1KNXGJ598ohEjRmj9+vVatmyZzp07p9tuu02nTp2q6tBCVuPGjfX8888rNzdXmzZt0i233KI777xT27dvr+rQQt7GjRv12muvqW3btlUdCqo5boGUlJ6eruuvv16vvvqqpAu/L5GcnKxHH31UY8aMqeLoQpvL5dKCBQt01113VXUo1cqRI0fUoEEDffLJJ7r55purOpxqIz4+Xi+++KKGDRtW1aGErJMnT+q6667Tn/70J02aNEnt27fX1KlTqzosVFOOryScPXtWubm5ysjI8C6LiIhQRkaG1q1bV4WRIZwVFBRIuvBHD+Y8Ho/mzp2rU6dOqVOnTlUdTkgbMWKE+vTp4/P/NKC8Kv0HnkLN0aNH5fF41LBhQ5/lDRs21I4dO6ooKoSz4uJijRo1SjfeeKNat25d1eGEtK1bt6pTp046c+aMateurQULFqhVq1ZVHVbImjt3rjZv3qyNGzdWdSgIE45PEoDKNmLECG3btk1r1qyp6lBCXosWLbRlyxYVFBRo/vz5Gjx4sD755BMSBT/y8vI0cuRILVu2jF85hG0cnyTUr19fkZGROnTokM/yQ4cOKTExsYqiQrjKzMzU4sWLtXr16pD+yfRQERUVpebNm0uS0tLStHHjRr388st67bXXqjiy0JObm6vDhw/ruuuu8y7zeDxavXq1Xn31VRUVFSkyMrIKI0R15Pg5CVFRUUpLS9OKFSu8y4qLi7VixQqufcI2hmEoMzNTCxYs0MqVK9WsWbOqDqlaKi4uVlFRUVWHEZJ69OihrVu3asuWLd5Hhw4dNHDgQG3ZsoUEAeXi+EqCJI0ePVqDBw9Whw4d1LFjR02dOlWnTp3S0KFDqzq0kHTy5Ent3r3b+3zPnj3asmWL4uPj1aRJkyqMLHSNGDFCc+bM0Ycffqg6deooPz9fkhQbG6uaNWtWcXShaezYserVq5eaNGmiEydOaM6cOcrJydHSpUurOrSQVKdOnRJzXGrVqqV69eox9wXlRpIgqX///jpy5IjGjRun/Px8tW/fXtnZ2SUmM+KCTZs2qXv37t7no0ePliQNHjxYs2bNqqKoQtuMGTMkSd26dfNZ/tZbb2nIkCGVH1A1cPjwYQ0aNEgHDx5UbGys2rZtq6VLl+rWW2+t6tAAx6BPAgAA8MvxcxIAAIB/JAkAAMAvkgQAAOAXSQIAAPCLJAEAAPhFkgAAAPwiSQAAAH6RJAAAAL9IEgAAgF8kCQAAwC+SBAAA4BdJAgAA8Ov/A21Tx0bbW49hAAAAAElFTkSuQmCC\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "def plot_as_patches(j):\n", " # Reorder axes so the diagonals will each form a contiguous patch.\n", " j = tf.transpose(j, [1, 0, 3, 2])\n", " # Pad in between each patch.\n", " lim = tf.reduce_max(abs(j))\n", " j = tf.pad(j, [[0, 0], [1, 1], [0, 0], [1, 1]],\n", " constant_values=-lim)\n", " # Reshape to form a single image.\n", " s = j.shape\n", " j = tf.reshape(j, [s[0]*s[1], s[2]*s[3]])\n", " imshow_zero_center(j, extent=[-0.5, s[2]-0.5, s[0]-0.5, -0.5])\n", "\n", "plot_as_patches(j)\n", "_ = plt.title('All (batch, batch) slices are diagonal')" ] }, { "cell_type": "markdown", "metadata": { "id": "OXpTBKyeK84z" }, "source": [ "期待どおりの結果を得るためには、重複する `batch` 次元を合計するか、`tf.einsum` を使用して対角を選択します。" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:26.405798Z", "iopub.status.busy": "2022-12-14T20:58:26.405150Z", "iopub.status.idle": "2022-12-14T20:58:26.410927Z", "shell.execute_reply": "2022-12-14T20:58:26.410298Z" }, "id": "v65OAjEgLQwl" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(7, 6, 5)\n", "(7, 6, 5)\n" ] } ], "source": [ "j_sum = tf.reduce_sum(j, axis=2)\n", "print(j_sum.shape)\n", "j_select = tf.einsum('bxby->bxy', j)\n", "print(j_select.shape)" ] }, { "cell_type": "markdown", "metadata": { "id": "zT_VfR6lcwxD" }, "source": [ "そもそも、余分な次元なしで計算を行う方がはるかに効率的です。`tf.GradientTape.batch_jacobian` メソッドは、まさにそれを行います。" ] }, { "cell_type": "code", "execution_count": 35, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:26.414129Z", "iopub.status.busy": "2022-12-14T20:58:26.413644Z", "iopub.status.idle": "2022-12-14T20:58:26.558664Z", "shell.execute_reply": "2022-12-14T20:58:26.558087Z" }, "id": "YJLIl9WpHqYq" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "WARNING:tensorflow:5 out of the last 5 calls to .f at 0x7f07c83a4430> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.\n" ] }, { "data": { "text/plain": [ "TensorShape([7, 6, 5])" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "jb = tape.batch_jacobian(y, x)\n", "jb.shape" ] }, { "cell_type": "code", "execution_count": 36, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:26.561959Z", "iopub.status.busy": "2022-12-14T20:58:26.561442Z", "iopub.status.idle": "2022-12-14T20:58:26.567036Z", "shell.execute_reply": "2022-12-14T20:58:26.566427Z" }, "id": "-5t_q5SfHw7T" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "0.0\n" ] } ], "source": [ "error = tf.reduce_max(abs(jb - j_sum))\n", "assert error < 1e-3\n", "print(error.numpy())" ] }, { "cell_type": "markdown", "metadata": { "id": "IUeY2ZCiL31I" }, "source": [ "注意: `tf.GradientTape.batch_jacobian` は、ソースとターゲットの最初の次元が一致することのみを検証します。勾配が実際に独立していることは確認しません。意味のある場所でのみ `batch_jacobian` を使用できるかは、ユーザー次第です。たとえば、`tf.keras.layers.BatchNormalization` を追加すると、`batch` 次元の全体にわたって正規化するため、独立性は破壊されます。" ] }, { "cell_type": "code", "execution_count": 37, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:26.570168Z", "iopub.status.busy": "2022-12-14T20:58:26.569707Z", "iopub.status.idle": "2022-12-14T20:58:26.898976Z", "shell.execute_reply": "2022-12-14T20:58:26.898282Z" }, "id": "tnDugVc-L4fj" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "WARNING:tensorflow:6 out of the last 6 calls to .f at 0x7f07c844b550> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "j.shape: (7, 6, 7, 5)\n" ] } ], "source": [ "x = tf.random.normal([7, 5])\n", "\n", "layer1 = tf.keras.layers.Dense(8, activation=tf.nn.elu)\n", "bn = tf.keras.layers.BatchNormalization()\n", "layer2 = tf.keras.layers.Dense(6, activation=tf.nn.elu)\n", "\n", "with tf.GradientTape(persistent=True, watch_accessed_variables=False) as tape:\n", " tape.watch(x)\n", " y = layer1(x)\n", " y = bn(y, training=True)\n", " y = layer2(y)\n", "\n", "j = tape.jacobian(y, x)\n", "print(f'j.shape: {j.shape}')" ] }, { "cell_type": "code", "execution_count": 38, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:26.902466Z", "iopub.status.busy": "2022-12-14T20:58:26.901812Z", "iopub.status.idle": "2022-12-14T20:58:27.118559Z", "shell.execute_reply": "2022-12-14T20:58:27.117924Z" }, "id": "SNyZ1WhJMVLm" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAgkAAAIkCAYAAACUULgQAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAABi8ElEQVR4nO3de1xU1fo/8M8AMgPCgMRNBEUlRVTkCEdF83KURFPTzKMWHcXMbpIaXdTz7ahd0bTSyrQss5s/LdMyT2HezSuKkWmiopKoASICAgHC7N8fHadGhrU2MAMj83m/XrxeMs/i2WtvBlw8e+9naxRFUUBERER0E4fGngARERHZJi4SiIiIyCwuEoiIiMgsLhKIiIjILC4SiIiIyCwuEoiIiMgsLhKIiIjILC4SiIiIyCwuEoiIiMgsLhJuYTt37oRGo8G6desaeyoWtWrVKmg0GmRmZhpfGzBgAAYMGNBoc6LascfvYVPfP7JPXCTYGI1Go+pj586djT1VaoJKS0sxb948vr+ICADg1NgTIFOffPKJyecff/wxtmzZUu31Tp064cSJEw05tUb1/fffN/YU7EJpaSmef/55ALD4X8X8HhLderhIsDEPPPCAyecHDhzAli1bqr0OwK4WCc7Ozo09BYsrLS2Fq6trY0+jwTTF7yFRU8fTDU2AwWDAyy+/jMDAQOh0OgwaNAgZGRnVxh08eBBDhgyBh4cHXF1d0b9/f+zdu9dkzLVr1zBjxgwEBwdDq9XC19cXd955J44cOVLrXDV566230LlzZ7i6uqJFixaIiorC6tWrhV9j7nxvWVkZ5s2bhw4dOkCn06Fly5YYPXo0zpw5Y3JsFi9ejM6dO0On08HPzw+PPPIIrl69apLr8OHDiI2Nhbe3N1xcXNC2bVs8+OCD0n35+uuvMWzYMAQEBECr1aJ9+/Z48cUXUVVVVW3+Xbp0QWpqKvr16wdXV1f8+9//BgCUl5dj7ty5CAkJgVarRVBQEJ599lmUl5dLt38j7y+//IJ//OMfcHV1RatWrfDqq69WG5ubm4vJkyfDz88POp0O3bp1w0cffWSMZ2ZmwsfHBwDw/PPPG09tzZs3TziH48ePY+DAgXBxcUFgYCBeeuklGAwGs3P96/ewoqICc+bMQWRkJDw8PNC8eXP07dsXO3bsqPa1V65cwb/+9S/o9Xp4enpi4sSJ+Omnn6DRaLBq1SqTsdu3b0ffvn3RvHlzeHp6YuTIkdUW1PPmzYNGo0FGRgbi4+Ph6ekJDw8PTJo0CaWlpSZjP/zwQwwcOBC+vr7QarUICwvDsmXLhMeEqKlgJaEJmD9/PhwcHPD000+jsLAQr776KuLi4nDw4EHjmO3bt2Po0KGIjIzE3Llz4eDgYPzl98MPP6BHjx4AgEcffRTr1q1DQkICwsLCcOXKFezZswcnTpxA9+7da5XLnBUrVmDatGkYM2YMpk+fjrKyMhw9ehQHDx7E/fffr3qfq6qqMHz4cGzbtg3jx4/H9OnTce3aNWzZsgXHjh1D+/btAQCPPPIIVq1ahUmTJmHatGk4d+4c3n77bfz444/Yu3cvmjVrhtzcXAwePBg+Pj6YNWsWPD09kZmZifXr10vnsWrVKri5uSExMRFubm7Yvn075syZg6KiIixcuNBk7JUrVzB06FCMHz8eDzzwAPz8/GAwGHD33Xdjz549ePjhh9GpUyf8/PPPeOONN3Dq1Cl89dVX0jlcvXoVQ4YMwejRozF27FisW7cOM2fORNeuXTF06FAAwO+//44BAwYgIyMDCQkJaNu2Lb744gvEx8ejoKAA06dPh4+PD5YtW4bHHnsM99xzD0aPHg0ACA8Pr3Hb2dnZ+Mc//oHKykrMmjULzZs3x3vvvQcXFxfpvIuKivD+++/jvvvuw5QpU3Dt2jV88MEHiI2NRUpKCiIiIgD8sdAbMWIEUlJS8NhjjyE0NBRff/01Jk6cWC3n1q1bMXToULRr1w7z5s3D77//jrfeegt9+vTBkSNHEBwcbDJ+7NixaNu2LZKSknDkyBG8//778PX1xYIFC4xjli1bhs6dO+Puu++Gk5MTvvnmGzz++OMwGAyYOnWqdD+JbmkK2bSpU6cqNX2bduzYoQBQOnXqpJSXlxtfX7JkiQJA+fnnnxVFURSDwaDcfvvtSmxsrGIwGIzjSktLlbZt2yp33nmn8TUPDw9l6tSpNc6nNrnMGTlypNK5c2fhmA8//FABoJw7d874Wv/+/ZX+/fsbP1+5cqUCQHn99dfNzlFRFOWHH35QACifffaZSTw5Odnk9Q0bNigAlEOHDgnnZU5paWm11x555BHF1dVVKSsrM5k/AGX58uUmYz/55BPFwcFB+eGHH0xeX758uQJA2bt3r3D7N/J+/PHHxtfKy8sVf39/5d577zW+tnjxYgWA8umnnxpfq6ioUKKjoxU3NzelqKhIURRFuXz5sgJAmTt3rnznFUWZMWOGAkA5ePCg8bXc3FzFw8ND+j2srKw0ed8qiqJcvXpV8fPzUx588EHja19++aUCQFm8eLHxtaqqKmXgwIEKAOXDDz80vh4REaH4+voqV65cMb72008/KQ4ODsqECROMr82dO1cBYLIdRVGUe+65R7nttttMXjP3PY6NjVXatWtn8trN+0fUFPB0QxMwadIkk/O9ffv2BQCcPXsWAJCWlobTp0/j/vvvx5UrV5CXl4e8vDyUlJRg0KBB2L17t7E87OnpiYMHD+LSpUtmt1WbXOZ4enriwoULOHToUL32+csvv4S3tzeeeOKJajGNRgMA+OKLL+Dh4YE777zTOM+8vDxERkbCzc3NWNb29PQEAGzatAnXr1+v1Tz++hfztWvXkJeXh759+6K0tBTp6ekmY7VaLSZNmmTy2hdffIFOnTohNDTUZI4DBw4EALOl95u5ubmZXLPi7OyMHj16GL//APDtt9/C398f9913n/G1Zs2aYdq0aSguLsauXbtqtd9/zdurVy+T6pGPjw/i4uKkX+vo6Gh83xoMBuTn56OyshJRUVEmp7eSk5PRrFkzTJkyxfiag4NDtb/if/vtN6SlpSE+Ph5eXl7G18PDw3HnnXfi22+/rTaHRx991OTzvn374sqVKygqKjK+9tfvcWFhIfLy8tC/f3+cPXsWhYWF0v0kupXxdEMT0Lp1a5PPW7RoAQDG8+6nT58GALPl2RsKCwvRokULvPrqq5g4cSKCgoIQGRmJu+66CxMmTEC7du1qncucmTNnYuvWrejRowdCQkIwePBg3H///ejTp4/Kvf3DmTNn0LFjRzg51fwWPn36NAoLC+Hr62s2npubCwDo378/7r33Xjz//PN44403MGDAAIwaNQr3338/tFqtcB7Hjx/Hc889h+3bt5v8xwKg2n8grVq1qnbx3unTp3HixAnjtQA1zVEkMDDQuDC6oUWLFjh69Kjx819//RW33347HBxM/y7o1KmTMV4Xv/76K3r27Fnt9Y4dO6r6+o8++givvfYa0tPTTRZobdu2NdlGy5Ytq13kGRISUm0uNW27U6dO2Lx5M0pKStC8eXPj66KfHb1eDwDYu3cv5s6di/3791e7XqGwsBAeHh6q9pXoVsRFQhPg6Oho9nVFUQDA+Jf9woULjed5b+bm5gbgj3O0ffv2xYYNG/D9999j4cKFWLBgAdavX4+hQ4fWKpc5nTp1wsmTJ7Fp0yYkJyfjyy+/xDvvvIM5c+YYb72zFIPBAF9fX3z22Wdm4zf+Y77RkOrAgQP45ptvsHnzZjz44IN47bXXcODAgRr3p6CgAP3794der8cLL7yA9u3bQ6fT4ciRI5g5c2a1ioq58/QGgwFdu3bF66+/bnYbQUFB0v2Uff9t1aeffor4+HiMGjUKzzzzDHx9feHo6IikpCSTi0+tSXbszpw5g0GDBiE0NBSvv/46goKC4OzsjG+//RZvvPGGsGpG1BRwkWAHblzEp9frERMTIx3fsmVLPP7443j88ceRm5uL7t274+WXX8bQoUNrncuc5s2bY9y4cRg3bhwqKiowevRovPzyy5g9ezZ0Op3qfTp48CCuX7+OZs2a1Thm69at6NOnj6oL6Xr16oVevXrh5ZdfxurVqxEXF4c1a9bgoYceMjt+586duHLlCtavX49+/foZXz937pyqfbgxx59++gmDBg2qVg2wpDZt2uDo0aMwGAwm1YQbp0TatGkDALWeQ5s2bYzVpb86efKk9GvXrVuHdu3aYf369SbbnTt3brVt7Nixo9otozffwXNjH8xtOz09Hd7e3iZVBDW++eYblJeXY+PGjSZVBzWngYiaAl6TYAciIyPRvn17LFq0CMXFxdXily9fBvDHHQM3l8h9fX0REBBgvB1Pba6aXLlyxeRzZ2dnhIWFQVGUWl0PcO+99yIvLw9vv/12tdiNvwLHjh2LqqoqvPjii9XGVFZWoqCgAMAfpeWb/+q+USUR3YZ446/Qv35tRUUF3nnnHdX7MXbsWFy8eBErVqyoFvv9999RUlKiOpfIXXfdhezsbKxdu9b4WmVlJd566y24ubmhf//+AGD8T/jGsVGT98CBA0hJSTG+dvny5RqrN39l7vgdPHgQ+/fvNxkXGxuL69evmxwjg8GApUuXmoxr2bIlIiIi8NFHH5nM/9ixY/j+++9x1113qdon2RwLCwvx4Ycf1joX0a2IlQQ74ODggPfffx9Dhw5F586dMWnSJLRq1QoXL17Ejh07oNfr8c033+DatWsIDAzEmDFj0K1bN7i5uWHr1q04dOgQXnvttVrlqsngwYPh7++PPn36wM/PDydOnMDbb7+NYcOGwd3dXfU+TZgwAR9//DESExORkpKCvn37oqSkBFu3bsXjjz+OkSNHon///njkkUeQlJSEtLQ0DB48GM2aNcPp06fxxRdfYMmSJRgzZgw++ugjvPPOO7jnnnvQvn17XLt2DStWrIBerxf+x9K7d2+0aNECEydOxLRp06DRaPDJJ5/Uqsz/r3/9C59//jkeffRR7NixA3369EFVVRXS09Px+eefY/PmzYiKilKdryYPP/ww3n33XcTHxyM1NRXBwcFYt24d9u7di8WLFxuPvYuLC8LCwrB27Vp06NABXl5e6NKlC7p06WI277PPPotPPvkEQ4YMwfTp0423QN6oXIgMHz4c69evxz333INhw4bh3LlzWL58OcLCwkwWoKNGjUKPHj3w1FNPISMjA6Ghodi4cSPy8/MBmFY/Fi5ciKFDhyI6OhqTJ0823gLp4eEh7fdgzuDBg+Hs7IwRI0bgkUceQXFxMVasWAFfX1/89ttvtc5HdMtprNsqSB01t0B+8cUXJq+fO3eu2q1hiqIoP/74ozJ69GjltttuU7RardKmTRtl7NixyrZt2xRF+ePWuWeeeUbp1q2b4u7urjRv3lzp1q2b8s4771TbtixXTd59912lX79+xq9r37698swzzyiFhYXGMWpugVSUP25N+7//+z+lbdu2SrNmzRR/f39lzJgxypkzZ0zGvffee0pkZKTi4uKiuLu7K127dlWeffZZ5dKlS4qiKMqRI0eU++67T2ndurWi1WoVX19fZfjw4crhw4eF+6IoirJ3716lV69eiouLixIQEKA8++yzyubNmxUAyo4dO0zmX9OtnxUVFcqCBQuUzp07K1qtVmnRooUSGRmpPP/88ybHxZya8k6cOFFp06aNyWs5OTnKpEmTFG9vb8XZ2Vnp2rVrtfeIoijKvn37lMjISMXZ2VnV7ZBHjx5V+vfvr+h0OqVVq1bKiy++qHzwwQfS76HBYFBeeeUVpU2bNopWq1X+9re/KZs2bTI798uXLyv333+/4u7urnh4eCjx8fHK3r17FQDKmjVrTMZu3bpV6dOnj+Li4qLo9XplxIgRyi+//GIy5sYtkJcvXzZ53dx7b+PGjUp4eLii0+mU4OBgZcGCBcZbcGXvUaJbnUZRbPzqJiIiM7766ivcc8892LNnT63vjiEidbhIICKb9/vvv5tcfFpVVYXBgwfj8OHDyM7OVnVhKhHVHq9JICKb98QTT+D3339HdHQ0ysvLsX79euzbtw+vvPIKFwhEVsRKAhHZvNWrV+O1115DRkYGysrKEBISgsceewwJCQmNPTWiJo2LBCIiIjKLfRKIiIjILC4SiIiIyKwGv3DRYDDg0qVLcHd3t2obWiIisj2KouDatWsICAio9sCxhlBWVoaKigqr5HZ2dlbdWv5W0eCLhEuXLql6aA0RETVdWVlZCAwMbNBtlpWVwcfFBdUbyluGv78/zp0716QWCg2+SPiz9e6gxtg8ERE1qkoA22rVht1SKioqUAzgSQDih8DXXjmAN7KzUVFRwUVCffx5isEJgPmn9xERUdPWmKebXQBY+r/xpnqBX1PdLyIiIqon1vuJiMiuOMDyfyE31b+4uUggIiK7wkWCek11v4iIiKieWEkgIiK7wkqCek11v4iIiKiebLaS8OWX3wjjo+/IlSfx9hbHs7Prn0ONsjJhWOMRV/9tAFBK1ooHVFZKcxjc9MK4Q1mpNEeFk6sw7qTiXVcs6Xbi4TFCnkSFAwfE77Oef1PRmU3y/ZXuDAC4uQnDsu+Lmmk0b26ZY/af/4iP2YwZ8hxemUfEA7p0kSdJTxfHJccUAODvLwxrmo+T51BhyhTxMXtv3iV5EtkPjpr3maenfIyM5Lhr+iTVfxtWxkqCek11v4iIiKiebLaSQEREZA2sJKjXVPeLiIiI6omVBCIisiusJKjXVPeLiIiI6omVBCIisiusJKjHRQIREdkVDSz/n3rjPdPSumx2kSDrg7B6q680x/1jxPe45zgGSHP4FeeLB6h5brjsBnZLkW0nM1OaorJLd2H8cJq4BwIARESI4xkZ0hQI85ccdwvpebt4OzlXvaQ5/FqI4xW6+vc40KvoT6HTyb83ljBvnji+Z488R7+oUPEANc00JD0OsHOnNMXZqLHy7VjAe2+Lfxd16CL/XXRq0ylhvCK4gzSHc6Y4B9askebA00/Lx1CTYbOLBCIiImvQwPJ/+TfVSkJTPY1CRERE9cRKAhER2RXH/31YOmdTxEoCERERmcVKAhER2RXeAqleU90vIiIiqidWEoiIyK6wkqAeFwlERGRXuEhQz3YXCW5uwvD9w4ukKU5lipvYhISomEeZimZJMmoaw1hAqU7c+EcXIW8M5LxntzDeJaKfNIdsd2+7TZpC+v23FIOn+Jj4qWhiZIkfI1lPrtJKeaMkV1VzrT+H7EvCeL8oT2mOj9eJ92fCKPnPNzwl2xkzRpqiXUHDNO1CdrYwfGqrPIW+i7hZUtGx8/IkeXni+PDh8hySfaGmxXYXCURERFbASoJ6TXW/iIiIqJ5YSSAiIrvCSoJ6TXW/iIiIqJ5YSSAiIrvCSoJ6TXW/iIiIqJ5YSSAiIrvCR0WrV6dFwtKlS7Fw4UJkZ2ejW7dueOutt9CjRw/LzqysTBgucpLf898hxCDOUSwvpOh14kNUAWdpDmenSukYS3CF+D758xfk99p7R4n7IOiT10tz5PQZLYw3ayZNoeq4WoLsnv8K7wBpDllfCGeI34cAUFEpfi/K+igAgAHy769FSHbYoJPPQ9YH4XyBuMcJALT2lvSFUNOfRNZrwVIk26nQyfe3KF38Xn19TWtpjkRZ6wg1/UkKCuRjbJwDLP/UxqZalq/1fq1duxaJiYmYO3cujhw5gm7duiE2Nha5ubnWmB8RERE1klovEl5//XVMmTIFkyZNQlhYGJYvXw5XV1esXLnSGvMjIiKyKAcrfTRFtdqviooKpKamIiYm5s8EDg6IiYnB/v37LT45IiIiajy1uiYhLy8PVVVV8PPzM3ndz88P6enpZr+mvLwc5eXlxs+LilT0ZCciIrIS3gKpntX3KykpCR4eHsaPoKAga2+SiIiILKBWiwRvb284OjoiJyfH5PWcnBz4+/ub/ZrZs2ejsLDQ+JGVlVX32RIREdUTr0lQr1b75ezsjMjISGzbts34msFgwLZt2xAdHW32a7RaLfR6vckHERER2b5a90lITEzExIkTERUVhR49emDx4sUoKSnBpEmTrDE/IiIii+I1CerVepEwbtw4XL58GXPmzEF2djYiIiKQnJxc7WLGepM09dAXqOjL4CTJ4SbvULN9p7ipz8CIfPk8GkippJlOa/8KaY78YvH+ZkeIGyUBQLufvhfGDTGDpTkajKTJTaWKPliSvl9wcpL/+nAtFr+f8518pTm8dJLmQpYi2WFZgyoAMPiLm1S11snfq6+/LX6/JybIc6j6BltCcbEw7KymQVFGhjCcOEa+Ly4dxQ2Xft+VIp9HQzWgsiIuEtSrU8fFhIQEJCQkWHouREREZEP47AYiIrIrrCSo11T3i4iIyKYtXboUwcHB0Ol06NmzJ1JSxKd7CgoKMHXqVLRs2RJarRYdOnTAt99+a9U5spJARER2xRaeAnnjOUjLly9Hz549sXjxYsTGxuLkyZPw9a1+DVJFRQXuvPNO+Pr6Yt26dWjVqhV+/fVXeFr5GhEuEoiIiBrYX5+DBADLly/Hf//7X6xcuRKzZs2qNn7lypXIz8/Hvn370Ox/j9INDg62+jx5uoGIiOyKo5U+gD8ePfDXj78+luCGujwHaePGjYiOjsbUqVPh5+eHLl264JVXXkFVVVU9j4YYFwlEREQWEhQUZPIogqSkpGpjRM9Bys7ONpv37NmzWLduHaqqqvDtt9/iP//5D1577TW89NJLVtmPG2z3dIP85nN5Dsk90PkF8jXSwCjxA6n2HfOS5ugdoqKngwXoJG0fTmWIeyAAgIdH/ech64Pg8P578iRjxtR/IipUOInvtXdV0Y+j1E3ew6C+Obwyf5EnaYDSIwBpDxNs3SrPMWasOK7i5zvxUXFfiGlPi7+3ALB4sfxnokGsWiUf06uXOC77BQDg9827hXFNT0dhHACUi5Lv/y1AA8v/hXzjmoSsrCyTzsJardYi+Q0GA3x9ffHee+/B0dERkZGRuHjxIhYuXIi5c+daZBvm2O4igYiIyAqseQukmscP1OU5SC1btkSzZs3g6PjnQq5Tp07Izs5GRUUFnJ2ts+Dl6QYiIqIGVJfnIPXp0wcZGRkwGAzG106dOoWWLVtabYEAcJFARER2xhaeApmYmIgVK1bgo48+wokTJ/DYY4+ZPAdpwoQJmD17tnH8Y489hvz8fEyfPh2nTp3Cf//7X7zyyiuYOnVq3Q6CSjzdQERE1MBkz0E6f/48HBz+XHoEBQVh8+bNePLJJxEeHo5WrVph+vTpmDlzplXnyUUCERHZFVtpyyx6DtLOnTurvRYdHY0DBw7UYUt1x9MNREREZBYrCUREZFdspZJwK2iq+0VERET1ZLuVBFljEEmjJADShixeZeJGSWpy9I6qkKb4dmv9m+2oIes/1cFfvr8VOvH9vT4+9Z+H60MPyedR2TDrV2cngzBu8JZ/72QtbBwg3oYaFSFh9c7RUHL6SxolAfAryBcPkDVsAqS/A95cLD/upWUN9HeSZK6lT8+RpnDNPiseUFAgn4ek4ZZS4i1N8ew8eZMqW8dKgnq2u0ggIiKyAlt4CuStoqkufoiIiKieWEkgIiK78tenNloyZ1PESgIRERGZxUoCERHZFV64qF5T3S8iIiKqJ1YSiIjIrmhg+b+Qm+rdDTa7SNBo723sKdxymjcf0dhTuOVoHEc29hRuOZrbJjb2FG45mjbWfVIfkbXY7CKBiIjIGnhNgnpcJBARkV3hIkG9prpfREREVE+sJBARkV1hJUG9prpfREREVE+sJBARkV1hJUG9prpfREREVE+sJBARkV3ho6LVYyWBiIiIzGIlgYiI7AofFa0eFwlERGRXeOGiek11v4iIiKieWEkgIiK7wkqCek11v4iIiKieWEkgIiK7ooHl/0LmLZBERERkV1hJICIiu8JrEtSz2UXCkCHfCOPffZQrT1JcLI57e8tzlJWJ404qDmF2tjCs6TxTnkOF6GjxMVu3Tp4jYM/nwvgvXcZKc4QV7BMPUHPc3dyEYU2rR+Q5VFCCfhLGv11+XprjrgGlwvi/X3KV5nhlXoV0jMy+w87CeJ8+I+q9DQBQNj8hjB/xHizN0d3tlDD+8YEO0hwTYi5Jx8gcyQ4QxiMjLXPMUlPFP5vd01ZKcxyJeFCc48JGaY6DfncL4506SVNAn3FEGNdEzpUnoVuGzS4SiIiIrIGVBPWa6n4RERFRPbGSQEREdoWVBPW4SCAiIrvCp0Cq11QXP0RERFRPrCQQEZFd4VMg1WMlgYiIiMyy2UrCd1+K7z1/fbmvNEfi+ErxAFkPBEB+T39GhjxHYKB8jAXs3CmOHzsmzxHQq5cwHuYvv5+/tLK3MO56eLc8R1Q/6RhLuHRA3AfhrgHy+/Uv7RTf8x8TI59HfrG4x8GFC/Ic7dvLx1hC6R3iPgjd18jv+S8dL77nf/hwFfPQiXscqDlmoaHyMZbQ3Vv8Prt/q/h4AMDqMUXiAYHin10A6OmUL4wXwUuaQ1VvGBvHCxfVa6r7RURERPV06y8JiYiIaoGVBPWa6n4RERFRPbGSQEREdoWVBPW4SCAiIruigeX/U2czJSIiIrIrrCQQEZFd4ekG9ZrqfhEREVE92W4loVLcCCkxXtwUBABWfiVutvKgrDmJinlImy0BDdZ8xDnvkjDePVA+j/e+ai2MPzxcvA0AgKf4uEPSsAkAXJ0M8u1YQIC/eDv5B8SNkgAgYNnLwrj3M/8nzZGZKY7bUv8aVydxQ62KB+SNgVy3/FcYd7pzmDSHrFmSTidNIf3xtpRSb/HP1erl8t9F+47phfGoKHEcAJwvnBXG9U7F0hwN1RzOmlhJUK+p7hcRERHVkw39fUJERGR9rCSo11T3i4iIyKYtXboUwcHB0Ol06NmzJ1JSUlR93Zo1a6DRaDBq1CjrThBcJBARkZ1xsNJHbaxduxaJiYmYO3cujhw5gm7duiE2Nha5ubnCr8vMzMTTTz+Nvn371nKLdcNFAhER2RVbWCS8/vrrmDJlCiZNmoSwsDAsX74crq6uWLmy5qeoVlVVIS4uDs8//zzatWtXyy3WDRcJREREFlJUVGTyUV5eXm1MRUUFUlNTEfOX58g7ODggJiYG+/fvrzH3Cy+8AF9fX0yePNkqczeHiwQiIrIr1qwkBAUFwcPDw/iRlJRUbft5eXmoqqqCn5+fyet+fn7Izs42O+c9e/bggw8+wIoVK+qx57VX67sbdu/ejYULFyI1NRW//fYbNmzYYJ2LJ2Q3OWdkSFM8eEeeMP7YzA7SHEuWiOPO6enSHAgNlY+xgPOV4v4ErT1LpTkejhBfOPNtWg9pjogIcdz1cLI0R8WQu6VjLOG998Xr5DzxWwgA8MC/xH0Q8rTyru6hJYow/vPP8nmo6QtgCQd/dBbGfXzkOZpHifsg+Kg4ZmXHxcfM318+j2PH5GMswTXzF/GAsjJpjoiI7sL48uXyeTz6qLhE7QxxDwwA8gYVdi4rKwt6/Z89K7Rabb1zXrt2Df/617+wYsUKeKvpzWNBtV4klJSUoFu3bnjwwQcxevRoa8yJiIjIaqx5C6RerzdZJJjj7e0NR0dH5OTkmLyek5MDfzOr2zNnziAzMxMjRowwvmYw/NEIzsnJCSdPnkT79u3rtwM1qPUiYejQoRg6dKg15kJERNTkOTs7IzIyEtu2bTNW4g0GA7Zt24aEhIRq40NDQ/HzTeXE5557DteuXcOSJUsQFBRktbmymRIREdkVW2imlJiYiIkTJyIqKgo9evTA4sWLUVJSgkmTJgEAJkyYgFatWiEpKQk6nQ5dunQx+XpPT08AqPa6pVl9kVBeXm5ydWdRkYrnJRARETVh48aNw+XLlzFnzhxkZ2cjIiICycnJxosZz58/DweHxr+3wOqLhKSkJDz//PPW3gwREZEqGgAajfzi2FrlVMQX0pqTkJBg9vQCAOzcuVP4tatWrar19urC6suU2bNno7Cw0PiRlZVl7U0SERHVzMnJOh9NkNX3SqvVWuQWECIiImpYtV4kFBcXI+MvPQrOnTuHtLQ0eHl5oXVr8TPTiYiIGp2TE2Dh0w1QFKCy0rI5bUCtFwmHDx/GP/7xD+PniYmJAICJEyda9hyJrMuJmis6JY2OFiyQp0iW9P25Oyai3vOwlNaH1wvj+QPkfS28JI2fQsw3AzMRUHleGD/bRd4oqd2Fs/INWcDw4eK4ih432LpVHH+gXH6uslLSPKhkmzxHYaF0iEX0/Ju44U4FxM2WAGDTJnE85Cf5/ob3aSGMb//yqjRHcLB0iEXk3BYmjPvlHJXmcE0W/3zHx8t/vh94QBz//AF5o7MGO2hkE2q9SBgwYACUOlygQUREZBOsVUloghr//goiIiKySU3zckwiIqKasJKgGisJREREZBYrCUREZF8cHQFLdzP83wOXmhouEoiIyL44OXGRoBJPNxAREZFZtltJkPVBKC6W5wgJEYb1lfKHTd093E0YP3rMVZojPCJCOsYiJDf9e+Vdkufw9haGO3jnS1OU6sRNtcoy5dMwhLaTD7KAAH/x6j+/QL6OHjZMHFfTX6Xsiviip4F7NkpzlMbI+09YhKT9bJmKH82YGHHcTfxjBwAo+lXcB2Fgprz3QEVguHxDFuDnI36fVbSQz8M5MFAYV3PMPl8u/vn9JVv+HgoLbAIP6WMlQTVWEoiIiMgs260kEBERWQMrCaqxkkBERERmsZJARET2xdHxjw9LqqqybD4bwUoCERERmcVKAhER2RcnJ8tXEizd5tlGcJFARET2hYsE1Xi6gYiIiMyy3UrCsWPiuKRREgDgwgVxXNI4CACQmSkMh4eKG5wAwNjxzvLtWMKBA8Lw2cB+0hTtMk+JB6g47q55ucK4p6evNMfly9IhFnHwkHid7OMjz5GdLY5LeuAAADIyxHH/EHmTG7fmsr9kxM221LqULT5m/v7yHLL9zcuT59DpxPGQEHmDIn23TpIRKn7PqGCQ/D3m/OlKeZIhQ4RhB8nPHQBpZ6+wgn3SFJrOTeBvS1YSVGsC320iIiKyBtutJBAREVkDKwmqsZJAREREZrGSQERE9sXRUfqgMvoDKwlERERkFpdSRERkX5ycWElQiUeJiIjsCxcJqmkURVEacoNFRUXw8PAAEAugWUNumoiIGt11AJtRWFgIvV7foFu+8f9PYd++0Ft4kVBUWQmPH35olP2yJi6liIjIvrCSoBovXCQiIiKzuJQiIiL7Yo1bIBv2zH2DYSWBiIiIzGIlgYiI7Is1rklgJYGIiIjsCSsJRERkX1hJUI2LBCIisi9cJKjG0w1ERERkFisJRERkX1hJUI2VBCIiIjKLlQQiIrIv1mimZDBYNp+NYCWBiIiIzGIlgYiI7Is1rklgJYGIiIjsCSsJRERkX1hJUI2VBCIiIjLLZisJ0dHfCOP73v9FnqSgQBgu6tJbmkKfd1Y8oLi43vPQ9F8oz6FKoDCqnHlGnmL5cmH46AOvSlOEf/OyeICPj3weEppHxO8PtZRNjwrjl/42TJojoED8Xkx8P0ya4/UHjogHVFZKc1RE9BDGtdoR0hxqKMcXCONndfL9beddJIxvP6yX5hjYq1Q8QKeT5ti9R/x3Uv/+ljlmBw6I3689/ztHmuPomBeE8XA3ye8qAEcK2gnj3UMlxxQAduwQhjXDxb9DbAIrCarZ7CKBiIjIKrhIUI2nG4iIiMgsLhKIiMi+3GimZMkPR8daT2Pp0qUIDg6GTqdDz549kZKSUuPYFStWoG/fvmjRogVatGiBmJgY4XhL4SKBiIioga1duxaJiYmYO3cujhw5gm7duiE2Nha5ublmx+/cuRP33XcfduzYgf379yMoKAiDBw/GxYsXrTpPLhKIiMi+WLqKUIdrHF5//XVMmTIFkyZNQlhYGJYvXw5XV1esXLnS7PjPPvsMjz/+OCIiIhAaGor3338fBoMB27Zts8QRqREXCURERA2ooqICqampiImJMb7m4OCAmJgY7N+/X1WO0tJSXL9+HV5eXtaaJgDe3UBERPbGGnc3VFUBAIqKTG/v1Wq10Gq1Jq/l5eWhqqoKfn5+Jq/7+fkhPT1d1eZmzpyJgIAAk4WGNdjsImHfzgphfGOy/F7suweI78XWl5k/92MiUNx7QFWfhNBQ+RgLUMqXCOMvzHeW5pgza5YwHl58Xj6Rxx4TxyV9IwDIj7uF+iQYhor7IARAfltThbf4vfj6IhW3RpWJ3yNFla7SFG4N9dMseT+3S5P0fABgCO4ujEdEyKdRCvExuXZZnqPfHQ1z21rPTuLfRSuPi3sgAMCDoeLfiSj2lOboLulPUQp5fwrXv/1NOsaeBQUFmXw+d+5czJs3z6LbmD9/PtasWYOdO3dCp6IfSH3Y7CKBiIjIKqxYScjKyoJe/+di6+YqAgB4e3vD0dEROTk5Jq/n5OTA399fuJlFixZh/vz52Lp1K8LDwy0wcTFek0BERPbFirdA6vV6kw9ziwRnZ2dERkaaXHR44yLE6OjoGqf96quv4sUXX0RycjKioqIsf1zMYCWBiIiogSUmJmLixImIiopCjx49sHjxYpSUlGDSpEkAgAkTJqBVq1ZISkoCACxYsABz5szB6tWrERwcjOzsbACAm5sb3NzcrDZPLhKIiMi+WON0Qy3zjRs3DpcvX8acOXOQnZ2NiIgIJCcnGy9mPH/+PBwc/iz2L1u2DBUVFRgzZoxJHmtc8/BXXCQQERE1goSEBCQkJJiN7dy50+TzzMxM60/IDC4SiIjIvthAJeFWwQsXiYiIyKymufQhIiKqCSsJqtnuXmVkCMN36y5IU2g8mgvjSs7t8nmkpYnjl1V0bPHxkY+xhLffFobn6CqlKTS3iR8WouT8n3wenp7i+HffyXO4u8vHWIDDBXFzqFLv1tIcrp3bCuPbPzinYibixkDe3vIMXbqo2IwlVIrfR/mSRkkA4OCoEQ+4okhzvPSSOK6mh1lERMMUU0udxE2KHgzdJ83xcEJvYfzRR+Xtebuf+EwYd/3fvf5Ckvv4qWmx3UUCERGRNbCSoFrT3CsiIqKa3GimZOmcTRAvXCQiIiKzWEkgIiL7wtMNqrGSQERERGY1zaUPERFRTVhJUI2VBCIiIjKrVkufpKQkrF+/Hunp6XBxcUHv3r2xYMECdOzY0eITM4SGCeMOwcHSHEqVThjPL5CvkbyiJDeoS+4ZB9BwK8wZM8Tx4mJpCmWG+JidveAszdFOVyQe8M9/SnPIj9kKeQ4VKvzFfRB0Kr51FSfFfRAGFufLk+jEx73CSdxHoSFVQPwekOzKH2OqxH0QZP0rAGDePPH3Ts2PnZofX0tw1RnEAyIipDmWLxfHy8pUTKSL+GfP4CT/+XaAZF/wloqJNDJWElSrVSVh165dmDp1Kg4cOIAtW7bg+vXrGDx4MEpKSqw1PyIiImoktVr6JCcnm3y+atUq+Pr6IjU1Ff369bPoxIiIiKyCfRJUq9dRKiwsBAB4ecnbgRIREdkEnm5Qrc57ZTAYMGPGDPTp0wddBE3jy8vLUV5ebvy8qEhyvpqIiIhsQp0XCVOnTsWxY8ewZ88e4bikpCQ8//zzdd0MERGRZbGSoFqdboFMSEjApk2bsGPHDgQGBgrHzp49G4WFhcaPrKysOk2UiIiIGlatlj6KouCJJ57Ahg0bsHPnTrRtK35ELgBotVpotdo6T5CIiMiiWElQrVZ7NXXqVKxevRpff/013N3dkZ2dDQDw8PCAi4uLVSZIREREjaNWi4Rly5YBAAYMGGDy+ocffoj4+HhLzQkA4JBxSjxATceWggJh2Cs0VJpi7Hhxc5HP3xZvAwCQlycfYwnr1onjklNDAABJk6p2xw5LU2hGhgjjVVXiRlkA4LBpo3SMJcia6ThfOCvNceBCO2G8X/ZWaY7t3mOFcRW9w+DpKR9jCc6oEMavXpM35Nm5Uxz39BQ3SgKAkDYaYfzYNnHDJkBVDyOLkDVu88o4Js3hIHmzuqr4+e4xRnxcUxbvk+aAm5t8jK1jJUG1Wp9uICIiIvvQNJc+RERENWEzJdW4SCAiIvvC0w2q8SmQREREZFbTXPoQERHVhJUE1VhJICIiIrOa5tKHiIioJqwkqGa7e+XvL45fuCDPIbtxXEX/gs+Xi/sxPDzLV5rjvfkNdJhlN30fOiTPITvuKm4sr6oS34ut5t70o1t7SUaskCdRwbVS/MCxfE9xDwQACJG07DgbKO6BAAARnuK41/7/SnNg6FD5GEuQ3K9/8aK8T0KIuJWGqpYe6aniW7IHLp4gzVG6/GP5hizAS1cqjBeF9pDm0F/4RRgv9Zb3lkhZJc7x8eHe0hwTIo5Kx1DTYbuLBCIiImvgLZCq8ZoEIiIiMouVBCIisi+8JkG1prlXRERENeEiQTWebiAiIiKzmubSh4iIqCasJKjGSgIRERGZ1TSXPkRERDVhJUE1292rY8fEcVnTH0DecKmXrGEPgLQ0Yfi9WQXSFJrbguTbsYSvvhLH4+PlOfbsEcejoqQpHJYtFcaPbhohzaHx2yQdYwmXivXCeICbuNkSAJyvFOeQ9fQCADc3yYA775Qnkf3MWMipC67CeHf/S9IcuzMChHFJvyYAQHGxOH52nrxRUrtB0ZIR3vKJqGDQiY+ZPuFxeZLnnhOGXcvyazMlsya4rZeO0XRLrvd26NZhu4sEIiIia2AzJdV4TQIRERGZxUoCERHZF16ToFrT3CsiIqKacJGgGk83EBERkVlNc+lDRERUE1YSVGMlgYiIiMzSKIqiNOQGi4qK4OHhASAWQLOG3DQRETW66wA2o7CwEHq9uMeIpd34/6fw3DmLb7uoqAgebds2yn5ZEysJREREZFbTPIlCRERUE16ToBorCURERGRW01z6EBER1YSVBNVYSSAiIvtyY5Fg6Y9aWrp0KYKDg6HT6dCzZ0+kpKQIx3/xxRcIDQ2FTqdD165d8e2339b1CKjGRQIREVEDW7t2LRITEzF37lwcOXIE3bp1Q2xsLHJzc82O37dvH+677z5MnjwZP/74I0aNGoVRo0bhmJWf/spbIImIqAHZwC2QV69a5xbIFi1U71fPnj3x97//HW+//TYAwGAwICgoCE888QRmzZpVbfy4ceNQUlKCTZs2GV/r1asXIiIisHz5csvtyE1YSSAiIrKQoqIik4/y8vJqYyoqKpCamoqYmBjjaw4ODoiJicH+/fvN5t2/f7/JeACIjY2tcbylcJFARER2xQAHq3wAQFBQEDw8PIwfSUlJ1bafl5eHqqoq+Pn5mbzu5+eH7Oxss3POzs6u1XhLaZqXYxIRETWCrKwsk9MNWq22EWdTf1wkEBGRXams/OPD0jkBQK/XS69J8Pb2hqOjI3Jyckxez8nJgb+/v9mv8ff3r9V4S+HpBiIiogbk7OyMyMhIbNu2zfiawWDAtm3bEB0dbfZroqOjTcYDwJYtW2ocbymsJBARkV2xZiVBrcTEREycOBFRUVHo0aMHFi9ejJKSEkyaNAkAMGHCBLRq1cp4TcP06dPRv39/vPbaaxg2bBjWrFmDw4cP47333rPsjtyEiwQiIrIrtrBIGDduHC5fvow5c+YgOzsbERERSE5ONl6ceP78eTg4/Fns7927N1avXo3nnnsO//73v3H77bfjq6++QpcuXSy5G9WwTwIRETWgxu+TkJVl+W0XFRUhKMijyT0qmpUEIiKyK7ZQSbhV8MJFIiIiMstmKwnK5ifEA27qPFUXRcX1XyPpnUqlYyqcXIVxrXZEvecBANHR3wjj++Z9L81hiBksjDtkX5JPRPagEzVL7oICYVjTeaY8hwpDhoiP2XdLz8qTeHqK42Vl8hyyY+LtLc9x4YIwrOn4lDyHKouEUeWkpv6byMuTj9HpxHFLHLM+1Rvh1EWLFuL3Wf4xFT9XsqY5gYH1z+HmJs8h+d5oer4oz9HIqqos/5d/VZVl89kKVhKIiIjILJutJBAREVkDr0lQj5UEIiIiMouVBCIisiusJKjHSgIRERGZxUoCERHZFVYS1OMigYiI7ApvgVTPZhcJFQPE9+s7L3pVnmPGs8L4mjXyeTz8kEEYP39B3AMBAFr7V8g3ZAH74sUP+tDEnpDmqKoSH/citwBpDv1y8fdmY6j4+wIAd+9ZLB1jCd8tOCqMa9p/Kc2hnIwTD9i5Uz6RqChxPC1NnkPNffIWoOzKEcY1HRfKc6Q+Lx4g64EAAMeOieN79shzJCTIx1hAftp5YVzTar00h7K3h3iAmmMm65OweLE8x6pV8jHUZNjsIoGIiMgaeLpBPV64SERERGaxkkBERHaFlQT1WEkgIiIis1hJICIiu8JKgnqsJBAREZFZrCQQEZFdYSVBPS4SiIjIrrCZkno2u0i4elUcb/aQvCGPp2TvZI2SAAAHDgjDTsG95TnKyuRjLCEkRBhW9naR5zicIgxvypA0dAEQGiP+3tyt+0U+D8f+4vhCeWMoVYqLhWHl60hpih4PdBDGt24VxwHASfJedc3Lk+ZosD9l0tOFYWXtv6QpNJHi94CSI27qBUD6fld1PDIy5GMs4YcfhGFlrbxJmabPZnGOk97yeXh6iuMPPSTPoaZJFTUZNrtIICIisgaeblCPFy4SERGRWawkEBGRXWElQT1WEoiIiMgsVhKIiMiusJKgHisJREREZBYrCUREZFfYJ0G9Wi0Sli1bhmXLliEzMxMA0LlzZ8yZMwdDhw61+MR8fMRxi7QeyM6WDqmIEvdB8FdxBA3Qq51R/dxxh3geTs7SFLIfnDER8mk4lxUJ4xW6MHmO4GDJiOXyiagRFSWOq/hNkjJc3G/j/AV5wa61d6l4QEyMNEeD1TsfeEAc1+mkKZRy8VxPZcrfqx0CJccsPl6aQ/6L5EN5DjX++U9xXNYoA4BSPkoYv5QnP2YBsmMm+3kAVLzPLHTMrIinG9Sr1emGwMBAzJ8/H6mpqTh8+DAGDhyIkSNH4vjx49aaHxERETWSWlUSRowYYfL5yy+/jGXLluHAgQPo3LmzRSdGRERkDawkqFfnaxKqqqrwxRdfoKSkBNHR0ZacExEREdmAWi8Sfv75Z0RHR6OsrAxubm7YsGEDwsJqPsdcXl6O8vJy4+dFReLz1URERNbESoJ6tb4FsmPHjkhLS8PBgwfx2GOPYeLEifjll5of1pKUlAQPDw/jR1BQUL0mTERERA2j1osEZ2dnhISEIDIyEklJSejWrRuWLFlS4/jZs2ejsLDQ+JGVlVWvCRMREdXHjUqCpT+aonr3STAYDCanE26m1Wqh1WrruxkiIiJqYLVaJMyePRtDhw5F69atce3aNaxevRo7d+7E5s3i55wTERHZCjZTUq9Wi4Tc3FxMmDABv/32Gzw8PBAeHo7NmzfjzjvvtPjEZD1OXJ0q5Ekkb4JLCJCmCIBkO8Xyrk4VugZqpiRpyOKQcUqeI7iDMOx84aw0xfq0dsL46CGShi4AUFAgH2MJsjeaiiY3st82rSsvSFNs3Co+ZsOHy6fh0FDHTLYdNb99PT2F4Q5uedIUC94S//zOfEzFRdK2cszUkLwXA+Q9rPD6ci9hPPEhFcfMIp3sGhcvXFSvVouEDz74wFrzICIiIhvDZzcQEZFdYSVBPT4FkoiIiMxiJYGIiOwKKwnqsZJAREREZrGSQEREdoW3QKrHSgIRERGZZbOVBNnt6b9kOEtzBAaK46pu9z1wQBjO79JPmsKrIFfFhizg00+F4UsxE6QpAiRzfXOT+H5+AHjoIXH81AVXaY4OWz6UjrGIC5IeBsnJ8hy9eonjERHSFDH+4vgdd8insW95tnyQJUh+JvD++/Icjz4qjsfESFPMnJIvjGs8lklzKAct3+PFrPR0cfzpp+U55s4Vx1U8jTcx5qgwrvFYI82h/DReOsbW8ZoE9Wx2kUBERGQNXCSox9MNREREZBYrCUREZFdYSVCPlQQiIiIyi4sEIiKyKzcqCZb+sJb8/HzExcVBr9fD09MTkydPRnFxsXD8E088gY4dO8LFxQWtW7fGtGnTUFhYWOttc5FARERkw+Li4nD8+HFs2bIFmzZtwu7du/Hwww/XOP7SpUu4dOkSFi1ahGPHjmHVqlVITk7G5MmTa71tXpNARER25VZqpnTixAkkJyfj0KFDiIqKAgC89dZbuOuuu7Bo0SIEBFR/ZHqXLl3w5ZdfGj9v3749Xn75ZTzwwAOorKyEk6zHwF+wkkBERGSj9u/fD09PT+MCAQBiYmLg4OCAgwcPqs5TWFgIvV5fqwUCYMOVBNkqLyzUUO9t6HTyNVKpv7hZkhdK5Rty8lQ5o3oaL25yEpB5SprCENJBGJ82XkVjqDLx26qDtzxF4pmpkhEqmhyp4S/pYiRr+qOGij9ZXMvEx3XfHvlBW70mXPWU6uUvv6zMGj5cnkNwPhUAkJenfj41UEqelI55YZG8sZdFhIQIw4YDKdIUDnmSnz01fxp7i99HSvk8aYr3Vskb2dm6ykrA0dHyOQGgqKjI5HWtVgutVlvnvNnZ2fD19TV5zcnJCV5eXsjOVtdALS8vDy+++KLwFEVNWEkgIiK7Ys0LF4OCguDh4WH8SEpKMjuHWbNmQaPRCD/SZZ06VSgqKsKwYcMQFhaGefPm1frrbbaSQEREdKvJysqCXq83fl5TFeGpp55CfHy8MFe7du3g7++P3FzTKlJlZSXy8/PhL6mEXrt2DUOGDIG7uzs2bNiAZs2aqduJv+AigYiI7Io1Tzfo9XqTRUJNfHx84OPjIx0XHR2NgoICpKamIjIyEgCwfft2GAwG9OzZs8avKyoqQmxsLLRaLTZu3AidTqduR27C0w1EREQ2qlOnThgyZAimTJmClJQU7N27FwkJCRg/frzxzoaLFy8iNDQUKSl/XNtSVFSEwYMHo6SkBB988AGKioqQnZ2N7OxsVNXyNgxWEoiIyK7cSrdAAsBnn32GhIQEDBo0CA4ODrj33nvx5ptvGuPXr1/HyZMnUVr6x4X0R44cMd75EHLTRbPnzp1DcHCw6m1zkUBERGTDvLy8sHr16hrjwcHBUBTF+PmAAQNMPq8PLhKIiMiuVFYCDhY+2d5UH/Bks4uE5s1HNPYUbjka7b2NPYVbjua2iY09hVuOpo2shwXdTNPqkcaeAlGd2OwigYiIyBpYSVCPiwQiIrIrXCSox1sgiYiIyCxWEoiIyK6wkqAeKwlERERkFisJRERkV261ZkqNiZUEIiIiMouVBCIisiuVlYBGY/mcTRErCURERGQWKwlERGRXWElQj4sEIiKyK1wkqMfTDURERGQWKwlERGRXWElQj5UEIiIiMouVBCIisitVVZavJLCZEhEREdkVVhKIiMiuWOP6gaZ6TYLNLhLKsUkYv3BGqfc20tLkYwYMEMcLCuQ5mjcXx/39R8iTqLBNcswyP5AfswceEMeHDJHPY3tyhTB+Kc9ZmiMrSxzv1csyx0yZ3VUY//aOV6Q57vJOEcZ7JPSQ5kj59JR4gIrfQEWBYcK4h4dljtleyfvs8BL5+yw+XhyfOVM+j2VLDcL4pWx5oXTrVnF84kQLvc/elvzgtGwpzfGtbrQwfpfnPmmOg469hXF3d2kKhLmdF8Y1babKk9Atw2YXCURERNbASoJ6vCaBiIiIzGIlgYiI7AorCepxkUBERHbFGrcr8hZIIiIisiusJBARkV2prASU+t8gZ4KVBCIiIrIrNltJyDguXuaFvfSgNMf5eSuF8dHDxffzA0DOVfE9/U4qjmBDrTCLvxYfsx4j5X1I06PEOd5/Xz6PXzLqf8zKy+VjLOHIGHEfhLviw6U5Kg4fFcafe04+jyL/DsK4mouirv8uH2MJlbvE75GQ/vL32Yme4hzPPCOfx5E08d84mZnyHIGB8jEW8c9/CsODH/CVptgkbk8B7CyW5ujZq0gYv1Ssl+ZAcZl8jI1jJUE9VhKIiIjILJutJBAREVkDKwnqsZJAREREZrGSQEREdoWVBPW4SCAiIrtSVWX5RYJB/LyxWxZPNxAREZFZrCQQEZFdqawEHCz8JzIrCURERGRXbLaSEBwsjpe+LW6UBACtt24U54i5W5rD3V0cd3GRpmgwQ4aI48VX5CfhvBb9WxgvfU7cfAgAsrOlQ6Rk339L6R6cL4zLGiUBgHPGL8L43UNCpDnOXhA3oNLppClUjbGEqChxvOCi/H0WkDBaGC9atV6ao1jSOyhEftjh5iYfYwk5irhZ0veb5I3dZB21croNlqbwKzgvjAfoVDRK8veXj7FxrCSox0oCERERmWWzlQQiIiJrYCVBPVYSiIiIyCxWEoiIyK5UVVn+L39L912wFVwkEBGRXamsBDTyh5XWSlNdJPB0AxEREZnFSgIREdkVVhLUq9ciYf78+Zg9ezamT5+OxYsXW2hKf0hOFsdHD5ffV3wpStwHISD7rHwiGRnCsJp7k2W9FizloYfE8Vmz5DkKHhL3QWg35i5pjnbjxwvjZ++YIJ9IA7k/wUsYHzBAnuOhh8KE8QJH+W+jdhcvCuOlngHSHGUqbnG3hOXLxXFZvw4AyJsn7oPQzkN+zMILC4Xxo5l6aQ7Jj7fF+JVIftds3S/NcervccJ4QoJ8Ht8/dEAYz+k/VppD1muBmpY6LxIOHTqEd999F+Hh4ZacDxERkVWxkqBena5JKC4uRlxcHFasWIEWLVpYek5ERERkA+q0SJg6dSqGDRuGmJgYS8+HiIjIqiorrfPRFNX6dMOaNWtw5MgRHDp0SNX48vJylJeXGz8vKiqq7SaJiIioEdSqkpCVlYXp06fjs88+g07l02SSkpLg4eFh/AgKCqrTRImIiCzDAEWx7AfQNPsy12qRkJqaitzcXHTv3h1OTk5wcnLCrl278Oabb8LJyQlVVVXVvmb27NkoLCw0fmRlZVls8kRERLVXZaUP68jPz0dcXBz0ej08PT0xefJkFMseg/o/iqJg6NCh0Gg0+Oqrr2q97Vqdbhg0aBB+/vlnk9cmTZqE0NBQzJw5E46OjtW+RqvVQqvV1npiREREBMTFxeG3337Dli1bcP36dUyaNAkPP/wwVq9eLf3axYsXQ1OPWzlqtUhwd3dHly5dTF5r3rw5brvttmqvExER2SZr/OVvnUrCiRMnkJycjEOHDiEqKgoA8NZbb+Guu+7CokWLEBBQcw+VtLQ0vPbaazh8+DBatmxZp+3bbMfF4cPF8dJKZ2kOb2/JgGJP+UQk3XTcVVzR6tRAR/n998VxldUpsVWr5GMkXX083eQpGuqYrf5UfB7RoOKMnKyJkZeZ03DVZGfLx0i4qTiuliBr2qOmqdPvv4vjbiUl8iRpacKwX/ve0hSBgfLNWEKpfzthXHefOA4AHSrFDeS+XyP/AT+VJ26W1KGFvEkdnBrooN2ibr44v77V9P3798PT09O4QACAmJgYODg44ODBg7jnnnvMfl1paSnuv/9+LF26FP7+/nXefr1/Fe/cubO+KYiIiBqQ9SoJN1+cP3fuXMybN6/OWbOzs+Hr62vympOTE7y8vJAt+OPiySefRO/evTFy5Mg6bxuw4UoCERHRrSYrKwt6/Z8twWuqIsyaNQsLFiwQ5jpx4kSd5rBx40Zs374dP/74Y52+/q+4SCAiIjtjjVsW/8in1+tNFgk1eeqppxAfHy8c065dO/j7+yM3N9fk9crKSuTn59d4GmH79u04c+YMPD09TV6/99570bdv31qdAeAigYiIqIH5+PjAx8dHOi46OhoFBQVITU1FZGQkgD8WAQaDAT179jT7NbNmzcJDNz3xr2vXrnjjjTcwYsSIWs2TiwQiIrIzt87dDZ06dcKQIUMwZcoULF++HNevX0dCQgLGjx9vvLPh4sWLGDRoED7++GP06NED/v7+ZqsMrVu3Rtu2bWu1/To9u4GIiOjWZYDlGylZr+PiZ599htDQUAwaNAh33XUX7rjjDrz33nvG+PXr13Hy5EmUlpZafNusJBAREdkwLy8vYeOk4OBgKJJnVcviNbHZRYIlnqgly1Gp81KRRBx2TV4vTVEUM1q+HQuQ7a+a3gMFBZIcnr7iAQAKJPfJt543TZrj/NNvSsdYhKR5RJmT/AKka9fEcZ1OXrC7UFlzQxQAaJ22T5ojP1TeF8ASZH0Q1LR8kL1XXVxcpTmyvcX722HSXdIc55d/Kx1jCRZ5QqDkvVrkJP991sHtkjBucBK/DwHAoSBfOsb23TqnGxobTzcQERGRWTZbSSAiIrIOVhLUYiWBiIiIzGIlgYiI7Iz1mik1NawkEBERkVmsJBARkZ3hNQlqcZFARER2hosEtXi6gYiIiMyy2UqCmoYsMrIGJoGB8hyyxjGZofJGSWHjZU1dHOUTUSE9XRzPzJTnuOmhYdWEhspzyBoyHR4gb5Skb6ORjBgun4gKRRA3SyrIq/82JD1wAMjfZ2f95Y2S2o0ZKBnRXD4RFQ4fFsfVNA6SHZM8FcddliMjQd4oKa2B3me//y6O69NTpDkMUT3EOfJyhXEA0m+OmkZJ3WNUNKGzeawkqMVKAhEREZlls5UEIiIi67jxgCdL52x6WEkgIiIis1hJICIiO8NmSmqxkkBERERmsZJARER2hnc3qMVFAhER2RkuEtTSKIqiNOQGi4qK4OHhASAWQLOG3DQRETW66wA2o7CwEHq9uE+Jpf35/88eAG4Wzl4M4I5G2S9rYiWBiIjsDCsJavHCRSIiIjKLlQQiIrIzrCSoxUoCERERmcVKAhER2RkFlm9+1KD3ADQYVhKIiIjILFYSiIjIzvCaBLW4SCAiIjvDRYJaPN1AREREZrGSQEREdoaVBLVYSSAiIiKzWEkgIiI7w0qCWqwkEBERkVmsJBARkZ0xwPLNlCydzzawkkBERERmsZJARER2htckqMVKAhEREZlls5WEvXu/EcZ7h+TKk+h0wnB+pV6awtNTHHfIviSfh5ubMKzxiJPnUGHSJPExWzlfxTHLyxOGL3mGSVMEQHJMJNsAADiJ35qazjPlOVRQltwpjJ8dPk2ao51/qTD+3qeu0hwPx1eIB0iOhxoax5H1zgEAypkl4gGS9zsAlLr5CuOZmfJ53HabOO7nLv6+AECFk/h7o9WOkE9EhV9/Ff9sts7cLc2R36WfMO7lVCTNcalY/DsvwFvyPgTkP5sWep9ZlwGW/8u/aV6TYLOLBCIiIuvg6Qa1eLqBiIiIzGIlgYiI7AxvgVSLlQQiIiIyi5UEIiKyM7wmQS1WEoiIiMgsVhKIiMjOsJKgls0uEnpHiO9xPnVBfJ81AHQIlNwnXSmfh0OZOEeFd4A0h3NxvnxDFrByufge5+175MdsYJS4t0RlgYqJuIlzIDhYnqNSxTfHAnLGifsgtPvpe2kOQ/BgYXzMGPk8SiudhXFXJ/lFUaVlDVQY9PcXhnOuyftC+DmJ36thISrmIblf/+gx+TzCvVX0ObGA1k7i7byXLu6BAAAPR0l+nxWXSXMEeIt/No+mi9+HABDu9It0DDUdNrtIICIisg5WEtTiIoGIiOwMOy6qxQsXiYiIyCxWEoiIyM6wmZJarCQQERGRWVwkEBGRnamy0od15OfnIy4uDnq9Hp6enpg8eTKKi4ulX7d//34MHDgQzZs3h16vR79+/fD777/XattcJBAREdmwuLg4HD9+HFu2bMGmTZuwe/duPPzww8Kv2b9/P4YMGYLBgwcjJSUFhw4dQkJCAhwcavffPq9JICIiO3Pr3AJ54sQJJCcn49ChQ4iKigIAvPXWW7jrrruwaNEiBASY79Xz5JNPYtq0aZg1a5bxtY4dO9Z6+7a7SCgTNwbp4C1vHLL9gJcwPvAOcUMXAICTuPmIc6WaHA10mNPShOGBIeImOADw4IzWwvjKRSoaQ3l6iuMFBfIcku+/pfhpcoXxigHiRkkA4JxxShj3kjQfAoBTF/TCuIeHfPVfXi4dYhk68c+ErFESAODwYWG4qEtvaYq8PHE8NFQ+jdJKeTM0SyhyE2/n4SHn5UkyxeXlU05h0hTBnuJ4eIikYRMAFHvLx9ixoqIik8+1Wi20Wm2d8+3fvx+enp7GBQIAxMTEwMHBAQcPHsQ999xT7Wtyc3Nx8OBBxMXFoXfv3jhz5gxCQ0Px8ssv44477qjV9nm6gYiI7Iz1rkkICgqCh4eH8SMpKaleM83Ozoavr2m3XCcnJ3h5eSE7O9vs15w9exYAMG/ePEyZMgXJycno3r07Bg0ahNOnT9dq+7ZbSSAiIrIK651uyMrKgl7/Z2WwpirCrFmzsGDBAmHGEydO1GkmBsMft2M+8sgjmDRpEgDgb3/7G7Zt24aVK1fWauHCRQIREZGF6PV6k0VCTZ566inEx8cLx7Rr1w7+/v7IzTU9LVpZWYn8/Hz413Aqs2XLlgCAsDDTU1CdOnXC+fMqTm39BRcJRERkZxq/mZKPjw98fHyk46Kjo1FQUIDU1FRERkYCALZv3w6DwYCePXua/Zrg4GAEBATg5MmTJq+fOnUKQ4cOrdU8eU0CERGRjerUqROGDBmCKVOmICUlBXv37kVCQgLGjx9vvLPh4sWLCA0NRUpKCgBAo9HgmWeewZtvvol169YhIyMD//nPf5Ceno7JkyfXavusJBARkZ25tR7w9NlnnyEhIQGDBg2Cg4MD7r33Xrz55pvG+PXr13Hy5EmUlv55d8qMGTNQVlaGJ598Evn5+ejWrRu2bNmC9u3b12rbXCQQERHZMC8vL6xevbrGeHBwMBRFqfb6rFmzTPok1EWtFgnz5s3D888/b/Jax44dkZ6eXq9JmFPkJO5xoOZW+4EDJCu7dV9Jc1y6Y6ww7u/vLM3h4NYwa7H8kB7CuJo7X6R9EDIz5Um6dBHHZX0UAODCBfkYC1i91VcYV3NLcevgYPGAPXukOTrIbup395TmMPi4SsdYwi/p4rOUYYHyHhc57cV9EPzeelmaI3PE/wnjzk7yv+yuXm2YM656J0n/ARW/0M66hQvjHTK3S3OcwkBhPCRE/h4qKGuY95l13TrNlBpbrf/36ty5M7Zu3fpngoZqFEREREQNqtb/wzs5OdV42wUREZHtYyVBrVrX2k6fPo2AgAC0a9cOcXFxtb7nkoiIqHHdWk+BbEy1qiT07NkTq1atQseOHfHbb7/h+eefR9++fXHs2DG4u7ub/Zry8nKU/6Wp/M19rYmIiMg21WqR8NcmDOHh4ejZsyfatGmDzz//vMZ7L5OSkqpd7EhERNR4eLpBrXpd2uvp6YkOHTogIyOjxjGzZ89GYWGh8SMrK6s+myQiIqIGUq9FQnFxMc6cOWPsE22OVqs19rJW29OaiIjIegxW+mh6arVIePrpp7Fr1y5kZmZi3759uOeee+Do6Ij77rvPWvMjIiKiRlKraxIuXLiA++67D1euXIGPjw/uuOMOHDhwQNVDKmpLr6sQxytVNNsp9haGj4SIGyUBQHdv8TyQVyCfh04nH2MBXsXiO016Vqk4ZmXBwnBOq+7SFNvWieP39zorn0cD9d+4f7x49X/+gnwdnV8sbqjlJWu2BADHjonnETpYmqKGJ9JanKzvE4rlOWq4ztlo3z/EjZIAoPfyx4Xx0kXvSHP4+TTMX38VTuIGRM7e4t9VABAsufP8aLG4URIAhGd+L4wbQuTvM6+yS9Ixtu/WasvcmGr1m3jNmjXWmgcRERHZGLZLJCIiO8O7G9TiIoGIiOwMFwlqNczTTYiIiOiWw0oCERHZGWvcstg0L1xkJYGIiIjMYiWBiIjsDK9JUMt2FwllZcJwqX87aQpXJ3GPg+4RKspDleLwqQJfaYoO/jbyUCvpDe6Q9ifwcZOnuH+M+Liv/FT+vXtwVL58Q5ZQKf4GBwaKeyAA0rcqEBgozZHTXHxMWl/5RZoj3y1MOsYSHMpKhfEjGfKuqhER4njvKEl/EgBFXcR9EPSZ8mMGNT0sLMAZ4v05mhcgzREu6dnSpYv8vWroIu6D4JCtogeC5GeGmhbbXSQQERFZBSsJavGaBCIiIjKLlQQiIrIzrCSoxUUCERHZGT67QS2ebiAiIiKzWEkgIiI7w2ZKarGSQERERGaxkkBERHaGFy6qZbuLBDdx1x7XdAs0SpE0DgKACogblHRwkzcfeXOVvFFKgygokI/R6YRhh7w8eQ5PT2H4wZhiaYrew1vLt2MBsu/v1cvyHC4u4rhBJ29yI8txyVPeKCng/y2VjrGEnGuuwnj33/4rz9FqmDBeWCg/ZpJfEdB7e0tzXCoQ74ul5FwV70/4D/LvXVHwVGFczY+mv7847irtDAY8+JK8GRo1Hba7SCAiIrIKVhLU4jUJREREZBYrCUREZGdYSVCLiwQiIrIzXCSoxdMNREREZBYrCUREZGfYllktVhKIiIjILI2iKEpDbrCoqAgeHh4AYgE0a8hNExFRo7sOYDMKCwuh1+sbdMt//v9zLyz//891AF82yn5ZEysJREREZBavSSAiIjtTBcv/jcy7G4iIiMiOsJJARER2hpUEtbhIICIiO8NFglo83UBERERmsZJARER2xgDLNz9qms2UGnyR8GdbhsqG3jQRETW6P373N3CLHrNzsP2cja/BFwnXrl3737+2NfSmiYjIRly7du1/jY0ajrOzM/z9/ZGdvdUq+f39/eHs7GyV3I2lwTsuGgwGXLp0Ce7u7tBoNA25aaGioiIEBQUhKyurSXXLsiYes9rjMas9HrPas+VjpigKrl27hoCAADg4NPxlcWVlZaioqLBKbmdnZ+h0OqvkbiwNXklwcHBAYGBgQ29WNb1eb3M/VLaOx6z2eMxqj8es9mz1mDV0BeGvdDpdk/uP3Jp4dwMRERGZxUUCERERmcVFwv9otVrMnTsXWq22sadyy+Axqz0es9rjMas9HjOylAa/cJGIiIhuDawkEBERkVlcJBAREZFZXCQQERGRWVwkEBERkVlcJPzP0qVLERwcDJ1Oh549eyIlJaWxp2Szdu/ejREjRiAgIAAajQZfffVVY0/J5iUlJeHvf/873N3d4evri1GjRuHkyZONPS2btmzZMoSHhxsbAkVHR+O7775r7GndMubPnw+NRoMZM2Y09lToFsZFAoC1a9ciMTERc+fOxZEjR9CtWzfExsYiNze3sadmk0pKStCtWzcsXbq0sadyy9i1axemTp2KAwcOYMuWLbh+/ToGDx6MkpKSxp6azQoMDMT8+fORmpqKw4cPY+DAgRg5ciSOHz/e2FOzeYcOHcK7776L8PDwxp4K3eJ4CySAnj174u9//zvefvttAH88XyIoKAhPPPEEZs2a1cizs20ajQYbNmzAqFGjGnsqt5TLly/D19cXu3btQr9+/Rp7OrcMLy8vLFy4EJMnT27sqdis4uJidO/eHe+88w5eeuklREREYPHixY09LbpF2X0loaKiAqmpqYiJiTG+5uDggJiYGOzfv78RZ0ZNWWFhIYA//tMjuaqqKqxZswYlJSWIjo5u7OnYtKlTp2LYsGEmv9OI6qrBH/Bka/Ly8lBVVQU/Pz+T1/38/JCent5Is6KmzGAwYMaMGejTpw+6dOnS2NOxaT///DOio6NRVlYGNzc3bNiwAWFhYY09LZu1Zs0aHDlyBIcOHWrsqVATYfeLBKKGNnXqVBw7dgx79uxp7KnYvI4dOyItLQ2FhYVYt24dJk6ciF27dnGhYEZWVhamT5+OLVu28CmHZDF2v0jw9vaGo6MjcnJyTF7PycmBv79/I82KmqqEhARs2rQJu3fvtulHptsKZ2dnhISEAAAiIyNx6NAhLFmyBO+++24jz8z2pKamIjc3F927dze+VlVVhd27d+Ptt99GeXk5HB0dG3GGdCuy+2sSnJ2dERkZiW3bthlfMxgM2LZtG899ksUoioKEhARs2LAB27dvR9u2bRt7Srckg8GA8vLyxp6GTRo0aBB+/vlnpKWlGT+ioqIQFxeHtLQ0LhCoTuy+kgAAiYmJmDhxIqKiotCjRw8sXrwYJSUlmDRpUmNPzSYVFxcjIyPD+Pm5c+eQlpYGLy8vtG7duhFnZrumTp2K1atX4+uvv4a7uzuys7MBAB4eHnBxcWnk2dmm2bNnY+jQoWjdujWuXbuG1atXY+fOndi8eXNjT80mubu7V7vGpXnz5rjtttt47QvVGRcJAMaNG4fLly9jzpw5yM7ORkREBJKTk6tdzEh/OHz4MP7xj38YP09MTAQATJw4EatWrWqkWdm2ZcuWAQAGDBhg8vqHH36I+Pj4hp/QLSA3NxcTJkzAb7/9Bg8PD4SHh2Pz5s248847G3tqRHaDfRKIiIjILLu/JoGIiIjM4yKBiIiIzOIigYiIiMziIoGIiIjM4iKBiIiIzOIigYiIiMziIoGIiIjM4iKB6CZXr17FCy+8gNzc3MaeCtWBoih48803kZKS0thTIbrlcZFAdJMWLVpAq9UiLi4OBoPBJLZz505oNBoUFBQ0+Lzi4+MxatSoOn/9gAEDMGPGDIvNRyYzMxMajQZpaWk1jrHG8dRoNIiIiMC4ceOqPbiNiGqHiwQ7Fh8fD41GA41Gg2bNmsHPzw933nknVq5cWe0/R0ttb968ecbPg4ODsXjxYotvxxJmzpyJDh06mMzXkhr6P2wAWL9+PV588cUG3aZM7969jW2XLalfv36YP38+4uLiUFVVZdHcRPaEz26wc0OGDMGHH36Iqqoq5OTkIDk5GdOnT8e6deuwceNGODnZ71tk6dKljT0Fi/Ly8mrsKVTj7OxstUeyjxs3DuPGjbNKbiJ7wUqCndNqtfD390erVq3QvXt3/Pvf/8bXX3+N7777zuRhTefPn8fIkSPh5uYGvV6PsWPHmpRy582bh4iICHzyyScIDg6Gh4cHxo8fj2vXrpnd7oABA/Drr7/iySefNFYzzDFXsi4oKIBGo8HOnTsB/HENQVxcHHx8fODi4oLbb78dH374oXF8VlYWxo4dC09PT3h5eWHkyJHIzMys8zEDgL179yI8PBw6nQ69evXCsWPHjLErV67gvvvuQ6tWreDq6oquXbvi//2//2eMx8fHY9euXViyZIlx32/M5/jx4xg+fDj0ej3c3d3Rt29fnDlzxmTbixYtQsuWLXHbbbdh6tSpuH79uqo531y9+OSTTxAVFQV3d3f4+/vj/vvvr3Ydhmg+BoMBL7zwAgIDA6HVao0PRrtZeno6evfuDZ1Ohy5dumDXrl3G2M2nG2TH7sZ+TJs2Dc8++yy8vLzg7+9vtYoPkb3jIoGqGThwILp164b169cD+OM/g5EjRyI/Px+7du3Cli1bcPbs2Wp/pZ05cwZfffUVNm3ahE2bNmHXrl2YP3++2W2sX78egYGBeOGFF/Dbb7/ht99+q/N8//Of/+CXX37Bd999hxMnTmDZsmXw9vYGAFy/fh2xsbFwd3fHDz/8gL1798LNzQ1DhgxBRUVFnbf5zDPP4LXXXsOhQ4fg4+ODESNGGP+zLisrQ2RkJP773//i2LFjePjhh/Gvf/3LeCHdkiVLEB0djSlTphj3PSgoCBcvXkS/fv2g1Wqxfft2pKam4sEHH0RlZaVxuzt27MCZM2ewY8cOfPTRR1i1alWdn7x5/fp1vPjii/jpp5/w1VdfITMz0+SJlLL5LFmyBK+99hoWLVqEo0ePIjY2FnfffTdOnz5d7Vg99dRT+PHHHxEdHY0RI0bgypUrZuckO3Y3fPTRR2jevDkOHjyIV199FS+88AK2bNlSp+NARAIK2a2JEycqI0eONBsbN26c0qlTJ0VRFOX7779XHB0dlfPnzxvjx48fVwAoKSkpiqIoyty5cxVXV1elqKjIOOaZZ55RevbsWeP227Rpo7zxxhvCOZ47d04BoPz444/G165evaoAUHbs2KEoiqKMGDFCmTRpktmv/+STT5SOHTsqBoPB+Fp5ebni4uKibN68Wbhtc3bs2KEAUNasWWN87cqVK4qLi4uydu3aGr9u2LBhylNPPWX8vH///sr06dNNxsyePVtp27atUlFRYTbHxIkTlTZt2iiVlZXG1/75z38q48aNUzV3c9v8q0OHDikAlGvXrqmaT0BAgPLyyy+bvPb3v/9defzxxxVF+fN7N3/+fGP8+vXrSmBgoLJgwQJFUf48nlevXq1xXuaO3R133FFtuzNnzqwxBxHVDSsJZJaiKMZTACdOnEBQUBCCgoKM8bCwMHh6euLEiRPG14KDg+Hu7m78vGXLlg1yG+Fjjz2GNWvWICIiAs8++yz27dtnjP3000/IyMiAu7s73Nzc4ObmBi8vL5SVlVUr49dGdHS08d9eXl7o2LGj8VhUVVXhxRdfRNeuXeHl5QU3Nzds3rwZ58+fF+ZMS0tD37590axZsxrHdO7cGY6OjsbP63OMU1NTMWLECLRu3Rru7u7o378/ABjnKZpPUVERLl26hD59+pi83qdPH5P3BGB6rJycnBAVFVVtzA1qj114eLjJ5w31XiOyN/Z7VRoJnThxAm3btq3V19z8n4lGo6n3XRIODn+sYxVFMb528zn4oUOH4tdff8W3336LLVu2YNCgQZg6dSoWLVqE4uJiREZG4rPPPquW28fHp15zq8nChQuxZMkSLF68GF27dkXz5s0xY8YM6ekNFxcXaW5LHeOSkhLExsYiNjYWn332GXx8fHD+/HnExsYa56lmPpam9thZ471GRNWxkkDVbN++HT///DPuvfdeAECnTp2QlZWFrKws45hffvkFBQUFCAsLq/N2nJ2dpben3fiP/K/XLJi7797HxwcTJ07Ep59+isWLF+O9994DAHTv3h2nT5+Gr68vQkJCTD7qc9vdgQMHjP++evUqTp06hU6dOgH446LGkSNH4oEHHkC3bt3Qrl07nDp1yuTrze17eHg4fvjhB9UXItZHeno6rly5gvnz56Nv374IDQ2t9pe4aD56vR4BAQHYu3evyet79+6t9p7467GqrKxEamqq8VjdTM2xI6KGw0WCnSsvL0d2djYuXryII0eO4JVXXsHIkSMxfPhwTJgwAQAQExODrl27Ii4uDkeOHEFKSgomTJiA/v37Iyoqqs7bDg4Oxu7du3Hx4kXk5eWZHePi4oJevXph/vz5OHHiBHbt2oXnnnvOZMycOXPw9ddfIyMjA8ePH8emTZuM/wnFxcXB29sbI0eOxA8//IBz585h586dmDZtGi5cuFDnub/wwgvYtm0bjh07hvj4eHh7exsbHd1+++3YsmUL9u3bhxMnTuCRRx6p1tQnODgYBw8eRGZmJvLy8mAwGJCQkICioiKMHz8ehw8fxunTp/HJJ5/g5MmTdZ5nTVq3bg1nZ2e89dZbOHv2LDZu3Fith4JsPs888wwWLFiAtWvX4uTJk5g1axbS0tIwffp0kzxLly7Fhg0bkJ6ejqlTp+Lq1at48MEHzc5LzbEjoobDRYKdS05ORsuWLREcHIwhQ4Zgx44dePPNN/H1118bz31rNBp8/fXXaNGiBfr164eYmBi0a9cOa9eurde2X3jhBWRmZqJ9+/bC0v/KlStRWVmJyMhIzJgxAy+99JJJ3NnZGbNnz0Z4eDj69esHR0dHrFmzBgDg6uqK3bt3o3Xr1hg9ejQ6deqEyZMno6ysDHq9vs5znz9/PqZPn47IyEhkZ2fjm2++gbOzMwDgueeeQ/fu3REbG4sBAwbA39+/WqfEp59+Go6OjggLCzOW+m+77TZs374dxcXF6N+/PyIjI7FixQrhNQp15ePjg1WrVuGLL75AWFgY5s+fj0WLFpmMkc1n2rRpSExMxFNPPYWuXbsiOTkZGzduxO23317tWM2fPx/dunXDnj17sHHjRuPdJzdTc+yIqOFolL+e7CWiJis6OhqDBg2qtsgiIqoJKwlETVx5eTkOHz6M48ePo3Pnzo09HSK6hfDuBqIm4IcffsDQoUNrjDs4OODuu+/GmDFjGnBWRHSr4+kGoibg999/x8WLF2uMh4SENOBsiKip4CKBiIiIzOI1CURERGQWFwlERERkFhcJREREZBYXCURERGQWFwlERERkFhcJREREZBYXCURERGQWFwlERERk1v8HEReMkdNtonkAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_as_patches(j)\n", "\n", "_ = plt.title('These slices are not diagonal')\n", "_ = plt.xlabel(\"Don't use `batch_jacobian`\")" ] }, { "cell_type": "markdown", "metadata": { "id": "M_x7ih5sarvG" }, "source": [ "この場合、`batch_jacobian` は引き続き実行され、期待される形状の*何か*を返しますが、その内容の意味は不明確です。" ] }, { "cell_type": "code", "execution_count": 39, "metadata": { "execution": { "iopub.execute_input": "2022-12-14T20:58:27.121721Z", "iopub.status.busy": "2022-12-14T20:58:27.121486Z", "iopub.status.idle": "2022-12-14T20:58:27.460382Z", "shell.execute_reply": "2022-12-14T20:58:27.459722Z" }, "id": "k8_mICHoasCi" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "jb.shape: (7, 6, 5)\n" ] } ], "source": [ "jb = tape.batch_jacobian(y, x)\n", "print(f'jb.shape: {jb.shape}')" ] } ], "metadata": { "colab": { "collapsed_sections": [], "name": "advanced_autodiff.ipynb", "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.16" } }, "nbformat": 4, "nbformat_minor": 0 }