{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZjN_IJ8mhJ-4"
   },
   "source": [
    "##### Copyright 2020 The TensorFlow Authors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "cellView": "form",
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:12.101188Z",
     "iopub.status.busy": "2024-01-11T18:47:12.100961Z",
     "iopub.status.idle": "2024-01-11T18:47:12.104718Z",
     "shell.execute_reply": "2024-01-11T18:47:12.104142Z"
    },
    "id": "sY3Ffd83hK3b"
   },
   "outputs": [],
   "source": [
    "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "# https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "03Pw58e6mTHI"
   },
   "source": [
    "# TensorFlow の NumPy API"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7WpGysDJmZsg"
   },
   "source": [
    "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
    "  <td><a target=\"_blank\" href=\"https://www.tensorflow.org/guide/tf_numpy\"><img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\">TensorFlow.org で表示</a></td>\n",
    "  <td><a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs-l10n/blob/master/site/ja/guide/tf_numpy.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\">Google Colab で実行</a></td>\n",
    "  <td><a target=\"_blank\" href=\"https://github.com/tensorflow/docs-l10n/blob/master/site/ja/guide/tf_numpy.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\">GitHub でソースを表示</a></td>\n",
    "  <td><a href=\"https://storage.googleapis.com/tensorflow_docs/docs-l10n/site/ja/guide/tf_numpy.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\">ノートブックをダウンロード</a></td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "s2enCDi_FvCR"
   },
   "source": [
    "## 概要\n",
    "\n",
    "TensorFlow では、<code>tf.experimental.numpy</code>を利用して<a>NumPy API</a> のサブセットを実装します。これにより、TensorFlow により高速化された NumPy コードを実行し、TensorFlow のすべて API にもアクセスできます。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ob1HNwUmYR5b"
   },
   "source": [
    "## セットアップ\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:12.108567Z",
     "iopub.status.busy": "2024-01-11T18:47:12.108305Z",
     "iopub.status.idle": "2024-01-11T18:47:14.712532Z",
     "shell.execute_reply": "2024-01-11T18:47:14.711782Z"
    },
    "id": "AJR558zjAZQu"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-01-11 18:47:12.854741: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n",
      "2024-01-11 18:47:12.854782: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n",
      "2024-01-11 18:47:12.856314: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using TensorFlow version 2.15.0\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "import tensorflow.experimental.numpy as tnp\n",
    "import timeit\n",
    "\n",
    "print(\"Using TensorFlow version %s\" % tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "M6tacoy0DU6e"
   },
   "source": [
    "### NumPy 動作の有効化\n",
    "\n",
    "`tnp` を NumPy として使用するには、TensorFlow の NumPy の動作を有効にしてください。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:14.716312Z",
     "iopub.status.busy": "2024-01-11T18:47:14.715892Z",
     "iopub.status.idle": "2024-01-11T18:47:14.719443Z",
     "shell.execute_reply": "2024-01-11T18:47:14.718718Z"
    },
    "id": "TfCyofpFDQxm"
   },
   "outputs": [],
   "source": [
    "tnp.experimental_enable_numpy_behavior()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "et9D5wq0D1H2"
   },
   "source": [
    "この呼び出しによって、TensorFlow での型昇格が可能になり、リテラルからテンソルに変換される場合に、型推論も Numpy の標準により厳格に従うように変更されます。\n",
    "\n",
    "注意: この呼び出しは、`tf.experimental.numpy` モジュールだけでなく、TensorFlow 全体の動作を変更します。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yh2BwqUzH3C3"
   },
   "source": [
    "## TensorFlow NumPy ND 配列\n",
    "\n",
    "**ND 配列**と呼ばれる `tf.experimental.numpy.ndarray` は、特定のデバイスに配置されたある `dtype` の多次元の密な配列を表します。`tf.Tensor` のエイリアスです。`ndarray.T`、`ndarray.reshape`、`ndarray.ravel` などの便利なメソッドについては、ND 配列クラスをご覧ください。\n",
    "\n",
    "まず、ND 配列オブジェクトを作成してから、さまざまなメソッドを呼び出します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:14.723114Z",
     "iopub.status.busy": "2024-01-11T18:47:14.722633Z",
     "iopub.status.idle": "2024-01-11T18:47:16.992303Z",
     "shell.execute_reply": "2024-01-11T18:47:16.991532Z"
    },
    "id": "-BHJjxigJ2H1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Created ND array with shape = (5, 3), rank = 2, dtype = <dtype: 'float32'> on device = /job:localhost/replica:0/task:0/device:GPU:0\n",
      "\n",
      "Is `ones` an instance of tf.Tensor: True\n",
      "\n",
      "ndarray.T has shape (3, 5)\n",
      "narray.reshape(-1) has shape (15,)\n"
     ]
    }
   ],
   "source": [
    "# Create an ND array and check out different attributes.\n",
    "ones = tnp.ones([5, 3], dtype=tnp.float32)\n",
    "print(\"Created ND array with shape = %s, rank = %s, \"\n",
    "      \"dtype = %s on device = %s\\n\" % (\n",
    "          ones.shape, ones.ndim, ones.dtype, ones.device))\n",
    "\n",
    "# `ndarray` is just an alias to `tf.Tensor`.\n",
    "print(\"Is `ones` an instance of tf.Tensor: %s\\n\" % isinstance(ones, tf.Tensor))\n",
    "\n",
    "# Try commonly used member functions.\n",
    "print(\"ndarray.T has shape %s\" % str(ones.T.shape))\n",
    "print(\"narray.reshape(-1) has shape %s\" % ones.reshape(-1).shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-BOY8CGRKEhE"
   },
   "source": [
    "### 型昇格\n",
    "\n",
    "TensorFlow の型昇格には 4 つのオプションがあります。\n",
    "\n",
    "- デフォルトでは、混合型の演算に対し、TensorFlow は型を昇格する代わりにエラーを発します。\n",
    "- `tf.numpy.experimental_enable_numpy_behavior()` を実行すると、`NumPy` 型の昇格ルール（以下に説明）を使うように TensorFlow が切り替えられます。\n",
    "- TensorFlow 2.15 以降で使用できる以下の新しいオプションが 2 つあります（詳細は、[TF NumPy 型昇格](tf_numpy_type_promotion.ipynb)をご覧ください）。\n",
    "    - `tf.numpy.experimental_enable_numpy_behavior(dtype_conversion_mode=\"all\")` は、Jax の型昇格ルールを使用します。\n",
    "    - `tf.numpy.experimental_enable_numpy_behavior(dtype_conversion_mode=\"safe\")` は Jax の型昇格ルールを使用しますが、特定の安全でない昇格を許可しません。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SXskSHrX5J45"
   },
   "source": [
    "#### NumPy 型昇格\n",
    "\n",
    "TensorFlow NumPy API には、リテラルを ND 配列に変換するためと ND 配列入力で型昇格を実行するための明確に定義されたセマンティクスがあります。詳細については、[`np.result_type`](https://numpy.org/doc/1.16/reference/generated/numpy.result_type.html) をご覧ください。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vcRznNaMj27J"
   },
   "source": [
    "TensorFlow API は `tf.Tensor` 入力を変更せずそのままにし、それに対して型昇格を実行しませんが、TensorFlow NumPy API は NumPy 型昇格のルールに従って、すべての入力を昇格します。次の例では、型昇格を行います。まず、さまざまな型の ND 配列入力で加算を実行し、出力の型を確認します。これらの型昇格は、TensorFlow API では行えません。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:16.996094Z",
     "iopub.status.busy": "2024-01-11T18:47:16.995820Z",
     "iopub.status.idle": "2024-01-11T18:47:17.012619Z",
     "shell.execute_reply": "2024-01-11T18:47:17.011929Z"
    },
    "id": "uHmBi4KZI2t1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Type promotion for operations\n",
      "int32 + int64 => int64\n",
      "int32 + float32 => float64\n",
      "int32 + float64 => float64\n",
      "int64 + float32 => float64\n",
      "int64 + float64 => float64\n",
      "float32 + float64 => float64\n"
     ]
    }
   ],
   "source": [
    "print(\"Type promotion for operations\")\n",
    "values = [tnp.asarray(1, dtype=d) for d in\n",
    "          (tnp.int32, tnp.int64, tnp.float32, tnp.float64)]\n",
    "for i, v1 in enumerate(values):\n",
    "  for v2 in values[i + 1:]:\n",
    "    print(\"%s + %s => %s\" %\n",
    "          (v1.dtype.name, v2.dtype.name, (v1 + v2).dtype.name))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CrpIoOc7oqox"
   },
   "source": [
    "最後に、`ndarray.asarray` を使ってリテラルをND 配列に変換し、結果の型を確認します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:17.016286Z",
     "iopub.status.busy": "2024-01-11T18:47:17.015701Z",
     "iopub.status.idle": "2024-01-11T18:47:17.020633Z",
     "shell.execute_reply": "2024-01-11T18:47:17.019912Z"
    },
    "id": "1m1cp8_VooNk"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Type inference during array creation\n",
      "tnp.asarray(1).dtype == tnp.int64\n",
      "tnp.asarray(1.).dtype == tnp.float64\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(\"Type inference during array creation\")\n",
    "print(\"tnp.asarray(1).dtype == tnp.%s\" % tnp.asarray(1).dtype.name)\n",
    "print(\"tnp.asarray(1.).dtype == tnp.%s\\n\" % tnp.asarray(1.).dtype.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kd-_iccXoRL8"
   },
   "source": [
    "リテラルを ND 配列に変換する際、NumPy は `tnp.int64` や `tnp.float64` といった幅広い型を優先します。一方、`tf.convert_to_tensor` は、`tf.int32` と `tf.float32` の型を優先して定数を `tf.Tensor` に変換します。TensorFlow NumPy API は、整数に関しては NumPy の動作に従っています。浮動小数点数については、`experimental_enable_numpy_behavior` の `prefer_float32` 引数によって、`tf.float64` よりも `tf.float32` を優先するかどうかを制御することができます（デフォルトは `False` です）。以下に例を示します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:17.024133Z",
     "iopub.status.busy": "2024-01-11T18:47:17.023559Z",
     "iopub.status.idle": "2024-01-11T18:47:17.030874Z",
     "shell.execute_reply": "2024-01-11T18:47:17.030200Z"
    },
    "id": "4gKasnH0j84C"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "When prefer_float32 is True:\n",
      "tnp.asarray(1.).dtype == tnp.float32\n",
      "tnp.add(1., 2.).dtype == tnp.float32\n",
      "When prefer_float32 is False:\n",
      "tnp.asarray(1.).dtype == tnp.float64\n",
      "tnp.add(1., 2.).dtype == tnp.float64\n"
     ]
    }
   ],
   "source": [
    "tnp.experimental_enable_numpy_behavior(prefer_float32=True)\n",
    "print(\"When prefer_float32 is True:\")\n",
    "print(\"tnp.asarray(1.).dtype == tnp.%s\" % tnp.asarray(1.).dtype.name)\n",
    "print(\"tnp.add(1., 2.).dtype == tnp.%s\" % tnp.add(1., 2.).dtype.name)\n",
    "\n",
    "tnp.experimental_enable_numpy_behavior(prefer_float32=False)\n",
    "print(\"When prefer_float32 is False:\")\n",
    "print(\"tnp.asarray(1.).dtype == tnp.%s\" % tnp.asarray(1.).dtype.name)\n",
    "print(\"tnp.add(1., 2.).dtype == tnp.%s\" % tnp.add(1., 2.).dtype.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MwCCDxSZOfA1"
   },
   "source": [
    "### ブロードキャスト\n",
    "\n",
    "TensorFlow と同様に、NumPy は「ブロードキャスト」値の豊富なセマンティクスを定義します。詳細については、[NumPy ブロードキャストガイド](https://numpy.org/doc/stable/user/basics.broadcasting.html)を確認し、これを [TensorFlow ブロードキャストセマンティクス](https://www.tensorflow.org/guide/tensor#broadcasting)と比較してください。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:17.034447Z",
     "iopub.status.busy": "2024-01-11T18:47:17.033863Z",
     "iopub.status.idle": "2024-01-11T18:47:17.040634Z",
     "shell.execute_reply": "2024-01-11T18:47:17.039915Z"
    },
    "id": "qlyOShxIO0s2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Broadcasting shapes (2, 3), (3,) and (1, 2, 1) gives shape (1, 2, 3)\n"
     ]
    }
   ],
   "source": [
    "x = tnp.ones([2, 3])\n",
    "y = tnp.ones([3])\n",
    "z = tnp.ones([1, 2, 1])\n",
    "print(\"Broadcasting shapes %s, %s and %s gives shape %s\" % (\n",
    "    x.shape, y.shape, z.shape, (x + y + z).shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LEVr4ctRPrqR"
   },
   "source": [
    "### インデックス作成\n",
    "\n",
    "NumPy は、非常に洗練されたインデックス作成ルールを定義しています。[NumPy インデックスガイド](https://numpy.org/doc/stable/reference/arrays.indexing.html)を参照してください。以下では、インデックスとして ND 配列が使用されていることに注意してください。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:17.044089Z",
     "iopub.status.busy": "2024-01-11T18:47:17.043514Z",
     "iopub.status.idle": "2024-01-11T18:47:17.792859Z",
     "shell.execute_reply": "2024-01-11T18:47:17.792160Z"
    },
    "id": "lRsrtnd3YyMj"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Basic indexing\n",
      "tf.Tensor(\n",
      "[[[16 17 18 19]\n",
      "  [20 21 22 23]]], shape=(1, 2, 4), dtype=int64) \n",
      "\n",
      "Boolean indexing\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor(\n",
      "[[[ 0  1  2  3]\n",
      "  [ 8  9 10 11]]\n",
      "\n",
      " [[12 13 14 15]\n",
      "  [20 21 22 23]]], shape=(2, 2, 4), dtype=int64) \n",
      "\n",
      "Advanced indexing\n",
      "tf.Tensor([12 13 17], shape=(3,), dtype=int64)\n"
     ]
    }
   ],
   "source": [
    "x = tnp.arange(24).reshape(2, 3, 4)\n",
    "\n",
    "print(\"Basic indexing\")\n",
    "print(x[1, tnp.newaxis, 1:3, ...], \"\\n\")\n",
    "\n",
    "print(\"Boolean indexing\")\n",
    "print(x[:, (True, False, True)], \"\\n\")\n",
    "\n",
    "print(\"Advanced indexing\")\n",
    "print(x[1, (0, 0, 1), tnp.asarray([0, 1, 1])])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:17.796068Z",
     "iopub.status.busy": "2024-01-11T18:47:17.795822Z",
     "iopub.status.idle": "2024-01-11T18:47:17.800135Z",
     "shell.execute_reply": "2024-01-11T18:47:17.799543Z"
    },
    "id": "yRAaiGhlaNw7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Currently, TensorFlow NumPy does not support mutation.\n"
     ]
    }
   ],
   "source": [
    "# Mutation is currently not supported\n",
    "try:\n",
    "  tnp.arange(6)[1] = -1\n",
    "except TypeError:\n",
    "  print(\"Currently, TensorFlow NumPy does not support mutation.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5XfJ602j-GVD"
   },
   "source": [
    "### サンプルモデル\n",
    "\n",
    "次に、モデルを作成して推論を実行する方法を見てみます。この簡単なモデルは、relu レイヤーとそれに続く線形射影を適用します。後のセクションでは、TensorFlow の`GradientTape`を使用してこのモデルの勾配を計算する方法を示します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:17.803539Z",
     "iopub.status.busy": "2024-01-11T18:47:17.803017Z",
     "iopub.status.idle": "2024-01-11T18:47:18.036399Z",
     "shell.execute_reply": "2024-01-11T18:47:18.035666Z"
    },
    "id": "kR_KCh4kYEhm"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor(\n",
      "[[-1.5633821  0.1619502]\n",
      " [-1.5633821  0.1619502]], shape=(2, 2), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "class Model(object):\n",
    "  \"\"\"Model with a dense and a linear layer.\"\"\"\n",
    "\n",
    "  def __init__(self):\n",
    "    self.weights = None\n",
    "\n",
    "  def predict(self, inputs):\n",
    "    if self.weights is None:\n",
    "      size = inputs.shape[1]\n",
    "      # Note that type `tnp.float32` is used for performance.\n",
    "      stddev = tnp.sqrt(size).astype(tnp.float32)\n",
    "      w1 = tnp.random.randn(size, 64).astype(tnp.float32) / stddev\n",
    "      bias = tnp.random.randn(64).astype(tnp.float32)\n",
    "      w2 = tnp.random.randn(64, 2).astype(tnp.float32) / 8\n",
    "      self.weights = (w1, bias, w2)\n",
    "    else:\n",
    "      w1, bias, w2 = self.weights\n",
    "    y = tnp.matmul(inputs, w1) + bias\n",
    "    y = tnp.maximum(y, 0)  # Relu\n",
    "    return tnp.matmul(y, w2)  # Linear projection\n",
    "\n",
    "model = Model()\n",
    "# Create input data and compute predictions.\n",
    "print(model.predict(tnp.ones([2, 32], dtype=tnp.float32)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kSR7Ou5YcS38"
   },
   "source": [
    "## TensorFlow NumPy および NumPy\n",
    "\n",
    "TensorFlow NumPy は、完全な NumPy 仕様のサブセットを実装します。シンボルは、今後追加される予定ですが、近い将来にサポートされなくなる体系的な機能があります。これらには、NumPy C API サポート、Swig 統合、Fortran ストレージ優先順位、ビュー、`stride_tricks`、およびいくつかの`dtype`（`np.recarray`や<code> np.object</code>）が含まれます。詳細については、 <a>TensorFlow NumPy API ドキュメント</a>をご覧ください。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Jb1KXak2YlNN"
   },
   "source": [
    "### NumPy 相互運用性\n",
    "\n",
    "TensorFlow ND 配列は、NumPy 関数と相互運用できます。これらのオブジェクトは、`__array__`インターフェースを実装します。NumPy はこのインターフェースを使用して、関数の引数を処理する前に`np.ndarray`値に変換します。\n",
    "\n",
    "同様に、TensorFlow NumPy 関数は、`np.ndarray` などのさまざまなタイプの入力を受け入れることができます。これらの入力は、<code>ndarray.asarray</code> を呼び出すことにより、ND 配列に変換されます。\n",
    "\n",
    "ND 配列を`np.ndarray`との間で変換すると、実際のデータコピーがトリガーされる場合があります。詳細については、[バッファコピー](#Buffer-copies)のセクションを参照してください。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:18.040315Z",
     "iopub.status.busy": "2024-01-11T18:47:18.040030Z",
     "iopub.status.idle": "2024-01-11T18:47:18.052216Z",
     "shell.execute_reply": "2024-01-11T18:47:18.051583Z"
    },
    "id": "cMOCgzQmeXRU"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sum = 6.0. Class: <class 'numpy.float64'>\n",
      "sum = 6.0. Class: <class 'tensorflow.python.framework.ops.EagerTensor'>\n"
     ]
    }
   ],
   "source": [
    "# ND array passed into NumPy function.\n",
    "np_sum = np.sum(tnp.ones([2, 3]))\n",
    "print(\"sum = %s. Class: %s\" % (float(np_sum), np_sum.__class__))\n",
    "\n",
    "# `np.ndarray` passed into TensorFlow NumPy function.\n",
    "tnp_sum = tnp.sum(np.ones([2, 3]))\n",
    "print(\"sum = %s. Class: %s\" % (float(tnp_sum), tnp_sum.__class__))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:18.056141Z",
     "iopub.status.busy": "2024-01-11T18:47:18.055641Z",
     "iopub.status.idle": "2024-01-11T18:47:18.206279Z",
     "shell.execute_reply": "2024-01-11T18:47:18.205607Z"
    },
    "id": "ZaLPjzxft780"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAigAAAGdCAYAAAA44ojeAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAhE0lEQVR4nO3dfVCVdf7/8RdyJxrnsKhwOCsQ3Yl4v2ZI+TVbGUBZy6TZbM2sHJ1asEVaUyotq13S3HJzTWdn2twmacuZNMVdylAxJ3QL1zXNSB1LWzxQOXJEE1Gu3x/787jHO8I4XB/w+Zg5M57ruji8z2eYw9Pr3BBkWZYlAAAAg3SyewAAAIBzESgAAMA4BAoAADAOgQIAAIxDoAAAAOMQKAAAwDgECgAAMA6BAgAAjBNi9wCXo6mpSdXV1YqMjFRQUJDd4wAAgB/AsiwdPXpUbrdbnTpd+hxJuwyU6upqxcfH2z0GAAC4DAcPHlTPnj0veUy7DJTIyEhJ/72DDofD5mkAAMAP4fV6FR8f7/s9fintMlDOPK3jcDgIFAAA2pkf8vIMXiQLAACMQ6AAAADjECgAAMA4BAoAADAOgQIAAIxDoAAAAOMQKAAAwDgECgAAMA6BAgAAjEOgAAAA4xAoAADAOAQKAAAwDoECAACMQ6AAAADjhNg9AACc6+pZa+0eocW+fD7b7hGADoUzKAAAwDgECgAAMA6BAgAAjEOgAAAA4xAoAADAOAQKAAAwDoECAACMQ6AAAADjECgAAMA4BAoAADAOgQIAAIxDoAAAAOMQKAAAwDgECgAAMA6BAgAAjEOgAAAA4xAoAADAOAQKAAAwDoECAACM06JAKSoq0pAhQxQZGamYmBiNHTtWVVVVfseMGDFCQUFBfpeHHnrI75gDBw4oOztbXbp0UUxMjGbMmKFTp079+HsDAAA6hJCWHFxeXq7c3FwNGTJEp06d0uOPP66MjAx99tln6tq1q++4KVOm6JlnnvFd79Kli+/fp0+fVnZ2tlwulz766CMdOnRI9913n0JDQ/X73/++Fe4SgP919ay1do8AAC3WokApLS31u75s2TLFxMSosrJSw4cP923v0qWLXC7XBW/j/fff12effaYPPvhAsbGxGjhwoJ599lnNnDlTTz/9tMLCwi7jbgAAgI7kR70Gpa6uTpIUHR3tt3358uXq3r27+vbtq8LCQh0/fty3r6KiQv369VNsbKxvW2Zmprxer3bt2nXB79PQ0CCv1+t3AQAAHVeLzqD8r6amJuXn5+uWW25R3759fdt/9atfKTExUW63Wzt27NDMmTNVVVWld955R5Lk8Xj84kSS77rH47ng9yoqKtLcuXMvd1QAANDOXHag5ObmaufOndq8ebPf9qlTp/r+3a9fP8XFxWnkyJHat2+frr322sv6XoWFhSooKPBd93q9io+Pv7zBAQCA8S7rKZ68vDyVlJRow4YN6tmz5yWPTU1NlSTt3btXkuRyuVRTU+N3zJnrF3vdSnh4uBwOh98FAAB0XC0KFMuylJeXp5UrV2r9+vVKSkpq9mu2b98uSYqLi5MkpaWl6dNPP1Vtba3vmHXr1snhcCglJaUl4wAAgA6qRU/x5Obmqri4WO+++64iIyN9rxlxOp2KiIjQvn37VFxcrNGjR6tbt27asWOHpk+fruHDh6t///6SpIyMDKWkpGjixImaP3++PB6PnnzySeXm5io8PLz17yEAAGh3WnQGZcmSJaqrq9OIESMUFxfnu7z11luSpLCwMH3wwQfKyMhQcnKyHn30UeXk5GjNmjW+2wgODlZJSYmCg4OVlpame++9V/fdd5/f56YAAIArW4vOoFiWdcn98fHxKi8vb/Z2EhMT9fe//70l3xoAAFxB+Fs8AADAOAQKAAAwDoECAACMQ6AAAADjECgAAMA4BAoAADAOgQIAAIxDoAAAAOMQKAAAwDgECgAAMA6BAgAAjEOgAAAA4xAoAADAOAQKAAAwDoECAACMQ6AAAADjECgAAMA4BAoAADAOgQIAAIxDoAAAAOMQKAAAwDgECgAAMA6BAgAAjEOgAAAA4xAoAADAOAQKAAAwDoECAACMQ6AAAADjECgAAMA4BAoAADAOgQIAAIxDoAAAAOMQKAAAwDgECgAAMA6BAgAAjEOgAAAA4xAoAADAOAQKAAAwDoECAACMQ6AAAADjECgAAMA4BAoAADAOgQIAAIxDoAAAAOMQKAAAwDgECgAAMA6BAgAAjEOgAAAA4xAoAADAOAQKAAAwDoECAACMQ6AAAADjECgAAMA4BAoAADBOiwKlqKhIQ4YMUWRkpGJiYjR27FhVVVX5HXPixAnl5uaqW7duuuqqq5STk6Oamhq/Yw4cOKDs7Gx16dJFMTExmjFjhk6dOvXj7w0AAOgQWhQo5eXlys3N1ZYtW7Ru3To1NjYqIyNDx44d8x0zffp0rVmzRitWrFB5ebmqq6s1btw43/7Tp08rOztbJ0+e1EcffaS//vWvWrZsmebMmdN69woAALRrQZZlWZf7xd98841iYmJUXl6u4cOHq66uTj169FBxcbHuuusuSdLnn3+u3r17q6KiQkOHDtU//vEP/eIXv1B1dbViY2MlSUuXLtXMmTP1zTffKCwsrNnv6/V65XQ6VVdXJ4fDcbnjA1eEq2ettXuEK8KXz2fbPQJgvJb8/v5Rr0Gpq6uTJEVHR0uSKisr1djYqPT0dN8xycnJSkhIUEVFhSSpoqJC/fr188WJJGVmZsrr9WrXrl0/ZhwAANBBhFzuFzY1NSk/P1+33HKL+vbtK0nyeDwKCwtTVFSU37GxsbHyeDy+Y/43Ts7sP7PvQhoaGtTQ0OC77vV6L3dsAADQDlz2GZTc3Fzt3LlTf/vb31pzngsqKiqS0+n0XeLj4wP+PQEAgH0uK1Dy8vJUUlKiDRs2qGfPnr7tLpdLJ0+e1JEjR/yOr6mpkcvl8h1z7rt6zlw/c8y5CgsLVVdX57scPHjwcsYGAADtRIsCxbIs5eXlaeXKlVq/fr2SkpL89g8ePFihoaEqKyvzbauqqtKBAweUlpYmSUpLS9Onn36q2tpa3zHr1q2Tw+FQSkrKBb9veHi4HA6H3wUAAHRcLXoNSm5uroqLi/Xuu+8qMjLS95oRp9OpiIgIOZ1OTZ48WQUFBYqOjpbD4dC0adOUlpamoUOHSpIyMjKUkpKiiRMnav78+fJ4PHryySeVm5ur8PDw1r+HAACg3WlRoCxZskSSNGLECL/tr732mu6//35J0ksvvaROnTopJydHDQ0NyszM1CuvvOI7Njg4WCUlJXr44YeVlpamrl27atKkSXrmmWd+3D0BAAAdxo/6HBS78DkowA/H56C0DT4HBWhem30OCgAAQCAQKAAAwDgECgAAMA6BAgAAjEOgAAAA4xAoAADAOAQKAAAwDoECAACMQ6AAAADjECgAAMA4BAoAADAOgQIAAIxDoAAAAOMQKAAAwDgECgAAMA6BAgAAjEOgAAAA4xAoAADAOAQKAAAwDoECAACMQ6AAAADjECgAAMA4BAoAADAOgQIAAIxDoAAAAOMQKAAAwDgECgAAMA6BAgAAjEOgAAAA4xAoAADAOAQKAAAwDoECAACMQ6AAAADjhNg9ANCeXD1rrd0jAMAVgTMoAADAOAQKAAAwDoECAACMQ6AAAADjECgAAMA4BAoAADAOgQIAAIxDoAAAAOMQKAAAwDgECgAAMA6BAgAAjEOgAAAA4xAoAADAOAQKAAAwDoECAACMQ6AAAADjECgAAMA4BAoAADAOgQIAAIxDoAAAAOMQKAAAwDgtDpRNmzZpzJgxcrvdCgoK0qpVq/z233///QoKCvK7ZGVl+R1z+PBhTZgwQQ6HQ1FRUZo8ebLq6+t/1B0BAAAdR4sD5dixYxowYIAWL1580WOysrJ06NAh3+XNN9/02z9hwgTt2rVL69atU0lJiTZt2qSpU6e2fHoAANAhhbT0C0aNGqVRo0Zd8pjw8HC5XK4L7tu9e7dKS0v18ccf68Ybb5QkLVq0SKNHj9aCBQvkdrtbOhIAAOhgAvIalI0bNyomJka9evXSww8/rO+++863r6KiQlFRUb44kaT09HR16tRJW7duveDtNTQ0yOv1+l0AAEDH1eqBkpWVpddff11lZWWaN2+eysvLNWrUKJ0+fVqS5PF4FBMT4/c1ISEhio6OlsfjueBtFhUVyel0+i7x8fGtPTYAADBIi5/iac748eN9/+7Xr5/69++va6+9Vhs3btTIkSMv6zYLCwtVUFDgu+71eokUAAA6sIC/zfiaa65R9+7dtXfvXkmSy+VSbW2t3zGnTp3S4cOHL/q6lfDwcDkcDr8LAADouAIeKF9//bW+++47xcXFSZLS0tJ05MgRVVZW+o5Zv369mpqalJqaGuhxAABAO9Dip3jq6+t9Z0Mkaf/+/dq+fbuio6MVHR2tuXPnKicnRy6XS/v27dNjjz2m6667TpmZmZKk3r17KysrS1OmTNHSpUvV2NiovLw8jR8/nnfwAAAASZdxBuWTTz7RoEGDNGjQIElSQUGBBg0apDlz5ig4OFg7duzQ7bffrhtuuEGTJ0/W4MGD9eGHHyo8PNx3G8uXL1dycrJGjhyp0aNHa9iwYfrzn//cevcKAAC0ay0+gzJixAhZlnXR/e+9916ztxEdHa3i4uKWfmsAAHCF4G/xAAAA4xAoAADAOAQKAAAwDoECAACMQ6AAAADjECgAAMA4BAoAADAOgQIAAIxDoAAAAOMQKAAAwDgECgAAMA6BAgAAjEOgAAAA4xAoAADAOAQKAAAwDoECAACMQ6AAAADjECgAAMA4BAoAADAOgQIAAIxDoAAAAOMQKAAAwDgECgAAMA6BAgAAjEOgAAAA4xAoAADAOAQKAAAwTojdAwBAR3D1rLV2j9BiXz6fbfcIwEVxBgUAABiHQAEAAMYhUAAAgHEIFAAAYBwCBQAAGIdAAQAAxiFQAACAcQgUAABgHAIFAAAYh0ABAADGIVAAAIBxCBQAAGAcAgUAABiHQAEAAMYhUAAAgHEIFAAAYBwCBQAAGIdAAQAAxiFQAACAcQgUAABgHAIFAAAYh0ABAADGIVAAAIBxCBQAAGAcAgUAABiHQAEAAMYhUAAAgHFaHCibNm3SmDFj5Ha7FRQUpFWrVvnttyxLc+bMUVxcnCIiIpSenq49e/b4HXP48GFNmDBBDodDUVFRmjx5surr63/UHQEAAB1HiwPl2LFjGjBggBYvXnzB/fPnz9fLL7+spUuXauvWreratasyMzN14sQJ3zETJkzQrl27tG7dOpWUlGjTpk2aOnXq5d8LAADQoYS09AtGjRqlUaNGXXCfZVlauHChnnzySd1xxx2SpNdff12xsbFatWqVxo8fr927d6u0tFQff/yxbrzxRknSokWLNHr0aC1YsEBut/tH3B0AANARtOprUPbv3y+Px6P09HTfNqfTqdTUVFVUVEiSKioqFBUV5YsTSUpPT1enTp20devWC95uQ0ODvF6v3wUAAHRcrRooHo9HkhQbG+u3PTY21rfP4/EoJibGb39ISIiio6N9x5yrqKhITqfTd4mPj2/NsQEAgGHaxbt4CgsLVVdX57scPHjQ7pEAAEAAtWqguFwuSVJNTY3f9pqaGt8+l8ul2tpav/2nTp3S4cOHfcecKzw8XA6Hw+8CAAA6rlYNlKSkJLlcLpWVlfm2eb1ebd26VWlpaZKktLQ0HTlyRJWVlb5j1q9fr6amJqWmprbmOAAAoJ1q8bt46uvrtXfvXt/1/fv3a/v27YqOjlZCQoLy8/P13HPP6frrr1dSUpJmz54tt9utsWPHSpJ69+6trKwsTZkyRUuXLlVjY6Py8vI0fvx43sEDAAAkXUagfPLJJ7rtttt81wsKCiRJkyZN0rJly/TYY4/p2LFjmjp1qo4cOaJhw4aptLRUnTt39n3N8uXLlZeXp5EjR6pTp07KycnRyy+/3Ap3BwAAdARBlmVZdg/RUl6vV06nU3V1dbweBW3q6llr7R4BaDVfPp9t9wi4wrTk93e7eBcPAAC4shAoAADAOAQKAAAwDoECAACMQ6AAAADjECgAAMA4BAoAADAOgQIAAIxDoAAAAOMQKAAAwDgECgAAMA6BAgAAjEOgAAAA44TYPQCuXPxlYADAxXAGBQAAGIdAAQAAxiFQAACAcQgUAABgHAIFAAAYh0ABAADGIVAAAIBxCBQAAGAcAgUAABiHQAEAAMYhUAAAgHEIFAAAYBwCBQAAGIdAAQAAxiFQAACAcQgUAABgHAIFAAAYh0ABAADGIVAAAIBxCBQAAGAcAgUAABiHQAEAAMYhUAAAgHEIFAAAYBwCBQAAGIdAAQAAxiFQAACAcQgUAABgHAIFAAAYh0ABAADGCbF7AACAPa6etdbuEVrsy+ez7R4BbYQzKAAAwDgECgAAMA6BAgAAjEOgAAAA4xAoAADAOAQKAAAwDoECAACMQ6AAAADjECgAAMA4rR4oTz/9tIKCgvwuycnJvv0nTpxQbm6uunXrpquuuko5OTmqqalp7TEAAEA7FpAzKH369NGhQ4d8l82bN/v2TZ8+XWvWrNGKFStUXl6u6upqjRs3LhBjAACAdiogf4snJCRELpfrvO11dXV69dVXVVxcrJ///OeSpNdee029e/fWli1bNHTo0ECMAwAA2pmAnEHZs2eP3G63rrnmGk2YMEEHDhyQJFVWVqqxsVHp6em+Y5OTk5WQkKCKiopAjAIAANqhVj+DkpqaqmXLlqlXr146dOiQ5s6dq//7v//Tzp075fF4FBYWpqioKL+viY2NlcfjuehtNjQ0qKGhwXfd6/W29tgAAMAgrR4oo0aN8v27f//+Sk1NVWJiot5++21FRERc1m0WFRVp7ty5rTUiAAAwXMDfZhwVFaUbbrhBe/fulcvl0smTJ3XkyBG/Y2pqai74mpUzCgsLVVdX57scPHgwwFMDAAA7BTxQ6uvrtW/fPsXFxWnw4MEKDQ1VWVmZb39VVZUOHDigtLS0i95GeHi4HA6H3wUAAHRcrf4Uz29/+1uNGTNGiYmJqq6u1lNPPaXg4GDdc889cjqdmjx5sgoKChQdHS2Hw6Fp06YpLS2Nd/AAAACfVg+Ur7/+Wvfcc4++++479ejRQ8OGDdOWLVvUo0cPSdJLL72kTp06KScnRw0NDcrMzNQrr7zS2mMAAIB2LMiyLMvuIVrK6/XK6XSqrq6Op3vasatnrbV7BADtzJfPZ9s9An6Elvz+5m/xAAAA4xAoAADAOAQKAAAwDoECAACMQ6AAAADjECgAAMA4BAoAADAOgQIAAIxDoAAAAOMQKAAAwDgECgAAMA6BAgAAjEOgAAAA4xAoAADAOAQKAAAwDoECAACMQ6AAAADjECgAAMA4BAoAADAOgQIAAIxDoAAAAOMQKAAAwDgECgAAMA6BAgAAjEOgAAAA4xAoAADAOAQKAAAwDoECAACMQ6AAAADjECgAAMA4BAoAADAOgQIAAIxDoAAAAOMQKAAAwDgECgAAMA6BAgAAjEOgAAAA4xAoAADAOCF2DwAAwA919ay1do/QYl8+n233CO0SZ1AAAIBxOIPSQbTH/1UAAHAxnEEBAADGIVAAAIBxCBQAAGAcAgUAABiHQAEAAMYhUAAAgHEIFAAAYBwCBQAAGIdAAQAAxiFQAACAcfio+wvgY+MBALAXZ1AAAIBxCBQAAGAcAgUAABjH1tegLF68WC+88II8Ho8GDBigRYsW6aabbrJzJAAAWlV7fV3jl89n2/r9bTuD8tZbb6mgoEBPPfWUtm3bpgEDBigzM1O1tbV2jQQAAAxhW6C8+OKLmjJlih544AGlpKRo6dKl6tKli/7yl7/YNRIAADCELU/xnDx5UpWVlSosLPRt69Spk9LT01VRUXHe8Q0NDWpoaPBdr6urkyR5vd6AzNfUcDwgtwsAQHsRiN+xZ27Tsqxmj7UlUL799ludPn1asbGxfttjY2P1+eefn3d8UVGR5s6de972+Pj4gM0IAMCVzLkwcLd99OhROZ3OSx7TLj6orbCwUAUFBb7rTU1NOnz4sLp166agoCBJ/62y+Ph4HTx4UA6Hw65RjcF6nMVanMVanMVa+GM9zmItzmrttbAsS0ePHpXb7W72WFsCpXv37goODlZNTY3f9pqaGrlcrvOODw8PV3h4uN+2qKioC962w+G44n+g/hfrcRZrcRZrcRZr4Y/1OIu1OKs116K5Mydn2PIi2bCwMA0ePFhlZWW+bU1NTSorK1NaWpodIwEAAIPY9hRPQUGBJk2apBtvvFE33XSTFi5cqGPHjumBBx6wayQAAGAI2wLl7rvv1jfffKM5c+bI4/Fo4MCBKi0tPe+Fsz9UeHi4nnrqqfOeCrpSsR5nsRZnsRZnsRb+WI+zWIuz7FyLIOuHvNcHAACgDfG3eAAAgHEIFAAAYBwCBQAAGIdAAQAAxmn3gXL69GnNnj1bSUlJioiI0LXXXqtnn332B33Of3u3adMmjRkzRm63W0FBQVq1apXffsuyNGfOHMXFxSkiIkLp6enas2ePPcO2gUutR2Njo2bOnKl+/fqpa9eucrvduu+++1RdXW3fwAHU3M/G/3rooYcUFBSkhQsXttl8bemHrMXu3bt1++23y+l0qmvXrhoyZIgOHDjQ9sMGWHNrUV9fr7y8PPXs2VMRERG+P+TaERUVFWnIkCGKjIxUTEyMxo4dq6qqKr9jTpw4odzcXHXr1k1XXXWVcnJyzvuA0Y6iufU4fPiwpk2bpl69eikiIkIJCQl65JFHfH8bLxDafaDMmzdPS5Ys0Z/+9Cft3r1b8+bN0/z587Vo0SK7Rwu4Y8eOacCAAVq8ePEF98+fP18vv/yyli5dqq1bt6pr167KzMzUiRMn2njStnGp9Th+/Li2bdum2bNna9u2bXrnnXdUVVWl22+/3YZJA6+5n40zVq5cqS1btvygj51ur5pbi3379mnYsGFKTk7Wxo0btWPHDs2ePVudO3du40kDr7m1KCgoUGlpqd544w3t3r1b+fn5ysvL0+rVq9t40sArLy9Xbm6utmzZonXr1qmxsVEZGRk6duyY75jp06drzZo1WrFihcrLy1VdXa1x48bZOHXgNLce1dXVqq6u1oIFC7Rz504tW7ZMpaWlmjx5cuCGstq57Oxs68EHH/TbNm7cOGvChAk2TWQPSdbKlSt915uamiyXy2W98MILvm1HjhyxwsPDrTfffNOGCdvWuetxIf/85z8tSdZXX33VNkPZ5GJr8fXXX1s//elPrZ07d1qJiYnWSy+91OaztbULrcXdd99t3XvvvfYMZKMLrUWfPn2sZ555xm/bz372M+uJJ55ow8nsUVtba0myysvLLcv67+NlaGiotWLFCt8xu3fvtiRZFRUVdo3ZZs5djwt5++23rbCwMKuxsTEgM7T7Myg333yzysrK9MUXX0iS/v3vf2vz5s0aNWqUzZPZa//+/fJ4PEpPT/dtczqdSk1NVUVFhY2TmaOurk5BQUEX/btOHVlTU5MmTpyoGTNmqE+fPnaPY5umpiatXbtWN9xwgzIzMxUTE6PU1NRLPiXWkd18881avXq1/vOf/8iyLG3YsEFffPGFMjIy7B4t4M48VREdHS1JqqysVGNjo99jaHJyshISEq6Ix9Bz1+NixzgcDoWEBOYzX9t9oMyaNUvjx49XcnKyQkNDNWjQIOXn52vChAl2j2Yrj8cjSed9Mm9sbKxv35XsxIkTmjlzpu65554r8o+BzZs3TyEhIXrkkUfsHsVWtbW1qq+v1/PPP6+srCy9//77uvPOOzVu3DiVl5fbPV6bW7RokVJSUtSzZ0+FhYUpKytLixcv1vDhw+0eLaCampqUn5+vW265RX379pX038fQsLCw8/4DcyU8hl5oPc717bff6tlnn9XUqVMDNodtH3XfWt5++20tX75cxcXF6tOnj7Zv3678/Hy53W5NmjTJ7vFgoMbGRv3yl7+UZVlasmSJ3eO0ucrKSv3xj3/Utm3bFBQUZPc4tmpqapIk3XHHHZo+fbokaeDAgfroo4+0dOlS3XrrrXaO1+YWLVqkLVu2aPXq1UpMTNSmTZuUm5srt9vtdyaho8nNzdXOnTu1efNmu0cxQnPr4fV6lZ2drZSUFD399NMBm6PdB8qMGTN8Z1EkqV+/fvrqq69UVFR0RQeKy+WSJNXU1CguLs63vaamRgMHDrRpKvudiZOvvvpK69evvyLPnnz44Yeqra1VQkKCb9vp06f16KOPauHChfryyy/tG66Nde/eXSEhIUpJSfHb3rt37yvul9X333+vxx9/XCtXrlR2drYkqX///tq+fbsWLFjQYQMlLy9PJSUl2rRpk3r27Onb7nK5dPLkSR05csTvLEpNTY3v8bUjuth6nHH06FFlZWUpMjJSK1euVGhoaMBmafdP8Rw/flydOvnfjeDgYN//jK5USUlJcrlcKisr823zer3aunWr0tLSbJzMPmfiZM+ePfrggw/UrVs3u0eyxcSJE7Vjxw5t377dd3G73ZoxY4bee+89u8drU2FhYRoyZMh5by/94osvlJiYaNNU9mhsbFRjY+MV83hqWZby8vK0cuVKrV+/XklJSX77Bw8erNDQUL/H0KqqKh04cKBDPoY2tx7Sf3+HZGRkKCwsTKtXrw74O93a/RmUMWPG6He/+50SEhLUp08f/etf/9KLL76oBx980O7RAq6+vl579+71Xd+/f7+2b9+u6OhoJSQkKD8/X88995yuv/56JSUlafbs2XK73Ro7dqx9QwfQpdYjLi5Od911l7Zt26aSkhKdPn3a9zxydHS0wsLC7Bo7IJr72Tg3zkJDQ+VyudSrV6+2HjXgmluLGTNm6O6779bw4cN12223qbS0VGvWrNHGjRvtGzpAmluLW2+9VTNmzFBERIQSExNVXl6u119/XS+++KKNUwdGbm6uiouL9e677yoyMtL3eOB0OhURESGn06nJkyeroKBA0dHRcjgcmjZtmtLS0jR06FCbp299za3HmTg5fvy43njjDXm9Xnm9XklSjx49FBwc3PpDBeS9QW3I6/Vav/nNb6yEhASrc+fO1jXXXGM98cQTVkNDg92jBdyGDRssSeddJk2aZFnWf99qPHv2bCs2NtYKDw+3Ro4caVVVVdk7dABdaj32799/wX2SrA0bNtg9eqtr7mfjXB35bcY/ZC1effVV67rrrrM6d+5sDRgwwFq1apV9AwdQc2tx6NAh6/7777fcbrfVuXNnq1evXtYf/vAHq6mpyd7BA+Bijwevvfaa75jvv//e+vWvf2395Cc/sbp06WLdeeed1qFDh+wbOoCaW4+L/exIsvbv3x+QmYL+/2AAAADGaPevQQEAAB0PgQIAAIxDoAAAAOMQKAAAwDgECgAAMA6BAgAAjEOgAAAA4xAoAADAOAQKAAAwDoECAACMQ6AAAADjECgAAMA4/w9fuHw2HaXqHgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# It is easy to plot ND arrays, given the __array__ interface.\n",
    "labels = 15 + 2 * tnp.random.randn(1, 1000)\n",
    "_ = plt.hist(labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kF-Xyw3XWKqJ"
   },
   "source": [
    "### バッファコピー\n",
    "\n",
    "TensorFlow NumPy を NumPy コードと混在させると、データコピーがトリガーされる場合があります。これは、TensorFlow NumPy のメモリアライメントに関する要件が NumPy の要件よりも厳しいためです。\n",
    "\n",
    "`np.ndarray`が TensorFlow Numpy に渡されると、アライメント要件を確認し、必要に応じてコピーがトリガーされます。ND 配列 CPU バッファを NumPy に渡す場合、通常、バッファはアライメント要件を満たし、NumPy はコピーを作成する必要はありません。\n",
    "\n",
    "ND 配列は、ローカル CPU メモリ以外のデバイスに配置されたバッファを参照できます。このような場合、NumPy 関数を呼び出すと、必要に応じてネットワークまたはデバイス全体でコピーが作成されます。\n",
    "\n",
    "このため、NumPy API 呼び出しとの混合は通常、注意して行い、ユーザーはデータのコピーのオーバーヘッドに注意する必要があります。TensorFlow NumPy 呼び出しを TensorFlow 呼び出しとインターリーブすることは一般的に安全であり、データのコピーを避けられます。 詳細については、[TensorFlow の相互運用性](#tensorflow-interoperability)のセクションをご覧ください。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RwljbqkBc7Ro"
   },
   "source": [
    "### 演算子の優先順位\n",
    "\n",
    "TensorFlow NumPy は、NumPy よりも優先順位の高い`__array_priority__`を定義します。つまり、ND 配列と`np.ndarray`の両方を含む演算子の場合、前者が優先されます。`np.ndarray`入力は ND 配列に変換され、演算子の TensorFlow NumPy 実装が呼び出されます。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:18.209776Z",
     "iopub.status.busy": "2024-01-11T18:47:18.209549Z",
     "iopub.status.idle": "2024-01-11T18:47:18.214905Z",
     "shell.execute_reply": "2024-01-11T18:47:18.214279Z"
    },
    "id": "Cbw8a3G_WUO7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x = tf.Tensor([2. 2.], shape=(2,), dtype=float64)\n",
      "class = <class 'tensorflow.python.framework.ops.EagerTensor'>\n"
     ]
    }
   ],
   "source": [
    "x = tnp.ones([2]) + np.ones([2])\n",
    "print(\"x = %s\\nclass = %s\" % (x, x.__class__))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DNEab_Ctky83"
   },
   "source": [
    "## TF NumPy と TensorFlow\n",
    "\n",
    "TensorFlow NumPy は TensorFlow の上に構築されているため、TensorFlow とシームレスに相互運用できます。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fCcfgrlOnAhQ"
   },
   "source": [
    "### `tf.Tensor` と ND 配列\n",
    "\n",
    "ND 配列は `tf.Tensor` のエイリアスであるため、実際のデータのコピーを呼び出さずに混在させることが可能です。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:18.218363Z",
     "iopub.status.busy": "2024-01-11T18:47:18.217853Z",
     "iopub.status.idle": "2024-01-11T18:47:18.222569Z",
     "shell.execute_reply": "2024-01-11T18:47:18.221989Z"
    },
    "id": "BkHVauKwnky_"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor([1 2], shape=(2,), dtype=int32)\n",
      "tf.Tensor([1 2], shape=(2,), dtype=int32)\n",
      "tf.Tensor([1 2], shape=(2,), dtype=int32)\n",
      "[1 2] <class 'numpy.ndarray'>\n"
     ]
    }
   ],
   "source": [
    "x = tf.constant([1, 2])\n",
    "print(x)\n",
    "\n",
    "# `asarray` and `convert_to_tensor` here are no-ops.\n",
    "tnp_x = tnp.asarray(x)\n",
    "print(tnp_x)\n",
    "print(tf.convert_to_tensor(tnp_x))\n",
    "\n",
    "# Note that tf.Tensor.numpy() will continue to return `np.ndarray`.\n",
    "print(x.numpy(), x.numpy().__class__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_151HQVBooxG"
   },
   "source": [
    "### TensorFlow 相互運用性\n",
    "\n",
    "ND 配列は `tf.Tensor` のエイリアスにすぎないため、TensorFlow API に渡すことができます。前述のように、このような相互運用では、アクセラレータやリモートデバイスに配置されたデータであっても、データのコピーは行われません。\n",
    "\n",
    "逆に言えば、`tf.Tensor` オブジェクトを、データのコピーを実行せずに、`tf.experimental.numpy` API に渡すことができます。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:18.225775Z",
     "iopub.status.busy": "2024-01-11T18:47:18.225373Z",
     "iopub.status.idle": "2024-01-11T18:47:18.234684Z",
     "shell.execute_reply": "2024-01-11T18:47:18.234034Z"
    },
    "id": "-QvxNhrFoz09"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Output = tf.Tensor(6.0, shape=(), dtype=float32)\n",
      "Output = tf.Tensor(6.0, shape=(), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "# ND array passed into TensorFlow function.\n",
    "tf_sum = tf.reduce_sum(tnp.ones([2, 3], tnp.float32))\n",
    "print(\"Output = %s\" % tf_sum)\n",
    "\n",
    "# `tf.Tensor` passed into TensorFlow NumPy function.\n",
    "tnp_sum = tnp.sum(tf.ones([2, 3]))\n",
    "print(\"Output = %s\" % tnp_sum)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1b4HeAkhprF_"
   },
   "source": [
    "### 勾配とヤコビアン: tf.GradientTape\n",
    "\n",
    "TensorFlow の GradientTape は、TensorFlow と TensorFlow NumPy コードを介してバックプロパゲーションに使用できます。\n",
    "\n",
    "[サンプルモデル](#example-model)セクションで作成されたモデルを使用して、勾配とヤコビアンを計算します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:18.237697Z",
     "iopub.status.busy": "2024-01-11T18:47:18.237422Z",
     "iopub.status.idle": "2024-01-11T18:47:18.436745Z",
     "shell.execute_reply": "2024-01-11T18:47:18.436045Z"
    },
    "id": "T47C9KS8pbsP"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Parameter shapes: [TensorShape([32, 64]), TensorShape([64]), TensorShape([64, 2])]\n",
      "Gradient shapes: [TensorShape([32, 64]), TensorShape([64]), TensorShape([64, 2])]\n"
     ]
    }
   ],
   "source": [
    "def create_batch(batch_size=32):\n",
    "  \"\"\"Creates a batch of input and labels.\"\"\"\n",
    "  return (tnp.random.randn(batch_size, 32).astype(tnp.float32),\n",
    "          tnp.random.randn(batch_size, 2).astype(tnp.float32))\n",
    "\n",
    "def compute_gradients(model, inputs, labels):\n",
    "  \"\"\"Computes gradients of squared loss between model prediction and labels.\"\"\"\n",
    "  with tf.GradientTape() as tape:\n",
    "    assert model.weights is not None\n",
    "    # Note that `model.weights` need to be explicitly watched since they\n",
    "    # are not tf.Variables.\n",
    "    tape.watch(model.weights)\n",
    "    # Compute prediction and loss\n",
    "    prediction = model.predict(inputs)\n",
    "    loss = tnp.sum(tnp.square(prediction - labels))\n",
    "  # This call computes the gradient through the computation above.\n",
    "  return tape.gradient(loss, model.weights)\n",
    "\n",
    "inputs, labels = create_batch()\n",
    "gradients = compute_gradients(model, inputs, labels)\n",
    "\n",
    "# Inspect the shapes of returned gradients to verify they match the\n",
    "# parameter shapes.\n",
    "print(\"Parameter shapes:\", [w.shape for w in model.weights])\n",
    "print(\"Gradient shapes:\", [g.shape for g in gradients])\n",
    "# Verify that gradients are of type ND array.\n",
    "assert isinstance(gradients[0], tnp.ndarray)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:18.440246Z",
     "iopub.status.busy": "2024-01-11T18:47:18.439983Z",
     "iopub.status.idle": "2024-01-11T18:47:18.614894Z",
     "shell.execute_reply": "2024-01-11T18:47:18.614236Z"
    },
    "id": "TujVPDFwrdqp"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Output shape: (16, 2), input shape: (16, 32)\n",
      "Batch jacobian shape: (16, 2, 32)\n"
     ]
    }
   ],
   "source": [
    "# Computes a batch of jacobians. Each row is the jacobian of an element in the\n",
    "# batch of outputs w.r.t. the corresponding input batch element.\n",
    "def prediction_batch_jacobian(inputs):\n",
    "  with tf.GradientTape() as tape:\n",
    "    tape.watch(inputs)\n",
    "    prediction = model.predict(inputs)\n",
    "  return prediction, tape.batch_jacobian(prediction, inputs)\n",
    "\n",
    "inp_batch = tnp.ones([16, 32], tnp.float32)\n",
    "output, batch_jacobian = prediction_batch_jacobian(inp_batch)\n",
    "# Note how the batch jacobian shape relates to the input and output shapes.\n",
    "print(\"Output shape: %s, input shape: %s\" % (output.shape, inp_batch.shape))\n",
    "print(\"Batch jacobian shape:\", batch_jacobian.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MYq9wxfc1Dv_"
   },
   "source": [
    "### トレースコンパイル: tf.function\n",
    "\n",
    "Tensorflow の `tf.function` は、コードを「トレースコンパイル」し、これらのトレースを最適化してパフォーマンスを大幅に向上させます。[グラフと関数の概要](./intro_to_graphs.ipynb)を参照してください。\n",
    "\n",
    "また、`tf.function` を使用して、TensorFlow NumPy コードを最適化することもできます。以下は、スピードアップを示す簡単な例です。`tf.function` コードの本文には、TensorFlow NumPy API への呼び出しが含まれていることに注意してください。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:18.618231Z",
     "iopub.status.busy": "2024-01-11T18:47:18.617998Z",
     "iopub.status.idle": "2024-01-11T18:47:18.837391Z",
     "shell.execute_reply": "2024-01-11T18:47:18.836732Z"
    },
    "id": "05SrUulm1OlL"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Eager performance\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.6766205000058108 ms\n",
      "\n",
      "tf.function compiled performance\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7275895000020682 ms\n"
     ]
    }
   ],
   "source": [
    "inputs, labels = create_batch(512)\n",
    "print(\"Eager performance\")\n",
    "compute_gradients(model, inputs, labels)\n",
    "print(timeit.timeit(lambda: compute_gradients(model, inputs, labels),\n",
    "                    number=10) * 100, \"ms\")\n",
    "\n",
    "print(\"\\ntf.function compiled performance\")\n",
    "compiled_compute_gradients = tf.function(compute_gradients)\n",
    "compiled_compute_gradients(model, inputs, labels)  # warmup\n",
    "print(timeit.timeit(lambda: compiled_compute_gradients(model, inputs, labels),\n",
    "                    number=10) * 100, \"ms\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5w8YxR6ELmo1"
   },
   "source": [
    "### ベクトル化：tf.vectorized_map\n",
    "\n",
    "TensorFlow には、並列ループのベクトル化のサポートが組み込まれているため、10 倍から 100 倍のスピードアップが可能です。これらのスピードアップは、`tf.vectorized_map` API を介して実行でき、TensorFlow NumPy にも適用されます。\n",
    "\n",
    "w.r.t. （対応する入力バッチ要素）バッチで各出力の勾配を計算すると便利な場合があります。このような計算は、以下に示すように `tf.vectorized_map` を使用して効率的に実行できます。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:18.841233Z",
     "iopub.status.busy": "2024-01-11T18:47:18.840594Z",
     "iopub.status.idle": "2024-01-11T18:47:19.025163Z",
     "shell.execute_reply": "2024-01-11T18:47:19.024487Z"
    },
    "id": "PemSIrs5L-VJ"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Weight shape: (32, 64), batch size: 128, per example gradient shape: (128, 32, 64) \n",
      "Weight shape: (64,), batch size: 128, per example gradient shape: (128, 64) \n",
      "Weight shape: (64, 2), batch size: 128, per example gradient shape: (128, 64, 2) \n"
     ]
    }
   ],
   "source": [
    "@tf.function\n",
    "def vectorized_per_example_gradients(inputs, labels):\n",
    "  def single_example_gradient(arg):\n",
    "    inp, label = arg\n",
    "    return compute_gradients(model,\n",
    "                             tnp.expand_dims(inp, 0),\n",
    "                             tnp.expand_dims(label, 0))\n",
    "  # Note that a call to `tf.vectorized_map` semantically maps\n",
    "  # `single_example_gradient` over each row of `inputs` and `labels`.\n",
    "  # The interface is similar to `tf.map_fn`.\n",
    "  # The underlying machinery vectorizes away this map loop which gives\n",
    "  # nice speedups.\n",
    "  return tf.vectorized_map(single_example_gradient, (inputs, labels))\n",
    "\n",
    "batch_size = 128\n",
    "inputs, labels = create_batch(batch_size)\n",
    "\n",
    "per_example_gradients = vectorized_per_example_gradients(inputs, labels)\n",
    "for w, p in zip(model.weights, per_example_gradients):\n",
    "  print(\"Weight shape: %s, batch size: %s, per example gradient shape: %s \" % (\n",
    "      w.shape, batch_size, p.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:19.028531Z",
     "iopub.status.busy": "2024-01-11T18:47:19.028275Z",
     "iopub.status.idle": "2024-01-11T18:47:19.836980Z",
     "shell.execute_reply": "2024-01-11T18:47:19.836277Z"
    },
    "id": "_QZ5BjJmRAlG"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running vectorized computation\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7061765999878844 ms\n",
      "\n",
      "Running unvectorized computation\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "28.316628299990043 ms\n"
     ]
    }
   ],
   "source": [
    "# Benchmark the vectorized computation above and compare with\n",
    "# unvectorized sequential computation using `tf.map_fn`.\n",
    "@tf.function\n",
    "def unvectorized_per_example_gradients(inputs, labels):\n",
    "  def single_example_gradient(arg):\n",
    "    inp, label = arg\n",
    "    return compute_gradients(model,\n",
    "                             tnp.expand_dims(inp, 0),\n",
    "                             tnp.expand_dims(label, 0))\n",
    "\n",
    "  return tf.map_fn(single_example_gradient, (inputs, labels),\n",
    "                   fn_output_signature=(tf.float32, tf.float32, tf.float32))\n",
    "\n",
    "print(\"Running vectorized computation\")\n",
    "print(timeit.timeit(lambda: vectorized_per_example_gradients(inputs, labels),\n",
    "                    number=10) * 100, \"ms\")\n",
    "\n",
    "print(\"\\nRunning unvectorized computation\")\n",
    "per_example_gradients = unvectorized_per_example_gradients(inputs, labels)\n",
    "print(timeit.timeit(lambda: unvectorized_per_example_gradients(inputs, labels),\n",
    "                    number=10) * 100, \"ms\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UOTh-nkzaJd9"
   },
   "source": [
    "### デバイスに配置する\n",
    "\n",
    "TensorFlow NumPy は、CPU、GPU、TPU、およびリモートデバイスに演算を配置できます。デバイスにおける配置には標準の TensorFlow メカニズムを使用します。以下の簡単な例は、すべてのデバイスを一覧表示してから、特定のデバイスに計算を配置する方法を示しています。\n",
    "\n",
    "ここでは取り上げませんが、TensorFlow には、デバイス間で計算を複製し、集合的な削減を実行するための API もあります。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-0gHrwYYaTCE"
   },
   "source": [
    "#### デバイスをリストする\n",
    "\n",
    "使用するデバイスを見つけるには、`tf.config.list_logical_devices` および`tf.config.list_physical_devices` を使用します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:19.840818Z",
     "iopub.status.busy": "2024-01-11T18:47:19.840543Z",
     "iopub.status.idle": "2024-01-11T18:47:19.845064Z",
     "shell.execute_reply": "2024-01-11T18:47:19.844445Z"
    },
    "id": "NDEAd9m9aemS"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "All logical devices: [LogicalDevice(name='/device:CPU:0', device_type='CPU'), LogicalDevice(name='/device:GPU:0', device_type='GPU'), LogicalDevice(name='/device:GPU:1', device_type='GPU'), LogicalDevice(name='/device:GPU:2', device_type='GPU'), LogicalDevice(name='/device:GPU:3', device_type='GPU')]\n",
      "All physical devices: [PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU'), PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU'), PhysicalDevice(name='/physical_device:GPU:1', device_type='GPU'), PhysicalDevice(name='/physical_device:GPU:2', device_type='GPU'), PhysicalDevice(name='/physical_device:GPU:3', device_type='GPU')]\n"
     ]
    }
   ],
   "source": [
    "print(\"All logical devices:\", tf.config.list_logical_devices())\n",
    "print(\"All physical devices:\", tf.config.list_physical_devices())\n",
    "\n",
    "# Try to get the GPU device. If unavailable, fallback to CPU.\n",
    "try:\n",
    "  device = tf.config.list_logical_devices(device_type=\"GPU\")[0]\n",
    "except IndexError:\n",
    "  device = \"/device:CPU:0\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fihgfF_tahVx"
   },
   "source": [
    "#### 演算の配置：**`tf.device`**\n",
    "\n",
    "デバイスに演算を配置するには、`tf.device` スコープでデバイスを呼び出します。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:19.848218Z",
     "iopub.status.busy": "2024-01-11T18:47:19.847950Z",
     "iopub.status.idle": "2024-01-11T18:47:19.855490Z",
     "shell.execute_reply": "2024-01-11T18:47:19.854863Z"
    },
    "id": "c7ELvLmnazfV"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using device: LogicalDevice(name='/device:GPU:0', device_type='GPU')\n",
      "prediction is placed on /job:localhost/replica:0/task:0/device:GPU:0\n"
     ]
    }
   ],
   "source": [
    "print(\"Using device: %s\" % str(device))\n",
    "# Run operations in the `tf.device` scope.\n",
    "# If a GPU is available, these operations execute on the GPU and outputs are\n",
    "# placed on the GPU memory.\n",
    "with tf.device(device):\n",
    "  prediction = model.predict(create_batch(5)[0])\n",
    "\n",
    "print(\"prediction is placed on %s\" % prediction.device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "e-LK6wsHbBiM"
   },
   "source": [
    "#### デバイス間での ND 配列のコピー: **`tnp.copy`**\n",
    "\n",
    "特定のデバイススコープで `tnp.copy` を呼び出すと、データがそのデバイスに既に存在しない限り、そのデバイスにデータがコピーされます。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:19.858886Z",
     "iopub.status.busy": "2024-01-11T18:47:19.858321Z",
     "iopub.status.idle": "2024-01-11T18:47:19.863449Z",
     "shell.execute_reply": "2024-01-11T18:47:19.862845Z"
    },
    "id": "CCesyidaa-UT"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/job:localhost/replica:0/task:0/device:GPU:0\n",
      "/job:localhost/replica:0/task:0/device:CPU:0\n"
     ]
    }
   ],
   "source": [
    "with tf.device(\"/device:CPU:0\"):\n",
    "  prediction_cpu = tnp.copy(prediction)\n",
    "print(prediction.device)\n",
    "print(prediction_cpu.device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AiYzRDOtKzAH"
   },
   "source": [
    "## パフォーマンスの比較\n",
    "\n",
    "TensorFlow NumPy は、CPU、GPU、TPU にディスパッチできる高度に最適化された TensorFlow カーネルを使用します。TensorFlow は、演算の融合など、多くのコンパイラ最適化も実行し、パフォーマンスとメモリを向上します。詳細については、[Grappler を使用した TensorFlow グラフの最適化](./graph_optimization.ipynb)をご覧ください。\n",
    "\n",
    "ただし、TensorFlow では、NumPy と比較してディスパッチ演算のオーバーヘッドが高くなります。小規模な演算（約 10 マイクロ秒未満）で構成されるワークロードの場合、これらのオーバーヘッドがランタイムを支配する可能性があり、NumPy はより優れたパフォーマンスを提供する可能性があります。その他の場合、一般的に TensorFlow を使用するとパフォーマンスが向上するはずです。\n",
    "\n",
    "以下のベンチマークを実行して、さまざまな入力サイズでの NumPy と TensorFlow Numpy のパフォーマンスを比較します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "cellView": "code",
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:19.866853Z",
     "iopub.status.busy": "2024-01-11T18:47:19.866324Z",
     "iopub.status.idle": "2024-01-11T18:47:19.872151Z",
     "shell.execute_reply": "2024-01-11T18:47:19.871599Z"
    },
    "id": "RExwjI9_pJG0"
   },
   "outputs": [],
   "source": [
    "def benchmark(f, inputs, number=30, force_gpu_sync=False):\n",
    "  \"\"\"Utility to benchmark `f` on each value in `inputs`.\"\"\"\n",
    "  times = []\n",
    "  for inp in inputs:\n",
    "    def _g():\n",
    "      if force_gpu_sync:\n",
    "        one = tnp.asarray(1)\n",
    "      f(inp)\n",
    "      if force_gpu_sync:\n",
    "        with tf.device(\"CPU:0\"):\n",
    "          tnp.copy(one)  # Force a sync for GPU case\n",
    "\n",
    "    _g()  # warmup\n",
    "    t = timeit.timeit(_g, number=number)\n",
    "    times.append(t * 1000. / number)\n",
    "  return times\n",
    "\n",
    "\n",
    "def plot(np_times, tnp_times, compiled_tnp_times, has_gpu, tnp_times_gpu):\n",
    "  \"\"\"Plot the different runtimes.\"\"\"\n",
    "  plt.xlabel(\"size\")\n",
    "  plt.ylabel(\"time (ms)\")\n",
    "  plt.title(\"Sigmoid benchmark: TF NumPy vs NumPy\")\n",
    "  plt.plot(sizes, np_times, label=\"NumPy\")\n",
    "  plt.plot(sizes, tnp_times, label=\"TF NumPy (CPU)\")\n",
    "  plt.plot(sizes, compiled_tnp_times, label=\"Compiled TF NumPy (CPU)\")\n",
    "  if has_gpu:\n",
    "    plt.plot(sizes, tnp_times_gpu, label=\"TF NumPy (GPU)\")\n",
    "  plt.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-01-11T18:47:19.875410Z",
     "iopub.status.busy": "2024-01-11T18:47:19.874835Z",
     "iopub.status.idle": "2024-01-11T18:47:21.100278Z",
     "shell.execute_reply": "2024-01-11T18:47:21.099576Z"
    },
    "id": "p-fs_H1lkLfV"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAHHCAYAAABDUnkqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8g+/7EAAAACXBIWXMAAA9hAAAPYQGoP6dpAACTN0lEQVR4nOzdd3xT1f/H8Vea7r1bKC0FCmXTlr0sIENAFJWNAiI4GDKcIIiAMgSUjX5VQH6yBAUHS2Qoe7Vl7z26gO7d5P7+qA2kgw7apuPzfDzy0Nyce/NJSpN37znnHpWiKApCCCGEEOWEkaELEEIIIYQoShJuhBBCCFGuSLgRQgghRLki4UYIIYQQ5YqEGyGEEEKUKxJuhBBCCFGuSLgRQgghRLki4UYIIYQQ5YqEGyGEEEKUKxJuRJHw9vZmyJAhhi7jiVauXIlKpeLGjRt5ts3P67lx4wYqlYq5c+cWTYHFoF27dtSvX9/QZeSbSqVi1KhRhi5DCFHGSbgRT3T69Gl69epF1apVMTc3x8PDg06dOrFo0SJDlybEE7Vr1w6VSpXn7bPPPgMyAm1ubZKTk3N9nsyQq1Kp+OWXX7I9/tlnn6FSqbh//35xvdQnyvq6XF1dadu2LZs2bTJIPYWR+bPs0aNHtscM/UdG5s8382ZpaUndunWZNGkSsbGxBqlJgLGhCxCl18GDB2nfvj1eXl4MHz4cd3d3bt++zeHDh1mwYAGjR4/Wtb148SJGRqU7K7/22mv069cPMzMzQ5ciSsAnn3zCsGHDdPePHTvGwoULmThxInXq1NFtb9iwoe7//fz8eO+997Idy9TUNF/POW3aNF5++WVUKtVTVF70Hn9d9+7d49tvv+Xll19m2bJlvP322wauLv/+/PNPTpw4QePGjQ1dSjbLli3D2tqa+Ph4/vrrL7744gt2797NgQMHSt2/h4pAwo3I1RdffIGdnR3Hjh3D3t5e77GIiAi9+2UhMKjVatRqtaHLEFkoikJycjIWFhZFetxOnTrp3Tc3N2fhwoV06tSJdu3a5biPh4cHr776aqGez8/Pj5CQEDZt2sTLL79cqGMUl6yva9CgQfj4+PD111+XmXDj5eVFXFwcU6dO5ffffzd0Odn06tULZ2dnAN5++21eeeUVfv31Vw4fPkzLli0NXF3FU7r/1BYGdfXqVerVq5ct2AC4urrq3c9pjMqpU6cIDAzEwsKCKlWq8Pnnn7NixYps4168vb15/vnn2bt3L02aNMHCwoIGDRqwd+9eAH799VcaNGiAubk5jRs3Jjg4OFs9u3fvpm3btlhZWWFvb8+LL77I+fPn9drkNOZGURQ+//xzqlSpgqWlJe3bt+fs2bMFep8Avv76a6pWrYqFhQWBgYGcOXMmW5sLFy7Qq1cvHB0dMTc3p0mTJtk+pDNrPHDgAOPHj8fFxQUrKyteeuklIiMjsx1z27ZtBAYGYmNjg62tLU2bNmXNmjXZ2p07d4727dtjaWmJh4cHX375pd7je/fuRaVS8fPPPzN16lQ8PDywsbGhV69exMTEkJKSwtixY3F1dcXa2prXX3+dlJQUvWOsWLGCDh064OrqipmZGXXr1mXZsmXZasn8ee/YsUP38/72229zfW8///xzjIyM9LpCb926xYULF3LdxxD69etHrVq1mDZtGoqiPLFtbmO62rVrpxe8iuLnkhN3d3fq1KnD9evXARg8eDDOzs6kpaVla9u5c2d8fX1zPdaoUaOwtrYmMTEx22P9+/fH3d0djUYDwPHjx+nSpQvOzs5YWFhQrVo1hg4dmme9ADY2NowbN44//viDoKCgJ7bN7CrKKqfPgKL4/MlJhw4dALh+/Tp79uxBpVLl2BW4Zs0aVCoVhw4dytdxRf5IuBG5qlq1KidOnMjxizovd+/e1QWFCRMmMG7cOFavXs2CBQtybH/lyhUGDBhAjx49mDlzJlFRUfTo0YPVq1czbtw4Xn31VaZOncrVq1fp06cPWq1Wt+/ff/9Nly5diIiI4LPPPmP8+PEcPHiQ1q1b5zl4+NNPP2Xy5Mk0atSIOXPmUL16dTp37kxCQkK+X+uqVatYuHAhI0eOZMKECZw5c4YOHToQHh6ua3P27FlatGjB+fPn+fjjj5k3bx5WVlb07Nkzxw+80aNHc/LkSaZMmcI777zDH3/8kW2g7cqVK+nevTsPHz5kwoQJzJo1Cz8/P7Zv367XLioqiueee45GjRoxb948ateuzUcffcS2bduyPe/MmTPZsWMHH3/8MUOHDuXXX3/l7bffZujQoVy6dInPPvuMl19+mZUrVzJ79my9fZctW0bVqlWZOHEi8+bNw9PTkxEjRrBkyZJsz3Px4kX69+9Pp06dWLBgAX5+fjm+t5MmTeLTTz/l22+/1esGHTRokF7XUlFJS0vj/v37erecvrRzolarmTRpEidPnizy8SxP83PJSVpaGrdv38bJyQnI6LJ98OABO3bs0GsXFhbG7t27n3g2q2/fviQkJLBlyxa97YmJifzxxx/06tULtVpNREQEnTt35saNG3z88ccsWrSIgQMHcvjw4Xy/D2PGjMHBwUE3TqqoPM3nT26uXr0KgJOTE+3atcPT05PVq1dna7d69Wpq1KghZ3eKmiJELv766y9FrVYrarVaadmypfLhhx8qO3bsUFJTU7O1rVq1qjJ48GDd/dGjRysqlUoJDg7WbXvw4IHi6OioAMr169f19gWUgwcP6rbt2LFDARQLCwvl5s2buu3ffvutAih79uzRbfPz81NcXV2VBw8e6LadPHlSMTIyUgYNGqTbtmLFCr3njoiIUExNTZXu3bsrWq1W127ixIkKoPd6cnL9+nVdjXfu3NFtP3LkiAIo48aN02179tlnlQYNGijJycm6bVqtVmnVqpVSs2bNbDV27NhRr6Zx48YparVaiY6OVhRFUaKjoxUbGxulefPmSlJSkl5dj+8XGBioAMqqVat021JSUhR3d3fllVde0W3bs2ePAij169fX+/n2799fUalUSteuXfWeo2XLlkrVqlX1tiUmJmZ7j7p06aJUr15db1vmz3v79u3Z2gPKyJEjFUVRlPfee08xMjJSVq5cma1d5usqiA0bNmT7t5NTXVlvU6ZMeeJxM/8dzJkzR0lPT1dq1qypNGrUSPdzmDJligIokZGRes+V07+vwMBAJTAwUHe/KH4uVatWVTp37qxERkYqkZGRysmTJ5V+/fopgDJ69GhFURRFo9EoVapUUfr27au371dffaWoVCrl2rVrub5+rVareHh46P17UhRF+fnnnxVA+ffffxVFUZRNmzYpgHLs2LFcj5WbwMBApV69eoqiKMrUqVMVQDlx4oSiKPrvf6bM9zyrrJ8BivL0nz+Zz3Xx4kUlMjJSuX79uvLtt98qZmZmipubm5KQkKAoiqJMmDBBMTMz0/0OK0rGZ5CxsXGe/8ZEwcmZG5GrTp06cejQIV544QVOnjzJl19+SZcuXfDw8Mizz3v79u20bNlS7y9yR0dHBg4cmGP7unXr6v3l0rx5cyDj1K6Xl1e27deuXQMgNDSUkJAQhgwZgqOjo65dw4YN6dSpE1u3bs21xr///pvU1FRGjx6tdwp77NixT3xtWfXs2RMPDw/d/WbNmtG8eXPdcz98+JDdu3fTp08f4uLidGcEHjx4QJcuXbh8+TJ3797VO+abb76pV1Pbtm3RaDTcvHkTgJ07dxIXF8fHH3+Mubm53r5ZT8dbW1vr/eVtampKs2bNdO/h4wYNGoSJiYnufvPmzVEUJVvXQfPmzbl9+zbp6em6bY+PmYmJieH+/fsEBgZy7do1YmJi9PavVq0aXbp0yfb8kNFVOGrUKBYsWMBPP/3E4MGDs7XZu3dvnl0/hdG8eXN27typdxs0aFC+93/87M3mzZuLrK6n+bkA/PXXX7i4uODi4kKjRo3YsGEDr732mu4sj5GREQMHDuT3338nLi5Ot9/q1atp1aoV1apVy7U2lUpF79692bp1K/Hx8brt69evx8PDgzZt2gDourf//PPPHLu/8ivz7M3UqVMLfYysCvv58zhfX19cXFyoVq0ab731Fj4+PmzZsgVLS0sg42eYkpLCxo0bdfusX7+e9PT0Qo/zErmTcCOeqGnTpvz6669ERUVx9OhRJkyYQFxcHL169eLcuXO57nfz5k18fHyybc9pG6D3AQJgZ2cHgKenZ47bo6KidM8D5DgmoE6dOty/fz/XLqbMfWvWrKm33cXFBQcHhxz3yUnW/QFq1aql6xK7cuUKiqIwefJk3RdM5m3KlClA9gHaWd+PzHoyX3fmKe/8XMOmSpUq2QKPg4OD7lhPet4n/Ry0Wq1eaDlw4AAdO3bUjXtycXFh4sSJADmGm9ysWrWKJUuWsGjRIvr375/n6ytKzs7OdOzYUe9WvXr1Ah1j4MCB+Pj45GvsTX49zc8FHoW2v//+m4MHD3L//n1WrVqlF0gHDRpEUlKSrkvt4sWLnDhxgtdeey3P+vr27UtSUpLuj574+Hi2bt1K7969df/2AgMDeeWVV5g6dSrOzs68+OKLrFixIl9jhLK+xrFjx/L777/ne/xLXgr7+fO4X375hZ07d7J3716uXLnCmTNn9GZ11a5dm6ZNm+p1Ta1evZoWLVrk+rkoCk/CjcgXU1NTmjZtyowZM1i2bBlpaWls2LChyI6f2yym3LYXx1/txSWzf/7999/PdlYg85b1w60oX3dBjlXYn8PVq1d59tlnuX//Pl999RVbtmxh586djBs3DiDbGIUnzYxq3bo1bm5uLF68mIcPH+barrTKPHsTEhLCb7/9lmOb3KYGZw68zemYBdme9WebGdqeffZZWrZsmeMkgbp169K4cWN++uknAH766SdMTU3p06dPjs/xuBYtWuDt7c3PP/8MwB9//EFSUhJ9+/bVtVGpVGzcuJFDhw4xatQo7t69y9ChQ2ncuLHeGZ/8GDNmDPb29rmevSnp9xfgmWeeoWPHjgQGBlKjRo0c9xs0aBD//PMPd+7c4erVqxw+fFjO2hQTCTeiwJo0aQJkdAnlpmrVqly5ciXb9py2PY2qVasCGX9lZnXhwgWcnZ2xsrJ64r6XL1/W2x4ZGZnjX2a5ybo/wKVLl/D29gbQ/eVvYmKS7axA5s3GxibfzwfoPjwLM9i7OPzxxx+kpKTw+++/89Zbb9GtWzc6duxYqOndPj4+/PXXX9y7d4/nnntOr5ukrHj11Vfx8fFh6tSpOX4ROjg4EB0dnW175tlEQxk0aBC7d+8mNDSUNWvW0L1793yfxezTpw/bt28nNjaW9evX4+3tTYsWLbK1a9GiBV988QXHjx9n9erVnD17lnXr1hWozsyzN7/99luOZ28ya876Hhv6/e3Xrx9qtZq1a9eyevVqTExM9AKgKDoSbkSu9uzZk+MHc+ZYkidND+3SpQuHDh0iJCREt+3hw4c5zhZ4GpUqVcLPz48ff/xR74PszJkz/PXXX3Tr1i3XfTt27IiJiQmLFi3Se53z588vUA2bN2/WGzNz9OhRjhw5QteuXYGMafPt2rXj22+/zTEQ5jTFOy+dO3fGxsaGmTNnZrt6riHOamX+hfv4c8fExLBixYpCHa9hw4Zs3bqV8+fP06NHD5KSkvQeL41TwR/3+NmbnMan1ahRg8OHD5Oamqrb9ueff3L79u2SLDOb/v37o1KpGDNmDNeuXSvQWYW+ffuSkpLCjz/+yPbt27Od8YmKisr2bzNzTF5Bu6YgY2ycvb0906ZNy/ZYZvj/999/ddsSEhL48ccfC/w8RcnZ2ZmuXbvy008/sXr1ap577jndtXFE0ZKL+IlcjR49msTERF566SVq165NamoqBw8e1P1V9vrrr+e674cffshPP/1Ep06dGD16NFZWVnz//fd4eXnx8OHDIr1i55w5c+jatSstW7bkjTfeICkpiUWLFmFnZ/fEKaMuLi68//77zJw5k+eff55u3boRHBzMtm3bCvSB4+PjQ5s2bXjnnXdISUlh/vz5ODk58eGHH+raLFmyhDZt2tCgQQOGDx9O9erVCQ8P59ChQ9y5c4eTJ08W6DXb2try9ddfM2zYMJo2bcqAAQNwcHDg5MmTJCYmlviHeOfOnTE1NaVHjx689dZbxMfH89133+Hq6vrEM3xP0qJFC3777Te6detGr1692Lx5s25Qbebp/dLcPTlw4ECmT5+uF/AzDRs2jI0bN/Lcc8/Rp08frl69yk8//ZRrd0ZJcXFx4bnnnmPDhg3Y29vTvXv3fO8bEBCAj48Pn3zyCSkpKdnOSPz4448sXbqUl156iRo1ahAXF8d3332Hra3tE/8IyY2dnR1jxozJsWuqc+fOeHl58cYbb/DBBx+gVqtZvnw5Li4u3Lp1q8DPVZQGDRpEr169AJg+fbpBaynP5MyNyNXcuXNp3749W7duZfz48YwfP56jR48yYsQIjhw5kmO/fSZPT0/27NlDnTp1mDFjBvPnz2fw4MG62R1ZZ/g8jY4dO7J9+3acnJz49NNPmTt3Li1atODAgQNPHLgKGReImzp1KsHBwXzwwQdcvXqVv/76K9eurJwMGjSI0aNHs3jxYr744gvq1avH7t27qVSpkq5N3bp1OX78ON27d2flypWMHDmSb775BiMjIz799NNCve433niD33//HVtbW6ZPn85HH31EUFCQ7oxRSfL19WXjxo2oVCref/99vvnmG958803GjBnzVMft0KEDP//8M3/99RevvfZavq4vUloYGxszadKkHB/r0qUL8+bN49KlS4wdO5ZDhw7x559/UqVKlRKuMrvM2WF9+vQp8JXH+/btS1xcHD4+PgQEBOg9FhgYSJMmTVi3bh3vvvsuX375JTVr1mT37t15/p7mZuzYsbpBvo8zMTFh06ZN1KhRg8mTJ7Nw4UKGDRtWKhZl7dGjBw4ODtjZ2fHCCy8YupxyS6WU5j99RLkzduxYvv32W+Lj42UpBCFKod9++42ePXvy77//0rZtW0OXU+6kp6dTuXJlevTowQ8//GDocsotOXMjik3WcRIPHjzg//7v/2jTpo0EGyFKqe+++47q1avrrk8jitbmzZuJjIws0PWTRMHJmBtRbFq2bEm7du2oU6cO4eHh/PDDD8TGxjJ58mRDlyaEyGLdunWcOnWKLVu2sGDBAlnJuogdOXKEU6dOMX36dPz9/QkMDDR0SeWadEuJYjNx4kQ2btzInTt3UKlUBAQEMGXKFDp27Gjo0oQQWahUKqytrenbty/ffPMNxsbyt29RGjJkCD/99BN+fn6sXLkyXxfgFIUn4UYIIYQQ5YqMuRFCCCFEuVJqws2sWbNQqVR5Llq4YcMGateujbm5OQ0aNHjiwohCCCGEqHhKRafqsWPH+Pbbb2nYsOET2x08eJD+/fvrLrq2Zs0aevbsSVBQUL77L7VaLffu3cPGxkYGzAkhhBBlhKIoxMXFUblyZYyM8jg3oxhYXFycUrNmTWXnzp1KYGCgMmbMmFzb9unTR+nevbvetubNmytvvfVWvp/v9u3bCiA3uclNbnKTm9zK4O327dt5ftcb/MzNyJEj6d69Ox07duTzzz9/YttDhw4xfvx4vW1dunRh8+bNue6TkpKit26J8t/46du3b2Nra1v4woUQQghRYmJjY/H09MzXQsMGDTfr1q0jKCiIY8eO5at9WFgYbm5uetvc3NwICwvLdZ+ZM2fmuPaIra2thBshhBCijMnPkBKDDSi+ffs2Y8aMYfXq1UW6zlBWEyZMICYmRncz9Kq7QgghhCheBjtzc+LECSIiIvQWV9NoNPz7778sXryYlJSUbJfod3d3Jzw8XG9beHg47u7uuT6PmZlZgRd/E0IIIUTZZbAzN88++yynT58mJCREd2vSpAkDBw4kJCQkx7WHWrZsya5du/S27dy5k5YtW5ZU2UIIIYQo5Qx25sbGxibb9G0rKyucnJx02wcNGoSHhwczZ84EYMyYMQQGBjJv3jy6d+/OunXrOH78OP/73/+KvD6NRkNaWlqRH1cIUXgmJiay6KoQIk8Gny31JLdu3dKby96qVSvWrFnDpEmTmDhxIjVr1mTz5s1FukaHoiiEhYURHR1dZMcUQhQde3t73N3d5TpVQohcVbi1pWJjY7GzsyMmJibH2VKhoaFER0fj6uqKpaWlfIAKUUooikJiYiIRERHY29tTqVIlQ5ckhChBeX1/P65Un7kpaRqNRhdsnJycDF2OECILCwsLACIiInB1dZUuKiFEjkrN2lKlQeYYG0tLSwNXIoTITebvp4yJE0LkRsJNDqQrSojSS34/hRB5kXAjhBBCiHJFwo0QQgghyhUJN+XEkCFDUKlUzJo1S2/75s2bS+Q0vkql0t3s7Oxo3bo1u3fvLvbnFUIIIbKScFOOmJubM3v2bKKiogzy/CtWrCA0NJQDBw7g7OzM888/z7Vr1wxSixBCCMM4cTOKhwmpBq1Bwk050rFjR9zd3XVXdM7qs88+w8/PT2/b/Pnz8fb21t0fMmQIPXv2ZMaMGbi5uWFvb8+0adNIT0/ngw8+wNHRkSpVqrBixYpsx8+8uFr9+vVZtmwZSUlJ7Ny5k1WrVuHk5ERKSope+549e/Laa6899esWQghheIqi8L9/r9Ln20OM/zkErdZwl9GTcJMHRVFITE03yK2g11dUq9XMmDGDRYsWcefOnUK/5t27d3Pv3j3+/fdfvvrqK6ZMmcLzzz+Pg4MDR44c4e233+att9564nNkXo8kNTWV3r17o9Fo+P3333WPR0REsGXLFoYOHVroOoUQQpQO0YmpDF91nBlbL6DRKtiam5Cq0RqsHrmIXx6S0jTU/XSHQZ773LQuWJoW7Ef00ksv4efnx5QpU/jhhx8K9byOjo4sXLgQIyMjfH19+fLLL0lMTGTixIkATJgwgVmzZrF//3769euXbf/ExEQmTZqEWq0mMDAQCwsLBgwYwIoVK+jduzcAP/30E15eXrRr165QNQohhCgdQm5HM3J1EHejkzBVGzHlhboMaOZl0Ms2SLgph2bPnk2HDh14//33C7V/vXr19Nb0cnNz01u/S61W4+TkREREhN5+/fv3R61Wk5SUhIuLCz/88AMNGzYEYPjw4TRt2pS7d+/i4eHBypUrdYOghRBClD2KovDjwRt8sfU8aRoFL0dLlg4MoL6HnaFLk3CTFwsTNeemdTHYcxfGM888Q5cuXZgwYQJDhgzRbTcyMsrW1ZXTVV5NTEz07qtUqhy3abX6pxy//vprOnbsiJ2dHS4uLnqP+fv706hRI1atWkXnzp05e/YsW7ZsKczLE0IIYWCxyWl8/Msptp4OA6BrfXdm92qIrblJHnuWDAk3eVCpVAXuGioNZs2ahZ+fH76+vrptLi4uhIWFoSiK7oxJSEhIkT2nu7s7Pj4+uT4+bNgw5s+fz927d+nYsSOenp5F9txCCCFKxtl7MYxcHcSNB4mYqFVM7FaHIa28S9WZeBlQXE41aNCAgQMHsnDhQt22du3aERkZyZdffsnVq1dZsmQJ27ZtK7GaBgwYwJ07d/juu+9kILEQQpQxiqKw9ugtXlp6kBsPEvGwt+Dnt1ryeutqpSrYgISbcm3atGl6XUd16tRh6dKlLFmyhEaNGnH06NFCj8spDDs7O1555RWsra3p2bNniT2vEEKIp5OQks649SFM+PU0qelanq3typZ32+Dv5WDo0nKkUgo637iMi42Nxc7OjpiYGGxtbfUeS05O5vr161SrVg1zc3MDVVi+Pfvss9SrV0/vjJIQBSG/p0KUrEvhcbzz0wmuRiagNlLxQRdf3mxbHSOjkj1b86Tv76zK3mASUSZFRUWxd+9e9u7dy9KlSw1djhBCiHzYeOIOkzafJjlNi5utGYsHBNDU29HQZeVJwo0oEf7+/kRFRTF79my9Qc5CCCFKn6RUDVN+P8PPxzMu1tq2pjNf9/XD2drMwJXlj4QbUSJu3Lhh6BKEEELkw9XIeEauDuJCWBxGKhjbsRYj2/ugLuFuqKch4UYIIYQQAPxx8h4f/3KKhFQNztZmLOznRysfZ0OXVWASboQQQogKLiVdw+d/nuf/Dt8EoHk1Rxb198fVtmwO2pdwI4QQQlRgtx4kMnJNEKfvxgAwqr0PYzvWxFhddq8WI+FGCCGEqKC2nwnjg40niUtOx8HShK/6+tHe19XQZT01CTdCCCFEBZOarmX29gv8sP86AAFe9iweEEBlewsDV1Y0JNwIIYQQFcjd6CRGrQki+FY0AMPbVuPD52pjUoa7obIqP69EiAKYPHkyb775psGe/5tvvqFHjx4Ge34hRMW050IE3RfuI/hWNLbmxvzvtcZ80r1uuQo2IOGmXFCpVE+8ffbZZ9y4cSPHx1599dVcj9uuXTtUKhXr1q3T2z5//ny8vb2L+VWRrWYnJyc6d+5McHDwUx03LCyMBQsW8Mknn2TbPnr0aKpXr46ZmRmenp706NGDXbt26dp4e3vr6rGysiIgIIANGzboHh8yZEiO62bt3bsXlUpFdHQ0AEOHDiUoKIh9+/Y91WsRQoj8SNdkdEO9vvIY0YlpNKxix5Z329K5nruhSysWEm7KgdDQUN1t/vz52Nra6m17fHHMv//+W++xJUuWPPHY5ubmTJo0ibS0tOJ+GbnKrHnHjh3Ex8fTtWtXXUgojO+//55WrVpRtWpV3bYbN27QuHFjdu/ezZw5czh9+jTbt2+nffv2jBw5Um//adOmERoaSnBwME2bNqVv374cPHiwQDWYmpoyYMAAWWNLCFHswmOTGfD9EZbtvQrAkFbebHi7JZ6OlgaurPhIuCkH3N3ddTc7OztUKpXeNmtra11bJyenbO2fpH///kRHR/Pdd9/l2iansxVjx46lXbt2uvvt2rVj9OjRjB07FgcHB9zc3Pjuu+9ISEjg9ddfx8bGBh8fH7Zt25bt+Jk1N2nShLlz5xIeHs6RI0eYNm0a9evXz9bez8+PyZMn51rvunXrsnUJjRgxApVKxdGjR3nllVeoVasW9erVY/z48Rw+fFivrY2NDe7u7tSqVYslS5ZgYWHBH3/8kevz5aZHjx78/vvvJCUlFXhfIYTIj/2X79NtwT6OXn+ItZkxSwYE8NkL9TAzVhu6tGIl4SYvigKpCYa5lYIF221tbfnkk0+YNm0aCQkJT3WsH3/8EWdnZ44ePcro0aN555136N27N61atSIoKIjOnTvz2muvkZiYmOsxLCwyRvKnpqYydOhQzp8/z7Fjx3SPBwcHc+rUKV5//fUc93/48CHnzp2jSZMmetu2b9/OyJEjsbKyyraPvb19rvUYGxtjYmJCampqXi8/myZNmpCens6RI0cKvK8QQjyJRqvw9c5LvLb8CA8SUqlTyZY/Rrehe8NKhi6tRMhsqbykJcKMyoZ57on3wDT7l+3TaNWqFUZGjzLtvn378Pf3f+I+I0aMYMGCBXz11VdPPCOSl0aNGjFp0iQAJkyYwKxZs3B2dmb48OEAfPrppyxbtoxTp07RokWLbPtHR0czffp0rK2tadasGW5ubnTp0oUVK1bQtGlTAFasWEFgYCDVq1fPsYZbt26hKAqVKz/6mV65cgVFUahdu3aBXk9qairz5s0jJiaGDh06FGhfAEtLS+zs7Lh582aB9xVCiNxExqUwdn0wB648AKB/M0+m9KiHuUn5PlvzODlzU8GsX7+ekJAQ3a1u3bp57mNmZsa0adOYO3cu9+/fL/RzN2zYUPf/arUaJycnGjRooNvm5uYGQEREhN5+rVq1wtraGgcHB06ePMn69et1bYcPH87atWtJTk4mNTWVNWvWMHTo0FxryOwCMjd/dElxpYBnyD766COsra2xtLRk9uzZzJo1i+7duxfoGJksLCyeeKZKCCEK4si1B3RfuI8DVx5gYaLmqz6NmPlywwoVbEDO3OTNxDLjDIqhnruIeXp64uPjU+D9Xn31VebOncvnn3+ebaaUkZFRtoCQ0wBkExMTvfsqlUpvm0qVseKsVqvVa7d+/Xrq1q2Lk5NTti6iHj16YGZmxqZNmzA1NSUtLY1evXrl+jqcnTMWgIuKisLFxQWAmjVrolKpuHDhQq77Pe6DDz5gyJAhWFtb4+bmpqsbMrrxcjoTEx0djVqtztbt9fDhQ10dQghRWFqtwjf/XmXujotoFfBxtWbZwABqutkYujSDMOiZm2XLltGwYUNsbW2xtbWlZcuWOQ4ozbRy5cpsU5kf/wu8WKhUGV1DhripSs/y8kZGRsycOZNly5Zx48YNvcdcXFwIDQ3V2xYSElJkz+3p6UmNGjVyHPtibGzM4MGDWbFiBStWrKBfv366cTk5qVGjBra2tpw7d063zdHRkS5durBkyZIcxxVlnZnl7OyMj48P7u7uesEGwNfXl7Nnz5KSkqK3PSgoiGrVqumFuatXr5KcnJxnt6AQQjxJVEIqb/x4jC+3ZwSbl/09+H1U6wobbMDA4aZKlSrMmjWLEydOcPz4cTp06MCLL77I2bNnc90n6zRnGa9Qcrp3707z5s359ttv9bZ36NCB48ePs2rVKi5fvsyUKVM4c+ZMidU1bNgwdu/ezfbt25/YJQUZIa1jx47s379fb/uSJUvQaDQ0a9aMX375hcuXL3P+/HkWLlxIy5Yt813LwIEDUalUDBo0iBMnTnDlyhWWL1/O/Pnzee+99/Ta7tu3j+rVq1OjRo38v1ghhHhM0K0oui/cx56LkZgZGzH7lQbM69MIS9OK3TFj0HDTo0cPunXrRs2aNalVqxZffPEF1tbW2abePi7rNOfMsReiZMyePZvk5GS9bV26dGHy5Ml8+OGHNG3alLi4OAYNGlRiNdWsWZNWrVpRu3Ztmjdvnmf7YcOGsW7dOr3ur+rVqxMUFET79u157733qF+/Pp06dWLXrl0sW7Ys37XY29uzb98+0tLSeOGFF/Dz82PhwoV89dVXvPXWW3pt165dqxtMLYQQBaEoCt/vu0afbw5xLyaZas5WbBrRmr5NvbKdUa6IVEpBR1MWE41Gw4YNGxg8eDDBwcE5DnRduXIlw4YNw8PDA61WS0BAADNmzKBevXq5HjclJUWviyA2NhZPT09iYmKwtbXVa5ucnMz169epVq1a8Xd3iSKjKAo1a9ZkxIgRjB8/Pl/tmzdvzrhx4+jfv38JVJjd2bNn6dChA5cuXcrzWkNCn/yeioouJimNDzeeZMfZcAC6N6zErJcbYGNukseeZVtsbCx2dnY5fn9nZfDZUqdPn8ba2hozMzPefvttNm3alOsMHl9fX5YvX85vv/3GTz/9hFarpVWrVty5cyfX48+cORM7OzvdzdPTs7heijCAyMhIFi9eTFhYWK7XtslKpVLxv//9j/T09GKuLnehoaGsWrVKgo0QokBO34nh+UX72HE2HFO1EdNfrMfi/v7lPtgUlMHP3KSmpnLr1i1iYmLYuHEj33//Pf/880++piinpaVRp04d+vfvz/Tp03NsI2duyjeVSoWzszMLFixgwIABhi5HlAD5PRUVkaIo/HT4JtP/PE+qRksVBwuWDgygYRV7Q5dWYgpy5sbgI45MTU11U5MbN27MsWPHWLBgQbZBqzkxMTHB39+fK1eu5NrGzMwMMzOzIqtXlC6lpFdVCCGKTXxKOh//coo/T2XMSu1U1425vRphZylna3Jj8HCTlVarzTaNNjcajYbTp0/TrVu3Yq5KCCGEKHnnQ2MZuTqIa/cTMDZS8XHX2rzRppoMGs6DQcPNhAkT6Nq1K15eXsTFxbFmzRr27t3Ljh07ABg0aBAeHh7MnDkTyFiNuUWLFvj4+BAdHc2cOXO4efMmw4YNM+TLEEIIIYqUoihsOH6Hyb+dISVdSyU7cxYPCKBxVQdDl1YmGDTcREREMGjQIEJDQ7Gzs6Nhw4bs2LGDTp06ARnrAD2+DlJUVBTDhw8nLCwMBwcHGjduzMGDB/M1PkcIIYQoCxJT05m8+Sy/BGVMlmnn68JXffxwtDI1cGVlh8EHFJe0Jw1IkoGKQpR+8nsqyrMrEXGMWB3EpfB4jFTwXmdf3gmsgZGRdEOVqQHFQgghhIBNwXf4ZNMZElM1uNiYsai/Py2qOxm6rDJJwo0QQghhQMlpGqb+cZa1R28D0NrHifl9/XGxkZm+hWXwi/iJiuHGjRuoVCrdgpp79+5FpVJlW5SyoLy9vZk/f/5T1ycy/PDDD3Tu3Nlgz799+3b8/PyyrQwvRHl1/X4CLy09yNqjt1GpYMyzNVk1tLkEm6ck4aYcCQsLY/To0VSvXh0zMzM8PT3p0aMHu3btMnRpeHp6EhoaSv369UvsOb29vbOtIv/4bciQIQA5PtamTZtcjztkyBBUKhWzZs3S27558+YSm575eK12dna0bt2a3bt3P9Uxk5OTmTx5MlOmTNHbHhsbyyeffELt2rUxNzfH3d2djh078uuvv+quM9SuXTtdPebm5tStW5elS5fqjvHZZ5/h5+eX7Tmzht7nnnsOExMTVq9e/VSvRYiyYMupUHos2s/50FicrExZNbQZ4zrVQi3ja56ahJty4saNGzRu3Jjdu3czZ84cTp8+zfbt22nfvj0jR440dHmo1Wrc3d0xNi65ntBjx47pVo//5ZdfALh48aJu24IFC3RtV6xYobfa/O+///7EY5ubmzN79myioqKK9TU8SWbNBw4cwNnZmeeff55r164V+ngbN27E1taW1q1b67ZFR0fTqlUrVq1axYQJEwgKCuLff/+lb9++fPjhh8TExOjaDh8+nNDQUM6dO0efPn0YOXIka9euLXAdQ4YMYeHChYV+HUKUdinpGj77/Swj1wQRn5JOU28HtrzblrY1XQxdWrkh4aacGDFiBCqViqNHj/LKK69Qq1Yt6tWrx/jx4/VWWb916xYvvvgi1tbW2Nra0qdPH8LDw3WPZ/6FvXz5cry8vLC2tmbEiBFoNBq+/PJL3N3dcXV15YsvvtB7fpVKxbJly+jatSsWFhZUr16djRs36h7P+hd6Tvbv30/btm2xsLDA09OTd999l4SEBN3jERER9OjRAwsLC6pVq5bnX/cuLi661eMdHR0BcHV11W17fF0ne3t7vdXmM9vnpmPHjri7u+uuwZSTnM5WzJ8/H29vb939IUOG0LNnT2bMmIGbmxv29vZMmzaN9PR0PvjgAxwdHalSpQorVqzIdvzMmuvXr8+yZctISkpi586drFq1Cicnp2wXw+zZsyevvfZarvWuW7eOHj166G2bOHEiN27c4MiRIwwePJi6detSq1Ythg8fTkhICNbW1rq2lpaWuLu7U716dT777DNq1qyZZ0jMSY8ePTh+/DhXr14t8L5ClHa3HybS55tDrDx4A4C3A2uwdngL3O1k5l9RknCTB0VRSExLNMgtv7P0Hz58yPbt2xk5ciRWVlbZHre3twcyrv784osv8vDhQ/755x927tzJtWvX6Nu3r177q1evsm3bNrZv387atWv54Ycf6N69O3fu3OGff/5h9uzZTJo0iSNHjujtN3nyZF555RVOnjzJwIED6devH+fPn8/Xa7h69SrPPfccr7zyCqdOnWL9+vXs37+fUaNG6doMGTKE27dvs2fPHjZu3MjSpUuJiIjI1/GLmlqtZsaMGSxatOiJC7fmx+7du7l37x7//vsvX331FVOmTOH555/HwcGBI0eO8Pbbb/PWW2898XksLCyAjLXaevfujUaj0QsWERERbNmyhaFDh+Z6jP3799OkSRPdfa1Wy7p16xg4cCCVK1fO1t7a2vqJZ+IsLCxITU194mvPiZeXF25ubuzbt6/A+wpRmu08F073hfs4eScGOwsTlg9pwsdda2Oslq/ioiazpfKQlJ5E8zXNDfLcRwYcwdLEMs92V65cQVEUateu/cR2u3bt4vTp01y/fl23OvqqVauoV68ex44do2nTpkDGl9ry5cuxsbGhbt26tG/fnosXL7J161aMjIzw9fVl9uzZ7Nmzh+bNH703vXv31l0tevr06ezcuZNFixbpjb3IzcyZMxk4cCBjx44FoGbNmixcuJDAwECWLVvGrVu32LZtG0ePHtXV+cMPP1CnTp08j50f/fv3R61W6+7/9NNP9OzZ84n7vPTSS/j5+TFlyhR++OGHQj+3o6MjCxcu1L23X375JYmJiUycOBHIuJL3rFmz2L9/P/369cu2f2JiIpMmTUKtVhMYGIiFhQUDBgxgxYoV9O7dW/d6vLy8aNeuXY41REdHExMToxdi7t+/T1RUVJ7/rrLSaDSsXbuWU6dO8eabbxZo30yVK1fm5s2bhdpXiNImTaNl7o6LfPtvRrexn6c9iwf4U8Uh7893UTgSbsqB/J7hOX/+PJ6enrpgA1C3bl3s7e05f/68LjR4e3tjY2Oja+Pm5oZarda7WrSbm1u2syYtW7bMdv9J3VCPO3nyJKdOndLralIUBa1Wy/Xr17l06RLGxsY0btxY93jt2rV1Z6We1tdff03Hjh119ytVqpSv/WbPnk2HDh14//33C/3c9erVy/bePj7wWq1W4+TklO39zgxkSUlJuLi48MMPP9CwYUMgY/xL06ZNuXv3Lh4eHqxcuVI3EDonSUlJAHoXxSvo9T2XLl3K999/T2pqKmq1mnHjxvHOO+8U6BiZLCwsSExMLNS+QpQmoTFJjFoTzImbGePzhrauxsdda2NqLGdripOEmzxYGFtwZMCRvBsW03PnR82aNVGpVFy4cKFIntfERH+lWZVKleO2opyuGx8fz1tvvcW7776b7TEvLy8uXbpUZM+VE3d3d93q9AXxzDPP0KVLFyZMmKCbfZXJyMgoW0BIS0vLdozCvt+ZgczOzg4XF/2BiP7+/jRq1IhVq1bRuXNnzp49y5YtW3J9HU5OTqhUKr0B0i4uLtjb2+f739XAgQP55JNPsLCwoFKlSnqBzdbWVm/wcabMSwE8Pv4JMrpas74mIcqavRcjGLc+hKjENGzMjJnTuyHP1c/fH07i6Ui4yYNKpcpX15AhOTo60qVLF5YsWcK7776bbdxNdHQ09vb21KlTh9u3b3P79m3d2Ztz584RHR1dJOtzHT58mEGDBund9/f3z9e+AQEBnDt3LteAUbt2bdLT0zlx4oTuDNPFixef+jo5RWHWrFn4+fnh6+urt93FxYWwsDAURdGdMcnvmaz8yCuQDRs2jPnz53P37l06duyod8YuK1NTU+rWrcu5c+d017kxMjKiX79+/N///R9TpkzJNu4mPj4ec3Nz3bgbOzu7XOvx9fXlzp07hIeH4+bmptseFBSEubk5Xl5eum3JyclcvXo13/92hCht0jVa5v99mSV7r6AoUK+yLUsHBlDVKfuYSFE85LxYObFkyRI0Gg3NmjXjl19+4fLly5w/f56FCxfquos6duxIgwYNGDhwIEFBQRw9epRBgwYRGBioN5C0sDZs2MDy5cu5dOkSU6ZM4ejRo3oDgp/ko48+4uDBg4waNYqQkBAuX77Mb7/9ptvf19eX5557jrfeeosjR45w4sQJhg0bphtIa0iZ72nW6cvt2rUjMjKSL7/8kqtXr7JkyRK2bdtWYnUNGDCAO3fu8N133z1xIHGmLl26sH//fr1tX3zxBZ6enjRv3pxVq1Zx7tw5Ll++zPLly/H39yc+Pj5ftXTp0gVfX1/69+/PwYMHuXbtGhs3bmTSpEmMGTNGb7zT4cOHMTMzy9bNKURZEBGbzKs/HGHxnoxg82oLL355p5UEmxIm4aacqF69OkFBQbRv35733nuP+vXr06lTJ3bt2sWyZcuAjLNQv/32Gw4ODjzzzDN07NiR6tWrs379+iKpYerUqaxbt46GDRuyatUq1q5dm+8zQg0bNuSff/7h0qVLtG3bFn9/fz799FO9swUrVqygcuXKBAYG8vLLL/Pmm2/i6upaJLU/rWnTpmXrNqpTpw5Lly5lyZIlNGrUiKNHjz7V2JyCsrOz45VXXsHa2jrPwdEAb7zxBlu3btXrPnJ0dOTw4cO8+uqrfP755/j7+9O2bVvWrl3LnDlzsnUn5cbY2Ji//voLLy8v+vfvT/369ZkyZQpjxoxh+vTpem3Xrl3LwIEDsbQs3WdMhcjq4NX7dFu4n8PXHmJlqmZBPz8+79kAcxN13juLIiWrgj9GVhsuPJVKxaZNm/L1JSpKzrPPPku9evXyfVG83r17ExAQwIQJE4q5spzdv38fX19fjh8/TrVq1XJsI7+norTRahWW7LnC139fQqtAbXcblgwMoIaLdd47i3yTVcGFqOCioqLYu3cve/fuzddU/Exz5szhjz/+KMbKnuzGjRssXbo012AjRGnzID6FcT+f5N9LkQD0aVKFqS/Ux8JUztYYkoQbIcohf39/oqKimD17draBzk/i7e3N6NGji7GyJ2vSpEmRjP8SoiQcu/GQ0WuCCYtNxtzEiOkv1qd3k9wH7ouSI+FGFIkK1rtZ6t24ccPQJQhRbmm1Ct/tu8aXOy6i0SpUd7Fi2cDG+Lrb5L2zKBESboQQQoh8ik5M5f0NJ/n7fMZFNV/0q8yMlxpgZSZfp6WJ/DSEEEKIfAi5Hc3I1UHcjU7C1NiIKT3qMqCZV65X/haGI+FGCCGEeAJFUVh58AYztp4nTaNQ1cmSJQMCqO+Rv0shiJIn4UYIIYTIRWxyGh9tPMW2M2EAdK3vzuxeDbE1N8ljT2FIEm6EEEKIHJy5G8PINUHcfJCIiVrFJ93qMLiVt3RDlQESboQQQojHKIrCmqO3mPrHOVLTtXjYW7BkYAB+nvaGLk3kkyy/ICqkyZMn8+abbxq0hm+++YYePXoYtAYhhL6ElHTGrg/hk01nSE3X8mxtV7a820aCTRkj4aYcUKlUT7x99tln3LhxI8fHXn311VyP265dO1QqFevWrdPbPn/+fLy9vYv5VZGtZicnJzp37kxwcPBTHTcsLIwFCxbwySefZNs+ZswYfHx8MDc3x83NjdatW7Ns2TISExN17by9vXU1WVlZERAQwIYNG3SPDxkyJMdlKPbu3YtKpdKtZD506FCCgoLYt2/fU70eIUTRuBgWxwuL9/NbyD3URiomdK3Nd4OaYG9paujSRAFJuCkHQkNDdbf58+dja2urt+3xxRr//vtvvceWLFnyxGObm5szadIk0tLSivtl5Cqz5h07dhAfH0/Xrl11AaEwvv/+e1q1akXVqlV1265du4a/vz9//fUXM2bMIDg4mEOHDvHhhx/y559/8vfff+sdY9q0aYSGhhIcHEzTpk3p27cvBw8eLFAdpqamDBgwIN/rPgkhis/GE3d4ccl+rkYm4GZrxro3W/BWYA2MjGR8TVkk4aYccHd3193s7OxQqVR626ytHy3e5uTklK39k/Tv35/o6Gi+++67XNvkdKZi7NixtGvXTne/Xbt2jB49mrFjx+Lg4ICbmxvfffcdCQkJvP7669jY2ODj48O2bduyHT+z5iZNmjB37lzCw8M5cuQI06ZNo379+tna+/n5MXny5FzrXbduXbbuoBEjRmBsbMzx48fp06cPderUoXr16rz44ots2bIlW3sbGxvc3d2pVasWS5YswcLColBrMvXo0YPff/+dpKSkAu8rhHh6SakaPtx4kvc3nCQ5TUvbms5sfbctTb0dDV2aeAoSbvKgKAraxESD3ErDkga2trZ88sknTJs2jYSEhKc61o8//oizszNHjx5l9OjRvPPOO/Tu3ZtWrVoRFBRE586dee211/S6gLKysLAAIDU1laFDh3L+/HmOHTumezw4OJhTp07x+uuv57j/w4cPOXfunN76RQ8ePOCvv/5i5MiRWFlZ5bjfk2ZHGBsbY2JiQmpq6hNff06aNGlCeno6R44cKfC+QoinczUynpeWHuDn43cwUsH4TrVY+XoznKzNDF2aeEoyWyoPSlISFwMaG+S5fYNOoLK0LNJjtmrVCiOjR5l23759+Pv7P3GfESNGsGDBAr766qsnnhHJS6NGjZg0aRIAEyZMYNasWTg7OzN8+HAAPv30U5YtW8apU6do0aJFtv2jo6OZPn061tbWNGvWDDc3N7p06cKKFSto2rQpACtWrCAwMJDq1avnWMOtW7dQFIXKlSvrtl25cgVFUbItMOns7ExycjIAI0eOZPbs2dmOl5qayrx584iJiaFDhw4Ffk8sLS2xs7Pj5s2bBd5XCFF4v5+8x4RfTpGQqsHZ2oyF/fxo5eNs6LJEEZEzNxXM+vXrCQkJ0d3q1q2b5z5mZmZMmzaNuXPncv/+/UI/d8OGDXX/r1arcXJyokGDBrptbm5uAEREROjt16pVK6ytrXFwcODkyZOsX79e13b48OGsXbuW5ORkUlNTWbNmDUOHDs21hszuH3Nz8zzrPXr0KCEhIdSrV4+UlBS9xz766COsra2xtLRk9uzZzJo1i+7du+d5zJxYWFg88WyVEKLoJKdpmLT5NO+uDSYhVUOL6o5sHdNGgk05I2du8qCysMA36ITBnruoeXp64uPjU+D9Xn31VebOncvnn3+ebaaUkZFRti60nAYgm5joX9FTpVLpbcvs+tFqtXrt1q9fT926dXFycsLe3l7vsR49emBmZsamTZswNTUlLS2NXr165fo6nJ0zPsCioqJwcXEBwMfHB5VKxcWLF/XaZp79scjh5/DBBx8wZMgQrK2tcXNz0+u2srW1zfFMTHR0NGq1OlvX18OHD3W1CCGKz80HCYxcE8SZu7EAjO7gw5hna2Kslr/zyxsJN3lQqVRF3jVUFhkZGTFz5kxefvll3nnnHb3HXFxcOHPmjN62kJCQbGGmsDw9PalRo0aOjxkbGzN48GBWrFiBqakp/fr1yzGMZKpRowa2tracO3eOWrVqARkDljt16sTixYsZPXp0ruNuHufs7JxrSPT19WXdunWkpKRgZvao7z4oKIhq1arpvS9Xr14lOTk5z65BIcTT2X4mlA82nCIuJR0HSxO+7utHO19XQ5cliolB4+qyZcto2LAhtra22Nra0rJlyxxnyzxuw4YN1K5dG3Nzcxo0aMDWrVtLqFrRvXt3mjdvzrfffqu3vUOHDhw/fpxVq1Zx+fJlpkyZki3sFKdhw4axe/dutm/f/sQuKcgIaR07dmT//v1625cuXUp6ejpNmjRh/fr1nD9/nosXL/LTTz9x4cIF1Gp1vusZOHAgKpWKQYMGceLECa5cucLy5cuZP38+7733nl7bffv2Ub169VzDmxDi6aSma5n2xzne/imIuJR0Gld1YMu7bSXYlHMGDTdVqlRh1qxZnDhxguPHj9OhQwdefPFFzp49m2P7gwcP0r9/f9544w2Cg4Pp2bMnPXv2LNEv0opu9uzZukG2mbp06cLkyZP58MMPadq0KXFxcQwaNKjEaqpZsyatWrWidu3aNG/ePM/2w4YNY926dXrdXzVq1CA4OJiOHTsyYcIEGjVqRJMmTVi0aBHvv/8+06dPz3c99vb27Nu3j7S0NF544QX8/PxYuHAhX331FW+99ZZe27Vr1+oGVAshitbd6CT6fHuI5QeuA/DmM9VZ92YLKtsXfZe/KF1USmmYb/wYR0dH5syZwxtvvJHtsb59+5KQkMCff/6p29aiRQv8/Pz45ptv8nX82NhY7OzsiImJwdbWVu+x5ORkrl+/TrVq1fI14FSUDoqiULNmTUaMGMH48ePz1b558+aMGzeO/v37l0CFOTt79iwdOnTg0qVLeV5vSDwiv6ciP3ZfCGf8zyeJTkzD1tyYub0b0bmeu6HLEk/hSd/fWZWaUVQajYZ169aRkJBAy5Ytc2xz6NAhOnbsqLetS5cuHDp0qCRKFKVQZGQkixcvJiwsLNdr22SlUqn43//+R3p6ejFX92ShoaGsWrVKgo0QRShdo2X29gsMXXmc6MQ0GlaxY8u7bSXYVDAGH1B8+vRpWrZsSXJyMtbW1mzatCnX6clhYWG6KcCZ3NzcCAsLy/X4KSkpetN4Y2Nji6ZwUSq4urri7OzM//73PxwcHPK9n5+fH35+fsVXWD5kDepCiKcTHpvM6DXBHL3xEIAhrbyZ0K02Zsb5HzMnygeDhxtfX19CQkKIiYlh48aNDB48mH/++Sdf11/Jj5kzZzJ16tQiOZYofUpZr6oQwkD2XY5k7LoQHiSkYm1mzOxXGtK9YSVDlyUMxODdUqampvj4+NC4cWNmzpxJo0aNWLBgQY5t3d3dCQ8P19sWHh6Ou3vupxsnTJhATEyM7nb79u0irV8IIYThaLQKX+28xKDlR3mQkEqdSrb8MbqNBJsKzuDhJiutVpvtarCZWrZsya5du/S27dy5M9cxOpBxdd3MqeaZt7zI2QAhSi/5/RSZIuNSGLT8CAt3XUZRoH8zLzaNaEU157yvVSXKN4N2S02YMIGuXbvi5eVFXFwca9asYe/evezYsQOAQYMG4eHhwcyZMwEYM2YMgYGBzJs3j+7du7Nu3TqOHz/O//73vyKpJ/PiaomJiU+8EJwQwnAyl6ooqotEirLp8LUHjF4bTGRcChYmama8XJ+X/KsYuixRShg03ERERDBo0CBCQ0Oxs7OjYcOG7Nixg06dOgEZixw+vshjq1atWLNmDZMmTWLixInUrFmTzZs3U79+/SKpR61WY29vr1vbyNLS8omrQQshSo6iKCQmJhIREYG9vX2BLqwoyg+tVmHZP1eZ99dFtArUdLVm6cAAarrZGLo0UYqUuuvcFLe85skrikJYWBjR0dElX5wQIk/29va4u7vLHx4VUFRCKuN+DmHvxUgAXvb34POX6mNpavC5MaIEFOQ6N/IvIguVSkWlSpVwdXXNcfFHIYThmJiYyBmbCurEzShGrwniXkwyZsZGTHuxHn2aeErIFTmScJMLtVotH6JCCGFgiqLww/7rzNp2gXStQjVnK5YODKBOpbwnh4iKS8KNEEKIUikmMY0PNp7kr3MZlwB5vmElZr7cABtzGUwunkzCjRBCiFLn1J1oRq4J4vbDJEzVRkx+vg6vtqgq3VAiXyTcCCGEKDUUReH/Dt/k8z/Pk6rR4ulowdIBjWlQRdZgE/kn4UYIIUSpEJecxse/nmbLqVAAOtd1Y07vRthZSDeUKBgJN0IIIQzu3L1YRq4J4vr9BIyNVHzctTZvtKkm3VCiUCTcCCGEMBhFUfj5+G0+/e0sKelaKtmZs3hAAI2rOhi6NFGGSbgRQghhEImp6UzafIZfg+4C0M7Xha/6+OFoZWrgykRZJ+FGCCFEibscHseI1UFcjojHSAXvd/Hl7WdqYGQk3VDi6Um4EUIIUaI2Bd9h4q9nSErT4GpjxsL+/rSo7mToskQ5IuFGCCFEiUhO0zD1j7OsPXobgNY+Tszv64+LjZmBKxPljYQbIYQQxe76/QRGrA7ifGgsKhW826Em7z5bE7V0Q4liIOFGCCFEsdpyKpSPfjlFfEo6TlamzO/nR9uaLoYuS5RjEm6EEEIUi5R0DTO2nOfHQzcBaObtyKIB/rjZmhu4MlHeSbgRQghR5G4/TGTkmiBO3YkB4J12NXivUy2M1UYGrkxUBBJuhBBCFKmd58J57+cQYpPTsbc04as+jehQ283QZYkKRMKNEEKIIpGm0fLl9gt8t+86AP5e9iweEICHvYWBKxMVjYQbIYQQT+1edBKj1gQRdCsagDfaVOOj52pjaizdUKLkSbgRQgjxVPZejGDc+hCiEtOwMTdmTq9GPFff3dBliQpMwo0QQohCSddomf/3ZRbvuQJAfQ9blgwIoKqTlYErExWdhBshhBAFFhGbzLvrgjl87SEAr7bwYlL3upibqA1cmRASboQQQhTQwSv3eXddCPfjU7AyVTPzlYa80KiyocsSQkfCjRBCiHzRahUW77nC/L8voVWgtrsNSwYGUMPF2tClCaFHwo0QQog8PYhPYez6EPZdvg9AnyZVmPpCfSxMpRtKlD4SboQQQjzR0esPGb02iPDYFMxNjPi8ZwN6Na5i6LKEyJWEGyGEEDnSahX+t+8ac3ZcRKNVqOFixdKBjfF1tzF0aUI8kYQbIYQQ2UQnpvLezyfZdSECgBf9KjPjpQZYmcnXhij95F+pEEIIPcG3ohi1Jpi70UmYGhvxWY969G/miUqlMnRpQuSLhBshhBAAKIrCyoM3mLH1PGkahapOliwZEEB9DztDlyZEgUi4EUIIQWxyGh9tPMW2M2EAdK3vzuxeDbE1NzFwZUIUnIQbIYSo4M7cjWHkmiBuPkjERK3ik251GNzKW7qhRJkl4UYIISooRVFYfeQW0/48R2q6Fg97C5YMDMDP097QpQnxVAy6Fv3MmTNp2rQpNjY2uLq60rNnTy5evPjEfVauXIlKpdK7mZubl1DFQghRPsSnpDNmXQiTNp8hNV1LxzqubHm3jQQbUS4Y9MzNP//8w8iRI2natCnp6elMnDiRzp07c+7cOayscl9V1tbWVi8EyalTIYTIvwthsYxYHcS1yATURio+es6X4W2ry2epKDcMGm62b9+ud3/lypW4urpy4sQJnnnmmVz3U6lUuLu7F3d5QghR7mw4fpvJv50hOU2Lu605iwf408Tb0dBlCVGkDNotlVVMTAwAjo5P/kWLj4+natWqeHp68uKLL3L27NmSKE8IIcqspFQNH2w4yQcbT5GcpqVtTWe2vNtGgo0ol0rNgGKtVsvYsWNp3bo19evXz7Wdr68vy5cvp2HDhsTExDB37lxatWrF2bNnqVIl+1onKSkppKSk6O7HxsYWS/1CCFFaXY2MZ8RPQVwMj8NIBeM61mJkex+MjKQbSpRPKkVRFEMXAfDOO++wbds29u/fn2NIyU1aWhp16tShf//+TJ8+Pdvjn332GVOnTs22PSYmBltb26eqWQghSrvfQu4y8dfTJKRqcLY2Y2E/P1r5OBu6LCEKLDY2Fjs7u3x9f5eKcDNq1Ch+++03/v33X6pVq1bg/Xv37o2xsTFr167N9lhOZ248PT0l3AghyrXkNA3T/zzH6iO3AGhR3ZGF/f1xtZHZpaJsKki4MWi3lKIojB49mk2bNrF3795CBRuNRsPp06fp1q1bjo+bmZlhZmb2tKUKIUSZcfNBAiNWB3H2XiwqFYxq78OYZ2tirC5VwyyFKDYGDTcjR45kzZo1/Pbbb9jY2BAWlnHZbzs7OywsLAAYNGgQHh4ezJw5E4Bp06bRokULfHx8iI6OZs6cOdy8eZNhw4YZ7HUIIURpsf1MKB9sOEVcSjqOVqZ83dePwFouhi5LiBJl0HCzbNkyANq1a6e3fcWKFQwZMgSAW7duYWT06K+NqKgohg8fTlhYGA4ODjRu3JiDBw9St27dkipbCCFKndR0LTO3nWfFgRsANKnqwKIB/lSyszBsYUIYQKkYc1OSCtJnJ4QQZcGdqERGrgnm5O1oAN56pjrvd/HFRLqhRDlSZsbcCCGEeDq7zocz/ueTxCSlYWtuzLw+fnSq62bosoQwKAk3QghRBqVrtMz96xLf/HMVgEZV7Fg8IABPR0sDVyaE4Um4EUKIMiYsJpnRa4M4diMKgCGtvJnQrTZmxmoDVyZE6SDhRgghypB/L0Uydn0IDxNSsTYzZvYrDenesJKhyxKiVJFwI4QQZYBGq7Bg12UW7b6MokDdSrYsHRiAt7OVoUsTotSRcCOEEKVcZFwKY9YFc/DqAwD6N/NiSo+6mJtIN5QQOZFwI4QQpdihqw94d10wkXEpWJqqmfFSA3r6exi6LCFKNQk3QghRCmm1Csv+ucq8vy6iVaCWmzVLBwbg42pj6NKEKPUk3AghRCnzMCGVcetD+OdSJAAvB3jwec/6WJrKR7YQ+SG/KUIIUYqcuPmQUWuCCY1JxszYiOkv1qd3kyqoVCpDlyZEmSHhRgghSgFFUfh+33Vmb79AulahmrMVSwcGUKeSLBMjREFJuBFCCAOLSUzj/Y0n2XkuHIAejSoz8+UGWJvJR7QQhSG/OUIIYUCn7kQzYnUQd6KSMFUbMblHXV5t7iXdUEI8hacONykpKZiZmRVFLUIIUWEoisL/Hb7J53+eJ1WjxdPRgqUDGtOgip2hSxOizDMq6A7btm1j8ODBVK9eHRMTEywtLbG1tSUwMJAvvviCe/fuFUedQghRbsQlpzFqbTCf/naWVI2WznXd+HN0Wwk2QhQRlaIoSn4abtq0iY8++oi4uDi6detGs2bNqFy5MhYWFjx8+JAzZ86wb98+Dh06xJAhQ5g+fTouLi7FXX+BxcbGYmdnR0xMDLa2MlBPCFGyzt2LZeSaIK7fT8DYSMWEbnUY2tpbuqGEyENBvr/zHW5atmzJpEmT6Nq1K0ZGuZ/wuXv3LosWLcLNzY1x48YVrPISIOFGCGEIiqKw/thtpvx+lpR0LZXtzFk8MIAALwdDlyZEmVAs4aa8kHAjhChpianpTNp0hl+D7wLQ3teFr/r44WBlauDKhCg7CvL9XSSzpTQaDadPn6Zq1ao4OMhfIUIIkelyeBzvrA7iSkQ8Rip4v4svbz9TAyMj6YYSorgUeEAxwNixY/nhhx+AjGATGBhIQEAAnp6e7N27tyjrE0KIMuvXoDu8sPgAVyLicbUxY+3wFoxo5yPBRohiVqhws3HjRho1agTAH3/8wfXr17lw4QLjxo3jk08+KdIChRCirElO0/DxL6cY//NJktI0tPFxZuuYtjSv7mTo0oSoEAoVbu7fv4+7uzsAW7dupXfv3tSqVYuhQ4dy+vTpIi1QCCHKkmuR8fRccoB1x26jUsHYjjX5cWgznK3lemBClJRChRs3NzfOnTuHRqNh+/btdOrUCYDExETUanWRFiiEEGXFn6fu8cLiA1wIi8PJypT/G9qcsR1roZZuKCFKVKEGFL/++uv06dOHSpUqoVKp6NixIwBHjhyhdu3aRVqgEEKUdinpGmZsOc+Ph24C0KyaI4v6++Nma27gyoSomAoVbj777DPq16/P7du36d27t275BbVazccff1ykBQohRGl2+2EiI9cEcepODAAj2tVgfKdaGKsLdWJcCFEE5Do3QghRSH+dDeP9DSeJTU7H3tKEr/v40b62q6HLEqJcKpHr3Bw7dow9e/YQERGBVqvVe+yrr74q7GGFEKLUS9Nomb3tAt/vvw6Av5c9iwcE4GFvYeDKhBBQyHAzY8YMJk2ahK+vL25ubnprosj6KEKI8uxedBKj1gQRdCsagDfaVOOj52pjaizdUEKUFoUKNwsWLGD58uUMGTKkiMsRQojSa8/FCMavDyEqMQ0bc2Pm9GrEc/XdDV2WECKLQoUbIyMjWrduXdS1CCFEqZSu0fL135dYsucqAPU9bFk6oDFeTpYGrkwIkZNCnUcdN24cS5YsKepahBCi1ImITWbg90d0wea1FlXZ+HYrCTZClGKFOnPz/vvv0717d2rUqEHdunUxMTHRe/zXX38tkuKEEMKQDl65z7vrgrkfn4qVqZqZrzTkhUaVDV2WECIPhQo37777Lnv27KF9+/Y4OTnJIGIhRLmi0Sos3n2F+bsuoShQ292GJQMDqOFibejShBD5UKhw8+OPP/LLL7/QvXv3p3rymTNn8uuvv3LhwgUsLCxo1aoVs2fPxtfX94n7bdiwgcmTJ3Pjxg1q1qzJ7Nmz6dat21PVIoQQAPfjUxi3PoR9l+8D0LeJJ5+9UA8LU1laRoiyolBjbhwdHalRo8ZTP/k///zDyJEjOXz4MDt37iQtLY3OnTuTkJCQ6z4HDx6kf//+vPHGGwQHB9OzZ0969uzJmTNnnroeIUTFdvT6Q7ov3Me+y/cxNzFibu9GzO7VUIKNEGVMoa5QvGLFCrZv386KFSuwtCy6QXWRkZG4urryzz//8Mwzz+TYpm/fviQkJPDnn3/qtrVo0QI/Pz+++eabPJ9DrlAshMhKq1X49t9rzP3rIhqtQg0XK5YObIyvu42hSxNC/KfYr1C8cOFCrl69ipubG97e3tkGFAcFBRXmsMTEZKzN4ujomGubQ4cOMX78eL1tXbp0YfPmzYV6TiFExRaVkMp7G06y+0IEAD39KvPFSw2wMiv0BdyFEAZWqN/enj17FnEZoNVqGTt2LK1bt6Z+/fq5tgsLC8PNzU1vm5ubG2FhYTm2T0lJISUlRXc/Nja2aAoWQpR5wbeiGLUmmLvRSZgaGzH1hXr0a+opkySEKOMKFW6mTJlS1HUwcuRIzpw5w/79+4v0uDNnzmTq1KlFekwhRNmmKAorDtxg5rbzpGkUvJ0sWTIwgHqV7QxdmhCiCOR7QHFxLh4+atQo/vzzT/bs2UOVKlWe2Nbd3Z3w8HC9beHh4bi753wJ9AkTJhATE6O73b59u8jqFkKUPbHJabzzUxDT/jxHmkahWwN3fh/dRoKNEOVIvsNNvXr1WLduHampqU9sd/nyZd555x1mzZqV5zEVRWHUqFFs2rSJ3bt3U61atTz3admyJbt27dLbtnPnTlq2bJljezMzM2xtbfVuQoiK6czdGJ5fuJ/tZ8MwUav4rEddlgwIwNbcJO+dhRBlRr67pRYtWsRHH33EiBEj6NSpE02aNKFy5cqYm5sTFRXFuXPn2L9/P2fPnmXUqFG88847eR5z5MiRrFmzht9++w0bGxvduBk7OzssLCwAGDRoEB4eHsycOROAMWPGEBgYyLx58+jevTvr1q3j+PHj/O9//yvM6xdCVACKorD6yC2m/XGOVI0WD3sLlgwMwM/T3tClCSGKQYGngu/fv5/169ezb98+bt68SVJSEs7Ozvj7+9OlSxcGDhyIg4ND/p48l0F7K1as0K043q5dO7y9vVm5cqXu8Q0bNjBp0iTdRfy+/PLLfF/ET6aCC1GxxKekM/HX0/x+8h4AHeu4Mrd3I+wtTQ1cmRCiIAry/V2o69yUZRJuhKg4LoTFMuKnIK7dT0BtpOLj52ozrG01mQ0lRBlU7Ne5EUKI0u7n47f59LczJKdpcbc1Z/EAf5p4534NLSFE+SHhRghRriSlapj82xk2nrgDwDO1XPi6TyOcrM0MXJkQoqRIuBFClBtXIuIZuTqIi+FxGKlgfKdajGjng5GRdEMJUZFIuBFClAu/hdxlwq+nSUzV4GxtxsL+frSq4WzosoQQBiDhRghRpiWnaZj25znWHLkFQIvqjizs74+rjbmBKxNCGEq+L+KX1dWrV5k0aRL9+/cnIiJjwblt27Zx9uzZIitOCCGe5Mb9BF5eepA1R26hUsHoDj6sHtZCgo0QFVyhws0///xDgwYNOHLkCL/++ivx8fEAnDx5sljWnRJCiKy2nQ6lx6L9nAuNxdHKlJWvN+O9zr6oZXyNEBVeocLNxx9/zOeff87OnTsxNX10IawOHTpw+PDhIitOCCGySk3XMvWPs7yzOoi4lHSaVHVgy7ttCKzlYujShBClRKHG3Jw+fZo1a9Zk2+7q6sr9+/efuighhMjJnahERq4J5uTtaADeCqzO+519MVEXuoddCFEOFSrc2NvbExoamm2hy+DgYDw8PIqkMCGEeNyu8+GM//kkMUlp2FmYMK93IzrWdTN0WUKIUqhQf+7069ePjz76iLCwMFQqFVqtlgMHDvD+++8zaNCgoq5RCFGBpWm0zNx6njd+PE5MUhqNqtjx5+g2EmyEELkq1JmbGTNmMHLkSDw9PdFoNNStWxeNRsOAAQOYNGlSUdcohKigQmOSGL0mmOM3owAY0sqbid3qYGos3VBCiNw91cKZt27d4syZM8THx+Pv70/NmjWLsrZiIQtnClE2/HspkrHrQ3iYkIq1mTFf9mpItwaVDF2WEMJASmzhTC8vL7y8vJ7mEEIIoUejVVjw9yUW7bmCokDdSrYsHRiAt7OVoUsTQpQRhQo3iqKwceNG9uzZQ0REBFqtVu/xX3/9tUiKE0JULBFxyYxZG8Khaw8AGNDci0+fr4u5idrAlQkhypJChZuxY8fy7bff0r59e9zc3FCp5KJZQoinc+jqA95dF0xkXAqWpmpmvNSAnv4y+1IIUXCFCjf/93//x6+//kq3bt2Kuh4hRAWj1Sos++cq8/66iFaBWm7WLB3YGB9Xa0OXJoQoowoVbuzs7KhevXpR1yKEqGAeJqQybn0I/1yKBOCVgCpM71kPS1NZ01cIUXiFmk/52WefMXXqVJKSkoq6HiFEBXH8xkO6L9zHP5ciMTM24steDZnXp5EEGyHEUyvUp0ifPn1Yu3Ytrq6ueHt7Y2Jiovd4UFBQkRQnhCh/FEXhu33XmL39IhqtQnVnK5YMDKBOJbk0gxCiaBQq3AwePJgTJ07w6quvyoBiIUS+xSSm8d6Gk/x9PhyAHo0qM/PlBlibydkaIUTRKdQnypYtW9ixYwdt2rQp6nqEEOXUydvRjFwTxJ2oJEzVRkzuUZdXm3vJH0dCiCJXqHDj6ekpV/cVQuSLoiisOnSTz7ecI02j4OVoyZIBATSoYmfo0oQQ5VShBhTPmzePDz/8kBs3bhRxOUKI8iQuOY1Ra4KZ8vtZ0jQKXeq58cfoNhJshBDFqlBnbl599VUSExOpUaMGlpaW2QYUP3z4sEiKE0KUXefuxTJi9QluPEjE2EjFhG51GNraW7qhhCjP4iPgxn4wsQTf5wxWRqHCzfz584u4DCFEeaEoCuuO3WbK72dJTddS2c6cxQMDCPByMHRpQoiiFheWEWZuHsj47/1LGdu9WpW9cDN48OCirkMIUQ4kpKQzafMZNgXfBaC9rwtf9fHDwcrUwJUJIYpE7D24cQBu7s8IMw+uZG/jVh+qNC752h6T73ATGxurG0QcGxv7xLYy2FiIiudSeBwjVgdxJSIetZGK9zv78tYz1TEykm4oIcqsmLv/nZn5L8w8vJalgQrc64N3W6jaGqq2AktHg5T6uHyHGwcHB0JDQ3F1dcXe3j7HfnNFUVCpVGg0miItUghRuv1y4g6TNp8hKU2Dq40Zi/r707y6k6HLEkIUVPRt/TATdUP/cZURuDd4LMy0BIvS1+Wc73Cze/duHB0z0tiePXuKrSAhRNmRnKZhym9nWX/8NgBtfJyZ388PZ2szA1cmhMiXqJuPxsvc2A/RN/UfVxlBpUbg3QaqtgGvFmBhb5BSCyLf4SYwMFD3/9WqVcPT0zPb2RtFUbh9+3bRVSeEKLWuRcYzYnUQF8LiUKlgzLM1Gd2hJmrphhKidFKUjDMxujBzAGJu6bdRqaGy32NhpjmYl71LNxRqQHG1atV0XVSPe/jwIdWqVZNuKSHKuT9O3uPjX06RkKrB2dqUBf38ae3jbOiyhBCPU5SMMTKPh5nYO/ptVGrwCMjoYvJumxFmzGwMU28RKlS4yRxbk1V8fDzm5uZPXZQQonRKSdfwxZbzrDqUceq6WTVHFvX3x81Wfu+FMDhFgQdXH42XuXEA4u7ptzEyBo/G/4WZNuDZHMysDVNvMSpQuBk/fjwAKpWKyZMnY2lpqXtMo9Fw5MgR/Pz88n28f//9lzlz5nDixAlCQ0PZtGkTPXv2zLX93r17ad++fbbtoaGhuLu75/t5hRAFd+tBIiPXBHH6bgwAI9rVYHynWhirC3WhcyHE01KUjKnYN/Y9CjPxYfptjEygSpPHwkwzMLUyTL0lqEDhJjg4GMg4c3P69GlMTR9du8LU1JRGjRrx/vvv5/t4CQkJNGrUiKFDh/Lyyy/ne7+LFy/qTTfP2j0mhChaO86G8f6Gk8Qlp2NvacLXffxoX1t+74QoUYqScZG8G/sygsyN/ZAQod9GbQpVmj4KM1WagqllzscrxwoUbjJnSb3++ussWLDgqa9n07VrV7p27Vrg/TKnowshileaRsvsbRf4fv91APy97Fk8IAAPewsDVyZEBaAoEHnh0UymmwcgIVK/jdos42yMLsw0ARP5/SzUmJsVK1YUdR0F4ufnR0pKCvXr1+ezzz6jdevWBq1HiPLobnQSo9YEEXwrGoBhbarx4XO1MTWWbighioVWC5Hn9cNM4gP9NsbmGWdjvNuCd2vwaAImMuYtq0KFG0OpVKkS33zzDU2aNCElJYXvv/+edu3aceTIEQICAnLcJyUlhZSUFN39vK6uLISAPRciGPdzCNGJadiYGzO3dyO61JNxbUIUKa0WIs4+FmYOQlKWhaeNLTLOzOjCTGMwlutI5aVMhRtfX198fX1191u1asXVq1f5+uuv+b//+78c95k5cyZTp04tqRKFKNPSNVq+2nmJpXuvAtDAw44lAwLwcqp4ffZCFDmtBsLPPBr8e/MAJEfrtzGxzJjB5N0m41Y5AIxlbbaCKlPhJifNmjVj//79uT4+YcIE3SwvyDhz4+npWRKlCVGmhMcmM3ptMEevZ/zlOKhlVT7pXgczY7WBKxOijNJqIOzUo8G/Nw9CSox+GxOrjKv+6sKMP6hNDFNvOVLmw01ISAiVKlXK9XEzMzPMzOQUnhBPcuDKfcasC+Z+fCpWpmpmvdKQHo0qG7osIcoWTTqEnXwUZm4dgpQsQyFMbfTDTKVGEmaKgUHDTXx8PFeuPFou/fr164SEhODo6IiXlxcTJkzg7t27rFq1CoD58+dTrVo16tWrR3JyMt9//z27d+/mr7/+MtRLEKJM02gVFu2+zIJdl1EUqO1uw9KBAVR3KX8X9RKiyGnSIPTkozEztw5Dapx+GzNb8Gr5X5hpDe6NQF3mzyuUegZ9h48fP653Ub7M7qPBgwezcuVKQkNDuXXr0boXqampvPfee9y9exdLS0saNmzI33//neOF/YQQT3Y/PoWx60LYf+U+AH2beDL1xXqYm0g3lBA50qTBveBHYeb2EUiN129jZgdVWz0WZhqCkfxOlTSVoiiKoYsoSbGxsdjZ2RETE/PU1+kRoqw6cu0Bo9cGExGXgoWJms971ueVxlUMXZYQpUt6KtwLeizMHIW0BP025vb/XWPmv+vMuNWXMFNMCvL9LefGhKhAtFqFb/+9xty/LqLRKvi4WrN0YAC13Mr+QnlCPLX0FLh74r8xM/sywkx6kn4bC4dHF8zzbgOu9cBIrv1U2ki4EaKCiEpIZfzPIey5mHGF05f8Pfi8Z32szORjQFRQaclw9/ijMHPnGKQn67exdNIPMy51JMyUAfKpJkQFEHQrilGrg7gXk4ypsRHTXqhH36aeqFQqQ5cmRMlJS84IMJlX/719FDQp+m0snR8FGe824OwrYaYMknAjRDmmKArLD9xg5tbzpGsVvJ0sWTIwgHqV7QxdmhDFLzVRP8zcOQaaVP02Vq6PBv96twXnWiChv8yTcCNEORWTlMaHG0+y42w4AN0bVGLWKw2wMZdraohyKjUh42yMLswcB22afhtrd/0w4+QjYaYcknAjRDl0+k4MI9ac4PbDJEzUKiZ1r8ugllWlG0qULynxGdOxM8PM3ROgTddvY1P50Uwm77bgWF3CTAUg4UaIckRRFH46covpf5wjVaPFw96CpQMDaORpb+jShHh6KXFw6wjc/G9q9r3g7GHG1uPReJmqrSXMVFASboQoJ+JT0pnw62n+OHkPgI51XJnX2w87S+mGEmVUcmzGVX91YSYEFI1+GztP/TDj4C1hRki4EaI8uBAWy4ifgrh2PwG1kYqPn6vNsLbVpBtKlC3JMXDz0KMwE3oSFK1+G/uqWcJMVcPUKko1CTdClHE/H7/N5M1nSEnX4m5rzuIB/jTxdjR0WULkLSnqvzDz33Vmwk5nDzMO3v8Fmf8GAdt7GaRUUbZIuBGijEpMTWfy5rP8EnQHgMBaLnzd1w9HK1MDVyZELhIfws2Dj4WZM0CWFYAcq+uHGTtZFkQUnIQbIcqgKxFxjFgdxKXweIxU8F5nX94JrIGRkXRDiVIk8eF/Qea/bqbws2QLM04++mHGtrJBShXli4QbIcqYzcF3mbjpNImpGlxszFjYz5+WNZwMXZYQkHD/sTBzACLOZm/jXOvReBnvNmDjXvJ1inJPwo0QZURymoapf5xj7dFbALSs7sSC/n642pgbuDJRYcVH6IeZyPPZ27jUfhRmqrYGG7eSr1NUOBJuhCgDbtxPYMTqIM6FxqJSwej2PozpWAu1dEOJkhQX/mgm040DcP9i9jaudfXDjLVLydcpKjwJN0KUcltPh/LhxlPEp6TjaGXK1339CKwlXxiiBMSGPhr8e+MAPLicvY1bff0wYyVdpMLwJNwIUUqlpmuZsfU8Kw/eAKBJVQcWDfCnkp2FYQsT5VfMXf0w8/BqlgYqcK//3+DfNlC1FVjKZQdE6SPhRohS6PbDREatDebk7WgA3gqszvudfTFRGxm2MFG+RN/WDzNR17M0UEGlho+FmZZg4WCQUoUoCAk3QpQyf58L570NJ4lJSsPOwoSv+jTi2ToyCFMUgehbj8bL3NgH0Tf1H1cZQaVG/81kagteLcDC3iClCvE0JNwIUUqkabTM3XGRb/+9BkAjT3uWDPCnioOlgSsTZZKiZIQXXZjZDzG39Nuo1FDZ77Ew0xzM7QxSrhBFScKNEKVAaEwSo9YEc+JmFACvt/ZmQtc6mBpLN5TIJ0XJ6FZ6PMzE3tFvo1KDR4B+mDGzMUy9otxQFIXY1FjCEsJ0N1szW7pW62qwmiTcCGFg/1yKZNz6EB4mpGJjZsyXvRrStUElQ5clSjtFgYfXHo2XubEf4u7ptzEyBo/Gjy6Y59kczKwNU68os+JT4zNCS2JGcAlPDNcLMuGJ4SSlJ+nt4+/qL+FGiIpIo1WY//clFu+5gqJA3Uq2LB0YgLezlaFLE6WRosCDK4+WMrh5AOJC9dsYmWSEmcxVsz2bgan8exK5S0pPyjGwhCWGEZ6QsS0+LT5fx3Iwc8Ddyh03KzfqONYp5sqfTMKNEAYQEZfMmLUhHLr2AIABzb349Pm6mJuoDVyZKDUUBe5f0g8z8eH6bdSm4NHkvzDTGqo0A1MZoyUypGpS9UJLTgEmJiUmX8eyMbXB3codd0v3jP9m3iwzwoybpRvmxqXnaukSboQoYQev3ufdtSHcj0/B0lTNjJca0NPfw9BlCUNTFIi8oB9mEiL126jNoErTx8JMUzCR6x5VROnadCITI3VdRVlDS1hCGA+TH+brWJbGltkCi7uVO26WbrptliZlKzRLuBGihGi1Ckv3XuGrnZfQKlDLzZqlAxvj4ypjICokrTZjLabMadk3D0Liff02xub/hZm2GWHGowmYlJ6/jkXx0Gg1PEh+kGNgyTz7cj/pPlpFm+exzNRm2c6w6AUZK3dsTGxQqcrXUi4SboQoAQ/iUxj380n+vZTxl3ivxlWY/mJ9LEylG6rC0GozVsl+PMwkZfnL2tgiY5yMLsw0BmMzw9QrioWiKDxMfqh3xiVzbEtmcIlIjCBdSc/zWMZGxrhZ5hBYHus6sjezL3fBJT8k3AhRzI7feMioNcGExSZjbmLEtBfr06eJp6HLEsVNq4HwM49mMt06CElR+m1MLDNmMHn/NzW7cgAYmxqmXvHUcpoSrRvnkvgoyKRqU/M8lpHKCBcLlxwDS+bN0dwRI5VcLiInEm6EKCaKovDdvmvM3n4RjVahuosVSwcGUNvd1tClieKg1UDY6UdjZm4dhOQsgzVNrDKu+psZZir5SZgpQzKnRGcNLE+aEp0TFSqcLJyyBRY3KzfdNmcLZ4yN5Cu6sOSdE6IYRCem8v6Gk/x9PgKAHo0qM/PlBlibya9cuaFJh7BTjwb/3jwEWWeemFqDV8vHwkwjUJsYpl7xREnpSRndQ1kH6D7llOhsAea/biQT+XdQrOSTVogiFnI7mpGrg7gbnYSp2ohPe9RlYHOvCtnvXa5o0iH05H/jZf4LM6lx+m3MbB8LM23AvRGo5WPW0EpiSnTmuJfSNiW6opLfOiGKiKIo/HjwBl9sPU+aRsHL0ZKlAwOo7yFr9ZRJmjS4F/IozNw6DKlZ/nI3s4OqrR4LMw3BSAaJl6QnTYnODDEPkh/k61hZp0TrBuo+FmTK2pToikrCjRBFIDY5jY9/OcXW02EAdKnnxpe9GmFnIaeey4z0VLgX/FiYOQJpCfptzO3/W8rgvzDjVl/CTDHSKlruJ91/4gDdgk6JfvwMS0WYEl1RGTTc/Pvvv8yZM4cTJ04QGhrKpk2b6Nmz5xP32bt3L+PHj+fs2bN4enoyadIkhgwZUiL1CpGTs/diGLk6iBsPEjE2UjGxWx1eb+0tH5KlXXoK3A36b8zMfrh9FNIS9dtYODxal8m7DbjWAyOZnVIUsk6Jfny8S+YYF5kSLQrLoOEmISGBRo0aMXToUF5++eU821+/fp3u3bvz9ttvs3r1anbt2sWwYcOoVKkSXbp0KYGKhXhEURTWHr3NZ3+cJTVdi4e9BYsG+BPg5WDo0kRO0lPgzvGMszI39sHtY5B1Zoul03/dTG0zQo1rXQkzhfD4lOicxrfIlGhR3Awabrp27UrXrvlfNfSbb76hWrVqzJs3D4A6deqwf/9+vv76awk3okQlpKTzyabTbA7JWIW5Q21X5vVuhIOVTOstNdKS4c6x/8LM/oz/T0/Wb2Pp/GgmU9XW4FJbwkw+JKQl5BhYZEq0KC3K1L+YQ4cO0bFjR71tXbp0YezYsbnuk5KSQkpKiu5+bGxscZUnKohL4XG889MJrkYmoDZS8UEXX95sWx0jIznlbVBpSRldS7owcxw0KfptrFwfjZep2gZcfEG6KvTkNSU6PCGcuLS4vA9EzlOi9f5fpkSLYlKmwk1YWBhubm5629zc3IiNjSUpKQkLi+wLyM2cOZOpU6eWVIminNt44g6TNp8mOU2Lm60Zi/oH0Kyao6HLqphSE+H2kUdh5u4J0GTp5rB21w8zzjUrdJjJbUr042EmOiU6X8d6fEp0btdzkSnRwlDKVLgpjAkTJjB+/Hjd/djYWDw95dL3omCSUjVM+f0MPx+/A0AbH2fm9/PD2VrW/SkxqQkZYSbzCsB3g0Cbpt/GptKjwb9V24BTjQoTZrJOic7p7Et+p0RbGFvkPL5FpkSLMqJMhRt3d3fCw8P1toWHh2Nra5vjWRsAMzMzzMzkC0gU3tXIeEauDuJCWBwqFYx9thajOviglm6o4pUSD7cP/xdmDsC9INBmmTlj6/FYmGkNjtXLZZh5fEp0bgN0CzIl+vFZRTIlWpRHZSrctGzZkq1bt+pt27lzJy1btjRQRaK8++PkPT7+5RQJqRqcrU1Z0M+f1j7Ohi6rfEqO/e/MzL7/wkwwKBr9Nnaej4KMdxtw8C7zYSZzSrReaCmmKdFuVm44mDlIcBHlnkHDTXx8PFeuXNHdv379OiEhITg6OuLl5cWECRO4e/cuq1atAuDtt99m8eLFfPjhhwwdOpTdu3fz888/s2XLFkO9BFFOJadp+HzLOX46fAuA5tUcWdTfH1dbGUNQZJJjMq76mxlmQkMg65kHe69HM5m824BDVYOUWlglNSU6M8w4WTjJlGghMHC4OX78OO3bt9fdzxwbM3jwYFauXEloaCi3bt3SPV6tWjW2bNnCuHHjWLBgAVWqVOH777+XaeCiSN16kMiINSc4czdjZt3I9jUY17EWxmr50ngqSdFw69CjMTNhp7KHGQfvR+NlvFtnhJtSrKimRAM4Wzjnul6RTIkWomBUiqIohi6iJMXGxmJnZ0dMTAy2traGLkeUMtvPhPHBxpPEJadjb2nC1339aO/rauiyyqbEh1nCzGkgy8eNY3X9MGNXxSCl5iQ5PTnXwJJ5xqUwU6L1uoxkSrQQ+VaQ72/5M0AIIDVdy+ztF/hh/3UAArzsWTwggMr2OQ9UFzlIfPjftOz/pmaHnyFbmHHy0Q8ztpUNUmpRT4nOKbDIlGghDEfCjajw7kYnMWpNEMG3ogEY3rYaHz5XGxPphnqyhPv6YSbibPY2zrX012aycS/2sjKnROc0QLeop0S7WblhZWJVzK9ICFFQEm5EhbbnQgTjfg4hOjENG3Nj5vZuRJd6xf8FXCbFR2YsMJkZZiLPZ2/jUvtRmKnaGmzcsrd5ClpFy4OkBzkGloJOiTY1Ms11fEvmfVtTW5lZJEQZJOFGVEjpGi3zdl5i2d6rADTwsGPJgAC8nOTCZDpx4fph5v7F7G1c6+qHGWuXQj+doihEpUTlOqOoKKZEP35fpkQLUX5JuBEVTnhsMqPXBnP0+kMABrWsyifd62BmrDZwZQYWG/poKYMb++HB5ext3Oo/FmZagVX+rvmT15To8IRwwhPDScm6FlQOcpoS7WblptddJFOihajYJNyICmXf5UjGrgvhQUIq1mbGzHqlAc83NMygVoOLvfcoyNzYDw+vZmmgAvf6/w3+/S/MWOa8jlZOU6Izz7Zknn0p6JTorIFFpkQLIfJLPiFEhaDRKizcdZmFuy+jKFDb3YalAwOo7mJt6NJKTswd/TATdT1LAxVUavhYmGkJFg6PpkTHXCL8Xni28S4FmRJtb2af/WyLTIkWQhQxCTei3IuMS2Hs+mAOXMmYIdOvqSefvVAPc5Ny3g0VfevReJmb+yHqhv7jKiPS3BsQXiWAMNdahNk4E5b2X9fR3W2EXV5RsCnRJjY5BhaZEi2EKGkSbkS5duTaA0avDSYiLgULEzWf96zPK41Lz4XiioyiQPRNvTCTHn2L+2o1YcZqwoyNCbOzI8zWjTArh4xtmkQeJD+E+7sybk/wpCnRmYFGpkQLIUoLCTeiXNJqFb759ypzd1xEq4CPqzVLBwZQy83G0KUVCa1Ww4OwYMKu7Sbs7lHCHlwgLD2OMPV/QcZGzX17T7TZZgMlQ0ooPDZu90lTojP/X6ZECyHKEgk3otyJSkhl/M8h7LkYCcBL/h583rM+VmZl4597jlOiE8IIi7pCePQ1wpPuE65NJv3xsGGlArJfjjynKdFZp0fLlGghRHlTNj7thcinoFtRjFodxL2YZEyNjZj2Qj36NvUsNV/euU2Jzvr/eU6JVqkwUhRcUONu5oCbrSfuTnVwt/WUKdFCiApPwo0oFxRF4Yf915m17QLpWgVvJ0uWDAygXmW7Eq3j8SnROV3PpSBTop3SNbhr0nFP12TctArudt64uwfg7v0MztWexdhcFn8VQoisJNyIMi8mKY0PNpzkr3PhAHRvUIlZrzTAxrxopxQnpyfnGlgKNSXa1B53RYVbcjzu0fdwT4zBXaPBPT0dt3QNJmozqNIUav63yGSVpmAiC3kKIUReJNyIMu30nRhGrDnB7YdJmKhVTH6+Lq+1qFrgbqg0Tdqj4JIlsBR4leicpkRbuuGelorbg+u43TuNxc1DkHhKf0djc6jS7NEikx5NwESmTgshREFJuBFlkqIo/HT4JtP/PE+qRksVBwuWDAigkad9trbp2nTuJ93P/YxLYjgPkh6goOT5vDlNic46QNfKxAq0Wog4lzEt+8K+jCnaSQ/1D2ZsAZ6Ph5nGYGxWRO+QEEJUXBJuRJkTn5LOx7+c4s9ToYCWZ+pY8M6zDkRojvF/58KyjXmJTIp86lWi85wSrdVC+Bk4/2PG+kw3D0BSlH4bE0vwbJ7RxeTdFioHgLFp0bwpQgghdCTciFIrpynR5yJus/3CBRK097GqEYOxaRzBpPPmk69Bh7HKGDerjCnRua1ZVKAp0VoNhJ1+tNDkzQOQHKPfxsQKvFo8CjOV/CTMCCFECZBwIwziqaZEmz/6h6slY5VoZwvnXK+gWyRTojXpEHbqsTBzCFKyhBlT6//CTJuM9Zkq+4GskySEECVOwo0oFolpiTmObynMlGhHcyfSU2x5EGOJkm5HNXsPhrUMwMexSvGtEq1Jh9CTGWsy3dgPtw5DSqx+GzNb/TBTqRGo5VdKCCEMTT6JRYEV+ZToHFaJzhzjEh9vybtrz3AzIh4jFbzX2Zd3AmtgZFTEF+XTpMG9EP0wkxqv38bMLmOlbO82ULU1uDeUMCOEEKWQfDILPU+aEp25PSolKu8DkX1KtN6sov/CjIVx7tdt2RR8h4m/HiMpTYOLjRkL+/nTsoZT0bzQ9FS4F/xYmDkCaQn6bcztMkKMLsw0AKNyvpK4EEKUAxJuKpDcpkSHJzwKM4WZEp3bAN3CrhKdnKZh6h9nWXv0NgCtajixoJ8/LjZPMU06PRXunngUZm4fhbTELC/KQT/MuNWTMCOEEGWQhJtyQqtoeZD0QO+My+MXoCvMlGg3K7dsgaW4V4m+fj+BEauDOB8ai0oFozvUZMyzNVEXtBsqPSUjzNx4LMxkHeNj4fhoJlPV1uBaF4xkHSYhhCjrJNyUAY9Pic4aWDJnFYUnhpOuTc/zWMYqY1wtXR+Fl6edEl2EtpwK5aNfThGfko6jlSnz+/rxTC2X/O2clgx3jz8KM3eOQXqyfhtLZ/0w41JbwowQQpRDEm5KgRRNCjdjb2YLLAVaJZr8TYl2NHdEXcq6WlLSNczceoGVB28A0NTbgUX9A3C3e8LSA2lJGQHmxv6Mq//eOQZZ3yMrl0dX/63aBlx8oZSsDi6EEKL4SLgxsMjESF7b9hp34+/m2dbJ3CnHwJLZfeRi6VL0U6KL2e2HiYxaE8TJOxnXjHkrsDrvd/bFRJ3ljEpqItw5+ijM3D0OmlT9NtZuj8bLeLcF55oSZoQQogIqW9+E5dC8E/O4G38XS2NLvGy99AboZr38v6m6fF3ddue5cN77OYTY5HTsLEz4qk8jnq3jlvFgagLcPvJYmDkB2jT9A9hU0g8zTjUkzAghhJBwY0jHw46z5doWVKhY3mU59ZzrGbqkEpGm0TJnx0X+9+81ABp52rO0Vy084k7C399kBJp7QZB1DJGtx2Nhpg04VpcwI4QQIhsJNwaSpk3jiyNfANCrVq8KE2xCY5IYtSaYizfv0s7oEsM9Q2lpfB6jb4JB0eg3tvPUDzMO3hJmhBBC5EnCTRFS0tJICw1FSU9HSU+H9HQUjSbL/2tQ0tPYe2MXThcvUc3IimHWtYj57TdDl1+80pK4dfkUNy+EMEW5QxVVJGoUuAtxKgBTsHQC51rg7AsutcDKOWPfO8Cd08Bpw9VfXCpsWKuor5uK+zOvoC8bMMjsU0NTOzhg1bKlwZ5fpShK3ldsK0diY2Oxs7MjJiYGW1vbIjuuJjaWaz17kn4vtMiOKYQQQpRFFn5+eK9bW6THLMj3t5y5KSLJ5y/ogo3azg6MjVGp1aiMjbP9/93EUB6kR2NuZkUdl3qo1MZl/3ormjRIjoakqIxbSva1pZIwI9XEDhsHF4wsHcDYHCpWttZXYV97RX3dGdesqpAq6MsGKuzvuVmNGgZ9/lIRbpYsWcKcOXMICwujUaNGLFq0iGbNmuXYduXKlbz++ut628zMzEhOTs6xfUmxqF8P75/Xo6SnYxkQkGu742HHGbHjdVSYsLb7j1Qtq2NtEh/CzYOPLpoXfoasn2BJttXYFleDvSm+nDKuz7iXA3nRz8Mw9QohhKgwDB5u1q9fz/jx4/nmm29o3rw58+fPp0uXLly8eBFXV9cc97G1teXixYu6+6WhP9PIygqLhg2f2KZMDyJOeAA3D2QEmZsH/gszWTjXgqqt0VZtw4o7lfliXxRaBWq5WfP9wMb4uFqXfN1CCCEqHIOHm6+++orhw4frzsZ88803bNmyheXLl/Pxxx/nuI9KpcLd3b0kyywS6y6s40r0FezN7HnX/11Dl/Nk8ZH6YSbiXPY2zr6PXQG4Ndi48SA+hbHrQ9h3+T4AvRtXYdqL9bEwLV1XRRZCCFF+GTTcpKamcuLECSZMmKDbZmRkRMeOHTl06FCu+8XHx1O1alW0Wi0BAQHMmDGDevVyPguSkpJCSsqjy/LHxsYW3QsogMjESJaGLAVgTMAY7M3tDVJHruIjHnUx3TwAkReyt3Gp81+YaZ0RZqz1z6wdu/GQ0WuCCYtNxtzEiGkv1qdPE88SegFCCCFEBoOGm/v376PRaHBzc9Pb7ubmxoULOXy5Ar6+vixfvpyGDRsSExPD3LlzadWqFWfPnqVKlSrZ2s+cOZOpU6cWS/0F8dWJr4hPi6eBcwNervmyocuBuDD9MHP/UvY2rvX0w0zm1OwstFqF7/Zd48sdF9FoFaq7WLF0YAC13YtuNpoQQgiRXwbvliqoli1b0vKxufOtWrWiTp06fPvtt0yfPj1b+wkTJjB+/Hjd/djYWDw9S/ZswvGw4/x57U9UqPik+ScYqQwwMyr2XsYyBjf2ZYSZB1eyNFCBW339MGPpmOdhoxNTeX/DSf4+HwHAC40qM+PlBliblbl/WkIIIcoJg34DOTs7o1arCQ8P19seHh6e7zE1JiYm+Pv7c+VK1i/rDGZmZpiZmT11rYVlsEHEMXf0w8zDa1kaqMC9QcaaTN6twatlvsLM40JuRzNydRB3o5MwVRsx5YW6DGjmVSoGeAshhKi4DBpuTE1Nady4Mbt27aJnz54AaLVadu3axahRo/J1DI1Gw+nTp+nWrVsxVlp4mYOI7czsincQcfTt/7qY/utqirqh/7jKCNwbPhoA7NUCLBwK9VSKorDy4A1mbD1PmkbBy9GSpQMDqO9h9/SvQwghhHhKBu87GD9+PIMHD6ZJkyY0a9aM+fPnk5CQoJs9NWjQIDw8PJg5cyYA06ZNo0WLFvj4+BAdHc2cOXO4efMmw4YNM+TLyNHjg4jHBowt2kHEUTcfjZe5sQ+ib+k/rjKCSn76Ycb86cNHbHIaH208xbYzYQA8V8+dL3s3xNbc5KmPLYQQQhQFg4ebvn37EhkZyaeffkpYWBh+fn5s375dN8j41q1bGD129d6oqCiGDx9OWFgYDg4ONG7cmIMHD1K3bl1DvYRcFdkgYkXJOBOjCzP7Iea2fhuVGir7Z3QxebcFz+ZgXrQDes/cjWHkmiBuPkjE2EjFxG51eL21t3RDCSGEKFVkbalicjzsOK/veB0VKtZ2X1uwsTaKkjFG5vEwE3tXv42RMVQO+C/MtMkIM2Y2RfsidOUorDl6i6l/nCM1XYuHvQWLB/jj71W4bi0hhBCioGRtKQNL16Yz4+gMIJ+DiBUFHlx9NF7mxgGIu6ffxsgEPBrrhxlTq2J6BY8kpKQzcdNpfgvJqOfZ2q7M69MIe0vTYn9uIYQQojAk3BSDdRfWcTnqcv4GEUechw1Dsl80T20KHk0ehZkqzcDUsthqzsnFsDhGrD7B1cgE1EYqPujiy5ttq2NkJN1QQgghSi8JN0XsftJ9loQsAfIxiPjiNvhlGKTGZ4SZKs0eCzNNwcSiZIrOwcYTd5i0+TTJaVrcbM1YPCCApt4FmyouhBBCGIKEmyL21fGMQcT1nernPohYUWD/V7BrOqBkDADu/SNYOZVorTlJStUw5fcz/Hz8DgBtazrzdV8/nK0Nd60gIYQQoiAk3BSRm7E3GfH3CG7F3cq4EnGLXK5EnJoIv4+CM79k3G86HJ6bCWrDT6W+GhnPyNVBXAiLQ6WCcR1rMbK9D2rphhJCCFGGSLgpIlHJUdyKy7jWTK9avajvXD97o5i7sG4AhIZkzHbqNgeaDC3ZQnPxW8hdJv56moRUDc7Wpizo509rn5zXkhJCCCFKMwk3RaSKTRVMjExwMHNgTMCY7A1uH4V1AyEhAiydoM//ZYyvMbDkNA3T/zzH6iMZwax5NUcW9ffH1dbcwJUJIYQQhSPhpog4Wziz45Ud2JrZYqbOMj4lZA38MQY0qRmLU/ZbAw5VDVPoY24+SGDE6iDO3osFYGT7GozrWAtjtQEW9hRCCCGKiISbIuRi6aK/QZMOf0+BQ4sz7td+Hl76FsysS764LLafCeWDDaeIS0nHwdKEr/r60d7X1dBlCSGEEE9Nwk1xSYqGjUPh6q6M+4EfQ+BHYGTYsyKp6VpmbbvA8gPXAQjwsmfxgAAq2xtu2rkQQghRlCTcFIf7l2FtP3hwBUwsoecyqNfT0FVxNzqJkauDCLkdDcDwttX48LnamEg3lBBCiHJEwk1Ru/x3xhmblBiw88wYX1OpoaGrYveFcMb/fJLoxDRszY2Z27sRneu5G7osIYQQoshJuClKwaszrmGjaMGzBfT9Caxd8t6vGKVrtMz96xLf/HMVgIZV7FgyIABPx5JdykEIIYQoKRJuikrm4GFFC/6vQvevwdiwi0uGxSTz7tpgjt54CMDgllWZ2L0OZsZqg9YlhBBCFCcJN0Ul4hwkRIKZHTw/3+BXHN53OZKx60J4kJCKtZkxs15pwPMNKxu0JiGEEKIkSLgpKukpGf+1dDBosNFoFRbsusyi3ZdRFKhTyZalAwOo5mxlsJqEEEKIkiThpqhYOkKDPmBluCULIuNSGLs+mANXHgDQv5knU3rUw9xEuqGEEEJUHBJuiopTDXjlO4M9/eFrDxi9NpjIuBQsTNR88VJ9Xg6oYrB6hBBCCEORcFPGabUKy/65yry/LqJVwMfVmmUDA6jpZmPo0oQQQgiDkHBThkUlpDLu5xD2XowE4GV/Dz5/qT6WpvJjFUIIUXHJt2AZdeLmQ0atCSY0JhkzYyOmvlCPvk09UalUhi5NCCGEMCgJN2WMoij8sP86s7ZdIF2rUM3ZiiUDAqhb2dbQpQkhhBClgoSbMiQmMY33N55k57lwALo3rMSslxtgY27Ya+oIIYQQpYmEmzLi1J1oRq4J4vbDJEzVRkx+vg6vtqgq3VBCCCFEFhJuSjlFUfi/wzf5/M/zpGq0VHGwYOnAABpWsTd0aUIIIUSpJOGmFItLTuPjX0+z5VQoAJ3qujG3VyPsLKUbSgghhMiNhJtS6ty9WEauCeL6/QSMjVR83LU2b7SpJt1QQgghRB4k3JQyiqKw/thtpvx+lpR0LZXszFk8wJ/GVR0NXZoQQghRJki4KUUSU9OZtOkMvwbfBSCwlgtf9/XD0crUwJUJIYQQZYeEm1LicngcI1YHcTkiHiMVvNfZl3cCa2BkJN1QQgghREFIuDGwlHQNm4LuMvWPcySlaXCxMWNhP39a1nAydGlCCCFEmSThxgAURSHkdjS/BN3hj5OhxCSlAdDax4n5ff1xsTEzcIVCCCFE2SXhpgTdjU5ic/Bdfgm6w7XIBN12N1szhrSqxpvPVEct3VBCCCHEUzEydAEAS5YswdvbG3Nzc5o3b87Ro0ef2H7Dhg3Url0bc3NzGjRowNatW0uo0oJLSEln44k7DPjuMG1m72bOjotci0zA3MSIl/w9+L83mnHw42d5p10NCTZCCCFEETD4mZv169czfvx4vvnmG5o3b878+fPp0qULFy9exNXVNVv7gwcP0r9/f2bOnMnzzz/PmjVr6NmzJ0FBQdSvX98AryA7jVbh8LUH/HLiDtvOhJGUptE91qK6Iy8HVKFbg0pYmxn87RdCCCHKHZWiKIohC2jevDlNmzZl8eLFAGi1Wjw9PRk9ejQff/xxtvZ9+/YlISGBP//8U7etRYsW+Pn58c033+T5fLGxsdjZ2RETE4OtbdGupL33YgSrj9zizN0YQmOSddurOVvxsr8HPf098HS0LNLnFEIIISqCgnx/G/TUQWpqKidOnGDChAm6bUZGRnTs2JFDhw7luM+hQ4cYP3683rYuXbqwefPmHNunpKSQkpKiux8bG/v0hedg2+lQ3lkdpLtva27M840q80pAFQK87OXKwkIIIUQJMWi4uX//PhqNBjc3N73tbm5uXLhwIcd9wsLCcmwfFhaWY/uZM2cyderUoin4CRpXdaC+hy0Olqb0a+rFs3VcMTdRF/vzCiGEEEJfuR/0MWHCBL0zPbGxsXh6ehb587jamvPn6LZFflwhhBBCFIxBw42zszNqtZrw8HC97eHh4bi7u+e4j7u7e4Ham5mZYWYm140RQgghKgqDTgU3NTWlcePG7Nq1S7dNq9Wya9cuWrZsmeM+LVu21GsPsHPnzlzbCyGEEKJiMXi31Pjx4xk8eDBNmjShWbNmzJ8/n4SEBF5//XUABg0ahIeHBzNnzgRgzJgxBAYGMm/ePLp37866des4fvw4//vf/wz5MoQQQghRShg83PTt25fIyEg+/fRTwsLC8PPzY/v27bpBw7du3cLI6NEJplatWrFmzRomTZrExIkTqVmzJps3by4117gRQgghhGEZ/Do3Ja04r3MjhBBCiOJRkO/vUrH8ghBCCCFEUZFwI4QQQohyRcKNEEIIIcoVCTdCCCGEKFck3AghhBCiXJFwI4QQQohyRcKNEEIIIcoVCTdCCCGEKFck3AghhBCiXDH48gslLfOCzLGxsQauRAghhBD5lfm9nZ+FFSpcuImLiwPA09PTwJUIIYQQoqDi4uKws7N7YpsKt7aUVqvl3r172NjYoFKpivTYsbGxeHp6cvv2bVm3qgjJ+1o85H0tHvK+Fg95X4tHWXpfFUUhLi6OypUr6y2onZMKd+bGyMiIKlWqFOtz2Nralvp/JGWRvK/FQ97X4iHva/GQ97V4lJX3Na8zNplkQLEQQgghyhUJN0IIIYQoVyTcFCEzMzOmTJmCmZmZoUspV+R9LR7yvhYPeV+Lh7yvxaO8vq8VbkCxEEIIIco3OXMjhBBCiHJFwo0QQgghyhUJN0IIIYQoVyTcCCGEEKJckXBTQEuWLMHb2xtzc3OaN2/O0aNHn9h+w4YN1K5dG3Nzcxo0aMDWrVtLqNKypSDv63fffUfbtm1xcHDAwcGBjh075vlzqKgK+u8107p161CpVPTs2bN4CyyjCvq+RkdHM3LkSCpVqoSZmRm1atWSz4IcFPR9nT9/Pr6+vlhYWODp6cm4ceNITk4uoWpLv3///ZcePXpQuXJlVCoVmzdvznOfvXv3EhAQgJmZGT4+PqxcubLY6ywWisi3devWKaampsry5cuVs2fPKsOHD1fs7e2V8PDwHNsfOHBAUavVypdffqmc+//27j8m6vqPA/gTDg9oYiWMXw4obmEMwRimQ0QyWTiZxchkSgiR0yaXJf3AeRUuFamRq5jZKge5MSkNKpUUpFhDyBQhcFwoHMraONKZE7CAu3t9/+jL7Yvgd95Njrvj+dg+f/Dm/bl7fl47uBfvz+fDtbfLW2+9JTNmzJC2tjYbJ7dvltZ13bp1sm/fPmlubhatVitZWVly//33yx9//GHj5PbN0rqO6u7uljlz5kh8fLw888wztgnrQCyt69DQkCxYsEBWrlwp9fX10t3dLXV1ddLS0mLj5PbN0rqWlZWJu7u7lJWVSXd3t5w8eVICAgJk69atNk5uv6qqqkSj0UhFRYUAkMrKyv87X6fTyX333Se5ubnS3t4uxcXFolAo5MSJE7YJfA+xubHAwoULJScnx/y10WiUwMBA2bNnz4Tz16xZI8nJyWPGFi1aJJs2bZrUnI7G0rrezmAwiJeXl3z55ZeTFdEhWVNXg8Egixcvli+++EIyMzPZ3EzA0rru379fQkNDZXh42FYRHZKldc3JyZEnn3xyzFhubq7ExcVNak5HdTfNzZtvvikRERFjxtLS0iQpKWkSk00Onpa6S8PDw2hqakJiYqJ5zNXVFYmJiWhsbJxwn8bGxjHzASApKemO86cja+p6u1u3bmFkZASzZ8+erJgOx9q6vvvuu/D19cWLL75oi5gOx5q6fv/994iNjUVOTg78/Pwwb948FBQUwGg02iq23bOmrosXL0ZTU5P51JVOp0NVVRVWrlxpk8zOyJnes6bdB2da69q1azAajfDz8xsz7ufnh99//33CffR6/YTz9Xr9pOV0NNbU9XZ5eXkIDAwc90M5nVlT1/r6ehw4cAAtLS02SOiYrKmrTqfDjz/+iPT0dFRVVaGzsxObN2/GyMgI8vPzbRHb7llT13Xr1uHatWtYsmQJRAQGgwEvvfQStm/fbovITulO71k3b97E33//DU9PzylKZjmu3JBDKywsRHl5OSorK+Hh4THVcRxWf38/MjIy8Pnnn8PHx2eq4zgVk8kEX19ffPbZZ4iJiUFaWho0Gg0+/fTTqY7m0Orq6lBQUIBPPvkE58+fR0VFBY4fP46dO3dOdTSyA1y5uUs+Pj5QKBTo6+sbM97X1wd/f/8J9/H397do/nRkTV1HFRUVobCwEKdOnUJUVNRkxnQ4lta1q6sLly9fxqpVq8xjJpMJAODm5oaOjg6oVKrJDe0ArHm9BgQEYMaMGVAoFOax8PBw6PV6DA8PQ6lUTmpmR2BNXd9++21kZGRgw4YNAIDIyEgMDg5i48aN0Gg0cHXl3+6WutN71qxZsxxq1Qbgys1dUyqViImJQW1trXnMZDKhtrYWsbGxE+4TGxs7Zj4A1NTU3HH+dGRNXQHg/fffx86dO3HixAksWLDAFlEdiqV1ffTRR9HW1oaWlhbz9vTTT2PZsmVoaWlBUFCQLePbLWter3Fxcejs7DQ3iwBw8eJFBAQEsLH5L2vqeuvWrXENzGgDKfzIRKs41XvWVF/R7EjKy8vF3d1dSktLpb29XTZu3CgPPPCA6PV6ERHJyMiQbdu2meefPn1a3NzcpKioSLRareTn5/NW8AlYWtfCwkJRKpVy5MgR6e3tNW/9/f1TdQh2ydK63o53S03M0rr29PSIl5eXqNVq6ejokGPHjomvr6/s2rVrqg7BLlla1/z8fPHy8pJDhw6JTqeT6upqUalUsmbNmqk6BLvT398vzc3N0tzcLABk79690tzcLFeuXBERkW3btklGRoZ5/uit4G+88YZotVrZt28fbwWfLoqLiyU4OFiUSqUsXLhQfvnlF/P3EhISJDMzc8z8r7/+WsLCwkSpVEpERIQcP37cxokdgyV1DQkJEQDjtvz8fNsHt3OWvl7/F5ubO7O0rg0NDbJo0SJxd3eX0NBQ2b17txgMBhuntn+W1HVkZER27NghKpVKPDw8JCgoSDZv3ix//fWX7YPbqZ9++mnC35WjdczMzJSEhIRx+zz22GOiVColNDRUSkpKbJ77XnAR4fodEREROQ9ec0NEREROhc0NERERORU2N0RERORU2NwQERGRU2FzQ0RERE6FzQ0RERE5FTY3RERE5FTY3BCRw8rKykJKSspUxyCi//r555+xatUqBAYGwsXFBd9++63FjyEiKCoqQlhYGNzd3TFnzhzs3r3bosfgB2cSkcP66KOP+DlCRHZkcHAQ8+fPR3Z2NlJTU616jFdeeQXV1dUoKipCZGQkrl+/juvXr1v0GPwPxURERHTPubi4oLKycszq6tDQEDQaDQ4dOoQbN25g3rx5eO+99/DEE08AALRaLaKionDhwgXMnTvX6ufmaSkisntHjhxBZGQkPD094e3tjcTERAwODo45LXX58mW4uLiM20Z/aQJAfX094uPj4enpiaCgIGzZsgWDg4NTc1BE05BarUZjYyPKy8vR2tqK5557DitWrMClS5cAAEePHkVoaCiOHTuGhx9+GA899BA2bNhg8coNmxsismu9vb1Yu3YtsrOzodVqUVdXh9TU1HGno4KCgtDb22vempub4e3tjaVLlwIAurq6sGLFCjz77LNobW3FV199hfr6eqjV6qk4LKJpp6enByUlJTh8+DDi4+OhUqnw+uuvY8mSJSgpKQEA6HQ6XLlyBYcPH8bBgwdRWlqKpqYmrF692qLn4jU3RGTXent7YTAYkJqaipCQEABAZGTkuHkKhQL+/v4AgH/++QcpKSmIjY3Fjh07AAB79uxBeno6Xn31VQDAI488go8//hgJCQnYv38/PDw8bHI8RNNVW1sbjEYjwsLCxowPDQ3B29sbAGAymTA0NISDBw+a5x04cAAxMTHo6Oi461NVbG6IyK7Nnz8fy5cvR2RkJJKSkvDUU09h9erVePDBB++4T3Z2Nvr7+1FTUwNX138XqH/77Te0trairKzMPE9EYDKZ0N3djfDw8Ek/FqLpbGBgAAqFAk1NTVAoFGO+N3PmTABAQEAA3NzcxjRAoz+bPT09bG6IyDkoFArU1NSgoaEB1dXVKC4uhkajwZkzZyacv2vXLpw8eRK//vorvLy8zOMDAwPYtGkTtmzZMm6f4ODgSctPRP+Kjo6G0WjEn3/+ifj4+AnnxMXFwWAwoKurCyqVCgBw8eJFADCv3N4N3i1FRA7FaDQiJCQEubm5aG1txY0bN8z/S+Obb77B2rVr8cMPP2D58uVj9ktPT0dfXx9OnTo1BamJpoeBgQF0dnYC+LeZ2bt3L5YtW4bZs2cjODgYzz//PE6fPo0PPvgA0dHRuHr1KmpraxEVFYXk5GSYTCY8/vjjmDlzJj788EOYTCbk5ORg1qxZqK6uvuscvKCYiOzamTNnUFBQgHPnzqGnpwcVFRW4evXquNNIFy5cwPr165GXl4eIiAjo9Xro9XrzXRZ5eXloaGiAWq1GS0sLLl26hO+++44XFBPdQ+fOnUN0dDSio6MBALm5uYiOjsY777wDACgpKcH69evx2muvYe7cuUhJScHZs2fNq6eurq44evQofHx8sHTpUiQnJyM8PBzl5eUW5eDKDRHZNa1Wi61bt+L8+fO4efMmQkJC8PLLL0OtViMrK8u8clNaWooXXnhh3P4JCQmoq6sDAJw9exYajQaNjY0QEahUKqSlpWH79u02PioimkxsboiIiMip8LQUERERORU2N0RERORU2NwQERGRU2FzQ0RERE6FzQ0RERE5FTY3RERE5FTY3BAREZFTYXNDREREToXNDRERETkVNjdERETkVNjcEBERkVNhc0NERERO5T90crk4nUn20gAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Define a simple implementation of `sigmoid`, and benchmark it using\n",
    "# NumPy and TensorFlow NumPy for different input sizes.\n",
    "\n",
    "def np_sigmoid(y):\n",
    "  return 1. / (1. + np.exp(-y))\n",
    "\n",
    "def tnp_sigmoid(y):\n",
    "  return 1. / (1. + tnp.exp(-y))\n",
    "\n",
    "@tf.function\n",
    "def compiled_tnp_sigmoid(y):\n",
    "  return tnp_sigmoid(y)\n",
    "\n",
    "sizes = (2 ** 0, 2 ** 5, 2 ** 10, 2 ** 15, 2 ** 20)\n",
    "np_inputs = [np.random.randn(size).astype(np.float32) for size in sizes]\n",
    "np_times = benchmark(np_sigmoid, np_inputs)\n",
    "\n",
    "with tf.device(\"/device:CPU:0\"):\n",
    "  tnp_inputs = [tnp.random.randn(size).astype(np.float32) for size in sizes]\n",
    "  tnp_times = benchmark(tnp_sigmoid, tnp_inputs)\n",
    "  compiled_tnp_times = benchmark(compiled_tnp_sigmoid, tnp_inputs)\n",
    "\n",
    "has_gpu = len(tf.config.list_logical_devices(\"GPU\"))\n",
    "if has_gpu:\n",
    "  with tf.device(\"/device:GPU:0\"):\n",
    "    tnp_inputs = [tnp.random.randn(size).astype(np.float32) for size in sizes]\n",
    "    tnp_times_gpu = benchmark(compiled_tnp_sigmoid, tnp_inputs, 100, True)\n",
    "else:\n",
    "  tnp_times_gpu = None\n",
    "plot(np_times, tnp_times, compiled_tnp_times, has_gpu, tnp_times_gpu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ReK_9k5D8BZQ"
   },
   "source": [
    "## 参考資料\n",
    "\n",
    "- TensorFlow NumPy: 分散型画像分類のチュートリアル\n",
    "- TensorFlow NumPy: Keras と分散ストラテジー\n",
    "- [Trax と TensorFlow NumPy を使用したセンチメント分析](https://github.com/google/trax/blob/master/trax/tf_numpy_and_keras.ipynb)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "name": "tf_numpy.ipynb",
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}