{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZjN_IJ8mhJ-4"
   },
   "source": [
    "##### Copyright 2020 The TensorFlow Authors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "cellView": "form",
    "execution": {
     "iopub.execute_input": "2022-12-14T22:41:59.062623Z",
     "iopub.status.busy": "2022-12-14T22:41:59.061879Z",
     "iopub.status.idle": "2022-12-14T22:41:59.065915Z",
     "shell.execute_reply": "2022-12-14T22:41:59.065346Z"
    },
    "id": "sY3Ffd83hK3b"
   },
   "outputs": [],
   "source": [
    "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "# https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "03Pw58e6mTHI"
   },
   "source": [
    "# TensorFlow 上的 NumPy API"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7WpGysDJmZsg"
   },
   "source": [
    "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
    "  <td><a target=\"_blank\" href=\"https://tensorflow.google.cn/guide/tf_numpy\"><img src=\"https://tensorflow.google.cn/images/tf_logo_32px.png\">在  TensorFlow.org 上查看</a></td>\n",
    "  <td><a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs-l10n/blob/master/site/zh-cn/guide/tf_numpy.ipynb\"><img src=\"https://tensorflow.google.cn/images/colab_logo_32px.png\">在 Google Colab 中运行 </a></td>\n",
    "  <td><a target=\"_blank\" href=\"https://github.com/tensorflow/docs-l10n/blob/master/site/zh-cn/guide/tf_numpy.ipynb\">     <img src=\"https://tensorflow.google.cn/images/GitHub-Mark-32px.png\">     在 GitHub 上查看源代码</a></td>\n",
    "  <td><a href=\"https://storage.googleapis.com/tensorflow_docs/docs-l10n/site/zh-cn/guide/tf_numpy.ipynb\"><img src=\"https://tensorflow.google.cn/images/download_logo_32px.png\">下载笔记本</a></td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "s2enCDi_FvCR"
   },
   "source": [
    "## 概述\n",
    "\n",
    "TensorFlow 实现了一部分 [NumPy API](https://numpy.org/doc/1.16)，这些 API 以 `tf.experimental.numpy` 形式提供。这样可以运行由 TensorFlow 加速的 NumPy 代码，并使用 TensorFlow 的所有 API。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ob1HNwUmYR5b"
   },
   "source": [
    "## 安装\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:41:59.069799Z",
     "iopub.status.busy": "2022-12-14T22:41:59.069236Z",
     "iopub.status.idle": "2022-12-14T22:42:01.327438Z",
     "shell.execute_reply": "2022-12-14T22:42:01.326732Z"
    },
    "id": "AJR558zjAZQu"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2022-12-14 22:42:00.330214: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory\n",
      "2022-12-14 22:42:00.330313: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory\n",
      "2022-12-14 22:42:00.330323: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using TensorFlow version 2.11.0\n"
     ]
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "import tensorflow.experimental.numpy as tnp\n",
    "import timeit\n",
    "\n",
    "print(\"Using TensorFlow version %s\" % tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "M6tacoy0DU6e"
   },
   "source": [
    "### 启用 NumPy 行为\n",
    "\n",
    "要将 `tnp` 用作 NumPy，请为 TensorFlow 启用 NumPy 行为："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:01.331439Z",
     "iopub.status.busy": "2022-12-14T22:42:01.330692Z",
     "iopub.status.idle": "2022-12-14T22:42:01.334319Z",
     "shell.execute_reply": "2022-12-14T22:42:01.333669Z"
    },
    "id": "TfCyofpFDQxm"
   },
   "outputs": [],
   "source": [
    "tnp.experimental_enable_numpy_behavior()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "et9D5wq0D1H2"
   },
   "source": [
    "此调用可在 TensorFlow 中启用类型提升，并在将文字转换为张量时更改类型推断，以更严格地遵循 NumPy 标准。\n",
    "\n",
    "注意：此调用将更改整个 TensorFlow 的行为，而不仅仅是 `tf.experimental.numpy` 模块。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yh2BwqUzH3C3"
   },
   "source": [
    "## TensorFlow NumPy ND 数组\n",
    "\n",
    "称为 **ND Array** 的实例 `tf.experimental.numpy.ndarray` 表示放置在特定设备上的给定 `dtype` 的多维密集数组。它是 `tf.Tensor` 的别名。请查看 ND 数组类来获取有用的方法，例如 `ndarray.T`、`ndarray.reshape`、`ndarray.ravel` 等。\n",
    "\n",
    "首先，创建一个 ND 数组对象，然后调用不同的方法。 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:01.338175Z",
     "iopub.status.busy": "2022-12-14T22:42:01.337517Z",
     "iopub.status.idle": "2022-12-14T22:42:04.695338Z",
     "shell.execute_reply": "2022-12-14T22:42:04.694607Z"
    },
    "id": "-BHJjxigJ2H1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Created ND array with shape = (5, 3), rank = 2, dtype = <dtype: 'float32'> on device = /job:localhost/replica:0/task:0/device:GPU:0\n",
      "\n",
      "Is `ones` an instance of tf.Tensor: True\n",
      "\n",
      "ndarray.T has shape (3, 5)\n",
      "narray.reshape(-1) has shape (15,)\n"
     ]
    }
   ],
   "source": [
    "# Create an ND array and check out different attributes.\n",
    "ones = tnp.ones([5, 3], dtype=tnp.float32)\n",
    "print(\"Created ND array with shape = %s, rank = %s, \"\n",
    "      \"dtype = %s on device = %s\\n\" % (\n",
    "          ones.shape, ones.ndim, ones.dtype, ones.device))\n",
    "\n",
    "# `ndarray` is just an alias to `tf.Tensor`.\n",
    "print(\"Is `ones` an instance of tf.Tensor: %s\\n\" % isinstance(ones, tf.Tensor))\n",
    "\n",
    "# Try commonly used member functions.\n",
    "print(\"ndarray.T has shape %s\" % str(ones.T.shape))\n",
    "print(\"narray.reshape(-1) has shape %s\" % ones.reshape(-1).shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Mub8-dvJMUr4"
   },
   "source": [
    "### 类型提升\n",
    "\n",
    "TensorFlow NumPy API 具有明确定义的语义，可用于将文字转换为 ND 数组，以及对 ND 数组输入执行类型提升。有关更多详细信息，请参阅 [`np.result_type`](https://numpy.org/doc/1.16/reference/generated/numpy.result_type.html)。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vcRznNaMj27J"
   },
   "source": [
    "TensorFlow API 保持 `tf.Tensor` 输入不变并且不对其执行类型提升，而 TensorFlow NumPy API 则根据 NumPy 类型提升规则来提升所有输入。在下一个示例中，您将执行类型提升。首先，对不同类型的 ND 数组输入运行加法并记下输出类型。TensorFlow API 不允许这些类型提升。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:04.698803Z",
     "iopub.status.busy": "2022-12-14T22:42:04.698287Z",
     "iopub.status.idle": "2022-12-14T22:42:04.713310Z",
     "shell.execute_reply": "2022-12-14T22:42:04.712736Z"
    },
    "id": "uHmBi4KZI2t1"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Type promotion for operations\n",
      "int32 + int64 => int64\n",
      "int32 + float32 => float64\n",
      "int32 + float64 => float64\n",
      "int64 + float32 => float64\n",
      "int64 + float64 => float64\n",
      "float32 + float64 => float64\n"
     ]
    }
   ],
   "source": [
    "print(\"Type promotion for operations\")\n",
    "values = [tnp.asarray(1, dtype=d) for d in\n",
    "          (tnp.int32, tnp.int64, tnp.float32, tnp.float64)]\n",
    "for i, v1 in enumerate(values):\n",
    "  for v2 in values[i + 1:]:\n",
    "    print(\"%s + %s => %s\" % \n",
    "          (v1.dtype.name, v2.dtype.name, (v1 + v2).dtype.name))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CrpIoOc7oqox"
   },
   "source": [
    "最后，使用 `ndarray.asarray` 将文字转换为 ND 数组，并记录结果类型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:04.716452Z",
     "iopub.status.busy": "2022-12-14T22:42:04.715990Z",
     "iopub.status.idle": "2022-12-14T22:42:04.720364Z",
     "shell.execute_reply": "2022-12-14T22:42:04.719771Z"
    },
    "id": "1m1cp8_VooNk"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Type inference during array creation\n",
      "tnp.asarray(1).dtype == tnp.int64\n",
      "tnp.asarray(1.).dtype == tnp.float64\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(\"Type inference during array creation\")\n",
    "print(\"tnp.asarray(1).dtype == tnp.%s\" % tnp.asarray(1).dtype.name)\n",
    "print(\"tnp.asarray(1.).dtype == tnp.%s\\n\" % tnp.asarray(1.).dtype.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kd-_iccXoRL8"
   },
   "source": [
    "将文字转换为 ND 数组时，NumPy 倾向于使用 `tnp.int64` 和 `tnp.float64` 之类的宽类型。相反，`tf.convert_to_tensor` 则倾向于使用 `tf.int32` 和 `tf.float32` 类型将常量转换为 `tf.Tensor`。TensorFlow NumPy API 遵循整数的 NumPy 行为。对于浮点数，`experimental_enable_numpy_behavior` 的 `prefer_float32` 参数可让您控制倾向于使用 `tf.float32` 而不是 `tf.float64`（默认为 `False`）。例如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:04.723379Z",
     "iopub.status.busy": "2022-12-14T22:42:04.722916Z",
     "iopub.status.idle": "2022-12-14T22:42:04.729539Z",
     "shell.execute_reply": "2022-12-14T22:42:04.728972Z"
    },
    "id": "4gKasnH0j84C"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "When prefer_float32 is True:\n",
      "tnp.asarray(1.).dtype == tnp.float32\n",
      "tnp.add(1., 2.).dtype == tnp.float32\n",
      "When prefer_float32 is False:\n",
      "tnp.asarray(1.).dtype == tnp.float64\n",
      "tnp.add(1., 2.).dtype == tnp.float64\n"
     ]
    }
   ],
   "source": [
    "tnp.experimental_enable_numpy_behavior(prefer_float32=True)\n",
    "print(\"When prefer_float32 is True:\")\n",
    "print(\"tnp.asarray(1.).dtype == tnp.%s\" % tnp.asarray(1.).dtype.name)\n",
    "print(\"tnp.add(1., 2.).dtype == tnp.%s\" % tnp.add(1., 2.).dtype.name)\n",
    "\n",
    "tnp.experimental_enable_numpy_behavior(prefer_float32=False)\n",
    "print(\"When prefer_float32 is False:\")\n",
    "print(\"tnp.asarray(1.).dtype == tnp.%s\" % tnp.asarray(1.).dtype.name)\n",
    "print(\"tnp.add(1., 2.).dtype == tnp.%s\" % tnp.add(1., 2.).dtype.name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MwCCDxSZOfA1"
   },
   "source": [
    "### 广播\n",
    "\n",
    "与 TensorFlow 类似，NumPy 为“广播”值定义了丰富的语义。您可以查看 [NumPy 广播指南](https://numpy.org/doc/stable/user/basics.broadcasting.html)了解更多信息，还可将其与 [TensorFlow 广播语义](https://tensorflow.google.cn/guide/tensor#broadcasting)进行比较。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:04.732609Z",
     "iopub.status.busy": "2022-12-14T22:42:04.732089Z",
     "iopub.status.idle": "2022-12-14T22:42:04.738285Z",
     "shell.execute_reply": "2022-12-14T22:42:04.737735Z"
    },
    "id": "qlyOShxIO0s2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Broadcasting shapes (2, 3), (3,) and (1, 2, 1) gives shape (1, 2, 3)\n"
     ]
    }
   ],
   "source": [
    "x = tnp.ones([2, 3])\n",
    "y = tnp.ones([3])\n",
    "z = tnp.ones([1, 2, 1])\n",
    "print(\"Broadcasting shapes %s, %s and %s gives shape %s\" % (\n",
    "    x.shape, y.shape, z.shape, (x + y + z).shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LEVr4ctRPrqR"
   },
   "source": [
    "### 索引\n",
    "\n",
    "NumPy 定义了非常复杂的索引规则。请参阅 [NumPy 索引指南](https://numpy.org/doc/1.16/reference/arrays.indexing.html)。请注意，下面的示例将 ND 数组用作索引。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:04.741407Z",
     "iopub.status.busy": "2022-12-14T22:42:04.740960Z",
     "iopub.status.idle": "2022-12-14T22:42:04.790245Z",
     "shell.execute_reply": "2022-12-14T22:42:04.789628Z"
    },
    "id": "lRsrtnd3YyMj"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Basic indexing\n",
      "tf.Tensor(\n",
      "[[[16 17 18 19]\n",
      "  [20 21 22 23]]], shape=(1, 2, 4), dtype=int64) \n",
      "\n",
      "Boolean indexing\n",
      "tf.Tensor(\n",
      "[[[ 0  1  2  3]\n",
      "  [ 8  9 10 11]]\n",
      "\n",
      " [[12 13 14 15]\n",
      "  [20 21 22 23]]], shape=(2, 2, 4), dtype=int64) \n",
      "\n",
      "Advanced indexing\n",
      "tf.Tensor([12 13 17], shape=(3,), dtype=int64)\n"
     ]
    }
   ],
   "source": [
    "x = tnp.arange(24).reshape(2, 3, 4)\n",
    "\n",
    "print(\"Basic indexing\")\n",
    "print(x[1, tnp.newaxis, 1:3, ...], \"\\n\")\n",
    "\n",
    "print(\"Boolean indexing\")\n",
    "print(x[:, (True, False, True)], \"\\n\")\n",
    "\n",
    "print(\"Advanced indexing\")\n",
    "print(x[1, (0, 0, 1), tnp.asarray([0, 1, 1])])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:04.793383Z",
     "iopub.status.busy": "2022-12-14T22:42:04.792879Z",
     "iopub.status.idle": "2022-12-14T22:42:04.797184Z",
     "shell.execute_reply": "2022-12-14T22:42:04.796532Z"
    },
    "id": "yRAaiGhlaNw7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Currently, TensorFlow NumPy does not support mutation.\n"
     ]
    }
   ],
   "source": [
    "# Mutation is currently not supported\n",
    "try:\n",
    "  tnp.arange(6)[1] = -1\n",
    "except TypeError:\n",
    "  print(\"Currently, TensorFlow NumPy does not support mutation.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5XfJ602j-GVD"
   },
   "source": [
    "### 示例模型\n",
    "\n",
    "接下来，您将看到如何创建一个模型并在该模型上运行推断。此简单模型应用 ReLU 层，随后是线性投影。后面的部分将展示如何使用 TensorFlow 的 `GradientTape` 计算此模型的梯度。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:04.800362Z",
     "iopub.status.busy": "2022-12-14T22:42:04.799819Z",
     "iopub.status.idle": "2022-12-14T22:42:05.136750Z",
     "shell.execute_reply": "2022-12-14T22:42:05.135909Z"
    },
    "id": "kR_KCh4kYEhm"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor(\n",
      "[[0.12651756 0.06182817]\n",
      " [0.12651756 0.06182817]], shape=(2, 2), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "class Model(object):\n",
    "  \"\"\"Model with a dense and a linear layer.\"\"\"\n",
    "\n",
    "  def __init__(self):\n",
    "    self.weights = None\n",
    "\n",
    "  def predict(self, inputs):\n",
    "    if self.weights is None:\n",
    "      size = inputs.shape[1]\n",
    "      # Note that type `tnp.float32` is used for performance.\n",
    "      stddev = tnp.sqrt(size).astype(tnp.float32)\n",
    "      w1 = tnp.random.randn(size, 64).astype(tnp.float32) / stddev\n",
    "      bias = tnp.random.randn(64).astype(tnp.float32)\n",
    "      w2 = tnp.random.randn(64, 2).astype(tnp.float32) / 8\n",
    "      self.weights = (w1, bias, w2)\n",
    "    else:\n",
    "      w1, bias, w2 = self.weights\n",
    "    y = tnp.matmul(inputs, w1) + bias\n",
    "    y = tnp.maximum(y, 0)  # Relu\n",
    "    return tnp.matmul(y, w2)  # Linear projection\n",
    "\n",
    "model = Model()\n",
    "# Create input data and compute predictions.\n",
    "print(model.predict(tnp.ones([2, 32], dtype=tnp.float32)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kSR7Ou5YcS38"
   },
   "source": [
    "## TensorFlow NumPy 和 NumPy\n",
    "\n",
    "TensorFlow NumPy 实现了完整 NumPy 规范的子集。尽管随着时间的推移会添加更多符号，但一些系统功能在不久的将来将不再受支持。这些功能包括 NumPy C API 支持、Swig 集成、Fortran 存储顺序、视图和 `stride_tricks` 以及一些 `dtype`（例如 `np.recarray`、`np.object`）。有关更多详细信息，请参阅 [TensorFlow NumPy API 文档](https://tensorflow.google.cn/api_docs/python/tf/experimental/numpy)。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Jb1KXak2YlNN"
   },
   "source": [
    "### NumPy 互操作性\n",
    "\n",
    "TensorFlow ND 数组可与 NumPy 函数互操作。这些对象实现了 `__array__` 接口。NumPy 使用此接口先将函数参数转换为 `np.ndarray` 值，然后再对它们进行处理。\n",
    "\n",
    "同样，TensorFlow NumPy 函数可以接受不同类型的输入，包括 `np.ndarray`。通过在它们上面调用 `ndarray.asarray` 可将这些输入转换为 ND 数组。\n",
    "\n",
    "ND 数组与 `np.ndarray` 之间的转换可能会触发实际数据副本。有关更多详细信息，请参阅关于[缓冲区副本](#Buffer-copies)的部分。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:05.140260Z",
     "iopub.status.busy": "2022-12-14T22:42:05.139993Z",
     "iopub.status.idle": "2022-12-14T22:42:05.147447Z",
     "shell.execute_reply": "2022-12-14T22:42:05.146790Z"
    },
    "id": "cMOCgzQmeXRU"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sum = 6.0. Class: <class 'numpy.float64'>\n",
      "sum = 6.0. Class: <class 'tensorflow.python.framework.ops.EagerTensor'>\n"
     ]
    }
   ],
   "source": [
    "# ND array passed into NumPy function.\n",
    "np_sum = np.sum(tnp.ones([2, 3]))\n",
    "print(\"sum = %s. Class: %s\" % (float(np_sum), np_sum.__class__))\n",
    "\n",
    "# `np.ndarray` passed into TensorFlow NumPy function.\n",
    "tnp_sum = tnp.sum(np.ones([2, 3]))\n",
    "print(\"sum = %s. Class: %s\" % (float(tnp_sum), tnp_sum.__class__))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:05.150701Z",
     "iopub.status.busy": "2022-12-14T22:42:05.150095Z",
     "iopub.status.idle": "2022-12-14T22:42:05.290416Z",
     "shell.execute_reply": "2022-12-14T22:42:05.289843Z"
    },
    "id": "ZaLPjzxft780"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAikAAAGdCAYAAADXIOPgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAhEklEQVR4nO3df1BVdf7H8Re/ReNeFhUudwWkXyL+XnORcs1WBlDWMmk2WzMrR6cWbJHWlErLapdy3XJzTWdn2twmacuZNMVdyvwBOaEVrmuakTqUtnihcuSKJqKc7x/79bY3LUSB8wGej5k74z3ncO/7foaBp+f+IMCyLEsAAACGCbR7AAAAgAshUgAAgJGIFAAAYCQiBQAAGIlIAQAARiJSAACAkYgUAABgJCIFAAAYKdjuAS5FU1OTqqurFRERoYCAALvHAQAAF8GyLB0/flxut1uBgc2fJ+mQkVJdXa24uDi7xwAAAJfg8OHD6tOnT7PHdchIiYiIkPTfB+lwOGyeBgAAXAyv16u4uDjf7/HmdMhIOfcUj8PhIFIAAOhgLvalGrxwFgAAGIlIAQAARiJSAACAkYgUAABgJCIFAAAYiUgBAABGIlIAAICRiBQAAGAkIgUAABiJSAEAAEYiUgAAgJGIFAAAYCQiBQAAGIlIAQAARgq2ewAA+K6+8zbYPUKLffZ0lt0jAJ0OZ1IAAICRiBQAAGAkIgUAABiJSAEAAEYiUgAAgJF4dw/QyXXEd8oAgNTCMymFhYUaMWKEIiIiFB0drYkTJ6qystLvmDFjxiggIMDvct999/kdc+jQIWVlZal79+6Kjo7WnDlzdObMmct/NAAAoNNo0ZmU0tJS5eTkaMSIETpz5owefvhhpaen6+OPP1aPHj18x82YMUNPPPGE73r37t19/z579qyysrLkcrn03nvv6ciRI7rrrrsUEhKi3//+963wkAAAQGfQokgpKSnxu75y5UpFR0eroqJCo0eP9m3v3r27XC7XBW/j7bff1scff6x33nlHMTExGjp0qJ588knNnTtXjz/+uEJDQy/hYQAAgM7msl44W1dXJ0mKiory275q1Sr16tVLAwcOVEFBgU6ePOnbV15erkGDBikmJsa3LSMjQ16vV3v37r3g/TQ0NMjr9fpdAABA53bJL5xtampSXl6ebrjhBg0cONC3/Ve/+pUSEhLkdru1e/duzZ07V5WVlXrjjTckSR6Pxy9QJPmuezyeC95XYWGhFi5ceKmjAgCADuiSIyUnJ0d79uzRtm3b/LbPnDnT9+9BgwYpNjZWY8eO1cGDB3XVVVdd0n0VFBQoPz/fd93r9SouLu7SBgcAAB3CJT3dk5ubq+LiYm3ZskV9+vT5wWNTUlIkSQcOHJAkuVwu1dTU+B1z7vr3vY4lLCxMDofD7wIAADq3FkWKZVnKzc3VmjVrtHnzZiUmJjb7Nbt27ZIkxcbGSpJSU1P10Ucfqba21nfMxo0b5XA4lJyc3JJxAABAJ9aip3tycnJUVFSkN998UxEREb7XkDidToWHh+vgwYMqKirS+PHj1bNnT+3evVuzZ8/W6NGjNXjwYElSenq6kpOTNXXqVC1atEgej0ePPvqocnJyFBYW1vqPEAAAdEgtOpOyfPly1dXVacyYMYqNjfVdXnvtNUlSaGio3nnnHaWnpyspKUkPPvigsrOztX79et9tBAUFqbi4WEFBQUpNTdWdd96pu+66y+9zVQAAAFp0JsWyrB/cHxcXp9LS0mZvJyEhQf/4xz9actcAAKCL4Q8MAgAAIxEpAADASEQKAAAwEpECAACMRKQAAAAjESkAAMBIRAoAADASkQIAAIxEpAAAACMRKQAAwEhECgAAMBKRAgAAjESkAAAAIxEpAADASEQKAAAwEpECAACMRKQAAAAjESkAAMBIRAoAADASkQIAAIxEpAAAACMRKQAAwEhECgAAMBKRAgAAjESkAAAAIxEpAADASEQKAAAwEpECAACMRKQAAAAjESkAAMBIRAoAADASkQIAAIxEpAAAACMRKQAAwEhECgAAMBKRAgAAjESkAAAAIxEpAADASEQKAAAwEpECAACMRKQAAAAjESkAAMBIRAoAADASkQIAAIxEpAAAACMRKQAAwEhECgAAMBKRAgAAjESkAAAAIxEpAADASEQKAAAwEpECAACMRKQAAAAjESkAAMBIRAoAADASkQIAAIxEpAAAACO1KFIKCws1YsQIRUREKDo6WhMnTlRlZaXfMadOnVJOTo569uypK664QtnZ2aqpqfE75tChQ8rKylL37t0VHR2tOXPm6MyZM5f/aAAAQKfRokgpLS1VTk6Otm/fro0bN6qxsVHp6ek6ceKE75jZs2dr/fr1Wr16tUpLS1VdXa1Jkyb59p89e1ZZWVk6ffq03nvvPf3tb3/TypUrtWDBgtZ7VAAAoMMLsCzLutQv/vLLLxUdHa3S0lKNHj1adXV16t27t4qKinTbbbdJkj755BP1799f5eXlGjlypP75z3/qF7/4haqrqxUTEyNJWrFihebOnasvv/xSoaGhzd6v1+uV0+lUXV2dHA7HpY4PdAl9522we4Qu4bOns+weATBeS39/X9ZrUurq6iRJUVFRkqSKigo1NjYqLS3Nd0xSUpLi4+NVXl4uSSovL9egQYN8gSJJGRkZ8nq92rt37wXvp6GhQV6v1+8CAAA6t0uOlKamJuXl5emGG27QwIEDJUkej0ehoaGKjIz0OzYmJkYej8d3zP8Gyrn95/ZdSGFhoZxOp+8SFxd3qWMDAIAO4pIjJScnR3v27NHf//731pznggoKClRXV+e7HD58uM3vEwAA2Cv4Ur4oNzdXxcXFKisrU58+fXzbXS6XTp8+rWPHjvmdTampqZHL5fId8/777/vd3rl3/5w75rvCwsIUFhZ2KaMCAIAOqkVnUizLUm5urtasWaPNmzcrMTHRb//w4cMVEhKiTZs2+bZVVlbq0KFDSk1NlSSlpqbqo48+Um1tre+YjRs3yuFwKDk5+XIeCwAA6ERadCYlJydHRUVFevPNNxUREeF7DYnT6VR4eLicTqemT5+u/Px8RUVFyeFwaNasWUpNTdXIkSMlSenp6UpOTtbUqVO1aNEieTwePfroo8rJyeFsCQAA8GlRpCxfvlySNGbMGL/tL730ku6++25J0nPPPafAwEBlZ2eroaFBGRkZeuGFF3zHBgUFqbi4WPfff79SU1PVo0cPTZs2TU888cTlPRIAANCpXNbnpNiFz0kBLh6fk9I++JwUoHnt+jkpAAAAbYVIAQAARiJSAACAkYgUAABgJCIFAAAYiUgBAABGIlIAAICRiBQAAGAkIgUAABiJSAEAAEYiUgAAgJGIFAAAYCQiBQAAGIlIAQAARiJSAACAkYgUAABgJCIFAAAYiUgBAABGIlIAAICRiBQAAGAkIgUAABiJSAEAAEYiUgAAgJGIFAAAYCQiBQAAGIlIAQAARiJSAACAkYgUAABgJCIFAAAYiUgBAABGIlIAAICRiBQAAGAkIgUAABiJSAEAAEYiUgAAgJGIFAAAYKRguwcAgM6g77wNdo/QYp89nWX3CMAP4kwKAAAwEpECAACMRKQAAAAjESkAAMBIRAoAADASkQIAAIxEpAAAACMRKQAAwEhECgAAMBKRAgAAjESkAAAAIxEpAADASEQKAAAwEpECAACMRKQAAAAjESkAAMBIRAoAADASkQIAAIxEpAAAACMRKQAAwEgtjpSysjJNmDBBbrdbAQEBWrt2rd/+u+++WwEBAX6XzMxMv2OOHj2qKVOmyOFwKDIyUtOnT1d9ff1lPRAAANC5tDhSTpw4oSFDhmjZsmXfe0xmZqaOHDniu7z66qt++6dMmaK9e/dq48aNKi4uVllZmWbOnNny6QEAQKcV3NIvGDdunMaNG/eDx4SFhcnlcl1w3759+1RSUqIPPvhA1113nSRp6dKlGj9+vBYvXiy3293SkQAAQCfUJq9J2bp1q6Kjo9WvXz/df//9+vrrr337ysvLFRkZ6QsUSUpLS1NgYKB27NjRFuMAAIAOqMVnUpqTmZmpSZMmKTExUQcPHtTDDz+scePGqby8XEFBQfJ4PIqOjvYfIjhYUVFR8ng8F7zNhoYGNTQ0+K57vd7WHhsAABim1SNl8uTJvn8PGjRIgwcP1lVXXaWtW7dq7Nixl3SbhYWFWrhwYWuNCAAAOoA2fwvylVdeqV69eunAgQOSJJfLpdraWr9jzpw5o6NHj37v61gKCgpUV1fnuxw+fLitxwYAADZr80j54osv9PXXXys2NlaSlJqaqmPHjqmiosJ3zObNm9XU1KSUlJQL3kZYWJgcDoffBQAAdG4tfrqnvr7ed1ZEkqqqqrRr1y5FRUUpKipKCxcuVHZ2tlwulw4ePKiHHnpIV199tTIyMiRJ/fv3V2ZmpmbMmKEVK1aosbFRubm5mjx5Mu/sAQAAPi0+k/Lhhx9q2LBhGjZsmCQpPz9fw4YN04IFCxQUFKTdu3fr5ptv1rXXXqvp06dr+PDhevfddxUWFua7jVWrVikpKUljx47V+PHjNWrUKP3lL39pvUcFAAA6vBafSRkzZowsy/re/W+99VaztxEVFaWioqKW3jUAAOhC+Ns9AADASEQKAAAwEpECAACM1Oof5gZ0Zn3nbbB7BADoMjiTAgAAjESkAAAAIxEpAADASEQKAAAwEpECAACMRKQAAAAjESkAAMBIRAoAADASkQIAAIxEpAAAACMRKQAAwEhECgAAMBKRAgAAjESkAAAAIxEpAADASEQKAAAwEpECAACMRKQAAAAjESkAAMBIRAoAADASkQIAAIxEpAAAACMRKQAAwEhECgAAMBKRAgAAjESkAAAAIxEpAADASEQKAAAwEpECAACMRKQAAAAjESkAAMBIRAoAADASkQIAAIxEpAAAACMRKQAAwEhECgAAMBKRAgAAjESkAAAAIxEpAADASEQKAAAwEpECAACMRKQAAAAjESkAAMBIRAoAADASkQIAAIxEpAAAACMRKQAAwEhECgAAMBKRAgAAjESkAAAAIxEpAADASEQKAAAwEpECAACMRKQAAAAjtThSysrKNGHCBLndbgUEBGjt2rV++y3L0oIFCxQbG6vw8HClpaVp//79fsccPXpUU6ZMkcPhUGRkpKZPn676+vrLeiAAAKBzaXGknDhxQkOGDNGyZcsuuH/RokV6/vnntWLFCu3YsUM9evRQRkaGTp065TtmypQp2rt3rzZu3Kji4mKVlZVp5syZl/4oAABApxPc0i8YN26cxo0bd8F9lmVpyZIlevTRR3XLLbdIkl5++WXFxMRo7dq1mjx5svbt26eSkhJ98MEHuu666yRJS5cu1fjx47V48WK53e7LeDgAAKCzaNXXpFRVVcnj8SgtLc23zel0KiUlReXl5ZKk8vJyRUZG+gJFktLS0hQYGKgdO3Zc8HYbGhrk9Xr9LgAAoHNr1UjxeDySpJiYGL/tMTExvn0ej0fR0dF++4ODgxUVFeU75rsKCwvldDp9l7i4uNYcGwAAGKhDvLunoKBAdXV1vsvhw4ftHgkAALSxVo0Ul8slSaqpqfHbXlNT49vncrlUW1vrt//MmTM6evSo75jvCgsLk8Ph8LsAAIDOrVUjJTExUS6XS5s2bfJt83q92rFjh1JTUyVJqampOnbsmCoqKnzHbN68WU1NTUpJSWnNcQAAQAfW4nf31NfX68CBA77rVVVV2rVrl6KiohQfH6+8vDw99dRTuuaaa5SYmKj58+fL7XZr4sSJkqT+/fsrMzNTM2bM0IoVK9TY2Kjc3FxNnjyZd/YAAACfFkfKhx9+qJtuusl3PT8/X5I0bdo0rVy5Ug899JBOnDihmTNn6tixYxo1apRKSkrUrVs339esWrVKubm5Gjt2rAIDA5Wdna3nn3++FR4OAADoLAIsy7LsHqKlvF6vnE6n6urqeH0K2lXfeRvsHgFoNZ89nWX3COhiWvr7u0O8uwcAAHQ9RAoAADASkQIAAIxEpAAAACMRKQAAwEhECgAAMBKRAgAAjESkAAAAIxEpAADASEQKAAAwEpECAACMRKQAAAAjESkAAMBIRAoAADASkQIAAIxEpAAAACMRKQAAwEhECgAAMBKRAgAAjESkAAAAIxEpAADASEQKAAAwEpECAACMRKQAAAAjESkAAMBIRAoAADASkQIAAIxEpAAAACMF2z0AAMAefedtsHuEFvvs6Sy7R0A74kwKAAAwEpECAACMRKQAAAAjESkAAMBIRAoAADASkQIAAIxEpAAAACMRKQAAwEhECgAAMBKRAgAAjESkAAAAIxEpAADASEQKAAAwEpECAACMRKQAAAAjESkAAMBIRAoAADASkQIAAIxEpAAAACMRKQAAwEjBdg+ArqvvvA12jwAAMBhnUgAAgJGIFAAAYCQiBQAAGIlIAQAARiJSAACAkYgUAABgJCIFAAAYqdUj5fHHH1dAQIDfJSkpybf/1KlTysnJUc+ePXXFFVcoOztbNTU1rT0GAADo4NrkTMqAAQN05MgR32Xbtm2+fbNnz9b69eu1evVqlZaWqrq6WpMmTWqLMQAAQAfWJp84GxwcLJfLdd72uro6vfjiiyoqKtLPf/5zSdJLL72k/v37a/v27Ro5cmRbjAMAADqgNjmTsn//frndbl155ZWaMmWKDh06JEmqqKhQY2Oj0tLSfMcmJSUpPj5e5eXl33t7DQ0N8nq9fhcAANC5tXqkpKSkaOXKlSopKdHy5ctVVVWln/3sZzp+/Lg8Ho9CQ0MVGRnp9zUxMTHyeDzfe5uFhYVyOp2+S1xcXGuPDQAADNPqT/eMGzfO9+/BgwcrJSVFCQkJev311xUeHn5Jt1lQUKD8/Hzfda/XS6gAANDJtflbkCMjI3XttdfqwIEDcrlcOn36tI4dO+Z3TE1NzQVfw3JOWFiYHA6H3wUAAHRubR4p9fX1OnjwoGJjYzV8+HCFhIRo06ZNvv2VlZU6dOiQUlNT23oUAADQgbT60z2//e1vNWHCBCUkJKi6ulqPPfaYgoKCdMcdd8jpdGr69OnKz89XVFSUHA6HZs2apdTUVN7ZAwAA/LR6pHzxxRe644479PXXX6t3794aNWqUtm/frt69e0uSnnvuOQUGBio7O1sNDQ3KyMjQCy+80NpjAACADi7AsizL7iFayuv1yul0qq6ujtendGB9522wewQAHcxnT2fZPQIuQ0t/f/O3ewAAgJGIFAAAYCQiBQAAGIlIAQAARiJSAACAkYgUAABgJCIFAAAYiUgBAABGIlIAAICRiBQAAGAkIgUAABiJSAEAAEYiUgAAgJGIFAAAYCQiBQAAGIlIAQAARiJSAACAkYgUAABgJCIFAAAYiUgBAABGIlIAAICRiBQAAGAkIgUAABiJSAEAAEYiUgAAgJGIFAAAYCQiBQAAGIlIAQAARiJSAACAkYgUAABgJCIFAAAYiUgBAABGIlIAAICRiBQAAGAkIgUAABiJSAEAAEYiUgAAgJGC7R4AAICL1XfeBrtHaLHPns6ye4QOizMpAADASJxJ6SQ64v8uAAD4IZxJAQAARiJSAACAkYgUAABgJCIFAAAYiUgBAABGIlIAAICRiBQAAGAkIgUAABiJSAEAAEYiUgAAgJGIFAAAYCQiBQAAGIlIAQAARiJSAACAkYLtHgAAgM6s77wNdo9wST57OsvuETiTAgAAzMSZlAvoqNULAEBnYuuZlGXLlqlv377q1q2bUlJS9P7779s5DgAAMIhtkfLaa68pPz9fjz32mHbu3KkhQ4YoIyNDtbW1do0EAAAMYlukPPvss5oxY4buueceJScna8WKFerevbv++te/2jUSAAAwiC2vSTl9+rQqKipUUFDg2xYYGKi0tDSVl5efd3xDQ4MaGhp81+vq6iRJXq+3TeZrajjZJrcLAEBH0Ra/Y8/dpmVZF3W8LZHy1Vdf6ezZs4qJifHbHhMTo08++eS84wsLC7Vw4cLztsfFxbXZjAAAdGXOJW1328ePH5fT6Wz2uA7x7p6CggLl5+f7rjc1Neno0aPq2bOnAgICWnRbXq9XcXFxOnz4sBwOR2uP2uGwHv5Yj2+xFv5Yj2+xFv5Yj281txaWZen48eNyu90XdXu2REqvXr0UFBSkmpoav+01NTVyuVznHR8WFqawsDC/bZGRkZc1g8Ph6PLfTP+L9fDHenyLtfDHenyLtfDHenzrh9biYs6gnGPLC2dDQ0M1fPhwbdq0ybetqalJmzZtUmpqqh0jAQAAw9j2dE9+fr6mTZum6667Tj/96U+1ZMkSnThxQvfcc49dIwEAAIPYFim33367vvzySy1YsEAej0dDhw5VSUnJeS+mbW1hYWF67LHHznv6qKtiPfyxHt9iLfyxHt9iLfyxHt9q7bUIsC72fUAAAADtiD8wCAAAjESkAAAAIxEpAADASEQKAAAwUpeJlLNnz2r+/PlKTExUeHi4rrrqKj355JMX/fcDOrqysjJNmDBBbrdbAQEBWrt2rd9+y7K0YMECxcbGKjw8XGlpadq/f789w7axH1qLxsZGzZ07V4MGDVKPHj3kdrt11113qbq62r6B21hz3xv/67777lNAQICWLFnSbvO1p4tZi3379unmm2+W0+lUjx49NGLECB06dKj9h20Hza1HfX29cnNz1adPH4WHh/v+WGxnVFhYqBEjRigiIkLR0dGaOHGiKisr/Y45deqUcnJy1LNnT11xxRXKzs4+70NLO4vm1uPo0aOaNWuW+vXrp/DwcMXHx+uBBx7w/e29i9VlIuWZZ57R8uXL9ec//1n79u3TM888o0WLFmnp0qV2j9YuTpw4oSFDhmjZsmUX3L9o0SI9//zzWrFihXbs2KEePXooIyNDp06daudJ294PrcXJkye1c+dOzZ8/Xzt37tQbb7yhyspK3XzzzTZM2j6a+944Z82aNdq+fftFf5x1R9TcWhw8eFCjRo1SUlKStm7dqt27d2v+/Pnq1q1bO0/aPppbj/z8fJWUlOiVV17Rvn37lJeXp9zcXK1bt66dJ217paWlysnJ0fbt27Vx40Y1NjYqPT1dJ06c8B0ze/ZsrV+/XqtXr1Zpaamqq6s1adIkG6duO82tR3V1taqrq7V48WLt2bNHK1euVElJiaZPn96yO7K6iKysLOvee+/12zZp0iRrypQpNk1kH0nWmjVrfNebmposl8tl/eEPf/BtO3bsmBUWFma9+uqrNkzYfr67Fhfy/vvvW5Kszz//vH2GstH3rccXX3xh/fjHP7b27NljJSQkWM8991y7z9beLrQWt99+u3XnnXfaM5DNLrQeAwYMsJ544gm/bT/5yU+sRx55pB0ns0dtba0lySotLbUs678/M0NCQqzVq1f7jtm3b58lySovL7drzHbz3fW4kNdff90KDQ21GhsbL/p2u8yZlOuvv16bNm3Sp59+Kkn697//rW3btmncuHE2T2a/qqoqeTwepaWl+bY5nU6lpKSovLzcxsnMUFdXp4CAgMv+e1EdVVNTk6ZOnao5c+ZowIABdo9jm6amJm3YsEHXXnutMjIyFB0drZSUlB98eqyzu/7667Vu3Tr95z//kWVZ2rJliz799FOlp6fbPVqbO/e0RVRUlCSpoqJCjY2Nfj9Hk5KSFB8f3yV+jn53Pb7vGIfDoeDgi/8c2S4TKfPmzdPkyZOVlJSkkJAQDRs2THl5eZoyZYrdo9nO4/FI0nmf9hsTE+Pb11WdOnVKc+fO1R133NFl/3DYM888o+DgYD3wwAN2j2Kr2tpa1dfX6+mnn1ZmZqbefvtt3XrrrZo0aZJKS0vtHs8WS5cuVXJysvr06aPQ0FBlZmZq2bJlGj16tN2jtammpibl5eXphhtu0MCBAyX99+doaGjoef+Z6Qo/Ry+0Ht/11Vdf6cknn9TMmTNbdNu2fSx+e3v99de1atUqFRUVacCAAdq1a5fy8vLkdrs1bdo0u8eDgRobG/XLX/5SlmVp+fLldo9ji4qKCv3pT3/Szp07FRAQYPc4tmpqapIk3XLLLZo9e7YkaejQoXrvvfe0YsUK3XjjjXaOZ4ulS5dq+/btWrdunRISElRWVqacnBy53W6/MwqdTU5Ojvbs2aNt27bZPYoRmlsPr9errKwsJScn6/HHH2/RbXeZSJkzZ47vbIokDRo0SJ9//rkKCwu7fKS4XC5JUk1NjWJjY33ba2pqNHToUJumste5QPn888+1efPmLnsW5d1331Vtba3i4+N9286ePasHH3xQS5Ys0WeffWbfcO2sV69eCg4OVnJyst/2/v37d8lfVt98840efvhhrVmzRllZWZKkwYMHa9euXVq8eHGnjZTc3FwVFxerrKxMffr08W13uVw6ffq0jh075nc2paamxvcztjP6vvU45/jx48rMzFRERITWrFmjkJCQFt1+l3m65+TJkwoM9H+4QUFBvv8ddWWJiYlyuVzatGmTb5vX69WOHTuUmppq42T2OBco+/fv1zvvvKOePXvaPZJtpk6dqt27d2vXrl2+i9vt1pw5c/TWW2/ZPV67Cg0N1YgRI8572+mnn36qhIQEm6ayT2NjoxobG7vMz1XLspSbm6s1a9Zo8+bNSkxM9Ns/fPhwhYSE+P0crays1KFDhzrlz9Hm1kP67++R9PR0hYaGat26dZf0LrgucyZlwoQJ+t3vfqf4+HgNGDBA//rXv/Tss8/q3nvvtXu0dlFfX68DBw74rldVVWnXrl2KiopSfHy88vLy9NRTT+maa65RYmKi5s+fL7fbrYkTJ9o3dBv5obWIjY3Vbbfdpp07d6q4uFhnz571PZ8cFRWl0NBQu8ZuM819b3w30kJCQuRyudSvX7/2HrXNNbcWc+bM0e23367Ro0frpptuUklJidavX6+tW7faN3Qbam49brzxRs2ZM0fh4eFKSEhQaWmpXn75ZT377LM2Tt02cnJyVFRUpDfffFMRERG+nwtOp1Ph4eFyOp2aPn268vPzFRUVJYfDoVmzZik1NVUjR460efrW19x6nAuUkydP6pVXXpHX65XX65Uk9e7dW0FBQRd3R5f7tqOOwuv1Wr/5zW+s+Ph4q1u3btaVV15pPfLII1ZDQ4Pdo7WLLVu2WJLOu0ybNs2yrP++DXn+/PlWTEyMFRYWZo0dO9aqrKy0d+g28kNrUVVVdcF9kqwtW7bYPXqbaO5747s681uQL2YtXnzxRevqq6+2unXrZg0ZMsRau3atfQO3sebW48iRI9bdd99tud1uq1u3bla/fv2sP/7xj1ZTU5O9g7eB7/u58NJLL/mO+eabb6xf//rX1o9+9COre/fu1q233modOXLEvqHbUHPr8X3fO5Ksqqqqi76fgP+/MwAAAKN0mdekAACAjoVIAQAARiJSAACAkYgUAABgJCIFAAAYiUgBAABGIlIAAICRiBQAAGAkIgUAABiJSAEAAEYiUgAAgJGIFAAAYKT/A79whE7dGTaiAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# It is easy to plot ND arrays, given the __array__ interface.\n",
    "labels = 15 + 2 * tnp.random.randn(1, 1000)\n",
    "_ = plt.hist(labels)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kF-Xyw3XWKqJ"
   },
   "source": [
    "### 缓冲区副本\n",
    "\n",
    "混合使用 TensorFlow NumPy 与 NumPy 代码可能会触发数据副本。这是因为 TensorFlow NumPy 对内存对齐的要求比 NumPy 更严格。\n",
    "\n",
    "当 `np.ndarray` 传递给 TensorFlow NumPy 时，它将检查对齐要求，并在需要时触发副本。将 ND 数组 CPU 缓冲区传递给 NumPy 时，该缓冲区通常会满足对齐要求，并且 NumPy 无需创建副本。\n",
    "\n",
    "ND 数组可以引用放置在本地 CPU 内存以外设备上的缓冲区。在这种情况下，调用 NumPy 函数将根据需要触发网络或设备上的副本。\n",
    "\n",
    "有鉴于此，与 NumPy API 调用混合使用时通常应谨慎操作，并且用户应当注意复制数据的开销。将 TensorFlow NumPy 调用与 TensorFlow 调用交错通常是安全的，可避免复制数据。有关更多详细信息，请参阅关于 [Tensorflow 互操作性](#tensorflow-interoperability)的部分。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RwljbqkBc7Ro"
   },
   "source": [
    "### 算子优先级\n",
    "\n",
    "TensorFlow NumPy 定义了一个优先级高于 NumPy 的 `__array_priority__`。这意味着，对于同时涉及 ND 数组和 `np.ndarray` 的算子，前者将获得优先权，即 `np.ndarray` 输入将转换为 ND 数组，并且将调用该算子的 TensorFlow NumPy 实现。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:05.294025Z",
     "iopub.status.busy": "2022-12-14T22:42:05.293556Z",
     "iopub.status.idle": "2022-12-14T22:42:05.298392Z",
     "shell.execute_reply": "2022-12-14T22:42:05.297838Z"
    },
    "id": "Cbw8a3G_WUO7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "x = tf.Tensor([2. 2.], shape=(2,), dtype=float64)\n",
      "class = <class 'tensorflow.python.framework.ops.EagerTensor'>\n"
     ]
    }
   ],
   "source": [
    "x = tnp.ones([2]) + np.ones([2])\n",
    "print(\"x = %s\\nclass = %s\" % (x, x.__class__))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "DNEab_Ctky83"
   },
   "source": [
    "## TF NumPy 和 TensorFlow\n",
    "\n",
    "TensorFlow NumPy 在 TensorFlow 上构建，因此可与 TensorFlow 无缝互操作。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fCcfgrlOnAhQ"
   },
   "source": [
    "### `tf.Tensor` 和 ND 数组\n",
    "\n",
    "ND 数组是 `tf.Tensor` 的别名，因此显然可以在不触发实际数据副本的情况下将它们混合到一起。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:05.301139Z",
     "iopub.status.busy": "2022-12-14T22:42:05.300925Z",
     "iopub.status.idle": "2022-12-14T22:42:05.305873Z",
     "shell.execute_reply": "2022-12-14T22:42:05.305222Z"
    },
    "id": "BkHVauKwnky_"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tf.Tensor([1 2], shape=(2,), dtype=int32)\n",
      "tf.Tensor([1 2], shape=(2,), dtype=int32)\n",
      "tf.Tensor([1 2], shape=(2,), dtype=int32)\n",
      "[1 2] <class 'numpy.ndarray'>\n"
     ]
    }
   ],
   "source": [
    "x = tf.constant([1, 2])\n",
    "print(x)\n",
    "\n",
    "# `asarray` and `convert_to_tensor` here are no-ops.\n",
    "tnp_x = tnp.asarray(x)\n",
    "print(tnp_x)\n",
    "print(tf.convert_to_tensor(tnp_x))\n",
    "\n",
    "# Note that tf.Tensor.numpy() will continue to return `np.ndarray`.\n",
    "print(x.numpy(), x.numpy().__class__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_151HQVBooxG"
   },
   "source": [
    "### TensorFlow 互操作性\n",
    "\n",
    "ND 数组可以传递给 TensorFlow API，因为 ND 数组只是 `tf.Tensor` 的别名。如上文所述，即使是放置在加速器或远程设备上的数据，这种互操作也不会创建数据副本。\n",
    "\n",
    "相反，可以将 `tf.Tensor` 对象传递给 `tf.experimental.numpy` API，而无需执行数据副本。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:05.308889Z",
     "iopub.status.busy": "2022-12-14T22:42:05.308426Z",
     "iopub.status.idle": "2022-12-14T22:42:05.314650Z",
     "shell.execute_reply": "2022-12-14T22:42:05.314084Z"
    },
    "id": "-QvxNhrFoz09"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Output = tf.Tensor(6.0, shape=(), dtype=float32)\n",
      "Output = tf.Tensor(6.0, shape=(), dtype=float32)\n"
     ]
    }
   ],
   "source": [
    "# ND array passed into TensorFlow function.\n",
    "tf_sum = tf.reduce_sum(tnp.ones([2, 3], tnp.float32))\n",
    "print(\"Output = %s\" % tf_sum)\n",
    "\n",
    "# `tf.Tensor` passed into TensorFlow NumPy function.\n",
    "tnp_sum = tnp.sum(tf.ones([2, 3]))\n",
    "print(\"Output = %s\" % tnp_sum)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1b4HeAkhprF_"
   },
   "source": [
    "### 梯度和雅可比矩阵：tf.GradientTape\n",
    "\n",
    "TensorFlow 的 GradientTape 可通过 TensorFlow 和 TensorFlow NumPy 代码用于反向传播。\n",
    "\n",
    "使用在[示例模型](#example-modle)部分中创建的模型，并计算梯度和雅可比矩阵。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:05.317806Z",
     "iopub.status.busy": "2022-12-14T22:42:05.317360Z",
     "iopub.status.idle": "2022-12-14T22:42:05.338679Z",
     "shell.execute_reply": "2022-12-14T22:42:05.338096Z"
    },
    "id": "T47C9KS8pbsP"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Parameter shapes:"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " [TensorShape([32, 64]), TensorShape([64]), TensorShape([64, 2])]\n",
      "Gradient shapes: [TensorShape([32, 64]), TensorShape([64]), TensorShape([64, 2])]\n"
     ]
    }
   ],
   "source": [
    "def create_batch(batch_size=32):\n",
    "  \"\"\"Creates a batch of input and labels.\"\"\"\n",
    "  return (tnp.random.randn(batch_size, 32).astype(tnp.float32),\n",
    "          tnp.random.randn(batch_size, 2).astype(tnp.float32))\n",
    "\n",
    "def compute_gradients(model, inputs, labels):\n",
    "  \"\"\"Computes gradients of squared loss between model prediction and labels.\"\"\"\n",
    "  with tf.GradientTape() as tape:\n",
    "    assert model.weights is not None\n",
    "    # Note that `model.weights` need to be explicitly watched since they\n",
    "    # are not tf.Variables.\n",
    "    tape.watch(model.weights)\n",
    "    # Compute prediction and loss\n",
    "    prediction = model.predict(inputs)\n",
    "    loss = tnp.sum(tnp.square(prediction - labels))\n",
    "  # This call computes the gradient through the computation above.\n",
    "  return tape.gradient(loss, model.weights)\n",
    "\n",
    "inputs, labels = create_batch()\n",
    "gradients = compute_gradients(model, inputs, labels)\n",
    "\n",
    "# Inspect the shapes of returned gradients to verify they match the\n",
    "# parameter shapes.\n",
    "print(\"Parameter shapes:\", [w.shape for w in model.weights])\n",
    "print(\"Gradient shapes:\", [g.shape for g in gradients])\n",
    "# Verify that gradients are of type ND array.\n",
    "assert isinstance(gradients[0], tnp.ndarray)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:05.341610Z",
     "iopub.status.busy": "2022-12-14T22:42:05.341203Z",
     "iopub.status.idle": "2022-12-14T22:42:05.507494Z",
     "shell.execute_reply": "2022-12-14T22:42:05.506863Z"
    },
    "id": "TujVPDFwrdqp"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Output shape: (16, 2), input shape: (16, 32)\n",
      "Batch jacobian shape: (16, 2, 32)\n"
     ]
    }
   ],
   "source": [
    "# Computes a batch of jacobians. Each row is the jacobian of an element in the\n",
    "# batch of outputs w.r.t. the corresponding input batch element.\n",
    "def prediction_batch_jacobian(inputs):\n",
    "  with tf.GradientTape() as tape:\n",
    "    tape.watch(inputs)\n",
    "    prediction = model.predict(inputs)\n",
    "  return prediction, tape.batch_jacobian(prediction, inputs)\n",
    "\n",
    "inp_batch = tnp.ones([16, 32], tnp.float32)\n",
    "output, batch_jacobian = prediction_batch_jacobian(inp_batch)\n",
    "# Note how the batch jacobian shape relates to the input and output shapes.\n",
    "print(\"Output shape: %s, input shape: %s\" % (output.shape, inp_batch.shape))\n",
    "print(\"Batch jacobian shape:\", batch_jacobian.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MYq9wxfc1Dv_"
   },
   "source": [
    "### 跟踪编译：tf.function\n",
    "\n",
    "TensorFlow 的 `tf.function` 的工作方式是先“跟踪编译”代码，然后优化这些跟踪记录来提高性能。请参阅[计算图和函数简介](./intro_to_graphs.ipynb)。\n",
    "\n",
    "`tf.function` 也可用于优化 TensorFlow NumPy 代码。下面是一个演示加速的简单示例。请注意， `tf.function` 代码的主体包括对 TensorFlow NumPy API 的调用。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:05.510549Z",
     "iopub.status.busy": "2022-12-14T22:42:05.510307Z",
     "iopub.status.idle": "2022-12-14T22:42:05.697819Z",
     "shell.execute_reply": "2022-12-14T22:42:05.697130Z"
    },
    "id": "05SrUulm1OlL"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Eager performance\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.366622199951962 ms\n",
      "\n",
      "tf.function compiled performance\n",
      "0.5858368998815422 ms\n"
     ]
    }
   ],
   "source": [
    "inputs, labels = create_batch(512)\n",
    "print(\"Eager performance\")\n",
    "compute_gradients(model, inputs, labels)\n",
    "print(timeit.timeit(lambda: compute_gradients(model, inputs, labels),\n",
    "                    number=10) * 100, \"ms\")\n",
    "\n",
    "print(\"\\ntf.function compiled performance\")\n",
    "compiled_compute_gradients = tf.function(compute_gradients)\n",
    "compiled_compute_gradients(model, inputs, labels)  # warmup\n",
    "print(timeit.timeit(lambda: compiled_compute_gradients(model, inputs, labels),\n",
    "                    number=10) * 100, \"ms\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5w8YxR6ELmo1"
   },
   "source": [
    "### 向量化：tf.vectorized_map\n",
    "\n",
    "TensorFlow 内置对向量化并行循环的支持，可以将速度提高一到两个数量级。这些加速可通过 `tf.vectorized_map`  API 访问，并且也适用于 TensorFlow NumPy 代码。\n",
    "\n",
    "有时，计算一个批次中每个输出相对于相应的输入批次元素的梯度十分有用。可以使用 `tf.vectorized_map` 有效地完成此类计算，具体如下所示。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:05.701483Z",
     "iopub.status.busy": "2022-12-14T22:42:05.700982Z",
     "iopub.status.idle": "2022-12-14T22:42:05.926843Z",
     "shell.execute_reply": "2022-12-14T22:42:05.926180Z"
    },
    "id": "PemSIrs5L-VJ"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/autograph/pyct/static_analysis/liveness.py:83: Analyzer.lamba_check (from tensorflow.python.autograph.pyct.static_analysis.liveness) is deprecated and will be removed after 2023-09-23.\n",
      "Instructions for updating:\n",
      "Lambda fuctions will be no more assumed to be used in the statement where they are used, or at least in the same block. https://github.com/tensorflow/tensorflow/issues/56089\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Weight shape: (32, 64), batch size: 128, per example gradient shape: (128, 32, 64) \n",
      "Weight shape: (64,), batch size: 128, per example gradient shape: (128, 64) \n",
      "Weight shape: (64, 2), batch size: 128, per example gradient shape: (128, 64, 2) \n"
     ]
    }
   ],
   "source": [
    "@tf.function\n",
    "def vectorized_per_example_gradients(inputs, labels):\n",
    "  def single_example_gradient(arg):\n",
    "    inp, label = arg\n",
    "    return compute_gradients(model,\n",
    "                             tnp.expand_dims(inp, 0),\n",
    "                             tnp.expand_dims(label, 0))\n",
    "  # Note that a call to `tf.vectorized_map` semantically maps\n",
    "  # `single_example_gradient` over each row of `inputs` and `labels`.\n",
    "  # The interface is similar to `tf.map_fn`.\n",
    "  # The underlying machinery vectorizes away this map loop which gives\n",
    "  # nice speedups.\n",
    "  return tf.vectorized_map(single_example_gradient, (inputs, labels))\n",
    "\n",
    "batch_size = 128\n",
    "inputs, labels = create_batch(batch_size)\n",
    "\n",
    "per_example_gradients = vectorized_per_example_gradients(inputs, labels)\n",
    "for w, p in zip(model.weights, per_example_gradients):\n",
    "  print(\"Weight shape: %s, batch size: %s, per example gradient shape: %s \" % (\n",
    "      w.shape, batch_size, p.shape))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:05.930248Z",
     "iopub.status.busy": "2022-12-14T22:42:05.929756Z",
     "iopub.status.idle": "2022-12-14T22:42:06.414474Z",
     "shell.execute_reply": "2022-12-14T22:42:06.413620Z"
    },
    "id": "_QZ5BjJmRAlG"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running vectorized computation\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.587835499936773 ms\n",
      "\n",
      "Running unvectorized computation\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "29.57804869984102 ms\n"
     ]
    }
   ],
   "source": [
    "# Benchmark the vectorized computation above and compare with\n",
    "# unvectorized sequential computation using `tf.map_fn`.\n",
    "@tf.function\n",
    "def unvectorized_per_example_gradients(inputs, labels):\n",
    "  def single_example_gradient(arg):\n",
    "    inp, label = arg\n",
    "    return compute_gradients(model,\n",
    "                             tnp.expand_dims(inp, 0),\n",
    "                             tnp.expand_dims(label, 0))\n",
    "\n",
    "  return tf.map_fn(single_example_gradient, (inputs, labels),\n",
    "                   fn_output_signature=(tf.float32, tf.float32, tf.float32))\n",
    "\n",
    "print(\"Running vectorized computation\")\n",
    "print(timeit.timeit(lambda: vectorized_per_example_gradients(inputs, labels),\n",
    "                    number=10) * 100, \"ms\")\n",
    "\n",
    "print(\"\\nRunning unvectorized computation\")\n",
    "per_example_gradients = unvectorized_per_example_gradients(inputs, labels)\n",
    "print(timeit.timeit(lambda: unvectorized_per_example_gradients(inputs, labels),\n",
    "                    number=10) * 100, \"ms\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UOTh-nkzaJd9"
   },
   "source": [
    "### 设备放置\n",
    "\n",
    "TensorFlow NumPy 可以将运算置于 CPU、GPU、TPU 和远程设备上。它使用标准的 TensorFlow 机制完成设备放置。下面我们将给出一个简单的示例，先列出所有设备，然后在特定设备上放置一些计算。\n",
    "\n",
    "TenorFlow 还具有用于在设备之间复制计算并执行集体缩减的 API，这里不作介绍。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-0gHrwYYaTCE"
   },
   "source": [
    "#### 列出设备\n",
    "\n",
    "可以使用 `tf.config.list_logical_devices` 和 `tf.config.list_physical_devices` 查找要使用的设备。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:06.418695Z",
     "iopub.status.busy": "2022-12-14T22:42:06.418061Z",
     "iopub.status.idle": "2022-12-14T22:42:06.422470Z",
     "shell.execute_reply": "2022-12-14T22:42:06.421831Z"
    },
    "id": "NDEAd9m9aemS"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "All logical devices: [LogicalDevice(name='/device:CPU:0', device_type='CPU'), LogicalDevice(name='/device:GPU:0', device_type='GPU'), LogicalDevice(name='/device:GPU:1', device_type='GPU'), LogicalDevice(name='/device:GPU:2', device_type='GPU'), LogicalDevice(name='/device:GPU:3', device_type='GPU')]\n",
      "All physical devices: [PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU'), PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU'), PhysicalDevice(name='/physical_device:GPU:1', device_type='GPU'), PhysicalDevice(name='/physical_device:GPU:2', device_type='GPU'), PhysicalDevice(name='/physical_device:GPU:3', device_type='GPU')]\n"
     ]
    }
   ],
   "source": [
    "print(\"All logical devices:\", tf.config.list_logical_devices())\n",
    "print(\"All physical devices:\", tf.config.list_physical_devices())\n",
    "\n",
    "# Try to get the GPU device. If unavailable, fallback to CPU.\n",
    "try:\n",
    "  device = tf.config.list_logical_devices(device_type=\"GPU\")[0]\n",
    "except IndexError:\n",
    "  device = \"/device:CPU:0\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fihgfF_tahVx"
   },
   "source": [
    "#### 放置运算：**`tf.device`**\n",
    "\n",
    "通过在 `tf.device` 范围内调用设备，可以将运算放置在该设备上。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:06.426074Z",
     "iopub.status.busy": "2022-12-14T22:42:06.425535Z",
     "iopub.status.idle": "2022-12-14T22:42:06.432556Z",
     "shell.execute_reply": "2022-12-14T22:42:06.431930Z"
    },
    "id": "c7ELvLmnazfV"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Using device: LogicalDevice(name='/device:GPU:0', device_type='GPU')\n",
      "prediction is placed on /job:localhost/replica:0/task:0/device:GPU:0\n"
     ]
    }
   ],
   "source": [
    "print(\"Using device: %s\" % str(device))\n",
    "# Run operations in the `tf.device` scope.\n",
    "# If a GPU is available, these operations execute on the GPU and outputs are\n",
    "# placed on the GPU memory.\n",
    "with tf.device(device):\n",
    "  prediction = model.predict(create_batch(5)[0])\n",
    "\n",
    "print(\"prediction is placed on %s\" % prediction.device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "e-LK6wsHbBiM"
   },
   "source": [
    "#### 跨设备复制 ND 数组：**`tnp.copy`**\n",
    "\n",
    "调用位于某个设备范围内的 `tnp.copy` 会将数据复制到该设备，除非这些数据已存在于该设备上。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:06.435973Z",
     "iopub.status.busy": "2022-12-14T22:42:06.435468Z",
     "iopub.status.idle": "2022-12-14T22:42:06.440583Z",
     "shell.execute_reply": "2022-12-14T22:42:06.440022Z"
    },
    "id": "CCesyidaa-UT"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/job:localhost/replica:0/task:0/device:GPU:0\n",
      "/job:localhost/replica:0/task:0/device:CPU:0\n"
     ]
    }
   ],
   "source": [
    "with tf.device(\"/device:CPU:0\"):\n",
    "  prediction_cpu = tnp.copy(prediction)\n",
    "print(prediction.device)\n",
    "print(prediction_cpu.device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AiYzRDOtKzAH"
   },
   "source": [
    "## 性能比较\n",
    "\n",
    "TensorFlow NumPy 使用高度优化的 TensorFlow 内核，这些内核可在 CPU、GPU 和 TPU 上调度。TensorFlow 还执行许多编译器优化（如运算融合），这些优化可转化为性能和内存改进。要了解更多信息，请参阅[使用 Grappler 进行 TensorFlow 计算图优化](./guide/graph_optimization)。\n",
    "\n",
    "但是，与 NumPy 相比，TensorFlow 在调度运算上的开销更高。对于由小型运算（短于约 10 微秒）组成的工作负载，这些开销会占用大部分运行时，此时，NumPy 可以提供更好的性能。对于其他情况，TensorFlow 通常会提供更好的性能。\n",
    "\n",
    "运行以下基准测试来比较 NumPy 和 TensorFlow NumPy 在不同输入大小下的性能。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "cellView": "code",
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:06.443779Z",
     "iopub.status.busy": "2022-12-14T22:42:06.443324Z",
     "iopub.status.idle": "2022-12-14T22:42:06.449077Z",
     "shell.execute_reply": "2022-12-14T22:42:06.448512Z"
    },
    "id": "RExwjI9_pJG0"
   },
   "outputs": [],
   "source": [
    "def benchmark(f, inputs, number=30, force_gpu_sync=False):\n",
    "  \"\"\"Utility to benchmark `f` on each value in `inputs`.\"\"\"\n",
    "  times = []\n",
    "  for inp in inputs:\n",
    "    def _g():\n",
    "      if force_gpu_sync:\n",
    "        one = tnp.asarray(1)\n",
    "      f(inp)\n",
    "      if force_gpu_sync:\n",
    "        with tf.device(\"CPU:0\"):\n",
    "          tnp.copy(one)  # Force a sync for GPU case\n",
    "\n",
    "    _g()  # warmup\n",
    "    t = timeit.timeit(_g, number=number)\n",
    "    times.append(t * 1000. / number)\n",
    "  return times\n",
    "\n",
    "\n",
    "def plot(np_times, tnp_times, compiled_tnp_times, has_gpu, tnp_times_gpu):\n",
    "  \"\"\"Plot the different runtimes.\"\"\"\n",
    "  plt.xlabel(\"size\")\n",
    "  plt.ylabel(\"time (ms)\")\n",
    "  plt.title(\"Sigmoid benchmark: TF NumPy vs NumPy\")\n",
    "  plt.plot(sizes, np_times, label=\"NumPy\")\n",
    "  plt.plot(sizes, tnp_times, label=\"TF NumPy (CPU)\")\n",
    "  plt.plot(sizes, compiled_tnp_times, label=\"Compiled TF NumPy (CPU)\")\n",
    "  if has_gpu:\n",
    "    plt.plot(sizes, tnp_times_gpu, label=\"TF NumPy (GPU)\")\n",
    "  plt.legend()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2022-12-14T22:42:06.452303Z",
     "iopub.status.busy": "2022-12-14T22:42:06.451810Z",
     "iopub.status.idle": "2022-12-14T22:42:07.407116Z",
     "shell.execute_reply": "2022-12-14T22:42:07.406419Z"
    },
    "id": "p-fs_H1lkLfV"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAioAAAHHCAYAAACRAnNyAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjYuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8o6BhiAAAACXBIWXMAAA9hAAAPYQGoP6dpAACFLElEQVR4nO3dd3gU1dfA8e/uppPeKaGEEjpJaAIixdBEFJEeRUQsNAWsNGlKUUQ6+kNFeamCoCJSlCJFmiShhd5LICQhve/O+0fMypJNSEKS3STn8zz7wM7emTkzSXbP3nvnjEpRFAUhhBBCCDOkNnUAQgghhBC5kURFCCGEEGZLEhUhhBBCmC1JVIQQQghhtiRREUIIIYTZkkRFCCGEEGZLEhUhhBBCmC1JVIQQQghhtiRREUIIIYTZkkRF5FC9enUGDx5s6jDy9P3336NSqbh69eoj2+bneK5evYpKpWLOnDlFE2AxaN++PQ0bNjR1GPmmUqkYOXKkqcMQQpRykqiUIydPnqR3795Uq1YNGxsbKleuTKdOnVi4cKGpQxMiT+3bt0elUj3yMWXKFCArOc2tTWpqaq77yU5YVSoVP/30U47Xp0yZgkqlIioqqrgONU8PH5enpydt27Zl06ZNJomnMLJ/lj169Mjxmqm/MGT/fLMfdnZ21K9fn4kTJxIfH2+SmARYmDoAUTL+/vtvOnToQNWqVXn99dfx9vbmxo0bHDp0iPnz5zNq1Ch923PnzqFWm3cO+/LLL9O/f3+sra1NHYooARMmTGDo0KH650ePHmXBggWMHz+eevXq6Zc3btxY/39/f3/efffdHNuysrLK1z6nTZtGr169UKlUjxF50XvwuG7fvs3XX39Nr169WLp0KW+99ZaJo8u/3377jWPHjtG0aVNTh5LD0qVLsbe3JzExkR07dvDpp5+ya9cuDhw4YHa/D+WBJCrlxKeffoqTkxNHjx7F2dnZ4LXIyEiD56Xhw1+j0aDRaEwdhniIoiikpqZia2tbpNvt1KmTwXMbGxsWLFhAp06daN++vdF1KleuzEsvvVSo/fn7+xMWFsamTZvo1atXobZRXB4+rkGDBlGrVi2+/PLLUpOoVK1alYSEBKZOncqvv/5q6nBy6N27N+7u7gC89dZbvPjii2zcuJFDhw7RqlUrE0dX/pj312ZRZC5dukSDBg1yJCkAnp6eBs+Nzek4ceIE7dq1w9bWlipVqvDJJ5+wfPnyHPNEqlevzrPPPsuePXto1qwZtra2NGrUiD179gCwceNGGjVqhI2NDU2bNiU0NDRHPLt27aJt27ZUqFABZ2dnnn/+ec6cOWPQxtgcFUVR+OSTT6hSpQp2dnZ06NCB06dPF+g8AXz55ZdUq1YNW1tb2rVrx6lTp3K0OXv2LL1798bV1RUbGxuaNWuW4w03O8YDBw4wduxYPDw8qFChAi+88AL37t3Lsc2tW7fSrl07HBwccHR0pHnz5qxevTpHu/DwcDp06ICdnR2VK1fms88+M3h9z549qFQqfvzxR6ZOnUrlypVxcHCgd+/exMXFkZaWxujRo/H09MTe3p5XX32VtLQ0g20sX76cjh074unpibW1NfXr12fp0qU5Ysn+eW/fvl3/8/76669zPbeffPIJarXaYLjx+vXrnD17Ntd1TKF///7UqVOHadOm8agbzOc2B6p9+/YGSVRR/FyM8fb2pl69ely5cgWAV155BXd3dzIyMnK07dy5M35+frlua+TIkdjb25OcnJzjtQEDBuDt7Y1WqwXgn3/+oUuXLri7u2Nra0uNGjUYMmTII+MFcHBwYMyYMWzevJmQkJA822YPxzzM2HtAUbz/GNOxY0cArly5wu7du1GpVEaH21avXo1KpeLgwYP52q7IH0lUyolq1apx7Ngxox+6j3Lr1i39h/64ceMYM2YMq1atYv78+UbbX7x4kYEDB9KjRw9mzpzJ/fv36dGjB6tWrWLMmDG89NJLTJ06lUuXLtG3b190Op1+3T///JMuXboQGRnJlClTGDt2LH///Tdt2rR55MTZjz/+mEmTJtGkSRM+//xzfH196dy5M0lJSfk+1hUrVrBgwQJGjBjBuHHjOHXqFB07duTu3bv6NqdPn+aJJ57gzJkzfPTRR3zxxRdUqFCBnj17Gn3zGjVqFMePH2fy5MkMGzaMzZs355hk+v3339O9e3diYmIYN24cs2bNwt/fn23bthm0u3//Pl27dqVJkyZ88cUX1K1blw8//JCtW7fm2O/MmTPZvn07H330EUOGDGHjxo289dZbDBkyhPPnzzNlyhR69erF999/z+zZsw3WXbp0KdWqVWP8+PF88cUX+Pj4MHz4cBYvXpxjP+fOnWPAgAF06tSJ+fPn4+/vb/TcTpw4kY8//pivv/7aYKhx0KBBBsM3RSUjI4OoqCiDh7EPYGM0Gg0TJ07k+PHjRT7/43F+LsZkZGRw48YN3NzcgKxh0ejoaLZv327Q7s6dO+zatSvPXqZ+/fqRlJTEli1bDJYnJyezefNmevfujUajITIyks6dO3P16lU++ugjFi5cSHBwMIcOHcr3eXjnnXdwcXHRzysqKo/z/pObS5cuAeDm5kb79u3x8fFh1apVOdqtWrWKmjVrSq9LUVNEubBjxw5Fo9EoGo1GadWqlfLBBx8o27dvV9LT03O0rVatmvLKK6/on48aNUpRqVRKaGiofll0dLTi6uqqAMqVK1cM1gWUv//+W79s+/btCqDY2toq165d0y//+uuvFUDZvXu3fpm/v7/i6empREdH65cdP35cUavVyqBBg/TLli9fbrDvyMhIxcrKSunevbui0+n07caPH68ABsdjzJUrV/Qx3rx5U7/88OHDCqCMGTNGv+zpp59WGjVqpKSmpuqX6XQ6pXXr1krt2rVzxBgUFGQQ05gxYxSNRqPExsYqiqIosbGxioODg9KyZUslJSXFIK4H12vXrp0CKCtWrNAvS0tLU7y9vZUXX3xRv2z37t0KoDRs2NDg5ztgwABFpVIp3bp1M9hHq1atlGrVqhksS05OznGOunTpovj6+hosy/55b9u2LUd7QBkxYoSiKIry7rvvKmq1Wvn+++9ztMs+roJYv359jt8dY3E9/Jg8eXKe283+Pfj888+VzMxMpXbt2kqTJk30P4fJkycrgHLv3j2DfRn7/WrXrp3Srl07/fOi+LlUq1ZN6dy5s3Lv3j3l3r17yvHjx5X+/fsrgDJq1ChFURRFq9UqVapUUfr162ew7ty5cxWVSqVcvnw51+PX6XRK5cqVDX6fFEVRfvzxRwVQ9u7dqyiKomzatEkBlKNHj+a6rdy0a9dOadCggaIoijJ16lQFUI4dO6YoiuH5z5Z9zh/28HuAojz++0/2vs6dO6fcu3dPuXLlivL1118r1tbWipeXl5KUlKQoiqKMGzdOsba21v8NK0rWe5CFhcUjf8dEwUmPSjnRqVMnDh48yHPPPcfx48f57LPP6NKlC5UrV37kGPG2bdto1aqVwTdlV1dXgoODjbavX7++wTeKli1bAlndp1WrVs2x/PLlywBEREQQFhbG4MGDcXV11bdr3LgxnTp14vfff881xj///JP09HRGjRpl0E08evToPI/tYT179qRy5cr65y1atKBly5b6fcfExLBr1y769u1LQkKC/pt6dHQ0Xbp04cKFC9y6dctgm2+88YZBTG3btkWr1XLt2jUA/vjjDxISEvjoo4+wsbExWPfhLm97e3uDb8RWVla0aNFCfw4fNGjQICwtLfXPW7ZsiaIoObrnW7ZsyY0bN8jMzNQve3COSVxcHFFRUbRr147Lly8TFxdnsH6NGjXo0qVLjv1D1nDcyJEjmT9/PitXruSVV17J0WbPnj2PHF4pjJYtW/LHH38YPAYNGpTv9R/sVfn555+LLK7H+bkA7NixAw8PDzw8PGjSpAnr16/n5Zdf1ve+qNVqgoOD+fXXX0lISNCvt2rVKlq3bk2NGjVyjU2lUtGnTx9+//13EhMT9cvXrVtH5cqVefLJJwH0Q8i//fab0SGm/MruVZk6dWqht/Gwwr7/PMjPzw8PDw9q1KjBm2++Sa1atdiyZQt2dnZA1s8wLS2NDRs26NdZt24dmZmZhZ4XJXIniUo50rx5czZu3Mj9+/c5cuQI48aNIyEhgd69exMeHp7reteuXaNWrVo5lhtbBhi8GQA4OTkB4OPjY3T5/fv39fsBjI6h16tXj6ioqFyHcbLXrV27tsFyDw8PXFxcjK5jzMPrA9SpU0c/7HTx4kUURWHSpEn6D4vsx+TJk4Gck5MfPh/Z8WQfd3a3cn5qpFSpUiVH8uLi4qLfVl77zevnoNPpDBKQAwcOEBQUpJ8n5OHhwfjx4wGMJiq5WbFiBYsXL2bhwoUMGDDgkcdXlNzd3QkKCjJ4+Pr6FmgbwcHB1KpVK19zVfLrcX4u8F8C9ueff/L3338TFRXFihUrDJLLQYMGkZKSoh+2OnfuHMeOHePll19+ZHz9+vUjJSVF/wUmMTGR33//nT59+uh/99q1a8eLL77I1KlTcXd35/nnn2f58uX5mlPz8DGOHj2aX3/9Nd/zRR6lsO8/D/rpp5/4448/2LNnDxcvXuTUqVMGVyfVrVuX5s2bGwz/rFq1iieeeCLX90VReJKolENWVlY0b96cGTNmsHTpUjIyMli/fn2RbT+3q3FyW14c36aLS/Z49nvvvZfj23r24+E3qqI87oJsq7A/h0uXLvH0008TFRXF3Llz2bJlC3/88QdjxowByDGmn9cVPm3atMHLy4tFixYRExOTaztzld2rEhYWxi+//GK0TW6Xq2ZPOjW2zYIsf/hnm52APf3007Rq1croBPn69evTtGlTVq5cCcDKlSuxsrKib9++RvfxoCeeeILq1avz448/ArB582ZSUlLo16+fvo1KpWLDhg0cPHiQkSNHcuvWLYYMGULTpk0NemLy45133sHZ2TnXXpWSPr8ATz31FEFBQbRr146aNWsaXW/QoEH89ddf3Lx5k0uXLnHo0CHpTSkmkqiUc82aNQOyhl1yU61aNS5evJhjubFlj6NatWpA1re/h509exZ3d3cqVKiQ57oXLlwwWH7v3j2j35hy8/D6AOfPn6d69eoA+m/klpaWOb6tZz8cHBzyvT9A/0ZYmInOxWHz5s2kpaXx66+/8uabb/LMM88QFBRUqEuOa9WqxY4dO7h9+zZdu3Y1GIooLV566SVq1arF1KlTjX6oubi4EBsbm2N5di+fqQwaNIhdu3YRERHB6tWr6d69e757F/v27cu2bduIj49n3bp1VK9enSeeeCJHuyeeeIJPP/2Uf/75h1WrVnH69GnWrl1boDize1V++eUXo70q2TE/fI5NfX779++PRqNhzZo1rFq1CktLS4NkThQdSVTKid27dxt9k82ee5HXJYtdunTh4MGDhIWF6ZfFxMQYnfX+OCpWrIi/vz8//PCDwZvSqVOn2LFjB88880yu6wYFBWFpacnChQsNjnPevHkFiuHnn382mGNy5MgRDh8+TLdu3YCsS7nbt2/P119/bTS5M3bZ8aN07twZBwcHZs6cmaNqqil6m7K/eT6477i4OJYvX16o7TVu3Jjff/+dM2fO0KNHD1JSUgxeN8fLkx/0YK+KsflcNWvW5NChQ6Snp+uX/fbbb9y4caMkw8xhwIABqFQq3nnnHS5fvlygb/v9+vUjLS2NH374gW3btuXoibl//36O383sOWwFHf6BrLlkzs7OTJs2Lcdr2Yn83r179cuSkpL44YcfCryfouTu7k63bt1YuXIlq1atomvXrvraK6JoScG3cmLUqFEkJyfzwgsvULduXdLT0/n777/135ZeffXVXNf94IMPWLlyJZ06dWLUqFFUqFCBb775hqpVqxITE1OklRo///xzunXrRqtWrXjttddISUlh4cKFODk55XkZo4eHB++99x4zZ87k2Wef5ZlnniE0NJStW7cW6M2jVq1aPPnkkwwbNoy0tDTmzZuHm5sbH3zwgb7N4sWLefLJJ2nUqBGvv/46vr6+3L17l4MHD3Lz5k2OHz9eoGN2dHTkyy+/ZOjQoTRv3pyBAwfi4uLC8ePHSU5OLvE35M6dO2NlZUWPHj148803SUxMZNmyZXh6eubZ85aXJ554gl9++YVnnnmG3r178/PPP+snlGZ3oZvzEGBwcDDTp083SNazDR06lA0bNtC1a1f69u3LpUuXWLlyZa5DBiXFw8ODrl27sn79epydnenevXu+1w0MDKRWrVpMmDCBtLS0HD0FP/zwA0uWLOGFF16gZs2aJCQksGzZMhwdHfP8QpEbJycn3nnnHaPDP507d6Zq1aq89tprvP/++2g0Gr777js8PDy4fv16gfdVlAYNGkTv3r0BmD59ukljKcukR6WcmDNnDh06dOD3339n7NixjB07liNHjjB8+HAOHz5sdJw7m4+PD7t376ZevXrMmDGDefPm8corr+ivUnj4SpXHERQUxLZt23Bzc+Pjjz9mzpw5PPHEExw4cCDPSZuQVUxs6tSphIaG8v7773Pp0iV27NiR63CRMYMGDWLUqFEsWrSITz/9lAYNGrBr1y4qVqyob1O/fn3++ecfunfvzvfff8+IESP46quvUKvVfPzxx4U67tdee41ff/0VR0dHpk+fzocffkhISIi+J6ck+fn5sWHDBlQqFe+99x5fffUVb7zxBu+8885jbbdjx478+OOP7Nixg5dffjlf9SvMhYWFBRMnTjT6WpcuXfjiiy84f/48o0eP5uDBg/z2229UqVKlhKPMKfsqp759+xa44nS/fv1ISEigVq1aBAYGGrzWrl07mjVrxtq1a3n77bf57LPPqF27Nrt27Xrk32luRo8erZ/g+iBLS0s2bdpEzZo1mTRpEgsWLGDo0KFmccPLHj164OLigpOTE88995ypwymzVIo5f40RZm306NF8/fXXJCYmSjl7IczQL7/8Qs+ePdm7dy9t27Y1dThlTmZmJpUqVaJHjx58++23pg6nzJIeFZEvD88riI6O5v/+7/948sknJUkRwkwtW7YMX19fff0TUbR+/vln7t27V6D6PKLgZI6KyJdWrVrRvn176tWrx927d/n222+Jj49n0qRJpg5NCPGQtWvXcuLECbZs2cL8+fPljr9F7PDhw5w4cYLp06cTEBBAu3btTB1SmSZDPyJfxo8fz4YNG7h58yYqlYrAwEAmT55MUFCQqUMTQjxEpVJhb29Pv379+Oqrr7CwkO+kRWnw4MGsXLkSf39/vv/++3wVaxSFJ4mKEEIIIcyWzFERQgghhNmSREUIIYQQZqtUD1zqdDpu376Ng4ODTBYTQgghSglFUUhISKBSpUqo1Xn3mZTqROX27ds57ogphBBCiNLhxo0bjyyOWKoTleybv924cQNHR0cTRyOEEEKI/IiPj8fHxydfN3Et1YlK9nCPo6OjJCpCCCFEKZOfaRsymVYIIYQQZksSFSGEEEKYLUlUhBBCCGG2SvUclfzSarVkZGSYOgwhxAMsLS3lhpZCiEcq04mKoijcuXOH2NhYU4cihDDC2dkZb29vqYMkhMhVmU5UspMUT09P7Ozs5M1QCDOhKArJyclERkYCULFiRRNHJIQwV2U2UdFqtfokxc3NzdThCCEeYmtrC0BkZCSenp4yDCSEMKrMTqbNnpNiZ2dn4kiEELnJ/vuUOWRCiNyU2UQlmwz3CGG+5O9TCPEoZT5REUIIIUTpJYmKEEIIIcyWJCpmaPDgwahUKmbNmmWw/Oeffy6RrnKVSqV/ODk50aZNG3bt2lXs+xVCCCEeJomKmbKxsWH27Nncv3/fJPtfvnw5ERERHDhwAHd3d5599lkuX75skliEEEKYxt7z90jN0Jo0BklUzFRQUBDe3t7MnDnT6OtTpkzB39/fYNm8efOoXr26/vngwYPp2bMnM2bMwMvLC2dnZ6ZNm0ZmZibvv/8+rq6uVKlSheXLl+fYfnYhroYNG7J06VJSUlL4448/WLFiBW5ubqSlpRm079mzJy+//PJjH7cQQgjTy9Dq+OS3cAZ9d4Rpv4WbNJZylagoikJyemaJPxRFKXCsGo2GGTNmsHDhQm7evFnoY961axe3b99m7969zJ07l8mTJ/Pss8/i4uLC4cOHeeutt3jzzTfz3Ed2vYv09HT69OmDVqvl119/1b8eGRnJli1bGDJkSKHjFEIIYR5ux6bQ7+uDfLP/CgAVrDTodAX/HCsqZbbgmzEpGVrqf7y9xPcbPq0LdlYFP9UvvPAC/v7+TJ48mW+//bZQ+3Z1dWXBggWo1Wr8/Pz47LPPSE5OZvz48QCMGzeOWbNmsX//fvr3759j/eTkZCZOnIhGo6Fdu3bY2toycOBAli9fTp8+fQBYuXIlVatWpX379oWKUQghhHn46/w9Rq8N5X5yBg42Fszp04QuDbxNGlO5SlRKo9mzZ9OxY0fee++9Qq3foEED1Or/Os68vLxo2LCh/rlGo8HNzU1fyjzbgAED0Gg0pKSk4OHhwbfffkvjxo0BeP3112nevDm3bt2icuXKfP/99/oJwEIIIUofrU5h3p/nWbT7IooCDSs7smRgU6q6mb5oarlKVGwtNYRP62KS/RbWU089RZcuXRg3bhyDBw/WL1er1TmGlIxV97S0tDR4rlKpjC7T6XQGy7788kuCgoJwcnLCw8PD4LWAgACaNGnCihUr6Ny5M6dPn2bLli2FOTwhhBAmFpmQyjtrwjh4ORqAl56oysTu9bF5jM+uolSuEhWVSlWoIRhTmzVrFv7+/vj5+emXeXh4cOfOHRRF0fdkhIWFFdk+vb29qVWrVq6vDx06lHnz5nHr1i2CgoLw8fEpsn0LIYQoGYcuRzNqTSj3EtKws9Iws1cjnvevbOqwDJSrybSlVaNGjQgODmbBggX6Ze3bt+fevXt89tlnXLp0icWLF7N169YSi2ngwIHcvHmTZcuWySRaIYQoZXQ6hcW7LzJw2SHuJaRRx8ueX0c+aXZJCkiiUmpMmzbNYHimXr16LFmyhMWLF9OkSROOHDlS6HksheHk5MSLL76Ivb09PXv2LLH9CiGEeDz3k9IZ8sNRPt9+Dp0CvQIr8/OINtTytDd1aEaplMJcO2sm4uPjcXJyIi4uDkdHR4PXUlNTuXLlCjVq1MDGxsZEEZZtTz/9NA0aNDDo6RGiIOTvVIiSFXL9PiNXhXA7LhVrCzXTn29In2ZVSvxiiLw+vx9W+iZsCJO7f/8+e/bsYc+ePSxZssTU4QghhHgERVH47sBVZv5+hkydQg33CiweGEj9SnknCeZAEhVRYAEBAdy/f5/Zs2cbTPAVQghhfuJTM/hg/Qm2nb4DQPdGFZn1YiMcbCwfsaZ5kERFFNjVq1dNHYIQQoh8OHUrjhGrQ7gWnYylRsXE7vUZ1Kpaqap7JYmKEEIIUcYoisKaIzeYsvk06Zk6Kjvbsjg4EH8fZ1OHVmCSqAghhBBlSFJaJhN/PsWm0FsAPF3Xky/6NsHZzsrEkRWOJCpCCCFEGXHhbgLDVoVwMTIRjVrF+138eKOtL2p16RnqeZgkKkIIIUQZsCn0JuM3niIlQ4ungzULBwTQ0tfN1GE9NklUhBBCiFIsNUPL1M3hrDlyHYA2tdyY3z8Ad3trE0dWNCRREUIIIUqpq1FJDF8VQnhEPCoVvN2xNm8/XRtNKR7qeZiU0Bel3qRJk3jjjTdMtv+vvvqKHj16mGz/QojyaevJCHos3E94RDyuFaz44dUWjOlUp0wlKSCJitlRqVR5PqZMmcLVq1eNvvbSSy/lut327dujUqlYu3atwfJ58+ZRvXr1Yj4qcsTs5uZG586dCQ0Nfazt3rlzh/nz5zNhwoQcy0eNGoWvry/W1tb4+PjQo0cPdu7cqW9TvXp1fTwVKlQgMDCQ9evX618fPHiw0fsY7dmzB5VKRWxsLABDhgwhJCSEffv2PdaxCCFEfqRn6pi6+TTDVoWQkJZJ8+ou/P52W56q42Hq0IqFJCpmJiIiQv+YN28ejo6OBssevPHgn3/+afDa4sWL89y2jY0NEydOJCMjo7gPI1fZMW/fvp3ExES6deum/8AvjG+++YbWrVtTrVo1/bKrV6/StGlTdu3axeeff87JkyfZtm0bHTp0YMSIEQbrT5s2jYiICEJDQ2nevDn9+vXj77//LlAMVlZWDBw4UO55JIQodrdiU+j79UGWH7gKwJvtfFn9+hN4O5Xde2VJomJmvL299Q8nJydUKpXBMnv7/+5u6ebmlqN9XgYMGEBsbCzLli3LtY2xXoTRo0fTvn17/fP27dszatQoRo8ejYuLC15eXixbtoykpCReffVVHBwcqFWrFlu3bs2x/eyYmzVrxpw5c7h79y6HDx9m2rRpNGzYMEd7f39/Jk2alGu8a9euzTHsMnz4cFQqFUeOHOHFF1+kTp06NGjQgLFjx3Lo0CGDtg4ODnh7e1OnTh0WL16Mra0tmzdvznV/uenRowe//vorKSkpBV5XCCHyY/fZSLov2EfYjVgcbSxYNqgZ47rVw1JTtj/Ky/bRPUxRID2p5B9mcoNqR0dHJkyYwLRp00hKSnqsbf3www+4u7tz5MgRRo0axbBhw+jTpw+tW7cmJCSEzp078/LLL5OcnJzrNmxtbQFIT09nyJAhnDlzhqNHj+pfDw0N5cSJE7z66qtG14+JiSE8PJxmzZoZLNu2bRsjRoygQoUKOdZxdnbONR4LCwssLS1JT09/1OHn0KxZMzIzMzl8+HCB1xVCiLxkanV8tu0sr35/lNjkDBpXcWLL223pVN/L1KGViPJ11U9GMsyoVPL7HX8brHJ+aD6u1q1bo1b/l2vu27ePgICAPNcZPnw48+fPZ+7cuXn2VDxKkyZNmDhxIgDjxo1j1qxZuLu78/rrrwPw8ccfs3TpUk6cOMETTzyRY/3Y2FimT5+Ovb09LVq0wMvLiy5durB8+XKaN28OwPLly2nXrh2+vr5GY7h+/TqKolCp0n8/04sXL6IoCnXr1i3Q8aSnp/PFF18QFxdHx44dC7QugJ2dHU5OTly7dq3A6wohRG4i41MZtSaUw1diAHilVTXGd6+HtYXGxJGVnPLVo1LGrFu3jrCwMP2jfv36j1zH2tqaadOmMWfOHKKiogq978aNG+v/r9FocHNzo1GjRvplXl5ZmX5kZKTBeq1bt8be3h4XFxeOHz/OunXr9G1ff/111qxZQ2pqKunp6axevZohQ4bkGkP2MIuNzX9js0oBe68+/PBD7O3tsbOzY/bs2cyaNYvu3bsXaBvZbG1t8+xBEkKIgvj7UhTPLNjP4SsxVLDSsHBAAFOfb1iukhQwox6VWbNmMW7cON555x3mzZtXPDuxtMvq3ShplnbFslkfHx9q1apV4PVeeukl5syZwyeffJLjih+1Wp3jw97Y5FtLS8Pbg6tUKoNl2Xfm1Ol0Bu3WrVtH/fr1cXNzyzEM06NHD6ytrdm0aRNWVlZkZGTQu3fvXI/D3d0dgPv37+PhkTXbvXbt2qhUKs6ePZvreg96//33GTx4MPb29nh5eRncUdTR0dFoD0lsbCwajSbH0FJMTIw+DiGEKCydTmHx7ot8+ed5dArU9XZgSXAgvh72j165DDKLROXo0aN8/fXXBt/Si4VKVSxDMKWNWq1m5syZ9OrVi2HDhhm85uHhwalTpwyWhYWF5UhMCsvHx4eaNWsafc3CwoJXXnmF5cuXY2VlRf/+/fXzWIypWbMmjo6OhIeHU6dOHQBcXV3p0qULixcv5u23386RTMTGxhokSO7u7rkme35+fqxdu5a0tDSsrf+r8BgSEkKNGjUMzsmlS5dITU195NCbEELkJSYpndHrwth7/h4AfZtVYepzDbG1Kl+9KA8y+dBPYmIiwcHBLFu2DBcXF1OHU250796dli1b8vXXXxss79ixI//88w8rVqzgwoULTJ48OUfiUpyGDh3Krl272LZtW57DPpCVcAUFBbF//36D5YsXL0ar1dKiRQt++uknLly4wJkzZ1iwYAGtWrXKdyzBwcGoVCoGDRrEsWPHuHjxIt999x3z5s3j3XffNWi7b98+fH19c03ChBDiUY5di6H7gn3sPX8PG0s1n/duzGe9m5TrJAXMIFEZMWIE3bt3Jygo6JFt09LSiI+PN3iIwps9ezapqakGy7p06cKkSZP44IMPaN68OQkJCQwaNKjEYqpduzatW7embt26tGzZ8pHthw4dytq1aw2GmHx9fQkJCaFDhw68++67NGzYkE6dOrFz506WLl2a71icnZ3Zt28fGRkZPPfcc/j7+7NgwQLmzp3Lm2++adB2zZo1+onEQghREIqi8M2+y/T7+hARcan4elTg5xFt6NPMx9ShmQWVUtDZh0Vo7dq1fPrppxw9ehQbGxvat2+Pv79/rnNUpkyZwtSpU3Msj4uLw9HR0WBZamoqV65coUaNGgaTLYV5UxSF2rVrM3z4cMaOHZuv9i1btmTMmDEMGDCgBCLM6fTp03Ts2JHz588/spaNMCR/p6K8i0vJ4P31x9kRfheAHk0qMbNXI+ytzWJmRrGJj4/HycnJ6Of3w0zWo3Ljxg3eeecdVq1ale83qHHjxhEXF6d/3Lhxo5ijFCXp3r17LFq0iDt37uRaO+VhKpWK//3vf2RmZhZzdLmLiIhgxYoVkqQIIQrk5M04nl24jx3hd7HSqJnesyEL+vuX+SSloEx2No4dO0ZkZCSBgYH6ZVqtlr1797Jo0SLS0tLQaAzH5aytrQ0mNYqyxdPTE3d3d/73v/8VaL6Sv78//v7+xRfYI+Rn2FIIIbIpisLKw9eZvjmcdK2OKi62LAkOpHEVZ1OHZpZMlqg8/fTTnDx50mDZq6++St26dfnwww9zJCmi7DPhKKQQQpSIxLRMxm88ya/Hs0pldKrvxZzeTXCyK5orK8sikyUqDg4OOe7tUqFCBdzc3Ize80UIIYQozc7dSWDYqmNcvpeERq3io651Gdq2hkH9JpGTDIQJIYQQxWzDsZtM/PkkqRk6vB1tWDQwgGbVXU0dVqlgVonKnj17TB2CEEIIUWRSM7RM/uU06/7JuvijbW135vXzx81e5lvml1klKkIIIURZcfleIsNXhXD2TgIqFYwJqsOIDrXQqGWopyAkURFCCCGK2JYTEXz40wkS0zJxt7difv8A2tRyN3VYpZIkKkIIIUQRScvUMmPLGX44mHVD0xY1XFk4IAAvRyloWFgmL6EvSp+rV6+iUqkICwsDsuYWqVQqYmNjH2u71atXL747Z5dD3377LZ07dzbZ/rdt24a/v3+OO2gLUVbdiEmm71cH9UnKsPY1WT20pSQpj0kSFTN1584dRo0aha+vL9bW1vj4+NCjRw927txp6tDw8fEhIiKiRC8jr169OiqVKtfH4MGDAYy+9uSTT+a63cGDB6NSqZg1a5bB8p9//rnELhl8MFYnJyfatGnDrl27HmubqampTJo0icmTJxssj4+PZ8KECdStWxcbGxu8vb0JCgpi48aN+jo27du318djY2ND/fr1WbJkiX4bU6ZMMVpg7+EEtmvXrlhaWrJq1arHOhYhSoOdZ+7y7ML9HL8Zh5OtJd8NbsaHXetioZGP2cclZ9AMXb16laZNm7Jr1y4+//xzTp48ybZt2+jQoQMjRowwdXhoNBq8vb2xsCi5kcOjR48SERFBREQEP/30EwDnzp3TL5s/f76+7fLly/XLIyIi+PXXX/Pcto2NDbNnz+b+/fvFegx5yY75wIEDuLu78+yzz3L58uVCb2/Dhg04OjrSpk0b/bLY2Fhat27NihUrGDduHCEhIezdu5d+/frxwQcfEBcXp2/7+uuvExERQXh4OH379mXEiBGsWbOmwHEMHjyYBQsWFPo4hDB3mVodM7ee4bUf/iEuJYMmPs5seftJOtb1MnVoZYYkKmZo+PDhqFQqjhw5wosvvkidOnVo0KABY8eO5dChQ/p2169f5/nnn8fe3h5HR0f69u3L3bt39a9nf/P97rvvqFq1Kvb29gwfPhytVstnn32Gt7c3np6efPrppwb7V6lULF26lG7dumFra4uvry8bNmzQv/7wN2dj9u/fT9u2bbG1tcXHx4e3336bpKQk/euRkZH06NEDW1tbatSo8chv3R4eHnh7e+Pt7Y2ra1btAU9PT/2yB++z4+zsrF/+YPvcBAUF4e3tzcyZM3NtY6wXYd68eVSvXl3/fPDgwfTs2ZMZM2bg5eWFs7Mz06ZNIzMzk/fffx9XV1eqVKnC8uXLc2w/O+aGDRuydOlSUlJS+OOPP1ixYgVubm6kpaUZtO/Zsycvv/xyrvGuXbuWHj16GCwbP348V69e5fDhw7zyyivUr1+fOnXq8PrrrxMWFoa9vb2+rZ2dHd7e3vj6+jJlyhRq1679yITPmB49evDPP/9w6dKlAq8rhLm7E5fKwGWH+fqvrC8Vr7apzvo3W1HFxc7EkZUt5SpRURSF5IzkEn8UpDR8TEwM27ZtY8SIEVSoUCHH687OzgDodDqef/55YmJi+Ouvv/jjjz+4fPky/fr1M2h/6dIltm7dyrZt21izZg3ffvst3bt35+bNm/z111/Mnj2biRMncvjwYYP1Jk2axIsvvsjx48cJDg6mf//+nDlzJl/HcOnSJbp27cqLL77IiRMnWLduHfv372fkyJH6NoMHD+bGjRvs3r2bDRs2sGTJEiIjI/N9noqSRqNhxowZLFy4kJs3bz7Wtnbt2sXt27fZu3cvc+fOZfLkyTz77LO4uLhw+PBh3nrrLd58880892NrawtAeno6ffr0QavVGiQJkZGRbNmyhSFDhuS6jf3799OsWTP9c51Ox9q1awkODqZSpUo52tvb2+fZQ2Zra0t6enqex25M1apV8fLyYt++fQVeVwhztv9CFN0X7OPI1RjsrS1YEhzI5B4NsLIoVx+rJaJcXfWTkplCy9UtS3y/hwcexs4yfxn2xYsXURSFunXr5tlu586dnDx5kitXruDj4wPAihUraNCgAUePHqV58+ZA1gfUd999h4ODA/Xr16dDhw6cO3eO33//HbVajZ+fH7Nnz2b37t20bPnfuenTpw9Dhw4FYPr06fzxxx8sXLjQYK5CbmbOnElwcDCjR48GoHbt2ixYsIB27dqxdOlSrl+/ztatWzly5Ig+zm+//ZZ69erl6xw9yoABAwzuFbVy5Up69uyZ5zovvPAC/v7+TJ48mW+//bbQ+3Z1dWXBggX6c/vZZ5+RnJzM+PHjgaw7gM+aNYv9+/fTv3//HOsnJyczceJENBoN7dq1w9bWloEDB7J8+XL69OmjP56qVavSvn17ozHExsYSFxdnkJBERUVx//79R/5ePUyr1bJmzRpOnDjBG2+8UaB1s1WqVIlr164Val0hzI1Wp7Bw1wXm77yAokC9io4sDQ6kunvOL5aiaJSrRKU0yG/vy5kzZ/Dx8dEnKQD169fH2dmZM2fO6BOA6tWr4+DgoG/j5eWFRqNBrVYbLHu4N6NVq1Y5nuc11POg48ePc+LECYPhHEVR0Ol0XLlyhfPnz2NhYUHTpk31r9etW1ffW/S4vvzyS4M7GlesWDFf682ePZuOHTvy3nvvFXrfDRo0yHFuH5x0rNFocHNzy3G+s5OrlJQUPDw8+Pbbb2ncuDGQNV+kefPm3Lp1i8qVK/P999/rJwEbk5KSAmTNvclW0Bs+LlmyhG+++Yb09HQ0Gg1jxoxh2LBhBdpGNltbW5KTkwu1rhDmJCoxjTHrwth3IQqAAS18mNyjATaWchPd4lSuEhVbC1sODzz86IbFsN/8ql27NiqVirNnzxbJvi0tDe/IqVKpjC4ryktIExMTefPNN3n77bdzvFa1alXOnz9fZPsyxtvbm1q1ahV4vaeeeoouXbowbtw4/VVE2dRqdY4P+4yMjBzbKOz5zk6unJyc8PDwMHgtICCAJk2asGLFCjp37szp06fZsmVLrsfh5uaGSqUymBzs4eGBs7Nzvn+vgoODmTBhAra2tlSsWNEg+XJ0dDSYeJst+/L0B+cLQdZw5sPHJERpc/RqDCNXh3A3Pg1bSw2fvtCQXoFVTB1WuVCuEhWVSpXvIRhTcXV1pUuXLixevJi33347xzyV2NhYnJ2dqVevHjdu3ODGjRv6XpXw8HBiY2OpX7/+Y8dx6NAhBg0aZPA8ICAgX+sGBgYSHh6ea7JQt25dMjMzOXbsmL7n59y5c49dh6UozJo1C39/f/z8/AyWe3h4cOfOHRRF0fdk5LeHKT8elVwNHTqUefPmcevWLYKCggx60h5mZWVF/fr1CQ8P19dRUavV9O/fn//7v/9j8uTJOeapJCYmYmNjo5+n4uTklGs8fn5+3Lx5k7t37+Ll9d+VDSEhIdjY2FC1alX9stTUVC5dupTv3x0hzI2iKPxv72U+234OrU6hlqc9S4IDqePl8OiVRZGQWT9maPHixWi1Wlq0aMFPP/3EhQsXOHPmDAsWLNAPyQQFBdGoUSOCg4MJCQnhyJEjDBo0iHbt2hlMoiys9evX891333H+/HkmT57MkSNHDCbD5uXDDz/k77//ZuTIkYSFhXHhwgV++eUX/fp+fn507dqVN998k8OHD3Ps2DGGDh2qn0RqStnn9OFLatu3b8+9e/f47LPPuHTpEosXL2br1q0lFtfAgQO5efMmy5Yty3MSbbYuXbqwf/9+g2WffvopPj4+tGzZkhUrVhAeHs6FCxf47rvvCAgIIDExMV+xdOnSBT8/PwYMGMDff//N5cuX2bBhAxMnTuSdd94xmB906NAhrK2tcwwlClEaxCVn8PqKf5i59SxanUJP/0r8MqKNJCklTBIVM+Tr60tISAgdOnTg3XffpWHDhnTq1ImdO3eydOlSIKt36JdffsHFxYWnnnqKoKAgfH19WbduXZHEMHXqVNauXUvjxo1ZsWIFa9asyXdPTePGjfnrr784f/48bdu2JSAggI8//tjgW/zy5cupVKkS7dq1o1evXrzxxht4enoWSeyPa9q0aTmGZurVq8eSJUtYvHgxTZo04ciRI481l6WgnJycePHFF7G3t3/kxGCA1157jd9//91giMbV1ZVDhw7x0ksv8cknnxAQEEDbtm1Zs2YNn3/+eY4hm9xYWFiwY8cOqlatyoABA2jYsCGTJ0/mnXfeYfr06QZt16xZQ3BwMHZ25t2TKcTDjt+IpfvCffx5JhIrCzUzXmjEl/38qWBdrgYizIJKKegsOzMSHx+Pk5MTcXFxODo6GryWmprKlStXqFGjhsGkQvFoKpWKTZs25esDUZScp59+mgYNGuS7gFqfPn0IDAxk3LhxxRyZcVFRUfj5+fHPP/9Qo0YNo23k71SYG0VR+L9D1/jktzOka3VUdbVjSXAgDSvnL5EX+ZPX5/fDJDUUwszdv3+fPXv2sGfPnnxdHp7t888/Z/PmzcUYWd6uXr3KkiVLck1ShDA3CakZfLTxJFtORADQpYEXn/dpgqON5SPWFMVJEhUhzFxAQAD3799n9uzZOSb55qV69eqMGjWqGCPLW7NmzYpkvpQQJeFMRDzDV4VwJSoJC7WKcc/UY0ib6iV2zy+RO0lURA6leDSwTLp69aqpQxCiTPvx6A0m/XKKtEwdFZ1sWDQwkKbVXEwdlviXJCpCCCHKpZR0LZN+OcWGY1m3tGjv58Hcvv64VrAycWTiQZKoCCGEKHcuRiYyYlUI5+4moFbBu539GNauJmq1DPWYG0lUhBBClCu/Hr/NuJ9OkJSuxd3emgUD/Gld093UYYlcSKIihBCiXEjL1DL9t3BWHroOwBO+riwYEICng1wab84kURFCCFHmXY9OZsTqEE7eyiqCOLJDLUYH1cZCI3VPzZ0kKkIIIcq0Hafv8O764ySkZuJiZ8ncfv508DOPStji0SSVFKXepEmTeOONN0waw1dffUWPHj1MGoMQwlCGVsenW8J54/+OkZCaSUBVZ7a83VaSlFJGEhUzo1Kp8nxMmTKFq1evGn3tpZdeynW77du3R6VSsXbtWoPl8+bNo3r16sV8VOSI2c3Njc6dOxMaGvpY271z5w7z589nwoQJOZa/88471KpVCxsbG7y8vGjTpg1Lly4lOTlZ36569er6mCpUqEBgYCDr16/Xvz548GCjtxLYs2cPKpVKf8fnIUOGEBISwr59+x7reIQQRSMiLoX+/zvEsn1XAHjtyRqse6MVlZxNf/NTUTCSqJiZiIgI/WPevHk4OjoaLHvwRnh//vmnwWuLFy/Oc9s2NjZMnDiRjIyM4j6MXGXHvH37dhITE+nWrZv+w74wvvnmG1q3bk21atX0yy5fvkxAQAA7duxgxowZhIaGcvDgQT744AN+++03/vzzT4NtTJs2jYiICEJDQ2nevDn9+vXj77//LlAcVlZWDBw4MN/34RFCFJ+95+/RfcF+jl27j4O1BV+91JRJz9bHykI+8koj+amZGW9vb/3DyckJlUplsMze3l7f1s3NLUf7vAwYMIDY2FiWLVuWaxtjPQijR4+mffv2+uft27dn1KhRjB49GhcXF7y8vFi2bBlJSUm8+uqrODg4UKtWLbZu3Zpj+9kxN2vWjDlz5nD37l0OHz7MtGnTaNiwYY72/v7+TJo0Kdd4165dm2PIZfjw4VhYWPDPP//Qt29f6tWrh6+vL88//zxbtmzJ0d7BwQFvb2/q1KnD4sWLsbW1LdQ9cnr06MGvv/5KSkpKgdcVQjw+rU5h7o5zvLL8CDFJ6TSo5Mhvbz9J14bepg5NPIZylagoioIuObnEH+ZSkt7R0ZEJEyYwbdo0kpKSHmtbP/zwA+7u7hw5coRRo0YxbNgw+vTpQ+vWrQkJCaFz5868/PLLBsMsD7O1zeqCTU9PZ8iQIZw5c4ajR4/qXw8NDeXEiRO8+uqrRtePiYkhPDzc4H4y0dHR7NixgxEjRlChQgWj6+V17w4LCwssLS1JT0/P8/iNadasGZmZmRw+fLjA6wohHs+9hDRe/vYwC3ZdRFFgYMuq/DSsNdXcjL8PiNKjXF31o6SkcC6waYnv1y/kGCo7uyLfbuvWrVGr/8s19+3bR0BAQJ7rDB8+nPnz5zN37tw8eyoepUmTJkycOBGAcePGMWvWLNzd3Xn99dcB+Pjjj1m6dCknTpzgiSeeyLF+bGws06dPx97enhYtWuDl5UWXLl1Yvnw5zZs3B2D58uW0a9cOX19fozFcv34dRVGoVKmSftnFixdRFCXHzfvc3d1JTU0FYMSIEcyePTvH9tLT0/niiy+Ii4ujY8eOBT4ndnZ2ODk5ce3atQKvK4QovMOXoxm1JpTIhDTsrDTMeKERPQMqmzosUUTKVY9KWbNu3TrCwsL0j/r16z9yHWtra6ZNm8acOXOIiooq9L4bN26s/79Go8HNzY1GjRrpl3l5eQEQGRlpsF7r1q2xt7fHxcWF48ePs27dOn3b119/nTVr1pCamkp6ejqrV69myJAhucaQPcRiY/PoYk1HjhwhLCyMBg0akJaWZvDahx9+iL29PXZ2dsyePZtZs2bRvXv3R27TGFtb2zx7kYQQRUenU1iy5yIDlh0iMiGNOl72/DqyjSQpZUy56lFR2driF3LMJPstDj4+PtSqVavA67300kvMmTOHTz75JMcVP2q1OsdQlbHJt5aWlgbPVSqVwbLs4RWdTmfQbt26ddSvXx83NzecnZ0NXuvRowfW1tZs2rQJKysrMjIy6N27d67H4e6eVfL6/v37eHh4AFCrVi1UKhXnzp0zaJvdK2Nr5Gfx/vvvM3jwYOzt7fHy8jIYGnJ0dDTaQxIbG4tGo8kxvBQTE6OPRQhRfO4npfPu+uPsOpv1ZahXYGU+6dkQO6ty9bFWLpSrn6hKpSqWIZjSRq1WM3PmTHr16sWwYcMMXvPw8ODUqVMGy8LCwnIkJoXl4+NDzZo1jb5mYWHBK6+8wvLly7GysqJ///5GE4tsNWvWxNHRkfDwcOrUqQNkTdbt1KkTixYtYtSoUbnOU3mQu7t7rgmfn58fa9euJS0tDWtra/3ykJAQatSoYXBeLl26RGpq6iOH34QQjyf0+n1Grg7lVmwK1hZqpj3fgL7NfPKcfyZKLxn6Kae6d+9Oy5Yt+frrrw2Wd+zYkX/++YcVK1Zw4cIFJk+enCNxKU5Dhw5l165dbNu2Lc9hH8hKuIKCgti/f7/B8iVLlpCZmUmzZs1Yt24dZ86c4dy5c6xcuZKzZ8+i0WjyHU9wcDAqlYpBgwZx7NgxLl68yHfffce8efN49913Ddru27cPX1/fXBMxIcTjURSF5Qeu0Pfrg9yKTaG6mx2bhrehX/OqkqSUYZKolGOzZ8/WTzDN1qVLFyZNmsQHH3xA8+bNSUhIYNCgQSUWU+3atWndujV169alZcuWj2w/dOhQ1q5dazDEVLNmTUJDQwkKCmLcuHE0adKEZs2asXDhQt577z2mT5+e73icnZ3Zt28fGRkZPPfcc/j7+7NgwQLmzp3Lm2++adB2zZo1+snEQoiiFZ+awYjVIUzdHE6GVuGZRt5sHvUk9Ss5mjo0UcxUirlcO1sI8fHxODk5ERcXh6Oj4S9ramoqV65coUaNGvmabCnMg6Io1K5dm+HDhzN27Nh8tW/ZsiVjxoxhwIABJRChcadPn6Zjx46cP3/+kfVsxH/k71Tkx+nbcYxYFcLV6GQsNSomPFOPV1pXl16UUiyvz++Hlas5KsK83bt3j7Vr13Lnzp1ca6c8TKVS8b///Y+TJ08Wc3R5i4iIYMWKFZKkCFGEFEVh3dEbfPzradIzdVR2tmXRwAACqrqYOjRRgiRREWbD09MTd3d3/ve//+Hikv83In9/f/z9/YsvsHwICgoy6f6FKGuS0zOZuOkUG0NvAdCxridz+zbB2c7KxJGJkiaJijAbpXgUUghRhC7cTWD4qhAuRCaiUat4r7Mfbz7li1otQz3lkSQqQgghzMbPobcYt/EkKRlaPB2sWTgggJa+bqYOS5hQmU9U5Fu6EOZL/j5FttQMLVM3h7PmyHUA2tRyY16/ADwcrB+xpijrymyikl2IKzk5Oc+iYUII08m+3UBRFRQUpdO16CSGrQwhPCIelQpGdazNO0/XRiNDPYIynKhoNBqcnZ3195qxs7OTS9mEMBOKopCcnExkZCTOzs4FKsInypZtpyJ4f/0JEtIyca1gxbx+/jxVR25DIf5TZhMVAG9vbyDnjfGEEObB2dlZ/3cqypf0TB2ztp7luwNXAGhWzYWFAwOo6CQ94MJQmU5UVCoVFStWxNPT0+iN9YQQpmNpaSk9KeXUrdgURq4OIfR6LABvPOXL+138sNRIsXSRU5lOVLJpNBp5QxRCCDOw+1wkY9aFEZucgaONBXP6NKFzA+lVE7krF4mKEEII08rU6vjyz/Ms3n0JgEaVnVgSHIiPq9zRXuRNEhUhhBDFKjI+lbfXhnLocgwALz9RjYnP1sPaQnq6xaNJoiKEEKLY/H0pirfXhBGVmEYFKw0zX2zMc00qmTosUYpIoiKEEKLI6XQKS/ZcZO4f59Ep4OflwJKXAqnpYW/q0EQpI4mKEEKIIhWTlM6YdWH8df4eAL2bVmH68w2xtZKhHlFwkqgIIYQoMseu3Wfk6hAi4lKxtlAzvWdD+jbzMXVYohSTREUIIcRjUxSFb/dfYdbWs2TqFHzdK7A4OJB6FR1NHZoo5SRREUII8VjiUjL4YMNxtp++C8CzjSsy68XG2FvLR4x4fPJbJIQQotBO3Ypj+KoQrsckY6VRM+nZerz0RDW5t5ooMpKoCCGEKDBFUVh95DpTN4eTnqmjiostS4IDaVzF2dShiTJGEhUhhBAFkpSWyfhNJ/kl7DYAQfW8+KJPE5zsLE0cmSiLJFERQgiRb+fvJjBs5TEu3UtCo1bxYVc/Xm/rK0M9othIoiKEECJfNobcZMKmU6RkaPF2tGHRwACaVXc1dViijJNERQghRJ5SM7RM+fU0a4/eAKBtbXfm9fPHzd7axJGJ8kASFSGEELm6EpXEsJXHOHsnAZUKRj9dh5Eda6FRy1CPKBmSqAghhDBqy4kIPvzpBIlpmbjbWzG/fwBtarmbOixRzkiiIoQQwkB6po4Zv5/h+7+vAtCiuisLBwbg5Whj2sBEuSSJihBCCL2b95MZsTqU4zdiAXirXU3e61wHC43atIGJcksSFSGEEADsPHOXsT8eJy4lAydbS+b2bcLT9bxMHZYo5yRREUKIci5Tq2POjvN89dclAJpUcWLRwEB8XO1MHJkQYNK+vKVLl9K4cWMcHR1xdHSkVatWbN261ZQhCSFEuXI3PpWByw7rk5TBrauz/q3WkqQIs2HSHpUqVaowa9YsateujaIo/PDDDzz//POEhobSoEEDU4YmhBBl3v4LUbyzNpTopHTsrS2Y/WJjujeuaOqwhDCgUhRFMXUQD3J1deXzzz/ntddee2Tb+Ph4nJyciIuLw9HRsQSiE0KI0k+rU1i06yLzdp5HUaCutwNLggPx9bA3dWiinCjI57fZzFHRarWsX7+epKQkWrVqZbRNWloaaWlp+ufx8fElFZ4QQpQJ0YlpjF4Xxr4LUQD0b+7DlOcaYGOpMXFkQhhn8kTl5MmTtGrVitTUVOzt7dm0aRP169c32nbmzJlMnTq1hCMUQoiy4ejVGEatDuVOfCo2lmo+6dmI3k2rmDosIfJk8qGf9PR0rl+/TlxcHBs2bOCbb77hr7/+MpqsGOtR8fHxkaEfIYTIg6IoLNt3mdnbzqHVKdT0qMCS4Kb4eTuYOjRRThVk6MfkicrDgoKCqFmzJl9//fUj28ocFSGEyFtccgbvrj/On2fuAvBck0rM7NWICtYm71AX5VipnKOSTafTGfSaCCGEKJwTN2MZviqEm/dTsNKo+bhHfYJbVkWlkhsKitLDpInKuHHj6NatG1WrViUhIYHVq1ezZ88etm/fbsqwhBCiVFMUhZWHrjH9tzOka3X4uNqyNLgpDSs7mTo0IQrMpIlKZGQkgwYNIiIiAicnJxo3bsz27dvp1KmTKcMSQohSKzEtk3EbT7L5+G0AOtf34vM+TXCytTRxZEIUjkkTlW+//daUuxdCiDLl7J14hq8M4XJUEhZqFR91q8trT9aQoR5RqpndHBUhhBAFt/6fG0z65RSpGToqOtmwaGAgTau5mDosIR6bJCpCCFGKpaRr+fiXU6w/dhOAdnU8+LKfP64VrEwcmRBFQxIVIYQopS7dS2TEqhDO3klArYKxneowvH0t1GoZ6hFlhyQqQghRCm0+fpuPfjpBUroWd3trFgzwp3VNd1OHJUSRk0RFCCFKkbRMLZ9uOcOKg9cAaFnDlYUDAvB0tDFxZEIUD0lUhBCilLgRk8yI1SGcuBkHwIgONRkTVAcLjdrEkQlRfCRREUKIUuCP8Lu8+2MY8amZONtZ8mU/fzr4eZo6LCGKnSQqQghhxjK0Oj7ffo7/7b0MQEBVZxYNDKSys62JIxOiZEiiIoQQZioiLoVRq0P559p9AIa0qcFH3epiZSFDPaL8kERFCCHM0N7z9xi9LoyYpHQcrC34vE9jujasaOqwhChxkqgIIYQZ0eoU5u+8wMJdF1AUaFDJkSXBgVRzq2Dq0IQwCUlUhBDCTNxLSGP0ulAOXIwGYGDLqnz8bH1sLDUmjkwI05FERQghzMDhy9GMWhNKZEIatpYaZvRqyAsBVUwdlhAmJ4mKEEKYkE6n8PXey8zZcQ6tTqG2pz1LggOp7eVg6tCEMAuSqAghhInEJqfz7o/H2Xk2EoAXAirz6QsNsbOSt2YhsslfgxBCmEDYjVhGrArhVmwKVhZqpj7XgP7NfVCp5IaCQjxIEhUhhChBiqLww99X+fT3M2RoFaq52bEkOJAGlZxMHZoQZkkSFSGEKCEJqRl89NNJtpyMAKBbQ29m926Mo42liSMTwnxJoiKEECUg/HY8w1cd42p0MhZqFeOfqcerbarLUI8QjyCJihBCFCNFUfjxnxt8/Mtp0jJ1VHKyYVFwIIFVXUwdmhClgiQqQghRTJLTM5n48yk2htwCoIOfB3P7+uNSwcrEkQlRekiiIoQQxeBiZALDV4Vw/m4iahW818WPt56qiVotQz1CFIQkKkIIUcR+CbvFuI0nSU7X4ulgzYIBATzh62bqsIQolSRREUKIIpKaoWX6b+GsOnwdgNY13ZjfPwAPB2sTRyZE6fXYiUpaWhrW1vJHKIQo365HJzN89TFO3YpHpYJRHWrxTlAdNDLUI8RjURd0ha1bt/LKK6/g6+uLpaUldnZ2ODo60q5dOz799FNu375dHHEKIYTZ2nbqDt0X7uPUrXhcK1jx/astGNvZT5IUIYqASlEUJT8NN23axIcffkhCQgLPPPMMLVq0oFKlStja2hITE8OpU6fYt28fBw8eZPDgwUyfPh0PD49iDT4+Ph4nJyfi4uJwdHQs1n0JIcTDMrQ6Zm89yzf7rwDQtJoLiwYGUNHJ1sSRCWHeCvL5ne9EpVWrVkycOJFu3bqhVufeEXPr1i0WLlyIl5cXY8aMKVjkBSSJihDCVG7HpjBydQgh12MBeL1tDT7oWhdLTYE7qoUod4olUTFHkqgIIUxh97lIxq4L435yBg42FnzRpwmdG3ibOiwhSo2CfH4XyVU/Wq2WkydPUq1aNVxcpNqiEKJsytTqmPfnBRbtvghAo8pOLB4YSFU3OxNHJkTZVag+ytGjR/Ptt98CWUlKu3btCAwMxMfHhz179hRlfEIIYRYiE1J56dvD+iTl5Seqsf6tVpKkCFHMCpWobNiwgSZNmgCwefNmrly5wtmzZxkzZgwTJkwo0gCFEMLUDl6K5pn5+zl0OQY7Kw3z+/szvWdDbCw1pg5NiDKvUIlKVFQU3t5Z47G///47ffr0oU6dOgwZMoSTJ08WaYBCCGEqOp3C4t0XCf7mEFGJafh5OfDryCd53r+yqUMTotwoVKLi5eVFeHg4Wq2Wbdu20alTJwCSk5PRaOQbhhCi9LuflM6QH47y+fZz6BR4MbAKP49oQy1Pe1OHJkS5UqjJtK+++ip9+/alYsWKqFQqgoKCADh8+DB169Yt0gCFEKKkHbt2n1GrQ7gdl4q1hZrpzzekb3MfU4clRLlUqERlypQpNGzYkBs3btCnTx99CX2NRsNHH31UpAEKIURJURSF7w5cZebvZ8jUKdRwr8CS4EDqVZTyB0KYitRREUIIID41gw/Wn2Db6TsAdG9UkVkvNsLBxtLEkQlR9pRIHZWjR4+ye/duIiMj0el0Bq/NnTu3sJsVQogSd+pWHMNXhXA9JhlLjYqJ3eszqFU1VCq5V48QplaoRGXGjBlMnDgRPz8/vLy8DP6Y5Q9bCFFaKIrC6iPXmbo5nPRMHZWdbVkSHEgTH2dThyaE+FehEpX58+fz3XffMXjw4CIORwghSkZSWiYTNp3k57CsO74/XdeTL/o2wdnOysSRCSEeVKhERa1W06ZNm6KORQghSsSFuwkMWxXCxchENGoVH3Tx4/W2vqjV0iMshLkpVB2VMWPGsHjx4qKORQghit3GkJs8t+gAFyMT8XK0Zs3rT/Bmu5qSpAhhpgrVo/Lee+/RvXt3atasSf369bG0NJwVv3HjxiIJTgghikpqhpapm0+z5sgNAJ6s5c68/v6421ubODIhRF4Klai8/fbb7N69mw4dOuDm5iYTaIUQZu1qVBLDV4UQHhGPSgXvPF2bUR1ro5FeFCHMXqESlR9++IGffvqJ7t27F3U8QghRpLaejOD9DSdITMvErYIV8/r707a2h6nDEkLkU6ESFVdXV2rWrFnUsQghRJFJz9Qxc+sZlh+4CkDz6i4sHBCIt5ONaQMTQhRIoSbTTpkyhcmTJ5OcnFzU8QghxGO7FZtC368P6pOUt9rVZM3rT0iSIkQpVKgelQULFnDp0iW8vLyoXr16jsm0ISEhRRKcEEIU1K6zdxn743FikzNwsrXkiz5NCKrvZeqwhBCFVKhEpWfPnkUchhBCPJ5MrY65f5xnyZ5LADSp4sSigYH4uNqZODIhxOOQmxIKIUq9u/GpjFoTypErMQAMbl2dcc/UxdpCY+LIhBDGFMtNCRVFkcuQhRBm58DFKN5ZG0pUYjr21hbMerERzzauZOqwhBBFJN+TaRs0aMDatWtJT0/Ps92FCxcYNmwYs2bNeuzghBAiNzqdwoKdF3jp28NEJaZT19uBX0e2kSRFiDIm3z0qCxcu5MMPP2T48OF06tSJZs2aUalSJWxsbLh//z7h4eHs37+f06dPM3LkSIYNG1accQshyrHoxDRGrwtj34UoAPo182Hq8w2wsZShHiHKmgLPUdm/fz/r1q1j3759XLt2jZSUFNzd3QkICKBLly4EBwfj4uJSXPEakDkqQpQ//1yNYeTqUO7Ep2JjqeaTno3o3bSKqcMSQhRAQT6/ZTKtEKJUUBSFb/ZdYda2s2h1Cr4eFVga3BQ/bwdThyaEKKBimUwrhBCmEpecwXsbjvNH+F0AejSpxMxejbC3lrcwIco6+SsXQpi1kzfjGL76GDdiUrDSqJnUoz4vtawqVyEKUU5IoiKEMEuKorDy0DWm/3aGdK0OH1dblgxsSqMqTqYOTQhRgiRREUKYncS0TMZtPMnm47cB6FTfizm9m+BkZ/mINYUQZY0kKkIIs3L2TjzDV4Vw+V4SFmoVH3Wry2tP1pChHiHKqULdPRng0qVLTJw4kQEDBhAZGQnA1q1bOX36dJEFJ4QoX9b/c4Oeiw9w+V4S3o42rH3jCYa29ZUkRYhyrFCJyl9//UWjRo04fPgwGzduJDExEYDjx48zefLkIg1QCFH2paRr+WDDcd7fcILUDB1P1fFgy9tP0qy6q6lDE0KYWKESlY8++ohPPvmEP/74AysrK/3yjh07cujQoSILTghR9l2+l8gLSw7w4z83Uavg3U51+H5wc9zsrU0dmhDCDBRqjsrJkydZvXp1juWenp5ERUU9dlBCiPLhtxO3+XDDCZLStbjbW7GgfwCta7mbOiwhhBkpVKLi7OxMREQENWrUMFgeGhpK5cqViyQwIUTZlZapZcaWM/xw8BoALWq4smhAAJ6ONiaOTAhhbgo19NO/f38+/PBD7ty5g0qlQqfTceDAAd577z0GDRpU1DEKIcqQGzHJ9P3qoD5JGd6+JquHtpQkRQhhVKESlRkzZlC3bl18fHxITEykfv36PPXUU7Ru3ZqJEyfmezszZ86kefPmODg44OnpSc+ePTl37lxhQhJClAJ/ht+l+4J9HL8Zh5OtJd8NbsYHXetioSn0BYhCiDLusW5KeP36dU6dOkViYiIBAQHUrl27QOt37dqV/v3707x5czIzMxk/fjynTp0iPDycChUqPHJ9uSmhEKVDhlbHnB3n+PqvywD4+zizaGAAVVzsTByZEMIUSu3dk+/du4enpyd//fUXTz311CPbS6IihPm7E5fKqDUhHL16H4AhbWrwUbe6WFlIL4oQ5VWx3z1ZURQ2bNjA7t27iYyMRKfTGby+cePGwmyWuLg4AFxdjddOSEtLIy0tTf88Pj6+UPsRQpSMfRfuMXptGNFJ6ThYW/BZ78Z0a1TR1GEJIUqRQiUqo0eP5uuvv6ZDhw54eXkVSdVInU7H6NGjadOmDQ0bNjTaZubMmUydOvWx9yWEKF5ancKCnRdYsOsCigL1KzqyJDiQ6u6PHtIVQogHFWrox9XVlZUrV/LMM88UWSDDhg1j69at7N+/nypVqhhtY6xHxcfHR4Z+hDAjUYlpjF4bxv6LWTWVBrSoyuQe9bGx1Jg4MiGEuSj2oR8nJyd8fX0LFZwxI0eO5LfffmPv3r25JikA1tbWWFtLtUohzNWRKzGMXB1CZEIatpYaZvRqyAsBuf9NCyHEoxRqNtuUKVOYOnUqKSkpj7VzRVEYOXIkmzZtYteuXTkKyAkhSgedTuGrvy4xYNkhIhPSqOVpz68j20iSIoR4bIXqUenbty9r1qzB09OT6tWrY2lpafB6SEhIvrYzYsQIVq9ezS+//IKDgwN37twBsnpsbG1tCxOaEKKExSan8+6Px9l5Nusu6j39K/HpC42oYF2otxchhDBQqHeSV155hWPHjvHSSy891mTapUuXAtC+fXuD5cuXL2fw4MGF2qYQouSE3YhlxKoQbsWmYGWhZkqPBgxo4VMkE+yFEAIKmahs2bKF7du38+STTz7Wzs2ohIsQogAUReGHv6/y6e9nyNAqVHOzY/HAQBpWdjJ1aEKIMqZQiYqPj49cZSNEOZWQmsFHP51ky8kIALo28OazPo1xtLF8xJpCCFFwhZpM+8UXX/DBBx9w9erVIg5HCGHOwm/H89yiA2w5GYGFWsXHz9Zn6UuBkqQIIYpNoXpUXnrpJZKTk6lZsyZ2dnY5JtPGxMQUSXBCCPOgKAo//nODj385TVqmjkpONiwKDiSwqoupQxNClHGFSlTmzZtXxGEIIcxVcnomk34+zU8hNwFo7+fBl339calgZeLIhBDlQaGv+hFClH0XIxMZvuoY5+8molbBu539GNauJmq1XNUjhCgZ+U5U4uPj9RNoH3UzQJloK0Tp90vYLcZtPElyuhYPB2sW9A+gVU03U4clhChn8p2ouLi4EBERgaenJ87OzkbrJCiKgkqlQqvVFmmQQoiSk5qh5ZMt4aw8dB2AVr5uzB/gj6eDjYkjE0KUR/lOVHbt2oWrqysAu3fvLraAhBCmcz06meGrj3HqVlav6aiOtRgdVAeNDPUIIUwk34lKu3bt9P+vUaMGPj45q08qisKNGzeKLjohRInZfvoO760/TkJqJi52lnzZz5/2fp6mDksIUc4VajJtjRo19MNAD4qJiaFGjRoy9CNEKZKh1TF761m+2X8FgMCqziwaGEglZ7nflhDC9AqVqGTPRXlYYmIiNjYyji1EaXE7NoWRq0MIuR4LwNAna/Bht7pYagpVC1IIIYpcgRKVsWPHAqBSqZg0aRJ2dnb617RaLYcPH8bf379IAxRCFI+/zt9j9NpQ7idn4GBjwee9m9C1obepwxJCCAMFSlRCQ0OBrB6VkydPYmX1X8EnKysrmjRpwnvvvVe0EQohipRWpzDvz/Ms2n0RRYEGlRxZEhxINbcKpg5NCCFyKFCikn21z6uvvsr8+fOlXooQpUxkQirvrAnj4OVoAF56oioTu9fHxlJj4siEEMK4Qs1RWb58eVHHIYQoZocuRzNqTSj3EtKws9Iws1cjnvevbOqwhBAiT4VKVIQQpYdOp7D0r0t8seMcOgXqeNmzJLgptTztTR2aEEI8kiQqQpRh95PSGftjGLvP3QOgV2BlPunZEDsr+dMXQpQO8m4lRBkVcv0+I1eFcDsuFWsLNdOfb0ifZlWMlhYQQghzJYmKEGWMoih8d+AqM38/Q6ZOoYZ7BRYPDKR+JZn8LoQofSRREaIMiU/N4IP1J9h2+g4A3RtVZNaLjXCwsTRxZEIIUTiSqAhRRpy6FceI1SFci07GUqNiYvf6DGpVTYZ6hBClmiQqQpRyiqKw5sgNpmw+TXqmjsrOtiwODsTfx9nUoQkhxGOTREWIUiwpLZOJP59iU+gtAJ6u68kXfZvgbGf1iDWFEKJ0kERFiFLqwt0Ehq0K4WJkIhq1ive7+PFGW1/UahnqEUKUHZKoCFEKbQq9yfiNp0jJ0OLpYM3CAQG09HUzdVhCCFHkJFERohRJzdAydXM4a45cB6BNLTfm9w/A3d7axJEJIUTxkERFiFLialQSw1eFEB4Rj0oFb3eszdtP10YjQz1CiDJMEhUhSoGtJyP4YMMJEtIyca1gxbx+/jxVx8PUYQkhRLGTREUIM5aeqWPm1jMsP3AVgGbVXFg0MBBvJxvTBiaEECVEEhUhzNSt2BRGrAoh7EYsAG8+5ct7Xfyw1KhNG5gQQpQgSVSEMEO7z0Yy5scwYpMzcLSx4Iu+/nSq72XqsIQQosRJoiKEGcnU6pj7x3mW7LkEQOMqTiweGIiPq52JIxNCCNOQREUIMxEZn8qoNaEcvhIDwKBW1ZjQvR7WFhoTRyaEEKYjiYoQZuDvS1G8vSaMqMQ0KlhpmPViY3o0qWTqsIQQwuQkURHChHQ6hcW7L/Lln+fRKVDX24HFwYHU9LA3dWhCCGEWJFERwkRiktIZvS6MvefvAdCnaRWmPd8QWysZ6hFCmJBOB7HX4O7prIdnPaj/nMnCkURFCBM4di2GkatDiYhLxcZSzbTnG9K3mY+pwxJClDep8RAZDndPwZ1TWYlJZDikJ/7XpkEvSVSEKC8UReHb/VeYtfUsmToFX48KLAkOpK63o6lDE0KUZTotxFzOSkiye0runoLY68bba6zAoy54NYQaT5VsrA+RREWIEhKXksH764+zI/wuAD2aVGJmr0bYW8ufoRCiCCXHPJCQZPeSnIHMVOPtHSuDV4OspCT7X7eaoLEs2bhzIe+QQpSAkzfjGL76GDdiUrDSqJnUoz4vtayKSiU3FBRCFJI2A6IuGCYkd09Dwm3j7S1swau+YVLiWR/sXEs27gKSREWIYqQoCisPX2f65nDStTqquNiyJDiQxlWcTR2aEKK0UBRIjHxo2OY03DsLugzj67hUf6CH5N/ExKU6qEvfZH1JVIQoJolpmYzfeJJfj2d9u+lU34s5vZvgZGce3alCCDOUkQpR5/6b2JqdnCRHGW9v5fBAMpLdU1IfrB1KNu5iJImKEMXg3J0Ehq06xuV7SWjUKj7qWpehbWvIUI8QIouiQPytnMM2URdA0eZsr1KDa82H5pI0AOeqUMbfVyRREaKIbTh2k4k/nyQ1Q4e3ow2LBgbQrLp5jwELIYpRelLWZNaHr7hJjTPe3tbl32TkgYTEoy5Ylc97fkmiIkQRSc3QMvmX06z75wYAbWu7M6+fP2721iaOTAhRIh4ulHb3VNYj5gqg5GyvtgD3Og8N2zQAh4plvpekICRREaIIXL6XyPBVIZy9k4BKBWOC6jCiQy00anmzEaJMSo2Du+GGvSQPF0p7kL1XzoTEvQ5YyBeZR5FERYjHtOVEBB/+dILEtEzc7a2Y3z+ANrXcTR2WEKIoPFwoLXuSa1w+CqV5NQDvhuDZAOw9SjbuMkQSFSEKKS1Ty4wtZ/jh4DUAWtRwZeGAALwcbUwcmRCiUApcKK1Kzitu3GqBRj5ai5KcTSEK4UZMMiNXh3D8ZtZkuGHta/JupzpYaNQmjkwI8UiZ6RBtrFBahPH2lnZZhdEMrripnzXpVRQ7SVSEKKCdZ+4y9sfjxKVk4GRryZf9mtCxrpepwxJCPKycF0orKyRRESKfMrU6Pt9xjq//ugxAEx9nFg8MoIpL+bxkUAizkpGalYAYXHGTR6E0a8ecwzae9cpUobSyQhIVIfLhTlwqb68J5cjVGAAGt67O+GfqYWUhQz1ClKgHC6XdOflfYhJ98dGF0rwfqE3i5COXAJcSkqgI8Qj7L0TxztpQopPSsbe24LPejXmmUUVThyVE2SeF0gSSqAiRK61OYeGuC8zfeQFFgXoVHVkSHEgN9wqmDk2IskWng9irOYdt8iyU5mekUJq39JKUQZKoCGFEVGIaY9aFse9C1vh2/+Y+THmuATaWMqFOiMdSJIXS/MDCqmTjFiYjiYoQDzl6NYaRq0O4G5+GraWGT3o25MWmVUwdlhCli04L0ZdyXnGTa6E0a/CsazhsI4XSBJKoCKGnKAr/23uZz7afQ6tTqOlRgaUvNaWOl1wFIESekqIh8oGqrXdPZV2B86hCad4N/+spca0phdKEUfJbIQQQl5zBu+vD+PNMJADP+1dixguNqGAtfyJC6EmhNGEC8i4syr3jN2IZsTqEm/dTsNKomfxcfQa2qIpKJuWJ8kpRIPGukUJp5/IolFbjoYSkQdYytVzCLx6PJCqi3FIUhf87dI1PfjtDulZHVVc7lgQH0rCyk6lDE6LkGC2UdgqSo423l0JpZUaGNoPo1OisR0rWIyoliujUf//993l7n/a82+xdk8UpiYoolxLTMvnopxP8diKry7pLAy8+79MERxtLE0cmRDFRFIi7mXPYJq9CaW61cl5xI4XSzFqmLpP7qfdzJBvZz2NSYohKiSIqNYq4tFzq0TzkRsKNYo46b5KoiHLnTEQ8I1aFcDkqCQu1inHP1GNIm+oy1CPKjrTErF6SByu33j0NuX0w2br+O7m1kWGhNEvbko1bGKVTdDmSj+iUaMNkJDXr3/up91GM1Z7JhYXKAlcbV9xs3XCzdcPd1h03m6x/3W3dcbN1o5J9pWI8unzEaNK9C1HCfjx6g0m/nCItU0dFJxsWDQykaTWZ2CdKKSmUVmopikJcWlyuPR/ZwzFRKVHcT72P1livVy7UKjUu1i76RCM7+dD/39YNd5usf52snVCrzHsekSQqolxISdcy6ZdTbDh2E4D2fh7M7euPawUpGiVKiZTYrMJoBkM34ZCRZLy9vbeRQml1pFBaMVIUhYSMBIOkw9jcj6iUKGJSY8jUZRZo+y7WLrn3fPybiLjZuuFi7YKmDN3tWRIVUeZdjExkxKoQzt1NQK2Cdzv7MaxdTdRq+QYpzJA2E2Iu57y/TVwu8wRyFEr7998K7iUbdxmlKArJmcm5TjR9sOcjOiWadF16gbbvaOWYo5fj4UTEzdYNFxsXLNXlcw6dJCqiTPv1+G3G/XSCpHQt7vbWLBjgT+ua8gYuzERSdM6EJK9CaU4+Oa+4kUJphZKSmfLIno/seSApmSkF2ra9pT3utu642rgaDL88nHy42rhipZEerkcx6W/33r17+fzzzzl27BgRERFs2rSJnj17mjIkUUakZWqZ/ls4Kw9llet+wteVBQMC8HSwMXFkolzKTIeo8zmvuEm8Y7y9ZYWswmgP9pB41gdb5xINu7RJ16Y/sucje3lSbkNmubC1sM2RaBjr+XCzccPGQt5nipJJE5WkpCSaNGnCkCFD6NWrlylDEWXI9ehkRqwO4eStrCscRnaoxeig2lhozHvCmCgDpFBakcvQZWRdUpuay9Uu/875iE6NJiE9oUDbttZY55jfkdvcDztLu2I6QvEoJk1UunXrRrdu3UwZgihjdpy+w7vrj5OQmomLnSVz+/nTwc/T1GGJsigjxUihtNN5FEpzyqVQmn3Jxm0GtDot99PuG7/a5aFkJDYttkDbtlBbGO3lMDb8UsGygpQlKAVK1cBmWloaaWlp+ufx8fEmjEaYkwytjs+2nWXZvisABFR1ZvHAQCo5Sx0I8ZhyFEo79UChNF3O9jkKpf1bm8SpSpm+BFin6IhNizVIPmJSY4wmIwWt9aFRaQx6PR5MQh7u+XC0cpTko4wpVYnKzJkzmTp1qqnDEGYmIi6FkatDOXbtPgCvPVmDD7vWxcpCus5FAaUlQuSZnEM3eRVK825oOGxThgqlKYpCfHp8rj0fDw7HxKTGFKjWhwqVvtCYsbkfDw6/lIZaH6L4lKpEZdy4cYwdO1b/PD4+Hh8fHxNGJExt7/l7jF4XRkxSOg7WFnzepwldG3qbOixh7rILpd05ZThsc/+K8fZqS/Dwyzl0Y+9V6npJFEUhMSPRaLJhbALq49T6eLjnw+By2zJW60MUn1KVqFhbW2NtbW3qMIQZ0OoU5v95noW7L6Io0KCSI0uCA6nmVsHUoQlzU04KpSVnJOd6tYvBPV5SogpV6yO3K1weTETKc60PUXxKVaIiBMC9hDTeWRvK35eyJi0ObFmVj5+tj42lfDsr17SZEHMp57BNnoXS6hkO25hZobTUzNQ8J5o+mIwUptbHwxNNjc39kFofwtRMmqgkJiZy8eJF/fMrV64QFhaGq6srVatWNWFkwlwdvhzNqDWhRCakYWelYcYLjegZUNnUYYmSlqNQ2kmIPAvaNOPt9YXSHqje6uprkkJp2bU+cr3Hy2PW+jA23PLgpbfZy6TWhygtTJqo/PPPP3To0EH/PHv+ySuvvML3339voqiEOdLpFL7ae4k528+hU6C2pz1LXwqklqeDqUMTxamUFErLrvXxcLJh7B4vBa31YaW2+q93w9bVaM9H9nOp9SHKIpMmKu3bt0dR8n+Jmiif7iel8+764+w6GwlAr4DKfPJCQ+ysZOSyzFAUSLiTMyGJOge5TeZ09c1ZKM25epEVSsuu9fHISqcp0dxPu1+gbVuoLfLu+bD5b7m9pb1cbivKNXmnF2Yt9Pp9Rq4O5VZsClYWaqY914B+zX3kjbs0M2GhtOxaH/m5x8v9tPvojNVJyYVGpdHf28XV1tUg2Xg4KZFaH0LknyQqwiwpisL3f19lxu9nyNAqVHezY3FwIA0qOZk6NJFfipI1kfXhXpI8C6XVznnFzSMKpWXX+shPz0d0anSBa3242LgYTTYevuGcs7Wz1PoQohhIoiLMTnxqBh/9dILfT2bNQ+jW0JvZvRvjaCOXPZotg0JpD1wC/MhCaY0eKJTmpy+Ull3rIzolmqi7x/Kc+xGdEk1GbvfRyYWztXOe93h5MPmwUMvbpBCmJH+Bwqycvh3HiFUhXI1OxlKjYvwz9Rjcurp0k5sLnS6rKNrDwzb5LJSW7F6LaMeKRKkhOru8emo0Udd+Ifqs4aW3abldwZMLByuHXCeaPjj3w9XWVWp9CFGKSKIizIKiKKw7eoOPfz1NeqaOys62LBoYQEBVF1OHVn6lxOZMSCLP5CiUlqpSEW2hIcrBi2jXqkQ5eBJt60S0pRXRaIlKjSE69SZRV8NIuViwWh8VLCsYJhu5XO3iauuKtUaKQQpRFkmiIkwuOT2TiZtOsTH0FgAd63ryRZ8muFSQIlMlwkihtIy7p4lOvE2URkO0Rv3vvxqiHKyItqxAlI09MRaWRKEl8cEqp7rbEHcbchnxAbDR2Bi9k62xsuu2FmXjnjlCiMKTREWY1MXIBIatDOFCZCIatYr3Ovvx5lO+qNUy1FPUMnQZ3I+5RNTto0RHniIq5gLR8TeJTokiWs1/yYhGTbyrBlzzKqSXaXDZsJXa6r9ejkfc48XOwk6G8oQQ+SaJijCZn0NvMW7jSVIytHg6WLNwQAAtfd1MHVap8mCtjwdvMBeVdJfouKtEJdzKmnCamUSsokV5OEGwJKtImhEWKg2u+ez5cLB0kORDCFEsJFERJS41Q8vUzeGsOXIdgDa13JjXLwAPB5ljAFm1PuLS4vJ1j5cC1fpQqVArCq6KCneNDW7WLrjZV8TduTpuTr6423kY9IA4WjvK5bZCCJOTREWUqGvRSQxbGUJ4RDwqFYzqWJt3nq6NpowP9TxY6yPPe7ykRBOTGkOmkks1ViNUioKLToebVou7VoubVod7phY3tSVu9pVxc/HF3aM+bhWb4lyxKRpbqUUjhCg9JFERJWbbqQjeX3+ChLRMXCtYMa+fP0/V8TB1WIWmKApJGUlGi4zFZF96+8BwTEFrfThZ2OGussRdq8M1LQn35DjctJm4ax9MSrS46MDiwUJp3v/WJnGsnGehNCGEKA0kURHFLj1Tx6ytZ/nuQFatjWbVXFg4MICKTuZ5RUdyRnLOno9U45VOU7WpBdq2g6WD4Z1srRxxy9TinpqAW2IUbvdv4nbvIm4pcRit9GHn9m/F1gfub+NRFyzlTrhCiLJJEhVRrG7FpjBydQih12MBeOMpX97v4oelpmTnPqRmphq9m62xuR/JmckF2radhZ3RAmMGE1CtXXBLTcA66sJ/lwGf+wPuXzW+UbVlVgLy8D1u7D2ll0QIUa5IoiKKze5zkYxZF0ZscgaONhbM6dOEzg28i2z7GdqMXJOPh3s+EjISCrTt7FofD9/J9uFKp242bthZ2hmunHI/q3z83dNwd++/hdLCISOXBMihYs7727jVBgupIyOEEJKoiCKXqdXx5Z/nWbz7EgCNKjuxJDgQH1e7R6wJmbpMYlJjHt3zkRpNXG73kcmFpdrS6GW2xsqu56vWhzYz6wZ7D95w7+5piL9pvL2FTdZdfx9MSDwbQAW5JFsIIXIjiYooUpHxqby9NpRDl2MAePmJaox7pg7J2njOxdzIc+5HTGoM91Pvo6Dke38WKgtcbV1zJBrGbjj3WLU+kqIeSkhOQeRZyO1+NE5VH5jc+u+cEldfUGsKt38hhCinJFERhaJTdMSnxROVEqVPNv65eY1Nx8+RqovFvloiFV0z2Z8eT6u1Mfmv9QGoVWpcbVxz9nzY5Kx0WuS1PjLTIOq84f1t7p6GxLvG21vZg2f9h4Zu6oONXAIshBBFQRIVoacoCgkZCQZDLbnN/YhJyaXWhyP6q1XuPHBBjAoVLjYuuNq4/tfzYeNutNKps7UzmuLueVAUSLjzbyJy8r+EJOq8QWn4B48AV1/DYRuvBuBcDdRSFE0IIYqLJCplXHatj7wmmj6YjBS01oejlRPpaXYkJtuiZDpQy60izzbww9vesOy6i40LFmoT/bplpGTd9ffhOwGnxBhvb+NkmIx4Ncy6AsfavmTjFkIIIYlKaZWckZxrsvHg3I/HrfWR19yPa5FqRq89yb24VKwt1Ezv2ZC+zXyK6YjzQVEg9nrOhCTmEhgbelJpwL12zitupFCaEEKYDUlUzEiaNu2/pMPIRNMHe0QKU+vj4Tof+uTjgbkfrjau2FjkXTxMURS+3X+FWVvPkqlT8HWvwOLgQOpVdHycwy+YtIR/e0n+TUbunMq6BDgt3nj77EJp2VVbvRqAu58UShNCCDMniUoxe7DWx6Pu8VLQWh/WGuv/5ng82PPx4KTT3Gp9FFJcSgYfbDjO9tNZk0u7N67IrF6NcLAxWkf18em0WUXRHr7iRgqlCSFEuSCJShFKzUxl6fGlnIo6pe8RKUytj4eTDYMJqA8Mv1SwrFD4y20L4dStOIavCuF6TDKWGhWTnq3Py09UK7oY9IXSTv2XmESeyaNQWqWcwzbutUFTTEmTEEKIEieJShGae2wua86uybHcQmWRdbltHgXGsns+HK0cSzT5yA9FUVh95DpTN4eTnqmjiostiwcG0sTHuXAbfOxCaf8mJXauhT4mIYQQpYMkKkXkwK0D+iTlvWbv4efqp+8VcbJ2KtpaHyUoKS2T8ZtO8kvYbQCC6nnyRR9/nOzy2WuReC/nsM29c7kXSnOumvOKGymUJoQQ5ZYkKkZkRkdz7eVBpF++jIW3NxoHe9T2Dqgd7NHYO6B2cDBYlmqjYd3pBdRTKTxVtyv9bZ6EZLIexJChuv/QHlR5P324R+Vxnz+0g5wvG1//clQS4zee4FpMCp4qFcPa1yS4ZRVUCTEYTqdRZSUeMZfh3nmIOpv1772zkBz18OFlhWPpAB5+/z7q/vuoDdaOOeOJy2WCrD5c8zhfhX2e8/wU8fbNrIdOCCEKQqUoSv7rlZuZ+Ph4nJyciIuLw9Gx6K44ifrfMu7NnVtk2xPCbJW2pKuUbz/ndxTZfp7bf8SXiGI9ntJ2ropx+xVaPoH7W29SlAry+S09KkbYNW8GgMbNDZ+vvkKXEI82IRFdYgLahAR0+v8ncvPOOa7cDqdCGvhaeGORnA6Z/1U2zZEFPpwXPubzot6+oihodQq6f5erUdCo/t2TouRcPwfVf/+qjARYevPisulRvx8F3dxjrW367QshcrJwNe2NUyVRMcIuIIBqq1djXasmmjwyvZsJN3l7c2+SMjS8HfA2zzZ+vQSjLEL/Fkq7e/EY23buxC3pAvVU16mhvoua/BRK+7c2iWOlx74EOEcHn5kndjkTvRw7KOLt5/FRXdTbLvZjKe7tF+/xmHz7OU5g+T6ex9p+gbddtOeqwNsv4XNlWbkSpiSJSi7sAgPyfF2r0zJh/wSSMpII8AxgSMMhJRTZY0pLeOAS4H8nuP5bKM0LeAXgwXmrdu7/3f23BAqlPXK+yeNuv0i3JoQQorhJolJIy08vJyQyBDsLO2Y8OaP4b6JXUA8XSrvzb22S2GtGm6cpFlxUKhNlV5vAFk/iUK3Jf4XShBBCCBORRKUQzkSfYXHYYgA+avERVRyqmDag5JisXpEH72+Tj0Jp8U5+LDtvy/Yody4rFRnazo/3OtfBQlM6L6UWQghR9kiiUkCpmal8tO8jMnWZPF31aXrW6llyOzcolPbA0E38LePtLWwfKpTWQF8obeeZu4z98ThxKRk42Vrydd8mPF3Pq+SORQghhMgHSVQKaF7IPC7HXcbd1p3JrSYXX40Ko4XSzoI23Xh7faG0hg8USquRo1BaplbHnK1n+eqvSwA0qeLEooGB+LgWzb2AhBBCiKIkiUoB/H37b1adWQXAtNbTcLFxefyNZqZlVWp9cNjm7mlIijTe3so+5w33POuBjdMjd3U3PpVRq0M5cjUGgMGtqzP+mXpYWchQjxBCCPMkiUo+xaXFMWn/JAD6+fWjbZW2BduAokBChGFCcucURF8AXaaRFVTgVjPnsI1TVVAXPLHYfyGKd9aGEp2Ujr21BbNfbEz3xhULvB0hhBCiJEmikg+KojDt4DQiUyKp7lidd5u9m/cK6clw74zhDffunsq6O7AxNs4572/jWResKjx27FqdwqJdF5m38zyKAnW9HVgSHIivh/1jb1sIIYQobpKo5MNvl39jx7UdWKgsmNV2FrYWtsYb3g6Dze9AxHGM1tBUacC9jmFCUkSF0oyJTkxj9Low9l2IAqB/cx+mPNcAG0szu5RaCCGEyIUkKo9wO/E2Mw7PAOCtJm/RwL2B8YYhK2DLe//dFdigUNq/CYmHH1hYl0jcR6/GMGp1KHfiU7GxVPNJz0b0bmriy6iFEEKIApJEJQ9anZbx+8eTmJFIE48mvNbotZyNMlKyEpSwlVnP63SDZ+dm9ZKYgKIoLNt3mdnbzqHVKdT0qMCS4Kb4eTuYJB4hhBDicUiikocfwn/g2N1j2FnYMfPJmVioHzpdMZfhx0Fw5ySo1NBxErQZXajJrkUhLjmDd9cf588zdwF4rkklZvZqRAVr+TELIYQoneQTzIid13Yyes9o/fMPW3yIj6OPYaNzW2Hjm5AWlzXM0/s78G1XsoE+4MTNWIavCuHm/RSsNGo+7lGf4JZVi6/OixBCCFECJFExIlP573Lhjj4deaHWC/+9qM2E3Z/C/rlZz6u0gL4/mHSoZ+Wha0z/7QzpWh0+rrYsDW5Kw8qPrqsihBBCmDtJVIxo5tWMQM9AMnQZTG79QPXZxHvw0xC4sjfrecu3oNN0sLAySZyJaZmM23iSzcdvA9C5vhef92mCk62lSeIRQgghipokKka42brxQ7cfDBfeOAI/vgIJt8GyAjy3ABr1Nk2AwNk78QxfGcLlqCQs1Co+6laX156sIUM9QgghyhRJVB5FUeDI/2D7+KwKsu51oO//ZRVkM5H1/9xg0i+nSM3QUdHJhkUDA2hazdVk8QghhBDFRRKVvKQlwua34dRPWc8bvADPLQRr01zqm5Ku5eNfTrH+2E0Anqrjwbx+/rhWMM3QkxBCCFHcJFExJvEehP8Mv7+X9VxtAZ0/yZqTYqKhlUv3EhmxKoSzdxJQq2BMUB1GdKiFWi1DPUIIIcouSVSMubTzvyTFoSL0+R6qPmGycDYfv81HP50gKV2Lu701C/r707qWu8niEUIIIUqKJCrGVGkOti5Z/z6/GOw9TRJGWqaWT7ecYcXBawC0rOHKwgEBeDramCQeIYQQoqRJomKMW0344IrJhnkAbsQkM2J1CCduxgEwokNNxgTVwUJjmqq3QgghhClIopIbEyYpf4Tf5d0fw4hPzcTZzpIv+/rToa5penWEEEIIU5JExYxkaHV8vv0c/9t7GYCAqs4sGhhIZWdbE0cmhBBCmIYkKmYiIi6FUatD+efafQCGtKnBR93qYmUhQz1CCCHKL0lUzMDe8/cYvS6MmKR0HKwt+LxPY7o2rGjqsIQQQgiTk0TFhLQ6hfk7L7Bw1wUUBRpUcmRJcCDV3CqYOjQhhBDCLEiiYiL3EtIYvS6UAxejARjYsiofP1sfG0uNiSMTQgghzIckKiZw+HI0o9aEEpmQhq2lhhm9GvJCQBVThyWEEEKYHUlUSpBOp/D13svM2XEOrU6htqc9S4IDqe1lmnsHCSGEEOZOEpUSEpuczrs/Hmfn2UgAXgiozKcvNMTOSn4EQgghRG7kU7IEhN2IZcSqEG7FpmBloWbqcw3o39wHlQmLygkhhBClgSQqxUhRFH74+yqf/n6GDK1CNTc7lgQH0qCSk6lDE0IIIUoFSVSKSUJqBh/9dJItJyMA6NbQm9m9G+NoY2niyIQQQojSQxKVYhB+O57hq45xNToZC7WK8c/U49U21WWoRwghhCggSVSKUHJ6Jj+F3OKT38JJy9RRycmGRcGBBFZ1MXVoQgghRKkkicpjUBSFK1FJ7D53jz3nIjl8OYZ0rQ6ADn4ezO3rj0sFKxNHKYQQQpReZpGoLF68mM8//5w7d+7QpEkTFi5cSIsWLUwdllGpGVoOXo5mz9lI9py/x7XoZIPXKzvbMrh1dV57sgZqtQz1CCGEEI/D5InKunXrGDt2LF999RUtW7Zk3rx5dOnShXPnzuHp6Wnq8AC4Fp3EnnP32H0ukoOXoknL1Olfs9SoaFHDlQ5+nrT386Cmh73MRRFCCCGKiEpRFMWUAbRs2ZLmzZuzaNEiAHQ6HT4+PowaNYqPPvooz3Xj4+NxcnIiLi4OR0fHIo3r1K04NobcYs+5SC5HJRm8VtHJhvZ+nnTw86B1LXfsrU2e7wkhhBClRkE+v036CZuens6xY8cYN26cfplarSYoKIiDBw/maJ+WlkZaWpr+eXx8fLHEdfhyNAO/OYxWl5XDWahVNK3mQoe6nnTw86SOl/SaCCGEECXBpIlKVFQUWq0WLy8vg+VeXl6cPXs2R/uZM2cyderUYo/rTnwqFaw01K3oyKutq9OmtrvUPxFCCCFMoFSNWYwbN46xY8fqn8fHx+Pj41Pk+3nevzLt6nigQoWTnSQoQgghhKmYNFFxd3dHo9Fw9+5dg+V3797F29s7R3tra2usra1LJDZnO7msWAghhDA1tSl3bmVlRdOmTdm5c6d+mU6nY+fOnbRq1cqEkQkhhBDCHJh86Gfs2LG88sorNGvWjBYtWjBv3jySkpJ49dVXTR2aEEIIIUzM5IlKv379uHfvHh9//DF37tzB39+fbdu25ZhgK4QQQojyx+R1VB5HcdZREUIIIUTxKMjnt0nnqAghhBBC5EUSFSGEEEKYLUlUhBBCCGG2JFERQgghhNmSREUIIYQQZksSFSGEEEKYLUlUhBBCCGG2JFERQgghhNmSREUIIYQQZsvkJfQfR3ZR3fj4eBNHIoQQQoj8yv7czk9x/FKdqCQkJADg4+Nj4kiEEEIIUVAJCQk4OTnl2aZU3+tHp9Nx+/ZtHBwcUKlURbrt+Ph4fHx8uHHjhtxHqAjJeS0ecl6Lh5zX4iHntXiUpvOqKAoJCQlUqlQJtTrvWSilukdFrVZTpUqVYt2Ho6Oj2f/ASyM5r8VDzmvxkPNaPOS8Fo/Scl4f1ZOSTSbTCiGEEMJsSaIihBBCCLMliUourK2tmTx5MtbW1qYOpUyR81o85LwWDzmvxUPOa/Eoq+e1VE+mFUIIIUTZJj0qQgghhDBbkqgIIYQQwmxJoiKEEEIIsyWJihBCCCHMVrlOVBYvXkz16tWxsbGhZcuWHDlyJM/269evp27dutjY2NCoUSN+//33Eoq0dCnIeV22bBlt27bFxcUFFxcXgoKCHvlzKK8K+vuabe3atahUKnr27Fm8AZZSBT2vsbGxjBgxgooVK2JtbU2dOnXkvcCIgp7XefPm4efnh62tLT4+PowZM4bU1NQSitb87d27lx49elCpUiVUKhU///zzI9fZs2cPgYGBWFtbU6tWLb7//vtij7NYKOXU2rVrFSsrK+W7775TTp8+rbz++uuKs7OzcvfuXaPtDxw4oGg0GuWzzz5TwsPDlYkTJyqWlpbKyZMnSzhy81bQ8zpw4EBl8eLFSmhoqHLmzBll8ODBipOTk3Lz5s0Sjty8FfS8Zrty5YpSuXJlpW3btsrzzz9fMsGWIgU9r2lpaUqzZs2UZ555Rtm/f79y5coVZc+ePUpYWFgJR27eCnpeV61apVhbWyurVq1Srly5omzfvl2pWLGiMmbMmBKO3Hz9/vvvyoQJE5SNGzcqgLJp06Y821++fFmxs7NTxo4dq4SHhysLFy5UNBqNsm3btpIJuAiV20SlRYsWyogRI/TPtVqtUqlSJWXmzJlG2/ft21fp3r27wbKWLVsqb775ZrHGWdoU9Lw+LDMzU3FwcFB++OGH4gqxVCrMec3MzFRat26tfPPNN8orr7wiiYoRBT2vS5cuVXx9fZX09PSSCrFUKuh5HTFihNKxY0eDZWPHjlXatGlTrHGWVvlJVD744AOlQYMGBsv69eundOnSpRgjKx7lcugnPT2dY8eOERQUpF+mVqsJCgri4MGDRtc5ePCgQXuALl265Nq+PCrMeX1YcnIyGRkZuLq6FleYpU5hz+u0adPw9PTktddeK4kwS53CnNdff/2VVq1aMWLECLy8vGjYsCEzZsxAq9WWVNhmrzDntXXr1hw7dkw/PHT58mV+//13nnnmmRKJuSwqS59ZpfqmhIUVFRWFVqvFy8vLYLmXlxdnz541us6dO3eMtr9z506xxVnaFOa8PuzDDz+kUqVKOf7AyrPCnNf9+/fz7bffEhYWVgIRlk6FOa+XL19m165dBAcH8/vvv3Px4kWGDx9ORkYGkydPLomwzV5hzuvAgQOJioriySefRFEUMjMzeeuttxg/fnxJhFwm5faZFR8fT0pKCra2tiaKrODKZY+KME+zZs1i7dq1bNq0CRsbG1OHU2olJCTw8ssvs2zZMtzd3U0dTpmi0+nw9PTkf//7H02bNqVfv35MmDCBr776ytShlWp79uxhxowZLFmyhJCQEDZu3MiWLVuYPn26qUMTZqBc9qi4u7uj0Wi4e/euwfK7d+/i7e1tdB1vb+8CtS+PCnNes82ZM4dZs2bx559/0rhx4+IMs9Qp6Hm9dOkSV69epUePHvplOp0OAAsLC86dO0fNmjWLN+hSoDC/rxUrVsTS0hKNRqNfVq9ePe7cuUN6ejpWVlbFGnNpUJjzOmnSJF5++WWGDh0KQKNGjUhKSuKNN95gwoQJqNXynbqgcvvMcnR0LFW9KVBOe1SsrKxo2rQpO3fu1C/T6XTs3LmTVq1aGV2nVatWBu0B/vjjj1zbl0eFOa8An332GdOnT2fbtm00a9asJEItVQp6XuvWrcvJkycJCwvTP5577jk6dOhAWFgYPj4+JRm+2SrM72ubNm24ePGiPvEDOH/+PBUrVpQk5V+FOa/Jyck5kpHsZFCR29EVSpn6zDL1bF5TWbt2rWJtba18//33Snh4uPLGG28ozs7Oyp07dxRFUZSXX35Z+eijj/TtDxw4oFhYWChz5sxRzpw5o0yePFkuTzaioOd11qxZipWVlbJhwwYlIiJC/0hISDDVIZilgp7Xh8lVP8YV9Lxev35dcXBwUEaOHKmcO3dO+e233xRPT0/lk08+MdUhmKWCntfJkycrDg4Oypo1a5TLly8rO3bsUGrWrKn07dvXVIdgdhISEpTQ0FAlNDRUAZS5c+cqoaGhyrVr1xRFUZSPPvpIefnll/Xtsy9Pfv/995UzZ84oixcvlsuTS6OFCxcqVatWVaysrJQWLVoohw4d0r/Wrl075ZVXXjFo/+OPPyp16tRRrKyslAYNGihbtmwp4YhLh4Kc12rVqilAjsfkyZNLPnAzV9Df1wdJopK7gp7Xv//+W2nZsqVibW2t+Pr6Kp9++qmSmZlZwlGbv4Kc14yMDGXKlClKzZo1FRsbG8XHx0cZPny4cv/+/ZIP3Ezt3r3b6Htl9nl85ZVXlHbt2uVYx9/fX7GyslJ8fX2V5cuXl3jcRUGlKNKvJoQQQgjzVC7nqAghhBCidJBERQghhBBmSxIVIYQQQpgtSVSEEEIIYbYkURFCCCGE2ZJERQghhBBmSxIVIYQQQpgtSVSEECY3ePBgevbsaeowhBD/2rt3Lz169KBSpUqoVCp+/vnnAm9DURTmzJlDnTp1sLa2pnLlynz66acF3k65vCmhEMK8zJ8/X+7pIoQZSUpKokmTJgwZMoRevXoVahvvvPMOO3bsYM6cOTRq1IiYmBhiYmIKvB2pTCuEEEKIXKlUKjZt2mTQ65mWlsaECRNYs2YNsbGxNGzYkNmzZ9O+fXsAzpw5Q+PGjTl16hR+fn6PtX8Z+hFClJgNGzbQqFEjbG1tcXNzIygoiKSkJIOhn6tXr6JSqXI8st8AAfbv30/btm2xtbXFx8eHt99+m6SkJNMclBDl0MiRIzl48CBr167lxIkT9OnTh65du3LhwgUANm/ejK+vL7/99hs1atSgevXqDB06tFA9KpKoCCFKREREBAMGDGDIkCGcOXOGPXv20KtXrxxDPj4+PkREROgfoaGhuLm58dRTTwFw6dIlunbtyosvvsiJEydYt24d+/fvZ+TIkaY4LCHKnevXr7N8+XLWr19P27ZtqVmzJu+99x5PPvkky5cvB+Dy5ctcu3aN9evXs2LFCr7//nuOHTtG7969C7w/maMihCgRERERZGZm0qtXL6pVqwZAo0aNcrTTaDR4e3sDkJqaSs+ePWnVqhVTpkwBYObMmQQHBzN69GgAateuzYIFC2jXrh1Lly7FxsamRI5HiPLq5MmTaLVa6tSpY7A8LS0NNzc3AHQ6HWlpaaxYsULf7ttvv6Vp06acO3euQMNBkqgIIUpEkyZNePrpp2nUqBFdunShc+fO9O7dGxcXl1zXGTJkCAkJCfzxxx+o1VkdwMePH+fEiROsWrVK305RFHQ6HVeuXKFevXrFfixClGeJiYloNBqOHTuGRqMxeM3e3h6AihUrYmFhYZDMZP9tXr9+XRIVIYT50Wg0/PHHH/z999/s2LGDhQsXMmHCBA4fPmy0/SeffML27ds5cuQIDg4O+uWJiYm8+eabvP322znWqVq1arHFL4TIEhAQgFarJTIykrZt2xpt06ZNGzIzM7l06RI1a9YE4Pz58wD6HtX8kqt+hBAmodVqqVatGmPHjuXEiRPExsbqazX89NNPDBgwgK1bt/L0008brBccHMzdu3f5888/TRC1EOVDYmIiFy9eBLISk7lz59KhQwdcXV2pWrUqL730EgcOHOCLL74gICCAe/fusXPnTho3bkz37t3R6XQ0b94ce3t75s2bh06nY8SIETg6OrJjx44CxSKTaYUQJeLw4cPMmDGDf/75h+vXr7Nx40bu3buXY6jm1KlTDBo0iA8//JAGDRpw584d7ty5o79a4MMPP+Tvv/9m5MiRhIWFceHCBX755ReZTCtEEfrnn38ICAggICAAgLFjxxIQEMDHH38MwPLlyxk0aBDvvvsufn5+9OzZk6NHj+p7NdVqNZs3b8bd3Z2nnnqK7t27U69ePdauXVvgWKRHRQhRIs6cOcOYMWMICQkhPj6eatWqMWrUKEaOHMngwYP1PSrff/89r776ao7127Vrx549ewA4evQoEyZM4ODBgyiKQs2aNenXrx/jx48v4aMSQhQ3SVSEEEIIYbZk6EcIIYQQZksSFSGEEEKYLUlUhBBCCGG2JFERQgghhNmSREUIIYQQZksSFSGEEEKYLUlUhBBCCGG2JFERQgghhNmSREUIIYQQZksSFSGEEEKYLUlUhBBCCGG2JFERQgghhNn6f77cFosfAEfDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 640x480 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Define a simple implementation of `sigmoid`, and benchmark it using\n",
    "# NumPy and TensorFlow NumPy for different input sizes.\n",
    "\n",
    "def np_sigmoid(y):\n",
    "  return 1. / (1. + np.exp(-y))\n",
    "\n",
    "def tnp_sigmoid(y):\n",
    "  return 1. / (1. + tnp.exp(-y))\n",
    "\n",
    "@tf.function\n",
    "def compiled_tnp_sigmoid(y):\n",
    "  return tnp_sigmoid(y)\n",
    "\n",
    "sizes = (2 ** 0, 2 ** 5, 2 ** 10, 2 ** 15, 2 ** 20)\n",
    "np_inputs = [np.random.randn(size).astype(np.float32) for size in sizes]\n",
    "np_times = benchmark(np_sigmoid, np_inputs)\n",
    "\n",
    "with tf.device(\"/device:CPU:0\"):\n",
    "  tnp_inputs = [tnp.random.randn(size).astype(np.float32) for size in sizes]\n",
    "  tnp_times = benchmark(tnp_sigmoid, tnp_inputs)\n",
    "  compiled_tnp_times = benchmark(compiled_tnp_sigmoid, tnp_inputs)\n",
    "\n",
    "has_gpu = len(tf.config.list_logical_devices(\"GPU\"))\n",
    "if has_gpu:\n",
    "  with tf.device(\"/device:GPU:0\"):\n",
    "    tnp_inputs = [tnp.random.randn(size).astype(np.float32) for size in sizes]\n",
    "    tnp_times_gpu = benchmark(compiled_tnp_sigmoid, tnp_inputs, 100, True)\n",
    "else:\n",
    "  tnp_times_gpu = None\n",
    "plot(np_times, tnp_times, compiled_tnp_times, has_gpu, tnp_times_gpu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ReK_9k5D8BZQ"
   },
   "source": [
    "## 延伸阅读\n",
    "\n",
    "- [TensorFlow NumPy：分布式图像分类教程](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/ops/numpy_ops/g3doc/TensorFlow_Numpy_Distributed_Image_Classification.ipynb)\n",
    "- [TensorFlow NumPy：Keras 和分布策略](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/ops/numpy_ops/g3doc/TensorFlow_NumPy_Keras_and_Distribution_Strategy.ipynb)\n",
    "- [使用 Trax 和 TensorFlow NumPy 进行情感分析](https://github.com/google/trax/blob/master/trax/tf_numpy_and_keras.ipynb)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "tf_numpy.ipynb",
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}