{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "rX8mhOLljYeM" }, "source": [ "##### Copyright 2022 The TensorFlow Authors." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "cellView": "form", "execution": { "iopub.execute_input": "2024-07-19T03:10:58.267342Z", "iopub.status.busy": "2024-07-19T03:10:58.266797Z", "iopub.status.idle": "2024-07-19T03:10:58.270797Z", "shell.execute_reply": "2024-07-19T03:10:58.270238Z" }, "id": "BZSlp3DAjdYf" }, "outputs": [], "source": [ "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ] }, { "cell_type": "markdown", "metadata": { "id": "3wF5wszaj97Y" }, "source": [ "# Quickstart for the TensorFlow Core APIs" ] }, { "cell_type": "markdown", "metadata": { "id": "DUNzJc4jTj6G" }, "source": [ "\n", " \n", " \n", " \n", " \n", "
\n", " View on TensorFlow.org\n", " \n", " Run in Google Colab\n", " \n", " View source on GitHub\n", " \n", " Download notebook\n", "
" ] }, { "cell_type": "markdown", "metadata": { "id": "04QgGZc9bF5D" }, "source": [ "This quickstart tutorial demonstrates how you can use the [TensorFlow Core low-level APIs](https://www.tensorflow.org/guide/core) to build and train a multiple linear regression model that predicts fuel efficiency. It uses the [Auto MPG](https://archive.ics.uci.edu/ml/datasets/auto+mpg){:.external} dataset which contains fuel efficiency data for late-1970s and early 1980s automobiles.\n", "\n", "You will follow the typical stages of a machine learning process:\n", "\n", "1. Load the dataset.\n", "2. Build an [input pipeline](../data.ipynb).\n", "3. Build a multiple [linear regression](https://developers.google.com/machine-learning/glossary#linear-regression){:.external} model.\n", "4. Evaluate the performance of the model." ] }, { "cell_type": "markdown", "metadata": { "id": "nnrWf3PCEzXL" }, "source": [ "## Setup\n", "\n", "Import TensorFlow and other necessary libraries to get started:" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2024-07-19T03:10:58.274266Z", "iopub.status.busy": "2024-07-19T03:10:58.274014Z", "iopub.status.idle": "2024-07-19T03:11:00.922030Z", "shell.execute_reply": "2024-07-19T03:11:00.921199Z" }, "id": "0trJmd6DjqBZ" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2024-07-19 03:10:58.529724: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n", "2024-07-19 03:10:58.550668: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n", "2024-07-19 03:10:58.557191: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "TensorFlow version: 2.17.0\n" ] } ], "source": [ "import tensorflow as tf\n", "import pandas as pd\n", "import matplotlib\n", "from matplotlib import pyplot as plt\n", "print(\"TensorFlow version:\", tf.__version__)\n", "# Set a random seed for reproducible results \n", "tf.random.set_seed(22)" ] }, { "cell_type": "markdown", "metadata": { "id": "7NAbSZiaoJ4z" }, "source": [ "## Load and preprocess the dataset\n", "\n", "Next, you need to load and preprocess the [Auto MPG dataset](https://archive.ics.uci.edu/ml/datasets/auto+mpg){:.external} from the [UCI Machine Learning Repository](https://archive.ics.uci.edu/ml/){:.external}. This dataset uses a variety of quantitative and categorical features such as cylinders, displacement, horsepower and weight to predict the fuel efficiencies of automobiles in the late-1970s and early 1980s.\n", "\n", "The dataset contains a few unknown values. Make sure to drop any missing values with `pandas.DataFrame.dropna`, and convert the dataset to a `tf.float32` tensor type with the `tf.convert_to_tensor` and `tf.cast` functions." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2024-07-19T03:11:00.926163Z", "iopub.status.busy": "2024-07-19T03:11:00.925752Z", "iopub.status.idle": "2024-07-19T03:11:03.303182Z", "shell.execute_reply": "2024-07-19T03:11:03.302360Z" }, "id": "HglhDsUfrJ98" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n", "I0000 00:00:1721358661.581093 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358661.584998 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358661.588229 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358661.591436 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358661.602597 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358661.605610 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358661.608434 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358661.611361 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358661.614314 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358661.617207 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358661.620201 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358661.623143 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.860014 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.862049 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.864099 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.866247 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.868335 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.870190 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.872147 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.874136 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.876124 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.877956 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.879910 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.881921 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.920748 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.922684 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.924672 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.926706 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.928705 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.930574 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.932534 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.934542 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.936548 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.938894 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.941284 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721358662.943710 188183 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
MPGCylindersDisplacementHorsepowerWeightAccelerationModel YearOrigin
39327.04140.086.02790.015.6821
39444.0497.052.02130.024.6822
39532.04135.084.02295.011.6821
39628.04120.079.02625.018.6821
39731.04119.082.02720.019.4821
\n", "
" ], "text/plain": [ " MPG Cylinders Displacement Horsepower Weight Acceleration \\\n", "393 27.0 4 140.0 86.0 2790.0 15.6 \n", "394 44.0 4 97.0 52.0 2130.0 24.6 \n", "395 32.0 4 135.0 84.0 2295.0 11.6 \n", "396 28.0 4 120.0 79.0 2625.0 18.6 \n", "397 31.0 4 119.0 82.0 2720.0 19.4 \n", "\n", " Model Year Origin \n", "393 82 1 \n", "394 82 2 \n", "395 82 1 \n", "396 82 1 \n", "397 82 1 " ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "url = 'http://archive.ics.uci.edu/ml/machine-learning-databases/auto-mpg/auto-mpg.data'\n", "column_names = ['MPG', 'Cylinders', 'Displacement', 'Horsepower', 'Weight',\n", " 'Acceleration', 'Model Year', 'Origin']\n", "\n", "dataset = pd.read_csv(url, names=column_names, na_values='?', comment='\\t',\n", " sep=' ', skipinitialspace=True)\n", "\n", "dataset = dataset.dropna()\n", "dataset_tf = tf.convert_to_tensor(dataset, dtype=tf.float32)\n", "dataset.tail()" ] }, { "cell_type": "markdown", "metadata": { "id": "0vgoDL3hYesB" }, "source": [ "Next, split the dataset into training and test sets. Make sure to shuffle the dataset with `tf.random.shuffle` to avoid biased splits." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2024-07-19T03:11:03.307273Z", "iopub.status.busy": "2024-07-19T03:11:03.307019Z", "iopub.status.idle": "2024-07-19T03:11:03.326900Z", "shell.execute_reply": "2024-07-19T03:11:03.326281Z" }, "id": "0mJU4kt6YiAp" }, "outputs": [], "source": [ "dataset_shuffled = tf.random.shuffle(dataset_tf, seed=22)\n", "train_data, test_data = dataset_shuffled[100:], dataset_shuffled[:100]\n", "x_train, y_train = train_data[:, 1:], train_data[:, 0]\n", "x_test, y_test = test_data[:, 1:], test_data[:, 0]" ] }, { "cell_type": "markdown", "metadata": { "id": "Bscb2Vsbi3TE" }, "source": [ "Perform basic feature engineering by one-hot-encoding the `\"Origin\"` feature. The `tf.one_hot` function is useful for transforming this categorical column into 3 separate binary columns." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2024-07-19T03:11:03.330570Z", "iopub.status.busy": "2024-07-19T03:11:03.329959Z", "iopub.status.idle": "2024-07-19T03:11:03.351740Z", "shell.execute_reply": "2024-07-19T03:11:03.351119Z" }, "id": "_B8N9IV1i6IV" }, "outputs": [ { "data": { "text/plain": [ "array([[ 4., 140., 72., ..., 1., 0., 0.],\n", " [ 4., 120., 74., ..., 0., 0., 1.],\n", " [ 4., 122., 88., ..., 0., 1., 0.],\n", " ...,\n", " [ 8., 318., 150., ..., 1., 0., 0.],\n", " [ 4., 156., 105., ..., 1., 0., 0.],\n", " [ 6., 232., 100., ..., 1., 0., 0.]], dtype=float32)" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "def onehot_origin(x):\n", " origin = tf.cast(x[:, -1], tf.int32)\n", " # Use `origin - 1` to account for 1-indexed feature\n", " origin_oh = tf.one_hot(origin - 1, 3)\n", " x_ohe = tf.concat([x[:, :-1], origin_oh], axis = 1)\n", " return x_ohe\n", "\n", "x_train_ohe, x_test_ohe = onehot_origin(x_train), onehot_origin(x_test)\n", "x_train_ohe.numpy()" ] }, { "cell_type": "markdown", "metadata": { "id": "qnoCDzzedite" }, "source": [ "This example shows a multiple regression problem with predictors or features on vastly different scales. Therefore, it is beneficial to standardize the data so that each feature has zero mean and unit variance. Use the `tf.reduce_mean` and `tf.math.reduce_std` functions for standardization. The regression model's prediction can then be unstandardized to obtain its value in terms of the original units." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2024-07-19T03:11:03.355159Z", "iopub.status.busy": "2024-07-19T03:11:03.354726Z", "iopub.status.idle": "2024-07-19T03:11:03.359012Z", "shell.execute_reply": "2024-07-19T03:11:03.358434Z" }, "id": "dJJFdvqydhyp" }, "outputs": [], "source": [ "class Normalize(tf.Module):\n", " def __init__(self, x):\n", " # Initialize the mean and standard deviation for normalization\n", " self.mean = tf.math.reduce_mean(x, axis=0)\n", " self.std = tf.math.reduce_std(x, axis=0)\n", "\n", " def norm(self, x):\n", " # Normalize the input\n", " return (x - self.mean)/self.std\n", "\n", " def unnorm(self, x):\n", " # Unnormalize the input\n", " return (x * self.std) + self.mean" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2024-07-19T03:11:03.362185Z", "iopub.status.busy": "2024-07-19T03:11:03.361703Z", "iopub.status.idle": "2024-07-19T03:11:03.520736Z", "shell.execute_reply": "2024-07-19T03:11:03.519892Z" }, "id": "5BONV6fYYwZb" }, "outputs": [], "source": [ "norm_x = Normalize(x_train_ohe)\n", "norm_y = Normalize(y_train)\n", "x_train_norm, y_train_norm = norm_x.norm(x_train_ohe), norm_y.norm(y_train)\n", "x_test_norm, y_test_norm = norm_x.norm(x_test_ohe), norm_y.norm(y_test)" ] }, { "cell_type": "markdown", "metadata": { "id": "BPZ68wASog_I" }, "source": [ "## Build a machine learning model\n", "\n", "Build a linear regression model with the TensorFlow Core APIs. The equation for multiple linear regression is as follows:\n", "\n", "$${\\mathrm{Y}} = {\\mathrm{X}}w + b$$\n", "\n", "where\n", "\n", "* $\\underset{m\\times 1}{\\mathrm{Y}}$: target vector\n", "* $\\underset{m\\times n}{\\mathrm{X}}$: feature matrix\n", "* $\\underset{n\\times 1}w$: weight vector\n", "* $b$: bias\n", "\n", "By using the `@tf.function` decorator, the corresponding Python code is traced to generate a callable TensorFlow graph. This approach is beneficial for saving and loading the model after training. It can also provide a performance boost for models with many layers and complex operations. " ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "execution": { "iopub.execute_input": "2024-07-19T03:11:03.525085Z", "iopub.status.busy": "2024-07-19T03:11:03.524803Z", "iopub.status.idle": "2024-07-19T03:11:03.531966Z", "shell.execute_reply": "2024-07-19T03:11:03.531333Z" }, "id": "h3IKyzTCDNGo" }, "outputs": [], "source": [ "class LinearRegression(tf.Module):\n", "\n", " def __init__(self):\n", " self.built = False\n", "\n", " @tf.function\n", " def __call__(self, x):\n", " # Initialize the model parameters on the first call\n", " if not self.built:\n", " # Randomly generate the weight vector and bias term\n", " rand_w = tf.random.uniform(shape=[x.shape[-1], 1])\n", " rand_b = tf.random.uniform(shape=[])\n", " self.w = tf.Variable(rand_w)\n", " self.b = tf.Variable(rand_b)\n", " self.built = True\n", " y = tf.add(tf.matmul(x, self.w), self.b)\n", " return tf.squeeze(y, axis=1)" ] }, { "cell_type": "markdown", "metadata": { "id": "l2hiez2eIUz8" }, "source": [ "For each example, the model returns a prediction for the input automobile's MPG by computing the weighted sum of its features plus a bias term. This prediction can then be unstandardized to obtain its value in terms of the original units." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "execution": { "iopub.execute_input": "2024-07-19T03:11:03.535132Z", "iopub.status.busy": "2024-07-19T03:11:03.534871Z", "iopub.status.idle": "2024-07-19T03:11:03.734307Z", "shell.execute_reply": "2024-07-19T03:11:03.733611Z" }, "id": "OeOrNdnkEEcR" }, "outputs": [ { "data": { "text/plain": [ "array([6.8007355], dtype=float32)" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "lin_reg = LinearRegression()\n", "prediction = lin_reg(x_train_norm[:1])\n", "prediction_unnorm = norm_y.unnorm(prediction)\n", "prediction_unnorm.numpy()" ] }, { "cell_type": "markdown", "metadata": { "id": "FIHANxNSvWr9" }, "source": [ "## Define a loss function\n", "\n", "Now, define a loss function to evaluate the model's performance during the training process.\n", "\n", "Since regression problems deal with continuous outputs, the mean squared error (MSE) is an ideal choice for the loss function. The MSE is defined by the following equation:\n", "\n", "$$MSE = \\frac{1}{m}\\sum_{i=1}^{m}(\\hat{y}_i -y_i)^2$$\n", "\n", "where\n", "\n", "* $\\hat{y}$: vector of predictions\n", "* $y$: vector of true targets\n", "\n", "The goal of this regression problem is to find the optimal weight vector, $w$, and bias, $b$, that minimizes the MSE loss function. " ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "execution": { "iopub.execute_input": "2024-07-19T03:11:03.737583Z", "iopub.status.busy": "2024-07-19T03:11:03.737325Z", "iopub.status.idle": "2024-07-19T03:11:03.740614Z", "shell.execute_reply": "2024-07-19T03:11:03.739978Z" }, "id": "8tYNVUkmw35s" }, "outputs": [], "source": [ "def mse_loss(y_pred, y):\n", " return tf.reduce_mean(tf.square(y_pred - y))" ] }, { "cell_type": "markdown", "metadata": { "id": "htI-7aJPqclK" }, "source": [ "## Train and evaluate your model\n", "\n", "Using mini-batches for training provides both memory efficiency and faster convergence. The `tf.data.Dataset` API has useful functions for batching and shuffling. The API enables you to build complex input pipelines from simple, reusable pieces. Learn more about building TensorFlow input pipelines in [this guide](https://www.tensorflow.org/guide/data)." ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "execution": { "iopub.execute_input": "2024-07-19T03:11:03.743779Z", "iopub.status.busy": "2024-07-19T03:11:03.743531Z", "iopub.status.idle": "2024-07-19T03:11:04.735966Z", "shell.execute_reply": "2024-07-19T03:11:04.735234Z" }, "id": "kxST2w_Nq0C5" }, "outputs": [], "source": [ "batch_size = 64\n", "train_dataset = tf.data.Dataset.from_tensor_slices((x_train_norm, y_train_norm))\n", "train_dataset = train_dataset.shuffle(buffer_size=x_train.shape[0]).batch(batch_size)\n", "test_dataset = tf.data.Dataset.from_tensor_slices((x_test_norm, y_test_norm))\n", "test_dataset = test_dataset.shuffle(buffer_size=x_test.shape[0]).batch(batch_size)" ] }, { "cell_type": "markdown", "metadata": { "id": "C9haUW8Yq3xD" }, "source": [ "Next, write a training loop to iteratively update your model's parameters by making use of the MSE loss function and its gradients with respect to the input parameters.\n", "\n", "This iterative method is referred to as [gradient descent](https://developers.google.com/machine-learning/glossary#gradient-descent){:.external}. At each iteration, the model's parameters are updated by taking a step in the opposite direction of their computed gradients. The size of this step is determined by the learning rate, which is a configurable hyperparameter. Recall that the gradient of a function indicates the direction of its steepest ascent; therefore, taking a step in the opposite direction indicates the direction of steepest descent, which ultimately helps to minimize the MSE loss function." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "execution": { "iopub.execute_input": "2024-07-19T03:11:04.740171Z", "iopub.status.busy": "2024-07-19T03:11:04.739906Z", "iopub.status.idle": "2024-07-19T03:11:08.071002Z", "shell.execute_reply": "2024-07-19T03:11:08.070005Z" }, "id": "y7suUbJXVLqP" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Mean squared error for step 0: 2.866\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Mean squared error for step 10: 0.453\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Mean squared error for step 20: 0.285\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Mean squared error for step 30: 0.231\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Mean squared error for step 40: 0.209\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Mean squared error for step 50: 0.203\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Mean squared error for step 60: 0.194\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Mean squared error for step 70: 0.184\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Mean squared error for step 80: 0.186\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Mean squared error for step 90: 0.176\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\n", "Final train loss: 0.177\n", "Final test loss: 0.157\n" ] } ], "source": [ "# Set training parameters\n", "epochs = 100\n", "learning_rate = 0.01\n", "train_losses, test_losses = [], []\n", "\n", "# Format training loop\n", "for epoch in range(epochs):\n", " batch_losses_train, batch_losses_test = [], []\n", "\n", " # Iterate through the training data\n", " for x_batch, y_batch in train_dataset:\n", " with tf.GradientTape() as tape:\n", " y_pred_batch = lin_reg(x_batch)\n", " batch_loss = mse_loss(y_pred_batch, y_batch)\n", " # Update parameters with respect to the gradient calculations\n", " grads = tape.gradient(batch_loss, lin_reg.variables)\n", " for g,v in zip(grads, lin_reg.variables):\n", " v.assign_sub(learning_rate * g)\n", " # Keep track of batch-level training performance \n", " batch_losses_train.append(batch_loss)\n", " \n", " # Iterate through the testing data\n", " for x_batch, y_batch in test_dataset:\n", " y_pred_batch = lin_reg(x_batch)\n", " batch_loss = mse_loss(y_pred_batch, y_batch)\n", " # Keep track of batch-level testing performance \n", " batch_losses_test.append(batch_loss)\n", "\n", " # Keep track of epoch-level model performance\n", " train_loss = tf.reduce_mean(batch_losses_train)\n", " test_loss = tf.reduce_mean(batch_losses_test)\n", " train_losses.append(train_loss)\n", " test_losses.append(test_loss)\n", " if epoch % 10 == 0:\n", " print(f'Mean squared error for step {epoch}: {train_loss.numpy():0.3f}')\n", "\n", "# Output final losses\n", "print(f\"\\nFinal train loss: {train_loss:0.3f}\")\n", "print(f\"Final test loss: {test_loss:0.3f}\")" ] }, { "cell_type": "markdown", "metadata": { "id": "4mDAAPFqVVgn" }, "source": [ "Plot the changes in MSE loss over time. Calculating performance metrics on a designated [validation set](https://developers.google.com/machine-learning/glossary#validation-set){:.external} or [test set](https://developers.google.com/machine-learning/glossary#test-set){:.external} ensures the model does not overfit to the training dataset and can generalize well to unseen data." ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "execution": { "iopub.execute_input": "2024-07-19T03:11:08.074765Z", "iopub.status.busy": "2024-07-19T03:11:08.074484Z", "iopub.status.idle": "2024-07-19T03:11:08.268290Z", "shell.execute_reply": "2024-07-19T03:11:08.267480Z" }, "id": "F7dTAzgHDUh7" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAwAAAAIjCAYAAAC0znyiAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/TGe4hAAAACXBIWXMAAA9hAAAPYQGoP6dpAACDkUlEQVR4nOzdd3xT9f7H8ddJ2ibde0IpZchegiCguECWA7xuvYLzqiAqV72X68KJP/fAPcCFE0VFUdkKsoeyZ9ktm+6ZnN8foYHaFhpok9K+n49HHm3OOTn5ZCjn3e8yTNM0ERERERGResHi6wJERERERMR7FABEREREROoRBQARERERkXpEAUBEREREpB5RABARERERqUcUAERERERE6hEFABERERGRekQBQERERESkHlEAEBERERGpRxQAREQq0LhxY4YOHerrMuoUwzAYPXr0CT3Wl5/HydTtC1u2bMEwDMaPH+/rUkSkllIAEBGvGz9+PIZhYBgGc+bMKbffNE2Sk5MxDIOLLrqozL6cnBweffRR2rZtS3BwMNHR0XTs2JG7776bXbt2uY8bPXq0+zkqumVkZNT46zzV/PTTT6fUha6v/PHHH4wePZpDhw75tI4JEybw8ssv+7QGETk1+fm6ABGpv+x2OxMmTOCss84qs3327Nns2LEDm81WZntxcTG9evVi7dq1DBkyhLvuuoucnBxWrVrFhAkTGDx4MElJSWUe8+abbxISElLuuSMiIqr99ZzqfvrpJ15//fUaCwH5+fn4+Z3YPzvr1q3DYvHN36z+Xvcff/zBY489xtChQ336PZowYQIrV67knnvuKbM9JSWF/Px8/P39fVOYiNR6CgAi4jMDBgzgq6++4tVXXy1zgTVhwgQ6d+7Mvn37yhw/adIkli1bxqeffsq1115bZl9BQQFFRUXlnuPyyy8nJiamZl5APVZSUoLT6SQgIKDKj7Hb7Sf8fH8Pg950MnV7Ii8vj6CgoJM+j2EYXqtZRE5N6gIkIj5zzTXXsH//fqZOnereVlRUxNdff13uAh9g06ZNAPTs2bPcPrvdTlhYWM0VC2zevJkrrriCqKgogoKCOPPMM/nxxx/LHffaa6/Rpk0bgoKCiIyMpEuXLkyYMMG9Pzs7m3vuuYfGjRtjs9mIi4ujT58+LF26tNLn/vrrrzEMg9mzZ5fb9/bbb2MYBitXrgQgIyODG2+8kYYNG2Kz2UhMTOTSSy9ly5YtlZ5/6NChvP766wBlukrBkT7lzz//PC+//DJNmzbFZrOxevVqioqKeOSRR+jcuTPh4eEEBwdz9tlnM3PmzHLP8fe+9KXdtDZu3Oj+a3p4eDg33ngjeXl5ZR779zEApd3I5s6dy8iRI4mNjSU4OJjBgwezd+/eMo91Op2MHj2apKQkgoKCOO+881i9enWVxxUcXffo0aO5//77AUhNTXW/T0e/t5988gmdO3cmMDCQqKgorr76arZv317mnOeeey5t27ZlyZIl9OrVi6CgIP73v/8B8N133zFw4ECSkpKw2Ww0bdqUJ554AofDUebxP/74I1u3bnXX0Lhx4zKf19/HAMyYMYOzzz6b4OBgIiIiuPTSS1mzZk2ZYzz5TKZOncpZZ51FREQEISEhtGjRwv0aRKR2UwuAiPhM48aN6d69O5999hn9+/cHYMqUKWRmZnL11Vfz6quvljk+JSUFgI8++oiHHnrIfYF6LAcOHCi3zc/Pz+OuG7t376ZHjx7k5eUxYsQIoqOj+fDDD7nkkkv4+uuvGTx4MADvvvsuI0aM4PLLL+fuu++moKCAv/76iwULFrhDze23387XX3/N8OHDad26Nfv372fOnDmsWbOG008/vcLnHzhwICEhIXz55Zecc845ZfZ98cUXtGnThrZt2wLwj3/8g1WrVnHXXXfRuHFj9uzZw9SpU9m2bZv7IvHv/vWvf7Fr1y6mTp3Kxx9/XOEx48aNo6CggNtuuw2bzUZUVBRZWVm89957XHPNNdx6661kZ2fz/vvv07dvXxYuXEjHjh2P+95eeeWVpKamMmbMGJYuXcp7771HXFwc//d//3fcx951111ERkby6KOPsmXLFl5++WWGDx/OF1984T5m1KhRPPvss1x88cX07duXP//8k759+1JQUHDc8//dZZddxvr16/nss8946aWX3K1LsbGxADz11FM8/PDDXHnlldxyyy3s3buX1157jV69erFs2bIy37v9+/fTv39/rr76aq6//nri4+MBV7gJCQlh5MiRhISEMGPGDB555BGysrJ47rnnAHjwwQfJzMxkx44dvPTSSwAVdnUrNW3aNPr370+TJk0YPXo0+fn5vPbaa/Ts2ZOlS5eW+14c7zNZtWoVF110Ee3bt+fxxx/HZrOxceNG5s6d6/F7KiI+YIqIeNm4ceNMwFy0aJE5duxYMzQ01MzLyzNN0zSvuOIK87zzzjNN0zRTUlLMgQMHuh+Xl5dntmjRwgTMlJQUc+jQoeb7779v7t69u9xzPProoyZQ4a1FixbHrTElJcUcMmSI+/4999xjAubvv//u3padnW2mpqaajRs3Nh0Oh2mapnnppZeabdq0Oea5w8PDzWHDhh23hr+75pprzLi4OLOkpMS9LT093bRYLObjjz9umqZpHjx40ATM5557zuPzDxs2zKzon4W0tDQTMMPCwsw9e/aU2VdSUmIWFhaW2Xbw4EEzPj7evOmmm8psB8xHH33Ufb/0M/r7cYMHDzajo6PLbPv751H6Herdu7fpdDrd2++9917TarWahw4dMk3TNDMyMkw/Pz9z0KBBZc43evRoEyhzzsr8ve7nnnvOBMy0tLQyx23ZssW0Wq3mU089VWb7ihUrTD8/vzLbzznnHBMw33rrrXLPV/rfwtH+9a9/mUFBQWZBQYF728CBA82UlJRyx5Z+XuPGjXNv69ixoxkXF2fu37/fve3PP/80LRaLecMNN7i3VfUzeemll0zA3Lt3b7nnF5HaT12ARMSnrrzySvLz85k8eTLZ2dlMnjy5wu4/AIGBgSxYsMDdBWP8+PHcfPPNJCYmctddd1FYWFjuMRMnTmTq1KllbuPGjfO4zp9++omuXbuWGbAcEhLCbbfdxpYtW1i9ejXgGly8Y8cOFi1aVOm5IiIiWLBgQZlZi6riqquuYs+ePcyaNcu97euvv8bpdHLVVVcBrvcoICCAWbNmcfDgQY/Ofzz/+Mc/3H/pLmW1Wt3jAJxOJwcOHKCkpIQuXbocs0vT0W6//fYy988++2z2799PVlbWcR972223lWkJOvvss3E4HGzduhWA6dOnU1JSwp133lnmcXfddVeVavPEN998g9Pp5Morr2Tfvn3uW0JCAs2bNy/XLcpms3HjjTeWO09gYKD79+zsbPbt28fZZ59NXl4ea9eu9biu9PR0li9fztChQ4mKinJvb9++PX369OGnn34q95jjfSalLRnfffcdTqfT45pExLcUAETEp2JjY+nduzcTJkzgm2++weFwcPnll1d6fHh4OM8++yxbtmxhy5YtvP/++7Ro0YKxY8fyxBNPlDu+V69e9O7du8yte/fuHte5detWWrRoUW57q1at3PsB/vOf/xASEkLXrl1p3rw5w4YNK9ct4tlnn2XlypUkJyfTtWtXRo8ezebNm49bQ79+/QgPDy/TveWLL76gY8eOnHbaaYDrovL//u//mDJlCvHx8fTq1Ytnn322WqY9TU1NrXD7hx9+SPv27bHb7URHRxMbG8uPP/5IZmZmlc7bqFGjMvcjIyMBqhRgjvfY0s+lWbNmZY6LiopyH1tdNmzYgGmaNG/enNjY2DK3NWvWsGfPnjLHN2jQoMJB1KtWrWLw4MGEh4cTFhZGbGws119/PUCV39Ojlb4HlX1/9+3bR25ubpntx3tfr7rqKnr27Mktt9xCfHw8V199NV9++aXCgMgpQgFARHzu2muvZcqUKbz11lv079+/yv3zU1JSuOmmm5g7dy4RERF8+umnNVtoFbRq1Yp169bx+eefc9ZZZzFx4kTOOussHn30UfcxV155JZs3b+a1114jKSmJ5557jjZt2jBlypRjnttmszFo0CC+/fZbSkpK2LlzJ3PnznX/9b/UPffcw/r16xkzZgx2u52HH36YVq1asWzZspN6bUf/ZbrUJ598wtChQ2natCnvv/8+P//8M1OnTuX888+v8sWg1WqtcLtpmjX62OrmdDoxDMP9Hvz99vbbb5c5vqL389ChQ5xzzjn8+eefPP744/zwww9MnTrV3ffeWxfYx3tfAwMD+e2335g2bRr//Oc/+euvv7jqqqvo06dPmcHKIlI7KQCIiM8NHjwYi8XC/PnzK+3+cyyRkZE0bdqU9PT0GqjOJSUlhXXr1pXbXtolo3SAMkBwcDBXXXUV48aNY9u2bQwcOJCnnnqqzKDTxMRE7rzzTiZNmkRaWhrR0dE89dRTx63jqquuYt++fUyfPp2vvvoK0zTLBQCApk2b8u9//5tff/2VlStXUlRUxAsvvHDMc1dlUPXfff311zRp0oRvvvmGf/7zn/Tt25fevXuf0ADbmlD6uWzcuLHM9v37959wF6nK3qemTZtimiapqanlWp169+7NmWeeedxzz5o1i/379zN+/HjuvvtuLrroInr37l1ha0VVP6/S96Cy729MTAzBwcFVOtfRLBYLF1xwAS+++CKrV6/mqaeeYsaMGRXOACUitYsCgIj4XEhICG+++SajR4/m4osvrvS4P//8s9zaAODq4rB69eoKuzhUlwEDBrBw4ULmzZvn3pabm8s777xD48aNad26NeC6sDxaQEAArVu3xjRNiouLcTgc5bpxxMXFkZSUVOEYhr/r3bs3UVFRfPHFF3zxxRd07dq1TNecvLy8chffTZs2JTQ09LjnL70I9GSF29K/FB/9F/cFCxaUeZ986YILLsDPz48333yzzPaxY8ee8Dkre58uu+wyrFYrjz32WLkWCNM0y303KlLR+1lUVMQbb7xRYR1V6RKUmJhIx44d+fDDD8vUvHLlSn799VcGDBhw3HP8XUWza5XO+FSV77GI+JamARWRWmHIkCHHPWbq1Kk8+uijXHLJJZx55pmEhISwefNmPvjgAwoLCytcwfbrr7+ucHrEPn36uKddrIr//ve/7ulKR4wYQVRUFB9++CFpaWlMnDjRvUrthRdeSEJCAj179iQ+Pp41a9YwduxYBg4cSGhoKIcOHaJhw4ZcfvnldOjQgZCQEKZNm8aiRYuO+xd6AH9/fy677DI+//xzcnNzef7558vsX79+PRdccAFXXnklrVu3xs/Pj2+//Zbdu3dz9dVXH/PcnTt3BmDEiBH07dsXq9V63MdcdNFFfPPNNwwePJiBAweSlpbGW2+9RevWrcnJyTnu66lp8fHx3H333bzwwgtccskl9OvXjz///JMpU6YQExNzQq0epe/Tgw8+yNVXX42/vz8XX3wxTZs25cknn2TUqFFs2bKFQYMGERoaSlpaGt9++y233XYb99133zHP3aNHDyIjIxkyZAgjRozAMAw+/vjjCrs0de7cmS+++IKRI0dyxhlnEBISUmmAfu655+jfvz/du3fn5ptvdk8DGh4efkIrPz/++OP89ttvDBw4kJSUFPbs2cMbb7xBw4YNy63sLSK1kG8mHxKR+uzoaUCP5e/TgG7evNl85JFHzDPPPNOMi4sz/fz8zNjYWHPgwIHmjBkzyjz2WNOAAubMmTOP+9x/nyJy06ZN5uWXX25GRESYdrvd7Nq1qzl58uQyx7z99ttmr169zOjoaNNms5lNmzY177//fjMzM9M0TdMsLCw077//frNDhw5maGioGRwcbHbo0MF84403jvOuHTF16lQTMA3DMLdv315m3759+8xhw4aZLVu2NIODg83w8HCzW7du5pdffnnc85aUlJh33XWXGRsbaxqG4Z4StHRayYqmFnU6nebTTz9tpqSkmDabzezUqZM5efJkc8iQIeWmqKSSaUD/PpVk6ffj6Gk2K5sG9O/foZkzZ5b7fEtKSsyHH37YTEhIMAMDA83zzz/fXLNmjRkdHW3efvvtx31f/l63aZrmE088YTZo0MC0WCzlap04caJ51llnmcHBwWZwcLDZsmVLc9iwYea6devcx5xzzjmVThc7d+5c88wzzzQDAwPNpKQk84EHHjB/+eWXcq8rJyfHvPbaa82IiAj31LimWfE0oKZpmtOmTTN79uxpBgYGmmFhYebFF19srl69uswxVf1Mpk+fbl566aVmUlKSGRAQYCYlJZnXXHONuX79+uO+nyLie4Zp+mCklIiIiA8dOnSIyMhInnzySR588EFflyMi4lUaAyAiInVafn5+uW0vv/wyAOeee653ixERqQU0BkBEROq0L774gvHjxzNgwABCQkKYM2cOn332GRdeeCE9e/b0dXkiIl6nACAiInVa+/bt8fPz49lnnyUrK8s9MPjJJ5/0dWkiIj7h0y5Ab775Ju3btycsLIywsDC6d+9+3IVwvvrqK1q2bIndbqddu3YVLmEuIiJS6vTTT2fatGns27ePoqIitm/fzssvv1zh7FAiIvWBTwNAw4YNeeaZZ1iyZAmLFy/m/PPP59JLL2XVqlUVHv/HH39wzTXXcPPNN7Ns2TIGDRrEoEGDWLlypZcrFxERERE5NdW6WYCioqJ47rnnuPnmm8vtu+qqq8jNzWXy5MnubWeeeSYdO3bkrbfe8maZIiIiIiKnpFozBsDhcPDVV1+Rm5tL9+7dKzxm3rx5jBw5ssy2vn37MmnSpErPW1hYWGZVQqfTyYEDB4iOjj6hBWBERERERGoj0zTJzs4mKSnJvUBlRXweAFasWEH37t0pKCggJCSEb7/9ltatW1d4bEZGRrmVO+Pj48nIyKj0/GPGjOGxxx6r1ppFRERERGqr7du307Bhw0r3+zwAtGjRguXLl5OZmcnXX3/NkCFDmD17dqUhwFOjRo0q02qQmZlJo0aN2L59O2FhYdXyHCIiIiIivpaVlUVycjKhoaHHPM7nASAgIIBmzZoB0LlzZxYtWsQrr7zC22+/Xe7YhIQEdu/eXWbb7t27SUhIqPT8NpsNm81WbnvpzEMiIiIiInXJ8bq517qVgJ1OZ5k++0fr3r0706dPL7Nt6tSplY4ZEBERERGRsnzaAjBq1Cj69+9Po0aNyM7OZsKECcyaNYtffvkFgBtuuIEGDRowZswYAO6++27OOeccXnjhBQYOHMjnn3/O4sWLeeedd3z5MkREREREThk+DQB79uzhhhtuID09nfDwcNq3b88vv/xCnz59ANi2bVuZEcw9evRgwoQJPPTQQ/zvf/+jefPmTJo0ibZt2/rqJYiIiIiInFJq3ToANS0rK4vw8HAyMzM1BkBERERqLdM0KSkpweFw+LoUqSWsVit+fn6V9vGv6nWuzwcBi4iIiEhZRUVFpKenk5eX5+tSpJYJCgoiMTGRgICAEz6HAoCIiIhILeJ0OklLS8NqtZKUlERAQIAWLxVM06SoqIi9e/eSlpZG8+bNj7nY17EoAIiIiIjUIkVFRTidTpKTkwkKCvJ1OVKLBAYG4u/vz9atWykqKsJut5/QeWrdNKAiIiIiwgn/dVfqtur4XuibJSIiIiJSjygAiIiIiIjUIwoAIiIiIlIrNW7cmJdffrnKx8+aNQvDMDh06FCN1QQwfvx4IiIiavQ5apICgIiIiIicFMMwjnkbPXr0CZ130aJF3HbbbVU+vkePHu4FZqVymgVIRERERE5Kenq6+/cvvviCRx55hHXr1rm3hYSEuH83TROHw4Gf3/EvQ2NjYz2qIyAggISEBI8eUx+pBUBERESkFjNNk7yiEp/cTNOsUo0JCQnuW3h4OIZhuO+vXbuW0NBQpkyZQufOnbHZbMyZM4dNmzZx6aWXEh8fT0hICGeccQbTpk0rc96/dwEyDIP33nuPwYMHExQURPPmzfn+++/d+//eBai0q84vv/xCq1atCAkJoV+/fmUCS0lJCSNGjCAiIoLo6Gj+85//MGTIEAYNGuTR5/Tmm2/StGlTAgICaNGiBR9//HGZz3D06NE0atQIm81GUlISI0aMcO9/4403aN68OXa7nfj4eC6//HKPnttTagEQERERqcXyix20fuQXnzz36sf7EhRQPZeL//3vf3n++edp0qQJkZGRbN++nQEDBvDUU09hs9n46KOPuPjii1m3bh2NGjWq9DyPPfYYzz77LM899xyvvfYa1113HVu3biUqKqrC4/Py8nj++ef5+OOPsVgsXH/99dx33318+umnAPzf//0fn376KePGjaNVq1a88sorTJo0ifPOO6/Kr+3bb7/l7rvv5uWXX6Z3795MnjyZG2+8kYYNG3LeeecxceJEXnrpJT7//HPatGlDRkYGf/75JwCLFy9mxIgRfPzxx/To0YMDBw7w+++/e/DOek4BQERERERq3OOPP06fPn3c96OioujQoYP7/hNPPMG3337L999/z/Dhwys9z9ChQ7nmmmsAePrpp3n11VdZuHAh/fr1q/D44uJi3nrrLZo2bQrA8OHDefzxx937X3vtNUaNGsXgwYMBGDt2LD/99JNHr+35559n6NCh3HnnnQCMHDmS+fPn8/zzz3Peeeexbds2EhIS6N27N/7+/jRq1IiuXbsCsG3bNoKDg7nooosIDQ0lJSWFTp06efT8nlIA8KLCEge/rd9HYYmDi9on+bocEREROQUE+ltZ/Xhfnz13denSpUuZ+zk5OYwePZoff/yR9PR0SkpKyM/PZ9u2bcc8T/v27d2/BwcHExYWxp49eyo9PigoyH3xD5CYmOg+PjMzk927d7svxgGsViudO3fG6XRW+bWtWbOm3GDlnj178sorrwBwxRVX8PLLL9OkSRP69evHgAEDuPjii/Hz86NPnz6kpKS49/Xr18/dxammaAyAF+UXObj1o8UMn7CMEkfVv1QiIiJSfxmGQVCAn09uhmFU2+sIDg4uc/++++7j22+/5emnn+b3339n+fLltGvXjqKiomOex9/fv9z7c6yL9YqOr+rYhuqSnJzMunXreOONNwgMDOTOO++kV69eFBcXExoaytKlS/nss89ITEzkkUceoUOHDjU6lakCgBfZ/I6k6MISBQARERGpv+bOncvQoUMZPHgw7dq1IyEhgS1btni1hvDwcOLj41m0aJF7m8PhYOnSpR6dp1WrVsydO7fMtrlz59K6dWv3/cDAQC6++GJeffVVZs2axbx581ixYgUAfn5+9O7dm2effZa//vqLLVu2MGPGjJN4ZcemLkBeZPM7krcKS5wE23xYjIiIiIgPNW/enG+++YaLL74YwzB4+OGHPep2U13uuusuxowZQ7NmzWjZsiWvvfYaBw8e9Kj14/777+fKK6+kU6dO9O7dmx9++IFvvvnGPavR+PHjcTgcdOvWjaCgID755BMCAwNJSUlh8uTJbN68mV69ehEZGclPP/2E0+mkRYsWNfWSFQC8yWIxCLBaKHI4KSh2+LocEREREZ958cUXuemmm+jRowcxMTH85z//ISsry+t1/Oc//yEjI4MbbrgBq9XKbbfdRt++fbFaqz7+YdCgQbzyyis8//zz3H333aSmpjJu3DjOPfdcACIiInjmmWcYOXIkDoeDdu3a8cMPPxAdHU1ERATffPMNo0ePpqCggObNm/PZZ5/Rpk2bGnrFYJje7gTlY1lZWYSHh5OZmUlYWJjXn7/do7+QXVjCzPvOJTUm+PgPEBERkXqloKCAtLQ0UlNTsdvtvi6n3nE6nbRq1Yorr7ySJ554wtfllHOs70dVr3PVAuBlNn8r2YUlagEQERERqQW2bt3Kr7/+yjnnnENhYSFjx44lLS2Na6+91tel1RgNAvay0nEAGgQsIiIi4nsWi4Xx48dzxhln0LNnT1asWMG0adNo1aqVr0urMWoB8DKb/+EAoBYAEREREZ9LTk4uN4NPXacWAC+zH54KtEAtACIiIiLiAwoAXqYWABERERHxJQUAL9MYABERERHxJQUAL7P7H+4CpBYAEREREfEBBQAvUwuAiIiIiPiSAoCXlbYAKACIiIiIiC8oAHhZaQuAugCJiIiInJjRo0fTsWPHGn+eoUOHMmjQoBp/Hm9TAPAym59aAERERKRuMQzjmLfRo0ef1LknTZpUZtt9993H9OnTT67oekwLgXmZvXQa0BK1AIiIiEjdkJ6e7v79iy++4JFHHmHdunXubSEhIdX6fCEhIdV+zvpELQBe5m4BKFYLgIiIiFSBaUJRrm9uplmlEhMSEty38PBwDMMos+3zzz+nVatW2O12WrZsyRtvvOF+bFFREcOHDycxMRG73U5KSgpjxowBoHHjxgAMHjwYwzDc9//eBai0q87zzz9PYmIi0dHRDBs2jOLiYvcx6enpDBw4kMDAQFJTU5kwYQKNGzfm5ZdfrvJHUVhYyIgRI4iLi8Nut3PWWWexaNEi9/6DBw9y3XXXERsbS2BgIM2bN2fcuHHHfZ3ephYALzsyC5BaAERERKQKivPg6STfPPf/dkFA8Emd4tNPP+WRRx5h7NixdOrUiWXLlnHrrbcSHBzMkCFDePXVV/n+++/58ssvadSoEdu3b2f79u0ALFq0iLi4OMaNG0e/fv2wWq2VPs/MmTNJTExk5syZbNy4kauuuoqOHTty6623AnDDDTewb98+Zs2ahb+/PyNHjmTPnj0evZYHHniAiRMn8uGHH5KSksKzzz5L37592bhxI1FRUTz88MOsXr2aKVOmEBMTw8aNG8nPzwc45uv0NgUALzuyDoBaAERERKTue/TRR3nhhRe47LLLAEhNTWX16tW8/fbbDBkyhG3bttG8eXPOOussDMMgJSXF/djY2FgAIiIiSEhIOObzREZGMnbsWKxWKy1btmTgwIFMnz6dW2+9lbVr1zJt2jQWLVpEly5dAHjvvfdo3rx5lV9Hbm4ub775JuPHj6d///4AvPvuu0ydOpX333+f+++/n23bttGpUyf3c5S2WADHfJ3epgDgZTaNARARERFP+Ae5/hLvq+c+Cbm5uWzatImbb77Z/Zd4gJKSEsLDwwFX950+ffrQokUL+vXrx0UXXcSFF17o8XO1adOmTAtBYmIiK1asAGDdunX4+flx+umnu/c3a9aMyMjIKp9/06ZNFBcX07NnT/c2f39/unbtypo1awC44447+Mc//sHSpUu58MILGTRoED169KjW11kdFAC8zN0FSC0AIiIiUhWGcdLdcHwlJycHcP2lvFu3bmX2lV6sn3766aSlpTFlyhSmTZvGlVdeSe/evfn66689ei5/f/8y9w3DwOn07vVW//792bp1Kz/99BNTp07lggsuYNiwYTz//PPV9jqrgwYBe5m7C5BaAERERKSOi4+PJykpic2bN9OsWbMyt9TUVPdxYWFhXHXVVbz77rt88cUXTJw4kQMHDgCuC3uH4+Sum1q0aEFJSQnLli1zb9u4cSMHDx6s8jmaNm1KQEAAc+fOdW8rLi5m0aJFtG7d2r0tNjaWIUOG8Mknn/Dyyy/zzjvvuPcd63V6k1oAvEwtACIiIlKfPPbYY4wYMYLw8HD69etHYWEhixcv5uDBg4wcOZIXX3yRxMREOnXqhMVi4auvviIhIYGIiAjA1Y9++vTp9OzZE5vN5lG3nVItW7akd+/e3Hbbbbz55pv4+/vz73//m8DAQAzDqNI5goODueOOO7j//vuJioqiUaNGPPvss+Tl5XHzzTcD8Mgjj9C5c2fatGlDYWEhkydPplWrVgDHfZ3epADgZVoITEREROqTW265haCgIJ577jnuv/9+goODadeuHffccw8AoaGhPPvss2zYsAGr1coZZ5zBTz/9hMXi+qPpCy+8wMiRI3n33Xdp0KABW7ZsOaE6PvroI26++WZ69epFQkICY8aMYdWqVdjt9iqf45lnnsHpdPLPf/6T7OxsunTpwi+//OIOJQEBAYwaNYotW7YQGBjI2Wefzeeff16l1+lNhmlWcYLXOiIrK4vw8HAyMzMJCwvz+vP/sWkf1767gOZxIUwdeY7Xn19ERERqt4KCAtLS0khNTfXo4lQ8s2PHDpKTk5k2bRoXXHCBr8upsmN9P6p6nasWAC9TC4CIiIiI982YMYOcnBzatWtHeno6DzzwAI0bN6ZXr16+Ls3rFAC8zK5pQEVERES8rri4mP/9739s3ryZ0NBQevTowaefflpu9qD6QAHAy0pbALQQmIiIiIj39O3bl759+/q6jFpB04B6mXsWILUAiIiIiIgPKAB4Wek6AIUlTurZ+GsRERHxgK4TpCLV8b1QAPAy2+ExAKYJRQ51AxIREZGySvuk5+Xl+bgSqY1KvxcnM3ZBYwC8rLQLELhaAUrHBIiIiIgAWK1WIiIi2LNnDwBBQUFVXqxK6i7TNMnLy2PPnj1ERERgtZ74NaQCgJcFWC0YhqsFoLDYCZreV0RERP4mISEBwB0CREpFRES4vx8nSgHAywzDwOZnoaDYSUGxBgKLiIhIeYZhkJiYSFxcHMXFxb4uR2oJf3//k/rLfykFAB+w+VkpKHZqMTARERE5JqvVWi0XfCJH0yBgHyhdDEwtACIiIiLibQoAPlA68FctACIiIiLibQoAPqDFwERERETEVxQAfMC9GFixWgBERERExLsUAHxALQAiIiIi4isKAD5QuhqwxgCIiIiIiLcpAPiA/fAgYM0CJCIiIiLepgDgA2oBEBERERFfUQDwgdIWAA0CFhERERFvUwDwAZsWAhMRERERH1EA8AEtBCYiIiIivqIA4ANHxgCoBUBEREREvEsBwAds7lmA1AIgIiIiIt6lAOADWghMRERERHxFAcAH7P4aAyAiIiIivqEA4AOlLQCaBUhEREREvE0BwAeOdAFSC4CIiIiIeJcCgA+UdgFSC4CIiIiIeJtPA8CYMWM444wzCA0NJS4ujkGDBrFu3bpjPmb8+PEYhlHmZrfbvVRx9VALgIiIiIj4ik8DwOzZsxk2bBjz589n6tSpFBcXc+GFF5Kbm3vMx4WFhZGenu6+bd261UsVVw9b6SBgTQMqIiIiIl7m58sn//nnn8vcHz9+PHFxcSxZsoRevXpV+jjDMEhISKjScxQWFlJYWOi+n5WVdWLFViN76SBgTQMqIiIiIl5Wq8YAZGZmAhAVFXXM43JyckhJSSE5OZlLL72UVatWVXrsmDFjCA8Pd9+Sk5OrteYToRYAEREREfGVWhMAnE4n99xzDz179qRt27aVHteiRQs++OADvvvuOz755BOcTic9evRgx44dFR4/atQoMjMz3bft27fX1EuoMru/xgCIiIiIiG/4tAvQ0YYNG8bKlSuZM2fOMY/r3r073bt3d9/v0aMHrVq14u233+aJJ54od7zNZsNms1V7vSfD5lfaAqAuQCIiIiLiXbUiAAwfPpzJkyfz22+/0bBhQ48e6+/vT6dOndi4cWMNVVf9NAuQiIiIiPiKT7sAmabJ8OHD+fbbb5kxYwapqaken8PhcLBixQoSExNroMKaUboOQJHDidNp+rgaEREREalPfNoCMGzYMCZMmMB3331HaGgoGRkZAISHhxMYGAjADTfcQIMGDRgzZgwAjz/+OGeeeSbNmjXj0KFDPPfcc2zdupVbbrnFZ6/DU6UtAOBqBQgMsPqwGhERERGpT3waAN58800Azj333DLbx40bx9ChQwHYtm0bFsuRC+aDBw9y6623kpGRQWRkJJ07d+aPP/6gdevW3ir7pJUNAA4FABERERHxGsM0zXrVByUrK4vw8HAyMzMJCwvzWR3N/vcTJU6TBf+7gPiwU2slYxERERGpfap6nVtrpgGtb0pbAQo0E5CIiIiIeJECgI+4FwPTTEAiIiIi4kUKAD5iVwuAiIiIiPiAAoCPqAVARERERHxBAcBH3IuBFSsAiIiIiIj3KAD4SGkLgLoAiYiIiIg3KQD4iLsFQF2ARERERMSLFAB8xO4eA6AWABERERHxHgUAHzmyDoBaAERERETEexQAfORIFyC1AIiIiIiI9ygA+Ihd04CKiIiIiA8oAPiITQuBiYiIiIgPKAD4iM1PLQAiIiIi4n0KAD5i99dCYCIiIiLifQoAPlLaAlCgQcAiIiIi4kUKAD5iUwuAiIiIiPiAAoCP2DUNqIiIiIj4gAKAj9gOTwOqhcBERERExJsUAHxEC4GJiIiIiC8oAPiIeyEwtQCIiIiIiBcpAPiIWgBERERExBcUAHzE3QKghcBERERExIsUAHyktAWgoFgtACIiIiLiPQoAPlK6EJhaAERERETEmxQAfMReuhCYAoCIiIiIeJECgI+UtgCoC5CIiIiIeJMCgI/Y1AIgIiIiIj6gAOAj9sMtAA6nSYlDIUBEREREvEMBwEdKWwAACtQKICIiIiJeogDgIwHWI299ocYBiIiIiIiXKAD4iMViEOCncQAiIiIi4l0KAD6kxcBERERExNsUAHxIi4GJiIiIiLcpAPiQFgMTEREREW9TAPAhdQESEREREW9TAPAhu7+6AImIiIiIdykA+JBaAERERETE2xQAfEiDgEVERETE2xQAfMg9CFgtACIiIiLiJQoAPlTaAlCgFgARERER8RIFAB+yqQVARERERLxMAcCH7BoDICIiIiJepgDgQ2oBEBERERFvUwDwodJpQNUCICIiIiLeogDgQ1oITERERES8TQHAh7QQmIiIiIh4mwKAD2khMBERERHxNgUAH3IvBFaiFgARERER8Q4FAB9yLwRWrBYAEREREfEOBQAfsqkFQERERES8TAHAh9xjANQCICIiIiJeogDgQ6UtAAVqARARERERL1EA8CG7WgBERERExMsUAHxILQAiIiIi4m0KAD5UuhCYWgBERERExFsUAHzI7q+FwERERETEuxQAfKi0BaCgWF2ARERERMQ7FAB8yD0NaIkT0zR9XI2IiIiI1AcKAD5k9z/y9hc51A1IRERERGqeAoAPlbYAABRoILCIiIiIeIECgA/5Ww0Mw/V7oaYCFREREREvUADwIcMwtBiYiIiIiHiVAoCPlS4GphYAEREREfEGBQAfK20B0BgAEREREfEGjwPAhx9+yI8//ui+/8ADDxAREUGPHj3YunVrtRZXHxxpAVAAEBEREZGa53EAePrppwkMDARg3rx5vP766zz77LPExMRw7733VnuBdV3pYmCFWgxMRERERLzA4wCwfft2mjVrBsCkSZP4xz/+wW233caYMWP4/fffPTrXmDFjOOOMMwgNDSUuLo5Bgwaxbt264z7uq6++omXLltjtdtq1a8dPP/3k6cuoNez+RxYDExERERGpaR4HgJCQEPbv3w/Ar7/+Sp8+fQCw2+3k5+d7dK7Zs2czbNgw5s+fz9SpUykuLubCCy8kNze30sf88ccfXHPNNdx8880sW7aMQYMGMWjQIFauXOnpS6kVSlsACtQCICIiIiJe4OfpA/r06cMtt9xCp06dWL9+PQMGDABg1apVNG7c2KNz/fzzz2Xujx8/nri4OJYsWUKvXr0qfMwrr7xCv379uP/++wF44oknmDp1KmPHjuWtt97y9OX4XOliYGoBEBERERFv8LgF4PXXX6d79+7s3buXiRMnEh0dDcCSJUu45pprTqqYzMxMAKKioio9Zt68efTu3bvMtr59+zJv3rwKjy8sLCQrK6vMrTaxaxpQEREREfEij1sAIiIiGDt2bLntjz322EkV4nQ6ueeee+jZsydt27at9LiMjAzi4+PLbIuPjycjI6PC48eMGXPStdUkm6YBFREREREv8rgF4Oeff2bOnDnu+6+//jodO3bk2muv5eDBgydcyLBhw1i5ciWff/75CZ+jIqNGjSIzM9N92759e7We/2S5ZwFSC4CIiIiIeIHHAeD+++93d6NZsWIF//73vxkwYABpaWmMHDnyhIoYPnw4kydPZubMmTRs2PCYxyYkJLB79+4y23bv3k1CQkKFx9tsNsLCwsrcahNb6SxAagEQERERES/wOACkpaXRunVrACZOnMhFF13E008/zeuvv86UKVM8OpdpmgwfPpxvv/2WGTNmkJqaetzHdO/enenTp5fZNnXqVLp37+7Rc9cW7lmA1AIgIiIiIl7gcQAICAggLy8PgGnTpnHhhRcCroG7ng6wHTZsGJ988gkTJkwgNDSUjIwMMjIyykwnesMNNzBq1Cj3/bvvvpuff/6ZF154gbVr1zJ69GgWL17M8OHDPX0ptYJ7JWC1AIiIiIiIF3g8CPiss85i5MiR9OzZk4ULF/LFF18AsH79+uN23/m7N998E4Bzzz23zPZx48YxdOhQALZt24bFciSn9OjRgwkTJvDQQw/xv//9j+bNmzNp0qRjDhyuzeyaBlREREREvMjjADB27FjuvPNOvv76a958800aNGgAwJQpU+jXr59H5zJN87jHzJo1q9y2K664giuuuMKj56qtSlsAtBCYiIiIiHiDxwGgUaNGTJ48udz2l156qVoKqm/UAiAiIiIi3uRxAABwOBxMmjSJNWvWANCmTRsuueQSrFZrtRZXH9i0EJiIiIiIeJHHAWDjxo0MGDCAnTt30qJFC8C12FZycjI//vgjTZs2rfYi6zItBCYiIiIi3uTxLEAjRoygadOmbN++naVLl7J06VK2bdtGamoqI0aMqIka6zS7WgBERERExIs8bgGYPXs28+fPJyoqyr0tOjqaZ555hp49e1ZrcfWBTWMARERERMSLPG4BsNlsZGdnl9uek5NDQEBAtRRVn7gXAlMXIBERERHxAo8DwEUXXcRtt93GggULME0T0zSZP38+t99+O5dccklN1Fin2f1LWwDUBUhEREREap7HAeDVV1+ladOmdO/eHbvdjt1up2fPnjRr1oxXXnmlJmqs00pbALQSsIiIiIh4g8djACIiIvjuu+/YsGEDa9euBaBVq1Y0a9as2ourDzQNqIiIiIh40wmtAwDQvHlzmjdvXp211EvuhcDUAiAiIiIiXlClADBy5Mgqn/DFF1884WLqo9IWgAK1AIiIiIiIF1QpACxbtqxKJzMM46SKqY9KpwEtdpg4nCZWi95DEREREak5VQoAM2fOrOk66q3ShcAAikqcBAZYfViNiIiIiNR1Hs8CJNUrwHrkIygoVjcgEREREalZCgA+5me14He4249WAxYRERGRmqYAUAtoMTARERER8RYFgFqgdDGwAk0FKiIiIiI1zKMAUFxczE033URaWlpN1VMvqQVARERERLzFowDg7+/PxIkTa6qWequ0BUBjAERERESkpnncBWjQoEFMmjSpBkqpvwLcXYDUAiAiIiIiNatK6wAcrXnz5jz++OPMnTuXzp07ExwcXGb/iBEjqq24+sLdBUhjAERERESkhnkcAN5//30iIiJYsmQJS5YsKbPPMAwFgBOgLkAiIiIi4i0eBwANAK5+tsMtAOoCJCIiIiI17aSmATVNE9M0q6uWesuuFgARERER8ZITCgAfffQR7dq1IzAwkMDAQNq3b8/HH39c3bXVG2oBEBERERFv8bgL0IsvvsjDDz/M8OHD6dmzJwBz5szh9ttvZ9++fdx7773VXmRdpzEAIiIiIuItHgeA1157jTfffJMbbrjBve2SSy6hTZs2jB49WgHgBNj9SwOAWgBEREREpGZ53AUoPT2dHj16lNveo0cP0tPTq6Wo+sbmV9oFSC0AIiIiIlKzPA4AzZo148svvyy3/YsvvqB58+bVUlR9c6QLkFoARERERKRmedwF6LHHHuOqq67it99+c48BmDt3LtOnT68wGMjxuRcC0xgAEREREalhHrcA/OMf/2DhwoXExMQwadIkJk2aRExMDAsXLmTw4ME1UWOdV9oCoFmARERERKSmedQCUFxczL/+9S8efvhhPvnkk5qqqd5RC4CIiIiIeItHLQD+/v5MnDixpmqpt9xjADQIWERERERqmMddgAYNGsSkSZNqoJT6y6ZpQEVERETESzweBNy8eXMef/xx5s6dS+fOnQkODi6zf8SIEdVWXH1hPzwNqFoARERERKSmeRwA3n//fSIiIliyZAlLliwps88wDAWAE6AWABERERHxFo8CgGmazJo1i7i4OAIDA2uqpnpHC4GJiIiIiLd4NAbANE2aN2/Ojh07aqqeesmuFgARERER8RKPAoDFYqF58+bs37+/puqpl0pbADQNqIiIiIjUNI9nAXrmmWe4//77WblyZU3UUy9pITARERER8RaPBwHfcMMN5OXl0aFDBwICAsqNBThw4EC1FVdfaCEwEREREfEWjwPAyy+/XANl1G9HtwCYpolhGD6uSERERETqKo8DwJAhQ2qijnqtdAyA04QSp4m/VQFARERERGqGx2MAADZt2sRDDz3ENddcw549ewCYMmUKq1atqtbi6ovSdQBA3YBEREREpGZ5HABmz55Nu3btWLBgAd988w05OTkA/Pnnnzz66KPVXmB9UNoFCDQQWERERERqlscB4L///S9PPvkkU6dOJSAgwL39/PPPZ/78+dVaXH1hGIY7BKgFQERERERqkscBYMWKFQwePLjc9ri4OPbt21ctRdVH7gCgFgARERERqUEeB4CIiAjS09PLbV+2bBkNGjSolqLqI9vhqUALitUCICIiIiI1x+MAcPXVV/Of//yHjIwMDMPA6XQyd+5c7rvvPm644YaaqLFesPuXdgFSC4CIiIiI1ByPA8DTTz9Ny5YtSU5OJicnh9atW9OrVy969OjBQw89VBM11gulU4FqDICIiIiI1CSP1wEICAjg3Xff5ZFHHmHFihXk5OTQqVMnmjdvXhP11RtHLwYmIiIiIlJTPA4ApZKTk0lOTq7OWuo1u79aAERERESk5p3QQmBS/TQNqIiIiIh4gwJALaEuQCIiIiLiDQoAtYS7C5ACgIiIiIjUIAUAXzDNcptCbK7hGFkFJd6uRkRERETqkSoNAv7rr7+qfML27dufcDF1Xs4eeL0bFOXCQ7vBMNy7YkJtAOzLKfRVdSIiIiJSD1QpAHTs2BHDMDBNE+Ooi9aKOBzqwlIpWyjkH3D9XpgF9nD3rpiQ0gBQ5IvKRERERKSeqFIXoLS0NDZv3kxaWhoTJ04kNTWVN954g2XLlrFs2TLeeOMNmjZtysSJE2u63lObfyD4Bbp+zz9YZldMSAAA+7LVAiAiIiIiNadKLQApKSnu36+44gpeffVVBgwY4N7Wvn17kpOTefjhhxk0aFC1F1mnBEVB1k7IOwCRjd2bS1sA9ucqAIiIiIhIzfF4EPCKFStITU0ttz01NZXVq1dXS1F1WmCk62dpV6DDoktbANQFSERERERqkMcBoFWrVowZM4aioiMXqkVFRYwZM4ZWrVpVa3F1UmkAyPt7FyBXC8DBvCJKHFoMTERERERqRpW6AB3trbfe4uKLL6Zhw4buGX/++usvDMPghx9+qPYC65ygKNfPv7UARAYFYDHAacKBvCLiQu0+KE5ERERE6jqPA0DXrl3ZvHkzn376KWvXrgXgqquu4tprryU4OLjaC6xzAg8HgLyyAcBqMYgKDmBfThH7shUARERERKRmeBwAAIKDg7ntttuqu5b6wd0CcLDcrpgQG/tyijQQWERERERqzAmtBPzxxx9z1llnkZSUxNatWwF46aWX+O6776q1uDopsOIuQHD0QGAFABERERGpGR4HgDfffJORI0fSv39/Dh486F74KzIykpdffrm666t73IOAywcA92Jg2ZoJSERERERqhscB4LXXXuPdd9/lwQcfxM/vSA+iLl26sGLFCo/O9dtvv3HxxReTlJSEYRhMmjTpmMfPmjULwzDK3TIyMjx9Gb5TySBgOCoAqAuQiIiIiNQQjwNAWloanTp1KrfdZrORm5vr0blyc3Pp0KEDr7/+ukePW7duHenp6e5bXFycR4/3qUoGAcNRXYDUAiAiIiIiNcTjQcCpqaksX768zOrAAD///LPH6wD079+f/v37e1oCcXFxREREePy4WuE4g4BBqwGLiIiISM3xOACMHDmSYcOGUVBQgGmaLFy4kM8++4wxY8bw3nvv1USN5XTs2JHCwkLatm3L6NGj6dmzZ6XHFhYWUlh45II6KyvLGyVWrrQFoDALHMVg9XfvitEgYBERERGpYR4HgFtuuYXAwEAeeugh8vLyuPbaa0lKSuKVV17h6quvroka3RITE3nrrbfo0qULhYWFvPfee5x77rksWLCA008/vcLHjBkzhscee6xG6/KIPfzI7/mHICTWfVeDgEVERESkpnkUAEpKSpgwYQJ9+/bluuuuIy8vj5ycHK/1wW/RogUtWrRw3+/RowebNm3ipZde4uOPP67wMaNGjWLkyJHu+1lZWSQnJ9d4rZWy+rlCQEGmayBwBQFgf24hpmliGIavqhQRERGROsqjQcB+fn7cfvvtFBQUABAUFOTzAbhdu3Zl48aNle632WyEhYWVuflcJQOBo4JdXYCKHSZZ+SXerkpERERE6gGPZwHq2rUry5Ytq4laTsjy5ctJTEz0dRmeqWQqULu/lVC7q1FGU4GKiIiISE3weAzAnXfeyb///W927NhB586dCQ4OLrO/ffv2VT5XTk5Omb/ep6WlsXz5cqKiomjUqBGjRo1i586dfPTRRwC8/PLLpKam0qZNGwoKCnjvvfeYMWMGv/76q6cvw7cCjz0TUHZBCfuyC2kaG+LlwkRERESkrvM4AJQO9B0xYoR7m2EY7j7rpSsDV8XixYs577zz3PdL++oPGTKE8ePHk56ezrZt29z7i4qK+Pe//83OnTsJCgqiffv2TJs2rcw5TglBla8FEBMSQNq+XPblaCCwiIiIiFQ/jwNAWlpatT35ueeei2male4fP358mfsPPPAADzzwQLU9v88ERrp+VrAacHSw1gIQERERkZrjcQD4+wJgcgKOsRpwTGjpasAKACIiIiJS/TwOAKVWr17Ntm3bKCoq21XlkksuOemi6rxKBgHDUWsB5KoLkIiIiIhUP48DwObNmxk8eDArVqxw9/0H3HPWezIGoN4q7QKUV34QcLR7MTC1AIiIiIhI9fN4GtC7776b1NRU9uzZQ1BQEKtWreK3336jS5cuzJo1qwZKrIOCKp8FKDbkcBegHAUAEREREal+HrcAzJs3jxkzZhATE4PFYsFisXDWWWcxZswYRowYUavWCKi1AivvAhTtXg1YXYBEREREpPp53ALgcDgIDQ0FICYmhl27dgGuwcHr1q2r3urqKncXoAPwt1mQYtQFSERERERqkMctAG3btuXPP/8kNTWVbt268eyzzxIQEMA777xDkyZNaqLGuqe0C5CjEIrzIODIYmoxh7sA5RY5yC9yEBhg9UWFIiIiIlJHedwC8NBDD+F0OgF4/PHHSUtL4+yzz+ann37i1VdfrfYC66SAELD4u37/21SgITY/AvxcH4vGAYiIiIhIdfO4BaBv377u35s1a8batWs5cOAAkZGR7pmA5DgMw9UKkLPbNQ4gIvmoXQaxITZ2HspnX04hyVFBPixUREREROoaj1sAKhIVFaWLf08FVj4TUPThbkD7czQQWERERESql8ctAOedd94xL/ZnzJhxUgXVG0cPBP4b90BgdQESERERkWrmcQDo2LFjmfvFxcUsX76clStXMmTIkOqqq+475mrAh1sANBWoiIiIiFQzjwPASy+9VOH20aNHk5OTc9IF1RtVWA14r6YCFREREZFqVi1jAACuv/56Pvjgg+o6Xd13zBYAdQESERERkZpRbQFg3rx52O326jpd3Vc6CLjCMQAaBCwiIiIiNcPjLkCXXXZZmfumaZKens7ixYt5+OGHq62wOi+o8lmA1AIgIiIiIjXF4wAQHh5e5r7FYqFFixY8/vjjXHjhhdVWWJ1XOgbgGF2ANAhYRERERKqbxwFg3LhxNVFH/XOMLkCl6wAczCuixOHEz1ptPbVEREREpJ7TlaWvHGMQcGRQABYDTBMOqBVARERERKqRxy0AkZGRVV7198CB8he3cph7JeBD4HSAxereZbUYRAUHsC+niH05RcSFaXC1iIiIiFQPjwPAww8/zJNPPknfvn3p3r074JoB6JdffuHhhx8mKiqq2ousk0rHAGBCQeaRFoHDYkJshwOABgKLiIiISPXxOADMnTuXxx9/nOHDh7u3jRgxgrFjxzJt2jQmTZpUnfXVXX4BEBAKRdmumYD+FgCi3asBKwCIiIiISPXxeAzAL7/8Qr9+/cpt79evH9OmTauWouoN92rAx1gMLFtjAERERESk+ngcAKKjo/nuu+/Kbf/uu++Ijo6ulqLqjaDjTwWqLkAiIiIiUp087gL02GOPccsttzBr1iy6desGwIIFC/j555959913q73AOq0KU4Hu02rAIiIiIlKNPA4AQ4cOpVWrVrz66qt88803ALRq1Yo5c+a4A4FU0TGmAlULgIiIiIjUBI8DAEC3bt349NNPq7uW+ucYLQAxGgQsIiIiIjXA4zEAS5cuZcWKFe773333HYMGDeJ///sfRUXqruIRdwvAwXK7NAhYRERERGqCxwHgX//6F+vXrwdg8+bNXHXVVQQFBfHVV1/xwAMPVHuBdVrg8QcB788txDRNb1YlIiIiInWYxwFg/fr1dOzYEYCvvvqKc845hwkTJjB+/HgmTpxY3fXVbcfoAhQV7OoCVOwwycov8WZVIiIiIlKHeRwATNPE6XQCMG3aNAYMGABAcnIy+/btq97q6rpjDAK2+1sJtbuGaOzVQGARERERqSYeB4AuXbrw5JNP8vHHHzN79mwGDhwIQFpaGvHx8dVeYJ3mbgEoPwYAjuoGpAAgIiIiItXE4wDw8ssvs3TpUoYPH86DDz5Is2bNAPj666/p0aNHtRdYp7kXAqssAGgtABERERGpXh5PA9q+ffsyswCVeu6557BardVSVL1ROgi4OBdKCsHPVma31gIQERERkermcQtAZex2O/7+/tV1uvrBFg7G4Y/gGKsBqwuQiIiIiFSXagsAcgIslipNBbpXXYBEREREpJooAPjaMaYCjdYgYBERERGpZgoAvnaMqUBj3YOAFQBEREREpHooAPhaaQtABTMBHRkErC5AIiIiIlI9PJ4FyOFwMH78eKZPn86ePXvci4KVmjFjRrUVVy+UjgFQFyARERER8QKPA8Ddd9/N+PHjGThwIG3btsUwjJqoq/44Rheg0nUAcosc5Bc5CAzQNKsiIiIicnI8DgCff/45X375JQMGDKiJeuofdwtA+S5AITY/AvwsFJU42ZdTSHJUkJeLExEREZG6xuMxAAEBAe7Vf6UaHKMFwDAMYrUYmIiIiIhUI48DwL///W9eeeUVTNOsiXrqn2NMAwpHugFpILCIiIiIVAePuwDNmTOHmTNnMmXKFNq0aVNu9d9vvvmm2oqrF4IqnwUINBBYRERERKqXxwEgIiKCwYMH10Qt9dMxVgKGo1sAFABERERE5OR5HADGjRtXE3XUX0evA2Ca8LdZlaK1FoCIiIiIVCMtBOZrpV2AnCVQmFVud4wGAYuIiIhINfK4BQDg66+/5ssvv2Tbtm0UFZX9y/TSpUurpbB6wz8Q/AKhJN81ENgeXma3ugCJiIiISHXyuAXg1Vdf5cYbbyQ+Pp5ly5bRtWtXoqOj2bx5M/3796+JGuu+Y0wFWjoN6J4sBQAREREROXkeB4A33niDd955h9dee42AgAAeeOABpk6dyogRI8jMzKyJGuu+wMpnAkqJCQZg+8E8ih1Ob1YlIiIiInWQxwFg27Zt9OjRA4DAwECys7MB+Oc//8lnn31WvdXVF4ERrp8VrAacGGYn0N9KscNk+4E879YlIiIiInWOxwEgISGBAwdcXVUaNWrE/PnzAUhLS9PiYCfqGF2ALBaD1MOtAJv25nqzKhERERGpgzwOAOeffz7ff/89ADfeeCP33nsvffr04aqrrtL6ACfqOKsBN40LAWDz3hxvVSQiIiIidZTHswC98847OJ2uvujDhg0jOjqaP/74g0suuYR//etf1V5gvXCMFgCAprGlLQAKACIiIiJycjwOABaLBYvlSMPB1VdfzdVXX12tRdU7x2sBiHW1AKgLkIiIiIicrBNaCOz333/n+uuvp3v37uzcuROAjz/+mDlz5lRrcfVGYKTrZwWzAAE0OdwCoC5AIiIiInKyPA4AEydOpG/fvgQGBrJs2TIKC13z02dmZvL0009Xe4H1wnG6ADWJcbUAHMwr5kBuUYXHiIiIiIhUhccB4Mknn+Stt97i3Xffxd/f3729Z8+eWgX4RB2nC1BggJUGEYGAxgGIiIiIyMnxOACsW7eOXr16ldseHh7OoUOHqqOm+ieo8oXASpXOBLRpjwKAiIiIiJy4E1oHYOPGjeW2z5kzhyZNmlRLUfVOaQtAYRY4iis8pHQmoM37NBBYRERERE6cxwHg1ltv5e6772bBggUYhsGuXbv49NNPue+++7jjjjtqosa6r3QlYID8QxUe0iRWLQAiIiIicvI8ngb0v//9L06nkwsuuIC8vDx69eqFzWbjvvvu46677qqJGus+ixXs4VCQ6RoIHBJb7hCtBSAiIiIi1cHjAGAYBg8++CD3338/GzduJCcnh9atWxMSElIT9dUfgVGuAFDJQOBmh1sAth3Io7DEgc3P6s3qRERERKSO8DgAlAoICKB169bVWUv9FhQFB9MqnQo0NtRGqM2P7MIStu3Po3l8qJcLFBEREZG6oMoB4KabbqrScR988MEJF1OvHWcqUMMwaBIbzJ87Mtm0N0cBQEREREROSJUHAY8fP56ZM2dy6NAhDh48WOnNE7/99hsXX3wxSUlJGIbBpEmTjvuYWbNmcfrpp2Oz2WjWrBnjx4/36DlrrYhk18/0Pys9pGnpQOC9mglIRERERE5MlVsA7rjjDj777DPS0tK48cYbuf7664mKijqpJ8/NzaVDhw7cdNNNXHbZZcc9Pi0tjYEDB3L77bfz6aefMn36dG655RYSExPp27fvSdXic837wuIPYN0UGPAcGEa5Q7QWgIiIiIicrCq3ALz++uukp6fzwAMP8MMPP5CcnMyVV17JL7/8gmmaJ/Tk/fv358knn2Tw4MFVOv6tt94iNTWVF154gVatWjF8+HAuv/xyXnrppRN6/lqlyTngHwRZOyptBXDPBKS1AERERETkBHm0DoDNZuOaa65h6tSprF69mjZt2nDnnXfSuHFjcnJq/q/S8+bNo3fv3mW29e3bl3nz5lX6mMLCQrKyssrcaiX/QGh6vuv3dT9VeEjpWgCb9+SccOgSERERkfrN44XA3A+0WDAMA9M0cTgc1VlTpTIyMoiPjy+zLT4+nqysLPLz8yt8zJgxYwgPD3ffkpOTvVHqiWk50PVz7Y8V7k6JDsJiQHZhCXuzC71YmIiIiIjUFR4FgMLCQj777DP69OnDaaedxooVKxg7dizbtm2rtesAjBo1iszMTPdt+/btvi6pcs37gmGB3Svh4JZyu21+VhpFBQGwUQuCiYiIiMgJqHIAuPPOO0lMTOSZZ57hoosuYvv27Xz11VcMGDAAi+WEGxI8kpCQwO7du8ts2717N2FhYQQGBlb4GJvNRlhYWJlbrRUcDY16uH5fN6XCQ0pnAtqsmYBERERE5ARUeRagt956i0aNGtGkSRNmz57N7NmzKzzum2++qbbi/q579+789FPZ/vFTp06le/fuNfacXtdyAGyd4+oGdOYd5XY3iQ1m+lrYpBYAERERETkBVQ4AN9xwA0YFU1OejJycHDZu3Oi+n5aWxvLly4mKiqJRo0aMGjWKnTt38tFHHwFw++23M3bsWB544AFuuukmZsyYwZdffsmPP1bcZ/6U1GIA/PI/2PqHa1GwoLJTrWotABERERE5GVUOADWx4NbixYs577zz3PdHjhwJwJAhQxg/fjzp6els27bNvT81NZUff/yRe++9l1deeYWGDRvy3nvvnfprABwtKhXiWsOe1bDhV+hwdZndWgtARERERE5GlQNATTj33HOPOZ1lRaHj3HPPZdmyZTVYVS3QcqArAKz9sXwAONwCsCszn/wiB4EBVl9UKCIiIiKnKO+M3hXPtBjg+rlxOhQXlNkVFRxAZJA/pglpWhBMRERERDykAFAbJXWC0CQozoW038rtbuIeB6BuQCIiIiLiGQWA2sgwoEV/1+/ryg9wbhobDCgAiIiIiIjnFABqq5aHuwGtmwJOZ5ldWgtARERERE6UAkBt1bgX2MIgZzfsXFJmV1N1ARIRERGRE6QAUFv5BUCz3q7f/9YNqMnhLkCb9+bidFY+i5KIiIiIyN8pANRmLQe6fq4tu/pxclQQ/laD/GIH6VkFFTxQRERERKRiCgC1WbPeYPGDfetg35EVk/2tFlKiS1sB1A1IRERERKpOAaA2C4yAxme7fv9bNyD3TEBaEVhEREREPKAAUNuVdgNaN6XM5iNrAWgmIBERERGpOgWA2q7p+a6fO5dASeGRzZoJSEREREROgAJAbRfVBIKiwVEE6X+5Nzc9aiYgEREREZGqUgCo7QwDGp7h+n3HIvfm0i5AGVkF5BSW+KIyERERETkFKQCcCtwBYKF7U3igP7GhNgD+2nHIB0WJiIiIyKlIAeBUkNzV9XP7ojKbL2gZB8Bbszd7uyIREREROUUpAJwKkk4HwwJZOyBrl3vzsPOa4Wcx+G39XhZvOeDDAkVERETkVKEAcCqwhUBcG9fvR40DSI4K4oouDQF4adp6X1QmIiIiIqcYBYBTRfLhcQDbF5bZPOy8ZvhbDeZu3M+Czft9UJiIiIiInEoUAE4V7oHAi8tujgziyi7JgFoBREREROT4FABOFQ0PDwTetQxKisrsGnZeMwKsFuZvPsAfm/b5oDgREREROVUoAJwqoptCYCQ4CiFjRZldSRGBXN3V1Qrw8tQNmKbpiwpFRERE5BSgAHCqqGRBsFJ3ntuMAD8LC7ccYO5GjQUQERERkYopAJxKSrsB7VhYbldCuJ1ruzYC4MWp69QKICIiIiIVUgA4lTTs4vq5vXwLAMCd5zbF5mdh6bZD/LZBYwFEREREpDwFgFNJg86AAZnbIDuj3O64MDv/PDMFgBenrlcrgIiIiIiUowBwKrGHQVxr1+8VjAMA+Nc5TbH7W/hz+yFmrN3jxeJERERE5FSgAHCqKe0GVEkAiA21MaR7YwDemLXJS0WJiIiIyKlCAeBUk3x4IHAl4wAAbj4rlQCrhSVbD7J020EvFSYiIiIipwIFgFPN0QuCOYorPCQuzM4lHZMAeO/3zd6qTEREREROAQoAp5roZmAPh5J82L2y0sNuPisVgJ9XZrD9QJ63qhMRERGRWk4B4FRjsRxZEOwY3YBaJYZxdvMYnCZ8MDfNS8WJiIiISG2nAHAqOsaCYEe75ewmAHy5aDuZ+RV3FxIRERGR+kUB4FR0nJmASvVqHsNp8SHkFjn4fOE2LxQmIiIiIrWdAsCpqGEXwICDWyBnb6WHGYbBLWe5WgHG/7GFYofTO/WJiIiISK2lAHAqsodDbEvX78fpBnRppyRiQmykZxbw41/pXihORERERGozBYBTVRW7Adn8rAzpngLAe3M2Y5pmTVcmIiIiIrWYAsCpqgoLgpW67swU7P4WVu7MYv7mAzVcmIiIiIjUZgoApyr3gmBLwVFyzEOjggP4x+kNAS0MJiIiIlLfKQCcqmJOA1s4FOfB1jnHPfzms1IxDJi+dg8b9+R4oUARERERqY0UAE5VFgu0Hez6/edR4Dj2PP9NYkO4oGU8oIXBREREROozBYBT2QWPQlA07FkN814/7uG3nJ0KwMQlO9iTVVDT1YmIiIhILaQAcCoLioILn3T9PusZOLj1mId3S43i9EYRFJY4eWPWJi8UKCIiIiK1jQLAqa7DNZByFpTkw0/3wzGm+TQMg/subAHAhAXb2Hko31tVioiIiEgtoQBwqjMMuOhFsPjDhl9gzQ/HPLxHsxjObBJFkcPJ2BkbvFSkiIiIiNQWCgB1QWwL6Hm36/cp/4HC7GMe/u/DrQBfLt7Bln25NV2diIiIiNQiCgB1Ra/7ILIxZO+CmU8f89AzGkdxzmmxOJwmr05XK4CIiIhIfaIAUFf4B8KAF1y/L3gLdi0/5uH/vvA0AL5dvpONe47dYiAiIiIidYcCQF3SvDe0GQymEybfC05HpYe2bxjBha3jMU14aZpaAURERETqCwWAuqbvGLCFwa6l8MX1sPIbKMiq8NCRF56GYcCPf6WzelfFx4iIiIhI3aIAUNeEJcKFT7h+X/cTfH0jPNcUPrkcFo+D7N3uQ1smhHFR+yQAXpy63hfVioiIiIiXKQDURZ2Hwi0zXDMDRTUFRxFsnAqT74EXWsCnV7iDwD29m2MxYNqa3SzffsiXVYuIiIiIFygA1FUNO0Ofx+GuJTBsIZz/MCSdDpiw4Vd45xzYvpCmsSFcdnpDAF74dZ1vaxYRERGRGqcAUNcZhmudgF73wW0z4c4FENMCstNh3ABY9D53n98MP4vB7xv28fH8rZjHWE1YRERERE5tCgD1TVxLuHU6tLoEnMXw40iSf3+AoV0TAHh40kqueme+pgYVERERqaMUAOojWyhc+RH0fgwMCyz/hAcz7mHMeeEE+ltZmHaA/q/8zou/rqOguPKpREVERETk1KMAUF8ZBpx1D1z/DQRGYWT8yTXL/8mciw5xfotYih0mr87YSP9XfuePjft8Xa2IiIiIVBMFgPqu6Xnwr9mQ2BHyDxA95V+87/cMH14aQ1yojbR9uVz73gKembLW15WKiIiISDVQABCIaAQ3/QLnjgJrAMam6Zwz/RJ+O3MxN3VLBOCt2ZtYsHm/jwsVERERkZOlACAu/nY4979wxzxIPQdKCrD/PoZHdtzGg61dXYAe/m4lxQ6njwsVERERkZOhACBlxTSDG76Dy96D4FjYv4FbN4/gkcCvWL87h/Fzt/i6QhERERE5CQoAUp5hQPsrYPhiOOMWAG4yv6WTsYGXpq0nPTPfxwWKiIiIyIlSAJDKBUbAwBeg4/UAPBf8CflFxTwxebVv6xIRERGRE6YAIMfX+1GwhdGsZANX+v3GTysymL1+r6+rEhEREZEToAAgxxcSB+c8AMDD9q8IJY9Hv1upRcJERERETkEKAFI1Xf8F0c0JKTnIf4O+Y8v+PN75bbOvqxIRERERDykASNX4BUC/ZwC4xpxCU2MnY2duZOv+XB8XJiIiIiKeUACQqmveG07rj8Us4aWwzykqcTD6+1WYpunrykRERESkimpFAHj99ddp3Lgxdrudbt26sXDhwkqPHT9+PIZhlLnZ7XYvVlvP9X0KrAG0L1xCX7+lzFy3l6vens/KnZm+rkxEREREqsDnAeCLL75g5MiRPProoyxdupQOHTrQt29f9uzZU+ljwsLCSE9Pd9+2bt3qxYrrueim0H04AC+EfUGYfwkLtxzg4rFz+O/Ev9ibXejjAkVERETkWHweAF588UVuvfVWbrzxRlq3bs1bb71FUFAQH3zwQaWPMQyDhIQE9y0+Pt6LFQtn/xtCEwnJ28Gcs9dyacckTBM+X7Sd856fxTu/baKoxOnrKkVERESkAj4NAEVFRSxZsoTevXu7t1ksFnr37s28efMqfVxOTg4pKSkkJydz6aWXsmrVqkqPLSwsJCsrq8xNTpItBPo8DkDY/Od45cAw/mr8Kp+Gvc7/HG9R/Oto3n72fpav2+jjQkVERETk73waAPbt24fD4Sj3F/z4+HgyMjIqfEyLFi344IMP+O677/jkk09wOp306NGDHTt2VHj8mDFjCA8Pd9+Sk5Or/XXUS+2ugMZng7MY9qwiLGM+PYvmcq3fDIb5fc9dRe8RNuFilq3d5OtKRUREROQohunDKVx27dpFgwYN+OOPP+jevbt7+wMPPMDs2bNZsGDBcc9RXFxMq1atuOaaa3jiiSfK7S8sLKSw8Ei/9KysLJKTk8nMzCQsLKx6Xkh9VVIEe1ZD3n7IPwh5ByD/AEXZ+8hZ/i1Rjn0sMVti3DCJ05sm+rpaERERkTotKyuL8PDw417n+nmxpnJiYmKwWq3s3r27zPbdu3eTkJBQpXP4+/vTqVMnNm6suLuJzWbDZrOddK1SAb8ASOpYbnMAENx5KLnvXkhn1jLlo5tYdtMndEqJ9nqJIiIiIlKWT7sABQQE0LlzZ6ZPn+7e5nQ6mT59epkWgWNxOBysWLGCxET9hbk2sSW1xe+aTynBj/7GHywbdw9/7Tjk67JERERE6j2fzwI0cuRI3n33XT788EPWrFnDHXfcQW5uLjfeeCMAN9xwA6NGjXIf//jjj/Prr7+yefNmli5dyvXXX8/WrVu55ZZbfPUSpBK2087DcdGrANzE93z/3hNaL0BERETEx3zaBQjgqquuYu/evTzyyCNkZGTQsWNHfv75Z/fA4G3btmGxHMkpBw8e5NZbbyUjI4PIyEg6d+7MH3/8QevWrX31EuQYbF2uozBrO7bfxjDKfJ973ovhjluH0TpJ4y9EREREfMGng4B9oaqDI6QamSZF3w4j4K9PyTNtXFvyMGFNu9G/bQIXto4nOkRjNEREREROVlWvcxUAxDscxZR8cgV+aTPJMoN4s+QSxjsupNCwc0bjKPq3TaBf20QSwu2+rlRERETklKQAUAkFAB8qyIKPB8POxQAcNCJ4qehSPnecTxH+GAbcdnYT7uvbAn+rz4eniIiIiJxSFAAqoQDgY04HrPgKZj4Nh7YCkG1P5MOAa3hxz+k4sdAxOYLXrulEclSQj4sVEREROXUoAFRCAaCWKCmCZR/B7Ocgx7Xqc05oEx7OHsy3BacTavfn//7RngHtNL2riIiISFUoAFRCAaCWKcqDRe/CnJdcqwkD6/1a8EjeFcx3tua6bo14+KLW2P2tPi5UREREpHZTAKiEAkAtVZAJf7wG816H4jwAZjo68GzJ1ZjxbXliUFs6JUfgp7EBIiIiIhVSAKiEAkAtl50Bs5+FpR+CswQnBt85evB88ZVk2RLp1iSK7k1j6NksmtPiQrFYDF9XLCIiIlIrKABUQgHgFLF/E8x4ElZ9A0AOgfyn6FZ+dJ7pPiQ6OIBzTovlgX4tNX2oiIiI1HsKAJVQADjF7FoGU/4D2xcAsCrpcl6yDGXu1lzyix0ARAS5Bgz3bZPgy0pFREREfEoBoBIKAKcgRwnMehp+fxEwIb4tRZd9wNLcGJ76cQ0rdmYCcF23Rjw0sDWBARowLCIiIvVPVa9zNaJSaj+rH1zwCFw/EYJiYPdKAt47jzNzpjPxjh78q1cTAD5dsI1Lxs5hTXqWjwsWERERqb3UAiCnlqx0+OZW2PK7637LiyAomj37D7B2Wzp+JXmEWAppEOZHVM+bMLr9CwwNFBYREZG6T12AKqEAUAc4Ha6Zgmb/H3Dsr+/0oH7Mb/kgLRtE0aZBGE1jQ/DXVKIiIiJSBykAVEIBoA7ZtgA2/AJ+gRAQDAHBmAEhTN+cy4olcxlhfIHVMJnraMMdxXeTRQgBVgs9mkXz9OB2JEUE+voViIiIiFQbBYBKKADUD4UlDjIWf0fS1GH4O/LYZW3ALSUPsLowFoDIIH9evroT55wW6+NKRURERKqHBgFLvWbzs5Jy5mX43zoVwpNJcuzkx6DRzL7CjzZJYRzMK2bouIW8+Os6HM56lYFFRESknlMAkLotoS3cMh0adMHIP0jKj9cxqdt6ru2ajGnCqzM2csMHC9iXU+jrSkVERES8QgFA6r7QeBg6GdoMBmcx/lNG8vSBkXzY20Ggv5W5G/cz8NXfWbTlgK8rFREREalxGgMg9YfTCX+86ppBqDgXgJzU/ty55xJ+2x+OxYA2SeG0bRBO2wZhtGsQzmnxodj9tbCYiIiI1H4aBFwJBQAhOwNmPg3LPgbTiWnx47fwS7knvQ9OLDQ3dtDcspPmxk6aW3Zxml8Gh+wNWZV8HcVNe5McHUJyZBCJ4Xb8NKWoiIiI1BIKAJVQABC3PWtg6iOw4VcATMOKYTqO+ZBNzkTecwzgG8fZOCw2msWFcG+f07iwdTyGFhwTERERH1IAqIQCgJSzaSb8+jDsXuG6H94IYltgxpxGZkhTNhTH4J82nRY7JhLozAFgvxnGx47efFzSh/2Ec0HLOEZf0obkqKCKn6O4AHYtg23zwBYKZ9yiFYpFRESkWikAVEIBQCrkdMLBNAiJB1tIxccUZsPSj2H+m5C5DQATg51mNJucSWwxGpDSoiM9u3XHP7ox7FsPW/9wXfTvXAqOo2Ya6nYH9BujECAiIiLVRgGgEgoActIcJbDmO/hjLOxaWvXHBcdCYgfYOM11v+c90Hu0QoCIiIhUi6pe5/p5sSaRusHqB23/4brl7IX9GzD3bWDD6qXs3vQXDZ07aWTsId0Sz6bAduyJ6EReYjdCElvQMCqINo2/JmTaAzD3ZfAPhHP/6+tXJCIiIvWIWgBEqlFmXjHP/rKWzxZuwWlWPkPQQ9GzuCX3HQDMC0ZjnH2vlyoUERGRukpdgCqhACDesDe7kLR9uew4mMeOg/nsPJjPjkN5bD+Qz7YDeQDcYf2e//h/DsAPSSMIPGs4PZvFEBigdQdERETEcwoAlVAAEF/bnVXAzLV7mL52Dx03vsEwy0QA/ld8M5ONc+idYuWCZAtdE0xijRzI2w9hSdDsAtcMQiIiIiIVUACohAKA1CYFRSXs+XYUjda8c9xjnZYAChr2wGg5AHvrARgRyWX2F5U4ySooJiu/mNxCB03jggkKqKFhPgWZMHMMbF8AF70ESR1r5nlERESkyhQAKqEAILWOacKvD2HOex0DE4fhR6YRTkZJCPvNUDIJprWxlSaWjDIPW2ekssz/dHY4ItlVHEx6SQgHzFAOmGEcIoRUWzbXNSuib2IOCcW7YP9G180WCmfeCe2ucA1o9rTWVd/Az6MgZ7drW1A03DgFYltU0xsiIiIiJ0IBoBIKAFJr5R0AixVsYWAYHMorYvb6vcxat5et+3MJyk6jU948zjYX0dlYj9U4yf90IxvDWSOhwzXgF3D84w9shh/vg03TXfejmkJAEGSsgNAkuOlniEw5uZpERETkhCkAVEIBQE51+UUODu7dScnaXwjIWEpg8UFsRQfxLzyAJX8/Rt4BwMRp8We3NZGVhbFsdiawxUxgb0BDLo3N4PwDXxBccgiAHHsiG5rfwoHTriI1IZKU6GCslqPWJigphLmvwG/PuxYzswbA2f92rWNQlAvjB8DetRCZ6goBoQm+eFtERETqPQWASigASJ3ndLj66NvCwOrHnqwCvly8nc8WbmfnoXwAAingWusM/uU3mTjjEAB7zTD2meHYDAdBVgd2ixOb4SDALMBa4pq5KD2mO9NS72ezM4H9OUWYwKVNDC6YdwPGoa0Q1xqG/ghBUdXzOjbNgGWfQFgD6PMYWP1P/rwiIiJ1lAJAJRQApL5yOE1+27CX1buyyCksIbewhML8XDrt+4E+Bz8j2rGv0sfuNcN5vPif/ODsDpRfubhd4H4mWEcTWrIfGnSBG74DWwgApmmy42A+i7ceYPGWg+QWltC+YQSdGkXQJimcAL+/rZeQsxeWfQxLxsGhbUe2n9YPrhjvWjztePZvgpA4zZokIiL1igJAJRQARCpQUgjb5uNwOtmT62TLwSI2Hyxm0/4iNh0oYLsZS1hICDEhAUQFBxAVbCMmJID9uUV8s3QHu7MKOc3YzpcBjxNh5JIe3Y1f2r/Kwh25LN5ykD3ZhRU+bYCfhbZJYXRKjuCC4E203vk14Wk/YTiLXQfYw6HVxbDiaygpgJSz4JrPMG2hHMorJr/YQUKYHUtpl6XCHPj1IVd4sIdDtzvgzNshMNJLb6SIiIjvKABUQgFApHqVOJz8tmEvny/czr51f/CR31OEGAVsdCbxmeM8vnWcTbY1nDZJ4XRJiSQs0J/l2w+xbNtBHHmHGGydw3XWaZxm2ek+559mU362D2RNTG9iIyJIzf2ToVv/Q5CZx2qjKUMKH2Cv0/XX/eAAKy0SQrkwbBvX7XqK0LztZevzC2ZN8lVMj7yCzbmBFDucNI4JJjUmmCaHf0YFB2AY5Vs2qk3ufkhfBruWg6MIml8IDTrDiT6nacKi9+CvL+C0vtDpnxp7ISIiCgCVUQAQqTl7swuZP/0bLvjzXoJM13gDp8Uf87T+WDsPgabngWGBnUsxF7+PuXIilpICAAqwMcU4m/cKzmOVmVru3G2MND4KeIZoI5sNzgZcXzSK/ZZoDGcxd/tN5A7r91gNk51mNA8U30Y4udzlN4lWFlc3ojzTxqeOC/jO0YM44xDJxl4aGntJNvaSYt1LrCWH1WFns7r5v4hLTCYl2hUOIoP8y4QDh9Mkr6iE/CIHRQ4nITY/Qmx++FkPd2UqyoPt82HnEti1HDN9OUbmjnKvpyg4kbwmAyhpcRHWlO4EBQZg86vCKtBFufD9CFj59ZFthhVa9IcuN0KT88Fiqfzx1cVRDAe3VHDbCpnbXIPC218F7S53dccSEZEapwBQCQUAES8oyHR121n2MexadmR7WEMIinRNHVoqrjV0uQnaXwn2cApLHGRkFrDrUAG7DuWTnukKErGhNlLMHZw++yYCctMxw5Nx9nuW4ulPYd+3EoC5wb15pOgGNmW71jeIDfbj0qC/+GfRl6QUrq9S6bmmjfccA3i3ZCA5BBFqd13g5xU53Bf9R7PgpI2xhfP9V3K2dSXtzbUEUFLuvJudCawwm2DByXmW5YQYBe59e81wfnV0YYrfeewObUd8eCDxYXbiw2zEh9mJDgkg1O5PTOE2ms28A9uBdZgWP+j6L4ydS1yBo1REIzh9CLQcCNHNPV/r4bDMvGLW7c5mffpBAjK30tCxlYSCNCJzNxGSuQG/Q5uPdNU6FsMKzS6guO0V/GZ0Y3u2k84pUbROCsNalA2Z211jPbIzXNPIJnasnkHk5V7QTlj4NhxIg75Pw98W0quV9m+CH+6G3L3Q5wk47UJfVyQitZwCQCUUAES8LGMFLP3Y1V2l4JBrm9UGbQa7LvyTu3rWFebQNvjoUte6BKUCo+Dil6H1pYDr4tXmb8Huf/gv6qYJG6fB7y/AnjUQnuy62IxIoTgsmT3WBDIy80leMZa47FWupyGU14ov5RNHbwpxrZNg4KSBsY/mxk5aWHbQ3pJGd2MlkUZOmRJ3mtEsdrZghTOVlWYqq5yNMexhRIfYCPS34ijKp13hUs4u+YNzzcWEG7nux650NuZjRx++d3QnH7t7e1/LQp73f5tQI589ZgR3Fo1gmdEKm5+F1tYdXGFMZ4BzNqEcOVex4c9uWyr7Q08jO7wlhdGtKYlqRpFfKA6LDRMT0wSnCcUOJ2l7czi4Yy2Be/4kpXAd7S2baGNsJcioeAxHnmljX0ADSsIbEZLQjJiGp2GJSoWwRNg6D+efn2PZtcR9fLYZyFJnc2KNTBpa9hF2VK1Hc4Q3Ji+mHQfC27AzqBWZkW2Ij42hYUQgMSG2I2M+qiJjBfwx1tVi4jwczMIawg2TIKZ51c9TVY4S13fTWeIKMoFR7nU2TNOk2GGWH/j+d6YJi9+HXx+G4rwj21tdAv2egfAG1V+3iNQJCgCVUAAQ8ZHiAlg/xdU60OqSk/srb84e+Pgy2L3C1Z/+kteqpw+8acKa72H6E7B/g6vskAbkJHQjMHMTAYc2YDn6guwwZ0AoBQ16cDDxLHZHn8leWzKBAX5EhwQQHWwjKjig8ou+kiIcm2fj+PMr/NZOwuJwXWwXWkNYGtWfX+z96HRgCpfmurr8LKYVdxbexR4zotyp7BQywLKAy62/0d6yuUwrw98Vmn5kE0S2GUg2QRTiz2nGDsKN8q+v0LCz078Rm41k1jkasLwwkVXFDdhFNEfPChVq9+PMJtGc2SSaTXtzmLIincj8rQyyzmGwZS7Jlr3lzn3QDGGnGUOmNZJkM51GZJQ7xmEarDZTWORsyVJasSOsA8FRiSSEBRJq98PubyUowHWz+1sJ8rfQKHMhzTaMIyL9d/d5zJSzIGc3xv4NFNuimdn1LVY6Uth+MJ+dB/PJKSyhoMRBYbGTgmKH61biJDjASsuEMFomhtIqMYyWCaG0SAglyJkLu1e5QkbGCti90hUwS8q+74WWIDKNMPY4gtnhiGJ5UA/2NexNs0YNaJMURpukcKKCDy/Gl5UO3w93BVaAxmdDQjtY8DaYDggIgfMehK63edS6U+xwciC3iMJiJ0kR9iNd1jzgdJpk5hezL6eQvTmFZOUXExTgR3igv/sWFuiP1cD13+iBza7bwbTDv6e5xsB0vdU1bsVShS5vtZhpmjU7duhElRSB6QR/+/GPlTpHAaASCgAidURxAexbBwntT3wwbWUcJbD8U5j1DGTvKrvPGuDqWhPXEuJaQeNergG9J9jVpoy8A67nXfS+66Lp77oPh96jMS1+5BY5yCkooajESWGJg8ISJ4UlTopKnBSUOMjJL6LkwBZse1cRcmgtUTnric/fSGxJ+QvsMi/dEkB+dGsCkrsQ0KiL67VFNys3riC/yMHWA7nM27SfuRv3s2DzfrILy3d9ig4OYEC7RC5qF88ZlvVY9q/HEZLI2oJIft9tY9bWfJZuPeTuWhVGDm0tW+gWsIVOfmm0MjcR69hT7rybnIksN5tiAEEUEkQBQUYhQRQSYWSTZBxwvR7T4CdnN94puYiVNCHGyOYDvzG0s2whywzk5qL7WWS2PO5H408JLY1tdLRspKNlI52MTTSxpFd4bD52Ckx/wsnBUsmK3YWmH7OdHZjs6M405+mEh0dwdfASbsl8jWBHFg5LALu6PIDtrGH4W/3Yu3EJsbP/S+SB5QBs9W/KS7bb2Whrhd3PFXzs/hZs/lbsflbyi0vYl1PE/pxC9ucWcSjvSHctf6tBSnQwTWODaRobQrO4EFIjrFiydpC/dwslB7ZhydpOQM4uQgp2ke+wMtvZgR8KOrLJGV/h67HgpJtlDQMt8+lnXUyMkXnM97MwqgVc+CS2lsfp1nRgs6t7WEQKhCZWOL4lp7CE/elbKEybj3XnYoL3/YnTz05Bo3Owt+pHXGo7/I41vqakCPZvhKJcnE7Xf08FRcWHQ2AJmYUmOwtsbMnxZ1O2H2mZJew4mM++nELaJIVxfos4zmsZR4eGERW2Th3IzGLViqXs2riCgkMZBEQnE9OwBSnN2tAkKeaEwlgp0zQpcZoUlTgp2fUXfkvHEbj2a3A6OJjUi20JF7I+vCf7im0cyiui2GFydvMYzmoeU7UxRx4ocTjJLighu6CEAD8LcaEettadSkwTstMhJL7WBVkFgEooAIhIlRXnuy7I8w5AbAuIbQVRTarnYv9YnE7YPMMVBNb/DP5BcOnr0GZQ9Zy7KBsKsqAw68jPohyIauoak3G4y4onShxOVu7KYu7GfSzecoC4UDsXdUike5Po417g5Bc5WLUrE7u/1d1qUqbFJGsXbP0Dx5a5OLb8QcD+tcetJ9+w85NfHz40B7CuIJLCkiNjNyKt+YyzvUhH5yqKDBu/tn0OZ7M+hB1uTbD7W7FbHITvW0bo9plYts/DtncFVmdRuefZaUaz2pnCGrMRa5wprDZT2GbGYWLB7gdd4i10jnXSNtLBaaFFRGevw1g1keCsTUdqNQNYazaik2Uj4OoGdm/xnWwwG5Z5LgMnV1ln8V+/z4g43G0sx7Sz3wxjP2HsN8PZd/j3vWYEGWYkGWYUGWYUe4nAsFixWgxsJTm0NrbS1pJGG8sW2hppNDV2Ya0krBxtg7MBv1u6sDywOxkhrUjNW0G3/N/p5Zhf5qLfYRrsMmPYYsaz9ahbsrGXu/y+ddc/z+jI19G3Y4lvjZ/VoLCohPjsVbTJnkOHvD9ILjmyFkgR/qRb4tllJLDLksBeM4Lk4s10YD0NjcrXMdluxrLEvzObI7qTHdmO8PytxOWuJ7FgI42KNtHIsQ3/CsbtVCbfDCCTYDLNYLIJIscMJIdAiv2CiYyMJikuloCSLEr2rCckJ404x95Kg+AeM4L9AUkUhjSi2B5NvjWYfCOIfEsweUYQeUYQmYSw0xlNRnEQOYeDf/bhtVycxfn0NxZwvd80Ols2VPgchaY/vznb86OjG3Oc7Yg0smkVsJcL4nM5PfQQDcx0LJk7ILEDnDsKM7ope3MK2bg7hw17ctiVmU9BkcM1Dupwy1je4fs5hSVkFxSTXVBCXpGjzPMG+FlIjgykUVSQ6xYdTFJ4+dYnv8JDhGRtIDAkgoiUtsRHhuFfwf8zsguKWb3jAHvWzsN/yyxiM//CtIdhi21GbEpL4lNaYUQ1cbUGV+GPQnuyCli5K5OVO7NYtSuT/TlF5Be7XmN+0ZGfVotB26Rwuic4ONdvFaflLCJox28YObsh5jQY8Dxmai+yCkrYnVVAemYBuzMLuLxzQ58EIAWASigAiMgpJTsDLH4QHOPrSmqPvAOwbT7sWeUaTxIQ5OoaExDsCksBwa7WGXu4+yEFxQ6yC0pwOE1iQ21YHQXw5RDY8Ivr/b3sHdc6ExunwYZfYdNMKPzbX7HtEa4WkYZdyIzuwOaAFhT4R2KaJiYcHk/h+j0u1EazuJAKL2QwTdizGlZ+Aysnult7TCwsTB7KxJDr2HKomB0H8kjPKsA0ISbERnJUIMmRQZwWWsDAjDdpvON7DKr2T7hpWFx/rbQGuFbtrkA+dvb5xZEVkEheUBIloQ0gohExZJK4exbBGQswnEddKBtWV7ek0ucIjKSwaX8Opg5kd1QXDhZZyMwrJjO/mEN5xRzKL2JvdiEH9+2m74GPudr8mQDDgcM0+NpxDk4MeluXEntUkCg2rWSYUSQYB/A3yl5gHs1hGqRZG7PZ1pqMsLb4F2VxWtY82pWsJMA4/sV9lhlIphmCCTixYAKGYcEwDAKMEsKMPAKduYf3eC6HYA4FpWCGxGPNSScsfwchZs7xH3iUAtOfXWY06WY06URTaPrT37qAqMNjkIpNK786z+Brow+FAREMtC7g3JI/aOAoPwtZZRxY+JbzebZgMHs4/vopFpw0MnbTwtjBacZ2TrPsoKVlB36UsN0Zyw4zhh1mHNvNWHaYsRwihObGDlpbttLa2Epry9Yy4a3Q9GOdmcwma1MygluQGd6aYmsQYRnzaJW/hDMtqwkz8o9ZU4nFTn54Uw5EdWBfRHsywtpzIKABhSVOMvOLWbUri5U7Mytcn8aPEiLJJsbIIsbIJJosTrPs4GzLX7SzbKn0OX80e/B44bXs5kjX1oUPXkBcqPe7YSkAVEIBQEREANdUpt/efnhKVQP+fnEXGAXN+0CT86DhGRDdtPq7m5mma6asjdOh6fnQsHOZ3UUlTpymeWRA+9EKs1197XP3uWYKyt17+Pc9ruCYne5qPcnOKHOhDkB4I0hs7/qrb2IHV1e64/3lNP8QbJoO66a4QlJBpmuRvZYXuVqnUs8Bq3+VX3rWznU4pz5KxJYpZV+zXwi7485mb4MLyG54DkZgJFYc2HN3EZizjcDc7dhzthGQtwdL7GnYm5xJYOOuFa787SzIIXPNDIrW/kLw9lmE5O0g255EVnhL8qJaURjTFjO+LX5RKYTY/QkMsBIc4Ifd31K+f7/TebjV7JDrvSg45Fp8sDAbR34mO3fvYefu3ezfv598bPjHtyA+tS2ntelEdFzDcu+tM/cAu7asIT1tDbkZGwkoOoTdmYfdmeP66cjB5sglsPgQQcUHKn0fnaENcHQagqXLEKxhfxsLVRo2V38HqybBvnWYtjDyQlJIc8SxKCuc1YUx7DPDuc46jd5W16xt+WYA3wZcxPzEG4iJjSfYZiXIzySxaCtJuauJy15FZOZqQrI2YnVUPEmAJ3Zb4gly5pSZxKAyuZZQdkV1pahhDw5kZlG4ZyOBOdtINjNIMvbjZzjLPWafGcYyZzNWm43xo4Rwcgk38ogPKCTOv4AISx5BJQcJKDp217XN1lSmFbVlpqM9G5wNucvvG663TsNqmOSYdt6yXMnMsMHEhIfw1OC2NIwMOuH35EQpAFRCAUBERNycDvjpPlj8get+UifXwPLmF7p+r2X9e0+I0+EKB1k7Xd3a4lqf/FSrjmLXoN6oVI8u+iu0bT4sfPdwmBjgaok5gW5ox2WarkHIfrbqP3dNKyl0fX6ZOw//3OFqCWt8luu7WtVuiUW5rlayw2HE6TRZuu0g09bswd9q0NW6js7rXyFo92LX8fYI12rs+zZA+p9QUsFf3/3sENvS9b2KO/zTz+6ase3QNji01fXz4FbI2++afSuhvWtwe0I7SGjraq0zTZwHtpC9ZQmF25ZhyVhO8IFVBJTkkB3XhYAWFxDU4gJXYP3bf5dFJU5W7DzEok27Sdu4hoisdbR1rqNl8VoaF2/EnypMWVzKsEBQDATHQkgshDWA1F6uPwSExlNQ7GBNehar07MIDvCjSfFGWiwZjW33Utfj41rDgOehcc+qP2c1UgCohAKAiIiUYZqwa6lretDQige5itQbpukaezTtMdi7puy+gFBI6ggNToek010X8JGNay4om6brdjKLG5YUusLL9oWuiSP8gyEwwhU67OGukGMPh6Bo10V/YKTnz+d0wvJPYOqjkH+4teb6idCs94nXfYIUACqhACAiIiJyHE6Ha4xKxl8Q16bSGcHkKHkHYPrjrj8o3DKj5ieMqIACQCUUAERERESkxpQU+qyrWVWvcxXjRERERESqyykwzkQBQERERESkHlEAEBERERGpRxQARERERETqEQUAEREREZF6RAFARERERKQeUQAQEREREalHFABEREREROoRBQARERERkXpEAUBEREREpB5RABARERERqUcUAERERERE6hEFABERERGRekQBQERERESkHlEAEBERERGpRxQARERERETqEQUAEREREZF6RAFARERERKQe8fN1Ad5mmiYAWVlZPq5ERERERKT6lF7fll7vVqbeBYDs7GwAkpOTfVyJiIiIiEj1y87OJjw8vNL9hnm8iFDHOJ1Odu3aRWhoKIZheP35s7KySE5OZvv27YSFhXn9+cW39PnXb/r86zd9/vWbPv/6zVufv2maZGdnk5SUhMVSeU//etcCYLFYaNiwoa/LICwsTP8DqMf0+ddv+vzrN33+9Zs+//rNG5//sf7yX0qDgEVERERE6hEFABERERGRekQBwMtsNhuPPvooNpvN16WID+jzr9/0+ddv+vzrN33+9Vtt+/zr3SBgEREREZH6TC0AIiIiIiL1iAKAiIiIiEg9ogAgIiIiIlKPKACIiIiIiNQjCgBe9Prrr9O4cWPsdjvdunVj4cKFvi5JasCYMWM444wzCA0NJS4ujkGDBrFu3boyxxQUFDBs2DCio6MJCQnhH//4B7t37/ZRxVKTnnnmGQzD4J577nFv0+dft+3cuZPrr7+e6OhoAgMDadeuHYsXL3bvN02TRx55hMTERAIDA+nduzcbNmzwYcVSnRwOBw8//DCpqakEBgbStGlTnnjiCY6ec0Xfgbrjt99+4+KLLyYpKQnDMJg0aVKZ/VX5rA8cOMB1111HWFgYERER3HzzzeTk5NRo3QoAXvLFF18wcuRIHn30UZYuXUqHDh3o27cve/bs8XVpUs1mz57NsGHDmD9/PlOnTqW4uJgLL7yQ3Nxc9zH33nsvP/zwA1999RWzZ89m165dXHbZZT6sWmrCokWLePvtt2nfvn2Z7fr8666DBw/Ss2dP/P39mTJlCqtXr+aFF14gMjLSfcyzzz7Lq6++yltvvcWCBQsIDg6mb9++FBQU+LByqS7/93//x5tvvsnYsWNZs2YN//d//8ezzz7La6+95j5G34G6Izc3lw4dOvD6669XuL8qn/V1113HqlWrmDp1KpMnT+a3337jtttuq9nCTfGKrl27msOGDXPfdzgcZlJSkjlmzBgfViXesGfPHhMwZ8+ebZqmaR46dMj09/c3v/rqK/cxa9asMQFz3rx5vipTqll2drbZvHlzc+rUqeY555xj3n333aZp6vOv6/7zn/+YZ511VqX7nU6nmZCQYD733HPubYcOHTJtNpv52WefeaNEqWEDBw40b7rppjLbLrvsMvO6664zTVPfgboMML/99lv3/ap81qtXrzYBc9GiRe5jpkyZYhqGYe7cubPGalULgBcUFRWxZMkSevfu7d5msVjo3bs38+bN82Fl4g2ZmZkAREVFAbBkyRKKi4vLfB9atmxJo0aN9H2oQ4YNG8bAgQPLfM6gz7+u+/777+nSpQtXXHEFcXFxdOrUiXfffde9Py0tjYyMjDKff3h4ON26ddPnX0f06NGD6dOns379egD+/PNP5syZQ//+/QF9B+qTqnzW8+bNIyIigi5duriP6d27NxaLhQULFtRYbX41dmZx27dvHw6Hg/j4+DLb4+PjWbt2rY+qEm9wOp3cc8899OzZk7Zt2wKQkZFBQEAAERERZY6Nj48nIyPDB1VKdfv8889ZunQpixYtKrdPn3/dtnnzZt58801GjhzJ//73PxYtWsSIESMICAhgyJAh7s+4on8P9PnXDf/973/JysqiZcuWWK1WHA4HTz31FNdddx2AvgP1SFU+64yMDOLi4srs9/PzIyoqqka/DwoAIjVo2LBhrFy5kjlz5vi6FPGS7du3c/fddzN16lTsdruvyxEvczqddOnShaeffhqATp06sXLlSt566y2GDBni4+rEG7788ks+/fRTJkyYQJs2bVi+fDn33HMPSUlJ+g5IraEuQF4QExOD1WotN8vH7t27SUhI8FFVUtOGDx/O5MmTmTlzJg0bNnRvT0hIoKioiEOHDpU5Xt+HumHJkiXs2bOH008/HT8/P/z8/Jg9ezavvvoqfn5+xMfH6/OvwxITE2ndunWZba1atWLbtm0A7s9Y/x7UXffffz///e9/ufrqq2nXrh3//Oc/uffeexkzZgyg70B9UpXPOiEhodyEMCUlJRw4cKBGvw8KAF4QEBBA586dmT59unub0+lk+vTpdO/e3YeVSU0wTZPhw4fz7bffMmPGDFJTU8vs79y5M/7+/mW+D+vWrWPbtm36PtQBF1xwAStWrGD58uXuW5cuXbjuuuvcv+vzr7t69uxZbtrf9evXk5KSAkBqaioJCQllPv+srCwWLFigz7+OyMvLw2Ipe3lltVpxOp2AvgP1SVU+6+7du3Po0CGWLFniPmbGjBk4nU66detWc8XV2PBiKePzzz83bTabOX78eHP16tXmbbfdZkZERJgZGRm+Lk2q2R133GGGh4ebs2bNMtPT0923vLw89zG333672ahRI3PGjBnm4sWLze7du5vdu3f3YdVSk46eBcg09fnXZQsXLjT9/PzMp556ytywYYP56aefmkFBQeYnn3ziPuaZZ54xIyIizO+++87866+/zEsvvdRMTU018/PzfVi5VJchQ4aYDRo0MCdPnmympaWZ33zzjRkTE2M+8MAD7mP0Hag7srOzzWXLlpnLli0zAfPFF180ly1bZm7dutU0zap91v369TM7depkLliwwJwzZ47ZvHlz85prrqnRuhUAvOi1114zGzVqZAYEBJhdu3Y158+f7+uSpAYAFd7GjRvnPiY/P9+88847zcjISDMoKMgcPHiwmZ6e7ruipUb9PQDo86/bfvjhB7Nt27amzWYzW7Zsab7zzjtl9judTvPhhx824+PjTZvNZl5wwQXmunXrfFStVLesrCzz7rvvNhs1amTa7XazSZMm5oMPPmgWFha6j9F3oO6YOXNmhf/mDxkyxDTNqn3W+/fvN6+55hozJCTEDAsLM2+88UYzOzu7Rus2TPOopelERERERKRO0xgAEREREZF6RAFARERERKQeUQAQEREREalHFABEREREROoRBQARERERkXpEAUBEREREpB5RABARERERqUcUAERERERE6hEFABERqXUMw2DSpEm+LkNEpE5SABARkTKGDh2KYRjlbv369fN1aSIiUg38fF2AiIjUPv369WPcuHFlttlsNh9VIyIi1UktACIiUo7NZiMhIaHMLTIyEnB1z3nzzTfp378/gYGBNGnShK+//rrM41esWMH5559PYGAg0dHR/9/O/YTC1sdxHP8cEWZQNJkmGwuahqJEmdjIQpSikdSkYTNNmGyUmsiINTuzECuiZqFm4U+xnBIbw2JYq0nIhik241l4mprmdrvufa77PM95v1bn/H6nc76/3e/TOd8jv9+vl5eXnGs2NzfV2Nio4uJiORwOTU1N5cw/Pj5qcHBQFotF9fX1isViv3fRAGASBAAAwKfNz8/L4/EokUjI6/VqZGREyWRSkpROp9XT06PKykqdn58rGo3q+Pg4Z4MfiUQ0OTkpv9+vq6srxWIx1dXV5TxjcXFRw8PDury8VF9fn7xer56enr50nQDwf2S8v7+//+kiAAD/HmNjY9ra2lJJSUnOeCgUUigUkmEYCgQCikQi2bn29na1tLRobW1N6+vrmp2d1e3traxWqyRpf39f/f39SqVSstvtqqmp0fj4uJaXl79Zg2EYmpub09LSkqSPUFFWVqaDgwN6EQDgF9EDAADI09XVlbPBl6Sqqqrssdvtzplzu926uLiQJCWTSTU3N2c3/5LU0dGhTCajm5sbGYahVCql7u7u79bQ1NSUPbZaraqoqND9/f3PLgkA8DcCAAAgj9Vqzfsk559SWlr6Q9cVFRXlnBuGoUwm8ztKAgBToQcAAPBpp6eneecul0uS5HK5lEgklE6ns/PxeFwFBQVyOp0qLy9XbW2tTk5OvrRmAMAH3gAAAPK8vb3p7u4uZ6ywsFA2m02SFI1G1draqs7OTm1vb+vs7EwbGxuSJK/Xq4WFBfl8PoXDYT08PCgYDGp0dFR2u12SFA6HFQgEVF1drd7eXj0/PysejysYDH7tQgHAhAgAAIA8h4eHcjgcOWNOp1PX19eSPv7Qs7u7q4mJCTkcDu3s7KihoUGSZLFYdHR0pOnpabW1tcliscjj8WhlZSV7L5/Pp9fXV62urmpmZkY2m01DQ0Nft0AAMDH+AgQA+BTDMLS3t6eBgYE/XQoA4CfQAwAAAACYCAEAAAAAMBF6AAAAn8KXowDw38YbAAAAAMBECAAAAACAiRAAAAAAABMhAAAAAAAmQgAAAAAATIQAAAAAAJgIAQAAAAAwEQIAAAAAYCJ/AdKrXYzvYHVKAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "matplotlib.rcParams['figure.figsize'] = [9, 6]\n", "\n", "plt.plot(range(epochs), train_losses, label = \"Training loss\")\n", "plt.plot(range(epochs), test_losses, label = \"Testing loss\")\n", "plt.xlabel(\"Epoch\")\n", "plt.ylabel(\"Mean squared error loss\")\n", "plt.legend()\n", "plt.title(\"MSE loss vs training iterations\");" ] }, { "cell_type": "markdown", "metadata": { "id": "Aj8NrlzlJqDG" }, "source": [ "It seems like the model does a good job of fitting the training data while also generalizing well to the unseen test data." ] }, { "cell_type": "markdown", "metadata": { "id": "AUNIPubuPYDR" }, "source": [ "## Save and load the model\n", "\n", "Start by making an export module that takes in raw data and performs the following operations:\n", "- Feature extraction \n", "- Normalization \n", "- Prediction\n", "- Unnormalization" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "execution": { "iopub.execute_input": "2024-07-19T03:11:08.272529Z", "iopub.status.busy": "2024-07-19T03:11:08.271998Z", "iopub.status.idle": "2024-07-19T03:11:08.277614Z", "shell.execute_reply": "2024-07-19T03:11:08.276846Z" }, "id": "g-uOrGa9ZehG" }, "outputs": [], "source": [ "class ExportModule(tf.Module):\n", " def __init__(self, model, extract_features, norm_x, norm_y):\n", " # Initialize pre and postprocessing functions\n", " self.model = model\n", " self.extract_features = extract_features\n", " self.norm_x = norm_x\n", " self.norm_y = norm_y\n", "\n", " @tf.function(input_signature=[tf.TensorSpec(shape=[None, None], dtype=tf.float32)]) \n", " def __call__(self, x):\n", " # Run the ExportModule for new data points\n", " x = self.extract_features(x)\n", " x = self.norm_x.norm(x)\n", " y = self.model(x)\n", " y = self.norm_y.unnorm(y)\n", " return y " ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "execution": { "iopub.execute_input": "2024-07-19T03:11:08.280915Z", "iopub.status.busy": "2024-07-19T03:11:08.280350Z", "iopub.status.idle": "2024-07-19T03:11:08.283976Z", "shell.execute_reply": "2024-07-19T03:11:08.283139Z" }, "id": "YPYYLQ8EZiU8" }, "outputs": [], "source": [ "lin_reg_export = ExportModule(model=lin_reg,\n", " extract_features=onehot_origin,\n", " norm_x=norm_x,\n", " norm_y=norm_y)" ] }, { "cell_type": "markdown", "metadata": { "id": "6v8xi06XZWiC" }, "source": [ "If you want to save the model at its current state, use the `tf.saved_model.save` function. To load a saved model for making predictions, use the `tf.saved_model.load` function." ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "execution": { "iopub.execute_input": "2024-07-19T03:11:08.287321Z", "iopub.status.busy": "2024-07-19T03:11:08.286911Z", "iopub.status.idle": "2024-07-19T03:11:08.558410Z", "shell.execute_reply": "2024-07-19T03:11:08.557760Z" }, "id": "K1IvMoHbptht" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Assets written to: /tmpfs/tmp/tmp82n361w_/lin_reg_export/assets\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Assets written to: /tmpfs/tmp/tmp82n361w_/lin_reg_export/assets\n" ] } ], "source": [ "import tempfile\n", "import os\n", "\n", "models = tempfile.mkdtemp()\n", "save_path = os.path.join(models, 'lin_reg_export')\n", "tf.saved_model.save(lin_reg_export, save_path)" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "execution": { "iopub.execute_input": "2024-07-19T03:11:08.562372Z", "iopub.status.busy": "2024-07-19T03:11:08.561773Z", "iopub.status.idle": "2024-07-19T03:11:08.651770Z", "shell.execute_reply": "2024-07-19T03:11:08.650882Z" }, "id": "rYb6DrEH0GMv" }, "outputs": [ { "data": { "text/plain": [ "array([28.097498, 26.193336, 33.564373, 27.719315, 31.787922, 24.014559,\n", " 24.421043, 13.459579, 28.562454, 27.368692], dtype=float32)" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "lin_reg_loaded = tf.saved_model.load(save_path)\n", "test_preds = lin_reg_loaded(x_test)\n", "test_preds[:10].numpy()" ] }, { "cell_type": "markdown", "metadata": { "id": "-47O6_GLdRuT" }, "source": [ "## Conclusion\n", "\n", "Congratulations! You have trained a regression model using the TensorFlow Core low-level APIs.\n", "\n", "For more examples of using TensorFlow Core APIs, check out the following guides:\n", "* [Logistic regression](./logistic_regression_core.ipynb) for binary classification\n", "* [Multi-layer perceptrons](./mlp_core.ipynb) for hand-written digit recognition\n" ] } ], "metadata": { "colab": { "collapsed_sections": [ "rX8mhOLljYeM" ], "name": "quickstart_core.ipynb", "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.19" } }, "nbformat": 4, "nbformat_minor": 0 }