{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "Tce3stUlHN0L" }, "source": [ "##### Copyright 2021 The TensorFlow Authors." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "cellView": "form", "execution": { "iopub.execute_input": "2025-06-21T11:29:46.632121Z", "iopub.status.busy": "2025-06-21T11:29:46.631883Z", "iopub.status.idle": "2025-06-21T11:29:46.635903Z", "shell.execute_reply": "2025-06-21T11:29:46.635274Z" }, "id": "IcfrhafzkZbH" }, "outputs": [], "source": [ "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ] }, { "cell_type": "markdown", "metadata": { "id": "qFdPvlXBOdUN" }, "source": [ "# Sparse weights using structural pruning" ] }, { "cell_type": "markdown", "metadata": { "id": "MfBg1C5NB3X0" }, "source": [ "\n", " \n", " \n", " \n", " \n", "
\n", " View on TensorFlow.org\n", " \n", " Run in Google Colab\n", " \n", " View source on GitHub\n", " \n", " Download notebook\n", "
" ] }, { "cell_type": "markdown", "metadata": { "id": "FbORZA_bQx1G" }, "source": [ "Structural pruning weights from your model to make it sparse in specific pattern can accelerate model inference time with appropriate HW supports.\n", "\n", "This tutorial shows you how to:\n", "* Define and train a model on the mnist dataset with a specific structural sparsity\n", "* Convert the pruned model to tflite format\n", "* Visualize structure of the pruned weights\n", "\n", "For a general overview of the pruning technique for the model optimization, see the [pruning overview](https://www.tensorflow.org/model_optimization/guide/pruning). For tutorial on general weight pruning, see [Pruning in Keras](https://www.tensorflow.org/model_optimization/guide/pruning/pruning_with_keras)." ] }, { "cell_type": "markdown", "metadata": { "id": "0f4SoBcoXNcb" }, "source": [ "## Structural pruning of weights" ] }, { "cell_type": "markdown", "metadata": { "id": "rn_a9362Wr_B" }, "source": [ "Structural pruning systematically zeroes out model weights at the beginning of the training process. You apply this pruning techniques to regular blocks of weights to speed up inference on supporting HWs, for example: grouping weights in the model by blocks of four and zeroing out two of those weights in each block, known as a _2 by 4_ reduction. This technique applies only to the last dimension of the weight tensor for the model that is converted by TensorFlow Lite. For example, `Conv2D` layer weights in TensorFlow Lite have the structure `[channel_out, height, width, channel_in]` and `Dense` layer weights have the structure `[channel_out, channel_in]`. The sparsity pattern is applied to the weights in the last dimension: `channel_in`.\n", "\n", "Compare to the random sparsity, the structured sparsity generally has lower accuracy due to restrictive structure, however, it can reduce inference time significantly on the supported hardware.\n", "\n", "Pruning can be applied to a model together with other model compression techniques for better compression rate. See quantization and clustering examples in [collaborative optimization technique](https://blog.tensorflow.org/2021/10/Collaborative-Optimizations.html) for more details." ] }, { "cell_type": "markdown", "metadata": { "id": "nuABqZnXVDvO" }, "source": [ "## Setup" ] }, { "cell_type": "markdown", "metadata": { "id": "u9mRDekZEfnR" }, "source": [ "Prepare your development environment and data." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "cellView": "both", "execution": { "iopub.execute_input": "2025-06-21T11:29:46.639373Z", "iopub.status.busy": "2025-06-21T11:29:46.639126Z", "iopub.status.idle": "2025-06-21T11:29:52.606499Z", "shell.execute_reply": "2025-06-21T11:29:52.605201Z" }, "id": "lvpH1Hg7ULFz" }, "outputs": [], "source": [ "! pip install -q tensorflow\n", "! pip install -q tensorflow-model-optimization\n", "! pip install -q matplotlib" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2025-06-21T11:29:52.610242Z", "iopub.status.busy": "2025-06-21T11:29:52.609924Z", "iopub.status.idle": "2025-06-21T11:29:56.182292Z", "shell.execute_reply": "2025-06-21T11:29:56.181261Z" }, "id": "_hn5e5_gWr_E" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2025-06-21 11:29:52.922961: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:467] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n", "WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n", "E0000 00:00:1750505392.944354 24882 cuda_dnn.cc:8579] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n", "E0000 00:00:1750505392.950893 24882 cuda_blas.cc:1407] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n", "W0000 00:00:1750505392.968012 24882 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.\n", "W0000 00:00:1750505392.968035 24882 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.\n", "W0000 00:00:1750505392.968037 24882 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.\n", "W0000 00:00:1750505392.968040 24882 computation_placer.cc:177] computation placer already registered. Please check linkage and avoid linking the same target more than once.\n" ] } ], "source": [ "import tensorflow as tf\n", "from tensorflow import keras\n", "\n", "import tensorflow_model_optimization as tfmot\n", "prune_low_magnitude = tfmot.sparsity.keras.prune_low_magnitude" ] }, { "cell_type": "markdown", "metadata": { "id": "TZyLYFTER4aP" }, "source": [ "## Download and normalize image data from the [MNIST](https://www.tensorflow.org/datasets/catalog/mnist) dataset" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2025-06-21T11:29:56.186104Z", "iopub.status.busy": "2025-06-21T11:29:56.185626Z", "iopub.status.idle": "2025-06-21T11:29:56.634263Z", "shell.execute_reply": "2025-06-21T11:29:56.633219Z" }, "id": "hSf4jYKGWr_E" }, "outputs": [], "source": [ "# Load MNIST dataset.\n", "mnist = keras.datasets.mnist\n", "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()\n", "\n", "# Normalize the input image so that each pixel value is between 0 and 1.\n", "train_images = train_images / 255.0\n", "test_images = test_images / 255.0" ] }, { "cell_type": "markdown", "metadata": { "id": "LKaL3XH1XO0Q" }, "source": [ "## Define structural pruning parameters" ] }, { "cell_type": "markdown", "metadata": { "id": "s9_33ta-Wr_E" }, "source": [ "Define parameters for pruning and specify the type of structural pruning. Set the parameters for pruning to `(2, 4)`.\n", "These settings mean that in a block of four elements, at least two with the lowest magnitude are set to zero.\n", "\n", "You don't have to set the `pruning_schedule` parameter. By default, the pruning mask is defined at the first step and it is not updated during the training." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2025-06-21T11:29:56.638336Z", "iopub.status.busy": "2025-06-21T11:29:56.638015Z", "iopub.status.idle": "2025-06-21T11:29:56.642139Z", "shell.execute_reply": "2025-06-21T11:29:56.641329Z" }, "id": "1EXNYAPJWr_F" }, "outputs": [], "source": [ "pruning_params_2_by_4 = {\n", " 'sparsity_m_by_n': (2, 4),\n", "}" ] }, { "cell_type": "markdown", "metadata": { "id": "mMKdsdAUWr_F" }, "source": [ "Define parameters for random pruning with the target sparsity of 50%." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2025-06-21T11:29:56.645408Z", "iopub.status.busy": "2025-06-21T11:29:56.644679Z", "iopub.status.idle": "2025-06-21T11:29:56.649172Z", "shell.execute_reply": "2025-06-21T11:29:56.648310Z" }, "id": "un24AZUOWr_F" }, "outputs": [], "source": [ "pruning_params_sparsity_0_5 = {\n", " 'pruning_schedule': tfmot.sparsity.keras.ConstantSparsity(target_sparsity=0.5,\n", " begin_step=0,\n", " frequency=100)\n", "}" ] }, { "cell_type": "markdown", "metadata": { "id": "jV4Yt0v5Wr_G" }, "source": [ "Define the model architecture and specify which layers to prune. Structural pruning is applied based on the layers of the model you select.\n", "\n", "In the example below, we prune only some of the layers. We prune the second `Conv2D` layer and the first `Dense` layer.\n", "\n", "Notice that the first `Conv2D` layer cannot be pruned structurally. To be pruned structurally, it should have more than one input channels. Instead, we prune the first `Conv2D` layer with random pruning." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2025-06-21T11:29:56.652658Z", "iopub.status.busy": "2025-06-21T11:29:56.651860Z", "iopub.status.idle": "2025-06-21T11:29:57.415536Z", "shell.execute_reply": "2025-06-21T11:29:57.414581Z" }, "id": "BDGzC6YlWr_G" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2025-06-21 11:29:56.776140: E external/local_xla/xla/stream_executor/cuda/cuda_platform.cc:51] failed call to cuInit: INTERNAL: CUDA error: Failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Model: \"sequential\"\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "_________________________________________________________________\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " Layer (type) Output Shape Param # \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "=================================================================\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " prune_low_magnitude_prunin (None, 28, 28, 32) 1634 \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " g_sparsity_0_5 (PruneLowMa \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " gnitude) \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " max_pooling2d (MaxPooling2 (None, 14, 14, 32) 0 \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " D) \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " prune_low_magnitude_struct (None, 14, 14, 64) 102466 \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " ural_pruning (PruneLowMagn \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " itude) \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " batch_normalization (Batch (None, 14, 14, 64) 256 \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " Normalization) \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " re_lu (ReLU) (None, 14, 14, 64) 0 \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " max_pooling2d_1 (MaxPoolin (None, 7, 7, 64) 0 \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " g2D) \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " flatten (Flatten) (None, 3136) 0 \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " prune_low_magnitude_struct (None, 1024) 6423554 \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " ural_pruning_dense (PruneL \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " owMagnitude) \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " dropout (Dropout) (None, 1024) 0 \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " dense (Dense) (None, 10) 10250 \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " \n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "=================================================================\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Total params: 6538160 (24.94 MB)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Trainable params: 3274762 (12.49 MB)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Non-trainable params: 3263398 (12.45 MB)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "_________________________________________________________________\n" ] } ], "source": [ "model = keras.Sequential([\n", " prune_low_magnitude(\n", " keras.layers.Conv2D(\n", " 32, 5, padding='same', activation='relu',\n", " input_shape=(28, 28, 1),\n", " name=\"pruning_sparsity_0_5\"),\n", " **pruning_params_sparsity_0_5),\n", " keras.layers.MaxPooling2D((2, 2), (2, 2), padding='same'),\n", " prune_low_magnitude(\n", " keras.layers.Conv2D(\n", " 64, 5, padding='same',\n", " name=\"structural_pruning\"),\n", " **pruning_params_2_by_4),\n", " keras.layers.BatchNormalization(),\n", " keras.layers.ReLU(),\n", " keras.layers.MaxPooling2D((2, 2), (2, 2), padding='same'),\n", " keras.layers.Flatten(),\n", " prune_low_magnitude(\n", " keras.layers.Dense(\n", " 1024, activation='relu',\n", " name=\"structural_pruning_dense\"),\n", " **pruning_params_2_by_4),\n", " keras.layers.Dropout(0.4),\n", " keras.layers.Dense(10)\n", "])\n", "\n", "model.compile(optimizer='adam',\n", " loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n", " metrics=['accuracy'])\n", "\n", "model.summary()" ] }, { "cell_type": "markdown", "metadata": { "id": "U_ddzMppWr_G" }, "source": [ "Train and evaluate the model." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "execution": { "iopub.execute_input": "2025-06-21T11:29:57.419028Z", "iopub.status.busy": "2025-06-21T11:29:57.418729Z", "iopub.status.idle": "2025-06-21T11:30:31.322479Z", "shell.execute_reply": "2025-06-21T11:30:31.321705Z" }, "id": "F4CnppA1Wr_H" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Pruned test accuracy: 0.9865999817848206\n" ] } ], "source": [ "batch_size = 128\n", "epochs = 2\n", "\n", "model.fit(\n", " train_images,\n", " train_labels,\n", " batch_size=batch_size,\n", " epochs=epochs,\n", " verbose=0,\n", " callbacks=tfmot.sparsity.keras.UpdatePruningStep(),\n", " validation_split=0.1)\n", "\n", "_, pruned_model_accuracy = model.evaluate(test_images, test_labels, verbose=0)\n", "print('Pruned test accuracy:', pruned_model_accuracy)" ] }, { "cell_type": "markdown", "metadata": { "id": "bA8EDPHMWr_H" }, "source": [ "Remove the pruning wrapper so that it is not included in the model when you convert it to TensorFlow Lite format." ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "execution": { "iopub.execute_input": "2025-06-21T11:30:31.325952Z", "iopub.status.busy": "2025-06-21T11:30:31.325328Z", "iopub.status.idle": "2025-06-21T11:30:31.364954Z", "shell.execute_reply": "2025-06-21T11:30:31.364176Z" }, "id": "3wn-OQ_gWr_H" }, "outputs": [], "source": [ "model = tfmot.sparsity.keras.strip_pruning(model)" ] }, { "cell_type": "markdown", "metadata": { "id": "eM28m66YWr_H" }, "source": [ "## Convert model to tflite format" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "execution": { "iopub.execute_input": "2025-06-21T11:30:31.367848Z", "iopub.status.busy": "2025-06-21T11:30:31.367614Z", "iopub.status.idle": "2025-06-21T11:30:32.800142Z", "shell.execute_reply": "2025-06-21T11:30:32.799284Z" }, "id": "EJ7DsA6-Wr_I" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Assets written to: /tmpfs/tmp/tmpvliqo0lw/assets\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Assets written to: /tmpfs/tmp/tmpvliqo0lw/assets\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Saved converted pruned model to: /tmpfs/tmp/tmpr4fup215.tflite\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n", "W0000 00:00:1750505432.319830 24882 tf_tfl_flatbuffer_helpers.cc:365] Ignored output_format.\n", "W0000 00:00:1750505432.319870 24882 tf_tfl_flatbuffer_helpers.cc:368] Ignored drop_control_dependency.\n", "I0000 00:00:1750505432.328462 24882 mlir_graph_optimization_pass.cc:425] MLIR V1 optimization pass is not enabled\n" ] } ], "source": [ "import tempfile\n", "\n", "converter = tf.lite.TFLiteConverter.from_keras_model(model)\n", "tflite_model = converter.convert()\n", "\n", "_, tflite_file = tempfile.mkstemp('.tflite')\n", "print('Saved converted pruned model to:', tflite_file)\n", "with open(tflite_file, 'wb') as f:\n", " f.write(tflite_model)" ] }, { "cell_type": "markdown", "metadata": { "id": "S44x9Rz3Wr_I" }, "source": [ "## Visualize and check weights" ] }, { "cell_type": "markdown", "metadata": { "id": "_CTu0wxFWr_J" }, "source": [ "Now visualize the structure of weights in the `Dense` layer pruned with 2 by 4 sparsity. Extract the weights from the tflite file." ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "execution": { "iopub.execute_input": "2025-06-21T11:30:32.803495Z", "iopub.status.busy": "2025-06-21T11:30:32.803197Z", "iopub.status.idle": "2025-06-21T11:30:32.815115Z", "shell.execute_reply": "2025-06-21T11:30:32.814408Z" }, "id": "fOIp6QB5Wr_J" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/lite/python/interpreter.py:457: UserWarning: Warning: tf.lite.Interpreter is deprecated and is scheduled for deletion in\n", " TF 2.20. Please use the LiteRT interpreter from the ai_edge_litert package.\n", " See the [migration guide](https://ai.google.dev/edge/litert/migration)\n", " for details.\n", " \n", " warnings.warn(_INTERPRETER_DELETION_WARNING)\n", "/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/lite/python/interpreter.py:465: UserWarning: Warning: Enabling `experimental_preserve_all_tensors` with the BUILTIN or AUTO op resolver is intended for debugging purposes only. Be aware that this can significantly increase memory usage by storing all intermediate tensors. If you encounter memory problems or are not actively debugging, consider disabling this option.\n", " warnings.warn(\n", "INFO: Created TensorFlow Lite XNNPACK delegate for CPU.\n" ] } ], "source": [ "# Load tflite file with the created pruned model\n", "interpreter = tf.lite.Interpreter(model_path=tflite_file, experimental_preserve_all_tensors=True)\n", "interpreter.allocate_tensors()\n", "\n", "details = interpreter.get_tensor_details()\n", "\n", "# Weights of the dense layer that has been pruned.\n", "tensor_name = 'structural_pruning_dense/MatMul'\n", "detail = [x for x in details if tensor_name in x[\"name\"]]\n", "\n", "# We need the first layer.\n", "tensor_data = interpreter.tensor(detail[0][\"index\"])()" ] }, { "cell_type": "markdown", "metadata": { "id": "yy0jTs_QWr_K" }, "source": [ "To verify that we selected the correct layer that has been pruned, print the shape of the weight tensor." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "execution": { "iopub.execute_input": "2025-06-21T11:30:32.818290Z", "iopub.status.busy": "2025-06-21T11:30:32.817912Z", "iopub.status.idle": "2025-06-21T11:30:32.821529Z", "shell.execute_reply": "2025-06-21T11:30:32.820923Z" }, "id": "mCDkwMUPWr_K" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Shape of Dense layer is (1, 1024)\n" ] } ], "source": [ "print(f\"Shape of Dense layer is {tensor_data.shape}\")" ] }, { "cell_type": "markdown", "metadata": { "id": "mvYTILeUWr_K" }, "source": [ "Now we visualize the structure for a small subset of the weight tensor. The structure of the weight tensor is sparse in the last dimension, using the `(2,4)` pattern: two elements out of four are zeros. To make the visualization more clear, we replace all non-zero values with ones." ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "execution": { "iopub.execute_input": "2025-06-21T11:30:32.824319Z", "iopub.status.busy": "2025-06-21T11:30:32.823926Z", "iopub.status.idle": "2025-06-21T11:30:32.828119Z", "shell.execute_reply": "2025-06-21T11:30:32.827560Z" }, "id": "WZfn34bRWr_K" }, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "import numpy as np\n", "\n", "# The value 24 is chosen for convenience.\n", "width = height = 24\n", "\n", "subset_values_to_display = tensor_data[0:height, 0:width]\n", "\n", "val_ones = np.ones([height, width])\n", "val_zeros = np.zeros([height, width])\n", "subset_values_to_display = np.where(abs(subset_values_to_display) > 0, val_ones, val_zeros)" ] }, { "cell_type": "markdown", "metadata": { "id": "fOfWvKwKWr_L" }, "source": [ "Define the auxiliary function to draw separation lines to see the structure clearly." ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "execution": { "iopub.execute_input": "2025-06-21T11:30:32.830914Z", "iopub.status.busy": "2025-06-21T11:30:32.830567Z", "iopub.status.idle": "2025-06-21T11:30:32.835947Z", "shell.execute_reply": "2025-06-21T11:30:32.835322Z" }, "id": "LUplruw9Wr_L" }, "outputs": [], "source": [ "def plot_separation_lines(height, width):\n", "\n", " block_size = [1, 4]\n", "\n", " # Add separation lines to the figure.\n", " num_hlines = int((height - 1) / block_size[0])\n", " num_vlines = int((width - 1) / block_size[1])\n", " line_y_pos = [y * block_size[0] for y in range(1, num_hlines + 1)]\n", " line_x_pos = [x * block_size[1] for x in range(1, num_vlines + 1)]\n", "\n", " for y_pos in line_y_pos:\n", " plt.plot([-0.5, width], [y_pos - 0.5 , y_pos - 0.5], color='w')\n", "\n", " for x_pos in line_x_pos:\n", " plt.plot([x_pos - 0.5, x_pos - 0.5], [-0.5, height], color='w')" ] }, { "cell_type": "markdown", "metadata": { "id": "sbyjrRgLWr_L" }, "source": [ "Now visualize the subset of the weight tensor." ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "execution": { "iopub.execute_input": "2025-06-21T11:30:32.838745Z", "iopub.status.busy": "2025-06-21T11:30:32.838373Z", "iopub.status.idle": "2025-06-21T11:30:33.053212Z", "shell.execute_reply": "2025-06-21T11:30:33.052452Z" }, "id": "ATeyf5vCWr_L" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdAAAAGkCAYAAACB7nTXAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8ekN5oAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAzHUlEQVR4nO3deXQUVcL+8acTSIdFFgUSjZFAEAFBYBLIi8AgGg2KIL4yIiKEKLhGkYwL+FOCogFlERSUQREX5LC5byBGmFGJMKIgLiiiLMNrAgwiEAwh3fX7g0kPTdIhXVUkKer7OafPMZXauvvGy13qPh7DMAwBAICwRFT3DQAA4ERUoAAAmEAFCgCACVSgAACYQAUKAIAJVKAAAJhABQoAgAlUoAAAmEAFCgCACVSgCJKQkKDhw4dX922YNnz4cCUkJFTb9SdPnqyWLVsqMjJSnTp1qrb7OJVcdNFFuuiii6r7NoAyTukKdOPGjRo4cKCaN2+u6OhoxcXF6dJLL9XTTz8dtF9OTo7efPPN6rnJGnQPsObDDz/Ufffdp+7du2vevHnKyck5qdcbPny4PB5P4FW/fn21bNlSAwcO1GuvvSa/339Srw+4Xa3qvoGTZfXq1erdu7fOOeccjRw5UrGxsdqxY4c+//xzzZgxQ3feeWdg35ycHA0cOFADBgyotvutCfdwKnjuueeqreL4+OOPFRERoblz5yoqKqpKrun1evX8889Lkv744w9t27ZN77zzjgYOHKiLLrpIb731lho0aFAl9wK4zSlbgT722GNq2LCh/vnPf6pRo0ZBv9u1a5fp8xYWFqpevXoW765q+P1+FRcXKzo6urpvJcjJ/Axr1659Us5bGbt27VKdOnVsqzwNw1BRUZHq1KkTcp9atWrphhtuCNr26KOPatKkSRo7dqxGjhypRYsW2XI/KKum/o2hapyyXbhbtmzR+eefX6bylKRmzZoF/tvj8aiwsFAvvfRSoCusdAxw/Pjx8ng8+u6773T99dercePG6tGjh6TQ4zLljcH5/X7NmDFDHTp0UHR0tJo2bao+ffroiy++OOE9hBrTK723Y3k8HmVmZurVV1/V+eefL6/Xq2XLlkmSpkyZogsvvFBnnHGG6tSpo6SkJC1durQSn2RZW7dulcfj0ZQpU/Tkk0+qefPmqlOnjnr16qVvvvmmzOdRv359bdmyRVdccYVOO+00DRkyRFLo8dbjP9tVq1bJ4/Fo8eLFeuyxx3T22WcrOjpal1xyiX766acy1zv28zr2XufMmaPExER5vV516dJF//znP8tce8mSJWrXrp2io6PVvn17vfHGG5UaV/V4PJo3b54KCwsD3+GLL74oSSopKdGECRMC105ISNADDzygw4cPB50jISFBV155pZYvX67k5GTVqVNHf/vb3yq8bihjxozRZZddpiVLlujHH38M+t0HH3ygnj17ql69ejrttNPUt29fffvtt0H7lH5vO3fu1IABA1S/fn01bdpU99xzj3w+X9C+CxcuVFJSkk477TQ1aNBAHTp00IwZM4L22bdvn+6++27Fx8fL6/WqVatWevzxx031FhQXF2vcuHFKSkpSw4YNVa9ePfXs2VMrV64M7GMYhhISEnTVVVeVOb6oqEgNGzbULbfcEth2+PBhZWdnq1WrVvJ6vYqPj9d9991X5juq6G8M7nPKtkCbN2+uvLw8ffPNN2rfvn3I/V555RWNGDFCXbt21c033yxJSkxMDNrnL3/5i84991zl5OTITPrbTTfdpBdffFGXX365RowYoZKSEn3yySf6/PPPlZycXKl7qKyPP/5YixcvVmZmppo0aRL4H/+MGTPUv39/DRkyRMXFxVq4cKH+8pe/6N1331Xfvn1NXevll1/WgQMHdMcdd6ioqEgzZszQxRdfrI0bNyomJiawX0lJidLS0tSjRw9NmTJFdevWNXW9SZMmKSIiQvfcc49+//13PfHEExoyZIjWrFlzwmMXLFigAwcO6JZbbpHH49ETTzyh//3f/9XPP/8caLW+9957GjRokDp06KCJEyfqt99+00033aS4uLgTnv+VV17RnDlztHbt2kCX6oUXXihJGjFihF566SUNHDhQf/3rX7VmzRpNnDhR33//vd54442g8/zwww8aPHiwbrnlFo0cOVLnnXdeuB9TwNChQ/Xhhx9qxYoVat26deA+09PTlZaWpscff1yHDh3Ss88+qx49euirr74K+oeCz+dTWlqaUlJSNGXKFH300UeaOnWqEhMTddttt0mSVqxYocGDB+uSSy7R448/Lkn6/vvv9dlnn2nUqFGSpEOHDqlXr17auXOnbrnlFp1zzjlavXq1xo4dq19//VXTp08P633t379fzz//vAYPHqyRI0fqwIEDmjt3rtLS0rR27Vp16tRJHo9HN9xwg5544gnt3btXp59+euD4d955R/v37w+03P1+v/r3769PP/1UN998s9q2bauNGzfqySef1I8//lhmbkKovzG4kHGK+vDDD43IyEgjMjLS6Natm3HfffcZy5cvN4qLi8vsW69ePSM9Pb3M9uzsbEOSMXjw4DK/69Wrl9GrV68y29PT043mzZsHfv74448NScZdd91VZl+/33/Cezj+fMff27EkGREREca3335bZv9Dhw4F/VxcXGy0b9/euPjii4O2N2/evNz7ONYvv/xiSDLq1Klj/Otf/wpsX7NmjSHJGD16dND9SzLGjBlT5jyhrnX8Z7ty5UpDktG2bVvj8OHDge0zZswwJBkbN24Mut6xn1fpvZ5xxhnG3r17A9vfeustQ5LxzjvvBLZ16NDBOPvss40DBw4Etq1atcqQVO53cLz09HSjXr16QdvWr19vSDJGjBgRtP2ee+4xJBkff/xx0OchyVi2bNkJrxXqesf66quvgr6PAwcOGI0aNTJGjhwZtF9+fr7RsGHDoO2l39sjjzwStG/nzp2NpKSkwM+jRo0yGjRoYJSUlIS8jwkTJhj16tUzfvzxx6DtY8aMMSIjI43t27dX+D6PLw8lJSVB5cAwDOO3334zYmJijBtvvDGw7YcffjAkGc8++2zQvv379zcSEhICf3+vvPKKERERYXzyySdB+82ePduQZHz22WeBbRX9jcF9Ttku3EsvvVR5eXnq37+/NmzYoCeeeEJpaWmKi4vT22+/Hda5br31VtP38dprr8nj8Sg7O7vM747vgrVDr1691K5duzLbjx1H++233/T777+rZ8+e+vLLL01fa8CAAUGts65duyolJUXvv/9+mX1LWyxWZGRkBI0v9uzZU5L0888/n/DYQYMGqXHjxiGP/b//+z9t3LhRw4YNU/369QP79erVSx06dDB9z6WfRVZWVtD2v/71r5KOtnqP1aJFC6WlpZm+3rFK38eBAwckHW0t7tu3T4MHD9aePXsCr8jISKWkpAR1gZY6vuz37Nkz6PNu1KiRCgsLtWLFipD3sWTJEvXs2VONGzcOum5qaqp8Pp/+8Y9/hPW+IiMjA+XA7/dr7969KikpUXJyclB5bt26tVJSUvTqq68Gtu3du1cffPCBhgwZEvj7W7Jkidq2bas2bdoE3d/FF18sSWU+l1B/Y3CfU7YLV5K6dOmi119/XcXFxdqwYYPeeOMNPfnkkxo4cKDWr19f6T+CFi1amL6HLVu26KyzzgrqQjqZQt3ru+++q0cffVTr168PGtexUomfe+65Zba1bt1aixcvDtpWq1YtnX322aavU+qcc84J+rm0Qvztt98sH7tt2zZJUqtWrcoc26pVK9P/0Ni2bZsiIiLKnDc2NlaNGjUKXLeUlbJ2vIMHD0qSTjvtNEnS5s2bJSlQMRzv+Nm6peP1x2rcuHHQ53377bdr8eLFuvzyyxUXF6fLLrtM1157rfr06RPYZ/Pmzfr666/LnKuUmUl9L730kqZOnapNmzbpyJEjge3Hf37Dhg1TZmamtm3bpubNm2vJkiU6cuSIhg4dGnR/33//faXvz87vCM52SlegpaKiotSlSxd16dJFrVu3VkZGhpYsWVJuq7A85c2C9Hg85Y6HHj/BwqpQFVyo65R3r5988on69++vP//5z3rmmWd05plnqnbt2po3b54WLFhg6/2Wx+v1KiKibGdHRe8tMjKyzPbytkmq1Li0lWPtUNl/qFQ04zZcpRO6Sivv0gk7r7zyimJjY8vsX6tW8P8OQn1mx2rWrJnWr1+v5cuX64MPPtAHH3ygefPmadiwYXrppZcC17300kt13333lXuO0vHZypo/f76GDx+uAQMG6N5771WzZs0UGRmpiRMnasuWLUH7XnfddRo9erReffVVPfDAA5o/f76Sk5ODxpb9fr86dOigadOmlXu9+Pj4oJ/t/I7gbK6oQI+VnJwsSfr1118D28y0who3blxu1+HxLYrExEQtX768zESG44W6h8aNG2vfvn0nvE5FXnvtNUVHR2v58uXyer2B7fPmzav0OcpT2qI51o8//ljpSRUVvbeWLVtaurdwNW/eXJLKzOoNtS2c8/r9fm3evFlt27YNbC8oKNC+ffsC1z0ZXnnlFXk8Hl166aWS/jsxrVmzZkpNTbXtOlFRUerXr5/69esnv9+v22+/XX/729/00EMPqVWrVkpMTNTBgwdtu+bSpUvVsmVLvf7660F/N+X9g/j0009X37599eqrr2rIkCH67LPPykxaSkxM1IYNG3TJJZeclGEVnLpO2THQlStXltu6KB2TOvZfoPXq1Sv3f+QVSUxM1KZNm7R79+7Atg0bNuizzz4L2u+aa66RYRh6+OGHy5zj2PsLdQ+JiYn6/fff9fXXXwe2/frrr2Vmb1YkMjJSHo8nqNW6detWyysfvfnmm9q5c2fg57Vr12rNmjW6/PLLK3V8YmKiPv/8cxUXFwe2vfvuu9qxY4el+zLjrLPOUvv27fXyyy8Huj4l6e9//7s2btxo+rxXXHGFJJX5n3Zpa8fsDOgTmTRpkj788EMNGjQo0NWelpamBg0aKCcnJ6jbs9SxZbmy/v3vfwf9HBERoQsuuECSAkMF1157rfLy8rR8+fIyx+/bt08lJSVhXbO0ZXzs38+aNWuUl5dX7v5Dhw7Vd999p3vvvVeRkZG67rrrgn5/7bXXaufOnXruuefKHPvHH3+osLAwrPuDe5yyLdA777xThw4d0tVXX602bdqouLhYq1ev1qJFi5SQkKCMjIzAvklJSfroo480bdo0nXXWWWrRooVSUlIqPP+NN96oadOmKS0tTTfddJN27dql2bNn6/zzz9f+/fsD+/Xu3VtDhw7VU089pc2bN6tPnz7y+/365JNP1Lt3b2VmZlZ4D9ddd53uv/9+XX311brrrrsCjx20bt260uNyffv21bRp09SnTx9df/312rVrl2bNmqVWrVoFVczhatWqlXr06KHbbrtNhw8f1vTp03XGGWeE7Ko73ogRI7R06VL16dNH1157rbZs2aL58+ebfoTHqpycHF111VXq3r27MjIy9Ntvv2nmzJlq3759UKUajo4dOyo9PV1z5szRvn371KtXL61du1YvvfSSBgwYoN69e1u655KSEs2fP1/S0ecbt23bprfffltff/21evfurTlz5gT2bdCggZ599lkNHTpUf/rTn3TdddepadOm2r59u9577z11795dM2fODOv6I0aM0N69e3XxxRfr7LPP1rZt2/T000+rU6dOgRb3vffeq7fffltXXnmlhg8frqSkJBUWFmrjxo1aunSptm7dqiZNmlT6mldeeaVef/11XX311erbt69++eUXzZ49W+3atSv3e+rbt6/OOOMMLVmyRJdffnnQc+DS0Qp28eLFuvXWW7Vy5Up1795dPp9PmzZt0uLFiwPP5QJlVOMM4JPqgw8+MG688UajTZs2Rv369Y2oqCijVatWxp133mkUFBQE7btp0ybjz3/+s1GnTh1DUuDRitJHRXbv3l3uNebPn2+0bNnSiIqKMjp16mQsX7683MdOSkpKjMmTJxtt2rQxoqKijKZNmxqXX365sW7duhPeg2EcfSSnffv2RlRUlHHeeecZ8+fPD/kYyx133FHuvc6dO9c499xzDa/Xa7Rp08aYN29euecI5zGWyZMnG1OnTjXi4+MNr9dr9OzZ09iwYUPQvid61GLq1KlGXFyc4fV6je7duxtffPFFyMdYlixZUu59zJs3L+h65T3GMnny5DLXlmRkZ2cHbVu4cKHRpk0bw+v1Gu3btzfefvtt45prrjHatGlT4WdS0Xs9cuSI8fDDDxstWrQwateubcTHxxtjx441ioqKgvZr3ry50bdv3xNe59jrSQq86tatayQkJBjXXHONsXTpUsPn85V73MqVK420tDSjYcOGRnR0tJGYmGgMHz7c+OKLL074Xo4vM0uXLjUuu+wyo1mzZkZUVJRxzjnnGLfccovx66+/Bh134MABY+zYsUarVq2MqKgoo0mTJsaFF15oTJkypdxHy451fHnw+/1GTk6O0bx5c8Pr9RqdO3c23n333ZCPfBmGYdx+++2GJGPBggXl/r64uNh4/PHHjfPPP9/wer1G48aNjaSkJOPhhx82fv/998B+Ff2NwX08hlFFsyhwyti6datatGihyZMn65577qnu2znpOnXqpKZNm1b4qAZqttGjR2vu3LnKz883vZAHcLxTdgwUCNeRI0fKjMetWrVKGzZsIE7LwYqKijR//nxdc801VJ6w1Sk7BgqEa+fOnUpNTdUNN9ygs846S5s2bdLs2bMVGxtraTENVI9du3bpo48+0tKlS/Xvf/87sLQgYBcqUOA/GjdurKSkJD3//PPavXu36tWrp759+2rSpEk644wzqvv2EKbvvvtOQ4YMUbNmzfTUU08RcA7bMQYKAHC0f/zjH5o8ebLWrVsXeMzvRNnKq1atUlZWlr799lvFx8frwQcfLDcdqiKMgQIAHK2wsFAdO3bUrFmzKrX/L7/8or59+6p3795av3697r77bo0YMaLcZ5UrQgsUAHDK8Hg8J2yB3n///XrvvfeC8ouvu+467du3L6x8V8ZAAQC2KCoqClpZzArDMMosrej1eoOWIzUrLy+vzNKSaWlpuvvuu8M6T6UrUH9+2eSNqpJ2VifTxy7/v/XmDvTUUUTM0VV6+tW/QUWHDp/ggJrF1Ps+5j37Cy6QjD/svalKsPJdmxVd16t3Dh5dzae63rcVZj6zY9+zE8u3RBk3Y4V/yUk7d1FRkVo0r6/8XfYEatSvX7/MylLZ2dkaP3685XPn5+crJiYmaFtMTIz279+vP/74o9KBAbRAAQCWFRcXK3+XT7+sa64Gp1mbXrP/gF8tkrZpx44dQTF7drQ+7UQFCgCwTYPTIixXoIFzNWhQJqfWDrGxsSooKAjaVlBQoAYNGoQVV0cFCgCwjc/wy2dxaqrP8NtzMyF069YtkMxVasWKFerWrVtY5+ExFgCAbfwybHmF4+DBg1q/fr3Wr18v6ehjKuvXr9f27dslSWPHjtWwYcMC+9966636+eefdd9992nTpk165plntHjxYo0ePTqs61KBAgAc7YsvvlDnzp3VuXNnSVJWVpY6d+6scePGSTqaoVxamUpSixYt9N5772nFihXq2LGjpk6dqueff15paWlhXZcuXACAbfzyy2oHbLhnuOiii1TRkgYvvvhiucd89dVX4d5aECpQAIBtfIYhn8X1eaweX1XowgUAwARaoAAA25iZBFTeOZyAChQAYBu/DPmoQAEACI+bWqCMgQIAYELlW6Ceyi9vZLfouhbWPzR93/89zmvl+tXF1PuuE/zfnpA7njSWvmuTgr/f6nnfVpj5zI59z44s3xJlvIZy0yxc8kABAJbt379fDRs21KbvY3SaxbVwDxzwq03bAv3+++8nZS1cu9CFCwCACZXPAy244GTeR4WuSjR/7be2fG3yyDqKiFkjSRrY7CYddlheorn3/d/37C9IkVT1WYlWvmuzvHW9WrprrqTqe99WmPnMjn3PTizfEmXcjNIM2JPJZ8MsXKvHV5XKj4FWY8iwpbBfs/d9zNjI4UOHnRc4bOZ9B40H/VEt33n1f87V876tsPqZObJ8S5TxGspnyIY0Fnvu5WSjCxcAABN4DhQAYBv/f15Wz+EEVKAAANv45ZHP4vNBfoc8S0YXLgAAJtACBQDYxm8cfVk9hxNQgQIAbOOzoQvX6vFVhQoUAGAbN1WgjIECAGACLVAAgG38hkd+w+IsXIvHVxUqUACAbejCBQAAFSIPNCTyQN2SlUgeqAPLt0QZr6F8ipDPYtvMZ9O9nGzkgQIALCvNA83deI7qWcwDLTzg1yUdtpMHCgDAqYg80JDIA3VLViJ5oM4r3xJl3IyqyQN1zyQi8kBDIQ/UpVmJ5IE6BmW8RvIZEfIZFsdAHTKwSBcuAAAm8BwoAMA2fnnkt9g288sZTVAqUACAbRgDBQDABHvGQJ3RAmUMFAAAE2iBAgBsc3QM1OJi8nThAgDcxm/DUn5OmUREFy4AACbQAgUA2MZNk4ioQAEAtvErgudAyyDOzFmIeqo04swcWL4lyjiqHXFmAADLSuPMXvmqg+qeFmnpXIcO+DS088YaH2dGFy4AwDb2BGo7o11HnFlIxJm5JeqJODPnlW+JMm5GVcSZuQlxZqEQZ+bSqCfizByDMl4j+Y0I+S3OwvU7ZGSRLlwAgG3owgUAwAS/JJ9hdSk/Z2AlIgAATKAFCgCwjT0LKTijbUcFCgCwjT1L+TmjAnXGXQIAUMPQAgUA2IY8UAAATKALFwAAVIgWKADANvYspOCMth0VKADANn7DI7/VhRQsHl9VyAMNiTxQt2QlkgfqwPItUcZR7cgDBQBYVpoHOumfvRRd31rnZtHBEo3p8nfyQAEA7mFPGsspNgZKHqizIojISqw88kCdV74lyrgZVZEH6pNHPot941aPryrkgYZCHqhLsxLJA3UMyjiqGV24AADb0IULAIAJPlnvgvXZcysnnTOqeQAAahhaoAAA29CFCwCACSwmDwCAg8yaNUsJCQmKjo5WSkqK1q5dW+H+06dP13nnnac6deooPj5eo0ePVlFRUVjXpAIFANjG+E8eqJWXEeYkpEWLFikrK0vZ2dn68ssv1bFjR6WlpWnXrl3l7r9gwQKNGTNG2dnZ+v777zV37lwtWrRIDzzwQFjXpQIFANimtAvX6isc06ZN08iRI5WRkaF27dpp9uzZqlu3rl544YVy91+9erW6d++u66+/XgkJCbrssss0ePDgE7Zaj0cFCgCokfbv3x/0Ony47CIUxcXFWrdunVJTUwPbIiIilJqaqry8vHLPe+GFF2rdunWBCvPnn3/W+++/ryuuuCKs+2MSEQDANnbGmcXHxwdtz87O1vjx44O27dmzRz6fTzExMUHbY2JitGnTpnLPf/3112vPnj3q0aOHDMNQSUmJbr311rC7cIkzC4k4M7dEPRFn5sDyLVHGayg7A7V37NgRlMbi9drz2a1atUo5OTl65plnlJKSop9++kmjRo3ShAkT9NBDD1X6PMSZAQAsK40zu+vTq+StX9vSuQ4fPKKnerxVqTiz4uJi1a1bV0uXLtWAAQMC29PT07Vv3z699dZbZY7p2bOn/ud//keTJ08ObJs/f75uvvlmHTx4UBERlfsHAGOgAADHioqKUlJSknJzcwPb/H6/cnNz1a1bt3KPOXToUJlKMjIyUpIUTpuSOLOQiDNzS9QTcWbOK98SZdyMqogz8ytCfotts3CPz8rKUnp6upKTk9W1a1dNnz5dhYWFysjIkCQNGzZMcXFxmjhxoiSpX79+mjZtmjp37hzown3ooYfUr1+/QEVaGcSZhUKcmUujnogzcwzKeI3kMzzyWZxEFO7xgwYN0u7duzVu3Djl5+erU6dOWrZsWWBi0fbt24NanA8++KA8Ho8efPBB7dy5U02bNlW/fv302GOPhXVdZuECABwvMzNTmZmZ5f5u1apVQT/XqlVL2dnZys7OtnRNKlAAgG3sfIylpqMCBQDYxrAhjcVgMXkAAE5dtEABALbxySOfxRUqrB5fVahAAQC28RvWxzD9Dlnehy5cAABMoAUKALCN34ZJRFaPrypUoAAA25SGYls9hxNQgQIAbFMdKxFVF+LMQiLOzC1RT8SZObB8S5RxVDvizAAAlpXGmV2Xe4Oi6kdZOlfxwWItvGR+peLMqhNduAAA2/hlw1J+DukGIs4sJOLM3BL1RJyZ88q3RBk3oyrizNyEOLNQiDNzadQTcWaOQRmvkQwbZuEap1oLFACAE3FTGosznlYFAKCGoQUKALANKxEBAGACXbgAAKBCtEABALZhLVwAAExwUxcuFSgAwDZuqkAZAwUAwARaoAAA27ipBUoFCgCwDRVoecgDdRayEiuNPFAHlm+JMo5qRx4oAMCy0jzQ1PdvUa161v6RUFJ4WB9d8TfyQAEA7kEXbjnIA3VWBBFZiZVHHqjzyrdEGTeDPFB7kQcaCnmgLs1KJA/UMSjjNRItUAAATHBTBcpCCgAAmEALFABgGze1QKlAAQC2MQyPDIsVoNXjqwoVKADANm6KM2MMFAAAE2iBAgBswxgoAAAmuGkMlC5cAABMoAUKALANXbjlIc7MWYh6qjTizBxYviXKeA3lpi5c4swAAJaVxpklvTbaljizddc8SZwZAMA9DBu6cJ3SAiXOLCTizNwS9UScmfPKt0QZN6Mq4swMSVb7NZ3SLUqcWSjEmbk06ok4M8egjKOa0YULALCNXx55XLKUHxUoAMA2bpqFSwUKALCN3/DI45LnQFmJCAAAE2iBAgBsYxg2zMJ1yDRcKlAAgG3cNAZKFy4AACbQAgUA2MZNLVAqUACAbZiFCwAAKkQLFABgG2bhloc8UGchK7HSyAN1YPmWKOM11NEK1OoYqE03c5KRBwoAsKw0D/Tc+WMUWTfa0rl8h4q0+YZJ5IECANyDWbjlIA/UWRFEZCVWHnmgzivfEmXcjCrLA7XhHE5AHmgo5IG6NCuRPFDHoIzXSG5qgfIYCwAAJjAGCgCwj4v6cGmBAgDs858uXCsvmejCnTVrlhISEhQdHa2UlBStXbu2wv337dunO+64Q2eeeaa8Xq9at26t999/P6xr0gIFADjaokWLlJWVpdmzZyslJUXTp09XWlqafvjhBzVr1qzM/sXFxbr00kvVrFkzLV26VHFxcdq2bZsaNWoU1nWpQAEAtqmOlYimTZumkSNHKiMjQ5I0e/Zsvffee3rhhRc0ZsyYMvu/8MIL2rt3r1avXq3atWtLkhISEsK+T7pwAQC2sdp9e+ws3v379we9Dh8uO4O5uLhY69atU2pqamBbRESEUlNTlZeXV+49vv322+rWrZvuuOMOxcTEqH379srJyZHP5wvrvVKBAgBqpPj4eDVs2DDwmjhxYpl99uzZI5/Pp5iYmKDtMTExys/PL/e8P//8s5YuXSqfz6f3339fDz30kKZOnapHH300rPujCxcAYB+Tk4DKnEPSjh07gpby83rtWUfY7/erWbNmmjNnjiIjI5WUlKSdO3dq8uTJys7OrvR5qEABALaxcwy0QYMGJ1wLt0mTJoqMjFRBQUHQ9oKCAsXGxpZ7zJlnnqnatWsrMjIysK1t27bKz89XcXGxoqKiKnWfdOECAOxj2PSqpKioKCUlJSk3Nzewze/3Kzc3V926dSv3mO7du+unn36S3+8PbPvxxx915plnVrrylIgzqwBxZm6JeiLOzIHlW6KMIyArK0vp6elKTk5W165dNX36dBUWFgZm5Q4bNkxxcXGBMdTbbrtNM2fO1KhRo3TnnXdq8+bNysnJ0V133RXWdStdgUbEmF2U3bp3DlbbpSUpsOi2m5QuuF3Vqvu7rq73bYXVz8yN5Vtybxk/2apjLdxBgwZp9+7dGjdunPLz89WpUyctW7YsMLFo+/btioj4b4drfHy8li9frtGjR+uCCy5QXFycRo0apfvvvz+s65IHCgCwrDQP9Jw54xRRx1oeqP+PIm2/+ZFTJw+UODNnJSgQ9VR5xJk5r3xLlHEzqiLOzE2IMwuFODOXRj0RZ+YYlPEayU1xZjzGAgCwD2ksAACgIrRAAQA28sj680F04QIA3IYuXAAAUBFaoAAA+7ioBUoFCgCwj41pLDUdFSgAwDZ2prHUdIyBAgBgAi1QAIB9GAMtB3FmzkLUU6URZ+bA8i1RxmsqF42BksYCALCsNI3l7KcesSWN5V93jTt10lgAADgRj3H0ZfUcTkCcWUjEmbkl6ok4M+eVb4kybkaVxJkxBloO4szM30N1IOrJJOLMHIMyjmpGFy4AwD4umkREBQoAsI+LunBZSAEAABNogQIA7OOiFigVKADAPlSgAACY4KJJRIyBAgBgAi1QAIBtWIkIAAAzXDQGShcuAAAmUIECAGACeaAhkQfqlqxE8kAdWL4lyngN5ZENY6C23MnJRx4oAMCy0jzQ5o8/qohoi3mgRUXadv+D5IECAFzERc+BkgcaEnmgbslKJA/UeeVbooybQR6ovcgDDYU8UJdmJZIH6hiUcVQzunABAPahBQoAQPhYiQgAADNc1AJlIQUAAEygBQoAsI+LWqBUoAAA27hpDJQuXAAATKAFCgCwDysRAQBggovGQOnCBQDABOLMQiLOzC1RT8SZObB8S5TxGspNk4iIMwMAWFYaZ9ZyXI4tcWY/P/JAjY8zowsXAAATiDMLiTgzt0Q9EWfmvPItUcbNqKo4M8tdsA7pFyXOLBTizFwa9UScmWNQxmsmF83C5TEWAIB9XFSBMgYKAIAJtEABALZx02MstEABADCBChQAABPowgUA2MdFk4ioQAEAtmEMFAAAVIgWKADAXg5pQVpFBQoAsI+LxkDpwgUAwATyQEMiD9QtWYnkgTqwfEuU8RrKTZOIyAMFAFhWmgd67r05ivRaywP1HS7S5sk1Pw+UMVAAgG3c1AIlDzQk8kDdkpVIHqjzyrdEGTejSvJAXYQ80FDIA3VpViJ5oI5BGa+ZqmkW7qxZszR58mTl5+erY8eOevrpp9W1a9cTHrdw4UINHjxYV111ld58882wrsksXACAfQybXmFYtGiRsrKylJ2drS+//FIdO3ZUWlqadu3aVeFxW7du1T333KOePXuGd8H/oAIFADjatGnTNHLkSGVkZKhdu3aaPXu26tatqxdeeCHkMT6fT0OGDNHDDz+sli1bmrouFSgAwDalk4isvqSjM3uPfR0+XLb7u7i4WOvWrVNqampgW0REhFJTU5WXlxfyPh955BE1a9ZMN910k+n3SgUKALCPjV248fHxatiwYeA1ceLEMpfbs2ePfD6fYmJigrbHxMQoPz+/3Fv89NNPNXfuXD333HOW3iqPsQAAaqQdO3YEPQfq9VpfhOLAgQMaOnSonnvuOTVp0sTSuahAAQD2sXEWboMGDU64kEKTJk0UGRmpgoKCoO0FBQWKjY0ts/+WLVu0detW9evXL7DN7/dLkmrVqqUffvhBiYmJlbpNunABALaxcwy0MqKiopSUlKTc3NzANr/fr9zcXHXr1q3M/m3atNHGjRu1fv36wKt///7q3bu31q9fr/j4+EpfmxYoAMDRsrKylJ6eruTkZHXt2lXTp09XYWGhMjIyJEnDhg1TXFycJk6cqOjoaLVv3z7o+EaNGklSme0nQgUKALBPNSykMGjQIO3evVvjxo1Tfn6+OnXqpGXLlgUmFm3fvl0REfZ3uFKBAgBsU11r4WZmZiozM7Pc361atarCY1988cXwLyjizCpAnJlbop6IM3Ng+ZYo4zWViwK1iTMDAFhWGmfW9g574sy+n0WcGQDATVzUAiXOLCTizNwS9UScmfPKt0QZN6Mq4sw8st4z7pRRFOLMQiHOzKVRT8SZOQZlHNWMLlwAgH3owgUAIHzV9RhLdWApPwAATKAFCgCwD124AACY5JAK0Cq6cAEAMIEWKADANm6aREQFCgCwD2OgAACEz00tUMZAAQAwgTizkIgzc0vUE3FmDizfEmW8pnJRFy5xZgAAy0rjzC64MUeRURbjzIqL9PULNT/OjC5cAABMIM4sJOLM3BL1RJyZ88q3RBk3oyrizNzUhUucWSjEmbk06ok4M8egjNdMLqpA6cIFAMAEngMFANjGTc+BUoECAOxDFy4AAKgILVAAgG08hiGPxeUFrB5fVahAAQD2cVEXLhUoAMA2bppExBgoAAAm0AIFANiHLlwAAMJHFy4AAKgQeaAhkQfqlqxE8kAdWL4lynhN5aIuXPJAAQCWleaBJg16zJY80HWL/h95oAAAnIrIAw2JPFC3ZCWSB+q88i1Rxs0gD9Re5IGGQh6oS7MSyQN1DMp4jeWUWbRW0YULAIAJPAcKALCPYRx9WT2HA1CBAgBs46aFFKhAAQD2cdEkIsZAAQAwgRYoAMA2Hv/Rl9VzOAEVKADAPnThAgCAitACBQDYhlm4AACYwXOg5SDOzFmIeqo04swcWL4lyjiqHXFmAADLSuPMUvpNUK3a1uLMSo4Uac07D9X4ODO6cAEA9nHRLFzizEIizswtUU/EmTmvfEuUcTOqJM7MRYgzC4U4M5dGPRFn5hiU8RqJWbgAAJjBLFwAAMLnphYoKxEBAGACLVAAgH2YhQsAQPjowgUAABWiBQoAsI/fOPqyeg4HoAIFANjHRWOgdOECAGACLVAAgG08smESkS13cvJRgQIA7MNKROUgD9RZyEqsNPJAHVi+Jco4qh15oAAAy0rzQHtcPF61alnMAy0p0qcfjw8rD3TWrFmaPHmy8vPz1bFjRz399NPq2rVrufs+99xzevnll/XNN99IkpKSkpSTkxNy/1CYRAQAsI9h0ysMixYtUlZWlrKzs/Xll1+qY8eOSktL065du8rdf9WqVRo8eLBWrlypvLw8xcfH67LLLtPOnTvDum6lW6DkgTorgoisxMojD9R55VuijJtxMvNAS1ugPS/KtqUF+smqhyvdAk1JSVGXLl00c+ZMSZLf71d8fLzuvPNOjRkz5oTH+3w+NW7cWDNnztSwYcMqfZ/kgYZCHqhLsxLJA3UMyvgpb//+/UE/e71eeb3BY8jFxcVat26dxo4dG9gWERGh1NRU5eXlVeo6hw4d0pEjR3T66aeHdX904QIA7OO36SUpPj5eDRs2DLwmTpxY5nJ79uyRz+dTTExM0PaYmBjl5+dX6pbvv/9+nXXWWUpNTQ3rrfIYCwDANh7DkMfi3NTS43fs2BHUhXt869MOkyZN0sKFC7Vq1SpFR4fX9UwFCgCokRo0aHDCMdAmTZooMjJSBQUFQdsLCgoUGxtb4bFTpkzRpEmT9NFHH+mCC8Ifm6YLFwBgnyqehRsVFaWkpCTl5uYGtvn9fuXm5qpbt24hj3viiSc0YcIELVu2TMnJyWG8wf+iBQoAsE81rESUlZWl9PR0JScnq2vXrpo+fboKCwuVkZEhSRo2bJji4uICY6iPP/64xo0bpwULFighISEwVlq/fn3Vr1+/0telAgUAONqgQYO0e/dujRs3Tvn5+erUqZOWLVsWmFi0fft2RUT8t8P12WefVXFxsQYOHBh0nuzsbI0fP77S16UCBQDYxmPYsJi8ieMzMzOVmZlZ7u9WrVoV9PPWrVvDv0A5qEABAPZx0WLyTCICAMAEWqAAANt4/EdfVs/hBMSZhUScmVuinogzc2D5lijjNZWLunCJMwMAWFa6mPxFXf6fLYvJr/rnY2HFmVUHxkABADCh0l24xJk5K0GBqKfKI87MeeVbooybcTLjzErZuRZuTUecWSjEmbk06ok4M8egjNdMLhoDpQsXAAATeIwFAGAfQ4E8T0vncAAqUACAbdw0BkoXLgAAJtACBQDYx5ANk4hsuZOTjgoUAGAfZuECAICK0AIFANjHL+trDJ9yi8kDAHACbpqFSwUKALCPi8ZAiTMLiTgzt0Q9EWfmwPItUcZR7YgzAwBYVhpndkm7e1Qr0to/Ekp8h5X73ZQaH2dGFy4AwD504ZZFnJmzEhSIeqo84sycV74lyrgZVRFn5ibEmYVCnJlLo56IM3MMynjNxGMsAACEz02PsbASEQAAJtACBQDYh0lEAACY4Dckj8UK0O+MCpQuXAAATKAFCgCwD124AACYYUMF6pBEbSpQAIB9XNQCZQwUAAATaIECAOzjN2S5C9Yhs3CpQAEA9jH8R19Wz+EA5IGGRB6oW7ISyQN1YPmWKOOoduSBAgAsK80DTY2/TbUiLOaB+g/rox3PkgcKAHARxkDLIg/UWRFEZCVWHnmgzivfEmXcDPJA7UUeaCjkgbo0K5E8UMegjNdMLnoOlC5cAIB9DNlQgdpyJycdCykAAGACLVAAgH3owgUAwAS/X5LFhRD8p9pCCgAAnIiLWqCMgQIAYAItUACAfVzUAqUCBQDYx0UrEdGFCwCACbRAAQC2MQy/DItxZFaPryrEmYVEnJlbop6IM3Ng+ZYo4zWVYVjvgnXIGChxZgAAy0rjzC5pNEy1PFGWzlViFCt338vEmQEAXMSwYRKRQ9p1xJmFRJyZW6KeiDNzXvmWKONmVEmcmd8veSyOYZ5yY6DEmZm/h+pA1JNJxJk5BmUc1YwuXACAfejCBQAgfIbfL8NiF+6p9xgLAAAn4qIWKCsRAQBgAi1QAIB9/IbkcUcLlAoUAGAfw5DlQG2HVKB04QIAYAItUACAbQy/IcNiF65TVpilBQoAsI/ht+cVplmzZikhIUHR0dFKSUnR2rVrK9x/yZIlatOmjaKjo9WhQwe9//77YV+TChQA4GiLFi1SVlaWsrOz9eWXX6pjx45KS0vTrl27yt1/9erVGjx4sG666SZ99dVXGjBggAYMGKBvvvkmrOtSgQIAbGP4DVte4Zg2bZpGjhypjIwMtWvXTrNnz1bdunX1wgsvlLv/jBkz1KdPH917771q27atJkyYoD/96U+aOXNmWNelAgUA2KeKu3CLi4u1bt06paamBrZFREQoNTVVeXl55R6Tl5cXtL8kpaWlhdw/lEpPIoqI3RzWie20oppXdaqSBIMaJiLGbIqNNdX9XVfX+7bC6mfmxvItubeMn2wlOmJ5IaISHZF0NGP0WF6vV15vcCD5nj175PP5FBMTE7Q9JiZGmzZtKvf8+fn55e6fn58f1n0yCxcAYFlUVJRiY2P1aX74k3HKU79+fcXHxwdty87O1vjx4205vx2oQAEAlkVHR+uXX35RcXGxLeczDEMeT1D+XJnWpyQ1adJEkZGRKigoCNpeUFCg2NjYcs8dGxsb1v6hUIECAGwRHR2t6OjoKr1mVFSUkpKSlJubqwEDBkiS/H6/cnNzlZmZWe4x3bp1U25uru6+++7AthUrVqhbt25hXZsKFADgaFlZWUpPT1dycrK6du2q6dOnq7CwUBkZGZKkYcOGKS4uThMnTpQkjRo1Sr169dLUqVPVt29fLVy4UF988YXmzJkT1nWpQAEAjjZo0CDt3r1b48aNU35+vjp16qRly5YFJgpt375dERH/fejkwgsv1IIFC/Tggw/qgQce0Lnnnqs333xT7du3D+u6HsMpayYBAFCD8BwoAAAmUIECAGACFSgAACZQgQIAYAIVKAAAJlCBAgBgAhUoAAAmUIECAGACFSgAACZQgQIAYAIVKAAAJlCBAgBgwv8HimeDSS/TatAAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot_separation_lines(height, width)\n", "\n", "plt.axis('off')\n", "plt.imshow(subset_values_to_display)\n", "plt.colorbar()\n", "plt.title(\"Structural pruning for Dense layer\")\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "id": "72f7VlOLWr_M" }, "source": [ "Visualize weights for the `Conv2D` layer. The structural sparsity is applied in the last channel, similar to the `Dense` layer. Only the second `Conv2D` layer is structurally pruned as pointed out above." ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "execution": { "iopub.execute_input": "2025-06-21T11:30:33.056113Z", "iopub.status.busy": "2025-06-21T11:30:33.055841Z", "iopub.status.idle": "2025-06-21T11:30:33.061027Z", "shell.execute_reply": "2025-06-21T11:30:33.060415Z" }, "id": "_Dkbt7eRWr_M" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Shape of the weight tensor is (64, 5, 5, 32)\n" ] } ], "source": [ "# Get weights of the convolutional layer that has been pruned with 2 by 4 sparsity.\n", "op_details = interpreter._get_ops_details()\n", "op_name = 'CONV_2D'\n", "op_detail = [x for x in op_details if op_name in x[\"op_name\"]]\n", "tensor_data = interpreter.tensor(op_detail[1][\"inputs\"][1])()\n", "print(f\"Shape of the weight tensor is {tensor_data.shape}\")" ] }, { "cell_type": "markdown", "metadata": { "id": "m7a6uTdLWr_M" }, "source": [ "Similar to the weights of `Dense` layer, the last dimension of the kernel has a (2, 4) structure." ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "execution": { "iopub.execute_input": "2025-06-21T11:30:33.063626Z", "iopub.status.busy": "2025-06-21T11:30:33.063395Z", "iopub.status.idle": "2025-06-21T11:30:33.197818Z", "shell.execute_reply": "2025-06-21T11:30:33.197088Z" }, "id": "wyvLpfa6Wr_M" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdQAAAGkCAYAAACIBdStAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8ekN5oAAAACXBIWXMAAA9hAAAPYQGoP6dpAABAPElEQVR4nO3de3hM18I/8O9MSCJJiWtCmgpKo6g4IWloXCqEalB1KSqkyq/nNJQ8TlWLKG+pRtW9Doq2OK59e9pSSvA2JaWltKrUpa5torTukmH2+v3hzDQjE5mZtffsjP39PM88Dzv7tmbvmTVr7bX31ySEECAiIiIpZr13gIiI6F7ACpWIiEgFrFCJiIhUwAqViIhIBaxQiYiIVMAKlYiISAWsUImIiFTACpWIiEgFrFCJiIhUwArVC6KiojBo0CD7/7dv3w6TyYTt27frtk++wmQyYcKECbrug8zxsi27du1a9XfMBfn5+ejZsyeqVq0Kk8mEGTNm6LIfRsfPvDG4XaH+8MMP6NmzJ2rXro3AwEBERESgQ4cOmD17tsN8kydPxscff6zWfnqkLOwDkatWrFiheoU3cuRIbNq0CWPGjMGHH36ITp06qbp+ZwoKCvDOO+8gPj4elSpVQmBgIBo0aID09HT8/PPPmm+/NBcuXEBWVhZat26N6tWrIzQ0FI8++ihWrVpVbN6lS5fCZDLZX4GBgahVqxaSk5Mxa9YsXLlyRYcSUFlVzp2Zd+7ciXbt2uGBBx7AkCFDEB4ejtOnT+Prr7/GzJkzMWzYMPu8kydPRs+ePdG9e3e199llZWEfyPe1bt0aN27cgL+/v6bbWbFiBQ4cOIARI0aots6tW7eiW7duGDVqlGrrvJvz58+jU6dO2LNnD5588kn069cPISEhOHz4MFauXIkFCxbAYrF4ZV9Kkpubi9deew1PPPEExo4di3LlymHdunV45plncPDgQbz++uvFlpk4cSLq1KmDmzdvIi8vD9u3b8eIESMwffp0fPLJJ3jkkUd0KAmVNW5VqG+88QYqVaqEb775BqGhoQ5/O3funMc7ce3aNQQHB3u8vDcpigKLxYLAwEC9d0UVt27dgqIomlcWvsxsNvvs8T537lyxz6qMgoIC+Pv7w2x23rk1aNAgfPfdd1i7di2efvpph79NmjQJr732mmr74qlGjRrhyJEjqF27tn3aP/7xDyQlJWHq1Kl4+eWXi30fde7cGc2bN7f/f8yYMdi6dSuefPJJdO3aFT/99BMqVKjgtTKozZe+g8syt7p8jx07hkaNGjn9gNaoUcP+b5PJhGvXruH999+3d5XYriFOmDABJpMJBw8eRL9+/VC5cmU89thjAIC2bduibdu2xdY9aNAgREVFOUxTFAUzZ85EkyZNEBgYiOrVq6NTp0749ttvS90HZ+srum9FmUwmpKenY/ny5WjUqBECAgKwceNGAMC0adPQsmVLVK1aFRUqVEBsbKxH18oyMzNRvnx5/P7778X+NnToUISGhqKgoKDE5QcNGoSQkBAcP34cycnJCA4ORq1atTBx4kQUDRM6ceIETCYTpk2bhhkzZqBevXoICAjAwYMH7V1bJ06ccFi3s2s/bdu2RePGjXHw4EG0a9cOQUFBiIiIwFtvvVVs3woLC5GZmYkHH3wQAQEBiIyMxMsvv4zCwsJi840cORLVq1fHfffdh65du+LMmTOlvndCCFSrVg0ZGRn2aYqiIDQ0FH5+frh48aJ9+tSpU1GuXDlcvXrVPu3QoUPo2bMnqlSpgsDAQDRv3hyffPJJqe8BAMydOxd169ZFhQoVEBcXh5ycnBLPYUVR8MYbb+D+++9HYGAg2rdvj6NHjzq8p+vXr8fJkyft52vRc3T27Nlo1KgRgoKCULlyZTRv3hwrVqwo8X2xHU8hBObOnWtfp83x48fRq1cvVKlSBUFBQXj00Uexfv16p+VeuXIlxo4di4iICAQFBeHy5ctOt7lr1y6sX78egwcPLlaZAkBAQACmTZvmMG3r1q1ITExEcHAwQkND0a1bN/z0008O89g+l0ePHsWgQYMQGhqKSpUqIS0tDdevX7fP17hxY7Rr167YdhVFQUREBHr27AkAqFOnjkNlCtz+nHfv3h2FhYU4fvy40/Ld6fHHH8e4ceNw8uRJLFu2zKVlisrJyUGvXr3wwAMP2D8bI0eOxI0bN+zzLFmyBCaTCd99912x5SdPngw/Pz+cPXvWPm3Xrl3o1KkTKlWqhKCgILRp0wY7duxwWO5u38Ekx60Wau3atZGbm4sDBw6gcePGJc734Ycf4vnnn0dcXByGDh0KAKhXr57DPL169UL9+vUxefJkeJIgN3jwYCxduhSdO3fG888/j1u3biEnJwdff/01mjdv7tI+uGrr1q1YvXo10tPTUa1aNfsX3cyZM9G1a1f0798fFosFK1euRK9evfDZZ5+hS5cuLq9/wIABmDhxIlatWoX09HT7dIvFYv+lX1oLyWq1olOnTnj00Ufx1ltvYePGjcjMzMStW7cwceJEh3mXLFmCgoICDB06FAEBAahSpYrrb8Z//fnnn+jUqRN69OiB3r17Y+3atRg9ejSaNGmCzp07A7j9Rda1a1d89dVXGDp0KBo2bIgffvgB77zzDn7++WeH69vPP/88li1bhn79+qFly5bYunWrS++hyWRCq1at8OWXX9qnff/997h06RLMZjN27NhhX09OTg6aNWuGkJAQAMCPP/6IVq1aISIiAq+88gqCg4OxevVqdO/eHevWrcNTTz1V4nbfffddpKenIzExESNHjsSJEyfQvXt3VK5cGffff3+x+d98802YzWaMGjUKly5dwltvvYX+/ftj165dAIDXXnsNly5dwpkzZ/DOO+8AgH0/Fy5ciOHDh6Nnz5546aWXUFBQgO+//x67du1Cv379nO5f69at8eGHH2LAgAHo0KEDUlNT7X/Lz89Hy5Ytcf36dQwfPhxVq1bF+++/j65du2Lt2rXFyj1p0iT4+/tj1KhRKCwsLLE3w/ZDZMCAASW+b0Vt2bIFnTt3Rt26dTFhwgTcuHEDs2fPRqtWrbB3795iP3p79+6NOnXqYMqUKdi7dy8WLVqEGjVqYOrUqQCAPn36YMKECcjLy0N4eLh9ua+++gq//vornnnmmbvuT15eHgCgWrVqLu2/rayvvvoqvvjiCwwZMsTl5QBgzZo1uH79Ov7+97+jatWq2L17N2bPno0zZ85gzZo1AICePXvixRdfxPLly9GsWTOH5ZcvX462bdsiIiICwO3vqc6dOyM2NhaZmZkwm81YsmQJHn/8ceTk5CAuLs5hednvYHJCuOGLL74Qfn5+ws/PTyQkJIiXX35ZbNq0SVgslmLzBgcHi4EDBxabnpmZKQCIvn37FvtbmzZtRJs2bYpNHzhwoKhdu7b9/1u3bhUAxPDhw4vNqyhKqftw5/ru3LeiAAiz2Sx+/PHHYvNfv37d4f8Wi0U0btxYPP744w7Ta9eu7bAf27ZtEwDEtm3b7NMSEhJEfHy8w3IfffRRsfmcGThwoAAghg0bZp+mKIro0qWL8Pf3F7///rsQQohffvlFABAVK1YU586dc1jHkiVLBADxyy+/OEx3tq9t2rQRAMQHH3xgn1ZYWCjCw8PF008/bZ/24YcfCrPZLHJychzWOX/+fAFA7NixQwghxL59+wQA8Y9//MNhvn79+gkAIjMz867lz8rKEn5+fuLy5ctCCCFmzZolateuLeLi4sTo0aOFEEJYrVYRGhoqRo4caV+uffv2okmTJqKgoMDhfWvZsqWoX79+ie9BYWGhqFq1qmjRooW4efOmfb6lS5cKAA7nsG3Zhg0bisLCQvv0mTNnCgDihx9+sE/r0qWL0/OyW7duolGjRnd9D0oCQLz44osO00aMGCEAOByXK1euiDp16oioqChhtVod9r1u3brFznVnnnrqKQFA/Pnnny7tW0xMjKhRo4a4cOGCfdr+/fuF2WwWqamp9mm2z+Vzzz1XbHtVq1a1///w4cMCgJg9e7bDfP/4xz9ESEjIXctw4cIFUaNGDZGYmOgw3fa5+Oabb0pctlKlSqJZs2Z3Lauzz5Gz/ZkyZYowmUzi5MmT9ml9+/YVtWrVsh8XIYTYu3evACCWLFkihLh93tavX18kJyc7fAdev35d1KlTR3To0ME+7W7fwSTHrS7fDh06IDc3F127dsX+/fvx1ltvITk5GREREcW6yUrzwgsvuDV/UevWrYPJZEJmZmaxv93ZZauGNm3a4OGHHy42veg1kz///BOXLl1CYmIi9u7d6/Y2UlNTsWvXLhw7dsw+bfny5YiMjESbNm1cWkfR1q2tq9pisWDLli0O8z399NOoXr262/tYVEhICJ599ln7//39/REXF+fQXbZmzRo0bNgQ0dHROH/+vP31+OOPAwC2bdsGANiwYQMAYPjw4Q7bcHVwTmJiIqxWK3bu3Angdks0MTERiYmJyMnJAQAcOHAAFy9eRGJiIgDgjz/+wNatW9G7d29cuXLFvm8XLlxAcnIyjhw54tCVVtS3336LCxcuYMiQIShX7q9Onv79+6Ny5cpOl0lLS3No2dn2w5XuxdDQUJw5cwbffPONC+9G6TZs2IC4uDiHbr6QkBAMHToUJ06cwMGDBx3mHzhwoEvXB21dwffdd1+p8/7222/Yt28fBg0a5NBD8sgjj6BDhw72c6KoO78zEhMTceHCBft2GzRogJiYGIfRularFWvXrkVKSkqJZVAUBf3798fFixeL3a3gipCQEI9G+xbdn2vXruH8+fNo2bIlhBAOXbypqan49ddf7Z8X4PZ3Q4UKFexd6/v27cORI0fQr18/XLhwwX4+X7t2De3bt8eXX34JRVEcti/zHUzOuX3bTIsWLfDRRx/hzz//xO7duzFmzBhcuXIFPXv2LPZBvJs6deq4u2m7Y8eOoVatWh51VXqipH397LPP8OijjyIwMBBVqlRB9erV8e677+LSpUtub6NPnz4ICAjA8uXLAQCXLl3CZ599hv79+7v0I8FsNqNu3boO0xo0aAAAxa6Lyrz3Nvfff3+x/apcuTL+/PNP+/+PHDmCH3/8EdWrV3d42fbLNpDt5MmTMJvNxbrkH3roIZf25W9/+xuCgoLslaetQm3dujW+/fZbFBQU2P9mq0SOHj0KIQTGjRtXbP9sP9RKGmh38uRJAMCDDz7oML1cuXJOr80DwAMPPODwf1vFW/T9Ksno0aMREhKCuLg41K9fHy+++GKx62LuOHnypNP3tmHDhva/F+Xq+VKxYkUAcKlysW2jpP2wVQZFufIe9unTBzt27LD/GNq+fTvOnTuHPn36lLgvw4YNw8aNG7Fo0SI0bdq01H2/09WrV136EXGnU6dO2X9QhISEoHr16vYfz0W/Qzp06ICaNWvavxsURcG///1vdOvWzb7dI0eOALj94+fO83nRokUoLCws9r2kxvcAOXLrGmpR/v7+aNGiBVq0aIEGDRogLS0Na9ascdpqdMbZr0XbIIo7Wa1WT3fTqZIqqJK242xfc3Jy0LVrV7Ru3Rrz5s1DzZo1Ub58eSxZsuSug0VKUrlyZTz55JNYvnw5xo8fj7Vr16KwsNChFaiWkt57Z0p6T/z8/JxOL3r8FEVBkyZNMH36dKfzRkZGlrarLilfvjzi4+Px5Zdf4ujRo8jLy0NiYiLCwsJw8+ZN7Nq1Czk5OYiOjra3zG2/1keNGoXk5GSn672zwpThyvtVkoYNG+Lw4cP47LPPsHHjRqxbtw7z5s3D+PHjnd7ioTZXR69GR0cDuH2vuq0FriZX3sM+ffpgzJgxWLNmDUaMGIHVq1ejUqVKJd5/+/rrr2PevHl48803Xb72W9SZM2dw6dIlt88Vq9WKDh064I8//sDo0aMRHR2N4OBgnD17FoMGDXJoTfr5+aFfv35YuHAh5s2bhx07duDXX391+G6wzZ+VlYWYmBin27Rdk7fx5VHJZZXHFWpRtuHkv/32m32aJ12vlStXdtoFducv5nr16mHTpk34448/7tpKLWkfKleu7DD6s6Tt3M26desQGBiITZs2ISAgwD59yZIlLq/jTqmpqejWrRu++eYb+yCERo0aubSsoig4fvy4vfUHwH4TfUmtpqJsv/bvfF/ceU/uVK9ePezfvx/t27e/6/lQu3ZtKIqCY8eOObRYDh8+7PK2EhMTMXXqVGzZsgXVqlVDdHQ0TCYTGjVqhJycHOTk5ODJJ5+0z29rzZcvXx5JSUlulcs2QvTo0aMOo0pv3bqFEydOeHxP4t3eo+DgYPTp0wd9+vSBxWJBjx498MYbb2DMmDFu39JTu3Ztp+/toUOH7H/3REpKCqZMmYJly5aVWqHatlHSflSrVs2j2zjq1KmDuLg4+wC/jz76CN27d3f4jNrMnTsXEyZMwIgRIzB69Gi3twXcHoAJoMQfZSX54Ycf8PPPP+P99993GDC2efNmp/Onpqbi7bffxqefforPP/8c1atXd9imrXenYsWKbp/PpB63uny3bdvm9Be17XpH0S/D4OBgp5XW3dSrVw+HDh1yuH1k//79xbq3nn76aQghnP46L7p/Je1DvXr1cOnSJXz//ff2ab/99hv+93//1+V99fPzg8lkcmjBnThxQurJTJ07d0a1atUwdepU/N///Z/brdM5c+bY/y2EwJw5c1C+fHm0b9++1GVtH8iio2WtVisWLFjg1j4U1bt3b5w9exYLFy4s9rcbN27Yu/Rso4JnzZrlMI87Tw1KTExEYWEhZsyYgccee8xeOSUmJuLDDz/Er7/+6vAlX6NGDbRt2xb/+te/HH4I2ji7hcmmefPmqFq1KhYuXIhbt27Zpy9fvtylLtySBAcHO71ccOHCBYf/+/v74+GHH4YQAjdv3nR7O0888QR2796N3Nxc+7Rr165hwYIFiIqKcjpewBUJCQno1KkTFi1a5PRzYLFY7A+YqFmzJmJiYvD+++87fEYPHDiAL774Ak888YRH+wDcbqV+/fXXWLx4Mc6fP++0u3fVqlUYPnw4+vfvX2IPSmm2bt2KSZMmoU6dOujfv79by9pa20W/r4QQmDlzptP5H3nkETzyyCNYtGiR/SEURa/fx8bGol69epg2bZrDbWE2dzufST1utVCHDRuG69ev46mnnkJ0dDQsFgt27tyJVatWISoqCmlpafZ5Y2NjsWXLFkyfPh21atVCnTp1EB8ff9f1P/fcc5g+fTqSk5MxePBgnDt3DvPnz0ejRo0c7n1r164dBgwYgFmzZuHIkSPo1KkTFEVBTk4O2rVrZx+cU9I+PPPMMxg9ejSeeuopDB8+HNevX8e7776LBg0auDygqEuXLpg+fTo6deqEfv364dy5c5g7dy4efPBBh4raHeXLl8czzzyDOXPmwM/PD3379nV52cDAQGzcuBEDBw5EfHw8Pv/8c6xfvx6vvvqqSwOQGjVqhEcffRRjxoyxt/xXrlzpUGG4a8CAAVi9ejVeeOEFbNu2Da1atYLVasWhQ4ewevVqbNq0Cc2bN0dMTAz69u2LefPm4dKlS2jZsiWys7Md7tMsTUJCAsqVK4fDhw/bb5MCbt8+8u677wJAsVbT3Llz8dhjj6FJkyYYMmQI6tati/z8fOTm5uLMmTPYv3+/0235+/tjwoQJGDZsGB5//HH07t0bJ06cwNKlS1GvXj2PB8bFxsZi1apVyMjIQIsWLRASEoKUlBR07NgR4eHhaNWqFcLCwvDTTz9hzpw56NKli0fX7l555RX8+9//RufOnTF8+HBUqVIF77//Pn755ResW7euxIc2uOKDDz5Ax44d0aNHD6SkpKB9+/YIDg7GkSNHsHLlSvz222/2e1GzsrLQuXNnJCQkYPDgwfbbZipVqiT1/ObevXtj1KhRGDVqFKpUqVKsxbZ7926kpqaiatWqaN++vf3apE3Lli2LjUf4/PPPcejQIdy6dQv5+fnYunUrNm/ejNq1a+OTTz5xu5cgOjoa9erVw6hRo3D27FlUrFgR69atu+sPstTUVPsPkjt/bJvNZixatAidO3dGo0aNkJaWhoiICJw9exbbtm1DxYoV8emnn7q1j+QBd4YEf/755+K5554T0dHRIiQkRPj7+4sHH3xQDBs2TOTn5zvMe+jQIdG6dWtRoUIFAcB+24htyLbtVo47LVu2TNStW1f4+/uLmJgYsWnTJqe3udy6dUtkZWWJ6Oho4e/vL6pXry46d+4s9uzZU+o+CHH7FqDGjRsLf39/8dBDD4lly5aVeNvMnbcd2Lz33nuifv36IiAgQERHR4slS5Y4XYcrt83Y7N69WwAQHTt2dLpNZwYOHCiCg4PFsWPHRMeOHUVQUJAICwsTmZmZDkPtbbfNZGVlOV3PsWPHRFJSkggICBBhYWHi1VdfFZs3b3Z624yz2zicHSeLxSKmTp0qGjVqJAICAkTlypVFbGyseP3118WlS5fs8924cUMMHz5cVK1aVQQHB4uUlBRx+vRpl26bsWnRooUAIHbt2mWfdubMGQFAREZGlljm1NRUER4eLsqXLy8iIiLEk08+KdauXWufp6TjZbs9JyAgQMTFxYkdO3aI2NhY0alTp2LLrlmzxmFZ27Gw3fYghBBXr14V/fr1E6GhoQKA/b3817/+JVq3bi2qVq0qAgICRL169cQ///lPh/evJCWdv8eOHRM9e/YUoaGhIjAwUMTFxYnPPvvMYZ6S9r00169fF9OmTRMtWrSwf0/Ur19fDBs2TBw9etRh3i1btohWrVqJChUqiIoVK4qUlBRx8OBBh3lK+s4o6VYvIYRo1aqVACCef/75Yn+zLVfSq+gxuXNef39/ER4eLjp06CBmzpxpv1WrNM7OoYMHD4qkpCQREhIiqlWrJoYMGSL2799fbB9sfvvtN+Hn5ycaNGhQ4na+++470aNHD/u5Urt2bdG7d2+RnZ1tn6e072DynEkI3tFbluzfvx8xMTH44IMPXB4kMWjQIKxdu9ZpVw95j6IoqF69Onr06OG0m5tIxvnz51GzZk2MHz8e48aN03t3yAnGt5UxCxcuREhICHr06KH3rtBdFBQUFBtP8MEHH+CPP/5w+uhBIllLly6F1Wr1aDQyeYcqo3xJ3qeffoqDBw9iwYIFSE9P54Oqy7ivv/4aI0eORK9evVC1alXs3bsX7733Hho3boxevXrpvXt0D9m6dSsOHjyIN954A927d3dp1D7pgxVqGTFs2DDk5+fjiSee8Mq9hSQnKioKkZGRmDVrln0QV2pqKt58800m95CqJk6ciJ07d6JVq1YePcmJvIfXUImI6J7y5ZdfIisrC3v27LHfEllaLvb27duRkZGBH3/8EZGRkRg7dqw9ocxVvIZKRET3lGvXrqFp06aYO3euS/P/8ssv6NKlC9q1a4d9+/ZhxIgReP7557Fp0ya3tssWKhER3bNMJlOpLdTRo0dj/fr1OHDggH3aM888g4sXL9rzr13Ba6hERKSJgoICWCwWVdYlhCj20JSAgACnj5V0V25ubrEHgCQnJ7uceGXjcYXawazfSMZNv+7z7gZNFWAOu/30o5SQZ1FwvdCj1Xh9v4tIrhXj9jKBQQH49OoyAHLllqHnsVbyHwHEDY9W48n7XZRMuWWPtZ7lluHRe2bQz7bNZmWNejtyh4KCAtSpHYK8c+qEm4SEhBS71z4zM1PqqVo2eXl5CAsLc5gWFhaGy5cv48aNGy4HCbCFSkREqrNYLMg7Z8Uve2qj4n1yw3UuX1FQJ/YkTp8+bY8JBKBK61RNrFCJiEgzFe8zS1eo9nVVrOhQoaolPDwc+fn5DtPy8/NRsWJFt2LuWKESEZFmrEKBVXLoq1Uopc8kISEhwZ6aZrN582YkJCS4tR7eNkNERJpRIFR5uePq1avYt28f9u3bB+D2bTH79u3DqVOnAABjxoxxyKF94YUXcPz4cbz88ss4dOgQ5s2bh9WrV2PkyJFubZcVKhER3VO+/fZbNGvWDM2aNQMAZGRkoFmzZhg/fjyA2/nXtsoVuB1Mv379emzevBlNmzbF22+/jUWLFrkdHM8uXyIi0owCBbIdtu6uoW3btsXCK4paunSp02W+++47d3fNAStUIiLSjFUIWCWfHyS7vLewy5eIiEgFbKESEZFmPBlU5GwdvoAVKhERaUaBgJUVKhERkRwjtVB5DZWIiEgFHrdQA4N0fIaiyfVHQanjr+0FyJTb6/v9F0+OV9GySpVbho7HGqgAmEqc8a6kPx8S5ZY91rqWW4ZH75kxP9sAvBZ2YaRRvsxDJSIyoA7mXpqmzVy+fBmVKlXCoZ/CcJ/ks3yvXFEQ3TAfly5d0uRZvmphly8REZEKPO7yVfIf8Xij3ep5vqys/xz73oOlKsActgsA0LPGYBR62FXi2bZvk33PZMut5McD8Cwj07eOt/7HWpYn73dAUADWnnsPgFy5Zejznul/jsuWW8/PlyusKozylV3eWzwf5ethADHgvb57pzzZ7yLXkwqvF3q+/3q+Z5LlBm54vP8+dbzLwLGWJft+S5Vbhh7vWVk4xyXLrevnywVWARXSZtTZF62xy5eIiEgFvA+ViIg0o/z3JbsOX8AKlYiINKPABKun92EVWYcvYJcvERGRCthCJSIizSji9kt2Hb6AFSoREWnGqkKXr+zy3sIKlYiINGOkCpXXUImIiFTAFioREWlGESYoQnKUr+Ty3sIKlYiINMMuXyIiInKL5y1UL+c1qsZHMxP1ydcsI9mgMtwut/7HWhazb91RBs5xyXKX+TxUmGGVbLtZVdoXrTEPlYjIgJS8+jCHH9Fs/bY81OwfHkCwZB7qtSsK2jc5xTxUIiIiI/C4yzcl5FmPN6pn/p+v5qHKYkamq9Q51rKMWG6981CNVW7vMdKgJI8rVJ/N//PRPFRZzMh0kVrHWpYRy61zHqqhyu1FVmGGVUheQ/WRC5Ps8iUiIlIB70MlIiLNKDBBkWy7KfCNJiorVCIi0gyvoRIREalAnWuovtFC5TVUIiIiFbCFSkREmrl9DVXy4fjs8iUiIqNTVHj0oK8MSmKXLxERkQrYQiUiIs0YaVASK1QiItKMAjPvQy2NL8YVeb5tRnoZJ9JLpWMty4jl1jm+zVjlxj3/yEM9ML6NiMiAvBXf9uF3TRB0n5/Uuq5fsWJAsx/KfHwbu3yJiEgz6gSM+0a7j/FtLmF8m5IfD8D7XUQyx9oTZSGyDvDd+DajnuPePk+Lutfj33wJ49tcwfg2ADd02X/d4tPA+DaPGPQc1/M8LevXQhVhhiI5ylfxkSuT7PIlIiLNsMuXiIhIBQoAq5B99KBv4JOSiIiIVMAWKhERaUadBzv4RtuPFSoREWlGnUcP+kaF6ht7SUREVMaxhUpERJphHioREZEK2OVLREREbmELlYiINKPOgx18o+3HCpWIiDSjCBMU2Qc7SC7vLcxDdQnzUIEK0GNcgNSx9kCZyIAFfDcP1aDnuLfPUweevudl/BnAvoh5qEREBuStPNQ3v2mDwBC5ztCCq7fwSov/Yx4qEREZlzppM/f4NVQlX7/8Pz0zMo2SCwqUjWxQo+aC+mrmrwx9cj3/KjfzULVhhQlWyetFsst7i+ctVJ/OPZRhvFxQQMdsUIPmgvps5q8MPb5THL6nmYdKctjlS0REmmGXLxERkQqskO+ytaqzK5rzjWqfiIiojGMLlYiINMMuXyIiIhXw4fhEREQ+bO7cuYiKikJgYCDi4+Oxe/fuu84/Y8YMPPTQQ6hQoQIiIyMxcuRIFBQUuLVNVqhERKQZ8d88VJmXcHNQ06pVq5CRkYHMzEzs3bsXTZs2RXJyMs6dO+d0/hUrVuCVV15BZmYmfvrpJ7z33ntYtWoVXn31Vbe2ywqViIg0Y+vylX25Y/r06RgyZAjS0tLw8MMPY/78+QgKCsLixYudzr9z5060atUK/fr1Q1RUFDp27Ii+ffuW2qq9EytUIiLyCZcvX3Z4FRYWf6CGxWLBnj17kJSUZJ9mNpuRlJSE3Nxcp+tt2bIl9uzZY69Ajx8/jg0bNuCJJ55wa/84KImIiDSjZnxbZGSkw/TMzExMmDDBYdr58+dhtVoRFhbmMD0sLAyHDh1yuv5+/frh/PnzeOyxxyCEwK1bt/DCCy+43eXreYXqYzFNMowYYwaUkSgzg8aY+WxEoQxdvlMqOP6b8W2qUzNg/PTp0w5pMwEB6rzv27dvx+TJkzFv3jzEx8fj6NGjeOmllzBp0iSMGzfO5fUwvo2IyIA6mHths7JGs/Xb4tuGf9UNASHlpdZVePUmZj32H5fi2ywWC4KCgrB27Vp0797dPn3gwIG4ePEi/vOf/xRbJjExEY8++iiysrLs05YtW4ahQ4fi6tWrMJtd+0HAa6hERHTP8Pf3R2xsLLKzs+3TFEVBdnY2EhISnC5z/fr1YpWmn58fAMCdNqfHXb4pIc96uqiucUOexCwxvs045VYrso7nuPv0PseNdLy9SYEZimTbzd3lMzIyMHDgQDRv3hxxcXGYMWMGrl27hrS0NABAamoqIiIiMGXKFABASkoKpk+fjmbNmtm7fMeNG4eUlBR7xeoKjytUr8dLqUQ+ZsmY8W1GLLee8W0yeI57xrjHW1tWYYJVclCSu8v36dMHv//+O8aPH4+8vDzExMRg48aN9oFKp06dcmiRjh07FiaTCWPHjsXZs2dRvXp1pKSk4I033nBruxzlS0RE95z09HSkp6c7/dv27dsd/l+uXDlkZmYiMzNTapusUImISDNq3jZT1rFCJSIizQgV0mYEH45PRERkHGyhEhGRZqwwwSr5NBzZ5b2FFSoREWlGEfLXQBUfefwQu3yJiIhUwBYqERFpRlFhUJLs8t7CCpWIiDRjCwmXXYcvYIVKRESa0eNJSXrxuEL1fryUOjzZb8a3AUYpt2qRdTzH3ab3OW6k4w2U/UcW+iLGtxERGZCSVx/m8COard8W3/ZM9rPwD/GXWpflqgUr2y9zKb5NT+zyJSIizShQ4dGD9/o1VCXf88ggPeOGPItZqgBz2C4ActFWvlxuo0RbqRVjJnusZd4zPePbeI67T89zhdTleQtVIq5I1757T/bb4ceR59FWvlxuY0Zb6Xisdf188Rz3yrb/S89zxRuECqN8xb3eQiUiIiqNkdJmfONuWSIiojKOLVQiItIMn5RERESkAnb5EhERkVvYQiUiIs3wWb5EREQqMFKXLytUIiLSjJEqVF5DJSIiUgFbqEREpBkjtVBZoRIRkWZYobpCIv9Pj9xDO4/2u4Ljvz08tr5cbqNkRaqVCyp9rL38+Soz5Zbho+e4budKGX8GsC9iHioRkQF1MPfCZmWNZuu35aEmbfh/KBcs96Ph1rVCbHniX8xDJSIi42KXrwv0zEP11axIGXrkTBYtt3GyIvXPx5Qle47LlFuGPu+Z/lnHsuXWM4OWHPlmHqqPZkXK0DVnEgbKiiwD+ZiyZN8zqXLL0OM9KwtZx5Ll1vu7oTRsoRIREanASBUqH+xARESkArZQiYhIM0ZqobJCJSIizQhhgpCsEGWX9xZWqEREpBkjxbfxGioREZEK2EIlIiLN8BoqERGRCox0DZVdvkRERCpgC5WIiDTDLl9X+GJckYfbVivaSoYesVhFy22caCv947xkyZ7jUuWWoct7VgaiGSXL7em2vfXIQiN1+TK+jYjIgLwV3xa7bqQq8W17nn6H8W1ERGRcQoUuX19poeoS3ybL23FFZSHGTJaesXW+FG1VFmLMAD3OFcbWGSma0ZsEANl+UF/pRtUlvk2WnnFFRo30MmK0lW4xZoD3zxXG1sGo0YykHnb5EhGRZhSYYDLIowdZoRIRkWaMNMqXFSoREWlGESaYDHIfKp+UREREpAK2UImISDNCqDDK10eG+bJCJSIizRjpGiq7fImIiFTAFioREWnGSC1UVqhERKQZjvIlIiIit7CFSkREmuEoX1f4WN6jjLKQCypLzxxYX8qKLBO5oIAO5wpzYI2UdQx4Mw9V/hqor1SozEMlIjIgJa8+zOFHNFu/LQ+1/rJX4BcUKLUu6/UCHHn2TeahEhGRcXGUrwtk8lBl8/9kMhdlMxONmhWpV7m9f678lQuqVwYs4LvnuAx9Ph/658Dqea54g4B8nqmvdKPqkocq3Xev47aNmhWpV7m9fq44/BDWKQMW8N1zXIYen48ykAOr57niDUZqofK2GSIiIhXwGioREWnHQH2+bKESEZF2/tvlK/OCB12+c+fORVRUFAIDAxEfH4/du3ffdf6LFy/ixRdfRM2aNREQEIAGDRpgw4YNbm2TLVQiIrqnrFq1ChkZGZg/fz7i4+MxY8YMJCcn4/Dhw6hRo0ax+S0WCzp06IAaNWpg7dq1iIiIwMmTJxEaGurWdlmhEhGRZvR4UtL06dMxZMgQpKWlAQDmz5+P9evXY/HixXjllVeKzb948WL88ccf2LlzJ8qXLw8AiIqKcns/2eVLRESake3uLTpK+PLlyw6vwsLiI6QtFgv27NmDpKQk+zSz2YykpCTk5uY63cdPPvkECQkJePHFFxEWFobGjRtj8uTJsFqtbpWVFSoREfmEyMhIVKpUyf6aMmVKsXnOnz8Pq9WKsLAwh+lhYWHIy8tzut7jx49j7dq1sFqt2LBhA8aNG4e3334b//M//+PW/rHLl4iItOPhoKJi6wBw+vRph0cPBgSo8xxkRVFQo0YNLFiwAH5+foiNjcXZs2eRlZWFzMxMl9fDCpWIiDSj5jXUihUrlvos32rVqsHPzw/5+fkO0/Pz8xEeHu50mZo1a6J8+fLw8/OzT2vYsCHy8vJgsVjg7+/v0n6yy5eIiLQjVHq5yN/fH7GxscjOzrZPUxQF2dnZSEhIcLpMq1atcPToUSiKYp/2888/o2bNmi5XpoBO8W3ScUVe3jbj2/Qrt/fPlQqO/9Yjsg7w3XNchi6fD/1j63Q7V8r4IwtlZGRkYODAgWjevDni4uIwY8YMXLt2zT7qNzU1FREREfZrsH//+98xZ84cvPTSSxg2bBiOHDmCyZMnY/jw4W5tl/FtREQG1MHcC5uVNZqt3xbf9sCC8TBLxrcp1wtwauhEt+Lb5syZg6ysLOTl5SEmJgazZs1CfHw8AKBt27aIiorC0qVL7fPn5uZi5MiR2LdvHyIiIjB48GCMHj3aoRu4NKxQiYgMyKsVagXJCvWG+xWqHjzu8k0JedbjjcrGDcnEHXm2bXUinmT4crSVDO+XW534NlmykV7uKhrfpldsnR7fC0YtN2nD4wpVKnJIsu/e69tWK+JJhi9HW8nwdrlVim+Tpct7badTbJ2e3wsAjFtubRkpvo23zRARkXaYNkNERETuYAuViIg0ZILH9585rKPsY4VKRETaYZcvERERuYMtVCIi0o6BWqisUImISDsqps2UdaxQiYhIM2qmzZR1vIZKRESkArZQiYhIO7yGWjqpyCHJmCbvb1uliCcZvhxtJcPr5VYnvk2WdKSXmxyPr06xdTp8Lxi13IAXH1looGuoTJshIjIgb6XN3D9roippM2eGj79302aIiIhKYxK3X7Lr8AWMb3OJOjFmMuWWjWjy1XLLcvd9KxrnpWeZffUcl6F3RCHj2zTCa6il89m4Ij3j2yTKLX29w0fLLUvmfdOzzD57jsvQOaKQ8W0ki12+RESkHQMNSmKFSkRE2jFQly8f7EBERKQCtlCJiEg7BmqhskIlIiLtsEIlIiJSgYEGJfEaKhERkQrYQiUiIs3wSUlERERqMNA1VHb5EhERqYAVKhERkQqYh+oSlXJBJcotnY/po+WW5e77VrScepbZZ89xGTpn/jIPVRsmqHANVZU90R7zUImIDMhbeai1p/4PzIGSeagFBTg5eizzUImIyMAMdB+qxxWqku95Bp8+2Z6eb7toRqZMZqIMPTIPy0I2qPfPFXXyMWV5+3irdY7rmc0pmwPLPFSNGGiUr+ctVF/L9lRr2xKZiTL0zjw0TA6sSvmYsvQ93jrlgsqSzIFlHirJYpcvERFphy1UIiIieXxSEhERkRoM1ELlgx2IiIhUwBYqERFpx0AtVFaoRESkGSNdQ2WXLxERkQrYQiUiIu3wSUlEREQqMNA1VHb5EhERqcDzFqrPRZF5vm3HOCvPI55kSL9nHigLUWbeP1fUifOS5e3jrdY5rsd5aicZW8f4Nm0YaVAS49uIiAzIW/FtdcdPViW+7fjEV8t8fBu7fImIiFTgk/FtMmQjnowTYwaoVW4ZstFW7tP/WMuSjSg0zrEGjHq8vUqFLl9fGZTkm/FtMiQjngwTYwaoV24Z3o5PKwPHWpbscTLMsQZ4vL3BQKN8edsMERFpx0AVKq+hEhERqYAtVCIi0oyRbpthC5WIiEgFrFCJiIhUwC5fIiLSjoEGJbFCJSIizfAaKhEREbmFLVQiItKWj7QwZbFCJSIi7RjoGiq7fImIiFTgm3moMiQzE42TCwqoVm4ZklmR7tP/WMuSzfw1zrEGjHq8AeahaoF5qEREBqTk1Yc5/Ihm67flodb/52T4BcjloVoLC3Akq+znofIaKhERacZILVSfzEP1dq5o0axIJT8egGdRTcyBdZ/3z5W/yixzrGV5+1xRKw/Vt441YMxznLTim3moumax3vB4+8yBdZ/XzxVT0f94fqxl6XmuGOZYA8Y8x71Np1G+c+fORVZWFvLy8tC0aVPMnj0bcXFxpS63cuVK9O3bF926dcPHH3/s1jY5ypeIiLQjVHq5YdWqVcjIyEBmZib27t2Lpk2bIjk5GefOnbvrcidOnMCoUaOQmJjo3gb/ixUqERHdU6ZPn44hQ4YgLS0NDz/8MObPn4+goCAsXry4xGWsViv69++P119/HXXr1vVou6xQiYhIM7ZBSbIv4PbI4aKvwsLi3eUWiwV79uxBUlKSfZrZbEZSUhJyc3NL3M+JEyeiRo0aGDx4sMdlZYVKRETaUbHLNzIyEpUqVbK/pkyZUmxz58+fh9VqRVhYmMP0sLAw5OXlOd3Fr776Cu+99x4WLlwoVVTeNkNERD7h9OnTDvehBgTIP4TkypUrGDBgABYuXIhq1apJrYsVKhERaUfFUb4VK1Ys9cEO1apVg5+fH/Lz8x2m5+fnIzw8vNj8x44dw4kTJ5CSkmKfpigKAKBcuXI4fPgw6tWr59JussuXiIg0o+Y1VFf4+/sjNjYW2dnZ9mmKoiA7OxsJCQnF5o+OjsYPP/yAffv22V9du3ZFu3btsG/fPkRGRrq8bbZQiYjonpKRkYGBAweiefPmiIuLw4wZM3Dt2jWkpaUBAFJTUxEREYEpU6YgMDAQjRs3dlg+NDQUAIpNLw0rVCIi0o4OD3bo06cPfv/9d4wfPx55eXmIiYnBxo0b7QOVTp06BbNZ/Q5aVqhERKQZvZ7lm56ejvT0dKd/2759+12XXbp0qfsbhK/Gt3l5246xThXueDydtttWDWPrXFTB8d8eHmtZ3j5XVItv86ljDRjzHP8vbz2y0EAB44xvIyIyIG/FtzV8UZ34tp/mMr6NiIiMzEAtVI8r1JSQZ9XcD7d4P65I/0gvPaLfjBnppX+clyxGFLpDnc+275Xbe0yQv3Ki05UXt3lcofpcFJmMMhDppev7DQNFW5WBOC9ZjCh0g0qfbZ8rN2mCXb5ERKQddvkSERHJ0+u2GT3w0YNEREQqYAuViIi0wy5fIiIilfhIhSiLXb5EREQqYAuViIg0Y6RBSaxQiYhIO7yGSkREJM9ILVReQyUiIlKBxy1U34sik6F/pJce77cxI730j/OSxYhCd6jz2fa9coPxbRpgfBsRkQF5K77tkecmw89fMr7NUoDvF5f9+DZ2+RIREamA8W0uMWrEkxHLXTbK7O3Iu7IQ3yb7ufbVcssq6/FtRuryZXybK4wa8WTEcpeVMusZeadXuSU/1z5bblllPb7NQBUqu3yJiIhUwPtQiYhIM0a6D5UVKhERaYddvkREROQOtlCJiEgzJiFgknzcgezy3sIKlYiItGOgLl9WqEREpBkjDUriNVQiIiIVsIVKRETaYZcvERGRPHb5EhERkVuYh+oSo2YmGrHcZaTMXs6QLRN5qJKfa58ttyzmoZYZzEMlIjIgb+WhxvZ5Q5U81D2rXmMeKhERkREwD9UlZSMjU4av5qF6OyOzaD5mzxqDUehhLJeeGZWyuaAy5Zahz3v21zmu1/HWMzvXKwzU5cs8VFeUlYxMGb6ah6pjRmbh9ULPl9cxo1L2PJMqtww93rMi57hex1vP7Fxv8ZVRurLY5UtERKQC3odKRETaEeL2S3YdPoAVKhERacZID3ZghUpERNox0KAkXkMlIiJSAVuoRESkGZNy+yW7Dl/ACpWIiLTDLl8iIiJyB1uoRESkGY7yJSIiUgPvQy2dT8YVeayMRHrJ8NX4Ni9HehWN8wrQcb9lyMaYSZVbhi7v2V/b1Ot46xb15wOPLPQ1jG8jIjKgDuZe2Kys0Wz9tvi2+JRJKFdeLr7t1s0C7Pp0XJmPb2OXLxERacdAo3x1iW/z5WgrmRgzGXpEv6lVbiPGt8ny1YhCGb58jsvQM9aR1KVPfJsPR1vJxJjJ0DX6DQDj27zMRyMKZfjyOS5D/3Jri6N8iYiI1MBRvkRERPKM1ELlk5KIiIhUwBYqERFph6N8iYiI5LHLl4iIiNzCFioREWlHEbdfsuvwAaxQiYhIOwa6hsouXyIiIhWwhUpERJoxQYVBSarsifZYoRIRkXb4pKTS6ZlxKUM2K1ImF1SGHlmqapXbkHmosnw081eGL5/jMvTKSb7XnyGsB+ahEhEZkJJXH+bwI5qt35aH+tjjE1CunGQe6q0CfLV1glt5qHPnzkVWVhby8vLQtGlTzJ49G3FxcU7nXbhwIT744AMcOHAAABAbG4vJkyeXOH9JOCiJiIi0I1R6uWHVqlXIyMhAZmYm9u7di6ZNmyI5ORnnzp1zOv/27dvRt29fbNu2Dbm5uYiMjETHjh1x9uxZt7brcQtVyfc8w082/08mK1LPPFQ9cw89e8/+ysiUyQb1pfxb5qEaKxfUuJ9tAOKGV1qoiW0zVWmh5mx/3eUWanx8PFq0aIE5c+YAABRFQWRkJIYNG4ZXXnml1OWtVisqV66MOXPmIDU11eX99HxQkkRuoHTfvZ7b1isXVJYn+1zkepJUNqiP5t8yD9W79L+mZ6DPto+6fPmyw/8DAgIQEOB4DdpisWDPnj0YM2aMfZrZbEZSUhJyc3Nd2s7169dx8+ZNVKlSxa39Y5cvERFpR1HpBSAyMhKVKlWyv6ZMmVJsc+fPn4fVakVYWJjD9LCwMOTl5bm0y6NHj0atWrWQlJTkVlF52wwREWnGJARMkmNfbcufPn3aocv3ztapGt58802sXLkS27dvR2Cge13VrFCJiMgnVKxYsdRrqNWqVYOfnx/y8/Mdpufn5yM8PPyuy06bNg1vvvkmtmzZgkcecf+6OLt8iYhIO14e5evv74/Y2FhkZ2fbpymKguzsbCQkJJS43FtvvYVJkyZh48aNaN68uRsF/AtbqEREpB0dnpSUkZGBgQMHonnz5oiLi8OMGTNw7do1pKWlAQBSU1MRERFhvwY7depUjB8/HitWrEBUVJT9WmtISAhCQkJc3i4rVCIiuqf06dMHv//+O8aPH4+8vDzExMRg48aN9oFKp06dgtn8Vwftu+++C4vFgp49ezqsJzMzExMmTHB5u6xQiYhIMyahwsPxPVg+PT0d6enpTv+2fft2h/+fOHHC/Q04wQqViIi0Y6CH43NQEhERkQrYQiUiIs2YlNsv2XX4As8rVIl4Kem4Ii9vu0zEmMny6D37axmpKDMfiutjfNt//22QGDPjfrbhvUcWGqjLl/FtREQG5K34trYtXlPl4fjbv3nDrfg2PfAaKhERkQo87vLVM75NhmyMmbEinhjfZsT4Nr2OtT6xjkb9bHuPms/yLet8M75NhmSMmaEinhjfZsj4Nr2OtS6xjkb9bHuTga6hssuXiIhIBbxthoiItCNgzzOVWocPYIVKRESaMdI1VHb5EhERqYAtVCIi0o6ACoOSVNkTzbFCJSIi7XCULxEREbmDLVQiItKOAvlnQ9/zD8cnIiIqhZFG+bJCJSIi7RjoGqpvxrfJkIwxM1bEE+PbdKNjfJtex1qfWEejfrZR9h9Z6IMY30ZEZEDeim9r//AolPOT+8Fxy1qI7IPTynx8G7t8iYhIO+zyLV1KyLMeb1Q2bkgmKokRT+7Qv9zePlfUim/z5XNcr9g6fWLIjFpu0oLHFapUXJFk373Xt23UiKeyUG4dzxU9I+v0PMd1i63T45qeUcvtTbxthoiISJ6Rbpvhk5KIiIhUwBYqERFph4OSiIiIVKAIwCRZISq+UaGyy5eIiEgFbKESEZF22OVLRESkBhUqVB9JGGeFSkRE2jFQC5XXUImIiFTAFioREWlHEZDusvWRUb6sUImISDtCuf2SXYcP8LhClcr/k8x69P62jZqZWAbK7eVzRbU8VB8+x3XLgdUlN9eo5ca9/wxhHTAPlYjIgLyVh5oU+XeUM0vmoSqF2HL6XeahEhGRgfEaaulk8lBleT8/UJ3MRD1zDz3J1yyaDWrEPFQ9s29lys1j7bqyUG5ZzFMtO/TJQ5Xl7b5/tTITdbxmIX+8jJeHqmv2rUS5eaw9ZaCsY28y0H2o7PIlIiLtCKhQoaqyJ5rjgx2IiIhUwBYqERFph12+REREKlAUAJIPZlDu8Qc7EBERlcpALVReQyUiIlIBW6hERKQdA7VQWaESEZF2DPSkJHb5EhERqYAtVCIi0owQCoRk/Jrs8t6iT3ybLK/HHakU8aRXTBM8O16OZTVefJuuUX0S5eaxdl2ZKLcsT983bz2yUAj5LlsfuYbK+DYiIgPyVnxb+9BUlDP5S63rlrAg++IHZT6+jddQiYgMKLlWjHc2ZBvlK/vyAR53+Sr5nscV+V7UEePbZKKtZHj7XDFunNdf57hRjjXgeLyN9Nn2KkUBTJLXQO/1a6j6xktJ8GS/Gd8GmWgrGbqeK0aK83K4dmjEY23kzzaphaN8iYhIO0KF+1Dv9S5fIiKi0ghFgZDs8r3nb5shIiIqlYFaqBzlS0REpAK2UImISDuKAEzGaKGyQiUiIu0IAemAcR+pUNnlS0REpAK2UImISDNCERCSXb6+8oRctlCJiEg7QlHn5aa5c+ciKioKgYGBiI+Px+7du+86/5o1axAdHY3AwEA0adIEGzZscHubrFCJiOiesmrVKmRkZCAzMxN79+5F06ZNkZycjHPnzjmdf+fOnejbty8GDx6M7777Dt27d0f37t1x4MABt7bLCpWIiDQjFKHKyx3Tp0/HkCFDkJaWhocffhjz589HUFAQFi9e7HT+mTNnolOnTvjnP/+Jhg0bYtKkSfjb3/6GOXPmuLVdVqhERKQdL3f5WiwW7NmzB0lJSfZpZrMZSUlJyM3NdbpMbm6uw/wAkJycXOL8JfF4UJJMjt5m33iKlFOfXl2m9y54RPY9N4fpk6ah57kiU2ZfPseNeKwB4362tXYLN6UflHQLNwHczlgtKiAgAAEBjuHu58+fh9VqRVhYmMP0sLAwHDp0yOn68/LynM6fl5fn1n5ylC8REanO398f4eHh+CrP/cE9zoSEhCAyMtJhWmZmJiZMmKDK+tXACpWIiFQXGBiIX375BRaLRZX1CSFgMjnkDBZrnQJAtWrV4Ofnh/z8fIfp+fn5CA8Pd7ru8PBwt+YvCStUIiLSRGBgIAIDA726TX9/f8TGxiI7Oxvdu3cHACiKguzsbKSnpztdJiEhAdnZ2RgxYoR92ubNm5GQkODWtlmhEhHRPSUjIwMDBw5E8+bNERcXhxkzZuDatWtIS0sDAKSmpiIiIgJTpkwBALz00kto06YN3n77bXTp0gUrV67Et99+iwULFri1XVaoRER0T+nTpw9+//13jB8/Hnl5eYiJicHGjRvtA49OnToFs/mvm1xatmyJFStWYOzYsXj11VdRv359fPzxx2jcuLFb2zUJX3mmExERURnG+1CJiIhUwAqViIhIBaxQiYiIVMAKlYiISAWsUImIiFTACpWIiEgFrFCJiIhUwAqViIhIBaxQiYiIVMAKlYiISAWsUImIiFTACpWIiEgF/x898tebGO26LwAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "weights_to_display = tf.reshape(tensor_data, [tf.reduce_prod(tensor_data.shape[:-1]), -1])\n", "weights_to_display = weights_to_display[0:width, 0:height]\n", "\n", "val_ones = np.ones([height, width])\n", "val_zeros = np.zeros([height, width])\n", "subset_values_to_display = np.where(abs(weights_to_display) > 1e-9, val_ones, val_zeros)\n", "\n", "plot_separation_lines(height, width)\n", "\n", "plt.axis('off')\n", "plt.imshow(subset_values_to_display)\n", "plt.colorbar()\n", "plt.title(\"Structurally pruned weights for Conv2D layer\")\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "id": "1aX2O8w0Wr_M" }, "source": [ "Let's see how those randomly pruned weights look. We extract them and display a subset of the weight tensor." ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "execution": { "iopub.execute_input": "2025-06-21T11:30:33.200954Z", "iopub.status.busy": "2025-06-21T11:30:33.200698Z", "iopub.status.idle": "2025-06-21T11:30:33.205252Z", "shell.execute_reply": "2025-06-21T11:30:33.204534Z" }, "id": "eEHu5nizWr_M" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Shape of the weight tensor is (32, 5, 5, 1)\n" ] } ], "source": [ "# Get weights of the convolutional layer that has been pruned with random pruning.\n", "tensor_name = 'pruning_sparsity_0_5/Conv2D'\n", "detail = [x for x in details if tensor_name in x[\"name\"]]\n", "tensor_data = interpreter.tensor(detail[0][\"index\"])()\n", "print(f\"Shape of the weight tensor is {tensor_data.shape}\")" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "execution": { "iopub.execute_input": "2025-06-21T11:30:33.207690Z", "iopub.status.busy": "2025-06-21T11:30:33.207458Z", "iopub.status.idle": "2025-06-21T11:30:33.335957Z", "shell.execute_reply": "2025-06-21T11:30:33.335218Z" }, "id": "Cimzp3kVWr_M" }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAdEAAAGkCAYAAABuLB/pAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjkuNCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8ekN5oAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA/5UlEQVR4nO3de1hU1cI/8O8MchERouTihSNmdtQk9WBeShRfSdLCY2mpZSiaWYqaHLucfiaapaXpodRSK/Wklqi93U5mKWaZ0g0vecmOqXkNxMy8g8ys3x++MzEywMxeSxab+X6eZ55HN3vPXvsys2atvfb+WoQQAkREROQ1q+4CEBERmRUrUSIiIoNYiRIRERnESpSIiMggVqJEREQGsRIlIiIyiJUoERGRQaxEiYiIDGIlSkREZBAr0Wpiw4YNsFgs2LBhg+6iaBMbG4shQ4ZoLcMvv/wCi8WCxYsXG172pZdeUl8wD5w9exYPPfQQoqOjYbFY8Nhjj2kph6+TOYfIfLyqRCdNmgSLxYITJ064/XurVq2QmJioolxuTZ06Fe+///5Ve3+zlIHMb/Xq1Zg0aZLS95w6dSoWL16MRx99FEuWLMGDDz6o9P3dsdlsWLRoERITE3HttdciMDAQsbGxSEtLw/fff3/V11+Z8+fPY+7cuejRowfq16+PunXrom3btnjttddgs9lc5nX8kHW8AgMDERUVhcTEREydOhWFhYWatoKqM1O1RKtDBVYdykBXT+PGjXHhwoWrXgGtXr0akydPVvqe69evR8eOHZGZmYlBgwYhPj5e6ftf6cKFC7jrrrswdOhQCCHw9NNP47XXXkNqaipyc3PRvn17HDly5KqWoTL79+/H6NGjIYRARkYGXnrpJTRp0gQjR47E0KFD3S4zZswYLFmyBAsWLMDjjz+Oa6+9FpmZmWjRogXWr19fxVtA1V0t3QW4Ws6dO4c6deroLka1YrfbUVxcjKCgIN1FqbYsFotp98/x48fRsmVLZe9XUlICu92OgIAAt39//PHHsWbNGvzrX/8q03WcmZmJf/3rX8rKYlR0dDR27NiBm266yTltxIgRGDp0KBYtWoRnnnkGN9xwg8syCQkJ6Nevn8u07du3o0ePHujbty92796N+vXrV0n5rwZ+N6p1VVuiju6RFStW4Pnnn0ejRo0QFBSE7t274+eff3aZd+/evejbty+io6MRFBSERo0aYcCAAfjjjz8AXP5yO3fuHP797387u1sc188c3cy7d+/G/fffj/DwcHTu3BkAkJiY6LaLeciQIYiNjXWZZrfb8fLLLyMuLg5BQUGIiIjAHXfc4eyWqqgMAHD06FEMHToUUVFRCAwMxE033YSFCxeWWfeRI0fQp08f1KlTB5GRkRg3bhyKioo82qeObd2zZw/uu+8+hIaG4rrrrsPYsWNx8eJFl3ktFgvS09OxbNky3HTTTQgMDMSaNWvKvf7q7lrOkCFDEBISgqNHj6JPnz4ICQlBREQExo8fX6Y7zG63IysrCzfddBOCgoIQFRWFESNG4Pfff3eZTwiB5557Do0aNUJwcDC6deuGXbt2ebT9f/vb33DPPfe4TIuLi4PFYsEPP/zgnJadnQ2LxYIff/zROc2T41Pe9ayVK1eiZcuWCAoKQqtWrfDee++5PYccFixYgKZNmyIwMBC33HILvvvuO+ffhgwZgrlz5wKAS/ehw/LlyxEfH4+6desiNDQUcXFxePnll8vdJ47jeeDAAXz88cfO9/vll18AXK5chw0bhqioKAQFBaF169b497//7Xa7X3rpJWRlZTnLvnv3brfrPHLkCObPn4/bb7/d7bVXPz8/jB8/Ho0aNXJO27p1K3r27InQ0FCEhISge/fu+Prrr12WW7x4MSwWCzZt2oSMjAxERESgTp06uPvuu126U++66y5cf/31bsvWqVMntGvXDgBQr149lwrU4e677wYAl/OjIq1bt0ZWVhZOnTqFOXPmeLRMaT/88AOGDBmC66+/HkFBQYiOjsbQoUPx22+/Oef5/PPPYbFY8N5775VZ/u2334bFYkFubq5z2p49e9CvXz9ce+21CAoKQrt27fDhhx+6LOfYn1988QVGjhyJyMhIl2NC8qqkJfrCCy/AarVi/Pjx+OOPPzB9+nQ88MAD+OabbwAAxcXFSE5ORlFREUaPHo3o6GgcPXoU//nPf3Dq1CmEhYVhyZIleOihh9C+fXs8/PDDAICmTZu6rOfee+9Fs2bNMHXqVBhJeBs2bBgWL16Mnj174qGHHkJJSQk2btyIr7/+Gu3atauwDAUFBejYsaOz4oqIiMAnn3yCYcOG4fTp084vmgsXLqB79+44dOgQxowZgwYNGmDJkiVedxPdd999iI2NxbRp0/D111/jlVdewe+//4633nrLZb7169djxYoVSE9PR7169RAbG4tTp055tS6bzYbk5GR06NABL730EtatW4eZM2eiadOmePTRR53zjRgxAosXL0ZaWhrGjBmDAwcOYM6cOdi6dSs2bdoEf39/AMDEiRPx3HPPoVevXujVqxe2bNmCHj16oLi4uNKyJCQk4J133nH+/+TJk9i1axesVis2btyIm2++GQCwceNGREREoEWLFgA8Pz7ufPzxx+jfvz/i4uIwbdo0/P777xg2bBgaNmzodv63334bZ86cwYgRI2CxWDB9+nTcc8892L9/P/z9/TFixAgcO3YMa9euxZIlS1yWXbt2LQYOHIju3bvjxRdfBHD5i37Tpk0YO3as2/W1aNECS5Yswbhx49CoUSP84x//AABERETgwoULSExMxM8//4z09HQ0adIEK1euxJAhQ3Dq1Kky77lo0SJcvHgRDz/8MAIDA3Httde6Xecnn3yCkpISj7u9d+3ahYSEBISGhuKJJ56Av78/5s+fj8TERHzxxRfo0KGDy/yjR49GeHg4MjMz8csvvyArKwvp6enIzs4GAPTv3x+pqan47rvvcMsttziXO3jwIL7++mvMmDGjwvLk5+cDuFzJeqpfv34YNmwYPvvsMzz//PMeLwdcPq779+9HWloaoqOjsWvXLixYsAC7du3C119/DYvFgsTERMTExGDZsmXOSt5h2bJlaNq0KTp16gTg8v687bbb0LBhQzz11FOoU6cOVqxYgT59+uDdd98ts/zIkSMRERGBiRMn4ty5c16VnSohvJCZmSkAiMLCQrd/v+mmm0TXrl2d///8888FANGiRQtRVFTknP7yyy8LAGLHjh1CCCG2bt0qAIiVK1dWuP46deqIwYMHl1uugQMHlvlb165dXcrkMHjwYNG4cWPn/9evXy8AiDFjxpSZ1263V1qGYcOGifr164sTJ064TB8wYIAICwsT58+fF0IIkZWVJQCIFStWOOc5d+6cuOGGGwQA8fnnn5d5b3fb2rt3b5fpI0eOFADE9u3bndMACKvVKnbt2uUyr+O4XLmuAwcOCABi0aJFzmmDBw8WAMSzzz7rMm/btm1FfHy88/8bN24UAMSyZctc5luzZo3L9OPHj4uAgABx5513uuzXp59+WgBwu29LW7lypQAgdu/eLYQQ4sMPPxSBgYGid+/eon///s75br75ZnH33Xc7/+/p8XG3D+Li4kSjRo3EmTNnnNM2bNggALicQ45lr7vuOnHy5Enn9A8++EAAEB999JFz2qhRo4S7j9/YsWNFaGioKCkpqXA/uNO4cWNx5513ukxznG9Lly51TisuLhadOnUSISEh4vTp0y5lDw0NFcePH690XePGjRMAxNatWz0qW58+fURAQIDYt2+fc9qxY8dE3bp1RZcuXZzTFi1aJACIpKQkl/Nj3Lhxws/PT5w6dUoIIcQff/whAgMDxT/+8Q+X9UyfPl1YLBZx8ODBcstSVFQkWrZsKZo0aSIuXbrknO74XFT0PdS6dWsRHh5e4ba6O4cc51dp77zzjgAgvvzyS+e0f/7znyIwMNC5nUJc/szUqlVLZGZmOqd1795dxMXFiYsXLzqn2e12ceutt4pmzZo5pzn2Z+fOnQ2dU1S5KhlYlJaW5nJdJSEhAcDli/4AEBYWBgD49NNPcf78ecPreeSRRwwv++6778JisSAzM7PM30p3tbkjhMC7776LlJQUCCFw4sQJ5ys5ORl//PEHtmzZAuDygJL69eu7XHMJDg52tmw9NWrUKJf/jx492vn+pXXt2lXJdbIr921CQoLz+AGXuzvDwsJw++23u2x/fHw8QkJC8PnnnwMA1q1bh+LiYowePdplv3p6O4bj3Pnyyy8BXG5x3nLLLbj99tuxceNGAMCpU6ewc+dO57zeHJ8rHTt2DDt27EBqaipCQkKc07t27Yq4uDi3y/Tv3x/h4eFlylx6f5Xnmmuuwblz57B27dpK5/XE6tWrER0djYEDBzqn+fv7Y8yYMTh79iy++OILl/n79u2LiIiISt/39OnTAIC6detWOq/NZsNnn32GPn36uHTB1q9fH/fffz+++uor5/s5PPzwwy7nR0JCAmw2Gw4ePAgACA0NRc+ePbFixQqXXqfs7Gx07NgRf/nLX8otT3p6Onbv3o05c+agVi3vOuNCQkJw5swZr5YBgNq1azv/ffHiRZw4cQIdO3YEAJdzLzU1FUVFRVi1apVzWnZ2NkpKSjBo0CAAl3tf1q9fj/vuuw9nzpxxnsu//fYbkpOTsXfvXhw9etRl/cOHD4efn5/X5abKKa9E3VU4V57Qji8Yx7WyJk2aICMjA2+88Qbq1auH5ORkzJ0713k91FNNmjQxWGpg3759aNCgQbndVxUpLCzEqVOnsGDBAkRERLi80tLSAFy+LgVc7m664YYbyuynv/71r16ts1mzZi7/b9q0KaxWq/M6mIPMPnFwXB8uLTw83OVa5969e/HHH38gMjKyzD44e/asy/a7K39ERIRLxVOeqKgoNGvWzFlhbty4EQkJCejSpQuOHTuG/fv3Y9OmTbDb7c7Ky5vjcyVHea8cfFLeNKDy870iI0eOxI033oiePXuiUaNGGDp0KNasWVPpcuU5ePAgmjVrBqvV9aPu6OZ2bJ+Dp+dLaGgoAHhUoRQWFuL8+fNuz/EWLVrAbrfj8OHDLtM92Yf9+/fH4cOHndcJ9+3bh7y8PPTv37/cssyYMQOvv/46pkyZgl69elVa9iudPXvWox8OVzp58iTGjh2LqKgo1K5dGxEREc59Xfp7rnnz5rjllluwbNky57Rly5ahY8eOzvPt559/hhACzzzzTJnz2dEIuPJ8VvE9QO559TPMMWrxwoULbv9+/vx5tyMby/sFVPoX5MyZMzFkyBB88MEH+OyzzzBmzBjn9T5PL4SX/rXnYLFY3F4fvXJQjAy73Q4AGDRoEAYPHux2Hse1uqulvNZyefvEnfL2iSe/YO12OyIjI10+/KV50rrxVOfOnZGTk4MLFy4gLy8PEydORKtWrXDNNddg48aN+PHHHxESEoK2bds6ywZU3fHx5HwvT2RkJLZt24ZPP/0Un3zyCT755BMsWrQIqampZQYDXQ3uzhd3mjdvDgDYsWMH2rRpo7wcnuzDlJQUBAcHY8WKFbj11luxYsUKWK1W3HvvvW6XXbx4MZ588kk88sgjmDBhgtdlunTpEv773/+iVatWXi973333YfPmzXj88cfRpk0bhISEwG6344477nCenw6pqakYO3Ysjhw5gqKiInz99dcug5kc848fPx7Jyclu13flDzxPjyt5z6tKtHHjxgCAn376CTExMS5/O3/+PA4fPowePXoYLkxcXBzi4uIwYcIEbN68GbfddhvmzZuH5557DkDl3aruhIeHu+1Gu/IXeNOmTfHpp5/i5MmTFbZG3ZUhIiICdevWhc1mQ1JSUoXlady4MXbu3AkhhMt7/fTTT5Vtiou9e/e6/Lr8+eefYbfbyx0tWprjV/2VA4yu3CfeaNq0KdatW4fbbrutwg+s4xzau3evS9deYWGhRy014HLX3qJFi7B8+XLYbDbceuutsFqt6Ny5s7MSvfXWW51fxN4cn/LKe+Vo8vKmeaqiczkgIAApKSlISUmB3W7HyJEjMX/+fLe3Y1SmcePG+OGHH2C3211ao3v27HH+3YiePXvCz88PS5curXRwUUREBIKDg92e43v27IHVai3zfeKJOnXq4K677sLKlSsxa9YsZGdnIyEhAQ0aNCgz7wcffICHHnoI99xzj3NktLdWrVqFCxculFtxlef3339HTk4OJk+ejIkTJzqn79271+38AwYMQEZGBt555x1cuHAB/v7+Lq1rx+fG39/f6/OZ1POqO7d79+4ICAjAa6+9VubX04IFC1BSUoKePXt6XYjTp0+jpKTEZVpcXBysVqvLrR916tTxemRp06ZNsWfPHpfh8du3b8emTZtc5uvbty+EEG5vgC/969ddGfz8/NC3b1+8++672LlzZ5nlS6+7V69eOHbsmMs1j/Pnz2PBggVebdeVXwSzZ88GAI/2f+PGjeHn5+e8rujw6quvelWG0u677z7YbDZMmTKlzN9KSkqc+ywpKQn+/v6YPXu2y37NysryeF2ObtoXX3wRN998s/OaekJCAnJycvD999875wG8Oz5XatCgAVq1aoW33noLZ8+edU7/4osvsGPHDo/LfCXHfXpXnkulb3kAAKvV6mwle3obVGm9evVCfn6+c1QrcPl4zJ49GyEhIejatavX7wkAMTExGD58OD777DPnuVea3W7HzJkzceTIEfj5+aFHjx744IMPXC43FBQU4O2330bnzp2d3cPe6t+/P44dO4Y33ngD27dvd9uV++WXX2LAgAHo0qULli1bVqZr2xPbt2/HY489hvDw8DLjESrj+DF3ZU9Eeed8vXr10LNnTyxduhTLli3DHXfc4TKKODIyEomJiZg/fz5+/fXXMsvzyUpVy6uWaGRkJCZOnIgJEyagS5cu6N27N4KDg7F582a888476NGjB1JSUrwuxPr165Geno57770XN954I0pKSrBkyRLnl59DfHw81q1bh1mzZqFBgwZo0qRJmaHxVxo6dChmzZqF5ORkDBs2DMePH8e8efNw0003uQxm6NatGx588EG88sor2Lt3r7ObZePGjejWrRvS09MrLMMLL7yAzz//HB06dMDw4cPRsmVLnDx5Elu2bMG6detw8uRJAJcv8M+ZMwepqanIy8tD/fr1sWTJEgQHB3u1zw4cOIDevXvjjjvuQG5uLpYuXYr7778frVu3rnTZsLAw3HvvvZg9ezYsFguaNm2K//znP+VeF/RE165dMWLECEybNg3btm1Djx494O/vj71792LlypV4+eWX0a9fP+c9ptOmTcNdd92FXr16YevWrfjkk088vt3ghhtuQHR0NH766SfngCoA6NKlC5588kkAcKlEAXh8fNyZOnUq/v73v+O2225DWloafv/9d8yZMwetWrVyqVi94Xia0JgxY5CcnAw/Pz8MGDAADz30EE6ePIn/+Z//QaNGjXDw4EHMnj0bbdq0cV7H9MbDDz+M+fPnY8iQIcjLy0NsbCxWrVqFTZs2ISsry9D1PYeZM2di3759GDNmDP73f/8Xd911F8LDw3Ho0CGsXLkSe/bswYABAwAAzz33HNauXYvOnTtj5MiRqFWrFubPn4+ioiJMnz7dcBl69eqFunXrYvz48WW+L4DLvSu9e/eGxWJBv379sHLlSpe/33zzzWW68jdu3IiLFy/CZrPht99+w6ZNm/Dhhx8iLCwM7733HqKjo70qY2hoKLp06YLp06fj0qVLaNiwIT777DMcOHCg3GVSU1Odgw/d/TCdO3cuOnfujLi4OAwfPhzXX389CgoKkJubiyNHjmD79u1elZEkGBnSu3TpUtGxY0dRp04dERgYKJo3by4mT57sMtxaiPKHjF85BHz//v1i6NChomnTpiIoKEhce+21olu3bmLdunUuy+3Zs0d06dJF1K5d2+V2iMpuvVm6dKm4/vrrRUBAgGjTpo349NNPy9ziIoQQJSUlYsaMGaJ58+YiICBAREREiJ49e4q8vLxKyyCEEAUFBWLUqFEiJiZG+Pv7i+joaNG9e3exYMECl/UcPHhQ9O7dWwQHB4t69eqJsWPHOm8F8fQWl927d4t+/fqJunXrivDwcJGeni4uXLjgMi8AMWrUKLfvU1hYKPr27SuCg4NFeHi4GDFihNi5c6fbW1zq1KlTbjmutGDBAhEfHy9q164t6tatK+Li4sQTTzwhjh075pzHZrOJyZMni/r164vatWuLxMREsXPnTtG4ceNKb3FxuPfeewUAkZ2d7ZxWXFwsgoODRUBAQJl9IYRnx8fd7QlCCLF8+XLRvHlzERgYKFq1aiU+/PBD0bdvX9G8efMyy86YMaPMugG43KJQUlIiRo8eLSIiIoTFYnHuy1WrVokePXqIyMhIERAQIP7yl7+IESNGiF9//bXSfeLuFhfHdqelpYl69eqJgIAAERcXV2b7Kip7RUpKSsQbb7whEhISRFhYmPD39xeNGzcWaWlpZW5/2bJli0hOThYhISEiODhYdOvWTWzevNllHsctGd99953L9PJuyxJCiAceeMB5W8yVHMuV9yp9TK6c19/fX0RERIguXbqI559/3qNbf4Rwfw4dOXJE3H333eKaa64RYWFh4t577xXHjh0rUwaHoqIiER4eLsLCwtyey0IIsW/fPpGamiqio6OFv7+/aNiwobjrrrvEqlWrnPOUtz9JHYsQBp5KQNpMmjQJkydPRmFhoVc3ipN6bdq0QUREhLLbUYgcSkpK0KBBA6SkpODNN9/UXRyqgKkeQE+kw6VLl8pcs9+wYQO2b99+VVOLyHe9//77KCwsRGpqqu6iUCVq7APoiVQ5evQokpKSMGjQIDRo0AB79uzBvHnzEB0dLfWAD6IrffPNN/jhhx8wZcoUtG3b1vDAL6o6rESJKhEeHo74+Hi88cYbKCwsRJ06dXDnnXfihRdewHXXXae7eFSDvPbaa1i6dCnatGnDUG+T4DVRIiIyvS+//BIzZsxAXl4efv31V7z33nvo06dPhcts2LABGRkZ2LVrF2JiYjBhwgSXZC5P8JooERGZ3rlz59C6dWuPH6Zx4MAB3HnnnejWrRu2bduGxx57DA899BA+/fRTr9bLligREdUojlzWilqiTz75JD7++GOXB7AMGDAAp06d8up51bwmSkRESl28eNGjfODKiCsejwoAgYGBCAwMlH7v3NzcMo9NTE5O9jhRysGrSvR2q/sHO3vi02PbDC8LAMkN2lT9ui21YY36AQCQEjIIF897/9g1swkKDsRHZ5cCkNtm2eMtw8i5Unq77QU3A8J9yEKNw3Ncyzbr/D5ca19Z+UwSLl68iCaNQ5B/XD7kIyQkpMxTwTIzMzFp0iTp987Pz0dUVJTLtKioKJw+fRoXLlzw+KH9bIkSEZEyxcXFyD9uw4G8xgita3zYzekzdjSJP4jDhw+7PFtZRStUJVaiRESkXGhdq1Ql6nyf0FDDAQUViY6ORkFBgcu0goIChIaGehUdx0qUiIiUswk7bBLDVm3CXvlMEjp16oTVq1e7TFu7di06derk1fvwFhciIlLODiH98sbZs2exbds2bNu2DcDlW1i2bduGQ4cOAQD++c9/ujxG8ZFHHsH+/fvxxBNPYM+ePXj11VexYsUKjBs3zqv1shIlIiLT+/7779G2bVu0bdsWAJCRkYG2bds6g9B//fVXZ4UKAE2aNMHHH3+MtWvXonXr1pg5cybeeOMNr0PX2Z1LRETK2WGHTIest0snJiaWCT4vzd1jFBMTE7F161Zvi+aClSgRESlnEwI2iWf5yCxbldidS0REZBBbokREpJyRwUFXLm8GrESJiEg5OwRsrESJiIi85ystUV4TJSIiMsirlmhQsMQzCy2eP0ap+qz7z+UCZdZvIqW3U2qbJY+3DCPniuu21gYs5c5aw/Ac10Ln92EV8ZXRucwTJSIiZU6fPo2wsDDs+TEKdSWenXvmjB3NWxTgjz/+uCrPzlWF3blEREQGedWdmxIy6GqVo1If7PvB8LJ/b3qzoeUCgwOx6vibAIB+kcNQVMW5gzLbbFxtWKO+AaBnm3UpfaztBR0AeJ8navQ8U8H4ueJ7x1vFsdZN5lxzZKlebTbJ0bkyy1YlrypRrYG9EiHJKspddL6o6rdfRzB0qWuBWra5WrhgaN+b8vPh88fb2LHWzQzHySYgmeKirixXE7tziYiIDOJ9okREpJz9/14yy5sBK1EiIlLODgtsEveK2U1ynxm7c4mIiAxiS5SIiJSzi8svmeXNgJUoEREpZ5PszpVZtiqxEiUiIuV8pRLlNVEiIiKD2BIlIiLl7MICu5AYnSuxbFViJUpERMqxO5eIiIgqVHV5orIk8veMllt77qCWTE7fy5cE1OSJmvPz4XvHuyZkx5oiTxRW2CTaaTaFZbmamCdKRETKOPJEc3b8BXUk8kTPnbGje9wh5okSERHVVFWWJ6onG1OW3qxF2X1mJHNQVdaiTN6hWbdbNk9Ud36sWbM1vad/m3WeK9aoqjnPfGVgUdXliZows0971qLkPpMvr/GsRZ3niq7tll6v5vxYs2Zreq0abLMpzxUv2YQVNiFxTdQkFxrZnUtERGQQ7xMlIiLl7LDALtFOs8McTVFWokREpByviRIRERkkf03UHC1RXhMlIiIyiC1RIiJS7vI1UYkH0LM7l4iIfJVd8rF/ZhlYxO5cIiIig9gSJSIi5XxlYBErUSIiUs4OK+8TvZJU/I6WWC9ZmmOiJPeZkeOlKiZK57mia7ul46k0R9+ZNRbMe/q32ZznCrnDKDQiIlLGEYW2ZGscguv6GX6f82dseLDtjmofhcbuXCIiUk4+lNsc7TuvKlF7gVx8jy5GY4dKx2PpiEKTZSwuSW/8G+C7UWgyjO8z+VgwnbF3xuiPQtOpqqLQfIV3LVETxO+4oyLCTEsUmiwjx0t3/Bvgu1FoMozuMwWxYKaLSKwGUWi+wC6ssEuMzrWb5Eoju3OJiEg5ducSEREZZAdgEzKP/TMHPrGIiIjIILZEiYhIOfmHLZijjcdKlIiIlJN/7J85KlFzlJKIiKgaYkuUiIiUY54oERGRQezOJSIiogqxJUpERMrJP2zBHG08VqJERKScXVhgl3nYgsSyVcm7StSkGXZGs/tKZ0xqyROVZeh4ac5QBXw3T1SG4X0mn61pvpxh/XmiVHMwT5SIiJRx5Im+8F1XBIUY7+y8eLYET93yBfNEiYjI98inuNTAa6IpIYMMr0hHRqQsFRmTOunK1TS6bp2qQ56ozGdEZ2aunkzQy8x6juvcZ1WVJ2qDBTaJvnKZZauSV5WoztxA/Vme5ssd1JWrqWbdOmnKE5U4v7Rm5mr8XJj2HDfZdwmVj925RESkHLtziYiIDLJBrkvWpq4oV5U5qnoiIqJqiC1RIiJSjt25REREBvEB9ERERCYzd+5cxMbGIigoCB06dMC3335b4fxZWVn461//itq1ayMmJgbjxo3DxYsXPV4fK1EiIlJO/F+eqNGXMDAoKTs7GxkZGcjMzMSWLVvQunVrJCcn4/jx427nf/vtt/HUU08hMzMTP/74I958801kZ2fj6aef9nidrESJiEg5R3euzMtbs2bNwvDhw5GWloaWLVti3rx5CA4OxsKFC93Ov3nzZtx22224//77ERsbix49emDgwIGVtl5LYyVKRETV1unTp11eRUXuH3JRXFyMvLw8JCUlOadZrVYkJSUhNzfX7TK33nor8vLynJXm/v37sXr1avTq1cvj8nFgERERKacqCi0mJsZlemZmJiZNmlRm/hMnTsBmsyEqKsplelRUFPbs2eN2Hffffz9OnDiBzp07QwiBkpISPPLII15153pVieqMPNIRM6UiHksnXZFgRtetU7WIQpP4jGiN+9MYkWjac9yksZLeUBXKffjwYZcUl8BAdd8tGzZswNSpU/Hqq6+iQ4cO+PnnnzF27FhMmTIFzzzzjEfvwSg0IiJSxhGFNuarvyMwxN/w+xSdvYRXOn/gcRRacXExgoODsWrVKvTp08c5ffDgwTh16hQ++OCDMsskJCSgY8eOmDFjhnPa0qVL8fDDD+Ps2bOwWiv/EcBrokREZHoBAQGIj49HTk6Oc5rdbkdOTg46derkdpnz58+XqSj9/PwAAJ62L73qzrUXmCveSl5tWKO+AWA8JkonY3FLarbZzPFYuiLBdMf96dhu3dvsS3F/Dh+dXVol67HDCrtEO83IshkZGRg8eDDatWuH9u3bIysrC+fOnUNaWhoAIDU1FQ0bNsS0adMAACkpKZg1axbatm3r7M595plnkJKS4qxMK+PdwCJfi+8pda3EcEyUTkaOl6ptNnE8lq5IMN3nl47t1r3Nvhv3d/XZhAU2iYFFRpbt378/CgsLMXHiROTn56NNmzZYs2aNc7DRoUOHXFqeEyZMgMViwYQJE3D06FFEREQgJSUFzz//vMfr5OhcIiKqMdLT05Genu72bxs2bHD5f61atZCZmYnMzEzD62MlSkREyqm6xaW6YyVKRETKCckUF8EH0BMREdVsbIkSEZFyNlhgk3hCjcyyVYmVKBERKWcXctc17SZ5DBC7c4mIiAxiS5SIiJSzSw4sklm2KrESJSIi5Rzh2jLLmwErUSIiUk7HE4t08K4S9YH4Hld/bq/hmCidDB0vRdts4ngsXZFguuP+dGy37m32pbg/ujoYhUZERMo4otAG5AxCQEiA4fcpPluM5d2XehyFpgu7c4mISDk7JB/7VxOvieqMQjNzZJK5MApN17GWOceN729zx/0ZYfZjLauqotB8hWmi0PTHDhmPTDIVRqFB17GWKrfR8po97k+aCY+1SQjJ0bmiJrZEiYiIPOErKS7muJuViIioGmJLlIiIlOMTi4iIiAxidy4RERFViC1RIiJSjs/OJSIiMshXunNZiRIRkXK+UonymigREZFBbIkSEZFyvtISZSVKRETKsRJ1x2QZkbJU5Q6aC/NEdR1rqXPc8P42eWauAaY/1lStME+UiIiUceSJJq0egVp1jP9YKDlXhHW95jNPlIiIfA+7c91ICRl0tcpRLenOHZTNHDSWMak/T9TM262LijxRM2bmymbHmvFYA3KfL2uUvqzfmsirStQXMvDKV/W5g9L720h5q0GeqKm3WxcFeaJmzMyVPU6mPNaAKY4TW6JEREQG+UolyoctEBERGcSWKBERKecrLVFWokREpJwQFgiJilBm2arESpSIiJTzlSg0XhMlIiIyiC1RIiJSjtdEiYiIDPKVa6LsziUiIjKILVEiIlKO3blu+Fp8j+7IJOn9bSgeS38Umqm3WxcFUWhmjPuTjb0z5bEGtEYNespXunMZhUZERMo4otDi3x0nHYWW1/dfjEIjIiLfIyS7c83SEq2yKDSZ6B5ZRqO1VEQmmW27VcW/ycSZye4zXfFYOo+1ceaOgDOiOpzjsswQhSYAyPRzmqWLtOqi0DRG96iIOjIcmWTq7TYejaXzXNEWj2WCeKoyzB4BJ03TOS7LjOdaDcXuXCIiUs4OCyw+8Ng/VqJERKScr4zOZSVKRETK2YUFFh+4T5RPLCIiIjKILVEiIlJOCMnRuSYZnstKlIiIlPOVa6LsziUiIjKILVEiIlLOV1qirESJiEg5js4lIiKiCrElSkREynF0rhtSOY8a8++MlltJ7qDJtltVhqrOc0VbxqQJMh7LMnmOqgHV4hyXZYJz7XIlKnNNVGFhriLmiRIRkTKOPNFmS5+CX3CQ4fexnb+IvYNeYJ4oERH5Ho7OdcNeoC8/T4bOPFGzUbXNMnmHsjmNxtb9Z66m0YxJPeWWW7fuHFWzHmtZOs+VKs0TlVzeDLxriZo0w05rnqiJSW2zxLkivZ+NrNvlR6+xjEkt5Va1bujJUTXrsZal81ypKr7SEuUtLkRERAbxmigREannI/25bIkSEZF6/9eda/QFg925c+fORWxsLIKCgtChQwd8++23Fc5/6tQpjBo1CvXr10dgYCBuvPFGrF692uP1sSVKREQ1QnZ2NjIyMjBv3jx06NABWVlZSE5Oxk8//YTIyMgy8xcXF+P2229HZGQkVq1ahYYNG+LgwYO45pprPF4nK1EiIlJOxxOLZs2aheHDhyMtLQ0AMG/ePHz88cdYuHAhnnrqqTLzL1y4ECdPnsTmzZvh7+8PAIiNjfVqnezOJSIi5WS6ckuP7D19+rTLq6jI/cjm4uJi5OXlISkpyTnNarUiKSkJubm5bpf58MMP0alTJ4waNQpRUVFo1aoVpk6dCpvN5vF2shIlIqJqKyYmBmFhYc7XtGnT3M534sQJ2Gw2REVFuUyPiopCfn6+22X279+PVatWwWazYfXq1XjmmWcwc+ZMPPfccx6Xj925RESknsTgIOfyAA4fPuzy2L/AQHXPLLbb7YiMjMSCBQvg5+eH+Ph4HD16FDNmzEBmZqZH78FKlIiIlFN1TTQ0NNSjZ+fWq1cPfn5+KCgocJleUFCA6Ohot8vUr18f/v7+8PPzc05r0aIF8vPzUVxcjICAgErXy+5cIiJSTyh4eSEgIADx8fHIyclxTrPb7cjJyUGnTp3cLnPbbbfh559/ht1ud07773//i/r163tUgQLetkRNEL/jjtYoNJNRts0S54p0xJShddd2/beBXig95ZZbt+4IOLMea1k6z5WaLCMjA4MHD0a7du3Qvn17ZGVl4dy5c87RuqmpqWjYsKHzuuqjjz6KOXPmYOzYsRg9ejT27t2LqVOnYsyYMR6vk1FoRESkjCMK7S8LJsIqEYVmP38Rhx5+1usotDlz5mDGjBnIz89HmzZt8Morr6BDhw4AgMTERMTGxmLx4sXO+XNzczFu3Dhs27YNDRs2xLBhw/Dkk0+6dPFWhJUoEREp41KJ1paoRC8Yq0SrmlfduSkhgwyvSCa6R58/I5N0xETJMhK3VB2i0PSQP9ayfDEKTQ81x1pnBJyMj84u1bbumsirSlQqvscE0T1llLpWoiMmSpZs3JKuKDQtVBxrWT4YhaaFqmOt+XhVd74ShcZbXIiISD2muBAREVFF2BIlIqKrwAK5+4fYnUtERL6K3blERERUEbZEiYhIPR9pibISJSIi9RSluFR3rESJiEg5VSku1R2viRIRERnEligREanHa6JlScX3mDK6588y64iJkmXkeFWHKDQ9FBxrWT4YhaaHomOtMwLODHzkmihTXIiISBlHikujV56VTnE5MmZizUpxISIi8oRFXH7JLG8GXlWi9gLj8T2y0T+6Y6LsBR0AVG1ahZ59pj8STJau7TZfJBhQervNeI4boftzrYLMfquyKDReE3VDZ/SP9tihC1Ue+aRln1WHSDBZurbbbJFgwBWPJzXhOS6t6rdZBf37jRzYnUtEROr5yMAiVqJERKSej3Tn8mELREREBrElSkRE6vlIS5SVKBERqcdKlIiIyCAfGVjEa6JEREQGsSVKRETK8YlFRERERvnINVF25xIRERnESpSIiMgg77pzdebnac5aBGpf8ZzRq0/PPqsGuZqydG236XI1gdLbbcpz3ADdn2sVzJBHaoHkNVFlJbm6mCdKRETKOPJEG7/4HKxBEnmiFy/i4JMTmCdKREQ+yEfuE62yPFFZunMHzZqt6S1VWYsyx0s2l9PIulVst45z1MH4PjN/fqy3asI5LsMaVUXr9pHRuVWWJypLd36eabM1pRjPWpTaV5LnmfxxMrbdWs8Po/usJuTHSjHnOU7VB7tziYhIPbZEiYiIjOETi4iIiIzykZYoH7ZARERkEFuiRESkno+0RFmJEhGRcr5yTZTduURERAaxJUpEROrxiUVEREQG+cg1UXbnEhERGVRlUWiydEcmmTYWzEuqYqKkjpfkeWZk3Sq2W2s8leF9VgOi77xUE85xM/CVgUWMQiMiImUcUWjXT5wqHYW2/9mnq30UGrtziYiIDPKqOzclZNDVKke1pCoyyVz+jMbStc2ykWLGYqb0b7cMo/tMRdyfTKyX7mMtE/+mc7tlfHR2adWsSLI71ywDi7yqRH0vJqk045FJpuJyfUjPNkufZ0bKXA22W4aKz6bhKDSJfaX7WEvFv+ncbjPwkdG5vMWFiIjU85FKlNdEiYiIDGJLlIiIlPOVW1zYEiUiIjKIlSgREZFB7M4lIiL1fGRgEStRIiJSjtdEiYiIqEJsiRIR0dVhktakDFaiRESkno9cE2V3LhERkUFetUS15iVqoCp30Fxqu/5bwzZLn2eGshr1b7cMo/tMSWauRDam7mMtlaGqc7tNwFcGFjFPlIiIlHHkiTZ7fCr8Ao3nidqKLmLvjOqfJ8prokREpJyvtES9qkTtBcYz8HTm5xnP/ZPPmJTZbpm8QqPrrg4ZqjrOFRW5mrJ05FNWh+2uaqq2WfbzqYs1ypzlrq68a4maNT/PaLkVZExKbbdkpqX8PjdpnqgkqYxJGZo/X9q2WyNdeaI+QdPo3Llz52LGjBnIz89H69atMXv2bLRv377S5ZYvX46BAwfi73//O95//32P18fRuUREpJ5Q8PJSdnY2MjIykJmZiS1btqB169ZITk7G8ePHK1zul19+wfjx45GQkOD1OlmJEhFRjTBr1iwMHz4caWlpaNmyJebNm4fg4GAsXLiw3GVsNhseeOABTJ48Gddff73X62QlSkREyjkGFsm8gMujfUu/iorcd78XFxcjLy8PSUlJzmlWqxVJSUnIzc0tt5zPPvssIiMjMWzYMEPbyUqUiIjUU9SdGxMTg7CwMOdr2rRpbld34sQJ2Gw2REVFuUyPiopCfn6+22W++uorvPnmm3j99dcNbyZvcSEiomrr8OHDLveJBgaqeVDFmTNn8OCDD+L1119HvXr1DL8PK1EiIlJP0ejc0NBQjx62UK9ePfj5+aGgoMBlekFBAaKjo8vMv2/fPvzyyy9ISUlxTrPb7QCAWrVq4aeffkLTpk0rXS+7c4mISDlV10Q9FRAQgPj4eOTk5Din2e125OTkoFOnTmXmb968OXbs2IFt27Y5X71790a3bt2wbds2xMTEeLRetkSJiKhGyMjIwODBg9GuXTu0b98eWVlZOHfuHNLS0gAAqampaNiwIaZNm4agoCC0atXKZflrrrkGAMpMrwgrUSIiUk/Dwxb69++PwsJCTJw4Efn5+WjTpg3WrFnjHGx06NAhWK1qO2BZiRIRkXK6np2bnp6O9PR0t3/bsGFDhcsuXrzY6/V5V4maNfrHcLnl47Gktltifxtdd3WIf9NxriiJBJOl4fNVLba7iinbZsnPZ43nI6HcjEIjIiJlHFFoLUbJR6H9OJdRaERE5It8pCXqVSWaEjLoapWjWlIRmaQj3kpGTYiJMmsEnAwVUWg64v50MPuxBuT2+UdnlyosSfkskLsapOFKkiFeVaK+FpNUmuHIJLPGx8G8MVFmjYCToeZc0RD3p535jjVg9n1es7A7l4iI1GN3LhERkTG6bnGpanzsHxERkUFsiRIRkXrsziUiIpJgkopQBrtziYiIDGJLlIiIlPOVgUWsRImISD1eEyUiIjLGV1qivCZKRERkkFctUa1xZhooiUwyWXxcTYiJMmsEnAwVUWha4v40MPuxBkyyz32kO5dRaEREpIwjCu3moVPhFyARhVZ8ET8srP5RaOzOJSIiMsir7lx7gfH4HbPFJQE1IzLJe7VhjfoGAKPQqpru6DszRqEZO8/0n+M691lVRaH5Sneud6NzTRzrJc+ckUleK3V9iFFoVUv/Z8SEUWhGjlM1OMf1H+sq4COVKLtziYiIDOJ9okREpJyv3CfKSpSIiNRjdy4RERFVhC1RIiJSziIELBKPIZBZtiqxEiUiIvV8pDuXlSgRESnnKwOLeE2UiIjIILZEiYhIPXbnEhERGcPuXCIiIqqQdy1Rk2VjyqoJuYPe+/MYM0+0aunOjzVlnqih80z/OW7G70Ov+Uh3LvNEiYhIGUeeaHz/56XzRPOy/x/zRImIiGoqn8gTNZ77J587qDNzUDZr0ay5mroyJs2WoQrozxOV3WfMjvUe80TV8o08UaPlVpE7qHOfSWYtmjZXU1fGpKkzVAEteaKS+4zZsdWbWUbYymB3LhERkUG8T5SIiNQT4vJLZnkTYCVKRETK+crDFliJEhGRej4ysIjXRImIiAxiS5SIiJSz2C+/ZJY3A1aiRESkHrtziYiIqCJsiRIRkXIcnUtERGQU7xN1w6zRP4bLrSAySec+k4yJMm0kmK54LJPFvwHVIApNcp8x9o50YxQaEREp44hC65AyBbX8jUehlVy6iG8+eqbaR6GxO5eIiNTzkdG5VRaFJktHdFB1iEySwZgoz5Xebl1RaGbdbhm6o9BktllnzKGMKotC8xFVFoUmS390kJ7IJBmMiTJGVxSaabdbhuZ9JrXNZo2GrCIcnUtERGQUR+cSEREZ4ystUT6xiIiIyCC2RImISD2OziUiIjKG3blERERUIbZEiYhIPbu4/JJZ3gRYiRIRkXo+ck2U3blEREQGsSVKRETKWSA5sEhZSa4uVqJERKQen1jkhgnzEmVUh9xBGcxa9Fzp7daVJ2ra7ZahOU9UapvNmq9MSjFPlIiIlHHkiXb+n0moVUsiT7TkIr5aP8nrPNG5c+dixowZyM/PR+vWrTF79my0b9/e7byvv/463nrrLezcuRMAEB8fj6lTp5Y7vzscWEREROoJBS8vZWdnIyMjA5mZmdiyZQtat26N5ORkHD9+3O38GzZswMCBA/H5558jNzcXMTEx6NGjB44ePerxOr1qicrkierMzzOe+1cb1qhvABjP1tS53UaoyhM13/H+81jryNWUpeIc953tlv9cm5k1Si7DtTKOlmhCYqZ0S3TjhsletUQ7dOiAW265BXPmzAEA2O12xMTEYPTo0XjqqacqXd5msyE8PBxz5sxBamqqR+ussjxRrfl5Rsvtcj3QWLamuXMDjeeJmu54lzrWWnI1ZSk4x31muxV8rqnqnD592uX/gYGBCAwse025uLgYeXl5+Oc//+mcZrVakZSUhNzcXI/Wdf78eVy6dAnXXnutx+Vjdy4REalnV/ACEBMTg7CwMOdr2rRpbld34sQJ2Gw2REVFuUyPiopCfn6+R0V+8skn0aBBAyQlJXm8mbzFhYiIlLMIAYvEuFXHsocPH3bpznXXClXhhRdewPLly7FhwwYEBXneDc1KlIiIqq3Q0FCPronWq1cPfn5+KCgocJleUFCA6OjoCpd96aWX8MILL2DdunW4+WbvxnOwO5eIiNSr4tG5AQEBiI+PR05OjnOa3W5HTk4OOnXqVO5y06dPx5QpU7BmzRq0a9fOu5WCLVEiIroaNDyxKCMjA4MHD0a7du3Qvn17ZGVl4dy5c0hLSwMApKamomHDhs7rqi+++CImTpyIt99+G7Gxsc5rpyEhIQgJCfFonaxEiYioRujfvz8KCwsxceJE5Ofno02bNlizZo1zsNGhQ4dgtf7ZAfvaa6+huLgY/fr1c3mfzMxMTJo0yaN1shIlIiLlLELyAfQGl01PT0d6errbv23YsMHl/7/88ouxlZTCSpSIiNTzkQfQc2ARERGRQWyJEhGRchb75ZfM8mZQZVFoWqN/DJe7tuu/DcSCmS3ySFUUmvmO95/LaIkEk6XgHPed7Zb/XJMHfKQ7l1FoRESkjOMB9Im3/D/pB9Bv+O55r6PQqhqviRIRERnkVXduSsggwysyHtUkz2gsl6pYMKNk48R0xkSZOQpNV+ydns+IfBSaTLl1n+My8W9m/E4DgI/OLlVYkvKpenZudedVJSoVk6QxbkhNvFPVRyZJl1tjTJSZo9C0xd7p+IyoiELTGZGoM/bO9N9pV5mPXBNldy4REZFBvMWFiIjUE3Bmghpe3gRYiRIRkXK+ck2U3blEREQGsSVKRETqCUgOLFJWkquKlSgREanH0blERERUEbZEiYhIPTvknktcIx9AT0RE5AFfGZ3LSpSIiNTzkWuiXlWiUvFWEjFqsoyWW1UsmFHScWIaY6LMHIWmLfZOy2dEQRSazohEnbF3JvxOI/UYhUZERMo4otC6txyPWn7GK/sSWxFydr9U7aPQ2J1LRETqsTu3LJ1RaDqitVREocmUW8c+0x3/po/eSDBZOuP+zBx7JxOFZlZVFYXmK0wThaY/+kdDPJb2fVb18W/aaI4Ek6Uz7s/MsXdSUWhUMd7iQkREZIyv3OLCJxYREREZxJYoERGpx4FFREREBtkFYJGoCO3mqETZnUtERGQQW6JERKQeu3OJiIiMkqxETZLKzUqUiIjU85GWKK+JEhERGcSWKBERqWcXkOqSNcnoXFaiRESknrBffsksbwKmyRPVkZ+nIk/UbPtMd4aqPnpzNWXpzMw1c3asVJ4oEZgnSkRECjnyRJNiHkUtq0SeqL0I6w6/xjxRIiLyQbwmWpa9QF9uoJ5cTnPnDspmLcrkierMUTVGf54oM3O9J5uZK/O5lim77LGWWbc1Sl/ubU3kXUvUrHmJRstt9txByaxFmTxRnTmqhlSDPFH955fvZeZKfa4lyi59rM2Q8+sj94myO5eIiNQTkKxElZXkquLDFoiIiAxiS5SIiNRjdy4REZFBdjsAiQcm2GvgwxaIiIg84iMtUV4TJSIiMogtUSIiUs9HWqKsRImISD0feWIRu3OJiIgMYkuUiIiUE8IOIRFnJrNsVfKuEjVh1BMAiXKbPDJJMiZKJgpNZwScMfqj0Bj3VzXrLr3NUp9ribJLH2uN38UeE0KuS9Yk10QZhUZERMo4otC6X5OKWpYAw+9TIoqRc+otRqEREZEPEpIDi0zSvvOqEk0JGWR4RWaPidIRj6UnLklN/JueOLPLdMZj6aIi7k9HFJosnXF/ZlVlUWh2O2CRuK5ZE6+JmjnySJaOeCwtcUmq4t/MGpsHH4q9A5RE32ndVxrj/ogAducSEdHVwO5cIiIiY4TdDiHRnVszb3EhIiLyhI+0RPnEIiIiIoPYEiUiIvXsArDU/JYoK1EiIlJPCEiFcpukEmV3LhERkUFsiRIRkXLCLiAkunPN8kRatkSJiEg9YZd/GTB37lzExsYiKCgIHTp0wLffflvh/CtXrkTz5s0RFBSEuLg4rF692qv1sRIlIqIaITs7GxkZGcjMzMSWLVvQunVrJCcn4/jx427n37x5MwYOHIhhw4Zh69at6NOnD/r06YOdO3d6vE5WokREpJywC+mXt2bNmoXhw4cjLS0NLVu2xLx58xAcHIyFCxe6nf/ll1/GHXfcgccffxwtWrTAlClT8Le//Q1z5szxeJ2sRImISL0q7s4tLi5GXl4ekpKSnNOsViuSkpKQm5vrdpnc3FyX+QEgOTm53Pnd8Wpg0Vr7Sm9mV2qt5idAfXR2aZWv0xe3WQXZ/WbW7ZZlNN1D93kqo8oSTXxQCS5JPbCoBJcAXM4nLS0wMBCBgWVDzU+cOAGbzYaoqCiX6VFRUdizZ4/bdeTn57udPz8/3+NycnQuEREpExAQgOjoaHyV790AHXdCQkIQExPjMi0zMxOTJk2Sfm9VWIkSEZEyQUFBOHDgAIqLi6XfSwgBi8Ulu85tKxQA6tWrBz8/PxQUFLhMLygoQHR0tNtloqOjvZrfHVaiRESkVFBQEIKCgqp0nQEBAYiPj0dOTg769OkDALDb7cjJyUF6errbZTp16oScnBw89thjzmlr165Fp06dPF4vK1EiIqoRMjIyMHjwYLRr1w7t27dHVlYWzp07h7S0NABAamoqGjZsiGnTpgEAxo4di65du2LmzJm48847sXz5cnz//fdYsGCBx+tkJUpERDVC//79UVhYiIkTJyI/Px9t2rTBmjVrnIOHDh06BKv1z5tSbr31Vrz99tuYMGECnn76aTRr1gzvv/8+WrVq5fE6LcIsz1YiIiKqZnifKBERkUGsRImIiAxiJUpERGQQK1EiIiKDWIkSEREZxEqUiIjIIFaiREREBrESJSIiMoiVKBERkUGsRImIiAxiJUpERGQQK1EiIiKD/j8VchUbWwY8VQAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "weights_to_display = tf.reshape(tensor_data, [tensor_data.shape[0],tf.reduce_prod(tensor_data.shape[1:])])\n", "weights_to_display = weights_to_display[0:width, 0:height]\n", "\n", "val_ones = np.ones([height, width])\n", "val_zeros = np.zeros([height, width])\n", "subset_values_to_display = np.where(abs(weights_to_display) > 0, val_ones, val_zeros)\n", "\n", "plot_separation_lines(height, width)\n", "\n", "plt.axis('off')\n", "plt.imshow(subset_values_to_display)\n", "plt.colorbar()\n", "plt.title(\"Unstructed pruned weights for Conv2D layer\")\n", "plt.show()" ] }, { "cell_type": "markdown", "metadata": { "id": "vqsfZdMpWr_N" }, "source": [ "The TensorFlow Model Optimization Toolkit includes a python script that can be used to check whether which layers in the model from the given tflite file have the structurally pruned weights: [`check_sparsity_m_by_n.py`](https://github.com/tensorflow/model-optimization/blob/master/tensorflow_model_optimization/python/core/sparsity/keras/tools/check_sparsity_m_by_n.py). The following command demonstrates how to use this tool to check for 2 by 4 sparsity in a specific model." ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "execution": { "iopub.execute_input": "2025-06-21T11:30:33.339059Z", "iopub.status.busy": "2025-06-21T11:30:33.338803Z", "iopub.status.idle": "2025-06-21T11:30:33.567340Z", "shell.execute_reply": "2025-06-21T11:30:33.566209Z" }, "id": "7HDYffebWr_N" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "python3: can't open file '/tmpfs/src/temp/tensorflow_model_optimization/g3doc/guide/pruning/./tensorflow_model_optimization/python/core/sparsity/keras/tools/check_sparsity_m_by_n.py': [Errno 2] No such file or directory\r\n" ] } ], "source": [ "! python3 ./tensorflow_model_optimization/python/core/sparsity/keras/tools/check_sparsity_m_by_n.py --model_tflite=pruned_model.tflite --m_by_n=2,4\n" ] } ], "metadata": { "colab": { "collapsed_sections": [ "Tce3stUlHN0L" ], "name": "pruning_with_sparsity_2_by_4.ipynb", "provenance": [], "toc_visible": true }, "interpreter": { "hash": "5be03e09ac1816611305450014280c0b9eb46a3a95e12dcae8d73de01e2da776" }, "kernelspec": { "display_name": "Python 3.6.9 64-bit ('mo': venv)", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.23" } }, "nbformat": 4, "nbformat_minor": 0 }