{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_-GR0EDHM1SO"
      },
      "source": [
        "##### Copyright 2019 The TensorFlow Authors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "cellView": "form",
        "id": "R3yYtBPkM2qZ"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6Y8E0lw5eYWm"
      },
      "source": [
        "# トレーニング後のダイナミックレンジ量子化"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "CIGrZZPTZVeO"
      },
      "source": [
        "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
        "  <td>     <a target=\"_blank\" href=\"https://www.tensorflow.org/lite/performance/post_training_quant\"><img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\">TensorFlow.org で表示</a>\n",
        "</td>\n",
        "  <td>     <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs-l10n/blob/master/site/ja/lite/performance/post_training_quant.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\">Google Colab で実行</a>\n",
        "</td>\n",
        "  <td>     <a target=\"_blank\" href=\"https://github.com/tensorflow/docs-l10n/blob/master/site/ja/lite/performance/post_training_quant.ipynb\"><img src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\">GitHub でソースを表示</a>\n",
        "</td>\n",
        "  <td><a href=\"https://storage.googleapis.com/tensorflow_docs/docs-l10n/site/ja/lite/performance/post_training_quant.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\"> ノートブックをダウンロード</a></td>\n",
        "  <td>     <a href=\"https://tfhub.dev/google/imagenet/resnet_v2_101/classification/4\"><img src=\"https://www.tensorflow.org/images/hub_logo_32px.png\">\tTF Hub モデルを参照</a> </td>\n",
        "</table>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BTC1rDAuei_1"
      },
      "source": [
        "## 概要\n",
        "\n",
        "[TensorFlow Lite](https://www.tensorflow.org/lite/) では、TensorFlow グラフ定義から TensorFlow Lite フラットバッファ形式へのモデル変換の一部として、重みを 8 ビット精度に変換できるようになりました。ダイナミックレンジ量子化は、モデルサイズを 4 分の 1 に削減します。さらに、TFLite は、アクティベーションのオンザフライの量子化および逆量子化をサポートし、以下を可能にします。\n",
        "\n",
        "1. 可能な場合、より高速な実装のために量子化されたカーネルを使用する。\n",
        "2. グラフの異なる部分に浮動小数点カーネルと量子化カーネルを使用する。\n",
        "\n",
        "アクティベーションは常に浮動小数点で保存されます。量子化カーネルをサポートする演算の場合、アクティベーションは処理の前に動的に 8 ビットの精度に量子化され、処理後に浮動小数点精度に逆量子化されます。変換されるモデルによって異なりますが、純粋な浮動小数点の計算より高速になる可能性があります。\n",
        "\n",
        "[量子化認識トレーニング](https://github.com/tensorflow/tensorflow/tree/r1.14/tensorflow/contrib/quantize)とは対照的に、この方法では、重みはトレーニング後に量子化され、アクティベーションは推論時に動的に量子化されます。したがって、モデルの重みは再量子化されず、量子化による誤差が補正されません。量子化モデルの精度をチェックして、精度低下が許容範囲内であることを確認することが重要です。\n",
        "\n",
        "このチュートリアルでは、MNIST モデルを新規にトレーニングし、TensorFlow でその精度を確認してから、モデルをダイナミックレンジ量子化を使用した Tensorflow Lite フラットバッファに変換します。最後に、変換されたモデルの精度を確認し、元の float モデルと比較します。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2XsEP17Zelz9"
      },
      "source": [
        "## MNIST モデルの構築"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "dDqqUIZjZjac"
      },
      "source": [
        "### セットアップ"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "gyqAw1M9lyab"
      },
      "outputs": [],
      "source": [
        "import logging\n",
        "logging.getLogger(\"tensorflow\").setLevel(logging.DEBUG)\n",
        "\n",
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "import numpy as np\n",
        "import pathlib"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eQ6Q0qqKZogR"
      },
      "source": [
        "### TensorFlow モデルのトレーニング"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "hWSAjQWagIHl"
      },
      "outputs": [],
      "source": [
        "# Load MNIST dataset\n",
        "mnist = keras.datasets.mnist\n",
        "(train_images, train_labels), (test_images, test_labels) = mnist.load_data()\n",
        "\n",
        "# Normalize the input image so that each pixel value is between 0 to 1.\n",
        "train_images = train_images / 255.0\n",
        "test_images = test_images / 255.0\n",
        "\n",
        "# Define the model architecture\n",
        "model = keras.Sequential([\n",
        "  keras.layers.InputLayer(input_shape=(28, 28)),\n",
        "  keras.layers.Reshape(target_shape=(28, 28, 1)),\n",
        "  keras.layers.Conv2D(filters=12, kernel_size=(3, 3), activation=tf.nn.relu),\n",
        "  keras.layers.MaxPooling2D(pool_size=(2, 2)),\n",
        "  keras.layers.Flatten(),\n",
        "  keras.layers.Dense(10)\n",
        "])\n",
        "\n",
        "# Train the digit classification model\n",
        "model.compile(optimizer='adam',\n",
        "              loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),\n",
        "              metrics=['accuracy'])\n",
        "model.fit(\n",
        "  train_images,\n",
        "  train_labels,\n",
        "  epochs=1,\n",
        "  validation_data=(test_images, test_labels)\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "5NMaNZQCkW9X"
      },
      "source": [
        "この例では、モデルを 1 エポックでトレーニングしたので、トレーニングの精度は 96％ 以下になります。\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xl8_fzVAZwOh"
      },
      "source": [
        "### TensorFlow Lite モデルに変換する\n",
        "\n",
        "TensorFlow Lite [Converter](https://www.tensorflow.org/lite/models/convert) を使用して、トレーニング済みモデルを TensorFlow Lite モデルに変換できるようになりました。\n",
        "\n",
        "次に、`TFLiteConverter`を使用してモデルを読み込みます。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "_i8B2nDZmAgQ"
      },
      "outputs": [],
      "source": [
        "converter = tf.lite.TFLiteConverter.from_keras_model(model)\n",
        "tflite_model = converter.convert()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F2o2ZfF0aiCx"
      },
      "source": [
        "tflite ファイルに書き込みます。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vptWZq2xnclo"
      },
      "outputs": [],
      "source": [
        "tflite_models_dir = pathlib.Path(\"/tmp/mnist_tflite_models/\")\n",
        "tflite_models_dir.mkdir(exist_ok=True, parents=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Ie9pQaQrn5ue"
      },
      "outputs": [],
      "source": [
        "tflite_model_file = tflite_models_dir/\"mnist_model.tflite\"\n",
        "tflite_model_file.write_bytes(tflite_model)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7BONhYtYocQY"
      },
      "source": [
        "エクスポート時にモデルを量子化するには、`optimizations`フラグを設定してサイズを最適化します。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "g8PUvLWDlmmz"
      },
      "outputs": [],
      "source": [
        "converter.optimizations = [tf.lite.Optimize.DEFAULT]\n",
        "tflite_quant_model = converter.convert()\n",
        "tflite_model_quant_file = tflite_models_dir/\"mnist_model_quant.tflite\"\n",
        "tflite_model_quant_file.write_bytes(tflite_quant_model)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "PhMmUTl4sbkz"
      },
      "source": [
        "生成されるファイルのサイズが約`1/4`であることに注意してください。"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "JExfcfLDscu4"
      },
      "outputs": [],
      "source": [
        "!ls -lh {tflite_models_dir}"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L8lQHMp_asCq"
      },
      "source": [
        "## TFLite モデルを実行する\n",
        "\n",
        "Python TensorFlow Lite インタープリタを使用して TensorFlow Lite モデルを実行します。\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Ap_jE7QRvhPf"
      },
      "source": [
        "### モデルをインタープリタに読み込む"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "Jn16Rc23zTss"
      },
      "outputs": [],
      "source": [
        "interpreter = tf.lite.Interpreter(model_path=str(tflite_model_file))\n",
        "interpreter.allocate_tensors()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "J8Pztk1mvNVL"
      },
      "outputs": [],
      "source": [
        "interpreter_quant = tf.lite.Interpreter(model_path=str(tflite_model_quant_file))\n",
        "interpreter_quant.allocate_tensors()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2opUt_JTdyEu"
      },
      "source": [
        "### 1 つの画像でモデルをテストする"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "AKslvo2kwWac"
      },
      "outputs": [],
      "source": [
        "test_image = np.expand_dims(test_images[0], axis=0).astype(np.float32)\n",
        "\n",
        "input_index = interpreter.get_input_details()[0][\"index\"]\n",
        "output_index = interpreter.get_output_details()[0][\"index\"]\n",
        "\n",
        "interpreter.set_tensor(input_index, test_image)\n",
        "interpreter.invoke()\n",
        "predictions = interpreter.get_tensor(output_index)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "XZClM2vo3_bm"
      },
      "outputs": [],
      "source": [
        "import matplotlib.pylab as plt\n",
        "\n",
        "plt.imshow(test_images[0])\n",
        "template = \"True:{true}, predicted:{predict}\"\n",
        "_ = plt.title(template.format(true= str(test_labels[0]),\n",
        "                              predict=str(np.argmax(predictions[0]))))\n",
        "plt.grid(False)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LwN7uIdCd8Gw"
      },
      "source": [
        "### モデルを評価する"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "05aeAuWjvjPx"
      },
      "outputs": [],
      "source": [
        "# A helper function to evaluate the TF Lite model using \"test\" dataset.\n",
        "def evaluate_model(interpreter):\n",
        "  input_index = interpreter.get_input_details()[0][\"index\"]\n",
        "  output_index = interpreter.get_output_details()[0][\"index\"]\n",
        "\n",
        "  # Run predictions on every image in the \"test\" dataset.\n",
        "  prediction_digits = []\n",
        "  for test_image in test_images:\n",
        "    # Pre-processing: add batch dimension and convert to float32 to match with\n",
        "    # the model's input data format.\n",
        "    test_image = np.expand_dims(test_image, axis=0).astype(np.float32)\n",
        "    interpreter.set_tensor(input_index, test_image)\n",
        "\n",
        "    # Run inference.\n",
        "    interpreter.invoke()\n",
        "\n",
        "    # Post-processing: remove batch dimension and find the digit with highest\n",
        "    # probability.\n",
        "    output = interpreter.tensor(output_index)\n",
        "    digit = np.argmax(output()[0])\n",
        "    prediction_digits.append(digit)\n",
        "\n",
        "  # Compare prediction results with ground truth labels to calculate accuracy.\n",
        "  accurate_count = 0\n",
        "  for index in range(len(prediction_digits)):\n",
        "    if prediction_digits[index] == test_labels[index]:\n",
        "      accurate_count += 1\n",
        "  accuracy = accurate_count * 1.0 / len(prediction_digits)\n",
        "\n",
        "  return accuracy"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "DqXBnDfJ7qxL"
      },
      "outputs": [],
      "source": [
        "print(evaluate_model(interpreter))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Km3cY9ry8ZlG"
      },
      "source": [
        "ダイナミックレンジ量子化モデルの評価を繰り返して、以下を取得する\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "-9cnwiPp6EGm"
      },
      "outputs": [],
      "source": [
        "print(evaluate_model(interpreter_quant))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "L7lfxkor8pgv"
      },
      "source": [
        "この例では、圧縮されたモデルの精度は同じです。"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M0o1FtmWeKZm"
      },
      "source": [
        "## 既存のモデルの最適化\n",
        "\n",
        "事前アクティべーレイヤーを備えた Resnet (Resnet-v2) は、ビジョンアプリケーションで広く使用されています。resnet-v2-101 の事前トレーニング済み凍結グラフは、[Tensorflow Hub](https://tfhub.dev/google/imagenet/resnet_v2_101/classification/4) で入手できます。\n",
        "\n",
        "次の方法で、量子化された凍結グラフを TensorFLow Lite フラットバッファに変換できます。\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "jrXZxSJiJfYN"
      },
      "outputs": [],
      "source": [
        "import tensorflow_hub as hub\n",
        "\n",
        "resnet_v2_101 = tf.keras.Sequential([\n",
        "  keras.layers.InputLayer(input_shape=(224, 224, 3)),\n",
        "  hub.KerasLayer(\"https://tfhub.dev/google/imagenet/resnet_v2_101/classification/4\")\n",
        "])\n",
        "\n",
        "converter = tf.lite.TFLiteConverter.from_keras_model(resnet_v2_101)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "LwnV4KxwVEoG"
      },
      "outputs": [],
      "source": [
        "# Convert to TF Lite without quantization\n",
        "resnet_tflite_file = tflite_models_dir/\"resnet_v2_101.tflite\"\n",
        "resnet_tflite_file.write_bytes(converter.convert())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "2qkZD0VoVExe"
      },
      "outputs": [],
      "source": [
        "# Convert to TF Lite with quantization\n",
        "converter.optimizations = [tf.lite.Optimize.DEFAULT]\n",
        "resnet_quantized_tflite_file = tflite_models_dir/\"resnet_v2_101_quantized.tflite\"\n",
        "resnet_quantized_tflite_file.write_bytes(converter.convert())"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "vhOjeg1x9Knp"
      },
      "outputs": [],
      "source": [
        "!ls -lh {tflite_models_dir}/*.tflite"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qqHLaqFMCjRZ"
      },
      "source": [
        "モデルサイズが 171 MB から 43 MB に削減されます。imagenet でのこのモデルの精度は、[TFLite 精度測定](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/tools/evaluation/tasks/imagenet_image_classification)用に提供されているスクリプトを使用して評価できます。\n",
        "\n",
        "最適化されたモデルの top-1 精度は、浮動小数点のモデルと同じく 76.8 です。"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [],
      "name": "post_training_quant.ipynb",
      "toc_visible": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}