{ "cells": [ { "cell_type": "markdown", "id": "g_nWetWWd_ns", "metadata": { "id": "g_nWetWWd_ns" }, "source": [ "##### Copyright 2021 The TensorFlow Authors." ] }, { "cell_type": "code", "execution_count": 1, "id": "2pHVBk_seED1", "metadata": { "cellView": "form", "execution": { "iopub.execute_input": "2024-07-19T11:25:52.210979Z", "iopub.status.busy": "2024-07-19T11:25:52.210756Z", "iopub.status.idle": "2024-07-19T11:25:52.214274Z", "shell.execute_reply": "2024-07-19T11:25:52.213718Z" }, "id": "2pHVBk_seED1" }, "outputs": [], "source": [ "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ] }, { "cell_type": "markdown", "id": "M7vSdG6sAIQn", "metadata": { "id": "M7vSdG6sAIQn" }, "source": [ "# Signatures in TensorFlow Lite" ] }, { "cell_type": "markdown", "id": "fwc5GKHBASdc", "metadata": { "id": "fwc5GKHBASdc" }, "source": [ "\n", " \n", " \n", " \n", " \n", "
\n", " View on TensorFlow.org\n", " \n", " Run in Google Colab\n", " \n", " View source on GitHub\n", " \n", " Download notebook\n", "
" ] }, { "cell_type": "markdown", "id": "9ee074e4", "metadata": { "id": "9ee074e4" }, "source": [ "TensorFlow Lite supports converting TensorFlow model's input/output\n", "specifications to TensorFlow Lite models. The input/output specifications are\n", "called \"signatures\". Signatures can be specified when building a SavedModel or\n", "creating concrete functions.\n", "\n", "Signatures in TensorFlow Lite provide the following features:\n", "\n", "* They specify inputs and outputs of the converted TensorFlow Lite model by\n", " respecting the TensorFlow model's signatures.\n", "* Allow a single TensorFlow Lite model to support multiple entry points.\n", "\n", "The signature is composed of three pieces:\n", "\n", "* Inputs: Map for inputs from input name in the signature to an input tensor.\n", "* Outputs: Map for output mapping from output name in signature to an output\n", " tensor.\n", "* Signature Key: Name that identifies an entry point of the graph.\n", "\n" ] }, { "cell_type": "markdown", "id": "UaWdLA3fQDK2", "metadata": { "id": "UaWdLA3fQDK2" }, "source": [ "## Setup" ] }, { "cell_type": "code", "execution_count": 2, "id": "9j4MGqyKQEo4", "metadata": { "execution": { "iopub.execute_input": "2024-07-19T11:25:52.217669Z", "iopub.status.busy": "2024-07-19T11:25:52.217421Z", "iopub.status.idle": "2024-07-19T11:25:55.253934Z", "shell.execute_reply": "2024-07-19T11:25:55.253250Z" }, "id": "9j4MGqyKQEo4" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2024-07-19 11:25:52.465411: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:485] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n", "2024-07-19 11:25:52.486411: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:8454] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n", "2024-07-19 11:25:52.493033: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1452] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n" ] } ], "source": [ "import tensorflow as tf" ] }, { "cell_type": "markdown", "id": "FN2N6hPEP-Ay", "metadata": { "id": "FN2N6hPEP-Ay" }, "source": [ "## Example model\n", "\n", "Let's say we have two tasks, e.g., encoding and decoding, as a TensorFlow model:" ] }, { "cell_type": "code", "execution_count": 3, "id": "d8577c80", "metadata": { "execution": { "iopub.execute_input": "2024-07-19T11:25:55.258031Z", "iopub.status.busy": "2024-07-19T11:25:55.257555Z", "iopub.status.idle": "2024-07-19T11:25:55.264676Z", "shell.execute_reply": "2024-07-19T11:25:55.264100Z" }, "id": "d8577c80" }, "outputs": [], "source": [ "class Model(tf.Module):\n", "\n", " @tf.function(input_signature=[tf.TensorSpec(shape=[None], dtype=tf.float32)])\n", " def encode(self, x):\n", " result = tf.strings.as_string(x)\n", " return {\n", " \"encoded_result\": result\n", " }\n", "\n", " @tf.function(input_signature=[tf.TensorSpec(shape=[None], dtype=tf.string)])\n", " def decode(self, x):\n", " result = tf.strings.to_number(x)\n", " return {\n", " \"decoded_result\": result\n", " }" ] }, { "cell_type": "markdown", "id": "9c814c6e", "metadata": { "id": "9c814c6e" }, "source": [ "In the signature wise, the above TensorFlow model can be summarized as follows:\n", "\n", "* Signature\n", "\n", " - Key: encode\n", " - Inputs: {\"x\"}\n", " - Output: {\"encoded_result\"}\n", "\n", "* Signature\n", "\n", " - Key: decode\n", " - Inputs: {\"x\"}\n", " - Output: {\"decoded_result\"}" ] }, { "cell_type": "markdown", "id": "c4099f20", "metadata": { "id": "c4099f20" }, "source": [ "## Convert a model with Signatures\n", "\n", "TensorFlow Lite converter APIs will bring the above signature information into\n", "the converted TensorFlow Lite model.\n", "\n", "This conversion functionality is available on all the converter APIs starting\n", "from TensorFlow version 2.7.0. See example usages.\n" ] }, { "cell_type": "markdown", "id": "Qv0WwFQkQgnO", "metadata": { "id": "Qv0WwFQkQgnO" }, "source": [ "\n", "### From Saved Model" ] }, { "cell_type": "code", "execution_count": 4, "id": "96c8fc79", "metadata": { "execution": { "iopub.execute_input": "2024-07-19T11:25:55.268535Z", "iopub.status.busy": "2024-07-19T11:25:55.267964Z", "iopub.status.idle": "2024-07-19T11:25:57.870957Z", "shell.execute_reply": "2024-07-19T11:25:57.870035Z" }, "id": "96c8fc79" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n", "I0000 00:00:1721388355.766954 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388355.770387 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388355.773500 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388355.776724 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388355.964425 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388355.969002 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388355.971855 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388355.974821 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388355.977723 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388355.980746 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388355.983554 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388355.986509 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.226874 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.228952 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.231036 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.233978 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.236025 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.237934 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.239855 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.241877 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.243815 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.245725 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.247642 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.249650 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.288270 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.290288 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.292249 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.294272 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.296252 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.298189 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.300125 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.302117 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.304075 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.307401 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.309796 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.312221 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Assets written to: content/saved_models/coding/assets\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Assets written to: content/saved_models/coding/assets\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "{'decode': {'inputs': ['x'], 'outputs': ['decoded_result']}, 'encode': {'inputs': ['x'], 'outputs': ['encoded_result']}}\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "WARNING: All log messages before absl::InitializeLog() is called are written to STDERR\n", "W0000 00:00:1721388357.770988 11098 tf_tfl_flatbuffer_helpers.cc:392] Ignored output_format.\n", "W0000 00:00:1721388357.771028 11098 tf_tfl_flatbuffer_helpers.cc:395] Ignored drop_control_dependency.\n", "2024-07-19 11:25:57.816402: W tensorflow/compiler/mlir/lite/flatbuffer_export.cc:3463] TFLite interpreter needs to link Flex delegate in order to run the model since it contains the following Select TFop(s):\n", "Flex ops: FlexAsString, FlexStringToNumber\n", "Details:\n", "\ttf.AsString(tensor) -> (tensor) : {device = \"\", fill = \"\", precision = -1 : i64, scientific = false, shortest = false, width = -1 : i64}\n", "\ttf.StringToNumber(tensor) -> (tensor) : {device = \"\", out_type = f32}\n", "See instructions: https://www.tensorflow.org/lite/guide/ops_select\n", "INFO: Created TensorFlow Lite delegate for select TF ops.\n", "I0000 00:00:1721388357.825869 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.827905 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.829889 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.831931 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.834022 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one " ] }, { "name": "stderr", "output_type": "stream", "text": [ "NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.835949 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.837883 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.839915 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.841875 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.843821 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.845762 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.847746 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.849850 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.851771 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.853685 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.855687 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.857634 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.859594 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.861531 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388357.863545 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "INFO: TfLiteFlexDelegate delegate: 1 nodes delegated out of 1 nodes with 1 partitions.\n", "\n", "2024-07-19 11:25:57.867316: E tensorflow/core/framework/node_def_util.cc:676] NodeDef mentions attribute use_inter_op_parallelism which is not in the op definition: Op output:out_type; attr=out_type:type,default=DT_FLOAT,allowed=[DT_FLOAT, DT_DOUBLE, DT_INT32, DT_INT64, DT_UINT32, DT_UINT64]> This may be expected if your graph generating binary is newer than this binary. Unknown attributes will be ignored. NodeDef: {{node StringToNumber}}\n" ] } ], "source": [ "model = Model()\n", "\n", "# Save the model\n", "SAVED_MODEL_PATH = 'content/saved_models/coding'\n", "\n", "tf.saved_model.save(\n", " model, SAVED_MODEL_PATH,\n", " signatures={\n", " 'encode': model.encode.get_concrete_function(),\n", " 'decode': model.decode.get_concrete_function()\n", " })\n", "\n", "# Convert the saved model using TFLiteConverter\n", "converter = tf.lite.TFLiteConverter.from_saved_model(SAVED_MODEL_PATH)\n", "converter.target_spec.supported_ops = [\n", " tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.\n", " tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.\n", "]\n", "tflite_model = converter.convert()\n", "\n", "# Print the signatures from the converted model\n", "interpreter = tf.lite.Interpreter(model_content=tflite_model)\n", "signatures = interpreter.get_signature_list()\n", "print(signatures)" ] }, { "cell_type": "markdown", "id": "5baa9f17", "metadata": { "id": "5baa9f17" }, "source": [ "### From Keras Model" ] }, { "cell_type": "code", "execution_count": 5, "id": "71f29229", "metadata": { "execution": { "iopub.execute_input": "2024-07-19T11:25:57.875240Z", "iopub.status.busy": "2024-07-19T11:25:57.874986Z", "iopub.status.idle": "2024-07-19T11:25:58.212555Z", "shell.execute_reply": "2024-07-19T11:25:58.211904Z" }, "id": "71f29229" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/keras/src/layers/core/dense.py:87: UserWarning: Do not pass an `input_shape`/`input_dim` argument to a layer. When using Sequential models, prefer using an `Input(shape)` object as the first layer in the model instead.\n", " super().__init__(activity_regularizer=activity_regularizer, **kwargs)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Assets written to: /tmpfs/tmp/tmpls6_w79q/assets\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Assets written to: /tmpfs/tmp/tmpls6_w79q/assets\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Saved artifact at '/tmpfs/tmp/tmpls6_w79q'. The following endpoints are available:\n", "\n", "* Endpoint 'serve'\n", " args_0 (POSITIONAL_ONLY): TensorSpec(shape=(None, 4), dtype=tf.float32, name='keras_tensor')\n", "Output Type:\n", " TensorSpec(shape=(None, 1), dtype=tf.float32, name=None)\n", "Captures:\n", " 140465497142544: TensorSpec(shape=(), dtype=tf.resource, name=None)\n", " 140465497142368: TensorSpec(shape=(), dtype=tf.resource, name=None)\n", " 140465497145008: TensorSpec(shape=(), dtype=tf.resource, name=None)\n", " 140465497145184: TensorSpec(shape=(), dtype=tf.resource, name=None)\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "{'serving_default': {'inputs': ['keras_tensor'], 'outputs': ['output_0']}}\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "W0000 00:00:1721388358.135245 11098 tf_tfl_flatbuffer_helpers.cc:392] Ignored output_format.\n", "W0000 00:00:1721388358.135270 11098 tf_tfl_flatbuffer_helpers.cc:395] Ignored drop_control_dependency.\n" ] } ], "source": [ "# Generate a Keras model.\n", "keras_model = tf.keras.Sequential(\n", " [\n", " tf.keras.layers.Dense(2, input_dim=4, activation='relu', name='x'),\n", " tf.keras.layers.Dense(1, activation='relu', name='output'),\n", " ]\n", ")\n", "\n", "# Convert the keras model using TFLiteConverter.\n", "# Keras model converter API uses the default signature automatically.\n", "converter = tf.lite.TFLiteConverter.from_keras_model(keras_model)\n", "tflite_model = converter.convert()\n", "\n", "# Print the signatures from the converted model\n", "interpreter = tf.lite.Interpreter(model_content=tflite_model)\n", "\n", "signatures = interpreter.get_signature_list()\n", "print(signatures)" ] }, { "cell_type": "markdown", "id": "e4d30f85", "metadata": { "id": "e4d30f85" }, "source": [ "### From Concrete Functions" ] }, { "cell_type": "code", "execution_count": 6, "id": "c9e8a742", "metadata": { "execution": { "iopub.execute_input": "2024-07-19T11:25:58.216246Z", "iopub.status.busy": "2024-07-19T11:25:58.216007Z", "iopub.status.idle": "2024-07-19T11:25:58.408050Z", "shell.execute_reply": "2024-07-19T11:25:58.406764Z" }, "id": "c9e8a742" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "INFO:tensorflow:Assets written to: /tmpfs/tmp/tmpvj1us7a_/assets\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "INFO:tensorflow:Assets written to: /tmpfs/tmp/tmpvj1us7a_/assets\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "{'decode': {'inputs': ['x'], 'outputs': ['decoded_result']}, 'encode': {'inputs': ['x'], 'outputs': ['encoded_result']}}\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "W0000 00:00:1721388358.314179 11098 tf_tfl_flatbuffer_helpers.cc:392] Ignored output_format.\n", "W0000 00:00:1721388358.314199 11098 tf_tfl_flatbuffer_helpers.cc:395] Ignored drop_control_dependency.\n", "2024-07-19 11:25:58.350819: W tensorflow/compiler/mlir/lite/flatbuffer_export.cc:3463] TFLite interpreter needs to link Flex delegate in order to run the model since it contains the following Select TFop(s):\n", "Flex ops: FlexAsString, FlexStringToNumber\n", "Details:\n", "\ttf.AsString(tensor) -> (tensor) : {device = \"\", fill = \"\", precision = -1 : i64, scientific = false, shortest = false, width = -1 : i64}\n", "\ttf.StringToNumber(tensor) -> (tensor) : {device = \"\", out_type = f32}\n", "See instructions: https://www.tensorflow.org/lite/guide/ops_select\n", "I0000 00:00:1721388358.360182 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.362471 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.364913 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.367496 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.369588 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS ha" ] }, { "name": "stderr", "output_type": "stream", "text": [ "d negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.371599 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.373508 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.375561 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.377521 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.379552 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.381482 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.383506 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.385611 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.387661 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.389715 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.391770 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.393767 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.395837 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See mo" ] }, { "name": "stderr", "output_type": "stream", "text": [ "re at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.397790 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.399833 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n" ] } ], "source": [ "model = Model()\n", "\n", "# Convert the concrete functions using TFLiteConverter\n", "converter = tf.lite.TFLiteConverter.from_concrete_functions(\n", " [model.encode.get_concrete_function(),\n", " model.decode.get_concrete_function()], model)\n", "converter.target_spec.supported_ops = [\n", " tf.lite.OpsSet.TFLITE_BUILTINS, # enable TensorFlow Lite ops.\n", " tf.lite.OpsSet.SELECT_TF_OPS # enable TensorFlow ops.\n", "]\n", "tflite_model = converter.convert()\n", "\n", "# Print the signatures from the converted model\n", "interpreter = tf.lite.Interpreter(model_content=tflite_model)\n", "signatures = interpreter.get_signature_list()\n", "print(signatures)" ] }, { "cell_type": "markdown", "id": "b5e85934", "metadata": { "id": "b5e85934" }, "source": [ "## Run Signatures\n", "\n", "TensorFlow inference APIs support the signature-based executions:\n", "\n", "* Accessing the input/output tensors through the names of the inputs and\n", " outputs, specified by the signature.\n", "* Running each entry point of the graph separately, identified by the\n", " signature key.\n", "* Support for the SavedModel's initialization procedure.\n", "\n", "Java, C++ and Python language bindings are currently available. See example the\n", "below sections.\n" ] }, { "cell_type": "markdown", "id": "ZRBMFciMQmiB", "metadata": { "id": "ZRBMFciMQmiB" }, "source": [ "\n", "### Java" ] }, { "cell_type": "markdown", "id": "04c5a4fc", "metadata": { "id": "04c5a4fc" }, "source": [ "```\n", "try (Interpreter interpreter = new Interpreter(file_of_tensorflowlite_model)) {\n", " // Run encoding signature.\n", " Map inputs = new HashMap<>();\n", " inputs.put(\"x\", input);\n", " Map outputs = new HashMap<>();\n", " outputs.put(\"encoded_result\", encoded_result);\n", " interpreter.runSignature(inputs, outputs, \"encode\");\n", "\n", " // Run decoding signature.\n", " Map inputs = new HashMap<>();\n", " inputs.put(\"x\", encoded_result);\n", " Map outputs = new HashMap<>();\n", " outputs.put(\"decoded_result\", decoded_result);\n", " interpreter.runSignature(inputs, outputs, \"decode\");\n", "}\n", "```" ] }, { "cell_type": "markdown", "id": "5ba86c64", "metadata": { "id": "5ba86c64" }, "source": [ "### C++" ] }, { "cell_type": "markdown", "id": "397ad6fd", "metadata": { "id": "397ad6fd" }, "source": [ "```\n", "SignatureRunner* encode_runner =\n", " interpreter->GetSignatureRunner(\"encode\");\n", "encode_runner->ResizeInputTensor(\"x\", {100});\n", "encode_runner->AllocateTensors();\n", "\n", "TfLiteTensor* input_tensor = encode_runner->input_tensor(\"x\");\n", "float* input = GetTensorData(input_tensor);\n", "// Fill `input`.\n", "\n", "encode_runner->Invoke();\n", "\n", "const TfLiteTensor* output_tensor = encode_runner->output_tensor(\n", " \"encoded_result\");\n", "float* output = GetTensorData(output_tensor);\n", "// Access `output`.\n", "```" ] }, { "cell_type": "markdown", "id": "0f4c6ad4", "metadata": { "id": "0f4c6ad4" }, "source": [ "### Python" ] }, { "cell_type": "code", "execution_count": 7, "id": "ab7b1963", "metadata": { "execution": { "iopub.execute_input": "2024-07-19T11:25:58.413849Z", "iopub.status.busy": "2024-07-19T11:25:58.413276Z", "iopub.status.idle": "2024-07-19T11:25:58.465769Z", "shell.execute_reply": "2024-07-19T11:25:58.465092Z" }, "id": "ab7b1963" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Signature: {'decode': {'inputs': ['x'], 'outputs': ['decoded_result']}, 'encode': {'inputs': ['x'], 'outputs': ['encoded_result']}}\n", "Input: tf.Tensor([1. 2. 3.], shape=(3,), dtype=float32)\n", "Encoded result: {'encoded_result': array([b'1.000000', b'2.000000', b'3.000000'], dtype=object)}\n", "Decoded result: {'decoded_result': array([1., 2., 3.], dtype=float32)}\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "I0000 00:00:1721388358.416569 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.418650 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.420607 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.422634 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.424605 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.426617 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.428553 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.430571 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.432535 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.434545 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.436448 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.438490 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.440605 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.442635 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.444583 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.446612 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.448591 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.450643 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.452608 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n", "I0000 00:00:1721388358.454631 11098 cuda_executor.cc:1015] successful NUMA node read from SysFS had negative value (-1), but there must be at least one NUMA node, so returning NUMA node zero. See more at https://github.com/torvalds/linux/blob/v6.0/Documentation/ABI/testing/sysfs-bus-pci#L344-L355\n" ] } ], "source": [ "# Load the TFLite model in TFLite Interpreter\n", "interpreter = tf.lite.Interpreter(model_content=tflite_model)\n", "\n", "# Print the signatures from the converted model\n", "signatures = interpreter.get_signature_list()\n", "print('Signature:', signatures)\n", "\n", "# encode and decode are callable with input as arguments.\n", "encode = interpreter.get_signature_runner('encode')\n", "decode = interpreter.get_signature_runner('decode')\n", "\n", "# 'encoded' and 'decoded' are dictionaries with all outputs from the inference.\n", "input = tf.constant([1, 2, 3], dtype=tf.float32)\n", "print('Input:', input)\n", "encoded = encode(x=input)\n", "print('Encoded result:', encoded)\n", "decoded = decode(x=encoded['encoded_result'])\n", "print('Decoded result:', decoded)" ] }, { "cell_type": "markdown", "id": "81b42e5b", "metadata": { "id": "81b42e5b" }, "source": [ "## Known limitations\n", "\n", "* As TFLite interpreter does not gurantee thread safety, the signature runners\n", " from the same interpreter won't be executed concurrently.\n", "* Support for iOS/Swift is not available yet.\n", "\n" ] }, { "cell_type": "markdown", "id": "3032Iof6QqmJ", "metadata": { "id": "3032Iof6QqmJ" }, "source": [ "## Updates\n", "\n", "* Version 2.7\n", " - The multiple signature feature is implemented.\n", " - All the converter APIs from version two generate signature-enabled\n", " TensorFlow Lite models.\n", "* Version 2.5\n", " - Signature feature is available through the `from_saved_model` converter\n", " API." ] } ], "metadata": { "colab": { "collapsed_sections": [], "name": "Signatures in TensorFlow Lite", "provenance": [], "toc_visible": true }, "id": "a1b42e5b", "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.19" } }, "nbformat": 4, "nbformat_minor": 5 }