{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "g_nWetWWd_ns" }, "source": [ "##### Copyright 2021 The TensorFlow Authors." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "cellView": "form", "execution": { "iopub.execute_input": "2023-11-07T21:48:11.380483Z", "iopub.status.busy": "2023-11-07T21:48:11.380204Z", "iopub.status.idle": "2023-11-07T21:48:11.384610Z", "shell.execute_reply": "2023-11-07T21:48:11.383843Z" }, "id": "2pHVBk_seED1" }, "outputs": [], "source": [ "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# https://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License." ] }, { "cell_type": "markdown", "metadata": { "id": "M7vSdG6sAIQn" }, "source": [ "# TFLite 创作工具" ] }, { "cell_type": "markdown", "metadata": { "id": "fwc5GKHBASdc" }, "source": [ "\n", " \n", " \n", " \n", " \n", "
在 TensorFlow.org 上查看\n", "在 Google Colab 中运行 在 GitHub 上查看源代码\n", " 下载笔记本
" ] }, { "cell_type": "markdown", "metadata": { "id": "9ee074e4" }, "source": [ "TensorFlow Lite Authoring API 提供了一种方式来维护与 TensorFlow Lite 兼容的 `tf.function` 模型。\n" ] }, { "cell_type": "markdown", "metadata": { "id": "UaWdLA3fQDK2" }, "source": [ "## 安装" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2023-11-07T21:48:11.388545Z", "iopub.status.busy": "2023-11-07T21:48:11.388059Z", "iopub.status.idle": "2023-11-07T21:48:13.929958Z", "shell.execute_reply": "2023-11-07T21:48:13.929152Z" }, "id": "DWjLcy2CvgxH" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2023-11-07 21:48:11.853226: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n", "2023-11-07 21:48:11.853276: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n", "2023-11-07 21:48:11.855004: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n" ] } ], "source": [ "import tensorflow as tf" ] }, { "cell_type": "markdown", "metadata": { "id": "CmkXJRDj5hTi" }, "source": [ "## TensorFlow 到 TensorFlow Lite 的兼容性问题\n", "\n", "如果您想在设备端使用您的 TF 模型，您需要将其转换为 TFLite 模型，以便从 TFLite 解释器使用它。在转换过程中，您可能会遇到兼容性错误，因为 TFLite 内置运算集不支持 TensorFlow 运算。\n", "\n", "这是一个令人讨厌的问题。如何能够更早地检测到该问题呢，比如在模型创作时间？\n", "\n", "请注意，以下代码将在调用 `converter.convert()` 时失败。\n" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2023-11-07T21:48:13.934410Z", "iopub.status.busy": "2023-11-07T21:48:13.933937Z", "iopub.status.idle": "2023-11-07T21:48:16.233373Z", "shell.execute_reply": "2023-11-07T21:48:16.232536Z" }, "id": "LHKqKFm5OvyQ" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "result = [1.]\n" ] } ], "source": [ "@tf.function(input_signature=[\n", " tf.TensorSpec(shape=[None], dtype=tf.float32)\n", "])\n", "def f(x):\n", " return tf.cosh(x)\n", "\n", "# Evaluate the tf.function\n", "result = f(tf.constant([0.0]))\n", "print (f\"result = {result}\")" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2023-11-07T21:48:16.236993Z", "iopub.status.busy": "2023-11-07T21:48:16.236679Z", "iopub.status.idle": "2023-11-07T21:48:16.335168Z", "shell.execute_reply": "2023-11-07T21:48:16.334063Z" }, "id": "BS5bOoD50zaU" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Got an exception: Could not translate MLIR to FlatBuffer. UNKNOWN: /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py:695:1: error: 'tf.Cosh' op is neither a custom op nor a flex op\n", " self._concrete_variable_creation_fn = tracing_compilation.trace_function(\n", "^\n", "/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py:178:1: note: called from\n", " concrete_function = _maybe_define_function(\n", "^\n", "/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py:283:1: note: called from\n", " concrete_function = _create_concrete_function(\n", "^\n", "/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py:310:1: note: called from\n", " traced_func_graph = func_graph_module.func_graph_from_py_func(\n", "^\n", "/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py:1059:1: note: called from\n", " func_outputs = python_func(*func_args, **func_kwargs)\n", "^\n", "/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py:598:1: note: called from\n", " out = weak_wrapped_fn().__wrapped__(*args, **kwds)\n", "^\n", "/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/autograph_util.py:41:1: note: called from\n", " return api.converted_call(\n", "^\n", "/tmpfs/tmp/ipykernel_34429/885400331.py:5:1: note: called from\n", "/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/weak_tensor_ops.py:88:1: note: called from\n", " return op(*args, **kwargs)\n", "^\n", "/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/gen_math_ops.py:2612:1: note: called from\n", " _, _, _op, _outputs = _op_def_library._apply_op_helper(\n", "^\n", "/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py:695:1: note: Error code: ERROR_NEEDS_FLEX_OPS\n", " self._concrete_variable_creation_fn = tracing_compilation.trace_function(\n", "^\n", ":0: error: failed while converting: 'main': \n", "Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select \n", "TF Select ops: Cosh\n", "Details:\n", "\ttf.Cosh(tensor) -> (tensor) : {device = \"\"}\n", "\n", "\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2023-11-07 21:48:16.310019: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:378] Ignored output_format.\n", "2023-11-07 21:48:16.310058: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:381] Ignored drop_control_dependency.\n", "Summary on the non-converted ops:\n", "---------------------------------\n", " * Accepted dialects: tfl, builtin, func\n", " * Non-Converted Ops: 1, Total Ops 4, % non-converted = 25.00 %\n", " * 1 TF ops\n", "\n", "\n", "\n", "\n", "- tf.Cosh: 1 occurrences (f32: 1)\n", "loc(fused[\"Cosh:\", callsite(\"Cosh\"(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py\":695:1) at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py\":178:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py\":283:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py\":310:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py\":1059:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/pyt" ] }, { "name": "stderr", "output_type": "stream", "text": [ "hon/eager/polymorphic_function/polymorphic_function.py\":598:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/autograph_util.py\":41:1 at callsite(\"/tmpfs/tmp/ipykernel_34429/885400331.py\":5:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/weak_tensor_ops.py\":88:1 at \"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/gen_math_ops.py\":2612:1)))))))))]): error: 'tf.Cosh' op is neither a custom op nor a flex op\n", "error: failed while converting: 'main': \n", "Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select \n", "TF Select ops: Cosh\n", "Details:\n", "\ttf.Cosh(tensor) -> (tensor) : {device = \"\"}\n", "\n" ] } ], "source": [ "# Convert the tf.function\n", "converter = tf.lite.TFLiteConverter.from_concrete_functions(\n", " [f.get_concrete_function()], f)\n", "try:\n", " fb_model = converter.convert()\n", "except Exception as e:\n", " print(f\"Got an exception: {e}\")" ] }, { "cell_type": "markdown", "metadata": { "id": "eLU0Y9V8g_Wk" }, "source": [ "## 简单的目标感知创作用法\n", "\n", "我们引入了 Authoring API 来检测模型创作期间的 TensorFlow Lite 兼容性问题。\n", "\n", "您只需添加 `@tf.lite.experimental.authoring.compatible` 修饰器来封装您的 `tf.function` 模型，以检查 TFLite 兼容性。\n", "\n", "之后，当您评估模型时，将自动检查兼容性。" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2023-11-07T21:48:16.339657Z", "iopub.status.busy": "2023-11-07T21:48:16.339378Z", "iopub.status.idle": "2023-11-07T21:48:16.450614Z", "shell.execute_reply": "2023-11-07T21:48:16.449669Z" }, "id": "zVSh6VCDhbPz" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "COMPATIBILITY WARNING: op 'tf.Cosh' require(s) \"Select TF Ops\" for model conversion for TensorFlow Lite. https://www.tensorflow.org/lite/guide/ops_select\n", "Op: tf.Cosh\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/gen_math_ops.py:2612\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/weak_tensor_ops.py:88\n", " - /tmpfs/tmp/ipykernel_34429/885400331.py:5\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/autograph_util.py:41\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py:598\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py:1059\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py:310\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py:283\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py:178\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py:695\n", "\n", "result = [1.]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2023-11-07 21:48:16.415556: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:378] Ignored output_format.\n", "2023-11-07 21:48:16.415597: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:381] Ignored drop_control_dependency.\n", "Summary on the non-converted ops:\n", "---------------------------------\n", " * Accepted dialects: tfl, builtin, func\n", " * Non-Converted Ops: 1, Total Ops 4, % non-converted = 25.00 %\n", " * 1 TF ops\n", "\n", "\n", "\n", "\n", "- tf.Cosh: 1 occurrences (f32: 1)\n", "loc(fused[\"Cosh:\", callsite(\"Cosh\"(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py\":695:1) at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py\":178:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py\":283:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py\":310:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py\":1059:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py\":598:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/autograph_util.py\":41:1 at callsite(\"/tmpfs/tmp/ipykernel_34429/885400331.py\":5:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/weak_tensor_ops.py\":88:1 at \"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/gen_math_ops.py\":2612:1)))))))))]): error: 'tf.Cosh' op is neither a custom op nor a flex op\n", "error: failed while converting: 'main': \n", "Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select \n", "TF Select ops: Cosh\n", "Details:\n", "\ttf.Cosh(tensor) -> (tensor) : {device = \"\"}\n", "\n" ] } ], "source": [ "@tf.lite.experimental.authoring.compatible\n", "@tf.function(input_signature=[\n", " tf.TensorSpec(shape=[None], dtype=tf.float32)\n", "])\n", "def f(x):\n", " return tf.cosh(x)\n", "\n", "# Evaluate the tf.function\n", "result = f(tf.constant([0.0]))\n", "print (f\"result = {result}\")\n" ] }, { "cell_type": "markdown", "metadata": { "id": "ZWkBEqv-eUwV" }, "source": [ "如果发现任何 TensorFlow Lite 兼容性问题，它将显示 `COMPATIBILITY WARNING` 或 `COMPATIBILITY ERROR` 以及有问题的运算的确切位置。在本例中，它显示了 tf.unction 模型中 `tf.Cosh` 运算的位置。\n", "\n", "您还可以使用 `.get_compatibility_log()` 方法检查兼容性日志。" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2023-11-07T21:48:16.454979Z", "iopub.status.busy": "2023-11-07T21:48:16.454228Z", "iopub.status.idle": "2023-11-07T21:48:16.458718Z", "shell.execute_reply": "2023-11-07T21:48:16.458019Z" }, "id": "irwO2qdv2RPA" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "compatibility_log = COMPATIBILITY WARNING: op 'tf.Cosh' require(s) \"Select TF Ops\" for model conversion for TensorFlow Lite. https://www.tensorflow.org/lite/guide/ops_select\n", "Op: tf.Cosh\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/gen_math_ops.py:2612\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/weak_tensor_ops.py:88\n", " - /tmpfs/tmp/ipykernel_34429/885400331.py:5\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/autograph_util.py:41\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py:598\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py:1059\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py:310\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py:283\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py:178\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py:695\n", "\n" ] } ], "source": [ "compatibility_log = '\\n'.join(f.get_compatibility_log())\n", "print (f\"compatibility_log = {compatibility_log}\")" ] }, { "cell_type": "markdown", "metadata": { "id": "-LTVE00CiqpS" }, "source": [ "## 引发不兼容性异常\n", "\n", "您可以为 `@tf.lite.experimental.authoring.compatible` 修饰器提供一个选项。当您尝试评估经过修饰的模型时，`raise_exception` 选项会引发异常。" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2023-11-07T21:48:16.462051Z", "iopub.status.busy": "2023-11-07T21:48:16.461778Z", "iopub.status.idle": "2023-11-07T21:48:16.561868Z", "shell.execute_reply": "2023-11-07T21:48:16.560774Z" }, "id": "YfPfOJm5jST4" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "COMPATIBILITY WARNING: op 'tf.Cosh' require(s) \"Select TF Ops\" for model conversion for TensorFlow Lite. https://www.tensorflow.org/lite/guide/ops_select\n", "Op: tf.Cosh\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/gen_math_ops.py:2612\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/weak_tensor_ops.py:88\n", " - /tmpfs/tmp/ipykernel_34429/885400331.py:5\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/autograph_util.py:41\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py:598\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py:1059\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py:310\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py:283\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py:178\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py:695\n", "\n", "Got an exception: CompatibilityException at \n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2023-11-07 21:48:16.539202: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:378] Ignored output_format.\n", "2023-11-07 21:48:16.539249: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:381] Ignored drop_control_dependency.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "Summary on the non-converted ops:\n", "---------------------------------\n", " * Accepted dialects: tfl, builtin, func\n", " * Non-Converted Ops: 1, Total Ops 4, % non-converted = 25.00 %\n", " * 1 TF ops\n", "\n", "\n", "\n", "\n", "- tf.Cosh: 1 occurrences (f32: 1)\n", "loc(fused[\"Cosh:\", callsite(\"Cosh\"(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py\":695:1) at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py\":178:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py\":283:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py\":310:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py\":1059:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py\":598:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/autograph_util.py\":41:1 at callsite(\"/tmpfs/tmp/ipykernel_34429/885400331.py\":5:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/weak_tensor_ops.py\":88:1 at \"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/gen_math_ops.py\":2612:1)))))))))]): error: 'tf.Cosh' op is neither a custom op nor a flex op\n", "error: failed while converting: 'main': \n", "Some ops are not supported by the native TFLite runtime, you can enable TF kernels fallback using TF Select. See instructions: https://www.tensorflow.org/lite/guide/ops_select \n", "TF Select ops: Cosh\n", "Details:\n", "\ttf.Cosh(tensor) -> (tensor) : {device = \"\"}\n", "\n" ] } ], "source": [ "@tf.lite.experimental.authoring.compatible(raise_exception=True)\n", "@tf.function(input_signature=[\n", " tf.TensorSpec(shape=[None], dtype=tf.float32)\n", "])\n", "def f(x):\n", " return tf.cosh(x)\n", "\n", "# Evaluate the tf.function\n", "try:\n", " result = f(tf.constant([0.0]))\n", " print (f\"result = {result}\")\n", "except Exception as e:\n", " print(f\"Got an exception: {e}\")" ] }, { "cell_type": "markdown", "metadata": { "id": "WXywHrR0Xjop" }, "source": [ "## 指定 \"Select TF ops\" 用法\n", "\n", "如果您已了解 [Select Tf op](https://tensorflow.google.cn/lite/guide/ops_select) 用法，可以通过设置 `converter_target_spec` 将其用于 Authoring API。它与您将在 [tf.lite.TFLiteConverter](https://tensorflow.google.cn/api_docs/python/tf/lite/TFLiteConverter) API 中使用的对象相同。\n" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "execution": { "iopub.execute_input": "2023-11-07T21:48:16.566402Z", "iopub.status.busy": "2023-11-07T21:48:16.566105Z", "iopub.status.idle": "2023-11-07T21:48:16.700363Z", "shell.execute_reply": "2023-11-07T21:48:16.699336Z" }, "id": "B483OwYQYG8A" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "result = [1.]\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2023-11-07 21:48:16.665435: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:378] Ignored output_format.\n", "2023-11-07 21:48:16.665473: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:381] Ignored drop_control_dependency.\n", "Summary on the non-converted ops:\n", "---------------------------------\n", " * Accepted dialects: tfl, builtin, func\n", " * Non-Converted Ops: 1, Total Ops 4, % non-converted = 25.00 %\n", " * 1 TF ops\n", "\n", "\n", "\n", "\n", "- tf.Cosh: 1 occurrences (f32: 1)\n", "2023-11-07 21:48:16.680080: W tensorflow/compiler/mlir/lite/flatbuffer_export.cc:2921] TFLite interpreter needs to link Flex delegate in order to run the model since it contains the following Select TFop(s):\n", "Flex ops: FlexCosh\n", "Details:\n", "\ttf.Cosh(tensor) -> (tensor) : {device = \"\"}\n", "See instructions: https://www.tensorflow.org/lite/guide/ops_select\n" ] } ], "source": [ "target_spec = tf.lite.TargetSpec()\n", "target_spec.supported_ops = [\n", " tf.lite.OpsSet.TFLITE_BUILTINS,\n", " tf.lite.OpsSet.SELECT_TF_OPS,\n", "]\n", "@tf.lite.experimental.authoring.compatible(converter_target_spec=target_spec, raise_exception=True)\n", "@tf.function(input_signature=[\n", " tf.TensorSpec(shape=[None], dtype=tf.float32)\n", "])\n", "def f(x):\n", " return tf.cosh(x)\n", "\n", "# Evaluate the tf.function\n", "result = f(tf.constant([0.0]))\n", "print (f\"result = {result}\")\n" ] }, { "cell_type": "markdown", "metadata": { "id": "mtept13-C6uD" }, "source": [ "## 检查 GPU 兼容性\n", "\n", "如果您希望确保您的模型与 TensorFlow Lite 的 [GPU 委托](https://tensorflow.google.cn/lite/performance/gpu)兼容，您可以设置 [tf.lite.TargetSpec](https://tensorflow.google.cn/api_docs/python/tf/lite/TargetSpec) 的 `experimental_supported_backends`。\n", "\n", "以下示例展示如何确保您的模型的 GPU 委托兼容性。请注意，此模型存在兼容性问题，因为它使用带有 tf.slice 算子和不受支持的 tf.cosh 算子的二维张量。您将看到两个带有位置信息的 `COMPATIBILITY WARNING`。" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "execution": { "iopub.execute_input": "2023-11-07T21:48:16.703970Z", "iopub.status.busy": "2023-11-07T21:48:16.703666Z", "iopub.status.idle": "2023-11-07T21:48:16.866907Z", "shell.execute_reply": "2023-11-07T21:48:16.865910Z" }, "id": "_DzHV3KVC0T0" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "'tfl.slice' op is not GPU compatible: SLICE supports for 3 or 4 dimensional tensors only, but node has 2 dimensional tensors.\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/util/dispatch.py:1260\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/util/traceback_utils.py:150\n", " - /tmpfs/tmp/ipykernel_34429/3833138856.py:13\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/autograph_util.py:41\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py:598\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py:1059\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py:310\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py:283\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py:178\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py:695\n", "\n", "'tf.Cosh' op is not GPU compatible: Not supported custom op FlexCosh\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/gen_math_ops.py:2612\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/weak_tensor_ops.py:88\n", " - /tmpfs/tmp/ipykernel_34429/3833138856.py:12\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/autograph_util.py:41\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py:598\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py:1059\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py:310\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py:283\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py:178\n", " - /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py:695\n", "\n", "COMPATIBILITY WARNING: op 'tf.Cosh, tfl.slice' aren't compatible with TensorFlow Lite GPU delegate. https://www.tensorflow.org/lite/performance/gpu\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2023-11-07 21:48:16.820800: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:378] Ignored output_format.\n", "2023-11-07 21:48:16.820846: W tensorflow/compiler/mlir/lite/python/tf_tfl_flatbuffer_helpers.cc:381] Ignored drop_control_dependency.\n", "Summary on the non-converted ops:\n", "---------------------------------\n", " * Accepted dialects: tfl, builtin, func\n", " * Non-Converted Ops: 2, Total Ops 7, % non-converted = 28.57 %\n", " * 1 ARITH ops, 1 TF ops\n", "\n", "- arith.constant: 1 occurrences (i32: 1)\n", "\n", "\n", "\n", "- tf.Cosh: 1 occurrences (f32: 1)\n", " (f32: 1)\n", " (f32: 1)\n", "2023-11-07 21:48:16.837705: W tensorflow/compiler/mlir/lite/flatbuffer_export.cc:2921] TFLite interpreter needs to link Flex delegate in order to run the model since it contains the following Select TFop(s):\n", "Flex ops: FlexCosh\n", "Details:\n", "\ttf.Cosh(tensor<4x4xf32>) -> (tensor<4x4xf32>) : {device = \"\"}\n", "See instructions: https://www.tensorflow.org/lite/guide/ops_select\n", "loc(fused[\"Cosh:\", callsite(\"Cosh\"(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py\":695:1) at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py\":178:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py\":283:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py\":310:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py\":1059:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py\":598:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/autograph_util.py\":41:1 at callsite(\"/tmpfs/tmp/ipykernel_34429/3833138856.py\":12:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/weak_tensor_ops.py\":88:1 at \"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/ops/gen_math_ops.py\":2612:1)))))))))]): error: 'tf.Cosh' op is not GPU compatible: Not supported custom op FlexCosh\n", "loc(fused[\"Slice:\", callsite(\"Slice\"(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py\":695:1) at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py\":178:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py\":283:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compilation.py\":310:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/framework/func_graph.py\":1059:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py\":598:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/eager/polymorphic_function/autograph_util.py\":41:1 at callsite(\"/tmpfs/tmp/ipykernel_34429/3833138856.py\":13:1 at callsite(\"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/util/traceback_utils.py\":150:1 at \"/tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow/python/util/dispatch.py\":1260:1)))))))))]): error: 'tfl.slice' op is not GPU compatible: SLICE supports for 3 or 4 dimensional tensors only, but node has 2 dimensional tensors.\n" ] } ], "source": [ "target_spec = tf.lite.TargetSpec()\n", "target_spec.supported_ops = [\n", " tf.lite.OpsSet.TFLITE_BUILTINS,\n", " tf.lite.OpsSet.SELECT_TF_OPS,\n", "]\n", "target_spec.experimental_supported_backends = [\"GPU\"]\n", "@tf.lite.experimental.authoring.compatible(converter_target_spec=target_spec)\n", "@tf.function(input_signature=[\n", " tf.TensorSpec(shape=[4, 4], dtype=tf.float32)\n", "])\n", "def func(x):\n", " y = tf.cosh(x)\n", " return y + tf.slice(x, [1, 1], [1, 1])\n", "\n", "result = func(tf.ones(shape=(4,4), dtype=tf.float32))" ] }, { "cell_type": "markdown", "metadata": { "id": "JvLEtCWRvvy8" }, "source": [ "## 阅读更多\n", "\n", "有关更多信息，请参阅：\n", "\n", "- [tf.function](https://tensorflow.google.cn/api_docs/python/tf/function) API 文档\n", "- [使用 tf.function 提升性能](https://tensorflow.google.cn/guide/function)\n", "- [TensorFlow Lite Converter](https://tensorflow.google.cn/lite/models/convert)\n", "- [TensorFlow Lite Model Analyzer](https://tensorflow.google.cn/lite/guide/model_analyzer)\n", "- [TensorFlow Lite GPU 委托](https://tensorflow.google.cn/lite/performance/gpu)" ] } ], "metadata": { "colab": { "collapsed_sections": [], "name": "authoring.ipynb", "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.18" } }, "nbformat": 4, "nbformat_minor": 0 }