"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"file = tf.keras.utils.get_file(\n",
" \"grace_hopper.jpg\",\n",
" \"https://storage.googleapis.com/download.tensorflow.org/example_images/grace_hopper.jpg\")\n",
"img = tf.keras.utils.load_img(file, target_size=[224, 224])\n",
"plt.imshow(img)\n",
"plt.axis('off')\n",
"x = tf.keras.utils.img_to_array(img)\n",
"x = tf.keras.applications.mobilenet.preprocess_input(\n",
" x[tf.newaxis,...])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "sqVcFL10JkF0"
},
"source": [
"我们会使用 Grace Hopper 的一张照片作为运行示例，并使用一个预先训练的 Keras 图像分类模型，因为它简单易用。您也可以使用自定义模型，后文会作详细介绍。"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:36.223136Z",
"iopub.status.busy": "2022-12-14T22:18:36.222908Z",
"iopub.status.idle": "2022-12-14T22:18:36.235883Z",
"shell.execute_reply": "2022-12-14T22:18:36.235306Z"
},
"id": "JhVecdzJTsKE"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Downloading data from https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\r",
" 8192/10484 [======================>.......] - ETA: 0s"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
"10484/10484 [==============================] - 0s 0us/step\n"
]
}
],
"source": [
"labels_path = tf.keras.utils.get_file(\n",
" 'ImageNetLabels.txt',\n",
" 'https://storage.googleapis.com/download.tensorflow.org/data/ImageNetLabels.txt')\n",
"imagenet_labels = np.array(open(labels_path).read().splitlines())"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:36.239238Z",
"iopub.status.busy": "2022-12-14T22:18:36.238722Z",
"iopub.status.idle": "2022-12-14T22:18:41.144538Z",
"shell.execute_reply": "2022-12-14T22:18:41.143775Z"
},
"id": "aEHSYjW6JZHV"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/mobilenet/mobilenet_1_0_224_tf.h5\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\r",
" 8192/17225924 [..............................] - ETA: 0s"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
" 8077312/17225924 [=============>................] - ETA: 0s"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
"14557184/17225924 [========================>.....] - ETA: 0s"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
"17225924/17225924 [==============================] - 0s 0us/step\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"Result before saving:\n",
" ['military uniform' 'bow tie' 'suit' 'bearskin' 'pickelhaube']\n"
]
}
],
"source": [
"pretrained_model = tf.keras.applications.MobileNet()\n",
"result_before_save = pretrained_model(x)\n",
"\n",
"decoded = imagenet_labels[np.argsort(result_before_save)[0,::-1][:5]+1]\n",
"\n",
"print(\"Result before saving:\\n\", decoded)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "r4KIsQDZJ5PS"
},
"source": [
"对该图像的顶部预测是“军服”。"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:41.148123Z",
"iopub.status.busy": "2022-12-14T22:18:41.147478Z",
"iopub.status.idle": "2022-12-14T22:18:49.049817Z",
"shell.execute_reply": "2022-12-14T22:18:49.049017Z"
},
"id": "8nfznDmHCW6F"
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING:absl:Found untraced functions such as _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op, _jit_compiled_convolution_op while saving (showing 5 of 28). These functions will not be directly callable after loading.\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"INFO:tensorflow:Assets written to: /tmpfs/tmp/tmpe_362t4g/mobilenet/1/assets\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:tensorflow:Assets written to: /tmpfs/tmp/tmpe_362t4g/mobilenet/1/assets\n"
]
}
],
"source": [
"mobilenet_save_path = os.path.join(tmpdir, \"mobilenet/1/\")\n",
"tf.saved_model.save(pretrained_model, mobilenet_save_path)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "pyX-ETE3wX63"
},
"source": [
"保存路径遵循 TensorFlow Serving 使用的惯例，路径的最后一个部分（此处为 `1/`）是模型的版本号——它可以让 Tensorflow Serving 之类的工具推断相对新鲜度。\n",
"\n",
"您可以使用 `tf.saved_model.load` 将 SavedModel 加载回 Python，并查看 Admiral Hopper 的图像是如何分类的。"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:49.057974Z",
"iopub.status.busy": "2022-12-14T22:18:49.057689Z",
"iopub.status.idle": "2022-12-14T22:18:51.473591Z",
"shell.execute_reply": "2022-12-14T22:18:51.472894Z"
},
"id": "NP2UpVFRV7N_"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"['serving_default']\n"
]
}
],
"source": [
"loaded = tf.saved_model.load(mobilenet_save_path)\n",
"print(list(loaded.signatures.keys())) # [\"serving_default\"]"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "K5srGzowfWff"
},
"source": [
"导入的签名总是会返回字典。要自定义签名名称和输出字典键，请参阅[在导出过程中指定签名](#specifying_signatures_during_export)。"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:51.477076Z",
"iopub.status.busy": "2022-12-14T22:18:51.476817Z",
"iopub.status.idle": "2022-12-14T22:18:51.480462Z",
"shell.execute_reply": "2022-12-14T22:18:51.479874Z"
},
"id": "ChFLpegYfQGR"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'predictions': TensorSpec(shape=(None, 1000), dtype=tf.float32, name='predictions')}\n"
]
}
],
"source": [
"infer = loaded.signatures[\"serving_default\"]\n",
"print(infer.structured_outputs)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "cJYyZnptfuru"
},
"source": [
"从 SavedModel 运行推断会产生与原始模型相同的结果。"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:51.484015Z",
"iopub.status.busy": "2022-12-14T22:18:51.483389Z",
"iopub.status.idle": "2022-12-14T22:18:51.928249Z",
"shell.execute_reply": "2022-12-14T22:18:51.927505Z"
},
"id": "9WjGEaS3XfX7"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Result after saving and loading:\n",
" ['military uniform' 'bow tie' 'suit' 'bearskin' 'pickelhaube']\n"
]
}
],
"source": [
"labeling = infer(tf.constant(x))[pretrained_model.output_names[0]]\n",
"\n",
"decoded = imagenet_labels[np.argsort(labeling)[0,::-1][:5]+1]\n",
"\n",
"print(\"Result after saving and loading:\\n\", decoded)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "SJEkdXjTWbtl"
},
"source": [
"## 在 TensorFlow Serving 中运行 SavedModel\n",
"\n",
"可以通过 Python 使用 SavedModel（下文中有详细介绍），但是，生产环境通常会使用专门服务进行推理，而不会运行 Python 代码。使用 TensorFlow Serving 时，这很容易从 SavedModel 进行设置。\n",
"\n",
"请参阅 [TensorFlow Serving REST 教程](https://tensorflow.google.cn/tfx/tutorials/serving/rest_simple)了解端到端 tensorflow-serving 示例。"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Bi0ILzu1XdWw"
},
"source": [
"## 磁盘上的 SavedModel 格式\n",
"\n",
"SavedModel 是一个包含序列化签名和运行这些签名所需的状态的目录，其中包括变量值和词汇表。\n"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:51.932113Z",
"iopub.status.busy": "2022-12-14T22:18:51.931844Z",
"iopub.status.idle": "2022-12-14T22:18:52.116442Z",
"shell.execute_reply": "2022-12-14T22:18:52.115344Z"
},
"id": "6u3YZuYZXyTO"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"assets\tfingerprint.pb\tsaved_model.pb\tvariables\r\n"
]
}
],
"source": [
"!ls {mobilenet_save_path}"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "ple4X5utX8ue"
},
"source": [
"`saved_model.pb` 文件用于存储实际 TensorFlow 程序或模型，以及一组已命名的签名——每个签名标识一个接受张量输入和产生张量输出的函数。\n",
"\n",
"SavedModel 可能包含模型的多个变体（多个 `v1.MetaGraphDefs`，通过 `saved_model_cli` 的 `--tag_set` 标记进行标识），但这种情况很少见。可以为模型创建多个变体的 API 包括 [tf.Estimator.experimental_export_all_saved_models](https://tensorflow.google.cn/api_docs/python/tf/estimator/Estimator#experimental_export_all_saved_models) 和 TensorFlow 1.x 中的 `tf.saved_model.Builder`。"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:52.120743Z",
"iopub.status.busy": "2022-12-14T22:18:52.120443Z",
"iopub.status.idle": "2022-12-14T22:18:54.650648Z",
"shell.execute_reply": "2022-12-14T22:18:54.649521Z"
},
"id": "Pus0dOYTYXbI"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"2022-12-14 22:18:53.272172: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory\r\n",
"2022-12-14 22:18:53.272277: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory\r\n",
"2022-12-14 22:18:53.272288: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.\r\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"The given SavedModel MetaGraphDef contains SignatureDefs with the following keys:\r\n",
"SignatureDef key: \"__saved_model_init_op\"\r\n",
"SignatureDef key: \"serving_default\"\r\n"
]
}
],
"source": [
"!saved_model_cli show --dir {mobilenet_save_path} --tag_set serve"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "eALHpGvRZOhk"
},
"source": [
"`variables` 目录包含一个标准训练检查点（参阅[训练检查点指南](./checkpoint.ipynb)）。"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:54.655192Z",
"iopub.status.busy": "2022-12-14T22:18:54.654907Z",
"iopub.status.idle": "2022-12-14T22:18:54.833368Z",
"shell.execute_reply": "2022-12-14T22:18:54.832539Z"
},
"id": "EDYqhDlNZAC2"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"variables.data-00000-of-00001 variables.index\r\n"
]
}
],
"source": [
"!ls {mobilenet_save_path}/variables"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "VKmaZQpHahGh"
},
"source": [
"`assets` 目录包含 TensorFlow 计算图使用的文件，例如，用于初始化词汇表的文本文件。本例中没有使用这种文件。\n",
"\n",
"SavedModel 可能有一个用于保存 TensorFlow 计算图未使用的任何文件的 `assets.extra` 目录，例如，为使用者提供的关于如何处理 SavedModel 的信息。TensorFlow 本身并不会使用此目录。"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "zIceoF_CYmaF"
},
"source": [
"## 保存自定义模型\n",
"\n",
"`tf.saved_model.save` 支持保存 `tf.Module` 对象及其子类，如 `tf.keras.Layer` 和 `tf.keras.Model`。\n",
"\n",
"我们来看一个保存和恢复 `tf.Module` 的示例。\n"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:54.837435Z",
"iopub.status.busy": "2022-12-14T22:18:54.837155Z",
"iopub.status.idle": "2022-12-14T22:18:54.850849Z",
"shell.execute_reply": "2022-12-14T22:18:54.850263Z"
},
"id": "6EPvKiqXMm3d"
},
"outputs": [],
"source": [
"class CustomModule(tf.Module):\n",
"\n",
" def __init__(self):\n",
" super(CustomModule, self).__init__()\n",
" self.v = tf.Variable(1.)\n",
"\n",
" @tf.function\n",
" def __call__(self, x):\n",
" print('Tracing with', x)\n",
" return x * self.v\n",
"\n",
" @tf.function(input_signature=[tf.TensorSpec([], tf.float32)])\n",
" def mutate(self, new_v):\n",
" self.v.assign(new_v)\n",
"\n",
"module = CustomModule()"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "J4FcP-Co3Fnw"
},
"source": [
"当您保存 `tf.Module` 时，任何 `tf.Variable` 特性、`tf.function` 装饰的方法以及通过递归遍历找到的 `tf.Module` 都会得到保存。（参阅[检查点教程](./checkpoint.ipynb)，了解此递归便利的详细信息。）但是，所有 Python 特性、函数和数据都会丢失。也就是说，当您保存 `tf.function` 时，不会保存 Python 代码。\n",
"\n",
"如果不保存 Python 代码，SavedModel 如何知道怎样恢复函数？\n",
"\n",
"简单地说，`tf.function` 的工作原理是，通过跟踪 Python 代码来生成 ConcreteFunction（一个可调用的 `tf.Graph` 包装器）。当您保存 `tf.function` 时，实际上保存的是 `tf.function` 的 ConcreteFunction 缓存。\n",
"\n",
"要详细了解 `tf.function` 与 ConcreteFunction 之间的关系，请参阅 [tf.function 指南](function.ipynb)。"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:54.854422Z",
"iopub.status.busy": "2022-12-14T22:18:54.853998Z",
"iopub.status.idle": "2022-12-14T22:18:54.988312Z",
"shell.execute_reply": "2022-12-14T22:18:54.987624Z"
},
"id": "85PUO9iWH7xn"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Tracing with Tensor(\"x:0\", shape=(), dtype=float32)\n",
"Saving model...\n",
"Tracing with Tensor(\"x:0\", shape=(), dtype=float32)\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"INFO:tensorflow:Assets written to: /tmpfs/tmp/tmpe_362t4g/module_no_signatures/assets\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:tensorflow:Assets written to: /tmpfs/tmp/tmpe_362t4g/module_no_signatures/assets\n"
]
}
],
"source": [
"module_no_signatures_path = os.path.join(tmpdir, 'module_no_signatures')\n",
"module(tf.constant(0.))\n",
"print('Saving model...')\n",
"tf.saved_model.save(module, module_no_signatures_path)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "2ujwmMQg7OUo"
},
"source": [
"## 加载和使用自定义模型"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "QpxQy5Eb77qJ"
},
"source": [
"在 Python 中加载 SavedModel 时，所有 `tf.Variable` 特性、`tf.function` 装饰方法和 `tf.Module` 都会按照与原始保存的 `tf.Module` 相同对象结构进行恢复。"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:54.992059Z",
"iopub.status.busy": "2022-12-14T22:18:54.991505Z",
"iopub.status.idle": "2022-12-14T22:18:55.039402Z",
"shell.execute_reply": "2022-12-14T22:18:55.038758Z"
},
"id": "EMASjADPxPso"
},
"outputs": [],
"source": [
"imported = tf.saved_model.load(module_no_signatures_path)\n",
"assert imported(tf.constant(3.)).numpy() == 3\n",
"imported.mutate(tf.constant(2.))\n",
"assert imported(tf.constant(3.)).numpy() == 6"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "CDiauvb_99uk"
},
"source": [
"由于没有保存 Python 代码，所以使用新输入签名调用 `tf.function` 会失败：\n",
"\n",
"```python\n",
"imported(tf.constant([3.]))\n",
"```\n",
"\n",
"ValueError: Could not find matching function to call for canonicalized inputs ((<tf.Tensor 'args_0:0' shape=(1,) dtype=float32>,), {}). Only existing signatures are [((TensorSpec(shape=(), dtype=tf.float32, name=u'x'),), {})].\n",
" "
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "4Vsva3UZ-2sf"
},
"source": [
"### 基本微调\n",
"\n",
"可以使用变量对象，还可以通过导入的函数向后传播。对于简单情形，这足以支持 SavedModel 的微调（即重新训练）。"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:55.043219Z",
"iopub.status.busy": "2022-12-14T22:18:55.042639Z",
"iopub.status.idle": "2022-12-14T22:18:55.048448Z",
"shell.execute_reply": "2022-12-14T22:18:55.047801Z"
},
"id": "PEkQNarJ-7nT"
},
"outputs": [],
"source": [
"optimizer = tf.keras.optimizers.SGD(0.05)\n",
"\n",
"def train_step():\n",
" with tf.GradientTape() as tape:\n",
" loss = (10. - imported(tf.constant(2.))) ** 2\n",
" variables = tape.watched_variables()\n",
" grads = tape.gradient(loss, variables)\n",
" optimizer.apply_gradients(zip(grads, variables))\n",
" return loss"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:55.051798Z",
"iopub.status.busy": "2022-12-14T22:18:55.051213Z",
"iopub.status.idle": "2022-12-14T22:18:55.215627Z",
"shell.execute_reply": "2022-12-14T22:18:55.214853Z"
},
"id": "p41NM6fF---3"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"loss=36.00 v=3.20\n",
"loss=12.96 v=3.92\n",
"loss=4.67 v=4.35\n",
"loss=1.68 v=4.61\n",
"loss=0.60 v=4.77\n",
"loss=0.22 v=4.86\n",
"loss=0.08 v=4.92\n",
"loss=0.03 v=4.95\n",
"loss=0.01 v=4.97\n",
"loss=0.00 v=4.98\n"
]
}
],
"source": [
"for _ in range(10):\n",
" # \"v\" approaches 5, \"loss\" approaches 0\n",
" print(\"loss={:.2f} v={:.2f}\".format(train_step(), imported.v.numpy()))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "XuXtkHSD_KSW"
},
"source": [
"### 一般微调\n",
"\n",
"与普通 `__call__` 相比，Keras 的 SavedModel 提供了[更多详细信息](https://github.com/tensorflow/community/blob/master/rfcs/20190509-keras-saved-model.md#serialization-details)来解决更复杂的微调情形。TensorFlow Hub 建议在共享的 SavedModel 中提供以下详细信息（如果适用），以便进行微调：\n",
"\n",
"- 如果模型使用随机失活，或者是训练与推理之间的前向传递不同的另一种技术（如批次归一化），则 `__call__` 方法会获取一个可选的 Python 值 `training=` 参数。该参数的默认值为 `False`，但可将其设置为 `True`。\n",
"- 对于变量的对应列表，除了 `__call__` 特性，还有 `.variable` 和 `.trainable_variable` 特性。在微调过程中，`.trainable_variables` 省略了一个变量，该变量原本可训练，但打算将其冻结。\n",
"- 对于 Keras 等将权重正则化项表示为层或子模型特性的框架，还有一个 `.regularization_losses` 特性。它包含一个零参数函数的列表，这些函数的值应加到总损失中。\n",
"\n",
"回到初始 MobileNet 示例，您可以看到一些具体操作："
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:55.219052Z",
"iopub.status.busy": "2022-12-14T22:18:55.218792Z",
"iopub.status.idle": "2022-12-14T22:18:57.608196Z",
"shell.execute_reply": "2022-12-14T22:18:57.607122Z"
},
"id": "Y6EUFdY8_PRD"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"MobileNet has 83 trainable variables: conv1/kernel:0, conv1_bn/gamma:0, conv1_bn/beta:0, conv_dw_1/depthwise_kernel:0, conv_dw_1_bn/gamma:0, ...\n"
]
}
],
"source": [
"loaded = tf.saved_model.load(mobilenet_save_path)\n",
"print(\"MobileNet has {} trainable variables: {}, ...\".format(\n",
" len(loaded.trainable_variables),\n",
" \", \".join([v.name for v in loaded.trainable_variables[:5]])))"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:57.612053Z",
"iopub.status.busy": "2022-12-14T22:18:57.611749Z",
"iopub.status.idle": "2022-12-14T22:18:57.617307Z",
"shell.execute_reply": "2022-12-14T22:18:57.616493Z"
},
"id": "B-mQJ8iP_R0h"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"MobileNet also has 54 non-trainable variables: conv1_bn/moving_mean:0, conv1_bn/moving_variance:0, conv_dw_1_bn/moving_mean:0, ...\n"
]
}
],
"source": [
"trainable_variable_ids = {id(v) for v in loaded.trainable_variables}\n",
"non_trainable_variables = [v for v in loaded.variables\n",
" if id(v) not in trainable_variable_ids]\n",
"print(\"MobileNet also has {} non-trainable variables: {}, ...\".format(\n",
" len(non_trainable_variables),\n",
" \", \".join([v.name for v in non_trainable_variables[:3]])))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "qGlHlbd3_eyO"
},
"source": [
"## 导出时指定签名\n",
"\n",
"TensorFlow Serving 之类的工具和 `saved_model_cli` 可以与 SavedModel 交互。为了帮助这些工具确定要使用的 ConcreteFunction，我们需要指定应用签名。`tf.keras.Model` 会自动指定应用签名，但是，对于自定义模块，我们必须明确声明应用签名。\n",
"\n",
"重要提示：除非您需要使用 Python 将模型导出到 TensorFlow 2.x 之外的环境，否则您不需要明确导出签名。如果您在寻找为特定函数强制输入签名的方式，请参阅 `tf.function` 的 [`input_signature`](https://tensorflow.google.cn/api_docs/python/tf/function#args_1) 参数。\n",
"\n",
"默认情况下，自定义 `tf.Module` 中不会声明签名。"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:57.620717Z",
"iopub.status.busy": "2022-12-14T22:18:57.620482Z",
"iopub.status.idle": "2022-12-14T22:18:57.624069Z",
"shell.execute_reply": "2022-12-14T22:18:57.623303Z"
},
"id": "h-IB5Xa0NxLa"
},
"outputs": [],
"source": [
"assert len(imported.signatures) == 0"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "BiNtaMZSI8Tb"
},
"source": [
"要声明服务上线签名，请使用 `signatures` 关键字参数指定 ConcreteFunction。当指定单个签名时，签名键为 `'serving_default'`，并将保存为常量 `tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY`。"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:57.627311Z",
"iopub.status.busy": "2022-12-14T22:18:57.627054Z",
"iopub.status.idle": "2022-12-14T22:18:57.702542Z",
"shell.execute_reply": "2022-12-14T22:18:57.701895Z"
},
"id": "_pAdgIORR2yH"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Tracing with Tensor(\"x:0\", dtype=float32)\n",
"Tracing with Tensor(\"x:0\", dtype=float32)\n",
"INFO:tensorflow:Assets written to: /tmpfs/tmp/tmpe_362t4g/module_with_signature/assets\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:tensorflow:Assets written to: /tmpfs/tmp/tmpe_362t4g/module_with_signature/assets\n"
]
}
],
"source": [
"module_with_signature_path = os.path.join(tmpdir, 'module_with_signature')\n",
"call = module.__call__.get_concrete_function(tf.TensorSpec(None, tf.float32))\n",
"tf.saved_model.save(module, module_with_signature_path, signatures=call)"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:57.705805Z",
"iopub.status.busy": "2022-12-14T22:18:57.705237Z",
"iopub.status.idle": "2022-12-14T22:18:57.740051Z",
"shell.execute_reply": "2022-12-14T22:18:57.739262Z"
},
"id": "nAzRHR0UT4hv"
},
"outputs": [
{
"data": {
"text/plain": [
"['serving_default']"
]
},
"execution_count": 23,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"imported_with_signatures = tf.saved_model.load(module_with_signature_path)\n",
"list(imported_with_signatures.signatures.keys())\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "_gH91j1IR4tq"
},
"source": [
"要导出多个签名，请将签名键的字典传递给 ConcreteFunction。每个签名键对应一个 ConcreteFunction。"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:57.743620Z",
"iopub.status.busy": "2022-12-14T22:18:57.742985Z",
"iopub.status.idle": "2022-12-14T22:18:57.814057Z",
"shell.execute_reply": "2022-12-14T22:18:57.813397Z"
},
"id": "6VYAiQmLUiox"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"INFO:tensorflow:Assets written to: /tmpfs/tmp/tmpe_362t4g/module_with_multiple_signatures/assets\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:tensorflow:Assets written to: /tmpfs/tmp/tmpe_362t4g/module_with_multiple_signatures/assets\n"
]
}
],
"source": [
"module_multiple_signatures_path = os.path.join(tmpdir, 'module_with_multiple_signatures')\n",
"signatures = {\"serving_default\": call,\n",
" \"array_input\": module.__call__.get_concrete_function(tf.TensorSpec([None], tf.float32))}\n",
"\n",
"tf.saved_model.save(module, module_multiple_signatures_path, signatures=signatures)"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:57.817524Z",
"iopub.status.busy": "2022-12-14T22:18:57.816993Z",
"iopub.status.idle": "2022-12-14T22:18:57.854696Z",
"shell.execute_reply": "2022-12-14T22:18:57.853915Z"
},
"id": "8IPx_0RWEx07"
},
"outputs": [
{
"data": {
"text/plain": [
"['serving_default', 'array_input']"
]
},
"execution_count": 25,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"imported_with_multiple_signatures = tf.saved_model.load(module_multiple_signatures_path)\n",
"list(imported_with_multiple_signatures.signatures.keys())"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "43_Qv2W_DJZZ"
},
"source": [
"默认情况下，输出张量名称非常通用，如 `output_0`。为了控制输出的名称，请修改 `tf.function`，以便返回将输出名称映射到输出的字典。输入的名称来自 Python 函数参数名称。"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:57.858170Z",
"iopub.status.busy": "2022-12-14T22:18:57.857942Z",
"iopub.status.idle": "2022-12-14T22:18:57.942795Z",
"shell.execute_reply": "2022-12-14T22:18:57.942214Z"
},
"id": "ACKPl1X8G1gw"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"INFO:tensorflow:Assets written to: /tmpfs/tmp/tmpe_362t4g/module_with_output_name/assets\n"
]
},
{
"name": "stderr",
"output_type": "stream",
"text": [
"INFO:tensorflow:Assets written to: /tmpfs/tmp/tmpe_362t4g/module_with_output_name/assets\n"
]
}
],
"source": [
"class CustomModuleWithOutputName(tf.Module):\n",
" def __init__(self):\n",
" super(CustomModuleWithOutputName, self).__init__()\n",
" self.v = tf.Variable(1.)\n",
"\n",
" @tf.function(input_signature=[tf.TensorSpec([], tf.float32)])\n",
" def __call__(self, x):\n",
" return {'custom_output_name': x * self.v}\n",
"\n",
"module_output = CustomModuleWithOutputName()\n",
"call_output = module_output.__call__.get_concrete_function(tf.TensorSpec(None, tf.float32))\n",
"module_output_path = os.path.join(tmpdir, 'module_with_output_name')\n",
"tf.saved_model.save(module_output, module_output_path,\n",
" signatures={'serving_default': call_output})"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {
"execution": {
"iopub.execute_input": "2022-12-14T22:18:57.946172Z",
"iopub.status.busy": "2022-12-14T22:18:57.945555Z",
"iopub.status.idle": "2022-12-14T22:18:57.969831Z",
"shell.execute_reply": "2022-12-14T22:18:57.969242Z"
},
"id": "1yGVy4MuH-V0"
},
"outputs": [
{
"data": {
"text/plain": [
"{'custom_output_name': TensorSpec(shape=(), dtype=tf.float32, name='custom_output_name')}"
]
},
"execution_count": 27,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"imported_with_output_name = tf.saved_model.load(module_output_path)\n",
"imported_with_output_name.signatures['serving_default'].structured_outputs"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Co6fDbzw_UnD"
},
"source": [
"## 在 C++ 中加载 SavedModel\n",
"\n",
"SavedModel [加载器](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/cc/saved_model/loader.h)的 C++ 版本提供了一个从路径中加载 SavedModel 的 API，同时允许使用 SessionOption 和 RunOption。您必须指定与计算图相关联的标记才能加载模型。加载的 SavedModel 版本称为 SavedModelBundle，其中包含 MetaGraphDef 以及加载该版本所处的会话。\n",
"\n",
"```C++\n",
"const string export_dir = ...\n",
"SavedModelBundle bundle;\n",
"...\n",
"LoadSavedModel(session_options, run_options, export_dir, {kSavedModelTagTrain},\n",
" &bundle);\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "b33KuyEuAO3Z"
},
"source": [
"$ saved_model_cli show --dir /tmp/saved_model_dir --tag_set serve,gpu \n",
"\n",
"要显示特定 `SignatureDef` 的所有输入和输出 TensorInfo，请将 `SignatureDef` 键传递给 `signature_def` 选项。如果您想知道输入张量的张量键值、dtype 和形状，以便随后执行计算图，这会非常有用。例如：\n",
"\n",
"```\n",
"$ saved_model_cli show --dir \\\n",
"/tmp/saved_model_dir --tag_set serve --signature_def serving_default\n",
"The given SavedModel SignatureDef contains the following input(s):\n",
" inputs['x'] tensor_info:\n",
" dtype: DT_FLOAT\n",
" shape: (-1, 1)\n",
" name: x:0\n",
"The given SavedModel SignatureDef contains the following output(s):\n",
" outputs['y'] tensor_info:\n",
" dtype: DT_FLOAT\n",
" shape: (-1, 1)\n",
" name: y:0\n",
"Method name is: tensorflow/serving/predict\n",
"```\n",
"\n",
"要显示 SavedModel 中的所有可用信息，请使用 `--all` 选项。例如：\n",
"\n",
"$ saved_model_cli show --dir /tmp/saved_model_dir --all<br>MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:<br><br>signature_def['classify_x2_to_y3']:<br> The given SavedModel SignatureDef contains the following input(s):<br> inputs['inputs'] tensor_info:<br> dtype: DT_FLOAT<br> shape: (-1, 1)<br> name: x2:0<br> The given SavedModel SignatureDef contains the following output(s):<br> outputs['scores'] tensor_info:<br> dtype: DT_FLOAT<br> shape: (-1, 1)<br> name: y3:0<br> Method name is: tensorflow/serving/classify<br><br>...<br><br>signature_def['serving_default']:<br> The given SavedModel SignatureDef contains the following input(s):<br> inputs['x'] tensor_info:<br> dtype: DT_FLOAT<br> shape: (-1, 1)<br> name: x:0<br> The given SavedModel SignatureDef contains the following output(s):<br> outputs['y'] tensor_info:<br> dtype: DT_FLOAT<br> shape: (-1, 1)<br> name: y:0<br> Method name is: tensorflow/serving/predict \n",
"\n",
"### `run` 命令\n",
"\n",
"调用 `run` 命令即可运行计算图计算，传递输入，然后显示输出，还可以选择保存。具体语法如下：\n",
"\n",
"```\n",
"usage: saved_model_cli run [-h] --dir DIR --tag_set TAG_SET --signature_def\n",
" SIGNATURE_DEF_KEY [--inputs INPUTS]\n",
" [--input_exprs INPUT_EXPRS]\n",
" [--input_examples INPUT_EXAMPLES] [--outdir OUTDIR]\n",
" [--overwrite] [--tf_debug]\n",
"```\n",
"\n",
"要将输入传递给模型，`run` 命令提供了以下三种方式：\n",
"\n",
"- `--inputs` 选项：可传递文件中的 NumPy ndarray。\n",
"- `--input_exprs` 选项：可传递 Python 表达式。\n",
"- `--input_examples` 选项：可传递 `tf.train.Example`。\n",
"\n",
"#### `--inputs`\n",
"\n",
"要传递文件中的输入数据，请指定 `--inputs` 选项，一般格式如下：\n",
"\n",
"```bsh\n",
"--inputs \n",
"```\n",
"\n",
"其中，*INPUTS* 采用以下格式之一：\n",
"\n",
"- `=`\n",
"- `=[]`\n",
"\n",
"您可以传递多个 *INPUTS*。如果确实要传递多个输入，请使用分号分隔每个 *INPUTS*。\n",
"\n",
"`saved_model_cli` 使用 `numpy.load` 来加载 *filename*。*filename* 可能为以下任何格式：\n",
"\n",
"- `.npy`\n",
"- `.npz`\n",
"- pickle 格式\n",
"\n",
"`.npy` 文件始终包含一个 NumPy ndarray。因此，从 `.npy` 文件加载时，会将内容直接分配给指定的输入张量。如果使用 `.npy` 文件来指定 *variable_name*，则会忽略 *variable_name*，并且会发出警告。\n",
"\n",
"从 `.npz` (zip) 文件加载时，您可以选择指定 *variable_name*，以便标识 zip 文件中要为输入张量键加载的变量。如果不指定 *variable_name*，SavedModel CLI 会检查 zip 文件中是否只包含一个文件。如果是，则为指定的输入张量键加载该文件。\n",
"\n",
"从 pickle 文件加载时，如果未在方括号中指定 `variable_name`，则会将该 pickle 文件中的任何内容全部传递给指定的输入张量键。否则，SavedModel CLI 会假设该 pickle 文件中有一个字典，并将使用与 *variable_name* 对应的值。\n",
"\n",
"#### `--input_exprs`\n",
"\n",
"要通过 Python 表达式传递输入，请指定 `--input_exprs` 选项。当您没有现成的数据文件，而又想使用与模型的 `SignatureDef` 的 dtype 和形状相符的一些简单输入来对模型进行健全性检查时，这非常有用。例如：\n",
"\n",
"```bsh\n",
"`=[[1],[2],[3]]`\n",
"```\n",
"\n",
"除了 Python 表达式，您还可以传递 NumPy 函数。例如：\n",
"\n",
"```bsh\n",
"`=np.ones((32,32,3))`\n",
"```\n",
"\n",
"（请注意，`numpy` 模块已作为 `np` 提供。）\n",
"\n",
"#### `--input_examples`\n",
"\n",
"要传递 `tf.train.Example` 作为输入，请指定 `--input_examples` 选项。对于每个输入键，它会获取一个字典列表，其中每个字典是 `tf.train.Example` 的一个实例。字典键就是特征，而值则是每个特征的值列表。例如：\n",
"\n",
"```bsh\n",
"`=[{\"age\":[22,24],\"education\":[\"BS\",\"MS\"]}]`\n",
"```\n",
"\n",
"#### 保存输出\n",
"\n",
"默认情况下，SavedModel CLI 会将输出写入 stdout。如果将字典传递给 `--outdir` 选项，则会将输出保存为以给定字典下的输出张量键命名的 `.npy` 文件。\n",
"\n",
"使用 `--overwrite` 可重写现有输出文件。\n"
]
}
],
"metadata": {
"colab": {
"collapsed_sections": [],
"name": "saved_model.ipynb",
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.16"
}
},
"nbformat": 4,
"nbformat_minor": 0
}
![](\"https://tensorflow.google.cn/images/tf_logo_32px.png\"){kind=logo}
在 TensorFlow.org 上查看![](\"https://tensorflow.google.cn/images/colab_logo_32px.png\"){kind=logo}
在 Google Colab 中运行![](\"https://tensorflow.google.cn/images/GitHub-Mark-32px.png\")
在 GitHub 上查看源代码![](\"https://tensorflow.google.cn/images/download_logo_32px.png\"){kind=logo}
下载笔记本