{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "qNLUPuRpkFv_" }, "source": [ "##### Copyright 2022 The TensorFlow Hub Authors.\n", "\n", "Licensed under the Apache License, Version 2.0 (the \"License\");" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "cellView": "form", "execution": { "iopub.execute_input": "2024-03-09T12:17:37.422703Z", "iopub.status.busy": "2024-03-09T12:17:37.422131Z", "iopub.status.idle": "2024-03-09T12:17:37.426496Z", "shell.execute_reply": "2024-03-09T12:17:37.425684Z" }, "id": "DQcWZm0FkPk-" }, "outputs": [], "source": [ "#@title Copyright 2022 The TensorFlow Hub Authors. All Rights Reserved.\n", "#\n", "# Licensed under the Apache License, Version 2.0 (the \"License\");\n", "# you may not use this file except in compliance with the License.\n", "# You may obtain a copy of the License at\n", "#\n", "# http://www.apache.org/licenses/LICENSE-2.0\n", "#\n", "# Unless required by applicable law or agreed to in writing, software\n", "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", "# See the License for the specific language governing permissions and\n", "# limitations under the License.\n", "# ==============================================================================" ] }, { "cell_type": "markdown", "metadata": { "id": "Exbxve1rHlrF" }, "source": [ "# Frame interpolation using the FILM model\n" ] }, { "cell_type": "markdown", "metadata": { "id": "jMWFVTlbrQ8m" }, "source": [ "\n", " \n", " \n", " \n", " \n", " \n", "
\n", " View on TensorFlow.org\n", " \n", " Run in Google Colab\n", " \n", " View on GitHub\n", " \n", " Download notebook\n", " \n", " See TF Hub model\n", "
" ] }, { "cell_type": "markdown", "metadata": { "id": "61H28S7ArUAZ" }, "source": [ "Frame interpolation is the task of synthesizing many in-between images from a given set of images. The technique is often used for frame rate upsampling or creating slow-motion video effects.\n", "\n", "In this colab, you will use the FILM model to do frame interpolation. The colab also provides code snippets to create videos from the interpolated in-between images.\n", "\n", "For more information on FILM research, you can read more here:\n", "- Google AI Blog: [Large Motion Frame Interpolation](https://ai.googleblog.com/2022/10/large-motion-frame-interpolation.html)\n", "- Project Page: FILM: [Frame Interpolation for Large Motion](https://film-net.github.io/)\n" ] }, { "cell_type": "markdown", "metadata": { "id": "dVX7s6zMulsu" }, "source": [ "## Setup" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2024-03-09T12:17:37.430206Z", "iopub.status.busy": "2024-03-09T12:17:37.429601Z", "iopub.status.idle": "2024-03-09T12:17:54.856013Z", "shell.execute_reply": "2024-03-09T12:17:54.855076Z" }, "id": "oi5t2OEJsGBW" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Collecting mediapy\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ " Downloading mediapy-1.2.0-py3-none-any.whl.metadata (4.8 kB)\r\n", "Requirement already satisfied: ipython in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from mediapy) (8.18.1)\r\n", "Requirement already satisfied: matplotlib in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from mediapy) (3.8.3)\r\n", "Requirement already satisfied: numpy in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from mediapy) (1.26.4)\r\n", "Requirement already satisfied: Pillow in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from mediapy) (10.2.0)\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: decorator in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipython->mediapy) (5.1.1)\r\n", "Requirement already satisfied: jedi>=0.16 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipython->mediapy) (0.19.1)\r\n", "Requirement already satisfied: matplotlib-inline in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipython->mediapy) (0.1.6)\r\n", "Requirement already satisfied: prompt-toolkit<3.1.0,>=3.0.41 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipython->mediapy) (3.0.43)\r\n", "Requirement already satisfied: pygments>=2.4.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipython->mediapy) (2.17.2)\r\n", "Requirement already satisfied: stack-data in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipython->mediapy) (0.6.3)\r\n", "Requirement already satisfied: traitlets>=5 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipython->mediapy) (5.14.1)\r\n", "Requirement already satisfied: typing-extensions in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipython->mediapy) (4.10.0)\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: exceptiongroup in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipython->mediapy) (1.2.0)\r\n", "Requirement already satisfied: pexpect>4.3 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipython->mediapy) (4.9.0)\r\n", "Requirement already satisfied: contourpy>=1.0.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from matplotlib->mediapy) (1.2.0)\r\n", "Requirement already satisfied: cycler>=0.10 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from matplotlib->mediapy) (0.12.1)\r\n", "Requirement already satisfied: fonttools>=4.22.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from matplotlib->mediapy) (4.49.0)\r\n", "Requirement already satisfied: kiwisolver>=1.3.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from matplotlib->mediapy) (1.4.5)\r\n", "Requirement already satisfied: packaging>=20.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from matplotlib->mediapy) (23.2)\r\n", "Requirement already satisfied: pyparsing>=2.3.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from matplotlib->mediapy) (3.1.2)\r\n", "Requirement already satisfied: python-dateutil>=2.7 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from matplotlib->mediapy) (2.9.0.post0)\r\n", "Requirement already satisfied: importlib-resources>=3.2.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from matplotlib->mediapy) (6.1.3)\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: zipp>=3.1.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from importlib-resources>=3.2.0->matplotlib->mediapy) (3.17.0)\r\n", "Requirement already satisfied: parso<0.9.0,>=0.8.3 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jedi>=0.16->ipython->mediapy) (0.8.3)\r\n", "Requirement already satisfied: ptyprocess>=0.5 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from pexpect>4.3->ipython->mediapy) (0.7.0)\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: wcwidth in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from prompt-toolkit<3.1.0,>=3.0.41->ipython->mediapy) (0.2.13)\r\n", "Requirement already satisfied: six>=1.5 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib->mediapy) (1.16.0)\r\n", "Requirement already satisfied: executing>=1.2.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from stack-data->ipython->mediapy) (2.0.1)\r\n", "Requirement already satisfied: asttokens>=2.1.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from stack-data->ipython->mediapy) (2.4.1)\r\n", "Requirement already satisfied: pure-eval in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from stack-data->ipython->mediapy) (0.2.2)\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Downloading mediapy-1.2.0-py3-none-any.whl (25 kB)\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Installing collected packages: mediapy\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Successfully installed mediapy-1.2.0\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\r", "Reading package lists... 0%\r", "\r", "Reading package lists... 100%\r", "\r", "Reading package lists... Done\r", "\r\n", "\r", "Building dependency tree... 0%\r", "\r", "Building dependency tree... 0%\r" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\r", "Building dependency tree... 50%\r", "\r", "Building dependency tree... 50%\r" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\r", "Building dependency tree \r", "\r\n", "\r", "Reading state information... 0%\r", "\r", "Reading state information... 0%\r" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\r", "Reading state information... Done\r", "\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "The following packages were automatically installed and are no longer required:\r\n", " libatasmart4 libblockdev-fs2 libblockdev-loop2 libblockdev-part-err2\r\n", " libblockdev-part2 libblockdev-swap2 libblockdev-utils2 libblockdev2\r\n", " libparted-fs-resize0 libxmlb2\r\n", "Use 'sudo apt autoremove' to remove them.\r\n", "The following additional packages will be installed:\r\n", " i965-va-driver intel-media-va-driver libaacs0 libaom0 libass9 libavcodec58\r\n", " libavdevice58 libavfilter7 libavformat58 libavresample4 libavutil56\r\n", " libbdplus0 libbluray2 libbs2b0 libcdio-cdda2 libcdio-paranoia2 libcdio18\r\n", " libchromaprint1 libcodec2-0.9 libdc1394-22 libfftw3-double3 libflite1\r\n", " libgme0 libgsm1 libigdgmm11 liblilv-0-0 libmysofa1 libopenal-data libopenal1\r\n", " libopenmpt0 libpostproc55 librubberband2 libsdl2-2.0-0 libserd-0-0 libshine3\r\n", " libsnappy1v5 libsndio7.0 libsord-0-0 libsratom-0-0 libssh-gcrypt-4\r\n", " libswresample3 libswscale5 libva-drm2 libva-x11-2 libva2 libvidstab1.1\r\n", " libwebpmux3 libx264-155 libx265-179 libxvidcore4 libzvbi-common libzvbi0\r\n", " mesa-va-drivers ocl-icd-libopencl1 va-driver-all\r\n", "Suggested packages:\r\n", " ffmpeg-doc i965-va-driver-shaders libbluray-bdj libfftw3-bin libfftw3-dev\r\n", " libportaudio2 serdi sndiod sordi\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "The following NEW packages will be installed:\r\n", " ffmpeg i965-va-driver intel-media-va-driver libaacs0 libaom0 libass9\r\n", " libavcodec58 libavdevice58 libavfilter7 libavformat58 libavresample4\r\n", " libavutil56 libbdplus0 libbluray2 libbs2b0 libcdio-cdda2 libcdio-paranoia2\r\n", " libcdio18 libchromaprint1 libcodec2-0.9 libdc1394-22 libfftw3-double3\r\n", " libflite1 libgme0 libgsm1 libigdgmm11 liblilv-0-0 libmysofa1 libopenal-data\r\n", " libopenal1 libopenmpt0 libpostproc55 librubberband2 libsdl2-2.0-0\r\n", " libserd-0-0 libshine3 libsnappy1v5 libsndio7.0 libsord-0-0 libsratom-0-0\r\n", " libssh-gcrypt-4 libswresample3 libswscale5 libva-drm2 libva-x11-2 libva2\r\n", " libvidstab1.1 libwebpmux3 libx264-155 libx265-179 libxvidcore4\r\n", " libzvbi-common libzvbi0 mesa-va-drivers ocl-icd-libopencl1 va-driver-all\r\n", "0 upgraded, 56 newly installed, 0 to remove and 188 not upgraded.\r\n", "Need to get 42.5 MB of archives.\r\n", "After this operation, 156 MB of additional disk space will be used.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\r", "0% [Working]\r", " \r", "Get:1 http://us-east1.gce.archive.ubuntu.com/ubuntu focal-updates/universe amd64 libaom0 amd64 1.0.0.errata1-3+deb11u1build0.20.04.1 [1161 kB]\r\n", "\r", "0% [1 libaom0 13.7 kB/1161 kB 1%]\r", " \r", "3% [Working]\r", " \r", "Get:2 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libva2 amd64 2.7.0-2 [53.5 kB]\r\n", "\r", "3% [2 libva2 13.7 kB/53.5 kB 26%]\r", " \r", "3% [Waiting for headers]\r", " \r", "Get:3 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libva-drm2 amd64 2.7.0-2 [7044 B]\r\n", "\r", "3% [3 libva-drm2 7044 B/7044 B 100%]\r", " \r", "3% [Waiting for headers]\r", " \r", "Get:4 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libva-x11-2 amd64 2.7.0-2 [11.9 kB]\r\n", "\r", "3% [4 libva-x11-2 11.9 kB/11.9 kB 100%]\r", " \r", "4% [Waiting for headers]\r", " \r", "Get:5 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/main amd64 ocl-icd-libopencl1 amd64 2.2.11-1ubuntu1 [30.3 kB]\r\n", "\r", "4% [5 ocl-icd-libopencl1 6627 B/30.3 kB 22%]\r", " \r", "4% [Waiting for headers]\r", " \r", "Get:6 http://us-east1.gce.archive.ubuntu.com/ubuntu focal-updates/universe amd64 libavutil56 amd64 7:4.2.7-0ubuntu0.1 [241 kB]\r\n", "\r", "4% [6 libavutil56 13.7 kB/241 kB 6%]\r", " \r", "5% [Waiting for headers]\r", " \r", "Get:7 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libcodec2-0.9 amd64 0.9.2-2 [7886 kB]\r\n", "\r", "5% [7 libcodec2-0.9 10.8 kB/7886 kB 0%]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\r", " \r", "20% [Waiting for headers]\r", " \r", "Get:8 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libgsm1 amd64 1.0.18-2 [24.4 kB]\r\n", "\r", "20% [8 libgsm1 24.4 kB/24.4 kB 100%]\r", " \r", "21% [Working]\r", " \r", "Get:9 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libshine3 amd64 3.1.1-2 [23.2 kB]\r\n", "\r", "21% [9 libshine3 8192 B/23.2 kB 35%]\r", " \r", "21% [Waiting for headers]\r", " \r", "Get:10 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/main amd64 libsnappy1v5 amd64 1.1.8-1build1 [16.7 kB]\r\n", "\r", "21% [10 libsnappy1v5 16.7 kB/16.7 kB 100%]\r", " \r", "21% [Working]\r", " \r", "Get:11 http://us-east1.gce.archive.ubuntu.com/ubuntu focal-updates/universe amd64 libswresample3 amd64 7:4.2.7-0ubuntu0.1 [57.1 kB]\r\n", "\r", "21% [11 libswresample3 8192 B/57.1 kB 14%]\r", " \r", "22% [Waiting for headers]\r", " \r", "Get:12 http://us-east1.gce.archive.ubuntu.com/ubuntu focal-updates/main amd64 libwebpmux3 amd64 0.6.1-2ubuntu0.20.04.3 [19.5 kB]\r\n", "\r", "22% [12 libwebpmux3 19.5 kB/19.5 kB 100%]\r", " \r", "22% [Working]\r", " \r", "Get:13 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libx264-155 amd64 2:0.155.2917+git0a84d98-2 [521 kB]\r\n", "\r", "22% [13 libx264-155 4096 B/521 kB 1%]\r", " \r", "24% [Waiting for headers]\r", " \r", "Get:14 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libx265-179 amd64 3.2.1-1build1 [1060 kB]\r\n", "\r", "24% [14 libx265-179 34.0 kB/1060 kB 3%]\r", " \r", "26% [Waiting for headers]\r", " \r", "Get:15 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libxvidcore4 amd64 2:1.3.7-1 [201 kB]\r\n", "\r", "26% [15 libxvidcore4 22.2 kB/201 kB 11%]\r", " \r", "27% [Waiting for headers]\r", " \r", "Get:16 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libzvbi-common all 0.2.35-17 [32.5 kB]\r\n", "\r", "27% [16 libzvbi-common 17.1 kB/32.5 kB 53%]\r", " \r", "27% [Working]\r", " \r", "Get:17 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libzvbi0 amd64 0.2.35-17 [237 kB]\r\n", "\r", "27% [17 libzvbi0 0 B/237 kB 0%]\r", " \r", "28% [Waiting for headers]\r", " \r", "Get:18 http://us-east1.gce.archive.ubuntu.com/ubuntu focal-updates/universe amd64 libavcodec58 amd64 7:4.2.7-0ubuntu0.1 [4878 kB]\r\n", "\r", "28% [18 libavcodec58 8665 B/4878 kB 0%]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\r", " \r", "37% [Waiting for headers]\r", " \r", "Get:19 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libass9 amd64 1:0.14.0-2 [88.0 kB]\r\n", "\r", "37% [19 libass9 20.5 kB/88.0 kB 23%]\r", " \r", "38% [Waiting for headers]\r", " \r", "Get:20 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libbluray2 amd64 1:1.2.0-1 [138 kB]\r\n", "\r", "38% [20 libbluray2 13.7 kB/138 kB 10%]\r", " \r", "39% [Waiting for headers]\r", " \r", "Get:21 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libchromaprint1 amd64 1.4.3-3build1 [37.6 kB]\r\n", "\r", "39% [21 libchromaprint1 13.7 kB/37.6 kB 36%]\r", " \r", "39% [Waiting for headers]\r", " \r", "Get:22 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libgme0 amd64 0.6.2-1build1 [123 kB]\r\n", "\r", "39% [22 libgme0 8033 B/123 kB 7%]\r", " \r", "40% [Waiting for headers]\r", " \r", "Get:23 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libopenmpt0 amd64 0.4.11-1build1 [599 kB]\r\n", "\r", "40% [23 libopenmpt0 993 B/599 kB 0%]\r", " \r", "41% [Waiting for headers]\r", " \r", "Get:24 http://us-east1.gce.archive.ubuntu.com/ubuntu focal-updates/main amd64 libssh-gcrypt-4 amd64 0.9.3-2ubuntu2.5 [202 kB]\r\n", "\r", "41% [24 libssh-gcrypt-4 41.1 kB/202 kB 20%]\r", " \r", "42% [Waiting for headers]\r", " \r", "Get:25 http://us-east1.gce.archive.ubuntu.com/ubuntu focal-updates/universe amd64 libavformat58 amd64 7:4.2.7-0ubuntu0.1 [985 kB]\r\n", "\r", "42% [25 libavformat58 34.9 kB/985 kB 4%]\r", " \r", "44% [Waiting for headers]\r", " \r", "Get:26 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libbs2b0 amd64 3.1.0+dfsg-2.2build1 [10.2 kB]\r\n", "\r", "44% [26 libbs2b0 10.2 kB/10.2 kB 100%]\r", " \r", "44% [Working]\r", " \r", "Get:27 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libflite1 amd64 2.1-release-3 [12.8 MB]\r\n", "\r", "44% [27 libflite1 4096 B/12.8 MB 0%]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\r", " \r", "69% [Waiting for headers]\r", " \r", "Get:28 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libserd-0-0 amd64 0.30.2-1 [46.6 kB]\r\n", "\r", "69% [28 libserd-0-0 12.3 kB/46.6 kB 26%]\r", " \r", "69% [Waiting for headers]\r", " \r", "Get:29 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libsord-0-0 amd64 0.16.4-1 [19.5 kB]\r\n", "\r", "69% [29 libsord-0-0 12.1 kB/19.5 kB 62%]\r", " \r", "70% [Waiting for headers]\r", " \r", "Get:30 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libsratom-0-0 amd64 0.6.4-1 [16.9 kB]\r\n", "\r", "70% [30 libsratom-0-0 16.9 kB/16.9 kB 100%]\r", " \r", "70% [Waiting for headers]\r", " \r", "Get:31 http://us-east1.gce.archive.ubuntu.com/ubuntu focal-updates/universe amd64 liblilv-0-0 amd64 0.24.6-1ubuntu0.1 [40.6 kB]\r\n", "\r", "70% [31 liblilv-0-0 12.3 kB/40.6 kB 30%]\r", " \r", "71% [Waiting for headers]\r", " \r", "Get:32 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libmysofa1 amd64 1.0~dfsg0-1 [39.2 kB]\r\n", "\r", "71% [32 libmysofa1 8035 B/39.2 kB 20%]\r", " \r", "71% [Working]\r", " \r", "Get:33 http://us-east1.gce.archive.ubuntu.com/ubuntu focal-updates/universe amd64 libpostproc55 amd64 7:4.2.7-0ubuntu0.1 [55.0 kB]\r\n", "\r", "71% [33 libpostproc55 13.7 kB/55.0 kB 25%]\r", " \r", "71% [Waiting for headers]\r", " \r", "Get:34 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/main amd64 libfftw3-double3 amd64 3.3.8-2ubuntu1 [728 kB]\r\n", "\r", "71% [34 libfftw3-double3 8033 B/728 kB 1%]\r", " \r", "73% [Waiting for headers]\r", " \r", "Get:35 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 librubberband2 amd64 1.8.2-1build1 [89.4 kB]\r\n", "\r", "73% [35 librubberband2 28.3 kB/89.4 kB 32%]\r", " \r", "74% [Waiting for headers]\r", " \r", "Get:36 http://us-east1.gce.archive.ubuntu.com/ubuntu focal-updates/universe amd64 libswscale5 amd64 7:4.2.7-0ubuntu0.1 [157 kB]\r\n", "\r", "74% [36 libswscale5 4022 B/157 kB 3%]\r", " \r", "74% [Waiting for headers]\r", " \r", "Get:37 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libvidstab1.1 amd64 1.1.0-2 [35.0 kB]\r\n", "\r", "74% [37 libvidstab1.1 35.0 kB/35.0 kB 100%]\r", " \r", "75% [Working]\r", " \r", "Get:38 http://us-east1.gce.archive.ubuntu.com/ubuntu focal-updates/universe amd64 libavfilter7 amd64 7:4.2.7-0ubuntu0.1 [1085 kB]\r\n", "\r", "75% [38 libavfilter7 6623 B/1085 kB 1%]\r", " \r", "77% [Waiting for headers]\r", " \r", "Get:39 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/main amd64 libcdio18 amd64 2.0.0-2 [58.6 kB]\r\n", "\r", "77% [39 libcdio18 36.6 kB/58.6 kB 62%]\r", " \r", "78% [Working]\r", " \r", "Get:40 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/main amd64 libcdio-cdda2 amd64 10.2+2.0.0-1 [17.6 kB]\r\n", "\r", "78% [40 libcdio-cdda2 0 B/17.6 kB 0%]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\r", " \r", "78% [Waiting for headers]\r", " \r", "Get:41 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/main amd64 libcdio-paranoia2 amd64 10.2+2.0.0-1 [16.2 kB]\r\n", "\r", "78% [41 libcdio-paranoia2 16.2 kB/16.2 kB 100%]\r", " \r", "78% [Working]\r", " \r", "Get:42 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libdc1394-22 amd64 2.2.5-2.1 [79.6 kB]\r\n", "\r", "78% [42 libdc1394-22 0 B/79.6 kB 0%]\r", " \r", "79% [Waiting for headers]\r", " \r", "Get:43 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libopenal-data all 1:1.19.1-1 [162 kB]\r\n", "\r", "79% [43 libopenal-data 57.3 kB/162 kB 35%]\r", " \r", "80% [Waiting for headers]\r", " \r", "Get:44 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libsndio7.0 amd64 1.5.0-3 [24.5 kB]\r\n", "\r", "80% [44 libsndio7.0 24.5 kB/24.5 kB 100%]\r", " \r", "80% [Working]\r", " \r", "Get:45 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libopenal1 amd64 1:1.19.1-1 [492 kB]\r\n", "\r", "80% [45 libopenal1 3460 B/492 kB 1%]\r", " \r", "81% [Waiting for headers]\r", " \r", "Get:46 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libsdl2-2.0-0 amd64 2.0.10+dfsg1-3 [407 kB]\r\n", "\r", "81% [46 libsdl2-2.0-0 35.0 kB/407 kB 9%]\r", " \r", "82% [Working]\r", " \r", "Get:47 http://us-east1.gce.archive.ubuntu.com/ubuntu focal-updates/universe amd64 libavdevice58 amd64 7:4.2.7-0ubuntu0.1 [74.3 kB]\r\n", "\r", "82% [47 libavdevice58 20.5 kB/74.3 kB 28%]\r", " \r", "83% [Waiting for headers]\r", " \r", "Get:48 http://us-east1.gce.archive.ubuntu.com/ubuntu focal-updates/universe amd64 libavresample4 amd64 7:4.2.7-0ubuntu0.1 [54.2 kB]\r\n", "\r", "83% [48 libavresample4 12.0 kB/54.2 kB 22%]\r", " \r", "83% [Waiting for headers]\r", " \r", "Get:49 http://us-east1.gce.archive.ubuntu.com/ubuntu focal-updates/universe amd64 ffmpeg amd64 7:4.2.7-0ubuntu0.1 [1453 kB]\r\n", "\r", "83% [49 ffmpeg 13.7 kB/1453 kB 1%]\r", " \r", "86% [Waiting for headers]\r", " \r", "Get:50 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libigdgmm11 amd64 20.1.1+ds1-1 [111 kB]\r\n", "\r", "86% [50 libigdgmm11 11.7 kB/111 kB 11%]\r", " \r", "87% [Waiting for headers]\r", " \r", "Get:51 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 intel-media-va-driver amd64 20.1.1+dfsg1-1 [1764 kB]\r\n", "\r", "87% [51 intel-media-va-driver 30.9 kB/1764 kB 2%]\r", " \r", "91% [Waiting for headers]\r", " \r", "Get:52 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libaacs0 amd64 0.9.0-2 [50.1 kB]\r\n", "\r", "91% [52 libaacs0 4096 B/50.1 kB 8%]\r", " \r", "91% [Waiting for headers]\r", " \r", "Get:53 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 libbdplus0 amd64 0.1.2-3 [47.3 kB]\r\n", "\r", "91% [53 libbdplus0 20.5 kB/47.3 kB 43%]\r", " \r", "92% [Waiting for headers]\r", " \r", "Get:54 http://us-east1.gce.archive.ubuntu.com/ubuntu focal-updates/universe amd64 mesa-va-drivers amd64 21.2.6-0ubuntu0.1~20.04.2 [2970 kB]\r\n", "\r", "92% [54 mesa-va-drivers 2914 B/2970 kB 0%]" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\r", " \r", "98% [Waiting for headers]\r", " \r", "Get:55 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 i965-va-driver amd64 2.4.0-0ubuntu1 [924 kB]\r\n", "\r", "98% [55 i965-va-driver 47.5 kB/924 kB 5%]\r", " \r", "100% [Waiting for headers]\r", " \r", "Get:56 http://us-east1.gce.archive.ubuntu.com/ubuntu focal/universe amd64 va-driver-all amd64 2.7.0-2 [4020 B]\r\n", "\r", "100% [56 va-driver-all 4020 B/4020 B 100%]\r", " \r", "100% [Working]\r", " \r", "Fetched 42.5 MB in 1s (71.4 MB/s)\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\r", "Extracting templates from packages: 53%" ] }, { "name": "stdout", "output_type": "stream", "text": [ "\r", "Extracting templates from packages: 100%\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libaom0:amd64.\r\n", "(Reading database ... \r" ] }, { "name": "stdout", "output_type": "stream", "text": [ "(Reading database ... 5%\r", "(Reading database ... 10%\r", "(Reading database ... 15%\r", "(Reading database ... 20%\r", "(Reading database ... 25%\r", "(Reading database ... 30%\r", "(Reading database ... 35%\r", "(Reading database ... 40%\r", "(Reading database ... 45%\r", "(Reading database ... 50%\r", "(Reading database ... 55%\r", "(Reading database ... 60%\r", "(Reading database ... 65%\r", "(Reading database ... 70%\r", "(Reading database ... 75%\r", "(Reading database ... 80%\r", "(Reading database ... 85%\r" ] }, { "name": "stdout", "output_type": "stream", "text": [ "(Reading database ... 90%\r", "(Reading database ... 95%\r", "(Reading database ... 100%\r", "(Reading database ... 143631 files and directories currently installed.)\r\n", "Preparing to unpack .../00-libaom0_1.0.0.errata1-3+deb11u1build0.20.04.1_amd64.deb ...\r\n", "Unpacking libaom0:amd64 (1.0.0.errata1-3+deb11u1build0.20.04.1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libva2:amd64.\r\n", "Preparing to unpack .../01-libva2_2.7.0-2_amd64.deb ...\r\n", "Unpacking libva2:amd64 (2.7.0-2) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libva-drm2:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../02-libva-drm2_2.7.0-2_amd64.deb ...\r\n", "Unpacking libva-drm2:amd64 (2.7.0-2) ...\r\n", "Selecting previously unselected package libva-x11-2:amd64.\r\n", "Preparing to unpack .../03-libva-x11-2_2.7.0-2_amd64.deb ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Unpacking libva-x11-2:amd64 (2.7.0-2) ...\r\n", "Selecting previously unselected package ocl-icd-libopencl1:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../04-ocl-icd-libopencl1_2.2.11-1ubuntu1_amd64.deb ...\r\n", "Unpacking ocl-icd-libopencl1:amd64 (2.2.11-1ubuntu1) ...\r\n", "Selecting previously unselected package libavutil56:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../05-libavutil56_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\r\n", "Unpacking libavutil56:amd64 (7:4.2.7-0ubuntu0.1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libcodec2-0.9:amd64.\r\n", "Preparing to unpack .../06-libcodec2-0.9_0.9.2-2_amd64.deb ...\r\n", "Unpacking libcodec2-0.9:amd64 (0.9.2-2) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libgsm1:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../07-libgsm1_1.0.18-2_amd64.deb ...\r\n", "Unpacking libgsm1:amd64 (1.0.18-2) ...\r\n", "Selecting previously unselected package libshine3:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../08-libshine3_3.1.1-2_amd64.deb ...\r\n", "Unpacking libshine3:amd64 (3.1.1-2) ...\r\n", "Selecting previously unselected package libsnappy1v5:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../09-libsnappy1v5_1.1.8-1build1_amd64.deb ...\r\n", "Unpacking libsnappy1v5:amd64 (1.1.8-1build1) ...\r\n", "Selecting previously unselected package libswresample3:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../10-libswresample3_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\r\n", "Unpacking libswresample3:amd64 (7:4.2.7-0ubuntu0.1) ...\r\n", "Selecting previously unselected package libwebpmux3:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../11-libwebpmux3_0.6.1-2ubuntu0.20.04.3_amd64.deb ...\r\n", "Unpacking libwebpmux3:amd64 (0.6.1-2ubuntu0.20.04.3) ...\r\n", "Selecting previously unselected package libx264-155:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../12-libx264-155_2%3a0.155.2917+git0a84d98-2_amd64.deb ...\r\n", "Unpacking libx264-155:amd64 (2:0.155.2917+git0a84d98-2) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libx265-179:amd64.\r\n", "Preparing to unpack .../13-libx265-179_3.2.1-1build1_amd64.deb ...\r\n", "Unpacking libx265-179:amd64 (3.2.1-1build1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libxvidcore4:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../14-libxvidcore4_2%3a1.3.7-1_amd64.deb ...\r\n", "Unpacking libxvidcore4:amd64 (2:1.3.7-1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libzvbi-common.\r\n", "Preparing to unpack .../15-libzvbi-common_0.2.35-17_all.deb ...\r\n", "Unpacking libzvbi-common (0.2.35-17) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libzvbi0:amd64.\r\n", "Preparing to unpack .../16-libzvbi0_0.2.35-17_amd64.deb ...\r\n", "Unpacking libzvbi0:amd64 (0.2.35-17) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libavcodec58:amd64.\r\n", "Preparing to unpack .../17-libavcodec58_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\r\n", "Unpacking libavcodec58:amd64 (7:4.2.7-0ubuntu0.1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libass9:amd64.\r\n", "Preparing to unpack .../18-libass9_1%3a0.14.0-2_amd64.deb ...\r\n", "Unpacking libass9:amd64 (1:0.14.0-2) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libbluray2:amd64.\r\n", "Preparing to unpack .../19-libbluray2_1%3a1.2.0-1_amd64.deb ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Unpacking libbluray2:amd64 (1:1.2.0-1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libchromaprint1:amd64.\r\n", "Preparing to unpack .../20-libchromaprint1_1.4.3-3build1_amd64.deb ...\r\n", "Unpacking libchromaprint1:amd64 (1.4.3-3build1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libgme0:amd64.\r\n", "Preparing to unpack .../21-libgme0_0.6.2-1build1_amd64.deb ...\r\n", "Unpacking libgme0:amd64 (0.6.2-1build1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libopenmpt0:amd64.\r\n", "Preparing to unpack .../22-libopenmpt0_0.4.11-1build1_amd64.deb ...\r\n", "Unpacking libopenmpt0:amd64 (0.4.11-1build1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libssh-gcrypt-4:amd64.\r\n", "Preparing to unpack .../23-libssh-gcrypt-4_0.9.3-2ubuntu2.5_amd64.deb ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Unpacking libssh-gcrypt-4:amd64 (0.9.3-2ubuntu2.5) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libavformat58:amd64.\r\n", "Preparing to unpack .../24-libavformat58_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\r\n", "Unpacking libavformat58:amd64 (7:4.2.7-0ubuntu0.1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libbs2b0:amd64.\r\n", "Preparing to unpack .../25-libbs2b0_3.1.0+dfsg-2.2build1_amd64.deb ...\r\n", "Unpacking libbs2b0:amd64 (3.1.0+dfsg-2.2build1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libflite1:amd64.\r\n", "Preparing to unpack .../26-libflite1_2.1-release-3_amd64.deb ...\r\n", "Unpacking libflite1:amd64 (2.1-release-3) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libserd-0-0:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../27-libserd-0-0_0.30.2-1_amd64.deb ...\r\n", "Unpacking libserd-0-0:amd64 (0.30.2-1) ...\r\n", "Selecting previously unselected package libsord-0-0:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../28-libsord-0-0_0.16.4-1_amd64.deb ...\r\n", "Unpacking libsord-0-0:amd64 (0.16.4-1) ...\r\n", "Selecting previously unselected package libsratom-0-0:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../29-libsratom-0-0_0.6.4-1_amd64.deb ...\r\n", "Unpacking libsratom-0-0:amd64 (0.6.4-1) ...\r\n", "Selecting previously unselected package liblilv-0-0:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../30-liblilv-0-0_0.24.6-1ubuntu0.1_amd64.deb ...\r\n", "Unpacking liblilv-0-0:amd64 (0.24.6-1ubuntu0.1) ...\r\n", "Selecting previously unselected package libmysofa1:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../31-libmysofa1_1.0~dfsg0-1_amd64.deb ...\r\n", "Unpacking libmysofa1:amd64 (1.0~dfsg0-1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libpostproc55:amd64.\r\n", "Preparing to unpack .../32-libpostproc55_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\r\n", "Unpacking libpostproc55:amd64 (7:4.2.7-0ubuntu0.1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libfftw3-double3:amd64.\r\n", "Preparing to unpack .../33-libfftw3-double3_3.3.8-2ubuntu1_amd64.deb ...\r\n", "Unpacking libfftw3-double3:amd64 (3.3.8-2ubuntu1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package librubberband2:amd64.\r\n", "Preparing to unpack .../34-librubberband2_1.8.2-1build1_amd64.deb ...\r\n", "Unpacking librubberband2:amd64 (1.8.2-1build1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libswscale5:amd64.\r\n", "Preparing to unpack .../35-libswscale5_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Unpacking libswscale5:amd64 (7:4.2.7-0ubuntu0.1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libvidstab1.1:amd64.\r\n", "Preparing to unpack .../36-libvidstab1.1_1.1.0-2_amd64.deb ...\r\n", "Unpacking libvidstab1.1:amd64 (1.1.0-2) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libavfilter7:amd64.\r\n", "Preparing to unpack .../37-libavfilter7_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\r\n", "Unpacking libavfilter7:amd64 (7:4.2.7-0ubuntu0.1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libcdio18:amd64.\r\n", "Preparing to unpack .../38-libcdio18_2.0.0-2_amd64.deb ...\r\n", "Unpacking libcdio18:amd64 (2.0.0-2) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libcdio-cdda2:amd64.\r\n", "Preparing to unpack .../39-libcdio-cdda2_10.2+2.0.0-1_amd64.deb ...\r\n", "Unpacking libcdio-cdda2:amd64 (10.2+2.0.0-1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libcdio-paranoia2:amd64.\r\n", "Preparing to unpack .../40-libcdio-paranoia2_10.2+2.0.0-1_amd64.deb ...\r\n", "Unpacking libcdio-paranoia2:amd64 (10.2+2.0.0-1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libdc1394-22:amd64.\r\n", "Preparing to unpack .../41-libdc1394-22_2.2.5-2.1_amd64.deb ...\r\n", "Unpacking libdc1394-22:amd64 (2.2.5-2.1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libopenal-data.\r\n", "Preparing to unpack .../42-libopenal-data_1%3a1.19.1-1_all.deb ...\r\n", "Unpacking libopenal-data (1:1.19.1-1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libsndio7.0:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../43-libsndio7.0_1.5.0-3_amd64.deb ...\r\n", "Unpacking libsndio7.0:amd64 (1.5.0-3) ...\r\n", "Selecting previously unselected package libopenal1:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../44-libopenal1_1%3a1.19.1-1_amd64.deb ...\r\n", "Unpacking libopenal1:amd64 (1:1.19.1-1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libsdl2-2.0-0:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../45-libsdl2-2.0-0_2.0.10+dfsg1-3_amd64.deb ...\r\n", "Unpacking libsdl2-2.0-0:amd64 (2.0.10+dfsg1-3) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libavdevice58:amd64.\r\n", "Preparing to unpack .../46-libavdevice58_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Unpacking libavdevice58:amd64 (7:4.2.7-0ubuntu0.1) ...\r\n", "Selecting previously unselected package libavresample4:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../47-libavresample4_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\r\n", "Unpacking libavresample4:amd64 (7:4.2.7-0ubuntu0.1) ...\r\n", "Selecting previously unselected package ffmpeg.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../48-ffmpeg_7%3a4.2.7-0ubuntu0.1_amd64.deb ...\r\n", "Unpacking ffmpeg (7:4.2.7-0ubuntu0.1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libigdgmm11:amd64.\r\n", "Preparing to unpack .../49-libigdgmm11_20.1.1+ds1-1_amd64.deb ...\r\n", "Unpacking libigdgmm11:amd64 (20.1.1+ds1-1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package intel-media-va-driver:amd64.\r\n", "Preparing to unpack .../50-intel-media-va-driver_20.1.1+dfsg1-1_amd64.deb ...\r\n", "Unpacking intel-media-va-driver:amd64 (20.1.1+dfsg1-1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package libaacs0:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../51-libaacs0_0.9.0-2_amd64.deb ...\r\n", "Unpacking libaacs0:amd64 (0.9.0-2) ...\r\n", "Selecting previously unselected package libbdplus0:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../52-libbdplus0_0.1.2-3_amd64.deb ...\r\n", "Unpacking libbdplus0:amd64 (0.1.2-3) ...\r\n", "Selecting previously unselected package mesa-va-drivers:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../53-mesa-va-drivers_21.2.6-0ubuntu0.1~20.04.2_amd64.deb ...\r\n", "Unpacking mesa-va-drivers:amd64 (21.2.6-0ubuntu0.1~20.04.2) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package i965-va-driver:amd64.\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Preparing to unpack .../54-i965-va-driver_2.4.0-0ubuntu1_amd64.deb ...\r\n", "Unpacking i965-va-driver:amd64 (2.4.0-0ubuntu1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Selecting previously unselected package va-driver-all:amd64.\r\n", "Preparing to unpack .../55-va-driver-all_2.7.0-2_amd64.deb ...\r\n", "Unpacking va-driver-all:amd64 (2.7.0-2) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Setting up libgme0:amd64 (0.6.2-1build1) ...\r\n", "Setting up libssh-gcrypt-4:amd64 (0.9.3-2ubuntu2.5) ...\r\n", "Setting up libshine3:amd64 (3.1.1-2) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Setting up libgsm1:amd64 (1.0.18-2) ...\r\n", "Setting up libx264-155:amd64 (2:0.155.2917+git0a84d98-2) ...\r\n", "Setting up libx265-179:amd64 (3.2.1-1build1) ...\r\n", "Setting up libaom0:amd64 (1.0.0.errata1-3+deb11u1build0.20.04.1) ...\r\n", "Setting up libmysofa1:amd64 (1.0~dfsg0-1) ...\r\n", "Setting up libdc1394-22:amd64 (2.2.5-2.1) ...\r\n", "Setting up libcdio18:amd64 (2.0.0-2) ...\r\n", "Setting up libxvidcore4:amd64 (2:1.3.7-1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Setting up libsnappy1v5:amd64 (1.1.8-1build1) ...\r\n", "Setting up libass9:amd64 (1:0.14.0-2) ...\r\n", "Setting up libva2:amd64 (2.7.0-2) ...\r\n", "Setting up libigdgmm11:amd64 (20.1.1+ds1-1) ...\r\n", "Setting up libcodec2-0.9:amd64 (0.9.2-2) ...\r\n", "Setting up intel-media-va-driver:amd64 (20.1.1+dfsg1-1) ...\r\n", "Setting up libaacs0:amd64 (0.9.0-2) ...\r\n", "Setting up libfftw3-double3:amd64 (3.3.8-2ubuntu1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Setting up librubberband2:amd64 (1.8.2-1build1) ...\r\n", "Setting up libsndio7.0:amd64 (1.5.0-3) ...\r\n", "Setting up libbdplus0:amd64 (0.1.2-3) ...\r\n", "Setting up libvidstab1.1:amd64 (1.1.0-2) ...\r\n", "Setting up libflite1:amd64 (2.1-release-3) ...\r\n", "Setting up libva-drm2:amd64 (2.7.0-2) ...\r\n", "Setting up ocl-icd-libopencl1:amd64 (2.2.11-1ubuntu1) ...\r\n", "Setting up libbs2b0:amd64 (3.1.0+dfsg-2.2build1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Setting up libopenal-data (1:1.19.1-1) ...\r\n", "Setting up mesa-va-drivers:amd64 (21.2.6-0ubuntu0.1~20.04.2) ...\r\n", "Setting up libbluray2:amd64 (1:1.2.0-1) ...\r\n", "Setting up libsdl2-2.0-0:amd64 (2.0.10+dfsg1-3) ...\r\n", "Setting up libva-x11-2:amd64 (2.7.0-2) ...\r\n", "Setting up libwebpmux3:amd64 (0.6.1-2ubuntu0.20.04.3) ...\r\n", "Setting up libopenmpt0:amd64 (0.4.11-1build1) ...\r\n", "Setting up libzvbi-common (0.2.35-17) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Setting up i965-va-driver:amd64 (2.4.0-0ubuntu1) ...\r\n", "Setting up libserd-0-0:amd64 (0.30.2-1) ...\r\n", "Setting up libzvbi0:amd64 (0.2.35-17) ...\r\n", "Setting up libcdio-cdda2:amd64 (10.2+2.0.0-1) ...\r\n", "Setting up libcdio-paranoia2:amd64 (10.2+2.0.0-1) ...\r\n", "Setting up libopenal1:amd64 (1:1.19.1-1) ...\r\n", "Setting up libavutil56:amd64 (7:4.2.7-0ubuntu0.1) ...\r\n", "Setting up va-driver-all:amd64 (2.7.0-2) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Setting up libpostproc55:amd64 (7:4.2.7-0ubuntu0.1) ...\r\n", "Setting up libsord-0-0:amd64 (0.16.4-1) ...\r\n", "Setting up libsratom-0-0:amd64 (0.6.4-1) ...\r\n", "Setting up libswscale5:amd64 (7:4.2.7-0ubuntu0.1) ...\r\n", "Setting up liblilv-0-0:amd64 (0.24.6-1ubuntu0.1) ...\r\n", "Setting up libswresample3:amd64 (7:4.2.7-0ubuntu0.1) ...\r\n", "Setting up libavresample4:amd64 (7:4.2.7-0ubuntu0.1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Setting up libavcodec58:amd64 (7:4.2.7-0ubuntu0.1) ...\r\n", "Setting up libchromaprint1:amd64 (1.4.3-3build1) ...\r\n", "Setting up libavformat58:amd64 (7:4.2.7-0ubuntu0.1) ...\r\n", "Setting up libavfilter7:amd64 (7:4.2.7-0ubuntu0.1) ...\r\n", "Setting up libavdevice58:amd64 (7:4.2.7-0ubuntu0.1) ...\r\n", "Setting up ffmpeg (7:4.2.7-0ubuntu0.1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Processing triggers for man-db (2.9.1-1) ...\r\n" ] }, { "name": "stdout", "output_type": "stream", "text": [ "Processing triggers for libc-bin (2.31-0ubuntu9.12) ...\r\n" ] } ], "source": [ "!pip install mediapy\n", "!sudo apt-get install -y ffmpeg" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2024-03-09T12:17:54.860759Z", "iopub.status.busy": "2024-03-09T12:17:54.860145Z", "iopub.status.idle": "2024-03-09T12:17:57.989230Z", "shell.execute_reply": "2024-03-09T12:17:57.987893Z" }, "id": "BA1tq39MjOiF" }, "outputs": [], "source": [ "import tensorflow as tf\n", "import tensorflow_hub as hub\n", "\n", "import requests\n", "import numpy as np\n", "\n", "from typing import Generator, Iterable, List, Optional\n", "import mediapy as media" ] }, { "cell_type": "markdown", "metadata": { "id": "GTgXmeYGnT7q" }, "source": [ "## Load the model from TFHub\n", "\n", "To load a model from TensorFlow Hub you need the tfhub library and the model handle which is its documentation url." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2024-03-09T12:17:57.997854Z", "iopub.status.busy": "2024-03-09T12:17:57.995505Z", "iopub.status.idle": "2024-03-09T12:18:03.528916Z", "shell.execute_reply": "2024-03-09T12:18:03.528034Z" }, "id": "GojhvyAtjUt0" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2024-03-09 12:18:00.216249: E external/local_xla/xla/stream_executor/cuda/cuda_driver.cc:282] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected\n" ] } ], "source": [ "model = hub.load(\"https://tfhub.dev/google/film/1\")" ] }, { "cell_type": "markdown", "metadata": { "id": "DOQJPsu2CwPk" }, "source": [ "## Util function to load images from a url or locally\n", "\n", "This function loads an image and make it ready to be used by the model later." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2024-03-09T12:18:03.533485Z", "iopub.status.busy": "2024-03-09T12:18:03.532725Z", "iopub.status.idle": "2024-03-09T12:18:03.538714Z", "shell.execute_reply": "2024-03-09T12:18:03.538063Z" }, "id": "BPnh5uhQvFln" }, "outputs": [], "source": [ "_UINT8_MAX_F = float(np.iinfo(np.uint8).max)\n", "\n", "def load_image(img_url: str):\n", " \"\"\"Returns an image with shape [height, width, num_channels], with pixels in [0..1] range, and type np.float32.\"\"\"\n", "\n", " if (img_url.startswith(\"https\")):\n", " user_agent = {'User-agent': 'Colab Sample (https://tensorflow.org)'}\n", " response = requests.get(img_url, headers=user_agent)\n", " image_data = response.content\n", " else:\n", " image_data = tf.io.read_file(img_url)\n", "\n", " image = tf.io.decode_image(image_data, channels=3)\n", " image_numpy = tf.cast(image, dtype=tf.float32).numpy()\n", " return image_numpy / _UINT8_MAX_F\n" ] }, { "cell_type": "markdown", "metadata": { "id": "yjDFns1zp5y6" }, "source": [ "FILM's model input is a dictionary with the keys `time`, `x0`, `x1`:\n", "\n", "- `time`: position of the interpolated frame. Midway is `0.5`.\n", "- `x0`: is the initial frame.\n", "- `x1`: is the final frame.\n", "\n", "Both frames need to be normalized (done in the function `load_image` above) where each pixel is in the range of `[0..1]`.\n", "\n", "`time` is a value between `[0..1]` and it says where the generated image should be. 0.5 is midway between the input images.\n", "\n", "All three values need to have a batch dimension too." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2024-03-09T12:18:03.542052Z", "iopub.status.busy": "2024-03-09T12:18:03.541591Z", "iopub.status.idle": "2024-03-09T12:18:05.397625Z", "shell.execute_reply": "2024-03-09T12:18:05.396783Z" }, "id": "VEQNQlHGsWSM" }, "outputs": [], "source": [ "# using images from the FILM repository (https://github.com/google-research/frame-interpolation/)\n", "\n", "image_1_url = \"https://github.com/google-research/frame-interpolation/blob/main/photos/one.png?raw=true\"\n", "image_2_url = \"https://github.com/google-research/frame-interpolation/blob/main/photos/two.png?raw=true\"\n", "\n", "time = np.array([0.5], dtype=np.float32)\n", "\n", "image1 = load_image(image_1_url)\n", "image2 = load_image(image_2_url)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2024-03-09T12:18:05.401951Z", "iopub.status.busy": "2024-03-09T12:18:05.401227Z", "iopub.status.idle": "2024-03-09T12:18:12.907807Z", "shell.execute_reply": "2024-03-09T12:18:12.906978Z" }, "id": "r6_MQE9EuF_K" }, "outputs": [], "source": [ "input = {\n", " 'time': np.expand_dims(time, axis=0), # adding the batch dimension to the time\n", " 'x0': np.expand_dims(image1, axis=0), # adding the batch dimension to the image\n", " 'x1': np.expand_dims(image2, axis=0) # adding the batch dimension to the image\n", "}\n", "mid_frame = model(input)" ] }, { "cell_type": "markdown", "metadata": { "id": "nZkzYE2bptfD" }, "source": [ "The model outputs a couple of results but what you'll use here is the `image` key, whose value is the interpolated frame." ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "execution": { "iopub.execute_input": "2024-03-09T12:18:12.912254Z", "iopub.status.busy": "2024-03-09T12:18:12.911665Z", "iopub.status.idle": "2024-03-09T12:18:12.915789Z", "shell.execute_reply": "2024-03-09T12:18:12.915144Z" }, "id": "eClVbNFhA5Py" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "dict_keys(['forward_flow_pyramid', 'backward_residual_flow_pyramid', 'x0_warped', 'image', 'x1_warped', 'backward_flow_pyramid', 'forward_residual_flow_pyramid'])\n" ] } ], "source": [ "print(mid_frame.keys())" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "execution": { "iopub.execute_input": "2024-03-09T12:18:12.919151Z", "iopub.status.busy": "2024-03-09T12:18:12.918582Z", "iopub.status.idle": "2024-03-09T12:18:13.099625Z", "shell.execute_reply": "2024-03-09T12:18:13.098891Z" }, "id": "rE2csH3u8ePe" }, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", "
input image one
\n", "
\n", "
generated image
\n", "
\n", "
input image two
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "frames = [image1, mid_frame['image'][0].numpy(), image2]\n", "\n", "media.show_images(frames, titles=['input image one', 'generated image', 'input image two'], height=250)" ] }, { "cell_type": "markdown", "metadata": { "id": "fS1AT8kn-f_l" }, "source": [ "Let's create a video from the generated frames" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "execution": { "iopub.execute_input": "2024-03-09T12:18:13.107981Z", "iopub.status.busy": "2024-03-09T12:18:13.107330Z", "iopub.status.idle": "2024-03-09T12:18:13.562489Z", "shell.execute_reply": "2024-03-09T12:18:13.561520Z" }, "id": "oFc53B3p37SH" }, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", "
FILM interpolated video
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "media.show_video(frames, fps=3, title='FILM interpolated video')" ] }, { "cell_type": "markdown", "metadata": { "id": "x5AOFNkj-lfO" }, "source": [ "## Define a Frame Interpolator Library\n", "\n", "As you can see, the transition is not too smooth. \n", "\n", "To improve that you'll need many more interpolated frames.\n", "\n", "You could just keep running the model many times with intermediary images but there is a better solution.\n", "\n", "To generate many interpolated images and have a smoother video you'll create an interpolator library." ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "execution": { "iopub.execute_input": "2024-03-09T12:18:13.567502Z", "iopub.status.busy": "2024-03-09T12:18:13.566716Z", "iopub.status.idle": "2024-03-09T12:18:13.577636Z", "shell.execute_reply": "2024-03-09T12:18:13.576970Z" }, "id": "tsoDv_9geoZn" }, "outputs": [], "source": [ "\"\"\"A wrapper class for running a frame interpolation based on the FILM model on TFHub\n", "\n", "Usage:\n", " interpolator = Interpolator()\n", " result_batch = interpolator(image_batch_0, image_batch_1, batch_dt)\n", " Where image_batch_1 and image_batch_2 are numpy tensors with TF standard\n", " (B,H,W,C) layout, batch_dt is the sub-frame time in range [0..1], (B,) layout.\n", "\"\"\"\n", "\n", "\n", "def _pad_to_align(x, align):\n", " \"\"\"Pads image batch x so width and height divide by align.\n", "\n", " Args:\n", " x: Image batch to align.\n", " align: Number to align to.\n", "\n", " Returns:\n", " 1) An image padded so width % align == 0 and height % align == 0.\n", " 2) A bounding box that can be fed readily to tf.image.crop_to_bounding_box\n", " to undo the padding.\n", " \"\"\"\n", " # Input checking.\n", " assert np.ndim(x) == 4\n", " assert align > 0, 'align must be a positive number.'\n", "\n", " height, width = x.shape[-3:-1]\n", " height_to_pad = (align - height % align) if height % align != 0 else 0\n", " width_to_pad = (align - width % align) if width % align != 0 else 0\n", "\n", " bbox_to_pad = {\n", " 'offset_height': height_to_pad // 2,\n", " 'offset_width': width_to_pad // 2,\n", " 'target_height': height + height_to_pad,\n", " 'target_width': width + width_to_pad\n", " }\n", " padded_x = tf.image.pad_to_bounding_box(x, **bbox_to_pad)\n", " bbox_to_crop = {\n", " 'offset_height': height_to_pad // 2,\n", " 'offset_width': width_to_pad // 2,\n", " 'target_height': height,\n", " 'target_width': width\n", " }\n", " return padded_x, bbox_to_crop\n", "\n", "\n", "class Interpolator:\n", " \"\"\"A class for generating interpolated frames between two input frames.\n", "\n", " Uses the Film model from TFHub\n", " \"\"\"\n", "\n", " def __init__(self, align: int = 64) -> None:\n", " \"\"\"Loads a saved model.\n", "\n", " Args:\n", " align: 'If >1, pad the input size so it divides with this before\n", " inference.'\n", " \"\"\"\n", " self._model = hub.load(\"https://tfhub.dev/google/film/1\")\n", " self._align = align\n", "\n", " def __call__(self, x0: np.ndarray, x1: np.ndarray,\n", " dt: np.ndarray) -> np.ndarray:\n", " \"\"\"Generates an interpolated frame between given two batches of frames.\n", "\n", " All inputs should be np.float32 datatype.\n", "\n", " Args:\n", " x0: First image batch. Dimensions: (batch_size, height, width, channels)\n", " x1: Second image batch. Dimensions: (batch_size, height, width, channels)\n", " dt: Sub-frame time. Range [0,1]. Dimensions: (batch_size,)\n", "\n", " Returns:\n", " The result with dimensions (batch_size, height, width, channels).\n", " \"\"\"\n", " if self._align is not None:\n", " x0, bbox_to_crop = _pad_to_align(x0, self._align)\n", " x1, _ = _pad_to_align(x1, self._align)\n", "\n", " inputs = {'x0': x0, 'x1': x1, 'time': dt[..., np.newaxis]}\n", " result = self._model(inputs, training=False)\n", " image = result['image']\n", "\n", " if self._align is not None:\n", " image = tf.image.crop_to_bounding_box(image, **bbox_to_crop)\n", " return image.numpy()" ] }, { "cell_type": "markdown", "metadata": { "id": "ZeGYaNBd_7a5" }, "source": [ "## Frame and Video Generation Utility Functions" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "execution": { "iopub.execute_input": "2024-03-09T12:18:13.581156Z", "iopub.status.busy": "2024-03-09T12:18:13.580666Z", "iopub.status.idle": "2024-03-09T12:18:13.588038Z", "shell.execute_reply": "2024-03-09T12:18:13.587406Z" }, "id": "gOJxup6s_1DP" }, "outputs": [], "source": [ "def _recursive_generator(\n", " frame1: np.ndarray, frame2: np.ndarray, num_recursions: int,\n", " interpolator: Interpolator) -> Generator[np.ndarray, None, None]:\n", " \"\"\"Splits halfway to repeatedly generate more frames.\n", "\n", " Args:\n", " frame1: Input image 1.\n", " frame2: Input image 2.\n", " num_recursions: How many times to interpolate the consecutive image pairs.\n", " interpolator: The frame interpolator instance.\n", "\n", " Yields:\n", " The interpolated frames, including the first frame (frame1), but excluding\n", " the final frame2.\n", " \"\"\"\n", " if num_recursions == 0:\n", " yield frame1\n", " else:\n", " # Adds the batch dimension to all inputs before calling the interpolator,\n", " # and remove it afterwards.\n", " time = np.full(shape=(1,), fill_value=0.5, dtype=np.float32)\n", " mid_frame = interpolator(\n", " np.expand_dims(frame1, axis=0), np.expand_dims(frame2, axis=0), time)[0]\n", " yield from _recursive_generator(frame1, mid_frame, num_recursions - 1,\n", " interpolator)\n", " yield from _recursive_generator(mid_frame, frame2, num_recursions - 1,\n", " interpolator)\n", "\n", "\n", "def interpolate_recursively(\n", " frames: List[np.ndarray], num_recursions: int,\n", " interpolator: Interpolator) -> Iterable[np.ndarray]:\n", " \"\"\"Generates interpolated frames by repeatedly interpolating the midpoint.\n", "\n", " Args:\n", " frames: List of input frames. Expected shape (H, W, 3). The colors should be\n", " in the range[0, 1] and in gamma space.\n", " num_recursions: Number of times to do recursive midpoint\n", " interpolation.\n", " interpolator: The frame interpolation model to use.\n", "\n", " Yields:\n", " The interpolated frames (including the inputs).\n", " \"\"\"\n", " n = len(frames)\n", " for i in range(1, n):\n", " yield from _recursive_generator(frames[i - 1], frames[i],\n", " times_to_interpolate, interpolator)\n", " # Separately yield the final frame.\n", " yield frames[-1]" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "execution": { "iopub.execute_input": "2024-03-09T12:18:13.591343Z", "iopub.status.busy": "2024-03-09T12:18:13.590779Z", "iopub.status.idle": "2024-03-09T12:18:16.839624Z", "shell.execute_reply": "2024-03-09T12:18:16.838710Z" }, "id": "X1R2KjhEAHu0" }, "outputs": [], "source": [ "times_to_interpolate = 6\n", "interpolator = Interpolator()" ] }, { "cell_type": "markdown", "metadata": { "id": "AZUo8tg1AYvZ" }, "source": [ "## Running the Interpolator" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "execution": { "iopub.execute_input": "2024-03-09T12:18:16.844162Z", "iopub.status.busy": "2024-03-09T12:18:16.843493Z", "iopub.status.idle": "2024-03-09T12:23:33.778243Z", "shell.execute_reply": "2024-03-09T12:23:33.777362Z" }, "id": "QMMNjs7sAWTG" }, "outputs": [], "source": [ "input_frames = [image1, image2]\n", "frames = list(\n", " interpolate_recursively(input_frames, times_to_interpolate,\n", " interpolator))\n" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "execution": { "iopub.execute_input": "2024-03-09T12:23:33.783012Z", "iopub.status.busy": "2024-03-09T12:23:33.782281Z", "iopub.status.idle": "2024-03-09T12:23:35.855835Z", "shell.execute_reply": "2024-03-09T12:23:35.854713Z" }, "id": "s9mHHyCAAhrM" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "video with 65 frames\n" ] }, { "data": { "text/html": [ "
\n", "
\n", "
FILM interpolated video
" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "print(f'video with {len(frames)} frames')\n", "media.show_video(frames, fps=30, title='FILM interpolated video')" ] }, { "cell_type": "markdown", "metadata": { "id": "_0AZKeMVFwAc" }, "source": [ "For more information, you can visit [FILM's model repository](https://github.com/google-research/frame-interpolation).\n" ] }, { "cell_type": "markdown", "metadata": { "id": "8764ry3SGDks" }, "source": [ "## Citation\n", "\n", "If you find this model and code useful in your works, please acknowledge it appropriately by citing:\n", "\n", "```\n", "@inproceedings{reda2022film,\n", " title = {FILM: Frame Interpolation for Large Motion},\n", " author = {Fitsum Reda and Janne Kontkanen and Eric Tabellion and Deqing Sun and Caroline Pantofaru and Brian Curless},\n", " booktitle = {The European Conference on Computer Vision (ECCV)},\n", " year = {2022}\n", "}\n", "```\n", "\n", "```\n", "@misc{film-tf,\n", " title = {Tensorflow 2 Implementation of \"FILM: Frame Interpolation for Large Motion\"},\n", " author = {Fitsum Reda and Janne Kontkanen and Eric Tabellion and Deqing Sun and Caroline Pantofaru and Brian Curless},\n", " year = {2022},\n", " publisher = {GitHub},\n", " journal = {GitHub repository},\n", " howpublished = {\\url{https://github.com/google-research/frame-interpolation}}\n", "}\n", "```" ] } ], "metadata": { "accelerator": "GPU", "colab": { "name": "tf_hub_film_example.ipynb", "toc_visible": true }, "kernelspec": { "display_name": "Python 3", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.18" } }, "nbformat": 4, "nbformat_minor": 0 }