{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VFMCdVJIIraw"
   },
   "source": [
    "##### Copyright 2019 The TensorFlow Hub Authors.\n",
    "\n",
    "Licensed under the Apache License, Version 2.0 (the \"License\");"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "cellView": "code",
    "execution": {
     "iopub.execute_input": "2023-11-07T18:07:32.706260Z",
     "iopub.status.busy": "2023-11-07T18:07:32.705746Z",
     "iopub.status.idle": "2023-11-07T18:07:32.709844Z",
     "shell.execute_reply": "2023-11-07T18:07:32.709237Z"
    },
    "id": "ZxMYj8OpIrCp"
   },
   "outputs": [],
   "source": [
    "# Copyright 2019 The TensorFlow Hub Authors. All Rights Reserved.\n",
    "#\n",
    "# Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "#     http://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License.\n",
    "# =============================================================================="
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0fO2R2BBKx3l"
   },
   "source": [
    "# Multilingual Universal Sentence Encoder Q&A 检索\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MfBg1C5NB3X0"
   },
   "source": [
    "<table class=\"tfo-notebook-buttons\" align=\"left\">\n",
    "  <td>     <a target=\"_blank\" href=\"https://tensorflow.google.cn/hub/tutorials/retrieval_with_tf_hub_universal_encoder_qa\"><img src=\"https://tensorflow.google.cn/images/tf_logo_32px.png\">在 TensorFlow.org 上查看</a> </td>\n",
    "  <td><a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs-l10n/blob/master/site/zh-cn/hub/tutorials/retrieval_with_tf_hub_universal_encoder_qa.ipynb\"><img src=\"https://tensorflow.google.cn/images/colab_logo_32px.png\">在 Google Colab 中运行</a></td>\n",
    "  <td>     <a target=\"_blank\" href=\"https://github.com/tensorflow/docs-l10n/blob/master/site/zh-cn/hub/tutorials/retrieval_with_tf_hub_universal_encoder_qa.ipynb\"><img src=\"https://tensorflow.google.cn/images/GitHub-Mark-32px.png\">在 GitHub 上查看源代码</a> </td>\n",
    "  <td>     <a href=\"https://storage.googleapis.com/tensorflow_docs/docs-l10n/site/zh-cn/hub/tutorials/retrieval_with_tf_hub_universal_encoder_qa.ipynb\"><img src=\"https://tensorflow.google.cn/images/download_logo_32px.png\">下载笔记本</a> </td>\n",
    "  <td data-parent-segment-id=\"12900598\">     <a href=\"https://tfhub.dev/s?q=google%2Funiversal-sentence-encoder-multilingual-qa%2F3%20OR%20google%2Funiversal-sentence-encoder-qa%2F3\"><img src=\"https://tensorflow.google.cn/images/hub_logo_32px.png\">\t查看 TF Hub 模型</a> </td>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zsDm_WgMNlJQ"
   },
   "source": [
    "这是使用 [Univeral Encoder Multilingual Q&amp;A 模型](https://tfhub.dev/google/universal-sentence-encoder-multilingual-qa/3)进行文本问答检索的演示，其中对模型的 **question_encoder** 和 **response_encoder** 的用法进行了说明。我们使用来自 [SQuAD](https://rajpurkar.github.io/SQuAD-explorer/) 段落的句子作为演示数据集，每个句子及其上下文（句子周围的文本）都使用 **response_encoder** 编码为高维嵌入向量。这些嵌入向量存储在使用 [simpleneighbors](https://pypi.org/project/simpleneighbors/) 库构建的索引中，用于问答检索。\n",
    "\n",
    "检索时，从 [SQuAD](https://rajpurkar.github.io/SQuAD-explorer/) 数据集中随机选择一个问题，并使用 **question_encoder** 将其编码为高维嵌入向量，然后查询 simpleneighbors 索引会返回语义空间中最近邻的列表。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "U0eOW2LTWiLg"
   },
   "source": [
    "### 更多模型\n",
    "\n",
    "您可以在[此处](https://tfhub.dev/s?module-type=text-embedding)找到所有当前托管的文本嵌入向量模型，还可以在[此处](https://tfhub.dev/s?dataset=squad)找到所有在 SQuADYou 上训练过的模型。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ORy-KvWXGXBo"
   },
   "source": [
    "## 安装\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "cellView": "both",
    "execution": {
     "iopub.execute_input": "2023-11-07T18:07:32.713935Z",
     "iopub.status.busy": "2023-11-07T18:07:32.713371Z",
     "iopub.status.idle": "2023-11-07T18:08:08.771758Z",
     "shell.execute_reply": "2023-11-07T18:08:08.770756Z"
    },
    "id": "x00t_uJCEbeb"
   },
   "outputs": [],
   "source": [
    "%%capture\n",
    "#@title Setup Environment\n",
    "# Install the latest Tensorflow version.\n",
    "!pip install -q \"tensorflow-text==2.11.*\"\n",
    "!pip install -q simpleneighbors[annoy]\n",
    "!pip install -q nltk\n",
    "!pip install -q tqdm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "cellView": "form",
    "execution": {
     "iopub.execute_input": "2023-11-07T18:08:08.775916Z",
     "iopub.status.busy": "2023-11-07T18:08:08.775510Z",
     "iopub.status.idle": "2023-11-07T18:08:12.053101Z",
     "shell.execute_reply": "2023-11-07T18:08:12.052280Z"
    },
    "id": "DmeFAuVsyWxg"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-11-07 18:08:09.796782: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-11-07 18:08:10.465902: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory\n",
      "2023-11-07 18:08:10.465997: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory\n",
      "2023-11-07 18:08:10.466007: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package punkt to /home/kbuilder/nltk_data...\n",
      "[nltk_data]   Unzipping tokenizers/punkt.zip.\n"
     ]
    }
   ],
   "source": [
    "#@title Setup common imports and functions\n",
    "import json\n",
    "import nltk\n",
    "import os\n",
    "import pprint\n",
    "import random\n",
    "import simpleneighbors\n",
    "import urllib\n",
    "from IPython.display import HTML, display\n",
    "from tqdm.notebook import tqdm\n",
    "\n",
    "import tensorflow.compat.v2 as tf\n",
    "import tensorflow_hub as hub\n",
    "from tensorflow_text import SentencepieceTokenizer\n",
    "\n",
    "nltk.download('punkt')\n",
    "\n",
    "\n",
    "def download_squad(url):\n",
    "  return json.load(urllib.request.urlopen(url))\n",
    "\n",
    "def extract_sentences_from_squad_json(squad):\n",
    "  all_sentences = []\n",
    "  for data in squad['data']:\n",
    "    for paragraph in data['paragraphs']:\n",
    "      sentences = nltk.tokenize.sent_tokenize(paragraph['context'])\n",
    "      all_sentences.extend(zip(sentences, [paragraph['context']] * len(sentences)))\n",
    "  return list(set(all_sentences)) # remove duplicates\n",
    "\n",
    "def extract_questions_from_squad_json(squad):\n",
    "  questions = []\n",
    "  for data in squad['data']:\n",
    "    for paragraph in data['paragraphs']:\n",
    "      for qas in paragraph['qas']:\n",
    "        if qas['answers']:\n",
    "          questions.append((qas['question'], qas['answers'][0]['text']))\n",
    "  return list(set(questions))\n",
    "\n",
    "def output_with_highlight(text, highlight):\n",
    "  output = \"<li> \"\n",
    "  i = text.find(highlight)\n",
    "  while True:\n",
    "    if i == -1:\n",
    "      output += text\n",
    "      break\n",
    "    output += text[0:i]\n",
    "    output += '<b>'+text[i:i+len(highlight)]+'</b>'\n",
    "    text = text[i+len(highlight):]\n",
    "    i = text.find(highlight)\n",
    "  return output + \"</li>\\n\"\n",
    "\n",
    "def display_nearest_neighbors(query_text, answer_text=None):\n",
    "  query_embedding = model.signatures['question_encoder'](tf.constant([query_text]))['outputs'][0]\n",
    "  search_results = index.nearest(query_embedding, n=num_results)\n",
    "\n",
    "  if answer_text:\n",
    "    result_md = '''\n",
    "    <p>Random Question from SQuAD:</p>\n",
    "    <p>&nbsp;&nbsp;<b>%s</b></p>\n",
    "    <p>Answer:</p>\n",
    "    <p>&nbsp;&nbsp;<b>%s</b></p>\n",
    "    ''' % (query_text , answer_text)\n",
    "  else:\n",
    "    result_md = '''\n",
    "    <p>Question:</p>\n",
    "    <p>&nbsp;&nbsp;<b>%s</b></p>\n",
    "    ''' % query_text\n",
    "\n",
    "  result_md += '''\n",
    "    <p>Retrieved sentences :\n",
    "    <ol>\n",
    "  '''\n",
    "\n",
    "  if answer_text:\n",
    "    for s in search_results:\n",
    "      result_md += output_with_highlight(s, answer_text)\n",
    "  else:\n",
    "    for s in search_results:\n",
    "      result_md += '<li>' + s + '</li>\\n'\n",
    "\n",
    "  result_md += \"</ol>\"\n",
    "  display(HTML(result_md))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1kbkT8i3FL_C"
   },
   "source": [
    "运行以下代码块，下载并将 SQuAD 数据集提取到：\n",
    "\n",
    "- **句子**是（文本, 上下文）元组的列表，SQuAD 数据集中的每个段落都用 NLTK 库拆分成句子，并且句子和段落文本构成（文本, 上下文）元组。\n",
    "- **问题**是（问题, 答案）元组的列表。\n",
    "\n",
    "注：您可以选择下面的 **squad_url**，使用本演示为 SQuAD 训练数据集或较小的 dev 数据集（1.1 或 2.0）建立索引。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "cellView": "both",
    "execution": {
     "iopub.execute_input": "2023-11-07T18:08:12.057802Z",
     "iopub.status.busy": "2023-11-07T18:08:12.056980Z",
     "iopub.status.idle": "2023-11-07T18:08:12.584442Z",
     "shell.execute_reply": "2023-11-07T18:08:12.583735Z"
    },
    "id": "iYqV2GAty_Eh"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10455 sentences, 10552 questions extracted from SQuAD https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v1.1.json\n",
      "\n",
      "Example sentence and context:\n",
      "\n",
      "sentence:\n",
      "\n",
      "('While official data does not capture the real extent of private schooling in '\n",
      " 'the country, various studies have reported unpopularity of government '\n",
      " 'schools and an increasing number of private schools.')\n",
      "\n",
      "context:\n",
      "\n",
      "('Legally, only non-profit trusts and societies can run schools in India. They '\n",
      " 'will have to satisfy a number of infrastructure and human resource related '\n",
      " 'criteria to get Recognition (a form of license) from the government. Critics '\n",
      " 'of this system point out that this leads to corruption by school inspectors '\n",
      " 'who check compliance and to fewer schools in a country that has the largest '\n",
      " 'adult illiterate population in the world. While official data does not '\n",
      " 'capture the real extent of private schooling in the country, various studies '\n",
      " 'have reported unpopularity of government schools and an increasing number of '\n",
      " 'private schools. The Annual Status of Education Report (ASER), which '\n",
      " 'evaluates learning levels in rural India, has been reporting poorer academic '\n",
      " 'achievement in government schools than in private schools. A key difference '\n",
      " 'between the government and private schools is that the medium of education '\n",
      " 'in private schools is English while it is the local language in government '\n",
      " 'schools.')\n",
      "\n"
     ]
    }
   ],
   "source": [
    "#@title Download and extract SQuAD data\n",
    "squad_url = 'https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v1.1.json' #@param [\"https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v2.0.json\", \"https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v2.0.json\", \"https://rajpurkar.github.io/SQuAD-explorer/dataset/train-v1.1.json\", \"https://rajpurkar.github.io/SQuAD-explorer/dataset/dev-v1.1.json\"]\n",
    "\n",
    "squad_json = download_squad(squad_url)\n",
    "sentences = extract_sentences_from_squad_json(squad_json)\n",
    "questions = extract_questions_from_squad_json(squad_json)\n",
    "print(\"%s sentences, %s questions extracted from SQuAD %s\" % (len(sentences), len(questions), squad_url))\n",
    "\n",
    "print(\"\\nExample sentence and context:\\n\")\n",
    "sentence = random.choice(sentences)\n",
    "print(\"sentence:\\n\")\n",
    "pprint.pprint(sentence[0])\n",
    "print(\"\\ncontext:\\n\")\n",
    "pprint.pprint(sentence[1])\n",
    "print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "9x3u-2uSGbDf"
   },
   "source": [
    "以下代码块使用 [Universal Encoder Multilingual Q&amp;A 模型](https://tfhub.dev/google/universal-sentence-encoder-multilingual-qa/3)的 **question_encoder** 和 **response_encoder** 签名对 TensorFlow 计算图 **g** 和**会话**进行设置。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-07T18:08:12.588453Z",
     "iopub.status.busy": "2023-11-07T18:08:12.587680Z",
     "iopub.status.idle": "2023-11-07T18:08:24.419807Z",
     "shell.execute_reply": "2023-11-07T18:08:24.418910Z"
    },
    "id": "44I0uCRQRiFO"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2023-11-07 18:08:17.196246: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory\n",
      "2023-11-07 18:08:17.196359: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcublas.so.11'; dlerror: libcublas.so.11: cannot open shared object file: No such file or directory\n",
      "2023-11-07 18:08:17.196424: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcublasLt.so.11'; dlerror: libcublasLt.so.11: cannot open shared object file: No such file or directory\n",
      "2023-11-07 18:08:17.196485: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcufft.so.10'; dlerror: libcufft.so.10: cannot open shared object file: No such file or directory\n",
      "2023-11-07 18:08:17.255270: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcusparse.so.11'; dlerror: libcusparse.so.11: cannot open shared object file: No such file or directory\n",
      "2023-11-07 18:08:17.255499: W tensorflow/core/common_runtime/gpu/gpu_device.cc:1934] Cannot dlopen some GPU libraries. Please make sure the missing libraries mentioned above are installed properly if you would like to use GPU. Follow the guide at https://www.tensorflow.org/install/gpu for how to download and setup the required libraries for your platform.\n",
      "Skipping registering GPU devices...\n"
     ]
    }
   ],
   "source": [
    "#@title Load model from tensorflow hub\n",
    "module_url = \"https://tfhub.dev/google/universal-sentence-encoder-multilingual-qa/3\" #@param [\"https://tfhub.dev/google/universal-sentence-encoder-multilingual-qa/3\", \"https://tfhub.dev/google/universal-sentence-encoder-qa/3\"]\n",
    "model = hub.load(module_url)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SCQpDmTZG0O6"
   },
   "source": [
    "以下代码块计算所有文本的嵌入向量和上下文元组，并使用 <strong>response_encoder</strong> 将它们存储在 [simpleneighbors](https://pypi.org/project/simpleneighbors/) 索引中。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-11-07T18:08:24.424279Z",
     "iopub.status.busy": "2023-11-07T18:08:24.423916Z",
     "iopub.status.idle": "2023-11-07T18:28:25.924812Z",
     "shell.execute_reply": "2023-11-07T18:28:25.923753Z"
    },
    "id": "FwDUryIfSLp2"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Computing embeddings for 10455 sentences\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "55e129e860264b2083a6ecca8818a628",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/104 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "simpleneighbors index for 10455 sentences built.\n"
     ]
    }
   ],
   "source": [
    "#@title Compute embeddings and build simpleneighbors index\n",
    "batch_size = 100\n",
    "\n",
    "encodings = model.signatures['response_encoder'](\n",
    "  input=tf.constant([sentences[0][0]]),\n",
    "  context=tf.constant([sentences[0][1]]))\n",
    "index = simpleneighbors.SimpleNeighbors(\n",
    "    len(encodings['outputs'][0]), metric='angular')\n",
    "\n",
    "print('Computing embeddings for %s sentences' % len(sentences))\n",
    "slices = zip(*(iter(sentences),) * batch_size)\n",
    "num_batches = int(len(sentences) / batch_size)\n",
    "for s in tqdm(slices, total=num_batches):\n",
    "  response_batch = list([r for r, c in s])\n",
    "  context_batch = list([c for r, c in s])\n",
    "  encodings = model.signatures['response_encoder'](\n",
    "    input=tf.constant(response_batch),\n",
    "    context=tf.constant(context_batch)\n",
    "  )\n",
    "  for batch_index, batch in enumerate(response_batch):\n",
    "    index.add_one(batch, encodings['outputs'][batch_index])\n",
    "\n",
    "index.build()\n",
    "print('simpleneighbors index for %s sentences built.' % len(sentences))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ZkNcjoPzHJpP"
   },
   "source": [
    "检索时，使用 **question_encoder** 对问题进行编码，而问题嵌入向量用于查询 simpleneighbors 索引。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "cellView": "both",
    "execution": {
     "iopub.execute_input": "2023-11-07T18:28:25.928818Z",
     "iopub.status.busy": "2023-11-07T18:28:25.928270Z",
     "iopub.status.idle": "2023-11-07T18:28:27.591899Z",
     "shell.execute_reply": "2023-11-07T18:28:27.590876Z"
    },
    "id": "J0xTw2w3UViK"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "    <p>Random Question from SQuAD:</p>\n",
       "    <p>&nbsp;&nbsp;<b>Who was the MVP of Super Bowl I and II?</b></p>\n",
       "    <p>Answer:</p>\n",
       "    <p>&nbsp;&nbsp;<b>Bart Starr</b></p>\n",
       "    \n",
       "    <p>Retrieved sentences :\n",
       "    <ol>\n",
       "  <li> <b>Bart Starr</b> (MVP of Super Bowls I and II) and Chuck Howley (MVP of Super Bowl V) appeared via video, while Peyton Manning (MVP of Super Bowl XLI and current Broncos quarterback) was shown in the locker room preparing for the game.</li>\n",
       "<li> Denver linebacker Von Miller was named Super Bowl MVP, recording five solo tackles, 2½ sacks, and two forced fumbles.</li>\n",
       "<li> This was the first Super Bowl to feature a quarterback on both teams who was the #1 pick in their draft classes.</li>\n",
       "<li> Peyton Manning became the first quarterback ever to lead two different teams to multiple Super Bowls.</li>\n",
       "<li> Kony Ealy tied a Super Bowl record with three sacks.</li>\n",
       "<li> No plans were announced regarding the recognition of Harvey Martin, co-MVP of Super Bowl XII, who died in 2001.</li>\n",
       "<li> Manning became the oldest quarterback ever to win a Super Bowl at age 39, and the first quarterback ever to win a Super Bowl with two different teams, while Gary Kubiak became the first head coach to win a Super Bowl with the same franchise he went to the Super Bowl with as a player.</li>\n",
       "<li> The Broncos made their second Super Bowl appearance in three years, having reached Super Bowl XLVIII, while the Panthers made their second Super Bowl appearance in franchise history, their other appearance being Super Bowl XXXVIII.</li>\n",
       "<li> In honor of the 50th Super Bowl, the pregame ceremony featured the on-field introduction of 39 of the 43 previous Super Bowl Most Valuable Players.</li>\n",
       "<li> This was the first fumble return touchdown in a Super Bowl since Super Bowl XXVIII at the end of the 1993 season.</li>\n",
       "<li> The Broncos took an early lead in Super Bowl 50 and never trailed.</li>\n",
       "<li> Cornerbacks Aqib Talib (three interceptions) and Chris Harris, Jr. (two interceptions) were the other two Pro Bowl selections from the defense.</li>\n",
       "<li> They defeated the Arizona Cardinals 49–15 in the NFC Championship Game and advanced to their second Super Bowl appearance since the franchise was founded in 1995.</li>\n",
       "<li> Jordan Norwood's 61-yard punt return set a new record, surpassing the old record of 45 yards set by John Taylor in Super Bowl XXIII.</li>\n",
       "<li> Manning was the #1 selection of the 1998 NFL draft, while Newton was picked first in 2011.</li>\n",
       "<li> Coincidentally, both teams were coached by John Fox in their last Super Bowl appearance prior to Super Bowl 50.</li>\n",
       "<li> Denver took the opening kickoff and started out strong with Peyton Manning completing an 18-yard pass to tight end Owen Daniels and a 22-yard throw to receiver Andre Caldwell.</li>\n",
       "<li> The two teams' combined third down conversion percentage of 13.8 was a Super Bowl low.</li>\n",
       "<li> Despite Manning's problems with interceptions during the season, he didn't throw any in their two playoff games.</li>\n",
       "<li> The Panthers finished the regular season with a 15–1 record, and quarterback Cam Newton was named the NFL Most Valuable Player (MVP).</li>\n",
       "<li> They then beat the defending Super Bowl XLIX champion New England Patriots in the AFC Championship Game, 20–18, by intercepting a pass on New England's 2-point conversion attempt with 17 seconds left on the clock.</li>\n",
       "<li> Carolina's secondary featured Pro Bowl safety Kurt Coleman, who led the team with a career high seven interceptions, while also racking up 88 tackles and Pro Bowl cornerback Josh Norman, who developed into a shutdown corner during the season and had four interceptions, two of which were returned for touchdowns.</li>\n",
       "<li> For the third straight season, the number one seeds from both conferences met in the Super Bowl.</li>\n",
       "<li> The Panthers offense, which led the NFL in scoring (500 points), was loaded with talent, boasting six Pro Bowl selections.</li>\n",
       "<li> Linebacker DeMarcus Ware was selected to play in the Pro Bowl for the ninth time in his career, ranking second on the team with 7½ sacks.</li>\n",
       "</ol>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#@title Retrieve nearest neighbors for a random question from SQuAD\n",
    "num_results = 25 #@param {type:\"slider\", min:5, max:40, step:1}\n",
    "\n",
    "query = random.choice(questions)\n",
    "display_nearest_neighbors(query[0], query[1])"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [
    "VFMCdVJIIraw"
   ],
   "name": "retrieval_with_tf_hub_universal_encoder_qa.ipynb",
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.18"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "1c110858e013456dad3fd8818daab72c": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HTMLStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "StyleView",
       "background": null,
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "1d737b5d38b14e3e9c6f1531e36146e3": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "2.0.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border_bottom": null,
       "border_left": null,
       "border_right": null,
       "border_top": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "3392ecedd7ab47b1a3e3c95f5622c1df": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "2.0.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_allow_html": false,
       "layout": "IPY_MODEL_e94fe5a5a85b4578ab6ae199ded5bcc1",
       "placeholder": "​",
       "style": "IPY_MODEL_e75c917ae3d24a02b846936723cac4b4",
       "tabbable": null,
       "tooltip": null,
       "value": " 104/104 [19:59&lt;00:00, 11.59s/it]"
      }
     },
     "3d810c92292b4442810e3454a97b6934": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "2.0.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border_bottom": null,
       "border_left": null,
       "border_right": null,
       "border_top": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "41e58c6262054ce4a96fb743842ee22f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "FloatProgressModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "FloatProgressModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "2.0.0",
       "_view_name": "ProgressView",
       "bar_style": "success",
       "description": "",
       "description_allow_html": false,
       "layout": "IPY_MODEL_d384fa2cd4c644a4b00259ad2ae35c56",
       "max": 104.0,
       "min": 0.0,
       "orientation": "horizontal",
       "style": "IPY_MODEL_d80c03a5363d44629a40e006b3bb2b22",
       "tabbable": null,
       "tooltip": null,
       "value": 104.0
      }
     },
     "55e129e860264b2083a6ecca8818a628": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HBoxModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HBoxModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "2.0.0",
       "_view_name": "HBoxView",
       "box_style": "",
       "children": [
        "IPY_MODEL_c89c74d2827f431b831f68bb9a706c9f",
        "IPY_MODEL_41e58c6262054ce4a96fb743842ee22f",
        "IPY_MODEL_3392ecedd7ab47b1a3e3c95f5622c1df"
       ],
       "layout": "IPY_MODEL_1d737b5d38b14e3e9c6f1531e36146e3",
       "tabbable": null,
       "tooltip": null
      }
     },
     "c89c74d2827f431b831f68bb9a706c9f": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HTMLModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/controls",
       "_view_module_version": "2.0.0",
       "_view_name": "HTMLView",
       "description": "",
       "description_allow_html": false,
       "layout": "IPY_MODEL_3d810c92292b4442810e3454a97b6934",
       "placeholder": "​",
       "style": "IPY_MODEL_1c110858e013456dad3fd8818daab72c",
       "tabbable": null,
       "tooltip": null,
       "value": "100%"
      }
     },
     "d384fa2cd4c644a4b00259ad2ae35c56": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "2.0.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border_bottom": null,
       "border_left": null,
       "border_right": null,
       "border_top": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "d80c03a5363d44629a40e006b3bb2b22": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "ProgressStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "ProgressStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "StyleView",
       "bar_color": null,
       "description_width": ""
      }
     },
     "e75c917ae3d24a02b846936723cac4b4": {
      "model_module": "@jupyter-widgets/controls",
      "model_module_version": "2.0.0",
      "model_name": "HTMLStyleModel",
      "state": {
       "_model_module": "@jupyter-widgets/controls",
       "_model_module_version": "2.0.0",
       "_model_name": "HTMLStyleModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "StyleView",
       "background": null,
       "description_width": "",
       "font_size": null,
       "text_color": null
      }
     },
     "e94fe5a5a85b4578ab6ae199ded5bcc1": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "2.0.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "2.0.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "2.0.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border_bottom": null,
       "border_left": null,
       "border_right": null,
       "border_top": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}