{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "moGQn-psFggi"
   },
   "source": [
    "##### Copyright 2020 The TensorFlow Authors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "cellView": "form",
    "execution": {
     "iopub.execute_input": "2024-07-19T09:16:00.339771Z",
     "iopub.status.busy": "2024-07-19T09:16:00.339375Z",
     "iopub.status.idle": "2024-07-19T09:16:00.343290Z",
     "shell.execute_reply": "2024-07-19T09:16:00.342700Z"
    },
    "id": "cUGG66UTFwJ-"
   },
   "outputs": [],
   "source": [
    "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
    "# you may not use this file except in compliance with the License.\n",
    "# You may obtain a copy of the License at\n",
    "#\n",
    "# https://www.apache.org/licenses/LICENSE-2.0\n",
    "#\n",
    "# Unless required by applicable law or agreed to in writing, software\n",
    "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
    "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
    "# See the License for the specific language governing permissions and\n",
    "# limitations under the License."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YpbnPF_MEv4h"
   },
   "source": [
    "# Model Remediation Case Study\n",
    "\n",
    "<div class=\"devsite-table-wrapper\"><table class=\"tfo-notebook-buttons\" align=\"left\">\n",
    "  <td><a target=\"_blank\" href=\"https://www.tensorflow.org/responsible_ai/model_remediation/min_diff/tutorials/min_diff_keras\">\n",
    "  <img src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" />View on TensorFlow.org</a>\n",
    "</td>\n",
    "<td>\n",
    "  <a target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/model-remediation/blob/master/docs/min_diff/tutorials/min_diff_keras.ipynb\">\n",
    "  <img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\">Run in Google Colab</a>\n",
    "</td>\n",
    "<td>\n",
    "  <a target=\"_blank\" href=\"https://github.com/tensorflow/model-remediation/blob/master/docs/min_diff/tutorials/min_diff_keras.ipynb\">\n",
    "  <img width=32px src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\">View source on GitHub</a>\n",
    "</td>\n",
    "<td>\n",
    "  <a target=\"_blank\" href=\"https://storage.googleapis.com/tensorflow_docs/model-remediation/docs/min_diff/tutorials/min_diff_keras.ipynb\"><img src=\"https://www.tensorflow.org/images/download_logo_32px.png\" />Download notebook</a>\n",
    "</td>\n",
    "</table></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yMcQGRPHnjP9"
   },
   "source": [
    "In this notebook, we’ll train a text classifier to identify written content that could be considered toxic or harmful, and apply MinDiff to remediate some fairness concerns. In our workflow, we will:\n",
    "1.   Evaluate our baseline model’s performance on text containing references to sensitive groups. \n",
    "2.   Improve performance on any underperforming groups by training with MinDiff. \n",
    "3.   Evaluate the new model’s performance on our chosen metric.\n",
    "\n",
    "Our purpose is to demonstrate usage of the MinDiff technique with a very minimal workflow, not to lay out a principled approach to fairness in machine learning. As such, our evaluation will only focus on one sensitive category and a single metric. We also don’t address potential shortcomings in the dataset, nor tune our configurations. In a production setting, you would want to approach each of these with rigor. For more information on evaluating for fairness, see [this guide](https://www.tensorflow.org/responsible_ai/fairness_indicators/guide/guidance).\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_FCDIaf8wsll"
   },
   "source": [
    "## Setup\n",
    "\n",
    "We begin by installing Fairness Indicators and TensorFlow Model Remediation.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "cellView": "form",
    "execution": {
     "iopub.execute_input": "2024-07-19T09:16:00.346955Z",
     "iopub.status.busy": "2024-07-19T09:16:00.346526Z",
     "iopub.status.idle": "2024-07-19T09:19:47.621611Z",
     "shell.execute_reply": "2024-07-19T09:19:47.620460Z"
    },
    "id": "WoA7i_6ShG6Y"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting tensorflow-model-remediation\r\n",
      "  Using cached tensorflow_model_remediation-0.1.7.1-py3-none-any.whl.metadata (4.8 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting dill (from tensorflow-model-remediation)\r\n",
      "  Using cached dill-0.3.8-py3-none-any.whl.metadata (10 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting mock (from tensorflow-model-remediation)\r\n",
      "  Using cached mock-5.1.0-py3-none-any.whl.metadata (3.0 kB)\r\n",
      "Requirement already satisfied: pandas in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow-model-remediation) (2.2.2)\r\n",
      "Requirement already satisfied: tensorflow-hub in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow-model-remediation) (0.16.1)\r\n",
      "Requirement already satisfied: tensorflow>=2.0.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow-model-remediation) (2.17.0)\r\n",
      "Requirement already satisfied: absl-py>=1.0.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (2.1.0)\r\n",
      "Requirement already satisfied: astunparse>=1.6.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (1.6.3)\r\n",
      "Requirement already satisfied: flatbuffers>=24.3.25 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (24.3.25)\r\n",
      "Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (0.6.0)\r\n",
      "Requirement already satisfied: google-pasta>=0.1.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (0.2.0)\r\n",
      "Requirement already satisfied: h5py>=3.10.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (3.11.0)\r\n",
      "Requirement already satisfied: libclang>=13.0.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (18.1.1)\r\n",
      "Requirement already satisfied: ml-dtypes<0.5.0,>=0.3.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (0.4.0)\r\n",
      "Requirement already satisfied: opt-einsum>=2.3.2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (3.3.0)\r\n",
      "Requirement already satisfied: packaging in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (24.1)\r\n",
      "Requirement already satisfied: protobuf!=4.21.0,!=4.21.1,!=4.21.2,!=4.21.3,!=4.21.4,!=4.21.5,<5.0.0dev,>=3.20.3 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (3.20.3)\r\n",
      "Requirement already satisfied: requests<3,>=2.21.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (2.32.3)\r\n",
      "Requirement already satisfied: setuptools in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (71.0.3)\r\n",
      "Requirement already satisfied: six>=1.12.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (1.16.0)\r\n",
      "Requirement already satisfied: termcolor>=1.1.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (2.4.0)\r\n",
      "Requirement already satisfied: typing-extensions>=3.6.6 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (4.12.2)\r\n",
      "Requirement already satisfied: wrapt>=1.11.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (1.16.0)\r\n",
      "Requirement already satisfied: grpcio<2.0,>=1.24.3 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (1.65.1)\r\n",
      "Requirement already satisfied: tensorboard<2.18,>=2.17 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (2.17.0)\r\n",
      "Requirement already satisfied: keras>=3.2.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (3.4.1)\r\n",
      "Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (0.37.1)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: numpy<2.0.0,>=1.23.5 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow>=2.0.0->tensorflow-model-remediation) (1.26.4)\r\n",
      "Requirement already satisfied: python-dateutil>=2.8.2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from pandas->tensorflow-model-remediation) (2.9.0.post0)\r\n",
      "Requirement already satisfied: pytz>=2020.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from pandas->tensorflow-model-remediation) (2024.1)\r\n",
      "Requirement already satisfied: tzdata>=2022.7 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from pandas->tensorflow-model-remediation) (2024.1)\r\n",
      "Requirement already satisfied: tf-keras>=2.14.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow-hub->tensorflow-model-remediation) (2.17.0)\r\n",
      "Requirement already satisfied: wheel<1.0,>=0.23.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from astunparse>=1.6.0->tensorflow>=2.0.0->tensorflow-model-remediation) (0.43.0)\r\n",
      "Requirement already satisfied: rich in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from keras>=3.2.0->tensorflow>=2.0.0->tensorflow-model-remediation) (13.7.1)\r\n",
      "Requirement already satisfied: namex in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from keras>=3.2.0->tensorflow>=2.0.0->tensorflow-model-remediation) (0.0.8)\r\n",
      "Requirement already satisfied: optree in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from keras>=3.2.0->tensorflow>=2.0.0->tensorflow-model-remediation) (0.12.1)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: charset-normalizer<4,>=2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorflow>=2.0.0->tensorflow-model-remediation) (3.3.2)\r\n",
      "Requirement already satisfied: idna<4,>=2.5 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorflow>=2.0.0->tensorflow-model-remediation) (3.7)\r\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorflow>=2.0.0->tensorflow-model-remediation) (2.2.2)\r\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests<3,>=2.21.0->tensorflow>=2.0.0->tensorflow-model-remediation) (2024.7.4)\r\n",
      "Requirement already satisfied: markdown>=2.6.8 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorboard<2.18,>=2.17->tensorflow>=2.0.0->tensorflow-model-remediation) (3.6)\r\n",
      "Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorboard<2.18,>=2.17->tensorflow>=2.0.0->tensorflow-model-remediation) (0.7.2)\r\n",
      "Requirement already satisfied: werkzeug>=1.0.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorboard<2.18,>=2.17->tensorflow>=2.0.0->tensorflow-model-remediation) (3.0.3)\r\n",
      "Requirement already satisfied: importlib-metadata>=4.4 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from markdown>=2.6.8->tensorboard<2.18,>=2.17->tensorflow>=2.0.0->tensorflow-model-remediation) (8.0.0)\r\n",
      "Requirement already satisfied: MarkupSafe>=2.1.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from werkzeug>=1.0.1->tensorboard<2.18,>=2.17->tensorflow>=2.0.0->tensorflow-model-remediation) (2.1.5)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: markdown-it-py>=2.2.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from rich->keras>=3.2.0->tensorflow>=2.0.0->tensorflow-model-remediation) (3.0.0)\r\n",
      "Requirement already satisfied: pygments<3.0.0,>=2.13.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from rich->keras>=3.2.0->tensorflow>=2.0.0->tensorflow-model-remediation) (2.18.0)\r\n",
      "Requirement already satisfied: zipp>=0.5 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard<2.18,>=2.17->tensorflow>=2.0.0->tensorflow-model-remediation) (3.19.2)\r\n",
      "Requirement already satisfied: mdurl~=0.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from markdown-it-py>=2.2.0->rich->keras>=3.2.0->tensorflow>=2.0.0->tensorflow-model-remediation) (0.1.2)\r\n",
      "Using cached tensorflow_model_remediation-0.1.7.1-py3-none-any.whl (142 kB)\r\n",
      "Using cached dill-0.3.8-py3-none-any.whl (116 kB)\r\n",
      "Using cached mock-5.1.0-py3-none-any.whl (30 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Installing collected packages: mock, dill, tensorflow-model-remediation\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Successfully installed dill-0.3.8 mock-5.1.0 tensorflow-model-remediation-0.1.7.1\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting fairness-indicators\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading fairness_indicators-0.46.0-py3-none-any.whl.metadata (12 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting tensorflow<2.16,>=2.15 (from fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading tensorflow-2.15.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (4.2 kB)\r\n",
      "Requirement already satisfied: tensorflow-hub<1.0.0,>=0.16.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from fairness-indicators) (0.16.1)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting tensorflow-data-validation<2.0.0,>=1.15.1 (from fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading tensorflow_data_validation-1.15.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (18 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting tensorflow-model-analysis<0.47,>=0.46 (from fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading tensorflow_model_analysis-0.46.0-py3-none-any.whl.metadata (20 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting witwidget<2,>=1.4.4 (from fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading witwidget-1.8.1-py3-none-any.whl.metadata (1.4 kB)\r\n",
      "Requirement already satisfied: protobuf<5,>=3.20.3 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from fairness-indicators) (3.20.3)\r\n",
      "Requirement already satisfied: absl-py>=1.0.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.16,>=2.15->fairness-indicators) (2.1.0)\r\n",
      "Requirement already satisfied: astunparse>=1.6.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.16,>=2.15->fairness-indicators) (1.6.3)\r\n",
      "Requirement already satisfied: flatbuffers>=23.5.26 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.16,>=2.15->fairness-indicators) (24.3.25)\r\n",
      "Requirement already satisfied: gast!=0.5.0,!=0.5.1,!=0.5.2,>=0.2.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.16,>=2.15->fairness-indicators) (0.6.0)\r\n",
      "Requirement already satisfied: google-pasta>=0.1.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.16,>=2.15->fairness-indicators) (0.2.0)\r\n",
      "Requirement already satisfied: h5py>=2.9.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.16,>=2.15->fairness-indicators) (3.11.0)\r\n",
      "Requirement already satisfied: libclang>=13.0.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.16,>=2.15->fairness-indicators) (18.1.1)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting ml-dtypes~=0.3.1 (from tensorflow<2.16,>=2.15->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading ml_dtypes-0.3.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (20 kB)\r\n",
      "Requirement already satisfied: numpy<2.0.0,>=1.23.5 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.16,>=2.15->fairness-indicators) (1.26.4)\r\n",
      "Requirement already satisfied: opt-einsum>=2.3.2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.16,>=2.15->fairness-indicators) (3.3.0)\r\n",
      "Requirement already satisfied: packaging in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.16,>=2.15->fairness-indicators) (24.1)\r\n",
      "Requirement already satisfied: setuptools in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.16,>=2.15->fairness-indicators) (71.0.3)\r\n",
      "Requirement already satisfied: six>=1.12.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.16,>=2.15->fairness-indicators) (1.16.0)\r\n",
      "Requirement already satisfied: termcolor>=1.1.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.16,>=2.15->fairness-indicators) (2.4.0)\r\n",
      "Requirement already satisfied: typing-extensions>=3.6.6 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.16,>=2.15->fairness-indicators) (4.12.2)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting wrapt<1.15,>=1.11.0 (from tensorflow<2.16,>=2.15->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading wrapt-1.14.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.7 kB)\r\n",
      "Requirement already satisfied: tensorflow-io-gcs-filesystem>=0.23.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.16,>=2.15->fairness-indicators) (0.37.1)\r\n",
      "Requirement already satisfied: grpcio<2.0,>=1.24.3 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow<2.16,>=2.15->fairness-indicators) (1.65.1)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting tensorboard<2.16,>=2.15 (from tensorflow<2.16,>=2.15->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading tensorboard-2.15.2-py3-none-any.whl.metadata (1.7 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting tensorflow-estimator<2.16,>=2.15.0 (from tensorflow<2.16,>=2.15->fairness-indicators)\r\n",
      "  Downloading tensorflow_estimator-2.15.0-py2.py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting keras<2.16,>=2.15.0 (from tensorflow<2.16,>=2.15->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading keras-2.15.0-py3-none-any.whl.metadata (2.4 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting absl-py>=1.0.0 (from tensorflow<2.16,>=2.15->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading absl_py-1.4.0-py3-none-any.whl.metadata (2.3 kB)\r\n",
      "Requirement already satisfied: joblib>=1.2.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators) (1.4.2)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting pandas<2,>=1.0 (from tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading pandas-1.5.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting pyarrow<11,>=10 (from tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading pyarrow-10.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)\r\n",
      "Collecting pyfarmhash<0.4,>=0.2.2 (from tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading pyfarmhash-0.3.2.tar.gz (99 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Preparing metadata (setup.py) ... \u001b[?25l-"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \bdone\r\n",
      "\u001b[?25hRequirement already satisfied: tensorflow-metadata<1.16,>=1.15.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators) (1.15.0)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting tfx-bsl<1.16,>=1.15.1 (from tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n",
      "  Downloading tfx_bsl-1.15.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (11 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting apache-beam<3,>=2.47 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading apache_beam-2.57.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.6 kB)\r\n",
      "Requirement already satisfied: tf-keras>=2.14.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow-hub<1.0.0,>=0.16.1->fairness-indicators) (2.17.0)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting ipython<8,>=7 (from tensorflow-model-analysis<0.47,>=0.46->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading ipython-7.34.0-py3-none-any.whl.metadata (4.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting ipywidgets<8,>=7 (from tensorflow-model-analysis<0.47,>=0.46->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading ipywidgets-7.8.2-py2.py3-none-any.whl.metadata (1.9 kB)\r\n",
      "Requirement already satisfied: pillow>=9.4.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (10.4.0)\r\n",
      "Collecting rouge-score<2,>=0.1.2 (from tensorflow-model-analysis<0.47,>=0.46->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading rouge_score-0.1.2.tar.gz (17 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Preparing metadata (setup.py) ... \u001b[?25l-"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \bdone\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[?25hCollecting sacrebleu<4,>=2.3 (from tensorflow-model-analysis<0.47,>=0.46->fairness-indicators)\r\n",
      "  Downloading sacrebleu-2.4.2-py3-none-any.whl.metadata (58 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: scipy<2,>=1.4.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (1.13.1)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-api-python-client>=1.7.8 (from witwidget<2,>=1.4.4->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_api_python_client-2.137.0-py2.py3-none-any.whl.metadata (6.7 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting oauth2client>=4.1.3 (from witwidget<2,>=1.4.4->fairness-indicators)\r\n",
      "  Downloading oauth2client-4.1.3-py2.py3-none-any.whl.metadata (1.2 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting crcmod<2.0,>=1.7 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading crcmod-1.7.tar.gz (89 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Preparing metadata (setup.py) ... \u001b[?25ldone\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[?25hCollecting orjson<4,>=3.9.7 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n",
      "  Downloading orjson-3.10.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (50 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting dill<0.3.2,>=0.3.1.1 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading dill-0.3.1.1.tar.gz (151 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Preparing metadata (setup.py) ... \u001b[?25l-"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \bdone\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[?25hCollecting cloudpickle~=2.2.1 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading cloudpickle-2.2.1-py3-none-any.whl.metadata (6.9 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting fastavro<2,>=0.23.6 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n",
      "  Downloading fastavro-1.9.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (5.5 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting fasteners<1.0,>=0.3 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading fasteners-0.19-py3-none-any.whl.metadata (4.9 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting hdfs<3.0.0,>=2.1.0 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n",
      "  Downloading hdfs-2.7.3.tar.gz (43 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Preparing metadata (setup.py) ... \u001b[?25l-"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \bdone\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[?25hCollecting httplib2<0.23.0,>=0.8 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading httplib2-0.22.0-py3-none-any.whl.metadata (2.6 kB)\r\n",
      "Requirement already satisfied: jsonschema<5.0.0,>=4.0.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators) (4.23.0)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting jsonpickle<4.0.0,>=3.0.0 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading jsonpickle-3.2.2-py3-none-any.whl.metadata (7.2 kB)\r\n",
      "Collecting objsize<0.8.0,>=0.6.1 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading objsize-0.7.0-py3-none-any.whl.metadata (12 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting pymongo<5.0.0,>=3.8.0 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading pymongo-4.8.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (22 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting proto-plus<2,>=1.7.1 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading proto_plus-1.24.0-py3-none-any.whl.metadata (2.2 kB)\r\n",
      "Collecting pydot<2,>=1.2.0 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading pydot-1.4.2-py2.py3-none-any.whl.metadata (8.0 kB)\r\n",
      "Requirement already satisfied: python-dateutil<3,>=2.8.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators) (2.9.0.post0)\r\n",
      "Requirement already satisfied: pytz>=2018.3 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators) (2024.1)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting redis<6,>=5.0.0 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading redis-5.0.7-py3-none-any.whl.metadata (9.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting regex>=2020.6.8 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading regex-2024.5.15-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (40 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting requests!=2.32.*,<3.0.0,>=2.24.0 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting zstandard<1,>=0.18.0 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading zstandard-0.23.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting pyarrow-hotfix<1 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading pyarrow_hotfix-0.6-py3-none-any.whl.metadata (3.6 kB)\r\n",
      "Collecting js2py<1,>=0.74 (from apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading Js2Py-0.74-py3-none-any.whl.metadata (868 bytes)\r\n",
      "Requirement already satisfied: cachetools<6,>=3.1.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators) (5.4.0)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-api-core<3,>=2.0.0 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_api_core-2.19.1-py3-none-any.whl.metadata (2.7 kB)\r\n",
      "Collecting google-apitools<0.5.32,>=0.5.31 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google-apitools-0.5.31.tar.gz (173 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Preparing metadata (setup.py) ... \u001b[?25l-"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \bdone\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[?25hCollecting google-auth<3,>=1.18.0 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_auth-2.32.0-py2.py3-none-any.whl.metadata (4.7 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-auth-httplib2<0.3.0,>=0.1.0 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_auth_httplib2-0.2.0-py2.py3-none-any.whl.metadata (2.2 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-cloud-datastore<3,>=2.0.0 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_cloud_datastore-2.19.0-py2.py3-none-any.whl.metadata (5.6 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-cloud-pubsub<3,>=2.1.0 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_cloud_pubsub-2.22.0-py2.py3-none-any.whl.metadata (9.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-cloud-pubsublite<2,>=1.2.0 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n",
      "  Downloading google_cloud_pubsublite-1.11.0-py2.py3-none-any.whl.metadata (5.6 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-cloud-storage<3,>=2.16.0 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_cloud_storage-2.17.0-py2.py3-none-any.whl.metadata (6.6 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-cloud-bigquery<4,>=2.0.0 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_cloud_bigquery-3.25.0-py2.py3-none-any.whl.metadata (8.9 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-cloud-bigquery-storage<3,>=2.6.3 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_cloud_bigquery_storage-2.25.0-py2.py3-none-any.whl.metadata (5.6 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-cloud-core<3,>=2.0.0 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_cloud_core-2.4.1-py2.py3-none-any.whl.metadata (2.7 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-cloud-bigtable<3,>=2.19.0 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_cloud_bigtable-2.24.0-py2.py3-none-any.whl.metadata (5.2 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-cloud-spanner<4,>=3.0.0 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_cloud_spanner-3.47.0-py2.py3-none-any.whl.metadata (10 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-cloud-dlp<4,>=3.0.0 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_cloud_dlp-3.18.1-py2.py3-none-any.whl.metadata (5.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-cloud-language<3,>=2.0 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n",
      "  Downloading google_cloud_language-2.13.4-py2.py3-none-any.whl.metadata (5.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-cloud-videointelligence<3,>=2.0 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n",
      "  Downloading google_cloud_videointelligence-2.13.4-py2.py3-none-any.whl.metadata (5.7 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-cloud-vision<4,>=2 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_cloud_vision-3.7.3-py2.py3-none-any.whl.metadata (5.2 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-cloud-recommendations-ai<0.11.0,>=0.1.0 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_cloud_recommendations_ai-0.10.11-py2.py3-none-any.whl.metadata (5.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-cloud-aiplatform<2.0,>=1.26.0 (from apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_cloud_aiplatform-1.59.0-py2.py3-none-any.whl.metadata (31 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: wheel<1.0,>=0.23.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from astunparse>=1.6.0->tensorflow<2.16,>=2.15->fairness-indicators) (0.43.0)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting uritemplate<5,>=3.0.1 (from google-api-python-client>=1.7.8->witwidget<2,>=1.4.4->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading uritemplate-4.1.1-py2.py3-none-any.whl.metadata (2.9 kB)\r\n",
      "Requirement already satisfied: jedi>=0.16 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.19.1)\r\n",
      "Requirement already satisfied: decorator in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (5.1.1)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting pickleshare (from ipython<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading pickleshare-0.7.5-py2.py3-none-any.whl.metadata (1.5 kB)\r\n",
      "Requirement already satisfied: traitlets>=4.2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (5.14.3)\r\n",
      "Requirement already satisfied: prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (3.0.47)\r\n",
      "Requirement already satisfied: pygments in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (2.18.0)\r\n",
      "Collecting backcall (from ipython<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading backcall-0.2.0-py2.py3-none-any.whl.metadata (2.0 kB)\r\n",
      "Requirement already satisfied: matplotlib-inline in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.1.7)\r\n",
      "Requirement already satisfied: pexpect>4.3 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipython<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (4.9.0)\r\n",
      "Requirement already satisfied: comm>=0.1.3 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.2.2)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting ipython-genutils~=0.2.0 (from ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators)\r\n",
      "  Downloading ipython_genutils-0.2.0-py2.py3-none-any.whl.metadata (755 bytes)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting widgetsnbextension~=3.6.7 (from ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators)\r\n",
      "  Downloading widgetsnbextension-3.6.7-py2.py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting jupyterlab-widgets<3,>=1.0.0 (from ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators)\r\n",
      "  Downloading jupyterlab_widgets-1.1.8-py3-none-any.whl.metadata (3.7 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting pyasn1>=0.1.7 (from oauth2client>=4.1.3->witwidget<2,>=1.4.4->fairness-indicators)\r\n",
      "  Downloading pyasn1-0.6.0-py2.py3-none-any.whl.metadata (8.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting pyasn1-modules>=0.0.5 (from oauth2client>=4.1.3->witwidget<2,>=1.4.4->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading pyasn1_modules-0.4.0-py3-none-any.whl.metadata (3.4 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting rsa>=3.1.4 (from oauth2client>=4.1.3->witwidget<2,>=1.4.4->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading rsa-4.9-py3-none-any.whl.metadata (4.2 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting nltk (from rouge-score<2,>=0.1.2->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading nltk-3.8.1-py3-none-any.whl.metadata (2.8 kB)\r\n",
      "Collecting portalocker (from sacrebleu<4,>=2.3->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading portalocker-2.10.1-py3-none-any.whl.metadata (8.5 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting tabulate>=0.8.9 (from sacrebleu<4,>=2.3->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading tabulate-0.9.0-py3-none-any.whl.metadata (34 kB)\r\n",
      "Collecting colorama (from sacrebleu<4,>=2.3->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading colorama-0.4.6-py2.py3-none-any.whl.metadata (17 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting lxml (from sacrebleu<4,>=2.3->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators)\r\n",
      "  Downloading lxml-5.2.2-cp39-cp39-manylinux_2_28_x86_64.whl.metadata (3.4 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-auth-oauthlib<2,>=0.5 (from tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_auth_oauthlib-1.2.1-py2.py3-none-any.whl.metadata (2.7 kB)\r\n",
      "Requirement already satisfied: markdown>=2.6.8 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->fairness-indicators) (3.6)\r\n",
      "Requirement already satisfied: tensorboard-data-server<0.8.0,>=0.7.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->fairness-indicators) (0.7.2)\r\n",
      "Requirement already satisfied: werkzeug>=1.0.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->fairness-indicators) (3.0.3)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO: pip is looking at multiple versions of tf-keras to determine which version is compatible with other requirements. This could take a while.\r\n",
      "Collecting tf-keras>=2.14.1 (from tensorflow-hub<1.0.0,>=0.16.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading tf_keras-2.16.0-py3-none-any.whl.metadata (1.6 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading tf_keras-2.15.1-py3-none-any.whl.metadata (1.7 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-api-python-client>=1.7.8 (from witwidget<2,>=1.4.4->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_api_python_client-1.12.11-py2.py3-none-any.whl.metadata (4.2 kB)\r\n",
      "Collecting tensorflow-serving-api<3,>=2.13.0 (from tfx-bsl<1.16,>=1.15.1->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading tensorflow_serving_api-2.16.1-py2.py3-none-any.whl.metadata (1.8 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting uritemplate<4dev,>=3.0.0 (from google-api-python-client>=1.7.8->witwidget<2,>=1.4.4->fairness-indicators)\r\n",
      "  Downloading uritemplate-3.0.1-py2.py3-none-any.whl.metadata (4.6 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting googleapis-common-protos<2.0.dev0,>=1.56.2 (from google-api-core<3,>=2.0.0->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n",
      "  Downloading googleapis_common_protos-1.63.2-py2.py3-none-any.whl.metadata (1.5 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting requests-oauthlib>=0.7.0 (from google-auth-oauthlib<2,>=0.5->tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->fairness-indicators)\r\n",
      "  Downloading requests_oauthlib-2.0.0-py2.py3-none-any.whl.metadata (11 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-cloud-resource-manager<3.0.0dev,>=1.3.3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_cloud_resource_manager-1.12.4-py2.py3-none-any.whl.metadata (5.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting shapely<3.0.0dev (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading shapely-2.0.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (7.0 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting pydantic<3 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading pydantic-2.8.2-py3-none-any.whl.metadata (125 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting docstring-parser<1 (from google-cloud-aiplatform<2.0,>=1.26.0->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading docstring_parser-0.16-py3-none-any.whl.metadata (3.0 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-resumable-media<3.0dev,>=0.6.0 (from google-cloud-bigquery<4,>=2.0.0->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading google_resumable_media-2.7.1-py2.py3-none-any.whl.metadata (2.2 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting grpc-google-iam-v1<1.0.0dev,>=0.12.4 (from google-cloud-bigtable<3,>=2.19.0->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpc_google_iam_v1-0.13.1-py2.py3-none-any.whl.metadata (3.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.65.1-py3-none-any.whl.metadata (1.1 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: overrides<8.0.0,>=6.0.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from google-cloud-pubsublite<2,>=1.2.0->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators) (7.7.0)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting sqlparse>=0.4.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading sqlparse-0.5.1-py3-none-any.whl.metadata (3.9 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting grpc-interceptor>=0.15.4 (from google-cloud-spanner<4,>=3.0.0->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpc_interceptor-0.15.4-py3-none-any.whl.metadata (8.4 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting google-crc32c<2.0dev,>=1.0 (from google-cloud-storage<3,>=2.16.0->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n",
      "  Downloading google_crc32c-1.5.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (2.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting docopt (from hdfs<3.0.0,>=2.1.0->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n",
      "  Downloading docopt-0.6.2.tar.gz (25 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Preparing metadata (setup.py) ... \u001b[?25l-"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \bdone\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[?25hRequirement already satisfied: pyparsing!=3.0.0,!=3.0.1,!=3.0.2,!=3.0.3,<4,>=2.4.2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from httplib2<0.23.0,>=0.8->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators) (3.1.2)\r\n",
      "Requirement already satisfied: parso<0.9.0,>=0.8.3 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jedi>=0.16->ipython<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.8.4)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting tzlocal>=1.2 (from js2py<1,>=0.74->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n",
      "  Downloading tzlocal-5.2-py3-none-any.whl.metadata (7.8 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting pyjsparser>=2.5.1 (from js2py<1,>=0.74->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading pyjsparser-2.7.1.tar.gz (24 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Preparing metadata (setup.py) ... \u001b[?25l-"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \bdone\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[?25hRequirement already satisfied: attrs>=22.2.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jsonschema<5.0.0,>=4.0.0->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators) (23.2.0)\r\n",
      "Requirement already satisfied: jsonschema-specifications>=2023.03.6 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jsonschema<5.0.0,>=4.0.0->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators) (2023.12.1)\r\n",
      "Requirement already satisfied: referencing>=0.28.4 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jsonschema<5.0.0,>=4.0.0->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators) (0.35.1)\r\n",
      "Requirement already satisfied: rpds-py>=0.7.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jsonschema<5.0.0,>=4.0.0->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators) (0.19.0)\r\n",
      "Requirement already satisfied: importlib-metadata>=4.4 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from markdown>=2.6.8->tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->fairness-indicators) (8.0.0)\r\n",
      "Requirement already satisfied: ptyprocess>=0.5 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from pexpect>4.3->ipython<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.7.0)\r\n",
      "Requirement already satisfied: wcwidth in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from prompt-toolkit!=3.0.0,!=3.0.1,<3.1.0,>=2.0.0->ipython<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.2.13)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting dnspython<3.0.0,>=1.16.0 (from pymongo<5.0.0,>=3.8.0->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading dnspython-2.6.1-py3-none-any.whl.metadata (5.8 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting async-timeout>=4.0.3 (from redis<6,>=5.0.0->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading async_timeout-4.0.3-py3-none-any.whl.metadata (4.2 kB)\r\n",
      "Requirement already satisfied: charset-normalizer<4,>=2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests!=2.32.*,<3.0.0,>=2.24.0->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators) (3.3.2)\r\n",
      "Requirement already satisfied: idna<4,>=2.5 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests!=2.32.*,<3.0.0,>=2.24.0->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators) (3.7)\r\n",
      "Requirement already satisfied: urllib3<3,>=1.21.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests!=2.32.*,<3.0.0,>=2.24.0->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators) (2.2.2)\r\n",
      "Requirement already satisfied: certifi>=2017.4.17 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from requests!=2.32.*,<3.0.0,>=2.24.0->apache-beam<3,>=2.47->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators) (2024.7.4)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO: pip is looking at multiple versions of tensorflow-serving-api to determine which version is compatible with other requirements. This could take a while.\r\n",
      "Collecting tensorflow-serving-api<3,>=2.13.0 (from tfx-bsl<1.16,>=1.15.1->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading tensorflow_serving_api-2.15.1-py2.py3-none-any.whl.metadata (1.8 kB)\r\n",
      "Requirement already satisfied: MarkupSafe>=2.1.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from werkzeug>=1.0.1->tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->fairness-indicators) (2.1.5)\r\n",
      "Requirement already satisfied: notebook>=4.4.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (7.2.1)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: click in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from nltk->rouge-score<2,>=0.1.2->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (8.1.7)\r\n",
      "Requirement already satisfied: tqdm in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from nltk->rouge-score<2,>=0.1.2->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (4.66.4)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO: pip is looking at multiple versions of grpcio-status to determine which version is compatible with other requirements. This could take a while.\r\n",
      "Collecting grpcio-status>=1.33.2 (from google-cloud-pubsub<3,>=2.1.0->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.64.1-py3-none-any.whl.metadata (1.1 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.64.0-py3-none-any.whl.metadata (1.1 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.63.0-py3-none-any.whl.metadata (1.1 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.62.2-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.62.1-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.62.0-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.60.1-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO: pip is still looking at multiple versions of grpcio-status to determine which version is compatible with other requirements. This could take a while.\r\n",
      "  Downloading grpcio_status-1.60.0-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.59.3-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.59.2-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.59.0-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.58.0-py3-none-any.whl.metadata (1.3 kB)\r\n",
      "INFO: This is taking longer than usual. You might need to provide the dependency resolver with stricter constraints to reduce runtime. See https://pip.pypa.io/warnings/backtracking for guidance. If you want to abort this run, press Ctrl + C.\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.57.0-py3-none-any.whl.metadata (1.2 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.56.2-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.56.0-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.55.3-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.54.3-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.54.2-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.54.0-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.53.2-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.53.1-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.53.0-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.51.3-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.51.1-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.50.0-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.49.1-py3-none-any.whl.metadata (1.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading grpcio_status-1.48.2-py3-none-any.whl.metadata (1.2 kB)\r\n",
      "Requirement already satisfied: zipp>=0.5 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from importlib-metadata>=4.4->markdown>=2.6.8->tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->fairness-indicators) (3.19.2)\r\n",
      "Requirement already satisfied: jupyter-server<3,>=2.4.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (2.14.2)\r\n",
      "Requirement already satisfied: jupyterlab-server<3,>=2.27.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (2.27.3)\r\n",
      "Requirement already satisfied: jupyterlab<4.3,>=4.2.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (4.2.4)\r\n",
      "Requirement already satisfied: notebook-shim<0.3,>=0.2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.2.4)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: tornado>=6.2.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (6.4.1)\r\n",
      "Collecting annotated-types>=0.4.0 (from pydantic<3->google-cloud-aiplatform<2.0,>=1.26.0->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading annotated_types-0.7.0-py3-none-any.whl.metadata (15 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting pydantic-core==2.20.1 (from pydantic<3->google-cloud-aiplatform<2.0,>=1.26.0->apache-beam[gcp]<3,>=2.47; python_version < \"3.11\"->tensorflow-data-validation<2.0.0,>=1.15.1->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading pydantic_core-2.20.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting oauthlib>=3.0.0 (from requests-oauthlib>=0.7.0->google-auth-oauthlib<2,>=0.5->tensorboard<2.16,>=2.15->tensorflow<2.16,>=2.15->fairness-indicators)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Downloading oauthlib-3.2.2-py3-none-any.whl.metadata (7.5 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: anyio>=3.1.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (4.4.0)\r\n",
      "Requirement already satisfied: argon2-cffi>=21.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (23.1.0)\r\n",
      "Requirement already satisfied: jinja2>=3.0.3 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (3.1.4)\r\n",
      "Requirement already satisfied: jupyter-client>=7.4.4 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (8.6.2)\r\n",
      "Requirement already satisfied: jupyter-core!=5.0.*,>=4.12 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (5.7.2)\r\n",
      "Requirement already satisfied: jupyter-events>=0.9.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.10.0)\r\n",
      "Requirement already satisfied: jupyter-server-terminals>=0.4.4 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.5.3)\r\n",
      "Requirement already satisfied: nbconvert>=6.4.4 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (7.16.4)\r\n",
      "Requirement already satisfied: nbformat>=5.3.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (5.10.4)\r\n",
      "Requirement already satisfied: prometheus-client>=0.9 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.20.0)\r\n",
      "Requirement already satisfied: pyzmq>=24 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (26.0.3)\r\n",
      "Requirement already satisfied: send2trash>=1.8.2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (1.8.3)\r\n",
      "Requirement already satisfied: terminado>=0.8.3 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.18.1)\r\n",
      "Requirement already satisfied: websocket-client>=1.7 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (1.8.0)\r\n",
      "Requirement already satisfied: async-lru>=1.0.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyterlab<4.3,>=4.2.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (2.0.4)\r\n",
      "Requirement already satisfied: httpx>=0.25.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyterlab<4.3,>=4.2.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.27.0)\r\n",
      "Requirement already satisfied: ipykernel>=6.5.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyterlab<4.3,>=4.2.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (6.29.5)\r\n",
      "Requirement already satisfied: jupyter-lsp>=2.0.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyterlab<4.3,>=4.2.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (2.2.5)\r\n",
      "Requirement already satisfied: tomli>=1.2.2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyterlab<4.3,>=4.2.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (2.0.1)\r\n",
      "Requirement already satisfied: babel>=2.10 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyterlab-server<3,>=2.27.1->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (2.15.0)\r\n",
      "Requirement already satisfied: json5>=0.9.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyterlab-server<3,>=2.27.1->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.9.25)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: sniffio>=1.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from anyio>=3.1.0->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (1.3.1)\r\n",
      "Requirement already satisfied: exceptiongroup>=1.0.2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from anyio>=3.1.0->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (1.2.2)\r\n",
      "Requirement already satisfied: argon2-cffi-bindings in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from argon2-cffi>=21.1->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (21.2.0)\r\n",
      "Requirement already satisfied: httpcore==1.* in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from httpx>=0.25.0->jupyterlab<4.3,>=4.2.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (1.0.5)\r\n",
      "Requirement already satisfied: h11<0.15,>=0.13 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from httpcore==1.*->httpx>=0.25.0->jupyterlab<4.3,>=4.2.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.14.0)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: debugpy>=1.6.5 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipykernel>=6.5.0->jupyterlab<4.3,>=4.2.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (1.8.2)\r\n",
      "Requirement already satisfied: nest-asyncio in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipykernel>=6.5.0->jupyterlab<4.3,>=4.2.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (1.6.0)\r\n",
      "Requirement already satisfied: psutil in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from ipykernel>=6.5.0->jupyterlab<4.3,>=4.2.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (6.0.0)\r\n",
      "Requirement already satisfied: platformdirs>=2.5 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyter-core!=5.0.*,>=4.12->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (4.2.2)\r\n",
      "Requirement already satisfied: python-json-logger>=2.0.4 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (2.0.7)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: pyyaml>=5.3 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (6.0.1)\r\n",
      "Requirement already satisfied: rfc3339-validator in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.1.4)\r\n",
      "Requirement already satisfied: rfc3986-validator>=0.1.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.1.1)\r\n",
      "Requirement already satisfied: beautifulsoup4 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (4.12.3)\r\n",
      "Requirement already satisfied: bleach!=5.0.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (6.1.0)\r\n",
      "Requirement already satisfied: defusedxml in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.7.1)\r\n",
      "Requirement already satisfied: jupyterlab-pygments in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.3.0)\r\n",
      "Requirement already satisfied: mistune<4,>=2.0.3 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (3.0.2)\r\n",
      "Requirement already satisfied: nbclient>=0.5.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.10.0)\r\n",
      "Requirement already satisfied: pandocfilters>=1.4.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (1.5.1)\r\n",
      "Requirement already satisfied: tinycss2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (1.3.0)\r\n",
      "Requirement already satisfied: fastjsonschema>=2.15 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from nbformat>=5.3.0->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (2.20.0)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: webencodings in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from bleach!=5.0.0->nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (0.5.1)\r\n",
      "Requirement already satisfied: fqdn in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (1.5.1)\r\n",
      "Requirement already satisfied: isoduration in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (20.11.0)\r\n",
      "Requirement already satisfied: jsonpointer>1.13 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (3.0.0)\r\n",
      "Requirement already satisfied: uri-template in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (1.3.0)\r\n",
      "Requirement already satisfied: webcolors>=24.6.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (24.6.0)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: cffi>=1.0.1 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from argon2-cffi-bindings->argon2-cffi>=21.1->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (1.16.0)\r\n",
      "Requirement already satisfied: soupsieve>1.2 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from beautifulsoup4->nbconvert>=6.4.4->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (2.5)\r\n",
      "Requirement already satisfied: pycparser in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from cffi>=1.0.1->argon2-cffi-bindings->argon2-cffi>=21.1->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (2.22)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: arrow>=0.15.0 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from isoduration->jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (1.3.0)\r\n",
      "Requirement already satisfied: types-python-dateutil>=2.8.10 in /tmpfs/src/tf_docs_env/lib/python3.9/site-packages (from arrow>=0.15.0->isoduration->jsonschema[format-nongpl]>=4.18.0->jupyter-events>=0.9.0->jupyter-server<3,>=2.4.0->notebook>=4.4.1->widgetsnbextension~=3.6.7->ipywidgets<8,>=7->tensorflow-model-analysis<0.47,>=0.46->fairness-indicators) (2.9.0.20240316)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading fairness_indicators-0.46.0-py3-none-any.whl (24 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading tensorflow-2.15.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (475.2 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading tensorflow_data_validation-1.15.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (19.0 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading tensorflow_model_analysis-0.46.0-py3-none-any.whl (1.9 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading witwidget-1.8.1-py3-none-any.whl (1.5 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading absl_py-1.4.0-py3-none-any.whl (126 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading apache_beam-2.57.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (14.9 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading ipython-7.34.0-py3-none-any.whl (793 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading ipywidgets-7.8.2-py2.py3-none-any.whl (124 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading keras-2.15.0-py3-none-any.whl (1.7 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading ml_dtypes-0.3.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.2 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading pandas-1.5.3-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.2 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading pyarrow-10.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (35.9 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading sacrebleu-2.4.2-py3-none-any.whl (106 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading tensorboard-2.15.2-py3-none-any.whl (5.5 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading tensorflow_estimator-2.15.0-py2.py3-none-any.whl (441 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading tf_keras-2.15.1-py3-none-any.whl (1.7 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading tfx_bsl-1.15.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (22.5 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_api_python_client-1.12.11-py2.py3-none-any.whl (62 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading wrapt-1.14.1-cp39-cp39-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (77 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading cloudpickle-2.2.1-py3-none-any.whl (25 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading fastavro-1.9.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading fasteners-0.19-py3-none-any.whl (18 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_api_core-2.19.1-py3-none-any.whl (139 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_auth-2.32.0-py2.py3-none-any.whl (195 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_auth_httplib2-0.2.0-py2.py3-none-any.whl (9.3 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_auth_oauthlib-1.2.1-py2.py3-none-any.whl (24 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_cloud_aiplatform-1.59.0-py2.py3-none-any.whl (5.1 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_cloud_bigquery-3.25.0-py2.py3-none-any.whl (239 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_cloud_bigquery_storage-2.25.0-py2.py3-none-any.whl (199 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_cloud_bigtable-2.24.0-py2.py3-none-any.whl (373 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_cloud_core-2.4.1-py2.py3-none-any.whl (29 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_cloud_datastore-2.19.0-py2.py3-none-any.whl (176 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_cloud_dlp-3.18.1-py2.py3-none-any.whl (180 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_cloud_language-2.13.4-py2.py3-none-any.whl (146 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_cloud_pubsub-2.22.0-py2.py3-none-any.whl (276 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_cloud_pubsublite-1.11.0-py2.py3-none-any.whl (303 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_cloud_recommendations_ai-0.10.11-py2.py3-none-any.whl (183 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_cloud_spanner-3.47.0-py2.py3-none-any.whl (384 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_cloud_storage-2.17.0-py2.py3-none-any.whl (126 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_cloud_videointelligence-2.13.4-py2.py3-none-any.whl (244 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_cloud_vision-3.7.3-py2.py3-none-any.whl (466 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading httplib2-0.22.0-py3-none-any.whl (96 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading ipython_genutils-0.2.0-py2.py3-none-any.whl (26 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading Js2Py-0.74-py3-none-any.whl (1.0 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading jsonpickle-3.2.2-py3-none-any.whl (41 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading jupyterlab_widgets-1.1.8-py3-none-any.whl (237 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading objsize-0.7.0-py3-none-any.whl (11 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading orjson-3.10.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (140 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading proto_plus-1.24.0-py3-none-any.whl (50 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading pyarrow_hotfix-0.6-py3-none-any.whl (7.9 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading pyasn1-0.6.0-py2.py3-none-any.whl (85 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading pyasn1_modules-0.4.0-py3-none-any.whl (181 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading pydot-1.4.2-py2.py3-none-any.whl (21 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading pymongo-4.8.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (921 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading redis-5.0.7-py3-none-any.whl (252 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading regex-2024.5.15-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (774 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading requests-2.31.0-py3-none-any.whl (62 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading rsa-4.9-py3-none-any.whl (34 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading tabulate-0.9.0-py3-none-any.whl (35 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading tensorflow_serving_api-2.15.1-py2.py3-none-any.whl (26 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading uritemplate-3.0.1-py2.py3-none-any.whl (15 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading widgetsnbextension-3.6.7-py2.py3-none-any.whl (1.5 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading zstandard-0.23.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.4 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading backcall-0.2.0-py2.py3-none-any.whl (11 kB)\r\n",
      "Downloading colorama-0.4.6-py2.py3-none-any.whl (25 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading lxml-5.2.2-cp39-cp39-manylinux_2_28_x86_64.whl (5.0 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading nltk-3.8.1-py3-none-any.whl (1.5 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading pickleshare-0.7.5-py2.py3-none-any.whl (6.9 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading portalocker-2.10.1-py3-none-any.whl (18 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading async_timeout-4.0.3-py3-none-any.whl (5.7 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading dnspython-2.6.1-py3-none-any.whl (307 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading docstring_parser-0.16-py3-none-any.whl (36 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_cloud_resource_manager-1.12.4-py2.py3-none-any.whl (339 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_crc32c-1.5.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (32 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading google_resumable_media-2.7.1-py2.py3-none-any.whl (81 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading googleapis_common_protos-1.63.2-py2.py3-none-any.whl (220 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading grpc_google_iam_v1-0.13.1-py2.py3-none-any.whl (24 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading grpc_interceptor-0.15.4-py3-none-any.whl (20 kB)\r\n",
      "Downloading grpcio_status-1.48.2-py3-none-any.whl (14 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading pydantic-2.8.2-py3-none-any.whl (423 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading pydantic_core-2.20.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.1 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading requests_oauthlib-2.0.0-py2.py3-none-any.whl (24 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading shapely-2.0.5-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading sqlparse-0.5.1-py3-none-any.whl (44 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading tzlocal-5.2-py3-none-any.whl (17 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading annotated_types-0.7.0-py3-none-any.whl (13 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading oauthlib-3.2.2-py3-none-any.whl (151 kB)\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Building wheels for collected packages: pyfarmhash, rouge-score, crcmod, dill, google-apitools, hdfs, pyjsparser, docopt\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Building wheel for pyfarmhash (setup.py) ... \u001b[?25l-"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \b\\"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \b|"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \b/"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \bdone\r\n",
      "\u001b[?25h  Created wheel for pyfarmhash: filename=pyfarmhash-0.3.2-cp39-cp39-linux_x86_64.whl size=101503 sha256=d7b76372a8ab929b265e06683cb9e071074e8861f1f0f03bb9a32e85900c7176\r\n",
      "  Stored in directory: /home/kbuilder/.cache/pip/wheels/de/2b/b1/c541160670d70f4b08c4786f4e155337d4baeaa3e01d9d1400\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Building wheel for rouge-score (setup.py) ... \u001b[?25l-"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \bdone\r\n",
      "\u001b[?25h  Created wheel for rouge-score: filename=rouge_score-0.1.2-py3-none-any.whl size=24934 sha256=f5e8a2d481a91183c66831ba35fd4148b1f22eb807b4db59a5d134e89a82cde8\r\n",
      "  Stored in directory: /home/kbuilder/.cache/pip/wheels/9b/3d/39/09558097d3119ca0a4d462df68f22c6f3c1b345ac63a09b86e\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Building wheel for crcmod (setup.py) ... \u001b[?25l-"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \bdone\r\n",
      "\u001b[?25h  Created wheel for crcmod: filename=crcmod-1.7-cp39-cp39-linux_x86_64.whl size=36917 sha256=9c265b885106f9a14dabcda869355b0910d93d64c5368eb6520a6bf52b0c300e\r\n",
      "  Stored in directory: /home/kbuilder/.cache/pip/wheels/4a/6c/a6/ffdd136310039bf226f2707a9a8e6857be7d70a3fc061f6b36\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Building wheel for dill (setup.py) ... \u001b[?25l-"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \b\\"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \bdone\r\n",
      "\u001b[?25h  Created wheel for dill: filename=dill-0.3.1.1-py3-none-any.whl size=78542 sha256=e220e00800b739c59522fa05187ad0f13735d6aed9d9d07af9bba35993f8d198\r\n",
      "  Stored in directory: /home/kbuilder/.cache/pip/wheels/4f/0b/ce/75d96dd714b15e51cb66db631183ea3844e0c4a6d19741a149\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Building wheel for google-apitools (setup.py) ... \u001b[?25l-"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \b\\"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \bdone\r\n",
      "\u001b[?25h  Created wheel for google-apitools: filename=google_apitools-0.5.31-py3-none-any.whl size=131014 sha256=3d21fb973e4f4d3bbe2ccb9b59f60e4ce8bc49e0fee7ff54f9e0f8912854a250\r\n",
      "  Stored in directory: /home/kbuilder/.cache/pip/wheels/6c/f8/60/b9e91899dbaf25b6314047d3daee379bdd8d61b1dc3fd5ec7f\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Building wheel for hdfs (setup.py) ... \u001b[?25l-"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \b\\\b \bdone\r\n",
      "\u001b[?25h  Created wheel for hdfs: filename=hdfs-2.7.3-py3-none-any.whl size=34324 sha256=4245d86faf0e86f1302d4671f0a686f14545b8c1df7ae3d7721d684dd8fb1e0a\r\n",
      "  Stored in directory: /home/kbuilder/.cache/pip/wheels/05/6f/21/aa8d233f90da3017b4ef7c61829589dc267402d376dd3efcf5\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Building wheel for pyjsparser (setup.py) ... \u001b[?25l-"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \bdone\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[?25h  Created wheel for pyjsparser: filename=pyjsparser-2.7.1-py3-none-any.whl size=25983 sha256=ca9f3b52c43c845bf01f836a2ac20372be3d688eea85bb4ef52a4d5584fc9134\r\n",
      "  Stored in directory: /home/kbuilder/.cache/pip/wheels/f0/70/61/f42dc45dcf0fbe8c495ce579b04730787081499bfb5b8bc60e\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Building wheel for docopt (setup.py) ... \u001b[?25l-"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b \bdone\r\n",
      "\u001b[?25h  Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13705 sha256=a7bb214bd3e852f3b78fd770855e81cbc7012a7ce445e38e0e590120b138a5ab\r\n",
      "  Stored in directory: /home/kbuilder/.cache/pip/wheels/70/4a/46/1309fc853b8d395e60bafaf1b6df7845bdd82c95fd59dd8d2b\r\n",
      "Successfully built pyfarmhash rouge-score crcmod dill google-apitools hdfs pyjsparser docopt\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Installing collected packages: pyjsparser, pyfarmhash, pickleshare, ipython-genutils, docopt, crcmod, backcall, zstandard, wrapt, uritemplate, tzlocal, tensorflow-estimator, tabulate, sqlparse, shapely, requests, regex, pydot, pydantic-core, pyasn1, pyarrow-hotfix, pyarrow, proto-plus, portalocker, orjson, objsize, oauthlib, ml-dtypes, lxml, keras, jupyterlab-widgets, jsonpickle, httplib2, grpc-interceptor, googleapis-common-protos, google-crc32c, fasteners, fastavro, docstring-parser, dnspython, dill, colorama, cloudpickle, async-timeout, annotated-types, absl-py, sacrebleu, rsa, requests-oauthlib, redis, pymongo, pydantic, pyasn1-modules, pandas, nltk, js2py, ipython, hdfs, grpcio-status, google-resumable-media, rouge-score, oauth2client, grpc-google-iam-v1, google-auth, google-auth-oauthlib, google-auth-httplib2, google-apitools, google-api-core, apache-beam, tensorboard, google-cloud-core, google-api-python-client, tensorflow, google-cloud-vision, google-cloud-videointelligence, google-cloud-storage, google-cloud-spanner, google-cloud-resource-manager, google-cloud-recommendations-ai, google-cloud-pubsub, google-cloud-language, google-cloud-dlp, google-cloud-datastore, google-cloud-bigtable, google-cloud-bigquery-storage, google-cloud-bigquery, tf-keras, tensorflow-serving-api, google-cloud-pubsublite, google-cloud-aiplatform, tfx-bsl, tensorflow-data-validation, widgetsnbextension, ipywidgets, witwidget, tensorflow-model-analysis, fairness-indicators\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Attempting uninstall: wrapt\r\n",
      "    Found existing installation: wrapt 1.16.0\r\n",
      "    Uninstalling wrapt-1.16.0:\r\n",
      "      Successfully uninstalled wrapt-1.16.0\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Attempting uninstall: requests\r\n",
      "    Found existing installation: requests 2.32.3\r\n",
      "    Uninstalling requests-2.32.3:\r\n",
      "      Successfully uninstalled requests-2.32.3\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Attempting uninstall: pydot\r\n",
      "    Found existing installation: pydot 3.0.1\r\n",
      "    Uninstalling pydot-3.0.1:\r\n",
      "      Successfully uninstalled pydot-3.0.1\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Attempting uninstall: ml-dtypes\r\n",
      "    Found existing installation: ml-dtypes 0.4.0\r\n",
      "    Uninstalling ml-dtypes-0.4.0:\r\n",
      "      Successfully uninstalled ml-dtypes-0.4.0\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Attempting uninstall: keras\r\n",
      "    Found existing installation: keras 3.4.1\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Uninstalling keras-3.4.1:\r\n",
      "      Successfully uninstalled keras-3.4.1\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Attempting uninstall: jupyterlab-widgets\r\n",
      "    Found existing installation: jupyterlab_widgets 3.0.11\r\n",
      "    Uninstalling jupyterlab_widgets-3.0.11:\r\n",
      "      Successfully uninstalled jupyterlab_widgets-3.0.11\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Attempting uninstall: dill\r\n",
      "    Found existing installation: dill 0.3.8\r\n",
      "    Uninstalling dill-0.3.8:\r\n",
      "      Successfully uninstalled dill-0.3.8\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Attempting uninstall: absl-py\r\n",
      "    Found existing installation: absl-py 2.1.0\r\n",
      "    Uninstalling absl-py-2.1.0:\r\n",
      "      Successfully uninstalled absl-py-2.1.0\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Attempting uninstall: pandas\r\n",
      "    Found existing installation: pandas 2.2.2\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Uninstalling pandas-2.2.2:\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      Successfully uninstalled pandas-2.2.2\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Attempting uninstall: ipython\r\n",
      "    Found existing installation: ipython 8.18.1\r\n",
      "    Uninstalling ipython-8.18.1:\r\n",
      "      Successfully uninstalled ipython-8.18.1\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Attempting uninstall: tensorboard\r\n",
      "    Found existing installation: tensorboard 2.17.0\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Uninstalling tensorboard-2.17.0:\r\n",
      "      Successfully uninstalled tensorboard-2.17.0\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Attempting uninstall: tensorflow\r\n",
      "    Found existing installation: tensorflow 2.17.0\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Uninstalling tensorflow-2.17.0:\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "      Successfully uninstalled tensorflow-2.17.0\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Attempting uninstall: tf-keras\r\n",
      "    Found existing installation: tf_keras 2.17.0\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "    Uninstalling tf_keras-2.17.0:\r\n",
      "      Successfully uninstalled tf_keras-2.17.0\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Attempting uninstall: widgetsnbextension\r\n",
      "    Found existing installation: widgetsnbextension 4.0.11\r\n",
      "    Uninstalling widgetsnbextension-4.0.11:\r\n",
      "      Successfully uninstalled widgetsnbextension-4.0.11\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  Attempting uninstall: ipywidgets\r\n",
      "    Found existing installation: ipywidgets 8.1.3\r\n",
      "    Uninstalling ipywidgets-8.1.3:\r\n",
      "      Successfully uninstalled ipywidgets-8.1.3\r\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Successfully installed absl-py-1.4.0 annotated-types-0.7.0 apache-beam-2.57.0 async-timeout-4.0.3 backcall-0.2.0 cloudpickle-2.2.1 colorama-0.4.6 crcmod-1.7 dill-0.3.1.1 dnspython-2.6.1 docopt-0.6.2 docstring-parser-0.16 fairness-indicators-0.46.0 fastavro-1.9.5 fasteners-0.19 google-api-core-2.19.1 google-api-python-client-1.12.11 google-apitools-0.5.31 google-auth-2.32.0 google-auth-httplib2-0.2.0 google-auth-oauthlib-1.2.1 google-cloud-aiplatform-1.59.0 google-cloud-bigquery-3.25.0 google-cloud-bigquery-storage-2.25.0 google-cloud-bigtable-2.24.0 google-cloud-core-2.4.1 google-cloud-datastore-2.19.0 google-cloud-dlp-3.18.1 google-cloud-language-2.13.4 google-cloud-pubsub-2.22.0 google-cloud-pubsublite-1.11.0 google-cloud-recommendations-ai-0.10.11 google-cloud-resource-manager-1.12.4 google-cloud-spanner-3.47.0 google-cloud-storage-2.17.0 google-cloud-videointelligence-2.13.4 google-cloud-vision-3.7.3 google-crc32c-1.5.0 google-resumable-media-2.7.1 googleapis-common-protos-1.63.2 grpc-google-iam-v1-0.13.1 grpc-interceptor-0.15.4 grpcio-status-1.48.2 hdfs-2.7.3 httplib2-0.22.0 ipython-7.34.0 ipython-genutils-0.2.0 ipywidgets-7.8.2 js2py-0.74 jsonpickle-3.2.2 jupyterlab-widgets-1.1.8 keras-2.15.0 lxml-5.2.2 ml-dtypes-0.3.2 nltk-3.8.1 oauth2client-4.1.3 oauthlib-3.2.2 objsize-0.7.0 orjson-3.10.6 pandas-1.5.3 pickleshare-0.7.5 portalocker-2.10.1 proto-plus-1.24.0 pyarrow-10.0.1 pyarrow-hotfix-0.6 pyasn1-0.6.0 pyasn1-modules-0.4.0 pydantic-2.8.2 pydantic-core-2.20.1 pydot-1.4.2 pyfarmhash-0.3.2 pyjsparser-2.7.1 pymongo-4.8.0 redis-5.0.7 regex-2024.5.15 requests-2.31.0 requests-oauthlib-2.0.0 rouge-score-0.1.2 rsa-4.9 sacrebleu-2.4.2 shapely-2.0.5 sqlparse-0.5.1 tabulate-0.9.0 tensorboard-2.15.2 tensorflow-2.15.1 tensorflow-data-validation-1.15.1 tensorflow-estimator-2.15.0 tensorflow-model-analysis-0.46.0 tensorflow-serving-api-2.15.1 tf-keras-2.15.1 tfx-bsl-1.15.1 tzlocal-5.2 uritemplate-3.0.1 widgetsnbextension-3.6.7 witwidget-1.8.1 wrapt-1.14.1 zstandard-0.23.0\r\n"
     ]
    }
   ],
   "source": [
    "#@title Installs\n",
    "!pip install --upgrade tensorflow-model-remediation\n",
    "!pip install --upgrade fairness-indicators\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SlyU3HZpob8i"
   },
   "source": [
    "Import all necessary components, including MinDiff and Fairness Indicators for evaluation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "cellView": "form",
    "execution": {
     "iopub.execute_input": "2024-07-19T09:19:47.626798Z",
     "iopub.status.busy": "2024-07-19T09:19:47.626495Z",
     "iopub.status.idle": "2024-07-19T09:19:51.451642Z",
     "shell.execute_reply": "2024-07-19T09:19:51.450946Z"
    },
    "id": "JYLW8UIsIMrE"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "2024-07-19 09:19:48.131764: E external/local_xla/xla/stream_executor/cuda/cuda_dnn.cc:9261] Unable to register cuDNN factory: Attempting to register factory for plugin cuDNN when one has already been registered\n",
      "2024-07-19 09:19:48.131805: E external/local_xla/xla/stream_executor/cuda/cuda_fft.cc:607] Unable to register cuFFT factory: Attempting to register factory for plugin cuFFT when one has already been registered\n",
      "2024-07-19 09:19:48.133491: E external/local_xla/xla/stream_executor/cuda/cuda_blas.cc:1515] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered\n"
     ]
    }
   ],
   "source": [
    "#@title Imports\n",
    "import copy\n",
    "import os\n",
    "import requests\n",
    "import tempfile\n",
    "import zipfile\n",
    "\n",
    "import tensorflow_model_remediation.min_diff as md\n",
    "from tensorflow_model_remediation.tools.tutorials_utils import min_diff_keras_utils\n",
    "\n",
    "from fairness_indicators.tutorial_utils import util as fi_util\n",
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "import tensorflow_hub as hub\n",
    "from tensorflow_model_analysis.addons.fairness.view import widget_view"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zPkyRv5_ozdC"
   },
   "source": [
    "We use a utility function to download the preprocessed data and prepare the labels to match the model’s output shape. The function also downloads the data as TFRecords to make later evaluation quicker. Alternatively, you may convert the Pandas DataFrame into TFRecords with any available utility conversion function.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-07-19T09:19:51.456649Z",
     "iopub.status.busy": "2024-07-19T09:19:51.455787Z",
     "iopub.status.idle": "2024-07-19T09:20:11.674654Z",
     "shell.execute_reply": "2024-07-19T09:20:11.673858Z"
    },
    "id": "-Hw5HdppwuBs"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://storage.googleapis.com/civil_comments_dataset/train_df_processed.csv\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "     8192/345699197 [..............................] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    81920/345699197 [..............................] - ETA: 5:04"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "   278528/345699197 [..............................] - ETA: 2:32"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "   786432/345699197 [..............................] - ETA: 1:15"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "  2351104/345699197 [..............................] - ETA: 32s "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "  7200768/345699197 [..............................] - ETA: 12s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 13172736/345699197 [>.............................] - ETA: 8s "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 17547264/345699197 [>.............................] - ETA: 7s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 19136512/345699197 [>.............................] - ETA: 8s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 26320896/345699197 [=>............................] - ETA: 6s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 28442624/345699197 [=>............................] - ETA: 7s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 36020224/345699197 [==>...........................] - ETA: 5s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 37888000/345699197 [==>...........................] - ETA: 6s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 43040768/345699197 [==>...........................] - ETA: 5s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 50339840/345699197 [===>..........................] - ETA: 5s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 57499648/345699197 [===>..........................] - ETA: 4s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 62537728/345699197 [====>.........................] - ETA: 4s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 67117056/345699197 [====>.........................] - ETA: 4s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 74858496/345699197 [=====>........................] - ETA: 4s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 79552512/345699197 [=====>........................] - ETA: 4s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 83894272/345699197 [======>.......................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 91865088/345699197 [======>.......................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 93954048/345699197 [=======>......................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "100687872/345699197 [=======>......................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "107175936/345699197 [========>.....................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "113147904/345699197 [========>.....................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "118669312/345699197 [=========>....................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "125050880/345699197 [=========>....................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "130514944/345699197 [==========>...................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "134225920/345699197 [==========>...................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "140779520/345699197 [===========>..................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "148422656/345699197 [===========>..................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "151003136/345699197 [============>.................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "158433280/345699197 [============>.................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "164200448/345699197 [=============>................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "171819008/345699197 [=============>................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "176087040/345699197 [==============>...............] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "182902784/345699197 [==============>...............] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "188751872/345699197 [===============>..............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "196075520/345699197 [================>.............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "200318976/345699197 [================>.............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "207970304/345699197 [=================>............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "213696512/345699197 [=================>............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "220692480/345699197 [==================>...........] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "225214464/345699197 [==================>...........] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "232595456/345699197 [===================>..........] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "238247936/345699197 [===================>..........] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "244154368/345699197 [====================>.........] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "250454016/345699197 [====================>.........] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "255549440/345699197 [=====================>........] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "261070848/345699197 [=====================>........] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "265895936/345699197 [======================>.......] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "272957440/345699197 [======================>.......] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "278413312/345699197 [=======================>......] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "284876800/345699197 [=======================>......] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "290160640/345699197 [========================>.....] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "297254912/345699197 [========================>.....] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "301867008/345699197 [=========================>....] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "307716096/345699197 [=========================>....] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "314621952/345699197 [==========================>...] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "319160320/345699197 [==========================>...] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "326639616/345699197 [===========================>..] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "331808768/345699197 [===========================>..] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "338100224/345699197 [============================>.] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "343482368/345699197 [============================>.] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "345699197/345699197 [==============================] - 4s 0us/step\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://storage.googleapis.com/civil_comments_dataset/validate_df_processed.csv\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "     8192/229970098 [..............................] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    81920/229970098 [..............................] - ETA: 3:46"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "   262144/229970098 [..............................] - ETA: 1:57"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "   638976/229970098 [..............................] - ETA: 1:06"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "  1654784/229970098 [..............................] - ETA: 32s "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "  4423680/229970098 [..............................] - ETA: 14s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "  9863168/229970098 [>.............................] - ETA: 7s "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 12591104/229970098 [>.............................] - ETA: 6s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 15392768/229970098 [=>............................] - ETA: 6s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 18022400/229970098 [=>............................] - ETA: 5s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 21471232/229970098 [=>............................] - ETA: 5s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 24190976/229970098 [==>...........................] - ETA: 5s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 26689536/229970098 [==>...........................] - ETA: 4s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 29933568/229970098 [==>...........................] - ETA: 4s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 32473088/229970098 [===>..........................] - ETA: 4s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 35078144/229970098 [===>..........................] - ETA: 4s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 37486592/229970098 [===>..........................] - ETA: 4s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 39837696/229970098 [====>.........................] - ETA: 4s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 43073536/229970098 [====>.........................] - ETA: 4s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 46366720/229970098 [=====>........................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 49807360/229970098 [=====>........................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 54157312/229970098 [======>.......................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 57573376/229970098 [======>.......................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 61153280/229970098 [======>.......................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 64790528/229970098 [=======>......................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 68198400/229970098 [=======>......................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 71000064/229970098 [========>.....................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 75505664/229970098 [========>.....................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 81158144/229970098 [=========>....................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 84811776/229970098 [==========>...................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 87662592/229970098 [==========>...................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 91348992/229970098 [==========>...................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 94461952/229970098 [===========>..................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 97026048/229970098 [===========>..................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "101408768/229970098 [============>.................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "106586112/229970098 [============>.................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "109404160/229970098 [=============>................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "113074176/229970098 [=============>................] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "115548160/229970098 [==============>...............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "117137408/229970098 [==============>...............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "120184832/229970098 [==============>...............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "125526016/229970098 [===============>..............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "130482176/229970098 [================>.............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "134430720/229970098 [================>.............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "139608064/229970098 [=================>............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "142974976/229970098 [=================>............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "147857408/229970098 [==================>...........] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "153174016/229970098 [==================>...........] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "158089216/229970098 [===================>..........] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "159391744/229970098 [===================>..........] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "162152448/229970098 [====================>.........] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "164569088/229970098 [====================>.........] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "167198720/229970098 [====================>.........] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "169943040/229970098 [=====================>........] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "173006848/229970098 [=====================>........] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "177520640/229970098 [======================>.......] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "179535872/229970098 [======================>.......] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "182034432/229970098 [======================>.......] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "185393152/229970098 [=======================>......] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "190332928/229970098 [=======================>......] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "195821568/229970098 [========================>.....] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "200777728/229970098 [=========================>....] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "205996032/229970098 [=========================>....] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "210993152/229970098 [==========================>...] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "215629824/229970098 [===========================>..] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "219791360/229970098 [===========================>..] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "222945280/229970098 [============================>.] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "227688448/229970098 [============================>.] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "229970098/229970098 [==============================] - 4s 0us/step\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Downloading data from https://storage.googleapis.com/civil_comments_dataset/validate_tf_processed.tfrecord\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\r",
      "     8192/324941336 [..............................] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "    81920/324941336 [..............................] - ETA: 5:16"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "   262144/324941336 [..............................] - ETA: 2:42"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "   688128/324941336 [..............................] - ETA: 1:26"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "  1835008/324941336 [..............................] - ETA: 41s "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "  5029888/324941336 [..............................] - ETA: 18s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 10174464/324941336 [..............................] - ETA: 10s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 16556032/324941336 [>.............................] - ETA: 7s "
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 21676032/324941336 [=>............................] - ETA: 6s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 27860992/324941336 [=>............................] - ETA: 5s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 33153024/324941336 [==>...........................] - ETA: 4s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 36462592/324941336 [==>...........................] - ETA: 5s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 43171840/324941336 [==>...........................] - ETA: 5s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 51011584/324941336 [===>..........................] - ETA: 4s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 56156160/324941336 [====>.........................] - ETA: 4s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 61964288/324941336 [====>.........................] - ETA: 4s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 67084288/324941336 [=====>........................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 71901184/324941336 [=====>........................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 79364096/324941336 [======>.......................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 84664320/324941336 [======>.......................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 90644480/324941336 [=======>......................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      " 96141312/324941336 [=======>......................] - ETA: 3s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "100638720/324941336 [========>.....................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "108199936/324941336 [========>.....................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "112295936/324941336 [=========>....................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "117743616/324941336 [=========>....................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "123789312/324941336 [==========>...................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "129310720/324941336 [==========>...................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "135258112/324941336 [===========>..................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "140779520/324941336 [===========>..................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "145874944/324941336 [============>.................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "152510464/324941336 [=============>................] - ETA: 2s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "157360128/324941336 [=============>................] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "162029568/324941336 [=============>................] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "168173568/324941336 [==============>...............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "173506560/324941336 [===============>..............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "174702592/324941336 [===============>..............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "182460416/324941336 [===============>..............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "190111744/324941336 [================>.............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "197763072/324941336 [=================>............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "205086720/324941336 [=================>............] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "209281024/324941336 [==================>...........] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "216645632/324941336 [===================>..........] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "220741632/324941336 [===================>..........] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "227729408/324941336 [====================>.........] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "232210432/324941336 [====================>.........] - ETA: 1s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "239419392/324941336 [=====================>........] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "243671040/324941336 [=====================>........] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "250798080/324941336 [======================>.......] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "255164416/324941336 [======================>.......] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "262365184/324941336 [=======================>......] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "266625024/324941336 [=======================>......] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "273891328/324941336 [========================>.....] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "278069248/324941336 [========================>.....] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "285458432/324941336 [=========================>....] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "289529856/324941336 [=========================>....] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "296738816/324941336 [==========================>...] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "301031424/324941336 [==========================>...] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "307847168/324941336 [===========================>..] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "310386688/324941336 [===========================>..] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "315834368/324941336 [============================>.] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "322863104/324941336 [============================>.] - ETA: 0s"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\b\r",
      "324941336/324941336 [==============================] - 4s 0us/step\n"
     ]
    }
   ],
   "source": [
    "# We use a helper utility to preprocessed data for convenience and speed.\n",
    "data_train, data_validate, validate_tfrecord_file, labels_train, labels_validate = min_diff_keras_utils.download_and_process_civil_comments_data()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LGum4JXSo-Qu"
   },
   "source": [
    "We define a few useful constants.  We will train the model on the `’comment_text’` feature, with our target label as `’toxicity’`. Note that the batch size here is chosen arbitrarily, but in a production setting you would need to tune it for best performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-07-19T09:20:11.679054Z",
     "iopub.status.busy": "2024-07-19T09:20:11.678752Z",
     "iopub.status.idle": "2024-07-19T09:20:11.682231Z",
     "shell.execute_reply": "2024-07-19T09:20:11.681612Z"
    },
    "id": "Ular7EPMU_Y1"
   },
   "outputs": [],
   "source": [
    "TEXT_FEATURE = 'comment_text'\n",
    "LABEL = 'toxicity'\n",
    "BATCH_SIZE = 512"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tyRduaSapFqt"
   },
   "source": [
    "Set random seeds. (Note that this does not fully stabilize results.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-07-19T09:20:11.685818Z",
     "iopub.status.busy": "2024-07-19T09:20:11.685153Z",
     "iopub.status.idle": "2024-07-19T09:20:11.688659Z",
     "shell.execute_reply": "2024-07-19T09:20:11.688052Z"
    },
    "id": "taGEqZGB_FWN"
   },
   "outputs": [],
   "source": [
    "#@title Seeds\n",
    "np.random.seed(1)\n",
    "tf.random.set_seed(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "D_r-uFyQpbkW"
   },
   "source": [
    "## Define and train the baseline model\n",
    "\n",
    "To reduce runtime, we use a pretrained model by default. It is a simple Keras sequential model with an initial embedding and convolution layers, outputting a toxicity prediction. If you prefer, you can change this and train from scratch using our utility function to create the model. (Note that since your environment is likely different from ours, you would need to customize the tuning and evaluation thresholds.)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-07-19T09:20:11.691896Z",
     "iopub.status.busy": "2024-07-19T09:20:11.691328Z",
     "iopub.status.idle": "2024-07-19T09:20:29.568594Z",
     "shell.execute_reply": "2024-07-19T09:20:29.567886Z"
    },
    "id": "KcRceFceKyE_"
   },
   "outputs": [],
   "source": [
    "use_pretrained_model = True #@param {type:\"boolean\"}\n",
    "\n",
    "if use_pretrained_model:\n",
    "  URL = 'https://storage.googleapis.com/civil_comments_model/baseline_model.zip'\n",
    "  BASE_PATH = tempfile.mkdtemp()\n",
    "  ZIP_PATH = os.path.join(BASE_PATH, 'baseline_model.zip')\n",
    "  MODEL_PATH = os.path.join(BASE_PATH, 'tmp/baseline_model')\n",
    "\n",
    "  r = requests.get(URL, allow_redirects=True)\n",
    "  open(ZIP_PATH, 'wb').write(r.content)\n",
    "\n",
    "  with zipfile.ZipFile(ZIP_PATH, 'r') as zip_ref:\n",
    "    zip_ref.extractall(BASE_PATH)\n",
    "  baseline_model = tf.keras.models.load_model(\n",
    "      MODEL_PATH, custom_objects={'KerasLayer' : hub.KerasLayer})\n",
    "else:\n",
    "  optimizer = tf.keras.optimizers.Adam(learning_rate=0.001)\n",
    "  loss = tf.keras.losses.BinaryCrossentropy()\n",
    "\n",
    "  baseline_model = min_diff_keras_utils.create_keras_sequential_model()\n",
    "  \n",
    "  baseline_model.compile(optimizer=optimizer, loss=loss, metrics=['accuracy'])\n",
    "\n",
    "  baseline_model.fit(x=data_train[TEXT_FEATURE],\n",
    "                     y=labels_train,\n",
    "                     batch_size=BATCH_SIZE,\n",
    "                     epochs=20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "m8nsimBdp-lh"
   },
   "source": [
    "We save the model in order to evaluate using [Fairness Indicators](https://www.tensorflow.org/responsible_ai/fairness_indicators)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-07-19T09:20:29.573239Z",
     "iopub.status.busy": "2024-07-19T09:20:29.572871Z",
     "iopub.status.idle": "2024-07-19T09:20:31.513029Z",
     "shell.execute_reply": "2024-07-19T09:20:31.512380Z"
    },
    "id": "DRLTXwDfN6a7"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /tmpfs/tmp/saved_models2j171elm/model_export_baseline/assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /tmpfs/tmp/saved_models2j171elm/model_export_baseline/assets\n"
     ]
    }
   ],
   "source": [
    "base_dir = tempfile.mkdtemp(prefix='saved_models')\n",
    "baseline_model_location = os.path.join(base_dir, 'model_export_baseline')\n",
    "baseline_model.save(baseline_model_location, save_format='tf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7ZRoxj-iqNqm"
   },
   "source": [
    "Next we run Fairness Indicators. As a reminder, we’re just going to perform sliced evaluation for comments referencing one category, *religious groups*. In a production environment, we recommend taking a thoughtful approach to determining which categories and metrics to evaluate across. \n",
    "\n",
    "To compute model performance, the utility function makes a few convenient choices for metrics, slices, and classifier thresholds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-07-19T09:20:31.517387Z",
     "iopub.status.busy": "2024-07-19T09:20:31.516841Z",
     "iopub.status.idle": "2024-07-19T09:29:14.334042Z",
     "shell.execute_reply": "2024-07-19T09:29:14.333281Z"
    },
    "id": "4d6CZj2d-jrw"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:absl:Tensorflow version (2.15.1) found. Note that TFMA support for TF 2.0 is currently in beta\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:apache_beam.runners.interactive.interactive_environment:Dependencies required for Interactive Beam PCollection visualization are not available, please use: `pip install apache-beam[interactive]` to install necessary dependencies to enable all data visualization features.\n"
     ]
    },
    {
     "data": {
      "application/javascript": [
       "\n",
       "        if (typeof window.interactive_beam_jquery == 'undefined') {\n",
       "          var jqueryScript = document.createElement('script');\n",
       "          jqueryScript.src = 'https://code.jquery.com/jquery-3.4.1.slim.min.js';\n",
       "          jqueryScript.type = 'text/javascript';\n",
       "          jqueryScript.onload = function() {\n",
       "            var datatableScript = document.createElement('script');\n",
       "            datatableScript.src = 'https://cdn.datatables.net/1.10.20/js/jquery.dataTables.min.js';\n",
       "            datatableScript.type = 'text/javascript';\n",
       "            datatableScript.onload = function() {\n",
       "              window.interactive_beam_jquery = jQuery.noConflict(true);\n",
       "              window.interactive_beam_jquery(document).ready(function($){\n",
       "                \n",
       "              });\n",
       "            }\n",
       "            document.head.appendChild(datatableScript);\n",
       "          };\n",
       "          document.head.appendChild(jqueryScript);\n",
       "        } else {\n",
       "          window.interactive_beam_jquery(document).ready(function($){\n",
       "            \n",
       "          });\n",
       "        }"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:apache_beam.io.tfrecordio:Couldn't find python-snappy so the implementation of _TFRecordUtil._masked_crc32c is not as fast as it could be.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow_model_analysis/writers/metrics_plots_and_validations_writer.py:112: tf_record_iterator (from tensorflow.python.lib.io.tf_record) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use eager execution and: \n",
      "`tf.data.TFRecordDataset(path)`\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /tmpfs/src/tf_docs_env/lib/python3.9/site-packages/tensorflow_model_analysis/writers/metrics_plots_and_validations_writer.py:112: tf_record_iterator (from tensorflow.python.lib.io.tf_record) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use eager execution and: \n",
      "`tf.data.TFRecordDataset(path)`\n"
     ]
    }
   ],
   "source": [
    "# We use a helper utility to hide the evaluation logic for readability.\n",
    "base_dir = tempfile.mkdtemp(prefix='eval')\n",
    "eval_dir = os.path.join(base_dir, 'tfma_eval_result')\n",
    "eval_result = fi_util.get_eval_results(\n",
    "    baseline_model_location, eval_dir, validate_tfrecord_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "S1X0wtRXmHPX"
   },
   "source": [
    "###  Render Evaluation Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-07-19T09:29:14.338030Z",
     "iopub.status.busy": "2024-07-19T09:29:14.337379Z",
     "iopub.status.idle": "2024-07-19T09:29:14.357286Z",
     "shell.execute_reply": "2024-07-19T09:29:14.356634Z"
    },
    "id": "ynbJR3Qc-j0D"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "88972b92aec44535969b86414a4efeea",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FairnessIndicatorViewer(slicingMetrics=[{'sliceValue': 'Overall', 'slice': 'Overall', 'metrics': {'accuracy': …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "widget_view.render_fairness_indicator(eval_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "o2cYxsxNqT_P"
   },
   "source": [
    "Let’s look at the evaluation results. Try selecting the metric false positive rate (FPR) with threshold 0.450. We can see that the model does not perform as well for some religious groups as for others, displaying a much higher FPR. Note the wide confidence intervals on some groups because they have too few examples. This makes it difficult to say with certainty that there is a significant difference in performance for these slices. We may want to collect more examples to address this issue. We can, however, attempt to apply MinDiff for the two groups that we are confident are underperforming.\n",
    "\n",
    "We’ve chosen to focus on FPR, because a higher FPR means that comments referencing these identity groups are more likely to be incorrectly flagged as toxic than other comments. This could lead to inequitable outcomes for users engaging in dialogue about religion, but note that disparities in other metrics can lead to other types of harm."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CRG6SHR8ryMV"
   },
   "source": [
    "## Define and Train the MinDiff Model\n",
    "\n",
    "Now, we’ll try to improve the FPR for underperforming religious groups. We’ll attempt to do so using [MinDiff](https://arxiv.org/abs/1910.11779), a remediation technique that seeks to balance error rates across slices of your data by penalizing disparities in performance during training. When we apply MinDiff, model performance may degrade slightly on other slices. As such, our goals with MinDiff will be:\n",
    "*   Improved performance for underperforming groups\n",
    "*   Limited degradation for other groups and overall performance\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "opFCpJjadf7g"
   },
   "source": [
    "### Prepare your data\n",
    "\n",
    "To use MinDiff, we create two additional data splits:\n",
    "* A split for non-toxic examples referencing minority groups: In our case, this will include comments with references to our underperforming identity terms.  We don’t include some of the groups because there are too few examples, leading to higher uncertainty with wide confidence interval ranges.\n",
    "* A split for non-toxic examples referencing the majority group.\n",
    "\n",
    "It’s important to have sufficient examples belonging to the underperforming classes. Based on your model architecture, data distribution, and MinDiff configuration, the amount of data needed can vary significantly. In past applications, we have seen MinDiff work well with 5,000 examples in each data split.\n",
    "\n",
    "In our case, the groups in the minority splits have example quantities of 9,688 and 3,906. Note the class imbalances in the dataset; in practice, this could be cause for concern, but we won’t seek to address them in this notebook since our intention is just to demonstrate MinDiff.  \n",
    "\n",
    "We select only negative examples for these groups, so that MinDiff can optimize on getting these examples right. It may seem counterintuitive to carve out sets of ground truth *negative* examples if we’re primarily concerned with disparities in *false positive rate*, but remember that a false positive prediction is a ground truth negative example that’s incorrectly classified as positive, which is the issue we’re trying to address."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1QilngDumRfI"
   },
   "source": [
    "#### Create MinDiff DataFrames"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-07-19T09:29:14.365056Z",
     "iopub.status.busy": "2024-07-19T09:29:14.364444Z",
     "iopub.status.idle": "2024-07-19T09:29:15.367203Z",
     "shell.execute_reply": "2024-07-19T09:29:15.366465Z"
    },
    "id": "jj4dychpyrqM"
   },
   "outputs": [],
   "source": [
    "# Create masks for the sensitive and nonsensitive groups\n",
    "minority_mask = data_train.religion.apply(\n",
    "    lambda x: any(religion in x for religion in ('jewish', 'muslim')))\n",
    "majority_mask = data_train.religion.apply(lambda x: x == \"['christian']\")\n",
    "\n",
    "# Select nontoxic examples, so MinDiff will be able to reduce sensitive FP rate.\n",
    "true_negative_mask = data_train['toxicity'] == 0\n",
    "\n",
    "data_train_main = copy.copy(data_train)\n",
    "data_train_sensitive = data_train[minority_mask & true_negative_mask]\n",
    "data_train_nonsensitive = data_train[majority_mask & true_negative_mask]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "3lR_w3LHt6QK"
   },
   "source": [
    "We also need to convert our Pandas DataFrames into Tensorflow Datasets for MinDiff input.  Note that unlike the Keras model API for Pandas DataFrames, using Datasets means that we need to provide the model’s input features and labels together in one Dataset. Here we provide the `'comment_text'` as an input feature and reshape the label to match the model's expected output. \n",
    "\n",
    "We batch the Dataset at this stage, too, since MinDiff requires batched Datasets.  Note that we tune the batch size selection the same way it is tuned for the baseline model, taking into account training speed and hardware considerations while balancing with model performance. Here we have chosen the same batch size for all three datasets but this is not a requirement, although it’s good practice to have the two MinDiff batch sizes be equivalent."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yA4Kw9tsmopa"
   },
   "source": [
    "#### Create MinDiff Datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-07-19T09:29:15.371757Z",
     "iopub.status.busy": "2024-07-19T09:29:15.371071Z",
     "iopub.status.idle": "2024-07-19T09:29:15.632856Z",
     "shell.execute_reply": "2024-07-19T09:29:15.632132Z"
    },
    "id": "C0DkFMTOuIQT"
   },
   "outputs": [],
   "source": [
    "# Convert the pandas DataFrames to Datasets.\n",
    "dataset_train_main = tf.data.Dataset.from_tensor_slices(\n",
    "    (data_train_main['comment_text'].values, \n",
    "     data_train_main.pop(LABEL).values.reshape(-1,1) * 1.0)).batch(BATCH_SIZE)\n",
    "dataset_train_sensitive = tf.data.Dataset.from_tensor_slices(\n",
    "    (data_train_sensitive['comment_text'].values, \n",
    "     data_train_sensitive.pop(LABEL).values.reshape(-1,1) * 1.0)).batch(BATCH_SIZE)\n",
    "dataset_train_nonsensitive = tf.data.Dataset.from_tensor_slices(\n",
    "    (data_train_nonsensitive['comment_text'].values, \n",
    "     data_train_nonsensitive.pop(LABEL).values.reshape(-1,1) * 1.0)).batch(BATCH_SIZE)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XRGvjZ8VuBvz"
   },
   "source": [
    "### Train and evaluate the model\n",
    "\n",
    "To train with MinDiff, simply take the original model and wrap it in a MinDiffModel with a corresponding `loss` and `loss_weight`.  We are using 1.5 as the default `loss_weight`, but this is a parameter that needs to be tuned for your use case, since it depends on your model and product requirements.  You can experiment with changing the value to see how it impacts the model, noting that increasing it pushes the performance of the minority and majority groups closer together but may come with more pronounced tradeoffs.\n",
    "\n",
    "Then we compile the model normally (using the regular non-MinDiff loss) and fit to train."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "eP_eTUpYm6U-"
   },
   "source": [
    "#### Train MinDiffModel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-07-19T09:29:15.637652Z",
     "iopub.status.busy": "2024-07-19T09:29:15.637120Z",
     "iopub.status.idle": "2024-07-19T09:29:25.882838Z",
     "shell.execute_reply": "2024-07-19T09:29:25.882141Z"
    },
    "id": "xutVGl9fyikP"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /tmpfs/tmp/saved_modelsi6zpos5k/model_export_min_diff/assets\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:tensorflow:Assets written to: /tmpfs/tmp/saved_modelsi6zpos5k/model_export_min_diff/assets\n"
     ]
    }
   ],
   "source": [
    "use_pretrained_model = True #@param {type:\"boolean\"}\n",
    "\n",
    "base_dir = tempfile.mkdtemp(prefix='saved_models')\n",
    "min_diff_model_location = os.path.join(base_dir, 'model_export_min_diff')\n",
    "\n",
    "if use_pretrained_model:\n",
    "  BASE_MIN_DIFF_PATH = tempfile.mkdtemp()\n",
    "  MIN_DIFF_URL = 'https://storage.googleapis.com/civil_comments_model/min_diff_model.zip'\n",
    "  ZIP_PATH = os.path.join(BASE_PATH, 'min_diff_model.zip')\n",
    "  MIN_DIFF_MODEL_PATH = os.path.join(BASE_MIN_DIFF_PATH, 'tmp/min_diff_model')\n",
    "  DIRPATH = '/tmp/min_diff_model'\n",
    "\n",
    "  r = requests.get(MIN_DIFF_URL, allow_redirects=True)\n",
    "  open(ZIP_PATH, 'wb').write(r.content)\n",
    "\n",
    "  with zipfile.ZipFile(ZIP_PATH, 'r') as zip_ref:\n",
    "    zip_ref.extractall(BASE_MIN_DIFF_PATH)\n",
    "  min_diff_model = tf.keras.models.load_model(\n",
    "      MIN_DIFF_MODEL_PATH, custom_objects={'KerasLayer' : hub.KerasLayer})\n",
    "  \n",
    "  min_diff_model.save(min_diff_model_location, save_format='tf')\n",
    "\n",
    "else:\n",
    "  min_diff_weight = 1.5 #@param {type:\"number\"}\n",
    "\n",
    "  # Create the dataset that will be passed to the MinDiffModel during training.\n",
    "  dataset = md.keras.utils.input_utils.pack_min_diff_data(\n",
    "      dataset_train_main, dataset_train_sensitive, dataset_train_nonsensitive)\n",
    "\n",
    "  # Create the original model.\n",
    "  original_model = min_diff_keras_utils.create_keras_sequential_model()\n",
    "  \n",
    "  # Wrap the original model in a MinDiffModel, passing in one of the MinDiff\n",
    "  # losses and using the set loss_weight.\n",
    "  min_diff_loss = md.losses.MMDLoss()\n",
    "  min_diff_model = md.keras.MinDiffModel(original_model,\n",
    "                                         min_diff_loss,\n",
    "                                         min_diff_weight)\n",
    "\n",
    "  # Compile the model normally after wrapping the original model.  Note that\n",
    "  # this means we use the baseline's model's loss here.\n",
    "  optimizer = tf.keras.optimizers.Adam(learning_rate=0.001)\n",
    "  loss = tf.keras.losses.BinaryCrossentropy()\n",
    "  min_diff_model.compile(optimizer=optimizer, loss=loss, metrics=['accuracy'])\n",
    "\n",
    "  min_diff_model.fit(dataset, epochs=20)\n",
    "\n",
    "  min_diff_model.save_original_model(min_diff_model_location, save_format='tf')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "doJhbIKVwQdp"
   },
   "source": [
    "Next we evaluate the results.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-07-19T09:29:25.887584Z",
     "iopub.status.busy": "2024-07-19T09:29:25.886922Z",
     "iopub.status.idle": "2024-07-19T09:37:47.786534Z",
     "shell.execute_reply": "2024-07-19T09:37:47.785664Z"
    },
    "id": "0CBdOQCH5IR6"
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:absl:Tensorflow version (2.15.1) found. Note that TFMA support for TF 2.0 is currently in beta\n"
     ]
    }
   ],
   "source": [
    "min_diff_eval_subdir = os.path.join(base_dir, 'tfma_eval_result')\n",
    "min_diff_eval_result = fi_util.get_eval_results(\n",
    "    min_diff_model_location,\n",
    "    min_diff_eval_subdir,\n",
    "    validate_tfrecord_file,\n",
    "    slice_selection='religion')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "JObiq-mVwUzL"
   },
   "source": [
    "To ensure we evaluate a new model correctly, we need to select a threshold the same way that we would the baseline model. In a production setting, this would mean ensuring that evaluation metrics meet launch standards. In our case, we will pick the threshold that results in a similar overall FPR to the baseline model. This threshold may be different from the one you selected for the baseline model.  Try selecting false positive rate with threshold 0.400.  (Note that the subgroups with very low quantity examples have very wide confidence range intervals and don’t have predictable results.)   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2024-07-19T09:37:47.791121Z",
     "iopub.status.busy": "2024-07-19T09:37:47.790846Z",
     "iopub.status.idle": "2024-07-19T09:37:47.806062Z",
     "shell.execute_reply": "2024-07-19T09:37:47.805405Z"
    },
    "id": "A3_PEjYBO3Dq"
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "de0e9b637e034835a901de1caa1fcbed",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "FairnessIndicatorViewer(slicingMetrics=[{'sliceValue': 'Overall', 'slice': 'Overall', 'metrics': {'accuracy': …"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "widget_view.render_fairness_indicator(min_diff_eval_result)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sVRurkbZwYMj"
   },
   "source": [
    "Reviewing these results, you may notice that the FPRs for our target groups have improved. The gap between our lowest performing group and the majority group has improved from .024 to .006. Given the improvements we’ve observed and the continued strong performance for the majority group, we’ve satisfied both of our goals. Depending on the product, further improvements may be necessary, but this approach has gotten our model one step closer to performing equitably for all users."
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [],
   "name": "min_diff_keras.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.19"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "state": {
     "3b5c7409e69f4cc39681b4eb65b634c5": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "88972b92aec44535969b86414a4efeea": {
      "model_module": "tensorflow_model_analysis",
      "model_module_version": "0.46.0",
      "model_name": "FairnessIndicatorModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "tensorflow_model_analysis",
       "_model_module_version": "0.46.0",
       "_model_name": "FairnessIndicatorModel",
       "_view_count": null,
       "_view_module": "tensorflow_model_analysis",
       "_view_module_version": "0.46.0",
       "_view_name": "FairnessIndicatorView",
       "evalName": "",
       "evalNameCompare": "",
       "js_events": [],
       "layout": "IPY_MODEL_3b5c7409e69f4cc39681b4eb65b634c5",
       "slicingMetrics": [
        {
         "metrics": {
          "accuracy": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "auc": {
           "boundedValue": {
            "lowerBound": 0.7916632597466121,
            "upperBound": 0.7955818151604456,
            "value": 0.7936224743972262
           }
          },
          "example_count": {
           "doubleValue": 721950.0
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.4963198646762778,
            "upperBound": 0.508845088048845,
            "value": 0.50258242023459
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.4863579235809003,
            "upperBound": 0.5002303176573045,
            "value": 0.4932941297843805
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.4755528987664636,
            "upperBound": 0.4916472089813904,
            "value": 0.48360022087244614
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.465055722557109,
            "upperBound": 0.4808339096586112,
            "value": 0.47294494162271083
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.4545948226943026,
            "upperBound": 0.4710787740856254,
            "value": 0.4628367619896822
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.44219820521105585,
            "upperBound": 0.4573987332329044,
            "value": 0.44979834430057314
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.4383885200501699,
            "upperBound": 0.4524078188119348,
            "value": 0.44539804675939626
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.42715941898479526,
            "upperBound": 0.4409330441332223,
            "value": 0.43404592240696754
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.41507484724900723,
            "upperBound": 0.4298511813705339,
            "value": 0.422462648119526
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.8192593370195388,
            "upperBound": 0.8247575267720234,
            "value": 0.8220084289070057
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.8274936933310938,
            "upperBound": 0.8331736853813455,
            "value": 0.8303337018101423
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.8352745922606607,
            "upperBound": 0.8416633571089499,
            "value": 0.8384689788586431
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.8432646733204443,
            "upperBound": 0.8495292963322238,
            "value": 0.8463969877020865
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.8510917008619685,
            "upperBound": 0.8575582766910681,
            "value": 0.85432499654553
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.8626740958313095,
            "upperBound": 0.868706312185112,
            "value": 0.865690203122841
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.8674322874574354,
            "upperBound": 0.8736205987677448,
            "value": 0.8705264612408457
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.8734836373851431,
            "upperBound": 0.8795907709662347,
            "value": 0.8765372391875086
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.8808119279506524,
            "upperBound": 0.8868403905315038,
            "value": 0.8838261710653585
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.0671890010802722,
            "upperBound": 0.06854619474585538,
            "value": 0.0678675989293145
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.06773493637929051,
            "upperBound": 0.06911522600809022,
            "value": 0.06842508298176393
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.06827384985345476,
            "upperBound": 0.06966658695952853,
            "value": 0.06897022050466015
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.06880356432400593,
            "upperBound": 0.07019685227361898,
            "value": 0.0695002106152945
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.06933223404025546,
            "upperBound": 0.07073733901988047,
            "value": 0.07003478942979034
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.07010196536447898,
            "upperBound": 0.07151655672735803,
            "value": 0.07080926284618765
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.07041752709570358,
            "upperBound": 0.07186813032503302,
            "value": 0.0711428305247913
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.07081295487589219,
            "upperBound": 0.07227619556018587,
            "value": 0.07154457790559973
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.07131716049747189,
            "upperBound": 0.07276198961413435,
            "value": 0.07203957742037212
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.01539779450216285,
            "upperBound": 0.015961106643121583,
            "value": 0.0156794477557548
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.014112672492153646,
            "upperBound": 0.014689206273517197,
            "value": 0.014400937273173568
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.012849740287634004,
            "upperBound": 0.013527638527488036,
            "value": 0.01318868646224554
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.011705304645736749,
            "upperBound": 0.012328889884439639,
            "value": 0.012017094995286528
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.010660734607681414,
            "upperBound": 0.011226044228223153,
            "value": 0.010943387134178847
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.009332052453618357,
            "upperBound": 0.009813983861997893,
            "value": 0.009573016652260208
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.008830737599209175,
            "upperBound": 0.009300320809082158,
            "value": 0.009065527803461766
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.008058247508838193,
            "upperBound": 0.008452459820450087,
            "value": 0.00825535272733844
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.00721998284083081,
            "upperBound": 0.00759809008823257,
            "value": 0.007409036012131543
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9709252838625962,
            "upperBound": 0.9716829239512527,
            "value": 0.971304106932613
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9727725145553655,
            "upperBound": 0.9735229291322879,
            "value": 0.9731477249116974
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9744685988227888,
            "upperBound": 0.9753617138524815,
            "value": 0.9749151603296627
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9761939188522741,
            "upperBound": 0.9770632248942104,
            "value": 0.976628575386107
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9778386814295567,
            "upperBound": 0.9786652233939804,
            "value": 0.9782519565066833
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9800383502460381,
            "upperBound": 0.9808093496286858,
            "value": 0.9804238520673176
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.980876921730722,
            "upperBound": 0.9816800385071651,
            "value": 0.9812784818893275
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.982138763243456,
            "upperBound": 0.9828726596648762,
            "value": 0.982505713692084
           }
          },
          "fairness_indicators_metrics/negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9835094627365902,
            "upperBound": 0.9842279121499923,
            "value": 0.98386868896738
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.02831707604874723,
            "upperBound": 0.029074716137403773,
            "value": 0.02869589306738694
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.02647707086771205,
            "upperBound": 0.02722748544463439,
            "value": 0.026852275088302513
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.024638286147518427,
            "upperBound": 0.02553140117721098,
            "value": 0.02508483967033728
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.022936775105789425,
            "upperBound": 0.023806081147725644,
            "value": 0.023371424613892928
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.021334776606019446,
            "upperBound": 0.022161318570443612,
            "value": 0.021748043493316712
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.019190650371314264,
            "upperBound": 0.01996164975396161,
            "value": 0.01957614793268232
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.01831996149283538,
            "upperBound": 0.019123078269277748,
            "value": 0.018721518110672485
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.017127340335123812,
            "upperBound": 0.017861236756544042,
            "value": 0.017494286307916062
           }
          },
          "fairness_indicators_metrics/positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.015772087850007714,
            "upperBound": 0.016490537263409884,
            "value": 0.01613131103261999
           }
          },
          "fairness_indicators_metrics/precision@0.4000": {
           "boundedValue": {
            "lowerBound": 0.4911549119511553,
            "upperBound": 0.5036801353237225,
            "value": 0.49741757976541007
           }
          },
          "fairness_indicators_metrics/precision@0.4125": {
           "boundedValue": {
            "lowerBound": 0.49976968234269525,
            "upperBound": 0.5136420764190998,
            "value": 0.5067058702156195
           }
          },
          "fairness_indicators_metrics/precision@0.4250": {
           "boundedValue": {
            "lowerBound": 0.5083527910186095,
            "upperBound": 0.5244471012335364,
            "value": 0.5163997791275539
           }
          },
          "fairness_indicators_metrics/precision@0.4375": {
           "boundedValue": {
            "lowerBound": 0.5191660903413888,
            "upperBound": 0.534944277442891,
            "value": 0.5270550583772892
           }
          },
          "fairness_indicators_metrics/precision@0.4500": {
           "boundedValue": {
            "lowerBound": 0.5289212259143748,
            "upperBound": 0.5454051773056973,
            "value": 0.5371632380103178
           }
          },
          "fairness_indicators_metrics/precision@0.4675": {
           "boundedValue": {
            "lowerBound": 0.5426012667670956,
            "upperBound": 0.5578017947889442,
            "value": 0.5502016556994269
           }
          },
          "fairness_indicators_metrics/precision@0.4750": {
           "boundedValue": {
            "lowerBound": 0.5475921811880653,
            "upperBound": 0.5616114799498302,
            "value": 0.5546019532406037
           }
          },
          "fairness_indicators_metrics/precision@0.4875": {
           "boundedValue": {
            "lowerBound": 0.5590669558667777,
            "upperBound": 0.5728405810152047,
            "value": 0.5659540775930325
           }
          },
          "fairness_indicators_metrics/precision@0.5000": {
           "boundedValue": {
            "lowerBound": 0.570148818629466,
            "upperBound": 0.584925152750993,
            "value": 0.577537351880474
           }
          },
          "fairness_indicators_metrics/recall@0.4000": {
           "boundedValue": {
            "lowerBound": 0.17524247322797662,
            "upperBound": 0.18074066298046118,
            "value": 0.17799157109299432
           }
          },
          "fairness_indicators_metrics/recall@0.4125": {
           "boundedValue": {
            "lowerBound": 0.1668263146186546,
            "upperBound": 0.1725063066689062,
            "value": 0.16966629818985768
           }
          },
          "fairness_indicators_metrics/recall@0.4250": {
           "boundedValue": {
            "lowerBound": 0.15833664289105026,
            "upperBound": 0.16472540773933936,
            "value": 0.16153102114135692
           }
          },
          "fairness_indicators_metrics/recall@0.4375": {
           "boundedValue": {
            "lowerBound": 0.1504707036677762,
            "upperBound": 0.15673532667955548,
            "value": 0.1536030122979135
           }
          },
          "fairness_indicators_metrics/recall@0.4500": {
           "boundedValue": {
            "lowerBound": 0.14244172330893187,
            "upperBound": 0.14890829913803152,
            "value": 0.1456750034544701
           }
          },
          "fairness_indicators_metrics/recall@0.4675": {
           "boundedValue": {
            "lowerBound": 0.1312936878148875,
            "upperBound": 0.13732590416869048,
            "value": 0.13430979687715905
           }
          },
          "fairness_indicators_metrics/recall@0.4750": {
           "boundedValue": {
            "lowerBound": 0.12637940123225516,
            "upperBound": 0.1325677125425645,
            "value": 0.12947353875915435
           }
          },
          "fairness_indicators_metrics/recall@0.4875": {
           "boundedValue": {
            "lowerBound": 0.12040922903376537,
            "upperBound": 0.12651636261485655,
            "value": 0.12346276081249137
           }
          },
          "fairness_indicators_metrics/recall@0.5000": {
           "boundedValue": {
            "lowerBound": 0.1131596094684965,
            "upperBound": 0.11918807204934738,
            "value": 0.11617382893464143
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9840388933568781,
            "upperBound": 0.9846022054978371,
            "value": 0.9843205522442452
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9853107937264828,
            "upperBound": 0.9858873275078462,
            "value": 0.9855990627268264
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9864723614725119,
            "upperBound": 0.9871502597123658,
            "value": 0.9868113135377544
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9876711101155604,
            "upperBound": 0.9882946953542635,
            "value": 0.9879829050047134
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9887739557717768,
            "upperBound": 0.9893392653923186,
            "value": 0.9890566128658211
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9901860161380018,
            "upperBound": 0.9906679475463818,
            "value": 0.9904269833477398
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9906996791909177,
            "upperBound": 0.9911692624007908,
            "value": 0.9909344721965382
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9915475401795497,
            "upperBound": 0.991941752491162,
            "value": 0.9917446472726615
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9924019099117672,
            "upperBound": 0.9927800171591691,
            "value": 0.9925909639878685
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.17524247322797662,
            "upperBound": 0.18074066298046118,
            "value": 0.17799157109299432
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.1668263146186546,
            "upperBound": 0.1725063066689062,
            "value": 0.16966629818985768
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.15833664289105026,
            "upperBound": 0.16472540773933936,
            "value": 0.16153102114135692
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.1504707036677762,
            "upperBound": 0.15673532667955548,
            "value": 0.1536030122979135
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.14244172330893187,
            "upperBound": 0.14890829913803152,
            "value": 0.1456750034544701
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.1312936878148875,
            "upperBound": 0.13732590416869048,
            "value": 0.13430979687715905
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.12637940123225516,
            "upperBound": 0.1325677125425645,
            "value": 0.12947353875915435
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.12040922903376537,
            "upperBound": 0.12651636261485655,
            "value": 0.12346276081249137
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.1131596094684965,
            "upperBound": 0.11918807204934738,
            "value": 0.11617382893464143
           }
          }
         },
         "slice": "Overall",
         "sliceValue": "Overall"
        },
        {
         "metrics": {
          "accuracy": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "auc": {
           "boundedValue": {
            "lowerBound": 0.7138955253038639,
            "upperBound": 0.7387764058470738,
            "value": 0.7263361238651004
           }
          },
          "example_count": {
           "doubleValue": 8436.0
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.41768459204433966,
            "upperBound": 0.4866101488163424,
            "value": 0.4521410579345088
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.4145516935522646,
            "upperBound": 0.488598053884179,
            "value": 0.451568894952251
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.3959020563459898,
            "upperBound": 0.47669180013907014,
            "value": 0.4362818590704648
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.3875180154858759,
            "upperBound": 0.47615566450663177,
            "value": 0.4318181818181818
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.3784832836949393,
            "upperBound": 0.47005554485489437,
            "value": 0.42424242424242425
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.3687057434113893,
            "upperBound": 0.4629902750683331,
            "value": 0.4158215010141988
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.35930757656748363,
            "upperBound": 0.4528389229492243,
            "value": 0.4060475161987041
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.3555261046769302,
            "upperBound": 0.44923727201658853,
            "value": 0.4023529411764706
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.3597420994733457,
            "upperBound": 0.44130108078417607,
            "value": 0.4005037783375315
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.7472244119424597,
            "upperBound": 0.7946361144280681,
            "value": 0.7709320695102686
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.7661315108115437,
            "upperBound": 0.8104851254921398,
            "value": 0.7883096366508688
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.7778170323315403,
            "upperBound": 0.8261827152796017,
            "value": 0.8020010531858873
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.7902496566423393,
            "upperBound": 0.8411339831845529,
            "value": 0.8156924697209057
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.8037135046181862,
            "upperBound": 0.8561056879306974,
            "value": 0.8299104791995787
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.8248977968670029,
            "upperBound": 0.871780568330215,
            "value": 0.8483412322274881
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.83271322516903,
            "upperBound": 0.8776577713347417,
            "value": 0.8551869404949973
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.8456473651429774,
            "upperBound": 0.8868419462367207,
            "value": 0.866245392311743
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.8556915958088132,
            "upperBound": 0.8936461053338536,
            "value": 0.8746708794102159
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.17963722172181676,
            "upperBound": 0.20350724469908146,
            "value": 0.19157288667887987
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.18251335080394077,
            "upperBound": 0.20616521467836413,
            "value": 0.19433986758405816
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.18400359940499936,
            "upperBound": 0.20806615359171884,
            "value": 0.19603552580769726
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.18595529205177092,
            "upperBound": 0.2102071373376901,
            "value": 0.19808184143222507
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.18780951735868912,
            "upperBound": 0.21244307298641385,
            "value": 0.20012698412698413
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.1903105543415027,
            "upperBound": 0.21532828353120145,
            "value": 0.20282009316379201
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.1914043562944103,
            "upperBound": 0.2159691949054037,
            "value": 0.20368744512730466
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.19336680379676746,
            "upperBound": 0.21731720424666373,
            "value": 0.20534265385095493
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.194489379530961,
            "upperBound": 0.21874458119923187,
            "value": 0.20661773852469212
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.04899495302051357,
            "upperBound": 0.06084235317102955,
            "value": 0.05491815817653358
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.04492405372709993,
            "upperBound": 0.05634657643995772,
            "value": 0.05063484778950589
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.03972278458379907,
            "upperBound": 0.04930956899828589,
            "value": 0.0445158329508949
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.03572121804794063,
            "upperBound": 0.04566238989306059,
            "value": 0.04069144867676304
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.03206661982204661,
            "upperBound": 0.04075017493290583,
            "value": 0.03640813828973535
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.027554416795080906,
            "upperBound": 0.03516615841782837,
            "value": 0.03135995104788129
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.02512731447519485,
            "upperBound": 0.03239179829268107,
            "value": 0.028759369741471624
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.023042719648318236,
            "upperBound": 0.02927511358346195,
            "value": 0.026158788435061954
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.021087220703763718,
            "upperBound": 0.02755925172329534,
            "value": 0.02432308398347866
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.8977938142485167,
            "upperBound": 0.9139646249146951,
            "value": 0.9058795637743006
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.905631462714042,
            "upperBound": 0.920588877850826,
            "value": 0.9131104788999526
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9138525659313149,
            "upperBound": 0.9280152169188655,
            "value": 0.9209340919867236
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.919486364540124,
            "upperBound": 0.934472585746334,
            "value": 0.9269796111901375
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9261630854534315,
            "upperBound": 0.9408353866291601,
            "value": 0.9334992887624467
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9353709037353233,
            "upperBound": 0.9477487815546508,
            "value": 0.9415599810336652
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9390164833310556,
            "upperBound": 0.9512157839034684,
            "value": 0.9451161688003793
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9440948136667623,
            "upperBound": 0.9551464519245382,
            "value": 0.9496206733048839
           }
          },
          "fairness_indicators_metrics/negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.947281119070566,
            "upperBound": 0.9585981365095494,
            "value": 0.9529397818871503
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.08603537508530501,
            "upperBound": 0.10220618575148338,
            "value": 0.09412043622569938
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.07941112214917402,
            "upperBound": 0.09436853728595833,
            "value": 0.08688952110004741
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.0719847830811345,
            "upperBound": 0.08614743406868476,
            "value": 0.07906590801327644
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.06552741425366598,
            "upperBound": 0.08051363545987608,
            "value": 0.07302038880986249
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.05916461337083983,
            "upperBound": 0.07383691454656849,
            "value": 0.06650071123755334
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.052251218445349405,
            "upperBound": 0.06462909626467676,
            "value": 0.05844001896633476
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.048784216096531594,
            "upperBound": 0.06098351666894457,
            "value": 0.054883831199620675
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.04485354807546199,
            "upperBound": 0.0559051863332381,
            "value": 0.05037932669511617
           }
          },
          "fairness_indicators_metrics/positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.041401863490450594,
            "upperBound": 0.05271888092943415,
            "value": 0.04706021811284969
           }
          },
          "fairness_indicators_metrics/precision@0.4000": {
           "boundedValue": {
            "lowerBound": 0.5133898511836574,
            "upperBound": 0.5823154079556603,
            "value": 0.5478589420654912
           }
          },
          "fairness_indicators_metrics/precision@0.4125": {
           "boundedValue": {
            "lowerBound": 0.5114019461158212,
            "upperBound": 0.5854483064477352,
            "value": 0.548431105047749
           }
          },
          "fairness_indicators_metrics/precision@0.4250": {
           "boundedValue": {
            "lowerBound": 0.5233081998609298,
            "upperBound": 0.60409794365401,
            "value": 0.5637181409295352
           }
          },
          "fairness_indicators_metrics/precision@0.4375": {
           "boundedValue": {
            "lowerBound": 0.5238443354933682,
            "upperBound": 0.6124819845141243,
            "value": 0.5681818181818182
           }
          },
          "fairness_indicators_metrics/precision@0.4500": {
           "boundedValue": {
            "lowerBound": 0.5299444551451057,
            "upperBound": 0.6215167163050604,
            "value": 0.5757575757575758
           }
          },
          "fairness_indicators_metrics/precision@0.4675": {
           "boundedValue": {
            "lowerBound": 0.5370097249316668,
            "upperBound": 0.6312942565886106,
            "value": 0.5841784989858012
           }
          },
          "fairness_indicators_metrics/precision@0.4750": {
           "boundedValue": {
            "lowerBound": 0.5471610770507757,
            "upperBound": 0.6406924234325165,
            "value": 0.593952483801296
           }
          },
          "fairness_indicators_metrics/precision@0.4875": {
           "boundedValue": {
            "lowerBound": 0.5507627279834114,
            "upperBound": 0.6444738953230699,
            "value": 0.5976470588235294
           }
          },
          "fairness_indicators_metrics/precision@0.5000": {
           "boundedValue": {
            "lowerBound": 0.558698919215824,
            "upperBound": 0.6402579005266542,
            "value": 0.5994962216624685
           }
          },
          "fairness_indicators_metrics/recall@0.4000": {
           "boundedValue": {
            "lowerBound": 0.20536388557193178,
            "upperBound": 0.2527755880575402,
            "value": 0.22906793048973143
           }
          },
          "fairness_indicators_metrics/recall@0.4125": {
           "boundedValue": {
            "lowerBound": 0.18951487450786023,
            "upperBound": 0.2338684891884559,
            "value": 0.21169036334913113
           }
          },
          "fairness_indicators_metrics/recall@0.4250": {
           "boundedValue": {
            "lowerBound": 0.17381728472039834,
            "upperBound": 0.2221829676684596,
            "value": 0.1979989468141127
           }
          },
          "fairness_indicators_metrics/recall@0.4375": {
           "boundedValue": {
            "lowerBound": 0.15886601681544701,
            "upperBound": 0.20975034335766077,
            "value": 0.18430753027909427
           }
          },
          "fairness_indicators_metrics/recall@0.4500": {
           "boundedValue": {
            "lowerBound": 0.14389431206930264,
            "upperBound": 0.19628649538181364,
            "value": 0.17008952080042128
           }
          },
          "fairness_indicators_metrics/recall@0.4675": {
           "boundedValue": {
            "lowerBound": 0.128219431669785,
            "upperBound": 0.17510220313299696,
            "value": 0.15165876777251186
           }
          },
          "fairness_indicators_metrics/recall@0.4750": {
           "boundedValue": {
            "lowerBound": 0.1223422286652586,
            "upperBound": 0.16728677483097007,
            "value": 0.14481305950500264
           }
          },
          "fairness_indicators_metrics/recall@0.4875": {
           "boundedValue": {
            "lowerBound": 0.1131580537632794,
            "upperBound": 0.15435263485702272,
            "value": 0.13375460768825698
           }
          },
          "fairness_indicators_metrics/recall@0.5000": {
           "boundedValue": {
            "lowerBound": 0.10635389466614659,
            "upperBound": 0.144308404191187,
            "value": 0.1253291205897841
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9391576468289706,
            "upperBound": 0.951005046979486,
            "value": 0.9450818418234664
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9436534235600423,
            "upperBound": 0.9550759462729,
            "value": 0.9493651522104941
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.950690431001714,
            "upperBound": 0.9602772154162008,
            "value": 0.9554841670491051
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9543376101069396,
            "upperBound": 0.9642787819520595,
            "value": 0.959308551323237
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9592498250670942,
            "upperBound": 0.9679333801779535,
            "value": 0.9635918617102647
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9648338415821718,
            "upperBound": 0.9724455832049193,
            "value": 0.9686400489521187
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9676082017073188,
            "upperBound": 0.9748726855248052,
            "value": 0.9712406302585284
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9707248864165382,
            "upperBound": 0.9769572803516821,
            "value": 0.9738412115649381
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9724407482767048,
            "upperBound": 0.9789127792962362,
            "value": 0.9756769160165213
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.20536388557193178,
            "upperBound": 0.2527755880575402,
            "value": 0.22906793048973143
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.18951487450786023,
            "upperBound": 0.2338684891884559,
            "value": 0.21169036334913113
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.17381728472039834,
            "upperBound": 0.2221829676684596,
            "value": 0.1979989468141127
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.15886601681544701,
            "upperBound": 0.20975034335766077,
            "value": 0.18430753027909427
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.14389431206930264,
            "upperBound": 0.19628649538181364,
            "value": 0.17008952080042128
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.128219431669785,
            "upperBound": 0.17510220313299696,
            "value": 0.15165876777251186
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.1223422286652586,
            "upperBound": 0.16728677483097007,
            "value": 0.14481305950500264
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.1131580537632794,
            "upperBound": 0.15435263485702272,
            "value": 0.13375460768825698
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.10635389466614659,
            "upperBound": 0.144308404191187,
            "value": 0.1253291205897841
           }
          }
         },
         "slice": "religion:muslim",
         "sliceValue": "muslim"
        },
        {
         "metrics": {
          "accuracy": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "auc": {
           "boundedValue": {
            "lowerBound": 0.8002613266080761,
            "upperBound": 0.8289154692181747,
            "value": 0.8145889771490287
           }
          },
          "example_count": {
           "doubleValue": 16172.0
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.4783807161890599,
            "upperBound": 0.5404898405117385,
            "value": 0.5094339622641509
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.46392625679428734,
            "upperBound": 0.5342191565518196,
            "value": 0.49906542056074765
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.45888583337217115,
            "upperBound": 0.534912010078889,
            "value": 0.4968944099378882
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.448697957691406,
            "upperBound": 0.5266983880467367,
            "value": 0.48769574944071586
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.42022232216903144,
            "upperBound": 0.5036477122066955,
            "value": 0.4619289340101523
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.40339243799259245,
            "upperBound": 0.4907484707473793,
            "value": 0.4470588235294118
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.3930988106468059,
            "upperBound": 0.47027827186124266,
            "value": 0.43167701863354035
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.3767324676082144,
            "upperBound": 0.4554607732860107,
            "value": 0.4160839160839161
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.3659046426715885,
            "upperBound": 0.4603719493338666,
            "value": 0.41312741312741313
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.7858436245597641,
            "upperBound": 0.8239774554439826,
            "value": 0.8049113233287858
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.7962898237198227,
            "upperBound": 0.8380890469205288,
            "value": 0.8171896316507503
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.8143695308253989,
            "upperBound": 0.8541157955223426,
            "value": 0.8342428376534788
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.8248732404250635,
            "upperBound": 0.862712559094846,
            "value": 0.8437926330150068
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.8365763464490261,
            "upperBound": 0.8742018725109748,
            "value": 0.8553888130968622
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.8543074150453464,
            "upperBound": 0.8892142112966895,
            "value": 0.8717598908594816
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.8585584853489555,
            "upperBound": 0.8917839906905182,
            "value": 0.8751705320600273
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.8688092843646242,
            "upperBound": 0.9033603391225038,
            "value": 0.8860845839017736
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.8787174703985396,
            "upperBound": 0.9139161458306415,
            "value": 0.8963165075034106
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.0720218772939201,
            "upperBound": 0.07936760274943916,
            "value": 0.07569439989736353
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.07298096201188209,
            "upperBound": 0.08024598301461353,
            "value": 0.07661316109228113
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.07433567678587866,
            "upperBound": 0.08157037252078722,
            "value": 0.07795270571738161
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.07509936967945277,
            "upperBound": 0.08223040786947111,
            "value": 0.07866454689984102
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.07595099493210293,
            "upperBound": 0.0830051379518698,
            "value": 0.07947775383445303
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.07738987321690069,
            "upperBound": 0.08405587633977535,
            "value": 0.08072258716523496
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.07760793333599558,
            "upperBound": 0.08428538127727497,
            "value": 0.08094637223974764
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.078318096095139,
            "upperBound": 0.0852226786224697,
            "value": 0.08177011204834446
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.07911071575399456,
            "upperBound": 0.0860378153841922,
            "value": 0.0825739961038145
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.018253246051958077,
            "upperBound": 0.02213860067659027,
            "value": 0.020195838433292534
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.016367716899530992,
            "upperBound": 0.019944170925041273,
            "value": 0.018155854753161976
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.014415152776088347,
            "upperBound": 0.018224868678681463,
            "value": 0.016319869441044473
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.01294388745303383,
            "upperBound": 0.016704094568845418,
            "value": 0.014823881408948728
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.01077375961858758,
            "upperBound": 0.013978120841444583,
            "value": 0.012375900992792057
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.00901186022551594,
            "upperBound": 0.011660017277320005,
            "value": 0.0103359173126615
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.008285300351691222,
            "upperBound": 0.010618571137332332,
            "value": 0.009451924384604923
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.006927412112915863,
            "upperBound": 0.009256523757804272,
            "value": 0.008091935264517883
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.006015225863718591,
            "upperBound": 0.008536698357961293,
            "value": 0.00727594179246566
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9610864568069181,
            "upperBound": 0.966813204058324,
            "value": 0.9639500371011624
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9640248484983238,
            "upperBound": 0.9698109346768977,
            "value": 0.9669181301014098
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9672195051458935,
            "upperBound": 0.9730470536568706,
            "value": 0.9701335641850112
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9694963779737423,
            "upperBound": 0.9752224232522245,
            "value": 0.9723596339351966
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9730032627993259,
            "upperBound": 0.9782701458839325,
            "value": 0.9756369032896364
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9766074259206605,
            "upperBound": 0.981344148675539,
            "value": 0.9789760079149147
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9777854241794984,
            "upperBound": 0.982392235840195,
            "value": 0.9800890427900074
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9799215907540186,
            "upperBound": 0.9847081613382409,
            "value": 0.9823151125401929
           }
          },
          "fairness_indicators_metrics/negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9815506176432959,
            "upperBound": 0.9864182515633683,
            "value": 0.9839846648528321
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.03318679594167618,
            "upperBound": 0.03891354319308215,
            "value": 0.036049962898837495
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.030189065323102164,
            "upperBound": 0.03597515150167609,
            "value": 0.033081869898590154
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.026952946343129475,
            "upperBound": 0.03278049485410634,
            "value": 0.029866435814988868
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.02477757674777498,
            "upperBound": 0.03050362202625759,
            "value": 0.027640366064803364
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.0217298541160676,
            "upperBound": 0.02699673720067404,
            "value": 0.02436309671036359
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.01865585132446091,
            "upperBound": 0.023392574079339465,
            "value": 0.021023992085085333
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.017607764159804796,
            "upperBound": 0.022214575820501434,
            "value": 0.01991095720999258
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.015291838661759174,
            "upperBound": 0.02007840924598123,
            "value": 0.017684887459807074
           }
          },
          "fairness_indicators_metrics/positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.013581748436631964,
            "upperBound": 0.018449382356703993,
            "value": 0.016015335147167943
           }
          },
          "fairness_indicators_metrics/precision@0.4000": {
           "boundedValue": {
            "lowerBound": 0.45951015948826146,
            "upperBound": 0.5216192838109401,
            "value": 0.49056603773584906
           }
          },
          "fairness_indicators_metrics/precision@0.4125": {
           "boundedValue": {
            "lowerBound": 0.46578084344818027,
            "upperBound": 0.5360737432057128,
            "value": 0.5009345794392523
           }
          },
          "fairness_indicators_metrics/precision@0.4250": {
           "boundedValue": {
            "lowerBound": 0.46508798992111094,
            "upperBound": 0.5411141666278286,
            "value": 0.5031055900621118
           }
          },
          "fairness_indicators_metrics/precision@0.4375": {
           "boundedValue": {
            "lowerBound": 0.4733016119532633,
            "upperBound": 0.551302042308594,
            "value": 0.5123042505592841
           }
          },
          "fairness_indicators_metrics/precision@0.4500": {
           "boundedValue": {
            "lowerBound": 0.49635228779330465,
            "upperBound": 0.5797776778309685,
            "value": 0.5380710659898477
           }
          },
          "fairness_indicators_metrics/precision@0.4675": {
           "boundedValue": {
            "lowerBound": 0.509251529252621,
            "upperBound": 0.5966075620074072,
            "value": 0.5529411764705883
           }
          },
          "fairness_indicators_metrics/precision@0.4750": {
           "boundedValue": {
            "lowerBound": 0.5297217281387573,
            "upperBound": 0.6069011893531943,
            "value": 0.5683229813664596
           }
          },
          "fairness_indicators_metrics/precision@0.4875": {
           "boundedValue": {
            "lowerBound": 0.5445392267139889,
            "upperBound": 0.6232675323917859,
            "value": 0.583916083916084
           }
          },
          "fairness_indicators_metrics/precision@0.5000": {
           "boundedValue": {
            "lowerBound": 0.5396280506661336,
            "upperBound": 0.6340953573284115,
            "value": 0.5868725868725869
           }
          },
          "fairness_indicators_metrics/recall@0.4000": {
           "boundedValue": {
            "lowerBound": 0.1760225445560175,
            "upperBound": 0.2141563754402359,
            "value": 0.19508867667121418
           }
          },
          "fairness_indicators_metrics/recall@0.4125": {
           "boundedValue": {
            "lowerBound": 0.16191095307947104,
            "upperBound": 0.2037101762801771,
            "value": 0.18281036834924966
           }
          },
          "fairness_indicators_metrics/recall@0.4250": {
           "boundedValue": {
            "lowerBound": 0.1458842044776573,
            "upperBound": 0.1856304691746011,
            "value": 0.16575716234652116
           }
          },
          "fairness_indicators_metrics/recall@0.4375": {
           "boundedValue": {
            "lowerBound": 0.13728744090515432,
            "upperBound": 0.17512675957493667,
            "value": 0.15620736698499318
           }
          },
          "fairness_indicators_metrics/recall@0.4500": {
           "boundedValue": {
            "lowerBound": 0.12579812748902514,
            "upperBound": 0.16342365355097377,
            "value": 0.1446111869031378
           }
          },
          "fairness_indicators_metrics/recall@0.4675": {
           "boundedValue": {
            "lowerBound": 0.11078578870331053,
            "upperBound": 0.1456925849546539,
            "value": 0.12824010914051842
           }
          },
          "fairness_indicators_metrics/recall@0.4750": {
           "boundedValue": {
            "lowerBound": 0.10821600930948194,
            "upperBound": 0.14144151465104438,
            "value": 0.12482946793997271
           }
          },
          "fairness_indicators_metrics/recall@0.4875": {
           "boundedValue": {
            "lowerBound": 0.09663966087749624,
            "upperBound": 0.13119071563537535,
            "value": 0.11391541609822646
           }
          },
          "fairness_indicators_metrics/recall@0.5000": {
           "boundedValue": {
            "lowerBound": 0.08608385416935849,
            "upperBound": 0.1212825296014603,
            "value": 0.10368349249658936
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9778613993234094,
            "upperBound": 0.9817467539480419,
            "value": 0.9798041615667075
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9800558290749589,
            "upperBound": 0.983632283100469,
            "value": 0.981844145246838
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9817751313213183,
            "upperBound": 0.9855848472239117,
            "value": 0.9836801305589555
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9832959054311544,
            "upperBound": 0.9870561125469665,
            "value": 0.9851761185910513
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9860218791585554,
            "upperBound": 0.9892262403814124,
            "value": 0.9876240990072079
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9883399827226799,
            "upperBound": 0.9909881397744841,
            "value": 0.9896640826873385
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9893814288626678,
            "upperBound": 0.9917146996483085,
            "value": 0.990548075615395
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9907434762421958,
            "upperBound": 0.9930725878870843,
            "value": 0.9919080647354821
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9914633016420388,
            "upperBound": 0.9939847741362816,
            "value": 0.9927240582075343
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.1760225445560175,
            "upperBound": 0.2141563754402359,
            "value": 0.19508867667121418
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.16191095307947104,
            "upperBound": 0.2037101762801771,
            "value": 0.18281036834924966
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.1458842044776573,
            "upperBound": 0.1856304691746011,
            "value": 0.16575716234652116
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.13728744090515432,
            "upperBound": 0.17512675957493667,
            "value": 0.15620736698499318
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.12579812748902514,
            "upperBound": 0.16342365355097377,
            "value": 0.1446111869031378
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.11078578870331053,
            "upperBound": 0.1456925849546539,
            "value": 0.12824010914051842
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.10821600930948194,
            "upperBound": 0.14144151465104438,
            "value": 0.12482946793997271
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.09663966087749624,
            "upperBound": 0.13119071563537535,
            "value": 0.11391541609822646
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.08608385416935849,
            "upperBound": 0.1212825296014603,
            "value": 0.10368349249658936
           }
          }
         },
         "slice": "religion:christian",
         "sliceValue": "christian"
        },
        {
         "metrics": {
          "accuracy": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "auc": {
           "boundedValue": {
            "lowerBound": 0.723580952287136,
            "upperBound": 0.7712838691831497,
            "value": 0.747431411686431
           }
          },
          "example_count": {
           "doubleValue": 3015.0
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.3987972632023776,
            "upperBound": 0.5364327047775099,
            "value": 0.4675925925925926
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.3774501561339672,
            "upperBound": 0.52417172129096,
            "value": 0.45077720207253885
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.3539844547284153,
            "upperBound": 0.503244251568267,
            "value": 0.42857142857142855
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.3368683705306989,
            "upperBound": 0.4808317581443297,
            "value": 0.4088050314465409
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.3256667756808867,
            "upperBound": 0.460637047413825,
            "value": 0.3931034482758621
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.3023598574792469,
            "upperBound": 0.47899022308571926,
            "value": 0.390625
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.29541383553607825,
            "upperBound": 0.4713986855060607,
            "value": 0.38333333333333336
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.27611926220034555,
            "upperBound": 0.4561964393350866,
            "value": 0.36607142857142855
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.257249046486836,
            "upperBound": 0.45453620904412173,
            "value": 0.3557692307692308
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.7367845655760584,
            "upperBound": 0.8122535733692849,
            "value": 0.7745098039215687
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.7542025258459397,
            "upperBound": 0.830128014374415,
            "value": 0.792156862745098
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.765120650181439,
            "upperBound": 0.8427427968292657,
            "value": 0.803921568627451
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.7788723102871971,
            "upperBound": 0.852520607988061,
            "value": 0.8156862745098039
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.791513064186816,
            "upperBound": 0.8634032446960229,
            "value": 0.8274509803921568
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.8101024439099057,
            "upperBound": 0.884030606682313,
            "value": 0.8470588235294118
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.817077757042845,
            "upperBound": 0.8927447337263993,
            "value": 0.8549019607843137
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.822529522455699,
            "upperBound": 0.8990516624553164,
            "value": 0.8607843137254902
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.8284222108390691,
            "upperBound": 0.9088394024023734,
            "value": 0.8686274509803922
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.1286192101764739,
            "upperBound": 0.15362745977126416,
            "value": 0.1411218292247231
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.13057870318912523,
            "upperBound": 0.15574562875567055,
            "value": 0.14316087880935507
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.13158663750188693,
            "upperBound": 0.15714878960758605,
            "value": 0.1443661971830986
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.13286469448051133,
            "upperBound": 0.15845507343109463,
            "value": 0.14565826330532214
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.13361602339298112,
            "upperBound": 0.16046405347295917,
            "value": 0.1470383275261324
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.1361312499309725,
            "upperBound": 0.16314449620125834,
            "value": 0.1496363006581226
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.1371227946343537,
            "upperBound": 0.16408952626648884,
            "value": 0.15060449050086355
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.13756263011958667,
            "upperBound": 0.16488570955859064,
            "value": 0.15122287289011369
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.13804868552371036,
            "upperBound": 0.1663163093868001,
            "value": 0.15218138096873926
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.03313103084886327,
            "upperBound": 0.04750863676022702,
            "value": 0.04031936127744511
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.02847205650545983,
            "upperBound": 0.040989633708866303,
            "value": 0.03473053892215569
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.024293661793870797,
            "upperBound": 0.035586064315158386,
            "value": 0.029940119760479042
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.020960537506234743,
            "upperBound": 0.030934996778486212,
            "value": 0.02594810379241517
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.018459509489249844,
            "upperBound": 0.02704870650393554,
            "value": 0.022754491017964073
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.015085334633290658,
            "upperBound": 0.024833235842339065,
            "value": 0.01996007984031936
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.013955869962845938,
            "upperBound": 0.02276927945469759,
            "value": 0.018363273453093812
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.012412594283336397,
            "upperBound": 0.02032035889321083,
            "value": 0.016367265469061875
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.011097553997479581,
            "upperBound": 0.01844240722999656,
            "value": 0.014770459081836327
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9188936201484533,
            "upperBound": 0.9378223530276447,
            "value": 0.9283582089552239
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9272477975085798,
            "upperBound": 0.9447250899557074,
            "value": 0.9359867330016584
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9332643594642465,
            "upperBound": 0.950650581855931,
            "value": 0.9419568822553898
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9389600712826441,
            "upperBound": 0.9555688707989703,
            "value": 0.9472636815920398
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9440670890365574,
            "upperBound": 0.9597492982151634,
            "value": 0.9519071310116086
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9498502793570888,
            "upperBound": 0.9652435248608638,
            "value": 0.9575456053067993
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9523568901735926,
            "upperBound": 0.9680438292517564,
            "value": 0.9601990049751243
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.955273677676467,
            "upperBound": 0.970433251939494,
            "value": 0.9628524046434495
           }
          },
          "fairness_indicators_metrics/negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9581156239105267,
            "upperBound": 0.9728972979746592,
            "value": 0.9655058043117745
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.06217764697235528,
            "upperBound": 0.08110637985154631,
            "value": 0.07164179104477612
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.05527491004429261,
            "upperBound": 0.07275220249142011,
            "value": 0.06401326699834163
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.04934941814406928,
            "upperBound": 0.06673564053575365,
            "value": 0.05804311774461028
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.04443112920102982,
            "upperBound": 0.06103992871735567,
            "value": 0.0527363184079602
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.04025070178483632,
            "upperBound": 0.05593291096344223,
            "value": 0.04809286898839138
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.03475647513913629,
            "upperBound": 0.050149720642911046,
            "value": 0.04245439469320066
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.031956170748243515,
            "upperBound": 0.04764310982640721,
            "value": 0.03980099502487562
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.029566748060505972,
            "upperBound": 0.04472632232353293,
            "value": 0.03714759535655058
           }
          },
          "fairness_indicators_metrics/positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.02710270202534105,
            "upperBound": 0.0418843760894729,
            "value": 0.03449419568822554
           }
          },
          "fairness_indicators_metrics/precision@0.4000": {
           "boundedValue": {
            "lowerBound": 0.4635672952224901,
            "upperBound": 0.6012027367976225,
            "value": 0.5324074074074074
           }
          },
          "fairness_indicators_metrics/precision@0.4125": {
           "boundedValue": {
            "lowerBound": 0.47582827870903976,
            "upperBound": 0.6225498438660328,
            "value": 0.5492227979274611
           }
          },
          "fairness_indicators_metrics/precision@0.4250": {
           "boundedValue": {
            "lowerBound": 0.496755748431733,
            "upperBound": 0.6460155452715849,
            "value": 0.5714285714285714
           }
          },
          "fairness_indicators_metrics/precision@0.4375": {
           "boundedValue": {
            "lowerBound": 0.5191682418556702,
            "upperBound": 0.6631316294693013,
            "value": 0.5911949685534591
           }
          },
          "fairness_indicators_metrics/precision@0.4500": {
           "boundedValue": {
            "lowerBound": 0.5393629525861751,
            "upperBound": 0.6743332243191132,
            "value": 0.6068965517241379
           }
          },
          "fairness_indicators_metrics/precision@0.4675": {
           "boundedValue": {
            "lowerBound": 0.5210097769142806,
            "upperBound": 0.6976401425207531,
            "value": 0.609375
           }
          },
          "fairness_indicators_metrics/precision@0.4750": {
           "boundedValue": {
            "lowerBound": 0.5286013144939397,
            "upperBound": 0.7045861644639217,
            "value": 0.6166666666666667
           }
          },
          "fairness_indicators_metrics/precision@0.4875": {
           "boundedValue": {
            "lowerBound": 0.5438035606649139,
            "upperBound": 0.7238807377996543,
            "value": 0.6339285714285714
           }
          },
          "fairness_indicators_metrics/precision@0.5000": {
           "boundedValue": {
            "lowerBound": 0.5454637909558783,
            "upperBound": 0.7427509535131639,
            "value": 0.6442307692307693
           }
          },
          "fairness_indicators_metrics/recall@0.4000": {
           "boundedValue": {
            "lowerBound": 0.1877464266307152,
            "upperBound": 0.2632154344239416,
            "value": 0.22549019607843138
           }
          },
          "fairness_indicators_metrics/recall@0.4125": {
           "boundedValue": {
            "lowerBound": 0.1698719856255852,
            "upperBound": 0.2457974741540604,
            "value": 0.20784313725490197
           }
          },
          "fairness_indicators_metrics/recall@0.4250": {
           "boundedValue": {
            "lowerBound": 0.15725720317073447,
            "upperBound": 0.23487934981856082,
            "value": 0.19607843137254902
           }
          },
          "fairness_indicators_metrics/recall@0.4375": {
           "boundedValue": {
            "lowerBound": 0.1474793920119389,
            "upperBound": 0.22112768971280286,
            "value": 0.1843137254901961
           }
          },
          "fairness_indicators_metrics/recall@0.4500": {
           "boundedValue": {
            "lowerBound": 0.13659675530397672,
            "upperBound": 0.20848693581318392,
            "value": 0.17254901960784313
           }
          },
          "fairness_indicators_metrics/recall@0.4675": {
           "boundedValue": {
            "lowerBound": 0.11596939331768712,
            "upperBound": 0.18989755609009423,
            "value": 0.15294117647058825
           }
          },
          "fairness_indicators_metrics/recall@0.4750": {
           "boundedValue": {
            "lowerBound": 0.10725526627360041,
            "upperBound": 0.1829222429571549,
            "value": 0.1450980392156863
           }
          },
          "fairness_indicators_metrics/recall@0.4875": {
           "boundedValue": {
            "lowerBound": 0.10094833754468402,
            "upperBound": 0.17747047754430062,
            "value": 0.1392156862745098
           }
          },
          "fairness_indicators_metrics/recall@0.5000": {
           "boundedValue": {
            "lowerBound": 0.09116059759762651,
            "upperBound": 0.17157778916093103,
            "value": 0.13137254901960785
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9524913632397729,
            "upperBound": 0.966868969151137,
            "value": 0.9596806387225549
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9590103662911336,
            "upperBound": 0.9715279434945405,
            "value": 0.9652694610778443
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9644139356848417,
            "upperBound": 0.9757063382061294,
            "value": 0.9700598802395209
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9690650032215137,
            "upperBound": 0.9790394624937654,
            "value": 0.9740518962075848
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9729512934960646,
            "upperBound": 0.98154049051075,
            "value": 0.9772455089820359
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.975166764157661,
            "upperBound": 0.9849146653667095,
            "value": 0.9800399201596807
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9772307205453026,
            "upperBound": 0.9860441300371541,
            "value": 0.9816367265469061
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9796796411067891,
            "upperBound": 0.9875874057166637,
            "value": 0.9836327345309381
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9815575927700034,
            "upperBound": 0.9889024460025205,
            "value": 0.9852295409181637
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.1877464266307152,
            "upperBound": 0.2632154344239416,
            "value": 0.22549019607843138
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.1698719856255852,
            "upperBound": 0.2457974741540604,
            "value": 0.20784313725490197
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.15725720317073447,
            "upperBound": 0.23487934981856082,
            "value": 0.19607843137254902
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.1474793920119389,
            "upperBound": 0.22112768971280286,
            "value": 0.1843137254901961
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.13659675530397672,
            "upperBound": 0.20848693581318392,
            "value": 0.17254901960784313
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.11596939331768712,
            "upperBound": 0.18989755609009423,
            "value": 0.15294117647058825
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.10725526627360041,
            "upperBound": 0.1829222429571549,
            "value": 0.1450980392156863
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.10094833754468402,
            "upperBound": 0.17747047754430062,
            "value": 0.1392156862745098
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.09116059759762651,
            "upperBound": 0.17157778916093103,
            "value": 0.13137254901960785
           }
          }
         },
         "slice": "religion:jewish",
         "sliceValue": "jewish"
        },
        {
         "metrics": {
          "accuracy": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "auc": {
           "boundedValue": {
            "lowerBound": 0.7285679101553432,
            "upperBound": 0.9087725211892049,
            "value": 0.8187695924764891
           }
          },
          "example_count": {
           "doubleValue": 138.0
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4000": {
           "boundedValue": {
            "lowerBound": -0.2561640169782696,
            "upperBound": 0.6561640169782698,
            "value": 0.2
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4125": {
           "boundedValue": {
            "lowerBound": -0.2561640169782696,
            "upperBound": 0.6561640169782698,
            "value": 0.2
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.6908995446638296,
            "upperBound": 0.9461259736774782,
            "value": 0.8181818181818182
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.6908995446638296,
            "upperBound": 0.9461259736774782,
            "value": 0.8181818181818182
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.6908995446638296,
            "upperBound": 0.9461259736774782,
            "value": 0.8181818181818182
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.7287390823737594,
            "upperBound": 0.9989642669085372,
            "value": 0.8636363636363636
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.7287390823737594,
            "upperBound": 0.9989642669085372,
            "value": 0.8636363636363636
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.7826583452208713,
            "upperBound": 1.035722835002414,
            "value": 0.9090909090909091
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.7826583452208713,
            "upperBound": 1.035722835002414,
            "value": 0.9090909090909091
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.7826583452208713,
            "upperBound": 1.035722835002414,
            "value": 0.9090909090909091
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.7826583452208713,
            "upperBound": 1.035722835002414,
            "value": 0.9090909090909091
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.07754094813636753,
            "upperBound": 0.1932116980003497,
            "value": 0.13533834586466165
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.07754094813636753,
            "upperBound": 0.1932116980003497,
            "value": 0.13533834586466165
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.07720123341813795,
            "upperBound": 0.19152682321218067,
            "value": 0.13432835820895522
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.07778607846313601,
            "upperBound": 0.20376242684189694,
            "value": 0.14074074074074075
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.07778607846313601,
            "upperBound": 0.20376242684189694,
            "value": 0.14074074074074075
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.07993441009790565,
            "upperBound": 0.2142376470767529,
            "value": 0.14705882352941177
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.07993441009790565,
            "upperBound": 0.2142376470767529,
            "value": 0.14705882352941177
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.07993441009790565,
            "upperBound": 0.2142376470767529,
            "value": 0.14705882352941177
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.07993441009790565,
            "upperBound": 0.2142376470767529,
            "value": 0.14705882352941177
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": -0.00953584614767224,
            "upperBound": 0.026793613491701226,
            "value": 0.008620689655172414
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": -0.00953584614767224,
            "upperBound": 0.026793613491701226,
            "value": 0.008620689655172414
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9350759824292768,
            "upperBound": 0.9924785641477506,
            "value": 0.9637681159420289
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9350759824292768,
            "upperBound": 0.9924785641477506,
            "value": 0.9637681159420289
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9446794451941365,
            "upperBound": 0.9973795722918006,
            "value": 0.9710144927536232
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9546147563703262,
            "upperBound": 1.0019429490034706,
            "value": 0.9782608695652174
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9546147563703262,
            "upperBound": 1.0019429490034706,
            "value": 0.9782608695652174
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9651821526328792,
            "upperBound": 1.0058624056089536,
            "value": 0.9855072463768116
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9651821526328792,
            "upperBound": 1.0058624056089536,
            "value": 0.9855072463768116
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9651821526328792,
            "upperBound": 1.0058624056089536,
            "value": 0.9855072463768116
           }
          },
          "fairness_indicators_metrics/negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9651821526328792,
            "upperBound": 1.0058624056089536,
            "value": 0.9855072463768116
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.007521435852249101,
            "upperBound": 0.06492401757072311,
            "value": 0.036231884057971016
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.007521435852249101,
            "upperBound": 0.06492401757072311,
            "value": 0.036231884057971016
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.0026204277081993757,
            "upperBound": 0.05532055480586326,
            "value": 0.028985507246376812
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": -0.001942949003470567,
            "upperBound": 0.04538524362967382,
            "value": 0.021739130434782608
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": -0.001942949003470567,
            "upperBound": 0.04538524362967382,
            "value": 0.021739130434782608
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": -0.005862405608953448,
            "upperBound": 0.03481784736712097,
            "value": 0.014492753623188406
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": -0.005862405608953448,
            "upperBound": 0.03481784736712097,
            "value": 0.014492753623188406
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": -0.005862405608953448,
            "upperBound": 0.03481784736712097,
            "value": 0.014492753623188406
           }
          },
          "fairness_indicators_metrics/positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": -0.005862405608953448,
            "upperBound": 0.03481784736712097,
            "value": 0.014492753623188406
           }
          },
          "fairness_indicators_metrics/precision@0.4000": {
           "boundedValue": {
            "lowerBound": 0.34383598302173046,
            "upperBound": 1.25616401697827,
            "value": 0.8
           }
          },
          "fairness_indicators_metrics/precision@0.4125": {
           "boundedValue": {
            "lowerBound": 0.34383598302173046,
            "upperBound": 1.25616401697827,
            "value": 0.8
           }
          },
          "fairness_indicators_metrics/precision@0.4250": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.4375": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.4500": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.4675": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.4750": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.4875": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.5000": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/recall@0.4000": {
           "boundedValue": {
            "lowerBound": 0.05387402632252164,
            "upperBound": 0.3091004553361706,
            "value": 0.18181818181818182
           }
          },
          "fairness_indicators_metrics/recall@0.4125": {
           "boundedValue": {
            "lowerBound": 0.05387402632252164,
            "upperBound": 0.3091004553361706,
            "value": 0.18181818181818182
           }
          },
          "fairness_indicators_metrics/recall@0.4250": {
           "boundedValue": {
            "lowerBound": 0.05387402632252164,
            "upperBound": 0.3091004553361706,
            "value": 0.18181818181818182
           }
          },
          "fairness_indicators_metrics/recall@0.4375": {
           "boundedValue": {
            "lowerBound": 0.0010357330914629037,
            "upperBound": 0.2712609176262404,
            "value": 0.13636363636363635
           }
          },
          "fairness_indicators_metrics/recall@0.4500": {
           "boundedValue": {
            "lowerBound": 0.0010357330914629037,
            "upperBound": 0.2712609176262404,
            "value": 0.13636363636363635
           }
          },
          "fairness_indicators_metrics/recall@0.4675": {
           "boundedValue": {
            "lowerBound": -0.03572283500241402,
            "upperBound": 0.21734165477912853,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/recall@0.4750": {
           "boundedValue": {
            "lowerBound": -0.03572283500241402,
            "upperBound": 0.21734165477912853,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/recall@0.4875": {
           "boundedValue": {
            "lowerBound": -0.03572283500241402,
            "upperBound": 0.21734165477912853,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/recall@0.5000": {
           "boundedValue": {
            "lowerBound": -0.03572283500241402,
            "upperBound": 0.21734165477912853,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9732063865082985,
            "upperBound": 1.009535846147672,
            "value": 0.9913793103448276
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9732063865082985,
            "upperBound": 1.009535846147672,
            "value": 0.9913793103448276
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.05387402632252164,
            "upperBound": 0.3091004553361706,
            "value": 0.18181818181818182
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.05387402632252164,
            "upperBound": 0.3091004553361706,
            "value": 0.18181818181818182
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.05387402632252164,
            "upperBound": 0.3091004553361706,
            "value": 0.18181818181818182
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.0010357330914629037,
            "upperBound": 0.2712609176262404,
            "value": 0.13636363636363635
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.0010357330914629037,
            "upperBound": 0.2712609176262404,
            "value": 0.13636363636363635
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": -0.03572283500241402,
            "upperBound": 0.21734165477912853,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": -0.03572283500241402,
            "upperBound": 0.21734165477912853,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": -0.03572283500241402,
            "upperBound": 0.21734165477912853,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": -0.03572283500241402,
            "upperBound": 0.21734165477912853,
            "value": 0.09090909090909091
           }
          }
         },
         "slice": "religion:other_religion",
         "sliceValue": "other_religion"
        },
        {
         "metrics": {
          "accuracy": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "auc": {
           "boundedValue": {
            "lowerBound": 0.7441683209035564,
            "upperBound": 0.8433629407345317,
            "value": 0.7937895437895437
           }
          },
          "example_count": {
           "doubleValue": 545.0
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.27348432533733014,
            "upperBound": 0.6550573922043875,
            "value": 0.4642857142857143
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.3391867566415423,
            "upperBound": 0.7007282631213034,
            "value": 0.52
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.291513419358009,
            "upperBound": 0.6648925659610196,
            "value": 0.4782608695652174
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.291513419358009,
            "upperBound": 0.6648925659610196,
            "value": 0.4782608695652174
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.23118462360023673,
            "upperBound": 0.6688858958356079,
            "value": 0.45
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.23118462360023673,
            "upperBound": 0.6688858958356079,
            "value": 0.45
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.23118462360023673,
            "upperBound": 0.6688858958356079,
            "value": 0.45
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.17039718018091393,
            "upperBound": 0.671725282839382,
            "value": 0.42105263157894735
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.18323324438190924,
            "upperBound": 0.758237343853385,
            "value": 0.47058823529411764
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.7235158137963938,
            "upperBound": 0.8868830982183832,
            "value": 0.8051948051948052
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.7712667168474338,
            "upperBound": 0.9170498658305725,
            "value": 0.8441558441558441
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.7712667168474338,
            "upperBound": 0.9170498658305725,
            "value": 0.8441558441558441
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.7712667168474338,
            "upperBound": 0.9170498658305725,
            "value": 0.8441558441558441
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.7799294238990924,
            "upperBound": 0.934340332498035,
            "value": 0.8571428571428571
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.7799294238990924,
            "upperBound": 0.934340332498035,
            "value": 0.8571428571428571
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.7799294238990924,
            "upperBound": 0.934340332498035,
            "value": 0.8571428571428571
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.7799294238990924,
            "upperBound": 0.934340332498035,
            "value": 0.8571428571428571
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.808812843635322,
            "upperBound": 0.9574945091211179,
            "value": 0.8831168831168831
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.08818643114499274,
            "upperBound": 0.15166901759902301,
            "value": 0.11992263056092843
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.09279592766531213,
            "upperBound": 0.1572126043590109,
            "value": 0.125
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.09239148726838364,
            "upperBound": 0.15665942935346966,
            "value": 0.12452107279693486
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.09239148726838364,
            "upperBound": 0.15665942935346966,
            "value": 0.12452107279693486
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.09335600871807179,
            "upperBound": 0.15807700943103611,
            "value": 0.12571428571428572
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.09335600871807179,
            "upperBound": 0.15807700943103611,
            "value": 0.12571428571428572
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.09335600871807179,
            "upperBound": 0.15807700943103611,
            "value": 0.12571428571428572
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.09313656300739558,
            "upperBound": 0.15781883388729068,
            "value": 0.12547528517110265
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.09753856476649318,
            "upperBound": 0.16004477240868814,
            "value": 0.12878787878787878
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.013168069572493106,
            "upperBound": 0.04238703355566385,
            "value": 0.027777777777777776
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.013168069572493106,
            "upperBound": 0.04238703355566385,
            "value": 0.027777777777777776
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.009950536417439928,
            "upperBound": 0.037058082872423896,
            "value": 0.023504273504273504
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.009950536417439928,
            "upperBound": 0.037058082872423896,
            "value": 0.023504273504273504
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.007371603078519315,
            "upperBound": 0.031087978746902244,
            "value": 0.019230769230769232
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.007371603078519315,
            "upperBound": 0.031087978746902244,
            "value": 0.019230769230769232
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.007371603078519315,
            "upperBound": 0.031087978746902244,
            "value": 0.019230769230769232
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.005241776826876697,
            "upperBound": 0.028945576462058066,
            "value": 0.017094017094017096
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.005241776826876697,
            "upperBound": 0.028945576462058066,
            "value": 0.017094017094017096
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9321065033146072,
            "upperBound": 0.9651476686163585,
            "value": 0.9486238532110092
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9368559000950745,
            "upperBound": 0.9714066460038917,
            "value": 0.9541284403669725
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9409141686883606,
            "upperBound": 0.974687686730459,
            "value": 0.9577981651376147
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9409141686883606,
            "upperBound": 0.974687686730459,
            "value": 0.9577981651376147
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9481845569175819,
            "upperBound": 0.9784238024807518,
            "value": 0.963302752293578
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9481845569175819,
            "upperBound": 0.9784238024807518,
            "value": 0.963302752293578
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9481845569175819,
            "upperBound": 0.9784238024807518,
            "value": 0.963302752293578
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9514959996203073,
            "upperBound": 0.9787806936877194,
            "value": 0.9651376146788991
           }
          },
          "fairness_indicators_metrics/negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9554555105489333,
            "upperBound": 0.982162668283918,
            "value": 0.9688073394495413
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.03485233138364163,
            "upperBound": 0.06789349668539302,
            "value": 0.05137614678899083
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.028593353996108663,
            "upperBound": 0.06314409990492531,
            "value": 0.045871559633027525
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.025312313269541014,
            "upperBound": 0.059085831311639456,
            "value": 0.04220183486238532
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.025312313269541014,
            "upperBound": 0.059085831311639456,
            "value": 0.04220183486238532
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.021576197519248306,
            "upperBound": 0.0518154430824181,
            "value": 0.03669724770642202
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.021576197519248306,
            "upperBound": 0.0518154430824181,
            "value": 0.03669724770642202
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.021576197519248306,
            "upperBound": 0.0518154430824181,
            "value": 0.03669724770642202
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.021219306312280386,
            "upperBound": 0.04850400037969277,
            "value": 0.03486238532110092
           }
          },
          "fairness_indicators_metrics/positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.01783733171608186,
            "upperBound": 0.04454448945106672,
            "value": 0.031192660550458717
           }
          },
          "fairness_indicators_metrics/precision@0.4000": {
           "boundedValue": {
            "lowerBound": 0.3449426077956125,
            "upperBound": 0.7265156746626699,
            "value": 0.5357142857142857
           }
          },
          "fairness_indicators_metrics/precision@0.4125": {
           "boundedValue": {
            "lowerBound": 0.29927173687869624,
            "upperBound": 0.6608132433584579,
            "value": 0.48
           }
          },
          "fairness_indicators_metrics/precision@0.4250": {
           "boundedValue": {
            "lowerBound": 0.33510743403898036,
            "upperBound": 0.7084865806419909,
            "value": 0.5217391304347826
           }
          },
          "fairness_indicators_metrics/precision@0.4375": {
           "boundedValue": {
            "lowerBound": 0.33510743403898036,
            "upperBound": 0.7084865806419909,
            "value": 0.5217391304347826
           }
          },
          "fairness_indicators_metrics/precision@0.4500": {
           "boundedValue": {
            "lowerBound": 0.33111410416439213,
            "upperBound": 0.7688153763997634,
            "value": 0.55
           }
          },
          "fairness_indicators_metrics/precision@0.4675": {
           "boundedValue": {
            "lowerBound": 0.33111410416439213,
            "upperBound": 0.7688153763997634,
            "value": 0.55
           }
          },
          "fairness_indicators_metrics/precision@0.4750": {
           "boundedValue": {
            "lowerBound": 0.33111410416439213,
            "upperBound": 0.7688153763997634,
            "value": 0.55
           }
          },
          "fairness_indicators_metrics/precision@0.4875": {
           "boundedValue": {
            "lowerBound": 0.3282747171606176,
            "upperBound": 0.8296028198190862,
            "value": 0.5789473684210527
           }
          },
          "fairness_indicators_metrics/precision@0.5000": {
           "boundedValue": {
            "lowerBound": 0.24176265614661502,
            "upperBound": 0.8167667556180909,
            "value": 0.5294117647058824
           }
          },
          "fairness_indicators_metrics/recall@0.4000": {
           "boundedValue": {
            "lowerBound": 0.11311690178161685,
            "upperBound": 0.27648418620360604,
            "value": 0.19480519480519481
           }
          },
          "fairness_indicators_metrics/recall@0.4125": {
           "boundedValue": {
            "lowerBound": 0.08295013416942738,
            "upperBound": 0.22873328315256652,
            "value": 0.15584415584415584
           }
          },
          "fairness_indicators_metrics/recall@0.4250": {
           "boundedValue": {
            "lowerBound": 0.08295013416942738,
            "upperBound": 0.22873328315256652,
            "value": 0.15584415584415584
           }
          },
          "fairness_indicators_metrics/recall@0.4375": {
           "boundedValue": {
            "lowerBound": 0.08295013416942738,
            "upperBound": 0.22873328315256652,
            "value": 0.15584415584415584
           }
          },
          "fairness_indicators_metrics/recall@0.4500": {
           "boundedValue": {
            "lowerBound": 0.06565966750196496,
            "upperBound": 0.22007057610090702,
            "value": 0.14285714285714285
           }
          },
          "fairness_indicators_metrics/recall@0.4675": {
           "boundedValue": {
            "lowerBound": 0.06565966750196496,
            "upperBound": 0.22007057610090702,
            "value": 0.14285714285714285
           }
          },
          "fairness_indicators_metrics/recall@0.4750": {
           "boundedValue": {
            "lowerBound": 0.06565966750196496,
            "upperBound": 0.22007057610090702,
            "value": 0.14285714285714285
           }
          },
          "fairness_indicators_metrics/recall@0.4875": {
           "boundedValue": {
            "lowerBound": 0.06565966750196496,
            "upperBound": 0.22007057610090702,
            "value": 0.14285714285714285
           }
          },
          "fairness_indicators_metrics/recall@0.5000": {
           "boundedValue": {
            "lowerBound": 0.04250549087888242,
            "upperBound": 0.19118715636467795,
            "value": 0.11688311688311688
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.957612966444336,
            "upperBound": 0.9868319304275068,
            "value": 0.9722222222222222
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.957612966444336,
            "upperBound": 0.9868319304275068,
            "value": 0.9722222222222222
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9629419171275759,
            "upperBound": 0.9900494635825603,
            "value": 0.9764957264957265
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9629419171275759,
            "upperBound": 0.9900494635825603,
            "value": 0.9764957264957265
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9689120212530977,
            "upperBound": 0.9926283969214811,
            "value": 0.9807692307692307
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9689120212530977,
            "upperBound": 0.9926283969214811,
            "value": 0.9807692307692307
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9689120212530977,
            "upperBound": 0.9926283969214811,
            "value": 0.9807692307692307
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9710544235379421,
            "upperBound": 0.9947582231731231,
            "value": 0.9829059829059829
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9710544235379421,
            "upperBound": 0.9947582231731231,
            "value": 0.9829059829059829
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.11311690178161685,
            "upperBound": 0.27648418620360604,
            "value": 0.19480519480519481
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.08295013416942738,
            "upperBound": 0.22873328315256652,
            "value": 0.15584415584415584
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.08295013416942738,
            "upperBound": 0.22873328315256652,
            "value": 0.15584415584415584
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.08295013416942738,
            "upperBound": 0.22873328315256652,
            "value": 0.15584415584415584
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.06565966750196496,
            "upperBound": 0.22007057610090702,
            "value": 0.14285714285714285
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.06565966750196496,
            "upperBound": 0.22007057610090702,
            "value": 0.14285714285714285
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.06565966750196496,
            "upperBound": 0.22007057610090702,
            "value": 0.14285714285714285
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.06565966750196496,
            "upperBound": 0.22007057610090702,
            "value": 0.14285714285714285
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.04250549087888242,
            "upperBound": 0.19118715636467795,
            "value": 0.11688311688311688
           }
          }
         },
         "slice": "religion:atheist",
         "sliceValue": "atheist"
        },
        {
         "metrics": {
          "accuracy": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "auc": {
           "boundedValue": {
            "lowerBound": 0.6143238184245766,
            "upperBound": 0.8375728604442824,
            "value": 0.7257575757575757
           }
          },
          "example_count": {
           "doubleValue": 218.0
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.39536639807160356,
            "upperBound": 0.9379669352617295,
            "value": 0.6666666666666666
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.2200406159989427,
            "upperBound": 0.8912689078105809,
            "value": 0.5555555555555556
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.194058156027422,
            "upperBound": 0.9492751773059112,
            "value": 0.5714285714285714
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.09199449965888834,
            "upperBound": 0.9080055003411116,
            "value": 0.5
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.09199449965888834,
            "upperBound": 0.9080055003411116,
            "value": 0.5
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4675": {
           "boundedValue": {
            "lowerBound": -0.042773901702433725,
            "upperBound": 0.8411072350357673,
            "value": 0.4
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4750": {
           "boundedValue": {
            "lowerBound": -0.042773901702433725,
            "upperBound": 0.8411072350357673,
            "value": 0.4
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4875": {
           "boundedValue": {
            "lowerBound": -0.042773901702433725,
            "upperBound": 0.8411072350357673,
            "value": 0.4
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.5000": {
           "boundedValue": {
            "lowerBound": -0.042773901702433725,
            "upperBound": 0.8411072350357673,
            "value": 0.4
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.6264716332369262,
            "upperBound": 0.9734333375233082,
            "value": 0.8
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.6264716332369262,
            "upperBound": 0.9734333375233082,
            "value": 0.8
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.6640101043881318,
            "upperBound": 1.035251591518301,
            "value": 0.85
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.6640101043881318,
            "upperBound": 1.035251591518301,
            "value": 0.85
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.6640101043881318,
            "upperBound": 1.035251591518301,
            "value": 0.85
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.6640101043881318,
            "upperBound": 1.035251591518301,
            "value": 0.85
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.6640101043881318,
            "upperBound": 1.035251591518301,
            "value": 0.85
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.6640101043881318,
            "upperBound": 1.035251591518301,
            "value": 0.85
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.6640101043881318,
            "upperBound": 1.035251591518301,
            "value": 0.85
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.039249386412145135,
            "upperBound": 0.1160445365704962,
            "value": 0.07766990291262135
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.03925067099139621,
            "upperBound": 0.11380789625830581,
            "value": 0.07655502392344497
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.03785413799467204,
            "upperBound": 0.12322045196050965,
            "value": 0.08056872037914692
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.03745357721140765,
            "upperBound": 0.12286427278101442,
            "value": 0.08018867924528301
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.03745357721140765,
            "upperBound": 0.12286427278101442,
            "value": 0.08018867924528301
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.0373977401731094,
            "upperBound": 0.12216697772357174,
            "value": 0.07981220657276995
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.0373977401731094,
            "upperBound": 0.12216697772357174,
            "value": 0.07981220657276995
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.0373977401731094,
            "upperBound": 0.12216697772357174,
            "value": 0.07981220657276995
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.0373977401731094,
            "upperBound": 0.12216697772357174,
            "value": 0.07981220657276995
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.013282103262478626,
            "upperBound": 0.06751172810471807,
            "value": 0.04040404040404041
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.0047398170621527555,
            "upperBound": 0.04575957884032676,
            "value": 0.025252525252525252
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.0014344266633226667,
            "upperBound": 0.03895642236946635,
            "value": 0.020202020202020204
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": -0.0019149092409657814,
            "upperBound": 0.03221143615513116,
            "value": 0.015151515151515152
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": -0.0019149092409657814,
            "upperBound": 0.03221143615513116,
            "value": 0.015151515151515152
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": -0.0045344214155947915,
            "upperBound": 0.024742533758027683,
            "value": 0.010101010101010102
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": -0.0045344214155947915,
            "upperBound": 0.024742533758027683,
            "value": 0.010101010101010102
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": -0.0045344214155947915,
            "upperBound": 0.024742533758027683,
            "value": 0.010101010101010102
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": -0.0045344214155947915,
            "upperBound": 0.024742533758027683,
            "value": 0.010101010101010102
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9130944959610575,
            "upperBound": 0.9768099856331637,
            "value": 0.944954128440367
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9322059184358903,
            "upperBound": 0.9852056278240212,
            "value": 0.9587155963302753
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9422466989271564,
            "upperBound": 0.9935146804496525,
            "value": 0.9678899082568807
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9474315956391165,
            "upperBound": 0.9975012279070691,
            "value": 0.9724770642201835
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9474315956391165,
            "upperBound": 0.9975012279070691,
            "value": 0.9724770642201835
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9525387291831472,
            "upperBound": 1.0015558309726527,
            "value": 0.9770642201834863
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9525387291831472,
            "upperBound": 1.0015558309726527,
            "value": 0.9770642201834863
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9525387291831472,
            "upperBound": 1.0015558309726527,
            "value": 0.9770642201834863
           }
          },
          "fairness_indicators_metrics/negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9525387291831472,
            "upperBound": 1.0015558309726527,
            "value": 0.9770642201834863
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.023190014366836147,
            "upperBound": 0.08690550403894304,
            "value": 0.05504587155963303
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.014794372175978544,
            "upperBound": 0.06779408156411029,
            "value": 0.04128440366972477
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.00648531955034759,
            "upperBound": 0.05775330107284346,
            "value": 0.03211009174311927
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.002498772092931152,
            "upperBound": 0.052568404360883315,
            "value": 0.027522935779816515
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.002498772092931152,
            "upperBound": 0.052568404360883315,
            "value": 0.027522935779816515
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": -0.0015558309726530217,
            "upperBound": 0.04746127081685309,
            "value": 0.022935779816513763
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": -0.0015558309726530217,
            "upperBound": 0.04746127081685309,
            "value": 0.022935779816513763
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": -0.0015558309726530217,
            "upperBound": 0.04746127081685309,
            "value": 0.022935779816513763
           }
          },
          "fairness_indicators_metrics/positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": -0.0015558309726530217,
            "upperBound": 0.04746127081685309,
            "value": 0.022935779816513763
           }
          },
          "fairness_indicators_metrics/precision@0.4000": {
           "boundedValue": {
            "lowerBound": 0.062033064738270305,
            "upperBound": 0.6046336019283962,
            "value": 0.3333333333333333
           }
          },
          "fairness_indicators_metrics/precision@0.4125": {
           "boundedValue": {
            "lowerBound": 0.10873109218941907,
            "upperBound": 0.7799593840010575,
            "value": 0.4444444444444444
           }
          },
          "fairness_indicators_metrics/precision@0.4250": {
           "boundedValue": {
            "lowerBound": 0.05072482269408879,
            "upperBound": 0.8059418439725781,
            "value": 0.42857142857142855
           }
          },
          "fairness_indicators_metrics/precision@0.4375": {
           "boundedValue": {
            "lowerBound": 0.09199449965888834,
            "upperBound": 0.9080055003411116,
            "value": 0.5
           }
          },
          "fairness_indicators_metrics/precision@0.4500": {
           "boundedValue": {
            "lowerBound": 0.09199449965888834,
            "upperBound": 0.9080055003411116,
            "value": 0.5
           }
          },
          "fairness_indicators_metrics/precision@0.4675": {
           "boundedValue": {
            "lowerBound": 0.1588927649642327,
            "upperBound": 1.0427739017024338,
            "value": 0.6
           }
          },
          "fairness_indicators_metrics/precision@0.4750": {
           "boundedValue": {
            "lowerBound": 0.1588927649642327,
            "upperBound": 1.0427739017024338,
            "value": 0.6
           }
          },
          "fairness_indicators_metrics/precision@0.4875": {
           "boundedValue": {
            "lowerBound": 0.1588927649642327,
            "upperBound": 1.0427739017024338,
            "value": 0.6
           }
          },
          "fairness_indicators_metrics/precision@0.5000": {
           "boundedValue": {
            "lowerBound": 0.1588927649642327,
            "upperBound": 1.0427739017024338,
            "value": 0.6
           }
          },
          "fairness_indicators_metrics/recall@0.4000": {
           "boundedValue": {
            "lowerBound": 0.026566662476692038,
            "upperBound": 0.3735283667630741,
            "value": 0.2
           }
          },
          "fairness_indicators_metrics/recall@0.4125": {
           "boundedValue": {
            "lowerBound": 0.026566662476692038,
            "upperBound": 0.3735283667630741,
            "value": 0.2
           }
          },
          "fairness_indicators_metrics/recall@0.4250": {
           "boundedValue": {
            "lowerBound": -0.03525159151830101,
            "upperBound": 0.3359898956118682,
            "value": 0.15
           }
          },
          "fairness_indicators_metrics/recall@0.4375": {
           "boundedValue": {
            "lowerBound": -0.03525159151830101,
            "upperBound": 0.3359898956118682,
            "value": 0.15
           }
          },
          "fairness_indicators_metrics/recall@0.4500": {
           "boundedValue": {
            "lowerBound": -0.03525159151830101,
            "upperBound": 0.3359898956118682,
            "value": 0.15
           }
          },
          "fairness_indicators_metrics/recall@0.4675": {
           "boundedValue": {
            "lowerBound": -0.03525159151830101,
            "upperBound": 0.3359898956118682,
            "value": 0.15
           }
          },
          "fairness_indicators_metrics/recall@0.4750": {
           "boundedValue": {
            "lowerBound": -0.03525159151830101,
            "upperBound": 0.3359898956118682,
            "value": 0.15
           }
          },
          "fairness_indicators_metrics/recall@0.4875": {
           "boundedValue": {
            "lowerBound": -0.03525159151830101,
            "upperBound": 0.3359898956118682,
            "value": 0.15
           }
          },
          "fairness_indicators_metrics/recall@0.5000": {
           "boundedValue": {
            "lowerBound": -0.03525159151830101,
            "upperBound": 0.3359898956118682,
            "value": 0.15
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9324882718952819,
            "upperBound": 0.9867178967375213,
            "value": 0.9595959595959596
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9542404211596731,
            "upperBound": 0.9952601829378471,
            "value": 0.9747474747474747
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.961043577630534,
            "upperBound": 0.9985655733366773,
            "value": 0.9797979797979798
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9677885638448692,
            "upperBound": 1.0019149092409658,
            "value": 0.9848484848484849
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9677885638448692,
            "upperBound": 1.0019149092409658,
            "value": 0.9848484848484849
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9752574662419724,
            "upperBound": 1.0045344214155947,
            "value": 0.98989898989899
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9752574662419724,
            "upperBound": 1.0045344214155947,
            "value": 0.98989898989899
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9752574662419724,
            "upperBound": 1.0045344214155947,
            "value": 0.98989898989899
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9752574662419724,
            "upperBound": 1.0045344214155947,
            "value": 0.98989898989899
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.026566662476692038,
            "upperBound": 0.3735283667630741,
            "value": 0.2
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.026566662476692038,
            "upperBound": 0.3735283667630741,
            "value": 0.2
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": -0.03525159151830101,
            "upperBound": 0.3359898956118682,
            "value": 0.15
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": -0.03525159151830101,
            "upperBound": 0.3359898956118682,
            "value": 0.15
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": -0.03525159151830101,
            "upperBound": 0.3359898956118682,
            "value": 0.15
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": -0.03525159151830101,
            "upperBound": 0.3359898956118682,
            "value": 0.15
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": -0.03525159151830101,
            "upperBound": 0.3359898956118682,
            "value": 0.15
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": -0.03525159151830101,
            "upperBound": 0.3359898956118682,
            "value": 0.15
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": -0.03525159151830101,
            "upperBound": 0.3359898956118682,
            "value": 0.15
           }
          }
         },
         "slice": "religion:hindu",
         "sliceValue": "hindu"
        },
        {
         "metrics": {
          "accuracy": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "auc": {
           "boundedValue": {
            "lowerBound": 0.7345257926336435,
            "upperBound": 0.9222467596147708,
            "value": 0.8283918788269417
           }
          },
          "example_count": {
           "doubleValue": 243.0
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.2714821791149228,
            "upperBound": 0.7942470916143479,
            "value": 0.5333333333333333
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.22185028995432465,
            "upperBound": 0.7007837426797081,
            "value": 0.46153846153846156
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.19812366871138615,
            "upperBound": 0.7091743110865936,
            "value": 0.45454545454545453
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.06969598176652592,
            "upperBound": 0.7274865579160139,
            "value": 0.4
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.11001827827483024,
            "upperBound": 0.7758944201378684,
            "value": 0.4444444444444444
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.08556417948052653,
            "upperBound": 0.6632453443289973,
            "value": 0.375
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.08556417948052653,
            "upperBound": 0.6632453443289973,
            "value": 0.375
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4875": {
           "boundedValue": {
            "lowerBound": -0.014533279718643932,
            "upperBound": 0.5831047082900724,
            "value": 0.2857142857142857
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.02415512280426818,
            "upperBound": 0.6391782105290651,
            "value": 0.3333333333333333
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.5622914008851065,
            "upperBound": 0.9544951344186703,
            "value": 0.7586206896551724
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.5622914008851065,
            "upperBound": 0.9544951344186703,
            "value": 0.7586206896551724
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.6348822105356009,
            "upperBound": 0.9509173768501933,
            "value": 0.7931034482758621
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.6348822105356009,
            "upperBound": 0.9509173768501933,
            "value": 0.7931034482758621
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.6715551017750003,
            "upperBound": 0.9831150875503616,
            "value": 0.8275862068965517
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.6715551017750003,
            "upperBound": 0.9831150875503616,
            "value": 0.8275862068965517
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.6715551017750003,
            "upperBound": 0.9831150875503616,
            "value": 0.8275862068965517
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.6715551017750003,
            "upperBound": 0.9831150875503616,
            "value": 0.8275862068965517
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.7187227564137592,
            "upperBound": 1.0049604849936205,
            "value": 0.8620689655172413
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.057188271063492606,
            "upperBound": 0.13572672272914715,
            "value": 0.09649122807017543
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.057005375128555906,
            "upperBound": 0.1342305076137319,
            "value": 0.09565217391304348
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.06228752784195435,
            "upperBound": 0.13592471393455383,
            "value": 0.09913793103448276
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.06194065717249306,
            "upperBound": 0.13542157874773786,
            "value": 0.09871244635193133
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.06372849186447743,
            "upperBound": 0.14133612008896454,
            "value": 0.10256410256410256
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.06332293538998968,
            "upperBound": 0.14087064956921685,
            "value": 0.10212765957446808
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.06332293538998968,
            "upperBound": 0.14087064956921685,
            "value": 0.10212765957446808
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.06284271872789399,
            "upperBound": 0.14048666975111723,
            "value": 0.1016949152542373
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.06688544982133787,
            "upperBound": 0.14402436512352573,
            "value": 0.10548523206751055
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.005171826668975678,
            "upperBound": 0.0695791000181886,
            "value": 0.037383177570093455
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.0029196707352063464,
            "upperBound": 0.053160804467967115,
            "value": 0.028037383177570093
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": -0.00040808525266645274,
            "upperBound": 0.047129040556607736,
            "value": 0.02336448598130841
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": -0.0040649206478243945,
            "upperBound": 0.04144063632671474,
            "value": 0.018691588785046728
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": -0.0040649206478243945,
            "upperBound": 0.04144063632671474,
            "value": 0.018691588785046728
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": -0.0019625898953578545,
            "upperBound": 0.030000529007060846,
            "value": 0.014018691588785047
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": -0.0019625898953578545,
            "upperBound": 0.030000529007060846,
            "value": 0.014018691588785047
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": -0.004012217301321992,
            "upperBound": 0.022697672161072338,
            "value": 0.009345794392523364
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": -0.004012217301321992,
            "upperBound": 0.022697672161072338,
            "value": 0.009345794392523364
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.8959780773258325,
            "upperBound": 0.9805680177300974,
            "value": 0.9382716049382716
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9078694076162348,
            "upperBound": 0.9851161208524848,
            "value": 0.9465020576131687
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9206004040541862,
            "upperBound": 0.9888588640029015,
            "value": 0.9547325102880658
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9269045658247366,
            "upperBound": 0.9907842018793167,
            "value": 0.9588477366255144
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9310397728543383,
            "upperBound": 0.994880393110692,
            "value": 0.9629629629629629
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9405241887846166,
            "upperBound": 0.9936196801739797,
            "value": 0.9670781893004116
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9405241887846166,
            "upperBound": 0.9936196801739797,
            "value": 0.9670781893004116
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9453515257088708,
            "upperBound": 0.9970311736920482,
            "value": 0.9711934156378601
           }
          },
          "fairness_indicators_metrics/negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9498914145930053,
            "upperBound": 1.0007207844548793,
            "value": 0.9753086419753086
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.01943198226990247,
            "upperBound": 0.10402192267416732,
            "value": 0.06172839506172839
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.014883879147515572,
            "upperBound": 0.09213059238376517,
            "value": 0.053497942386831275
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.011141135997098574,
            "upperBound": 0.07939959594581394,
            "value": 0.04526748971193416
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.009215798120683356,
            "upperBound": 0.07309543417526376,
            "value": 0.0411522633744856
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.00511960688930789,
            "upperBound": 0.06896022714566204,
            "value": 0.037037037037037035
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.006380319826020046,
            "upperBound": 0.05947581121538322,
            "value": 0.03292181069958848
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.006380319826020046,
            "upperBound": 0.05947581121538322,
            "value": 0.03292181069958848
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.002968826307951807,
            "upperBound": 0.054648474291129026,
            "value": 0.02880658436213992
           }
          },
          "fairness_indicators_metrics/positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": -0.0007207844548793099,
            "upperBound": 0.05010858540699475,
            "value": 0.024691358024691357
           }
          },
          "fairness_indicators_metrics/precision@0.4000": {
           "boundedValue": {
            "lowerBound": 0.20575290838565202,
            "upperBound": 0.7285178208850773,
            "value": 0.4666666666666667
           }
          },
          "fairness_indicators_metrics/precision@0.4125": {
           "boundedValue": {
            "lowerBound": 0.2992162573202918,
            "upperBound": 0.7781497100456755,
            "value": 0.5384615384615384
           }
          },
          "fairness_indicators_metrics/precision@0.4250": {
           "boundedValue": {
            "lowerBound": 0.2908256889134062,
            "upperBound": 0.8018763312886137,
            "value": 0.5454545454545454
           }
          },
          "fairness_indicators_metrics/precision@0.4375": {
           "boundedValue": {
            "lowerBound": 0.27251344208398603,
            "upperBound": 0.9303040182334736,
            "value": 0.6
           }
          },
          "fairness_indicators_metrics/precision@0.4500": {
           "boundedValue": {
            "lowerBound": 0.2241055798621317,
            "upperBound": 0.8899817217251698,
            "value": 0.5555555555555556
           }
          },
          "fairness_indicators_metrics/precision@0.4675": {
           "boundedValue": {
            "lowerBound": 0.33675465567100266,
            "upperBound": 0.9144358205194736,
            "value": 0.625
           }
          },
          "fairness_indicators_metrics/precision@0.4750": {
           "boundedValue": {
            "lowerBound": 0.33675465567100266,
            "upperBound": 0.9144358205194736,
            "value": 0.625
           }
          },
          "fairness_indicators_metrics/precision@0.4875": {
           "boundedValue": {
            "lowerBound": 0.416895291709927,
            "upperBound": 1.014533279718644,
            "value": 0.7142857142857143
           }
          },
          "fairness_indicators_metrics/precision@0.5000": {
           "boundedValue": {
            "lowerBound": 0.3608217894709348,
            "upperBound": 0.9758448771957317,
            "value": 0.6666666666666666
           }
          },
          "fairness_indicators_metrics/recall@0.4000": {
           "boundedValue": {
            "lowerBound": 0.04550486558132996,
            "upperBound": 0.43770859911489324,
            "value": 0.2413793103448276
           }
          },
          "fairness_indicators_metrics/recall@0.4125": {
           "boundedValue": {
            "lowerBound": 0.04550486558132996,
            "upperBound": 0.43770859911489324,
            "value": 0.2413793103448276
           }
          },
          "fairness_indicators_metrics/recall@0.4250": {
           "boundedValue": {
            "lowerBound": 0.04908262314980641,
            "upperBound": 0.3651177894643993,
            "value": 0.20689655172413793
           }
          },
          "fairness_indicators_metrics/recall@0.4375": {
           "boundedValue": {
            "lowerBound": 0.04908262314980641,
            "upperBound": 0.3651177894643993,
            "value": 0.20689655172413793
           }
          },
          "fairness_indicators_metrics/recall@0.4500": {
           "boundedValue": {
            "lowerBound": 0.016884912449638634,
            "upperBound": 0.3284448982249996,
            "value": 0.1724137931034483
           }
          },
          "fairness_indicators_metrics/recall@0.4675": {
           "boundedValue": {
            "lowerBound": 0.016884912449638634,
            "upperBound": 0.3284448982249996,
            "value": 0.1724137931034483
           }
          },
          "fairness_indicators_metrics/recall@0.4750": {
           "boundedValue": {
            "lowerBound": 0.016884912449638634,
            "upperBound": 0.3284448982249996,
            "value": 0.1724137931034483
           }
          },
          "fairness_indicators_metrics/recall@0.4875": {
           "boundedValue": {
            "lowerBound": 0.016884912449638634,
            "upperBound": 0.3284448982249996,
            "value": 0.1724137931034483
           }
          },
          "fairness_indicators_metrics/recall@0.5000": {
           "boundedValue": {
            "lowerBound": -0.004960484993620257,
            "upperBound": 0.28127724358624084,
            "value": 0.13793103448275862
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9304208999818111,
            "upperBound": 0.9948281733310242,
            "value": 0.9626168224299065
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.946839195532033,
            "upperBound": 0.9970803292647934,
            "value": 0.9719626168224299
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9528709594433926,
            "upperBound": 1.0004080852526664,
            "value": 0.9766355140186916
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9585593636732849,
            "upperBound": 1.0040649206478243,
            "value": 0.9813084112149533
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9585593636732849,
            "upperBound": 1.0040649206478243,
            "value": 0.9813084112149533
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9699994709929395,
            "upperBound": 1.0019625898953577,
            "value": 0.985981308411215
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9699994709929395,
            "upperBound": 1.0019625898953577,
            "value": 0.985981308411215
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9773023278389277,
            "upperBound": 1.0040122173013222,
            "value": 0.9906542056074766
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9773023278389277,
            "upperBound": 1.0040122173013222,
            "value": 0.9906542056074766
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.04550486558132996,
            "upperBound": 0.43770859911489324,
            "value": 0.2413793103448276
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.04550486558132996,
            "upperBound": 0.43770859911489324,
            "value": 0.2413793103448276
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.04908262314980641,
            "upperBound": 0.3651177894643993,
            "value": 0.20689655172413793
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.04908262314980641,
            "upperBound": 0.3651177894643993,
            "value": 0.20689655172413793
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.016884912449638634,
            "upperBound": 0.3284448982249996,
            "value": 0.1724137931034483
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.016884912449638634,
            "upperBound": 0.3284448982249996,
            "value": 0.1724137931034483
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.016884912449638634,
            "upperBound": 0.3284448982249996,
            "value": 0.1724137931034483
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.016884912449638634,
            "upperBound": 0.3284448982249996,
            "value": 0.1724137931034483
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": -0.004960484993620257,
            "upperBound": 0.28127724358624084,
            "value": 0.13793103448275862
           }
          }
         },
         "slice": "religion:buddhist",
         "sliceValue": "buddhist"
        }
       ],
       "slicingMetricsCompare": []
      }
     },
     "c5ba0d11de314572bc6b482487b5980a": {
      "model_module": "@jupyter-widgets/base",
      "model_module_version": "1.2.0",
      "model_name": "LayoutModel",
      "state": {
       "_model_module": "@jupyter-widgets/base",
       "_model_module_version": "1.2.0",
       "_model_name": "LayoutModel",
       "_view_count": null,
       "_view_module": "@jupyter-widgets/base",
       "_view_module_version": "1.2.0",
       "_view_name": "LayoutView",
       "align_content": null,
       "align_items": null,
       "align_self": null,
       "border": null,
       "bottom": null,
       "display": null,
       "flex": null,
       "flex_flow": null,
       "grid_area": null,
       "grid_auto_columns": null,
       "grid_auto_flow": null,
       "grid_auto_rows": null,
       "grid_column": null,
       "grid_gap": null,
       "grid_row": null,
       "grid_template_areas": null,
       "grid_template_columns": null,
       "grid_template_rows": null,
       "height": null,
       "justify_content": null,
       "justify_items": null,
       "left": null,
       "margin": null,
       "max_height": null,
       "max_width": null,
       "min_height": null,
       "min_width": null,
       "object_fit": null,
       "object_position": null,
       "order": null,
       "overflow": null,
       "overflow_x": null,
       "overflow_y": null,
       "padding": null,
       "right": null,
       "top": null,
       "visibility": null,
       "width": null
      }
     },
     "de0e9b637e034835a901de1caa1fcbed": {
      "model_module": "tensorflow_model_analysis",
      "model_module_version": "0.46.0",
      "model_name": "FairnessIndicatorModel",
      "state": {
       "_dom_classes": [],
       "_model_module": "tensorflow_model_analysis",
       "_model_module_version": "0.46.0",
       "_model_name": "FairnessIndicatorModel",
       "_view_count": null,
       "_view_module": "tensorflow_model_analysis",
       "_view_module_version": "0.46.0",
       "_view_name": "FairnessIndicatorView",
       "evalName": "",
       "evalNameCompare": "",
       "js_events": [],
       "layout": "IPY_MODEL_c5ba0d11de314572bc6b482487b5980a",
       "slicingMetrics": [
        {
         "metrics": {
          "accuracy": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "auc": {
           "boundedValue": {
            "lowerBound": 0.7797951604713586,
            "upperBound": 0.7841776661306442,
            "value": 0.7819864957775187
           }
          },
          "example_count": {
           "doubleValue": 721950.0
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.469204478219344,
            "upperBound": 0.4902122071143054,
            "value": 0.4797083682818246
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.46087203103036245,
            "upperBound": 0.4813806135956741,
            "value": 0.4711262798634812
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.4523040942513333,
            "upperBound": 0.47452529828670315,
            "value": 0.4634146341463415
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.44223719448521764,
            "upperBound": 0.46490629176362436,
            "value": 0.45357197491203916
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.43537868875193325,
            "upperBound": 0.45879218447550335,
            "value": 0.4470854555744199
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.42253930874982143,
            "upperBound": 0.44519968351765876,
            "value": 0.4338698390482855
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.416718014091682,
            "upperBound": 0.4405794115739155,
            "value": 0.4286491880613263
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.40743547840098326,
            "upperBound": 0.4326258531452144,
            "value": 0.4200310559006211
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.40179452919702724,
            "upperBound": 0.42849201527161523,
            "value": 0.4151436031331593
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.8579297130442171,
            "upperBound": 0.8635020192723147,
            "value": 0.8607157662014647
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.8633140451253758,
            "upperBound": 0.869033753200865,
            "value": 0.8661738289346415
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.8687461930596738,
            "upperBound": 0.8743795992627192,
            "value": 0.8715628022661324
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.873998203711224,
            "upperBound": 0.8792146478587922,
            "value": 0.8766063285891944
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.8792312112061028,
            "upperBound": 0.8845178116865055,
            "value": 0.8818743954677353
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.8857965827809283,
            "upperBound": 0.8906302603699957,
            "value": 0.8882133480724057
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.8887206491847511,
            "upperBound": 0.8937169432141977,
            "value": 0.8912187370457372
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.8944007157182627,
            "upperBound": 0.8991602460275137,
            "value": 0.8967804338814426
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9008127159001432,
            "upperBound": 0.9057370604681837,
            "value": 0.903274837639906
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.06999938566482607,
            "upperBound": 0.07107762565403843,
            "value": 0.0705385086863774
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.07033916778247597,
            "upperBound": 0.071462040367519,
            "value": 0.07090060794570903
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.07071559202242951,
            "upperBound": 0.07180825375489912,
            "value": 0.07126192641634138
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.0710613945435932,
            "upperBound": 0.07212866179075114,
            "value": 0.07159503213538052
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.07143268202183159,
            "upperBound": 0.07247477101028457,
            "value": 0.07195373044092218
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.07182691681443802,
            "upperBound": 0.0729238050794493,
            "value": 0.0723753655783527
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.07201323271852736,
            "upperBound": 0.0731437150796129,
            "value": 0.07257847852170476
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.07238369818041275,
            "upperBound": 0.07353153659940058,
            "value": 0.07295762218856004
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.07283176469242322,
            "upperBound": 0.07398977380977878,
            "value": 0.07341077381996841
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.010896487521765868,
            "upperBound": 0.01149627426771937,
            "value": 0.011196378607763827
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.010115337202414677,
            "upperBound": 0.010672132462561289,
            "value": 0.010393733039782908
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.00939640266459494,
            "upperBound": 0.00994540060188509,
            "value": 0.009670900258111539
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.008643444334215093,
            "upperBound": 0.00921655191391212,
            "value": 0.008929996656898385
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.008062494113737178,
            "upperBound": 0.008592780411503702,
            "value": 0.008327636005505576
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.00721290586767475,
            "upperBound": 0.007725639494657512,
            "value": 0.007469272077270824
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.006852027666840751,
            "upperBound": 0.00737874419276581,
            "value": 0.0071153851945775495
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.0062752476911990766,
            "upperBound": 0.006759837421153395,
            "value": 0.006517542248070187
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.005748357820687211,
            "upperBound": 0.0062235620331710035,
            "value": 0.005985958973216033
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.978238097535757,
            "upperBound": 0.9788254128617815,
            "value": 0.978531754276612
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9794051815753317,
            "upperBound": 0.9800102920088576,
            "value": 0.9797077359927973
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9805222883901044,
            "upperBound": 0.9810872441628105,
            "value": 0.9808047648729137
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.98158488450229,
            "upperBound": 0.9821965424008411,
            "value": 0.9818907126532308
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9825772407989113,
            "upperBound": 0.9831572302985261,
            "value": 0.9828672345730314
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9838731055057536,
            "upperBound": 0.9844571134907995,
            "value": 0.9841651083870074
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9844329544552144,
            "upperBound": 0.9850303070164665,
            "value": 0.9847316296142392
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9854774839781942,
            "upperBound": 0.9859776055960484,
            "value": 0.9857275434586883
           }
          },
          "fairness_indicators_metrics/negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9864818415732923,
            "upperBound": 0.9869927777899392,
            "value": 0.9867373086778862
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.021174587138218424,
            "upperBound": 0.021761902464242894,
            "value": 0.021468245723388048
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.019989707991142792,
            "upperBound": 0.020594818424668416,
            "value": 0.020292264007202715
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.018912755837189488,
            "upperBound": 0.019477711609895573,
            "value": 0.019195235127086364
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.01780345759915906,
            "upperBound": 0.01841511549771001,
            "value": 0.018109287346769167
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.016842769701474043,
            "upperBound": 0.017422759201088763,
            "value": 0.01713276542696863
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.015542886509200434,
            "upperBound": 0.016126894494246296,
            "value": 0.01583489161299259
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.014969692983533482,
            "upperBound": 0.015567045544785532,
            "value": 0.015268370385760786
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.014022394403951326,
            "upperBound": 0.014522516021805812,
            "value": 0.014272456541311726
           }
          },
          "fairness_indicators_metrics/positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.013007222210060693,
            "upperBound": 0.013518158426707692,
            "value": 0.01326269132211372
           }
          },
          "fairness_indicators_metrics/precision@0.4000": {
           "boundedValue": {
            "lowerBound": 0.5097877928856946,
            "upperBound": 0.5307955217806559,
            "value": 0.5202916317181754
           }
          },
          "fairness_indicators_metrics/precision@0.4125": {
           "boundedValue": {
            "lowerBound": 0.5186193864043258,
            "upperBound": 0.5391279689696378,
            "value": 0.5288737201365188
           }
          },
          "fairness_indicators_metrics/precision@0.4250": {
           "boundedValue": {
            "lowerBound": 0.525474701713297,
            "upperBound": 0.5476959057486668,
            "value": 0.5365853658536586
           }
          },
          "fairness_indicators_metrics/precision@0.4375": {
           "boundedValue": {
            "lowerBound": 0.5350937082363757,
            "upperBound": 0.5577628055147822,
            "value": 0.5464280250879608
           }
          },
          "fairness_indicators_metrics/precision@0.4500": {
           "boundedValue": {
            "lowerBound": 0.5412078155244968,
            "upperBound": 0.5646213112480667,
            "value": 0.5529145444255801
           }
          },
          "fairness_indicators_metrics/precision@0.4675": {
           "boundedValue": {
            "lowerBound": 0.5548003164823412,
            "upperBound": 0.5774606912501785,
            "value": 0.5661301609517145
           }
          },
          "fairness_indicators_metrics/precision@0.4750": {
           "boundedValue": {
            "lowerBound": 0.5594205884260846,
            "upperBound": 0.5832819859083178,
            "value": 0.5713508119386737
           }
          },
          "fairness_indicators_metrics/precision@0.4875": {
           "boundedValue": {
            "lowerBound": 0.5673741468547855,
            "upperBound": 0.592564521599017,
            "value": 0.5799689440993789
           }
          },
          "fairness_indicators_metrics/precision@0.5000": {
           "boundedValue": {
            "lowerBound": 0.5715079847283848,
            "upperBound": 0.5982054708029727,
            "value": 0.5848563968668408
           }
          },
          "fairness_indicators_metrics/recall@0.4000": {
           "boundedValue": {
            "lowerBound": 0.13649798072768524,
            "upperBound": 0.1420702869557828,
            "value": 0.1392842337985353
           }
          },
          "fairness_indicators_metrics/recall@0.4125": {
           "boundedValue": {
            "lowerBound": 0.1309662467991349,
            "upperBound": 0.136685954874624,
            "value": 0.13382617106535857
           }
          },
          "fairness_indicators_metrics/recall@0.4250": {
           "boundedValue": {
            "lowerBound": 0.12562040073728087,
            "upperBound": 0.13125380694032618,
            "value": 0.12843719773386764
           }
          },
          "fairness_indicators_metrics/recall@0.4375": {
           "boundedValue": {
            "lowerBound": 0.12078535214120772,
            "upperBound": 0.1260017962887755,
            "value": 0.12339367141080558
           }
          },
          "fairness_indicators_metrics/recall@0.4500": {
           "boundedValue": {
            "lowerBound": 0.11548218831349455,
            "upperBound": 0.12076878879389749,
            "value": 0.11812560453226476
           }
          },
          "fairness_indicators_metrics/recall@0.4675": {
           "boundedValue": {
            "lowerBound": 0.10936973963000388,
            "upperBound": 0.11420341721907179,
            "value": 0.11178665192759431
           }
          },
          "fairness_indicators_metrics/recall@0.4750": {
           "boundedValue": {
            "lowerBound": 0.10628305678580222,
            "upperBound": 0.11127935081524873,
            "value": 0.10878126295426281
           }
          },
          "fairness_indicators_metrics/recall@0.4875": {
           "boundedValue": {
            "lowerBound": 0.10083975397248619,
            "upperBound": 0.10559928428173719,
            "value": 0.10321956611855741
           }
          },
          "fairness_indicators_metrics/recall@0.5000": {
           "boundedValue": {
            "lowerBound": 0.09426293953181616,
            "upperBound": 0.09918728409985669,
            "value": 0.09672516236009396
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9885037257322807,
            "upperBound": 0.989103512478234,
            "value": 0.9888036213922362
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9893278675374388,
            "upperBound": 0.9898846627975855,
            "value": 0.9896062669602171
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.990054599398115,
            "upperBound": 0.9906035973354052,
            "value": 0.9903290997418884
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9907834480860876,
            "upperBound": 0.9913565556657847,
            "value": 0.9910700033431016
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9914072195884964,
            "upperBound": 0.991937505886263,
            "value": 0.9916723639944944
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9922743605053423,
            "upperBound": 0.9927870941323251,
            "value": 0.9925307279227292
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.992621255807234,
            "upperBound": 0.9931479723331591,
            "value": 0.9928846148054224
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9932401625788467,
            "upperBound": 0.9937247523088009,
            "value": 0.9934824577519298
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9937764379668292,
            "upperBound": 0.9942516421793124,
            "value": 0.9940140410267839
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.13649798072768524,
            "upperBound": 0.1420702869557828,
            "value": 0.1392842337985353
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.1309662467991349,
            "upperBound": 0.136685954874624,
            "value": 0.13382617106535857
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.12562040073728087,
            "upperBound": 0.13125380694032618,
            "value": 0.12843719773386764
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.12078535214120772,
            "upperBound": 0.1260017962887755,
            "value": 0.12339367141080558
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.11548218831349455,
            "upperBound": 0.12076878879389749,
            "value": 0.11812560453226476
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.10936973963000388,
            "upperBound": 0.11420341721907179,
            "value": 0.11178665192759431
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.10628305678580222,
            "upperBound": 0.11127935081524873,
            "value": 0.10878126295426281
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.10083975397248619,
            "upperBound": 0.10559928428173719,
            "value": 0.10321956611855741
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.09426293953181616,
            "upperBound": 0.09918728409985669,
            "value": 0.09672516236009396
           }
          }
         },
         "slice": "Overall",
         "sliceValue": "Overall"
        },
        {
         "metrics": {
          "accuracy": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "auc": {
           "boundedValue": {
            "lowerBound": 0.7526103353874609,
            "upperBound": 0.7790221153761607,
            "value": 0.7658157179490176
           }
          },
          "example_count": {
           "doubleValue": 16172.0
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.36581965644790865,
            "upperBound": 0.4892958875436613,
            "value": 0.427536231884058
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.3569614831502954,
            "upperBound": 0.47446359896778645,
            "value": 0.41568627450980394
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.328229189975096,
            "upperBound": 0.45106253539589913,
            "value": 0.38961038961038963
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.33522117500326387,
            "upperBound": 0.4590980358424439,
            "value": 0.39712918660287083
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.3354998697352214,
            "upperBound": 0.458346804344929,
            "value": 0.39690721649484534
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.3124270575485783,
            "upperBound": 0.4532843369698449,
            "value": 0.38285714285714284
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.3061231872111799,
            "upperBound": 0.45576419133573765,
            "value": 0.38095238095238093
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.28667522982681576,
            "upperBound": 0.45012394065121236,
            "value": 0.3684210526315789
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.2730550901314166,
            "upperBound": 0.44636881705785514,
            "value": 0.3597122302158273
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.8749764684382255,
            "upperBound": 0.909472266558293,
            "value": 0.8922237380627558
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.8823261443739079,
            "upperBound": 0.9143987846490669,
            "value": 0.8983628922237381
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.8877054325198835,
            "upperBound": 0.919933360467296,
            "value": 0.9038199181446112
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.899368511208557,
            "upperBound": 0.9287383620622485,
            "value": 0.9140518417462483
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9066950503194309,
            "upperBound": 0.9336905601588452,
            "value": 0.9201909959072305
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.913113347868541,
            "upperBound": 0.9395489265199842,
            "value": 0.9263301500682128
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9161922982944698,
            "upperBound": 0.9419269751092142,
            "value": 0.9290586630286494
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9213649586729523,
            "upperBound": 0.947672621948468,
            "value": 0.9345156889495225
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.924957472142162,
            "upperBound": 0.9536289520903954,
            "value": 0.9392905866302865
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.07629600348703965,
            "upperBound": 0.08827352197578649,
            "value": 0.08228485153497735
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.0766262067454812,
            "upperBound": 0.08885708420054841,
            "value": 0.08274172268643588
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.0769280320307762,
            "upperBound": 0.08930987548789804,
            "value": 0.08311900131735776
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.07792332998704592,
            "upperBound": 0.08996485306205713,
            "value": 0.08394412077930213
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.07839952198006768,
            "upperBound": 0.0904576320553369,
            "value": 0.08442858931030167
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.07879386064106093,
            "upperBound": 0.090987870179268,
            "value": 0.0848909170469463
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.07898184567644138,
            "upperBound": 0.09122552463033105,
            "value": 0.08510372406898276
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.07958822366625595,
            "upperBound": 0.09144791081236256,
            "value": 0.08551810237203496
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.07988768035590124,
            "upperBound": 0.09188292316683253,
            "value": 0.08588536144202583
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.006703281966192791,
            "upperBound": 0.009344683334355074,
            "value": 0.008023935808513532
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.006127195848503183,
            "upperBound": 0.008288807470243601,
            "value": 0.007207942336461309
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.005186638835221536,
            "upperBound": 0.007053362398633568,
            "value": 0.006119951040391677
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.0046604267573102185,
            "upperBound": 0.006627540829716503,
            "value": 0.005643954848361213
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.004186552120928038,
            "upperBound": 0.006285438812376538,
            "value": 0.005235958112335101
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.003441259971995898,
            "upperBound": 0.005670673815323977,
            "value": 0.004555963552291582
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.0032043890787065337,
            "upperBound": 0.005499595375880334,
            "value": 0.004351965184278526
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.0026054357299215136,
            "upperBound": 0.005010565268828492,
            "value": 0.00380796953624371
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.002287180016210257,
            "upperBound": 0.0045128658692408395,
            "value": 0.003399972800217598
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9808706129612795,
            "upperBound": 0.9849961651947597,
            "value": 0.9829334652485778
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9824680959602088,
            "upperBound": 0.9859957783494191,
            "value": 0.984232005936186
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9840960720043396,
            "upperBound": 0.9873359893207897,
            "value": 0.9857160524363097
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9853168946620134,
            "upperBound": 0.9888359750091786,
            "value": 0.9870764283947564
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.986202992808038,
            "upperBound": 0.9898049569719403,
            "value": 0.9880039574573337
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.987414714976693,
            "upperBound": 0.9909429665425866,
            "value": 0.9891788276032649
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9878772914662501,
            "upperBound": 0.9913460666201122,
            "value": 0.9896116744991343
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9886680972140104,
            "upperBound": 0.9925339647200974,
            "value": 0.9906010388325501
           }
          },
          "fairness_indicators_metrics/negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9894739133079704,
            "upperBound": 0.9933357885104968,
            "value": 0.9914048973534504
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.015003834805240193,
            "upperBound": 0.019129387038719973,
            "value": 0.01706653475142221
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.01400422165058071,
            "upperBound": 0.01753190403979121,
            "value": 0.015767994063814
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.012664010679210436,
            "upperBound": 0.015903927995660243,
            "value": 0.014283947563690329
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.011164024990821489,
            "upperBound": 0.014683105337986481,
            "value": 0.012923571605243631
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.0101950430280597,
            "upperBound": 0.013797007191962076,
            "value": 0.011996042542666337
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.009057033457413374,
            "upperBound": 0.012585285023307118,
            "value": 0.010821172396735098
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.00865393337988802,
            "upperBound": 0.012122708533749558,
            "value": 0.010388325500865693
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.0074660352799024565,
            "upperBound": 0.011331902785989814,
            "value": 0.009398961167449914
           }
          },
          "fairness_indicators_metrics/positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.006664211489503419,
            "upperBound": 0.010526086692029946,
            "value": 0.008595102646549592
           }
          },
          "fairness_indicators_metrics/precision@0.4000": {
           "boundedValue": {
            "lowerBound": 0.5107041124563386,
            "upperBound": 0.6341803435520913,
            "value": 0.572463768115942
           }
          },
          "fairness_indicators_metrics/precision@0.4125": {
           "boundedValue": {
            "lowerBound": 0.5255364010322136,
            "upperBound": 0.6430385168497047,
            "value": 0.5843137254901961
           }
          },
          "fairness_indicators_metrics/precision@0.4250": {
           "boundedValue": {
            "lowerBound": 0.5489374646041006,
            "upperBound": 0.6717708100249044,
            "value": 0.6103896103896104
           }
          },
          "fairness_indicators_metrics/precision@0.4375": {
           "boundedValue": {
            "lowerBound": 0.5409019641575561,
            "upperBound": 0.6647788249967361,
            "value": 0.6028708133971292
           }
          },
          "fairness_indicators_metrics/precision@0.4500": {
           "boundedValue": {
            "lowerBound": 0.5416531956550709,
            "upperBound": 0.6645001302647785,
            "value": 0.6030927835051546
           }
          },
          "fairness_indicators_metrics/precision@0.4675": {
           "boundedValue": {
            "lowerBound": 0.5467156630301552,
            "upperBound": 0.6875729424514218,
            "value": 0.6171428571428571
           }
          },
          "fairness_indicators_metrics/precision@0.4750": {
           "boundedValue": {
            "lowerBound": 0.5442358086642625,
            "upperBound": 0.69387681278882,
            "value": 0.6190476190476191
           }
          },
          "fairness_indicators_metrics/precision@0.4875": {
           "boundedValue": {
            "lowerBound": 0.5498760593487875,
            "upperBound": 0.7133247701731842,
            "value": 0.631578947368421
           }
          },
          "fairness_indicators_metrics/precision@0.5000": {
           "boundedValue": {
            "lowerBound": 0.553631182942145,
            "upperBound": 0.7269449098685836,
            "value": 0.6402877697841727
           }
          },
          "fairness_indicators_metrics/recall@0.4000": {
           "boundedValue": {
            "lowerBound": 0.09052773344170702,
            "upperBound": 0.12502353156177487,
            "value": 0.1077762619372442
           }
          },
          "fairness_indicators_metrics/recall@0.4125": {
           "boundedValue": {
            "lowerBound": 0.08560121535093328,
            "upperBound": 0.11767385562609219,
            "value": 0.10163710777626193
           }
          },
          "fairness_indicators_metrics/recall@0.4250": {
           "boundedValue": {
            "lowerBound": 0.08006663953270399,
            "upperBound": 0.11229456748011682,
            "value": 0.09618008185538882
           }
          },
          "fairness_indicators_metrics/recall@0.4375": {
           "boundedValue": {
            "lowerBound": 0.07126163793775167,
            "upperBound": 0.10063148879144329,
            "value": 0.08594815825375171
           }
          },
          "fairness_indicators_metrics/recall@0.4500": {
           "boundedValue": {
            "lowerBound": 0.06630943984115473,
            "upperBound": 0.09330494968056893,
            "value": 0.07980900409276943
           }
          },
          "fairness_indicators_metrics/recall@0.4675": {
           "boundedValue": {
            "lowerBound": 0.06045107348001564,
            "upperBound": 0.08688665213145899,
            "value": 0.07366984993178717
           }
          },
          "fairness_indicators_metrics/recall@0.4750": {
           "boundedValue": {
            "lowerBound": 0.05807302489078558,
            "upperBound": 0.08380770170552998,
            "value": 0.07094133697135062
           }
          },
          "fairness_indicators_metrics/recall@0.4875": {
           "boundedValue": {
            "lowerBound": 0.052327378051531544,
            "upperBound": 0.07863504132704753,
            "value": 0.06548431105047749
           }
          },
          "fairness_indicators_metrics/recall@0.5000": {
           "boundedValue": {
            "lowerBound": 0.046371047909604515,
            "upperBound": 0.07504252785783784,
            "value": 0.06070941336971351
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9906553166656451,
            "upperBound": 0.993296718033807,
            "value": 0.9919760641914864
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9917111925297564,
            "upperBound": 0.9938728041514964,
            "value": 0.9927920576635387
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9929466376013661,
            "upperBound": 0.9948133611647784,
            "value": 0.9938800489596084
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9933724591702835,
            "upperBound": 0.9953395732426897,
            "value": 0.9943560451516388
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9937145611876234,
            "upperBound": 0.995813447879072,
            "value": 0.9947640418876649
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.994329326184676,
            "upperBound": 0.9965587400280038,
            "value": 0.9954440364477084
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9945004046241196,
            "upperBound": 0.9967956109212933,
            "value": 0.9956480348157215
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9949894347311712,
            "upperBound": 0.9973945642700781,
            "value": 0.9961920304637563
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.995487134130759,
            "upperBound": 0.9977128199837897,
            "value": 0.9966000271997824
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.09052773344170702,
            "upperBound": 0.12502353156177487,
            "value": 0.1077762619372442
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.08560121535093328,
            "upperBound": 0.11767385562609219,
            "value": 0.10163710777626193
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.08006663953270399,
            "upperBound": 0.11229456748011682,
            "value": 0.09618008185538882
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.07126163793775167,
            "upperBound": 0.10063148879144329,
            "value": 0.08594815825375171
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.06630943984115473,
            "upperBound": 0.09330494968056893,
            "value": 0.07980900409276943
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.06045107348001564,
            "upperBound": 0.08688665213145899,
            "value": 0.07366984993178717
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.05807302489078558,
            "upperBound": 0.08380770170552998,
            "value": 0.07094133697135062
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.052327378051531544,
            "upperBound": 0.07863504132704753,
            "value": 0.06548431105047749
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.046371047909604515,
            "upperBound": 0.07504252785783784,
            "value": 0.06070941336971351
           }
          }
         },
         "slice": "religion:christian",
         "sliceValue": "christian"
        },
        {
         "metrics": {
          "accuracy": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "auc": {
           "boundedValue": {
            "lowerBound": 0.698991537626898,
            "upperBound": 0.7232982457035595,
            "value": 0.7111476189774205
           }
          },
          "example_count": {
           "doubleValue": 8436.0
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.3043797402715789,
            "upperBound": 0.4184836202293347,
            "value": 0.3614457831325301
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.2932766038525642,
            "upperBound": 0.411007465487603,
            "value": 0.3521739130434783
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.29028473378198144,
            "upperBound": 0.41606622537067256,
            "value": 0.3532110091743119
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.28473792414127397,
            "upperBound": 0.4204992993480404,
            "value": 0.3526570048309179
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.25880306797008357,
            "upperBound": 0.4112309593911583,
            "value": 0.33505154639175255
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.23473163649145298,
            "upperBound": 0.38740863355183697,
            "value": 0.3111111111111111
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.23302835110309478,
            "upperBound": 0.39781253656484133,
            "value": 0.31547619047619047
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.20792070351750652,
            "upperBound": 0.3794892398587673,
            "value": 0.29375
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.20002448217250396,
            "upperBound": 0.39298293841176296,
            "value": 0.296551724137931
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9044230223398905,
            "upperBound": 0.928121064745756,
            "value": 0.9162717219589257
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9107249230069621,
            "upperBound": 0.9323497588296081,
            "value": 0.9215376513954713
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9157225650507771,
            "upperBound": 0.9357783615735916,
            "value": 0.9257503949447078
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9194838301107863,
            "upperBound": 0.9393896683304349,
            "value": 0.9294365455502897
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9218964137349984,
            "upperBound": 0.9422429970857241,
            "value": 0.9320695102685624
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9251069760444783,
            "upperBound": 0.9442987806783569,
            "value": 0.9347024749868352
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9302154797470575,
            "upperBound": 0.9486687124686358,
            "value": 0.9394418114797262
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9311423444887145,
            "upperBound": 0.9498480160513308,
            "value": 0.9404949973670352
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.93627423338558,
            "upperBound": 0.9563001464075092,
            "value": 0.9462875197472354
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.2031817769895906,
            "upperBound": 0.22188374526831867,
            "value": 0.21253206302674973
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.2038351090670135,
            "upperBound": 0.222683333991034,
            "value": 0.21325859127467706
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.2045586376293381,
            "upperBound": 0.22328403528618865,
            "value": 0.21392066196154783
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.20524166583924358,
            "upperBound": 0.22373034271963455,
            "value": 0.21448535666545146
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.20549774280388847,
            "upperBound": 0.22401086978632037,
            "value": 0.21475370055811696
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.20574483827668363,
            "upperBound": 0.22424677823168734,
            "value": 0.21499515503875968
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.20660379842358154,
            "upperBound": 0.22494071854275635,
            "value": 0.21577164973391388
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.20668995884815997,
            "upperBound": 0.224920646735172,
            "value": 0.21580473658772353
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.20750000329730584,
            "upperBound": 0.22598305896421716,
            "value": 0.21674104450609094
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.010714925596105505,
            "upperBound": 0.01682071511305493,
            "value": 0.013767783386874713
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.009297794819478043,
            "upperBound": 0.015484304634573922,
            "value": 0.012391005048187242
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.008747022194922034,
            "upperBound": 0.014811147128834135,
            "value": 0.011779103564326144
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.008134247276099429,
            "upperBound": 0.014200116366684275,
            "value": 0.011167202080465045
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.007086152644774334,
            "upperBound": 0.012800189335919587,
            "value": 0.009943399112742848
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.0058972526570128195,
            "upperBound": 0.011235569356121597,
            "value": 0.008566620774055378
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.005351296498878605,
            "upperBound": 0.010863797867538367,
            "value": 0.008107694661159553
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.004631052655726978,
            "upperBound": 0.009748332334675393,
            "value": 0.007189842435367906
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.004021736054012624,
            "upperBound": 0.009133775673470639,
            "value": 0.006577940951506807
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9666126407674804,
            "upperBound": 0.974354819887113,
            "value": 0.9704836415362731
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9689213260475386,
            "upperBound": 0.9765504783848923,
            "value": 0.9727358937885253
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9706188092666905,
            "upperBound": 0.9776981418373041,
            "value": 0.974158368895211
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9719692059710255,
            "upperBound": 0.9789555097001174,
            "value": 0.9754623044096729
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.973884171399588,
            "upperBound": 0.9801228675183866,
            "value": 0.9770033191085823
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9756605938032811,
            "upperBound": 0.981665638911693,
            "value": 0.9786628733997155
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9770301828535622,
            "upperBound": 0.9831407401558588,
            "value": 0.9800853485064012
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9782137545858496,
            "upperBound": 0.9838536774504363,
            "value": 0.9810336652441916
           }
          },
          "fairness_indicators_metrics/negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9799442309395492,
            "upperBound": 0.9856793120732785,
            "value": 0.9828117591275486
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.025645180112887064,
            "upperBound": 0.0333873592325195,
            "value": 0.029516358463726886
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.023449521615107494,
            "upperBound": 0.031078673952461368,
            "value": 0.027264106211474633
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.022301858162695982,
            "upperBound": 0.029381190733308978,
            "value": 0.025841631104789
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.021044490299882486,
            "upperBound": 0.02803079402897466,
            "value": 0.02453769559032717
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.019877132481613376,
            "upperBound": 0.02611582860041201,
            "value": 0.022996680891417733
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.01833436108830704,
            "upperBound": 0.02433940619671888,
            "value": 0.021337126600284494
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.01685925984414109,
            "upperBound": 0.022969817146437952,
            "value": 0.01991465149359886
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.016146322549563375,
            "upperBound": 0.02178624541415009,
            "value": 0.01896633475580844
           }
          },
          "fairness_indicators_metrics/positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.014320687926721405,
            "upperBound": 0.02005576906045064,
            "value": 0.0171882408724514
           }
          },
          "fairness_indicators_metrics/precision@0.4000": {
           "boundedValue": {
            "lowerBound": 0.5815163797706651,
            "upperBound": 0.6956202597284211,
            "value": 0.6385542168674698
           }
          },
          "fairness_indicators_metrics/precision@0.4125": {
           "boundedValue": {
            "lowerBound": 0.5889925345123969,
            "upperBound": 0.7067233961474358,
            "value": 0.6478260869565218
           }
          },
          "fairness_indicators_metrics/precision@0.4250": {
           "boundedValue": {
            "lowerBound": 0.5839337746293273,
            "upperBound": 0.7097152662180185,
            "value": 0.6467889908256881
           }
          },
          "fairness_indicators_metrics/precision@0.4375": {
           "boundedValue": {
            "lowerBound": 0.5795007006519597,
            "upperBound": 0.7152620758587263,
            "value": 0.6473429951690821
           }
          },
          "fairness_indicators_metrics/precision@0.4500": {
           "boundedValue": {
            "lowerBound": 0.5887690406088415,
            "upperBound": 0.7411969320299161,
            "value": 0.6649484536082474
           }
          },
          "fairness_indicators_metrics/precision@0.4675": {
           "boundedValue": {
            "lowerBound": 0.6125913664481628,
            "upperBound": 0.7652683635085473,
            "value": 0.6888888888888889
           }
          },
          "fairness_indicators_metrics/precision@0.4750": {
           "boundedValue": {
            "lowerBound": 0.6021874634351587,
            "upperBound": 0.766971648896905,
            "value": 0.6845238095238095
           }
          },
          "fairness_indicators_metrics/precision@0.4875": {
           "boundedValue": {
            "lowerBound": 0.6205107601412326,
            "upperBound": 0.7920792964824933,
            "value": 0.70625
           }
          },
          "fairness_indicators_metrics/precision@0.5000": {
           "boundedValue": {
            "lowerBound": 0.6070170615882369,
            "upperBound": 0.7999755178274961,
            "value": 0.7034482758620689
           }
          },
          "fairness_indicators_metrics/recall@0.4000": {
           "boundedValue": {
            "lowerBound": 0.07187893525424395,
            "upperBound": 0.0955769776601093,
            "value": 0.08372827804107424
           }
          },
          "fairness_indicators_metrics/recall@0.4125": {
           "boundedValue": {
            "lowerBound": 0.06765024117039167,
            "upperBound": 0.08927507699303802,
            "value": 0.0784623486045287
           }
          },
          "fairness_indicators_metrics/recall@0.4250": {
           "boundedValue": {
            "lowerBound": 0.06422163842640834,
            "upperBound": 0.08427743494922287,
            "value": 0.07424960505529225
           }
          },
          "fairness_indicators_metrics/recall@0.4375": {
           "boundedValue": {
            "lowerBound": 0.060610331669565046,
            "upperBound": 0.08051616988921377,
            "value": 0.07056345444971038
           }
          },
          "fairness_indicators_metrics/recall@0.4500": {
           "boundedValue": {
            "lowerBound": 0.05775700291427566,
            "upperBound": 0.07810358626500152,
            "value": 0.0679304897314376
           }
          },
          "fairness_indicators_metrics/recall@0.4675": {
           "boundedValue": {
            "lowerBound": 0.055701219321643174,
            "upperBound": 0.07489302395552161,
            "value": 0.06529752501316483
           }
          },
          "fairness_indicators_metrics/recall@0.4750": {
           "boundedValue": {
            "lowerBound": 0.051331287531364345,
            "upperBound": 0.06978452025294245,
            "value": 0.06055818852027383
           }
          },
          "fairness_indicators_metrics/recall@0.4875": {
           "boundedValue": {
            "lowerBound": 0.050151983948669085,
            "upperBound": 0.06885765551128568,
            "value": 0.05950500263296472
           }
          },
          "fairness_indicators_metrics/recall@0.5000": {
           "boundedValue": {
            "lowerBound": 0.04369985359249096,
            "upperBound": 0.06372576661442006,
            "value": 0.053712480252764615
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9831792848869448,
            "upperBound": 0.9892850744038949,
            "value": 0.9862322166131253
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9845156953654262,
            "upperBound": 0.9907022051805221,
            "value": 0.9876089949518128
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9851888528711661,
            "upperBound": 0.991252977805078,
            "value": 0.9882208964356739
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9857998836333156,
            "upperBound": 0.9918657527239007,
            "value": 0.9888327979195349
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9871998106640807,
            "upperBound": 0.9929138473552257,
            "value": 0.9900566008872571
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9887644306438786,
            "upperBound": 0.9941027473429874,
            "value": 0.9914333792259447
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9891362021324617,
            "upperBound": 0.9946487035011213,
            "value": 0.9918923053388404
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9902516676653247,
            "upperBound": 0.9953689473442727,
            "value": 0.9928101575646321
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9908662243265293,
            "upperBound": 0.9959782639459874,
            "value": 0.9934220590484932
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.07187893525424395,
            "upperBound": 0.0955769776601093,
            "value": 0.08372827804107424
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.06765024117039167,
            "upperBound": 0.08927507699303802,
            "value": 0.0784623486045287
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.06422163842640834,
            "upperBound": 0.08427743494922287,
            "value": 0.07424960505529225
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.060610331669565046,
            "upperBound": 0.08051616988921377,
            "value": 0.07056345444971038
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.05775700291427566,
            "upperBound": 0.07810358626500152,
            "value": 0.0679304897314376
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.055701219321643174,
            "upperBound": 0.07489302395552161,
            "value": 0.06529752501316483
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.051331287531364345,
            "upperBound": 0.06978452025294245,
            "value": 0.06055818852027383
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.050151983948669085,
            "upperBound": 0.06885765551128568,
            "value": 0.05950500263296472
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.04369985359249096,
            "upperBound": 0.06372576661442006,
            "value": 0.053712480252764615
           }
          }
         },
         "slice": "religion:muslim",
         "sliceValue": "muslim"
        },
        {
         "metrics": {
          "accuracy": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "auc": {
           "boundedValue": {
            "lowerBound": 0.6813364902101924,
            "upperBound": 0.7349013359274488,
            "value": 0.7081257093655825
           }
          },
          "example_count": {
           "doubleValue": 3015.0
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.2528612979396553,
            "upperBound": 0.504101369843711,
            "value": 0.3783783783783784
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.23685203086112158,
            "upperBound": 0.5061843399138902,
            "value": 0.37142857142857144
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.23719226619441142,
            "upperBound": 0.5014139896561869,
            "value": 0.36923076923076925
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.24689144847156952,
            "upperBound": 0.5328512464738476,
            "value": 0.3898305084745763
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.2531776084570353,
            "upperBound": 0.5539295515865698,
            "value": 0.40350877192982454
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.20426737552079466,
            "upperBound": 0.5409016544100183,
            "value": 0.37254901960784315
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.21013641035366512,
            "upperBound": 0.5499491488861493,
            "value": 0.38
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.18352524213433102,
            "upperBound": 0.5606093424799234,
            "value": 0.37209302325581395
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.15405040322753957,
            "upperBound": 0.5288183030911024,
            "value": 0.34146341463414637
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.8796449769217708,
            "upperBound": 0.9399835034962664,
            "value": 0.9098039215686274
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.8844756854917382,
            "upperBound": 0.9429977433965469,
            "value": 0.9137254901960784
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.891629182801894,
            "upperBound": 0.9476112480262698,
            "value": 0.9196078431372549
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9035503477489257,
            "upperBound": 0.9552930568744984,
            "value": 0.9294117647058824
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9083906369199911,
            "upperBound": 0.9582964451223693,
            "value": 0.9333333333333333
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9138292712271455,
            "upperBound": 0.9606984812837999,
            "value": 0.9372549019607843
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9158229672555297,
            "upperBound": 0.96262810625927,
            "value": 0.9392156862745098
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9248530317293111,
            "upperBound": 0.9692832665744164,
            "value": 0.9470588235294117
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9248530317293111,
            "upperBound": 0.9692832665744164,
            "value": 0.9470588235294117
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.14892322572949956,
            "upperBound": 0.1666149822007852,
            "value": 0.15776946616797008
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.1493959845766103,
            "upperBound": 0.16707219129883655,
            "value": 0.15823429541595926
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.15022269245098716,
            "upperBound": 0.1677433602058763,
            "value": 0.15898305084745762
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.1511842889461013,
            "upperBound": 0.1695191772038483,
            "value": 0.16035182679296348
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.15187355205512196,
            "upperBound": 0.16996533054805255,
            "value": 0.16091954022988506
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.15191060749646149,
            "upperBound": 0.1706264280860703,
            "value": 0.1612685560053981
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.152345671172013,
            "upperBound": 0.17075718496779282,
            "value": 0.1615514333895447
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.15319515800634706,
            "upperBound": 0.17183855089837544,
            "value": 0.16251682368775236
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.15309930227533844,
            "upperBound": 0.17171580993731603,
            "value": 0.16240753194351043
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.007103556910270241,
            "upperBound": 0.015251768702304574,
            "value": 0.011177644710578843
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.006197356613773901,
            "upperBound": 0.014560989212522395,
            "value": 0.010379241516966068
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.005463746284754044,
            "upperBound": 0.013698008807819961,
            "value": 0.009580838323353293
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.004944821241501458,
            "upperBound": 0.013418531932961055,
            "value": 0.009181636726546906
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.004944821241501458,
            "upperBound": 0.013418531932961055,
            "value": 0.009181636726546906
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.003655801092190267,
            "upperBound": 0.011514104480673915,
            "value": 0.007584830339321357
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.003655801092190267,
            "upperBound": 0.011514104480673915,
            "value": 0.007584830339321357
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.002402102689950089,
            "upperBound": 0.010373093888394615,
            "value": 0.006387225548902195
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.0018973415524016811,
            "upperBound": 0.009281033825074805,
            "value": 0.005588822355289421
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9690156145474331,
            "upperBound": 0.9818962861128129,
            "value": 0.9754560530679933
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9705408036365822,
            "upperBound": 0.9830249826708387,
            "value": 0.9767827529021559
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9720302713412261,
            "upperBound": 0.9848524881511513,
            "value": 0.978441127694859
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9743470683287773,
            "upperBound": 0.9865156909495741,
            "value": 0.9804311774461029
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9752675585880402,
            "upperBound": 0.9869218153227961,
            "value": 0.981094527363184
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9780735220624326,
            "upperBound": 0.9880959492587433,
            "value": 0.9830845771144279
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9783062522441909,
            "upperBound": 0.988526611767312,
            "value": 0.9834162520729685
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9804933454309577,
            "upperBound": 0.9909826077572432,
            "value": 0.9857379767827529
           }
          },
          "fairness_indicators_metrics/negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9812338656273066,
            "upperBound": 0.9915687995631356,
            "value": 0.9864013266998342
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.018103713887187158,
            "upperBound": 0.030984385452567084,
            "value": 0.024543946932006632
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.01697501732916144,
            "upperBound": 0.029459196363417674,
            "value": 0.02321724709784411
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.015147511848848317,
            "upperBound": 0.02796972865877388,
            "value": 0.02155887230514096
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.013484309050425906,
            "upperBound": 0.025652931671222984,
            "value": 0.01956882255389718
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.013078184677203803,
            "upperBound": 0.024732441411959464,
            "value": 0.01890547263681592
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.01190405074125659,
            "upperBound": 0.021926477937567447,
            "value": 0.01691542288557214
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.011473388232687998,
            "upperBound": 0.02169374775580876,
            "value": 0.01658374792703151
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.009017392242756697,
            "upperBound": 0.019506654569042325,
            "value": 0.014262023217247097
           }
          },
          "fairness_indicators_metrics/positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.008431200436864367,
            "upperBound": 0.01876613437269362,
            "value": 0.013598673300165837
           }
          },
          "fairness_indicators_metrics/precision@0.4000": {
           "boundedValue": {
            "lowerBound": 0.49589863015628877,
            "upperBound": 0.7471387020603449,
            "value": 0.6216216216216216
           }
          },
          "fairness_indicators_metrics/precision@0.4125": {
           "boundedValue": {
            "lowerBound": 0.4938156600861099,
            "upperBound": 0.7631479691388783,
            "value": 0.6285714285714286
           }
          },
          "fairness_indicators_metrics/precision@0.4250": {
           "boundedValue": {
            "lowerBound": 0.4985860103438131,
            "upperBound": 0.7628077338055883,
            "value": 0.6307692307692307
           }
          },
          "fairness_indicators_metrics/precision@0.4375": {
           "boundedValue": {
            "lowerBound": 0.46714875352615226,
            "upperBound": 0.7531085515284306,
            "value": 0.6101694915254238
           }
          },
          "fairness_indicators_metrics/precision@0.4500": {
           "boundedValue": {
            "lowerBound": 0.44607044841343035,
            "upperBound": 0.7468223915429648,
            "value": 0.5964912280701754
           }
          },
          "fairness_indicators_metrics/precision@0.4675": {
           "boundedValue": {
            "lowerBound": 0.4590983455899814,
            "upperBound": 0.7957326244792057,
            "value": 0.6274509803921569
           }
          },
          "fairness_indicators_metrics/precision@0.4750": {
           "boundedValue": {
            "lowerBound": 0.45005085111385057,
            "upperBound": 0.7898635896463349,
            "value": 0.62
           }
          },
          "fairness_indicators_metrics/precision@0.4875": {
           "boundedValue": {
            "lowerBound": 0.43939065752007667,
            "upperBound": 0.8164747578656689,
            "value": 0.627906976744186
           }
          },
          "fairness_indicators_metrics/precision@0.5000": {
           "boundedValue": {
            "lowerBound": 0.47118169690889744,
            "upperBound": 0.8459495967724605,
            "value": 0.6585365853658537
           }
          },
          "fairness_indicators_metrics/recall@0.4000": {
           "boundedValue": {
            "lowerBound": 0.06001649650373374,
            "upperBound": 0.1203550230782293,
            "value": 0.09019607843137255
           }
          },
          "fairness_indicators_metrics/recall@0.4125": {
           "boundedValue": {
            "lowerBound": 0.05700225660345344,
            "upperBound": 0.11552431450826171,
            "value": 0.08627450980392157
           }
          },
          "fairness_indicators_metrics/recall@0.4250": {
           "boundedValue": {
            "lowerBound": 0.05238875197373014,
            "upperBound": 0.10837081719810601,
            "value": 0.0803921568627451
           }
          },
          "fairness_indicators_metrics/recall@0.4375": {
           "boundedValue": {
            "lowerBound": 0.04470694312550175,
            "upperBound": 0.09644965225107438,
            "value": 0.07058823529411765
           }
          },
          "fairness_indicators_metrics/recall@0.4500": {
           "boundedValue": {
            "lowerBound": 0.0417035548776308,
            "upperBound": 0.09160936308000914,
            "value": 0.06666666666666667
           }
          },
          "fairness_indicators_metrics/recall@0.4675": {
           "boundedValue": {
            "lowerBound": 0.03930151871620015,
            "upperBound": 0.08617072877285452,
            "value": 0.06274509803921569
           }
          },
          "fairness_indicators_metrics/recall@0.4750": {
           "boundedValue": {
            "lowerBound": 0.03737189374072991,
            "upperBound": 0.0841770327444703,
            "value": 0.060784313725490195
           }
          },
          "fairness_indicators_metrics/recall@0.4875": {
           "boundedValue": {
            "lowerBound": 0.030716733425583818,
            "upperBound": 0.07514696827068895,
            "value": 0.052941176470588235
           }
          },
          "fairness_indicators_metrics/recall@0.5000": {
           "boundedValue": {
            "lowerBound": 0.030716733425583818,
            "upperBound": 0.07514696827068895,
            "value": 0.052941176470588235
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9847482312976956,
            "upperBound": 0.9928964430897296,
            "value": 0.9888223552894212
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9854390107874778,
            "upperBound": 0.9938026433862264,
            "value": 0.9896207584830339
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.98630199119218,
            "upperBound": 0.9945362537152462,
            "value": 0.9904191616766467
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.986581468067039,
            "upperBound": 0.9950551787584986,
            "value": 0.9908183632734531
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.986581468067039,
            "upperBound": 0.9950551787584986,
            "value": 0.9908183632734531
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.988485895519326,
            "upperBound": 0.9963441989078096,
            "value": 0.9924151696606787
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.988485895519326,
            "upperBound": 0.9963441989078096,
            "value": 0.9924151696606787
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9896269061116055,
            "upperBound": 0.9975978973100498,
            "value": 0.9936127744510979
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9907189661749254,
            "upperBound": 0.998102658447598,
            "value": 0.9944111776447105
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.06001649650373374,
            "upperBound": 0.1203550230782293,
            "value": 0.09019607843137255
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.05700225660345344,
            "upperBound": 0.11552431450826171,
            "value": 0.08627450980392157
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.05238875197373014,
            "upperBound": 0.10837081719810601,
            "value": 0.0803921568627451
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.04470694312550175,
            "upperBound": 0.09644965225107438,
            "value": 0.07058823529411765
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.0417035548776308,
            "upperBound": 0.09160936308000914,
            "value": 0.06666666666666667
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.03930151871620015,
            "upperBound": 0.08617072877285452,
            "value": 0.06274509803921569
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.03737189374072991,
            "upperBound": 0.0841770327444703,
            "value": 0.060784313725490195
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.030716733425583818,
            "upperBound": 0.07514696827068895,
            "value": 0.052941176470588235
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.030716733425583818,
            "upperBound": 0.07514696827068895,
            "value": 0.052941176470588235
           }
          }
         },
         "slice": "religion:jewish",
         "sliceValue": "jewish"
        },
        {
         "metrics": {
          "accuracy": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "auc": {
           "boundedValue": {
            "lowerBound": 0.7005991278115193,
            "upperBound": 0.8082954601891787,
            "value": 0.7544400044400045
           }
          },
          "example_count": {
           "doubleValue": 545.0
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.3649210373613505,
            "upperBound": 0.8349690725287591,
            "value": 0.6
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.3649210373613505,
            "upperBound": 0.8349690725287591,
            "value": 0.6
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.4089357436201034,
            "upperBound": 0.8768701171857574,
            "value": 0.6428571428571429
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.4089357436201034,
            "upperBound": 0.8768701171857574,
            "value": 0.6428571428571429
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.39536639807160356,
            "upperBound": 0.9379669352617295,
            "value": 0.6666666666666666
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.3262862092516143,
            "upperBound": 0.874269346303941,
            "value": 0.6
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.3262862092516143,
            "upperBound": 0.874269346303941,
            "value": 0.6
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.3262862092516143,
            "upperBound": 0.874269346303941,
            "value": 0.6
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.22004061599894265,
            "upperBound": 0.891268907810581,
            "value": 0.5555555555555556
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.8657850416214014,
            "upperBound": 0.9783566873592794,
            "value": 0.922077922077922
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.8657850416214014,
            "upperBound": 0.9783566873592794,
            "value": 0.922077922077922
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.8827918743846173,
            "upperBound": 0.9873515299225257,
            "value": 0.935064935064935
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.8827918743846173,
            "upperBound": 0.9873515299225257,
            "value": 0.935064935064935
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.900400755047432,
            "upperBound": 0.9957443245861728,
            "value": 0.948051948051948
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.900400755047432,
            "upperBound": 0.9957443245861728,
            "value": 0.948051948051948
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.900400755047432,
            "upperBound": 0.9957443245861728,
            "value": 0.948051948051948
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.900400755047432,
            "upperBound": 0.9957443245861728,
            "value": 0.948051948051948
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.900400755047432,
            "upperBound": 0.9957443245861728,
            "value": 0.948051948051948
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.10744688283922887,
            "upperBound": 0.1604854592829964,
            "value": 0.1339622641509434
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.10744688283922887,
            "upperBound": 0.1604854592829964,
            "value": 0.1339622641509434
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.11007343780784895,
            "upperBound": 0.16112006242665694,
            "value": 0.13559322033898305
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.11007343780784895,
            "upperBound": 0.16112006242665694,
            "value": 0.13559322033898305
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.11156238421375657,
            "upperBound": 0.1623682432288825,
            "value": 0.13696060037523453
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.11127903318237949,
            "upperBound": 0.161626947046458,
            "value": 0.13644859813084112
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.11127903318237949,
            "upperBound": 0.161626947046458,
            "value": 0.13644859813084112
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.11127903318237949,
            "upperBound": 0.161626947046458,
            "value": 0.13644859813084112
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.11111764195108226,
            "upperBound": 0.16127851996791492,
            "value": 0.13619402985074627
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.006469817576176895,
            "upperBound": 0.03200405288008237,
            "value": 0.019230769230769232
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.006469817576176895,
            "upperBound": 0.03200405288008237,
            "value": 0.019230769230769232
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.006469817576176895,
            "upperBound": 0.03200405288008237,
            "value": 0.019230769230769232
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.006469817576176895,
            "upperBound": 0.03200405288008237,
            "value": 0.019230769230769232
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.0047252588112337655,
            "upperBound": 0.029471518612016703,
            "value": 0.017094017094017096
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.0030297257271027413,
            "upperBound": 0.022619866240613982,
            "value": 0.01282051282051282
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.0030297257271027413,
            "upperBound": 0.022619866240613982,
            "value": 0.01282051282051282
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.0030297257271027413,
            "upperBound": 0.022619866240613982,
            "value": 0.01282051282051282
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.0016338191459504148,
            "upperBound": 0.019738189236788446,
            "value": 0.010683760683760684
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9582396880929576,
            "upperBound": 0.9867033566482513,
            "value": 0.9724770642201835
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9582396880929576,
            "upperBound": 0.9867033566482513,
            "value": 0.9724770642201835
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9597568415042893,
            "upperBound": 0.9888544046087989,
            "value": 0.9743119266055046
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9597568415042893,
            "upperBound": 0.9888544046087989,
            "value": 0.9743119266055046
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9646805581391444,
            "upperBound": 0.9912760018845073,
            "value": 0.9779816513761468
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9694803816519655,
            "upperBound": 0.993815222615912,
            "value": 0.981651376146789
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9694803816519655,
            "upperBound": 0.993815222615912,
            "value": 0.981651376146789
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9694803816519655,
            "upperBound": 0.993815222615912,
            "value": 0.981651376146789
           }
          },
          "fairness_indicators_metrics/negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9728859642879935,
            "upperBound": 0.9940822969351659,
            "value": 0.9834862385321101
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.013296643351748707,
            "upperBound": 0.04176031190704256,
            "value": 0.027522935779816515
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.013296643351748707,
            "upperBound": 0.04176031190704256,
            "value": 0.027522935779816515
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.011145595391201416,
            "upperBound": 0.04024315849571075,
            "value": 0.025688073394495414
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.011145595391201416,
            "upperBound": 0.04024315849571075,
            "value": 0.025688073394495414
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.00872399811549254,
            "upperBound": 0.03531944186085548,
            "value": 0.022018348623853212
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.006184777384088154,
            "upperBound": 0.030519618348034662,
            "value": 0.01834862385321101
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.006184777384088154,
            "upperBound": 0.030519618348034662,
            "value": 0.01834862385321101
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.006184777384088154,
            "upperBound": 0.030519618348034662,
            "value": 0.01834862385321101
           }
          },
          "fairness_indicators_metrics/positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.005917703064834008,
            "upperBound": 0.027114035712006747,
            "value": 0.01651376146788991
           }
          },
          "fairness_indicators_metrics/precision@0.4000": {
           "boundedValue": {
            "lowerBound": 0.1650309274712413,
            "upperBound": 0.635078962638649,
            "value": 0.4
           }
          },
          "fairness_indicators_metrics/precision@0.4125": {
           "boundedValue": {
            "lowerBound": 0.1650309274712413,
            "upperBound": 0.635078962638649,
            "value": 0.4
           }
          },
          "fairness_indicators_metrics/precision@0.4250": {
           "boundedValue": {
            "lowerBound": 0.1231298828142422,
            "upperBound": 0.5910642563798969,
            "value": 0.35714285714285715
           }
          },
          "fairness_indicators_metrics/precision@0.4375": {
           "boundedValue": {
            "lowerBound": 0.1231298828142422,
            "upperBound": 0.5910642563798969,
            "value": 0.35714285714285715
           }
          },
          "fairness_indicators_metrics/precision@0.4500": {
           "boundedValue": {
            "lowerBound": 0.062033064738270305,
            "upperBound": 0.6046336019283962,
            "value": 0.3333333333333333
           }
          },
          "fairness_indicators_metrics/precision@0.4675": {
           "boundedValue": {
            "lowerBound": 0.1257306536960589,
            "upperBound": 0.6737137907483859,
            "value": 0.4
           }
          },
          "fairness_indicators_metrics/precision@0.4750": {
           "boundedValue": {
            "lowerBound": 0.1257306536960589,
            "upperBound": 0.6737137907483859,
            "value": 0.4
           }
          },
          "fairness_indicators_metrics/precision@0.4875": {
           "boundedValue": {
            "lowerBound": 0.1257306536960589,
            "upperBound": 0.6737137907483859,
            "value": 0.4
           }
          },
          "fairness_indicators_metrics/precision@0.5000": {
           "boundedValue": {
            "lowerBound": 0.10873109218941895,
            "upperBound": 0.7799593840010572,
            "value": 0.4444444444444444
           }
          },
          "fairness_indicators_metrics/recall@0.4000": {
           "boundedValue": {
            "lowerBound": 0.021643312640720357,
            "upperBound": 0.1342149583785984,
            "value": 0.07792207792207792
           }
          },
          "fairness_indicators_metrics/recall@0.4125": {
           "boundedValue": {
            "lowerBound": 0.021643312640720357,
            "upperBound": 0.1342149583785984,
            "value": 0.07792207792207792
           }
          },
          "fairness_indicators_metrics/recall@0.4250": {
           "boundedValue": {
            "lowerBound": 0.01264847007747407,
            "upperBound": 0.11720812561538288,
            "value": 0.06493506493506493
           }
          },
          "fairness_indicators_metrics/recall@0.4375": {
           "boundedValue": {
            "lowerBound": 0.01264847007747407,
            "upperBound": 0.11720812561538288,
            "value": 0.06493506493506493
           }
          },
          "fairness_indicators_metrics/recall@0.4500": {
           "boundedValue": {
            "lowerBound": 0.004255675413827119,
            "upperBound": 0.099599244952568,
            "value": 0.05194805194805195
           }
          },
          "fairness_indicators_metrics/recall@0.4675": {
           "boundedValue": {
            "lowerBound": 0.004255675413827119,
            "upperBound": 0.099599244952568,
            "value": 0.05194805194805195
           }
          },
          "fairness_indicators_metrics/recall@0.4750": {
           "boundedValue": {
            "lowerBound": 0.004255675413827119,
            "upperBound": 0.099599244952568,
            "value": 0.05194805194805195
           }
          },
          "fairness_indicators_metrics/recall@0.4875": {
           "boundedValue": {
            "lowerBound": 0.004255675413827119,
            "upperBound": 0.099599244952568,
            "value": 0.05194805194805195
           }
          },
          "fairness_indicators_metrics/recall@0.5000": {
           "boundedValue": {
            "lowerBound": 0.004255675413827119,
            "upperBound": 0.099599244952568,
            "value": 0.05194805194805195
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9679959471199178,
            "upperBound": 0.9935301824238232,
            "value": 0.9807692307692307
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9679959471199178,
            "upperBound": 0.9935301824238232,
            "value": 0.9807692307692307
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9679959471199178,
            "upperBound": 0.9935301824238232,
            "value": 0.9807692307692307
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9679959471199178,
            "upperBound": 0.9935301824238232,
            "value": 0.9807692307692307
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9705284813879835,
            "upperBound": 0.9952747411887661,
            "value": 0.9829059829059829
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9773801337593861,
            "upperBound": 0.9969702742728974,
            "value": 0.9871794871794872
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9773801337593861,
            "upperBound": 0.9969702742728974,
            "value": 0.9871794871794872
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9773801337593861,
            "upperBound": 0.9969702742728974,
            "value": 0.9871794871794872
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9802618107632118,
            "upperBound": 0.9983661808540493,
            "value": 0.9893162393162394
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.021643312640720357,
            "upperBound": 0.1342149583785984,
            "value": 0.07792207792207792
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.021643312640720357,
            "upperBound": 0.1342149583785984,
            "value": 0.07792207792207792
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.01264847007747407,
            "upperBound": 0.11720812561538288,
            "value": 0.06493506493506493
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.01264847007747407,
            "upperBound": 0.11720812561538288,
            "value": 0.06493506493506493
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.004255675413827119,
            "upperBound": 0.099599244952568,
            "value": 0.05194805194805195
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.004255675413827119,
            "upperBound": 0.099599244952568,
            "value": 0.05194805194805195
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.004255675413827119,
            "upperBound": 0.099599244952568,
            "value": 0.05194805194805195
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.004255675413827119,
            "upperBound": 0.099599244952568,
            "value": 0.05194805194805195
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.004255675413827119,
            "upperBound": 0.099599244952568,
            "value": 0.05194805194805195
           }
          }
         },
         "slice": "religion:atheist",
         "sliceValue": "atheist"
        },
        {
         "metrics": {
          "accuracy": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "auc": {
           "boundedValue": {
            "lowerBound": 0.5671703079973114,
            "upperBound": 0.797199630263518,
            "value": 0.6821969696969699
           }
          },
          "example_count": {
           "doubleValue": 218.0
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4000": {
           "boundedValue": {
            "lowerBound": -0.9425172802630517,
            "upperBound": 1.9425172802630517,
            "value": 0.5
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4125": {
           "boundedValue": {
            "lowerBound": -0.9425172802630517,
            "upperBound": 1.9425172802630517,
            "value": 0.5
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4250": {
           "boundedValue": {
            "lowerBound": -0.9425172802630517,
            "upperBound": 1.9425172802630517,
            "value": 0.5
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4375": {
           "boundedValue": {
            "lowerBound": -0.9425172802630517,
            "upperBound": 1.9425172802630517,
            "value": 0.5
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4500": {
           "boundedValue": {
            "lowerBound": "NaN",
            "upperBound": "NaN",
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4675": {
           "boundedValue": {
            "lowerBound": "NaN",
            "upperBound": "NaN",
            "value": "NaN"
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4750": {
           "boundedValue": {
            "lowerBound": "NaN",
            "upperBound": "NaN",
            "value": "NaN"
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4875": {
           "boundedValue": {
            "lowerBound": "NaN",
            "upperBound": "NaN",
            "value": "NaN"
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.5000": {
           "boundedValue": {
            "lowerBound": "NaN",
            "upperBound": "NaN",
            "value": "NaN"
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.8433975848679567,
            "upperBound": 1.056684974471568,
            "value": 0.95
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.8433975848679567,
            "upperBound": 1.056684974471568,
            "value": 0.95
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.8433975848679567,
            "upperBound": 1.056684974471568,
            "value": 0.95
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.8433975848679567,
            "upperBound": 1.056684974471568,
            "value": 0.95
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.052577727030225944,
            "upperBound": 0.12337510426368015,
            "value": 0.08796296296296297
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.052577727030225944,
            "upperBound": 0.12337510426368015,
            "value": 0.08796296296296297
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.052577727030225944,
            "upperBound": 0.12337510426368015,
            "value": 0.08796296296296297
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.052577727030225944,
            "upperBound": 0.12337510426368015,
            "value": 0.08796296296296297
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.055852696198463156,
            "upperBound": 0.12850439156946242,
            "value": 0.09216589861751152
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.055877581151165895,
            "upperBound": 0.12763049720879915,
            "value": 0.09174311926605505
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.055877581151165895,
            "upperBound": 0.12763049720879915,
            "value": 0.09174311926605505
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.055877581151165895,
            "upperBound": 0.12763049720879915,
            "value": 0.09174311926605505
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.055877581151165895,
            "upperBound": 0.12763049720879915,
            "value": 0.09174311926605505
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": -0.005569420199088093,
            "upperBound": 0.01567996562566676,
            "value": 0.005050505050505051
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": -0.005569420199088093,
            "upperBound": 0.01567996562566676,
            "value": 0.005050505050505051
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": -0.005569420199088093,
            "upperBound": 0.01567996562566676,
            "value": 0.005050505050505051
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": -0.005569420199088093,
            "upperBound": 0.01567996562566676,
            "value": 0.005050505050505051
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": -0.005569420199088093,
            "upperBound": 0.01567996562566676,
            "value": 0.005050505050505051
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9775428192674914,
            "upperBound": 1.0041045964454884,
            "value": 0.9908256880733946
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9775428192674914,
            "upperBound": 1.0041045964454884,
            "value": 0.9908256880733946
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9775428192674914,
            "upperBound": 1.0041045964454884,
            "value": 0.9908256880733946
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9775428192674914,
            "upperBound": 1.0041045964454884,
            "value": 0.9908256880733946
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.985767712398469,
            "upperBound": 1.005052477634381,
            "value": 0.9954128440366973
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": -0.00410459644548828,
            "upperBound": 0.022457180732508544,
            "value": 0.009174311926605505
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": -0.00410459644548828,
            "upperBound": 0.022457180732508544,
            "value": 0.009174311926605505
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": -0.00410459644548828,
            "upperBound": 0.022457180732508544,
            "value": 0.009174311926605505
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": -0.00410459644548828,
            "upperBound": 0.022457180732508544,
            "value": 0.009174311926605505
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": -0.005052477634381043,
            "upperBound": 0.01423228760153095,
            "value": 0.0045871559633027525
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/precision@0.4000": {
           "boundedValue": {
            "lowerBound": -0.9425172802630517,
            "upperBound": 1.9425172802630517,
            "value": 0.5
           }
          },
          "fairness_indicators_metrics/precision@0.4125": {
           "boundedValue": {
            "lowerBound": -0.9425172802630517,
            "upperBound": 1.9425172802630517,
            "value": 0.5
           }
          },
          "fairness_indicators_metrics/precision@0.4250": {
           "boundedValue": {
            "lowerBound": -0.9425172802630517,
            "upperBound": 1.9425172802630517,
            "value": 0.5
           }
          },
          "fairness_indicators_metrics/precision@0.4375": {
           "boundedValue": {
            "lowerBound": -0.9425172802630517,
            "upperBound": 1.9425172802630517,
            "value": 0.5
           }
          },
          "fairness_indicators_metrics/precision@0.4500": {
           "boundedValue": {
            "lowerBound": "NaN",
            "upperBound": "NaN",
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/precision@0.4675": {
           "boundedValue": {
            "lowerBound": "NaN",
            "upperBound": "NaN",
            "value": "NaN"
           }
          },
          "fairness_indicators_metrics/precision@0.4750": {
           "boundedValue": {
            "lowerBound": "NaN",
            "upperBound": "NaN",
            "value": "NaN"
           }
          },
          "fairness_indicators_metrics/precision@0.4875": {
           "boundedValue": {
            "lowerBound": "NaN",
            "upperBound": "NaN",
            "value": "NaN"
           }
          },
          "fairness_indicators_metrics/precision@0.5000": {
           "boundedValue": {
            "lowerBound": "NaN",
            "upperBound": "NaN",
            "value": "NaN"
           }
          },
          "fairness_indicators_metrics/recall@0.4000": {
           "boundedValue": {
            "lowerBound": -0.056684974471568325,
            "upperBound": 0.15660241513204304,
            "value": 0.05
           }
          },
          "fairness_indicators_metrics/recall@0.4125": {
           "boundedValue": {
            "lowerBound": -0.056684974471568325,
            "upperBound": 0.15660241513204304,
            "value": 0.05
           }
          },
          "fairness_indicators_metrics/recall@0.4250": {
           "boundedValue": {
            "lowerBound": -0.056684974471568325,
            "upperBound": 0.15660241513204304,
            "value": 0.05
           }
          },
          "fairness_indicators_metrics/recall@0.4375": {
           "boundedValue": {
            "lowerBound": -0.056684974471568325,
            "upperBound": 0.15660241513204304,
            "value": 0.05
           }
          },
          "fairness_indicators_metrics/recall@0.4500": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/recall@0.4675": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/recall@0.4750": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/recall@0.4875": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/recall@0.5000": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9843200343743331,
            "upperBound": 1.005569420199088,
            "value": 0.9949494949494949
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9843200343743331,
            "upperBound": 1.005569420199088,
            "value": 0.9949494949494949
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9843200343743331,
            "upperBound": 1.005569420199088,
            "value": 0.9949494949494949
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9843200343743331,
            "upperBound": 1.005569420199088,
            "value": 0.9949494949494949
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9843200343743331,
            "upperBound": 1.005569420199088,
            "value": 0.9949494949494949
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": -0.056684974471568325,
            "upperBound": 0.15660241513204304,
            "value": 0.05
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": -0.056684974471568325,
            "upperBound": 0.15660241513204304,
            "value": 0.05
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": -0.056684974471568325,
            "upperBound": 0.15660241513204304,
            "value": 0.05
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": -0.056684974471568325,
            "upperBound": 0.15660241513204304,
            "value": 0.05
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          }
         },
         "slice": "religion:hindu",
         "sliceValue": "hindu"
        },
        {
         "metrics": {
          "accuracy": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "auc": {
           "boundedValue": {
            "lowerBound": 0.5544230991004662,
            "upperBound": 0.8016680167161103,
            "value": 0.6780956112852664
           }
          },
          "example_count": {
           "doubleValue": 138.0
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.7836674170799884,
            "upperBound": 1.0347638884565804,
            "value": 0.9090909090909091
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.7836674170799884,
            "upperBound": 1.0347638884565804,
            "value": 0.9090909090909091
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.7836674170799884,
            "upperBound": 1.0347638884565804,
            "value": 0.9090909090909091
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.7836674170799884,
            "upperBound": 1.0347638884565804,
            "value": 0.9090909090909091
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.7836674170799884,
            "upperBound": 1.0347638884565804,
            "value": 0.9090909090909091
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.7836674170799884,
            "upperBound": 1.0347638884565804,
            "value": 0.9090909090909091
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.7836674170799884,
            "upperBound": 1.0347638884565804,
            "value": 0.9090909090909091
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.7836674170799884,
            "upperBound": 1.0347638884565804,
            "value": 0.9090909090909091
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.7836674170799884,
            "upperBound": 1.0347638884565804,
            "value": 0.9090909090909091
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.09157302339165083,
            "upperBound": 0.20250321387266165,
            "value": 0.14705882352941177
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.09157302339165083,
            "upperBound": 0.20250321387266165,
            "value": 0.14705882352941177
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.09157302339165083,
            "upperBound": 0.20250321387266165,
            "value": 0.14705882352941177
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.09157302339165083,
            "upperBound": 0.20250321387266165,
            "value": 0.14705882352941177
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.09157302339165083,
            "upperBound": 0.20250321387266165,
            "value": 0.14705882352941177
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.09157302339165083,
            "upperBound": 0.20250321387266165,
            "value": 0.14705882352941177
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.09157302339165083,
            "upperBound": 0.20250321387266165,
            "value": 0.14705882352941177
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.09157302339165083,
            "upperBound": 0.20250321387266165,
            "value": 0.14705882352941177
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.09157302339165083,
            "upperBound": 0.20250321387266165,
            "value": 0.14705882352941177
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9645244883547686,
            "upperBound": 1.006483880560576,
            "value": 0.9855072463768116
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9645244883547686,
            "upperBound": 1.006483880560576,
            "value": 0.9855072463768116
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9645244883547686,
            "upperBound": 1.006483880560576,
            "value": 0.9855072463768116
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9645244883547686,
            "upperBound": 1.006483880560576,
            "value": 0.9855072463768116
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9645244883547686,
            "upperBound": 1.006483880560576,
            "value": 0.9855072463768116
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9645244883547686,
            "upperBound": 1.006483880560576,
            "value": 0.9855072463768116
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9645244883547686,
            "upperBound": 1.006483880560576,
            "value": 0.9855072463768116
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9645244883547686,
            "upperBound": 1.006483880560576,
            "value": 0.9855072463768116
           }
          },
          "fairness_indicators_metrics/negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9645244883547686,
            "upperBound": 1.006483880560576,
            "value": 0.9855072463768116
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": -0.006483880560576295,
            "upperBound": 0.03547551164523126,
            "value": 0.014492753623188406
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": -0.006483880560576295,
            "upperBound": 0.03547551164523126,
            "value": 0.014492753623188406
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": -0.006483880560576295,
            "upperBound": 0.03547551164523126,
            "value": 0.014492753623188406
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": -0.006483880560576295,
            "upperBound": 0.03547551164523126,
            "value": 0.014492753623188406
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": -0.006483880560576295,
            "upperBound": 0.03547551164523126,
            "value": 0.014492753623188406
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": -0.006483880560576295,
            "upperBound": 0.03547551164523126,
            "value": 0.014492753623188406
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": -0.006483880560576295,
            "upperBound": 0.03547551164523126,
            "value": 0.014492753623188406
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": -0.006483880560576295,
            "upperBound": 0.03547551164523126,
            "value": 0.014492753623188406
           }
          },
          "fairness_indicators_metrics/positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": -0.006483880560576295,
            "upperBound": 0.03547551164523126,
            "value": 0.014492753623188406
           }
          },
          "fairness_indicators_metrics/precision@0.4000": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.4125": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.4250": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.4375": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.4500": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.4675": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.4750": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.4875": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.5000": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/recall@0.4000": {
           "boundedValue": {
            "lowerBound": -0.034763888456580364,
            "upperBound": 0.21633258292001167,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/recall@0.4125": {
           "boundedValue": {
            "lowerBound": -0.034763888456580364,
            "upperBound": 0.21633258292001167,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/recall@0.4250": {
           "boundedValue": {
            "lowerBound": -0.034763888456580364,
            "upperBound": 0.21633258292001167,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/recall@0.4375": {
           "boundedValue": {
            "lowerBound": -0.034763888456580364,
            "upperBound": 0.21633258292001167,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/recall@0.4500": {
           "boundedValue": {
            "lowerBound": -0.034763888456580364,
            "upperBound": 0.21633258292001167,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/recall@0.4675": {
           "boundedValue": {
            "lowerBound": -0.034763888456580364,
            "upperBound": 0.21633258292001167,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/recall@0.4750": {
           "boundedValue": {
            "lowerBound": -0.034763888456580364,
            "upperBound": 0.21633258292001167,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/recall@0.4875": {
           "boundedValue": {
            "lowerBound": -0.034763888456580364,
            "upperBound": 0.21633258292001167,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/recall@0.5000": {
           "boundedValue": {
            "lowerBound": -0.034763888456580364,
            "upperBound": 0.21633258292001167,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": -0.034763888456580364,
            "upperBound": 0.21633258292001167,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": -0.034763888456580364,
            "upperBound": 0.21633258292001167,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": -0.034763888456580364,
            "upperBound": 0.21633258292001167,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": -0.034763888456580364,
            "upperBound": 0.21633258292001167,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": -0.034763888456580364,
            "upperBound": 0.21633258292001167,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": -0.034763888456580364,
            "upperBound": 0.21633258292001167,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": -0.034763888456580364,
            "upperBound": 0.21633258292001167,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": -0.034763888456580364,
            "upperBound": 0.21633258292001167,
            "value": 0.09090909090909091
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": -0.034763888456580364,
            "upperBound": 0.21633258292001167,
            "value": 0.09090909090909091
           }
          }
         },
         "slice": "religion:other_religion",
         "sliceValue": "other_religion"
        },
        {
         "metrics": {
          "accuracy": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "auc": {
           "boundedValue": {
            "lowerBound": 0.7001188173707207,
            "upperBound": 0.8703523258274064,
            "value": 0.7852078633580406
           }
          },
          "example_count": {
           "doubleValue": 243.0
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.4875": {
           "boundedValue": {
            "lowerBound": "NaN",
            "upperBound": "NaN",
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_discovery_rate@0.5000": {
           "boundedValue": {
            "lowerBound": "NaN",
            "upperBound": "NaN",
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.7768699178456813,
            "upperBound": 1.0160773966533574,
            "value": 0.896551724137931
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.7768699178456813,
            "upperBound": 1.0160773966533574,
            "value": 0.896551724137931
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.8302541920988559,
            "upperBound": 1.0317140759038947,
            "value": 0.9310344827586207
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.8302541920988559,
            "upperBound": 1.0317140759038947,
            "value": 0.9310344827586207
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.8302541920988559,
            "upperBound": 1.0317140759038947,
            "value": 0.9310344827586207
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.8302541920988559,
            "upperBound": 1.0317140759038947,
            "value": 0.9310344827586207
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.8302541920988559,
            "upperBound": 1.0317140759038947,
            "value": 0.9310344827586207
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.8917219160079988,
            "upperBound": 1.0391248553560135,
            "value": 0.9655172413793104
           }
          },
          "fairness_indicators_metrics/false_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.8917219160079988,
            "upperBound": 1.0391248553560135,
            "value": 0.9655172413793104
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.06613288843674422,
            "upperBound": 0.15052081573685883,
            "value": 0.10833333333333334
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.06613288843674422,
            "upperBound": 0.15052081573685883,
            "value": 0.10833333333333334
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.0704337686443717,
            "upperBound": 0.15361578445715623,
            "value": 0.11203319502074689
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.0704337686443717,
            "upperBound": 0.15361578445715623,
            "value": 0.11203319502074689
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.0704337686443717,
            "upperBound": 0.15361578445715623,
            "value": 0.11203319502074689
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.0704337686443717,
            "upperBound": 0.15361578445715623,
            "value": 0.11203319502074689
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.0704337686443717,
            "upperBound": 0.15361578445715623,
            "value": 0.11203319502074689
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.07291070766562549,
            "upperBound": 0.1584734210391969,
            "value": 0.11570247933884298
           }
          },
          "fairness_indicators_metrics/false_omission_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.07291070766562549,
            "upperBound": 0.1584734210391969,
            "value": 0.11570247933884298
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/false_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.0,
            "upperBound": 0.0,
            "value": 0.0
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 0.9736320094551186,
            "upperBound": 1.00167829526612,
            "value": 0.9876543209876543
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 0.9736320094551186,
            "upperBound": 1.00167829526612,
            "value": 0.9876543209876543
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 0.9798493857853189,
            "upperBound": 1.0036857805787922,
            "value": 0.9917695473251029
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 0.9798493857853189,
            "upperBound": 1.0036857805787922,
            "value": 0.9917695473251029
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 0.9798493857853189,
            "upperBound": 1.0036857805787922,
            "value": 0.9917695473251029
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 0.9798493857853189,
            "upperBound": 1.0036857805787922,
            "value": 0.9917695473251029
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 0.9798493857853189,
            "upperBound": 1.0036857805787922,
            "value": 0.9917695473251029
           }
          },
          "fairness_indicators_metrics/negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 0.9872423635434671,
            "upperBound": 1.0045195966066498,
            "value": 0.9958847736625515
           }
          },
          "fairness_indicators_metrics/negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 0.9872423635434671,
            "upperBound": 1.0045195966066498,
            "value": 0.9958847736625515
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": -0.0016782952661200008,
            "upperBound": 0.02636799054488142,
            "value": 0.012345679012345678
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": -0.0016782952661200008,
            "upperBound": 0.02636799054488142,
            "value": 0.012345679012345678
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": -0.003685780578792054,
            "upperBound": 0.020150614214680992,
            "value": 0.00823045267489712
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": -0.003685780578792054,
            "upperBound": 0.020150614214680992,
            "value": 0.00823045267489712
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": -0.003685780578792054,
            "upperBound": 0.020150614214680992,
            "value": 0.00823045267489712
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": -0.003685780578792054,
            "upperBound": 0.020150614214680992,
            "value": 0.00823045267489712
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": -0.003685780578792054,
            "upperBound": 0.020150614214680992,
            "value": 0.00823045267489712
           }
          },
          "fairness_indicators_metrics/positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": -0.004519596606649851,
            "upperBound": 0.012757636456533139,
            "value": 0.00411522633744856
           }
          },
          "fairness_indicators_metrics/positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": -0.004519596606649851,
            "upperBound": 0.012757636456533139,
            "value": 0.00411522633744856
           }
          },
          "fairness_indicators_metrics/precision@0.4000": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.4125": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.4250": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.4375": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.4500": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.4675": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.4750": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.4875": {
           "boundedValue": {
            "lowerBound": "NaN",
            "upperBound": "NaN",
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/precision@0.5000": {
           "boundedValue": {
            "lowerBound": "NaN",
            "upperBound": "NaN",
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/recall@0.4000": {
           "boundedValue": {
            "lowerBound": -0.016077396653357334,
            "upperBound": 0.22313008215431873,
            "value": 0.10344827586206896
           }
          },
          "fairness_indicators_metrics/recall@0.4125": {
           "boundedValue": {
            "lowerBound": -0.016077396653357334,
            "upperBound": 0.22313008215431873,
            "value": 0.10344827586206896
           }
          },
          "fairness_indicators_metrics/recall@0.4250": {
           "boundedValue": {
            "lowerBound": -0.0317140759038947,
            "upperBound": 0.16974580790114396,
            "value": 0.06896551724137931
           }
          },
          "fairness_indicators_metrics/recall@0.4375": {
           "boundedValue": {
            "lowerBound": -0.0317140759038947,
            "upperBound": 0.16974580790114396,
            "value": 0.06896551724137931
           }
          },
          "fairness_indicators_metrics/recall@0.4500": {
           "boundedValue": {
            "lowerBound": -0.0317140759038947,
            "upperBound": 0.16974580790114396,
            "value": 0.06896551724137931
           }
          },
          "fairness_indicators_metrics/recall@0.4675": {
           "boundedValue": {
            "lowerBound": -0.0317140759038947,
            "upperBound": 0.16974580790114396,
            "value": 0.06896551724137931
           }
          },
          "fairness_indicators_metrics/recall@0.4750": {
           "boundedValue": {
            "lowerBound": -0.0317140759038947,
            "upperBound": 0.16974580790114396,
            "value": 0.06896551724137931
           }
          },
          "fairness_indicators_metrics/recall@0.4875": {
           "boundedValue": {
            "lowerBound": -0.039124855356013746,
            "upperBound": 0.10827808399200099,
            "value": 0.034482758620689655
           }
          },
          "fairness_indicators_metrics/recall@0.5000": {
           "boundedValue": {
            "lowerBound": -0.039124855356013746,
            "upperBound": 0.10827808399200099,
            "value": 0.034482758620689655
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4000": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4125": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4250": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4375": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4500": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4675": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4750": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.4875": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_negative_rate@0.5000": {
           "boundedValue": {
            "lowerBound": 1.0,
            "upperBound": 1.0,
            "value": 1.0
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4000": {
           "boundedValue": {
            "lowerBound": -0.016077396653357334,
            "upperBound": 0.22313008215431873,
            "value": 0.10344827586206896
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4125": {
           "boundedValue": {
            "lowerBound": -0.016077396653357334,
            "upperBound": 0.22313008215431873,
            "value": 0.10344827586206896
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4250": {
           "boundedValue": {
            "lowerBound": -0.0317140759038947,
            "upperBound": 0.16974580790114396,
            "value": 0.06896551724137931
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4375": {
           "boundedValue": {
            "lowerBound": -0.0317140759038947,
            "upperBound": 0.16974580790114396,
            "value": 0.06896551724137931
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4500": {
           "boundedValue": {
            "lowerBound": -0.0317140759038947,
            "upperBound": 0.16974580790114396,
            "value": 0.06896551724137931
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4675": {
           "boundedValue": {
            "lowerBound": -0.0317140759038947,
            "upperBound": 0.16974580790114396,
            "value": 0.06896551724137931
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4750": {
           "boundedValue": {
            "lowerBound": -0.0317140759038947,
            "upperBound": 0.16974580790114396,
            "value": 0.06896551724137931
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.4875": {
           "boundedValue": {
            "lowerBound": -0.039124855356013746,
            "upperBound": 0.10827808399200099,
            "value": 0.034482758620689655
           }
          },
          "fairness_indicators_metrics/true_positive_rate@0.5000": {
           "boundedValue": {
            "lowerBound": -0.039124855356013746,
            "upperBound": 0.10827808399200099,
            "value": 0.034482758620689655
           }
          }
         },
         "slice": "religion:buddhist",
         "sliceValue": "buddhist"
        }
       ],
       "slicingMetricsCompare": []
      }
     }
    },
    "version_major": 2,
    "version_minor": 0
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}