![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
\n",
"\n",
""
],
"metadata": {
"id": "oS1XWW4yBcYR"
}
},
{
"cell_type": "markdown",
"source": [
"# Exploring Global Precipitation with Zarr on Google Cloud Storage and PARDIG\n",
"\n",
"**Welcome!** This section demonstrates how to load, slice, and visualize the **PARDIG** (*Precipitation AI Reanalysis — Densely Inferred GPM*) dataset available in Zarr format on Google Cloud Storage, developed by Google DeepMind and Google Research.\n",
"\n",
"**Dataset Overview**\n",
"\n",
"PARDIG is an experimental dataset of global, AI-driven precipitation estimates developed by Google DeepMind and Google Research. PARDIG is trained directly on precipitation retrievals from the Combined Radar-Radiometer Algorithm (CORRA) aboard the NASA/JAXA Global Precipitation Measurement (GPM) Core Observatory. Relying on satellite observations and ECMWF atmospheric fields as inputs, PARDIG produces global precipitation rate estimates **every 15 minutes** at **0.1° (~10 km) spatial resolution** from **1999 to the present**.\n",
"\n",
"PARDIG is also used as one of the precipitation training targets for WeatherNext 3 (`experimental_tp_1hr`).\n",
"\n",
"PARDIG is released in two compatible variants:\n",
"* **PARDIG-NRT (Near-Real-Time):** 2016 to present (starts `2016-01-01T07:45:00` UTC; updated every 6 hours on Cloud Storage with ~5 hours average, ~2–8 hours effective latency)\n",
"* **PARDIG-LR (Long-Record):** 1999–2023 (`1999-01-01T00:00:00` to `2023-12-31T23:45:00` UTC; static 25-year archive for climatological studies)\n",
"\n",
"**Notebook Purpose**\n",
"\n",
"This section aims to:\n",
"\n",
"* Familiarize users with the directory structure, chunking layouts, and variable conventions of the PARDIG dataset in Zarr on Google Cloud Storage.\n",
"* Demonstrate how to query, slice, and aggregate instantaneous precipitation rates using Xarray and obstore.\n",
"* Showcase both spatial map querying and high-throughput multi-year point time-series analysis using the dedicated time-series chunked Zarr store.\n",
"\n",
"**Copyright Information**\n",
"\n",
"Copyright 2026 Google LLC.\n",
"\n",
"All materials and data are licensed under the [Creative Commons Attribution 4.0 International License (CC-BY 4.0)](https://creativecommons.org/licenses/by/4.0/). Publicly accessible with any Google Cloud account (no access request form required).\n",
"\n",
"**Additional Documentation**\n",
"\n",
"* [PARDIG Developer Guide](https://developers.google.com/weathernext/guides/pardig)\n",
"* [Xarray Library Documentation](https://docs.xarray.dev)"
],
"metadata": {
"id": "RXeYspjIbJt5"
}
},
{
"cell_type": "markdown",
"metadata": {
"id": "cell_1793952622267731146_5"
},
"source": [
"# PARDIG Dataset Schema & Cloud Storage Structure\n",
"\n",
"PARDIG Zarr datasets are published in Google Cloud Storage under:\n",
"`gs://pardig_1/`\n",
"\n",
"### Directory Structure & Storage Paths\n",
"\n",
"```none\n",
"gs://pardig_1/\n",
"├── LR/\n",
"│ ├── zarr/\n",
"│ │ └── 1999_to_2023/\n",
"│ │ └── predictions.zarr # Spatial access: chunked (4, 1801, 3600)\n",
"│ └── zarr_rechunked_as_timeseries/\n",
"│ └── 1999_to_2023/\n",
"│ └── predictions.zarr # Time-series access: chunked (time: -1, 5, 5)\n",
"└── NRT/\n",
" ├── zarr/\n",
" │ ├── 2016_to_2025/\n",
" │ │ └── predictions.zarr # Spatial access: chunked (4, 1801, 3600); starts 2016-01-01T07:45:00\n",
" │ └── 2026_to_present/ # Live operational 15-minute frames (updated every 6 hours)\n",
" │ ├── <xarray.Dataset> Size: 9TB\n",
"Dimensions: (time: 350657, latitude: 1801, longitude: 3600)\n",
"Coordinates:\n",
" * time (time) datetime64[ns] 3MB 2016-01-01T07:45:00 ... 2025-12-...\n",
" * latitude (latitude) float64 14kB -90.0 -89.9 -89.8 ... 89.8 89.9 90.0\n",
" * longitude (longitude) float64 29kB 0.0 0.1 0.2 ... 359.7 359.8 359.9\n",
"Data variables:\n",
" precipitation (time, latitude, longitude) float32 9TB dask.array<chunksize=(4, 1801, 3600), meta=np.ndarray><xarray.Dataset> Size: 9TB\n",
"Dimensions: (time: 350657, latitude: 1801, longitude: 3600)\n",
"Coordinates:\n",
" * time (time) datetime64[ns] 3MB 2016-01-01T07:45:00 ... 2025-12-...\n",
" * latitude (latitude) float64 14kB -90.0 -89.9 -89.8 ... 89.8 89.9 90.0\n",
" * longitude (longitude) float64 29kB 0.0 0.1 0.2 ... 359.7 359.8 359.9\n",
"Data variables:\n",
" precipitation (time, latitude, longitude) float32 9TB dask.array<chunksize=(350657, 5, 5), meta=np.ndarray>