# Downloading Data from Google Cloud Storage

This page explains, in simple terms, how to download data files from the Climate Atlas related buckets in Google Cloud Storage (GCS).

You do not need to be a cloud expert, but you do need:

- A Google account.
- The Google Cloud SDK installed (`gcloud` and `gsutil`).
- A terminal (Command Prompt, PowerShell, or a shell in VS Code).

The examples below use `gsutil`, which is a command line tool for working with GCS.

> Note: The `nclimgrid-daily` dataset is in a different bucket than the main Climate Atlas COGs. It uses the bucket `gs://noaa-ncei-nclimgrid-daily/...`.

---

## 1. Log In and Check Access

1. Open a terminal.
2. Run:

   ```bash
   gcloud auth login
   ```

3. Follow the prompts in your browser to sign in.
4. In the same terminal, run a simple list command to confirm access, for example:

   ```bash
   gsutil ls gs://noaa-ncei-nclimgrid-daily/
   ```

If you see a list of folders or files, your setup is working.

---

## 2. Basic `gsutil cp` Example

The simplest way to download a single file is with a `cp` command/

Example:

```bash
gsutil cp gs://noaa-ncei-nclimgrid-daily/cog/1952/nclimgrid-daily-19520101.tif ./
```

This copies `nclimgrid-daily-19520101` from the bucket to your current folder.

- `gs://noaa-ncei-nclimgrid-daily/...` is the source in GCS.
- `./` is the destination on your local machine (the current directory).

---

## 3. Downloading Many Files at Once with `-m`

To download many files at once, you can:

- List all the file paths.
- Pass them all to one `gsutil cp` command.
- Add `-m` to run the copies in parallel (faster).

### Example: First 10 days of nClimGrid data from January 1952

```bash
gsutil -m cp \
  "gs://noaa-ncei-nclimgrid-daily/cog/1952/nclimgrid-daily-19520101.tif" \
  "gs://noaa-ncei-nclimgrid-daily/cog/1952/nclimgrid-daily-19520102.tif" \
  "gs://noaa-ncei-nclimgrid-daily/cog/1952/nclimgrid-daily-19520103.tif" \
  "gs://noaa-ncei-nclimgrid-daily/cog/1952/nclimgrid-daily-19520104.tif" \
  "gs://noaa-ncei-nclimgrid-daily/cog/1952/nclimgrid-daily-19520105.tif" \
  "gs://noaa-ncei-nclimgrid-daily/cog/1952/nclimgrid-daily-19520106.tif" \
  "gs://noaa-ncei-nclimgrid-daily/cog/1952/nclimgrid-daily-19520107.tif" \
  "gs://noaa-ncei-nclimgrid-daily/cog/1952/nclimgrid-daily-19520108.tif" \
  "gs://noaa-ncei-nclimgrid-daily/cog/1952/nclimgrid-daily-19520109.tif" \
  "gs://noaa-ncei-nclimgrid-daily/cog/1952/nclimgrid-daily-19520110.tif" \
  ./
```

Notes:

- Each line after the first is another file to copy.
- The final `./` is the local destination folder.
- The backslashes (`\`) at the end of each line are a shell line continuation. They work in bash and many Unix like shells. On Windows PowerShell, you can put the command on one long line instead.

---

## 4. Using Wildcards

`gsutil` supports wildcards for simple patterns.

For example, to download all January 1952 nclimgrid daily files:

```bash
gsutil -m cp "gs://noaa-ncei-nclimgrid-daily/cog/1952/nclimgrid-daily-195201*.tif" ./
```

This matches any file that starts with `nclimgrid-daily-195201` and ends with `.tif`.

Be careful with wildcards:

- Always run a list first to see what you will get:

  ```bash
  gsutil ls "gs://noaa-ncei-nclimgrid-daily/cog/1952/nclimgrid-daily-195201*.tif"
  ```

- If the list looks correct, then run the `cp` command.

---

## 5. Downloading a Climate Atlas COG Example

Most Climate Atlas COGs live in a bucket like this:

```text
gs://noaa-ncei-ipg/climate_atlas/cog/v1/
```

Here is an example COG URL for a daily normals file:

```text
https://storage.googleapis.com/noaa-ncei-ipg/climate_atlas/cog/v1/daily_normals/nclimgrid-1991-2020-daily-normals-day_100.tif
```

You can turn that into a `gs://` path for `gsutil` like this:

```text
gs://noaa-ncei-ipg/climate_atlas/cog/v1/daily_normals/nclimgrid-1991-2020-daily-normals-day_100.tif
```

To download that single file into the current folder:

```bash
gsutil cp "gs://noaa-ncei-ipg/climate_atlas/cog/v1/daily_normals/nclimgrid-1991-2020-daily-normals-day_100.tif" ./
```

If you want to download several similar files, you can adapt the pattern examples above by changing the bucket and path to match `noaa-ncei-ipg/climate_atlas/cog/v1/...`.

## 6. Troubleshooting Tips

If something goes wrong:

- **Permission errors**: check that you are logged in with a Google account that has access to the bucket.
- **Typos in paths**: use `gsutil ls` to verify the exact bucket name and folder structure.
- **Slow downloads**: use `-m` for parallel copies, but be mindful of network limits.

When in doubt, start by listing paths with `gsutil ls`, confirm they are correct, and only then run the `cp` commands.