Admin on Apache Org Exposed for 2.5 Years via Deleted PyPI Package

TL;DR: Deleting a PyPI package does not remove it from PyPI’s servers. We recovered and scanned 600,000 package releases, discovering 190 unique secrets, including a GitHub PAT with admin access to the Apache and Astronomer organizations.

Deleted packages recovered

678,376

Unique live secrets

190

Packages with secrets

161

Case Study: Admin Access to Apache & Astronomer

We discovered a GitHub Personal Access Token (PAT) lurking in a package from April 2023. This token granted admin access to the Apache and Astronomer GitHub organizations.

The PAT was base64 encoded inside a helpers.py file. This package also contained some hardcoded AWS keys (which were invalid) as well as some other OAuth tokens.

Given that these organizations support hundreds of thousands of users, a malicious actor could have potentially compromised critical repositories and the downstream supply chain. We disclosed the PAT to both organizations; they verified the vulnerability and revoked the credentials within hours.

The Persistence Gap

Hundreds of Python packages are published to PyPI every day, and some get deleted by their owners. We found that deleting a PyPI package doesn’t actually remove the artifact. PyPI’s UI and API (pip install <package>) return a clean 404 when requesting a deleted package, but the underlying object storage keeps serving the artifacts to anyone who knows the direct URL, indefinitely.

Package deletion should mean permanent removal, unfortunately in PyPI, it doesn’t. PyPI only hard-deletes artifacts from object storage when PII is reported, and only on request. For everything else, the files persist. We call this gap Ghost Persistence, and it allowed us to recover the code of over 600,000 deleted packages.

Enumerating “deleted” Packages

That URL is only one piece of the puzzle. The object storage path follows a hash-based directory structure: three path components derived from the file’s blake2b-256 digest, appended with the package name and version.

Brute-forcing the blake2b-256 path components isn’t realistic, and you need the whole hash to access the package. Instead, we leveraged PyPI’s own public metadata to reconstruct these paths.

PyPI maintains a public BigQuery dataset containing metadata for every package ever uploaded. Crucially, this data is never removed, even if a release or project is deleted. By querying this dataset, we can reconstruct the exact object storage download URLs for packages that no longer exist.

“The table is meant to be a data dump of metadata from every release on PyPI, which means that the rows in this BigQuery table are immutable and are not removed even if a release or project is deleted.” - https://docs.pypi.org/api/bigquery/

Next, we needed to identify which packages had been deleted. To do this we saved all the unique package names of entries in the BigQuery dataset and ran these through a simple python script to detect availability.

select distinct(name) from bigquery-public-data.pypi.distribution_metadata;

The python script checked the response code of requests to this endpoint https://pypi.org/pypi/{package}/json a 404 meant the package was “deleted”.

def check_package(name: str, session: requests.Session) -> dict:
    """Check if a package exists on PyPI."""
    url = PYPI_URL.format(package=name.lower().replace("_", "-").replace(".", "-"))
    for attempt in range(3):
        try:
            resp = session.head(url, allow_redirects=True, timeout=15)
            if resp.status_code == 429:
                time.sleep(2 ** attempt)
                print("rate limited")
                continue
            return {"name": name, "alive": resp.status_code == 200}
        except requests.RequestException:
            return {"name": name, "alive": False, "error": True}
    return {"name": name, "alive": False, "error": True}

We found 150,868 “deleted” packages. Some packages had multiple versions, so in total, we found 678,376 unique artifacts using this method. This methodology excludes packages where a new owner re-registered the original name.

The oldest package that contained a secret was published in 2018 and contained a live Telegram Bot Token. The latest package was published in January 2026 and contained valid Postgres credentials.

This is a significant gap in the deletion process, some of these secrets have permissive access to sensitive services and information. The packages are also believed to be deleted, so organizations aren’t looking here for leaks.

PyPI’s Response

We contacted PyPI regarding this issue. According to their policy, hard deletions from the underlying object storage are only performed when PII is reported. They also noted that deleting packages has limited effectiveness, as PyPI mirrors continuously archiving package code.