# Encryption

Data encryption and decryption is the process of transforming plaintext data to cipher-text and vice versa using a key or password.

## Encryption in flight

CockroachDB uses TLS 1.2 for inter-node and client-node authentication as well as setting up a secure communication channel. Once the secure channel is set up, all inter-node and client-node network communication is encrypted using a shared encryption key as per the TLS 1.2 protocol. This feature is enabled by default for all secure clusters and needs no additional configuration.

## Encryption at Rest (Enterprise)

Encryption at Rest provides transparent encryption of a node's data on the local disk. It allows encryption of all files on disk using AES in counter mode, with all key sizes allowed.

Encryption is performed in the storage layer and configured per store. All files used by the store, regardless of contents, are encrypted with the desired algorithm.

To allow arbitrary rotation schedules and ensure security of the keys, we use two layers of keys:

- Store keys are provided by the user in a file. They are used to encrypt the list of data keys (see below). This is known as a key encryption key: its only purpose is to encrypt other keys. Store keys are never persisted by CockroachDB. Since very little data is encrypted using this key, it can have a very long lifetime without risk of reuse.

- Data keys are automatically generated by CockroachDB. They are used to encrypt all files on disk. This is known as a data encryption key. Data keys are persisted in a key registry file, encrypted using the store key. The key has a short lifetime to avoid reuse.

Store keys are specified at node startup by passing a path to a locally readable file. The file must contain 32 bytes (the key ID) followed by the key (16, 24, or 32 bytes). The size of the key dictates the version of AES to use (AES-128, AES-192, or AES-256). For an example showing how to create a store key, see Generating key files below.

Also during node startup, CockroachDB uses a data key with the same length as the store key. If encryption has just been enabled, the key size has changed, or the data key is too old (default lifetime is one week), CockroachDB generates a new data key.

Any new file created by the store uses the currently-active data key. All data keys (both active and previous) are stored in a key registry file and encrypted with the active store key.

After startup, if the active data key is too old, CockroachDB generates a new data key and marks it as active, using it for all further encryption.

CockroachDB does not currently force re-encryption of older files but instead relies on normal RocksDB churn to slowly rewrite all data with the desired encryption.

### Rotating keys

Key rotation is necessary for Encryption at Rest for multiple reasons:

- To prevent key reuse with the same encryption parameters (after encrypting many files).
- To reduce the risk of key exposure.

Store keys are specified by the user and must be rotated by specifying different keys. This is done by setting the `key` parameter of the `--enterprise-encryption` flag to the path to the new key, and `old-key` to the previously-used key.

Data keys will automatically be rotated at startup if any of the following conditions are met:

- The active store key has changed.
- The encryption type has changed (different key size, or plaintext to/from encryption).
- The current data key is `rotation-period` old or more.

Data keys will automatically be rotated at runtime if the current data key is `rotation-period` old or more.

Once rotated, an old store key cannot be made the active key again.

Upon store key rotation the data keys registry is decrypted using the old key and encrypted with the new key. The newly-generated data key is used to encrypt all new data from this point on.

### Changing encryption type

The user can change the encryption type from plaintext to encryption, between different encryption algorithms (using various key sizes), or from encryption to plaintext.

When changing the encryption type to plaintext, the data key registry is no longer encrypted and all previous data keys are readable by anyone. All data on the store is effectively readable.

When changing from plaintext to encryption, it will take some time for all data to eventually be re-written and encrypted.

### Recommendations

There are a number of considerations to keep in mind when running with encryption.

#### Deployment configuration

To prevent key leakage, production deployments should:

- Use encrypted swap, or disable swap entirely.
- Disable core files.

CockroachDB attempts to disable core files at startup when encryption is requested, but it may fail.

#### Key handling

Key management is the most dangerous aspect of encryption. The following rules should be kept in mind:

- Make sure that only the UNIX user running the `cockroach` process has access to the keys.
- Do not store the keys on the same partition/drive as the CockroachDB data. It is best to load keys at run time from a separate system (e.g., Keywhiz, Vault).
- Rotate store keys frequently (every few weeks to months).
- Keep the data key rotation period low (default is one week).

#### Other recommendations

A few other recommendations apply for best security practices:

- Do not switch from encrypted to plaintext, this leaks data keys. When plaintext is selected, all previously encrypted data must be considered reachable.
- Do not copy the encrypted files, as the data keys are not easily available.
- If encryption is desired, start a node with it enabled from the first run, without ever running in plaintext.

Warning:

Note that backups taken with the `BACKUP` statement are not encrypted even if Encryption at Rest is enabled. Encryption at Rest only applies to the CockroachDB node's data on the local disk. If you want encrypted backups, you will need to encrypt your backup files using your preferred encryption method.

### Examples

#### Generating store key files

Cockroach determines which encryption algorithm to use based on the size of the key file. The key file must contain random data making up the key ID (32 bytes) and the actual key (16, 24, or 32 bytes depending on the encryption algorithm).

Algorithm | Key size | Key file size
--- | --- | ---
AES-128 | 128 bits (16 bytes) | 48 bytes
AES-192 | 192 bits (24 bytes) | 56 bytes
AES-256 | 256 bits (32 bytes) | 64 bytes

Generating a key file can be done using the `cockroach` CLI:

    $ cockroach gen encryption-key -s 128 /path/to/my/aes-128.key

Or the equivalent openssl CLI command:

    $ openssl rand -out /path/to/my/aes-128.key 48

#### Starting a node with encryption

Encryption at Rest is configured at node start time using the `--enterprise-encryption` command line flag. The flag specifies the encryption options for one of the stores on the node. If multiple stores exist, the flag must be specified for each store.

The flag takes the form: `--enterprise-encryption=path=<store path>,key=<key file>,old-key=<old key file>,rotation-period=<period>`.

The allowed components in the flag are:

Component | Requirement | Description
--- | --- | ---
`path` | Required | Path of the store to apply encryption to.
`key` | Required | Path to the key file to encrypt data with, or `plain` for plaintext.
`old-key` | Required | Path to the key file the data is encrypted with, or `plain` for plaintext.
`rotation-period` | Optional | How often data keys should be automatically rotated. Default: one week.

The `key` and `old-key` components must always be specified. They allow for transitions between encryption algorithms, and between plaintext and encrypted.

Starting a node for the first time using AES-128 encryption can be done using:

    $ cockroach start --store=cockroach-data --enterprise-encryption=path=cockroach-data,key=/path/to/my/aes-128.key,old-key=plain

Warning:

Once specified for a given store, the `--enterprise-encryption` flag must always be present.

#### Checking encryption status

Encryption status can be seen on the node's stores report, reachable through: `http(s)://nodeaddress:8080/#/reports/stores/local` (or replace `local` with the node ID). For example, if you are running a local cluster, you can see the node's stores report at `https://localhost:8080/#/reports/stores/local`.

The report shows encryption status for all stores on the selected node, including:

- Encryption algorithm.
- Active store key information.
- Active data key information.
- The fraction of files/bytes encrypted using the active data key.

CockroachDB relies on RocksDB compactions to write new files using the latest encryption key. It may take several days for all files to be replaced. Some files are only rewritten at startup, and some keep older copies around, requiring multiple restarts. You can force RocksDB compaction with the `cockroach debug compact` command (the node must first be stopped).

Information about keys is written to the logs, including:

- Active/old key information at startup.
- New key information after data key rotation.

Alternatively, you can use the `cockroach debug encryption-active-key` command to view information about a store's encryption algorithm and store key.

#### Changing encryption algorithm or keys

Encryption type and keys can be changed at any time by restarting the node. To change keys or encryption type, the `key` component of the `--enterprise-encryption` flag is set to the new key, while the key previously used must be specified in the `old-key` component.

For example, we can switch from AES-128 to AES-256 using:

    $ cockroach start --store=cockroach-data --enterprise-encryption=path=cockroach-data,key=/path/to/my/aes-256.key,old-key=/path/to/my/aes-128.key

Upon starting, the node will read the existing data keys using the old encryption key (`aes-128.key`), then rewrite the data keys using the new key (`aes-256.key`). A new data key will be generated to match the desired AES-256 algorithm.

To check that the new key is active, use the stores report page in the Admin UI to check the encryption status.

To disable encryption, specify `key=plain`. The data keys will be stored in plaintext and new data will not be encrypted.

To rotate keys, specify `key=/path/to/my/new-aes-128.key` and `old-key=/path/to/my/old-aes-128.key`. The data keys will be decrypted using the old key and then encrypted using the new key. A new data key will also be generated.

## Encrypted backups (Enterprise)

New in v20.1: You can encrypt full or incremental backups by using the `encryption_passphrase` option. Files written by the backup (including `BACKUP` manifests and data files) are encrypted using the specified passphrase to derive a key. To restore the encrypted backup, the same `encryption_passphrase` option (with the same passphrase) must be included in the `RESTORE` statement.

When used with incremental backups, the `encryption_passphrase` option is applied to all the backup file URLs, which means the same passphrase must be used when appending another incremental backup to an existing backup. Similarly, when used with locality-aware backups, the passphrase provided is applied to files in all localities.

Encryption is done using AES-256-GCM, and GCM is used to both encrypt and authenticate the files. A random salt is used to derive a once-per-backup AES key from the specified passphrase, and then a random initialization vector is used per-file. CockroachDB uses PBKDF2 with 64,000 iterations for the key derivation.

Note:

`BACKUP` and `RESTORE` will use more memory when using encryption, as both the plain-text and cipher-text of a given file are held in memory during encryption and decryption.

For an example of an encrypted backup, see Create an encrypted backup.

## Encryption caveats

### Higher CPU utilization

Enabling Encryption at Rest might result in a higher CPU utilization. We estimate a 5-10% increase in CPU utilization.

### Encryption for touchpoints with other services

- S3 backup encryption
- Encrypted comms with Kafka

## See also

- Client Connection Parameters
- Manual Deployment
- Orchestrated Deployment
- Local Deployment
- Other Cockroach Commands