Encrypt Data Source Credentials

Data source credentials and organization’s OIDC client secrets are vital pieces of metadata that must be safeguarded. While in GoodData Cloud we ensure your credentials are encrypted and stored securely on our end, in GoodData.CN the responsibility falls on the deployment administrator. This article explains how to secure your data source credentials in GoodData.CN.

We use the Google Tink library to encrypt this metadata using AES256-GCM authenticated encryption with associated data, with the keyset passed as a metadata-api service configuration parameter.

Follow these steps to ensure your data source credentials stored in GoodData.CN are encrypted:

Create and Manage Keysets
Configure Helm Chart

Create and Manage Keysets

Google Tink provides a key management tool, Tinkey. Use Tinkey to manage your keys.

Install Tinkey

Download and extract the Tinkey key management tool:

curl -fsL https://storage.googleapis.com/tinkey/tinkey-1.9.0.tar.gz | tar xzvf -

Create New Keyset

Generate a new keyset, with one key set as primary and enabled:

tinkey create-keyset --key-template AES256_GCM --out output_keyset.json

You can now configure the Helm chart.

Rotate Keysets

Key rotation is essential for maintaining the security of distributed systems. However, this process can be prone to race conditions. Follow these steps to ensure a smooth rotation:

Add New Key
Promote New Key
Encrypt Secrets
(Optional) Remove the oldest key

Add New Key

Add a new key to the keyset and propagate this change to all metadata-api services.

Steps:

Identify all existing keys in keyset:

tinkey list-keyset --in live_keyset.json |grep '  key_id:'

The output may look something like this:

key_id: 1088398113
key_id: 343627806

Add a new key to keyset:

tinkey add-key --key-template AES256_GCM --in live_keyset.json --out added_key_keyset.json

Perform a check by re-identifiying all existing keys in keyset:
```
tinkey list-keyset --in added_key_keyset.json |grep '  key_id:'
```
The output may look something like this:
```
key_id: 1088398113
key_id: 343627806
key_id: 1292234114
```
where key_id: 1292234114 is the new key.

Distribute the updated added_key_keyset.json to GoodData.CN deployments. Ensure all metadata-api services restart after the Secret is modified.

Promote New Key

Promoting a key means making it the primary key for encryption. After adding a new key, you’ll need to promote it:

tinkey promote-key --key-id <RECENTLY_ADDED_KEY_ID> --in added_key_keyset.json --out rotated_keyset.json

Replace <RECENTLY_ADDED_KEY_ID> with the actual ID of the recently added key (e.g., 1292234114 from the previous step).

Distribute the rotated_keyset.json to GoodData.CN deployments and ensure all metadata-api services restart.

Encrypt Secrets

With the new primary key set, older secrets remain encrypted with previous keys. Re-encrypt old data to ensure outdated keys aren’t utilized:

See Encrypt Credentials.

(Optional) Remove the Oldest Key

As the keyset grows with each rotation, it might be efficient to limit the number of keys by removing older ones. However, remember:

Ensure the key isn’t encrypting any secret.
SQL-level DB backups encrypted with a removed key will be rendered unusable.

tinkey delete-key --key-id <OLDEST_KEY_ID> --in added_key_keyset.json --out removed_old_key_keyset.json

Other Useful Commands

You may also use the following commands to list and delete keys:

List Keys

To view all keys in the keyset:

tinkey list-keyset --in keyset.json

The output, for example, shows the key ids and indicates the primary key:

primary_key_id: 343627806
key_info {
  type_url: "type.googleapis.com/google.crypto.tink.AesGcmKey"
  status: ENABLED
  key_id: 1088398113
  output_prefix_type: TINK
}
key_info {
  type_url: "type.googleapis.com/google.crypto.tink.AesGcmKey"
  status: ENABLED
  key_id: 343627806
  output_prefix_type: TINK
}

Delete Key

Reduce the keyset size by removing a key. Before doing so, confirm all data encrypted by this key has been re-encrypted by the current primary key:

tinkey delete-key --key-set input_keyset.json --key-id number --out output_keyset.json

Configure Helm Chart

Once you have your Tink keyset, update the following parameters in the helm chart options for GoodData.CN to enable the encryption of data source credentials in your GoodData.CN deployment:

metadataApi.encryptor.enabled
Enable or disable the use of encryption for your data source credentials. Enabled by default.
Encryption is Irreversible!
Be aware that once you enable encryption, your credentials become irreversibly encrypted. If you later change the `metadataApi.encryptor.enabled` setting from `true` to `false`, your GoodData.CN deployment will become unusable. To restore functionality, you will need to update all secrets via the API to store them in an unencrypted format. Additionally, because OAuth2 client credentials are also encrypted, users will lose access to the UI. However, token authentication will remain functional.
metadataApi.encryptor.existingSecret
Store your Tink keyset as a secret here. See Secret Management.
metadataApi.encryptor.keySet
Or store your Tink keyset as plain text here.

For security reasons we recommend you store your keyset using a secret. In case both existingSecret and keySet fields are populated, the backend will use existingSecret.

Note that if the encryption is enabled, but no keyset is provided, the metadata-api deployment will fail.

Encrypt Credentials

Whenever you store a new keyset, you have to (re)encrypt your data source credentials by calling the API endpoint http://localhost:9008/actuator/reencrypt:

kubectl exec <pod> -n <namespace> -- /bin/bash -c "curl -X POST -v http://localhost:9008/actuator/reencrypt"

For example:

kubectl -n gooddata-cn exec -c metadata-api deployments/gooddata-cn-metadata-api -- curl -X POST -v http://localhost:9008/actuator/reencrypt

Enable CSP