Secrets & encryption

← Documentation · Welcome

Your Git repo is world-readable to the platform and possibly other people - so secret values are encrypted with SOPS and age before they ever touch Git. This page covers the one-time setup (your personal key) and the day-to-day commands.

How it works

• Secret values live in one encrypted file per environment at the repo root: test.sops.yaml and prod.sops.yaml, as flat sops:KEY: value entries. Only the values are encrypted - the key names stay readable, so diffs still tell you what changed.
• Your Kubernetes Secret manifests never contain values, only <sops:KEY> placeholders plus an annotation pointing at the encrypted file.
• At sync time, ArgoCD (via the ArgoCD Vault Plugin) decrypts the file with the cluster's private key and substitutes the real values into your namespace.

age encryption is asymmetric: a file is encrypted to a list of public keys ("recipients"), and anyone holding a matching private key can decrypt it. The cluster key can decrypt, but you don't have its private half - which is why you need a personal keypair as an additional recipient. Otherwise you can encrypt new values but never read or edit the file again.

1. Install sops and age

Both are single binaries, packaged for most distros:

• sops - your package manager (apt install sops, brew install sops, …) or the official releases (install docs).
• age - your package manager (apt install age, brew install age, …) or the install instructions.

2. Create your personal age keypair

mkdir -p ~/.config/sops/age
age-keygen -o ~/.config/sops/age/keys.txt

This prints your public key (age1...) - that's what goes into .sops.yaml below, and it's safe to share anywhere. The file holds your private key: keep it out of Git, back it up, and treat it like an SSH key. sops looks in ~/.config/sops/age/keys.txt by default, so no further configuration is needed. Re-print your public key anytime with:

age-keygen -y ~/.config/sops/age/keys.txt

Optional: store the private key in GNOME Keyring

If you'd rather not keep the private key in a plaintext file, store it in your login keyring (unlocked with your session) and let sops fetch it on demand:

# generate straight into the keyring (never touches disk):
age-keygen | secret-tool store --label="sops age key" sops age-key

# print the public key from the stored secret:
secret-tool lookup sops age-key | age-keygen -y -

# tell sops how to fetch the key (add to your ~/.bashrc / ~/.zshrc; needs sops >= 3.10):
export SOPS_AGE_KEY_CMD="secret-tool lookup sops age-key"

Alternatively, encrypt the key file itself with a passphrase (age-keygen | age -p -o ~/.config/sops/age/keys.txt.age) and decrypt it manually into SOPS_AGE_KEY when you need it.

3. Configure recipients - .sops.yaml

.sops.yaml at the repo root tells sops which files to encrypt to which keys. Your repo ships with the cluster keys already in place - add your team's personal public keys so you can decrypt and edit the files later.

The clusters' public keys:

(Deploying to a different cluster? Its key is on that cluster's start page, in your repo's .sops.yaml, or ask support@inf.ethz.ch.)

Best practice: one variables file per environment, matching the manifest folder - test/ → test.sops.yaml, prod/ → prod.sops.yaml. Each cluster has its own key, and this keeps prod secrets undecryptable by the test cluster (and vice versa):

---
stores:
  yaml: {indent: 2}
creation_rules:
  - path_regex: "test\\.sops\\.yaml$"
    encrypted_regex: "^sops:.*$"    # only encrypt the sops:* values
    age:
      # staging cluster - k8s-isg-test
      - age1rz34zr0qa4z8aakpe8crv6aa537dqan483v35v54d0ac6r00fagsfrewxu
      # alice
      - age1<alice's public key>
      # bob
      - age1<bob's public key>
  - path_regex: "prod\\.sops\\.yaml$"
    encrypted_regex: "^sops:.*$"
    age:
      # production cluster - k8s-isg
      - age1c2uvxzjsde6ddm2c80jfj7k255p43f7tamx5jc3rgnh74w0jpslsufaf3h
      # alice
      - age1<alice's public key>

.sops.yaml only affects new encryption operations. After changing recipients, re-wrap the existing files and commit them:

sops updatekeys test.sops.yaml
sops updatekeys prod.sops.yaml

Someone left the team? Remove their key, run sops updatekeys, then rotate the encryption key with sops rotate -i test.sops.yaml - and change the secret values themselves: old Git revisions remain decryptable with the removed key forever.

4. Create, edit, and inspect the encrypted files

All of these encrypt/decrypt transparently using .sops.yaml:

sops test.sops.yaml                                          # create or edit: opens
                                                             # decrypted in $EDITOR,
                                                             # re-encrypts on save
sops -d test.sops.yaml                                       # view the whole file
sops -d --extract '["sops:DB_PASSWORD"]' test.sops.yaml      # view one value
sops set test.sops.yaml '["sops:DB_PASSWORD"]' '"s3cret"'    # set one value (scripts/CI)

The file content is flat key/value, every key prefixed with sops::

sops:DB_PASSWORD: s3cret
sops:API_TOKEN: abc123

Before committing, glance at git diff - every value must show as ENC[AES256_GCM,...]. If you can read a value in the diff, do not commit. The encrypted file (plus the sops metadata block it appends) is exactly what belongs in Git.

Without a personal key among the recipients you could still create the file (public keys suffice to encrypt) - but never read or change it again short of keeping a plaintext copy outside Git. Don't go there: set up your personal key first.

5. Use a secret in a Kubernetes resource

Directly, with the annotation

Reference values from your manifests with <sops:KEY> placeholders and tell the platform which file they live in via the avp.kubernetes.io/path annotation (the path is relative to the manifest's folder):

apiVersion: v1
kind: Secret
metadata:
  name: my-app
  annotations:
    avp.kubernetes.io/path: ../test.sops.yaml
type: Opaque
stringData:
  DB_PASSWORD: <sops:DB_PASSWORD>
  API_TOKEN: <sops:API_TOKEN>

Commit exactly this - the placeholders are substituted in the cluster at sync time and never appear resolved in Git. Two gotchas:

• Placeholders are only substituted in resources that carry the annotation.
• The placeholder wraps the full key name from the variables file, including the sops: prefix.

Via Helm

If you render your manifests from a Helm chart, nothing changes conceptually: put the placeholders in your values file and make sure the chart adds the annotation to the Secret, then commit the rendered output. For example (values file):

myapp:
  secretAnnotations:
    avp.kubernetes.io/path: "../test.sops.yaml"
  env:
    - name: DB_PASSWORD
      value: <sops:DB_PASSWORD>

helm template passes the placeholders through as literal strings into test/all.yaml, where AVP resolves them at sync time - see isg-example for a worked example of this pattern.

Quick reference