Encryption at Rest

Encryption at Rest

Encryption at Rest for the Secrets API

By default the API server writes Secrets to etcd 137,357 unencrypted, so an etcd backup or a stolen disk leaks every credential. An EncryptionConfiguration file, passed with --encryption-provider-config, makes the API server encrypt chosen resources before they reach etcd. On kind 14,561 the API server is a static Pod: copy the file into the node, add the flag and a mount to its manifest, and the kubelet restarts it. etcd is etcd's function:

Turning on secretbox encryption for Secrets and proving it in etcdYAML

etcd get /registry/secrets/booknest/demo-secret --print-value-only | grep -c 's3cr3t!'
cat > ~/encryption.yaml <<EOF
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- resources: [secrets]
  providers:
  - secretbox: { keys: [{ name: key1, secret: $(head -c 32 /dev/urandom | base64) }] }
  - identity: {}
EOF
N=l3-booknest-control-plane; M=/etc/kubernetes/manifests/kube-apiserver.yaml
export F=/etc/kubernetes/enc
docker exec $N mkdir -p $F && docker cp -q ~/encryption.yaml $N:$F/
docker exec $N cat $M | yq '
  .spec.containers[0].command += "--encryption-provider-config=" + strenv(F) + "/encryption.yaml" |
  .spec.containers[0].volumeMounts += {"name": "enc", "mountPath": strenv(F)} |
  .spec.volumes += {"name": "enc", "hostPath": {"path": strenv(F)}}' > ~/kube-apiserver.yaml
docker cp -q ~/kube-apiserver.yaml $N:$M
sleep 20; until kubectl get --raw /readyz >/dev/null 2>&1; do sleep 2; done
kubectl get secrets -A -o json | kubectl replace -f - | wc -l
etcd get /registry/secrets/booknest/demo-secret --print-value-only | head -c 26; echo

Output

1
4
k8s:enc:secretbox:v1:key1:

The password was stored in the clear; after the restart, kubectl 5,150 replace rewrote all four Secrets and each value starts with the provider and key name. The first provider encrypts and the rest can still decrypt, so to rotate, put key2 first, restart, rewrite all Secrets and remove key1; identity keeps old plaintext entries readable.

The key sits on the control-plane node, so this protects backups and disks, not a compromised node. The documentation rates secretbox strong and advises against aescbc and aesgcm; strongest is KMS v2 (stable since 1.29), where an external key service such as AWS 24 KMS or HashiCorp Vault 4,548 holds the key-encryption key.

MENU