Encryption

Encryption

Encryption at Rest and in Transit

In transit, every hop (client to engine, engine to catalog and object store) should use TLS, or anyone on the path reads the Parquet 129 bytes and the keys in request headers. At rest, the object store encrypts each object before it reaches a drive: S3 offers SSE-S3 (service-managed keys, the AWS 24 default since January 2023), SSE-KMS (your key in a key management service, with audit and revocation) and SSE-C (the client sends the key). MinIO 30,943 implements the same headers with keys from a KMS; for a demo, MINIO_KMS_SECRET_KEY sets one static key. security/minio_sec_up.sh starts such a server with a self-signed certificate, and then:

security/encrypt.sh: the customer file on TLS plus SSE-S3, compared with l1-minioShell

# Store the customer file on a MinIO with TLS and SSE-S3; compare raw drives with l1-minio.
bash "$DEMOS/security/minio_sec_up.sh"     # self-signed certificate, MINIO_KMS_SECRET_KEY
mc alias set -q sec https://localhost:31910 booknest-admin booknest-secret-2026 >/dev/null
mc mb -q sec/booknest-raw >/dev/null && mc encrypt set sse-s3 sec/booknest-raw
mc cp -q "$DEMOS/../ch03/data/customers.jsonl" sec/booknest-raw/shop/ >/dev/null
mc stat sec/booknest-raw/shop/customers.jsonl | grep -i encrypt
curl -sv --cacert sec/certs/public.crt https://localhost:31910/minio/health/live 2>&1 |
  grep -o "SSL connection using .*"
for c in l1-minio l1-minio-sec; do                 # look for one email in the drives' files
  echo "$c: $(docker exec $c grep -rl gus.evans12@example.com /data | wc -l) file(s) in clear"
done

Output

Auto encryption configuration has been set successfully for sec/booknest-raw
Encryption: SSE-S3
SSL connection using TLSv1.3 / TLS_AES_128_GCM_SHA256 / X25519MLKEM768 / RSASSA-PSS
l1-minio: 1 file(s) in clear
l1-minio-sec: 0 file(s) in clear

On the plain server a customer's email sits readable in a drive file; on the encrypted one it does not, and TLS 1.3 even negotiated the post-quantum hybrid key exchange X25519MLKEM768. It defeats stolen drives, not stolen access keys: whoever may GetObject gets plaintext. Keep keys in a real KMS (HashiCorp Vault, AWS KMS) and rotate them; Parquet modular encryption can protect single columns, such as the MyKad number, under separate keys.

MENU