Encryption in Transit with TLS

TLS gives a listener what it gives HTTPS: the client checks that the broker's certificate was signed by a certificate authority (CA) it trusts and names the host it dialed, then everything is encrypted. In production the CA is your company's or a cloud's; BookNest's compose/secure/certs.sh creates its own with OpenSSL and issues two certificates, one for the broker, whose subject alternative names cover both addresses clients use, and one for a client that will log in with it (mTLS and OAuth):

compose/secure/certs.sh (excerpt): issuing a certificate from the private CAShell

issue() {                                                     # issue <name> <subjectAltName>
  openssl req -newkey rsa:3072 -nodes -subj "$(sub $1)" -keyout $1.key -out $1.csr 2>/dev/null
  openssl x509 -req -in $1.csr -CA ca.pem -CAkey ca.key -days 90 -out $1.crt \
    -extfile <(printf "subjectAltName=%s\nextendedKeyUsage=serverAuth,clientAuth" "$2") 2>/dev/null
  cat $1.key $1.crt > $1-keystore.pem                         # key + chain for Kafka's PEM store
}
issue broker "DNS:l3-kafka-sec,DNS:localhost"
issue booknest-packer "DNS:booknest-packer"                  # a client identity for mutual TLS

Since Kafka 2.7 129 (KIP-651) a broker reads PEM files directly, so you no longer need keytool and the password-protected JKS stores most older guides build. Five lines of server.properties enable TLS for every listener: ssl.keystore.type=PEM, ssl.keystore.location=/certs/broker-keystore.pem, the same two for the truststore (ca.pem), and ssl.enabled.protocols=TLSv1.3. The CONTROLLER listener is encrypted too: in KRaft mode the metadata log carries every credential and ACL (The KRaft Metadata Quorum). OpenSSL shows what a client negotiates (listings/l0614_tls.sh):

Output of 82

$ openssl s_client -connect localhost:33093 -CAfile compose/secure/certs/ca.pem -brief
  </dev/null
Connecting to 127.0.0.1
Can't use SSL_get_servername
CONNECTION ESTABLISHED
Protocol version: TLSv1.3
Ciphersuite: TLS_AES_256_GCM_SHA384
Peer certificate: O=BookNest, CN=broker

Java clients check the host name against the certificate by default (ssl.endpoint.identification.algorithm=https), and librdkafka does since version 2.0, so connecting through an address the certificate does not list fails. TLS also disables Kafka's zero-copy send from page cache to socket (Throughput and Latency): bytes are encrypted in user space.

MENU