Securing the Cluster

Securing BookNest's Cluster End to End

Put together, l3-kafka-sec has four encrypted listeners, each with its own way of establishing who is calling, and one authorizer behind them:

BookNest's secured broker: encrypted listeners, principals and one authorizer

secure_clients.py then tries BookNest's Python clients (confluent-kafka 2.15.1) without TLS, with a wrong password, as each user, and with the packer's certificate. Only the connection settings differ between methods; mtls() sets security.protocol to SSL with ssl.certificate.location and ssl.key.location instead:

listings/secure_clients.py (excerpt): one client configuration per methodPython

TLS = {"ssl.ca.location": str(CERTS / "ca.pem")}            # trust BookNest's private CA
def scram(user, password=None):                             # SASL/SCRAM over TLS, port 33093
    return {"bootstrap.servers": "localhost:33093", "security.protocol": "SASL_SSL",
            "sasl.mechanisms": "SCRAM-SHA-512", "sasl.username": user,
            "sasl.password": password or PW[user], **TLS}
...
produce("plaintext client", {"bootstrap.servers": "localhost:33093"})
produce("wrong password", scram("producer", "guess"))
produce("producer", scram("producer"))
produce("analytics", scram("analytics"))
consume("analytics", scram("analytics"), "booknest-analytics")
consume("analytics", scram("analytics"), "booknest-packers")
consume("packer (mTLS)", mtls("booknest-packer"), "booknest-packers")

Output

plaintext client produce: 1/1 brokers are down
wrong password produce: SASL authentication error: Authentication failed during authentication
  due to invalid credentials with SASL mechanism SCRAM-SHA-512
producer       produce: ok, offset 0
analytics      produce: Failed to acquire idempotence PID: Broker: Cluster authorization failed
analytics      consume: key 1001
analytics      consume: FindCoordinator response error: Group authorization failed.
packer (mTLS)  consume: key 1001

Each layer refused what it should. The broker dropped the plaintext client, which librdkafka reports only as "brokers are down". analytics may read but not write, and how its write failed matters: an idempotent producer (the default since Kafka 3.0 129 ) first asks for a producer ID, which needs Write on some topic or IdempotentWrite on the cluster, so a read-only principal fails with a cluster authorization error before any topic is checked. It could not borrow the packers' group either; the packer read the order with its certificate alone. listings/run_0614.sh runs the whole chain.

MENU