Load-Testing a Producer

kafka-producer-perf-test.sh reports throughput and latency percentiles, and --payload-file sends real events. Run it twice: flat out for the ceiling, then at the planned rate for the latency users will see.

Producer throughput ceiling and latency at a planned rateShell

head -20000 data/order_events.jsonl > payloads.txt
docker cp payloads.txt l3-c1:/tmp/payloads.txt >/dev/null
for rate in -1 10000; do
  docker exec l3-c1 /opt/kafka/bin/kafka-producer-perf-test.sh --topic booknest.perf \
    --bootstrap-server l3-b4:9092 --num-records 300000 --throughput $rate \
    --payload-file /tmp/payloads.txt --command-property acks=all linger.ms=5 \
    compression.type=lz4 2>&1 | tail -1 | cut -d, -f2,5,8
done

Output

 22512.381810 records/sec (1.98 MB/sec), 8646 ms 50th, 10298 ms 99.9th.
 9993.670675 records/sec (0.88 MB/sec), 6 ms 50th, 45 ms 99.9th.

Flat out, records queued for seconds; at 10,000 a second the median was 6 ms. The ceiling ranged from 13,000 to 60,000 records a second across runs on this shared host, so compare settings within one session.

MENU