Producers compress whole batches, and brokers store them unchanged while the topic's compression.type is producer (the default). listings/l0655_codecs.sh sent all 390,737 events once per codec, with linger.ms=50, and measured each log file on disk.
| Codec | Log size | Ratio | Producer rate |
|---|---|---|---|
| none | 42.7 MB | 1.0x | 236,000-433,000 msg/s |
| gzip | 7.6 MB | 5.6x | 94,000-147,000 msg/s |
| snappy | 12.2 MB | 3.5x | 283,000-421,000 msg/s |
| lz4 | 12.8 MB | 3.3x | 318,000-417,000 msg/s |
| zstd 126 | 8.1 MB | 5.3x | 330,000-442,000 msg/s |
zstd (level 3 by default) came within 7% of gzip's size at no speed cost, while gzip ran at about a third of the rate. Prefer zstd for event streams.