Managing Topics

Managing Topics

Creating, Describing and Altering Topics

kafka-topics.sh creates, lists, describes, alters and deletes topics. The listing creates BookNest's customer topic with two per-topic settings, describes it, and tries to shrink and grow it.

Creating, describing and altering the customers topic

# Create the compacted customers topic, inspect it, and try two changes
K="docker exec -i l3-kafka /opt/kafka/bin"
B="--bootstrap-server l3-kafka:9092"
T="--topic booknest.customers"
$K/kafka-topics.sh $B $T --create --partitions 3 --replication-factor 1 \
  --config cleanup.policy=compact --config min.compaction.lag.ms=60000 | grep -v ^WARNING
$K/kafka-topics.sh $B $T --describe | tr '\t' ' ' | head -1 |
  sed -E 's/ TopicId: [^ ]+//; s/ Elr:.*//; s/ Configs: /\n  Configs: /'
$K/kafka-topics.sh $B $T --alter --partitions 2 2>&1 | grep -o 'InvalidPartitions.*'
$K/kafka-topics.sh $B $T --alter --partitions 6

Output

Created topic booknest.customers.
Topic: booknest.customers PartitionCount: 3 ReplicationFactor: 1
  Configs: min.insync.replicas=1,cleanup.policy=compact,min.compaction.lag.ms=60000
InvalidPartitionsException: The topic booknest.customers currently has 3 partition(s); 2
  would...

Kafka 129 can add partitions but never remove them, since each is a log with its own offsets; growing to six succeeded silently. On a keyed topic that sends new events for old keys elsewhere (Topics and Partitions). The filtered warning flags names mixing . and _, which collide in metric names.

MENU