Dead Letter Queues

Dead Letter Queues

Dead Letter Queues and Error Handling

By default one record that cannot be converted or transformed fails its task, as in Converters, blocking the partition. A sink can instead skip such records into a dead letter queue (DLQ) topic, with the error in the record's headers:

listings/l0688_dlq.sh: send failed records to booknest.dlq.order-eventsShell

# PATCH the sink to tolerate bad records and send them to a dead letter queue topic
C=localhost:33083/connectors/booknest-orders-sink
curl -s -X PATCH -H 'Content-Type: application/json' $C/config -d '{"errors.tolerance": "all",
  "errors.deadletterqueue.topic.name": "booknest.dlq.order-events",
  "errors.deadletterqueue.topic.replication.factor": "1",
  "errors.deadletterqueue.context.headers.enable": "true"}' >/dev/null && sleep 10
E='"type":"order_paid"'                                   # one good event and three bad ones
printf '%s\n' "900001|{\"event_id\":900001,\"ts\":\"2026-10-02T09:00:00Z\",$E,\"order_id\":7}" \
  '900002|not json' "900003|{\"event_id\":900003,\"ts\":\"2 Oct 2026\",$E,\"order_id\":7}" \
  "900004|{\"event_id\":900004,\"ts\":\"2026-10-02T09:00:00Z\",$E}" |
  docker exec -i l3-kafka /opt/kafka/bin/kafka-console-producer.sh --bootstrap-server \
    l3-kafka:9092 --topic booknest.order-events --reader-property parse.key=true \
    --reader-property key.separator='|'
sleep 15
docker exec l3-pg psql -U booknest -Atc "SELECT * FROM order_events WHERE event_id > 900000"
docker exec l3-kafka /opt/kafka/bin/kafka-console-consumer.sh --bootstrap-server l3-kafka:9092 \
  --topic booknest.dlq.order-events --from-beginning --timeout-ms 15000 \
  --formatter-property print.headers=true 2>/dev/null |       # stage and error of each record
  grep -o 'errors.stage:[A-Z_]*\|exception.message:[^,]\{0,60\}' | sed 's/^[a-z.]*://' | paste - -

Output

900001|2026-10-02 09:00:00|order_paid|7||
TRANSFORMATION   Could not parse timestamp: value (2 Oct 2026) does not match
VALUE_CONVERTER   Invalid null value for required INT64 field
VALUE_CONVERTER   Converting byte[] to Kafka Connect data failed due to serial

PATCH (Kafka 3.8 129 +) changed four keys. The good event reached PostgreSQL 1,289 ; the bad JSON and the missing order_id failed in the converter, the date in TimestampConverter, and the tasks kept running. Alert on DLQ traffic, or tolerance becomes silent data loss. Sinks must use Connect's errant-record reporter for their own failures to reach the DLQ; the Aiven JDBC sink does not, so a failing INSERT is retried max.retries (10) times, then fails the task. Sources have no DLQ.

MENU