Offsets and Time-Based Lookup

Each partition has a log start offset (oldest record kept), a log end offset (the next offset to be written) and between them the high watermark (High Watermark and Epochs). kafka-get-offsets.sh --time takes -2 (earliest), -1 (latest) or a timestamp in milliseconds, for which the broker returns, from the .timeindex files, the first offset at or after that time: how a consumer rewinds to a date.

Earliest, latest and time-based offsets, and the records around a timestamp

# Offsets by position and by time: where 2026 begins in each partition
K="docker exec l3-c1 /opt/kafka/bin"
B="--bootstrap-server l3-b4:9092"
T="--topic booknest.order-events"
new_year="$(date -ud 2026-01-01 +%s)000"                  # epoch milliseconds, UTC
for t in -2 -1 "$new_year"; do                            # earliest, latest, a timestamp
  echo "--time $t: $($K/kafka-get-offsets.sh $B $T --time $t | sort | cut -d: -f3 | xargs)"
done
p0=$($K/kafka-get-offsets.sh $B $T --time "$new_year" | grep ':0:' | cut -d: -f3)
$K/kafka-console-consumer.sh $B $T --partition 0 --offset $((p0 - 1)) --max-messages 2 \
  --formatter-property print.offset=true --formatter-property print.timestamp=true \
  2>/dev/null | cut -c1-95

Output

--time -2: 0 0 0
--time -1: 128971 131080 130686
--time 1767225600000: 85788 87147 87488
CreateTime:1767225060000   Offset:85787   {"event_id":260419,"ts":"2025-12-31T23:51:00Z","type"
  :"or
CreateTime:1767225774000   Offset:85788   {"event_id":260425,"ts":"2026-01-01T00:02:54Z","type"
  :"or

Offset 85,788 is partition 0's first event of 2026. Lookups are only as good as the timestamps: with the default message.timestamp.type=CreateTime the producer sets them; LogAppendTime has the broker stamp each batch. Kafka 4 129 .x rejects timestamps over an hour in the future (message.timestamp.after.max.ms) but accepts any age.

MENU