Registering Order Schemas

Registering BookNest's Order Schema

Treat schemas as code: keep them in Git 1,932 , test them in CI (Compatibility Modes), and register them at deployment, with producers set to auto.register.schemas=false. BookNest's step pins the rule, registers version 2 with a defaulted channel, and lands the Avro 129 topic in PostgreSQL 1,289 through AvroConverter:

listings/l0698_register.sh: pin the rule, register version 2, sink the Avro topicShell

# Schemas as code: pin the subject's rule, register version 2, then land the topic with Connect
R=localhost:33081; S=booknest.order-events-avro-value; H='Content-Type: application/json'
curl -s -X PUT -H "$H" -d '{"compatibility": "BACKWARD_TRANSITIVE"}' $R/config/$S >/dev/null
jq -c '.fields += [{name: "channel", type: "string", default: "web"}]' schemas/order_event.avsc |
  jq -Rn '{schema: input}' | curl -s -X POST -H "$H" -d @- "$R/subjects/$S/versions" |
  jq -c '{id, version}'
jq 'with_entries(select(.key | test("^(transforms|value)") | not)) + {
  "topics": "booknest.order-events-avro", "table.name.format": "order_events_avro",
  "value.converter": "io.confluent.connect.avro.AvroConverter",
  "value.converter.schema.registry.url": "http://l3-schema-registry:8081"}' \
  listings/orders-sink.json > avro-sink.json
curl -s -X PUT -H "$H" -d @avro-sink.json localhost:33083/connectors/booknest-avro-sink/config \
  >/dev/null
q() { docker exec l3-pg psql -U booknest -Atc "$1" 2>/dev/null; }
until [ "$(q 'SELECT count(*) FROM order_events_avro')" = 390737 ]; do sleep 2; done
q "SELECT string_agg(column_name || ' ' || udt_name, ', ' ORDER BY ordinal_position)
   FROM information_schema.columns WHERE table_name = 'order_events_avro'"

Output

{"id":4,"version":2}
event_id int8, ts timestamp, type text, order_id int8, customer_id int8, total float8

No connector setting names a schema: AvroConverter fetched version 1 by the records' ID and built the table, ts already a timestamp, with no transforms. Version 2 records would add channel, which the sink's auto.evolve adds as a column.

MENU