Iceberg 129 's refs come in two kinds: a tag names one snapshot (a month-end close, a model's training set) and can carry its own retention; a branch is a line of snapshots that moves independently of main. Branches enable write-audit-publish (WAP): a job writes to a branch, checks run against the branch, and only a passing branch is published to main with fast_forward (or cherrypick_snapshot). Here a rerun of the June 30 load, a classic source of duplicates, goes through WAP:
"""Write-audit-publish: a retried load lands on a branch, fails its audit, stays off main."""
from lake import spark
T = "booknest.orders"
spark.sql(f"ALTER TABLE {T} CREATE TAG `ch08-baseline` RETAIN 365 DAYS") # a named version
spark.sql(f"ALTER TABLE {T} SET TBLPROPERTIES ('write.wap.enabled' = 'true')")
spark.sql(f"ALTER TABLE {T} CREATE BRANCH audit")
retry = spark.table(T).where("order_ts >= '2026-06-30'").localCheckpoint() # rerun of June 30
spark.conf.set("spark.wap.branch", "audit") # 1. write: this session writes to audit
retry.writeTo(T).append()
spark.conf.unset("spark.wap.branch")
rows, ids = spark.sql(f"""SELECT count(*), count(DISTINCT order_id) -- 2. audit the branch
FROM {T} VERSION AS OF 'audit'""").first()
print(f"audit branch: {rows} rows, {ids} distinct order IDs")
if rows == ids: # 3. publish only a clean branch
spark.sql(f"CALL lake.system.fast_forward('{T}', 'main', 'audit')")
else:
print("audit failed: duplicate orders, main left untouched")
spark.sql(f"ALTER TABLE {T} DROP BRANCH audit")
spark.sql(f"ALTER TABLE {T} UNSET TBLPROPERTIES ('write.wap.enabled')")
for r in spark.sql(f"SELECT name, type, snapshot_id FROM {T}.refs").collect():
print(f"ref {r.name:<14} {r.type:<7} {r.snapshot_id}")audit branch: 100195 rows, 100000 distinct order IDs audit failed: duplicate orders, main left untouched ref main BRANCH 4637335077536227122 ref ch08-baseline TAG 4637335077536227122
The branch held 195 duplicated orders; readers of main never saw them, and dropping the branch discarded them. spark.wap.branch routed the writes without changing the job's code, and the tag keeps this state for a year. Data Quality replaces the hand-written check with real data-quality tools.