Time Travel and VACUUM

Delta Lake's Time Travel and VACUUM

Every version stays readable until its files are deleted: VERSION AS OF or TIMESTAMP AS OF in SQL, DeltaTable(uri, version=n) in delta-rs. Removed files linger until VACUUM deletes those no longer referenced by the current version and older than the retention, seven days by default (delta.deletedFileRetentionDuration):

rs_versions.py: delete, read the old version, vacuum it awayPython

"""Delta time travel and VACUUM with delta-rs: old versions live until VACUUM deletes them."""
from deltalake import DeltaTable
from rs_conf import S3
URI = "s3://warehouse/delta/orders"
rows = lambda **kw: DeltaTable(URI, storage_options=S3, **kw).to_pyarrow_dataset().count_rows()
dt = DeltaTable(URI, storage_options=S3)
dt.delete("status = 'pending'")                    # version 1: June's file is rewritten
print("now", rows(), "rows; version 0 had", rows(version=0))
print("vacuum, 7-day retention:", dt.vacuum(dry_run=True))
gone = dt.vacuum(retention_hours=0, enforce_retention_duration=False, dry_run=False)
print("vacuum, no retention:", [p.split("/")[-2] for p in gone])
try:
    rows(version=0)
except Exception as e:
    print("version 0 now:", type(e).__name__, str(e)[:60])

Output

now 99996 rows; version 0 had 100000
vacuum, 7-day retention: []
vacuum, no retention: ['order_month=2026-06']
version 0 now: FileNotFoundError Object at location delta/orders/order_month=2026-06/part-000

The delete rewrote June's file (delta-rs reads deletion vectors but rewrites on delete). With the default retention VACUUM removed nothing; forced to zero, it deleted the old June file, and version 0 broke. Keep the retention longer than your longest reader.

MENU