Reftable

The Reftable Backend and Git 3.0's Ref Storage Change

One file per ref hurts big repositories: each update creates and renames a file, deleting a packed ref rewrites all of packed-refs, multi-ref updates are not atomic, and on case-insensitive disks Fix and fix collide. Reftable, a format Shawn Pearce designed for JGit 427 at Google, stores refs and reflogs in sorted, compressed, append-only tables. Git 2.45 1,932 (April 2024) added it as a backend:

A repository with the reftable backendShell
cd ..
git init -q --ref-format=reftable rt-lab
cd rt-lab
ls .git/reftable
cat .git/HEAD
git config get extensions.refstorage
Output
0x000000000001-0x000000000001-d37a419a.ref
tables.list
ref: refs/heads/.invalid
reftable

Each update appends a small table listed in tables.list, compacted in the background; HEAD is a stub for older tools. Creating, listing and deleting 10,000 tags with git update-ref --stdin on this shared 4-CPU machine shows the difference:

files versus reftable for 10,000 refs
Operation (10,000 refs) files reftable
Create 879 ms 158 ms
List (for-each-ref) 371 ms 244 ms
Delete 797 ms 104 ms

Git 3.0 makes reftable the default for new repositories; git refs migrate --ref-format=reftable (Git 2.46) converts an existing one. Tools that read .git/refs directly instead of calling git for-each-ref break on reftable repositories, one more reason to script against plumbing.