git bisect

Finding a Regression with git bisect

Over the afternoon four more migrations and a README section land on main:

The rest of the migration workShell
# Add db/migrations/002-add-isbn.sql
git add db && git commit -qm "Add an isbn column to books"
# Add db/migrations/003-index-genre.sql
git add db && git commit -qm "Index books by genre"
# Add db/migrations/004-widen-price.sql
git add db && git commit -qm "Allow prices up to 999,999.99"
# Add db/migrations/005-add-created-at.sql
git add db && git commit -qm "Record when each book entered the catalog"
# Edit README.md: add a Database migrations section
git commit -qam "Document the migrations folder"
git log --oneline v1.1.0..
Output
c02cab3 Document the migrations folder
340f3db Record when each book entered the catalog
f2b4106 Allow prices up to 999,999.99
a2ce605 Index books by genre
a5f5aa2 Add an isbn column to books
a62d631 Apply numbered SQL migrations from db/migrations
2f90a6c Clamp ?limit= to 1-100 instead of failing with a 500

The tests pass, since they only check that prices are numbers, yet The Quiet Harbor now costs $15 instead of $14.99; 1.1.0 was fine. Sam writes a check that rebuilds the database with the current commit's code, and keeps it outside the repository so it stays put whichever commit is checked out:

A regression check that can run at any commitShell
cat > ../check-price.js <<'END'
// Rebuild the database with this commit's db/ code, then check one known price.
// Exit status for git bisect run: 0 good, 1 bad, 125 this commit cannot be tested.
const db = require(`${process.cwd()}/db`);
(async () => {
  await db.query('DROP SCHEMA public CASCADE; CREATE SCHEMA public');
  await db.init();
  const { rows } = await db.query('SELECT price::text AS price FROM books WHERE id = 1');
  console.log(`book 1 costs ${rows[0].price}`);
  process.exitCode = rows[0].price === '14.99' ? 0 : 1;
})().catch((err) => {
  console.log(`cannot test: ${err.message}`);
  process.exitCode = 125;
}).finally(() => db.close());
END
node ../check-price.js; echo "exit status $?"
Output
book 1 costs 15
exit status 1

Seven commits could be to blame. git bisect does a binary search: you name one bad and one good commit, Git 1,932 checks out the commit halfway between, you test it and say good or bad, and the range halves each time, so about log2(N) tests find the culprit, 10 for a thousand commits (11):

A manual bisect sessionShell
git bisect start main v1.1.0
node ../check-price.js
git bisect good
node ../check-price.js
git bisect bad
node ../check-price.js
git bisect good
git bisect reset
Output
Bisecting: 3 revisions left to test after this (roughly 2 steps)
[a5f5aa26a2a50f0e947b2b01d419fa4753dae30f] Add an isbn column to books
book 1 costs 14.99
Bisecting: 1 revision left to test after this (roughly 1 step)
[f2b410653316fff815ca6c89aa129ad9174d0a42] Allow prices up to 999,999.99
book 1 costs 15
Bisecting: 0 revisions left to test after this (roughly 0 steps)
[a2ce60518618931de609c43945b1b2ed8b588f28] Index books by genre
book 1 costs 14.99
f2b410653316fff815ca6c89aa129ad9174d0a42 is the first 'bad' commit
...

git bisect start <bad> <good> saves two commands. While bisecting, HEAD is detached; git bisect skip passes over an untestable commit, and reset returns you to your branch. To hunt for the commit that fixed something, --term-old=broken --term-new=fixed renames the two words.

Bisect halves the suspect range with each test: three tests for seven commits
Bisect halves the suspect range with each test: three tests for seven commits