Shaurya Singh

← writing

Git Is Not the Ground Truth

A merge-base six months old and a git log that lied about how far apart two lineages actually were. Reconciliation needed blob hashes, not branch diffs.

reconciled · forensic audit complete, disposition not yet executed

reconciled · forensic audit complete — disposition plan drafted, execution pending

Two copies of the same product exist right now, six months apart, and neither one is behind. One runs on a Mac Pro that’s been powered on since February, taking real production traffic. The other lives on GitHub, where most of the newer work actually landed. Somewhere between the two, git stopped being able to answer the question “what code exists.”

That’s not a metaphor. git log on either side gives you a true, complete answer to “what’s in this branch’s history,” and a false, incomplete answer to “what code exists that matters.” The gap between those two questions is this whole piece.

Where git quietly stops being complete

Git’s model assumes one thing that a long-lived, disconnected fork quietly breaks: that everything worth tracking eventually gets committed, and everything committed eventually gets pushed. Break the always-connected assumption for six months — a box that stays on and gets worked on directly, syncing with a remote only sporadically — and three failure modes show up that a normal git diff main..feature will never surface.

First: work that never became a commit at all. A directory of scripts, hand-tuned against real usage, living on disk, never git init’d. Second: a branch that did get pushed, did get reviewed, and then had its remote deleted when the review closed unmerged — so the only surviving copy of that work now hangs off a ref that a routine branch listing won’t even show. Third, and worst: two migration histories that both kept moving, independently, and each created database objects the other side already has. None of these three show up if you trust the log. All three showed up here.

The number that was wrong by ten

The starting claim, going into this, was that the box’s branch was “88 commits ahead of main.” Reasonable-sounding, round-ish, repeated a few times before anyone actually measured it.

It was wrong, and not in the direction anyone guessed:

claimed:  ric-v2 is 88 commits ahead of main
measured: merge-base 11e6e81de (2026-02-26)
          98 commits unique to ric-v2
          96 commits unique to main

“Ahead” was the wrong shape of claim before it was even a wrong number. Ninety-eight commits exist only on the box. Ninety-six exist only on GitHub. Neither side is ahead of the other — they’re both ahead, along different axes, from a common point six months back. A single “ahead by N” number can’t represent that; the honest representation is two counts and a shared ancestor, and the ancestor itself is a fact worth pinning, because how far back the two diverged turns out to matter more than how many commits once you’re deciding what’s safe to discard.

Hash the blob, not the branch

Once the shape of the divergence is right, the actual reconciliation question is narrower than it sounds: for every piece of code that exists anywhere — on disk, in a working tree, in a closed branch — does an identical copy exist somewhere in the other side’s full ref set. Not “does it exist on main.” Anywhere: open branches, closed branches, tags, pull-request refs, reflog. A diff between two branch tips answers a much smaller question than that, and answering the smaller question first is how you end up confidently deleting something that’s actually unique.

The mechanical version of this is unglamorous: take every candidate file, hash its content, and check that hash against every blob reachable from every ref on the remote — not just the ones checked out locally, not just the ones with recent activity.

shaurya@sps ~ % git cat-file -e <blob-sha> 2>/dev/null \
  && echo "byte-identical copy exists in some ref" \
  || echo "no match anywhere in the ref set"

Run that against a directory instead of a branch diff, and the question changes from “did main move past me” to the actual one: “does this exact content exist anywhere I haven’t looked yet.” The two questions produce different answers more often than you’d expect.

The clearest case: a telephony bridge, about 14.9KB across four files, hand-tuned against real calls — voice settings, endpointing thresholds, the kind of values you only get right by listening to actual production traffic. It had never been a git repository. No commit, on either side, anywhere, matched any of its bytes. Days of tuning work, sitting as plain files on one machine, with git having nothing to say about it one way or the other — not behind, not ahead, just absent from the record entirely. The fix wasn’t a merge. It was extraction: commit it fresh, using a scrub pattern already proven on an earlier piece of code, so secrets never enter history in the first place. Rescuing it wasn’t optional — a disk failure would have taken it with zero recovery path, because the one thing supposed to make code recoverable, git, had never had a chance to see it.

Migrations don’t diff — they replay, and replay can fail both ways

Database migrations are the sharpest version of “neither side is a superset,” because unlike application code, a migration isn’t just present or absent — it’s ordered, and replaying it against the wrong starting state doesn’t just fail to help, it can fail to run at all.

The live database on the box had 118 applied migrations, zero failed. Six of those existed only on the box, including one that created a persistence table for a feature GitHub’s main branch has no model for at all. Main, meanwhile, had 41 migrations the box had never applied — and one of those 41 tries to create tables under the same names the box-only migrations had already created.

box-only migrations:   6   (creates a table main has no model for at all)
main-only migrations: 41   (one of them creates tables under the same
                             names the box-only migrations already used)
result:  prisma migrate deploy fails in both directions

That’s not a merge conflict you resolve by picking a side. Running main’s migrations against the box’s database fails, because the tables already exist. Running the box’s migrations against a database built from main also fails, for the mirror reason. The only way through is a deliberate baseline step — telling the migration tool “these specific migrations are already satisfied, don’t replay them” — and that knowledge, right now, exists in exactly one place: whoever traced through both migration histories by hand. Nothing about cutover here is a standard deploy. It’s closer to reconciling two ledgers that both independently decided to number their entries the same way.

Stranded on a review that closed

The third failure mode is quieter and easier to lose entirely: work that did get pushed, did get reviewed, and then became unreachable through any normal path because the review closed without merging and the branch got deleted afterward. Git still has the commits — a closed, unmerged review keeps its own ref alive even after the branch is gone — but nothing about a routine branch listing shows that ref exists, and nobody goes looking for code behind a review they remember as closed.

One runtime component split exactly this way. Eighty-two of its 113 files turned out to be byte-identical to a copy already sitting behind that closed review — a copy that was actually ahead of what the box had, because it included a scrub pass that had stripped real phone numbers and internal hostnames before the code ever reached a shared history. Sixteen more were the reverse: the box’s original, pre-scrub versions of the same files, which is exactly the set that should never make that trip. Knowing which sixteen to leave behind only works because the byte-identity check gave a per-file answer, not a per-branch one — not “GitHub is ahead” or “the box is ahead,” but a file-by-file split that happened to run in opposite directions. That’s 98 of the 113 accounted for by hash. The remaining fifteen haven’t been run through the same check yet — UNKNOWN, not zero, and I’m not folding them into either bucket until they have been.

What counts as unshipped, measured properly

Touch 5,961 files at once and most of that number is noise — in this case, 5,866 of them were generated seed data, safe to exclude without individually checking each one. The real question sits in the remaining files, and even there, “changed” doesn’t mean “missing.” The same byte-identity check that found the bridge and the stranded runtime cleared most of those candidates automatically too: 37 turned out to be byte-identical to some ref’s current tip, 7 more existed somewhere in history even without being at a tip. Both groups were already safe, whatever the branch diff made them look like.

files touched:              5,961
generated seed data:        5,866   (excluded, not individually checked)
candidates:                    95
byte-identical to a tip:        37   (already safe)
found somewhere in history:      7   (already safe)

What was left after both passes sorted into three real categories: code with no counterpart anywhere — a reducer module and its test suite, whose inputs had shipped separately but whose actual logic hadn’t; documentation of failures nobody had written down anywhere else — a runbook naming four platform flags that had each broken the system silently, plus how each one was actually diagnosed; and tooling that only existed because someone had needed it once and never circled back to land it properly. None of those three categories is visible from a branch diff, because a branch diff only shows files that exist in both trees and differ. It can’t show a file that one tree has and the other tree’s history was never even checked against.

The arithmetic above says 95 candidates. An earlier working note from the same audit says 96. I haven’t re-run the count to find which one dropped a file or double-counted one, so I’m not going to quietly pick the rounder-sounding number and move on — that gap is UNKNOWN until someone re-runs the filter, not resolved.

Treat “neither side is a superset” as the default

The instinct, coming into a reconciliation like this, is to assume one side is basically current and the other side is basically behind, and the job is catching the behind one up. That instinct is what produced “88 commits ahead” in the first place — a plausible-sounding, directional claim that turned out to be the wrong shape of claim entirely.

The corrected default has to be the opposite: assume neither tree is a superset of the other until you’ve disproven it, per file, by hash — not by branch, not by folder, not by “this looks newer.” Everything downstream of that default falls out cleanly. Some code gets a fresh PR, some gets a straightforward re-push, some genuinely needs to stay exactly where it is because the other side’s version is the correct one and moving it would be the mistake. That last category is a real decision, not an oversight — pre-scrub files, an environment placeholder, anything that only exists on the box because it should only exist on the box. A disposition plan without a “deliberately leave here” column isn’t finished. It’s just optimistic.

What I don’t know yet

There’s one loose end I haven’t closed. A local-only branch, walking its full object set, reaches a particular server file’s exact blob — the hash is there — but that blob doesn’t appear at that path in any commit the branch actually contains. Something put that content into the object store without a commit ever pointing at it that way. It doesn’t change anything measured above, because everything here came from checking bytes directly rather than trusting any one branch’s history. But I don’t know how that blob got orphaned like that, and I’m not deleting the branch until I do.

I also don’t know if the migration baseline gets documented before someone runs a plain deploy against this database out of habit. The knowledge exists in one person’s head right now, and “documenting it is the fix” is true and also exactly the kind of fix that’s easy to keep meaning to get to.

And I don’t know whether this method — hash every blob, assume neither side is complete — actually needed six months of drift to become necessary, or whether it’s just as true after six weeks and nobody had measured it yet at that scale. I haven’t run it early enough on anything to know where the line sits.

— Blob by Blob

← back to writing