The bug my AI reviewer missed — and the second one that caught it
A reviewer that agrees with you is worth almost nothing. What's worth trusting is a reviewer whose whole job is to make you wrong.
I've been running a pipeline that finds real bugs in open-source projects, fixes them, and opens pull requests. The part I care about most isn't the fixing. It's the gate every fix has to pass before it's allowed anywhere near a PR. Last week the gate earned its whole existence in a single catch, and it's a good illustration of why "looks good to me" is one of the most dangerous phrases in software.
The fix
The target was a bug in git-machete: running git machete discover from different git worktrees produced different branch trees, silently dropping branches depending on where you ran it. The root cause was subtle. discover ranks branches by how recently you checked them out, using git reflog show on HEAD — but HEAD's reflog is per-worktree. So the ranking only ever saw the current worktree's history, and branches you'd recently used elsewhere got quietly pruned.
The fix aggregates every worktree's reflog instead of just the current one. Clean, well-scoped, one function. I reproduced the bug, wrote a regression test, confirmed it passed. Done, right?
The gate
Every fix goes through two independent reviewers, each running in its own clean checkout, each told nothing about the other. Their jobs are deliberately different. The first is asked to reproduce and confirm. The second is asked to try to prove the fix wrong.
The first reviewer did its job perfectly. It rebuilt the failure, confirmed the fix made it disappear, ran the whole test suite green, and independently checked that my regression test actually failed without the fix. Verdict: confirmed. If my process ended there — the way most reviews end, with a thumbs-up and a merge — the story would be over and the bug would be shipped.
The catch
The second reviewer refused to sign off. It went looking for the failure modes nobody had considered, and it found one.
My fix had replaced a git reflog show subprocess call with a direct file read — open(path). Looks equivalent. It isn't. The old subprocess decoded its output as strict UTF-8, independent of the machine's locale. A bare open() in Python uses the locale's default encoding. On a system without a UTF-8 locale — a minimal container, a stripped-down CI runner — a reflog containing a non-ASCII committer name like Zoé would throw an uncaught UnicodeDecodeError and crash the command outright.
It was invisible on my machine. It would have been invisible on the maintainer's machine. It would have surfaced as a mystery crash in some stranger's container three months later, and nobody would have connected it to a worktree fix from last summer. The reviewer handed me the exact repro: an accented committer name, LC_ALL=C, one command, guaranteed crash.
The fix for the fix was one line — decode as UTF-8 explicitly, with a fallback that can't throw. I applied it, sent it back through the same reviewer, it cleared, and the PR went out.
The point
The one-line patch isn't the interesting part. The interesting part is that the two reviewers disagreed, and the disagreement was the entire value.
A single reviewer that agrees with you is worth almost nothing. It's theater — a second signature on a decision you'd already made. What's actually worth trusting is a reviewer whose incentive is to make you wrong, who treats "I couldn't find a problem" as a failure to search hard enough rather than a passing grade. "LGTM" and "I tried to break it and here's how" feel like the same output — an approval — but they are completely different signals. Only one of them tells you anything.
Merged pull requests are a nice number to point at. But the thing quietly doing the work is the gate that stops the plausible-but-wrong fixes from ever becoming merged pull requests in the first place. That's not overhead around the product. It is the product.