A pull request says it fixes a bug and adds tests. Coretexa puts the source code back the way it was, leaves the new tests exactly where they are, and runs them again. If they still pass, the tests do not touch the thing the pull request changed — and you know that before you merge, not six months later.
runs on your runner — nothing executes on our infrastructure · no runtime dependencies of its own · no telemetry · MIT
a live, unmodified verdict — not a mock-up
the same discipline, applied to our own work:
7 fixes merged upstream — gson, buildtools, fastmcp, git-machete, szio, croniter, superfile
Agent-written pull requests arrive green. Green is not evidence.
A passing suite tells you the tests passed. It does not tell you the tests would have failed without the fix — and that is the only property that makes a test a gate.
The new test exercises the tidy helper function the change added, not the one line that wires it into the program. Delete the wiring and the test is still green.
An assertion loosened, a case skipped, a threshold widened. CI goes green because the bar moved, not because the bug went away.
The thing that wrote the fix also wrote the test that proves the fix. That is not review — it is a rubber stamp with extra steps.
There is no model in it, and on a standard project there is nothing to configure. It answers one question by running something, and everything it reports it established by observation.
Source changes on one side, new and modified tests on the other. Renames, docs, lockfiles and assets are classified and set aside.
Every behavioural hunk is reverted to the merge base at once. That is two test runs and usually the whole story. Only when the revert fails to compile — common in Go, Rust and Java — does it fall back to reverting each hunk on its own, so the verdict still points at a line rather than a pull request. That second stage is bounded by a time budget you control.
Unchanged, exactly as the author wrote them. If they fail, the tests genuinely gate the change. If they pass, they never did.
A finding is only reported when the mapping from test to code was proved, not inferred. When it cannot prove one, it says so instead of picking the interesting answer.
NO_GATEA behavioural change can be reverted with the pull request’s own tests still passing. The tests do not detect the fix.
GATE_HOLDSReverting the source makes the new tests fail. They really do gate the change. This is the good case, and the common one.
GATE_HOLDS_BUILDThe revert does not compile, so the change is load-bearing at build time. Weaker evidence than a failing assertion, and labelled as such.
NO_NEW_TESTSThere is nothing to run. The pull request changes behaviour and adds no test that could observe it.
INCONCLUSIVEThe experiment could not be run cleanly. Reported honestly, with the reason, rather than dressed up as a pass.
test-command
No account, no API key, no service to sign up for. Coretexa itself is pure Python standard library and pulls in nothing of its own — it detects and installs your project’s declared test dependencies on your runner, because it has to run your tests.
Read-only. The verdict goes to the job summary, so nothing needs write access — the safest thing to run on a public repo that takes fork pull requests. Want it as a PR comment instead? Add pull-requests: write and pass github-token. Non-blocking by default — it only fails the job if you opt in with fail-on.
The pull request did two things: it added a directory-size cache, and it added request tracking to keep stale results from being applied to the wrong path. The cache was well tested. The request tracking was not tested at all — and nothing in the review caught it, because a passing suite looks the same either way.
Five new symbols, none of them named in a single test file. Revert all three hunks and the suite is still green, so the change could break in any future edit and nothing would say so.
CodeRabbit had reviewed this pull request three times and posted rounds of design opinions on exactly that code. It never mentioned that the code had no tests. That is the gap this check is for: not whether the code is good, but whether anything would notice if it stopped working.
The contributor added request_tracking_test.go plus integration tests the next day — and while writing them, found and fixed a stale-response bug on the same path that our report had not named. That is the intended shape of this: the check points at an untested change, and the person who wrote it does the rest.
A tool that claims to catch everything is a tool you cannot trust on anything. This one answers a single narrow question, and these are the questions it does not answer.
NO_GATE. A subtly wrong fix with a test that pins the wrong behaviour is GATE_HOLDS. It measures whether a change is observable, never whether it is right.GATE_HOLDS means the tests notice this change. It says nothing about the case the author never considered — We have a worked example of our own: a SollanSystems/loop-engineer pull request that left SQLite sidecar files behind on a crash-recovered store. We refuted it by hand — and when we later ran this check against it, the verdict was GATE_HOLDS. The tests did gate the change. The change was still incomplete. Finding that needs a person, or the deeper adversarial pass, not this.Everything it says, it established by running something.
MIT licensed. Runs entirely on your runner — no telemetry and no analytics, and nothing is sent anywhere. It touches the network only for git and, unless you set install-deps: false, to install your project’s own declared test dependencies. Secrets are stripped from the environment of every subprocess that runs your repository’s code, so a pull request cannot read your workflow token out of a test.
One file in .github/workflows. It is free and MIT, it runs on your own runner — usually two extra test runs per pull request, and Actions is free on public repos — and it tells you the one thing a green check never will.
early, solo, and MIT — issues and pull requests welcome
Want this run across pull requests you have already merged?
We do that by hand — the check first, a person reading every finding, then a short write-up of the ones worth acting on.
Ask →