Every result crux produces is only as honest as the command you pass with -c. 3 rules separate a target that works from one that produces confident nonsense.
the exit code is the entire contract
0 holds. Anything else drifted. crux captures stdout & stderr so you can read them in replay & reports, but classification never reads text. Output parsing is where tools start lying.
commit everything the command touches
This is the rule people break first. Probes check out old commits, and worktrees in parallel mode contain tracked files only. A check script that exists only in your working tree works at HEAD & vanishes in every probe, which makes every historical commit look broken.
Commit the harnessWrite check.sh or check.cmd, commit it, point -c at the committed path. 1 commit removes an entire class of false results.
the command must be deterministic
Pin the seed. Skip the clock & the network. A target that flakes 1 time in 50 will eventually flip a probe mid-search and land the result on an innocent commit, presented with full confidence. Not sure about your target. Run it through replay & read the verdict.
examples
crux who -c "cargo test --quiet" -f v1.2.0..HEADcopiedcrux who -c "node render.mjs fixture.json | diff - golden.txt" -f HEAD~30..HEADcopiedThe second is the purest form: program output against a golden file. Any observable behavior can be written this way.
edit this page