Every number on this page comes from one campaign: 9 scenarios, git bisect versus crux, same predicate commands, same machine, raw data in the repository under benchmark/results/raw.tsv. Accuracy column means the tool named the actual breaking commit.

summary panel: crux fast mode beats git bisect on 5 of 9 scenarios, hyperfine head to head crux 1.183s vs bisect 1.215s, and only crux flags the interaction fault

wall time

9 scenarios: linear histories at 50, 200 & 1,000 commits, a merge, a large diff, an interaction fault, a dependency chain, a slow-test suite, and a real repository (fd, 25 synthetic commits on top of upstream history). Green is bisect, amber is crux fast, purple is crux full.

grouped bar chart of wall seconds per scenario for git bisect, crux fast and crux full the same bars on a log scale, showing the ordering holds at every magnitude

crux --fast wins 5 of 9 outright (S1, S3, S4, S5, R1) and loses 3 (dependency chain, slow tests, and one noisy S2 harness row that a 10-run hyperfine measurement puts back in crux's favor). Full mode pays extra probes for the explanations, which is the honest trade.

hyperfine head-to-head

10 timed runs each on the 200-commit scenario, warmup run first: crux --fast 1.183s vs git bisect 1.215s. Every individual run as a dot:

strip plot of every hyperfine run on a log scale with means marked, crux fast slightly ahead of git bisect

speedup factor

speedup ratio per scenario with a red parity line; bars above the line are crux wins, below are bisect wins

The ratio is bisect time over fastest crux mode. S3 (1,000 commits) is the clearest win at 1.38x: more commits means more probes, and crux's probe pipeline batches what bisect spawns one process at a time.

test executions

Every probe runs your command for real, so probe count is what multiplies into minutes on slow suites. crux probes 6-10 times across these scenarios versus bisect's 4-8; the counts are close because both are binary searching, and crux spends its extra probes verifying instead of trusting your range bounds.

test executions per scenario, git bisect versus crux

scaling with history size

3d surface of wall seconds across 50, 200 and 1000 commit linear repos for both tools

Both tools grow logarithmically, as they should. The surface shows the gap staying proportional at every size: whatever wins at 200 commits still wins at 1,000.

per-scenario detail

nine small panels, one per scenario, comparing bisect, crux fast and crux full wall times

micro-benchmarks

The search dominates wall time, so the in-process primitives barely matter. They're measured anyway: ddmin minimization over a 100-item set runs in 18 microseconds, content hashing 10 KB in 26 microseconds. The bottleneck is your test command, not crux.

divan micro-benchmark table for ddmin, variable and call extraction, and content hashing criterion means for the same primitives, matching the divan numbers

reproducing

The harness lives in benchmark/ (bench-all.ps1 regenerates every repo & reruns all tools), the chart generators in visuals/generate.py, and the raw numbers in benchmark/results/raw.tsv. Nothing on this page is hand-typed.

edit this page