Harness > Model: What Cursor's SQLite Experiment Tells Reviewers

Cursor published research on July 20 showing their agent swarm rebuilt SQLite in Rust, from the 835-page documentation manual alone, and passed 100% of the sqllogictest suite across all four model configurations tested. The finding that should concern every engineering team is not the 100% pass rate. It is the cost range: $1,339 to $10,565 for equivalent outcomes, depending entirely on which models orchestrated versus executed. Same harness, same task, same grade — roughly an 8x price spread. That gap is the harness, not the model.

What Cursor actually measured#

The experiment ran four configurations: GPT-5.5 as both planner and worker; Grok 4.5 as both; Opus 4.8 as planner with Composer 2.5 as worker; Fable 5 as planner with Composer 2.5 as worker. The new harness outperformed the old in every single mix. The old Grok 4.5 run had to be paused before its two-hour mark after accumulating more than 70,000 merge conflicts and 68,000 commits. The new run, same model, logged fewer than 1,000 conflicts over four hours.

The code volume tells the same story. The Opus mix under the old harness produced 19,013 lines of engine code at 97%. The new harness produced 4,645 lines at 100%. The Fable 5 mix under the old harness needed 64,305 lines; the new one used 9,908. The test suite cannot see any of this. A grader that only measures query results has no opinion on whether the implementation is 4,645 lines or 64,305.

The token-cost split matters for reviewers#

Workers carried at least 69% of the tokens in every run, and over 90% in most. But worker tokens are cheap. In the Opus 4.8 + Composer 2.5 run, the Opus planner produced a small fraction of the tokens and roughly two-thirds of the cost. Composer handled the vast majority of tokens for the remaining third. The GPT-5.5 solo run spent $9,373 on workers alone. The Opus hybrid's entire worker fleet cost $411.

This creates a structural fact that review processes currently ignore. A diff produced by a fast, instruction-following worker model has a different failure profile than one produced by a frontier reasoning model doing its own planning. Worker-produced code is precise within its instructions and fragile at their boundaries. Planner-produced code carries architectural decisions that may never surface in the diff at all. Neither failure mode is visible to a test suite. Both are visible to a reviewer who knows what to look for — but only if the reviewer knows which kind of agent wrote the code.

The artifacts agent-TDD leaves in diffs#

Cursor's harness used a test-driven approach: agents were graded against sqllogictest, a suite the swarm never saw directly but whose pass rate was the external measure of progress. This produces distinctive patterns in the resulting codebase. Coverage concentrates where the spec is explicit. Modules that implement well-documented SQL features get built out evenly; anything the documentation leaves ambiguous accumulates debt.

The specific coordination failures Cursor documented — split-brain design (54 crates vs. 9), megafiles (one file touched by 1,173 agents and collecting 7,771 conflicts), and ossification — all leave structural traces. Duplicate packages signal split-brain. Files with unusually high churn density in git history signal megafile contention. Core modules that were never touched despite architectural dependency suggest ossification. These are not security vulnerabilities. They are maintainability risks that arrive already passing CI, and they compound.

What review processes are missing#

The research from agenticcontrolplane.com, published July 10, draws the same conclusion from a different direction: metering a few hundred real tool-calling agent runs found that cost and reliability are mostly decided by the harness, not the model. That finding matches Cursor's data. The implication for review is that model identity is a weaker signal than harness configuration.

Current review tooling has no concept of provenance at the agent level. A PR arrives. Tests pass. The diff looks reasonable. Nothing in the standard pipeline asks whether the code came from a planner making architectural decisions or a worker following detailed instructions — and the answer determines where the failure modes live. Architectural brittleness hides in the former. Boundary-condition gaps hide in the latter. Both survive green CI.

As covered in what actually changes the week your team adopts an AI coding tool, generation rate climbs faster than review capacity almost immediately after adoption. Agent swarms accelerate that gap by orders of magnitude. Cursor's new VCS peaked at 1,000 commits per second. Human review tempo does not.

The delta between "passes tests" and "reviewable"#

The Cursor experiment is useful precisely because it controls for outcome. Every model configuration passed 100% of the held-out suite. The differences that survived that control — line count, crate structure, conflict history, architectural coherence — are the differences that matter for long-term maintainability and that review exists to catch.

Passing tests is a floor, not a ceiling. The gap between 4,645 lines at 100% and 19,013 lines at 97% is not a testing problem. It is a review problem. Engineering teams shipping agent-generated code at volume need review that operates on the full diff, not just the test signal. That is true regardless of which model mix produced the code, and regardless of whether the harness was a Cursor swarm or something a team assembled internally.

Hyrax is live at hyrax.dev.