The Draft Cost 0.11M Tokens. The Checking Cost 11.8M.

Fabrication detection went from 5 of 36 to 33 of 36. What each layer bought.

means across eight controlled topics. source: paper, tables 3 and 4.

A paper generator with no server behind it

verdict: the design is ordinary. the receipts are the contribution.

A single-pass draft from the same backbone caught 5 of 36 planted fabrications. The full pipeline caught 33 of the same 36. Closing that gap cost roughly a hundred times more tokens than writing the draft did, and that ratio is the finding worth your time.

Spark-to-Paper is thirteen composable skills that run inside Claude Code. Each skill handles one research task (plan, cite, write, refine, review, figure, assemble, experiment) and they communicate through files in a shared project directory. There is no orchestration server, no graph database, no cluster scheduler. You copy the skill folders in, paste an idea, and get a compiled PDF with a verified bibliography and editable vector figures.

Systems that write papers end to end are not new. AI Scientist, its v2, Agent Laboratory and CycleResearcher all ran that arc first, and each ships as a standalone application. What this team did that the others did not is instrument the cost of not fabricating, layer by layer, and publish the line items.

One model, four harnesses, 78 points of difference

verdict: the first layer is the expensive one. the last two are the bargain.

The team seeded 36 probes spanning ten failure families, drawn from three sources. Every probe went into the shared source material before drafting, at the same intervention point. The identical probe set and the identical detection procedure ran against all four configurations, so the rows differ by the quality stack and nothing else. Intervals are Wilson 95%.

Read the ladder from the bottom. Gates are the heavy line item: 8.1M extra tokens and 5.3 dollars, for 55 points. Self-review and adversarial review together add 3.7M tokens and 2.2 dollars, and add another 23 points. The least expensive quality sits at the end of the stack, and most teams stop before they reach it.

Sum the three deltas and the checking accounts for 11.8M tokens against a 0.11M draft. The authors warn that the paired three-topic subset does not have to reconcile with the eight-topic mean, so treat that as the shape rather than the exact bill. The shape is still the point: writing is a rounding error next to verifying.

Preregistration written as a data structure

verdict: the strongest idea in the paper, and it is four lines of design.

Inside every skill the work splits two ways. The model handles judgment: which literature is relevant, how to organize an argument, whether evidence supports a claim. Python handles anything with an explicit rule: template structure, citation resolution, notation checks, plotting, LaTeX compilation, file presence. Six gates enforce the second half and a stage proceeds after its checks pass.

Here is the part worth copying into your own agent. During planning, the system fixes the shape of every result table, and the numeric cells stay empty until the matching experiment finishes. Required evidence is specified before any result exists. That converts preregistration from an instruction the model can quietly drop into a structure a script can inspect. An empty cell is checkable. A promise in a prompt is not.

One caveat the authors state plainly: deterministic code verifies that a number traces to its source, but deciding whether that evidence supports the sentence is done by the model and written into a structured report. The audit is auditable. It is not machine-decided.

Seven cycles, then the run is allowed to say no

verdict: the most transferable engineering decision in the system.

The team names a failure mode they call the Self-Refutation Loop. The system tests a hypothesis, judges the evidence insufficient, revises the method or the analysis, runs again, judges it insufficient again, and keeps going. It looks like iteration. It is not. Productive iteration resolves a concrete weakness. This one keeps rejecting its own objective while refusing to abandon the direction.

96.4 for the machine, 58 for the humans

verdict: a real capability gap, and the code has already moved past the paper.

Figures route by role. Anything reporting measured results is plotted programmatically from the recorded data and exported as a native vector PDF, and an image model never touches it. Method diagrams take the other path: an image model produces a raster that is treated as a visual target, and the assistant then rebuilds it as real text, shapes, connections and layout, renders that, compares it against the reference, and adjusts until the difference closes. If reconstruction is unreliable, the raster stays.

Across roughly 1,900 ground-truth figure elements the system kept 96.4% of them editable. Human-written preprints audited the same way scored 58%. AI Scientist scored 0 of 210 elements and its v2 scored 3%. Researchers ship bitmaps, and a generator that does not is beating them on a dimension nobody was competing on.

If you go from the paper to the code expecting that reconstruction path, it is no longer there. Release v1.2.0 on the repo swaps it for a skill that runs PaperBanana to render candidates, learns the render's design language, redraws the figure natively from the paper's own facts, and repairs it over at least four rounds against a geometry audit script that uses nothing but the standard library. The audit catches canvas overflow, shapes painted over labels, arrowheads that render at triple size because of a units attribute, connectors docking on nothing, and embedded rasters dressed up as vectors. That script is the interesting artifact, and it is more useful to most readers than the paper is.

Read the headline metric literally

verdict: the paper discloses most of this itself, which is to its credit.

Two smaller drifts, both harmless and both a little funny in a paper about integrity gates. The repo header says fourteen skills while its own skills section says thirteen folders above a table listing fourteen rows, because v1.2.0 added one after submission. And the repository description advertises about ten dollars of model costs per paper, against a measured mean of 8.1 with an upper interval bound of 9.6.