| researchaudio.io · referee report |
| manuscript | arXiv 2611.09822v2 · quant-ph · nov 9 2026 |
| from | Google DeepMind + Google Quantum AI, 29 authors |
| agents | Gemini 4 Carbon teams, Lean 4 checker |
| referee read | real results, one headline that needs a footnote |
|
DeepMind's Quantum Agents Hit 290x. A Fuzzer Did the First 217x.
Agent teams wrote quantum programs nobody can run yet, and a theorem prover checked every one. The biggest number in the abstract owes the least to that prover. The smaller numbers are the ones that could not exist without it.
|
| R1 | what the paper claims | summary |
"Discovering formally verified quantum programs with AI" (Shirobokov, Jaderberg et al.) builds a quantum programming library inside the Lean theorem prover, then lets teams of Gemini 4 Carbon agents search for lower-cost quantum circuits. No circuit counts unless Lean proves two things: it computes the right answer, and it costs what it claims.
The cost unit is Q·T: logical qubits times T gates. T gates are the expensive operation on an error-corrected quantum computer, so think of it as floor space times machine hours. Lower is better.
exhibit a · the five results
| problem | used for | gain | can a laptop test it |
| Fixed-point arcsin | loading numbers into quantum states | 290x | mostly yes |
| Fusion stopping power | fusion fuel design | 10.9x | no, 4,374 qubits |
| Non-Born-Oppenheimer dynamics | light-driven chemistry | 8.08x | no, 6,057 qubits |
| Cytochrome P450 | how the body clears drugs | 2.49x | no, 333 qubits |
| Parallel Grover search | search with parallel queries | up to 2.55x | no, holds as N grows without limit |
gains in Q·T against the strongest baseline the authors could build; Grover is measured in expected oracle rounds
|
| R2 | how the loop works | mechanism |
A researcher writes a Lean file that reads like a contract with blanks. It states what the circuit must do and how its cost is counted, and leaves sorry placeholders for the circuit, its cost, and the proofs. The agents fill in the blanks. Nobody gives them problem-specific hints: the prompts are generic optimization prompts, and the domain knowledge lives in the spec.
exhibit b · one candidate's path
| 1 | Researcher writes the spec and pins it to a repo commit. |
| 2 | Agent team proposes a circuit and iterates on Lean kernel errors. Team design is up to each researcher: one agent or a hierarchy. |
| 3 | External verifier replays the submission in a clean sandbox against the frozen spec. |
| 4 | Shared wiki takes the result if it beats the best by at least 5%, plus reusable lemmas and dead-end notes. |
| 5 | Conference every four hours: teams critique each other's unverified ideas, which count for nothing until verified. |
|
| R3 | the 290x, split by stage | needs footnote |
Arcsin is the odd one out among the five, and the paper says so. Circuits for it can be written in a restricted gate set called kickmix, which a normal computer can fuzz-test in seconds. So the authors ran the search in two stages: fast fuzzing first, Lean second. Appendix F gives every waypoint. I divided them out.
exhibit c · arcsin Q·T at each stage
| stage | checked by | Q·T | vs baseline |
| Published baseline (Häner et al.) | | >108,000,000 | 1x |
| Kickmix search | fuzz tests | 498,168 | 217x |
| Lean search, still kickmix-expressible | Lean, fuzzable too | 415,200 | 260x |
| Lean search, gates kickmix cannot express | Lean, nothing else | 369,800 | 292x |
ratios are my division of appendix f and table s2 figures
So of the 290x in the abstract, the part that needed gates a fuzzer cannot check is the last 1.12x. The jump from a piecewise-polynomial method to a 100-qubit circuit happened inside the testable box.
Two notes in the paper's favor. First, the baseline is a lower bound: it was built for an error of about 10-6, looser than the 1.25×10-7 the agents had to hit, and it leaves out some registers. The real gap is likely bigger than 290x. Second, the authors state the lesson plainly: where a classical simulator can check circuits, use it, and put formal proof on top. That is the right engineering call, and it is worth stating loudly before the 290x travels.
|
| R4 | where proof was the sole judge | verified |
The three science results run on hundreds to thousands of logical qubits. No classical computer can simulate them, and no quantum computer exists to run them. Nothing but Lean can say these circuits are right. These are the real story.
Fusion, 10.9x. The task: simulate an alpha particle and 218 electrons moving among 216 fixed protons. The baseline works in momentum space and pays for every state the qubits could hold, including impossible ones. The agents' circuit moves electrons onto a real-space grid, where it can use a rule of physics: no more than two electrons (opposite spins) can sit on one grid point. Lean proved that, so the circuit stops paying for crowds that cannot happen. Result: 12.8x fewer steps at 1.52x the T gates per step, on 4,374 qubits instead of 5,650.
Light-driven chemistry, 8.08x. All 47 nuclei and 187 electrons are treated as quantum. Three moves: copy particle coordinates into scratch qubits and erase the copy with a measurement, instead of swapping particles back and forth; get 1/r by random sampling from a small table instead of computing a division; and treat a nucleus of charge 8 as eight unit charges at the same spot.
Drug metabolism, 2.49x. The baseline stores which Hamiltonian term is active as a binary number and looks up rotation angles from memory into a 247-qubit register. The agents' circuit gives each term its own qubit and flips one on. Earlier work found this idea narrowly worse because it needs more qubits. The agents' version uses the register twice, for the term weights and for the rotation angles, which covers over 95% of the per-step T-gate reduction. The 247-qubit register is gone: 481 qubits down to 333.
exhibit d · the authors made their own baselines harder to beat
| problem | vs published number | vs strengthened baseline | reported |
| Light-driven chemistry | 11.0x | 8.08x | 8.08x |
| Drug metabolism (P450) | 5.17x | 2.49x | 2.49x |
"vs published" is my division against the original estimates the paper cites (6.77×1015 and 2.26×1012)
For these two problems the authors folded later improvements into the baseline before comparing, and reported the smaller number. That is the opposite of how launch posts usually work.
|
| R5 | the referee lives in another sandbox | transferable |
For anyone building agent loops, Appendix B is the most useful page. The authors list how agents can game a local check without improving anything:
| how an agent can game it | what blocks it |
| weaken the theorem it must prove | statements must match the frozen spec |
| edit a helper definition | every definition the theorems depend on is compared |
| add a custom axiom | proofs may use three standard Lean axioms, no others |
| turn off local kernel checks | grading replays in a clean sandbox the agent never touches |
| misreport the cost | the cost theorem must reduce to the claimed number |
The checker is the open-source Lean Comparator. Compare this with the Hermes Agent self-refactor this newsletter covered in September, where agents edited the very tests that served as their oracle and test parity reported zero regressions. Same failure, opposite design: here the oracle sits outside the agent's reach, and humans review the spec instead of the solution.
Even then, one hole got through. Agents found a way to exploit how Toffoli gates were counted, which lowered the score without improving the circuit. The fix was not a better agent but a revised spec. A frozen referee is no better than the rulebook it reads.
|
| R6 | what the teams did for each other | after-the-fact credit |
The authors traced every verified gain back to where its idea started, once both sharing channels were running.
exhibit e · share of cost reduction by idea source
| problem | own team | wiki | conference |
| Fusion stopping power | 87% | 9% | 4% |
| Light-driven chemistry | 60% | 11% | 29% |
| Drug metabolism (P450) | 37% | 25% | 38% |
| Average | 61% | 15% | 24% |
fig. 5b; fusion conference share inferred from the stated 13% cross-team total
On the chemistry problem, the lead changed teams seven times, and the final circuit combines tricks from three teams. The biggest single drop, 34%, started as a Team 1 conference post that Team 2 refined and proved. A note proving why one shortcut fails stopped two other teams from chasing it.
Two wiki rules worth copying. A new record needs at least a 5% improvement, so teams are not constantly rebasing onto tiny gains. And a solution that cuts the Lean code by 20% while lowering cost is also accepted, because proof files kept growing past agent context limits. Code size was treated as a resource, the same as qubits.
|
| R7 | the spec is the ceiling | limit |
The authors are direct about this: agents cannot optimize beyond the spec. To match the baseline, the chemistry spec demands correct behavior even on physically impossible input states, which blocks gains like the fusion trick. The P450 spec fixes one way of writing the Hamiltonian, which rules out the newer representations behind recent human gains.
And none of this runs soon. Early fault-tolerant machines are projected at roughly one million logical operations. The lowest-cost circuit here, P450, still needs about 1.31 billion T gates. The paper says further cuts of 10x and more are needed. This is progress on the cost curve, not a path to a demo next year.
|
| R8 | open questions for the authors | unanswered |
| Compute | No token count, wall-clock time or agent-hours are reported for any problem. |
| Code | The Lean library, specs and solutions "will be open sourced shortly". |
| Ablations | Each researcher built a different team, so there is no controlled run with the wiki or conferences switched off. Exhibit E is attribution after the fact, and the authors say credit is hard to separate. |
| Baselines | The fusion and P450 baselines come from papers led by people who are co-authors here (Rubin, Low). They beat their own earlier work, which is fine, and worth knowing. |
| Model | All large runs used Gemini 4 Carbon. Nothing says how much the result depends on that model. |
|
| R9 | what to take into your own agent loop | recommend |
| 01 | Use the fast checker first and the strict checker where the fast one stops. Arcsin got 217x from fuzzing before proof added anything. |
| 02 | Grade outside the agent's sandbox, against a frozen copy of the rules. Anything the agent can edit, it eventually will. |
| 03 | Put human review on the spec, not the output. When a loophole shows up, fix the spec. |
| 04 | Share verified results and dead ends in one place, with a minimum-gain bar for new records. |
| 05 | Let agents trade unverified ideas, but give those ideas zero weight until a checker passes them. |
|
recommendation
[x] accept the science results as the headline [x] footnote the 290x
The durable idea is not that Gemini found faster circuits. It is that a frozen spec plus an outside checker let independent teams build on each other's work with no human code review, on problems no machine can test. |
|
|
|
| researchaudio.io · end of report · stage ratios and "vs published" figures are our division of the paper's own numbers |
|