Telling Opus 5 What to Remember Erased 89 Percent of Its Memory Gain

Anthropic's agent memory post is going around for its dreaming wiki chart. The sharper result sits in Figure 5: one reasonable-sounding memory rule wiped out most of what memory bought the strongest model, while steering of the same kind nearly doubled the weakest one.

On November 6, Anthropic Engineering published The components of agent memory, written by Lance Martin. It does three things: it splits agent memory into four layers, it runs five evaluations (a CL-Bench database task, Claude Plays Pokémon, a 420M-token internal corpus, Harvey LAB legal briefs, and a merger Q&A set), and it names the offline pass that edits memory dreaming, building on Letta and UC Berkeley's sleep-time compute paper.

The definition is narrow on purpose. Memory is what Claude learns from experience and keeps between sessions as context it can read later. Weight updates, in-context learning and compaction are all excluded.

The post's best line: memory is a prediction. Writing a note is a bet that a future session will need it. Writing down a verified fact is a bet that a later session will want to use it without checking it again.

If you have been on call, you already know this shape. During an incident you scribble notes as things happen. Some are wrong, because you wrote them before you knew how the incident ended. The next morning you run the postmortem: you check every note against the logs and fix the runbook. In-session notes are the scribbles. Dreaming is the postmortem. The transcript is the log.

On the CL-Bench 1.0 database task, Claude answers questions one at a time, each in a fresh session, sees the correct answer after each one, and can keep a memory folder between sessions. Here is every model Anthropic charted. The gain column is my subtraction from their Figure 3.

Two things stand out. The newest models gain less because they start higher, so Opus 5, Opus 5.5 and Sonnet 5.5 all land near 0.83 with memory. And Sonnet 4.6 barely gains at all: its folder held scattered files of untested guesses, and it often did not read them.

Then Anthropic tried steering. Each test gave the model one rule about what to write. The rule they reported told the model to record failures and nothing else. It sounds sensible: learn from mistakes. Here is what it did to the gain from memory.

changes from Anthropic's fig. 5 (−0.234, −0.150, −0.020); percentages are my arithmetic

The strongest model was hurt the most. My reading of why: Opus 5 was the first model to close the learning loop. Its notes held rules, how it knew each rule was true, answers to earlier questions, and notes about traps. A failures rule throws away the confirmed successes, which are exactly the verified facts a future session wants to reuse without checking again.

Flip it. For Sonnet 4.6, Anthropic added two instructions: read your memory before answering, and test your theories. Its score went from 0.330 to 0.641, above Sonnet 5 with memory (0.610). Sonnet 5 reads its memory before answering 9 times out of 10 without being told. Sonnet 4.6 did it less than half the time.

To test judgment about what to keep, Anthropic compared memory folders from one Pokémon Red run each, about 10,000 steps in.

Sonnet 3.7 kept a file per battle, per item and per character it met. The game state changes constantly, so that archive went stale as fast as it grew. Near step 10,000 it spent 17 steps creating six files and deleting nine, with zero moves in the game.

Fable 5 followed a rule worth stealing: do not store what the environment shows you every turn. Money and map position are on screen. Strategy and tested rules exist in nobody's head unless written down. Its global.md acted as a dashboard (current goal, gym plan, its own memory rules). After seven Poké Balls in a row failed on a Magikarp, it wrote itself a rule: after three straight misses, vary the timing of the button presses. Opus 5.5 later beat the game with a small folder it kept rewriting.

Mt. Moon has its entrance on the west and its exit on the east, with a one-way ledge near the entrance that drops you back onto the route you came from. Claude wrote down that the PokéCenter was on the east side. It was wrong. It walked out, said "Mt Moon is behind me", marked the cave done, and fell off the ledge.

It fell three times. After the third it wrote that it would stop rather than keep looping. The harness's automatic reply told it to continue, so it walked all the way back to Pewter City and never reached Cerulean. A run with no memory never got there either.

Then they ran a dreaming pass on that same folder. Claude checked the note against the session transcript, found the error, and rewrote it. With the corrected note it went straight back into the cave and reached Cerulean City at action 1,935. The fix did not need a smarter model. It needed a second look at the evidence.

Anthropic pointed Opus 5 agents at about 350,000 internal files (roughly 420M tokens) from Stainless, a company it acquired in May, and had them build a linked wiki over 128 dreaming sessions, coordinating through a shared log. They scored 25 questions against graded criteria.

The headline number, 13 to 48, never caught up with plain search at 53. The real win is the bottom row: wiki plus search beat search alone by 8 points and used about 20,000 fewer tokens per question.

Some rough math of my own. The chart's x-axis runs to 128M dreaming tokens. At 20,000 tokens cut per question, the wiki pays back its build cost after roughly 6,400 questions. That ignores price differences between token types and leaves the accuracy gain out of the math. For a team asking thousands of questions against one corpus, it pays. For a one-off research task, search alone gets you most of the way.

Anthropic also shows where wikis help. On 24 merger questions where most answers needed several documents plus judgment or math, Claude scored 87% with the wiki and 73% with search. On a LAB task where most answers sat in a single memo, the wiki showed no clear gain. On another, the wiki reached 203 KB against a 115 KB corpus, 1.77 times the source. The post calls this Borges's map the size of the empire: a summary that abstracts nothing.

On a LAB acquisition task graded on contract terms, four agents wrote to one shared provisions.md at the same time. One agent's write erased another's lines.

Any on-call engineer will recognize this as a classic lost-update race, and the fix is the classic one: single-writer ownership. Claude found it on its own through a loop that reflected on failures, proposed new build instructions, and rebuilt the wiki. A second dreaming pass on another wiki found nine links pointing at a page that did not exist, repointed them, and added three pages including an index (225 links grew to 278).

Promoting a note into every session costs tokens on every turn and carries any mistake into every session. Loading it is also no promise Claude acts on it. The Claude Code memory docs load the first 200 lines or 25 KB of MEMORY.md, whichever comes first, and tell you to keep each CLAUDE.md under 200 lines because longer files reduce adherence. So much of dreaming's job at the top is pushing detail down and leaving a link behind. The header becomes an index, not an encyclopedia.

Kierkegaard said life is understood backward but lived forward. The post's quiet argument is that agents need both halves: a forward guess written in the moment, and a backward pass that checks the guess against what actually happened.