ResearchAudio.io

Your Agent Improves Itself Without Touching a Single Weight

Schmidhuber and 11 co-authors give prompt edits, memory rewrites, and tool creation the same formal status as fine-tuning.

That folder of prompt files, memory notes, and tool definitions you keep tweaking: a new 12-author survey with Jürgen Schmidhuber on the author list just gave it formal standing. It sits in the same taxonomy as fine-tuning, preference training, and the Gödel Machine.

You were doing self-improvement research the whole time. You called it harness engineering.

Two loops, one agent

The survey is Self-Improvements in Modern Agentic Systems, posted July 14 by researchers from Jilin University, King Abdullah University of Science and Technology, and collaborators. It defines an agent as a pair, A = (θ, Σ): θ is the foundation model's weights, and Σ is the operational scaffold wrapped around them.

The scaffold decomposes into four parts: prompts (p), memory (m), tools (T), and control logic (g). Routing, scheduling, and safety constraints all live in g.

Self-improvement then gets a strict definition: a durable, self-induced update to θ or Σ. Chat history and the KV cache do not count, since transient state resets at task boundaries. The bar is a committed change the agent carries into future work.

That one definition splits the entire field into two loops.

Loop 1 is the familiar one: the model trains on signals it generated itself. The survey sorts those signals into three kinds: demonstrations the agent writes, judgments the agent makes about its own outputs, and trajectories from acting in a real or simulated environment.

Loop 2 is the one you run in production, probably without calling it learning. The agent rewrites its prompts, consolidates its memory, creates or refines tools, or patches its whole scaffold, with θ untouched.

Scaffold edits are not RL with extra steps

Here is the formal move I did not expect: scaffold updates sit outside classical reinforcement learning. RL assumes a fixed action space and state representation. Add a tool or restructure memory and you reshape the underlying Markov decision process itself.

The update mechanics differ in kind, too. Prompts, memory entries, and routing rules are discrete, structured artifacts, updated through search, generation, or symbolic edits rather than gradient descent. They stay explicit and inspectable instead of dissolving into weights.

Inspectable means reviewable. The survey keeps the full scaffold history in its formalism for exactly this reason: validate each change, roll back what degrades.

A skill is a self-improvement operator

Here's the part nobody's talking about. The survey formalizes skill as the reusable unit of self-improvement: a named, retained update to the agent's own configuration.

The substrate is secondary. A skill can serialize as a tool plus its calling convention, an instruction or workflow file, a memory entry, control logic, or consolidated weights. The skill is the update it encodes; the storage location just names where it lives.

Then the split that matters. Object-level skills act on the task, like a learned collect-wood routine, the agentic analog of options in hierarchical RL. Meta-level skills act on the agent itself: writing a new tool, refactoring a prompt, consolidating experience into memory.

Meta-level skills both edit the configuration and live inside it. Improve one, and the improver evolves together with the system it improves. That is the self-referential loop Schmidhuber described in 1987, showing up in your skills directory, and the structure the Gödel Machine formalized in 2003, minus the proof obligation.

The scaffold-audit sheet I use for this, the (p, m, T, g) mapping plus the acceptance checks, sits in the paid archive.

Quick Hits

In-context learning is 35 years old. The survey connects modern in-context adaptation mechanically to 1991 fast-weight programmers: the transformer's KV cache acts as an associative memory producing transient, context-dependent weight changes. Same trick, new scale.

Memory instead of rollouts. A system called GLoW keeps a two-scale textual world memory, a global frontier of high-value discoveries plus local reflections, to steer a Go-Explore-style agent through text games. It reports strong results with 100 to 800 times fewer real environment interactions than RL baselines.

Verifier gaming got easier. When the verifier is a linguistic object, a judge model and its prompt, an agent can satisfy the literal check while failing the task. The survey lists this alongside capability regression as a failure mode specific to foundation-model agents.

Self-improvement at inference time. Test-time self-improvement flags weak cases through uncertainty estimates, generates targeted training examples for those exact blind spots, and applies small low-rank updates on the spot. Per-instance gains at a fraction of the data cost.

Harness, meet scaffold. What industry calls the agent harness, the survey deliberately names the scaffold, stressing that it is the modifiable structure around a frozen model. Expect the term in papers from here on.

The Take

Fine-tuning is consolidation. The daily self-improvement loop for most teams shipping agents in 2026 is scaffold improvement, and it deserves the discipline we give code: review, tests, versioning, rollback.

The history section carries the sting. Eurisko's self-modification worked in the 1980s to the extent that a human served as the evaluation signal, interpreting outputs and pruning unproductive heuristics by hand. Four decades later the evaluator is still the weak link: judge models can end up scoring surface compliance above genuine improvement, and consistency signals break down when a model is confidently wrong.

Whoever cracks evaluation of scaffold edits, not generation of them, owns this loop.

The Open Question

What is your acceptance test for an agent-proposed edit to its own scaffold? Unit tests catch tool regressions. Nothing standard exists yet for a memory write or a prompt refactor.

If you run canary tasks, staged rollout for skills, or evals on scaffold diffs, hit reply and tell me what holds up. I read every reply.