LOOP ENGINEERING

Your AI agent says
it’s done. Is it?

Designing the system that assigns the work, checks the result, and knows when to stop.

7 MIN READ   ·   CONCEPTS, EVIDENCE, AND ONE WORKED EXAMPLE

Imagine a coding agent working through the night. At 8 a.m., it leaves a tidy summary: the bug is fixed, the tests pass, the branch is ready. You open the diff. It changed the implementation. It also changed the test that was supposed to catch the bug.

That hypothetical is a useful way into loop engineering. An agent can produce a plausible patch and a convincing account of its work. The surrounding system still has to decide whether the task is complete, whether another attempt would help, and what evidence you should see.

Once agents can take several steps on their own, those decisions become a substantial part of the engineering work.

01 / THE IDEA

Who writes the next prompt?

In Addy Osmani’s account, loop engineering means building the system that prompts your agents: it finds work, assigns it, checks results, and records progress. Coordination moves from the chat into software. [1]

Figure 1. A conceptual view, adapted from Osmani. Implementation boundaries vary. [1]

The inner loop handles tool calls and observations within a run. The outer loop chooses work and decides what follows. It can start runs on a schedule, respond to events, or pursue a bounded goal.

Terminology varies. Here, the focus is handing off coordination. One worker and ordinary application code can be enough.

02 / A CONCRETE EXAMPLE

One bug, three possible endings

Consider a CSV importer that drops the final record when a file has no trailing newline. In this illustrative workflow, the loop receives a confirmed bug report and gives a coding agent an isolated checkout, the failing fixture, and a narrow scope: repair the importer.

Before the first edit, the workflow reproduces the failure on the starting revision. After the patch, it runs that same check plus the existing parser suite, then inspects whether the change stayed within scope. The evidence is tied to the candidate revision. If the code changes again, the earlier test result no longer qualifies.

Figure 2. An example decision policy for a repair loop. “Submit for review” is this workflow’s success boundary.

If the fixture still fails, the next instruction includes the actual failure and the previous attempt. If the patch passes the required checks, the loop prepares it for review. If the agent keeps making the same edit, hits its limit, or discovers that the bug requires a wider change, the loop stops with a useful handoff.

The benefit is that you can review a patch with its evidence attached, without having to reconstruct the night’s conversation.

03 / VERIFICATION

Write the acceptance test first

“Make the importer reliable” leaves too much room for interpretation. “Preserve every record in these fixtures and pass the existing parser suite” gives the loop something concrete to check. It still leaves untested behavior, but it makes the acceptance boundary visible.

Anthropic’s guidance separates agent evaluation into code-based checks, model-based judgments, and human review. Each is useful for different questions. Executable checks can verify a specific result; a model can assess a diff against a rubric; a person can decide whether the proposed behavior is the behavior the product needs. Model graders also need calibration against human judgments. [3]

For the importer example, keep acceptance fixtures under the verifier’s control and reject unauthorized edits to them. Let the worker add tests, but do not let it silently weaken the checks that determine success. A second model’s approval is another signal to examine, not a substitute for running those checks.

04 / STATE

A fresh session needs a real handoff

A long conversation is an awkward place to store a project’s state. In its work on long-running agents, Anthropic documented failures that included unfinished work left for the next session and agents declaring completion too early. Its approach used incremental work, a progress file, and Git history to help the next session recover context. [2]

For our repair loop, the handoff can be small: task ID, starting and candidate revisions, attempted fixes, check results, remaining budget, and the next unresolved question. Store links to the actual logs. Treat a progress note as a claim to verify against the current checkout.

This matters after interruptions, too. If a run already opened a pull request, the next run should resume that work. A restart should not create a duplicate task simply because the conversation was lost.

05 / THE RESEARCH

The evidence is early

An August 2026 preprint by Jai Lal Lulla and colleagues examined traces of loop engineering in software repositories. From an initial dataset of 36,710 repositories, their heuristics identified 256 candidates; inspection confirmed autonomous agent processes in 217. Most confirmed cases were triggered by repository events. [4]

Figure 3. Lulla et al. (2026). Heuristic selection limits what these counts say about adoption across software teams. [4]

The authors emphasize an observation problem: runtime state and costs may live outside the repository, so missing committed evidence does not establish that a practice is absent. A controlled study of effort and outcomes was still proposed in this version. [4]

My reading: the practice is real enough to investigate, while broad productivity claims still need stronger evidence. For a team deciding whether to adopt it, its own task outcomes will matter more than the label.

06 / BUILDING IT

Start with a loop you can audit

A sensible first experiment is one recurring maintenance task with a reproducible failure and a reviewable output. Decide what the loop is allowed to do before choosing how many agents to run. Here is a small contract for the importer example:

Illustrative limits, not benchmark recommendations. Tune them to the task and enforce them outside the model.

Then evaluate whether the loop helped. Track accepted fixes, regressions, human review time, and total cost across both successful and failed attempts. Compare those results with your current workflow. A loop that produces more patches but leaves you more review work may be a poor trade.

The question to leave with is specific: if your agent stopped right now, what evidence would let someone else decide whether its task was finished? That answer is a good place to begin designing the loop.

SOURCES & NOTES

1. Addy Osmani, Loop Engineering

2. Anthropic, Effective harnesses for long-running agents

3. Anthropic, Demystifying evals for AI agents

4. Lulla et al., Loop Engineering: Building Blocks, Adoption, and Impact

The CSV scenario, contract, and decision policy are illustrative designs by ResearchAudio. Research discussed here is distinguished from our interpretation and implementation suggestions. Sources checked September 18, 2026.

ResearchAudio.io / AI research and engineering