In partnership with

Why are so many brands shifting ad spend to CTV?

People spend more time streaming than they do with cable, social media or video games, and advertisers know it. In fact, according to Advertiser Perceptions and Premion's 2026 CTV/OTT Advertiser Survey, 70% of advertisers are raising CTV budgets in 2026, with much of that budget diverted from social advertising.

For marketers who want a presence in streaming, Roku is the obvious place to go, with scale that speaks for itself. With more than 100 million households representing 47% of all streaming time in the US, Roku easily has more reach than every competitor.

With Roku Ads Manager, you can reach viewers across more than 100 premium apps in a single buy. Combine your customer data with Roku's insights to reach only the households you want, then measure and optimize to your heart's content.

Want to give it a try? Unleash your first campaign in just four steps: (1) Create your account, (2) define your objective and target audience, (3) upload creative, and (4) launch.

First-time spenders get a $5,000 match on the first $5,000 spent with code BEEHIIV5K. Terms apply.

ResearchAudio · Paper of the week

This AI can only read 8K tokens. It got 100% on 128K-token tests. No training.

7-minute read · 9 diagrams · Paper: arXiv 2611.10463

Llama3-8B has a short memory. It can read about 8,000 tokens at one time. That is a few pages of text.

Now give it a document of 128,000 tokens. That is the length of a full book. It should fail.

In a new paper from Microsoft Research Asia, it does not fail. On the recall tests, it gets 100%. It beats models that were re-trained on long text with hundreds of GPU-hours. And nobody changed a single weight in the model.

The trick is not a bigger brain. It is a better toolkit around the brain. Also, watch for Part 5. There, the AI’s first “improvement” made the results worse on long documents.

In this issue

1. Why long documents are hard (and the usual fix)
2. The big idea: split the job, keep notes outside
3. What a “harness” looks like inside
4. The results: where it wins, and where it does not
5. The step that went backwards
6. The cost, and the limits

Part 1 of 6

Why long documents are hard

A long task is not only “read a lot of text.” The model must find the important parts. It must remember results from the middle. Then it must put all the pieces together into one answer.

The usual fix is expensive. Teams make a large set of artificial long documents (“data synthesis”). Then they train the model again on that data (“post-training”). Methods such as NExtLong, EntropyLong, and PolicyLong work this way.

The authors say this path is costly, slow, and based mostly on trial and error. So they ask a different question: what if the model already has the skills, and only needs better tools?

Diagram 1 · Two ways to handle long text

Usual way: change the model

Make lots of long training data
↓
Train the model again on GPUs
↓
New model with a
bigger memory

LongCapHarness: change the tools

A strong AI writes small tools
↓
Tests and fixes them on examples
↓
Same small model,
now with a toolkit

Part 2 of 6

The big idea: split the job, keep notes outside

Think of a student with a 300-page book and a hard question. A bad student tries to hold the whole book in their head. A good student uses an index, writes notes on cards, and then reads the cards.

LongCapHarness turns the AI into the good student. The big task becomes small steps. Each step gets only the text it needs. The results go into notes outside the model’s memory. So the length of the document no longer depends on the length of the model’s memory.

Diagram 2 · One big call vs. many small calls

✗ One big call

128K-token document → 8K model
does not fit

✓ A harness

Step A
small piece of text
→ Step B
small piece of text
→ Step C
small piece of text
→ Answer
📝 Shared notes (kept outside the model): each step writes its result here, and the next step reads it.

The key sentence from the authors: a model without long-context training can still do well on long tasks, if it has the right reasoning tools.

Part 3 of 6

What a “harness” looks like inside

The paper uses four words. Here they are in plain English:

• Capability: one kind of job. For example, “put these passages in order of relevance.”
• Primitives: the small steps of that job. Each step is either plain code or a short LLM call.
• Control logic: the order of the steps, and when to stop.
• State: the shared notes between steps.

The rule for code vs. LLM is simple. If the rule is clear (count, sort, look up), use code. If the step needs to understand meaning, use the LLM.

Diagram 3 · Example: the “Reranking” harness
Capability: Reranking
Question + many passages → passages in the best order
↓ split into ↓
Primitive 1: Relevance judgement
LLM CALL
“Is this passage useful for the question?” Needs understanding.
→ Primitive 2: Ordering
PLAIN CODE
Sort the passages by the scores. A clear rule. No AI needed.

Other primitives in the paper: keyword matching, lookup, following relations, tracking state, counting, rule inference, classification, picking evidence, and joining evidence.

Who builds these harnesses? A strong AI agent (GPT-5.6-Sol). It works in a loop called recursive self-improvement: it builds a harness, runs it, compares the answer with the correct answer, and fixes the harness. Then it repeats.

When a harness is good enough, it is frozen and saved in a library with a short description. Later, the small model reads the descriptions, picks the right harness, and runs it.

Diagram 4 · Build once, use many times

Phase 1: Build

2,242 practice examples

Strong AI writes a harness
↓
Run it. Compare with the correct answer.
↓
Fix the steps, order, or notes
↺ repeat until close enough
Freeze → save to library

Phase 2: Use

59,826 new test examples

New question arrives
↓
Small model picks a harness from the library
↓
Harness runs: code steps + small LLM calls
↓
Answer
Keep reading: In Part 5, the AI replaces a 1990s-style keyword search with a smart LLM judge. On long documents, the smart version scores lower. Then the AI finds a fix that adds 26.75 points.

Part 4 of 6

The results

The team tested on three long-context benchmarks: HELMET, RULER, and LongBench-v2. They used two small models to run the harnesses: Llama3-8B and Qwen2.5-3B.

They compared against three groups: the plain model, models re-trained for 128K tokens, and RRSI, another method where an AI improves its own setup without training.

Diagram 5 · Average score, HELMET + RULER (out of 100)

Llama3-8B (8K memory)

LongCapHarness
79.3
NExtLong (re-trained, 128K)
70.2
RRSI (no training)
39.7
Plain model
15.1

Qwen2.5-3B (32K memory)

LongCapHarness
78.7
PolicyLong (re-trained, 128K)
64.1
Plain model
61.9
RRSI (no training)
38.5

The harnesses were built with Llama, then moved to Qwen. The score fell by only 0.59 points.

Why did RRSI do so badly? The authors read its code. RRSI kept trying to cut the text into pieces by length, like cutting a book every 10 pages. That is the physical shape of the text.

LongCapHarness starts from the logical shape of the task: what steps does the question need? That difference explains most of the gap.

But the average hides a split. Some task types improve a lot. Others do not improve at all.

Diagram 6 · Scoreboard by task type (Llama3-8B)
Task type Harness Best other Result
Recall · find a fact100.082.6BIG WIN
RULER · track, count, follow links94.687.4WIN
Rank · order passages39.731.5WIN
ICL · learn from examples83.983.5TIE
LongQA · answer about a book36.837.3TIE
RAG · answer from search results59.764.6LOSS
“Best other” = the highest score from NExtLong, EntropyLong, or RRSI for that row.

The pattern is clear. The harness wins where code can do the work: matching keywords, looking things up, counting, tracking. Code does not get tired at page 300. That is why Recall stays at 100% from 8K to 128K tokens.

The harness does not win where the task needs judgement about meaning: understanding examples, or joining evidence into an answer. In the authors’ words, harnesses organize these steps, but do not replace the judgement.

LongBench-v2 shows the same split. On medium documents (32K–128K tokens), the harness wins clearly. On very long documents (up to 2M tokens), the re-trained model still wins.

Diagram 7 · LongBench-v2 with Qwen2.5-3B

Medium: 32K–128K tokens

Harness
34.5
PolicyLong
27.1

Long: 128K–2M tokens

Harness
27.9
PolicyLong
31.5

+7.4 points on medium. −3.7 points on long.

Part 5 of 6

The step that went backwards

This is my favourite part of the paper. The authors show how one harness (Reranking) changed over three versions.

V0: classic keyword scoring (BM25), then sort. No AI at all.
V1: the LLM gives each passage a score, then sort. This sounds smarter.
V2: the LLM compares passages in small groups, then the code merges the groups.

You would expect V1 to beat V0. On short documents, it does. On long documents, it does not.

Diagram 8 · Reranking score, three versions (Qwen2.5-3B)

Short documents (8K tokens)

V0 · keywords
53.3
V1 · LLM scores
68.1 ↑
V2 · compare + merge
71.8

Long documents (128K tokens)

V0 · keywords
28.9
V1 · LLM scores
17.1 ↓ worse than keywords
V2 · compare + merge
43.8 (+26.75 vs V1)

Average over all lengths: V0 40.4 → V1 40.5 → V2 56.3.

The lesson: putting an LLM into one step is not enough. The way the steps fit together matters too. V2 changed both the question to the LLM (“which is better?” instead of “give a score”) and the code around it (group, then merge).

One honest note: with Llama3-8B, V2 helped much less (only +0.72 points on average). The best design depends on the model that runs it.

Part 6 of 6

The cost, and the limits

Building the harnesses used about 814 million input tokens on GPT-5.6-Sol. That sounds like a lot. But compare it with renting H100 GPUs to re-train a model:

Diagram 9 · Estimated cost to get ready (USD)
NExtLong training
$3,696
PolicyLong training
$1,859
LongCapHarness build
$541
RRSI build
$332
1.87M vs 89.6M
Output tokens at test time with Llama (harness vs RRSI). Code does the work, so the model writes much less.
$53.74 vs $29.39
Test-time cost with Qwen (harness vs RRSI). Here the harness costs more, because it reads more input.

So the build cost is less than one third of the cheaper training method. Running cost is about the same with Llama, but higher with Qwen.

Now the limits. Read these before you get excited:

It needs a strong builder. The harnesses were built by GPT-5.6-Sol. When the small models tried to build their own harnesses, performance dropped sharply. The authors say this is still an open problem.
Judgement is still the weak point. Where a task needs understanding (RAG, long-book QA), the harness does not help much. On very long inputs (128K–2M), the re-trained model still wins.
Practice matched the test. The practice set included examples of every task type in the test. The test questions were new, but the task types were not. Results on a truly new task type are not shown.
Costs are estimates. All dollar values use list prices and rental rates, not real bills.

The takeaway

When a model fails at long text, our first idea is: train it more.

This paper shows a second idea: give it better tools. Let code do what code does well. Let the model do only the parts that need understanding. Keep the notes outside.

It does not replace training for every task. But for find, count, track, and sort, a small model with a good harness beat models trained for 128K tokens, at less than a third of the cost.

Quick facts
PaperLongCapHarness: Capability-Guided Auto-Harnessing for Long-Context Reasoning
AuthorsTao Zhang, Yuhang He (Microsoft Research Asia)
Posted7 November 2026
Builder modelGPT-5.6-Sol
Runner modelsLlama3-8B, Qwen2.5-3B
BenchmarksHELMET · RULER · LongBench-v2 (2,242 practice / 59,826 test examples)
Read the full paper →

One question for you: for your own work, would you rather train a bigger model or build better tools around a small one?

Hit reply and tell me. I read every answer.

ResearchAudio · Hard research papers, in simple English.
Forward this to one friend who builds with LLMs.