In partnership with

Ads charge per click. Getting cited doesn't.

Every click from an ad has a price, and the price resets tomorrow.

When your buyers ask ChatGPT, Claude or Perplexity what to use, the answer isn't an auction. It's built from the sources AI trusts.

ChatFeatured is the AEO platform that does the work to get you into those sources:

  • See which sources AI cites in your category.

  • See how often you're recommended versus your competitors.

  • Get the content that earns the citation written and published for you.

  • Watch the lift week by week.

A carpet dyeing company landed a large commercial customer that found it through ChatGPT. It's a client of Impact IQ Marketing, an agency that uses ChatFeatured.

ResearchAudio · Paper of the week

An AI built a brain model with no training. Then it found a hidden error in a famous dataset.

7-minute read · 9 diagrams · Paper: arXiv 2611.18114

Most AI models are black boxes. They give good answers. But nobody can say why.

For science, this is a big problem. A scientist does not only want to predict the brain. A scientist wants to explain the brain.

A new paper tries a strange idea. It tells an AI agent: “Build a neural network. But you cannot train it. You must write every number by hand.”

The result works. And in one test, the AI found a problem in a dataset that many labs use. I will show you that part in Part 5. It is the best part.

In this issue

1. The big idea, in one picture
2. How the loop works
3. The “200-word test” for clear models
4. Test one: maths and language tasks
5. Test two: the brain (vision, language, sound)
6. What does not work yet

Part 1 of 6

The big idea: a glass box, not a black box

Usually, we train a network with backpropagation. This is an automatic method. It changes millions of numbers (“weights”) a little at a time. The final numbers work, but they do not tell a story.

The new method is called iWSP (interpretable weight-space programming). An LLM agent writes the weights directly in code. Each change must have a reason that a person can read.

This is not a new dream. In 1943, McCulloch and Pitts set every connection of small networks by hand. iWSP brings that old idea back, with an AI doing the hand work.

Diagram 1 · Two ways to build a model

Usual way: training

Lots of data
↓
Backpropagation changes the weights
↓
■ BLACK BOX
Good answers. No reasons.

iWSP: writing by hand

Agent’s knowledge + a score
↓
Agent writes each weight in code
↓
□ GLASS BOX
Good answers. Clear reasons.

Part 2 of 6

How the loop works

The agent works like a human scientist. It makes a guess. It tests the guess. It reads the score. Then it makes a better guess.

The rules are strict. The agent cannot use optimizers, fitting loops, or downloaded pretrained weights. It can only use basic tools, such as PyTorch.

Diagram 2 · The iWSP loop
1. Task
“Sort these numbers” or “Predict this brain signal”
→ 2. LLM agent
Writes weights in code
NO TRAINING
→ 3. Fixed evaluator
Tests the model. The agent cannot change it.
→ 4. Score
Goes back to the agent
↺ Repeat 30 to 900 times. Each change has a written reason.
After the loop: a second AI does an audit. It reads every saved model and log. It looks for hidden training, rule breaks, and edits that a person cannot understand. Models that fail are removed from the results.

One more check: the team ran the weight-writing code again on changed task data. The weights stayed the same. This shows that the models did not secretly learn from the data.

Part 3 of 6

The “200-word test”

“Interpretable” is a soft word. How do you measure it? The authors use a clever test.

If you truly understand a machine, you can describe it in a short note. Then another person can build it again from that note alone.

Diagram 3 · The rebuild test
Best model
(original)
→ A note of 200 words or less
written by the AI
→ A fresh AI session
rebuilds the model from the note only
Question: Does the rebuilt model perform like the original?
100%
Maths tasks: all 13 rebuilt models got every hidden test right.
−1.4 pts
Language tasks: rebuilt models lost only about 1.4 points of accuracy.

There was also a control group. Here, a fresh AI rebuilt the model with no note. On maths tasks, those models were about 2,043 times bigger than the original. With the note, they were only 1.74 times bigger. So the note carries real information.

Keep reading: In Part 5, the score suddenly jumps from 0.067 to 0.104 in one step. The AI did not get smarter. It found something wrong with the data.

Part 4 of 6

Test one: maths and language

First, the team gave the agent a small transformer (3 layers, width 64). There were 17 tasks:

• 13 algorithm tasks, for example multiplication, sorting, edit distance, and dates.
• 4 language tasks: question type (TREC), sentiment (SST-2), paraphrase (MRPC), and logic between sentences (SNLI).

Result with Claude Opus 5.5: all 26 runs on the algorithm tasks got 100% on hidden tests. On 3 of 4 language tasks, the models did much better than a simple guess. The paraphrase task failed.

The most interesting part is what happened after the models were correct. The agent did not stop. It made the models smaller and simpler.

Diagram 4 · First correct, then small

Average algorithm run: number of parameters

First correct model
~3,400
After compression
~200

The models hit 99% accuracy in the first 10% of the run. The other 90% of the time went to making them small. In the median run, the final model was 29 times smaller than the first correct one.

Smallest 100%-correct model (median), by AI model · log scale

Opus 5.5
166
Sonnet 5.5
41,600
Haiku 5.5
100,000

All three AI models got about 100% on the algorithm tasks. But the strongest model made far smaller networks. Smaller here means easier to read.

What do these hand-written networks look like? Often, the agent puts a well-known algorithm into the weights. For example, take the task “find the greatest common divisor” (GCD).

The agent used a very old method: subtract the small number from the large number, and repeat. It built this as one block of only 3 ReLU units, repeated 1,000 times.

Diagram 5 · GCD of 84 and 36, inside the network
84, 36 → 48, 36 → 12, 36 → 24, 12 → 12, 12 → 0, 12

Each arrow = one pass through the same 3-unit block.
Rule: replace the pair with (difference, smaller number). Stop at 0. Answer: 12.

Language is harder. There is no textbook algorithm for “is this review positive?” So the agent built simple rules from standard transformer parts.

The best sentiment model had 1,515 hand-written word patterns. It also had a negator: an attention layer that looks back for words like “not” and flips the positive meaning after it. The authors call it crude. But it handles “not good” better than a simple word list.

Part 5 of 6

Test two: the brain

Now the real goal. Neuroscientists build encoding models. These models take a picture, a story, or a sound, and predict the brain’s response.

Here, the agent writes code that turns the input into features. Then a fixed step (ridge regression) links those features to the brain data. The agent never sees the brain data directly.

The team tested three senses:

👁 5a. Vision: beats the clear models, close to the black box

The agent built a model from fMRI scans of 5 people. They looked at 25,052 natural images. After 600 iterations, the final program had 865 lines and 14 named computations.

Then came the hard test. The team froze the model and used it on a different dataset (NSD) with different people. No changes were made.

Diagram 6 · Prediction score on a new dataset (higher is better)

Early visual cortex (edges, simple shapes)

iWSP
0.443
Best clear baseline
0.400
AlexNet (black box): about the same as iWSP here.

Higher visual cortex (faces, places, objects)

AlexNet (black box)
0.391
iWSP
0.317
All 7 baselines together
0.283
Best clear baseline
0.256

iWSP beat all 7 interpretable baselines in every one of the 4 new subjects.

Because the model is clear, we can ask: which computation does each brain area use? Nobody told the agent which area is which. But a pattern appeared on its own.

Diagram 7 · Different areas, different maths

Early areas (V1, V2, V3)

Use more shading (“surface geometry”) and edge energy (Gabor filters).

Surface geometry: 38% of the prediction in V1.

→

Higher areas (FFA, PPA)

Use more contour curvature and patterns that occur together.

Surface geometry falls to 15% in PPA. Curvature rises to 15% in FFA.

The same rebuild test passed here too. A 185-word note gave a rebuilt model with a score of 0.451. The original scored 0.488.

💬 5b. Language: the score jump that was a trap

This is the part I promised. People in a scanner listened to more than 20 hours of podcast stories. The agent had to predict the brain response from the words.

For a long time, the score went up slowly. It stayed below GPT-2 XL, a strong black-box model. Then, in one step, the score jumped from 0.067 to 0.104. That is above GPT-2 XL.

Was it a breakthrough? No. The team read the new code. The new features did not describe word meaning at all. They only described where the word was in the story.

Diagram 8 · The jump, and the cause
iWSP before the jump
0.067
GPT-2 XL (black box)
0.079
Story position only
~0.090
iWSP after the jump
0.104

One fMRI scan, start to end (average brain signal):

TOO HIGH real story response TOO LOW
cut ~40 TRs cut ~40 TRs

The signal starts too high and ends too low in every scan. This follows the clock, not the story. A model that knows “where am I in the scan?” gets a free bonus. It appeared in all 3 subjects.

The position features follow this edge pattern with a correlation of 0.97. The authors suggest a fix: remove about 40 TRs (one TR is 2 seconds) from each end of every scan, in training and in testing.

To be fair, the authors say the dataset is still of very high quality. Past findings that use word-level or short-context models stay valid. The problem matters most for models that use long context or story position.

Why this matters: a black-box model could get the same free bonus. You would never see it. You would just think your model is better. With a glass box, you can read the trick. After the fix, iWSP still built models that focus on meaning, and they performed on par with GPT-2 XL.

🎧 5c. Sound: finds what Whisper misses

The last test used ECoG: electrodes placed directly on the brain of a patient. There were 166 electrodes, recorded over 7 days. The input was the raw audio in the room.

The comparison was Whisper, a strong speech model. The agent ran 902 iterations. It started with a basic sound map (a cochleagram). Then it added known tools by hand, such as modulation filters, slow context over past seconds, and wavelet scattering.

Alone, iWSP was a little below Whisper. But the clear features found something different.

Diagram 9 · Better together (hidden test, all 166 electrodes)
iWSP
0.065
Whisper
0.070
Whisper + iWSP
0.075

Who explains what? (share of the combined model’s explained variance)

6% 69% shared 25%
▲ iWSP only Whisper only

126 of 166 electrodes improved when iWSP features were added to Whisper. One single feature, spectral flatness (how noise-like a sound is), reached 29% of the full model’s score alone.

The team also tested 5 more patients. Whisper won on every patient. But the iWSP features still added new signal in 4 of the 6 patients in total.

Part 6 of 6

What does not work yet

The authors are honest about the limits. Here are the main ones:

No new ideas yet. The agent combines known tools (textbook algorithms, Gabor filters). It does not invent new principles.
Shallow models. Deep, many-layer networks are hard to write by hand. This is likely why iWSP trails black boxes in higher vision, hard language tasks, and speech.
Trust in the notes. The method assumes the agent’s written reasons match its real edits. Stronger agents will need more checks.
Cost. All experiments cost about $17,500 at API list prices.
Not proof of cause. The brain studies use few subjects and no interventions. A model shows which computations predict the brain. It does not prove the brain uses them.

The takeaway

For years, the choice was: accurate or understandable. Pick one.

This paper shows a third path. Let an AI agent do the slow hand work that human scientists did before. Then check that a short note can rebuild the result.

It does not beat the best black boxes yet. But it already found a data problem that black boxes would hide. For science, that may be worth more than a few points of accuracy.

Quick facts
PaperInterpretable weight-space programming builds transparent transformers and brain encoding models
AuthorsYash Mehta, Chandan Singh, Weijia Xu, Richard J. Antonello, Adeen Flinker, Nima Mesgarani, Michael F. Bonner, Nebojsa Jojic, Jianfeng Gao
Posted13 November 2026
AI agents usedClaude Opus / Sonnet / Haiku 5.5, Claude Opus 4.7, Gemini 4 Carbon
Data17 benchmark tasks · LAION-fMRI + NSD (vision) · story-listening fMRI · ECoG audio
Read the full paper →

One question for you: would you trust a model more if you could read every weight?

Hit reply and tell me. I read every answer.

ResearchAudio · Hard research papers, in simple English.
Forward this to one friend who likes AI or neuroscience.