| researchaudio.io · Issue 18 · 2026-07-06 |
A new MCP patterns paper sets a 10-15 tool ceiling. Most production servers are already over it.
|
| A 2-author ICSME industry paper catalogues 5 MCP server patterns, 4 anti-patterns, and the production telemetry that says your server is probably too big. |
|
| Claude Haiku 4.5 picks the right tool 91% of the time when you hand it 10 tools. Drop to 15 tools and it picks right 87% of the time. Claude Sonnet 4 buys you another notch: 95% at 10 tools, holds above 90% up to 20, drops below at 30. |
| That is the load-bearing finding in the new ICSME 2026 industry paper, “MCP Server Architecture Patterns for LLM-Integrated Applications” (Rodrigues & Vas, arXiv 2606.30317, submitted 2026-06-29). The five patterns get the press. The numbers are the story. |
|
The load-bearing finding
10 tools.
91% accuracy.
15 tools. 87%.
Most production MCP servers ship 20–50+ tools per context. The number is wrong.
|
|
What the paper says |
| Rodrigues (Celabe, operator of the ANSYR voice AI platform) and Vas (University of Waterloo) catalogue the architectural shapes MCP servers are taking in production. The paper is a patterns book in the Gamma et al. tradition: context, problem, solution, consequences, known uses. Five patterns, four anti-patterns, cross-cutting concerns. The contribution is not the structural skeletons — every pattern has a classical ancestor (Repository, Facade, Mediator, Adapter). The contribution is the LLM-client delta: the constraint that the client selects tools by reading natural-language descriptions, not by inspecting schemas or reading documentation. |
The corpus is fifteen servers: five production servers from ANSYR (anonymized Server-A through E) plus ten from the official modelcontextprotocol/servers registry. Coding was single-author open coding with secondary verification; an inter-rater reliability study on 54 held-out servers (Section VI-A) was the mitigation. |
The five patterns |
| 1 | Resource Gateway | Read-mostly backend data through a sanitization layer. MongoDB connector, GitHub issues-as-read. |
| 2 | Tool Orchestrator | Multi-system workflow as a single named tool. Create-ticket → notify → post. |
| 3 | Stateful Session Server | Multi-turn context preserved server-side. Code editor agent (open → edit → save). |
| 4 | Proxy Aggregator | Single endpoint across N upstream servers, namespaced tool names. Enterprise MCP gateway. |
| 5 | Domain-Specific Adapter | Wraps an LLM-hostile complex API. Salesforce, Kubernetes, FHIR. |
|
The four anti-patterns |
1. The God Tool — do_anything(action, params). Selection accuracy collapses.
2. Unsanitized Resource Content — user content returned as resource data. “Ignore previous instructions” becomes an instruction, not data.
3. Synchronous Long-Running Operations — video encoding as sync call. Return a job ID; expose poll_job(id).
4. Missing or Vague Tool Descriptions — send_message with no description. LLMs pick tools by reading descriptions, not schemas.
|
| “Transport overhead is dominated by network RTT, not by the protocol layer.” |
| Section VI-B. In-host: under 1 ms. Same-region remote: 30 ms p50, 180 ms p99. Two to three orders of magnitude. |
|
|
The gap to the ceiling
|
What the numbers actually show |
| Three measurements. The patterns are the vocabulary. These are the proof. |
A. Inter-rater reliability (N=54 held-out servers) |
| Cohen's κ between two LLM raters | 0.76 (95% CI [0.62, 0.88]) |
| Raw agreement | 81.5% |
| Author-vs-rater (Haiku 4.5) | 68.5% |
| Author-vs-rater (Sonnet 4) | 75.9% |
| Pilot on author-written descriptions | 97% (measures wording, not taxonomy) |
|
| The 97% pilot is the misleading number. It was on author-written canonical descriptions that named their own architecture. Strip that out — descriptions like “stage changes, commit, show diffs, switch branches in a repository” — and the agreement drops to 69–76%. |
B. Transport latency (Table III, ms) |
| Transport | Method | p50 | p95 | p99 |
| stdio (local) | measured | 0.01 | 0.02 | 0.02 |
| streamable-http (loopback) | measured | 0.39 | 0.45 | 0.48 |
| streamable-http (same-region remote) | modeled | 30.4 | 80.4 | 180.4 |
| Stateful Session Server (remote) | modeled | 38.4 | 100.4 | 216.4 |
| Proxy Aggregator (remote, single hop) | modeled | 62.4 | 160.4 | 308.4 |
|
C. Tool-count vs. accuracy (observational, ANSYR production) |
| Tools in context | Claude Haiku 4.5 | Claude Sonnet 4 |
| 10 | 91% (245 ms median) | 95% (410 ms median) |
| 15 | 87% | above 90% |
| 20 | — | ≥90% |
| 30 | — | drops below 90% |
|
| Wilson 95% CIs within ±4 percentage points. Prior work (Gan and Sun, Kate et al.) places the wider degradation onset beyond ~30 tools. |
How it actually works |
| In plain English: MCP is a standardized way for an AI model to call tools, read files, and use templates from a server. The server is a program that exposes those capabilities; the client is the AI app. The same server works with Claude, GPT, Gemini, or any compliant agent. |
MCP is built on JSON-RPC 2.0 with three primitives: tools (callable functions with name, description, and JSON schema), resources (URI-addressed data, static or dynamic), and prompts (parameterized templates managed server-side). Two transports: stdio for local in-process, streamable-http (HTTP plus optional server-sent events) for remote. The LSP parallel is exact: a single LSP server works across VS Code, Neovim, Emacs. MCP aims for the same decoupling between agent and capability provider. |
|
The proxy aggregator path: when the catalog outgrows one server
|
Where it works, where it collapses |
| In plain English: the patterns are right; the reliability study is honest; the limitations section is the bit most readers will skip. |
| What it gets right. The LLM-client delta framing is the right mental model. Every one of the five patterns shows up in the public MCP registry. The tool-count study is production data, N=200 per bucket, Wilson 95% CIs within ±4 points. The limitations section is honest. |
| Where it collapses. The 0.76 κ is the ceiling, not the floor — the 97% pilot is on author-written descriptions that named their own architecture. Three of five transport rows are modeled, not measured. Single-coder derivation is admitted as a limitation. The corpus is fifteen servers from one org plus the official registry; more than 8,000 public MCP servers exist. |
| What the framing skips. The paper's biggest idea is the inversion of API design: the LLM picks a tool by reading its natural-language description, not by consulting documentation. This is load-bearing. The “Missing or Vague Tool Descriptions” anti-pattern is the most underappreciated of the four, because it operates not at the architecture level but at the description-string level. Practitioners who treat tool descriptions as documentation comments will find their servers underperform. |
|
Qualitative failures
No HN signal on this paper. Per the newsletter rule, this section runs the paper's own failure modes instead of invented community reaction.
|
|
Failure 1 — the 10-15 tool ceiling for current Haiku-class models. ANSYR production telemetry, Wilson 95% CIs within ±4pp. Most production MCP servers today bundle 20–50+ tools. The paper's own mitigation: the scoped Proxy Aggregator variant (per-context tool filtering).
Failure 2 — 0.76 κ on architecture-neutral descriptions. The 97% pilot masked the structural problem. Statefulness, domain logic, and read-style tool behavior are invisible from a function description. The authors' fix (treat them as cross-cutting attributes) is a workaround that moves the disagreement rather than resolves it.
Failure 3 — three pattern-boundary ambiguities the paper localizes but does not solve. (1) Statefulness invisible — every stateful server classified as Tool Orchestrator. (2) Domain logic invisible — adapters split between Tool Orchestrator and Resource Gateway. (3) Read-style tools resemble gateways — retrieval-oriented orchestrators reclassified as Resource Gateways. The recommendation: draw on implementation signals, not a capability list alone.
Failure 4 — single-coder derivation, admitted. First-author open coding with secondary verification, not independent dual coding. The reliability study is the mitigation; the limitation remains. The replication package makes it reproducible, but reproducibility of a single-coder derivation is reproducibility of one person's reading.
Failure 5 — three of five transport rows are modeled, not measured. Table III labels each row. The relative ordering is robust. The absolute numbers depend on the network and the deployment region.
|
What this means for |
|
Junior engineers.
Read the four anti-patterns first. The patterns are vocabulary. The anti-patterns are the failure modes you will actually hit on your first MCP server. The God Tool is the most common. Missing or vague descriptions is the second.
|
|
|
Senior engineers.
The 10-15 tool ceiling is the load-bearing finding. Audit your current MCP servers. Count tools per context. If you are over 15 for any user-visible context, the scoped Proxy Aggregator variant is the next refactor. Co-locate the MCP server with the client if you have any latency budget under 100 ms p50.
|
|
|
Hiring managers.
The skill you are hiring for is description-writing, not just schema-design. A senior MCP engineer who cannot write a one-paragraph tool description that an LLM picks correctly is not a senior MCP engineer.
|
|
|
Founders.
The transport finding is “don't use remote MCP for latency-constrained deployments.” Voice agents, real-time copilots will pay 30–180 ms per remote tool call. That is not an optimization; it is the architecture. Co-locate. And: the unsanitized-resource-content anti-pattern is a prompt injection surface.
|
|
The metric that actually matters |
| Tool selection accuracy at 10 / 15 / 20 / 30 tools per context, ANSYR production telemetry, N=200 per bucket, Wilson 95% CI within ±4pp. |
Claude Haiku 4.5 Claude Sonnet 4
───────────────── ─────────────────
10 tools 91% ████████████████████░░░ 95% ███████████████████░░░░
15 tools 87% ██████████████████░░░░░ ~93% ██████████████████░░░░░
20 tools ~85% █████████████████░░░░░░ ≥90% ██████████████████░░░░
30 tools ~78% ███████████████░░░░░░░░ <90% █████████████████░░░░░
────────────────────────────────────────────────────
90% threshold · 10-tool budget
|
| The number to walk away with: 10 tools per context. The practical accuracy boundary for current Haiku-class models, measured on production traffic, with the paper's own mitigation (scoped Proxy Aggregator, retrieval-over-tools) as the only safe way to exceed it. |
|
Share this
An LLM picks a tool by reading its description. Not by consulting documentation. Not by inspecting the schema. Treat tool descriptions as load-bearing artifacts, not code comments.
|
|
| The five patterns are vocabulary. The tool-count ceiling is the architecture. The descriptions are the operationally important bit. Treat all three accordingly. |
|
Reader challenge |
| 1. Count the tools in your largest MCP server. If you are over 15 in any user-visible context, the scoped Proxy Aggregator variant is the next refactor. What is the smallest refactor that would partition the catalog? |
| 2. Read the tool descriptions in your MCP server. Pick three that a human engineer would find obvious. Would a 12-year-old reading only the description know when to use them? If not, rewrite. |
| 3. For latency-constrained deployments, is your MCP server co-located with the client? If not, what is the network RTT p50 / p95 / p99 between them? If p99 is over 200 ms, the transport table says you are paying 200 ms per tool call before the LLM does any work. |
Next issue |
| Issue 19 looks at Gan and Sun's RAG-MCP paper (arXiv 2505.03275) — the retrieval-over-tools approach the MCP patterns paper cites as the mitigation. When retrieval-over-tools works. When the retrieval model itself becomes the bottleneck. |
| -- researchaudio.io |
