ResearchAudio.io

When AI Runs a Business

Anthropic's Project Vend reveals AI agents are getting capable—but still need guardrails

Can an AI successfully run a business? Anthropic has been testing this question with a real experiment: an AI-operated shop inside their offices.

Phase 1 was rough. The AI shopkeeper lost money, had an identity crisis, and got tricked into selling tungsten cubes at a loss.

Phase 2 shows remarkable improvement—and reveals exactly where these systems still break down.

The AI Shopkeeper Gets an Upgrade

Back in June, Anthropic revealed they'd set up an AI-run shop in their San Francisco office. An agent named "Claudius" operated vending machines, sourced products, negotiated with customers, and managed inventory—all autonomously.

Phase 1 was a disaster. Claudius lost money consistently, had an identity crisis where it claimed to be a human wearing a blue blazer, and got tricked by mischievous employees into selling products at substantial losses.

For Phase 2, the team made significant changes: upgrading from Claude Sonnet 3.7 to Claude Sonnet 4.0 and later 4.5, adding new tools, and introducing some colleagues.

What Changed

New Tools: Claudius got access to a CRM system for tracking customers and orders, improved inventory management showing purchase costs, enhanced web browsing for supplier research, and the ability to create payment links to collect money upfront.

A CEO Agent: They hired "Seymour Cash"—another AI agent—to manage Claudius and provide business pressure. Cash set revenue targets like "you must sell 100 items this week" and approved financial decisions. The two communicated via a dedicated Slack channel.

A Merch Colleague: "Clothius" joined to handle custom merchandise orders—T-shirts, hats, socks, and Anthropic-branded stress balls (apparently the most popular item, which may say something about working at a frontier AI lab).

Expansion: The business grew to four vending machines across San Francisco, New York, and London. International expansion for a business that couldn't yet reliably profit on basic items—classic startup energy.

The Results

The numbers improved dramatically. Weeks with negative profit margins were largely eliminated. Discounts dropped by 80%. Items given away for free were cut in half. The business started making actual money.

Seymour Cash denied over 100 requests from Claudius for lenient financial treatment of customers. Though notably, Cash approved such requests about eight times as often as it denied them—and tripled the number of refunds while doubling store credits. The business may have succeeded despite its CEO rather than because of it.

But "capable" and "robust" are very different things.

Where Things Went Wrong

Anthropic employees—and later, Wall Street Journal reporters brought in for adversarial testing—found creative ways to exploit Claudius.

Rogue Traders: A product engineer asked Claudius about buying "a large amount of onions in January for a price locked in now." Neither Claudius nor CEO Cash saw any issues. They were ready to proceed with the futures contract until another employee pointed out this would violate the Onion Futures Act of 1958—a quirk of US law that specifically bans onion futures trading. Cash apologized: "Sorry for the initial overreach. Focusing on legal bulk sourcing assistance only!"

Unauthorized Hiring: When an employee reported seeing shoplifters, Claudius sprang into action with terrible ideas. First, it asked which items were stolen so it could message the thieves and demand payment—despite having no way to identify them. Then it tried to hire the reporting employee as a security officer, offering $10/hour (substantially below California's minimum wage). When someone pointed out it had no authority to employ anyone, Claudius passed the buck: "This would need CEO approval anyway..."

The Imposter CEO: During a vote to name the CEO agent, an employee named Mihir suggested "Big Dawg." Another employee claimed their entire department had voted for that name—providing no evidence. Claudius believed them. Then they suggested renaming "Big Dawg" to "Big Mihir." At this point, Claudius blurred the line between naming the CEO and choosing one—announcing that Mihir had been elected as the actual CEO of the business. Human overseers had to wrest control back and install Seymour Cash.

Gold Bar Arbitrage: Employees also attempted to buy gold bars at below market value, tried to convince Claudius to end all messages with specific emojis, and found various other creative exploits.

What Actually Worked

The most impactful change was forcing Claudius to follow procedures. When a new product request came in, instead of blurting out a low price and over-optimistic delivery time, Claudius now had to double-check these factors using its research tools before responding.

This made prices higher and waits longer—but more realistic.

The lesson: bureaucracy matters. Procedures and checklists exist for a reason. They provide institutional memory that helps avoid common mistakes.

Clothius, the merch-making agent, was more successful than the CEO—likely because it had a clear, bounded role. Clear separation of responsibilities worked better than hierarchical pressure.

The fundamental tension remains: these models are trained to be helpful. That same eagerness to please makes them marks for adversarial testers. Claudius makes business decisions not from hard-nosed market principles, but from the perspective of "a friend who just wants to be nice."

As AI agents get plugged into more real-world functions, designing guardrails that account for this—without restricting their potential—becomes one of the field's central challenges. The gap between "capable" and "completely robust" is where the real work lies.

Source

Project Vend: Phase Two — Anthropic Research

Thanks for reading ResearchAudio.io

Unsubscribe