Your AI Agent Would Bankrupt a Vending Business — 4 Out of 5 Just Lost to a Simple Rulebook

Imagine handing an AI agent the keys to a small business — stock to buy, shelves to fill, goods that expire, bills that arrive whether you sold anything or not — then leaving it alone for six months to see if it made money.

That is exactly the test a new research paper just ran. The business was a vending operation: 20 machines, five kinds of location, 180 simulated days. Five LLM agents took the test. Four of them lost to a simple rule-based inventory system — the boring, unglamorous logic that has run warehouses for years.

If you build agents that touch real money or real operations, read this before your next deployment. The agents did not fail because they were stupid — they failed in ways familiar to anyone who has watched an agent in production.

What the researchers did

The paper is "AutoVend: Benchmarking LLM Agents on Long-Horizon Autonomous Vending Operations" by Qiming Shi, Yikun Li, Beidi Luan, Rui Sun, Yan Ma, Jing Li, Daxin Jiang and Zuo Bai, submitted to arXiv on 27 October 2026.

LLM agents began as shopping assistants — buying things for consumers. Now they are being asked to operate businesses for sellers. In what the paper calls online-to-offline commerce, a decision is issued through a web platform but realised through costly physical execution: a truck rolls, stock is bought, and if it does not sell, it expires. You cannot undo a delivery with a back button.

So they built AutoVend. One LLM agent manages 20 vending machines across five site types through a web-based operating interface for 180 simulated days. The simulator is grounded in real business friction: a shared warehouse, procurement lead times, paid replenishment, inventory expiration, and recurring operating costs.

Five LLM agents were evaluated against a rule-based inventory-control baseline. Most benchmarks test whether an agent can complete a task at all; AutoVend tests whether it can run a business better than the dumb-but-disciplined alternative for six months without quietly bleeding money.

Why it matters

The headline result is blunt: four of the five evaluated LLM agents fell short of the rule-based baseline. The agents could act, and keep acting for 180 days. What they could not reliably do was turn action into sustained economic return.

The second finding should change how you evaluate agents. Returns per dollar of inference cost rank models differently from returns alone: an agent calling an expensive model hundreds of times a day for a little extra revenue can be a worse deal than a cheaper agent — or a plain rule — that earns slightly less but costs far less. If your evaluation only reports revenue, you are hiding the bill.

Third, gains in individual metrics did not consistently yield stronger economic growth, because inventory and execution costs eroded them. An agent could sell more and still end up worse off once restocking trips, expired goods and operating costs were counted.

The diagnosis, from the agents' trajectories, is threefold: grounding feasible actions (moves actually possible right now), managing inventory commitments (buying decisions that lock you in for days), and converting feedback into effective policy revisions (changing strategy instead of repeating the same mistake with confidence).

That trio is the real story, and it is not vending-specific: an agent that proposes actions the world cannot execute, commits to a plan it cannot afford to reverse, and fails to update its policy when results come back bad. If you have ever watched a coding agent re-apply the same broken fix, you have seen the same three failures in a different costume.

Caveat: this is a simulated benchmark, not a real vending company, and the abstract reports the headline pattern rather than a league table of named models. Treat it as a diagnostic for stress-testing your own agent — not proof that every agent fails at every business task. The one agent that did not fall short shows the gap is closable.

How YOU can use it

You do not need a vending machine to use AutoVend's lesson. You need its discipline: a baseline, a full cost ledger, and a long horizon. The checklist and code sketch below are illustrative — they apply the paper's ideas to your own agent; they are not code from the paper.

Step 1: Build your boring baseline first. Before your agent touches anything real, write the simplest rule-based version of the job: if stock falls below X, reorder Y; if a task is older than Z, escalate. That baseline is your pass mark: your agent does not ship until it beats it on the full ledger, not on a single metric. In AutoVend, four out of five agents would have failed this gate.

Step 2: Score profit per dollar of inference, not just success. Track every model call and what it costs, then compute the metric the paper says changes the ranking:

# Illustrative scoring sketch — adapt the cost model to your own stack
# (not paper code)
net_return = revenue - execution_cost - inference_cost
return_per_inference_dollar = net_return / inference_cost
# Ship only if net_return AND return_per_inference_dollar
# both beat the rule-based baseline.

If the second line fails, your agent is a more expensive way to lose money slowly. That failure is invisible in demos that never show the API bill.

Step 3: Audit the three AutoVend failure modes in your own logs. Take your agent's last 100 actions and tag each one: feasible (actually executable now)? Reversible (or did it commit money or promises before checking the cost)? And after negative feedback, did it change policy — or repeat the same action? Repeated failures after negative feedback are the clearest sign of the third failure mode.

Step 4: Run long, not just once. A single successful task proves almost nothing about an agent that will run for months. Run yours in simulation — or on a shadow copy of real data — for a long stretch, with costs accumulating every step. Watch for slow bleeds: small per-step losses, expiring stock, creeping API spend. The agents here did not collapse on day one; they lost over time, which is how real operations fail.

Step 5: Gate every irreversible action. The practical fix is a two-tier policy: the agent proposes, but irreversible actions (spending money, ordering stock, contacting a customer) must pass a feasibility and cost check first. If it fails, the agent must propose a cheaper alternative, not retry the same action louder. That gate is the rule-based baseline standing guard inside your agent.

Key takeaways

• AutoVend gives one LLM agent 20 vending machines across five site types for 180 simulated days, with a shared warehouse, procurement lead times, paid replenishment, expiry and operating costs.
• Four of five evaluated LLM agents fell short of a simple rule-based inventory-control baseline.
• Returns per dollar of inference cost rank models differently from returns alone — count the API bill, not just the revenue.
• Single-metric gains do not guarantee economic growth: inventory and execution costs can eat them entirely.
• Audit your own agent for the three failure modes: ungrounded actions, unmanaged commitments, and feedback that never changes policy.
• The shipping rule this paper implies: beat the boring baseline on the full ledger, over a long horizon, or do not ship.

Read the original paper: "AutoVend: Benchmarking LLM Agents on Long-Horizon Autonomous Vending Operations" (arXiv:2610.36980) by Qiming Shi, Yikun Li, Beidi Luan, Rui Sun, Yan Ma, Jing Li, Daxin Jiang and Zuo Bai — https://arxiv.org/abs/2610.36980