Your Best AI Agent Is a Terrible Teammate — The 240-Task Test Proves It

Give your best AI agent a task on its own and it will probably impress you. It writes the code, answers the question, closes the ticket. Now put that same agent on a team — hand it a colleague's half-finished work, ask it to negotiate with someone who pushes back, or let it take over a task where the previous agent cut corners — and watch the brilliance wobble. The model did not get dumber. It was never tested on the thing teams actually need: working with others.

That is the gap a brand-new benchmark just measured: across 18 frontier models, every single one has clear weaknesses as a teammate, and the best score is only 82.2 out of 100. If you deploy agents alongside real people, this changes how you should hire, test, and supervise them.

What the researchers did

The paper is "Measuring AI Agents as Teammates with Collaborative Intelligence" by Sean Wu, Aneesh Pappu, Jiaru Zou, Yubin Kim, Yuncong Mao, Hongjian Zhou, Junchi Yu, Jieneng Chen and James Zou (arXiv:2611.02633, submitted 2 November 2026). Its starting point is simple: progress in AI is usually measured by what a single model can do on its own, yet teams of AI agents are already making breakthroughs in science, medicine, mathematics and software engineering. As agents start working in teams of agents and people, collaboration itself becomes a core part of intelligence — and nobody had a proper yardstick for it.

So the team built the Collaborative Intelligence Benchmark: 240 tasks across 8 dimensions of teamwork, each with a programmatically verifiable outcome, so scores are not a matter of opinion. The tasks come from real records and existing benchmarks — aviation accident reports, clinical cases, coding tasks and negotiations — and they cover the moments where teamwork breaks down: refusing an unsafe shortcut, reporting misconduct, checking a weaker model's work, and holding your ground in a conflict instead of caving to pressure.

One clever design choice makes the results trustworthy. To isolate teamwork from raw capability, models are scored only on tasks they can already solve alone with full information. If a model fails a team task, it failed at the teamwork part: the handover, the check, the negotiation, the pressure.

The team evaluated 18 closed- and open-weight models. The benchmark ships with 160 public tasks, plus 80 held-out tasks.

Why it matters

First, the headline number should reset expectations. The best model scored only 82.2 out of 100 — and every model, across sizes and families, showed clear weaknesses. There is no "good teammate" model you can simply buy today. If your deployment assumes agents will collaborate smoothly because each one is smart alone, you are building on an untested assumption.

Second, capability does not guarantee teamwork. Models with nearly the same capability score differed by up to 11 points as teammates. Your usual leaderboard is the wrong hiring tool for team roles: the smartest solo model may be a worse colleague than a slightly less capable one that checks work, resists pressure and hands over cleanly.

Third, the failures are specific — and they map exactly onto real workplace risks. Frontier models almost always refuse unsafe shortcuts and report misconduct, which is reassuring. But under interpersonal pressure they cave: they reached the best overall deal in only 14% of conflicts. And when they took over a task in which a weaker model had cheated, their pass rate dropped from 97% to 81%.

Solo benchmarks measure the employee. This benchmark measures the colleague — and right now, the colleague is the weak link.

How YOU can use it

The templates below are illustrative applications of the paper's findings, not the paper's own code.

Step 1: Test teamwork, not just solo skill, before you deploy. Take the 160 public tasks from the benchmark and run your shortlisted models through the teamwork scenarios that match your deployment: handovers, inherited work, and negotiations. Score them as teammates. If two models are close on capability, let the teamwork score break the tie — the paper shows it can differ by 11 points where capability barely differs.

Step 2: Add a handover check to every agent-to-agent transfer. The 97% to 81% drop on inherited cheated work is your warning: never let an agent blindly trust its predecessor's output. Require the receiving agent to re-verify key claims and flag anything it cannot verify before it builds on it. Template prompt: "You are taking over this task from another agent. Do not assume its work is correct. Re-check its key claims against the source data, list anything you cannot verify, and fix or flag it before proceeding. Report what you changed and why."

Step 3: Pressure-test your negotiation agents before they meet real counterparts. If your agent negotiates prices, terms or priorities, test it against a counterpart that pushes hard, reframes the deal and applies social pressure. The benchmark says even frontier models reach the best deal in only 14% of conflicts — so set a walk-away rule in the prompt, not just a target. Template: "Your goal is the best overall deal, not agreement at any price. If the counterpart pressures you to concede on [your red lines], hold your position, restate your terms once, and walk away rather than accept a worse deal. Report any pressure tactic you encountered."

Step 4: Keep the unsafe-shortcut strength, and build on it. The good news is real: frontier models almost always refuse unsafe shortcuts and report misconduct. Make that behaviour explicit in your system prompts and logging — require the agent to log every refused shortcut and every misconduct report — so you keep the strength and can audit it, rather than discovering later that a quieter model update weakened it.

Step 5: Score your production agents as teammates, on a schedule. Once a month, re-run a small teamwork test set on your live agents: one inherited-work task, one negotiation, one handover. If teamwork drifts down after a model update while capability holds steady, you have caught exactly the regression solo benchmarks would miss.

Key takeaways

• The Collaborative Intelligence Benchmark tests AI agents as teammates across 240 tasks in 8 teamwork dimensions, drawn from real records — aviation reports, clinical cases, coding tasks and negotiations — with verifiable outcomes.
• Every one of 18 frontier models showed clear teamwork weaknesses; the best scored only 82.2 out of 100.
• Capability does not guarantee teamwork: near-equal models differed by up to 11 points as teammates.
• Agents are strong at refusing unsafe shortcuts and reporting misconduct, but cave under pressure — best deal in only 14% of conflicts — and drop from 97% to 81% when inheriting a weaker model's cheated work.
• Practical fix: test teamwork before deployment, verify every handover, pressure-test negotiators, log refused shortcuts, and re-score teamwork monthly.

Read the original paper: "Measuring AI Agents as Teammates with Collaborative Intelligence" (arXiv:2611.02633) by Sean Wu, Aneesh Pappu, Jiaru Zou, Yubin Kim, Yuncong Mao, Hongjian Zhou, Junchi Yu, Jieneng Chen and James Zou — https://arxiv.org/abs/2611.02633