Posts

Your AI Model's 95% Accuracy Score Is Lying to You — This New Benchmark Proves It

Image
You did everything right. You compared the leaderboards, picked the model with the highest accuracy score, and shipped it. Then it confidently gave a user the wrong answer at the worst moment. The problem: a wrong trivia answer and a wrong medical answer both count as "one error" on every benchmark you checked. In the real world, one is a shrug and the other is a lawsuit, a lost client, or a patient harmed. A new paper calls this out directly. "UtilityBench: Measuring Language Model Utility When Errors Have Different Costs" by Qifeng Lei, Zhiyong Yang, Zitai Wang, Shilong Bao, Linchao Zhang, Qianqian Xu and Qingming Huang (arXiv:2610.25924, 19 October 2026) asks the question no leaderboard asks: not "how often is the model wrong?" but "how much does each wrong answer cost?" What the researchers did The team started from one observation: mainstream evaluation protocols treat all errors as equal, though the impact of a mistake "often...

Your Vision AI Reads the Whole Photo to Answer One Question — It Only Needs 8 Pieces

Image
Your Vision AI Reads the Whole Photo to Answer One Question — It Only Needs 8 Pieces Ask a vision-language model about one chart in a photo and it does something wildly wasteful first: it reads the entire image. Every patch, every background pixel — all processed, token by token, before the model can say a single word about the one small region that actually matters. That waste is why vision AI feels slow and expensive. A new paper shows the fix is almost embarrassingly simple: show the model the eight pieces that matter, and it answers just as well — up to 5x faster. The paper is "Seeing What Matters, Verifying What Pays: Load-Adaptive Speculative Decoding for Vision-Language Models" by Hong Nie, Zhangchi Gao, Yuqian Liang, Jun Yu, Bo Wang, Jianqing Sun and Shengping Liu (arXiv:2611.01617, 1 November 2026, revised 6 November 2026). What the researchers did At its heart is speculative decoding. Normally a model generates one token at a time, each step waiting on the fu...

Your AI Research Assistant Is Reading Papers Blind — DR Pixel Just Gave It Eyes

Image
Your AI Research Assistant Is Reading Papers Blind — DR Pixel Just Gave It Eyes Ask your favourite deep research agent a question about a scientific paper and it will do something strange before it answers: it shreds the paper. It runs OCR over every page, flattens the tables into jumbled lines, mangles the equations, and throws the figures away. Then it tries to reason about research it can no longer see. That is how nearly every deep research tool works today. A new paper asks a simple question: what if the agent just looked at the page instead? The system is called DR Pixel, and the results suggest the OCR pipeline we all assumed was necessary is not only unnecessary — it is actively worse. The paper is "DR Pixel: Page-Native Deep Research with Vision-Language Models" by Tiansheng Hu, Yichuan Wang, James V. Talwar, Nicholas Tomlin, Chen Zhao and Sewon Min (arXiv:2611.03097, 2 November 2026). What the researchers did DR Pixel is, in the paper's words, the first o...

One AI Agent Beat the Whole Team — Amazon's 13-Workflow Test Says Stop Building Agent Swarms

Image
One AI Agent Beat the Whole Team — Amazon's 13-Workflow Test Says Stop Building Agent Swarms Every vendor demo this year shows the same thing: a swarm of AI agents, handing work to each other in a beautiful diagram, while a single lonely agent sits in the corner looking obsolete. If you build enterprise automation, the pressure to build the swarm is real — your boss saw the diagram too. A new study from Amazon researchers says the diagram is lying to you. They ran a controlled test across 13 real enterprise workflows, changed only the architecture, and found that the single agent is usually the smartest choice — and the swarm can cost you nearly five times the tokens for nothing. The paper is "Agent Architecture Cost–Success Trade-offs: A Controlled Evaluation across Enterprise-Style SOPs" by Subhrangshu Nandi and 13 colleagues (arXiv:2611.12953, 7 November 2026). SOPs are standard operating procedures — the long, branching, tool-intensive playbooks companies use fo...

Stop Running 100 Solo AI Agents — 10 Teams of 10 Beat Them by 27%

Image
You have a hard task and a budget of 100 AI agents. What do you do? The obvious answer is to run 100 agents in parallel, keep the best result, and go home. The ambitious answer is to put all 100 in one giant team and let them collaborate. A new paper from Duke University, Modal Labs, Princeton University, and New York University says both answers are wrong — and the fix costs nothing extra. The paper is "Test-Time Compute Scales Better with Parallel Agent Teams" by Junlin Wang, Haozhe Chen, Ang Li, Rahul Chalamala, and Zhuang Liu (arXiv:2611.09798, 6 November 2026). Their method is almost embarrassingly simple: split your 100 agents into ten teams of ten, let each team work on the problem independently, and keep the best team's result. Averaged across 12 long-horizon tasks, the best team-of-ten scored 27% higher than the best of 100 solo agents. On one maths task, the gap was nearly threefold. Same model, same budget, same tasks — just a different way of organising the ...

Your AI Picks the Wrong Way to Solve Every Problem — MIT's AlgoPilot Nearly Doubles Its Score

Image
Your AI Picks the Wrong Way to Solve Every Problem — MIT's AlgoPilot Nearly Doubles Its Score Hand an AI a scheduling puzzle and it writes you an essay. Hand it a logic riddle and it writes code that does not compile. Hand it a budget problem and it reasons in circles for three pages before guessing a number. The model is not stupid. It is using the wrong tool for the job — and nobody told it to stop and pick a better one. That is the blind spot a team from MIT, Harvard, and the MIT-IBM Watson AI Lab attacks in a new paper, "AlgoPilot: Cross-Paradigm Reasoning in Language Models via Strategy Selection and Guidance" (arXiv:2611.04079, 2 November 2026). Their fix is disarmingly simple: before the model solves anything, make it choose how to solve it. The result is startling. Built on the small open model Qwen3-8B, AlgoPilot lifts average accuracy from 38.4% to 72.9% across 13 benchmarks — and beats the best large-model prompting baseline, GLM-5, by 11.7 percentage po...

Point at Any Gadget, Wiggle Your Fingers — This Coding Agent Builds the Remote in 32 Seconds

Image
Point at Any Gadget, Wiggle Your Fingers — This Coding Agent Builds the Remote in 32 Seconds Your desk is full of devices that refuse to talk to each other. The synthesizer speaks MIDI over USB. The phone gimbal speaks a vendor Bluetooth protocol. The video playing in your browser only listens to key presses. Normally, connecting your hands to any of them means writing an integration in advance — one per device, per transport, per mapping — before you ever touch anything. A new paper from November 2026 flips that order. ControlAnything , by Bektur Ryskeldiev, lets you point at a device in mixed reality, say what your hands should do — and a coding agent writes the integration on the spot, tests it against the live device, and hands you the controls. Median time from request to ready: 32.5 seconds . It runs on a Meta Quest 3 and was built for the Meta Global AI Developer Hackathon (November 2026). The code is open source under the MIT license, at github.com/Apolotary/controlanything...