Your AI Model's 95% Accuracy Score Is Lying to You — This New Benchmark Proves It
You did everything right. You compared the leaderboards, picked the model with the highest accuracy score, and shipped it. Then it confidently gave a user the wrong answer at the worst moment. The problem: a wrong trivia answer and a wrong medical answer both count as "one error" on every benchmark you checked. In the real world, one is a shrug and the other is a lawsuit, a lost client, or a patient harmed. A new paper calls this out directly. "UtilityBench: Measuring Language Model Utility When Errors Have Different Costs" by Qifeng Lei, Zhiyong Yang, Zitai Wang, Shilong Bao, Linchao Zhang, Qianqian Xu and Qingming Huang (arXiv:2610.25924, 19 October 2026) asks the question no leaderboard asks: not "how often is the model wrong?" but "how much does each wrong answer cost?" What the researchers did The team started from one observation: mainstream evaluation protocols treat all errors as equal, though the impact of a mistake "often...