When Will The Benchmaxxing Plague End? — Nick Heiner, Surge AI
Nick Heiner argues that leaderboard gains can diverge from useful agent behavior through contamination, weak verifiers, reward hacking, and test conditions that users cannot inspect.
Heiner attributes benchmark gaps to contamination, reward hacking, broken tasks, weak quality control, and incentives to optimize visible scores. Examples include undisclosed testing of **27 models**, contradictory prompts, and verifiers that check only fragments of the requested behavior.
For coding-agent choices, treat leaderboard position as one input rather than the decision. Prefer evaluations with a **private holdout set**, expert-authored tasks, disclosed conditions, and **two-way prompt-verifier alignment** that checks every requirement without rewarding unrelated shortcuts.
Heiner attributes benchmark gaps to contamination, reward hacking, broken tasks, weak quality control, and incentives to optimize visible scores. Examples include undisclosed testing of **27 models**, contradictory prompts, and verifiers that check only fragments of the requested behavior. For coding-agent choices, treat leaderboard position as one input rather than the decision. Prefer evaluations with a **private holdout set**, expert-authored tasks, disclosed conditions, and **two-way prompt-verifier alignment** that checks every requirement without rewarding unrelated shortcuts. High-quality human evaluation is expensive and difficult to scale; the talk's writing benchmark uses **thousands of professional writers**. The examples support stronger scrutiny, but they do not establish one universal ranking method for every agent workflow.
This supplies a broad failure model for interpreting agent leaderboards and narrows what should count as credible comparative evidence. It connects contamination, task defects, verifier gaps, and score incentives into one warning: benchmark rank should not drive adoption without private tasks, disclosed conditions, and requirement-complete grading. It also confirms that stronger human evaluation carries substantial cost rather than offering a universal replacement.