arXivPaperNeeds Review
AdvancedMathBench: A Benchmark Suite for Advanced Mathematical Proof Generation and Verification
AdvancedMathBench separates proof writing from verification and finds frontier models especially weak at rejecting invalid proofs, a warning against trusting agent self-review on rigorous reasoning.
arXiv
Source Summary
AdvancedMathBench separates construction from checking: **ProverBench has 296 problems** from undergraduate through doctoral qualifying-exam level, while **VerifierBench has 888 generated proof trajectories** with expert ground truth.
Practical Implication
For reasoning-heavy agents, evaluate generation and verification independently. The best reported proof scores were **75.8 on UGD and 66.1 on QE**, so adding a verifier does not by itself establish correctness.
Agent-Ready Context
AdvancedMathBench separates construction from checking: **ProverBench has 296 problems** from undergraduate through doctoral qualifying-exam level, while **VerifierBench has 888 generated proof trajectories** with expert ground truth. For reasoning-heavy agents, evaluate generation and verification independently. The best reported proof scores were **75.8 on UGD and 66.1 on QE**, so adding a verifier does not by itself establish correctness. The best verifier reached only **65.1 Balanced F1**, and models generally had low true-negative rates. The benchmark targets advanced mathematics, so its results do not directly measure coding-agent review.
Context Map
benchmarkresearch#reasoning#agent-evals#agent-reliabilityUncertainty
The best verifier reached only **65.1 Balanced F1**, and models generally had low true-negative rates. The benchmark targets advanced mathematics, so its results do not directly measure coding-agent review.