arXivPaperNeeds Review
AV-AIVAT: 74x Cheaper Agent Evaluation with Certified Anytime-Valid Stopping in Imperfect-Information Games
AV-AIVAT combines variance reduction with anytime-valid stopping, cutting the game samples needed to compare agents while preserving a recheckable confidence claim.
arXiv
Source Summary
AV-AIVAT combines AIVAT corrections with continuously monitored confidence sequences. Across **15 agent configurations** and **71,439 paired HUNL hands**, AIVAT reduced variance by a median **54×**.
Practical Implication
For costly agent comparisons, use sequential stopping rules instead of repeatedly checking ordinary confidence intervals. At 95% confidence and ±1 Big Blind precision, raw outcomes required a median **74×** more hands than corrected outcomes under AsympCS.
Agent-Ready Context
AV-AIVAT combines AIVAT corrections with continuously monitored confidence sequences. Across **15 agent configurations** and **71,439 paired HUNL hands**, AIVAT reduced variance by a median **54×**. For costly agent comparisons, use sequential stopping rules instead of repeatedly checking ordinary confidence intervals. At 95% confidence and ±1 Big Blind precision, raw outcomes required a median **74×** more hands than corrected outcomes under AsympCS. The 74× result is asymptotic screening, not exact finite-sample certification. EB-CS needs an independently justified payoff bound, and descriptive HUNL runs showed only a 1.37× stopping-time ratio.
Context Map
benchmark#agent-evals#agent-reliability#multi-agentUncertainty
The 74× result is asymptotic screening, not exact finite-sample certification. EB-CS needs an independently justified payoff bound, and descriptive HUNL runs showed only a 1.37× stopping-time ratio.