Direct Pass@$k$ Optimization for Thought Pruning: Balancing Quality and Diversity in Test-Time Reasoning
A pruning objective selects reasoning branches for their combined pass@k, retaining promising but nonredundant paths so test-time compute is spent on a stronger candidate pool.
The method scores a pool of partial solutions by the chance that at least one continuation will be correct. It was tested on **eight open-weight reasoning models** and outperformed quality-only and diversity-only pruning.
For agent workloads that sample several reasoning paths, optimize the surviving set rather than ranking branches independently. **Direct pass@k optimization** makes both predicted quality and overlap between candidates part of one selection criterion.
The method scores a pool of partial solutions by the chance that at least one continuation will be correct. It was tested on **eight open-weight reasoning models** and outperformed quality-only and diversity-only pruning. For agent workloads that sample several reasoning paths, optimize the surviving set rather than ranking branches independently. **Direct pass@k optimization** makes both predicted quality and overlap between candidates part of one selection criterion. The abstract provides no effect sizes or compute-cost comparison. Gains were strongest with a **moderate number of survivors**, so applicability to very small or very large candidate pools remains unclear.
This narrows test-time branch pruning to a set-level objective: retain candidates according to the probability that at least one continuation succeeds, rather than treating quality and diversity as separate heuristics. It supports evaluating reasoning as a multi-path inference system, but absent effect sizes and compute comparisons leave the practical advantage and useful pool sizes unresolved.