On Turn-Level Best-of-N Sampling for Scaling LLM Agents
Turn-level best-of-N sampling has a generator propose actions and a selector choose one at each step. In long-horizon terminal tasks, the paper reports 5–10% accuracy gains and more consistent attempts.
At each turn, a generator model proposes **N candidate actions** and a selector model chooses one to execute. Tests on long-horizon terminal tasks report **5–10% accuracy improvement** and greater consistency across repeated attempts, alongside fewer attempts and better instruction following, debugging, and testing.
For a workflow where each agent step can go wrong, consider testing candidate generation plus selection at the turn level. Compare task success and repeated-run consistency against a single-action baseline.
At each turn, a generator model proposes **N candidate actions** and a selector model chooses one to execute. Tests on long-horizon terminal tasks report **5–10% accuracy improvement** and greater consistency across repeated attempts, alongside fewer attempts and better instruction following, debugging, and testing. For a workflow where each agent step can go wrong, consider testing candidate generation plus selection at the turn level. Compare task success and repeated-run consistency against a single-action baseline. The paper evaluates off-the-shelf open-weight models on terminal tasks. The material does not specify the costs of generating and selecting candidates or show that the gains transfer to other agent tasks.
Turn-level Best-of-N adds a selection step within each agent task: generate several candidate actions, then choose one to execute. This gives a concrete reliability strategy to compare with routing approaches that select a model per turn or keep one model for the whole task. Its reported gains are limited to terminal tasks, and the material leaves the added inference cost unmeasured.