arXivPaperNeeds Review

FOCUS: Active Critic Learning for Long-Horizon Language Model Agents

FOCUS uses selective branching to improve value estimates for long-horizon language-model agents. Its results suggest a way to assign credit across segments without spending the same rollout budget on every state.

arXiv · Oct 26, 2026

Open Source Open Markdown Open JSON

Source Summary

FOCUS samples continuations from trajectory states expected to reduce critic error, then uses the resulting values to score segments between states. With matched per-step generation budgets, it outperforms GRPO and VinePPO on the reported benchmarks. At Qwen3.5-9B scale, it improves held-out accuracy by **7.2 points** on BrowseComp-Plus and **4.1 points** on DeonticBench.

Practical Implication

Builders training agents across long tasks can consider whether selective branching and compaction-based segment boundaries help allocate rollout work and credit assignment.

Agent-Ready Context

FOCUS samples continuations from trajectory states expected to reduce critic error, then uses the resulting values to score segments between states. With matched per-step generation budgets, it outperforms GRPO and VinePPO on the reported benchmarks. At Qwen3.5-9B scale, it improves held-out accuracy by **7.2 points** on BrowseComp-Plus and **4.1 points** on DeonticBench.

Builders training agents across long tasks can consider whether selective branching and compaction-based segment boundaries help allocate rollout work and credit assignment.

The evidence is limited to the reported legal reasoning, deep search, and code localization benchmarks; the material does not establish results for deployed coding agents.

Connected Context · Feed7 Judgment

FOCUS allocates continuation sampling to states expected to reduce critic error, then uses those values to score trajectory segments. Its matched per-step-budget gains suggest selective branching can improve long-horizon training efficiency on the reported benchmarks, while the evidence does not establish gains for deployed coding agents.

Reinforcement Learning without Verifiable Rewards — Will Brown, Prime IntellectBoth address supervision for agent tasks where direct outcome rewards are limited: FOCUS learns critic values from selected continuations, while the talk describes deriving proxy rewards from traces, simulations, and judges. The talk’s warning about proxy reward hacking remains relevant to learned value estimates.RRSI: Regularized Recursive Self-Improvement of Agent HarnessesBoth report ways to spend agent-training compute more selectively: FOCUS chooses rollout continuations to improve critic estimates, while RRSI constrains harness proposals and reuse. They optimize different parts of the training process.LLM-as-a-Verifier: A General-Purpose Verification FrameworkThe verifier framework scores agent outputs for evaluation, whereas FOCUS trains a critic to value trajectory segments. The distinction matters when using outcome verification to assess whether FOCUS-style training improves final task results.

Context Map

agentcodingresearch#harness-engineering#agent-reliability#agent-evals

Uncertainty

The evidence is limited to the reported legal reasoning, deep search, and code localization benchmarks; the material does not establish results for deployed coding agents.