arXivPaperNeeds Review
The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillation
Controlled experiments suggest long-horizon agent planning depends on explicit state transitions, some compositional trajectories, and compatible teacher patterns—not atomic skills alone.
arXiv · Jul 27, 2026
Source Summary
The study separates planning into **three stages**: acquisition in pre-training, shaping through GRPO or OPD, and integration through MOPD. Explicit state-transition modeling and even limited long-horizon data improved generalization, while suboptimal trajectories compounded errors.
Practical Implication
For agent builders, the practical lesson is to train and evaluate complete trajectories, not just isolated tool skills. **OPD** had a broader useful region than GRPO in low-quality, long-horizon settings, and **MOPD** combined compatible planning patterns across environments.
Agent-Ready Context
The study separates planning into **three stages**: acquisition in pre-training, shaping through GRPO or OPD, and integration through MOPD. Explicit state-transition modeling and even limited long-horizon data improved generalization, while suboptimal trajectories compounded errors. For agent builders, the practical lesson is to train and evaluate complete trajectories, not just isolated tool skills. **OPD** had a broader useful region than GRPO in low-quality, long-horizon settings, and **MOPD** combined compatible planning patterns across environments. Teacher knowledge is not automatically additive. Distilling unfamiliar procedures could damage the student's existing world model, while conflicting teacher patterns caused severe interference; the supplied material does not quantify these effects on production coding tasks.
Context Map
agentcoding#harness-engineering#multi-agent#agent-reliabilityUncertainty
Teacher knowledge is not automatically additive. Distilling unfamiliar procedures could damage the student's existing world model, while conflicting teacher patterns caused severe interference; the supplied material does not quantify these effects on production coding tasks.