GitHubGitHub RepoNeeds Review

radixark/miles

Miles targets large-scale RL post-training with decoupled rollout and training, rapid weight distribution, low-precision recipes, and connectors for coding-agent environments.

GitHub · Trending today

Open Source Open Markdown Open JSON

Source Summary

Miles **v0.1** is an open-source framework for large-scale LLM and VLM post-training. It combines SGLang rollouts with Megatron-LM training, supports asynchronous RL, and says in-loop weights can reach inference engines in seconds even at **trillion-parameter** scale.

Practical Implication

Model-training teams can use its connectors for coding and computer-use environments, task sandboxes, rollout routing replay, and recipes including GRPO, PPO, SFT, and on-policy distillation. Low-precision options include **MXFP8 and NVFP4**.

Agent-Ready Context

Miles **v0.1** is an open-source framework for large-scale LLM and VLM post-training. It combines SGLang rollouts with Megatron-LM training, supports asynchronous RL, and says in-loop weights can reach inference engines in seconds even at **trillion-parameter** scale.

Model-training teams can use its connectors for coding and computer-use environments, task sandboxes, rollout routing replay, and recipes including GRPO, PPO, SFT, and on-policy distillation. Low-precision options include **MXFP8 and NVFP4**.

This is infrastructure for teams post-training large models, not a drop-in improvement for everyday coding-agent users. The largest recipes and parallelism rely on Megatron-LM; FSDP2 exists for Hugging Face implementations but has narrower coverage.

Connected Context · Feed7 Judgment

Miles adds a systems layer for post-training agents and multimodal models at very large scale, joining rollout generation, asynchronous RL, weight propagation, sandboxes, and multiple training recipes. It does not itself establish better reasoning or computer use; it makes those improvements trainable and testable by well-resourced model teams, with the broadest path tied to Megatron-LM and narrower Hugging Face coverage.

Cliff: Learning Process Rewards from the First MistakeCliff proposes finer rollout supervision at the first reasoning error; Miles supplies general rollout and post-training infrastructure in which process-reward methods of that kind could be operationalized, though no direct integration is stated.Test-Time Scaling in Reasoning LLMs: Inference Regimes, Evaluation, and ReproducibilityMiles concerns training-time scale and rapid weight updates, while the paper requires precise accounting for inference-time scaling; together they separate post-training infrastructure claims from evaluation of the resulting model-and-inference system.AI4AI-Bench: Benchmarking LLM Agents in Algorithmic Design for Recursive Self-ImprovementMiles provides coding-environment connectors, sandboxes, and training machinery, while AI4AI-Bench shows that verified improvement to learning algorithms remains difficult; scalable infrastructure therefore does not establish recursive self-improvement capability.

Context Map

modelcodingresearch#reasoning#computer-use#agent-evals

Uncertainty

This is infrastructure for teams post-training large models, not a drop-in improvement for everyday coding-agent users. The largest recipes and parallelism rely on Megatron-LM; FSDP2 exists for Hugging Face implementations but has narrower coverage.