Scaling Laws for Tool-Use Agents via Controllable Environment Synthesis
The paper finds that training tool-use agents on more varied tools and databases can improve generalization more than adding tasks. Builders designing agent evals or synthetic data can use that as a guide for what to diversify.
The SynthAPI pipeline varies tools, tasks and databases across **458,752 tasks** and **849 models**. The paper finds that as data budgets grow, more tool and database variety helps more than simply adding tasks; increasing task complexity also improves results.
For builders training or evaluating tool-using agents, diversify the APIs and data they encounter, and include harder interactions as the dataset grows. Track whether gains transfer to held-out environments.
The SynthAPI pipeline varies tools, tasks and databases across **458,752 tasks** and **849 models**. The paper finds that as data budgets grow, more tool and database variety helps more than simply adding tasks; increasing task complexity also improves results. For builders training or evaluating tool-using agents, diversify the APIs and data they encounter, and include harder interactions as the dataset grows. Track whether gains transfer to held-out environments. The authors report improvements across four benchmarks, but the material does not give the benchmark scores or establish how these findings transfer to production agents or other training setups.
SynthAPI reports that growing tool and database variety, along with task complexity, helps more than simply adding tasks as data budgets increase. This makes environment coverage a training-set design choice, while the supplied evidence leaves transfer to production agents unresolved.