The Bitter Lesson of Tool Calling
Across BFCL v4, models usually handled tools as typed Python calls at least as well as native JSON, suggesting code-based orchestration is worth testing for capable coding agents.
The study compares programmatic tool calling with native JSON calls across **14 models** on BFCL v4. Python-stub calls matched or beat JSON in **11 of 14 models**, while the GPT-5.6 family improved by **10.6%**.
If your agents already write reliable code, test a typed-stub execution layer that lets one turn chain or parallelize calls. It also matched or exceeded JSON in **13 of 14 models** under parallel fan-out and stayed stable in the reported context-rot condition.
The study compares programmatic tool calling with native JSON calls across **14 models** on BFCL v4. Python-stub calls matched or beat JSON in **11 of 14 models**, while the GPT-5.6 family improved by **10.6%**. If your agents already write reliable code, test a typed-stub execution layer that lets one turn chain or parallelize calls. It also matched or exceeded JSON in **13 of 14 models** under parallel fan-out and stayed stable in the reported context-rot condition. Performance tracked model capability, so programmatic calls are not automatically better for every model. The evidence comes from one established function-calling benchmark; production safety, debugging, sandboxing, and task-level cost still need separate evaluation.
This makes tool-call representation a consequential harness variable rather than neutral syntax: capable code-writing models may perform better through typed programmatic stubs, including parallel calls. The result supports testing both interfaces per model, but does not justify replacing JSON calls without production evaluation of safety, observability, debugging, and cost.