Back
arXivPaperNeeds ReviewbenchmarkNew
The Bitter Lesson of Tool Calling
Test typed Python tool stubs: they matched or beat JSON calls in 11 of 14 models on BFCL v4.
arXivAug 6, 20262 min
Source Summary
Across BFCL v4, models usually handled tools as typed Python calls at least as well as native JSON, suggesting code-based orchestration is worth testing for capable coding agents.
Practical Implication
If your agents already write reliable code, test a typed-stub execution layer that lets one turn chain or parallelize calls. It also matched or exceeded JSON in 13 of 14 models under parallel fan-out and stayed stable in the reported context-rot condition.
Agent-Ready Context
The study compares programmatic tool calling with native JSON calls across **14 models** on BFCL v4. Python-stub calls matched or beat JSON in **11 of 14 models**, while the GPT-5.6 family improved by **10.6%**. If your agents already write reliable code, test a typed-stub execution layer that lets one turn chain or parallelize calls. It also matched or exceeded JSON in **13 of 14 models** under parallel fan-out and stayed stable in the reported context-rot condition. Performance tracked model capability, so programmatic calls are not automatically better for every model. The evidence comes from one established function-calling benchmark; production safety, debugging, sandboxing, and task-level cost still need separate evaluation.
Context Map
Uncertainty
Automatically selected from source material; feed7 has not independently tested the claim.
Rate This Item
Personal Note
No note yet. Notes are included in exported bundles.
Related — Every Edge Explained
No approved edges yet for this post.