From Tokenmaxxing to Trusted Throughput — Mingsheng Hong, Ironclad
Ironclad argues against optimizing token volume or spend alone: measure reviewed, merged code and fix review and CI bottlenecks before judging coding-agent ROI.
Ironclad combines vendor data to track token cost by team and individual, but treats that dashboard as a **smoke detector**, not a leaderboard. Its value metric evolved from open PRs to merged PRs, then to AI-assigned **complexity scores**.
Measure cost alongside code that survives review, tests, security checks, deployment, and customer use. Put AI review before human review, keep PRs small enough to inspect, and invest in flaky-test and CI latency reduction as generation volume rises.
Ironclad combines vendor data to track token cost by team and individual, but treats that dashboard as a **smoke detector**, not a leaderboard. Its value metric evolved from open PRs to merged PRs, then to AI-assigned **complexity scores**. Measure cost alongside code that survives review, tests, security checks, deployment, and customer use. Put AI review before human review, keep PRs small enough to inspect, and invest in flaky-test and CI latency reduction as generation volume rises. Merged-PR complexity is only a proxy for business value, and the scoring method has no standard definition. The framework suits teams past initial agent adoption; early cost controls could suppress useful experimentation before value is measurable.
This replaces token volume as a productivity signal with trusted throughput: reviewed, tested, secured, deployed code weighted by estimated complexity. It confirms that rising agent output makes review capacity and CI health first-class constraints, while narrowing the metric’s authority because complexity remains an undefined proxy for customer or business value.