How Mercor ships internal apps and billions of production tokens on one platform
Mercor describes using gateway failover, scoped model keys, and narrow agent permissions to handle bursty workloads and internal agents. Its reported batch times fell from over 12 hours to 2–3 hours.
Mercor says AI Gateway routed **18 million requests and 63 billion tokens in 10 days** without rate-limit hits. With additional inference capacity and provider failover, jobs that had taken more than 12 hours now take **2–3 hours**. Per-user keys can carry expiry, token, and spend limits.
For coding-agent builders, the operational pattern is to give experiments bounded credentials, track usage by user and project, and scope each agent’s permissions to its task. Mercor keeps model-use limits separate from the systems and actions an agent can access.
Mercor says AI Gateway routed **18 million requests and 63 billion tokens in 10 days** without rate-limit hits. With additional inference capacity and provider failover, jobs that had taken more than 12 hours now take **2–3 hours**. Per-user keys can carry expiry, token, and spend limits. For coding-agent builders, the operational pattern is to give experiments bounded credentials, track usage by user and project, and scope each agent’s permissions to its task. Mercor keeps model-use limits separate from the systems and actions an agent can access. These are results from Mercor’s workload and Vercel’s case study; they do not establish that the same latency or capacity gains will apply elsewhere. Mercor also continues to use multiple gateways and vendors.
Mercor’s reported traffic and job-time reductions offer a workload-specific example of gateway capacity, failover, and bounded per-user credentials in production. The figures do not establish similar gains elsewhere, and the account still uses multiple gateways and vendors, so the case supports an operating pattern rather than a universal platform choice.