# How Mercor ships internal apps and billions of production tokens on one platform

Source: [Vercel](https://vercel.com/blog/how-mercor-ships-internal-apps-and-billions-of-production-tokens-on-one-platform)  
Feed7 permalink: https://feed7.dev/p/how-mercor-ships-internal-apps-and-billions-of-production-tokens-on-one-1xui2e3  
Published: 2026-11-03T04:00:00.000Z  
Trust: Official Source (official_source)

## Why Included

Mercor describes using gateway failover, scoped model keys, and narrow agent permissions to handle bursty workloads and internal agents. Its reported batch times fell from over 12 hours to 2–3 hours.

## Source Summary

Mercor says AI Gateway routed **18 million requests and 63 billion tokens in 10 days** without rate-limit hits. With additional inference capacity and provider failover, jobs that had taken more than 12 hours now take **2–3 hours**. Per-user keys can carry expiry, token, and spend limits.

## Practical Implication

For coding-agent builders, the operational pattern is to give experiments bounded credentials, track usage by user and project, and scope each agent’s permissions to its task. Mercor keeps model-use limits separate from the systems and actions an agent can access.

## Agent-Ready Context

Mercor says AI Gateway routed **18 million requests and 63 billion tokens in 10 days** without rate-limit hits. With additional inference capacity and provider failover, jobs that had taken more than 12 hours now take **2–3 hours**. Per-user keys can carry expiry, token, and spend limits.

For coding-agent builders, the operational pattern is to give experiments bounded credentials, track usage by user and project, and scope each agent’s permissions to its task. Mercor keeps model-use limits separate from the systems and actions an agent can access.

These are results from Mercor’s workload and Vercel’s case study; they do not establish that the same latency or capacity gains will apply elsewhere. Mercor also continues to use multiple gateways and vendors.

## Connected Context

Feed7 judgment across 1009 accumulated Signals:

Mercor’s reported traffic and job-time reductions offer a workload-specific example of gateway capacity, failover, and bounded per-user credentials in production. The figures do not establish similar gains elsewhere, and the account still uses multiple gateways and vendors, so the case supports an operating pattern rather than a universal platform choice.

- [AI Gateway logs now have a dedicated page](https://feed7.dev/p/ai-gateway-logs-1272t5j) — Mercor’s usage limits and reported workload results pair with request-level cost, latency, routing, and fallback logs, which can help teams inspect whether similar controls work in their own workloads.
- [AI Gateway now supports team and project spend budgets](https://feed7.dev/p/ai-gateway-spend-budgets-and-alerts-1969711) — Mercor describes expiry, token, and spend limits on per-user keys; team and project budgets add broader enforcement layers, while the case study does not say Mercor used that feature.
- [Routing LLM Inference in Production: From Engine Signals to Policy — Qianru Lao & Lu Zhang, OpenAI](https://feed7.dev/p/routing-llm-inference-in-production-from-engine-signals-to-policy-qianru-0ug6kw6) — Mercor reports using provider failover to shorten job times, while the routing talk describes global optimization and local guards for latency, capacity, and failure handling; the case does not establish that it used that architecture.

## Context Map

- Layer: infra
- Domains: None
- Topics: gateways, observability, agent-reliability

## Uncertainty

- These are results from Mercor’s workload and Vercel’s case study; they do not establish that the same latency or capacity gains will apply elsewhere. Mercor also continues to use multiple gateways and vendors.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.