Computer-use models will agentify the web, not APIs — Dhruv Batra, Yutori
The long-tail web is unlikely to expose clean agent APIs. Browser agents need pixels as ground truth, with code and network access used opportunistically for speed rather than as universal substitutes.
The argument separates popular services with structured endpoints from a long tail of scanned PDFs, image menus, delayed network data, and state expressed only through rendered UI. It estimates roughly **200 million active websites** and about **1 billion total websites**.
Build computer-use systems that inspect pixels to verify outcomes, but let them read network responses or generate JavaScript when that is more efficient. For broad research, an orchestrator can dispatch multiple browser navigators in separate sandboxes and collect structured results.
The argument separates popular services with structured endpoints from a long tail of scanned PDFs, image menus, delayed network data, and state expressed only through rendered UI. It estimates roughly **200 million active websites** and about **1 billion total websites**. Build computer-use systems that inspect pixels to verify outcomes, but let them read network responses or generate JavaScript when that is more efficient. For broad research, an orchestrator can dispatch multiple browser navigators in separate sandboxes and collect structured results. The talk says current accuracy differences against frontier models are within statistical noise, while latency and cost are the stronger case for specialized models. The prediction of sub-penny, sub-100 ms browser tasks is an aspiration, not a demonstrated present capability.
This narrows the API-versus-GUI choice into a hybrid access strategy: use rendered pixels for user-visible truth, but exploit network data or generated code when efficient. It extends computer use toward parallel web research over the unstructured long tail, while making cost and latency—not demonstrated accuracy superiority—the present case for specialized models.