The Missing Layer in Agentic AI — Giedrius Šteimantas, Oxylabs
Web agents should validate fetched content before spending model tokens on it and reserve browser automation for interactive steps. Explicit failures beat silently passing CAPTCHAs or blocked HTML downstream.
In the example, only **3 of 10 pages** returned valid product content, so sending every response to the model wasted **70% of tokens** on blocked pages. Oxylabs’ search API returns compact JSON under **2,000 tokens** with average responses below **700 ms**.
Split web-agent work by need: use search for discovery, a scraper that validates and cleans content for verification, and a browser only for dynamic interaction such as checkout. Fail loudly on CAPTCHAs or blocks before content reaches the model, and preserve geolocation between verification and execution.
In the example, only **3 of 10 pages** returned valid product content, so sending every response to the model wasted **70% of tokens** on blocked pages. Oxylabs’ search API returns compact JSON under **2,000 tokens** with average responses below **700 ms**. Split web-agent work by need: use search for discovery, a scraper that validates and cleans content for verification, and a browser only for dynamic interaction such as checkout. Fail loudly on CAPTCHAs or blocks before content reaches the model, and preserve geolocation between verification and execution. The performance and reliability claims describe Oxylabs’ own products and one shopping-agent reconstruction. The talk provides no independent comparison, and protected sites, purchase flows, and regional behavior can still change.
This moves web-agent reliability ahead of model reasoning: discovery, verification, and interaction should use different access paths, and invalid pages should fail before consuming context. It supports a hybrid web stack but narrows browser automation to genuinely interactive work, while the vendor-specific latency and token figures remain insufficient to establish general superiority across protected, regional, or changing sites.