Can an Agent Debug a Web Corpus? Agentic Auditing of Slovak FineWeb2
A budgeted agent turned corpus anomalies into reusable tests, but static sampling found more valid selectors for less money. The useful pattern is agent discovery followed by deterministic checks.
Three agent runs inspected **26.5 million Slovak documents** through a read-only API and produced seven findings. Native-speaker review supported **6 of 7**, including a 1,408,437-document content farm.
Use the agent for exploratory debugging and have it compile findings into executable selectors. The resulting suite scanned the corpus in **0.7 seconds** without a GPU, making repeated checks deterministic and cheap.
Three agent runs inspected **26.5 million Slovak documents** through a read-only API and produced seven findings. Native-speaker review supported **6 of 7**, including a 1,408,437-document content farm. Use the agent for exploratory debugging and have it compile findings into executable selectors. The resulting suite scanned the corpus in **0.7 seconds** without a GPU, making repeated checks deterministic and cheap. The strongest static sampling baseline validated more selectors than any agent run at under half the cost. The agent added prevalence estimates, failure explanations, sampled precision checks, and reusable tests, but did not dominate discovery.
This qualifies the prior case for agent-led data inspection: agents were valuable for explaining failures, estimating prevalence, checking precision, and turning discoveries into reusable selectors, but a cheaper static baseline found more validated selectors. The practical pattern is exploratory model judgment followed by deterministic, replayable checks—not treating the agent as either the sole discoverer or the production scanner.