How We Solved Agent Building — Andrew Qu, Vercel
Vercel’s data agent improved after replacing prescriptive specialist pipelines with a filesystem, Bash and company-specific context. Eve packages that pattern as conventions.
Vercel’s internal data agent began around **30% on its evals**. Moving from tightly scoped specialist agents toward a sandboxed filesystem agent with Bash, file access and company context **roughly doubled the eval score**, according to the talk.
Builders should test whether a capable agent can navigate domain files and a few general tools before encoding every workflow edge. Keep proprietary semantics in inspectable files and skills; use framework conventions only where they reduce runtime, deployment and channel plumbing.
Vercel’s internal data agent began around **30% on its evals**. Moving from tightly scoped specialist agents toward a sandboxed filesystem agent with Bash, file access and company context **roughly doubled the eval score**, according to the talk. Builders should test whether a capable agent can navigate domain files and a few general tools before encoding every workflow edge. Keep proprietary semantics in inspectable files and skills; use framework conventions only where they reduce runtime, deployment and channel plumbing. The results describe one internal data workload rather than a published general benchmark. Eve packages the final architecture, but the talk does not establish how much improvement came from the filesystem, newer models, revised context or other simultaneous changes.
This adds unusually concrete internal eval evidence to the filesystem-agent pattern, but does not isolate which architectural change produced the gain. Against the candidates, it supports testing a general agent with inspectable domain context before building specialist pipelines, while strengthening the need for stable task contracts, trace-based diagnosis and controlled comparisons before generalizing the reported improvement.