# How We Solved Agent Building — Andrew Qu, Vercel

Source: [AI Engineer](https://www.youtube.com/watch?v=9dYcwOkpCE8)  
Feed7 permalink: https://feed7.dev/p/how-we-solved-agent-building-andrew-qu-vercel-1qhqr4p  
Published: 2026-09-14T17:00:06.000Z  
Trust: Source Linked (source_linked)

## Why Included

Vercel’s data agent improved after replacing prescriptive specialist pipelines with a filesystem, Bash and company-specific context. Eve packages that pattern as conventions.

## Source Summary

Vercel’s internal data agent began around **30% on its evals**. Moving from tightly scoped specialist agents toward a sandboxed filesystem agent with Bash, file access and company context **roughly doubled the eval score**, according to the talk.

## Practical Implication

Builders should test whether a capable agent can navigate domain files and a few general tools before encoding every workflow edge. Keep proprietary semantics in inspectable files and skills; use framework conventions only where they reduce runtime, deployment and channel plumbing.

## Agent-Ready Context

Vercel’s internal data agent began around **30% on its evals**. Moving from tightly scoped specialist agents toward a sandboxed filesystem agent with Bash, file access and company context **roughly doubled the eval score**, according to the talk.

Builders should test whether a capable agent can navigate domain files and a few general tools before encoding every workflow edge. Keep proprietary semantics in inspectable files and skills; use framework conventions only where they reduce runtime, deployment and channel plumbing.

The results describe one internal data workload rather than a published general benchmark. Eve packages the final architecture, but the talk does not establish how much improvement came from the filesystem, newer models, revised context or other simultaneous changes.

## Connected Context

Feed7 judgment across 778 accumulated Signals:

This adds unusually concrete internal eval evidence to the filesystem-agent pattern, but does not isolate which architectural change produced the gain. Against the candidates, it supports testing a general agent with inspectable domain context before building specialist pipelines, while strengthening the need for stable task contracts, trace-based diagnosis and controlled comparisons before generalizing the reported improvement.

- [Agents Without Code: Skills, YAML, and Filesystems Replaced Python — Philipp Schmid, Google DeepMind](https://feed7.dev/p/agents-without-code-skills-yaml-and-filesystems-replaced-python-philipp-0t7a4s8) — The pull-request agent independently reaches the same minimal architecture of general tools, files and skills, while emphasizing that outcome evals remain necessary.
- [The Unreasonable Effectiveness of Separating the Task from the Model — Maxime Rivest & Isaac Miller](https://feed7.dev/p/the-unreasonable-effectiveness-of-separating-the-task-from-the-model-max-0vu7lvy) — DSPy’s separation of task contracts from models and harnesses is a prerequisite for determining whether the reported eval gain came from architecture rather than simultaneous implementation changes.
- [Improving Agents is a Data Mining Problem — Vivek Trivedy, LangChain](https://feed7.dev/p/improving-agents-is-a-data-mining-problem-vivek-trivedy-langchain-1cgibr8) — Trace mining provides the next diagnostic step: recurring production failures can seed replayable evals and identify whether context, tools, orchestration or models need adjustment.
- [The Building Blocks of GTM Orchestration — Arman Vaziri, Ramp](https://feed7.dev/p/the-building-blocks-of-gtm-orchestration-arman-vaziri-ramp-1mjpli3) — Ramp shows that extending this pattern across business workflows requires shared entity context and durable execution, not merely a filesystem and Bash.

## Context Map

- Layer: agent
- Domains: coding, data
- Topics: harness-engineering, skills, agent-evals

## Uncertainty

- The results describe one internal data workload rather than a published general benchmark. Eve packages the final architecture, but the talk does not establish how much improvement came from the filesystem, newer models, revised context or other simultaneous changes.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.