Back
VercelEngineering PostOfficial SourcebenchmarkNew
Run Terminal-Bench and other Harbor evals on Vercel Sandbox
Harbor can run repeatable agent benchmarks in isolated microVMs, parallelizing model comparisons while keeping secrets outside each sandbox.
VercelSep 17, 20262 min
Source Summary
Harbor can run Terminal-Bench and related evals in isolated Vercel microVMs, enabling parallel model comparisons without putting injected credentials inside each sandbox.
Practical Implication
Move repeatable agent evaluations off a constrained local machine, parallelize trials, and swap the gateway `--model` value to compare providers while keeping the benchmark command stable.
Agent-Ready Context
**Harbor 0.22.0 or later** can run Terminal-Bench, SWE-bench, tau3-bench, OSWorld, and other registry evals on Vercel Sandbox with `--env vercel`. Each trial gets an isolated **Firecracker microVM**. Move repeatable agent evaluations off a constrained local machine, parallelize trials, and swap the gateway `--model` value to compare providers while keeping the benchmark command stable. Network policy is enforced outside the VM, and optional secrets are attached only to matching outbound requests. The material does not quantify cost, startup overhead, concurrency limits, or result reproducibility.
Context Map
Rate This Item
Personal Note
No note yet. Notes are included in exported bundles.
Related — Every Edge Explained
No approved edges yet for this post.