Harness design for long-running application development
An Anthropic harness for multi-hour app builds pairs a generator agent with a Playwright-driven evaluator to counter self-grading bias — a $200, 6-hour run versus $9 solo, and it got simpler on Opus 4.6.
**The gist** Anthropic built a GAN-style harness separating a generator from a **Playwright-driven evaluator**, plus a planner, for autonomous full-stack builds. A game-maker app took **6 hours and $200** through the harness versus **20 minutes and $9** solo; a simplified v2 on **Opus 4.6** built a digital audio workstation in 3h50m for **$124.70**.
**Why it matters** Models grade their own work leniently, so a separate evaluator that clicks through the live app catches bugs self-review misses. But the deeper lesson cuts the other way: each piece of scaffolding is a bet on what the model can't do yet, and **Opus 4.6** made the sprint structure and **context resets** unnecessary — audit your harness every model generation.
**The gist** Anthropic built a GAN-style harness separating a generator from a **Playwright-driven evaluator**, plus a planner, for autonomous full-stack builds. A game-maker app took **6 hours and $200** through the harness versus **20 minutes and $9** solo; a simplified v2 on **Opus 4.6** built a digital audio workstation in 3h50m for **$124.70**. **Why it matters** Models grade their own work leniently, so a separate evaluator that clicks through the live app catches bugs self-review misses. But the deeper lesson cuts the other way: each piece of scaffolding is a bet on what the model can't do yet, and **Opus 4.6** made the sprint structure and **context resets** unnecessary — audit your harness every model generation. **Watch out** The evaluator only pays off when tasks exceed the model's baseline, and it still missed **nested layout bugs** and couldn't judge audio at all. Even evaluation wording steers output — "museum quality" pushed designs toward **one converged aesthetic**.