{
  "schema_version": "1.0",
  "id": "archive:https://www.anthropic.com/engineering/harness-design-long-running-apps",
  "slug": "harness-design-long-running-apps-15vc0wu",
  "url": "https://feed7.dev/p/harness-design-long-running-apps-15vc0wu",
  "title": "Harness design for long-running application development",
  "why_included": "An Anthropic harness for multi-hour app builds pairs a generator agent with a Playwright-driven evaluator to counter self-grading bias — a $200, 6-hour run versus $9 solo, and it got simpler on Opus 4.6.",
  "summary": "**The gist** Anthropic built a GAN-style harness separating a generator from a **Playwright-driven evaluator**, plus a planner, for autonomous full-stack builds. A game-maker app took **6 hours and $200** through the harness versus **20 minutes and $9** solo; a simplified v2 on **Opus 4.6** built a digital audio workstation in 3h50m for **$124.70**.",
  "practical_implication": "**Why it matters** Models grade their own work leniently, so a separate evaluator that clicks through the live app catches bugs self-review misses. But the deeper lesson cuts the other way: each piece of scaffolding is a bet on what the model can't do yet, and **Opus 4.6** made the sprint structure and **context resets** unnecessary — audit your harness every model generation.",
  "agent_context": "**The gist** Anthropic built a GAN-style harness separating a generator from a **Playwright-driven evaluator**, plus a planner, for autonomous full-stack builds. A game-maker app took **6 hours and $200** through the harness versus **20 minutes and $9** solo; a simplified v2 on **Opus 4.6** built a digital audio workstation in 3h50m for **$124.70**.\n\n**Why it matters** Models grade their own work leniently, so a separate evaluator that clicks through the live app catches bugs self-review misses. But the deeper lesson cuts the other way: each piece of scaffolding is a bet on what the model can't do yet, and **Opus 4.6** made the sprint structure and **context resets** unnecessary — audit your harness every model generation.\n\n**Watch out** The evaluator only pays off when tasks exceed the model's baseline, and it still missed **nested layout bugs** and couldn't judge audio at all. Even evaluation wording steers output — \"museum quality\" pushed designs toward **one converged aesthetic**.",
  "source": {
    "name": "Anthropic",
    "url": "https://www.anthropic.com/engineering/harness-design-long-running-apps",
    "published_at": null
  },
  "source_class": "blog_post",
  "content_type": "Engineering Post",
  "layer": "agent",
  "domains": [
    "coding"
  ],
  "topics": [
    "harness-engineering",
    "multi-agent",
    "coding-agents"
  ],
  "verification": {
    "status": "official_source",
    "label": "Official Source",
    "method": "source_feed",
    "verified_at": null
  },
  "uncertainty": [
    "The evaluator only pays off when tasks exceed the model's baseline, and it still missed **nested layout bugs** and couldn't judge audio at all. Even evaluation wording steers output — \"museum quality\" pushed designs toward **one converged aesthetic**."
  ],
  "lifecycle": "Current",
  "published_at": null,
  "modified_at": null,
  "supersedes": [],
  "expires_at": null,
  "formats": {
    "html": "https://feed7.dev/p/harness-design-long-running-apps-15vc0wu",
    "json": "https://feed7.dev/p/harness-design-long-running-apps-15vc0wu.json",
    "markdown": "https://feed7.dev/p/harness-design-long-running-apps-15vc0wu.md"
  }
}