{
  "schema_version": "1.1",
  "id": "s13:https://arxiv.org/abs/2609.20804v1",
  "slug": "2609-20804v1-17ecr62",
  "url": "https://feed7.dev/p/2609-20804v1-17ecr62",
  "title": "An Empirical Study of Harness Design for Coding Agents",
  "why_included": "Harness components pay off differently by model and budget. Elide before summarizing, use planning selectively, and avoid elaborate tools when the model is already strong with bash.",
  "summary": "Researchers tested **176 matched settings** across **four models**, varying planning, action space, and context management on SWE-Bench Verified and Terminal-Bench 2.1. Context handling mattered more as budgets tightened, chiefly by preventing overflow.",
  "practical_implication": "Stage rule-based elision before LLM summarization. Use planning as an accuracy scaffold for weaker models and a cost control for stronger ones; offer predefined tools when bash skill is weak, but consider bash-only operation for capable models on CLI-heavy work.",
  "agent_context": "Researchers tested **176 matched settings** across **four models**, varying planning, action space, and context management on SWE-Bench Verified and Terminal-Bench 2.1. Context handling mattered more as budgets tightened, chiefly by preventing overflow.\n\nStage rule-based elision before LLM summarization. Use planning as an accuracy scaffold for weaker models and a cost control for stronger ones; offer predefined tools when bash skill is weak, but consider bash-only operation for capable models on CLI-heavy work.\n\nRecoverable elision added machinery without an accuracy gain because models rarely used recovery. The evidence spans **five context strategies** and **four window budgets**, but the abstract provides no effect sizes and covers only two benchmarks.",
  "source": {
    "name": "arXiv",
    "url": "https://arxiv.org/abs/2609.20804v1",
    "published_at": "2026-09-17T17:58:07.000Z"
  },
  "source_class": "blog_post",
  "content_type": "Paper",
  "layer": "agent",
  "domains": [
    "coding"
  ],
  "topics": [
    "harness-engineering",
    "context-engineering",
    "tool-use"
  ],
  "verification": {
    "status": "needs_review",
    "label": "Needs Review",
    "method": "unverified",
    "verified_at": null
  },
  "uncertainty": [
    "Recoverable elision added machinery without an accuracy gain because models rarely used recovery. The evidence spans **five context strategies** and **four window budgets**, but the abstract provides no effect sizes and covers only two benchmarks."
  ],
  "connected_context": {
    "meaning": "This supplies comparative evidence for choices that prior harness guidance treated mainly as design principles. It narrows the highest-leverage coding-agent controls to budget-aware context management, model-dependent planning, and action spaces matched to tool competence, while warning that extra recovery machinery is not automatically useful. Its two-benchmark scope does not establish the broader production controls needed for stateful or irreversible work.",
    "corpus_size": 807,
    "generated_at": "2026-09-18T10:07:02.227Z",
    "connections": [
      {
        "title": "Harness Engineering: Building the Production Cage for Powerful Domain Agents — Mike Chambers, AWS",
        "source_name": "AI Engineer",
        "source_url": "https://www.youtube.com/watch?v=gxVZ_1tuuq4",
        "feed7_url": "https://feed7.dev/p/harness-engineering-building-the-production-cage-for-powerful-domain-age-0la85zn",
        "reason": "The production-harness framework defines the surrounding system broadly; this study adds matched empirical evidence about which planning, context, and tool-interface choices matter within that boundary."
      },
      {
        "title": "Agents Without Code: Skills, YAML, and Filesystems Replaced Python — Philipp Schmid, Google DeepMind",
        "source_name": "AI Engineer",
        "source_url": "https://www.youtube.com/watch?v=fjF8EKnxKCU",
        "feed7_url": "https://feed7.dev/p/agents-without-code-skills-yaml-and-filesystems-replaced-python-philipp-0t7a4s8",
        "reason": "The reduced-orchestration approach favors general tools and instructions, while these results qualify that choice by showing that predefined tools can still help models with weaker bash competence."
      },
      {
        "title": "AI Agents Are Just Distributed Systems Now — Salman Munaf, TikTok",
        "source_name": "AI Engineer",
        "source_url": "https://www.youtube.com/watch?v=hD9-V56FNRI",
        "feed7_url": "https://feed7.dev/p/ai-agents-are-just-distributed-systems-now-salman-munaf-tiktok-1v4yc47",
        "reason": "The study measures accuracy and context behavior on coding benchmarks, whereas this candidate identifies external-state semantics and recovery controls that remain necessary beyond benchmark task completion."
      }
    ]
  },
  "lifecycle": "Current",
  "published_at": "2026-09-17T17:58:07.000Z",
  "modified_at": "2026-09-17T17:58:07.000Z",
  "supersedes": [],
  "expires_at": null,
  "formats": {
    "html": "https://feed7.dev/p/2609-20804v1-17ecr62",
    "json": "https://feed7.dev/p/2609-20804v1-17ecr62.json",
    "markdown": "https://feed7.dev/p/2609-20804v1-17ecr62.md"
  }
}