{
  "schema_version": "1.1",
  "id": "s13:https://arxiv.org/abs/2609.09133v1",
  "slug": "2609-09133v1-0zfcxb4",
  "url": "https://feed7.dev/p/2609-09133v1-0zfcxb4",
  "title": "ExecCritic: Learn to Test, Test to Improve for Coding Agents",
  "why_included": "ExecCritic separates test writing from repair and freezes qualified tests before code changes. On SWE-bench Verified, weak generated tests hurt resolution, while separately trained roles reached 72.6%.",
  "summary": "ExecCritic assigns testing and repair to separate Qwen-3.5-35B-A3B agents. A fail-closed harness qualifies and freezes repository-native tests, preventing the Repair agent from changing the target after seeing execution feedback.",
  "practical_implication": "Test quality determined whether feedback helped on SWE-bench Verified: the no-test baseline resolved **61.2%**, base-agent tests lowered that to **57.3%**, and GPT-5.6-sol tests raised it to **65.3%**. Builders should isolate test generation from patching and reject tests that fail qualification.",
  "agent_context": "ExecCritic assigns testing and repair to separate Qwen-3.5-35B-A3B agents. A fail-closed harness qualifies and freezes repository-native tests, preventing the Repair agent from changing the target after seeing execution feedback.\n\nTest quality determined whether feedback helped on SWE-bench Verified: the no-test baseline resolved **61.2%**, base-agent tests lowered that to **57.3%**, and GPT-5.6-sol tests raised it to **65.3%**. Builders should isolate test generation from patching and reject tests that fail qualification.\n\nSeparately post-training both Qwen roles produced a **72.6%** resolved rate, 11.4 points above the original no-test baseline without stronger-model or Oracle feedback at evaluation. These are benchmark results; repository-specific costs and generalization are not covered here.",
  "source": {
    "name": "arXiv",
    "url": "https://arxiv.org/abs/2609.09133v1",
    "published_at": "2026-09-08T17:53:37.000Z"
  },
  "source_class": "blog_post",
  "content_type": "Paper",
  "layer": "agent",
  "domains": [
    "coding"
  ],
  "topics": [
    "coding-agents",
    "harness-engineering",
    "agent-reliability"
  ],
  "verification": {
    "status": "needs_review",
    "label": "Needs Review",
    "method": "unverified",
    "verified_at": null
  },
  "uncertainty": [
    "Separately post-training both Qwen roles produced a **72.6%** resolved rate, 11.4 points above the original no-test baseline without stronger-model or Oracle feedback at evaluation. These are benchmark results; repository-specific costs and generalization are not covered here."
  ],
  "connected_context": {
    "meaning": "This makes test provenance and qualification first-class controls rather than assuming any execution feedback helps. The benchmark reversal shows weak generated tests can be worse than no tests, while isolated roles, frozen qualified tests, and separate post-training turn testing from an informal agent habit into a governed repair interface.",
    "corpus_size": 713,
    "generated_at": "2026-09-09T10:12:53.401Z",
    "connections": [
      {
        "title": "MineValiCoder: Reliable Code Generation with Test Case Quality Mining and Bipartite Graph-Based Mutual Validation",
        "source_name": "arXiv",
        "source_url": "https://arxiv.org/abs/2607.22471v1",
        "feed7_url": "https://feed7.dev/p/2607-22471v1-0kivszm",
        "reason": "Both prevent unreliable generated tests from steering code changes; ExecCritic adds strict role isolation and freezes repository-native tests before repair feedback is exposed."
      },
      {
        "title": "How to Kill the Code Review — Ankit Jain, Aviator",
        "source_name": "AI Engineer",
        "source_url": "https://www.youtube.com/watch?v=YgEv7IQzGdM",
        "feed7_url": "https://feed7.dev/p/how-to-kill-the-code-review-ankit-jain-aviator-0rku6kj",
        "reason": "Its qualified, immutable tests provide the kind of verification evidence reviewers can inspect, while the weak-test result supports retaining human judgment over generated checks."
      },
      {
        "title": "How to Get Your Org to Adopt Coding Agents (Without Shipping Garbage) — Eyal Blum, Figma",
        "source_name": "AI Engineer",
        "source_url": "https://www.youtube.com/watch?v=5Bn0xro2ol8",
        "feed7_url": "https://feed7.dev/p/how-to-get-your-org-to-adopt-coding-agents-without-shipping-garbage-eyal-04y3rj2",
        "reason": "It supplies a concrete safeguard for the adoption concern that verification quality, not generation alone, determines whether agent-written code is trustworthy."
      },
      {
        "title": "Loop Engineering from First Principles — Kyle Mistele, HumanLayer",
        "source_name": "AI Engineer",
        "source_url": "https://www.youtube.com/watch?v=xIt_mTQp6mY",
        "feed7_url": "https://feed7.dev/p/loop-engineering-from-first-principles-kyle-mistele-humanlayer-1nuq7gf",
        "reason": "Separating test creation from bounded repair gives the incremental coding loop a fail-closed feedback boundary before a patch proceeds to review."
      }
    ]
  },
  "lifecycle": "Current",
  "published_at": "2026-09-08T17:53:37.000Z",
  "modified_at": "2026-09-08T17:53:37.000Z",
  "supersedes": [],
  "expires_at": null,
  "formats": {
    "html": "https://feed7.dev/p/2609-09133v1-0zfcxb4",
    "json": "https://feed7.dev/p/2609-09133v1-0zfcxb4.json",
    "markdown": "https://feed7.dev/p/2609-09133v1-0zfcxb4.md"
  }
}