# ExecCritic: Learn to Test, Test to Improve for Coding Agents

Source: [arXiv](https://arxiv.org/abs/2609.09133v1)  
Feed7 permalink: https://feed7.dev/p/execcritic-learn-to-test-test-to-improve-for-coding-agen-79c903650b  
Published: 2026-09-08T00:00:00.000Z  
Trust: Needs Review (needs_review)

## Why Included

Separate test writing from repair, qualify repository-native tests, and freeze them before the repair agent sees execution feedback.

## Source Summary

ExecCritic separates test writing from repair and freezes qualified tests before code changes. On SWE-bench Verified, weak generated tests hurt resolution, while separately trained roles reached 72.6%.

## Practical Implication

Test quality determined whether feedback helped on SWE-bench Verified: the no-test baseline resolved 61.2%, base-agent tests lowered that to 57.3%, and GPT-5.6-sol tests raised it to 65.3%. Builders should isolate test generation from patching and reject tests that fail qualification.

## Agent-Ready Context

ExecCritic assigns testing and repair to separate Qwen-3.5-35B-A3B agents. A fail-closed harness qualifies and freezes repository-native tests, preventing the Repair agent from changing the target after seeing execution feedback.

Test quality determined whether feedback helped on SWE-bench Verified: the no-test baseline resolved **61.2%**, base-agent tests lowered that to **57.3%**, and GPT-5.6-sol tests raised it to **65.3%**. Builders should isolate test generation from patching and reject tests that fail qualification.

Separately post-training both Qwen roles produced a **72.6%** resolved rate, 11.4 points above the original no-test baseline without stronger-model or Oracle feedback at evaluation. These are benchmark results; repository-specific costs and generalization are not covered here.

## Context Map

- Layer: agent
- Domains: coding
- Topics: coding-agents, harness-engineering, agent-reliability

## Uncertainty

- Automatically selected from source material; feed7 has not independently tested the claim.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.