# An Empirical Study of Harness Design for Coding Agents

Source: [arXiv](https://arxiv.org/abs/2609.20804v1)  
Feed7 permalink: https://feed7.dev/p/an-empirical-study-of-harness-design-for-coding-agents-caaf99b872  
Published: 2026-09-17T00:00:00.000Z  
Trust: Needs Review (needs_review)

## Why Included

Harness tests favor rule-based elision before summarization, selective planning, and bash-only tools for models already strong at CLI work.

## Source Summary

Harness components pay off differently by model and budget. Elide before summarizing, use planning selectively, and avoid elaborate tools when the model is already strong with bash.

## Practical Implication

Stage rule-based elision before LLM summarization. Use planning as an accuracy scaffold for weaker models and a cost control for stronger ones; offer predefined tools when bash skill is weak, but consider bash-only operation for capable models on CLI-heavy work.

## Agent-Ready Context

Researchers tested **176 matched settings** across **four models**, varying planning, action space, and context management on SWE-Bench Verified and Terminal-Bench 2.1. Context handling mattered more as budgets tightened, chiefly by preventing overflow.

Stage rule-based elision before LLM summarization. Use planning as an accuracy scaffold for weaker models and a cost control for stronger ones; offer predefined tools when bash skill is weak, but consider bash-only operation for capable models on CLI-heavy work.

Recoverable elision added machinery without an accuracy gain because models rarely used recovery. The evidence spans **five context strategies** and **four window budgets**, but the abstract provides no effect sizes and covers only two benchmarks.

## Context Map

- Layer: agent
- Domains: coding
- Topics: harness-engineering, context-engineering, tool-use

## Uncertainty

- Automatically selected from source material; feed7 has not independently tested the claim.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.