Harness Engineering

Local coding agents could delete their own execution traces without guardrail alerts. Treat agent-controlled logs as untrusted and capture them through an independent mechanism.

RAPID turns one visual demonstration into an executable, testable robot program, showing how agent loops can derive their own specifications, tools, and verification environment.

Paris keynotes frame agent productivity as a systems problem: consolidate connectors and skills, support long-running sandboxed work, and keep permissions and company data under control.

Cursor cut agent token cost by shrinking static instructions, loading tools on demand, improving cache boundaries, and using subagents more selectively.

A learned harness moved recurring control into reusable code, cutting model calls and inference cost while keeping web-agent performance stable across 4B–120B deployment models.

Reliable agent skills need more than prompts: use hooks as automatic guardrails, test across models, and compile harness-specific variants when instruction-following differs.

onPanda turns the first bad token into an annotation boundary, regenerating from each correction to create on-policy supervision with less manual rewriting.

RRSI regularizes automated harness editing to curb benchmark overfitting, improving held-out tasks while using fewer policy tokens than unregularized evolution.

Large agent workloads turn inference into an orchestration problem: scheduling must account for workflow state, KV cache, hardware, retries, and cost per completed task.

CodeMidas derives verified coding-agent RL tasks directly from working source, widening training data beyond issues and commits while improving repair, construction, and terminal benchmarks.

Treat memory, context control, concurrency, and model routing as harness concerns. The workshop shows how these layers can improve agent behavior without changing model weights.

A safety prompt did not stop robot coding agents from colliding with obstacles. SafeHarness instead makes route verification, replanning, and contact constraints explicit in the execution loop.

Harness components pay off differently by model and budget. Elide before summarizing, use planning selectively, and avoid elaborate tools when the model is already strong with bash.

Delegating retrieval to a trained search subagent can keep noisy results out of the main agent’s context while cutting the token, latency, and cost burden of search-heavy workflows.

Knowledge agents need intent decomposition, specialist researchers, and multiple retrieval primitives; copying a coding-agent loop fails when evidence is ambiguous and context-dependent.

Jev gives agent loops typed decisions and probabilities without generating prose, offering a smaller control primitive for routing tools, retries, reviews, and guardrails.

A low-latency voice harness can pair fast provisional work with slower corrections. The design speculates on transcripts and responses, runs tools in parallel, and reuses cached speech prefixes.

Voice agents trade reasoning time for latency, then fail on transcription, data capture, and pronunciation. Treat inputs as typed fields, validate them, and normalize speech output yourself.

Stellar Colosseum coordinates parallel strategy search, falsification, decomposition, and verifier feedback for long research tasks. Its harness patterns may transfer to agents handling interdependent coding work.

HypoEvolve turns multi-agent scientific work into explicit population updates governed by a genetic algorithm. The result suggests agent collaboration is easier to test when selection and revision are formalized.

A GitHub review agent is rebuilt with progressively less orchestration code, ending as instructions and setup files around general tools. Evals still remain the builder’s job.

Vercel’s data agent improved after replacing prescriptive specialist pipelines with a filesystem, Bash and company-specific context. Eve packages that pattern as conventions.

PostHog treats every context source as part of an agent’s supply chain, scanning at build and use time while reserving blocking decisions for deterministic controls.

A production agent's harness is everything around the model: tools, memory, runtime, isolation, and deployment controls. The practical lesson is to standardize that layer across agents.

Dioxus found that generating more Rust was easy but merging it was not. Agents paid off on bounded research and maintenance work when architecture, real-device tests, and line-by-line review stayed human-owned.

AI SDK now runs GitHub Copilot behind the same HarnessAgent interface as nine other coding harnesses, using an official adapter and ACP connection.

Token counts do not show whether an agent was worth running. Define outcomes and cheap acceptance checks first, then reserve agents for work that is uncertain to execute but relatively easy to verify.

Design-to-code agents can silently introduce security, accessibility, and fidelity drift. Scan both artifacts with deterministic checks, review proposed fixes, and catch divergence before merge.

A spatial agent harness can make parallel work, dependencies, and ownership visible on a shared canvas. Bridging that canvas to coding tools lets its nodes perform real work beyond drawing.

A reported weekend Python-to-TypeScript port shows how agent-driven iteration can make once-prohibitive migrations plausible, provided verification is part of the loop.

Show-Harness puts a compact semantic action layer between VLM intent and robot-specific control, a harness pattern for reusing frontier or small open models across embodiments.

LinkedIn scales a large internal agent catalog through search, schema lookup, and execution rather than exposing every tool at once. Its playbooks add task-specific operating knowledge.

RLMs keep large inputs in a programmable environment, letting a model inspect them with code and delegate subsets to submodels instead of attending to every token at once.

Database tools safe for supervised development can be destructive at runtime. Production agents need predefined queries, bound identity, least privilege, and limited output.

Procedural Graphs externalize an agent’s workflow, guide each action from the relevant subgraph, and refine the structure from failed and working runs. This targets drift and repeated tool mistakes on long tasks.

Full expert-trajectory imitation can break the fit between a weaker model and its evolved harness. Correcting only the failing turn preserved the model’s planning style across seven enterprise tasks.

ExecCritic separates test writing from repair and freezes qualified tests before code changes. On SWE-bench Verified, weak generated tests hurt resolution, while separately trained roles reached 72.6%.

EDGE synthesizes multi-step tool-call training data by retaining only API links verified through live execution. Its 9B model nearly matched an untuned 27B sibling on Korean public-API tasks.

Sentinel-RL keeps topology outside the LLM: a graph encoder and constrained policy choose SOC actions, while the model writes critic-gated narratives for human approval.

Knowledge-work agents need code-like infrastructure around tools: centralized context, action records, verification, enforced permissions, and preflight checks for irreversible work.