Open Brain
Atlas / Agent

Harness Engineering

Open JSONConfidence: Auto-collectedLast updated 2026-11-06

Current Answer

No editorial synthesis yet — the evidence below is collected automatically from source labels. A current answer lands here once an editor approves one.

Evidence

CPU-ARC: Agentic Reasoning Caching for on-CPU Reasoning Reuse
arXiv · 2026-11-06

CPU-ARC turns prior reasoning traces into reusable CPU-executable procedures, reporting frontier-level performance on math and science tasks at 16–291× lower invocation cost. It suggests a way to avoid rerunning a model for every similar instance.

SwarmEvolve: Latent Agent Swarms for Evolutionary Discovery and Optimization
arXiv · 2026-11-05

SwarmEvolve uses latent communication between agent swarms to search for better programs. Its reported speed suggests a different way to structure expensive, iterative coding-agent experiments.

ReproBridge: Building Reproducibility into ML Research Artifacts
arXiv · 2026-11-05

ReproBridge finds that agent-ready ML artifacts improve reproduction rates and cut API spending across four evaluated models. The practical lesson is to preserve setup and execution decisions alongside code.

Our Smaller Model Beat Opus on Finance Tasks - Alexander Shevchenko, Ramp
AI Engineer · 2026-11-04

Ramp describes three ways applied research can improve production agents: a finance specialist model, prebuilt compressed context, and large parallel security sweeps. The gains target latency, cost, and coverage.

Strategist: Evolving Reasoning with Agents
arXiv · 2026-11-02

Strategist uses a reusable library of executable reasoning blueprints that agents retrieve, test, and revise across tasks. The paper reports similar average accuracy to its strongest baseline at lower inference cost.

Catching Bugs Before They Land: JiTTesting’s Human-on-the-Loop Approach at Full Scale
Meta AI · 2026-10-29

Meta generates tests that expose behavioral changes in risky diffs, then combines deterministic rules, an LLM ensemble, and aggregate human review to filter likely false positives before code lands.

GEPA-Curator: Curated Learning for Self Improving Agents
arXiv · 2026-10-27

GEPA-Curator filters noisy agent failures and retrieves relevant successful runs before prompt optimization, improving results over two baselines across four environments.

The fx adapter now runs the harness on the host
Vercel · 2026-10-27

The AI SDK's fx adapter moves the harness and model requests onto the host while keeping built-in shell and filesystem tools sandboxed, so model credentials stay off the sandbox.

Agents That Self Improve While You Sleep - Fuad Ali, Arize AI
AI Engineer · 2026-10-26

The workshop shows a loop that uses production traces to find recurring agent failures, propose code fixes, and turn failures into evals and regression data. Builders should keep human review and cost checks in that loop.

Profiling a Frontier Agent with EBench: From Direct Control to Harness with Policy
arXiv · 2026-10-26

An EBench study finds that a robot-control agent can complement post-trained action policies: a harness that invokes them matched the strongest compared policies while using fewer tokens. Invocation and handoff still limit performance.

FOCUS: Active Critic Learning for Long-Horizon Language Model Agents
arXiv · 2026-10-26

FOCUS uses selective branching to improve value estimates for long-horizon language-model agents. Its results suggest a way to assign credit across segments without spending the same rollout budget on every state.

Can an Agent Debug a Web Corpus? Agentic Auditing of Slovak FineWeb2
arXiv · 2026-10-22

A budgeted agent turned corpus anomalies into reusable tests, but static sampling found more valid selectors for less money. The useful pattern is agent discovery followed by deterministic checks.

Making every harness durable
Vercel · 2026-10-19

Vercel’s open-source durable-harness logs model and tool steps so cloud coding agents can resume after crashes without rerunning completed calls.

LLM Agents Can Easily Tamper With Their Own Traces
arXiv · 2026-09-24

Local coding agents could delete their own execution traces without guardrail alerts. Treat agent-controlled logs as untrusted and capture them through an independent mechanism.

RAPID: Robot Agentic Programming from Demonstrations
arXiv · 2026-09-24

RAPID turns one visual demonstration into an executable, testable robot program, showing how agent loops can derive their own specifications, tools, and verification environment.

AI Engineer Paris 2026 Opening Keynotes: Mistral, Langfuse & Sizzy | Day 1
AI Engineer · 2026-09-24

Paris keynotes frame agent productivity as a systems problem: consolidate connectors and skills, support long-running sandboxed work, and keep permissions and company data under control.

Improved token efficiency for longer agent runs
Cursor · 2026-09-23

Cursor cut agent token cost by shrinking static instructions, loading tools on demand, improving cache boundaries, and using subagents more selectively.

Grow the Harness, Not the Context: From Strategy-Free Scaffolds to Reusable Specialist Agents
arXiv · 2026-09-22

A learned harness moved recurring control into reusable code, cutting model calls and inference cost while keeping web-agent performance stable across 4B–120B deployment models.

The Dark Arts of Skill Engineering — Paul Bakaus, Renaissance Geek (Impeccable)
AI Engineer · 2026-09-21

Reliable agent skills need more than prompts: use hooks as automatic guardrails, test across models, and compile harness-specific variants when instruction-following differs.

onPanda: Efficient Annotation of On-Policy Alignment Data for LLMs and Agents via Token-Level Correction
arXiv · 2026-09-21

onPanda turns the first bad token into an annotation boundary, regenerating from each correction to create on-policy supervision with less manual rewriting.

RRSI: Regularized Recursive Self-Improvement of Agent Harnesses
arXiv · 2026-09-21

RRSI regularizes automated harness editing to curb benchmark overfitting, improving held-out tasks while using fewer policy tokens than unregularized evolution.

Operating Distributed Inference Systems at Scale — Nishant Gupta & Naman Ahuja, Meta
AI Engineer · 2026-09-19

Large agent workloads turn inference into an orchestration problem: scheduling must account for workflow state, KV cache, hardware, retries, and cost per completed task.

CodeMidas: Scaling Agentic Coding RL Environments from Code Itself
arXiv · 2026-09-18

CodeMidas derives verified coding-agent RL tasks directly from working source, widening training data beyond issues and commits while improving repair, construction, and terminal benchmarks.

Total Recall: Agent Memory and Harness Engineering — Ignacio Martinez, Oracle
AI Engineer · 2026-09-18

Treat memory, context control, concurrency, and model routing as harness concerns. The workshop shows how these layers can improve agent behavior without changing model weights.

Coding Agents with an Obstacle-Aware Harness for Safe Robot Manipulation
arXiv · 2026-09-17

A safety prompt did not stop robot coding agents from colliding with obstacles. SafeHarness instead makes route verification, replanning, and contact constraints explicit in the execution loop.

An Empirical Study of Harness Design for Coding Agents
arXiv · 2026-09-17

Harness components pay off differently by model and budget. Elide before summarizing, use planning selectively, and avoid elaborate tools when the model is already strong with bash.

Where RL Will Take Search — Maximilian-David Rumpf, SID.ai
AI Engineer · 2026-09-16

Delegating retrieval to a trained search subagent can keep noisy results out of the main agent’s context while cutting the token, latency, and cost burden of search-heavy workflows.

If we want them to do Knowledge Work, design them as Knowledge Agents — Benjamin Clavié, Mixedbread
AI Engineer · 2026-09-16

Knowledge agents need intent decomposition, specialist researchers, and multiple retrieval primitives; copying a coding-agent loop fails when evidence is ambiguous and context-dependent.

TypeSafe AI's Jev now available on AI Gateway
Vercel · 2026-09-16

Jev gives agent loops typed decisions and probabilities without generating prose, offering a smaller control primitive for routing tools, retries, reviews, and guardrails.

Realtime Voice Agents with Frontier Intelligence — Bohan Li, EliseAI
AI Engineer · 2026-09-15

A low-latency voice harness can pair fast provisional work with slower corrections. The design speculates on transcripts and responses, runs tools in parallel, and reuses cached speech prefixes.

5 Voice Agent Failure Modes You'll Hit in Week One — Venky B, Plivo
AI Engineer · 2026-09-15

Voice agents trade reasoning time for latency, then fail on transcription, data capture, and pronunciation. Treat inputs as typed fields, validate them, and normalize speech output yourself.

Stellar Colosseum: A Many-Agent Harness for Long-Horizon Research in Mathematics and Theoretical Computer Science
arXiv · 2026-09-14

Stellar Colosseum coordinates parallel strategy search, falsification, decomposition, and verifier feedback for long research tasks. Its harness patterns may transfer to agents handling interdependent coding work.

HypoEvolve: Genetic Algorithms Enable Multi-Agent LLMs to Discover Scientific Hypotheses
arXiv · 2026-09-14

HypoEvolve turns multi-agent scientific work into explicit population updates governed by a genetic algorithm. The result suggests agent collaboration is easier to test when selection and revision are formalized.

Agents Without Code: Skills, YAML, and Filesystems Replaced Python — Philipp Schmid, Google DeepMind
AI Engineer · 2026-09-14

A GitHub review agent is rebuilt with progressively less orchestration code, ending as instructions and setup files around general tools. Evals still remain the builder’s job.

How We Solved Agent Building — Andrew Qu, Vercel
AI Engineer · 2026-09-14

Vercel’s data agent improved after replacing prescriptive specialist pipelines with a filesystem, Bash and company-specific context. Eve packages that pattern as conventions.

We let an AI agent execute Bash and lived to talk about it — Sarah Sanders, PostHog
AI Engineer · 2026-09-14

PostHog treats every context source as part of an agent’s supply chain, scanning at build and use time while reserving blocking decisions for deterministic controls.

Harness Engineering: Building the Production Cage for Powerful Domain Agents — Mike Chambers, AWS
AI Engineer · 2026-09-14

A production agent's harness is everything around the model: tools, memory, runtime, isolation, and deployment controls. The practical lesson is to standardize that layer across agents.

Building ambitious software — Jonathan Kelley, Dioxus Labs & Cognition
AI Engineer · 2026-09-11

Dioxus found that generating more Rust was easy but merging it was not. Agents paid off on bounded research and maintenance work when architecture, real-device tests, and line-by-line review stayed human-owned.

GitHub Copilot is now available in the AI SDK harness layer
Vercel · 2026-09-10

AI SDK now runs GitHub Copilot behind the same HarnessAgent interface as nine other coding harnesses, using an official adapter and ACP connection.

Mousepower: agents that can’t be measured, can’t be managed. — Maximillian Piras, Yutori
AI Engineer · 2026-09-10

Token counts do not show whether an agent was worth running. Define outcomes and cheap acceptance checks first, then reserve agents for work that is uncertain to execute but relatively easy to verify.

Stable permalink · evidence auto-collected from source labels · synthesis maintained by feed7 editorial