# Benchmarking Coding Agents on New vs Legacy Codebases — Denys Linkov, Wisedocs

Source: [AI Engineer](https://www.youtube.com/watch?v=7vn4WpqNpck)  
Feed7 permalink: https://feed7.dev/p/benchmarking-coding-agents-on-new-vs-legacy-codebases-de-9e82a687ae  
Published: 2026-08-08T00:00:00.000Z  
Trust: Source Linked (source_linked)

## Why Included

Judge coding agents with explicit requirements and end-to-end tests, since fast output may be incomplete scaffolding.

## Source Summary

A production refactor shows why coding-agent evaluations need acceptance criteria and end-to-end verification: fast output can still be incomplete scaffolding.

## Practical Implication

Benchmark agents against explicit requirements, runnable end-to-end tests, deployment constraints, and hidden assumptions—not elapsed time or lines changed. Wisedocs also found a monorepo simpler for verification and sandbox setup across its former 10+ repositories.

## Agent-Ready Context

An early O3-assisted task took **3 hours** and produced 10 major mistakes; newer Sonnet 4.6 solved it after one extra iteration and Opus 4.8 nearly one-shot it. A broader GPT-5.5 attempt finished in **10m 22s** but mostly wrote scaffolding.

Benchmark agents against explicit requirements, runnable end-to-end tests, deployment constraints, and hidden assumptions—not elapsed time or lines changed. Wisedocs also found a monorepo simpler for verification and sandbox setup across its former **10+ repositories**.

These are task-specific observations from one refactor, not controlled cross-model results. Human review remained part of the project, and only **15 of 17 requirements** were met during the migration.

## Context Map

- Layer: benchmark
- Domains: coding
- Topics: agent-evals, agent-reliability, coding-agents

## Uncertainty

- None recorded.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.