# Form, Not Content? A Preregistered, Placebo-Controlled Evaluation of Learned Error-Conditioned Self-Repair Through Prompts and Weights in Frozen Small Code Models

Source: [arXiv](https://arxiv.org/abs/2607.12962v1)  
Feed7 permalink: https://feed7.dev/p/2607-12962v1-0q4i26c  
Published: 2026-07-14T16:59:42.000Z  
Trust: Needs Review (needs_review)

## Why Included

Placebo-controlled tests found no evidence that small frozen code models repaired failures because of the error content itself. Retry scaffolds and mismatched feedback performed as well or better.

## Source Summary

PoPE tested frozen **0.5–1.5B code models** with live errors and placebos through prompts and adapter training. In prompting, a form-only placebo unlocked **12 units versus 10** for live error patterns on a resistant 40-unit band.

## Practical Implication

When measuring self-repair, preserve the retry scaffold but remove or mismatch task-relevant feedback. If the placebo performs similarly, do not attribute gains to the compiler or test error; the retry format itself may be doing the work.

## Agent-Ready Context

PoPE tested frozen **0.5–1.5B code models** with live errors and placebos through prompts and adapter training. In prompting, a form-only placebo unlocked **12 units versus 10** for live error patterns on a resistant 40-unit band.

When measuring self-repair, preserve the retry scaffold but remove or mismatch task-relevant feedback. If the placebo performs similarly, do not attribute gains to the compiler or test error; the retry format itself may be doing the work.

In adapter training, error content tied the no-intervention baseline at **8–8**, while a deranged-error placebo reached **10 unlocks**. These are public-tier screening results, not equivalence evidence; hidden-tier confirmation was deferred.

## Context Map

- Layer: benchmark
- Domains: coding
- Topics: agent-evals, benchmark-integrity, agent-reliability

## Uncertainty

- In adapter training, error content tied the no-intervention baseline at **8–8**, while a deranged-error placebo reached **10 unlocks**. These are public-tier screening results, not equivalence evidence; hidden-tier confirmation was deferred.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.