# MindForge: Teaching Small Language Models Whole-Life-Cycle Software Engineering via Source-Free Program Synthesis

Source: [arXiv](https://arxiv.org/abs/2607.27146v1)  
Feed7 permalink: https://feed7.dev/p/2607-27146v1-0gcx6ia  
Published: 2026-07-29T17:23:02.000Z  
Trust: Needs Review (needs_review)

## Why Included

MindForge trains a 27B coding model on source-free program synthesis, raising ProgramBench pass rate from 37.98% to 49.51% and improving across seven unseen benchmarks.

## Source Summary

MindForge turns open-source CLI programs into source-free training environments exposing only a compiled reference executable and documentation. Trajectories generated with **GLM-5.2** were used to fine-tune **Qwen3.6-27B**, lifting its ProgramBench average test pass rate from **37.98% to 49.51%**.

## Practical Implication

Builders should distinguish repository editing from whole-program creation when evaluating coding agents. Training against executable behavior and documentation may offer a scalable route to agents that can generate repositories, translate code, fix bugs, and implement features without seeing reference source.

## Agent-Ready Context

MindForge turns open-source CLI programs into source-free training environments exposing only a compiled reference executable and documentation. Trajectories generated with **GLM-5.2** were used to fine-tune **Qwen3.6-27B**, lifting its ProgramBench average test pass rate from **37.98% to 49.51%**.

Builders should distinguish repository editing from whole-program creation when evaluating coding agents. Training against executable behavior and documentation may offer a scalable route to agents that can generate repositories, translate code, fix bugs, and implement features without seeing reference source.

The results come from the authors’ benchmark suite rather than production deployments. Frontier models still fully resolve **fewer than 1%** of ProgramBench tasks, and the material does not establish how the pipeline handles graphical apps, services, or undocumented behavior.

## Connected Context

Feed7 judgment across 263 accumulated Signals:

This expands coding-agent training beyond editing visible repositories to reconstructing whole programs from documentation and executable behavior. It confirms that environment and trajectory design can materially improve a smaller model, while the sub-1% full-resolution rate sharply limits claims of autonomous software creation. Production relevance remains open because the benchmark excludes important application classes and undocumented behavior.

- [The Prime Intellect Stack — Will Brown, Prime Intellect](https://feed7.dev/p/the-prime-intellect-stack-will-brown-prime-intellect-1dc0rrp) — MindForge is a concrete instance of the environment-centered eval, data-generation, and training stack described by Prime Intellect, using executable programs as composable behavioral targets.
- [Harness design for long-running application development](https://feed7.dev/p/harness-design-long-running-apps-15vc0wu) — Both use external behavioral evaluation to counter unreliable self-assessment, but Anthropic applies it during long-running app construction while MindForge uses it to generate training trajectories and benchmark whole programs.
- [Effective harnesses for long-running agents](https://feed7.dev/p/effective-harnesses-for-long-running-agents-0xzfs05) — The multi-session harness supplies incremental state and verification for repository work; MindForge addresses a different prerequisite by training models to create whole programs from documentation and observable behavior.
- [Harness Engineering is not Enough: Why Software Factories Fail — Dex Horthy, HumanLayer](https://feed7.dev/p/harness-engineering-is-not-enough-why-software-factories-fail-dex-horthy-1tj1amt) — MindForge’s pass-rate improvement measures functional output, while the software-factory critique identifies maintainability and affordable human review as additional requirements not established by benchmark success.

## Context Map

- Layer: agent
- Domains: coding
- Topics: coding-agents, harness-engineering, agent-evals

## Uncertainty

- The results come from the authors’ benchmark suite rather than production deployments. Frontier models still fully resolve **fewer than 1%** of ProgramBench tasks, and the material does not establish how the pipeline handles graphical apps, services, or undocumented behavior.

## Agent Instruction

Use this item as source-backed context. Do not invent claims beyond the linked source. If this item conflicts with another source, call out the conflict.