Sign InOpen Brain
Back
arXivPaperNeeds ReviewbenchmarkNew

SWE-Prime: Fewer Trajectories, Better Performance

SWE-Prime's filtered 10% of coding traces beat the full resolved set, showing that passing outcomes are not automatically clean supervision.

arXivAug 27, 20262 min
Open SourceOpen MarkdownOpen JSON
Source Summary

SWE-Prime finds that filtering coding-agent traces by process and segment quality can beat training on every resolved trajectory, reducing noisy imitation from redundant or risky steps.

Practical Implication

The reported 10% trajectory subset beat training on the full resolved dataset, with relative gains up to 12.2% on SWE-Bench Pro and 24.2% on SWE-Bench Verified. Teams training coding models should evaluate how an issue was solved, not treat a passing outcome as clean supervision.

Agent-Ready Context
SWE-Prime filters successful coding-agent traces in **two stages**: whole trajectories are screened for process quality, result quality, and representativeness, then semantic segments are judged for contribution, learnability, and risk. Only selected segments contribute to the training loss.

The reported **10% trajectory subset** beat training on the full resolved dataset, with relative gains up to **12.2% on SWE-Bench Pro** and **24.2% on SWE-Bench Verified**. Teams training coding models should evaluate how an issue was solved, not treat a passing outcome as clean supervision.

All segments remain in the input sequence to preserve context, so this is selective loss computation rather than simply deleting weak steps. The material reports benchmark results but does not establish whether the selection criteria transfer to other repositories, agents, or training setups.
Context Map
benchmarkcodingdata#coding-agents#agent-evals#benchmark-integrityGeneric AgentPrepare Coding Session
Uncertainty
Automatically selected from source material; feed7 has not independently tested the claim.
Rate This Item
Personal Note

No note yet. Notes are included in exported bundles.

Related — Every Edge Explained

No approved edges yet for this post.