Data and Environment Curation for Post-Training LLMs — Mahesh Sathiamoorthy, Bespoke Labs
Post-training gains depend heavily on task selection, rollout quality, and environment design. For many enterprise agents, curated SFT may deliver most of the value before costly RL.
Bespoke’s curation recipe selects source prompts, mixes and filters them, generates teacher answers, then filters again. Across its work, **multiple answers per question** helped, while the strongest model was not always the best teacher.
Treat data recipes and environments as versioned engineering assets. Start with **SFT** for the required behavior, run ablations at each curation stage, and reserve **RL** for gains that justify its added compute and infrastructure.
Bespoke’s curation recipe selects source prompts, mixes and filters them, generates teacher answers, then filters again. Across its work, **multiple answers per question** helped, while the strongest model was not always the best teacher. Treat data recipes and environments as versioned engineering assets. Start with **SFT** for the required behavior, run ablations at each curation stage, and reserve **RL** for gains that justify its added compute and infrastructure. Synthetic rewriting and task augmentation did not reliably help in the reported agent work. Production datasets can also be imbalanced, so fine-tuning may amplify rare-looking attributes unless the mix and outputs are checked carefully.
This moves agent improvement upstream into disciplined data and environment engineering. It favors curated SFT as the default, makes RL an evidence-gated escalation, and challenges assumptions that stronger teachers or more synthetic augmentation automatically improve results. Versioned recipes, per-stage ablations, and distribution checks become prerequisites for attributing gains rather than optional research hygiene.