arXivPaperNeeds Review
From Production Traffic to Post-Training: Building a Self-Hosted LLM That Covers the Corporate Request Mix
A production-derived post-training recipe consolidated more than 200 internal apps onto one self-hosted model by training separate experts for distinct quality gaps, then merging them.
arXiv
Source Summary
The team analyzed traffic from **200+ internal applications**, trained separate GRPO experts for instruction following, function calling, and internal task distribution, then merged them with **two-stage SLERP** to avoid cross-domain reward interference.
Practical Implication
For a shared internal agent model, derive eval slices from production errors and isolate objectives whose rewards conflict. Their non-reasoning recipe beat a roughly **7× larger baseline** on the in-house Arena, instruction following, and function calling.
Agent-Ready Context
The team analyzed traffic from **200+ internal applications**, trained separate GRPO experts for instruction following, function calling, and internal task distribution, then merged them with **two-stage SLERP** to avoid cross-domain reward interference. For a shared internal agent model, derive eval slices from production errors and isolate objectives whose rewards conflict. Their non-reasoning recipe beat a roughly **7× larger baseline** on the in-house Arena, instruction following, and function calling. These are internal benchmarks using deterministic verifiers or calibrated model judges, so external comparability is limited. The resulting model handled **116M requests per month**, but the material gives neither model identity nor an exact serving-cost figure.
Context Map
modelcoding#enterprise#model-selection#agent-reliabilityUncertainty
These are internal benchmarks using deterministic verifiers or calibrated model judges, so external comparability is limited. The resulting model handled **116M requests per month**, but the material gives neither model identity nor an exact serving-cost figure.