CodeQuant2: Enhanced Outlier Mitigation and Adaptive Clustering for Low-Precision LRMs
CodeQuant2 tunes low-precision quantization for reasoning models, including calibration that captures early decoding behavior. It reports up to 2.85× BF16 speed, but the material gives no benchmark details.
CodeQuant2 combines rotation and scaling to reduce activation outliers, clustering for non-uniform weights, and layer-specific codebook sizes. Its calibration samples are chosen to reflect early reasoning-time decoding, when activations fluctuate more. The authors report up to **2.85×** speed over BF16 with accuracy gains on reasoning benchmarks.
If you run reasoning models locally or serve them for coding agents, the paper points to calibration data as part of the quantization design: include early decoding states, not just representative prompts or later tokens.
CodeQuant2 combines rotation and scaling to reduce activation outliers, clustering for non-uniform weights, and layer-specific codebook sizes. Its calibration samples are chosen to reflect early reasoning-time decoding, when activations fluctuate more. The authors report up to **2.85×** speed over BF16 with accuracy gains on reasoning benchmarks. If you run reasoning models locally or serve them for coding agents, the paper points to calibration data as part of the quantization design: include early decoding states, not just representative prompts or later tokens. The material does not identify the benchmarks, model sizes, or hardware used for the reported speedup, so it does not establish how the approach compares with quantization options in a particular agent stack.
This offers a specific quantization design for reasoning models, including calibration on early decoding states when activations fluctuate. The reported speed and accuracy gains make it a candidate for local coding-agent inference, but the missing benchmark, model, and hardware details prevent judging its fit against alternatives in a particular stack.