Open Brain
arXivPaperNeeds Review

Minimally Invasive Steering of Language Models

MISVO steers frozen models toward test-time rewards while penalizing output-distribution disruption, offering a parameter-free adaptation method tested on preference and code generation.

arXiv · Sep 24, 2026
Open Source Open MarkdownOpen JSON
Source Summary

MISVO adds position-specific vectors before the output layer and penalizes them using local KL geometry. It works with frozen models of roughly **1B–14B parameters** and does not update their weights.

Practical Implication

Builders operating their own models could treat steering as constrained test-time adaptation: optimize for a task reward while explicitly limiting changes to the token distribution, instead of relying on unconstrained activation edits.

Agent-Ready Context
MISVO adds position-specific vectors before the output layer and penalizes them using local KL geometry. It works with frozen models of roughly **1B–14B parameters** and does not update their weights.

Builders operating their own models could treat steering as constrained test-time adaptation: optimize for a task reward while explicitly limiting changes to the token distribution, instead of relying on unconstrained activation edits.

MISVO had the highest mean reward in **six of seven settings**, while diversity and coherence stayed near Best-of-N. The material covers preference and code-generation tasks but does not establish deployment cost or behavior outside those settings.
Connected Context · Feed7 Judgment

MISVO adds a bounded test-time control option between model selection and open-ended search: frozen models can be optimized toward a reward while local KL geometry limits distributional drift. This complements efficiency and generate-evaluate-refine methods, but narrows the claim to tested preference and code tasks; it does not yet show that constrained steering is cheaper or more robust in deployed agent loops.

Prefix Sliding for efficient test-time scalingPrefix Sliding controls the retained reasoning context, whereas MISVO modifies output behavior under a divergence constraint; the two address different test-time scaling costs and could be evaluated independently or together.Post-Training Language Models for Gold-Medal Performance in Coding CompetitionsThe competition system improves coding through specialized post-training and iterative search, while MISVO offers weight-frozen reward optimization as a distinct test-time adaptation mechanism.DFM Mimir v1: An Open HRM Delivering Frontier Performance at 1B Parameters Using Only Permissible Post-Training DataMISVO’s tested size range includes compact models, making it relevant to deployments considering a 1B model such as Mimir, though neither candidate establishes the combined system’s quality or cost.Jev is the fastest-adopted model in AI Gateway historyJev provides typed decisions around an agent, while MISVO steers token distributions inside a frozen generator; they are complementary control-plane and model-level interventions rather than substitutes.
Context Map
modelcoding#reasoning
Uncertainty
MISVO had the highest mean reward in **six of seven settings**, while diversity and coherence stayed near Best-of-N. The material covers preference and code-generation tasks but does not establish deployment cost or behavior outside those settings.