Minimally Invasive Steering of Language Models
MISVO steers frozen models toward test-time rewards while penalizing output-distribution disruption, offering a parameter-free adaptation method tested on preference and code generation.
MISVO adds position-specific vectors before the output layer and penalizes them using local KL geometry. It works with frozen models of roughly **1B–14B parameters** and does not update their weights.
Builders operating their own models could treat steering as constrained test-time adaptation: optimize for a task reward while explicitly limiting changes to the token distribution, instead of relying on unconstrained activation edits.
MISVO adds position-specific vectors before the output layer and penalizes them using local KL geometry. It works with frozen models of roughly **1B–14B parameters** and does not update their weights. Builders operating their own models could treat steering as constrained test-time adaptation: optimize for a task reward while explicitly limiting changes to the token distribution, instead of relying on unconstrained activation edits. MISVO had the highest mean reward in **six of seven settings**, while diversity and coherence stayed near Best-of-N. The material covers preference and code-generation tasks but does not establish deployment cost or behavior outside those settings.
MISVO adds a bounded test-time control option between model selection and open-ended search: frozen models can be optimized toward a reward while local KL geometry limits distributional drift. This complements efficiency and generate-evaluate-refine methods, but narrows the claim to tested preference and code tasks; it does not yet show that constrained steering is cheaper or more robust in deployed agent loops.