Minimally Invasive Steering of Language Models
MISVO steers frozen models toward test-time rewards while penalizing output-distribution disruption, offering a parameter-free adaptation method tested on preference and code generation.
MISVO adds position-specific vectors before the output layer and penalizes them using local KL geometry. It works with frozen models of roughly **1B–14B parameters** and does not update their weights.
Builders operating their own models could treat steering as constrained test-time adaptation: optimize for a task reward while explicitly limiting changes to the token distribution, instead of relying on unconstrained activation edits.
MISVO adds position-specific vectors before the output layer and penalizes them using local KL geometry. It works with frozen models of roughly **1B–14B parameters** and does not update their weights. Builders operating their own models could treat steering as constrained test-time adaptation: optimize for a task reward while explicitly limiting changes to the token distribution, instead of relying on unconstrained activation edits. MISVO had the highest mean reward in **six of seven settings**, while diversity and coherence stayed near Best-of-N. The material covers preference and code-generation tasks but does not establish deployment cost or behavior outside those settings.
MISVO adds a middle path between choosing a different model and retraining one: optimize frozen-model behavior at inference time while explicitly constraining distributional drift. Its reward gains with near-Best-of-N diversity and coherence make constrained activation steering testable for owned models, but deployment cost and reliability beyond seven preference and coding settings remain open.