Open Brain
arXivPaperNeeds Review

Towards interpreting any residual stream direction by scaling activation inversion

MAEMs generate text that activates a chosen language-model residual direction, then verify it with a forward pass. The paper reports better held-out activation inversion than corpus search and a natural-language autoencoder.

arXiv · Nov 5, 2026
Open Source Open MarkdownOpen JSON
Source Summary

The paper introduces **Maximally-Activating Example Metamodels (MAEMs)**, trained to generate text whose activations align with a supplied residual-stream direction. A **Qwen3.6-27B MAEM** beat corpus search and a natural-language autoencoder on held-out activation inversion, and activated **89% of held-out SAE features**. For builders investigating model behavior, this offers a way to produce testable textual examples for internal directions, steering vectors, and some LoRA backdoor payloads. The generated example can be checked by running it through the subject model. These results describe interpretability experiments, not a general guarantee that generated explanations reveal a model's internal reasoning. The material also notes the MAEM verbalized some workspace inner monologue, without specifying how reliably this works.

Practical Implication

No separate implication has been recorded yet.

Agent-Ready Context
The paper introduces **Maximally-Activating Example Metamodels (MAEMs)**, trained to generate text whose activations align with a supplied residual-stream direction. A **Qwen3.6-27B MAEM** beat corpus search and a natural-language autoencoder on held-out activation inversion, and activated **89% of held-out SAE features**.
For builders investigating model behavior, this offers a way to produce testable textual examples for internal directions, steering vectors, and some LoRA backdoor payloads. The generated example can be checked by running it through the subject model.
These results describe interpretability experiments, not a general guarantee that generated explanations reveal a model's internal reasoning. The material also notes the MAEM verbalized some workspace inner monologue, without specifying how reliably this works.
Connected Context · Feed7 Judgment

MAEMs extend SAE feature verbalization from readable descriptions toward generated, model-checkable examples that activate a supplied residual direction. This makes internal directions easier to probe, while the results still support treating generated interpretations as hypotheses rather than explanations of the model’s reasoning.

Context Map
modelresearchsecurity#reasoning