Generative Latent Priors for Interpreting and Steering Protein Language Models
A diffusion prior for ESM-2 activations improves some protein-feature probes and isolated steering, but decoded sequences lose structural plausibility over repeated rounds. The work is outside everyday coding-agent practice.
Generative Latent Priors learn distributions of frozen ESM-2 activations and are evaluated with **ESM-2-8M** and **ESM-2-650M**. As probes, they perform strongly on discrete protein concepts; as an editing prior, they steer three biochemical properties in isolation.
For builders working with protein language models, the result suggests activation priors can help inspect representations and guide edits, but generated sequences need validation after decoding and repeated editing.
Generative Latent Priors learn distributions of frozen ESM-2 activations and are evaluated with **ESM-2-8M** and **ESM-2-650M**. As probes, they perform strongly on discrete protein concepts; as an editing prior, they steer three biochemical properties in isolation. For builders working with protein language models, the result suggests activation priors can help inspect representations and guide edits, but generated sequences need validation after decoding and repeated editing. Repeated decoding compromised structural plausibility, and only the Helix motif succeeded in the reported SAE-based structural steering experiments. The paper attributes naturalness problems to rewriting coverage and the decoder as well as describing limitations of the method.
This extends activation-based interpretation and steering into protein language models, while showing a sharper constraint: useful probes and isolated edits do not ensure plausible decoded sequences. The reported structural failures, including limited SAE-based steering success, make post-edit validation central to applying these methods.