The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.
Improving Steering Vectors by Targeting Sparse Autoencoder Features
Open accessAbstract
To control the behavior of language models, steering methods attempt to ensure that outputs of the model satisfy specific pre-defined properties. Adding steering vectors to the model is a promising method of model control that is easier than finetuning, and may be more robust than prompting. However, it can be difficult to anticipate the effects of steering vectors produced by methods such as CAA [Panickssery et al., 2024] or the direct use of SAE latents [Templeton et al., 2024]. In our work, we address this issue by using SAEs to measure the effects of steering vectors, giving us a method that can be used to understand the causal effect of any steering vector intervention. We use this method for measuring causal effects to develop an improved steering method, SAE-Targeted Steering (SAE-TS), which finds steering vectors to target specific SAE features while minimizing unintended side effects. We show that overall, SAE-TS balances steering effects with coherence better than CAA and SAE feature steering, when evaluated on a range of tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Increasing the steering factor monotonically reduces instruct score in all methods (larger steering vectors harm capabilities).
“We notice that increasing the factor monotonically reduces instruct score in all methods, i.e. larger steering vectors harm capabilities; this agrees with prior findings (Durmus et al., 2024; Chalnev et al., 2024).”
From AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders (2025) · cites this paper as Chalnev et al., 2024
Section 5 (Evaluations; page unavailable) reports that coherence eventually drops as steering scale increases, supporting the general point that excessive steering can harm output quality. But coherence initially rises for SAE-TS, so the paper does not support a monotonic decline in all methods; it also does not report an “instruct score.” (LLM, conf=0.88)
Experimental — accuracy not yet measuredRecent work reports mixed results when evaluating SAEs for steering; SAEs suffer from a tradeoff between model control and capabilities preservation.
“Recent work reports mixed results when evaluating SAEs for steering; SAEs (but also several other steering methods) suffer from a tradeoff between model control and capabilities preservation (Mayne et al., 2024; Chalnev et al., 2024; Durmus et al., 2024; Bhalla et al., 2025).”
From AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders (2025) · cites this paper as Chalnev et al., 2024
The Evaluations section reports mixed performance across steering methods and tasks: SAE-TS leads on seven of nine tasks, while CAA leads or ties on the other two. It also measures steering success alongside output coherence, which can fall as steering strength increases. But the Limitations section says the authors did not investigate tasks or features relevant to capabilities, so the paper does not establish a tradeoff with capabilities preservation. (LLM, conf=0.87)
Experimental — accuracy not yet measuredPrevious evaluation benchmarks for steering are not general-purpose; they either rely on a limited set of tasks or condition generation on a fixed prefix.
“Previous evaluation benchmarks for steering are not general-purpose; they either rely on a limited set of tasks (Zou et al., 2023; Makelov, 2024; Bhalla et al., 2025) or condition generation on a fixed prefix (Chalnev et al., 2024).”
From AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders (2025) · cites this paper as Chalnev et al., 2024
Mechanistic interpretability decomposition methods enable steering models toward a narrower range of behaviors with fewer side effects.
“Mechanistic interpretability decomposition methods enable the steering of models toward a narrower range of behaviors with fewer side effects (Chalnev et al., 2024).”
From Open Problems in Mechanistic Interpretability (2025) · cites this paper as Chalnev et al., 2024
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., et al. (2022) · arXiv · doi:10.48550/arxiv.2209.10652 · Cited by 51
Flagged first, then by citations
Research works printed but not linked: 19
- Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. Linear algebraic structure of word senses, with applications to polysemy. Transactions of the Association for Computational Linguistics, 6:483-495, 2018. doi: 10.1162/tacl_a_00034. URL https://aclanthology.org/Q18-1034. · DOI · Paper page
- Yuanpu Cao, Tianrong Zhang, Bochuan Cao, Ziyi Yin, Lu Lin, Fenglong Ma, and Jinghui Chen. Personalized steering of large language models: Versatile steering vectors through bi-directional preference optimization, 2024. URL https://arxiv.org/abs/2406.00045. · arXiv
- Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason Wei. Scaling instruction-finetuned language models, 2022. URL https://arxiv.org/abs/2210.11416. · arXiv
- Hoagy Cunningham, Aidan Ewart, Logan Riggs, et al. Sparse autoencoders find highly interpretable features in language models, 2023. URL https://arxiv.org/abs/2309.08600. · arXiv
- Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, et al. The llama 3 herd of models, 2024. URL https://arxiv.org/abs/2407.21783. · arXiv
- Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders, 2024. URL https://arxiv.org/abs/2406.04093. · arXiv
- Gemma Team. Gemma 2: Improving open language models at a practical size, 2024. URL https://arxiv.org/ abs/2408.00118. · arXiv
- Bruce W. Lee, Inkit Padhi, Karthikeyan Natesan Ramamurthy, Erik Miehling, Pierre Dognin, Manish Nagireddy, and Amit Dhurandhar. Programming refusal with conditional activation steering, 2024. URL https://arxiv.org/ abs/2409.05907. · arXiv
- Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah, and Neel Nanda. Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2, 2024. URL https://arxiv.org/abs/2408.05147. · arXiv
- Samuel Marks, Can Rager, Eric J. Michaud, et al. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models. Computing Research Repository, arXiv:2403.19647, 2024. URL https: //arxiv.org/abs/2403.19647. · arXiv
- Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom in: An introduction to circuits. Distill, 2020. doi: 10.23915/distill.00024.001. · DOI · Paper page
- Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback, 2022. URL https://arxiv.org/abs/2203.02155. · arXiv
- Nina Panickssery, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering Llama-2 via Contrastive Activation Addition, 2024. URL https://arxiv.org/abs/2312.06681. · arXiv
- Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, and Neel Nanda. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders, 2024. URL https://arxiv.org/abs/2407.14435. · arXiv
- Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J. Vazquez, Ulisse Mini, and Monte MacDiarmid. Activation addition: Steering language models without optimization, 2024. URL https://arxiv.org/abs/2308. 10248.
- Eric Wallace, Kai Xiao, Reimar Leike, Lilian Weng, Johannes Heidecke, and Alex Beutel. The instruction hierarchy: Training llms to prioritize privileged instructions, 2024. URL https://arxiv.org/abs/2404.13208. · arXiv
- Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does llm safety training fail?, 2023. URL https://arxiv.org/abs/2307.02483. · arXiv
- Yu Zhao, Alessio Devoto, Giwon Hong, Xiaotang Du, Aryo Pradipta Gema, Hongru Wang, Xuanli He, Kam-Fai Wong, and Pasquale Minervini. Steering knowledge selection behaviours in llms via sae-based representation engineering, 2024. URL https://arxiv.org/abs/2410.15999. · arXiv
- Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, J. Zico Kolter, and Dan Hendrycks. Representation engineering: A top-down approach to ai transparency, 2023. URL https://arxiv.org/abs/2310.01405. "<bos>I think back on my childhood and remember growing up in Atlanta with my family and best friend.\n\nWe spent our summers playing baseball in the neighborhood and learning to" "<bos>I think the idea of The Black Keys' new video for "Lonely Phaser" is inspired by a "Twilight" movie.\n\nShot in London and directed by" "<bos>I think we're officially in Fall. It's been a chilly season for the past couple months -and this week is no exception. While we'" "<bos>I think the new movie "Still Alice" based on the memoir of Alice Koda is one of the most compelling films of the year. The film follows the" · arXiv
Cited by
Sorted by citations
Open Problems in Mechanistic Interpretability
Sharkey, L., Chughtai, B., Batson, J., Lindsey, J., Wu, J., Bushnaq, L., Goldowsky-Dill, N., Heimersheim, S., Ortega, A., Bloom, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.16496 · Cited by 291
Self-citationAxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders
Wu, Z., Arora, A., Geiger, A., Wang, Z., Huang, J., Jurafsky, D., Manning, C. D., & Potts, C. (2025) · arXiv · doi:10.48550/arxiv.2501.17148
“Sparse autoencoders (SAEs) aim to enable self-supervised and thus scalable decomposition of the representation space into meaningful concepts (Tem-pleton et al., 2024; Chalnev et al., 2024; Makelov, 2024; O’Brien et al., 2024; Gao et al., 2024).”
Influential
Metadata
- Authors
- Chalnev, S., Siu, M., & Conmy, A.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2024
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2411.02193
- OpenAlex
- W4404354935
- Cited by
- 2
- References
- 1
- Citations by year
- 2026: 1 · 2025: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- steering vectors, sparse autoencoders, unintended side effects, model control
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
