The DOI matches its DataCite record; the references it cites are not checked yet.
Locating and Editing Factual Associations in GPT
Open accessAbstract
We analyze the storage and recall of factual associations in autoregressive transformer language models, finding evidence that these associations correspond to localized, directly-editable computations. We first develop a causal intervention for identifying neuron activations that are decisive in a model's factual predictions. This reveals a distinct set of steps in middle-layer feed-forward modules that mediate factual predictions while processing subject tokens. To test our hypothesis that these computations correspond to factual association recall, we modify feed-forward weights to update specific factual associations using Rank-One Model Editing (ROME). We find that ROME is effective on a standard zero-shot relation extraction (zsRE) model-editing task, comparable to existing methods. To perform a more sensitive evaluation, we also evaluate ROME on a new dataset of counterfactual assertions, on which it simultaneously maintains both specificity and generalization, whereas other methods sacrifice one or another. Our results confirm an important role for mid-layer feed-forward modules in storing factual associations and suggest that direct manipulation of computational mechanisms may be a feasible approach for model editing. The code, dataset, visualizations, and an interactive demo notebook are available at https://rome.baulab.info/
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“FT-EWC [20], ROME [18], MEMIT [19], and MEND [31] edit the long-term memory stored in the LLMs' model parameters, but they either do not support continual editing or have negative effects on irrelevant knowledge (poor locality).”
From WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models (2024) · cites this paper as [18]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Representations in later hidden layers propagate through residual connections without drastic changes [38, 18], enabling effective early exit in LLMs [39, 40].”
From WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models (2024) · cites this paper as [18]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Furthermore, some neurons exhibit task-specific activations and can be predictive of model performance on these tasks [28,30-32].”
From From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Task (2025) · cites this paper as [28]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“research has shown that neurons across different layers tend to specialize, with middle layers often containing neurons that represent higher-level contextual features [26-29].”
From From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Task (2025) · cites this paper as [28]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Notably, these units are mostly located in intermediate and deeper layers, matching results from previous research that show higher-level reasoning and conceptual encodings in LLMs can be found in these layers [26-29].”
From From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Task (2025) · cites this paper as [28]
Experimental — accuracy not yet measured
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These methods compute rank-one updates that modify model behavior for specific factual associations while preserving performance on unrelated tasks.”
From Subspace Collisions in Knowledge Editing: Orthogonal Low-Rank Updates for Scalable, Stable Model Edits (2026) · cites this paper as [6]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Following prior work [6,7], we target mid-to-late MLP layers where factual knowledge is most directly encoded.”
From Subspace Collisions in Knowledge Editing: Orthogonal Low-Rank Updates for Scalable, Stable Model Edits (2026) · cites this paper as [6]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“ROME [6] introduced causal tracing to identify key layers and applied rank-one updates to MLP weights, while MEMIT [7] extended this to mass editing by applying multiple rank-one updates sequentially.”
From Subspace Collisions in Knowledge Editing: Orthogonal Low-Rank Updates for Scalable, Stable Model Edits (2026) · cites this paper as [6]
The cited source says what the claim attributes to it.
In the citing paper
“The ZsRE question-answering task [46] is extensively studied within the model editing literature [18, 19, 31, 15, 11]”
From WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models (2024) · cites this paper as [18]
The cited source says what the claim attributes to it.
In the citing paper
“ROME [18] uses causal analysis to pinpoint knowledge within specific MLP layers and modifies the entire matrix through least squares approximation.”
From WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models (2024) · cites this paper as [18]
The cited source says what the claim attributes to it.
In the citing paper
“ROME identifies factual associations and edits efficiently using MLP-based memories [18], extended by MEMIT for mass-editing [19].”
From WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models (2024) · cites this paper as [18]
The cited source says what the claim attributes to it.
In the citing paper
“[55] Presents a related study which intervenes on specific parts of the Transformer architecture [56] in order to make counterfactual edits to the model’s encoding of factual knowledge (what if Pierre Curie’s area of work is medicine?). They then observe that the model’s subsequent inferences (e.g. expectations about Pierre Curie’s professional accomplishments) are updated accordingly.”
From Symbols and grounding in large language models (2023) · cites this paper as [55]
The cited source says what the claim attributes to it.
In the citing paper
“These 286 units were predominantly located in the middle layers of the model (approximately layers 12-22), consistent with the distribution of units encoding the agent's position (Fig. 3) and with previous studies reporting that higher-level conceptual processing in LLMs tends to emerge in intermediate layers [26-29].”
From From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Task (2025) · cites this paper as [28]
The cited source says what the claim attributes to it.
In the citing paper
“Early approaches, such as ROME [6] and MEMIT [7], demonstrated that factual edits can be performed efficiently using low-rank weight updates applied to specific layers.”
From Subspace Collisions in Knowledge Editing: Orthogonal Low-Rank Updates for Scalable, Stable Model Edits (2026) · cites this paper as [6]
In the citing paper
“these edits should affect some other bits of knowledge (e.g., that the previously current president is now the past president) but leave many other facts unaffected [160].”
From Dissociating language and thought in large language models (2024) · cites this paper as [160]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 28
- Adi, Y., Kermany, E., Belinkov, Y., Lavi, O., and Goldberg, Y. Fine-grained analysis of sentence embeddings using auxiliary prediction tasks. In International Conference on Learning Representations (ICLR) , April 2017.
- Anderson, J. A. A simple neural network generating an interactive memory. Mathematical biosciences , 14(3-4):197-220, 1972.
- Bau, D., Liu, S., Wang, T., Zhu, J.-Y., and Torralba, A. Rewriting a deep generative model. In Proceedings of the European Conference on Computer Vision (ECCV) , 2020.
- Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. Language models are few-shot learners. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F., and Lin, H. (eds.), Advances in Neural Information Processing Systems , volume 33, pp. 1877-1901. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper/ 2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf .
- Dai, D., Dong, L., Hao, Y., Sui, Z., Chang, B., and Wei, F. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pp. 8493-8502, 2022.
- De Cao, N., Aziz, W., and Titov, I. Editing factual knowledge in language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pp. 6491-6506, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. URL https://aclanthology.org/2021.emnlp-main.522 .
- Elazar, Y., Kassner, N., Ravfogel, S., Ravichander, A., Hovy, E., Sch¨ utze, H., and Goldberg, Y. Measuring and Improving Consistency in Pretrained Language Models. Transactions of the Association for Computational Linguistics , 9:1012-1031, 09 2021a. ISSN 2307-387X. doi: 10.1162/tacl a 00410. URL https://doi.org/10.1162/tacl\_a\_00410 . · DOI
- Elazar, Y., Ravfogel, S., Jacovi, A., and Goldberg, Y. Amnesic probing: Behavioral explanation with amnesic counterfactuals. Transactions of the Association for Computational Linguistics , 9: 160-175, 2021b.
- Feder, A., Oved, N., Shalit, U., and Reichart, R. CausaLM: Causal model explanation through counterfactual language models. Computational Linguistics , 47(2):333-386, 2021.
- Geva, M., Schuster, R., Berant, J., and Levy, O. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pp. 5484-5495, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. URL https://aclanthology.org/2021.emnlp-main.446 .
- Hase, P., Diab, M., Celikyilmaz, A., Li, X., Kozareva, Z., Stoyanov, V., Bansal, M., and Iyer, S. Do language models have beliefs? methods for detecting, updating, and visualizing model beliefs. arXiv preprint arXiv:2111.13654 , 2021. · arXiv
- Hupkes, D., Veldhoen, S., and Zuidema, W. Visualisation and 'diagnostic classifiers' reveal how recurrent and recursive neural networks process hierarchical structure. Journal of Artificial Intelligence Research , 61:907-926, 2018.
- Jiang, Z., Xu, F. F., Araki, J., and Neubig, G. How can we know what language models know? Transactions of the Association for Computational Linguistics , 8:423-438, 2020. doi: 10.1162/ tacl a 00324. URL https://aclanthology.org/2020.tacl-1.28 .
- Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In Bengio, Y. and LeCun, Y. (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings , 2015. URL http://arxiv.org/abs/1412. 6980 .
- Kohonen, T. Correlation matrix memories. IEEE transactions on computers , 100(4):353-359, 1972.
- Meng, K., Sen Sharma, A., Andonian, A., Belinkov, Y., and Bau, D. Mass-editing memory in a transformer. arXiv preprint arXiv:2210.07229 , 2022. · arXiv
- Mitchell, E., Lin, C., Bosselut, A., Finn, C., and Manning, C. D. Fast model editing at scale. In International Conference on Learning Representations , 2021.
- Pearl, J. Direct and indirect effects. In Proceedings of the Seventeenth conference on Uncertainty in artificial intelligence , pp. 411-420, 2001.
- Pearl, J. Causality: Models, Reasoning and Inference . Cambridge University Press, USA, 2nd edition, 2009. ISBN 052189560X.
- Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners. OpenAI blog , pp. 9, 2019.
- Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research , 21(140):1-67, 2020.
- Sundararajan, M., Taly, A., and Yan, Q. Axiomatic attribution for deep networks. In International conference on machine learning , pp. 3319-3328. PMLR, 2017.
- Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. Attention is all you need. In Advances in neural information processing systems , pp. 5998-6008, 2017.
- Vig, J., Gehrmann, S., Belinkov, Y., Qian, S., Nevo, D., Sakenis, S., Huang, J., Singer, Y ., and Shieber, S. Causal mediation analysis for interpreting neural NLP: The case of gender bias. arXiv preprint arXiv:2004.12265 , 2020a. · arXiv
- Vig, J., Gehrmann, S., Belinkov, Y., Qian, S., Nevo, D., Singer, Y., and Shieber, S. M. Investigating gender bias in language models using causal mediation analysis. In NeurIPS , 2020b.
- Zhang, Y., Galley, M., Gao, J., Gan, Z., Li, X., Brockett, C., and Dolan, W. B. Generating informative and diverse conversational responses via adversarial information maximization. In NeurIPS , 2018.
- Zhao, S., Pascual, D., Brunner, G., and Wattenhofer, R. Of non-linearity and commutativity in BERT. In 2021 International Joint Conference on Neural Networks (IJCNN) , pp. 1-8. IEEE, 2021.
- Zhu, C., Rawat, A. S., Zaheer, M., Bhojanapalli, S., Li, D., Yu, F., and Kumar, S. Modifying memories in transformer models. arXiv preprint arXiv:2012.00363 , 2020. · arXiv
Cited by
Sorted by citations
Explainability for Large Language Models: A Survey
Zhao, H., Chen, H., Yang, F., Liu, N., Deng, H., Cai, H., Wang, S., Yin, D., & Du, M. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3639372 · Cited by 681
RWKV: Reinventing RNNs for the Transformer Era
Peng, B., Alcaide, E., Anthony, Q., Albalak, A., Arcadinho, S., Biderman, S., Cao, H., Cheng, X., Chung, M., Derczynski, L., et al. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.936 · Cited by 390
Talking about Large Language Models
Shanahan, M. (2024) · Communications of the ACM · doi:10.1145/3624724 · Cited by 381
Dissociating language and thought in large language models
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.01.011 · Cited by 345
Empowering biomedical discovery with AI agents
Gao, S., Fang, A., Huang, Y., Giunchiglia, V., Noori, A., Schwarz, J. R., Ektefaie, Y., Kondic, J., & Zitnik, M. (2024) · Cell · doi:10.1016/j.cell.2024.09.022 · Cited by 315
Open Problems in Mechanistic Interpretability
Sharkey, L., Chughtai, B., Batson, J., Lindsey, J., Wu, J., Bushnaq, L., Goldowsky-Dill, N., Heimersheim, S., Ortega, A., Bloom, J., et al. (2025) · arXiv · doi:10.48550/arxiv.2501.16496 · Cited by 291
Self-citationErasing Concepts from Diffusion Models
Gandikota, R., Materzyńska, J., Fiotto-Kaufman, J., & Bau, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00230 · Cited by 361
Self-citationMedical large language models are vulnerable to data-poisoning attacks
Alber, D. A., Yang, Z., Alyakin, A., Yang, E., Rai, S., Valliani, A. A., Zhang, J., Rosenbaum, G. R., Amend-Thomas, A. K., Kurland, D. B., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03445-1 · Cited by 242
Retrieval-augmented generation for educational application: A systematic survey
Li, Z., Wang, Z., Wang, W., Hung, K., Xie, H., & Wang, F. L. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100417 · Cited by 151
Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey
Ling, C., Zhao, X., Lu, J., Deng, C., Zheng, C., Wang, J., Chowdhury, T., Li, Y., Cui, H., Zhang, X., et al. (2025) · ACM Computing Surveys · doi:10.1145/3764579 · Cited by 385
Ablating Concepts in Text-to-Image Diffusion Models
Kumari, N., Zhang, B., Wang, S.-Y., Shechtman, E., Zhang, R., & Zhu, J.-Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02074 · Cited by 207
Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks
Räuker, T., Ho, A., Casper, S., & Hadfield-Menell, D. (2023) · 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00039 · Cited by 122
Unified Concept Editing in Diffusion Models
Gandikota, R., Orgad, H., Belinkov, Y., Materzyńska, J., & Bau, D. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00503 · Cited by 201
Self-citationA comprehensive survey of large language models and multimodal large language models in medicine
Xiao, H., Zhou, F., Liu, X., Liu, T., Ли, Ж., Liu, X., & Huang, X. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102888 · Cited by 154
(2023) · doi:10.18653/v1/2023.emnlp-main.632 · Cited by 115
Language Model Behavior: A Comprehensive Survey
Chang, T. A., & Bergen, B. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00492 · Cited by 105
Barman, D., Guo, Z., & Conlan, O. (2024) · Machine Learning with Applications · doi:10.1016/j.mlwa.2024.100545 · Cited by 127
Symbols and grounding in large language models
Pavlick, E. (2023) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2022.0041 · Cited by 95
“[ 55] Presents a related study which intervenes on specific parts of the Transformer architecture [56] in order to make counterfactual edits to the model’s encoding of factual knowledge (what if Pierre Curie’s area of work is medicine ?).”
BackgroundLanguage Models as Agent Models
Andreas, J. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.423 · Cited by 92
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Casper, S., Davies, X., Shi, C., Gilbert, T. K., Scheurer, J., Rando, J., Freedman, R., Korbak, T., Lindner, D., Freire, P., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.15217 · Cited by 101
Continual Learning of Large Language Models: A Comprehensive Survey
Shi, H., Xu, Z., Wang, H., Qin, W., Wang, W., Wang, Y., Wang, Z., Ebrahimi, S., & Wang, H. (2025) · ACM Computing Surveys · doi:10.1145/3735633 · Cited by 291
MACE: Mass Concept Erasure in Diffusion Models
Lu, S., Wang, Z., Li, L., Liu, Y., & Adams Wai Kin, K. O. N. G. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00615 · Cited by 130
Black-Box Access is Insufficient for Rigorous AI Audits
Casper, S., Ezell, C., Siegmann, C., Kolt, N., Curtis, T. L., Bucknall, B., Haupt, A., Wei, K., Scheurer, J., Hobbhahn, M., et al. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3659037 · Cited by 86
Self-citationLanguage in Brains, Minds, and Machines
Tuckute, G., Kanwisher, N., & Fedorenko, E. G. (2024) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-120623-101142 · Cited by 78
The alignment problem from a deep learning perspective
Ngo, R., Chan, L., & Mindermann, S. (2022) · arXiv · doi:10.48550/arxiv.2209.00626 · Cited by 68
Barman, K. G., Wood, N. G., & Pawłowski, P. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09778-2 · Cited by 74
Han, S., Wang, M., Zhang, J., Li, D., & Duan, J. (2024) · Electronics · doi:10.3390/electronics13245040 · Cited by 81
LM vs LM: Detecting Factual Errors via Cross Examination
Cohen, R., Hamri, M., Geva, M., & Globerson, A. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.778 · Cited by 60
Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
Li, K., Patel, O., Viégas, F., Pfister, H., & Wattenberg, M. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1797 · Cited by 117
(2023) · doi:10.18653/v1/2023.emnlp-main.971 · Cited by 55
Can We Edit Factual Knowledge by In-Context Learning?
Zheng, C., Li, L., Dong, Q., Fan, Y., Wu, Z., Xu, J., & Chang, B. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.296 · Cited by 54
Dissecting Recall of Factual Associations in Auto-Regressive Language Models
Geva, M., Bastings, J., Filippova, K., & Globerson, A. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.751 · Cited by 53
Zhang, Z., Yao, Y., Zhang, A., Tang, X., Ma, X., He, Z., Wang, Y., Gerstein, M., Wang, R., Liu, G., et al. (2025) · ACM Computing Surveys · doi:10.1145/3719341 · Cited by 51
Evaluating language models for mathematics through interactions
Collins, K. M., Jiang, A. Q., Frieder, S., Wong, L., Zilka, M., Bhatt, U., Lukasiewicz, T., Wu, Y., Tenenbaum, J. B., Hart, W., et al. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2318124121 · Cited by 108
Towards Lifelong Learning of Large Language Models: A Survey
Zheng, J., Qiu, S., Shi, C., & Ma, Q. (2025) · ACM Computing Surveys · doi:10.1145/3716629 · Cited by 93
A Survey on Symbolic Knowledge Distillation of Large Language Models
Acharya, K., Velasquez, A., & Song, H. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3428519 · Cited by 56
Context-faithful Prompting for Large Language Models
Zhou, W., Zhang, S., Poon, H., & Chen, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.968 · Cited by 141
PMET: Precise Model Editing in a Transformer
Li, X., Li, S., Song, S., Yang, J., Ma, J., & Yu, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29818 · Cited by 125
Semnani, S. J., Yao, V., Zhang, H., & Lam, M. S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.157 · Cited by 78
Evaluating the Ripple Effects of Knowledge Editing in Language Models
Cohen, R., Biran, E., Yoran, O., Globerson, A., & Geva, M. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00644 · Cited by 42
In-Context Learning Creates Task Vectors
Hendel, R., Geva, M., & Globerson, A. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.624 · Cited by 229
Recent Advances in Interactive Machine Translation With Large Language Models
Wang, Y., Zhang, J., Shi, T., Deng, D., Tian, Y., & Matsumoto, T. (2024) · IEEE Access · doi:10.1109/access.2024.3487352 · Cited by 54
Explaining AI through mechanistic interpretability
Kästner, L., & Crook, B. (2024) · European Journal for Philosophy of Science · doi:10.1007/s13194-024-00614-4 · Cited by 59
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey
Kumar, S., Balachandran, V., Njoo, L., Anastasopoulos, A., & Tsvetkov, Y. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.241 · Cited by 38
DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Models
Wu, X., Li, J., Xu, M., Dong, W., Wu, S., Bian, C., & Xiong, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.174 · Cited by 119
Machine Unlearning: Taxonomy, Metrics, Applications, Challenges, and Prospects
Li, N., Zhou, C., Gao, Y., Chen, H., Zhang, Z., Kuang, B., & Fu, A. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3530988 · Cited by 75
Inseq: An Interpretability Toolkit for Sequence Generation Models
Sarti, G., Feldhus, N., Sickert, L., & van der Wal, O. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-demo.40 · Cited by 56
A comprehensive overview of Generative AI (GAI): Technologies, applications, and challenges
Zhang, Z., Zhang, J., Zhang, X., & Mai, W. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129645 · Cited by 45
Large language models for air transportation: A critical review
Liu, Y. (2024) · Journal of the Air Transport Research Society · doi:10.1016/j.jatrs.2024.100024 · Cited by 33
ArtAdapter: Text-to-Image Style Transfer using Multi-Level Style Encoder and Explicit Adaptation
Chen, D.-Y., Tennent, H., & Hsu, C.-W. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00823 · Cited by 41
Kenthapadi, K., Sameki, M., & Taly, A. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671467 · Cited by 36
Future Shock: Generative AI and the International AI Policy and Governance Crisis
Leslie, D., & Perini, A. M. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.88b4cc98 · Cited by 33
Emergent Linear Representations in World Models of Self-Supervised Sequence Models
Nanda, N., Lee, A., & Wattenberg, M. (2023) · Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/2023.blackboxnlp-1.2 · Cited by 55
Digital forgetting in large language models: a survey of unlearning methods
Blanco-Justicia, A., Jebreel, N., Manzanares-Salor, B., Sánchez, D., Domingo-Ferrer, J., Collell, G., & Eeik Tan, K. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-024-11078-6 · Cited by 31
The argument for near-term human disempowerment through AI
Dung, L. (2024) · AI & Society · doi:10.1007/s00146-024-01930-2 · Cited by 43
How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advances
Zhang, Z., Fang, M., Chen, L., Namazi‐Rad, M., & Wang, Y.-X. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.516 · Cited by 66
Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledge
Onoe, Y., Zhang, M., Padmanabhan, S., Durrett, G., & Choi, E. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.300 · Cited by 51
DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answering
Neeman, E., Aharoni, R., Honovich, O., Choshen, L., Szpektor, I., & Abend, O. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.559 · Cited by 66
MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRA
Yu, L., Chen, Q., Zhou, J., & He, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29916 · Cited by 51
Prompt Perturbation in Retrieval-Augmented Generation based Large Language Models
Hu, Z., Wang, C., Shu, Y., Paik, H.-Y., & Zhu, L. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671932 · Cited by 39
CatVersion: Concatenating Embeddings for Diffusion-Based Text-to-Image Personalization
Zhao, R., Zhu, M., Dong, S., Cheng, D., Wang, N., & Gao, X. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3531917 · Cited by 28
Red teaming large language models: A comprehensive review and critical analysis
Jabbar, M. S., Al-Azani, S., Alotaibi, A., & Ahmed, M. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104239 · Cited by 31
Survey on the Role of Mechanistic Interpretability in Generative AI
Ranaldi, L. (2025) · Big Data and Cognitive Computing · doi:10.3390/bdcc9080193 · Cited by 27
Diao, S., Xu, T., Xu, R., Wang, J., & Zhang, T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.280 · Cited by 43
DSI++: Updating Transformer Memory with New Documents
Mehta, S., Gupta, J., Tay, Y., Dehghani, M., Tran, V., Rao, J., Najork, M., Strubell, E., & Metzler, D. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.510 · Cited by 21
HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models
Gutiérrez, B. J., Shu, Y., Gu, Y., Yasunaga, M., & Su, Y. (2024) · arXiv · doi:10.48550/arxiv.2405.14831 · Cited by 152
Plisiecki, H., & Sobieszek, A. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02212-3 · Cited by 24
Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmark
Hoelscher-Obermaier, J., Persson, J., Kran, E., Konstas, I., & Barez, F. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.733 · Cited by 24
Chun, J., & Elkins, K. (2023) · International Journal of Digital Humanities · doi:10.1007/s42803-023-00069-8 · Cited by 22
“The nature of large DNN black box models like LLMs is fundamentally different from previous models and hence requires a different approach to XAI.”
SoK: Prompt Hacking of Large Language Models
Rababah, B., Wu, S. T., Kwiatkowski, M. R., Leung, C. K.-S., & Akçora, C. G. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825103 · Cited by 20
The Life Cycle of Knowledge in Big Language Models: A Survey
Cao, B., Lin, H., Han, X., & Sun, L. (2024) · Machine Intelligence Research · doi:10.1007/s11633-023-1416-x · Cited by 18
“, 2022a), causal-inspired (Meng et al., 2022), attention-based (Clark et al.”
BackgroundMethodInfluentialA Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models
Stolfo, A., Jin, Z., Shridhar, K., Schoelkopf, B., & Sachan, M. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.32 · Cited by 17
Measuring and Modifying Factual Knowledge in Large Language Models
Pezeshkpour, P. (2023) · International Conference on Machine Learning and Applications · doi:10.1109/icmla58977.2023.00122 · Cited by 31
Chain-of-Skills: A Configurable Model for Open-Domain Question Answering
Ma, K., Cheng, H., Zhang, Y., Liu, X., Nyberg, E., & Gao, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.89 · Cited by 29
Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models
Qi, J., Fernández, R., & Bisazza, A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.658 · Cited by 59
Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIP
Palit, V., Pandey, R., Arora, A., & Liang, P. P. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00307 · Cited by 30
On Early Detection of Hallucinations in Factual Question Answering
Snyder, B., Moisescu, M., & Zafar, M. B. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671796 · Cited by 46
Large language models for causal hypothesis generation in science
Cohrs, K.-H., Díaz, E., Sitokonstantinou, V., Varando, G., & Camps‐Valls, G. (2024) · Machine Learning Science and Technology · doi:10.1088/2632-2153/ada47f · Cited by 23
Memorization in Deep Learning: A Survey
Wei, J., Zhang, Y., Zhang, L. Y., Ding, M., Chen, C., Ong, K., Zhang, J., & Xiang, Y. (2025) · ACM Computing Surveys · doi:10.1145/3769076 · Cited by 26
Grounding neuroscience in behavioral changes using artificial neural networks
Lindsay, G. W. (2023) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2023.102816 · Cited by 16
How Can Deep Neural Networks Inform Theory in Psychological Science?
McGrath, S. W., Russin, J., Pavlick, E., & Feiman, R. (2024) · Current Directions in Psychological Science · doi:10.1177/09637214241268098 · Cited by 26
Editing Factual Knowledge and Explanatory Ability of Medical Large Language Models
Xu, D., Zhang, Z., Zhu, Z., Lin, Z., Liu, Q., Wu, X., Xu, T., Wang, W., Ye, Y., Zhao, X., et al. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679673 · Cited by 20
Stolfo, A., Belinkov, Y., & Sachan, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.435 · Cited by 107
Self-citationThe Geometry of Concepts: Sparse Autoencoder Feature Structure
Li, Y., Michaud, E. J., Baek, D. D., Engels, J., Sun, X., & Tegmark, M. (2025) · Entropy · doi:10.3390/e27040344 · Cited by 45
Physics of Language Models: Part 3.1, Knowledge Storage and ExtractionPreprint
Allen-Zhu, Z., & Li, Y. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5250633 · Cited by 212
Physics of Language Models: Part 3.1, Knowledge Storage and Extraction
Allen-Zhu, Z., & Li, Y. (2023) · arXiv · doi:10.48550/arxiv.2309.14316 · Cited by 14
He, X., Fan, W., Wang, R., Wang, Y., Wang, X. Y., Pan, S., & Wang, X. Y. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107317 · Cited by 18
Structured knowledge-based causal discovery: Agentic streams of thought
Meier, S., Raut, P. N., Mahr, F., Thielen, N., Franke, J. E., & Risch, F. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104202 · Cited by 14
Watson, E. ‘., Viana, T., & Zhang, S. (2023) · AI · doi:10.3390/ai4010007 · Cited by 18
FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledge
Feng, S., Balachandran, V., Bai, Y., & Tsvetkov, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.59 · Cited by 40
Chen, C., Merullo, J., & Eickhoff, C. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657841 · Cited by 19
A Survey on Knowledge Editing of Neural Networks
Mazzia, V., Pedrani, A., Caciolai, A., Rottmann, K., & Bernardi, D. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3498935 · Cited by 41
Multimodal Neurons in Pretrained Text-Only Transformers
Schwettmann, S., Chowdhury, N., Klein, S., Bau, D., & Torralba, A. B. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00308 · Cited by 18
Self-citationHu, X., Li, D., Hu, B., Zheng, Z., Liu, Z., & Zhang, M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29784 · Cited by 22
Overcoming Generic Knowledge Loss with Selective Parameter Update
Zhang, W., Janson, P., Aljundi, R., & Elhoseiny, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02270 · Cited by 15
LLMs Meet Library Evolution: Evaluating Deprecated API Usage in LLM-Based Code Completion
Wang, C., Huang, K., Zhang, J., Feng, Y., Zhang, L., Liu, Y., & Peng, X. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00245 · Cited by 17
Modularity in Deep Learning: A Survey
Sun, H., & Guyon, I. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-37963-5_40 · Cited by 15
Chen, Y., Cao, P., Chen, Y., Liu, K., & Zhao, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29735 · Cited by 12
Layered Bias: Interpreting Bias in Pretrained Large Language Models
Prakash, N., & Lee, R. K.-W. (2023) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2023.blackboxnlp-1.22 · Cited by 15
Wu, D., Yang, J., & Wang, K. (2024) · Patterns · doi:10.1016/j.patter.2024.101030 · Cited by 14
Metadata
- Authors
- Meng, K., Bau, D., Andonian, A., & Belinkov, Y.
- Affiliations
- Northeastern University (US)
- MIT Computer Science and Artificial Intelligence Laboratory (US)
- Technion – Israel Institute of Technology (IL)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2202.05262
- OpenAlex
- W4281657280
- Cited by
- 230
- References
- 0
- Citations by year
- 2026: 7 · 2025: 67 · 2024: 75 · 2023: 79 · 2022: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Open Philanthropy Project
- Azrieli Foundation
- Defense Advanced Research Projects Agency (HR0011, FA8750)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- model editing, Rome, causal intervention, neuronal activation, zero-shot relation extraction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
