Locating and Editing Factual Associations in GPT
Open accessAbstract
We analyze the storage and recall of factual associations in autoregressive transformer language models, finding evidence that these associations correspond to localized, directly-editable computations. We first develop a causal intervention for identifying neuron activations that are decisive in a model's factual predictions. This reveals a distinct set of steps in middle-layer feed-forward modules that mediate factual predictions while processing subject tokens. To test our hypothesis that these computations correspond to factual association recall, we modify feed-forward weights to update specific factual associations using Rank-One Model Editing (ROME). We find that ROME is effective on a standard zero-shot relation extraction (zsRE) model-editing task, comparable to existing methods. To perform a more sensitive evaluation, we also evaluate ROME on a new dataset of counterfactual assertions, on which it simultaneously maintains both specificity and generalization, whereas other methods sacrifice one or another. Our results confirm an important role for mid-layer feed-forward modules in storing factual associations and suggest that direct manipulation of computational mechanisms may be a feasible approach for model editing. The code, dataset, visualizations, and an interactive demo notebook are available at https://rome.baulab.info/
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
FT-EWC, ROME, MEMIT, and MEND edit long-term memory stored in LLM parameters, but they either do not support continual editing or have negative effects on irrelevant knowledge (poor locality).
“FT-EWC [20], ROME [18], MEMIT [19], and MEND [31] edit the long-term memory stored in the LLMs' model parameters, but they either do not support continual editing or have negative effects on irrelevant knowledge (poor locality).”
From WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models (2024) · cites this paper as [18]
Section 3 (p. 5) describes ROME as editing factual associations in model weights. Section 3.7 (p. 9) says it edits only one fact at a time, but does not establish that it cannot perform continual, sequential edits. The paper reports good specificity for ROME rather than poor locality; the other named methods have separate citations and are not assessed here. (LLM, conf=0.83)
Experimental — accuracy not yet measuredRepresentations in later hidden layers propagate through residual connections without drastic changes, enabling effective early exit in LLMs.
“Representations in later hidden layers propagate through residual connections without drastic changes [38, 18], enabling effective early exit in LLMs [39, 40].”
From WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models (2024) · cites this paper as [18]
Section 2.2 (p. 4) finds that higher-layer states’ causal effects depend little on subsequent MLP activity, and Section 2.3 (p. 4) proposes that information accumulated in middle layers is copied by attention at high layers. These passages support information persisting into later layers, but do not establish that representations propagate through residual connections without drastic changes. The early-exit claim is attached to other citations, not this source. (LLM, conf=0.72)
Experimental — accuracy not yet measuredSome neurons exhibit task-specific activations and can be predictive of model performance on these tasks.
“Furthermore, some neurons exhibit task-specific activations and can be predictive of model performance on these tasks [28,30-32].”
From From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Task (2025) · cites this paper as [28]
The Introduction (p. 1) reports that certain activations have a causal effect on factual predictions, and Section 2.3 (p. 4) localizes factual recall to middle-layer MLP modules. This supports a role for activations in a specific model behavior, but the paper does not show that individual neurons’ activations predict overall model performance on tasks. (LLM, conf=0.85)
Experimental — accuracy not yet measuredMiddle layers often contain neurons that represent higher-level contextual features.
“research has shown that neurons across different layers tend to specialize, with middle layers often containing neurons that represent higher-level contextual features [26-29].”
From From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Task (2025) · cites this paper as [28]
The causal-tracing results (Section 2, p. 2; Section 2.2, p. 4) identify a role for middle-layer MLPs in recalling factual associations, distinct from attention’s role in later layers. But the paper does not establish that individual middle-layer neurons represent higher-level contextual features; Section 3 (p. 5) explicitly distinguishes its MLP-level account from a per-neuron view. (LLM, conf=0.86)
Experimental — accuracy not yet measuredHigher-level reasoning and conceptual encodings in LLMs can be found in intermediate and deeper layers.
“Notably, these units are mostly located in intermediate and deeper layers, matching results from previous research that show higher-level reasoning and conceptual encodings in LLMs can be found in these layers [26-29].”
From From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Task (2025) · cites this paper as [28]
Section 2.3 (p. 4) locates factual associations in middle-layer MLPs and says high-layer attention copies that information to the last token. Figure 2 (p. 2) likewise identifies causal sites in middle and late layers during factual recall. These findings support a narrower layer-location claim, but the paper does not establish that higher-level reasoning or conceptual encodings are found in those layers. (LLM, conf=0.86)
Experimental — accuracy not yet measured
These methods compute rank-one updates that modify model behavior for specific factual associations while preserving performance on unrelated tasks.
“These methods compute rank-one updates that modify model behavior for specific factual associations while preserving performance on unrelated tasks.”
From Subspace Collisions in Knowledge Editing: Orthogonal Low-Rank Updates for Scalable, Stable Model Edits (2026) · cites this paper as [6]
Section 3.1 (p. 6) says ROME uses a rank-one weight update to insert a specific factual association. Section 3.2 (p. 6) evaluates specificity using accuracy on an unrelated fact, but the paper does not establish preserved performance across unrelated tasks; that part of the claim is broader than the evidence. (LLM, conf=0.86)
Experimental — accuracy not yet measuredFollowing prior work, mid-to-late MLP layers are targeted where factual knowledge is most directly encoded.
“Following prior work [6,7], we target mid-to-late MLP layers where factual knowledge is most directly encoded.”
From Subspace Collisions in Knowledge Editing: Orthogonal Low-Rank Updates for Scalable, Stable Model Edits (2026) · cites this paper as [6]
Section 2.3 (p. 4) locates factual associations in MLP modules at specific middle layers, supporting the choice of mid-layer MLPs. But Figure 2 (p. 2) distinguishes that middle-layer MLP site from a late-layer site where attention, not MLPs, is important; the source does not support extending the target to late MLP layers. (LLM, conf=0.88)
Experimental — accuracy not yet measuredROME introduced causal tracing to identify key layers and applied rank-one updates to MLP weights, while MEMIT extended this to mass editing by applying multiple rank-one updates sequentially.
“ROME [6] introduced causal tracing to identify key layers and applied rank-one updates to MLP weights, while MEMIT [7] extended this to mass editing by applying multiple rank-one updates sequentially.”
From Subspace Collisions in Knowledge Editing: Orthogonal Low-Rank Updates for Scalable, Stable Model Edits (2026) · cites this paper as [6]
The ZsRE question-answering task is extensively studied within the model editing literature.
“The ZsRE question-answering task [46] is extensively studied within the model editing literature [18, 19, 31, 15, 11]”
From WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models (2024) · cites this paper as [18]
ROME uses causal analysis to pinpoint knowledge within specific MLP layers and modifies the entire matrix through least squares approximation.
“ROME [18] uses causal analysis to pinpoint knowledge within specific MLP layers and modifies the entire matrix through least squares approximation.”
From WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models (2024) · cites this paper as [18]
ROME identifies factual associations and edits efficiently using MLP-based memories, extended by MEMIT for mass-editing.
“ROME identifies factual associations and edits efficiently using MLP-based memories [18], extended by MEMIT for mass-editing [19].”
From WISE: Rethinking the Knowledge Memory for Lifelong Model Editing of Large Language Models (2024) · cites this paper as [18]
Intervening on specific parts of the Transformer architecture to make counterfactual edits to the model's encoding of factual knowledge updates the model's subsequent inferences accordingly.
“[55] Presents a related study which intervenes on specific parts of the Transformer architecture [56] in order to make counterfactual edits to the model’s encoding of factual knowledge (what if Pierre Curie’s area of work is medicine?). They then observe that the model’s subsequent inferences (e.g. expectations about Pierre Curie’s professional accomplishments) are updated accordingly.”
From Symbols and grounding in large language models (2023) · cites this paper as [55]
These 286 units were predominantly located in middle layers, approximately layers 12-22.
“These 286 units were predominantly located in the middle layers of the model (approximately layers 12-22), consistent with the distribution of units encoding the agent's position (Fig. 3) and with previous studies reporting that higher-level conceptual processing in LLMs tends to emerge in intermediate layers [26-29].”
From From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Task (2025) · cites this paper as [28]
Early approaches ROME and MEMIT demonstrated that factual edits can be performed efficiently using low-rank weight updates applied to specific layers.
“Early approaches, such as ROME [6] and MEMIT [7], demonstrated that factual edits can be performed efficiently using low-rank weight updates applied to specific layers.”
From Subspace Collisions in Knowledge Editing: Orthogonal Low-Rank Updates for Scalable, Stable Model Edits (2026) · cites this paper as [6]
Edits to LLM world knowledge should affect some other bits of knowledge but leave many other facts unaffected.
“these edits should affect some other bits of knowledge (e.g., that the previously current president is now the past president) but leave many other facts unaffected [160].”
From Dissociating language and thought in large language models (2024) · cites this paper as [160]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 28
- Adi, Y., Kermany, E., Belinkov, Y., Lavi, O., and Goldberg, Y. Fine-grained analysis of sentence embeddings using auxiliary prediction tasks. In International Conference on Learning Representations (ICLR) , April 2017.
- Anderson, J. A. A simple neural network generating an interactive memory. Mathematical biosciences , 14(3-4):197-220, 1972.
- Bau, D., Liu, S., Wang, T., Zhu, J.-Y., and Torralba, A. Rewriting a deep generative model. In Proceedings of the European Conference on Computer Vision (ECCV) , 2020.
- Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J. D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C., McCandlish, S., Radford, A., Sutskever, I., and Amodei, D. Language models are few-shot learners. In Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M. F., and Lin, H. (eds.), Advances in Neural Information Processing Systems , volume 33, pp. 1877-1901. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper/ 2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf .
- Dai, D., Dong, L., Hao, Y., Sui, Z., Chang, B., and Wei, F. Knowledge neurons in pretrained transformers. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pp. 8493-8502, 2022.
- De Cao, N., Aziz, W., and Titov, I. Editing factual knowledge in language models. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pp. 6491-6506, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. URL https://aclanthology.org/2021.emnlp-main.522 .
- Elazar, Y., Kassner, N., Ravfogel, S., Ravichander, A., Hovy, E., Sch¨ utze, H., and Goldberg, Y. Measuring and Improving Consistency in Pretrained Language Models. Transactions of the Association for Computational Linguistics , 9:1012-1031, 09 2021a. ISSN 2307-387X. doi: 10.1162/tacl a 00410. URL https://doi.org/10.1162/tacl\_a\_00410 . · DOI
- Elazar, Y., Ravfogel, S., Jacovi, A., and Goldberg, Y. Amnesic probing: Behavioral explanation with amnesic counterfactuals. Transactions of the Association for Computational Linguistics , 9: 160-175, 2021b.
- Feder, A., Oved, N., Shalit, U., and Reichart, R. CausaLM: Causal model explanation through counterfactual language models. Computational Linguistics , 47(2):333-386, 2021.
- Geva, M., Schuster, R., Berant, J., and Levy, O. Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing , pp. 5484-5495, Online and Punta Cana, Dominican Republic, November 2021. Association for Computational Linguistics. URL https://aclanthology.org/2021.emnlp-main.446 .
- Hase, P., Diab, M., Celikyilmaz, A., Li, X., Kozareva, Z., Stoyanov, V., Bansal, M., and Iyer, S. Do language models have beliefs? methods for detecting, updating, and visualizing model beliefs. arXiv preprint arXiv:2111.13654 , 2021. · arXiv
- Hupkes, D., Veldhoen, S., and Zuidema, W. Visualisation and 'diagnostic classifiers' reveal how recurrent and recursive neural networks process hierarchical structure. Journal of Artificial Intelligence Research , 61:907-926, 2018.
- Jiang, Z., Xu, F. F., Araki, J., and Neubig, G. How can we know what language models know? Transactions of the Association for Computational Linguistics , 8:423-438, 2020. doi: 10.1162/ tacl a 00324. URL https://aclanthology.org/2020.tacl-1.28 .
- Kingma, D. P. and Ba, J. Adam: A method for stochastic optimization. In Bengio, Y. and LeCun, Y. (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings , 2015. URL http://arxiv.org/abs/1412. 6980 .
- Kohonen, T. Correlation matrix memories. IEEE transactions on computers , 100(4):353-359, 1972.
- Meng, K., Sen Sharma, A., Andonian, A., Belinkov, Y., and Bau, D. Mass-editing memory in a transformer. arXiv preprint arXiv:2210.07229 , 2022. · arXiv
- Mitchell, E., Lin, C., Bosselut, A., Finn, C., and Manning, C. D. Fast model editing at scale. In International Conference on Learning Representations , 2021.
- Pearl, J. Direct and indirect effects. In Proceedings of the Seventeenth conference on Uncertainty in artificial intelligence , pp. 411-420, 2001.
- Pearl, J. Causality: Models, Reasoning and Inference . Cambridge University Press, USA, 2nd edition, 2009. ISBN 052189560X.
- Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al. Language models are unsupervised multitask learners. OpenAI blog , pp. 9, 2019.
- Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research , 21(140):1-67, 2020.
- Sundararajan, M., Taly, A., and Yan, Q. Axiomatic attribution for deep networks. In International conference on machine learning , pp. 3319-3328. PMLR, 2017.
- Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. Attention is all you need. In Advances in neural information processing systems , pp. 5998-6008, 2017.
- Vig, J., Gehrmann, S., Belinkov, Y., Qian, S., Nevo, D., Sakenis, S., Huang, J., Singer, Y ., and Shieber, S. Causal mediation analysis for interpreting neural NLP: The case of gender bias. arXiv preprint arXiv:2004.12265 , 2020a. · arXiv
- Vig, J., Gehrmann, S., Belinkov, Y., Qian, S., Nevo, D., Singer, Y., and Shieber, S. M. Investigating gender bias in language models using causal mediation analysis. In NeurIPS , 2020b.
- Zhang, Y., Galley, M., Gao, J., Gan, Z., Li, X., Brockett, C., and Dolan, W. B. Generating informative and diverse conversational responses via adversarial information maximization. In NeurIPS , 2018.
- Zhao, S., Pascual, D., Brunner, G., and Wattenhofer, R. Of non-linearity and commutativity in BERT. In 2021 International Joint Conference on Neural Networks (IJCNN) , pp. 1-8. IEEE, 2021.
- Zhu, C., Rawat, A. S., Zaheer, M., Bhojanapalli, S., Li, D., Yu, F., and Kumar, S. Modifying memories in transformer models. arXiv preprint arXiv:2012.00363 , 2020. · arXiv
Cited by
Sorted by citations
Explainability for Large Language Models: A Survey
Zhao, H., Chen, H., Yang, F., Liu, N., Deng, H., Cai, H., Wang, S., Yin, D., & Du, M. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3639372 · Cited by 681 · see citing works
RWKV: Reinventing RNNs for the Transformer Era
Peng, B., Alcaide, E., Anthony, Q., Albalak, A., Arcadinho, S., Biderman, S., Cao, H., Cheng, X., Chung, M., Derczynski, L., et al. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.936 · Cited by 390 · see citing works
Talking about Large Language Models
Shanahan, M. (2024) · Communications of the ACM · doi:10.1145/3624724 · Cited by 381 · see citing works
Dissociating language and thought in large language models
Mahowald, K., Ivanova, A. A., Blank, I. A., Kanwisher, N., Tenenbaum, J. B., & Fedorenko, E. (2024) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2024.01.011 · Cited by 345 · see citing works
Empowering biomedical discovery with AI agents
Gao, S., Fang, A., Huang, Y., Giunchiglia, V., Noori, A., Schwarz, J. R., Ektefaie, Y., Kondic, J., & Zitnik, M. (2024) · Cell · doi:10.1016/j.cell.2024.09.022 · Cited by 315 · see citing works
Erasing Concepts from Diffusion Models
Gandikota, R., Materzyńska, J., Fiotto-Kaufman, J., & Bau, D. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00230 · Cited by 250 · see citing works
Self-citationMedical large language models are vulnerable to data-poisoning attacks
Alber, D. A., Yang, Z., Alyakin, A., Yang, E., Rai, S., Valliani, A. A., Zhang, J., Rosenbaum, G. R., Amend-Thomas, A. K., Kurland, D. B., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03445-1 · Cited by 195 · see citing works
Retrieval-augmented generation for educational application: A systematic survey
Li, Z., Wang, Z., Wang, W., Hung, K., Xie, H., & Wang, F. L. (2025) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2025.100417 · Cited by 151 · see citing works
Domain Specialization as the Key to Make Large Language Models Disruptive: A Comprehensive Survey
Ling, C., Zhao, X.-J., Lu, J., Deng, C.-Y., Zheng, C., Wang, J., Chowdhury, T., Li, Y.-Q., Cui, H., Zhang, X.-C., et al. (2023) · ACM Computing Surveys · doi:10.1145/3764579 · Cited by 321 · see citing works
Ablating Concepts in Text-to-Image Diffusion Models
Kumari, N., Zhang, B., Wang, S.-Y., Shechtman, E., Zhang, R., & Zhu, J.-Y. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.02074 · Cited by 150 · see citing works
Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks
Räuker, T., Ho, A., Casper, S., & Hadfield-Menell, D. (2023) · 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00039 · Cited by 122 · see citing works
Unified Concept Editing in Diffusion Models
Gandikota, R., Orgad, H., Belinkov, Y., Materzyńska, J., & Bau, D. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00503 · Cited by 127 · see citing works
Self-citationA comprehensive survey of large language models and multimodal large language models in medicine
Xiao, H., Zhou, F., Liu, X., Liu, T., Ли, Ж., Liu, X., & Huang, X. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102888 · Cited by 130 · see citing works
(2023) · doi:10.18653/v1/2023.emnlp-main.632 · Cited by 115 · see citing works
Language Model Behavior: A Comprehensive Survey
Chang, T. A., & Bergen, B. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00492 · Cited by 105 · see citing works
Barman, D., Guo, Z., & Conlan, O. (2024) · Machine Learning with Applications · doi:10.1016/j.mlwa.2024.100545 · Cited by 103 · see citing works
Symbols and grounding in large language models
Pavlick, E. (2023) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2022.0041 · Cited by 95 · see citing works
“[ 55] Presents a related study which intervenes on specific parts of the Transformer architecture [56] in order to make counterfactual edits to the model’s encoding of factual knowledge (what if Pierre Curie’s area of work is medicine ?).”
BackgroundLanguage Models as Agent Models
Andreas, J. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.423 · Cited by 92 · see citing works
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Casper, S., Davies, X., Shi, C., Gilbert, T. K., Scheurer, J., Rando, J., Freedman, R., Korbak, T., Lindner, D., Freire, P., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.15217 · Cited by 101 · see citing works
Continual Learning of Large Language Models: A Comprehensive Survey
Shi, H., Xu, Z., Wang, H., Qin, W., Wang, W., Wang, Y., Wang, Z., Ebrahimi, S., & Wang, H. (2025) · ACM Computing Surveys · doi:10.1145/3735633 · Cited by 136 · see citing works
MACE: Mass Concept Erasure in Diffusion Models
Lu, S., Wang, Z., Li, L., Liu, Y., & Adams Wai Kin, K. O. N. G. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00615 · Cited by 90 · see citing works
Black-Box Access is Insufficient for Rigorous AI Audits
Casper, S., Ezell, C., Siegmann, C., Kolt, N., Curtis, T. L., Bucknall, B., Haupt, A. A., Wei, K., Scheurer, J., Hobbhahn, M., et al. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659037 · Cited by 104 · see citing works
Self-citationLanguage in Brains, Minds, and Machines
Tuckute, G., Kanwisher, N., & Fedorenko, E. G. (2024) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-120623-101142 · Cited by 69 · see citing works
The alignment problem from a deep learning perspective
Ngo, R., Chan, L., & Mindermann, S. (2022) · arXiv · doi:10.48550/arxiv.2209.00626 · Cited by 68 · see citing works
Barman, K. G., Wood, N. G., & Pawłowski, P. (2024) · Ethics and Information Technology · doi:10.1007/s10676-024-09778-2 · Cited by 69 · see citing works
Han, S., Wang, M., Zhang, J., Li, D., & Duan, J. (2024) · Electronics · doi:10.3390/electronics13245040 · Cited by 72 · see citing works
LM vs LM: Detecting Factual Errors via Cross Examination
Cohen, R., Hamri, M., Geva, M., & Globerson, A. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.778 · Cited by 60 · see citing works
Inference-Time Intervention: Eliciting Truthful Answers from a Language Model
Li, K., Patel, O., Viégas, F., Pfister, H., & Wattenberg, M. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1797 · Cited by 117 · see citing works
(2023) · doi:10.18653/v1/2023.emnlp-main.971 · Cited by 55 · see citing works
Can We Edit Factual Knowledge by In-Context Learning?
Zheng, C., Li, L., Dong, Q., Fan, Y., Wu, Z., Xu, J., & Chang, B. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.296 · Cited by 54 · see citing works
Dissecting Recall of Factual Associations in Auto-Regressive Language Models
Geva, M., Bastings, J., Filippova, K., & Globerson, A. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.751 · Cited by 53 · see citing works
Zhang, Z., Yao, Y., Zhang, A., Tang, X., Ma, X., He, Z., Wang, Y., Gerstein, M., Wang, R., Liu, G., et al. (2025) · ACM Computing Surveys · doi:10.1145/3719341 · Cited by 51 · see citing works
Evaluating language models for mathematics through interactions
Collins, K. M., Jiang, A. Q., Frieder, S., Wong, L., Zilka, M., Bhatt, U., Lukasiewicz, T., Wu, Y.-H., Tenenbaum, J., Hart, W., et al. (2023) · Proceedings of the National Academy of Sciences of the United States of America · doi:10.1073/pnas.2318124121 · Cited by 130 · see citing works
Towards Lifelong Learning of Large Language Models: A Survey
Zheng, J., Qiu, S., Shi, C., & Ma, Q. (2025) · ACM Computing Surveys · doi:10.1145/3716629 · Cited by 64 · see citing works
A Survey on Symbolic Knowledge Distillation of Large Language Models
Acharya, K., Velasquez, A., & Song, H. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3428519 · Cited by 50 · see citing works
Context-faithful Prompting for Large Language Models
Zhou, W., Zhang, S., Poon, H., & Chen, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.968 · Cited by 70 · see citing works
PMET: Precise Model Editing in a Transformer
Li, X., Li, S., Song, S., Yang, J., Ma, J., & Yu, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29818 · Cited by 45 · see citing works
Semnani, S. J., Yao, V., Zhang, H., & Lam, M. S. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.157 · Cited by 52 · see citing works
Evaluating the Ripple Effects of Knowledge Editing in Language Models
Cohen, R., Biran, E., Yoran, O., Globerson, A., & Geva, M. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00644 · Cited by 72 · see citing works
In-Context Learning Creates Task Vectors
Hendel, R., Geva, M., & Globerson, A. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.624 · Cited by 43 · see citing works
Recent Advances in Interactive Machine Translation With Large Language Models
Wang, Y., Zhang, J., Shi, T., Deng, D., Tian, Y., & Matsumoto, T. (2024) · IEEE Access · doi:10.1109/access.2024.3487352 · Cited by 50 · see citing works
Explaining AI through mechanistic interpretability
Kästner, L., & Crook, B. (2024) · European Journal for Philosophy of Science · doi:10.1007/s13194-024-00614-4 · Cited by 48 · see citing works
Language Generation Models Can Cause Harm: So What Can We Do About It? An Actionable Survey
Kumar, S., Balachandran, V., Njoo, L., Anastasopoulos, A., & Tsvetkov, Y. (2023) · Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.241 · Cited by 38 · see citing works
DEPN: Detecting and Editing Privacy Neurons in Pretrained Language Models
Wu, X., Li, J., Xu, M., Dong, W., Wu, S., Bian, C., & Xiong, D. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.174 · Cited by 38 · see citing works
Machine Unlearning: Taxonomy, Metrics, Applications, Challenges, and Prospects
Li, N., Zhou, C., Gao, Y., Chen, H., Zhang, Z., Kuang, B., & Fu, A. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3530988 · Cited by 53 · see citing works
Inseq: An Interpretability Toolkit for Sequence Generation Models
Sarti, G., Feldhus, N., Sickert, L., & van der Wal, O. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-demo.40 · Cited by 35 · see citing works
A comprehensive overview of Generative AI (GAI): Technologies, applications, and challenges
Zhang, Z., Zhang, J., Zhang, X., & Mai, W. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.129645 · Cited by 37 · see citing works
Large language models for air transportation: A critical review
Liu, Y. (2024) · Journal of the Air Transport Research Society · doi:10.1016/j.jatrs.2024.100024 · Cited by 33 · see citing works
ArtAdapter: Text-to-Image Style Transfer using Multi-Level Style Encoder and Explicit Adaptation
Chen, D.-Y., Tennent, H., & Hsu, C.-W. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00823 · Cited by 33 · see citing works
Kenthapadi, K., Sameki, M., & Taly, A. (2024) · Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining · doi:10.1145/3637528.3671467 · Cited by 36 · see citing works
Future Shock: Generative AI and the International AI Policy and Governance Crisis
Leslie, D., & Perini, A. M. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.88b4cc98 · Cited by 33 · see citing works
Emergent Linear Representations in World Models of Self-Supervised Sequence Models
Nanda, N., Lee, A., & Wattenberg, M. (2023) · Proceedings of the 6th BlackboxNLP Workshop: Analyzing and Interpreting Neural Networks for NLP · doi:10.18653/v1/2023.blackboxnlp-1.2 · Cited by 55 · see citing works
Digital forgetting in large language models: a survey of unlearning methods
Blanco-Justicia, A., Jebreel, N., Manzanares-Salor, B., Sánchez, D., Domingo-Ferrer, J., Collell, G., & Eeik Tan, K. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-024-11078-6 · Cited by 31 · see citing works
The argument for near-term human disempowerment through AI
Dung, L. (2024) · AI & Society · doi:10.1007/s00146-024-01930-2 · Cited by 29 · see citing works
How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advances
Zhang, Z., Fang, M., Chen, L., Namazi‐Rad, M., & Wang, Y.-X. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.516 · Cited by 37 · see citing works
Can LMs Learn New Entities from Descriptions? Challenges in Propagating Injected Knowledge
Onoe, Y., Zhang, M., Padmanabhan, S., Durrett, G., & Choi, E. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.300 · Cited by 32 · see citing works
DisentQA: Disentangling Parametric and Contextual Knowledge with Counterfactual Question Answering
Neeman, E., Aharoni, R., Honovich, O., Choshen, L., Szpektor, I., & Abend, O. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.559 · Cited by 41 · see citing works
MELO: Enhancing Model Editing with Neuron-Indexed Dynamic LoRA
Yu, L., Chen, Q., Zhou, J., & He, L. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i17.29916 · Cited by 35 · see citing works
Prompt Perturbation in Retrieval-Augmented Generation based Large Language Models
Hu, Z., Wang, C., Shu, Y., Paik, H.-Y., & Zhu, L. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671932 · Cited by 27 · see citing works
CatVersion: Concatenating Embeddings for Diffusion-Based Text-to-Image Personalization
Zhao, R., Zhu, M., Dong, S., Cheng, D., Wang, N., & Gao, X. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3531917 · Cited by 23 · see citing works
Red teaming large language models: A comprehensive review and critical analysis
Jabbar, M. S., Al-Azani, S., Alotaibi, A., & Ahmed, M. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104239 · Cited by 25 · see citing works
Survey on the Role of Mechanistic Interpretability in Generative AI
Ranaldi, L. (2025) · Big Data and Cognitive Computing · doi:10.3390/bdcc9080193 · Cited by 22 · see citing works
Diao, S., Xu, T., Xu, R., Wang, J., & Zhang, T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.280 · Cited by 26 · see citing works
DSI++: Updating Transformer Memory with New Documents
Mehta, S., Gupta, J., Tay, Y., Dehghani, M., Tran, V., Rao, J., Najork, M., Strubell, E., & Metzler, D. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.510 · Cited by 21 · see citing works
HippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models
Gutiérrez, B. J., Shu, Y., Gu, Y., Yasunaga, M., & Su, Y. (2024) · arXiv · doi:10.48550/arxiv.2405.14831 · Cited by 152 · see citing works
Plisiecki, H., & Sobieszek, A. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02212-3 · Cited by 23 · see citing works
Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmark
Hoelscher-Obermaier, J., Persson, J., Kran, E., Konstas, I., & Barez, F. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.733 · Cited by 24 · see citing works
Chun, J., & Elkins, K. (2023) · International Journal of Digital Humanities · doi:10.1007/s42803-023-00069-8 · Cited by 19 · see citing works
“The nature of large DNN black box models like LLMs is fundamentally different from previous models and hence requires a different approach to XAI.”
SoK: Prompt Hacking of Large Language Models
Rababah, B., Wu, S. T., Kwiatkowski, M. R., Leung, C. K.-S., & Akçora, C. G. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825103 · Cited by 20 · see citing works
The Life Cycle of Knowledge in Big Language Models: A Survey
Cao, B., Lin, H., Han, X., & Sun, L. (2024) · Machine Intelligence Research · doi:10.1007/s11633-023-1416-x · Cited by 18 · see citing works
“, 2022a), causal-inspired (Meng et al., 2022), attention-based (Clark et al.”
BackgroundMethodInfluentialA Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models
Stolfo, A., Jin, Z., Shridhar, K., Schoelkopf, B., & Sachan, M. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.32 · Cited by 17 · see citing works
Measuring and Modifying Factual Knowledge in Large Language Models
Pezeshkpour, P. (2023) · International Conference on Machine Learning and Applications · doi:10.1109/icmla58977.2023.00122 · Cited by 20 · see citing works
Chain-of-Skills: A Configurable Model for Open-Domain Question Answering
Ma, K., Cheng, H., Zhang, Y., Liu, X., Nyberg, E., & Gao, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.89 · Cited by 22 · see citing works
Cross-Lingual Consistency of Factual Knowledge in Multilingual Language Models
Qi, J., Fernández, R., & Bisazza, A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.658 · Cited by 29 · see citing works
Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIP
Palit, V., Pandey, R., Arora, A., & Liang, P. P. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00307 · Cited by 16 · see citing works
On Early Detection of Hallucinations in Factual Question Answering
Snyder, B., Moisescu, M., & Zafar, M. B. (2024) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3637528.3671796 · Cited by 16 · see citing works
Large language models for causal hypothesis generation in science
Cohrs, K.-H., Díaz, E., Sitokonstantinou, V., Varando, G., & Camps‐Valls, G. (2024) · Machine Learning Science and Technology · doi:10.1088/2632-2153/ada47f · Cited by 16 · see citing works
Memorization in Deep Learning: A Survey
Wei, J., Zhang, Y., Zhang, L. Y., Ding, M., Chen, C., Ong, K., Zhang, J., & Xiang, Y. (2025) · ACM Computing Surveys · doi:10.1145/3769076 · Cited by 26 · see citing works
Grounding neuroscience in behavioral changes using artificial neural networks
Lindsay, G. W. (2023) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2023.102816 · Cited by 16 · see citing works
How Can Deep Neural Networks Inform Theory in Psychological Science?
McGrath, S. W., Russin, J., Pavlick, E., & Feiman, R. (2024) · Current Directions in Psychological Science · doi:10.1177/09637214241268098 · Cited by 15 · see citing works
Editing Factual Knowledge and Explanatory Ability of Medical Large Language Models
Xu, D., Zhang, Z., Zhu, Z., Lin, Z., Liu, Q., Wu, X., Xu, T., Wang, W., Ye, Y., Zhao, X., et al. (2024) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3627673.3679673 · Cited by 20 · see citing works
Stolfo, A., Belinkov, Y., & Sachan, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.435 · Cited by 14 · see citing works
Self-citationThe Geometry of Concepts: Sparse Autoencoder Feature Structure
Li, Y., Michaud, E. J., Baek, D. D., Engels, J., Sun, X., & Tegmark, M. (2025) · Entropy · doi:10.3390/e27040344 · Cited by 14 · see citing works
Physics of Language Models: Part 3.1, Knowledge Storage and ExtractionPreprint
Allen-Zhu, Z., & Li, Y. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5250633 · Cited by 51 · see citing works
Physics of Language Models: Part 3.1, Knowledge Storage and Extraction
Allen-Zhu, Z., & Li, Y. (2023) · arXiv · doi:10.48550/arxiv.2309.14316 · Cited by 14
He, X., Fan, W., Wang, R., Wang, Y., Wang, X. Y., Pan, S., & Wang, X. Y. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.107317 · Cited by 15 · see citing works
Structured knowledge-based causal discovery: Agentic streams of thought
Meier, S., Raut, P. N., Mahr, F., Thielen, N., Franke, J. E., & Risch, F. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104202 · Cited by 13 · see citing works
Watson, E. ‘., Viana, T., & Zhang, S. (2023) · AI · doi:10.3390/ai4010007 · Cited by 13 · see citing works
FactKB: Generalizable Factuality Evaluation using Language Models Enhanced with Factual Knowledge
Feng, S., Balachandran, V., Bai, Y., & Tsvetkov, Y. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.59 · Cited by 21 · see citing works
Chen, C., Merullo, J., & Eickhoff, C. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657841 · Cited by 13 · see citing works
A Survey on Knowledge Editing of Neural Networks
Mazzia, V., Pedrani, A., Caciolai, A., Rottmann, K., & Bernardi, D. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3498935 · Cited by 24 · see citing works
Multimodal Neurons in Pretrained Text-Only Transformers
Schwettmann, S., Chowdhury, N., Klein, S., Bau, D., & Torralba, A. B. (2023) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw60793.2023.00308 · Cited by 13 · see citing works
Self-citationHu, X., Li, D., Hu, B., Zheng, Z., Liu, Z., & Zhang, M. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29784 · Cited by 14 · see citing works
Overcoming Generic Knowledge Loss with Selective Parameter Update
Zhang, W., Janson, P., Aljundi, R., & Elhoseiny, M. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.02270 · Cited by 12 · see citing works
LLMs Meet Library Evolution: Evaluating Deprecated API Usage in LLM-Based Code Completion
Wang, C., Huang, K., Zhang, J., Feng, Y., Zhang, L., Liu, Y., & Peng, X. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00245 · Cited by 17 · see citing works
Modularity in Deep Learning: A Survey
Sun, H., & Guyon, I. (2023) · Lecture notes in networks and systems · doi:10.1007/978-3-031-37963-5_40 · Cited by 11 · see citing works
Chen, Y., Cao, P., Chen, Y., Liu, K., & Zhao, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i16.29735 · Cited by 12 · see citing works
Layered Bias: Interpreting Bias in Pretrained Large Language Models
Prakash, N., & Lee, R. K.-W. (2023) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2023.blackboxnlp-1.22 · Cited by 11 · see citing works
Wu, D., Yang, J., & Wang, K. (2024) · Patterns · doi:10.1016/j.patter.2024.101030 · Cited by 11 · see citing works
A Causal Explainable Guardrails for Large Language Models
Chu, Z., Wang, Y., Li, L., Wang, Z., Qin, Z., & Ren, K. (2024) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3658644.3690217 · Cited by 15 · see citing works
- Authors
- Meng, K., Bau, D., Andonian, A., & Belinkov, Y.
- Affiliations
- Northeastern University (US)
- MIT Computer Science and Artificial Intelligence Laboratory (US)
- Technion – Israel Institute of Technology (IL)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2022
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2202.05262
- OpenAlex
- W4281657280
- Cited by
- 230
- References
- 0
- Citations by year
- 2026: 7 · 2025: 67 · 2024: 75 · 2023: 79 · 2022: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Open Philanthropy Project
- Azrieli Foundation
- Defense Advanced Research Projects Agency (HR0011, FA8750)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- model editing, Rome, causal intervention, neuronal activation, zero-shot relation extraction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
