Physics of Language Models: Part 3.2, Knowledge Manipulation
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This suggests that simply spelling out factual equivalencies is not enough for less capable models and that the ability to effectively use factual knowledge in context-dependent reasoning tasks may be a key differentiator of more advanced AI systems40,41.”
From When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior (2025) · cites this paper as [40]
Citations
Cites
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 13,453
Self-citationChain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101
Siriwardhana, S., Weerasekera, R., Wen, E., Kaluarachchi, T., Rana, R., & Nanayakkara, S. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00530 · Cited by 337
LIMA: Less Is More for Alignment
Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L., et al. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2400 · Cited by 275
LIMA: Less Is More for Alignment
Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.11206 · Cited by 275
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies
Geva, M., Khashabi, D., Segal, E., Khot, T., Roth, D., & Berant, J. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00370 · Cited by 242
Physics of Language Models: Part 3.1, Knowledge Storage and ExtractionPreprint
Allen-Zhu, Z., & Li, Y. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5250633 · Cited by 234
Self-citationRecent Advances in Retrieval-Augmented Text Generation
Cai, D., Wang, Y., Liu, L., & Shi, S. (2022) · Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval · doi:10.1145/3477495.3532682 · Cited by 76
GPT-NeoX-20B: An Open-Source Autoregressive Language Model
Black, S., Biderman, S., Hallahan, E., Anthony, Q., Gao, L., Golding, L., He, H., Leahy, C., McDonell, K., Phang, J., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.06745 · Cited by 67
What Does My QA Model Know? Devising Controlled Probes Using Expert Knowledge
Richardson, K., & Sabharwal, A. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00331 · Cited by 48
A Semantics-aware Transformer Model of Relation Linking for Knowledge Base Question Answering
Naseem, T., Ravishankar, S., Mihindukulasooriya, N., Abdelaziz, I., Lee, Y.-S., Kapanipathi, P., Roukos, S., Gliozzo, A., & Gray, A. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers) · doi:10.18653/v1/2021.acl-short.34 · Cited by 29
Meet in the Middle: A New Pre-training Paradigm
Nguyen, A. K., Karampatziakis, N., & Chen, W. (2023) · neural information processing systems · doi:10.52202/075280-0225 · Cited by 28
Active Retrieval Augmented Generation
Jiang, Z., Xu, F. F., Gao, L., Sun, Z., Liu, Q., Dwivedi-Yu, J., Yang, Y., Callan, J., & Neubig, G. (2023) · arXiv · doi:10.48550/arxiv.2305.06983 · Cited by 18
Better & Faster Large Language Models via Multi-token Prediction
Gloeckle, F., Idrissi, B. Y., Rozière, B., Lopez-Paz, D., & Synnaeve, G. (2024) · arXiv · doi:10.48550/arxiv.2404.19737 · Cited by 15
Physics of Language Models: Part 3.1, Knowledge Storage and Extraction
Allen-Zhu, Z., & Li, Y. (2023) · arXiv · doi:10.48550/arxiv.2309.14316 · Cited by 14
Self-citationLet's Think Dot by Dot: Hidden Computation in Transformer Language Models
Pfau, J., Merrill, W., & Bowman, S. R. (2024) · arXiv · doi:10.48550/arxiv.2404.15758 · Cited by 6
Research works printed but not linked: 23
- Zeyuan Allen, 2023, Physics of Language Models: Part 1, Learning Hierarchical Language Structures
- Zeyuan Allen, 2024, Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws
- Lukas Berglund, 2023, The Reversal Curse: LLMs trained on "A is B" fail to learn
- Olga Golovneva, 2024, Reverse training to nurse the reversal curse
- Qingyan Guo, 2024, Mitigating reversal curse via semantic-aware permutation training
- Evan Hernandez, 2023, Measuring and manipulating knowledge representations in language models
- Alexandre Albert Q Jiang, 2023, Mistral 7b
- Mojtaba Komeili, 2021, Internet-augmented dialogue generation
- R�mi Lebret, 2016, Generating text from structured data with application to the biography domain
- Shangqing Liu, 2020, Retrieval-augmented generation for code summarization via hybrid gnn
- Yuning Mao, 2020, Generation-augmented retrieval for open-domain question answering
- Reham Omar, 2023, Chatgpt versus traditional question answering for knowledge graphs: Current status and future directions towards knowledge graph chatbots
- 2023, OpenAI. Gpt-4 technical report
- Md Rizwan Parvez, 2021, Retrieval augmented code generation and summarization
- Hao Peng, 2022, Copen: Probing conceptual knowledge in pre-trained language models
- Fabio Petroni, 2019, Language models as knowledge bases? arXiv preprint
- Alec Radford, 2019, Language models are unsupervised multitask learners, OpenAI blog, 1
- Ori Ram, 2023, -context retrieval-augmented language models
- Jianlin Su, 2021, Roformer: Enhanced transformer with rotary position embedding
- Kai Sun, 2023, Head-to-tail: How knowledgeable are large language models (llm)? aka will llms replace knowledge graphs? arXiv preprint
- Hugo Touvron, 2023, Open and efficient foundation language models
- Tian Ye, Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process
- Tian Ye, Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problems
Cited by
A Comprehensive Study of Knowledge Editing for Large Language ModelsPreprint
Zhang, N., Yao, Y., Tian, B., Wang, P., Deng, S., Wang, M., Xi, Z., Mao, S., Zhang, J., Ni, Y., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.01286 · Cited by 407
Exploring the role of large language models in the scientific method: from hypothesis to discovery
Zhang, Y., Khan, S. A., Mahmud, A., Yang, H., Lavin, A., Levin, M. G., Frey, J. G., Dunnmon, J., Evans, J. A., Bundy, A., et al. (2025) · npj Artificial Intelligence · doi:10.1038/s44387-025-00019-5 · Cited by 241
Chen, S., Gao, M., Sasse, K., Hartvigsen, T., Anthony, B., Fan, L., Aerts, H., Gallifant, J., & Bitterman, D. S. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02008-z · Cited by 112
Physics of Language Models: Part 1, Learning Hierarchical Language StructuresPreprint
Allen-Zhu, Z., & Li, Y. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5250639 · Cited by 140
Self-citationOn the Diversity of Synthetic Data and its Impact on Training Large Language ModelsPreprint
Chen, H., Waheed, A., Li, X., Wang, Y., Wang, J., Raj, B., & Abdin, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.15226 · Cited by 115
Metadata
- Authors
- Allen-Zhu, Z., & Li, Y.
- Affiliations
- Meta (United States) (US)
- Mohamed bin Zayed University of Artificial Intelligence (AE)
- Corresponding author
- Allen-Zhu, Zeyuan
- Venue
- SSRN Electronic Journal
- Publisher
- Elsevier BV
- Published
- 2025
- ISSN
- 1556-5068 (ISSN-L)
- OpenAlex
- W4410357125
- Cited by
- 8
- References
- 11
- Citations by year
- 2026: 5 · 2025: 3
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- SSRN Electronic Journal (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
- Keywords
- language models, knowledge manipulation, large language models, knowledge representation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.