Physics of Language Models: Part 3.2, Knowledge Manipulation
Open accessThe DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This suggests that simply spelling out factual equivalencies is not enough for less capable models and that the ability to effectively use factual knowledge in context-dependent reasoning tasks may be a key differentiator of more advanced AI systems40,41.”
From When helpfulness backfires: LLMs and the risk of false medical information due to sycophantic behavior (2025) · cites this paper as [40]
Citations
Cites
LoRA: Low-Rank Adaptation of Large Language Models
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., & Chen, W. (2021) · arXiv · doi:10.48550/arxiv.2106.09685 · Cited by 13,453
Self-citationChain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q., & Zhou, D. (2022) · arXiv · doi:10.48550/arxiv.2201.11903 · Cited by 4,263
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083
Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101
Siriwardhana, S., Weerasekera, R., Wen, E., Kaluarachchi, T., Rana, R., & Nanayakkara, S. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00530 · Cited by 337
LIMA: Less Is More for Alignment
Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L., et al. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2400 · Cited by 275
LIMA: Less Is More for Alignment
Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.11206 · Cited by 275
Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies
Geva, M., Khashabi, D., Segal, E., Khot, T., Roth, D., & Berant, J. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00370 · Cited by 242
Physics of Language Models: Part 3.1, Knowledge Storage and ExtractionPreprint
Allen-Zhu, Z., & Li, Y. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5250633 · Cited by 234
Self-citation
Research works printed but not linked: 23
- Zeyuan Allen, 2023, Physics of Language Models: Part 1, Learning Hierarchical Language Structures
- Zeyuan Allen, 2024, Physics of Language Models: Part 3.3, Knowledge Capacity Scaling Laws
- Lukas Berglund, 2023, The Reversal Curse: LLMs trained on "A is B" fail to learn
- Olga Golovneva, 2024, Reverse training to nurse the reversal curse
- Qingyan Guo, 2024, Mitigating reversal curse via semantic-aware permutation training
- Evan Hernandez, 2023, Measuring and manipulating knowledge representations in language models
- Alexandre Albert Q Jiang, 2023, Mistral 7b
- Mojtaba Komeili, 2021, Internet-augmented dialogue generation
- R�mi Lebret, 2016, Generating text from structured data with application to the biography domain
- Shangqing Liu, 2020, Retrieval-augmented generation for code summarization via hybrid gnn
- Yuning Mao, 2020, Generation-augmented retrieval for open-domain question answering
- Reham Omar, 2023, Chatgpt versus traditional question answering for knowledge graphs: Current status and future directions towards knowledge graph chatbots
- 2023, OpenAI. Gpt-4 technical report
- Md Rizwan Parvez, 2021, Retrieval augmented code generation and summarization
- Hao Peng, 2022, Copen: Probing conceptual knowledge in pre-trained language models
- Fabio Petroni, 2019, Language models as knowledge bases? arXiv preprint
- Alec Radford, 2019, Language models are unsupervised multitask learners, OpenAI blog, 1
- Ori Ram, 2023, -context retrieval-augmented language models
- Jianlin Su, 2021, Roformer: Enhanced transformer with rotary position embedding
- Kai Sun, 2023, Head-to-tail: How knowledgeable are large language models (llm)? aka will llms replace knowledge graphs? arXiv preprint
- Hugo Touvron, 2023, Open and efficient foundation language models
- Tian Ye, Physics of Language Models: Part 2.1, Grade-School Math and the Hidden Reasoning Process
- Tian Ye, Physics of Language Models: Part 2.2, How to Learn From Mistakes on Grade-School Math Problems
Cited by
A Comprehensive Study of Knowledge Editing for Large Language ModelsPreprint
Zhang, N., Yao, Y., Tian, B., Wang, P., Deng, S., Wang, M., Xi, Z., Mao, S., Zhang, J., Ni, Y., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2401.01286 · Cited by 407
Exploring the role of large language models in the scientific method: from hypothesis to discovery
Zhang, Y., Khan, S. A., Mahmud, A., Yang, H., Lavin, A., Levin, M. G., Frey, J. G., Dunnmon, J., Evans, J. A., Bundy, A., et al. (2025) · npj Artificial Intelligence · doi:10.1038/s44387-025-00019-5 · Cited by 241
Chen, S., Gao, M., Sasse, K., Hartvigsen, T., Anthony, B., Fan, L., Aerts, H., Gallifant, J., & Bitterman, D. S. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02008-z · Cited by 112
Physics of Language Models: Part 1, Learning Hierarchical Language StructuresPreprint
Allen-Zhu, Z., & Li, Y. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5250639 · Cited by 140
Self-citationOn the Diversity of Synthetic Data and its Impact on Training Large Language ModelsPreprint
Chen, H., Waheed, A., Li, X., Wang, Y., Wang, J., Raj, B., & Abdin, M. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.15226 · Cited by 115
Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks
Jain, S., Kirk, R., Lubana, E. S., Dick, R. P., Tanaka, H., Grefenstette, E., Rocktäschel, T., & Krueger, D. (2023) · International Conference on Learning Representations · doi:10.48550/arxiv.2311.12786 · Cited by 113
RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models
Jin, Z., Cao, P., Wang, C., He, Z., Yuan, H., Liu, K., Chen, Y., Liu, K., & Zhao, J. (2024) · neural information processing systems · doi:10.52202/079017-3117 · Cited by 190
Against The Achilles' Heel: A Survey on Red Teaming for Generative Models
Lin, L., Mu, H., Zhai, Z., Wang, M., Wang, Y., Wang, R., Gao, J., Zhang, Y., Che, W., Baldwin, T., et al. (2025) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.17654 · Cited by 110
Lee, S., Sim, W., Shin, D., Seo, W., Park, J., Lee, S. C., Hwang, S., Kim, S., & Kim, S. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3712701 · Cited by 93
Tell me about yourself: LLMs are aware of their learned behaviors
Betley, J., Bao, X., Soto, M., Sztyber, A., Chua, J., & Evans, O. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2501.11120 · Cited by 110
Scaling Laws for Fact Memorization of Large Language Models
Lu, X., Li, X., Cheng, Q., Ding, K., Huang, X., & Qiu, X. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.658 · Cited by 73
Changing Answer Order Can Decrease MMLU Accuracy
Gupta, V., Pantoja, D., Ross, C., Williams, A., & Ung, M. (2024) · arXiv · doi:10.48550/arxiv.2406.19470
Dynamic Parametric Retrieval Augmented Generation for Test-time Knowledge EnhancementPreprint
Tan, Y., He, S., Liao, H., Zhao, J., & Liu, K. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.23895 · Cited by 52
How Is LLM Reasoning Distracted by Irrelevant Context? An Analysis Using a Controlled Benchmark
Yang, M., Huang, E., Zhang, L., Surdeanu, M., Wang, W. Y., & Pan, L. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.674 · Cited by 65
Liu, Y., Guo, Z., Liang, T., Shareghi, E., Vulić, I., & Collier, N. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.02205 · Cited by 52
Connecting the Dots: LLMs can Infer and Verbalize Latent Structure from Disparate Training Data
Treutlein, J., Choi, D., Betley, J., Marks, S., Anil, C., Grosse, R., & Evans, O. (2024) · neural information processing systems · doi:10.52202/079017-4465 · Cited by 78
An Overview of Large Language Models for Statisticians
Ji, W., Yuan, W., Getzen, E. J., Cho, K., Jordan, M. I., Mei, S., Weston, J., Su, W., Xu, J., & Zhang, L. (2026) · The American Statistician · doi:10.1080/00031305.2026.2657480 · Cited by 51
Lessons from Studying Two-Hop Latent ReasoningPreprint
Balesni, M., Korbak, T., & Evans, O. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.16353 · Cited by 37
Yu, H., Cheng, T., Wang, Y., He, W., Wang, Q., Cheng, Y., Zhang, Y., Feng, R., & Zhang, X. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2501.09213 · Cited by 52
PonderLM: Pretraining Language Models to Ponder in Continuous SpacePreprint
Zeng, B., Song, S., Huang, S., Wang, Y., Li, H., He, Z., Wang, X., Li, Z., & Lin, Z. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.20674 · Cited by 38
Alignment faking in large language models
Greenblatt, R., Denison, C., Wright, B., Roger, F., MacDiarmid, M., Marks, S., Treutlein, J., Belonax, T., Chen, J., Duvenaud, D., et al. (2024) · arXiv · doi:10.48550/arxiv.2412.14093 · Cited by 30
(2025) · doi:10.52202/085713-1689 · Cited by 28
Towards Infinite-Long Prefix in Transformer
Liang, Y., Shi, Z., Song, Z., & Yang, C. (2025) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2025.emnlp-main.563 · Cited by 29
Compositional Capabilities of Autoregressive Transformers: A Study on Synthetic, Interpretable TasksPreprint
Ramesh, R., Lubana, E. S., Khona, M., Dick, R. P., & Tanaka, H. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2311.12997 · Cited by 25
Recent Advances in Attack and Defense Approaches of Large Language ModelsPreprint
Cui, J., Xu, Y., Huang, Z., Zhou, S., Jiao, J., & Zhang, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.03274 · Cited by 30
Roll the dice & look before you leap: Going beyond the creative limits of next-token predictionPreprint
Nagarajan, V., Wu, C. H., Ding, C., & Raghunathan, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.15266 · Cited by 21
Looking Inward: Language Models Can Learn About Themselves by Introspection
Binder, F. J., Chua, J., Korbak, T., Sleight, H., Hughes, J., Long, R., Perez, E., Turpin, M., & Evans, O. (2024) · arXiv · doi:10.48550/arxiv.2410.13787 · Cited by 20
A Percolation Model of Emergence: Analyzing Transformers Trained on a Formal Language
Lubana, E. S., Kawaguchi, K., Dick, R. P., & Tanaka, H. (2024) · International Conference on Learning Representations · doi:10.48550/arxiv.2408.12578 · Cited by 24
Physics of Language Models: Part 4.1, Architecture Design and the Magic of Canon LayersPreprint
Allen-Zhu, Z., & Alfarano, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5240330 · Cited by 37
Self-citationPonderLM-2: Pretraining LLM with Latent Thoughts in Continuous SpacePreprint
Zeng, B., Li, H., Song, S., Wang, Y., Wang, Z., He, Z., Wang, X., & Lin, Z. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.23184 · Cited by 20
Saito, K., Lee, C., Sohn, K., & Ushiku, Y. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.58 · Cited by 21
DiSK: A Diffusion Model for Structured KnowledgePreprint
Kitouni, O., Nolte, N. S., Hensman, J. J., & Mitra, B. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.05253 · Cited by 21
Do LLMs Really Think Step-by-step In Implicit Reasoning?Preprint
Yu, Y. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.15862 · Cited by 23
Structure-aware Domain Knowledge Injection for Large Language Models
Liu, K., Chen, Z., Fu, Z., Zhang, W., Jiang, R., Zhou, F., Chen, Y., Wu, Y., & Ye, J. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1425 · Cited by 20
Knowledge-Instruct: Effective Continual Pre-training from Limited Data using InstructionsPreprint
Ovadia, O., Brief, M., Lemberg, R., & Sheetrit, E. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.05571 · Cited by 17
Decomposing Elements of Problem Solving: What "Math" Does RL Teach?Preprint
Qin, T., Park, C., Kwun, M., Walsman, A., Malach, E., Anand, N., Tanaka, H., & Alvarez-Melis, D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.22756 · Cited by 15
Implicit Reasoning in Transformers is Reasoning through ShortcutsPreprint
Lin, T., Xie, J., Yuan, S., & Yang, D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2503.07604 · Cited by 15
Sun, J., Huang, W., Wu, J., Gu, C., Li, W., Zhang, S., Yan, H., & He, C. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.604 · Cited by 28
Can LLMs Reconcile Knowledge Conflicts in Counterfactual ReasoningPreprint
Yamin, K., Ghosal, G. R., & Wilder, B. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.15732 · Cited by 15
Jiang, Y., Rajendran, G., Ravikumar, P., & Aragam, B. (2024) · neural information processing systems · doi:10.52202/079017-2163 · Cited by 26
Extractive Structures Learned in Pretraining Enable Generalization on Finetuned FactsPreprint
Feng, J., Russell, S., & Steinhardt, J. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.04614 · Cited by 16
Analyzing Memorization in Large Language Models through the Lens of Model Attribution
Menta, T. R., Agrawal, S., & Agarwal, C. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.535 · Cited by 24
How Much Knowledge Can You Pack into a LoRA Adapter without Harming LLM?
Pletenev, S., Marina, M., Moskovskiy, D., Konovalov, V., Braslavski, P., Panchenko, A., & Salnikov, M. (2025) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2025.findings-naacl.243 · Cited by 25
Chen, X., Zhao, L., & Zou, D. (2024) · neural information processing systems · doi:10.52202/079017-3799 · Cited by 25
Hallucination, monofacts, and miscalibration: An empirical investigation
Miao, M. M., & Kearns, M. J. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2533582123 · Cited by 12
Towards a Theoretical Understanding of the 'Reversal Curse' via Training Dynamics
Zhu, H., Huang, B., Zhang, S., Jordan, M. I., Jiao, J., Tian, Y., & Russell, S. (2024) · neural information processing systems · doi:10.52202/079017-2872 · Cited by 44
Let's Think in Two Steps: Mitigating Agreement Bias in MLLMs with Self-Grounded Verification
Andrade, M., Cha, J., Ho, B., Srihari, V., Yadav, K., & Kira, Z. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2507.11662 · Cited by 11
ALPINE: Unveiling The Planning Capability of Autoregressive Learning in Language Models
Wang, S., Shen, Y., Feng, S., Sun, H., Teng, S., & Chen, W. (2024) · neural information processing systems · doi:10.52202/079017-3802 · Cited by 26
Cheng, S., Yin, X., Zhou, R., Li, Y., Wang, X., Pan, L., Wang, W. Y., & Zhong, V. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.01970 · Cited by 9
Self-citationGeneralization or Hallucination? Understanding Out-of-Context Reasoning in Transformers
Huang, Y., Zhu, H., Guo, T., Jiao, J., Sojoudi, S., Jordan, M. I., Russell, S. J., & Song, M. (2025) · neural information processing systems · doi:10.52202/085713-4670 · Cited by 14
Representation Shattering in Transformers: A Synthetic Study with Knowledge EditingPreprint
Nishi, K., Ramesh, R., Okawa, M., Khona, M., Tanaka, H., & Lubana, E. S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.17194 · Cited by 10
SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training
Chu, T., Zhai, Y., Yang, J., Tong, S., Xie, S., Schuurmans, D., Le, Q. V., Levine, S., & Ma, Y. (2025) · arXiv · doi:10.48550/arxiv.2501.17161 · Cited by 7
Lifelong Learning of Large Language Model Based Agents: A Roadmap
Zheng, J., Shi, C., Cai, X., Li, Q., Zhang, D., Li, C., Yu, D., & Ma, Q. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2025.3650546 · Cited by 7
DiscoLoop: Looping Discrete Embeddings and Continuous Hidden States for Multi-hop ReasoningPreprint
Fu, H., Guo, T., Wang, Z., Zhu, H., Lee, J. D., Jiao, J., Russell, S., & Mei, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.00341 · Cited by 8
The pitfalls of next-token prediction
Bachmann, G., & Nagarajan, V. (2024) · arXiv · doi:10.48550/arxiv.2403.06963 · Cited by 6
Rethinking the Reversal Curse of LLMs: a Prescription from Human Knowledge Reversal
Lu, Z., Jin, L., Li, P., Tian, Y., Zhang, L., Wang, S., Xu, G., Tian, C., & Cai, X. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.428 · Cited by 19
Quantifying Cross-Modality Memorization in Vision-Language Models
Wen, Y., Huang, Y., Goldstein, T., Kumar, R., Ghazi, B., & Zhang, C. (2025) · neural information processing systems · doi:10.52202/085713-4292 · Cited by 6
Evaluating Large Language Model with Knowledge Oriented Language Specific Simple Question AnsweringPreprint
Jiang, B., Zhu, R., Wu, J., Jiang, Z., He, Y., Gao, J., Yu, J., Min, R., Wang, Y., Yang, H., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.16591 · Cited by 7
FictionalQA: A Dataset for Studying Memorization and Knowledge AcquisitionPreprint
Kirchenbauer, J., Mongkolsupawan, J., Wen, Y., Goldstein, T., & Ippolito, D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2506.05639 · Cited by 9
MeMo: Memory as a ModelPreprint
Quek, R. W. H., Lee, S., Leong, A. W. L., Verma, A., Prakash, A., Chen, N. F., Low, B. K. H., Rus, D. L., & Solar-Lezama, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.15156 · Cited by 6
Implicit Reasoning in Transformers is Reasoning through Shortcuts
Lin, T., Xie, J., Yuan, S., & Yang, D. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.493 · Cited by 16
Representing Rule-based Chatbots with Transformers
Friedman, D., Panigrahi, A., & Chen, D. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-long.163 · Cited by 7
Who Reasons in the Large Language Models?
Shao, J., & Wu, J. (2025) · neural information processing systems · doi:10.52202/085713-3771 · Cited by 7
How Training Data Shapes the Use of Parametric and In-Context Knowledge in Language Models
Kim, M., Kim, D.-K., Kwon, J., Yang, N., Jung, K., & Cha, M. (2026) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2026.acl-long.1064 · Cited by 6
Large Language Models are Limited in Out-of-Context Knowledge Reasoning
Hu, P., Gao, C., Gao, R., Chen, J., & Huang, S. (2024) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2024.findings-emnlp.178 · Cited by 6
Wu, D., Zhang, R., Zucchelli, E. M., Chen, Y., & Linares, R. (2025) · Scientific Reports · doi:10.1038/s41598-025-91150-5 · Cited by 9
Benchmarking and Understanding Compositional Relational Reasoning of LLMs
Ni, R., Xiao, D., Meng, Q., Li, X., Zheng, S., & Liang, H. (2025) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v39i18.34170 · Cited by 9
Chen, S., Gao, M., Sasse, K., Hartvigsen, T., Anthony, B., Fan, L., Aerts, H. J. W. L., Gallifant, J., & Bitterman, D. S. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.20385 · Cited by 4
Unlocking the Theory Behind Scaling 1-Bit Neural NetworksPreprint
Daliri, M., Song, Z., & Yang, C. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2411.01663 · Cited by 4
Memorizing is Not Enough: Deep Knowledge Injection Through ReasoningPreprint
Xu, R., Ji, Y., Cao, B., Lu, Y., Lin, H., Han, X., He, B., Sun, Y., Li, X., & Sun, L. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.00472 · Cited by 4
Geometric Factual Recall in TransformersPreprint
Ravfogel, S., Yehudai, G., Bruna, J., & Bietti, A. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2605.12426 · Cited by 4
Do Large Language Models Know How Much They Know?
Prato, G., Huang, J., Parthasarathi, P., Sodhani, S., & Chandar, S. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.348 · Cited by 13
Toward Honest Language Models for Deductive ReasoningPreprint
Liu, J., Dhole, K., Wang, Y., Wen, H., Zhang, S., Mao, H., Li, G., Varshney, N., Liu, J., & Pan, X. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.09222 · Cited by 4
Do Large Language Models Know How Much They Know?Preprint
Prato, G., Huang, J., Parthasarathi, P., Sodhani, S., & Chandar, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.19573 · Cited by 3
DeFrame: Debiasing Large Language Models Against Framing EffectsPreprint
Lim, K., Kim, S., & Whang, S. E. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2602.04306 · Cited by 3
Synthetic continued pretraining
Yang, Z., Band, N., Li, S., Candès, E., & Hashimoto, T. (2024) · arXiv · doi:10.48550/arxiv.2409.07431 · Cited by 2
Library-Like Behavior In Language Models is Enhanced by Self-Referencing Causal Cycles
Nwadike, M. S., Iklassov, Z., Aremu, T., Hiraoka, T., Heinzerling, B., Bojković, V., AlQuabeh, H., Takáč, M., & Inui, K. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1232 · Cited by 2
Yadav, I., Schindler, S., Peters, D., & Klinger, R. (2026) · Semantic Web · doi:10.1177/22104968261453132 · Cited by 5
Towards principled knowledge editing methods for large language model reasoning
Yao, Y., Qin, J., Zhang, N., Xu, H., Zhu, Y., Yu, Z., Wang, M., Tang, Y., Gu, J.-C., Deng, S., et al. (2026) · Nature Machine Intelligence · doi:10.1038/s42256-026-01276-y · Cited by 15
Reflections on “Can AI Understand Our Universe?”
Wang, Y. (2024) · International Journal of Modern Physics D · doi:10.1142/s0218271825400103 · Cited by 2
Empirical Capacity Model for Self-Attention Neural NetworksPreprint
Härmä, A., Pietrasik, M., & Wilbik, A. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2407.15425 · Cited by 3
Double Descent as a Lens for Sample Efficiency in Autoregressive vs. Discrete Diffusion ModelsPreprint
Fraij, A., & Dauncey, S. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2509.24974 · Cited by 2
Rote Learning Considered Useful: Generalizing over Memorized Data in LLMs
Wu, Q., Das, S., Amani, M., Ghosh, B., Khan, M. A., Gummadi, K. P., & Zafar, M. B. (2025) · International Conference on Learning Representations · doi:10.48550/arxiv.2507.21914 · Cited by 6
Provable Knowledge Acquisition and Extraction in One-Layer TransformersPreprint
Xu, R., & Chen, K. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.00901 · Cited by 2
A Taxonomy of TranscendencePreprint
Abreu, N., Zhang, E., Malach, E., & Saphra, N. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2508.17669 · Cited by 2
Natural Emergent Misalignment from Reward Hacking in Production RL
MacDiarmid, M., Wright, B., Uesato, J., Benton, J., Kutasov, J., Price, S., Bouscal, N., Bowman, S., Bricken, T., Cloud, A., et al. (2025) · arXiv · doi:10.48550/arxiv.2511.18397 · Cited by 1
Are DeepSeek R1 And Other Reasoning Models More Faithful?
Chua, J., & Evans, O. (2025) · arXiv · doi:10.48550/arxiv.2501.08156 · Cited by 1
Source-Aware Training Enables Knowledge Attribution in Language Models
Khalifa, M., Wadden, D., Strubell, E., Lee, H., Wang, L., Beltagy, I., & Peng, H. (2024) · arXiv · doi:10.48550/arxiv.2404.01019 · Cited by 1
Out-of-Context Reasoning in Large Language Models
Shaki, J., Malfa, E. L., Wooldridge, M., & Kraus, S. (2025) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2025.findings-emnlp.1068 · Cited by 3
Rewriting History: A Recipe for Interventional Analyses to Study Data Effects on Model Behavior
Nadkarni, R., Elazar, Y., Gonen, H., & Smith, N. A. (2026) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl.a.740 · Cited by 2
MULFE: A Multi-Level Benchmark for Free Text Model Editing
Wang, C., Cao, P., Jin, Z., Chen, Y., Zeng, D., Liu, K., & Zhao, J. (2024) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2024.acl-long.732 · Cited by 4
What Makes the Preferred Thinking Direction for LLMs in Multiple-choice Questions?Preprint
Zhang, Y., Bai, R., Gu, Z., Zhang, R., Gu, J., Abbe, E. A., Samy Bengio, A., & Jaitly, N. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.18435 · Cited by 1
CASCADE Your Datasets for Cross-Mode Knowledge Retrieval of Language ModelsPreprint
Zhou, R., & Zhang, Y. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.01450 · Cited by 1
Nuotio, M. (2024) · Aaltodoc (Aalto University) · Cited by 3
Unifying Learning Dynamics and Generalization in Transformers Scaling LawPreprint
Yang, C. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2512.22088 · Cited by 2
Continually self-improving AIPreprint
Yang, Z. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.18073 · Cited by 1
Nemotron-H: A Family of Accurate and Efficient Hybrid Mamba-Transformer Models
NVIDIA, :, Blakeman, A., Basant, A., Khattar, A., Renduchintala, A., Bercovich, A., Ficek, A., Bjorlin, A., Taghibakhshi, A., et al. (2025) · arXiv · doi:10.48550/arxiv.2504.03624
The Logical Implication Steering Method for Conditional Interventions on Transformer GenerationPreprint
Kalajdzievski, D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.03618
On the Consistency of Commonsense in Large Language Models
Li, G., Wang, P., Ke, W., Xu, Z., Liu, J., & Shang, Z. (2025) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2025.findings-acl.834 · Cited by 2
Memorizing is Not Enough: Deep Knowledge Injection Through Reasoning
Xu, R., Ji, Y., Cao, B., Lu, Y., Lin, H., Han, X., He, B., Sun, Y., Li, X., & Sun, L. (2025) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2025.acl-long.1392
Metadata
- Authors
- Allen-Zhu, Z., & Li, Y.
- Affiliations
- Meta (United States) (US)
- Mohamed bin Zayed University of Artificial Intelligence (AE)
- Corresponding author
- Allen-Zhu, Zeyuan
- Venue
- SSRN Electronic Journal
- Publisher
- Elsevier BV
- Published
- 2025
- ISSN
- 1556-5068 (ISSN-L)
- OpenAlex
- W4410357125
- Cited by
- 8
- References
- 11
- Citations by year
- 2026: 5 · 2025: 3
- Type
- posted content
- Language
- en
- Open access
- green
- doi.org
- Locations
- SSRN Electronic Journal (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
- Keywords
- language models, knowledge manipulation, large language models, knowledge representation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.