The Effect of Scaling, Retrieval Augmentation and Form on the Factual Consistency of Language Models

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works

Survey of Hallucination in Natural Language Generation

Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · see citing works

LLaMA: Open and Efficient Foundation Language Models

Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602 · see citing works

Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Lewis, P., Perez, E., Piktus, A., Petroni, F., Karpukhin, V., Goyal, N., Küttler, H., Lewis, M., Yih, W.-T., Rocktäschel, T., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.11401 · Cited by 3,101 · see citing works

Natural Questions: A benchmark for question answering research

Kwiatkowski, T., Palomaki, J., Redfield, O., Collins, M., Parikh, A., Alberti, C., Epstein, D., Polosukhin, I., Devlin, J., Lee, K., et al. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00276 · Cited by 2,069 · see citing works

Language Models as Knowledge Bases?

Petroni, F., Rocktäschel, T., Riedel, S., Lewis, P., Bakhtin, A., Wu, Y., & Miller, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1250 · Cited by 1,834 · see citing works

Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering

Izacard, G., & Grave, E. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.74 · Cited by 1,109 · see citing works

LaMDA: Language Models for Dialog Applications

Thoppilan, R., De Freitas, D., Hall, J., Shazeer, N., Kulshreshtha, A., Cheng, H.-T., Jin, A., Bos, T., Baker, L., Du, Y., et al. (2022) · arXiv · doi:10.48550/arxiv.2201.08239 · Cited by 778 · see citing works

Wizard of Wikipedia: Knowledge-Powered Conversational agentsPreprint

Dinan, E., Roller, S., Shuster, K., Fan, A., Auli, M., & Weston, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.01241 · Cited by 610 · see citing works

Retrieval Augmentation Reduces Hallucination in Conversation

Shuster, K., Poff, S., Chen, M., Kiela, D., & Weston, J. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.320 · Cited by 575 · see citing works

When not to trust language models: Investigating effectiveness of parametric and non-parametric memories

Mallen, A., Asai, A., Zhong, V., Das, R., Khashabi, D., & Hajishirzi, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.546 · Cited by 340 · see citing works

Locating and Editing Factual Associations in GPT

Meng, K., Bau, D., Andonian, A., & Belinkov, Y. (2022) · arXiv · doi:10.48550/arxiv.2202.05262 · Cited by 230 · see citing works

Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyond

Feder, A., Keith, K. A., Manzoor, E., Pryzant, R., Sridhar, D., Wood-Doughty, Z., Eisenstein, J., Grimmer, J., Reichart, R., Roberts, M. E., et al. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00511 · Cited by 220 · see citing works

Measuring and improving consistency in pretrained language models

Elazar, Y., Kassner, N., Ravfogel, S., Ravichander, A., Hovy, E., Schütze, H., & Goldberg, Y. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00410 · Cited by 220 · see citing works

Atlas: Few-shot Learning with Retrieval Augmented Language Models

Izacard, G., Lewis, P., Lomeli, M., Hosseini, L., Petroni, F., Schick, T., Dwivedi-Yu, J., Joulin, A., Riedel, S., & Grave, E. (2022) · arXiv · doi:10.48550/arxiv.2208.03299 · Cited by 213 · see citing works

Unsupervised Dense Information Retrieval with Contrastive Learning

Izacard, G., Caron, M., Hosseini, L., Riedel, S., Bojanowski, P., Joulin, A., & Grave, E. (2021) · arXiv · doi:10.48550/arxiv.2112.09118 · Cited by 172 · see citing works

Large Language Models Struggle to Learn Long-Tail Knowledge

Kandpal, N., Deng, H., Roberts, A., Wallace, E., & Raffel, C. (2022) · arXiv · doi:10.48550/arxiv.2211.08411 · Cited by 143 · see citing works

Untitled work

(2020) · neural information processing systems · Cited by 142 · see citing works

Entity-Based Knowledge Conflicts in Question Answering

Longpre, S., Perisetla, K., Chen, A., Ramesh, N., DuBois, C., & Singh, S. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.565 · Cited by 101 · see citing works

Simple Entity-Centric Questions Challenge Dense Retrievers

Sciavolino, C., Zhong, Z., Lee, J., & Chen, D. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.496 · Cited by 98 · see citing works

Are Red Roses Red? Evaluating Consistency of Question-Answering Models

Ribeiro, M. T., Guestrin, C., & Singh, S. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1621 · Cited by 93 · see citing works

Generating Training Data with Language Models: Towards Zero-Shot Language UnderstandingPreprint

Yu, M., Huang, J., Zhang, Y., & Han, J. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2202.04538 · Cited by 91 · see citing works

Measure and Improve Robustness in NLP Models: A Survey

Wang, X., Wang, H., & Yang, D. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.339 · Cited by 90 · see citing works

Knowledgeable or Educated Guess? Revisiting Language Models as Knowledge Bases

Cao, B., Lin, H., Han, X., Sun, L., Yan, L., Liao, M., Xue, T., & Xu, J. (2021) · doi:10.18653/v1/2021.acl-long.146 · Cited by 90 · see citing works

When Do You Need Billions of Words of Pretraining Data?

Zhang, Y., Warstadt, A., Li, X., & Bowman, S. R. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.90 · Cited by 85 · see citing works

TIARA: Multi-grained Retrieval for Robust Question Answering over Large Knowledge Base

Shu, Y., Yu, Z., Li, Y., Karlsson, B. F., Ma, T., Qu, Y., & Lin, C.-Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.555 · Cited by 66 · see citing works

Generate, Annotate, and Learn: NLP with Synthetic Text

He, X., Nassar, I., Kiros, J., Haffari, G., & Norouzi, M. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00492 · Cited by 49 · see citing works

Adaptable and Interpretable Neural MemoryOver Symbolic Knowledge

Verga, P., Sun, H., Soares, L. B., & Cohen, W. W. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.288 · Cited by 47 · see citing works

An Efficient Memory-Augmented Transformer for Knowledge-Intensive NLP Tasks

Wu, Y., Zhao, Y. Q., Hu, B., Minervini, P., Stenetorp, P., & Riedel, S. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.346 · Cited by 39 · see citing works

Modeling Content and Context with Deep Relational Learning

Pacheco, M. L., & Goldwasser, D. (2021) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00357 · Cited by 28 · see citing works

Measuring Causal Effects of Data Statistics on Language Model's `Factual' PredictionsPreprint

Elazar, Y., Kassner, N., Ravfogel, S., Feder, A., Ravichander, A., Mosbach, M., Belinkov, Y., Schütze, H., & Goldberg, Y. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2207.14251 · Cited by 15 · see citing works

Transferring Knowledge from Vision to Language: How to Achieve it and how to Measure it?

Norlund, T., Hagström, L., & Johansson, R. (2021) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2021.blackboxnlp-1.10 · Cited by 10 · see citing works

Retrieval-Augmented Generation for AI-Generated Content: A Survey

Zhao, P., Zhang, H., Yu, Q., Wang, Z., Geng, Y., Fu, F., Yang, L., Zhang, W., Jiang, J., & Cui, B. (2026) · Data Science and Engineering · doi:10.1007/s41019-025-00335-5 · Cited by 549 · see citing works

Deciphering the Interplay of Parametric and Non-parametric Memory in Retrieval-augmented Language Models

Farahani, M., & Johansson, R. (2024) · Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2024.emnlp-main.943 · Cited by 5 · see citing works

From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistency

Ohmer, X. I., Bruni, E., & Hupkes, D. (2024) · Computational Linguistics · doi:10.1162/coli_a_00529 · Cited by 11 · see citing works

Enhancing Classification Models With Sophisticated Counterfactual Images

Li, X., Togo, R., Maeda, K., Ogawa, T., & Haseyama, M. (2025) · IEEE Open Journal of Signal Processing · doi:10.1109/ojsp.2025.3530843 · Cited by 1 · see citing works

Eliminating Retrieval Knowledge Conflicts: Cross-Validation Re-ranking with Large Language Models

Wu, Q., Lin, H., Zheng, H., Xie, R., Yang, S., Sun, X., Kang, Z., & Kim, H. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228012

Reframing SEO for the Age of Generative Engines

Murár, P. (2025) · Media & Marketing Identity · doi:10.34135/mmidentity-2025-80

A Survey of Text Deduplication: From Syntactic Matching to Semantic Understanding

Kim, J., & Park, Y. (2026) · IEEE Access · doi:10.1109/access.2026.3658439 · Cited by 2 · see citing works

ConsistencyAI: A Benchmark to Assess LLMs' Factual Consistency When Responding to Different Demographic GroupsPreprint

Banyas, P., Sharma, S., Simmons, A., & Vispute, A. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2510.13852

Latent-Space Intervention for Cross-Lingual Factual Consistency: Consistency Improvements without Accuracy Drops

Ghorbanpour, F., Fierro, C., Fraser, A., & Søgaard, A. (2026) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/2026.findings-emnlp.1206

LLMs as Repositories of Factual Knowledge: Limitations and Solutions

Mousavi, S. M., Alghisi, S., & Riccardi, G. (2025) · IEEE Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslpro.2026.3680709 · Cited by 19 · see citing works

Predicting Question-Answering Performance of Large Language Models through Semantic Consistency

Rabinovich, E., Ackerman, S., Raz, O., Farchi, E., & Anaby-Tavor, A. (2023) · arXiv · doi:10.48550/arxiv.2311.01152 · Cited by 3 · see citing works

Attend to Fragments: How Key Information Affects Large Language Models for Factual Inconsistency Detection

Guo, X., Xie, Z., & Chen, P. H. (2026) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3805712.3809915