Adaptive Chameleon or Stubborn Sloth: Revealing the Behavior of Large Language Models in Knowledge Conflicts
Open accessAbstract
By providing external information to large language models (LLMs), tool augmentation (including retrieval augmentation) has emerged as a promising solution for addressing the limitations of LLMs' static parametric memory. However, how receptive are LLMs to such external evidence, especially when the evidence conflicts with their parametric memory? We present the first comprehensive and controlled investigation into the behavior of LLMs when encountering knowledge conflicts. We propose a systematic framework to elicit high-quality parametric memory from LLMs and construct the corresponding counter-memory, which enables us to conduct a series of controlled experiments. Our investigation reveals seemingly contradicting behaviors of LLMs. On the one hand, different from prior wisdom, we find that LLMs can be highly receptive to external evidence even when that conflicts with their parametric memory, given that the external evidence is coherent and convincing. On the other hand, LLMs also demonstrate a strong confirmation bias when the external evidence contains some information that is consistent with their parametric memory, despite being presented with conflicting evidence at the same time. These results pose important implications that are worth careful consideration for the further development and deployment of tool- and retrieval-augmented LLMs. Resources are available at https://github.com/OSU-NLP-Group/LLM-Knowledge-Conflict.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
Sorted by citations
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3703155 · Cited by 2,142 · see citing works
Xiao, T., & Xu, P. (2024) · Energy and Buildings · doi:10.1016/j.enbuild.2024.114691 · Cited by 80 · see citing works
Large Language Model Instruction Following: A Survey of Progresses and Challenges
Lou, R., Zhang, K., & Yin, W. (2023) · Computational Linguistics · doi:10.1162/coli_a_00523 · Cited by 74 · see citing works
Self-citationCan ChatGPT Defend its Belief in Truth? Evaluating LLM Reasoning via Debate
Wang, B., Yue, X., & Sun, H. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.795 · Cited by 71 · see citing works
Automatic Evaluation of Attribution by Large Language Models
Yue, X., Wang, B., Chen, Z., Zhang, K., Su, Y., & Sun, H. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.307 · Cited by 62 · see citing works
Self-citationHippoRAG: Neurobiologically Inspired Long-Term Memory for Large Language Models
Gutiérrez, B. J., Shu, Y., Gu, Y., Yasunaga, M., & Su, Y. (2024) · arXiv · doi:10.48550/arxiv.2405.14831 · Cited by 152 · see citing works
“Due in part to its ease of use and the limitations of other techniques such as model editing [35], retrieval-augmented generation (RAG) has become the de facto solution for long-term memory in LLMs, allowing users to present new knowledge to a static model [28, 33, 50].”
Self-citationGranite Guardian: Comprehensive LLM Safeguarding
Padhi, I., Nagireddy, M., Cornacchia, G., Chaudhury, S., Pedapati, T., Dognin, P., Murugesan, K., Miehling, E., Cooper, M. S., Fraser, K., et al. (2025) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2025.naacl-industry.49 · Cited by 3 · see citing works
Neutralizing Backdoors through Information Conflicts for Large Language ModelsPreprint
Chen, Z., Sun, Y., Gong, X., Gao, J., & Lam, K. (2024) · Qeios · doi:10.32388/fv951p · Cited by 3 · see citing works
From RAG to Memory: Non-Parametric Continual Learning for Large Language Models
Gutiérrez, B. J., Shu, Y., Qi, W., Zhou, S., & Su, Y. (2025) · arXiv · doi:10.48550/arxiv.2502.14802 · Cited by 2 · see citing works
“Due to their simplicity and robustness (Zhong et al., 2023; Xie et al., 2024), RAG has quickly become the de facto continual learning solution for production LLM systems.”
Self-citationMeasuring Adversarial Datasets
Bai, Y., Huang, R., Viswanathan, V., Kuo, T.-S., & Wu, T. (2023) · doi:10.18653/v1/2023.artofsafety-1.4 · Cited by 2 · see citing works
Do as We Do, Not as You Think: the Conformity of Large Language Models
Weng, Z., Chen, G., & Wang, W. (2025) · arXiv · doi:10.48550/arxiv.2501.13381 · Cited by 1 · see citing works
Deceptive Grounding: Entity Attribution Failure in Clinical Retrieval-Augmented Generation
Caruzzo, C., Yoo, D., & Kim, T. S. (2026) · arXiv · doi:10.48550/arxiv.2607.09349
“A productive line of work studies model behavior when parametric knowledge contradicts retrieved context [Longpre et al., 2021, Mallen et al., 2023, Shi et al., 2023, Xie et al., 2024].”
SealQA: Raising the Bar for Reasoning in Search-Augmented Language Models
Pham, T., Nguyen, N., Zunjare, P., Chen, W., Tseng, Y.-M., & Vu, T. (2025) · arXiv · doi:10.48550/arxiv.2506.01062
“Still, Xie et al. [2024b] show that models remain highly receptive to contradictory external evidence when it is coherent and convincing.”
Integrity in the Age of AI: Detecting Synthetic Text Across Platforms and Purposes
Shah, R., & Mahato, M. (2025) · International Conference on Cognitive Computing in Engineering, Communications, Sciences and Biomedical Health Informatics (IC3ECSBHI) · doi:10.1109/ic3ecsbhi63591.2025.10990518
- Authors
- Xie, J., Zhang, K., Chen, J., Lou, R., & Su, Y.
- Affiliations
- Fudan University (CN)
- Pennsylvania State University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2305.13300
- OpenAlex
- W4377865953
- Cited by
- 9
- References
- 0
- Citations by year
- 2025: 2 · 2024: 4 · 2023: 3
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Cisco Systems
- Fudan University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- large language models, knowledge conflicts, parametric memory, retrieval-augmented generation, external evidence, confirmation bias, counter-memory
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
