The DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Anchoring Bias in Large Language Models: An Experimental Study
Open accessAbstract
Large Language Models (LLMs) like GPT-4 and Gemini have significantly advanced artificial intelligence by enabling machines to generate and comprehend human-like text. Despite their impressive capabilities, LLMs are not free of limitations. They have shown various biases. While much research has explored demographic biases, the cognitive biases in LLMs have not been equally studied. This study delves into anchoring bias, a cognitive bias where initial information disproportionately influences judgment. Utilizing an experimental dataset, we examine how anchoring bias manifests in LLMs and verify the effectiveness of various mitigation strategies. Our findings highlight the sensitivity of LLM responses to biased hints. At the same time, our experiments show that, to mitigate anchoring bias, one needs to collect hints from comprehensive angles to prevent the LLMs from being anchored to individual pieces of information, while simple algorithms such as Chain-of-Thought, Thoughts of Principles, Ignoring Anchor Hints, and Reflection are not sufficient.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Experimental work on anchoring bias shows that LLM responses can be influenced by biased initial information [13].”
From Machine Psychometrics: A Mathematical Psychology of Artificial Intelligence (2026) · cites this paper as [13]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bias and Fairness in Large Language Models: A Survey
Gallegos, I. O., Rossi, R. A., Barrow, J., Tanjim, M. M., Kim, S., Dernoncourt, F., Yu, T., Zhang, R., & Ahmed, N. K. (2024) · Computational Linguistics · doi:10.1162/coli_a_00524 · Cited by 1,245
A literature review of the anchoring effect
Furnham, A., & Boo, H. C. (2011) · The Journal of Socio-Economics · doi:10.1016/j.socec.2010.10.008 · Cited by 1,179
Measures of Anchoring in Estimation Tasks
Jacowitz, K. E., & Kahneman, D. J. (1995) · Personality and Social Psychology Bulletin · doi:10.1177/01461672952111004 · Cited by 753
Evaluation of ChatGPT-generated medical responses: A systematic review and meta-analysis
Wei, Q., Yao, Z., Cui, Y., Wei, B., Jin, Z., & Xu, X. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104620 · Cited by 233
Simmons, J. P., LeBoeuf, R. A., & Nelson, L. D. (2010) · Journal of Personality and Social Psychology · doi:10.1037/a0021540 · Cited by 229
Take a Step Back: Evoking Reasoning via Abstraction in Large Language ModelsPreprint
Zheng, H., Mishra, S. R., Chen, X., Cheng, H.-T., Ed, H., Le, Q. V., & Zhou, D. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.06117 · Cited by 226
Large Language Models are not Fair Evaluators
Wang, P., Li, L., Chen, L., Cai, Z., Zhu, D., Lin, B., Cao, Y., Kong, L., Liu, Q., Liu, T., et al. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.511 · Cited by 138
Wilke, A., & Mata, R. (2012) · Elsevier eBooks · doi:10.1016/b978-0-12-375000-6.00094-x · Cited by 77
(Ir)rationality and cognitive biases in large language models
Macmillan-Scott, O., & Musolesi, M. (2024) · Royal Society Open Science · doi:10.1098/rsos.240255 · Cited by 75
Cognitive bias in decision-making with LLMs
Echterhoff, J. M., Liu, Y., Alessa, A., McAuley, J., & He, Z. (2024) · Findings of the Association for Computational Linguistics: EMNLP 2024 · doi:10.18653/v1/2024.findings-emnlp.739 · Cited by 62
Research works printed but not linked: 6
- Shojaei M. "CHAT-GPT and artificial intelligence in Medical Endocrine System and interventions". _Eurasian Journal of Chemical, Medicinal and Petroleum Research_. 3 (1): 197–209, January 2024.
- Yeh KC, Chi JA, Lian DC, Hsieh SK. "Evaluating Interfaced LLM Bias". _The 35th Conference on Computational Linguistics and Speech Processing_, October 20-21, 2023.
- Schmidgall S, Harris C, Essien I, Olshvang D, Rahman T, Kim JW, Ziaei R, Eshraghian J, Abadir P, Chellappa R. Addressing cognitive bias in medical language models. _arXiv_. 2402.08113v3. 20 Feb 2024. Available from: https://arxiv.org/abs/2402.08113v3. · arXiv
- Pantana G, Castello M, Torre I. Examining Cognitive Biases in ChatGPT 3.5 and 4 through Human Evaluation and Linguistic Comparison. In: Proceedings of the 16th Conference of the Association for Machine Translation in the Americas; 2024 Sep 30 - Oct 2; Chicago, USA.
- Li R, Gao Y (2024). "Anchored Answers: Unravelling Positional Bias in GPT-2’s Multiple-Choice Questions". _arXiv_. arXiv: 2405.03205 [https://arxiv.org/abs/2405.03205]. 23 May 2024. · arXiv
- Wei J, Wang X, Schuurmans D, Bosma M, Chi E, Le Q, Zhou D. Chain of Thought Prompting Elicits Reasoning in Large Language Models. In: Proceedings of the 36th Conference on Neural Information Processing Systems; 2022.
Cited by
Sorted by citations
Theodorakopoulos, L., Theodoropoulou, A., & Halkiopoulos, C. (2025) · Electronics · doi:10.3390/electronics14193930 · Cited by 32
Gimmelberg, D., & Ludviga, I. (2025) · International Journal of Financial Studies · doi:10.3390/ijfs13040210 · Cited by 7
A Role-Aware Multi-Agent Framework for Financial Education QA
Zhu, A., & Du, Y. (2025) · ACM International Conference on AI in Finance (ICAIF) · doi:10.1145/3768292.3770345 · Cited by 1
How Does Prompt Anchoring Affect Large Language Model Outputs?
Kim, E. (2026) · Publications · doi:10.3390/publications14030043 · Cited by 1
Machine Psychometrics: A Mathematical Psychology of Artificial Intelligence
Bogdan, A., & de Valois-Franklin, A. (2026) · arXiv · doi:10.48550/arxiv.2605.23952
Closing the Empirical Loop: Autonomous AI Agents Conduct End‐to‐end Research With Human Participants
Wehr, G., Rideaux, R., Fox, A. J., Lightfoot, D. R., Tangen, J. M., Mattingley, J. B., & Ehrhardt, S. E. (2026) · Advanced Science · doi:10.1002/advs.76675
Taheri, A., Alaoui, H. E., Carrington, P., & Bigham, J. P. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791078
Takenami, Y., Huang, Y. J., Murawaki, Y., Takeuchi, K., & Chu, C. (2026) · Journal of Natural Language Processing · doi:10.5715/jnlp.33.473
Metadata
- Authors
- Lou, J., & Sun, Y.
- ORCID
- Lou, Jiaxu
- Venue
- Qeios
- Publisher
- Qeios Ltd
- Published
- 2025-01-28
- ISSN
- 2632-3834 (ISSN-L)
- OpenAlex
- W4406885083
- Cited by
- 5
- References
- 14
- Citations by year
- 2026: 2 · 2025: 3
- Type
- posted content
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Qeios (acceptedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- anchoring effect, large language models, bias mitigation, chain-of-thought prompting, GPT-4, Gemini
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
