Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Cited by 33

The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.

Human bias in AI models? Anchoring effects and mitigation strategies in large language models

Nguyen, Jeremy K. · 2024 · Journal of Behavioral and Experimental Finance

Abstract

This study builds on the seminal work of Tversky and Kahneman (1974), exploring the presence and extent of anchoring bias in forecasts generated by four Large Language Models (LLMs): GPT-4, Claude 2, Gemini Pro and GPT-3.5. In contrast to recent findings of advanced reasoning capabilities in LLMs, our randomised controlled trials reveal the presence of anchoring bias across all models: forecasts are significantly influenced by prior mention of high or low values. We examine two mitigation prompting strategies, ‘Chain of Thought’ and ‘ignore previous’, finding limited and varying degrees of effectiveness. Our results extend the anchoring bias research in finance beyond human decision-making to encompass LLMs, highlighting the importance of deliberate and informed prompting in AI forecasting in both ad hoc LLM use and in crafting few-shot examples.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.jbef.2024.100971

Cites

16 of 16 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 24

  1. Akter, 2021, Algorithmic bias in data-driven innovation in the age of AI, Int. J. Inf. Manag., 60, 10.1016/j.ijinfomgt.2021.102387 · DOI · Paper page
  2. Ashrafimoghari, V., Gürkan, N., Suchow, J.W., 2024. Evaluating Large Language Models on the GMAT: Implications for the Future of Business Education. https://doi.org/10.48550/arXiv.2401.02985. · arXiv · doi:10.2139/ssrn.4681307
  3. Belot, 2023, Australian academics apologise for false AI-generated allegations against big four consultancy firms, Guardian
  4. Cen, 2013, The role of anchoring bias in the equity market: evidence from analysts’ earnings forecasts and stock returns, J. Financ. Quant. Anal., 48, 47, 10.1017/S0022109012000609 · DOI · Paper page
  5. Chu, 2023, A survey of chain of thought reasoning: advances, Front. Future
  6. Coda-Forno, 2023, Inducing Anxiety Large Lang. Models Increases Explor. bias
  7. Dowling, 2023, ChatGPT for (Finance) research: the bananarama conjecture, Financ. Res. Lett., 53, 10.1016/j.frl.2023.103662 · DOI · Paper page
  8. Goodell, 2021, Artificial intelligence and machine learning in finance: identifying foundations, themes, and research clusters from bibliometric analysis, J. Behav. Exp. Financ., 32, 10.1016/j.jbef.2021.100577 · DOI · Paper page
  9. Gray, 2023, Measurement and mitigation of bias in artificial intelligence: a narrative literature review for regulatory science, Clin. Pharmacol. Ther.
  10. Gurdgiev, 2020, Herding and anchoring in cryptocurrency markets: investor reaction to fear and uncertainty, J. Behav. Exp. Financ., 25, 10.1016/j.jbef.2020.100271 · DOI · Paper page
  11. Gurnee, 2023, Lang. Models Represent Space Time
  12. Harvey, 2007, Use of heuristics: Insights from forecasting research, Think. Reason., 13, 5, 10.1080/13546780600872502 · DOI
  13. Hasan, 2022, Can Artificial Intelligence (AI) manage behavioural biases among financial planners?, J. Glob. Inf. Manag. JGIM, 31, 1
  14. Hendy, 2021, Unsticking credit card repayments from the minimum: advice, anchors and financial incentives, J. Behav. Exp. Financ., 30, 10.1016/j.jbef.2021.100505 · DOI
  15. Jacowitz, 1995, Measures of anchoring in estimation tasks, Pers. Soc. Psychol. Bull., 21, 1161, 10.1177/01461672952111004 · DOI · Paper page
  16. Königstorfer, 2020, Applications of Artificial Intelligence in commercial banks – a research agenda for behavioral finance, J. Behav. Exp. Financ., 27, 10.1016/j.jbef.2020.100352 · DOI · Paper page
  17. Koralus, P., Wang-Maścianica, V., 2023. Humans in Humans Out: On GPT Converging Toward Common Sense in both Success and Failure [WWW Document]. arXiv.org. URL 〈https://arxiv.org/abs/2303.17276v1〉 (accessed 2.7.24). · arXiv
  18. Li, 2023, Can ChatGPT reduce human financial analysts’ optimistic biases?, Econ. Polit. Stud., 0, 1
  19. Løhre, 2016, Numerical anchors and their strong effects on software development effort estimates, J. Syst. Softw., 116, 49, 10.1016/j.jss.2015.03.015 · DOI
  20. Meub, 2016, Can anchoring explain biased forecasts? Experimental evidence, J. Behav. Exp. Financ., 12, 1, 10.1016/j.jbef.2016.08.001 · DOI
  21. Milmo, 2023, Two US lawyers fined for submitting fake court citations from ChatGPT, Guardian
  22. Opedal, A., Stolfo, A., Shirakami, H., Jiao, Y., Cotterell, R., Schölkopf, B., Saparov, A., Sachan, M., 2024. Do Language Models Exhibit the Same Cognitive Biases in Problem Solving as Human Learners? https://doi.org/10.48550/arXiv.2401.18070. · DOI
  23. Shanmuganathan, 2020, Behavioural finance in an era of artificial intelligence: longitudinal case study of robo-advisors in investment decisions, J. Behav. Exp. Financ., 27, 10.1016/j.jbef.2020.100297 · DOI · Paper page
  24. Talboy, A.N., Fuller, E., 2023. Challenging the appearance of machine intelligence: Cognitive bias in LLMs. https://doi.org/10.48550/arXiv.2304.01358. · DOI

Cited by

See all 33

Metadata

Authors
Nguyen, J. K.
Venue
Journal of Behavioral and Experimental Finance
Publisher
Elsevier BV
Published
2024-09
Volume
43
Pages
100971
ISSN
2214-6350
Cited by
33
References
16
Language
en
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.