Confidence in judgment: persistence of the illusion of validity
Abstract
An accumulating body of research on clinical judgment, decision making, and probability estimation has documented a substantial lack of ability of both ex-perts and nonexperts. However, evidence shows that people have great confi-dence in their fallible judgment. This article examines how this contradiction can be resolved and, in so doing, discusses the relationship between learning and experience. The basic tasks that are considered involve judgments made for the purpose of choosing between actions. At some later time, outcome feedback is used for evaluating the accuracy of judgment. The manner in which judgments of the contingency between predictions and outcomes are made is discussed and is related to the difficulty people have in searching for discommoning informa-tion to test hypotheses. A model for learning and maintaining confidence in one's own judgment is developed that includes the effects of experience and both the frequency and importance of positive and negative feedback. Everyone complains of his memory and no one com-plains of his judgment. (La Rochefoucauld, 1959, p. 49) Although the study and cataloguing of judgmental fallability has had a long history in psychology (see, e.g., Guilford, 1954, chap. 12; Johnson, 1972), an accumulating body of recent research on clinical judgment, decision making, and probability estimation has docu-mented a substantial lack of ability across both individuals and situations (Slovic, Fisch-hoff, & Lichtenstein, 1977; Slovic &…
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Overconfidence may cause fixation and hinder learning and correction, particularly when physicians do not receive systematic feedback.
“Overconfidence may cause fixation [38] and hinder learning and correction, particularly when physicians do not receive systematic feedback on highly confident yet incorrect judgments [39] .”
From Enhancing human-AI collaboration: The case of colonoscopy (2024) · cites this paper as [39]
Cites
Availability: A heuristic for judging frequency and probability
Tversky, A., & Kahneman, D. (1973) · Cognitive Psychology · doi:10.1016/0010-0285(73)90033-9 · Cited by 9,984 · cited 5× · see citing works
“Such an encoding process will inevitably lead to an "availability" bias (Tversky & Kahneman, 1973) in recall and corresponding "illusory correlation" (Chapman & Chapman, 1969).”
BackgroundInfluentialOn the psychology of prediction.
Kahneman, D., & Tversky, A. (1973) · Psychological Review · doi:10.1037/h0034747 · Cited by 6,202 · see citing works
Rosenthal, R., & Jacobson, L. (1968) · The Urban Review · doi:10.1007/bf02322211 · Cited by 3,878 · see citing works
Human problem solving: The state of the theory in 1970.
Simon, H. A., & Newell, A. (1971) · American Psychologist · doi:10.1037/h0030806 · Cited by 635 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Confirmation Bias: A Ubiquitous Phenomenon in Many Guises
Nickerson, R. S. (1998) · Review of General Psychology · doi:10.1037/1089-2680.2.2.175 · Cited by 7,267 · see citing works
Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment.
Tversky, A., & Kahneman, D. (1983) · Psychological Review · doi:10.1037/0033-295x.90.4.293 · Cited by 3,991 · see citing works
Conditions for intuitive expertise: A failure to disagree.
Kahneman, D., & Klein, G. (2009) · American Psychologist · doi:10.1037/a0016755 · Cited by 2,443 · see citing works
“Clinical versus Actuarial Judgment.”
Dawes, R. M., Faust, D., & Meehl, P. E. (1989) · Science · doi:10.1126/science.2648573 · Cited by 2,298 · see citing works
Confirmation, disconfirmation, and information in hypothesis testing
Klayman, J., & Ha, Y.-W. (1987) · Psychological Review · doi:10.1037/0033-295x.94.2.211 · Cited by 2,094 · see citing works
Consumer Decision Making in Online Shopping Environments: The Effects of Interactive Decision Aids
Häubl, G., & Trifts, V. (2000) · Marketing Science · doi:10.1287/mksc.19.1.4.15178 · Cited by 1,731 · see citing works
Clinical versus mechanical prediction: a meta-analysis
Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000) · Psychological Assessment · doi:10.1037/1040-3590.12.1.19 · Cited by 1,640 · see citing works
Support theory: a nonextensional representation of subjective probability
Tversky, A., & Koehler, D. J. (1994) · Psychological Review · doi:10.1037/0033-295x.101.4.547 · Cited by 1,167 · see citing works
Mental contamination and mental correction: Unwanted influences on judgments and evaluations.
Wilson, T. D., & Brekke, N. (1994) · Psychological Bulletin · doi:10.1037/0033-2909.116.1.117 · Cited by 1,157 · see citing works
Dunning, D., Meyerowitz, J. A., & Holzberg, A. D. (1989) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.57.6.1082 · Cited by 758 · see citing works
Evaluating the Impact of DSS, Cognitive Effort, and Incentives on Strategy Selection
Todd, P., & Benbasat, I. (1999) · Information Systems Research · doi:10.1287/isre.10.4.356 · Cited by 312 · see citing works
Brill, T. M., Munoz, L., & Miller, R. J. (2019) · Journal of Marketing Management · doi:10.1080/0267257x.2019.1687571 · Cited by 291 · see citing works
The generality of the relation between familiarity and judged validity
Arkes, H. R., Hackett, C., & Boehm, L. (1989) · Journal of Behavioral Decision Making · doi:10.1002/bdm.3960020203 · Cited by 229 · see citing works
The effects of asymmetry among advisors on the aggregation of their opinions
Budescu, D. V., Rantilla, A. K., Yu, H.-T., & Karelitz, T. M. (2003) · Organizational Behavior and Human Decision Processes · doi:10.1016/s0749-5978(02)00516-2 · Cited by 117 · see citing works
Confident or Competent? How to Avoid Superstitious Learning in Alliance Portfolios
Heimeriks, K. H. (2010) · Long Range Planning · doi:10.1016/j.lrp.2009.10.004 · Cited by 70 · see citing works
Enhancing human-AI collaboration: The case of colonoscopy
Introzzi, L., Zonca, J., Cabitza, F., Cherubini, P., & Reverberi, C. (2024) · Digestive and Liver Disease · doi:10.1016/j.dld.2023.10.018 · Cited by 28 · see citing works
The influence of prior belief on scientific thinking
Evans, J. S. B. T. (2002) · The Cognitive Basis of Science · doi:10.1017/cbo9780511613517.011 · Cited by 22 · see citing works
- Authors
- Einhorn, H. J., & Hogarth, R. M.
- Affiliations
- University of Chicago (US)
- ORCID
- Hogarth, Robin M.
- Venue
- Psychological Review
- Published
- 1978-09
- Volume
- 85
- Issue
- 5
- Pages
- 395-416
- ISSN
- 0033-295X (ISSN-L)
- 1939-1471
- OpenAlex
- W2030364704
- Cited by
- 1,209
- References
- 4
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 12 · 2023: 8 · 2022: 19 · 2021: 14 · 2020: 16 · 2019: 15 · 2018: 14 · 2017: 25 · 2016: 21 · 2015: 32 · 2014: 18 · 2013: 18 · 2012: 26
- FWCI
- 40.70
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Review (publishedVersion)
- Topics
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Memory Processes and Influences
- Keywords
- confidence in judgments, clinical judgment, decision-making, probability estimation, outcome feedback, experiential learning, contingency judgment, disconfirming information, hypothesis testing, positive and negative feedback, predictive validity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review