The calibration of expert judgment: heuristics and biases beyond the laboratory

Koehler, Derek J.; Brenner, Lyle; Griffin, Dale

doi:10.1017/cbo9780511808098.041

Cited by 146Decision-Making and Behavioral EconomicsForecasting Techniques and ApplicationsClinical Reasoning and Diagnostic Skills

The calibration of expert judgment: heuristics and biases beyond the laboratory

Koehler, Derek J., Brenner, Lyle, Griffin, Dale·2002·Heuristics and Biases

DOI 10.1017/cbo9780511808098.041

Open this paper in CoChat — free account

Abstract

The study of how people use subjective probabilities is a remarkably modern concern, and was largely motivated by the increasing use of expert judgment during and after World War II (Cooke, 1991). Experts are often asked to quantify the likelihood of events such as a stock market collapse, a nuclear plant accident, or a presidential election (Ayton, 1992; Baron, 1998; Hammond, 1996). For applications such as these, it is essential to know how the probabilities experts attach to various outcomes match the relative frequencies of those outcomes; that is, whether experts are properly "calibrated." Despite this, relatively few studies have evaluated how well descriptive theories of probabilistic reasoning capture the behavior of experts in their natural environment. In this chapter, we examine the calibration of expert probabilistic predictions "in the wild" and assess how well the heuristics and biases perspective on judgment under uncertainty can account for the findings. We then review alternate theories of calibration in light of the expert data.

Integrity signals

Registry

Crossref · 10.1017/cbo9780511808098.041

Cites

96 of 96 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Miscalibration of probabilities can impact detection.
“Higher-order cognitive tendencies, such as the ones involved in misjudgments of probabilities (miscalibration [23 , 24] ), can also impact detection.”
From Enhancing human-AI collaboration: The case of colonoscopy (2024) · cites this paper as [24]

The theory of planned behavior
Ajzen, I. (1991) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(91)90020-t · Cited by 90,587 · see citing works
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524 · see citing works
Judgment under Uncertainty: Heuristics and Biases: Biases in judgments reveal some heuristics of thinking under uncertainty.
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810 · see citing works
The Framing of Decisions and the Psychology of Choice
Tversky, A., & Kahneman, D. (1981) · Science · doi:10.1126/science.7455683 · Cited by 17,604 · see citing works
Advances in prospect theory: Cumulative representation of uncertainty
Tversky, A., & Kahneman, D. (1992) · Journal of Risk and Uncertainty · doi:10.1007/bf00122574 · Cited by 15,467 · see citing works
Classification And Regression Trees
Breiman, L., Friedman, J. H., Olshen, R. A., & Stone, C. J. (2017) · doi:10.1201/9781315139470 · Cited by 11,417 · see citing works
Availability: A heuristic for judging frequency and probability
Tversky, A., & Kahneman, D. (1973) · Cognitive Psychology · doi:10.1016/0010-0285(73)90033-9 · Cited by 9,984 · see citing works
The case for motivated reasoning.
Kunda, Z. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.108.3.480 · Cited by 9,022 · see citing works
The Elaboration Likelihood Model of Persuasion
Petty, R. E., & Cacioppo, J. T. (1986) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60214-2 · Cited by 8,822 · see citing works
Bad Is Stronger Than Good
Baumeister, R. F., Bratslavsky, E., Finkenauer, C., & Vohs, K. D. (2001) · Review of General Psychology · doi:10.1037/1089-2680.5.4.323 · Cited by 7,678 · see citing works
Motivational processes affecting learning.
Dweck, C. S. (1986) · American Psychologist · doi:10.1037/0003-066x.41.10.1040 · Cited by 6,971 · see citing works
Optimism, coping, and health: Assessment and implications of generalized outcome expectancies.
Scheier, M. F., & Carver, C. S. (1985) · Health Psychology · doi:10.1037/0278-6133.4.3.219 · Cited by 6,690 · see citing works
Implicit social cognition: Attitudes, self-esteem, and stereotypes
Greenwald, A. G., & Banaji, M. R. (1995) · Psychological Review · doi:10.1037/0033-295x.102.1.4 · Cited by 6,682 · see citing works
Choices, values, and frames.
Kahneman, D., & Tversky, A. (1984) · American Psychologist · doi:10.1037/0003-066x.39.4.341 · Cited by 6,544 · see citing works
On the psychology of prediction.
Kahneman, D., & Tversky, A. (1973) · Psychological Review · doi:10.1037/h0034747 · Cited by 6,202 · see citing works
Controlled and automatic human information processing: I. Detection, search, and attention.
Schneider, W., & Shiffrin, R. M. (1977) · Psychological Review · doi:10.1037/0033-295x.84.1.1 · Cited by 5,902 · see citing works
Status quo bias in decision making
Samuelson, W., & Zeckhauser, R. (1988) · Journal of Risk and Uncertainty · doi:10.1007/bf00055564 · Cited by 5,897 · see citing works
Implementation intentions: Strong effects of simple plans.
Gollwitzer, P. M. (1999) · American Psychologist · doi:10.1037/0003-066x.54.7.493 · Cited by 5,478 · see citing works
Self-Perception Theory
Bem, D. J. (1972) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60024-6 · Cited by 5,241 · see citing works
Why People Obey the Law
Tyler, T. R. (2006) · doi:10.1515/9781400828609 · Cited by 5,156 · see citing works
Anomalies: The Endowment Effect, Loss Aversion, and Status Quo Bias
Kahneman, D., Knetsch, J. L., & Thaler, R. H. (1991) · Journal of Economic Perspectives · doi:10.1257/jep.5.1.193 · Cited by 5,019 · see citing works
Mood, misattribution, and judgments of well-being: Informative and directive functions of affective states.
Schwarz, N., & Clore, G. L. (1983) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.45.3.513 · Cited by 4,952 · see citing works
The principles of psychology, Vol I.
James, W. (1890) · doi:10.1037/10538-000 · Cited by 4,514 · see citing works
Perception in chess
Chase, W. G., & Simon, H. A. (1973) · Cognitive Psychology · doi:10.1016/0010-0285(73)90004-2 · Cited by 4,479 · see citing works
Negativity Bias, Negativity Dominance, and Contagion
Rozin, P., & Royzman, E. B. (2001) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0504_2 · Cited by 4,342 · see citing works
Human Cognitive Abilities: A Survey of Factor-Analytic Studies
Carroll, J. B. (1993) · doi:10.1017/cbo9780511571312 · Cited by 4,121 · see citing works
On the Self-Regulation of Behavior
Carver, C. S., & Scheier, M. F. (1998) · Cambridge University Press eBooks · doi:10.1017/cbo9781139174794 · Cited by 4,110 · see citing works
Regret Theory: An Alternative Theory of Rational Choice Under Uncertainty
Loomes, G., & Sugden, R. (1982) · The Economic Journal · doi:10.2307/2232669 · Cited by 4,042 · see citing works
Connectionism and cognitive architecture: A critical analysis
Fodor, J. A., & Pylyshyn, Z. W. (1988) · Cognition · doi:10.1016/0010-0277(88)90031-5 · Cited by 4,022 · see citing works
How the mind works
PINKER, S. (1999) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1999.tb08538.x · Cited by 3,738 · see citing works
The unbearable automaticity of being
Bargh, J. A., & Chartrand, T. L. (1999) · American Psychologist · doi:10.1037/0003-066x.54.7.462 · Cited by 3,699 · see citing works
The Psychology of Self-Affirmation: Sustaining the Integrity of the Self
Steele, C. M. (1988) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60229-4 · Cited by 3,697 · see citing works
Habit and intention in everyday life: The multiple processes by which past behavior predicts future behavior.
Ouellette, J. A., & Wood, W. (1998) · Psychological Bulletin · doi:10.1037/0033-2909.124.1.54 · Cited by 3,480 · see citing works
Mood and judgment: The affect infusion model (AIM).
Forgas, J. P. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.117.1.39 · Cited by 3,470 · see citing works
From acts to dispositions the attribution process in person perception
Jones, E. E., & Davis, K. E. (1965) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60107-0 · Cited by 3,424 · see citing works
Pretense and representation: The origins of "theory of mind."
Leslie, A. M. (1987) · Psychological Review · doi:10.1037/0033-295x.94.4.412 · Cited by 3,379 · see citing works
Choices, Values, and Frames
(2000) · doi:10.1017/cbo9780511803475 · Cited by 2,987 · see citing works
Self-perception: An alternative interpretation of cognitive dissonance phenomena.
Bem, D. J. (1967) · Psychological Review · doi:10.1037/h0024835 · Cited by 2,959 · see citing works
Integration of the cognitive and the psychodynamic unconscious
Epstein, S. (1994) · American Psychologist · doi:10.1037/0003-066x.49.8.709 · Cited by 2,883 · see citing works
Self-reports: How the questions shape the answers.
Schwarz, N. (1999) · American Psychologist · doi:10.1037/0003-066x.54.2.93 · Cited by 2,846 · see citing works
Downward comparison principles in social psychology
Wills, T. A. (1981) · Psychological Bulletin · doi:10.1037/0033-2909.90.2.245 · Cited by 2,705 · see citing works
Out of Control: Visceral Influences on Behavior
Loewenstein, G. (1996) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1996.0028 · Cited by 2,610 · see citing works
The psychology of sunk cost
Arkes, H. R., & Blumer, C. (1985) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(85)90049-4 · Cited by 2,573 · see citing works
Motivated closing of the mind: “Seizing” and “freezing”
Kruglanski, A. W., & Webster, D. M. (1996) · Psychological Review · doi:10.1037/0033-295x.103.2.263 · Cited by 2,379 · see citing works
“Clinical versus Actuarial Judgment.”
Dawes, R. M., Faust, D., & Meehl, P. E. (1989) · Science · doi:10.1126/science.2648573 · Cited by 2,298 · see citing works
A neuropsychological theory of positive affect and its influence on cognition.
Ashby, F. G., Isen, A. M., & Turken, A. U. (1999) · Psychological Review · doi:10.1037/0033-295x.106.3.529 · Cited by 2,270 · see citing works
Confirmation, disconfirmation, and information in hypothesis testing
Klayman, J., & Ha, Y.-W. (1987) · Psychological Review · doi:10.1037/0033-295x.94.2.211 · Cited by 2,094 · see citing works
Exposure and affect: Overview and meta-analysis of research, 1968–1987.
Bornstein, R. F. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.106.2.265 · Cited by 1,970 · see citing works
Adaptive strategy selection in decision making.
Payne, J. W., Bettman, J. R., & Johnson, E. J. (1988) · Journal of Experimental Psychology: Learning, Memory, and Cognition · doi:10.1037/0278-7393.14.3.534 · Cited by 1,955 · see citing works
The correspondence bias.
Gilbert, D. T., & Malone, P. S. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.117.1.21 · Cited by 1,897 · see citing works
Concepts, Kinds, and Cognitive Development
Keil, F. C. (1989) · doi:10.7551/mitpress/2065.001.0001 · Cited by 1,886 · see citing works
Variability in automatic activation as an unobtrusive measure of racial attitudes: A bona fide pipeline?
Fazio, R. H., Jackson, J. R., Dunton, B. C., & Williams, C. J. (1995) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.69.6.1013 · Cited by 1,793 · see citing works
The attribution of attitudes
Jones, E. E., & Harris, V. A. (1967) · Journal of Experimental Social Psychology · doi:10.1016/0022-1031(67)90034-0 · Cited by 1,783 · see citing works
Attitudes vs. actions
LaPiere, R. T. (1934) · Social Forces · doi:10.2307/2570339 · Cited by 1,720 · see citing works
Just world research and the attribution process: Looking back and ahead.
Lerner, M. J., & Miller, D. T. (1978) · Psychological Bulletin · doi:10.1037/0033-2909.85.5.1030 · Cited by 1,687 · see citing works
Illness cognition: Using common sense to understand treatment adherence and affect cognition interactions
Leventhal, H., Diefenbach, M., & Leventhal, E. A. (1992) · Cognitive Therapy and Research · doi:10.1007/bf01173486 · Cited by 1,573 · see citing works
Three-year-olds’ difficulty with false belief: The case for a conceptual deficit
Perner, J., Leekam, S. R., & Wimmer, H. (1987) · British Journal of Developmental Psychology · doi:10.1111/j.2044-835x.1987.tb01048.x · Cited by 1,550 · see citing works
Pragmatic reasoning schemas
Cheng, P. W., & Holyoak, K. J. (1985) · Cognitive Psychology · doi:10.1016/0010-0285(85)90014-3 · Cited by 1,463 · see citing works
Thinking too much: Introspection can reduce the quality of preferences and decisions.
Wilson, T. D., & Schooler, J. W. (1991) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.60.2.181 · Cited by 1,417 · see citing works
Ease of retrieval as information: another look at the availability heuristic
Schwarz, N., Bless, H., Strack, F., Klumpp, G., Rittenauer-Schatka, H., & Simons, A. (1991) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.61.2.195 · Cited by 1,412 · see citing works
Implementation intentions and effective goal pursuit.
Gollwitzer, P. M., & Brandstätter, V. (1997) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.73.1.186 · Cited by 1,343 · see citing works
Individuals with sociopathic behavior caused by frontal damage fail to respond autonomically to social stimuli
DAMASIO, A. (1990) · Behavioural Brain Research · doi:10.1016/0166-4328(90)90144-4 · Cited by 1,275 · see citing works
Duration neglect in retrospective evaluations of affective episodes.
Fredrickson, B. L., & Kahneman, D. (1993) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.65.1.45 · Cited by 1,228 · see citing works
At last, my research article on procrastination
Lay, C. H. (1986) · Journal of Research in Personality · doi:10.1016/0092-6566(86)90127-3 · Cited by 1,175 · see citing works
Support theory: a nonextensional representation of subjective probability
Tversky, A., & Koehler, D. J. (1994) · Psychological Review · doi:10.1037/0033-295x.101.4.547 · Cited by 1,167 · see citing works
Self-citation
Mental contamination and mental correction: Unwanted influences on judgments and evaluations.
Wilson, T. D., & Brekke, N. (1994) · Psychological Bulletin · doi:10.1037/0033-2909.116.1.117 · Cited by 1,157 · see citing works
Patients' memories of painful medical treatments: real-time and retrospective evaluations of two minimally invasive procedures
Redelmeier, D. A., & Kahneman, D. (1996) · Pain · doi:10.1016/0304-3959(96)02994-6 · Cited by 1,078 · see citing works
Generality of learned helplessness in man.
Hiroto, D. S., & Seligman, M. E. (1975) · Journal of Personality and Social Psychology · doi:10.1037/h0076270 · Cited by 896 · see citing works
They saw a game: A case study
Hastorf, A. H., & Cantril, H. (1954) · The Journal of Abnormal and Social Psychology · doi:10.1037/h0057880 · Cited by 882 · see citing works
Reluctance to vaccinate: Omission bias and ambiguity
Ritov, I., & Baron, J. (1990) · Journal of Behavioral Decision Making · doi:10.1002/bdm.3960030404 · Cited by 854 · see citing works
Ambiguity and self-evaluation: The role of idiosyncratic trait definitions in self-serving assessments of ability.
Dunning, D., Meyerowitz, J. A., & Holzberg, A. D. (1989) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.57.6.1082 · Cited by 758 · see citing works
Fault trees: sensitivity of estimated failure probabilities to problem representation
Fischhoff, B., Slovic, P., & Lichtenstein, S. (1978) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037/0096-1523.4.2.330 · Cited by 708 · see citing works
Unit weighting schemes for decision making
Einhorn, H. J., & Hogarth, R. M. (1975) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(75)90044-6 · Cited by 673 · see citing works
Dissociation of processes in belief: Source recollection, statement familiarity, and the illusion of truth.
Begg, I. M., Anas, A., & Farinacci, S. (1992) · Journal of Experimental Psychology: General · doi:10.1037//0096-3445.121.4.446 · Cited by 650 · see citing works
The dilution effect: Nondiagnostic information weakens the implications of diagnostic information
Nisbett, R. E., Zukier, H., & Lemley, R. E. (1981) · Cognitive Psychology · doi:10.1016/0010-0285(81)90010-4 · Cited by 605 · see citing works
Evidential impact of base rates
Tversky, A., & Kahneman, D. (1982) · Judgment under Uncertainty · doi:10.1017/cbo9780511809477.011 · Cited by 591 · see citing works
Events and subjective well-being: Only recent events matter.
Suh, E., Diener, E., & Fujita, F. (1996) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.70.5.1091 · Cited by 558 · see citing works
Overcoming the Inevitable Anchoring Effect: Considering the Opposite Compensates for Selective Accessibility
Mussweiler, T., Strack, F., & Pfeiffer, T. (2000) · Personality and Social Psychology Bulletin · doi:10.1177/01461672002611010 · Cited by 556 · see citing works
“How Probable is Probable? A Numerical Translation of Verbal Probability Expressions.”
Beyth‐Marom, R. (1982) · Journal of Forecasting · doi:10.1002/for.3980010305 · Cited by 527 · see citing works
Accountability and the Perseverance of First Impressions
Tetlock, P. E. (1983) · Social Psychology Quarterly · doi:10.2307/3033716 · Cited by 513 · see citing works
Managerial myopia: Self-serving biases in organizational planning.
Larwood, L., & Whittaker, W. (1977) · Journal of Applied Psychology · doi:10.1037/0021-9010.62.2.194 · Cited by 490 · see citing works
Do frequency representations eliminate conjunction effects? An exercise in adversarial collaboration
Mellers, B., Hertwig, R., & Kahneman, D. (2001) · Psychological Science · doi:10.1111/1467-9280.00350 · Cited by 448 · see citing works
Causes and effects of my disturbing little book
Meehl, P. E. (1986) · Journal of Personality Assessment · doi:10.1207/s15327752jpa5003_6 · Cited by 410 · see citing works
Subjective Construal, Social Inference, and Human Misunderstanding
Griffin, D. W., & Ross, L. (1991) · Advances in Experimental Social Psychology · doi:10.1016/s0065-2601(08)60333-0 · Cited by 404 · see citing works
Self-citation
The curse of expertise: The effects of expertise and debiasing methods on prediction of novice performance.
Hinds, P. J. (1999) · Journal of Experimental Psychology: Applied · doi:10.1037/1076-898x.5.2.205 · Cited by 397 · see citing works
Repetition and Memory
Hintzman, D. L. (1976) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60464-8 · Cited by 388 · see citing works
Reliability of Subjective Probability Forecasts of Precipitation and Temperature
Murphy, A. H., & Winkler, R. L. (1977) · Applied Statistics · doi:10.2307/2346866 · Cited by 386 · see citing works
Craving for Alcohol, Loss of Control, and Relapse: A Cognitive-Behavioral Analysis
Marlatt, G. A. (1978) · Alcoholism · doi:10.1007/978-1-4613-2874-2_11 · Cited by 362 · see citing works
Effects of procedural and outcome accountability on judgment quality
Siegel-Jacobs, K., & Yates, J. F. (1996) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1996.0001 · Cited by 327 · see citing works
When shall we use our heads instead of the formula?
Meehl, P. E. (1957) · Journal of Counseling Psychology · doi:10.1037/h0047554 · Cited by 310 · see citing works
Stated expectations as functions of probability and desirability of outcomes
IRWIN, F. W. (1953) · Journal of Personality · doi:10.1111/j.1467-6494.1953.tb01775.x · Cited by 281 · see citing works
Cross-Cultural Variations in Probability Judgment Accuracy: Beyond General Knowledge Overconfidence?
Yates, J. F., Lee, J.-W., Shinotsuka, H., Patalano, A. L., & Sieck, W. R. (1998) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1998.2771 · Cited by 220 · see citing works
HUMAN ABILITIES
Sternberg, R. J., & Kaufman, J. C. (1998) · Annual Review of Psychology · doi:10.1146/annurev.psych.49.1.479 · Cited by 213 · see citing works
Cultural differences in probabilistic thinking
Wright, G. N., Phillips, L. D., Whalley, P. C., Choo, G. T., Ng, K.-O., Tan, I., & Wisudha, A. (1978) · Journal of Cross-Cultural Psychology · doi:10.1177/002202217893002 · Cited by 168 · see citing works
The influence of motor processes on attitudes toward novel versus familiar semantic stimuli
Priester, J. R., Cacioppo, J. T., & Petty, R. E. (1996) · Personality and Social Psychology Bulletin · doi:10.1177/0146167296225002 · Cited by 144 · see citing works
Verification of forecasts expressed in terms of probability
BRIER, G. W. (1950) · Monthly Weather Review · doi:10.1175/1520-0493(1950)078%3c0001:vofeit%3e2.0.co;2

Flagged first, then by citations

Sorted by citations

Two-stage dynamic signal detection: A theory of choice, decision time, and confidence.
Pleskac, T. J., & Busemeyer, J. R. (2010) · Psychological Review · doi:10.1037/a0019737 · Cited by 722 · see citing works
“Accuracy of Forecasts in Strategic Intelligence.”
Mandel, D. R., & Barnes, A. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1406138111 · Cited by 101 · see citing works
Confidence Calibration in a Multiyear Geopolitical Forecasting Competition
Moore, D. A., Swift, S. A., Minster, A., Mellers, B., Ungar, L., Tetlock, P., Yang, H. H. J., & Tenney, E. R. (2017) · Management Science · doi:10.1287/mnsc.2016.2525 · Cited by 74 · see citing works
Known Unknowns: A Critical Determinant of Confidence and Calibration
Walters, D. J., Fernbach, P. M., Fox, C. R., & Sloman, S. A. (2017) · Management Science · doi:10.1287/mnsc.2016.2580 · Cited by 62 · see citing works
“Intelligence Analysis and Judgmental Calibration.”
RIEBER, S. (2004) · International Journal of Intelligence and CounterIntelligence · doi:10.1080/08850600490273431 · Cited by 32 · see citing works
Enhancing human-AI collaboration: The case of colonoscopy
Introzzi, L., Zonca, J., Cabitza, F., Cherubini, P., & Reverberi, C. (2024) · Digestive and Liver Disease · doi:10.1016/j.dld.2023.10.018 · Cited by 28 · see citing works

Authors: Koehler, D. J., Brenner, L., & Griffin, D.
Affiliations: University of Waterloo (CA)
University of Florida (US)
University of British Columbia (CA)
ORCID: Koehler, Derek J.
Venue: Heuristics and Biases
Publisher: Cambridge University Press
Published: 2002-07-08
Pages: 686-715
DOI: 10.1017/cbo9780511808098.041
OpenAlex: W2165863233
Cited by: 146
References: 96
Citations by year: 2026: 2 · 2025: 6 · 2024: 3 · 2023: 5 · 2022: 6 · 2021: 7 · 2020: 6 · 2019: 6 · 2018: 6 · 2017: 5 · 2016: 6 · 2015: 7 · 2014: 9 · 2013: 4 · 2012: 9
FWCI: 5.22
Type: book chapter
Language: en
Open access: closed
Licence: www.cambridge.org/core/terms (unspecified, 3620 d)
Locations: Cambridge University Press eBooks (publishedVersion)
Topics: Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
Social Sciences › Decision Sciences › Management Science and Operations Research › Forecasting Techniques and Applications
Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
Keywords: expert opinion, calibration, subjective probability, heuristics and biases, probabilistic reasoning, judgment under uncertainty, prediction accuracy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

The calibration of expert judgment: heuristics and biases beyond the laboratory

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Record

About these checks