The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Overconfidence in Interval Estimates.
Abstract
Judges were asked to make numerical estimates (e.g., "In what year was the first flight of a hot air balloon?"). Judges provided high and low estimates such that they were X% sure that the correct answer lay between them. They exhibited substantial overconfidence: The correct answer fell inside their intervals much less than X% of the time. This contrasts with choices between 2 possible answers to a question, which showed much less overconfidence. The authors show that overconfidence in interval estimates can result from variability in setting interval widths. However, the main cause is that subjective intervals are systematically too narrow given the accuracy of one's information-sometimes only 40% as large as necessary to be well calibrated. The degree of overconfidence varies greatly depending on how intervals are elicited. There are also substantial differences among domains and between male and female judges. The authors discuss the possible psychological mechanisms underlying this pattern of findings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
Statistical principles in experimental design.
Winer, B. J. (1962) · McGraw-Hill Book Company · doi:10.1037/11774-000 · Cited by 26,755
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
Continuous Univariate Distributions.
Omori, Y., Johnson, N. L., Kotz, S., & Balakrishnan, N. (1995) · Journal of the American Statistical Association · doi:10.2307/2291547 · Cited by 9,084
Experimental Design: Procedures for the Behavioral Sciences
Kirk, R. (2013) · doi:10.4135/9781483384733 · Cited by 8,539
Boys will be Boys: Gender, Overconfidence, and Common Stock Investment
Barber, B. M., & Odean, T. (2001) · The Quarterly Journal of Economics · doi:10.1162/003355301556400 · Cited by 5,276
Experimental Design: Procedures for the Behavioral Sciences.
Addelman, S., & Kirk, R. E. (1969) · Journal of the American Statistical Association · doi:10.2307/2283497 · Cited by 5,159
Statistical Principles in Experimental Design
Green, P. E., Winer, B. J., Brown, D. R., & Michels, K. M. (1992) · Journal of Marketing Research · doi:10.2307/3172747 · Cited by 4,333
Journal of Experimental Psychology: Human Learning and Memory
Bourne, L. E. (1977) · Cited by 4,328
Heuristics and Biases: The Psychology of Intuitive Judgment
Bottom, W. P., Gilovich, T., Griffin, D., & Kahneman, D. (2004) · The Academy of Management Review · doi:10.2307/20159081 · Cited by 3,536
Decision analysis and behavioral research
von Winterfeldt, D., & Edwards, W. (1986) · Medical Entomology and Zoology · Cited by 3,042
Calibration of probabilities: the state of the art to 1980
Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1982) · Judgment under Uncertainty · doi:10.1017/cbo9780511809477.023 · Cited by 1,967
The weighing of evidence and the determinants of confidence
Griffin, D., & Tversky, A. (1992) · Cognitive Psychology · doi:10.1016/0010-0285(92)90013-r · Cited by 1,841
Probabilistic mental models: a Brunswikian theory of confidence
Gigerenzer, G., Hoffrage, U., & Kleinbölting, H. (1991) · Psychological Review · doi:10.1037/0033-295x.98.4.506 · Cited by 1,506
Koriat, A., Lichtenstein, S., & Fischhoff, B. (1980) · Journal of Experimental Psychology: Human Learning and Memory · doi:10.1037/0278-7393.6.2.107 · Cited by 1,477
Making Hard Decisions, an Introduction to Decision Analysis
Liu, S. T., & Clemen, R. T. (1992) · Technometrics · doi:10.2307/1270059 · Cited by 1,450
Overconfidence in case-study judgments.
Oskamp, S. (1965) · Journal of Consulting Psychology · doi:10.1037/h0022125 · Cited by 1,060
A progress report on the training of probability assessors
Alpert, M., & Raiffa, H. (1982) · Judgment under Uncertainty · doi:10.1017/cbo9780511809477.022 · Cited by 842
Experimental design: Procedures for the behavioral sciences (3rd ed.).
Kirk, R. E. (1995) · Cited by 787
Overconfidence: It Depends on How, What, and Whom You Ask
Klayman, J., Soll, J. B., González-Vallejo, C., & Barlas, S. (1999) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1999.2847 · Cited by 785
Self-citationVarieties of Confirmation Bias
Klayman, J. (1995) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60315-1 · Cited by 755
Self-citationHighly confident but wrong: Gender differences and similarities in confidence judgments.
Lundeberg, M. A., Fox, P. W., & Punccohar, J. (1994) · Journal of Educational Psychology · doi:10.1037//0022-0663.86.1.114 · Cited by 678
Exceptional Paper—Probability Encoding in Decision Analysis
Spetzler, C. S., & Staël Von Holstein, C.-A. S. (1975) · Management Science · doi:10.1287/mnsc.22.3.340 · Cited by 662
Simultaneous over- and underconfidence: the role of error in judgment processes
Erev, I., Wallsten, T. S., & Budescu, D. V. (1994) · Psychological Review · doi:10.1037/0033-295x.101.3.519 · Cited by 644
Measurement and prediction, clinical and statistical.
Sawyer, J. (1966) · Psychological Bulletin · doi:10.1037/h0023624 · Cited by 594
Beware of samples! A cognitive-ecological sampling approach to judgment biases.
Fiedler, K. (2000) · Psychological Review · doi:10.1037//0033-295x.107.4.659 · Cited by 482
Physicians' use of probabilistic information in a real clinical setting.
Christensen-Szalanski, J. J. J., & Bushyhead, J. B. (1981) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.7.4.928 · Cited by 443
Counterfactual reasoning and accuracy in predicting personal events.
Hoch, S. J. (1985) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037//0278-7393.11.1-4.719 · Cited by 430
Incorporating the Irrelevant: Anchors in Judgments of Belief and Value
Chapman, G. B., & Johnson, E. J. (2002) · Heuristics and Biases · doi:10.1017/cbo9780511808098.008 · Cited by 410
Juslin, P., Winman, A., & Olsson, H. (2000) · Psychological Review · doi:10.1037//0033-295x.107.2.384 · Cited by 394
Reliability of Subjective Probability Forecasts of Precipitation and Temperature
Murphy, A. H., & Winkler, R. L. (1977) · Applied Statistics · doi:10.2307/2346866 · Cited by 386
(1991) · CaltechAUTHORS (California Institute of Technology) · Cited by 340
Juslin, P. (1994) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1994.1013 · Cited by 323
A note on aggregating opinions
Hogarth, R. M. (1978) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(78)90037-5 · Cited by 321
Graininess of judgment under uncertainty: An accuracy-informativeness trade-off.
Yaniv, I., & Foster, D. P. (1995) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.124.4.424 · Cited by 310
Facing uncertainty in the game of bridge: A calibration study
Keren, G. (1987) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(87)90047-1 · Cited by 302
Critical IT (information technology) issues: the next ten years.
Benjamin, R. I., & Blunt, J. (1992) · PubMed · Cited by 257
The False Consensus Effect and Overconfidence: Flaws in Judgment or Flaws in How We Study Judgment?
Dawes, R. M., & Mulford, M. (1996) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1996.0020 · Cited by 239
Soll, J. B. (1996) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1996.0011 · Cited by 229
Self-citationOverconfidence in estimation: Testing the anchoring-and-adjustment hypothesis
Block, R. A., & Harper, D. R. (1991) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(91)90048-x · Cited by 209
Precision and accuracy of judgmental estimation
Yaniv, I., & Foster, D. P. (1997) · Journal of Behavioral Decision Making · doi:10.1002/(sici)1099-0771(199703)10:1<21::aid-bdm243>3.0.co;2-g · Cited by 203
Format dependence in subjective probability calibration.
Juslin, P., Wennerholm, P., & Olsson, H. (1999) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037//0278-7393.25.4.1038 · Cited by 176
Decision making from a cognitive perspective
Busemeyer, J. R., Medin, D. L., & Hastie, R. (1995) · Academic Press eBooks · Cited by 175
Juslin, P., & Olsson, H. (1997) · Psychological Review · doi:10.1037//0033-295x.104.2.344 · Cited by 172
Gender Differences in the Self-Assessment of Accuracy on Cognitive Tasks
Pallier, G. (2003) · Sex Roles · doi:10.1023/a:1022877405718 · Cited by 170
When 90% confidence intervals are 50% certain: on the credibility of credible intervals
Teigen, K. H., & JØrgensen, M. (2005) · Applied Cognitive Psychology · doi:10.1002/acp.1085 · Cited by 167
The calibration of expert judgment: heuristics and biases beyond the laboratory
Koehler, D. J., Brenner, L., & Griffin, D. (2002) · Heuristics and Biases · doi:10.1017/cbo9780511808098.041 · Cited by 146
The Hard–Easy Effect in Subjective Probability Calibration
Suantak, L., Bolger, F., & Ferrell, W. R. (1996) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1996.0074 · Cited by 145
Juslin, P., Olsson, H., & BJÖRKMAN, M. (1997) · Journal of Behavioral Decision Making · doi:10.1002/(sici)1099-0771(199709)10:3<189::aid-bdm258>3.0.co;2-4 · Cited by 143
The calibration of subjective probabilities: Theories and models 1980-1993
McClelland, A. (1994) · UCL Discovery (University College London) · Cited by 142
Juslin, P. (1993) · The European Journal of Cognitive Psychology · doi:10.1080/09541449308406514 · Cited by 127
Intuitive Theories of Information: Beliefs about the Value of Redundancy
Soll, J. B. (1999) · Cognitive Psychology · doi:10.1006/cogp.1998.0699 · Cited by 113
Self-citationAre We Overconfident in the Belief That Probability Forecasters Are Overconfident?
Pfeifer, P. E. (1994) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1994.1034 · Cited by 106
Budescu, D. V., Erev, I., & Wallsten, T. S. (1997) · Journal of Behavioral Decision Making · doi:10.1002/(sici)1099-0771(199709)10:3<157::aid-bdm260>3.0.co;2-# · Cited by 89
Brenner, L. A. (2000) · Psychological Review · doi:10.1037//0033-295x.107.4.943 · Cited by 73
Confidence depends on level of aggregation
Sniezek, J. A., & Buckley, T. (1991) · Journal of Behavioral Decision Making · doi:10.1002/bdm.3960040404 · Cited by 73
(Non)Complementary updating of belief in two hypotheses
McKenzie, C. R. M. (1999) · Memory & Cognition · doi:10.3758/bf03201221 · Cited by 70
A random support model of the calibration of subjective probabilities
Brenner, L. A. (2003) · Organizational Behavior and Human Decision Processes · doi:10.1016/s0749-5978(03)00004-9 · Cited by 56
An examination of the accuracy of judgemental confidence intervals in time series forecasting
O’Connor, M. J., & Lawrence, M. J. (1989) · Journal of Forecasting · doi:10.1002/for.3980080207 · Cited by 54
When wrong predictions provide more support than right ones
McKenzie, C. R. M., & Amin, M. (2002) · Psychonomic Bulletin & Review · doi:10.3758/bf03196341 · Cited by 28
The Effects of Gender and Instructions on Calibration.
Lichtenstein, S., & Fischhoff, B. (1981) · Defense Technical Information Center (DTIC) · Cited by 20
Introduction to this Special Issue on Stochastic and Cognitive Models of Confidence
Budescu, D. V., Erev, I., Wallsten, T. S., & Yates, J. F. (1997) · Journal of Behavioral Decision Making · Cited by 13
Cited by
Gender Differences in Preferences
Croson, R., & Gneezy, U. (2009) · Journal of Economic Literature · doi:10.1257/jel.47.2.448 · Cited by 5,694
The trouble with overconfidence.
Moore, D. A., & Healy, P. J. (2008) · Psychological Review · doi:10.1037/0033-295x.115.2.502 · Cited by 2,358
Bonaccio, S., & Dalal, R. S. (2006) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2006.07.001 · Cited by 1,011
Eliciting Expert Knowledge in Conservation Science
MARTIN, T. G., BURGMAN, M. A., FIDLER, F., KUHNERT, P. M., LOW‐CHOY, S., MCBRIDE, M., & MENGERSEN, K. (2012) · Conservation Biology · doi:10.1111/j.1523-1739.2011.01806.x · Cited by 829
Niederle, M., & Vesterlund, L. (2011) · Annual Review of Economics · doi:10.1146/annurev-economics-111809-125122 · Cited by 805
Metadata
- Authors
- Soll, J. B., & Klayman, J.
- Venue
- Journal of Experimental Psychology: Learning, Memory, and Cognition
- Published
- 2004
- Volume
- 30
- Issue
- 2
- Pages
- 299-314
- ISSN
- 0278-7393 (ISSN-L)
- 1939-1285
- OpenAlex
- W1985907650
- Cited by
- 493
- References
- 62
- FWCI
- 6.42
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.