The DOI matches its Crossref record.
State of the Art—Encoding Subjective Probabilities: A Psychological and Psychometric Review
Abstract
In order to review the empirical literature on subjective probability encoding from a psychological and psychometric perspective, it is first suggested that the usual encoding techniques can be regarded as instances of the general methods used to scale psychological variables. It is then shown that well-established concepts and theories from measurement and psychometric theory can provide a general framework for evaluating and assessing subjective probability encoding. The actual review of the literature distinguishes between studies conducted with nonexperts and with experts. In the former class, findings related to the reliability, internal consistency, and external validity of the judgments are critically discussed. The latter class reviews work relevant to some of these characteristics separately for several fields of expertise. In die final section of the paper the results from these two classes of studies are summarized and related to a view of vague subjective probabilities. Problems deserving additional attention and research are identified.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment.
Tversky, A., & Kahneman, D. (1983) · Psychological Review · doi:10.1037/0033-295x.90.4.293 · Cited by 3,991
Heuristics and Biases: The Psychology of Intuitive Judgment
Bottom, W. P., Gilovich, T., Griffin, D., & Kahneman, D. (2004) · The Academy of Management Review · doi:10.2307/20159081 · Cited by 3,536
Representativeness revisited: Attribute substitution in intuitive judgment.
Kahneman, D., & Frederick, S. (2002) · Heuristics and Biases · doi:10.1017/cbo9780511808098.004 · Cited by 3,476
Norm theory: Comparing reality to its alternatives.
Kahneman, D., & Miller, D. T. (1986) · Psychological Review · doi:10.1037/0033-295x.93.2.136 · Cited by 3,024
Slovic, P., Finucane, M., Peters, E., & MacGregor, D. G. (2002) · Heuristics and Biases · doi:10.1017/cbo9780511808098.025 · Cited by 2,349
“Clinical versus Actuarial Judgment.”
Dawes, R. M., Faust, D., & Meehl, P. E. (1989) · Science · doi:10.1126/science.2648573 · Cited by 2,298
Heuristics and Biases: The Psychology of Intuitive Judgment
(2002) · Cambridge University Press eBooks · doi:10.1017/cbo9780511808098 · Cited by 2,192
The weighing of evidence and the determinants of confidence
Griffin, D., & Tversky, A. (1992) · Cognitive Psychology · doi:10.1016/0010-0285(92)90013-r · Cited by 1,841
The Hot Hand in Basketball: On the Misperception of Random Sequences
Gilovich, T., Vallone, R., & Tversky, A. (1985) · Cognitive Psychology · doi:10.1016/0010-0285(85)90010-6 · Cited by 1,649
The Role of Imagery in Information Processing: Review and Extensions
MacInnis, D. J., & Price, L. L. (1987) · Journal of Consumer Research · doi:10.1086/209082 · Cited by 1,276
Support theory: a nonextensional representation of subjective probability
Tversky, A., & Koehler, D. J. (1994) · Psychological Review · doi:10.1037/0033-295x.101.4.547 · Cited by 1,167
How we know--and sometimes misjudge--what others know: Imputing one's own knowledge to others.
Nickerson, R. S. (1999) · Psychological Bulletin · doi:10.1037//0033-2909.125.6.737 · Cited by 836
The use of statistical heuristics in everyday inductive reasoning.
Nisbett, R. E., Krantz, D. H., Jepson, C., & Kunda, Z. (1983) · Psychological Review · doi:10.1037/0033-295x.90.4.339 · Cited by 811
Statistical Methods for Eliciting Probability Distributions
Garthwaite, P. H., Kadane, J. B., & O'Hagan, A. (2005) · Journal of the American Statistical Association · doi:10.1198/016214505000000105 · Cited by 801
Dunning, D., Meyerowitz, J. A., & Holzberg, A. D. (1989) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.57.6.1082 · Cited by 758
Mood effects on subjective probability assessment
Wright, W. F., & Bower, G. H. (1992) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(92)90039-a · Cited by 681
Dunning, D., Meyerowitz, J. A., & Holzberg, A. D. (2002) · Heuristics and Biases · doi:10.1017/cbo9780511808098.020 · Cited by 678
Varieties of knowledge elicitation techniques
Cooke, N. J. (1994) · International Journal of Human-Computer Studies · doi:10.1006/ijhc.1994.1083 · Cited by 579
Toward a Contingency Model of Strategic Risk Taking
Baird, I. S., & Thomas, H. (1985) · Academy of Management Review · doi:10.5465/amr.1985.4278108 · Cited by 566
Calibration and probability judgements: Conceptual and methodological issues
Keren, G. (1991) · Acta Psychologica · doi:10.1016/0001-6918(91)90036-y · Cited by 549
Decision‐making competence: External validation through an individual‐differences approach
Parker, A. M., & Fischhoff, B. (2005) · Journal of Behavioral Decision Making · doi:10.1002/bdm.481 · Cited by 437
Unpacking, repacking, and anchoring: Advances in support theory.
Rottenstreich, Y., & Tversky, A. (1997) · Psychological Review · doi:10.1037/0033-295x.104.2.406 · Cited by 425
Incorporating the Irrelevant: Anchors in Judgments of Belief and Value
Chapman, G. B., & Johnson, E. J. (2002) · Heuristics and Biases · doi:10.1017/cbo9780511808098.008 · Cited by 410
Using Expectations Data to Study Subjective Income Expectations
Dominitz, J., & Manski, C. F. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10474041 · Cited by 398
Sloman, S. A. (2002) · Heuristics and Biases · doi:10.1017/cbo9780511808098.024 · Cited by 395
Quadrel, M. J., Fischhoff, B., & Davis, W. (1993) · American Psychologist · doi:10.1037//0003-066x.48.2.102 · Cited by 395
Juslin, P., Winman, A., & Olsson, H. (2000) · Psychological Review · doi:10.1037//0033-295x.107.2.384 · Cited by 394
Sherman, S. J., Cialdini, R. B., Schwartzman, D. F., & Reynolds, K. D. (1985) · Personality and Social Psychology Bulletin · doi:10.1177/0146167285111011 · Cited by 382
The Innovation Diffusion Process in a Heterogeneous Population: A Micromodeling Approach
Chatterjee, R. A., & Eliashberg, J. (1990) · Management Science · doi:10.1287/mnsc.36.9.1057 · Cited by 371
Identifying Expertise to Extract the Wisdom of Crowds
Budescu, D. V., & Chen, E. (2015) · Management Science · doi:10.1287/mnsc.2014.1909 · Cited by 360
Self-citationKadane, J., & Wolfson, L. J. (1998) · Journal of the Royal Statistical Society: Series D (The Statistician) · doi:10.1111/1467-9884.00113 · Cited by 331
Assessing uncertainty in physical constants
Henrion, M., & Fischhoff, B. (1986) · American Journal of Physics · doi:10.1119/1.14447 · Cited by 316
Introduction – Heuristics and Biases: Then and Now
Gilovich, T., & Griffin, D. (2002) · Heuristics and Biases · doi:10.1017/cbo9780511808098.002 · Cited by 305
Consistency in interpretation of probabilistic phrases
Budescu, D. V., & Wallsten, T. S. (1985) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(85)90007-x · Cited by 300
Self-citationJuslin, P., Olsson, N., & Winman, A. (1996) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037//0278-7393.22.5.1304 · Cited by 293
Norm Theory: Comparing Reality to Its Alternatives
Kahneman, D., & Miller, D. T. (2002) · Heuristics and Biases · doi:10.1017/cbo9780511808098.022 · Cited by 289
A framework for the formation and structure of consumer expectations: Review and propositions
Oliver, R. L., & Winer, R. S. (1987) · Journal of Economic Psychology · doi:10.1016/0167-4870(87)90037-7 · Cited by 277
Decisions based on numerically and verbally expressed uncertainties.
Budescu, D. V., Weinberg, S., & Wallsten, T. S. (1988) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.14.2.281 · Cited by 252
Self-citationHarvey, N. (1997) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(97)01014-0 · Cited by 239
Overconfidence among beginners: Is a little learning a dangerous thing?
Sanchez, C., & Dunning, D. (2018) · Journal of Personality and Social Psychology · doi:10.1037/pspa0000102 · Cited by 177
Mental Contamination and the Debiasing Problem
Wilson, T. D., Centerbar, D. B., & Brekke, N. (2002) · Heuristics and Biases · doi:10.1017/cbo9780511808098.012 · Cited by 160
The calibration of expert judgment: heuristics and biases beyond the laboratory
Koehler, D. J., Brenner, L., & Griffin, D. (2002) · Heuristics and Biases · doi:10.1017/cbo9780511808098.041 · Cited by 146
Stone, E. R., & Opel, R. B. (2000) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.2000.2910 · Cited by 101
Interpersonal comparison of subjective probabilities: Toward translating linguistic probabilities
Dhami, M. K., & Wallsten, T. S. (2005) · Memory & Cognition · doi:10.3758/bf03193213 · Cited by 75
Self-citation
Metadata
- Authors
- Wallsten, T. S., & Budescu, D. V.
- Venue
- Management Science
- Published
- 1983-02
- Volume
- 29
- Issue
- 2
- Pages
- 151-173
- ISSN
- 0025-1909 (ISSN-L)
- 1526-5501
- OpenAlex
- W2028076981
- Cited by
- 350
- References
- 0
- FWCI
- 9.00
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.