Interpersonal comparison of subjective probabilities: Toward translating linguistic probabilities
Open accessAbstract
Interpersonal variability in understanding linguistic probabilities can adversely affect decision making. Using the fact that everyone judges canonical probability events similarly in a manner consistent with axiom systems that yield a probability measure, we developed and tested a method for comparing the meanings of probability phrases across individuals. An experiment demonstrated that despite extreme heterogeneity in participants' linguistic probability lexicons, interpersonal similarity in phrase meaning is well predicted by phrase rank order within the lexicons. Thus, equally ranked phrases have similar meanings, and individual differences in linguistic probabilities may simply be explained by the phrases people use at each rank.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
When analysts use terms like high confidence, there is no reliable way to know whether they mean probabilities closer to 70 percent or to 90 percent.
“When analysts use terms like”
From The Value of Precision in Probability Assessment: Evidence from a Large-Scale Geopolitical Forecasting Tournament (2018) · cites this paper as Dhami and Wallsten 2005
The Abstract (page unavailable) says that the same probability phrase can refer to different probabilities for different people, supporting the concern about ambiguity. The paper does not establish what analysts mean by “high confidence” or compare 70% with 90%; its Between-Participants Comparisons section also suggests that individual meanings may be translated, so “no reliable way” is too absolute. (LLM, conf=0.84)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The Concept of a Linguistic Variable and Its Application to Approximate Reasoning—I
Zadeh, L. A. (1975) · Information Sciences · doi:10.1016/0020-0255(75)90036-5 · Cited by 12,187 · see citing works
The concept of a linguistic variable and its application to approximate reasoning-III
Zadeh, L. A. (1975) · Information Sciences · doi:10.1016/0020-0255(75)90017-1 · Cited by 12,172 · see citing works
Anderson, N. H. (2001) · Psychology Press eBooks · doi:10.4324/9781410600004-24 · Cited by 6,939 · see citing works
The Psychology of Survey Response
Tourangeau, R., Rips, L. J., & Rasinski, K. (2000) · doi:10.1017/cbo9780511819322 · Cited by 4,734 · see citing works
Decision analysis and behavioral research
von Winterfeldt, D., & Edwards, W. (1986) · Medical Entomology and Zoology · Cited by 3,060 · see citing works
Individual Choice Behavior: A Theoretical Analysis.
Cane, V. R., & Luce, R. D. (1960) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2343282 · Cited by 2,598 · see citing works
Utility Theory for Decision Making
Adelson, R. M., & Fishburn, P. C. (1971) · Operational Research Quarterly (1970-1977) · doi:10.2307/3008003 · Cited by 2,490 · see citing works
Lindley, D. V., & Savage, L. J. (1973) · The Mathematical Gazette · doi:10.2307/3615588 · Cited by 1,973 · see citing works
La prévision : ses lois logiques, ses sources subjectives
de Finetti, B. (1937) · French digital mathematics library (Numdam) · Cited by 1,659 · see citing works
UTILITY THEORY FOR DECISION MAKING
Fishburn, P. C. (1970) · doi:10.21236/ad0708563 · Cited by 1,552 · see citing works
Shafer, G. R., & Hacking, I. (1976) · Journal of the American Statistical Association · doi:10.2307/2285346 · Cited by 1,164 · see citing works
Individual choice behavior: A theoretical analysis.
Luce, R. D. (2005) · doi:10.1037/14396-000 · Cited by 1,157 · cited 1× · see citing works
“…the left of person B's function for β. Finally, since this technique maps judgments of events to intervals (or functions), which sometimes overlap, rather than to points, it yields a semiorder (Luce, 1959) rather than a linear order of subjective probabilities both within and between participants.”
BackgroundThe NEW ENGLAND JOURNAL of MEDICINE
Gelijns, A. C., Rosenberg, N., & Moskowitz, A. J. (2009) · WORLD SCIENTIFIC eBooks · doi:10.1142/9789814273596_0013 · Cited by 1,000 · see citing works
Fine, T. L., & Hacking, I. (1978) · The Philosophical Review · doi:10.2307/2184357 · Cited by 989 · see citing works
Kahneman, D. J., & Tversky, A. (1982) · Cognition · doi:10.1016/0010-0277(82)90023-3 · Cited by 791 · see citing works
Foundations of Measurement, Vol. I: Additive and Polynomial Representations
Krantz, D. H., Luce, D., Suppes, P., et al. (1971) · Cited by 760 · see citing works
Measures of similarity among fuzzy concepts: A comparative analysis
Zwick, R., Carlstein, E., & Budescu, D. (1987) · International Journal of Approximate Reasoning · doi:10.1016/0888-613x(87)90015-6 · Cited by 543 · see citing works
“How Probable is Probable? A Numerical Translation of Verbal Probability Expressions.”
Beyth‐Marom, R. (1982) · Journal of Forecasting · doi:10.1002/for.3980010305 · Cited by 527 · see citing works
Measuring the Vague Meanings of Probability Terms
Wallsten, T., Budescu, D., Rapoport, A., Zwick, R., & Forsyth, B. H. (1986) · Journal of Experimental Psychology General · doi:10.1037/0096-3445.115.4.348 · Cited by 504 · see citing works
Self-citationState of the Art—Encoding Subjective Probabilities: A Psychological and Psychometric Review
Wallsten, T. S., & Budescu, D. V. (1983) · Management Science · doi:10.1287/mnsc.29.2.151 · Cited by 354 · see citing works
Self-citationVerbal probabilities: Ambiguous, context-dependent, or both?
Brun, W., & Teigen, K. H. (1988) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(88)90036-2 · Cited by 334 · see citing works
Empirical scaling of common verbal phrases associated with numerical probabilities
Lichtenstein, S., & Newman, J. R. (1967) · Psychonomic Science · doi:10.3758/bf03327890 · Cited by 313 · see citing works
Consistency in interpretation of probabilistic phrases
Budescu, D. V., & Wallsten, T. S. (1985) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(85)90007-x · Cited by 300 · see citing works
Self-citationDecision making under uncertainty
Roush, F. W. (1984) · Mathematical Social Sciences · doi:10.1016/0165-4896(84)90027-1 · Cited by 298 · see citing works
Preferences and reasons for communicating probabilistic information in verbal or numerical terms
Wallsten, T. S., Budescu, D. V., Zwick, R., & Kemp, S. M. (1993) · Bulletin of the Psychonomic Society · doi:10.3758/bf03334162 · Cited by 291 · see citing works
Self-citationHow Medical Professionals Evaluate Expressions of Probability
Kong, A., Barnett, G. O., Mosteller, F. C., et al. (1986) · New England Journal of Medicine · doi:10.1056/nejm198609183151206 · Cited by 286 · see citing works
Bartoshuk, L. M., Duffy, V. B., Fast, K., Green, B. G., Prutkin, J. M., & Snyder, D. J. (2003) · Food Quality and Preference · doi:10.1016/s0950-3293(02)00077-0 · Cited by 282 · see citing works
Additivity, utility, and subjective probability
Tversky, A. (1967) · Journal of Mathematical Psychology · doi:10.1016/0022-2496(67)90049-1 · Cited by 280 · see citing works
Savage, L. J.: Foundations of Statistics. Dover Publ., Inc,. New York 1972. 310 S.
Bandemer, H. (1974) · Biometrische Zeitschrift · doi:10.1002/bimj.19740160812 · Cited by 270 · see citing works
Decisions based on numerically and verbally expressed uncertainties.
Budescu, D. V., Weinberg, S., & Wallsten, T. S. (1988) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/0096-1523.14.2.281 · Cited by 252 · see citing works
Self-citationComparing the calibration and coherence of numerical and verbal probability judgments
Wallsten, T., Budescu, D., & Zwick, R. (1993) · Management Science · doi:10.1287/mnsc.39.2.176 · Cited by 252 · see citing works
Self-citationVerbal versus numerical probabilities: Efficiency, biases, and the preference paradox
Erev, I., & Cohen, B. L. (1990) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(90)90002-q · Cited by 250 · see citing works
Laming, D. (1997) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780198523420.001.0001 · Cited by 244 · see citing works
Weber, E. U., & Hilton, D. (1990) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037/0096-1523.16.4.781 · Cited by 225 · see citing works
Base rate effects on the interpretations of probability and frequency expressions
Wallsten, T. S., Fillenbaum, S., & Cox, J. A. (1986) · Journal of Memory and Language · doi:10.1016/0749-596x(86)90012-4 · Cited by 216 · see citing works
Self-citationProcessing Linguistic Probabilities: General Principles and Empirical Evidence
Budescu, D. V., & Wallsten, T. S. (1995) · The Psychology of learning and motivation/The psychology of learning and motivation · doi:10.1016/s0079-7421(08)60313-8 · Cited by 203 · see citing works
Self-citationRecertification: Will We Retreat?
Bryant, G. D., & Gr, N. (1980) · New England Journal of Medicine · doi:10.1056/nejm198002143020718 · Cited by 184 · see citing works
Quantitative meanings of verbal probability expressions.
Reagan, R. T., Mosteller, F. C., & Youtz, C. (1989) · Journal of Applied Psychology · doi:10.1037/0021-9010.74.3.433 · Cited by 182 · see citing works
Quantifying Probabilistic Expressions
Mosteller, F. C., & Youtz, C. (1990) · Statistical Science · doi:10.1214/ss/1177012242 · Cited by 180 · see citing works
Decision making from a cognitive perspective
Busemeyer, J. R., Medin, D. L., & Hastie, R. (1995) · Academic Press eBooks · Cited by 175 · see citing works
Yes, but it is uncertain: Direction and communicative intention of verbal probabilistic terms
Teigen, K. H., & Brun, W. (1995) · Acta Psychologica · doi:10.1016/0001-6918(93)e0071-9 · Cited by 162 · see citing works
Karelitz, T. M., & Budescu, D. V. (2004) · Journal of Experimental Psychology Applied · doi:10.1037/1076-898x.10.1.25 · Cited by 141 · see citing works
Expressions of Probability: Words and Numbers
(1980) · New England Journal of Medicine · doi:10.1056/nejm198002143020717 · Cited by 141 · see citing works
Predicting the directionality of probability words from their membership functions
Budescu, D. V., Karelitz, T. M., & Wallsten, T. S. (2003) · Journal of Behavioral Decision Making · doi:10.1002/bdm.440 · Cited by 140 · see citing works
Self-citationSelection of verbal probabilities: A solution for some problems of verbal probability expression
Hamm, R. M. (1991) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(91)90012-i · Cited by 138 · see citing works
Prospect relativity: How choice options influence decision under risk.
Stewart, N., Chater, N., Stott, H. P., & Reimers, S. J. (2003) · Journal of Experimental Psychology General · doi:10.1037/0096-3445.132.1.23 · Cited by 137 · see citing works
Processing of Medical information in Aging Patients
(1999) · Psychology Press eBooks · doi:10.4324/9781410601070 · Cited by 132 · see citing works
Verbal uncertainty expressions: A critical review of two decades of research
Clark, D. A. (1990) · Current Psychology · doi:10.1007/bf02686861 · Cited by 112 · see citing works
The specific meanings of certain terms indicating differing degrees of frequency
Simpson, R. H. (1944) · Quarterly Journal of Speech · doi:10.1080/00335634409381009 · Cited by 107 · see citing works
The Concept of Probability in Psychological Experiments
von Holstein, C.-A. S. S. (1973) · Studies in logic and the foundations of mathematics · doi:10.1016/s0049-237x(09)70379-9 · Cited by 105 · see citing works
Dyadic decisions with numerical and verbal probabilities
Budescu, D. V., & Wallsten, T. S. (1990) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(90)90031-4 · Cited by 74 · see citing works
Self-citationComprehension of verbal probability expressions in children and adolescents
Mullet, É., & Rivet, I. (1991) · Language & Communication · doi:10.1016/0271-5309(91)90007-i · Cited by 70 · see citing works
Bradburn, N. M., & Miles, C. (1979) · Public Opinion Quarterly · doi:10.1086/268494 · Cited by 67 · see citing works
Direct and indirect scaling of membership functions of probability phrases
Rapoport, A., Wallsten, T. S., & Cox, J. A. (1987) · Mathematical Modelling · doi:10.1016/0270-0255(87)90506-9 · Cited by 62 · see citing works
Self-citationHow we may have been misled into believing in the interpersonal comparability of utility
Narens, L., & Luce, R. D. (1983) · Theory and Decision · doi:10.1007/bf00125671 · Cited by 51 · see citing works
Zétényi, T. (1988) · Cited by 50 · see citing works
Madjid Tavana, D., Kennedy, D. T., & Mohebbi, B. (1997) · Journal of Behavioral Decision Making · doi:10.1002/(sici)1099-0771(199706)10:2 · Cited by 45 · see citing works
Timed magnitude comparisons of numerical and nonnumerical expressions of uncertainty
Jaffe-Katz, A., Budescu, D. V., & Wallsten, T. S. (1989) · Memory & Cognition · doi:10.3758/bf03198463 · Cited by 45 · see citing works
Self-citationFillenbaum, S., Wallsten, T. S., Cohen, B. L., & Cox, J. A. (1991) · The American Journal of Psychology · doi:10.2307/1422850 · Cited by 43 · see citing works
Self-citationEffects of probability mode on preference reversal.
Gonzalez-Vallejo, C. C., & Wallsten, T. S. (1992) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037//0278-7393.18.4.855 · Cited by 37 · see citing works
Self-citationMeasuring Vague Uncertainties and Understanding Their Use in Decision Making
Wallsten, T. S. (1990) · Springer eBooks · doi:10.1007/978-94-015-7873-8_15 · Cited by 35 · see citing works
Self-citationDo Decision Quality and Preference Order Depend on Whether Probabilities Are Verbal or Numerical?
Gonzalez-Vallejo, C. C., Erev, I., & Wallsten, T. S. (1994) · The American Journal of Psychology · doi:10.2307/1423035 · Cited by 31 · see citing works
Self-citationWallsten, T. S. (1971) · Journal of Experimental Psychology · doi:10.1037/h0030669 · Cited by 27 · see citing works
Self-citationEffects of probability mode on preference reversal.
Gonzalez-Vallejo, C. C., & Wallsten, T. S. (1992) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/0278-7393.18.4.855 · Cited by 24 · see citing works
Self-citationNewstead, S. E. (1988) · Advances in psychology · doi:10.1016/s0166-4115(08)60481-1 · Cited by 22 · see citing works
Feynman Issues His Own Shuttle Report, Attacking NASA's Risk Estimates
Marshall, E. (1986) · Science · doi:10.1126/science.232.4758.1596 · Cited by 19 · see citing works
[Quantifying Probabilistic Expressions]: Comment
Wallsten, T. S., & Budescu, D. V. (1990) · Statistical Science · doi:10.1214/ss/1177012248 · Cited by 16 · see citing works
Self-citationThe limits of perceived magnitude: Comparison among individuals and among perceptual continua
Teghtsoonian, R., Teghtsoonian, M., & Karlsson, J.-G. (1981) · Acta Psychologica · doi:10.1016/0001-6918(81)90034-2 · Cited by 16 · see citing works
The effects of system failure error on predictions
O'Connor, R. M., Doherty, M. E., & Tweney, R. D. (1989) · Organizational Behavior and Human Decision Processes · doi:10.1016/0749-5978(89)90031-9 · Cited by 14 · see citing works
Talking probabilities: communicating probabilistic information with words and numbers
Renooij, S., & Witteman, C. L. M. (1999) · International Journal of Approximate Reasoning · doi:10.1016/s0888-613x(99)00027-4 · Cited by 13 · see citing works
[Quantifying Probabilistic Expressions]: Rejoinder
Mosteller, F. C., & Youtz, C. (1990) · Statistical Science · doi:10.1214/ss/1177012251 · Cited by 11 · see citing works
The Psychological Concept of Subjective Probability: A Measurement-Theoretic View
Wallsten, T. S. (1974) · Springer eBooks · doi:10.1007/978-94-010-2288-0_4 · Cited by 8 · see citing works
Self-citation[Quantifying Probabilistic Expressions]: Comment: On the Possible Dangers of Isolation
Tanur, J. M. (1990) · Statistical Science · doi:10.1214/ss/1177012247 · Cited by 5 · see citing works
Validation of a new technique for eliciting membership functions of probability phrases
Karelitz, T. M. (2000) · PsycEXTRA Dataset · doi:10.1037/e683322011-137 · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- [3]3. Borg, G. (1982). A category scale with ratio properties for intermodal and interindividual comparisons. In H. Geissler, P. Petzold, H. Buf- fart, & Y. Zabrodin (Eds.), Psychophysical judgment and the process of perception (pp. 25-34). Amsterdam: North-Holland.
- [26]26. Krantz, D. H., Luce, R. D., Suppes, P., & Tversky, A. (1971). Foun- dations of measurement 1: Additive and polynomial representation. New York: Academic Press.
- [35]35. Olson, M. J., & Budescu, D. V. (1997). Patterns of preference for numerical and verbal probabilities. Journal of Behavioral Decision Making, 10, 117-131.
- [52]52. Wallsten, T. S., & Budescu, D. V. (1983). Encoding subjective prob- abilities: A psychological and psychometric review. Management Sci- ence, 29, 151-173.
- [60]60. Zimmer, A. C. (1983). Verbal versus numerical processing of subjective probabilities. In R. W. Scholz (Ed.), Decision making under uncer- tainty (pp. 159-182). Amsterdam: North-Holland.
Cited by
Sorted by citations
“Improving Intelligence Analysis with Decision Science.”
Dhami, M. K., Mandel, D. R., Mellers, B. A., & Tetlock, P. E. (2015) · Perspectives on Psychological Science · doi:10.1177/1745691615598511 · Cited by 104 · see citing works
Self-citationFriedman, J. A., Baker, J. D., Mellers, B. A., Tetlock, P. E., & Zeckhauser, R. (2018) · International Studies Quarterly · doi:10.1093/isq/sqx078 · Cited by 58 · see citing works
- Authors
- Dhami, M. K., & Wallsten, T. S.
- Affiliations
- University of Cambridge (GB)
- University of Maryland, College Park (US)
- Corresponding author
- Dhami, Mandeep K.
- ORCID
- Dhami, Mandeep K.
- Venue
- Memory & Cognition
- Published
- 2005-09
- Volume
- 33
- Issue
- 6
- Pages
- 1057-1068
- ISSN
- 0090-502X (ISSN-L)
- 1532-5946
- OpenAlex
- W2100594827
- Cited by
- 75
- References
- 74
- Citations by year
- 2026: 2 · 2025: 1 · 2024: 1 · 2023: 2 · 2022: 17 · 2021: 3 · 2020: 5 · 2019: 3 · 2018: 4 · 2017: 4 · 2016: 3 · 2015: 8 · 2014: 5 · 2013: 5 · 2012: 1
- FWCI
- 0.97
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- www.springer.com/tdm (tdm)
- Locations
- Memory & Cognition (publishedVersion)
- PubMed (publishedVersion)
- Open Research (University of Surrey) (submittedVersion)
- Surrey Research Insight Open Access (The University of Surrey) (submittedVersion)
- Funders
- National Science Foundation (0196140)
- Topics
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Categorization, perception, and language
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Keywords
- subjective probability, interpersonal variability, probability judgment, decision-making, axiomatic systems, probability measures
- MeSH terms
- Decision Making, Humans, Linguistics, Probability Theory, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
