The DOI matches its Crossref record; 101 of the 101 references it cites were found in a registry or an index.
Statistical Methods for Eliciting Probability Distributions
Open accessAbstract
Elicitation is a key task for subjectivist Bayesians. Although skeptics hold that elicitation cannot (or perhaps should not) be done, in practice it brings statisticians closer to their clients and subject-matter expert colleagues. This article reviews the state of the art, reflecting the experience of statisticians informed by the fruits of a long line of psychological research into how people represent uncertain information cognitively and how they respond to questions about that information. In a discussion of the elicitation process, the first issue to address is what it means for an elicitation to be successful; that is, what criteria should be used. Our answer is that a successful elicitation faithfully represents the opinion of the person being elicited. It is not necessarily “true” in some objectivistic sense, and cannot be judged in that way. We see that elicitation as simply part of the process of statistical modeling. Indeed, in a hierarchical model at which point the likelihood ends and the prior begins is ambiguous. Thus the same kinds of judgment that inform statistical modeling in general also inform elicitation of prior distributions. The psychological literature suggests that people are prone to certain heuristics and biases in how they respond to situations involving uncertainty. As a result, some of the ways of asking questions about uncertain quantities are preferable to others, and appear to be more reliable. However, data are lacking on exactly how well the various methods work, because it is unclear, other than by asking using an elicitation method, just what the person believes. Consequently, one is reduced to indirect means of assessing elicitation methods. The tool chest of methods is growing. Historically, the first methods involved choosing hyperparameters using conjugate prior families, at a time when these were the only families for which posterior distributions could be computed. Modern computational methods, such as Markov chain Monte Carlo, have freed elicitation from this constraint. As a result, now both parametric and nonparametric methods are available for low-dimensional problems. High-dimensional problems are probably best thought of as lacking another hierarchical level, which has the effect of reducing the as-yet-unelicited parameter space. Special considerations apply to the elicitation of group opinions. Informal methods, such as Delphi, encourage the participants to discuss the issue in the hope of reaching consensus. Formal methods, such as weighted averages or logarithmic opinion pools, each have mathematical characteristics that are uncomfortable. Finally, there is the question of what a group opinion even means, because it is not necessarily the opinion of any participant.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
Availability: A heuristic for judging frequency and probability
Tversky, A., & Kahneman, D. (1973) · Cognitive Psychology · doi:10.1016/0010-0285(73)90033-9 · Cited by 9,984 · cited 2×
“Hence, most people judge that “r” is more likely to be the first letter of a word, rather than the third letter, although the reverse is true (Tversky and Kahneman, 1973).”
BackgroundSocial Choice and Individual Values.
Shackle, G. L. S., & Arrow, K. J. (1951) · Economica · doi:10.2307/2549613 · Cited by 7,792
On the psychology of prediction.
Kahneman, D., & Tversky, A. (1973) · Psychological Review · doi:10.1037/h0034747 · Cited by 6,202
The growth of logical thinking: From childhood to adolescence.
Inhelder, B., & Piaget, J. (1958) · Basic Books · doi:10.1037/10034-000 · Cited by 4,596
Degroot, M. H. (1974) · Journal of the American Statistical Association · doi:10.1080/01621459.1974.10480137 · Cited by 4,004
Belief in the law of small numbers.
Tversky, A., & Kahneman, D. (1971) · Psychological Bulletin · doi:10.1037/h0031322 · Cited by 3,246
Kendall's Advanced Theory of Statistics.
AA, Stuart, A., & Ord, K. (1995) · Journal of the American Statistical Association · doi:10.2307/2291187 · Cited by 2,619
Calibration of probabilities: the state of the art to 1980
Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1982) · Judgment under Uncertainty · doi:10.1017/cbo9780511809477.023 · Cited by 1,967
Decision Analysis: Introductory Lectures on Choices Under Uncertainty.
Klevorick, A. K., & Raiffa, H. (1969) · The Journal of Finance · doi:10.2307/2325714 · Cited by 1,928
The Statistical Evaluation of Medical Tests for Classification and Prediction
(2005) · Technometrics · doi:10.1198/tech.2005.s278 · Cited by 1,899
Judgment under Uncertainty: Heuristics and Biases
(2017) · Macat Library eBooks · doi:10.4324/9781912282562 · Cited by 1,843
Comparison of Bayesian and regression approaches to the study of information processing in judgment
Slovic, P., & Lichtenstein, S. (1971) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(71)90033-x · Cited by 1,819
Generalized Latent Variable Modeling: Multilevel, Longitudinal, and Structural Equation Models
Skrondal, A., & Rabe‐Hesketh, S. (2004) · Thammasat University Digital Collections · Cited by 1,505
Koriat, A., Lichtenstein, S., & Fischhoff, B. (1980) · Journal of Experimental Psychology: Human Learning and Memory · doi:10.1037/0278-7393.6.2.107 · Cited by 1,477
Support theory: a nonextensional representation of subjective probability
Tversky, A., & Koehler, D. J. (1994) · Psychological Review · doi:10.1037/0033-295x.101.4.547 · Cited by 1,167
Experts in uncertainty: opinion and subjective probability in science
(1992) · Choice Reviews Online · doi:10.5860/choice.29-5666 · Cited by 1,129
Smith, J. Q., Bernardo, J. M., Berger, J. O., Dawid, A. P., & Smith, A. F. M. (1993) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2982740 · Cited by 1,097
On narrow norms and vague heuristics: A reply to Kahneman and Tversky.
Gigerenzer, G. (1996) · Psychological Review · doi:10.1037//0033-295x.103.3.592 · Cited by 1,070
Ratio scales and category scales for a dozen perceptual continua.
Stevens, S. S., & Galanter, E. H. (1957) · Journal of Experimental Psychology · doi:10.1037/h0043680 · Cited by 1,051
Combining Probability Distributions: A Critique and an Annotated Bibliography
Genest, C., & Zidek, J. V. (1986) · Statistical Science · doi:10.1214/ss/1177013825 · Cited by 1,001
Man as an intuitive statistician.
Peterson, C. R., & Beach, L. R. (1967) · Psychological Bulletin · doi:10.1037/h0024722 · Cited by 963
Scoring Rules for Continuous Probability Distributions
Matheson, J. E., & Winkler, R. L. (1976) · Management Science · doi:10.1287/mnsc.22.10.1087 · Cited by 957
A progress report on the training of probability assessors
Alpert, M., & Raiffa, H. (1982) · Judgment under Uncertainty · doi:10.1017/cbo9780511809477.022 · Cited by 842
Howard, R. A., & Matheson, J. E. (2005) · Decision Analysis · doi:10.1287/deca.1050.0020 · Cited by 819
The base rate fallacy reconsidered: Descriptive, normative, and methodological challenges
Koehler, J. J. (1996) · Behavioral and Brain Sciences · doi:10.1017/s0140525x00041157 · Cited by 763
Fault trees: sensitivity of estimated failure probabilities to problem representation
Fischhoff, B., Slovic, P., & Lichtenstein, S. (1978) · Journal of Experimental Psychology: Human Perception and Performance · doi:10.1037/0096-1523.4.2.330 · Cited by 708
Readings on the Principles and Applications of Decision Analysis
Howard, R. A., & Matheson, J. E. (1989) · Cited by 685
Fischhoff, B., & Beyth, R. (1975) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(75)90002-1 · Cited by 685
Framing, Probability Distortions, and Insurance Decisions
Johnson, E. J., Hershey, J., Meszaros, J., & Kunreuther, H. (1993) · Making Decisions About Liability And Insurance · doi:10.1007/978-94-011-2192-7_3 · Cited by 668
Exceptional Paper—Probability Encoding in Decision Analysis
Spetzler, C. S., & Staël Von Holstein, C.-A. S. (1975) · Management Science · doi:10.1287/mnsc.22.3.340 · Cited by 662
Challenge problems: uncertainty in system response given uncertain parameters
Oberkampf, W. L., Helton, J. C., Joslyn, C., Wojtkiewicz, S. F., & Ferson, S. A. (2004) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2004.03.002 · Cited by 649
An overview of robust Bayesian analysis
Berger, J. O., Moreno, E., Pericchi, L. R., Bayarri, M. J., Bernardo, J. M., Cano, J. A., De la Horra, J., Martín, J., Ríos-Insúa, D., Betrò, B., et al. (1994) · Test · doi:10.1007/bf02562676 · Cited by 602
Self-citationThe Delphi method: Substance, context, a critique and an annotated bibliography
Pill, J. (1971) · Socio-Economic Planning Sciences · doi:10.1016/0038-0121(71)90041-3 · Cited by 579
Relative importance of probabilities and payoffs in risk taking.
Slovic, P., & Lichtenstein, S. (1968) · Journal of Experimental Psychology · doi:10.1037/h0026468 · Cited by 562
Calibration and probability judgements: Conceptual and methodological issues
Keren, G. (1991) · Acta Psychologica · doi:10.1016/0001-6918(91)90036-y · Cited by 549
“How Probable is Probable? A Numerical Translation of Verbal Probability Expressions.”
Beyth‐Marom, R. (1982) · Journal of Forecasting · doi:10.1002/for.3980010305 · Cited by 527
Measuring the vague meanings of probability terms.
Wallsten, T. S., Budescu, D. V., Rapoport, A., Zwick, R., & Forsyth, B. (1986) · Journal of Experimental Psychology: General · doi:10.1037/0096-3445.115.4.348 · Cited by 517
On the psychology of experimental surprises.
Slovic, P., & Fischhoff, B. (1977) · Journal of Experimental Psychology Human Perception & Performance · doi:10.1037//0096-1523.3.4.544 · Cited by 485
Calibration of Probabilities: The State of the Art
Lichtenstein, S., Fischhoff, B., & Phillips, L. D. (1977) · Decision Making and Change in Human Affairs · doi:10.1007/978-94-010-1276-8_19 · Cited by 483
Cognitive Processes and the Assessment of Subjective Probability Distributions
Hogarth, R. M. (1975) · Journal of the American Statistical Association · doi:10.1080/01621459.1975.10479858 · Cited by 479
Introduction to Bayesian Inference and Decision.
Hildebrand, D. K., & Winkler, R. L. (1974) · Journal of the American Statistical Association · doi:10.2307/2285548 · Cited by 450
The Assessment of Prior Distributions in Bayesian Analysis
Winkler, R. L. (1967) · Journal of the American Statistical Association · doi:10.1080/01621459.1967.10500894 · Cited by 430
Use of nonlinear, noncompensatory models as a function of task and amount of information
Einhorn, H. J. (1971) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(71)90002-x · Cited by 380
Lichtenstein, S., & Fischhoff, B. (1980) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(80)90052-5 · Cited by 375
Bayesian inference and decision techniques : essays in honor of Bruno de Finetti
Zellner, A., de Finetti, B., & Goel, P. K. (1986) · North-Holland eBooks · Cited by 373
THE CONCEPT OF CORRELATION IN ADULTS
Smedslund, J. (1963) · Scandinavian Journal of Psychology · doi:10.1111/j.1467-9450.1963.tb01324.x · Cited by 356
State of the Art—Encoding Subjective Probabilities: A Psychological and Psychometric Review
Wallsten, T. S., & Budescu, D. V. (1983) · Management Science · doi:10.1287/mnsc.29.2.151 · Cited by 350
Laud, P. W., & Ibrahim, J. G. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02028.x · Cited by 350
Correlations and Copulas for Decision and Risk Analysis
Clemen, R. T., & Reilly, T. (1999) · Management Science · doi:10.1287/mnsc.45.2.208 · Cited by 343
On the Reconciliation of Probability Assessments
Lindley, D. V., Tversky, A., & Brown, R. V. (1979) · Journal of the Royal Statistical Society. Series A (General) · doi:10.2307/2345078 · Cited by 340
Kadane, J., & Wolfson, L. J. (1998) · Journal of the Royal Statistical Society: Series D (The Statistician) · doi:10.1111/1467-9884.00113 · Cited by 331
Self-citationOn the subjective probability of compound events
Bar-Hillel, M. (1973) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(73)90061-5 · Cited by 330
Measuring psychological uncertainty: Verbal versus numeric methods.
Windschitl, P. D., & Wells, G. L. (1996) · Journal of Experimental Psychology: Applied · doi:10.1037/1076-898x.2.4.343 · Cited by 326
Empirical scaling of common verbal phrases associated with numerical probabilities
Lichtenstein, S., & Newman, J. R. (1967) · Psychonomic Science · doi:10.3758/bf03327890 · Cited by 313 · cited 2×
“Unfortu nately, there is considerable variation in the probabilities that different people attach to the same phrase, and the context also affects the probability that a person associates with a phrase (Lichtenstein and Newman 1967; Beyth-Marom 1982; Wallsten et al. 1986).”
BackgroundThe display of information and the judgment of contingency.
Ward, W. C., & Jenkins, H. M. (1965) · Canadian Journal of Psychology / Revue canadienne de psychologie · doi:10.1037/h0082908 · Cited by 311
O'Hagan, A. (1998) · Journal of the Royal Statistical Society: Series D (The Statistician) · doi:10.1111/1467-9884.00114 · Cited by 309
Self-citationPopular induction: Information is not necessarily informative
Nisbett, R. E., Borgida, E., Crandall, R., & Reed, H. (1982) · Judgment under Uncertainty · doi:10.1017/cbo9780511809477.008 · Cited by 295
Interactive Elicitation of Opinion for a Normal Linear Model
Kadane, J. B., Dickey, J. M., Winkler, R. L., Smith, W. S., & Peters, S. C. (1980) · Journal of the American Statistical Association · doi:10.1080/01621459.1980.10477562 · Cited by 294
Self-citationMorris, P. A. (1974) · Management Science · doi:10.1287/mnsc.20.9.1233 · Cited by 288
Slovic, P. (1972) · PsycEXTRA Dataset · doi:10.1037/e310462005-001 · Cited by 272
A New Perspective on Priors for Generalized Linear Models
Bedrick, E. J., Christensen, R., & Johnson, W. (1996) · Journal of the American Statistical Association · doi:10.1080/01621459.1996.10476713 · Cited by 271
Probability is perfect, but we can't elicit it perfectly
O'Hagan, A., & Oakley, J. E. (2004) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2004.03.014 · Cited by 236
Self-citationQuantifying Probabilistic Expressions
Mosteller, F., & Youtz, C. (1990) · Statistical Science · doi:10.1214/ss/1177012242 · Cited by 177
The psychometrics of everyday life
Kunda, Z., & Nisbett, R. E. (1986) · Cognitive Psychology · doi:10.1016/0010-0285(86)90012-5 · Cited by 156
Approximating Priors by Mixtures of Natural Conjugate Priors
Dalal, S. R., & Hall, W. J. (1983) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1983.tb01251.x · Cited by 154
The Scientist Speculates: An Anthology of Partly-baked Ideas
Page, T. (1964) · American Journal of Physics · doi:10.1119/1.1970298 · Cited by 151
Procedures Guide for Structural Expert Judgement in Accident Consequence Modelling
Cooke, R. M., & Goossens, L. H. J. (2000) · Radiation Protection Dosimetry · doi:10.1093/oxfordjournals.rpd.a033152 · Cited by 151
A case of radical probability estimation.
Hammerton, M. (1973) · Journal of Experimental Psychology · doi:10.1037/h0035224 · Cited by 149
Separating Probability Elicitation from Utilities
Winkler, R. L. (1999) · Cambridge University Press eBooks · doi:10.1017/cbo9781139173230.005 · Cited by 130
Subjective Probability and Its Measurement
Hampton, J. M., Moore, P. G., & Thomas, H. (1973) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2344419 · Cited by 129
Prior Elicitation, Variable Selection and Bayesian Computation for Logistic Regression Models
Chen, M., Ibrahim, J. G., & Yiannoutsos, C. T. (1999) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/1467-9868.00173 · Cited by 118
METHODS FOR QUANTIFYING SUBJECTIVE PROBABILITIES AND MULTI‐ATTRIBUTE UTILITIES*†
Huber, G. P. (1974) · Decision Sciences · doi:10.1111/j.1540-5915.1974.tb00630.x · Cited by 117
Assessment of a Beta Prior Distribution: PM Elicitation
Chaloner, K. M., & Duncan, G. T. (1983) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2987609 · Cited by 112
Eliciting subjective probability distributions on continuous variables
Seaver, D. A., von Winterfeldt, D., & Edwards, W. (1978) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(78)90061-2 · Cited by 107
Graphical Elicitation of a Prior Distribution for a Clinical Trial
Chaloner, K. M., Church, T. R., Louis, T. A., & Matts, J. P. (1993) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348469 · Cited by 106
Assessing Dependence: Some Experimental Results
Clemen, R. T., Fischer, G. W., & Winkler, R. L. (2000) · Management Science · doi:10.1287/mnsc.46.8.1100.12023 · Cited by 104
Kendall's advanced theory of statistics. Vol.2B: Bayesian inference
O’Hagan, A., & Forster, J. J. (2004) · Cited by 99
Self-citationA Predictive Approach to the Analysis of Designed Experiments
Ibrahim, J. G., & Laud, P. W. (1994) · Journal of the American Statistical Association · doi:10.1080/01621459.1994.10476472 · Cited by 95
Group Elicitation of Probability Distributions: Are Many Heads Better Than One?
Phillips, L. D. (1999) · Springer eBooks · doi:10.1007/978-1-4615-5089-1_17 · Cited by 76
Mode, median, and mean as optimal strategies.
Peterson, C. R., & Miller, A. J. (1964) · Journal of Experimental Psychology · doi:10.1037/h0040387 · Cited by 75
Madansky, A. (1964) · Cited by 75
Quantifying Expert Opinion in Linear Regression Problems
Garthwaite, P. H., & Dickey, J. (1988) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1988.tb01741.x · Cited by 74
Self-citationGokhale, D. V., & Press, S. J. (1982) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2981537 · Cited by 67
Elicitation of Prior Distributions for Variable-Selection Problems in Regression
Garthwaite, P. H., & Dickey, J. (1992) · The Annals of Statistics · doi:10.1214/aos/1176348886 · Cited by 63
Self-citationQuantifying Expert Opinion in the UK Water Industry: An Experimental Study
Garthwaite, P. H., & O’Hagan, A. (2000) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.1111/1467-9884.00246 · Cited by 58
Self-citationAbsolute judgments of discrete quantities randomly distributed over time.
Erlick, D. E. (1964) · Journal of Experimental Psychology · doi:10.1037/h0042698 · Cited by 57
Credible Interval Temperature Forecasting: Some Experimental Results
Murphy, A. H., & Winkler, R. L. (1974) · Monthly Weather Review · doi:10.1175/1520-0493(1974)102<0784:citfse>2.0.co;2 · Cited by 52
Percentage estimation of proportion as a function of element type, exposure time, and task.
Shuford, E. H. (1961) · Journal of Experimental Psychology · doi:10.1037/h0043335 · Cited by 49
A Comparison of Two Elicitation Methods for a Prior Distribution for a Binomial Parameter
Gavasakar, U. (1988) · Management Science · doi:10.1287/mnsc.34.6.784 · Cited by 39
Probability: Methods and Measurement.
Burrell, Q. L., & O’Hagan, A. (1991) · Journal of the Royal Statistical Society Series C (Applied Statistics) · doi:10.2307/2347926 · Cited by 38
Self-citationGoldstein, M. (2005) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess0986.pub2 · Cited by 37
Some properties of the dirichlet-multinomial distribution and its use in prior elicitation
Chaloner, K. M., & Duncan, G. T. (1987) · Communication in Statistics- Theory and Methods · doi:10.1080/03610928708829384 · Cited by 33
An elicitation method for multivariate normal distributions
Alawadhi, S., & Garthwaite, P. H. (1998) · Communication in Statistics- Theory and Methods · doi:10.1080/03610929808832149 · Cited by 32
Self-citationRanges of Posterior Probabilities for Quasiunimodal Priors With Specified Quantiles
O’Hagan, A., & Berger, J. O. (1988) · Journal of the American Statistical Association · doi:10.1080/01621459.1988.10478624 · Cited by 31
Self-citationAn elicitation method for multiple linear regression models
Garthwaite, P. H., & Dickey, J. (1991) · Journal of Behavioral Decision Making · doi:10.1002/bdm.3960040103 · Cited by 24
Self-citationQuantifying and using expert opinion for variable-selection problems in regression
Garthwaite, P. H., & Dickey, J. (1996) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/s0169-7439(96)00035-4 · Cited by 22
Self-citationBeliefs about beliefs, a theory for stochastic assessments of subjective probabilities
Dickey, J. M. (1980) · Trabajos de Estadistica y de Investigacion Operativa · doi:10.1007/bf02888364 · Cited by 20
The sequential judgment of proportion
Pitz, G. F. (1966) · Psychonomic Science · doi:10.3758/bf03342357 · Cited by 13
Specifying a Prior Distribution in Structured Regression Problems
Oman, S. D. (1985) · Journal of the American Statistical Association · doi:10.1080/01621459.1985.10477159 · Cited by 12
[Quantifying Probabilistic Expressions]: Rejoinder
Mosteller, F. C., & Youtz, C. (1990) · Statistical Science · doi:10.1214/ss/1177012251 · Cited by 11
Cited by
Strictly Proper Scoring Rules, Prediction, and Estimation
Gneiting, T., & Raftery, A. E. (2007) · Journal of the American Statistical Association · doi:10.1198/016214506000001437 · Cited by 5,793
Bayesian estimation supersedes the t-test
Kruschke, J. K. (2013) · Journal of Experimental Psychology: General · doi:10.1037/a0029146 · Cited by 1,530
Bayes factor design analysis: Planning for compelling evidence
Schönbrodt, F. D., & Wagenmakers, E.-J. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1230-y · Cited by 729
Use (and abuse) of expert elicitation in support of decision making for public policy
Morgan, M. G. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1319946111 · Cited by 702
Good practices for quantitative bias analysis
Lash, T. L., Fox, M. P., MacLehose, R. F., Maldonado, G., McCandless, L. C., & Greenland, S. (2014) · International Journal of Epidemiology · doi:10.1093/ije/dyu149 · Cited by 646
Metadata
- Authors
- Garthwaite, P. H., Kadane, J. B., & O'Hagan, A.
- Venue
- Journal of the American Statistical Association
- Publisher
- Informa UK Limited
- Published
- 2005-06
- Volume
- 100
- Issue
- 470
- Pages
- 680-701
- ISSN
- 0162-1459 (ISSN-L)
- 1537-274X
- OpenAlex
- W2079025608
- Cited by
- 801
- References
- 101
- FWCI
- 25.28
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.