Apples and Oranges (and Pears, Oh My!): The Search for Moderators in Meta-Analysis
The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Abstract
The purpose of this article is to review current practices with respect to detection and estimation of moderators in meta-analysis and to develop recommendations that are driven by the results of this review and previous research. The first purpose was accomplished through a review of the meta-analyses published in Journal of Applied Psychology from 1978 to 1997. Results show, first, that practices with respect to both the execution of and the reporting of results from searches for moderators are highly variable and, second, that findings relevant for detection of moderators (e.g., percentage variance attributable to artifacts, SDρ, etc.) are often highly inconsistent with what has been suggested in the past. These practices held regardless of time of publication, specificity of the question addressed in the paper, and content area. Detailed suggestions for modifications of current practices are offered.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Significance tests and goodness of fit in the analysis of covariance structures.
Bentler, P. M., & Bonett, D. G. (1980) · Psychological Bulletin · doi:10.1037//0033-2909.88.3.588 · Cited by 19,023 · cited 1×
“In structural equation modeling, indices of model fit, such as the normed fit index (Bentler & Bonnet, 1980), involve a comparison of the residual matrix of the hypothesized model to the residual matrix of a null model.2 Although the definition of “null” can be debated (Mulaik, James, Van Alstine,…”
BackgroundThe application of electronic computers to factor analysis
Kaiser, H. F. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000116 · Cited by 12,459 · cited 1×
“This convention stems from work by Kaiser (1960) and Guttman (1954) in which it was shown that a factor with an eigenvalue = 1 has a reliability of zero.”
Evaluating the use of exploratory factor analysis in psychological research.
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999) · Psychological Methods · doi:10.1037/1082-989x.4.3.272 · Cited by 9,587
Huber, P. J. (1981) · Wiley Series in Probability and Statistics · doi:10.1002/0471725250 · Cited by 7,066
Comparison of five rules for determining the number of components to retain.
Zwick, W. R., & Velicer, W. F. (1986) · Psychological Bulletin · doi:10.1037//0033-2909.99.3.432 · Cited by 3,129 · cited 1×
“Although it can certainly be argued that Kaiser’s criterion is overextended (Cattell & Vogelmann, 1977; Fabrigar, Wegener, MacCallum, & Strahan, 1999; Zwick & Velicer, 1986), it is useful as a baseline value for ruling out factors/ components.”
BackgroundMeta-Analytic review of leader–member exchange theory: Correlates and construct issues.
Gerstner, C. R., & Day, D. V. (1997) · Journal of Applied Psychology · doi:10.1037/0021-9010.82.6.827 · Cited by 3,005 · cited 1×
“…be followed up byt tests, informal consideration of percentage variance attributable to artifacts (e.g., Huffcutt, Roth, & McDaniel, 1996), Hedges and Olkin (1985)Q tests (e.g., Gerstner & Day, 1997), or more sophisticated techniques such as hierarchical linear modeling (van Eerde & Thierry, 1996).”
MethodEvaluation of goodness-of-fit indices for structural equation models.
Mulaik, S. A., James, L. R., Van Alstine, J., & Bennett, N. (1989) · Psychological Bulletin · doi:10.1037//0033-2909.105.3.430 · Cited by 2,923
Some necessary conditions for common-factor analysis
Guttman, L. (1954) · Psychometrika · doi:10.1007/bf02289162 · Cited by 1,689 · cited 2×
“This convention stems from work by Kaiser (1960) and Guttman (1954) in which it was shown that a factor with an eigenvalue = 1 has a reliability of zero. Thus, this can be thought of as a “point of no return” criterion such that we may not think much of factors with eigenvalues only slightly larger than 1, but we can…”
The five factor model of personality and job performance in the European Community.
Salgado, J. F. (1997) · Journal of Applied Psychology · doi:10.1037//0021-9010.82.1.30 · Cited by 1,581 · cited 1×
“…which a criterion was specified, 57% specified the “75% rule”, 19% specified aχ2 of some kind (e.g.,Q, U), 2% relied on a “credibility interval does not contain zero” rule, and 6% specified some other rule (e.g., Salgado, 1997: sampling error accounts for more than half of the observed variance).”
BackgroundRole of social desirability in personality testing for personnel selection: The red herring.
Ones, D. S., Viswesvaran, C., & Reiss, A. D. (1996) · Journal of Applied Psychology · doi:10.1037//0021-9010.81.6.660 · Cited by 924 · cited 4×
“For example, a meta-analysis reported in Ones, Viswesvaran, and Reiss (1996) resulted in a percentage variance value of 73%, but the standard deviation ofρ was only .034, resulting in a very small credibility interval.”
BackgroundMethodInfluentialDevelopment of a general solution to the problem of validity generalization.
Schmidt, F. L., & Hunter, J. E. (1977) · Journal of Applied Psychology · doi:10.1037/0021-9010.62.5.529 · Cited by 769 · cited 1×
“…been published there (e.g., Callender & Osburn, 1981; James et al., 1986; Johnson et al., 1995; Law, Schmidt, & Hunter, 1994a, 1994b; Raju & Burke, 1983; Sackett et al., 1987; Schmidt & Hunter, 1977; Spector & Levine, 1987; Switzer et al., 1992; Wanous, Sullivan, & Malinak, 1989; Whitener, 1990).”
Vroom's expectancy models and work-related criteria: A meta-analysis.
van Eerde, W., & Thierry, H. (1996) · Journal of Applied Psychology · doi:10.1037//0021-9010.81.5.575 · Cited by 687 · cited 1×
“…be followed up byt tests, informal consideration of percentage variance attributable to artifacts (e.g., Huffcutt, Roth, & McDaniel, 1996), Hedges and Olkin (1985)Q tests (e.g., Gerstner & Day, 1997), or more sophisticated techniques such as hierarchical linear modeling (van Eerde & Thierry, 1996).”
Method(1990) · Cited by 561
A Comprehensive Trial Of The Scree And Kg Criteria For Determining The Number Of Factors
Cattell, R. B., & Vogelmann, S. (1977) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr1203_2 · Cited by 506 · cited 1×
“Although it can certainly be argued that Kaiser’s criterion is overextended (Cattell & Vogelmann, 1977; Fabrigar, Wegener, MacCallum, & Strahan, 1999; Zwick & Velicer, 1986), it is useful as a baseline value for ruling out factors/ components.”
BackgroundConfusion of confidence intervals and credibility intervals in meta-analysis.
Whitener, E. M. (1990) · Journal of Applied Psychology · doi:10.1037/0021-9010.75.3.315 · Cited by 432 · cited 2×
“…& Arthur, 1995; Hunter & Schmidt, 1994; James et al., 1986; Johnson, Mullen, & Salas, 1995; Kemery, Mossholder, & Roth, 1987; Osburn, Callender, Greener, & Ashworth, 1983; Raju, Pappas, & Williams, 1989; Sackett et al., 1987; Spector & Levine, 1987; Switzer, Paese, & Drasgow, 1992; Whitener, 1990).”
Podsakoff, P. M., MacKenzie, S. B., & Bommer, W. H. (1996) · Journal of Applied Psychology · doi:10.1037//0021-9010.81.4.380 · Cited by 415 · cited 2×
“Therefore, the analyses reported here include values from the Podsakoff et al. (1996) study.”
MethodPearlman, K., Schmidt, F. L., & Hunter, J. E. (1980) · Journal of Applied Psychology · doi:10.1037//0021-9010.65.4.373 · Cited by 343 · cited 1×
“…third component were not coded for, a conservative estimate of .0368, which is the mean of theV values for the assumed distributions involvingryy = .6,rxx = .8, and ratios of restricted to unrestricted standard deviations of either 1.0 or .59, can be taken from Pearlman, Schmidt, and Hunter (1980).”
BackgroundInfluentialComparing meta-analytic moderator estimation techniques under realistic conditions.
Steel, P. D., & Kammeyer-Mueller, J. D. (2002) · Journal of Applied Psychology · doi:10.1037/0021-9010.87.1.96 · Cited by 309 · cited 2×
“Steel, R. P., & Ovalle, N. K. (1984). A review and meta-analysis of research on the relationship between behavioral intentions and employee turnover.”
BackgroundHow many discoveries have been lost by ignoring modern statistical methods?
Wilcox, R. R. (1998) · American Psychologist · doi:10.1037//0003-066x.53.3.300 · Cited by 298 · cited 1×
“Various authors have suggested that outliers be routinely removed from data sets so that the distributions conform better to expectations (e.g., Huber, 1981; Wilcox, 1997).”
BackgroundDevelopment of a new outlier statistic for meta-analytic data.
Huffcutt, A. I., & Arthur, W. (1995) · Journal of Applied Psychology · doi:10.1037//0021-9010.80.2.327 · Cited by 266 · cited 3×
“…on artifacts corrected for, appropriateness/accuracy of those corrections, estimation procedure (e.g., Hunter & Schmidt, 1994; Raju et al., 1989; Switzer et al., 1992), and consideration of outliers (e.g., Beal, Corey, & Dunlap, 2002; Huffcutt & Arthur, 1995; James et al., 1992; Raju et al., 1989).”
MethodMeta-Analysis in Education: How Has It Been Used?
Slavin, R. E. (1984) · Educational Researcher · doi:10.2307/1175907 · Cited by 191 · cited 1×
“Although previous authors have made recommendations regarding the conduct of meta-analyses (e.g., Hedges & Olkin, 1985; Hunter & Schmidt, 1990; Orwin & Cordray, 1985; Slavin, 1984; Wanous et al., 1989), few have been specific to moderator identification and estimation.”
Huffcutt, A. I., Roth, P. L., & McDaniel, M. A. (1996) · Journal of Applied Psychology · doi:10.1037//0021-9010.81.5.459 · Cited by 184 · cited 1×
“Thus, contrary to claims by Hunter and Schmidt (1990) and others, a relatively small percentage variance is typi cally attributable to artifacts, and considerable variability remains after correction of variance for artifacts.”
ResultThe role of judgment calls in meta-analysis.
Wanous, J. P., Sullivan, S. E., & Malinak, J. (1989) · Journal of Applied Psychology · doi:10.1037//0021-9010.74.2.259 · Cited by 171 · cited 1×
“Although previous authors have made recommendations regarding the conduct of meta-analyses (e.g., Hedges & Olkin, 1985; Hunter & Schmidt, 1990; Orwin & Cordray, 1985; Slavin, 1984; Wanous et al., 1989), few have been specific to moderator identification and estimation.”
BackgroundStatistical Methods for Behavioral Science Research.
Maisel, R., & Marascuilo, L. A. (1973) · Contemporary Sociology · doi:10.2307/2064181 · Cited by 167
On the logic and purpose of significance testing.
Cortina, J. M., & Dunlap, W. P. (1997) · Psychological Methods · doi:10.1037/1082-989x.2.2.161 · Cited by 166 · cited 1×
“The probabilistic nature of these statements can cause problems, but this is not especially likely (Cortina & Dunlap, 1997).”
BackgroundSelf-citationCorrelates of employee lateness: Some theoretical considerations.
Koslowsky, M., Sagie, A., Krausz, M., & Singer, A. D. (1997) · Journal of Applied Psychology · doi:10.1037//0021-9010.82.1.79 · Cited by 161
Sackett, P. R., Harris, M. M., & Orr, J. M. (1986) · Journal of Applied Psychology · doi:10.1037/0021-9010.71.2.302 · Cited by 152 · cited 3×
“…& Arthur, 1995; Hunter & Schmidt, 1994; James et al., 1986; Johnson, Mullen, & Salas, 1995; Kemery, Mossholder, & Roth, 1987; Osburn, Callender, Greener, & Ashworth, 1983; Raju, Pappas, & Williams, 1989; Sackett et al., 1987; Spector & Levine, 1987; Switzer, Paese, & Drasgow, 1992; Whitener, 1990).”
ResultComparison of three major meta-analytic approaches.
Johnson, B. T., Mullen, B., & Salas, E. (1995) · Journal of Applied Psychology · doi:10.1037//0021-9010.80.1.94 · Cited by 133 · cited 1×
“Much of the most important work on meta-analysis has been published there (e.g., Callender & Osburn, 1981; James et al., 1986; Johnson et al., 1995; Law, Schmidt, & Hunter, 1994a, 1994b; Raju & Burke, 1983; Sackett et al., 1987; Schmidt & Hunter, 1977; Spector & Levine, 1987; Switzer et al., 1992;…”
BackgroundWhen is it Acceptable to Accept a Null Hypothesis: No Way, Jose?
Cortina, J. M., & Folger, R. G. (1998) · Organizational Research Methods · doi:10.1177/109442819813004 · Cited by 130 · cited 1×
“Moderators provide boundary conditions for the effects that are hypothesized, thus informing researchers of the situations in which the effects in question do and do not hold (Cortina & Folger, 1998).”
BackgroundSelf-citationEffects of deficient reporting on meta-analysis: A conceptual framework and reanalysis.
Orwin, R. G., & Cordray, D. S. (1985) · Psychological Bulletin · doi:10.1037/0033-2909.97.1.134 · Cited by 102 · cited 1×
“Although previous authors have made recommendations regarding the conduct of meta-analyses (e.g., Hedges & Olkin, 1985; Hunter & Schmidt, 1990; Orwin & Cordray, 1985; Slavin, 1984; Wanous et al., 1989), few have been specific to moderator identification and estimation.”
Spector, P. E., & Levine, E. L. (1987) · Journal of Applied Psychology · doi:10.1037//0021-9010.72.1.3 · Cited by 97
Validity generalization in the context of situational models.
James, L. R., Demaree, R. G., Mulaik, S. A., & Ladd, R. T. (1992) · Journal of Applied Psychology · doi:10.1037//0021-9010.77.1.3 · Cited by 88 · cited 1×
“…on artifacts corrected for, appropriateness/accuracy of those corrections, estimation procedure (e.g., Hunter & Schmidt, 1994; Raju et al., 1989; Switzer et al., 1992), and consideration of outliers (e.g., Beal, Corey, & Dunlap, 2002; Huffcutt & Arthur, 1995; James et al., 1992; Raju et al., 1989).”
MethodValidity generalization results for two job groups in the petroleum industry.
Schmidt, F. L., Hunter, J. E., & Caplan, J. R. (1981) · Journal of Applied Psychology · doi:10.1037/0021-9010.66.3.261 · Cited by 80
Beal, D. J., Corey, D. M., & Dunlap, W. P. (2002) · Journal of Applied Psychology · doi:10.1037//0021-9010.87.3.583 · Cited by 79
A note on validity generalization procedures.
James, L. R., Demaree, R. G., & Mulaik, S. A. (1986) · Journal of Applied Psychology · doi:10.1037//0021-9010.71.3.440 · Cited by 72 · cited 6×
“James and his colleagues have pointed out (James, Demaree, Mulaik, & Mumford, 1988; James et al., 1986), the procedure based on the 75% rule (or any percentage cutoff) contains a fallacy known as the “affirming the consequent” fallacy.”
BackgroundInfluentialTwo new procedures for studying validity generalization.
Raju, N. S., & Burke, M. J. (1983) · Journal of Applied Psychology · doi:10.1037//0021-9010.68.3.382 · Cited by 60
Callender, J. C., & Osburn, H. G. (1981) · Journal of Applied Psychology · doi:10.1037/0021-9010.66.3.274 · Cited by 53 · cited 5×
“Fourth, Type I error is inflated in the 75% rule and in some other procedures but controlled for in the Callender and Osburn (1981), chi-square, andU procedures (Sackett et al., 1986; Spector & Levine, 1987).”
BackgroundMethodInfluentialA test of two refinements in procedures for meta-analysis.
Law, K. S., Schmidt, F. L., & Hunter, J. E. (1994) · Journal of Applied Psychology · doi:10.1037/0021-9010.79.6.978 · Cited by 49
Osburn, H. G., Callender, J. C., Greener, J. M., & Ashworth, S. D. (1983) · Journal of Applied Psychology · doi:10.1037/0021-9010.68.1.115 · Cited by 48 · cited 1×
“First, the 75% rule is the most powerful of the commonly used moderator detection techniques, although its power is not high if the difference in population correlations is less than .2, or ifkand meanNare small (e.g., Osburn et al., 1983; Sackett et al., 1986; Spector & Levine, 1987).”
BackgroundNonlinearity of range corrections in meta-analysis: Test of an improved procedure.
Law, K. S., Schmidt, F. L., & Hunter, J. E. (1994) · Journal of Applied Psychology · doi:10.1037/0021-9010.79.3.425 · Cited by 48
Hunter, J. E., & Schmidt, F. L. (1994) · Journal of Applied Psychology · doi:10.1037/0021-9010.79.2.171 · Cited by 43 · cited 4×
“All average percentages were calculated as the reciprocal of the average of the reciprocals as suggested by Hunter and Schmidt (1990). 2.”
MethodInfluentialThe power of the Schmidt and Hunter Additive Model of Validity Generalization.
Kemery, E. R., Mossholder, K. W., & Roth, L. (1987) · Journal of Applied Psychology · doi:10.1037/0021-9010.72.1.30 · Cited by 38
Bootstrap estimates of standard errors in validity generalization.
Switzer, F. S., Paese, P. W., & Drasgow, F. (1992) · Journal of Applied Psychology · doi:10.1037/0021-9010.77.2.123 · Cited by 38 · cited 2×
“…been published there (e.g., Callender & Osburn, 1981; James et al., 1986; Johnson et al., 1995; Law, Schmidt, & Hunter, 1994a, 1994b; Raju & Burke, 1983; Sackett et al., 1987; Schmidt & Hunter, 1977; Spector & Levine, 1987; Switzer et al., 1992; Wanous, Sullivan, & Malinak, 1989; Whitener, 1990).”
MethodAguinis, H., & Whitehead, R. (1997) · Journal of Applied Psychology · doi:10.1037//0021-9010.82.4.528 · Cited by 35 · cited 1×
“Various studies have evaluated one or more of these alternatives (e.g., Aguinis & Whitehead, 1997; Huffcutt & Arthur, 1995; Hunter & Schmidt, 1994; James et al., 1986; Johnson, Mullen, & Salas, 1995; Kemery, Mossholder, & Roth, 1987; Osburn, Callender, Greener, & Ashworth, 1983; Raju, Pappas, &…”
Raju, N. S., Pappas, S. H., & Williams, C. P. (1989) · Journal of Applied Psychology · doi:10.1037/0021-9010.74.6.901 · Cited by 25 · cited 4×
“Fifth, estimation of residual standard deviation and the standard deviation of ρ depends on artifacts corrected for, appropriateness/accuracy of those corrections, esti mation procedure (e.g., Hunter & Schmidt, 1994; Raju et al., 1989; Switzer et al., 1992), and consideration of outliers (e.”
MethodInfluentialValidity generalization: Rejoinder to Schmidt, Hunter, and Raju (1988).
James, L. R., Demaree, R. G., Mulaik, S. A., & Mumford, M. D. (1988) · Journal of Applied Psychology · doi:10.1037/0021-9010.73.4.673 · Cited by 24 · cited 1×
“One is a lack of moderators, but there are others, such as small average sample sizes (James et al., 1988), lack of between-study variability on a given moderator, and relatively small differences between population effect sizes.”
Background(1985)
Research works printed but not linked: 22
- 10.1037/0021-9010.82.4.528 · doi:10.1037/0021-9010.82.4.528 · Paper page
- 10.1037/0021-9010.87.3.583 · doi:10.1037/0021-9010.87.3.583 · Paper page
- 10.1037/0033-2909.88.3.588 · DOI · Paper page
- 10.1037/0021-9010.80.2.327 · doi:10.1037/0021-9010.80.2.327 · Paper page
- 10.1037/0021-9010.81.5.459 · doi:10.1037/0021-9010.81.5.459 · Paper page
- 10.1037/0021-9010.71.3.440 · doi:10.1037/0021-9010.71.3.440 · Paper page
- 10.1037/0021-9010.77.1.3 · doi:10.1037/0021-9010.77.1.3 · Paper page
- 10.1037/0021-9010.80.1.94 · doi:10.1037/0021-9010.80.1.94 · Paper page
- 10.1037/0021-9010.82.1.79 · doi:10.1037/0021-9010.82.1.79 · Paper page
- 10.1037/0033-2909.105.3.430 · DOI · Paper page
- 10.1037/0021-9010.81.6.660 · doi:10.1037/0021-9010.81.6.660 · Paper page
- 10.1037/0021-9010.65.4.373 · doi:10.1037/0021-9010.65.4.373 · Paper page
- 10.1037/0021-9010.81.4.380 · doi:10.1037/0021-9010.81.4.380 · Paper page
- 10.1037/0021-9010.68.3.382 · doi:10.1037/0021-9010.68.3.382 · Paper page
- 10.1037/0021-9010.82.1.30 · DOI · Paper page
- 10.3102/0013189X013008006 · doi:10.3102/0013189x013008006 · Paper page
- Society for Industrial and Organizational Psychology. (2000). 16th Annual Conference call for proposals. Bowling Green, OH: Author.
- 10.1037/0021-9010.72.1.3 · doi:10.1037/0021-9010.72.1.3 · Paper page
- 10.1037/0021-9010.81.5.575 · doi:10.1037/0021-9010.81.5.575 · Paper page
- 10.1037/0021-9010.74.2.259 · doi:10.1037/0021-9010.74.2.259 · Paper page
- 10.1037/0003-066X.53.3.300 · doi:10.1037/0003-066x.53.3.300 · Paper page
- 10.1037/0033-2909.99.3.432 · DOI · Paper page
Cited by
CHRISTIAN, M. S., GARZA, A. S., & SLAUGHTER, J. E. (2011) · Personnel Psychology · doi:10.1111/j.1744-6570.2010.01203.x · Cited by 2,903
“Cortina (2003) suggests that when moderators are present, an appropriate method is to break down the effect sizes into categories and test for differences.”
BackgroundMethodPerceived organizational support: a meta-analytic evaluation of organizational support theory.
Kurtessis, J. N., Eisenberger, R., Ford, M. T., Buffardi, L. C., Stewart, K. A., & Adis, C. S. (2017) · Journal of Management · doi:10.1177/0149206315575554 · Cited by 2,354
“Because caution has been urged in removing outliers (e.g., Cortina, 2003) and these two did not substantially impact results, we retained them.”
Intrinsic motivation and extrinsic incentives jointly predict performance: A 40-year meta-analysis.
Cerasoli, C. P., Nicklin, J. M., & Ford, M. T. (2014) · Psychological Bulletin · doi:10.1037/a0035661 · Cited by 1,981
Refining the relationship between personality and subjective well-being.
Steel, P., Schmidt, J., & Shultz, J. (2008) · Psychological Bulletin · doi:10.1037/0033-2909.134.1.138 · Cited by 1,794
A Review of Self-Determination Theory’s Basic Psychological Needs at Work
Van den Broeck, A., Ferris, D. L., Chang, C.-H., & Rosen, C. C. (2016) · Journal of Management · doi:10.1177/0149206316632058 · Cited by 1,465
“Additional subgroup analyses were performed to examine the effects of a priori moderation effects associated with specific study characteristics (Cortina, 2003) with a significant Q statistic.”
Method
Metadata
- Authors
- Cortina, J. M.
- Affiliations
- George Mason University
- Venue
- Organizational Research Methods
- Publisher
- SAGE Publications
- Published
- 2003-10
- Volume
- 6
- Issue
- 4
- Pages
- 415-439
- ISSN
- 1094-4281 (ISSN-L)
- 1552-7425
- OpenAlex
- W2075116339
- Cited by
- 168
- References
- 47
- Citations by year
- 2026: 6 · 2025: 5 · 2024: 6 · 2023: 7 · 2022: 6 · 2021: 8 · 2020: 6 · 2019: 9 · 2018: 9 · 2017: 10 · 2016: 13 · 2015: 11 · 2014: 11 · 2013: 9 · 2012: 9
- FWCI
- 2.29
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Organizational Research Methods (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
- Keywords
- moderators, heterogeneity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.