The Sources of Four Commonly Reported Cutoff Criteria: What Did They Really Say?
The DOI matches its Crossref record; 128 of the 128 references it cites were found in a registry or an index.
Abstract
Everyone can recite methodological “urban legends” that were taught in graduate school, learned over the years through experience publishing, or perhaps just heard through the grapevine. In this article, the authors trace four widely cited and reported cutoff criteria to their (alleged) original sources to determine whether they really said what they are cited as having said about the cutoff criteria, and if not, what the original sources really said. The authors uncover partial truths in tracing the history of each cutoff criterion and in the end endorse a set of 12 specific guidelines for effective academic referencing provided by Harzing that, if adopted, should help prevent the further perpetuation of methodological urban legends.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“variations considering instruments but, considering average consistency, the models performed above the standard of α > 0.7 adopted by Lance, Butts, and Michels (2006).”
From Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations (2024) · cites this paper as Lance, Butts, and Michels (2006)
In the citing paper
“Lance et al. (2006) pointed out that Nunnally (1978) and Carmines and Zeller (1979) recommended a reliability standard of 0.8 for the majority of studies, and reliability of 0.7 indicates the scale has only modest reliability.”
From Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations (2024) · cites this paper as Lance et al. (2006)
In the citing paper
“Cronbach's alpha of 0.7 only indicates modest reliability (Lance et al., 2006)”
From Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations (2024) · cites this paper as Lance et al., 2006
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788 · cited 4×
“The estimate of within-group interrater reliability provided by James et al. (1984) was used.”
BackgroundMethodInfluentialA Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Structural equation modeling in practice: A review and recommended two-step approach
Anderson, J. C., & Gerbing, D. W. (1988) · Psychological Bulletin · doi:10.1037/0033-2909.103.3.411 · Cited by 41,941 · cited 1×
“In fact, the jury is still out on the much broader question of how to best assess model fit (e.g., Anderson & Gerbing, 1988; Fraas & Newman, 1994; Hayduk & Glaser, 2000; Mulaik et al., 1989; Mulaik & Milsap, 2000; Williams & Holahan, 1994), as assessment of model fit still represents a large and…”
BackgroundAlternative Ways of Assessing Model Fit
Browne, M. W., & Cudeck, R. (1992) · Sociological Methods & Research · doi:10.1177/0049124192021002005 · Cited by 27,200 · cited 1×
“…models, Hoelter (1983) recommended a “critical N,” (the maximum sample size up to which a model cannot be rejected by the χ2 test) of 200, and Browne and Cudeck (1993) wrote that a value of about 0.08 or less for the root mean squared error of approximation would indicate a well-fitting…”
BackgroundInfluentialComparative fit indexes in structural models.
Bentler, P. M. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.107.2.238 · Cited by 25,266 · cited 1×
“In fact, they could not have, as the CFI was first reported 10 years later by Bentler (1990).”
BackgroundIntraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Significance tests and goodness of fit in the analysis of covariance structures.
Bentler, P. M., & Bonett, D. G. (1980) · Psychological Bulletin · doi:10.1037//0033-2909.88.3.588 · Cited by 19,023 · cited 16×
“The four cutoff criteria we discuss are (a) goodness-of-fit indices (GFIs) greater than .90 indicate wellfitting structural equation models (SEMs; Bentler & Bonett, 1980), (b) reliability at or above 202 Organizational Research Methods Volume 9 Number 2 April 2006 202-220 © 2006 Sage Publications…”
BackgroundMethodInfluentialTesting Structural Equation Models.
Mueller, C. W., Bollen, K. A., & Long, J. S. (1994) · Contemporary Sociology · doi:10.2307/2074955 · Cited by 14,120
The job demands-resources model of burnout.
Demerouti, E., Bakker, A. B., Nachreiner, F., & Schaufeli, W. B. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.3.499 · Cited by 13,643
The Scree Test For The Number Of Factors
Cattell, R. B. (1966) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr0102_10 · Cited by 13,554
Research works printed but not linked: 24
- 10.1037/0033-2909.112.3.400 · doi:10.1037/0033-2909.112.3.400 · Paper page
- 10.1037/0033-2909.88.3.588 · DOI · Paper page
- 10.1177/014920639802400202 · doi:10.1177/014920639802400202 · Paper page
- 10.1037/0033-2909.103.2.276 · doi:10.1037/0033-2909.103.2.276 · Paper page
- Dixon, W. J., 1992, BMDP statistical software manual
- 10.1037/0021-9010.75.6.698 · doi:10.1037/0021-9010.75.6.698 · Paper page
- Gorsuch, R. L., 1983, Factor analysis
- Harvey, R. J., meeting of the Society for Industrial and Organizational Psychology
- 10.1037/0022-3514.52.1.134 · doi:10.1037/0022-3514.52.1.134
- Hays, W. H., 1994, Statistics
- 10.1037/0033-2909.103.3.391 · doi:10.1037/0033-2909.103.3.391 · Paper page
- McAnally, M., 1999, Beach house on the moon [CD]
- 10.1037/0033-2909.105.3.430 · DOI · Paper page
- Nunnally, J. C., 1967, Psychometric theory
- Nunnally, J. C., 1978, Psychometric theory
- Nunnally, J. C., 1994, Psychometric theory
- 10.1037/0022-3514.83.3.735 · doi:10.1037/0022-3514.83.3.735 · Paper page
- SAS Institute Inc., 1999, SAS online Doc® [Computer software and manual]
- 10.1037/0021-9010.74.2.368 · doi:10.1037/0021-9010.74.2.368 · Paper page
- SPSS, 1999, SPSS 10.0 syntax reference guide
- 10.1037/0022-0167.34.4.414 · doi:10.1037/0022-0167.34.4.414 · Paper page
- 10.1016/S0149-2063_03_00033-3 · DOI · Paper page
- 10.1037/1082-989X.1.4.354 · doi:10.1037/1082-989x.1.4.354 · Paper page
- 10.1037/0033-2909.99.3.432 · DOI · Paper page
Cited by
Answers to 20 questions about interrater reliability and interrater agreement
LeBreton, J. M., & Senter, J. L. (2008) · Organizational Research Methods · doi:10.1177/1094428106296642 · Cited by 3,608
“The commonly used cut point of .70 is by no means a definitive rule; it is simply a heuristic that is frequently (and inappropriately) invoked in research (Lance et al., 2006).”
BackgroundMethodInfluentialUse Omega Rather than Cronbach’s Alpha for Estimating Reliability. But…
Hayes, A. F., & Coutts, J. J. (2020) · Communication Methods and Measures · doi:10.1080/19312458.2020.1718629 · Cited by 2,675
Cheung, G. W., Cooper-Thomas, H. D., Lau, R. S., & Wang, L. C. (2024) · Asia Pacific Journal of Management · doi:10.1007/s10490-023-09871-y · Cited by 2,262
“Although Cronbach’s alpha greater than 0.7 has been widely used as the standard for adequate reliability, Lance et al. (2006) pointed out that Nunnally (1978) and Carmines and Zeller (1979) recommended a reliability standard of 0.8 for the majority of studies, and reliability of 0.7 indicates the…”
InfluentialRauch, A., & Frese, M. (2007) · European Journal of Work and Organizational Psychology · doi:10.1080/13594320701595438 · Cited by 1,839
Richardson, H. A., Simmering, M. J., & Sturman, M. C. (2009) · Organizational Research Methods · doi:10.1177/1094428109332834 · Cited by 1,721
Metadata
- Authors
- Lance, C. E., Butts, M. M., & Michels, L. C.
- Affiliations
- University of Georgia (US)
- Venue
- Organizational Research Methods
- Publisher
- SAGE Publications
- Published
- 2006-04
- Volume
- 9
- Issue
- 2
- Pages
- 202-220
- ISSN
- 1094-4281 (ISSN-L)
- 1552-7425
- OpenAlex
- W1987634243
- Cited by
- 2,053
- References
- 128
- Citations by year
- 2026: 105 · 2025: 134 · 2024: 153 · 2023: 151 · 2022: 175 · 2021: 161 · 2020: 131 · 2019: 111 · 2018: 136 · 2017: 115 · 2016: 129 · 2015: 107 · 2014: 86 · 2013: 74 · 2012: 56
- FWCI
- 38.46
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Organizational Research Methods (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Business, Management and Accounting › Strategy and Management › Business Strategies and Management Research
- Keywords
- statistical significance thresholds, citation accuracy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Sources of Four Commonly Reported Cutoff Criteria: What Did They Really Say? https://app.cochat.ai/papers/10.1177/1094428105284919
- The paper
- Lance, C. E., Butts, M. M., & Michels, L. C. (2006). The Sources of Four Commonly Reported Cutoff Criteria: What Did They Really Say? Organizational Research Methods. https://doi.org/10.1177/1094428105284919
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.