The Sources of Four Commonly Reported Cutoff Criteria: What Did They Really Say?
The DOI matches its Crossref record; 128 of the 128 references it cites were found in a registry or an index.
Abstract
Everyone can recite methodological “urban legends” that were taught in graduate school, learned over the years through experience publishing, or perhaps just heard through the grapevine. In this article, the authors trace four widely cited and reported cutoff criteria to their (alleged) original sources to determine whether they really said what they are cited as having said about the cutoff criteria, and if not, what the original sources really said. The authors uncover partial truths in tracing the history of each cutoff criterion and in the end endorse a set of 12 specific guidelines for effective academic referencing provided by Harzing that, if adopted, should help prevent the further perpetuation of methodological urban legends.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Lance et al. (2006) pointed out that Nunnally (1978) and Carmines and Zeller (1979) recommended a reliability standard of 0.8 for the majority of studies, and reliability of 0.7 indicates the scale has only modest reliability.”
From Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations (2024) · cites this paper as Lance et al. (2006)
In the citing paper
“Cronbach's alpha of 0.7 only indicates modest reliability (Lance et al., 2006)”
From Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations (2024) · cites this paper as Lance et al., 2006
In the citing paper
“variations considering instruments but, considering average consistency, the models performed above the standard of α > 0.7 adopted by Lance, Butts, and Michels (2006).”
From Are Large Language Models Moral Hypocrites? A Study Based on Moral Foundations (2024) · cites this paper as Lance, Butts, and Michels (2006)
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788 · cited 4×
“The estimate of within-group interrater reliability provided by James et al. (1984) was used.”
BackgroundMethodInfluentialA Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Structural equation modeling in practice: A review and recommended two-step approach
Anderson, J. C., & Gerbing, D. W. (1988) · Psychological Bulletin · doi:10.1037/0033-2909.103.3.411 · Cited by 41,941 · cited 1×
“In fact, the jury is still out on the much broader question of how to best assess model fit (e.g., Anderson & Gerbing, 1988; Fraas & Newman, 1994; Hayduk & Glaser, 2000; Mulaik et al., 1989; Mulaik & Milsap, 2000; Williams & Holahan, 1994), as assessment of model fit still represents a large and…”
BackgroundAlternative Ways of Assessing Model Fit
Browne, M. W., & Cudeck, R. (1992) · Sociological Methods & Research · doi:10.1177/0049124192021002005 · Cited by 27,200 · cited 1×
“…models, Hoelter (1983) recommended a “critical N,” (the maximum sample size up to which a model cannot be rejected by the χ2 test) of 200, and Browne and Cudeck (1993) wrote that a value of about 0.08 or less for the root mean squared error of approximation would indicate a well-fitting…”
BackgroundInfluentialComparative fit indexes in structural models.
Bentler, P. M. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.107.2.238 · Cited by 25,266 · cited 1×
“In fact, they could not have, as the CFI was first reported 10 years later by Bentler (1990).”
BackgroundIntraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 24,532
Significance tests and goodness of fit in the analysis of covariance structures.
Bentler, P. M., & Bonett, D. G. (1980) · Psychological Bulletin · doi:10.1037//0033-2909.88.3.588 · Cited by 19,023 · cited 16×
“The four cutoff criteria we discuss are (a) goodness-of-fit indices (GFIs) greater than .90 indicate wellfitting structural equation models (SEMs; Bentler & Bonett, 1980), (b) reliability at or above 202 Organizational Research Methods Volume 9 Number 2 April 2006 202-220 © 2006 Sage Publications…”
BackgroundMethodInfluentialTesting Structural Equation Models.
Mueller, C. W., Bollen, K. A., & Long, J. S. (1994) · Contemporary Sociology · doi:10.2307/2074955 · Cited by 14,120
The job demands-resources model of burnout.
Demerouti, E., Bakker, A. B., Nachreiner, F., & Schaufeli, W. B. (2001) · Journal of Applied Psychology · doi:10.1037/0021-9010.86.3.499 · Cited by 13,643
The Scree Test For The Number Of Factors
Cattell, R. B. (1966) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr0102_10 · Cited by 13,554
The application of electronic computers to factor analysis
Kaiser, H. F. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000116 · Cited by 12,459 · cited 6×
“…(c) interrater agreement indices at or above .70 justify aggregation of individual-level data to group-level measures (James, Demaree, & Wolf, 1984), and (d) one should keep the number of factors for rotation and interpretation whose eigenvalues are greater than 1.00 (Guttman, 1954; Kaiser, 1960).”
BackgroundInfluentialHu, L.-T., & Bentler, P. M. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.424 · Cited by 12,298 · cited 3×
“It seemed so—at least up until the late 1990s, when Hu and Bentler (1998, 1999) published two studies investigating the effects of sample size, estimation method, violations of multivariate normality, and model misspecification on several popularly reported overall GFIs.”
Background- STATISTICAL PACKAGE FOR SOCIAL SCIENCES
(2014) · Statistical Methods in Psychiatry Research and SPSS · doi:10.1201/b17560-26 · Cited by 12,056
Review of Psychometric theory (2nd ed.).
SITGREAVES, R. (1979) · Contemporary Psychology: A Journal of Reviews · doi:10.1037/018882 · Cited by 11,425
Evaluating the use of exploratory factor analysis in psychological research.
Fabrigar, L. R., Wegener, D. T., MacCallum, R. C., & Strahan, E. J. (1999) · Psychological Methods · doi:10.1037/1082-989x.4.3.272 · Cited by 9,587 · cited 3×
“Still, current “best practice” recommendations are to use multiple factor retention approaches with interpretability of the (usually rotated) factor solution a primary concern (e.g., Fabrigar et al., 1999).”
BackgroundMethodA Rationale and Test for the Number of Factors in Factor Analysis
Horn, J. L. (1965) · Psychometrika · doi:10.1007/bf02289447 · Cited by 8,714 · cited 1×
“James (1982) did not even mention the rwg index, much less any cutoff criterion.”
BackgroundTesting Structural Equation Models.
Clogg, C. C., Bollen, K. A., & Long, J. S. (1995) · Social Forces · doi:10.2307/2580595 · Cited by 8,211
Statistical Theories of Mental Test Scores
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,022
Forming inferences about some intraclass correlation coefficients.
McGraw, K. O., & Wong, S. P. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.1.30 · Cited by 6,903 · cited 1×
“…indices have been used to determine whether more micro-level data are sufficiently homogeneous to justify aggregation, including various versions of the intraclass correlation (James, 1982; McGraw & Wong, 1996; Shrout & Fleiss, 1979) and η2 (Bliese & Halverson, 1998; W. H. Hays, 1994; James, 1982).”
MethodMarsh, H. W., Hau, K.-T., & Wen, Z. (2004) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem1103_2 · Cited by 6,691 · cited 2×
“More recently, however, Marsh, Hau, and Wen (2004) criticized the hypothesis-testing rationale underlying Hu and Bentler’s (1999) work, cautioned against the routine application of their more stringent cutoff criteria, and recommended a return to the χ2 statistic to evaluate model fit.”
BackgroundReliability and Validity Assessment
Carmines, E., & Zeller, R. (1979) · doi:10.4135/9781412985642 · Cited by 5,884 · cited 1×
“Carmines and Zeller (1979) made a similar recommendation: “As a general rule, we believe that reliabilities should not be below .80 for widely used scales” (p. 51).”
BackgroundReliability and Validity Assessment
(2011) · The Encyclopedia of Political Science · doi:10.4135/9781608712434.n1341 · Cited by 5,040
Klein, K. J., & Kozlowski, S. W. J. (2000) · Cited by 4,914
Estimating within-group interrater reliability with and without response bias
James, L. R., Demaree, R. G., & Wolf, G. (1984) · Journal of Applied Psychology · doi:10.1037/0021-9010.69.1.85 · Cited by 4,774 · cited 14×
“The other apparently original source is James’s (1988) chapter on organizational climate in which he discussed the development of the rwg index in great detail, and although he did not explicitly recommend a .70 cutoff for aggregation, he relied on the .70 cutoff extensively as indicating acceptable within-group agree…”
BackgroundMethodInfluentialGoodness-of-fit indexes in confirmatory factor analysis: The effect of sample size.
Marsh, H. W., Balla, J. R., & McDonald, R. P. (1988) · Psychological Bulletin · doi:10.1037//0033-2909.103.3.391 · Cited by 3,824 · cited 3×
“For example, Marsh, Balla, and McDonald (1988) wrote, “They cautioned that the absolute value of these indexes may be difficult to interpret but that values of less than .9 usually mean that the model can be improved upon substantially” (p. 393), and Homer and Kahle (1988) referred to “the Bentler…”
BackgroundMethodInfluentialComparison of five rules for determining the number of components to retain.
Zwick, W. R., & Velicer, W. F. (1986) · Psychological Bulletin · doi:10.1037//0033-2909.99.3.432 · Cited by 3,129 · cited 2×
“In one particularly comprehensive simulation study, Zwick and Velicer (1986) compared five methods for determining the number of components to retain: (a) Bartlett’s (1950) χ2 test, (b) the K1 criterion, (c) Velicer’s (1976) minimum average partial (MAP) correlation procedure, (d) Cattell’s (1966)…”
MethodFactor Retention Decisions in Exploratory Factor Analysis: a Tutorial on Parallel Analysis
Hayton, J. C., Allen, D. G., & Scarpello, V. (2004) · Organizational Research Methods · doi:10.1177/1094428104263675 · Cited by 3,104 · cited 3×
“…documentation of its tendency to retain too many, and oftentimes far too many, factors and (b) repeated admonitions by authors of major review articles to abandon its use.8 The follow-up: PA is an accurate alternative and is not as difficult to implement as some may think (see Hayton et al., 2004).”
BackgroundMethodInfluentialPredicting unit performance by assessing transformational and transactional leadership.
Bass, B. M., Avolio, B. J., Jung, D. I., & Berson, Y. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.2.207 · Cited by 3,026 · cited 1×
“Other authors have aggregated data if a certain percentage of groups reach some criterion on the rwg index (again, usually .70; e.g., Bass et al., 2003; Demerouti, Bakker, Nachreiner, & Schaufeli, 2001), and still others have aggregated data only for those groups that reach some cutoff criterion…”
BackgroundEvaluation of goodness-of-fit indices for structural equation models.
Mulaik, S. A., James, L. R., Van Alstine, J., & Bennett, N. (1989) · Psychological Bulletin · doi:10.1037//0033-2909.105.3.430 · Cited by 2,923 · cited 1×
“…still out on the much broader question of how to best assess model fit (e.g., Anderson & Gerbing, 1988; Fraas & Newman, 1994; Hayduk & Glaser, 2000; Mulaik et al., 1989; Mulaik & Milsap, 2000; Williams & Holahan, 1994), as assessment of model fit still represents a large and controversial area…”
BackgroundBMPD statistical software manual
Dixon, W. (1988) · University of California Press eBooks · Cited by 2,536
Determining the number of components from the matrix of partial correlations
Velicer, W. F. (1976) · Psychometrika · doi:10.1007/bf02293557 · Cited by 2,438
TESTS OF SIGNIFICANCE IN FACTOR ANALYSIS
BARTLETT, M. S. (1950) · British Journal of Statistical Psychology · doi:10.1111/j.2044-8317.1950.tb00285.x · Cited by 2,337 · cited 10×
“Sure! Jum Nunnally (1978) said this, right? Many authors have seemed to think so. For example, Rothbard and Edwards (2003) reported that “all reliabilities exceeded the .70 criterion suggested by Nunnally (1978) and were considered acceptable” (p. 713), and McAllister and Bigley (2002) wrote that “reliability assessme…”
BackgroundMethodInfluentialrwg: an assessment of within-group interrater agreement
James, L. R., Demaree, R. G., & Wolf, G. (1993) · Journal of Applied Psychology · doi:10.1037/0021-9010.78.2.306 · Cited by 2,221 · cited 1×
“…the interpretation of rwg as an IRR coefficient; Kozlowski and Hattrup (1992) argued that rwg is properly interpreted as an IRA index, not an IRR coefficient; James, Demaree, and Wolf (1993) concurred; and rwg is now more widely (and correctly) interpreted as an IRA index, not an IRR coefficient.”
BackgroundAggregation bias in estimates of perceptual agreement.
James, L. R. (1982) · Journal of Applied Psychology · doi:10.1037/0021-9010.67.2.219 · Cited by 2,139 · cited 6×
“…ratings from an individual to a group level of analysis” (p. 211), and Susskind, Kacmar, and Borchgrevink (2003) reported that “responses from the remaining 26 organizations that aggregated to the organizational level exceeded the recommended cutoff of .60 offered by James (1982)” (p. 82).”
BackgroundMethodInfluentialComparative Fit Indices in Structural Models
Bentler, P. M. (1990) · eScholarship (California Digital Library) · Cited by 2,044
A structural equation test of the value-attitude-behavior hierarchy.
Homer, P. M., & Kahle, L. R. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.54.4.638 · Cited by 1,883 · cited 2×
“…value of these indexes may be difficult to interpret but that values of less than .9 usually mean that the model can be improved upon substantially” (p. 393), and Homer and Kahle (1988) referred to “the Bentler and Bonett (1980) heuristic that model fits of less than .90 are inadequate” (p. 643).”
BackgroundMethodSome necessary conditions for common-factor analysis
Guttman, L. (1954) · Psychometrika · doi:10.1007/bf02289162 · Cited by 1,689 · cited 2×
“Guttman (1954) showed that the minimum rank of the correlation matrix with communalities in the diagonal (RC = R – U2, where R is the unadjusted correlation matrix and U2 is the diagonal matrix of unique variances) is greater than or equal to the number of eigenvalues ≥1.00 when the eigenvalues are…”
BackgroundInfluentialOn the fit of models to covariances and methodology to the Bulletin.
Bentler, P. M. (1992) · Psychological Bulletin · doi:10.1037//0033-2909.112.3.400 · Cited by 1,657 · cited 1×
“Even Peter Bentler (1992) himself, reflecting on his 1980 paper with Bonett and referring to the normed fit index (NFI) proposed therein, reported that “higher values indicate greater covariation accounted for, with excellent models having NFI values above .90 or so” (p. 401).”
BackgroundThe Analysis of Covariance Structures: Goodness-of-Fit Indices
HOELTER, J. W. (1983) · Sociological Methods & Research · doi:10.1177/0049124183011003003 · Cited by 1,624
Multivariate Analysis with Latent Variables: Causal Modeling
Bentler, P. M. (1980) · Annual Review of Psychology · doi:10.1146/annurev.ps.31.020180.002223 · Cited by 1,619 · cited 1×
“…(i.e., least squares, generalized least squares, and maximum likelihood), proper interpretation of the overall χ2 test of goodness of fit, the χ2 statistic’s dependency on sample size, tests of hierarchically nested models and χ2 tests, and incremental fit indices (see also Bentler, 1980).”
MethodPersonality, affect, and behavior in groups
George, J. M. (1990) · Journal of Applied Psychology · doi:10.1037/0021-9010.75.2.107 · Cited by 1,372 · cited 2×
“%) cited James et al. to justify their use of rwg but cited George (1990, or some other paper that then cited George, 1990) to document their use of the .70 cutoff, (c) 12 (15”
BackgroundUses of factor analysis in counseling psychology research.
Tinsley, H. E. A., & Tinsley, D. J. (1987) · Journal of Counseling Psychology · doi:10.1037//0022-0167.34.4.414 · Cited by 1,324
Contingencies of Self-Worth in College Students: Theory and Measurement.
Crocker, J., Luhtanen, R. K., Cooper, M. L., & Bouvrette, A. (2003) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.85.5.894 · Cited by 1,018 · cited 2×
“Second, and contrary to many authors’attributions (e.g., Ambrose & Schminke, 2003; Crocker et al., 2003; Posig & Kickul, 2003; Postmes & Branscombe, 2002; Tjosvold, Hui, & Yu, 2003; Thill, Holmbeck, & Bryant, 2003; Van der Vegt et al., 2001), Bentler and Bonett (1980) had nothing to say about the…”
BackgroundResultA Review and Evaluation of Exploratory Factor Analysis Practices in Organizational Research
Conway, J. M., & Huffcutt, A. I. (2003) · Organizational Research Methods · doi:10.1177/1094428103251541 · Cited by 938 · cited 2×
“…use Kaiser’s (1960) eigenvalues-greater-than-1 criterion, which is generally recognized as inadequate” (Wood, Tataryn, & Gorsuch, 1996, p. 260; see also Conway & Huffcutt, 2003) despite admonitions to abandon its use, for example, “we know of no study of this rule that shows it to work well”…”
BackgroundMethodInfluentialGeorge, J. M., & Bettenhausen, K. (1990) · Journal of Applied Psychology · doi:10.1037//0021-9010.75.6.698 · Cited by 917 · cited 2×
“George and Bettenhausen (1990) cited the same personal communication with James.”
BackgroundInfluentialVelicer, W. F., Eaton, C. A., & Fava, J. L. (2000) · Problems and Solutions in Human Assessment · doi:10.1007/978-1-4615-4397-8_3 · Cited by 866 · cited 1×
“Recently, Velicer, Eaton, and Fava (2000) compared the accuracy of the K1 criterion, six different implementations of PA, and three different implementations of MAP and concluded that (a) although the K1 criterion “is the easiest decision method to implement” (p. 67), it was also “the least…”
インターグループ (1986) · Medical Entomology and Zoology · Cited by 817
Why do ‘great minds’ think alike?: antecedents of team member schema agreement
Rentsch, J. R., & Klimoski, R. J. (2001) · Journal of Organizational Behavior · doi:10.1002/job.81 · Cited by 770 · cited 1×
“…criterion on the rwg index (again, usually .70; e.g., Bass et al., 2003; Demerouti, Bakker, Nachreiner, & Schaufeli, 2001), and still others have aggregated data only for those groups that reach some cutoff criterion (Aryee, Chen, & Budhwar, 2004; Rentsch & Klimoski, 2001; Susskind et al., 2003).”
BackgroundComponent Analysis versus Common Factor Analysis: Some issues in Selecting an Appropriate Procedure
Velicer, W. F., & Jackson, D. N. (1990) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2501_1 · Cited by 738 · cited 2×
“Discrepancies are rarely, if ever, of any practical importance in subsequent interpretations” (Velicer & Jackson, 1990, p. 5).”
BackgroundGlorfeld, L. W. (1995) · Educational and Psychological Measurement · doi:10.1177/0013164495055003002 · Cited by 729 · cited 3×
“…the presentation of tabled values for roots of random data correlation matrices (Lautenschlager, 1989) and embellishments to Horn’s (1965) PA to the analysis of multiple correlation matrices and averaging of eigenvalues across the matrices (e.g., Glorfeld, 1995; Hayton, Allen, & Scarpello, 2004).”
BackgroundMethodSchilling, M. A. (2002) · Academy of Management Journal · doi:10.2307/3069353 · Cited by 663
Foundations of Factor Analysis
Mulaik, S. A. (2009) · Chapman & Hall/CRC eBooks · doi:10.1201/b15851 · Cited by 639
Joint Impact of Interdependence and Group Diversity on Innovation
VanderVegt, G. (2003) · Journal of Management · doi:10.1016/s0149-2063(03)00033-3 · Cited by 580 · cited 1×
“…authors have reported the average or median rwg for their groups and aggregated lower-level data to group-level indexes if the median or mean rwg reaches some criterion (usually .70; e.g., Bunderson, 2003; Kirkman, Tesluk, & Rosen, 2001; Patterson, West, & Wall, 2004; Van der Vegt & Janssen, 2003).”
BackgroundAmbrose, M. L., & Schminke, M. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.2.295 · Cited by 518 · cited 10×
“Similarly, Ambrose and Schminke (2003) reported that “in general . . . IFI and CFI scores above .90 (Bentler & Bonnett [sic], 1990 [sic]) indicate a good model fit” (p. 299). Crocker, Luhtanen, and Cooper (2003) wrote that “values > .90 for the NNFI and CFI indicate . . . acceptable model fit (Bentler & Bonett, 1980)”…”
BackgroundResultInfluentialWhich comes first: Employee attitudes or organizational financial and market performance?
Schneider, B., Hanges, P. J., Smith, D. B., & Salvaggio, A. N. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.5.836 · Cited by 516
A disagreement about within-group agreement: disentangling issues of consistency versus consensus
Kozlowski, S. W., & Hattrup, K. (1992) · Journal of Applied Psychology · doi:10.1037/0021-9010.77.2.161 · Cited by 514 · cited 1×
“But by and by, Schmidt and Hunter (1989) criticized the interpretation of rwg as an IRR coefficient; Kozlowski and Hattrup (1992) argued that rwg is properly interpreted as an IRA index, not an IRR coefficient; James, Demaree, and Wolf (1993) concurred; and rwg is now more widely (and correctly)…”
BackgroundFactors Influencing Four Rules For Determining The Number Of Components To Retain
Zwick, W. R., & Velicer, W. F. (1982) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr1702_5 · Cited by 512
The eigenvalues-greater-than-one rule and the reliability of components.
Cliff, N. (1988) · Psychological Bulletin · doi:10.1037//0033-2909.103.2.276 · Cited by 488 · cited 2×
“Cliff (1988) offered a third, informal rationale “that a component is of little interest if it accounts for less variance than a single variable does” (p. 276).”
BackgroundMethodSusskind, A. M., Kacmar, K. M., & Borchgrevink, C. P. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.1.179 · Cited by 426
Common Factor Analysis versus Component Analysis: Some Well and Little Known Facts
Gorsuch, R. L. (1990) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2501_3 · Cited by 418 · cited 1×
“The second is that “the rules for determining the number of the factors are the same for components as for common factor analysis” (Gorsuch, 1990, p. 33).”
BackgroundWho stole the money, and when? Individual and situational determinants of employee theft
Greenberg, J. (2002) · Organizational Behavior and Human Decision Processes · doi:10.1016/s0749-5978(02)00039-0 · Cited by 403 · cited 1×
“…multiple item scale. . . . Values of 0.70 or higher are considered to be indicators of good agreement among raters (e.g., George & Bettenhausen, 1990)” (pp. 805- 806), and Greenberg (2002) wrote that “interrater reliability exceeded the minimum standards specified by Gibbs et al. (1992)” (p. 992).”
BackgroundInfluentialLOCUS OF CONTROL AND WELL-BEING AT WORK: HOW GENERALIZABLE ARE WESTERN FINDINGS?
Spector, P. E., Cooper, C. L., Sánchez, J. I., O’Driscoll, M. P., Sparks, K., Bernin, P., Büssing, A., Dewe, P. J., Hart, P., Lu, L., et al. (2002) · Academy of Management Journal · doi:10.5465/3069359 · Cited by 394 · cited 1×
“Also, Spector et al. (2002) reported that “these scales maintained adequate internal consistency reliabilities as assessed with the widely accepted .70 coefficient alpha standard (Nunnally, 1978)” (p. 458).”
BackgroundSackett, P. R., Schmitt, N., Ellingson, J. E., & Kabin, M. B. (2001) · American Psychologist · doi:10.1037/0003-066x.56.4.302 · Cited by 387
Assessment of Fit in Overidentified Models with Latent Variables
WHEATON, B. (1987) · Sociological Methods & Research · doi:10.1177/0049124187016001005 · Cited by 357 · cited 1×
“For example, Wheaton (1987) advocated χ2/df ratios in the range of 2 to 5 as indicating well-fitting models, Hoelter (1983) recommended a “critical N,” (the maximum sample size up to which a model cannot be rejected by the χ2 test) of 200, and Browne and Cudeck (1993) wrote that a value of about…”
MethodEffects of under- and overextraction on principal axis factor analysis with varimax rotation.
Wood, J. M., Tataryn, D. J., & Gorsuch, R. L. (1996) · Psychological Methods · doi:10.1037//1082-989x.1.4.354 · Cited by 354 · cited 1×
“In all fairness, it is a greater mistake to retain too few factors than it is to retain too many (Fava & Velicer, 1992, 1996; Wood et al., 1996), but this alone is not justification for using the K1 criterion.”
Lautenschlager, G. J. (1989) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2403_6 · Cited by 337 · cited 2×
“Other efforts included the presentation of tabled values for roots of random data correlation matrices (Lautenschlager, 1989) and embellishments to Horn’s (1965) PA to the analysis of multiple correlation matrices and averaging of eigenvalues across the matrices (e.g., Glorfeld, 1995; Hayton,…”
MethodVAN DER VEGT, G. S., EMANS, B. J. M., & VAN DE VLIERT, E. (2001) · Personnel Psychology · doi:10.1111/j.1744-6570.2001.tb00085.x · Cited by 326 · cited 2×
“And Van der Vegt, Emans, and Van der Vliert (2001) reported that the “CFI has been found to outperform many fit indices in simulation research and should ideally be greater than or equal to .90 (Bentler & Bonett, 1980)” (p. 60).”
BackgroundMoral Maturity: Measuring the Development of Sociomoral Reflection
Gibbs, J. C., Basinger, K. S., & Fuller, D. (1991) · Cited by 316
Mulaik, S. A., & Millsap, R. E. (2000) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem0701_02 · Cited by 299 · cited 1×
“…broader question of how to best assess model fit (e.g., Anderson & Gerbing, 1988; Fraas & Newman, 1994; Hayduk & Glaser, 2000; Mulaik et al., 1989; Mulaik & Milsap, 2000; Williams & Holahan, 1994), as assessment of model fit still represents a large and controversial area within the current SEM…”
BackgroundInterrater Agreement Reconsidered: An Alternative to the rwg Indices
Brown, R. D., & Hauenstein, N. M. A. (2005) · Organizational Research Methods · doi:10.1177/1094428105275376 · Cited by 297 · cited 1×
“LeBreton et al. also proposed a new analysis-of-variance–based rWGp index for assessing IRA that allows for multiple target true scores, and Brown and Hauenstein (2005) presented a family of IRA indices called awg that are derived from extensions of J. Cohen’s (1960) kappa.”
BackgroundDoes Work–Home Interference mediate the relationship between workload and well-being?
Geurts, S. A. E., Kompier, M. A. J., Roxburgh, S., & Houtman, I. L. D. (2003) · Journal of Vocational Behavior · doi:10.1016/s0001-8791(02)00025-8 · Cited by 293
Probst, T. M. (2003) · Journal of Occupational and Organizational Psychology · doi:10.1348/096317903322591587 · Cited by 271 · cited 1×
“…Houtman (2003) reported (in reference to the TLI) that “values less than .90 usually mean that the model can be improved substantially” (p. 543), and Probst (2003) wrote that “a general rule of thumb states that models with NNFIs below .90 can be improved upon substantially (Bentler & Bonett,…”
BackgroundHoukes, I., Janssen, P. P. M., de Jonge, J., & Bakker, A. B. (2003) · Journal of Occupational and Organizational Psychology · doi:10.1348/096317903322591578 · Cited by 269
Montanelli, R. G., & Humphreys, L. G. (1976) · Psychometrika · doi:10.1007/bf02293559 · Cited by 268 · cited 1×
“…eigenvalues that would be obtained from the analysis of a given number of random variables on a given sample size (e.g., Allen & Hubbard, 1986; Montenelli & Humphreys, 1976; Longman, Holden, & Fekken, 1991), against which researchers could compare eigenvalues from their real data to conduct…”
MethodWho stole feminism? : how women have betrayed women
Sommers, C. H. (1995) · Cited by 261
WRIGHT, P. M., GARDNER, T. M., MOYNIHAN, L. M., PARK, H. J., GERHART, B., & DELERY, J. E. (2001) · Personnel Psychology · doi:10.1111/j.1744-6570.2001.tb00235.x · Cited by 260 · cited 1×
“Others have noted similar misinterpretations (e.g., A. Cohen et al., 2001; LeBreton et al., 2003; Wright et al., 2001).”
Bunderson, J. S. (2003) · Academy of Management Journal · doi:10.2307/30040638 · Cited by 257 · cited 1×
“…authors have reported the average or median rwg for their groups and aggregated lower-level data to group-level indexes if the median or mean rwg reaches some criterion (usually .70; e.g., Bunderson, 2003; Kirkman, Tesluk, & Rosen, 2001; Patterson, West, & Wall, 2004; Van der Vegt & Janssen, 2003).”
BackgroundThe Correction for Attenuation
Muchinsky, P. M. (1996) · Educational and Psychological Measurement · doi:10.1177/0013164496056001004 · Cited by 255
A Revised Index of Interrater Agreement for Multi-Item Ratings of a Single Target
Lindell, M. K., Brandt, C. J., & Whitney, D. J. (1999) · Applied Psychological Measurement · doi:10.1177/01466219922031257 · Cited by 255 · cited 1×
“Furthermore, Lindell et al. (1999) showed that as J (the number of items in the composite scale) increases, rwg(J) and rwg(J)-MV remain very high even though responses tend toward a random pattern.”
BackgroundINVESTMENT IN WORK AND FAMILY ROLES: A TEST OF IDENTITY AND UTILITARIAN MOTIVES
ROTHBARD, N. P., & EDWARDS, J. R. (2003) · Personnel Psychology · doi:10.1111/j.1744-6570.2003.tb00755.x · Cited by 255 · cited 1×
“For example, Rothbard and Edwards (2003) reported that “all reliabilities exceeded the .70 criterion suggested by Nunnally (1978) and were considered acceptable” (p. 713), and McAllister and Bigley (2002) wrote that “reliability assessments for all scales exceeded the minimum standard of .70…”
BackgroundLebreton, J. M., Burgess, J. R. D., Kaiser, R. B., Atchley, E. K., & James, L. R. (2003) · Organizational Research Methods · doi:10.1177/1094428102239427 · Cited by 254 · cited 4×
“Others have noted similar misinterpretations (e.g., A. Cohen et al., 2001; LeBreton et al., 2003; Wright et al., 2001).”
BackgroundMethodResultInfluentialParsimony‐based fit indices for multiple‐indicator models: Do they work?
Williams, L. J., & Holahan, P. J. (1994) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705519409539970 · Cited by 253 · cited 1×
“…question of how to best assess model fit (e.g., Anderson & Gerbing, 1988; Fraas & Newman, 1994; Hayduk & Glaser, 2000; Mulaik et al., 1989; Mulaik & Milsap, 2000; Williams & Holahan, 1994), as assessment of model fit still represents a large and controversial area within the current SEM literature.”
BackgroundData analytic methods for the analysis of multilevel questions
Castro, S. L. (2002) · The Leadership Quarterly · doi:10.1016/s1048-9843(01)00105-9 · Cited by 250 · cited 3×
“For example, Castro (2003) wrote that “a .70 criterion has been commonly used . . . but adequate support and justification for this value has not been provided” (p. 73).”
BackgroundAryee, S., Chen, Z. X., & Budhwar, P. S. (2004) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2004.03.002 · Cited by 247
McAllister, D. J., & Bigley, G. A. (2002) · Academy of Management Journal · doi:10.2307/3069320 · Cited by 245 · cited 1×
“…Edwards (2003) reported that “all reliabilities exceeded the .70 criterion suggested by Nunnally (1978) and were considered acceptable” (p. 713), and McAllister and Bigley (2002) wrote that “reliability assessments for all scales exceeded the minimum standard of .70 suggested by Nunnally (1978)”…”
BackgroundWho Stole Feminism? How Women Have Betrayed Women
Davison, C. M. (1995) · Canadian women's studies · Cited by 243
Group size and measures of group-level properties: An examination of eta-squared and ICC values
Bliese, P. (1998) · Journal of Management · doi:10.1016/s0149-2063(99)80058-0 · Cited by 241 · cited 1×
“…indices have been used to determine whether more micro-level data are sufficiently homogeneous to justify aggregation, including various versions of the intraclass correlation (James, 1982; McGraw & Wong, 1996; Shrout & Fleiss, 1979) and η2 (Bliese & Halverson, 1998; W. H. Hays, 1994; James, 1982).”
MethodPostmes, T., & Branscombe, N. R. (2002) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.83.3.735 · Cited by 232 · cited 2×
“Second, and contrary to many authors’attributions (e.g., Ambrose & Schminke, 2003; Crocker et al., 2003; Posig & Kickul, 2003; Postmes & Branscombe, 2002; Tjosvold, Hui, & Yu, 2003; Thill, Holmbeck, & Bryant, 2003; Van der Vegt et al., 2001), Bentler and Bonett (1980) had nothing to say about the…”
BackgroundResultLOCUS OF CONTROL AND WELL-BEING AT WORK: HOW GENERALIZABLE ARE WESTERN FINDINGS?
Spector, P. E., Cooper, C. L., Sanchez, J. I., O'Driscoll, M., Sparks, K., Bernin, P., Bossing, A., Dewe, P., Hart, P., Lu, L., et al. (2002) · Academy of Management Journal · doi:10.2307/3069359 · Cited by 228
Hodgkinson, G. P., & Sadler‐Smith, E. (2003) · Journal of Occupational and Organizational Psychology · doi:10.1348/096317903765913722 · Cited by 222 · cited 1×
“….90 cutoff, including ∆χ2 tests of nested models (e.g., Houkes, Janssen, & de Jonge, 2003; Plomplun & Omar, 2003), the normed fit index (NFI; e.g., Hodgkinson & Sadler-Smith, 2003; Penley, Wiebe, & Nwosu, 2003), the χ2 statistic (e.g., McCue, Martin, & Buchanan, 2003), and its sensitivity to…”
Dunlap, W. P., Burke, M. J., & Smith-Crowe, K. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.2.356 · Cited by 219
Integrated manufacturing, empowerment, and company performance
Patterson, M. G., West, M. A., & Wall, T. D. (2004) · Journal of Organizational Behavior · doi:10.1002/job.261 · Cited by 194
Harzing, A. (2002) · Journal of Organizational Behavior · doi:10.1002/job.125 · Cited by 188
Jiving the Four-Step, Waltzing Around Factor Analysis, and Other Serious Fun
Hayduk, L. A., & Glaser, D. N. (2000) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem0701_01 · Cited by 179 · cited 1×
“In fact, the jury is still out on the much broader question of how to best assess model fit (e.g., Anderson & Gerbing, 1988; Fraas & Newman, 1994; Hayduk & Glaser, 2000; Mulaik et al., 1989; Mulaik & Milsap, 2000; Williams & Holahan, 1994), as assessment of model fit still represents a large and…”
BackgroundStatistical properties of the rWG(J) index of agreement.
Cohen, A., Doveh, E., & Eick, U. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.3.297 · Cited by 166 · cited 5×
“Others have noted similar misinterpretations (e.g., A. Cohen et al., 2001; LeBreton et al., 2003; Wright et al., 2001).”
BackgroundInfluentialCONFLICT MANAGEMENT AND TASK REFLEXIVITY FOR TEAM IN‐ROLE AND EXTRA‐ROLE PERFORMANCE IN CHINA
Tjosvold, D., Hui, C., & Yu, Z. (2003) · International Journal of Conflict Management · doi:10.1108/eb022895 · Cited by 165
A Monte Carlo Approach to the Number of Factors Problem
Linn, R. L. (1968) · Psychometrika · doi:10.1007/bf02289675 · Cited by 159 · cited 1×
“Early simulation studies (e.g., Browne, 1968; Linn, 1968) found that the eigenvalues-greater-than-1.”
BackgroundKIRKMAN, B. L., TESLUK, P. E., & ROSEN, B. (2001) · Personnel Psychology · doi:10.1111/j.1744-6570.2001.tb00226.x · Cited by 147
The Effects of Overextraction on Factor and Component Analysis
Fava, J. L., & Velicer, W. F. (1992) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2703_5 · Cited by 147 · cited 1×
“In all fairness, it is a greater mistake to retain too few factors than it is to retain too many (Fava & Velicer, 1992, 1996; Wood et al., 1996), but this alone is not justification for using the K1 criterion.”
Levels of analysis issues in group psychology: Using efficacy as an example of a multilevel model.
Moritz, S. E., & Watson, C. B. (1998) · Group Dynamics Theory Research and Practice · doi:10.1037/1089-2699.2.4.285 · Cited by 130 · cited 1×
“(George, 1990, p. 110) As a second example, Moritz and Watson (1998) posed the question, “Just how much interrater agreement is enough?” and answered it by stating that “guidelines from classical test theory can be applied to inform the judgment of whether an observed level of interrater agreement…”
BackgroundExtending our understanding of burnout: Test of an integrated model in nonservice occupations.
Posig, M., & Kickul, J. (2003) · Journal of Occupational Health Psychology · doi:10.1037/1076-8998.8.1.3 · Cited by 127 · cited 1×
“Second, and contrary to many authors’attributions (e.g., Ambrose & Schminke, 2003; Crocker et al., 2003; Posig & Kickul, 2003; Postmes & Branscombe, 2002; Tjosvold, Hui, & Yu, 2003; Thill, Holmbeck, & Bryant, 2003; Van der Vegt et al., 2001), Bentler and Bonett (1980) had nothing to say about the…”
ResultInterrater reliability coefficients cannot be computed when only one stimulus is rated.
Schmidt, F. L., & Hunter, J. E. (1989) · Journal of Applied Psychology · doi:10.1037//0021-9010.74.2.368 · Cited by 123
Situational specificity in assessment center ratings: A confirmatory factor analysis.
Bycio, P., Alvares, K. M., & Hahn, J. (1987) · Journal of Applied Psychology · doi:10.1037/0021-9010.72.3.463 · Cited by 123 · cited 1×
“…to substantiate the claim that “models with delta values of less than .90 should not be accepted, because such models can often be improved” (p. 137); Bycio, Alvares, and Hahn (1987) cited them to support the idea that “rho . . . values of .90 or higher reflect a reasonable model” (p. 466); and…”
BackgroundMohammed, S., Mathieu, J. E., & ‘Bart’ Bartlett, A. L. (2002) · Journal of Organizational Behavior · doi:10.1002/job.169 · Cited by 118 · cited 1×
“For example, Mohammed, Mathieu, and Bartlett (2002) reported that “estimates were computed using James, Demaree, and Wolf’s (1984) within-group agree- ment index (rwg) for a multiple item scale. . . . Values of 0.70 or higher are considered to be indicators of good agreement among raters (e.g.,…”
MethodInfluentialRecent Issues Regarding rWG, rWG, rWG(J), and rWG(J)
LeBreton, J. M., James, L. R., & Lindell, M. K. (2005) · Organizational Research Methods · doi:10.1177/1094428104272181 · Cited by 118 · cited 1×
“Also, LeBreton, James, and Lindell (2005) have shown that rwg(J) need not be derived as a Spearman-Brown adjusted rwg and that negative values for rwg can be avoided by opting for an alternative structural model of IRA.”
BackgroundGrawitch, M. J., Munz, D. C., Elliott, E. K., & Mathis, A. (2003) · Group Dynamics: Theory, Research, and Practice · doi:10.1037/1089-2699.7.3.200 · Cited by 112
Psychometric Properties of the Spanish Beck Depression Inventory-II in a Medical Sample.
Penley, J. A., Wiebe, J. S., & Nwosu, A. (2003) · Psychological Assessment · doi:10.1037/1040-3590.15.4.569 · Cited by 108
A test of the context dependency of three causal models of halo rater error.
Lance, C. E., LaPointe, J. A., & Stewart, A. M. (1994) · Journal of Applied Psychology · doi:10.1037/0021-9010.79.3.332 · Cited by 95 · cited 2×
“Interestingly, although the text of Nunnally’s (1978) “Standards of Reliability” section has remained relatively unchanged over its three editions, he did raise the bar for reliability of measures in “early stages of research” from “.60 or .50” (p. 226) in 1967 to .70 in 1978 and maintained the .70 standard in Nunnall…”
BackgroundSelf-citationThe Effects of Underextraction in Factor and Component Analyses
Fava, J. L., & Velicer, W. F. (1996) · Educational and Psychological Measurement · doi:10.1177/0013164496056006001 · Cited by 87 · cited 1×
“In all fairness, it is a greater mistake to retain too few factors than it is to retain too many (Fava & Velicer, 1992, 1996; Wood et al., 1996), but this alone is not justification for using the K1 criterion.”
Measuring Interrater Agreement for Ratings of a Single Target
Lindell, M. K., & Brandt, C. J. (1997) · Applied Psychological Measurement · doi:10.1177/01466216970213006 · Cited by 86
Structural-equation models of current drug use: Are appropriate models so simple(x)?
Hays, R. D., Widaman, K. F., DiMatteo, M. R., & Stacy, A. W. (1987) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.52.1.134 · Cited by 76
Construct validity of multiple job performance measures using confirmatory factor analysis.
Vance, R. J., MacCallum, R. C., Coovert, M. D., & Hedge, J. W. (1988) · Journal of Applied Psychology · doi:10.1037/0021-9010.73.1.74 · Cited by 74
Effects of the terrorist attacks of 9/11/01 on employee attitudes.
Ryan, A. M., West, B. J., & Carr, J. Z. (2003) · Journal of Applied Psychology · doi:10.1037/0021-9010.88.4.647 · Cited by 70
Assessor Cognitive Processes in an Operational Assessment Center.
Lance, C. E., Foster, M. R., Gentry, W. A., & Thoresen, J. D. (2004) · Journal of Applied Psychology · doi:10.1037/0021-9010.89.1.22 · Cited by 68
Self-citationRESEARCH MODELS OF THE FUTURE FOR INDUSTRIAL AND ORGANIZATIONAL PSYCHOLOGY1
Barrett, G. V. (1972) · Personnel Psychology · doi:10.1111/j.1744-6570.1972.tb01086.x · Cited by 58 · cited 8×
“Another 22 articles (69%) explicitly attributed the idea that GFIs (including the NFI, TLI, CFI, and various other GFIs) greater than .90 indicate a well-fitting SEM to Bentler and Bonett. Only 2 (6%) of the articles that cited the Bentler and Bonett (1980) article did so accurately: Geurts, Kompier, Roxburgh, and Hou…”
BackgroundInfluentialAllen, S. J., & Hubbard, R. (1986) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr2103_7 · Cited by 52 · cited 1×
“…the first several eigenvalues that would be obtained from the analysis of a given number of random variables on a given sample size (e.g., Allen & Hubbard, 1986; Montenelli & Humphreys, 1976; Longman, Holden, & Fekken, 1991), against which researchers could compare eigenvalues from their…”
BackgroundAllen, S. J., & Hubbard, R. (1986) · Multivariate Behavioral Research · Cited by 46
McCue, P., Martin, C., Buchanan, T., Rodgers, J., & Scholey, A. (2003) · Psychology, Health & Medicine · doi:10.1080/1354850310001604568 · Cited by 41
Organizational climate: Another look at a potentially important construct.
James, L. R. (1982) · doi:10.4324/9781315808086-27 · Cited by 38 · cited 2×
“Unfortunately, James’s early work on rwg contributed to this confusion by referring to rwg as an IRR index (e.g., James, 1988; James et al., 1984).”
BackgroundMethodApplications of interactionist psychology : essays in honor of Saul B. Sells
Sells, S. B., Cole, S. G., Demaree, R. G., & Curtis, B. (1988) · Medical Entomology and Zoology · Cited by 37
Statistical properties of the r-sub(WG(J) ) index of agreement.
Cohen, A., Doveh, E., & Eick, U. (2001) · Psychological Methods · doi:10.1037//1082-989x.6.3.297 · Cited by 32 · cited 5×
“Others have noted similar misinterpretations (e.g., A. Cohen et al., 2001; LeBreton et al., 2003; Wright et al., 2001).”
BackgroundInfluentialEnmeshed Lives: Adult Children With Developmental Disabilities and Their Aging Mothers.
Pruchno, R. A. (2003) · Psychology and Aging · doi:10.1037/0882-7974.18.4.851 · Cited by 28 · cited 1×
“…de Jonge, 2003; Plomplun & Omar, 2003), the normed fit index (NFI; e.g., Hodgkinson & Sadler-Smith, 2003; Penley, Wiebe, & Nwosu, 2003), the χ2 statistic (e.g., McCue, Martin, & Buchanan, 2003), and its sensitivity to sample size (e.g., Pruchno, 2003; Ryan, West, & Carr, 2003); the remaining 30 (55”
Thill, A. D. W., Holmbeck, G. N., Bryant, F. B., Nelson, C., Skocic, A., & Uli, N. (2003) · Journal of Personality Assessment · doi:10.1207/s15327752jpa8102_03 · Cited by 24
Fraas, J. W., & Newman, I. (1994) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519409539978 · Cited by 17 · cited 1×
“In fact, the jury is still out on the much broader question of how to best assess model fit (e.g., Anderson & Gerbing, 1988; Fraas & Newman, 1994; Hayduk & Glaser, 2000; Mulaik et al., 1989; Mulaik & Milsap, 2000; Williams & Holahan, 1994), as assessment of model fit still represents a large and…”
BackgroundEstimation of Quantiles for the Sampling Distribution of the rWG within Group Agreement Index
Charnes, J. M., & Schriesheim, C. A. (1995) · Educational and Psychological Measurement · doi:10.1177/0013164495055003007 · Cited by 12
A Note on Lower Bounds for the Number of Common Factors
Browne, M. W. (1968) · Psychometrika · doi:10.1007/bf02290155 · Cited by 10 · cited 1×
“Early simulation studies (e.g., Browne, 1968; Linn, 1968) found that the eigenvalues-greater-than-1.”
BackgroundAnomalies in the Allen and Hubbard Parallel Analysis Procedure
Cota, A. A., Longman, R. S., Holden, R. R., & Fekken, G. C. (1991) · Applied Psychological Measurement · doi:10.1177/014662169101500111 · Cited by 7
Do Minority Representative Reading Passages Provide Factorially Invariant Scores for All Students?
Pomplun, M. R., & Omar, M. H. (2003) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1207/s15328007sem1002_7 · Cited by 6 · cited 1×
“…to issues discussed by Bentler and Bonett other than the .90 cutoff, including ∆χ2 tests of nested models (e.g., Houkes, Janssen, & de Jonge, 2003; Plomplun & Omar, 2003), the normed fit index (NFI; e.g., Hodgkinson & Sadler-Smith, 2003; Penley, Wiebe, & Nwosu, 2003), the χ2 statistic (e.g.,…”
Background
Research works printed but not linked: 24
- 10.1037/0033-2909.112.3.400 · doi:10.1037/0033-2909.112.3.400 · Paper page
- 10.1037/0033-2909.88.3.588 · DOI · Paper page
- 10.1177/014920639802400202 · doi:10.1177/014920639802400202 · Paper page
- 10.1037/0033-2909.103.2.276 · doi:10.1037/0033-2909.103.2.276 · Paper page
- Dixon, W. J., 1992, BMDP statistical software manual
- 10.1037/0021-9010.75.6.698 · doi:10.1037/0021-9010.75.6.698 · Paper page
- Gorsuch, R. L., 1983, Factor analysis
- Harvey, R. J., meeting of the Society for Industrial and Organizational Psychology
- 10.1037/0022-3514.52.1.134 · doi:10.1037/0022-3514.52.1.134 · Paper page
- Hays, W. H., 1994, Statistics
- 10.1037/0033-2909.103.3.391 · doi:10.1037/0033-2909.103.3.391 · Paper page
- McAnally, M., 1999, Beach house on the moon [CD]
- 10.1037/0033-2909.105.3.430 · DOI · Paper page
- Nunnally, J. C., 1967, Psychometric theory
- Nunnally, J. C., 1978, Psychometric theory
- Nunnally, J. C., 1994, Psychometric theory
- 10.1037/0022-3514.83.3.735 · doi:10.1037/0022-3514.83.3.735 · Paper page
- SAS Institute Inc., 1999, SAS online Doc® [Computer software and manual]
- 10.1037/0021-9010.74.2.368 · doi:10.1037/0021-9010.74.2.368 · Paper page
- SPSS, 1999, SPSS 10.0 syntax reference guide
- 10.1037/0022-0167.34.4.414 · doi:10.1037/0022-0167.34.4.414 · Paper page
- 10.1016/S0149-2063_03_00033-3 · DOI · Paper page
- 10.1037/1082-989X.1.4.354 · doi:10.1037/1082-989x.1.4.354 · Paper page
- 10.1037/0033-2909.99.3.432 · DOI · Paper page
Cited by
Answers to 20 questions about interrater reliability and interrater agreement
LeBreton, J. M., & Senter, J. L. (2008) · Organizational Research Methods · doi:10.1177/1094428106296642 · Cited by 3,608
“The commonly used cut point of .70 is by no means a definitive rule; it is simply a heuristic that is frequently (and inappropriately) invoked in research (Lance et al., 2006).”
BackgroundMethodInfluentialUse Omega Rather than Cronbach’s Alpha for Estimating Reliability. But…
Hayes, A. F., & Coutts, J. J. (2020) · Communication Methods and Measures · doi:10.1080/19312458.2020.1718629 · Cited by 2,675
Cheung, G. W., Cooper-Thomas, H. D., Lau, R. S., & Wang, L. C. (2024) · Asia Pacific Journal of Management · doi:10.1007/s10490-023-09871-y · Cited by 2,262
“Although Cronbach’s alpha greater than 0.7 has been widely used as the standard for adequate reliability, Lance et al. (2006) pointed out that Nunnally (1978) and Carmines and Zeller (1979) recommended a reliability standard of 0.8 for the majority of studies, and reliability of 0.7 indicates the…”
InfluentialRauch, A., & Frese, M. (2007) · European Journal of Work and Organizational Psychology · doi:10.1080/13594320701595438 · Cited by 1,839
Richardson, H. A., Simmering, M. J., & Sturman, M. C. (2009) · Organizational Research Methods · doi:10.1177/1094428109332834 · Cited by 1,721
Metadata
- Authors
- Lance, C. E., Butts, M. M., & Michels, L. C.
- Affiliations
- University of Georgia (US)
- Venue
- Organizational Research Methods
- Publisher
- SAGE Publications
- Published
- 2006-04
- Volume
- 9
- Issue
- 2
- Pages
- 202-220
- ISSN
- 1094-4281 (ISSN-L)
- 1552-7425
- OpenAlex
- W1987634243
- Cited by
- 2,053
- References
- 128
- Citations by year
- 2026: 105 · 2025: 134 · 2024: 153 · 2023: 151 · 2022: 175 · 2021: 161 · 2020: 131 · 2019: 111 · 2018: 136 · 2017: 115 · 2016: 129 · 2015: 107 · 2014: 86 · 2013: 74 · 2012: 56
- FWCI
- 38.46
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Organizational Research Methods (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Business, Management and Accounting › Strategy and Management › Business Strategies and Management Research
- Keywords
- statistical significance thresholds, citation accuracy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.