Reliability of Scales With Second-Order Structure: Evaluation of Coefficient Alpha’s Population Slippage Using Latent Variable Modeling
Open accessThe DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
A readily applicable procedure is discussed that allows evaluation of the discrepancy between the popular coefficient alpha and the reliability coefficient of a scale with second-order factorial structure that is frequently of relevance in empirical educational and psychological research. The approach is developed within the framework of the widely used latent variable modeling methodology and permits point and interval estimation of the slippage of alpha from scale reliability in a population under investigation. The method is useful when examining the consistency of complex structure measuring instruments assessing higher order latent constructs and, under its assumptions, represents a generally recommendable alternative to coefficient alpha. The outlined procedure is illustrated using data from an authoritarianism study.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These and other measures with second-order factors violate the assumption of unidimensionality of measures (i.e., each indicator assesses a single underlying construct; Clark & Watson, 1995), and therefore Cronbach's alpha and CR are not appropriate reliability measures because they ignore the second-order factor structure (Raykov et al., 2018).”
From Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations (2024) · cites this paper as Raykov et al., 2018
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931 · cited 1×
“Coefficient alpha (e.g., Cronbach, 1951) is frequently used to appraise the reliability of multicomponent measuring instruments that are highly popular in these and cognate disciplines.”
BackgroundStructural Equations with Latent Variables.
Clogg, C. C., & Bollen, K. A. (1991) · Contemporary Sociology · doi:10.2307/2072165 · Cited by 17,442
Structural Equations with Latent Variables
Bollen, K. A. (1989) · Wiley series in probability and statistics · doi:10.1002/9781118619179 · Cited by 17,277 · cited 4×
“In particular, when the popular maximum likelihood (ML) method of parameter estimation is used (with observed variable normality; e.g., Bollen, 1989), Equation (8) renders the ML estimator of the population slippage of coefficient alpha from the reliability coefficient of the overall scale score X,…”
BackgroundMethodInfluential(2010) · Cited by 7,034
Casella, G., & Berger, R. (2024) · Chapman & Hall/CRC eBooks · doi:10.1201/9781003456285 · Cited by 5,711
Harman, H. H. (1960) · Cited by 5,148
Introduction to Classical and Modern Test Theory
Crocker, L. M., & Algina, J. J. (1986) · Cited by 5,090
On the use, the misuse, and the very limited usefulness of Cronbach’s alpha
Sijtsma, K. (2009) · Psychometrika · doi:10.1007/s11336-008-9101-0 · Cited by 2,814 · cited 1×
“When these conditions do not hold, however, alpha can be seriously misleading (e.g., Sijtsma, 2009), as can also be the closely related index ‘‘alpha if item deleted’’ that is currently often used by researchers involved in scale construction and development.”
BackgroundMerrifield, P. R., & Harman, H. (1969) · American Educational Research Journal · doi:10.2307/1161864 · Cited by 2,574
A First Course in Structural Equation Modeling
Raykov, T., & Marcoulides, G. A. (2012) · Routledge eBooks · doi:10.4324/9780203930687 · Cited by 1,612 · cited 2×
“(For an introduction to the Mplus syntax, see e.g., Raykov & Marcoulides, 2006.)”
BackgroundSelf-citationIntroduction to Psychometric Theory
Raykov, T., & Marcoulides, G. A. (2011) · Routledge eBooks · doi:10.4324/9780203841624 · Cited by 1,095 · cited 3×
“…is a lower bound of reliability, with the population discrepancy of the former from the latter possibly being of marked magnitude (e.g., Raykov, 1997; for the correlated error case, when that slippage of alpha can be considerable, see for instance Bollen, 1980, and Raykov & Marcoulides, 2011).”
BackgroundMethodSelf-citationBeyond SEM: General latent variable modeling
Muthén, B. O. (2002) · Behaviormetrika · doi:10.2333/bhmk.29.81 · Cited by 1,074 · cited 3×
“For further details on this study, we refer to Raykov et al. (2018), who used another portion of that data set for the purpose of illustrating a method for evaluating latent criterion validity.”
MethodIssues in the Comparative Measurement of Political Democracy
Bollen, K. A. (1980) · American Sociological Review · doi:10.2307/2095172 · Cited by 777 · cited 2×
“…is a lower bound of reliability, with the population discrepancy of the former from the latter possibly being of marked magnitude (e.g., Raykov, 1997; for the correlated error case, when that slippage of alpha can be considerable, see for instance Bollen, 1980, and Raykov & Marcoulides, 2011).”
BackgroundRaykov, T. (1997) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3204_2 · Cited by 481 · cited 4×
“In particular, Raykov (1997) showed that under restrictive yet testable conditions, coefficient alpha has minimal population slippage from scale reliability, which is negligible for most empirical purposes (see also Raykov & Marcoulides, 2015; Raykov, West, & Traynor, 2015).”
BackgroundInfluentialSelf-citationCoefficient Alpha and the Reliability of Composite Measurements
Novick, M. R., & Lewis, C. (1967) · Psychometrika · doi:10.1007/bf02289400 · Cited by 466 · cited 3×
“This follows from a main result in Novick and Lewis (1967), which states that with uncorrelated errors population alpha is a lower bound of scale reliability unless the scale components are essentially tau-equivalent (i.”
BackgroundInfluentialCOEFFICIENT ALPHA AND THE RELIABILITY OF COMPOSITE MEASUREMENTS
Novick, M. R., & Lewis, C. (1966) · ETS Research Bulletin Series · doi:10.1002/j.2333-8504.1966.tb00356.x · Cited by 446 · cited 2×
“However, as may be implied from Novick and Lewis (1967), it would in general be incorrect to use coefficient alpha then despite its still high popularity as an index of overall scale consistency.”
BackgroundGhosh, M., Casella, G., & Berger, R. L. (1994) · Journal of the American Statistical Association · doi:10.2307/2290879 · Cited by 411
The Impact of Categorization With Confirmatory Factor Analysis
DiStefano, C. (2002) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1207/s15328007sem0903_2 · Cited by 276 · cited 2×
“…which do not result from piling at scale end for an individual component(s), it may well be recommendable to use the robust ML method (MLR; L. K. Muthén & Muthén, 2017), possibly also with components having as few as 5 to 7 response options (e.g., DiStefano, 2002; see also the appendix).”
MethodRaykov, T., & Marcoulides, G. A. (2014) · Educational and Psychological Measurement · doi:10.1177/0013164414526039 · Cited by 63 · cited 1×
“In particular, Raykov (1997) showed that under restrictive yet testable conditions, coefficient alpha has minimal population slippage from scale reliability, which is negligible for most empirical purposes (see also Raykov & Marcoulides, 2015; Raykov, West, & Traynor, 2015).”
BackgroundSelf-citationKurzskala Autoritarismus (KSA-3)
Beierlein, C., Asbrock, F., Kauff, M., & Schmidt, P. (2015) · Open Access Repository for Measurement Instruments (ZIS) · doi:10.6102/zis228 · Cited by 34
Raykov, T., West, B. T., & Traynor, A. (2014) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2014.936081 · Cited by 20
Self-citationEvaluation of Validity and Reliability for Hierarchical Scales Using Latent Variable Modeling
Raykov, T., & Marcoulides, G. A. (2012) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2012.687675 · Cited by 18 · cited 3×
“…as mentioned above use of ML estimation is appropriate and yields ML estimates of the alpha to reliability discrepancy, as well as of the overall scale’s reliability and coefficient alpha if of interest (see the second term in the right-hand side of Equation 7, and Raykov & Marcoulides, 2012).”
BackgroundSelf-citationRaykov, T., Menold, N., & Marcoulides, G. A. (2017) · Educational and Psychological Measurement · doi:10.1177/0013164417698017 · Cited by 13 · cited 7×
“Several limitations of the discussed approach are worthwhile pointing out here (cf. Raykov et al., 2018).”
BackgroundMethodInfluentialSelf-citation
Research works printed but not linked: 1
- 11. Muthén L. K., Muthén B. (2017). Mplus user’s guide. Los Angeles, CA: Muthén & Muthén. [[Google Scholar](https://scholar.google.com/scholar_lookup?title=Mplus%20user%E2%80%99s%20guide&author=L.%20K.%20Muth%C3%A9n&author=B.%20Muth%C3%A9n&publication_year=2017&)]
Cited by
Cheung, G. W., Cooper-Thomas, H. D., Lau, R. S., & Wang, L. C. (2024) · Asia Pacific Journal of Management · doi:10.1007/s10490-023-09871-y · Cited by 2,262
“…violate the assumption of unidimensionality of measures (i.e., each indicator assesses a single underlying construct; Clark & Watson, 1995), and therefore Cronbach’s alpha and CR are not appropriate reliability measures because they ignore the second-order factor structure (Raykov et al., 2018).”
Estimating reliability: A comparison of Cronbach's α, McDonald's ωt and the greatest lower bound
Malkewitz, C. P., Schwall, P., Meesters, C., & Hardt, J. (2023) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2022.100368 · Cited by 199
Richmond, J., Boynton, M. H., Ozawa, S., Muessig, K. E., Cykert, S., & Ribisl, K. M. (2022) · Social Science & Medicine · doi:10.1016/j.socscimed.2022.114827 · Cited by 63
Troia, G. A., Wang, H., & Lawrence, F. (2022) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2022.102100 · Cited by 26
Raykov, T., & Marcoulides, G. A. (2023) · Measurement Interdisciplinary Research and Perspectives · doi:10.1080/15366367.2022.2031485 · Cited by 9
Self-citation
Metadata
- Authors
- Raykov, T., Goldammer, P., Marcoulides, G. A., Li, T., & Menold, N.
- Affiliations
- Michigan State University (US)
- ETH Zurich (CH)
- University of California, Santa Barbara (US)
- GESIS - Leibniz Institute for the Social Sciences (DE)
- Corresponding author
- Raykov, Tenko
- Venue
- Educational and Psychological Measurement
- Publisher
- SAGE Publications
- Published
- 2017-07-24
- Volume
- 78
- Issue
- 6
- Pages
- 1123-1135
- ISSN
- 0013-1644 (ISSN-L)
- 1552-3888
- PMC
- PMC6293413
- OpenAlex
- W2739196104
- Cited by
- 13
- References
- 23
- Citations by year
- 2025: 5 · 2024: 2 · 2023: 4 · 2022: 2
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Educational and Psychological Measurement (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Keywords
- Cronbach's alpha, scale reliability, latent variable models, interval estimation, higher-order constructs, authoritarianism
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.