Making Reliability Reliable: A Systematic Approach to Reliability Coefficients
The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.
Abstract
The current conventions for test score reliability coefficients are unsystematic and chaotic. Reliability coefficients have long been denoted using names that are unrelated to each other, with each formula being generated through different methods, and they have been represented inconsistently. Such inconsistency prevents organizational researchers from understanding the whole picture and misleads them into using coefficient alpha unconditionally. This study provides a systematic naming convention, formula-generating methods, and methods of representing each of the reliability coefficients. This study offers an easy-to-use solution to the issue of choosing between coefficient alpha and composite reliability. This study introduces a calculator that enables its users to obtain the values of various multidimensional reliability coefficients with a few mouse clicks. This study also presents illustrative numerical examples to provide a better understanding of the characteristics and computations of reliability coefficients.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Cronbach's alpha is not structural equation modeling (SEM) based, yet it is the most commonly-reported reliability coefficient in studies using SEM (Cho, 2016).”
From Reporting reliability, convergent and discriminant validity with structural equation modeling: A review and best-practice recommendations (2024) · cites this paper as Cho, 2016
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931 · cited 8×
“One estimator goes by the name of the fourth person to propose it (Cronbach, 1951), and others do not bear the name of any developers.”
BackgroundMethodInfluentialStructural Equations with Latent Variables.
Clogg, C. C., & Bollen, K. A. (1991) · Contemporary Sociology · doi:10.2307/2072165 · Cited by 17,442
Structural Equations with Latent Variables
Bollen, K. A. (1989) · Wiley series in probability and statistics · doi:10.1002/9781118619179 · Cited by 17,277 · cited 1×
“This study assumes that no specific factors exist to circumvent the controversial issue of specific factors and errors (Bollen, 1989).”
BackgroundStatistical Methods for Research Workers
(1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,880
EQS : structural equations program manual
Bentler, P. M. (1989) · Medical Entomology and Zoology · Cited by 9,669
What is coefficient alpha? An examination of theory and applications
Cortina, J. M. (1993) · Journal of Applied Psychology · doi:10.1037/0021-9010.78.1.98 · Cited by 8,810 · cited 2×
“Many users perform the reverse; they calculate reliability to identify unidimensionality, misconceiving that a high value of a unidimensional reliability coefficient indicates unidimensionality (Cortina, 1993; Green, Lissitz, & Mulaik, 1977; Hattie, 1985; Schmitt, 1996).”
BackgroundStatistical Theories of Mental Test Scores
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,022
Test Theory: A Unified Treatment
Zwick, R., & McDonald, R. P. (2000) · Journal of the American Statistical Association · doi:10.2307/2669496 · Cited by 3,603
Dunn, T. J., Baguley, T., & Brunsden, V. (2014) · British Journal of Psychology · doi:10.1111/bjop.12046 · Cited by 3,596 · cited 3×
“In addition to Jöreskog’s (1971) work, two approaches deserve mention. Let pi denote the ratio of the length of item i (i.e., li) to the total length (i.e., Pk i1⁄41 li), and let siX denote Pk j1⁄41 sij. Raju’s (1977) generalization of his previous formula (~ rR1977) is algebraically equivalent to rC .”
BackgroundMethodHandbook of Structural Equation Modeling
Howard, A. L. (2013) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705511.2013.769397 · Cited by 3,440
Research works printed but not linked: 10
- Armor D. J., 1974, Sociological methodology, 17
- Cronbach L. J., 1978, Current Contents, 13, 263
- Feldt L. S., 1989, Educational measurement, 105
- Hair J. F., 2010, Multivariate data analysis
- Kamata A., 2003, Estimating reliability for multidimensional composite scale scores
- 10.1037/1082-989X.5.3.343 · doi:10.1037/1082-989x.5.3.343 · Paper page
- Peters G. J. Y., 2014, European Health Psychologist, 16, 56
- Raju N. S., 1970, Proceedings of the 78th Annual Convention of APA, 5, 143
- Spearman C., 1910, British Journal Psychology, 3, 271
- Tang W., 2012, Paper presented at the annual meeting of the American Educational Research Association
Cited by
Cheung, G. W., Cooper-Thomas, H. D., Lau, R. S., & Wang, L. C. (2024) · Asia Pacific Journal of Management · doi:10.1007/s10490-023-09871-y · Cited by 2,262
“Using a simulation study, Voorhees et al. (2016) extended the HTMT method to covariance-based SEM and provided initial supporting evidence.”
InfluentialAn Updated Guideline for Assessing Discriminant Validity
Rönkkö, M., & Cho, E. (2022) · Organizational Research Methods · doi:10.1177/1094428120968614 · Cited by 1,309
Self-citationGabriel, A. S., Podsakoff, N. P., Beal, D. J., Scott, B. A., Sonnentag, S., Trougakos, J. P., & Butts, M. M. (2019) · Organizational Research Methods · doi:10.1177/1094428118802626 · Cited by 755
“Thus, we encourage researchers to employ the procedures described by Geldhof et al. in their article (Mplus syntax is provided) to estimate and report multilevel reliabilities for their Level 1 measures, identifying the specific type of reliability calculated (see also Cho, 2016).”
MethodUn viaje alrededor de alfa y omega para estimar la fiabilidad de consistencia interna
Viladrich, C., Angulo-Brunet, A., & Doval, E. (2017) · Anales de Psicología · doi:10.6018/analesps.33.3.268401 · Cited by 699
Li, G., Li, L., Choi, T., & Sethi, S. P. (2020) · Journal of Operations Management · doi:10.1002/joom.1061 · Cited by 423
Metadata
- Authors
- Cho, E.
- Affiliations
- Kwangwoon University (KR)
- Corresponding author
- Cho, Eunseong
- ORCID
- Cho, Eunseong
- Venue
- Organizational Research Methods
- Publisher
- SAGE Publications
- Published
- 2016-07-08
- Volume
- 19
- Issue
- 4
- Pages
- 651-682
- ISSN
- 1094-4281 (ISSN-L)
- 1552-7425
- OpenAlex
- W2474970179
- Cited by
- 378
- References
- 74
- Citations by year
- 2026: 35 · 2025: 46 · 2024: 51 · 2023: 56 · 2022: 42 · 2021: 56 · 2020: 40 · 2019: 23 · 2018: 21 · 2017: 8
- FWCI
- 18.96
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Organizational Research Methods (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Health Sciences › Nursing › Research and Theory › Health Education and Validation
- Keywords
- test score reliability, reliability coefficients, Cronbach's alpha, composite reliability, measurement reliability
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Making Reliability Reliable: A Systematic Approach to Reliability Coefficients. https://app.cochat.ai/papers/10.1177/1094428116656239
- The paper
- Cho, E. (2016). Making Reliability Reliable: A Systematic Approach to Reliability Coefficients. Organizational Research Methods. https://doi.org/10.1177/1094428116656239
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.