Estimating reliability: A comparison of Cronbach's α, McDonald's ωt and the greatest lower bound
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Cronbach's alpha as a reliability estimate for a set of items has been repeatedly criticised. Using McDonald's omega or the Greatest Lower Bound (GLB) has been suggested instead. A simulation study using a single scale was performed on data with n = 60, 80, 120, 200, 300, 500 and 900 cases utilizing 4, 8, 16 or 32 items. A total of 10% of each item had missing values vs. there were no missing data. Skewness of the items were 0, 1 and 2. Correlations were homogenous vs. heterogeneous. Standard deviations were homogenous vs. heterogeneous. Underlying constructs were homogenous vs. heterogenous. Conditions were combined in a block design, and 1000 replications for each one were calculated. As a result, alpha slightly underestimated reliability in some cases, omega minimally overestimated in large samples, the GLB overestimated strongly in small samples. Additionally, GLB estimates had less precision within the 1000 replications, i.e., standard deviations were almost double as large as for alpha and omega. Interestingly, the performances of alpha and omega were basically similar. However, various programs showed different results particularly in cases with missing data on the items. Out of the programs examined, the R package MBESS displayed best results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Cronbach's alpha, introduced by Cronbach (1951), has been widely used to assess internal consistency, but relies on several assumptions that may not always hold (Malkewitz et al., 2023;Agbo, 2010).”
From Can You Trust LLM Judgments? Reliability of LLM-as-a-Judge (2024) · cites this paper as Malkewitz et al., 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Missing data: our view of the state of the art
Schafer, J. L., & Graham, J. W. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.2.147 · Cited by 11,638
Thanks coefficient alpha, we’ll take it from here.
McNeish, D. (2018) · Psychological Methods · doi:10.1037/met0000144 · Cited by 2,533
Making Reliability Reliable: A Systematic Approach to Reliability Coefficients
Cho, E. (2016) · Organizational Research Methods · doi:10.1177/1094428116656239 · Cited by 378
Buctot, D. B., Kim, N., & Kim, S. H. (2020) · doi:10.2139/ssrn.3571366 · Cited by 81
Raykov, T., Goldammer, P., Marcoulides, G. A., Li, T., & Menold, N. (2018) · Educational and Psychological Measurement · doi:10.1177/0013164417719948 · Cited by 13
Research works printed but not linked: 34
- Al-Homidan, 2008, Semidefinite programming for the educational testing problem, Central European Journal of Operations Research, 16, 239, 10.1007/s10100-008-0056-0 · doi:10.1007/s10100-008-0056-0
- Beland, 2017, Using the McDonalds omega coefficient instead of cronbach's alpha, McGill Journal of Education, 52, 791
- van Buuren, 2010, Item imputation without specifying scale structure, Methodology, 6, 31, 10.1027/1614-2241/a000004 · DOI
- Cho, 2021, Neither Cronbachs alpha nor McDonalds omega: A commentary on Sijtsma and pfad, Psychometrika, 84, 877, 10.1007/s11336-021-09801-1 · DOI
- Coste, 2013, Health and Quality of Life Outcomes, 11, 44, 10.1186/1477-7525-11-44 · DOI
- Cronbach, 1951, Coefficient alpha and the internal structure of tests, Biometrica, 16, 297
- Cronbach, 2004, My current thoughts on coefficient alpha and succesor procedures, Educational and Psychological Measurement, 64, 391, 10.1177/0013164404266386 · DOI · Paper page
- Crutzen, 2017, Scale quality: Alpha is an inadequate estimate and factor-analytic evidenceis needed first of all, Health Psychology Review, 11, 242, 10.1080/17437199.2015.1124240 · DOI · Paper page
- Deng, 2017, Testing the difference between reliability coefficients alpha and omega, Educational and Psychological Measurement, 77, 185, 10.1177/0013164416658325 · DOI · Paper page
- Fowler, 2020, The role of personality and risk-taking on Tinder use, Social Sciences & Humanities Open, 2, 10.1016/j.ssaho.2020.100067 · DOI
- Hardt, 2012, Auxiliary variables in multiple imputation in regression with missing X: A warning against including too many in small sample research, BMC Medical Research Methodology, 12, 10.1186/1471-2288-12-184 · DOI · Paper page
- Jolani, 2014, Dual imputation model for incomplete longitudinal data, British Journal of Mathematical and Statistical Psychology, 67, 197, 10.1111/bmsp.12021 · DOI
- Komaroff, 1997, Effects of simultaneos violations of essential tau-equivalence and uncorrelated error on coefficient alpha, Applied Psychological Measurement, 21, 337, 10.1177/01466216970214004 · DOI · Paper page
- Marcusson-Claverts, 2019, Psychometric propertiesof the spontanwous and deliberate mind wandering scales, European Journal of Psychological Assessment, 35, 878, 10.1027/1015-5759/a000470 · DOI
- McDonald, 2000, A Monte Carlo study of missing item methods, Organizational Research Methods, 3, 70, 10.1177/109442810031003 · DOI · Paper page
- Oosterwijk, 2019, Using confidence intervals for assessing reliability of real tests, Assessment, 26, 1207, 10.1177/1073191117737375 · DOI
- Padilla, 2016, A Comparison of composite reliability estimators: Coefficient omega confidence intervalls in the current literature, Educational and Psychological Measurement, 76, 436, 10.1177/0013164415593776 · DOI · Paper page
- Pedersen, 2019, Psychometric properties of the severity indices of personality problems in two samples, European Journal of Psychological Assessment, 35, 698, 10.1027/1015-5759/a000436 · DOI
- Peugh, 2004, Missing data in educational research: A review of reporting and suggestions for improvement. 0, Review of Educational Research, 74, 525, 10.3102/00346543074004525 · DOI · Paper page
- Raju, 1977, A generalization of coefficient alpha, Psychometrika, 42, 549, 10.1007/BF02295978 · DOI
- Raykov, 1998, Coefficient alpha and composite reliability with interrelated nonhomogeneous items, Applied Psychological Measurement, 22, 375, 10.1177/014662169802200407 · DOI · Paper page
- Raykov, 2001, Bias of coefficient alpha for fixed congeneric measures with correlated errors, Applied Psychological Measurement, 25, 69, 10.1177/01466216010251005 · DOI · Paper page
- Raykov, 2019, Thanks coefficient alpha, we still need you, Educational and Psychological Measurement, 79, 200, 10.1177/0013164417725127 · DOI · Paper page
- Raykov, 2019, Multiple-component measurement instruments in heterogeneous populations: Is there a single coefficient alpha, Educational and Psychological Measurement, 79, 399, 10.1177/0013164417733305 · DOI
- Raykov, 2015, The of the assumption of uncorrelated errors in psychometric theory, Educational and Psychological Measurement, 75, 634, 10.1177/0013164414548217 · DOI
- Revelle, 2009, Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma, Psychometrika, 74, 145, 10.1007/s11336-008-9102-z · DOI · Paper page
- Sijtsma, 2009, Reliability beyond theory and into practice, Psychometrika, 74, 169, 10.1007/s11336-008-9103-y · DOI · Paper page
- Sijtsma, 2021, Part II: On the use, the misuse, and the very limited usefulness of Cronbachs alpha: Discussing lower bounds and correlated errors, Psychometrika, 86, 843, 10.1007/s11336-021-09789-8 · DOI
- TenBerge, 2004, The graetest lower bound to the reliability of a test and the hypothesis of unidimensinality, Psychometrika, 69, 613, 10.1007/BF02289858 · DOI · Paper page
- Thoemmes, 2014, A cautious note on auxiliary variables that can increase bias in missing data problems, Multivariate Behavioral Research, 49, 443, 10.1080/00273171.2014.931799 · DOI
- Trizano-Hermosilla, 2016, Best alternatives to cronbach's alpha reliability in realistic conditions: Congeneric and asymmetrical measurements, Frontiers in Psychology, 7, 769, 10.3389/fpsyg.2016.00769 · DOI · Paper page
- Woodhouse, 1977, Lower bounds for the reliability of the total score on a test composed of non-homogenous items: II: A search procedure to locate the graetest lower bound, Psychometrika, 42, 579, 10.1007/BF02295980 · DOI · Paper page
- Yang, 2019, Categorical omega with small sample sizes via Bayesian estimation: An alternative to frequentist estimatiors, Educational and Psychological Measurement, 79, 19, 10.1177/0013164417752008 · DOI
- Zhang, 2016, Robust coefficients alpha and omega and confidence intervalls with outlying observations and missing data: Methods and software, Educational and Psychological Measurement, 76, 387, 10.1177/0013164415594658 · DOI · Paper page
Cited by
A validity-guided workflow for robust large language model research in psychology
Lin, Z. (2026) · Behavior Research Methods · doi:10.3758/s13428-026-03073-2 · Cited by 8
Metadata
- Authors
- Malkewitz, C. P., Schwall, P., Meesters, C., & Hardt, J.
- Venue
- Social Sciences & Humanities Open
- Publisher
- Elsevier BV
- Published
- 2023
- Volume
- 7
- Issue
- 1
- Pages
- 100368
- ISSN
- 2590-2911 (ISSN-L)
- OpenAlex
- W4311221106
- Cited by
- 199
- References
- 5
- FWCI
- 21.39
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0/ (vor)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Estimating reliability: A comparison of Cronbach's α, McDonald's ωt and the greatest lower bound. https://app.cochat.ai/papers/10.1016/j.ssaho.2022.100368
- The paper
- Malkewitz, C. P., Schwall, P., Meesters, C., & Hardt, J. (2023). Estimating reliability: A comparison of Cronbach's α, McDonald's ωt and the greatest lower bound. Social Sciences & Humanities Open. https://doi.org/10.1016/j.ssaho.2022.100368
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.