CoChat

Estimating reliability: A comparison of Cronbach's α, McDonald's ωt and the greatest lower bound

Open access
Malkewitz, Camila Paola, Schwall, Philipp, Meesters, Christian, Hardt, Jochen · 2023 · Social Sciences & Humanities Open
Cited by 199

The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.

Abstract

Cronbach's alpha as a reliability estimate for a set of items has been repeatedly criticised. Using McDonald's omega or the Greatest Lower Bound (GLB) has been suggested instead. A simulation study using a single scale was performed on data with n = 60, 80, 120, 200, 300, 500 and 900 cases utilizing 4, 8, 16 or 32 items. A total of 10% of each item had missing values vs. there were no missing data. Skewness of the items were 0, 1 and 2. Correlations were homogenous vs. heterogeneous. Standard deviations were homogenous vs. heterogeneous. Underlying constructs were homogenous vs. heterogenous. Conditions were combined in a block design, and 1000 replications for each one were calculated. As a result, alpha slightly underestimated reliability in some cases, omega minimally overestimated in large samples, the GLB overestimated strongly in small samples. Additionally, GLB estimates had less precision within the 1000 replications, i.e., standard deviations were almost double as large as for alpha and omega. Interestingly, the performances of alpha and omega were basically similar. However, various programs showed different results particularly in cases with missing data on the items. Out of the programs examined, the R package MBESS displayed best results.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.ssaho.2022.100368

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 5 references

Research works printed but not linked: 34

  1. Al-Homidan, 2008, Semidefinite programming for the educational testing problem, Central European Journal of Operations Research, 16, 239, 10.1007/s10100-008-0056-0 · doi:10.1007/s10100-008-0056-0
  2. Beland, 2017, Using the McDonalds omega coefficient instead of cronbach's alpha, McGill Journal of Education, 52, 791
  3. van Buuren, 2010, Item imputation without specifying scale structure, Methodology, 6, 31, 10.1027/1614-2241/a000004 · DOI
  4. Cho, 2021, Neither Cronbachs alpha nor McDonalds omega: A commentary on Sijtsma and pfad, Psychometrika, 84, 877, 10.1007/s11336-021-09801-1 · DOI
  5. Coste, 2013, Health and Quality of Life Outcomes, 11, 44, 10.1186/1477-7525-11-44 · DOI
  6. Cronbach, 1951, Coefficient alpha and the internal structure of tests, Biometrica, 16, 297
  7. Cronbach, 2004, My current thoughts on coefficient alpha and succesor procedures, Educational and Psychological Measurement, 64, 391, 10.1177/0013164404266386 · DOI · Paper page
  8. Crutzen, 2017, Scale quality: Alpha is an inadequate estimate and factor-analytic evidenceis needed first of all, Health Psychology Review, 11, 242, 10.1080/17437199.2015.1124240 · DOI · Paper page
  9. Deng, 2017, Testing the difference between reliability coefficients alpha and omega, Educational and Psychological Measurement, 77, 185, 10.1177/0013164416658325 · DOI · Paper page
  10. Fowler, 2020, The role of personality and risk-taking on Tinder use, Social Sciences & Humanities Open, 2, 10.1016/j.ssaho.2020.100067 · DOI
  11. Hardt, 2012, Auxiliary variables in multiple imputation in regression with missing X: A warning against including too many in small sample research, BMC Medical Research Methodology, 12, 10.1186/1471-2288-12-184 · DOI · Paper page
  12. Jolani, 2014, Dual imputation model for incomplete longitudinal data, British Journal of Mathematical and Statistical Psychology, 67, 197, 10.1111/bmsp.12021 · DOI
  13. Komaroff, 1997, Effects of simultaneos violations of essential tau-equivalence and uncorrelated error on coefficient alpha, Applied Psychological Measurement, 21, 337, 10.1177/01466216970214004 · DOI · Paper page
  14. Marcusson-Claverts, 2019, Psychometric propertiesof the spontanwous and deliberate mind wandering scales, European Journal of Psychological Assessment, 35, 878, 10.1027/1015-5759/a000470 · DOI
  15. McDonald, 2000, A Monte Carlo study of missing item methods, Organizational Research Methods, 3, 70, 10.1177/109442810031003 · DOI · Paper page
  16. Oosterwijk, 2019, Using confidence intervals for assessing reliability of real tests, Assessment, 26, 1207, 10.1177/1073191117737375 · DOI
  17. Padilla, 2016, A Comparison of composite reliability estimators: Coefficient omega confidence intervalls in the current literature, Educational and Psychological Measurement, 76, 436, 10.1177/0013164415593776 · DOI · Paper page
  18. Pedersen, 2019, Psychometric properties of the severity indices of personality problems in two samples, European Journal of Psychological Assessment, 35, 698, 10.1027/1015-5759/a000436 · DOI
  19. Peugh, 2004, Missing data in educational research: A review of reporting and suggestions for improvement. 0, Review of Educational Research, 74, 525, 10.3102/00346543074004525 · DOI · Paper page
  20. Raju, 1977, A generalization of coefficient alpha, Psychometrika, 42, 549, 10.1007/BF02295978 · DOI
  21. Raykov, 1998, Coefficient alpha and composite reliability with interrelated nonhomogeneous items, Applied Psychological Measurement, 22, 375, 10.1177/014662169802200407 · DOI · Paper page
  22. Raykov, 2001, Bias of coefficient alpha for fixed congeneric measures with correlated errors, Applied Psychological Measurement, 25, 69, 10.1177/01466216010251005 · DOI · Paper page
  23. Raykov, 2019, Thanks coefficient alpha, we still need you, Educational and Psychological Measurement, 79, 200, 10.1177/0013164417725127 · DOI · Paper page
  24. Raykov, 2019, Multiple-component measurement instruments in heterogeneous populations: Is there a single coefficient alpha, Educational and Psychological Measurement, 79, 399, 10.1177/0013164417733305 · DOI
  25. Raykov, 2015, The of the assumption of uncorrelated errors in psychometric theory, Educational and Psychological Measurement, 75, 634, 10.1177/0013164414548217 · DOI
  26. Revelle, 2009, Coefficients alpha, beta, omega, and the glb: Comments on Sijtsma, Psychometrika, 74, 145, 10.1007/s11336-008-9102-z · DOI · Paper page
  27. Sijtsma, 2009, Reliability beyond theory and into practice, Psychometrika, 74, 169, 10.1007/s11336-008-9103-y · DOI · Paper page
  28. Sijtsma, 2021, Part II: On the use, the misuse, and the very limited usefulness of Cronbachs alpha: Discussing lower bounds and correlated errors, Psychometrika, 86, 843, 10.1007/s11336-021-09789-8 · DOI
  29. TenBerge, 2004, The graetest lower bound to the reliability of a test and the hypothesis of unidimensinality, Psychometrika, 69, 613, 10.1007/BF02289858 · DOI · Paper page
  30. Thoemmes, 2014, A cautious note on auxiliary variables that can increase bias in missing data problems, Multivariate Behavioral Research, 49, 443, 10.1080/00273171.2014.931799 · DOI
  31. Trizano-Hermosilla, 2016, Best alternatives to cronbach's alpha reliability in realistic conditions: Congeneric and asymmetrical measurements, Frontiers in Psychology, 7, 769, 10.3389/fpsyg.2016.00769 · DOI · Paper page
  32. Woodhouse, 1977, Lower bounds for the reliability of the total score on a test composed of non-homogenous items: II: A search procedure to locate the graetest lower bound, Psychometrika, 42, 579, 10.1007/BF02295980 · DOI · Paper page
  33. Yang, 2019, Categorical omega with small sample sizes via Bayesian estimation: An alternative to frequentist estimatiors, Educational and Psychological Measurement, 79, 19, 10.1177/0013164417752008 · DOI
  34. Zhang, 2016, Robust coefficients alpha and omega and confidence intervalls with outlying observations and missing data: Methods and software, Educational and Psychological Measurement, 76, 387, 10.1177/0013164415594658 · DOI · Paper page

Metadata

Authors
Malkewitz, C. P., Schwall, P., Meesters, C., & Hardt, J.
Venue
Social Sciences & Humanities Open
Publisher
Elsevier BV
Published
2023
Volume
7
Issue
1
Pages
100368
ISSN
2590-2911 (ISSN-L)
OpenAlex
W4311221106
Cited by
199
References
5
FWCI
21.39
Type
journal article
Language
en
Open access
gold
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Estimating reliability: A comparison of Cronbach's α, McDonald's ωt and the greatest lower bound. https://app.cochat.ai/papers/10.1016/j.ssaho.2022.100368
The paper
Malkewitz, C. P., Schwall, P., Meesters, C., & Hardt, J. (2023). Estimating reliability: A comparison of Cronbach's α, McDonald's ωt and the greatest lower bound. Social Sciences & Humanities Open. https://doi.org/10.1016/j.ssaho.2022.100368

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.