The DOI matches its Crossref record; 89 of the 89 references it cites were found in a registry or an index.
The reliability of multidimensional neuropsychological measures: from alpha to omega
Abstract
OBJECTIVE: To demonstrate that Coefficient omega, a model-based estimate, is more a more appropriate index of reliability than coefficient alpha for the multidimensional scales that are commonly employed by neuropsychologists. METHOD: As an illustration, a structural model of an overarching general factor and four first-order factors for the WAIS-IV based on the standardization sample of 2200 participants was identified and omega coefficients were subsequently computed for WAIS-IV composite scores. RESULTS: Alpha coefficients were ≥ .90 and omega coefficients ranged from .75 to .88 for WAIS-IV factor index scores, indicating that the blend of general and group factor variance in each index score created a reliable multidimensional composite. However, the amalgam of variance from general and group factors did not allow the precision of Full Scale IQ (FSIQ) and factor index scores to be disentangled. In contrast, omega hierarchical coefficients were low for all four factor index scores (.10-.41), indicating that most of the reliable variance of each factor index score was due to the general intelligence factor. In contrast, the omega hierarchical coefficient for the FSIQ score was .84. CONCLUSIONS: Meaningful interpretation of WAIS-IV factor index scores as unambiguous indicators of group factors is imprecise, thereby fostering unreliable identification of neurocognitive strengths and weaknesses, whereas the WAIS-IV FSIQ score can be interpreted as a reliable measure of general intelligence. It was concluded that neuropsychologists should base their clinical decisions on reliable scores as indexed by coefficient omega.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
What is coefficient alpha? An examination of theory and applications
Cortina, J. M. (1993) · Journal of Applied Psychology · doi:10.1037/0021-9010.78.1.98 · Cited by 8,810 · cited 1×
“If its basic assumptions are violated, alpha may either over or under estimate the population reliability (Cortina, 1993; Green & Hershberger, 2000; Green, Lissitz, & Mulaik, 1977; Novick & Lewis, 1967; Raykov, 2001a).”
BackgroundThe standards for educational and psychological testing.
Eignor, D. R. (2013) · APA handbook of testing and assessment in psychology, Vol. 1: Test theory and testing and assessment in industrial and organizational psychology. · doi:10.1037/14047-013 · Cited by 6,988
“Ethical Principles of Psychologists and Code of Conduct”
(2000) · Ethical Reasoning in the Mental Health Professions · doi:10.4324/9781482274400-4 · Cited by 4,251
Human Cognitive Abilities: A Survey of Factor-Analytic Studies
Carroll, J. B. (1993) · doi:10.1017/cbo9780511571312 · Cited by 4,121
Starting at the beginning: an introduction to coefficient alpha and internal consistency
Streiner, D. L. (2003) · Journal of Personality Assessment · doi:10.1207/s15327752jpa8001_18 · Cited by 3,794
Dunn, T. J., Baguley, T., & Brunsden, V. (2014) · British Journal of Psychology · doi:10.1111/bjop.12046 · Cited by 3,596
Human cognitive abilities a survey of factor-analytic studies
Carroll, J. B. (2004) · Cited by 3,400
Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0149-3 · Cited by 3,200
Standards for Educational and Psychological Testing
(2007) · Encyclopedia of Measurement and Statistics · doi:10.4135/9781412952644.n432 · Cited by 3,126
On the use, the misuse, and the very limited usefulness of Cronbach’s alpha
Sijtsma, K. (2009) · Psychometrika · doi:10.1007/s11336-008-9101-0 · Cited by 2,814
Wechsler Adult Intelligence Scale--Fourth Edition
Wechsler, D. (2012) · PsycTESTS Dataset · doi:10.1037/t15169-000 · Cited by 2,232
Introduction to Measurement Theory
Allen, M. J. (1979) · Cited by 2,124
The Rediscovery of Bifactor Measurement Models
Reise, S. P. (2012) · Multivariate Behavioral Research · doi:10.1080/00273171.2012.715555 · Cited by 2,112 · cited 10×
“In contrast, a robust ωhs coefficient suggests that most of the reliable variance of that subscale is independent of the general factor and clinical interpretation of an examinee’s strengths and weaknesses beyond the general factor can be conducted (Brunner et al., 2012; DeMars, 2013; Reise, 2012).”
BackgroundMethodResultInfluentialMy Current Thoughts on Coefficient Alpha and Successor Procedures
Cronbach, L. J., & Shavelson, R. J. (2004) · Educational and Psychological Measurement · doi:10.1177/0013164404266386 · Cited by 1,606
Evaluating Bifactor Models: Calculating and Interpreting Statistical Indices
Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016) · Psychological Methods · doi:10.1037/met0000045 · Cited by 1,599
Zinbarg, R. E., Revelle, W., Yovel, I., & Li, W. (2005) · Psychometrika · doi:10.1007/s11336-003-0974-7 · Cited by 1,439 · cited 1×
“Omega may also be computed via bifactor confirmatory analysis and exploratory factor analysis models with orthogonalization or target bifactor rotation (Brunner et al., 2012; Reise et al., 2013; Zinbarg et al., 2005).”
MethodThe Development of Hierarchical Factor Solutions
Schmid, J., & Leiman, J. M. (1957) · Psychometrika · doi:10.1007/bf02289209 · Cited by 1,098 · cited 1×
“As recommended by Carroll (1993, 1995), that hierarchical structure was then orthogonalized (Schmid & Leiman, 1957) to allow convenient computation of omega indices.”
BackgroundScoring and Modeling Psychological Measures in the Presence of Multidimensionality
Reise, S. P., Bonifay, W. E., & Haviland, M. G. (2013) · Journal of Personality Assessment · doi:10.1080/00223891.2012.725437 · Cited by 1,069
Understanding Internal Consistency Reliability Estimates: A Conceptual Primer on Coefficient Alpha
Henson, R. K. (2001) · Measurement and Evaluation in Counseling and Development · doi:10.1080/07481756.2002.12069034 · Cited by 1,058
Applying Bifactor Statistical Indices in the Evaluation of Psychological Measures
Rodriguez, A., Reise, S. P., & Haviland, M. G. (2016) · Journal of Personality Assessment · doi:10.1080/00223891.2015.1089249 · Cited by 990
Interpretation of Educational Measurements
Freeman, F. S., & Kelley, T. L. (1928) · The American Journal of Psychology · doi:10.2307/1414476 · Cited by 877
Handbook of Psychological Assessment
Groth‐Marnat, G., & Wright, A. J. (2016) · Wiley eBooks · doi:10.1002/9781394258918 · Cited by 769
Cronbach’s Coefficient Alpha: Well Known but Poorly Understood
Cho, E., & Kim, S. (2015) · Organizational Research Methods · doi:10.1177/1094428114555994 · Cited by 764
Graham, J. M. (2006) · Educational and Psychological Measurement · doi:10.1177/0013164406288165 · Cited by 687
Handbook of psychological assessment, 4th ed.
Groth‐Marnat, G. (2003) · Cited by 609
Chen, F. F., Hayes, A., Carver, C. S., Laurenceau, J., & Zhang, Z. (2012) · Journal of Personality · doi:10.1111/j.1467-6494.2011.00739.x · Cited by 536
Limitations of Coefficient Alpha as an Index of Test Unidimensionality1
Green, S. B., Lissitz, R. W., & Mulaik, S. A. (1977) · Educational and Psychological Measurement · doi:10.1177/001316447703700403 · Cited by 522
The New Psychometrics: Science, Psychology and Measurement
Kline, P. (1998) · Cited by 497
Raykov, T. (1997) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3204_2 · Cited by 481
Essentials of WISC-IV Assessment
Lichtenberger, E. O., & Kaufman, A. S. (2000) · Medical Entomology and Zoology · Cited by 473
Coefficient Alpha and the Reliability of Composite Measurements
Novick, M. R., & Lewis, C. (1967) · Psychometrika · doi:10.1007/bf02289400 · Cited by 466
A Tutorial on Hierarchically Structured Constructs
Brunner, M., Nagy, G., & Wilhelm, O. (2012) · Journal of Personality · doi:10.1111/j.1467-6494.2011.00749.x · Cited by 459
Rabin, L. A., Paolillo, E., & Barr, W. B. (2016) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acw007 · Cited by 448
COEFFICIENT ALPHA AND THE RELIABILITY OF COMPOSITE MEASUREMENTS
Novick, M. R., & Lewis, C. (1966) · ETS Research Bulletin Series · doi:10.1002/j.2333-8504.1966.tb00356.x · Cited by 446
Zinbarg, R. E., Yovel, I., Revelle, W., & McDonald, R. P. (2006) · Applied Psychological Measurement · doi:10.1177/0146621605278814 · Cited by 422
Raykov, T. (2001) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711001159582 · Cited by 415 · cited 2×
“…are likely violated in practice, and provided alternatives that are not dependent on such unrealistic assumptions (Green & Yang, 2009; McDonald, 1999; Raykov, 1997, 2001b; Sijtsma, 2009; Simsek & Noyan, 2013; Zinbarg, Revelle, Yovel, & Li, 2005; Zinbarg, Yovel, Revelle, & McDonald, 2006).”
BackgroundAssessment in Special and Inclusive Education
Salvia, Ysseldyke, & Bolt (2010) · Cited by 353
Bias of Coefficient afor Fixed Congeneric Measures with Correlated Errors
Raykov, T. (2001) · Applied Psychological Measurement · doi:10.1177/01466216010251005 · Cited by 338
Commentary on Coefficient Alpha: A Cautionary Tale
Green, S. B., & Yang, Y. (2009) · Psychometrika · doi:10.1007/s11336-008-9098-4 · Cited by 312
Ethical principles of psychologists and code of conduct
American Psychological Association (2002) · American Psychologist · doi:10.1037//0003-066x.57.12.1060 · Cited by 304
Huggins-Manley, A. C. (2022) · Measurement: Interdisciplinary Research and Perspectives · doi:10.1080/15366367.2021.1976089 · Cited by 302
Kelley, K., & Pornprasertmanit, S. (2016) · Psychological Methods · doi:10.1037/a0040086 · Cited by 259
Charter, R. A. (1999) · Journal of Clinical and Experimental Neuropsychology · doi:10.1076/jcen.21.4.559.889 · Cited by 215
Bifactor Modeling and the Estimation of Model-Based Reliability in the WAIS-IV
Gignac, G. E., & Watkins, M. W. (2013) · Multivariate Behavioral Research · doi:10.1080/00273171.2013.804398 · Cited by 200 · cited 3×
“…or confirmatory factor results with a standalone computer program (Watkins, 2013), by hand (Brunner et al., 2012), using a so-called ‘phantom variable’ within confirmatory factor models (Black et al., 2015; Gignac & Watkins, 2013), and within the R (R Development Core Team, 2016) system.”
BackgroundMethodResultSelf-citationBifactor structure of the Wechsler Preschool and Primary Scale of Intelligence—Fourth Edition.
Watkins, M. W., & Beaujean, A. A. (2013) · School Psychology Quarterly · doi:10.1037/spq0000038 · Cited by 190
Self-citationZhang, Z., & Yuan, K.-H. (2016) · Educational and Psychological Measurement · doi:10.1177/0013164415594658 · Cited by 186
A Tutorial on Interpreting Bifactor Model Scores
DeMars, C. E. (2013) · International Journal of Testing · doi:10.1080/15305058.2013.799067 · Cited by 176
Padilla, M. A., & Divers, J. (2016) · Educational and Psychological Measurement · doi:10.1177/0013164415593776 · Cited by 173
Canivez, G. L., & Watkins, M. W. (2010) · Psychological Assessment · doi:10.1037/a0020429 · Cited by 158
Self-citationCorrelated Errors in True Score Models and Their Effect on Coefficient Alpha
Green, S. B., & Hershberger, S. L. (2000) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1207/s15328007sem0702_6 · Cited by 141
Canivez, G. L., Watkins, M. W., & Dombrowski, S. C. (2016) · Psychological Assessment · doi:10.1037/pas0000238 · Cited by 113
Self-citationOn Methodology in the Study of Cognitive Abilities
Carroll, J. B. (1995) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3003_6 · Cited by 112
Psychopathology: History, Diagnosis, and Empirical Foundations
Craighead, W. E., Miklowitz, D. J., & Craighead, L. W. (2008) · Cited by 111
Learning disabilities: The need for neuropsychological evaluation
Silver, C., Ruff, R. M., Iverson, G. L., Barth, J. T., Broshek, D. K., BUSH, S., Koffler, S. P., & REYNOLDS, C. (2007) · Archives of Clinical Neuropsychology · doi:10.1016/j.acn.2007.09.006 · Cited by 111
Measurement and Evaluation in Psychology and Education
Thorndike-Christ (2011) · Cited by 110
Brooks, B. L., Strauss, E., Sherman, E. M. S., Iverson, G. L., & Slick, D. J. (2009) · Canadian Psychology/Psychologie canadienne · doi:10.1037/a0016066 · Cited by 108
Gignac, G. E. (2013) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000181 · Cited by 100
Psychological Assessment Training in Clinical Psychology Doctoral Programs
Mihura, J. L., Roy, M., & Graceffo, R. A. (2016) · Journal of Personality Assessment · doi:10.1080/00223891.2016.1201978 · Cited by 97
Analytic Estimation of Standard Error and Confidence Interval for Scale Reliability
Raykov, T. (2002) · Multivariate Behavioral Research · doi:10.1207/s15327906mbr3701_04 · Cited by 94
Nelson, J. M., Canivez, G. L., & Watkins, M. W. (2013) · Psychological Assessment · doi:10.1037/a0032086 · Cited by 91
Self-citationOn the Changing Role of Cronbach’s α in the Evaluation of the Quality of a Measure
SCHWEIZER, K. W. (2011) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000069 · Cited by 88
McCloskey, G. (2008) · Cited by 88
PROFESSIONAL PRACTICE ISSUES IN THE ASSESSMENT OF COGNITIVE FUNCTIONING FOR EDUCATIONAL APPLICATIONS
Decker, S. L., Hale, J. B., & Flanagan, D. P. (2013) · Psychology in the Schools · doi:10.1002/pits.21675 · Cited by 77
Şimşek, G. G., & Noyan, F. (2013) · Communications in Statistics - Simulation and Computation · doi:10.1080/03610918.2012.689062 · Cited by 68
Donders, J., & Strong, C.-A. H. (2014) · Assessment · doi:10.1177/1073191114530776 · Cited by 66
Charter, R. A., & Feldt, L. S. (2001) · Journal of Clinical and Experimental Neuropsychology · doi:10.1076/jcen.23.4.530.1227 · Cited by 65
Gomez, R., Vance, A., & Watson, S. D. (2016) · Frontiers in Psychology · doi:10.3389/fpsyg.2016.00737 · Cited by 55
On Competence and Ethicality in Psychodiagnostic Assessment
Weiner, I. B. (1989) · Journal of Personality Assessment · doi:10.1207/s15327752jpa5304_18 · Cited by 54
Canivez, G. L., & McGill, R. J. (2016) · Psychological Assessment · doi:10.1037/pas0000279 · Cited by 43
Larrabee, G. J. (2014) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acu049 · Cited by 40
Kelley, K., & Cheng, Y. (2011) · Methodology · doi:10.1027/1614-2241/a000036 · Cited by 39
Cucina, J. M., & Howardson, G. N. (2016) · Psychological Assessment · doi:10.1037/pas0000389 · Cited by 36
Reynolds, M. R., Ingram, P. B., Seeley, J. S., & Newby, K. D. (2013) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2013.06.029 · Cited by 33
Black, R. A., Yang, Y., Beitra, D., & McCaffrey, S. A. (2014) · Journal of Psychoeducational Assessment · doi:10.1177/0734282914553551 · Cited by 27
Estimating Between-Person and Within-Person Subscore Reliability with Profile Analysis
Bulut, O., Davison, M. L., & Rodriguez, M. C. (2016) · Multivariate Behavioral Research · doi:10.1080/00273171.2016.1253452 · Cited by 27
Strickland, T., Watkins, M. W., & Caterino, L. C. (2015) · Psychological Assessment · doi:10.1037/pas0000052 · Cited by 23
Self-citationDiscrepancy Score Reliabilities in the WAIS-IV Standardization Sample
Glass, L. A., Ryan, J. J., & Charter, R. A. (2009) · Journal of Psychoeducational Assessment · doi:10.1177/0734282909346710 · Cited by 22
Raykov, T., & Zinbarg, R. E. (2010) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711009x479714 · Cited by 16
Cui, Y. D., & Li, J. (2012) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/j.2044-8317.2012.02038.x · Cited by 16
The Neuropsychological Evaluation
Caccappolo, E. (2014) · Springer eBooks · doi:10.1007/978-1-4899-7482-2_26 · Cited by 16
Some supplementary methods for the analysis of WAIS‐IV index scores in neuropsychological assessment
Crawford, J. R., Garthwaite, P. H., Longman, R. S., & Batty, A. M. (2011) · Journal of Neuropsychology · doi:10.1111/j.1748-6653.2011.02022.x · Cited by 15
Bifactor model of WISC-IV: Applicability and measurement invariance in low and normal IQ groups.
Gomez, R., Vance, A. L. A., & Watson, S. (2016) · Psychological Assessment · doi:10.1037/pas0000369 · Cited by 14
(2016) · NSUWorks (Nova Southeastern University) · Cited by 13
Confirmatory Factor Analysis of the WAIS-IV and WMS-IV in Older Adults
Miller, D., Davidson, P., Schindler, D. C., & Messier, C. (2013) · Journal of Psychoeducational Assessment · doi:10.1177/0734282912467961 · Cited by 12
Assessment of neuropsychological functioning.
Puente, A. E., & Puente, A. N. (2013) · American Psychological Association eBooks · doi:10.1037/14048-009 · Cited by 11
Frazier, T. W., Youngstrom, E. A., Chelune, G. J., Naugle, R. I., & Lineweaver, T. T. (2004) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617704104049 · Cited by 7
Clinical Manual of Neuropsychiatry
Hales, R. E. (2011) · American Psychiatric Association Publishing eBooks · doi:10.1176/appi.books.9781615378302 · Cited by 3
Cited by
Kalkbrenner, M. T. (2023) · Counseling Outcome Research and Evaluation · doi:10.1080/21501378.2021.1940118 · Cited by 395
Cognitive profile analysis in school psychology: history, issues, and continued concerns
McGill, R. J., Dombrowski, S. C., & Canivez, G. L. (2018) · Journal of School Psychology · doi:10.1016/j.jsp.2018.10.007 · Cited by 91
Canivez, G. L., McGill, R. J., Dombrowski, S. C., Watkins, M. W., Pritchard, A. E., & Jacobson, L. A. (2020) · Assessment · doi:10.1177/1073191118811609 · Cited by 53
Self-citationCanivez, G. L., Dombrowski, S. C., & Watkins, M. W. (2018) · Psychology in the Schools · doi:10.1002/pits.22138 · Cited by 30
Self-citationdi Fronso, S., Robazza, C., Bondár, R. Z., & Bertollo, M. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19127152 · Cited by 22
Metadata
- Authors
- Watkins, M. W.
- Venue
- The Clinical Neuropsychologist
- Publisher
- Informa UK Limited
- Published
- 2017-04-21
- Volume
- 31
- Issue
- 6-7
- Pages
- 1113-1126
- ISSN
- 0920-1637 (ISSN-L)
- 1385-4046
- 1744-4144
- OpenAlex
- W2605982093
- Cited by
- 271
- References
- 89
- FWCI
- 12.33
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.