The DOI matches its Crossref record; 77 of the 77 references it cites were found in a registry or an index.
Applying Bifactor Statistical Indices in the Evaluation of Psychological Measures
Abstract
The purpose of this study was to apply a set of rarely reported psychometric indices that, nevertheless, are important to consider when evaluating psychological measures. All can be derived from a standardized loading matrix in a confirmatory bifactor model: omega reliability coefficients, factor determinacy, construct replicability, explained common variance, and percentage of uncontaminated correlations. We calculated these indices and extended the findings of 50 recent bifactor model estimation studies published in psychopathology, personality, and assessment journals. These bifactor derived indices (most not presented in the articles) provided a clearer and more complete picture of the psychometric properties of the assessment instruments. We reached 2 firm conclusions. First, although all measures had been tagged "multidimensional," unit-weighted total scores overwhelmingly reflected variance due to a single latent variable. Second, unit-weighted subscale scores often have ambiguous interpretations because their variance mostly reflects the general, not the specific, trait. Finally, we review the implications of our evaluations and consider the limits of inferences drawn from a bifactor modeling approach.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Omega is a reliability estimate that does not depend on the assumption of tau equivalence unlike its classic counterpart Cronbach’s alpha [39].”
From Dimensionality of the system usability scale among professionals using internet-based interventions for depression: a confirmatory factor analysis (2020) · cites this paper as [39]
In the citing paper
“When PUC is greater than .70 and ECV greater than .70 relative bias will be slight and the common variance can be regarded as essentially unidimensional [39].”
From Dimensionality of the system usability scale among professionals using internet-based interventions for depression: a confirmatory factor analysis (2020) · cites this paper as [39]
Citations
Cites
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Constructing validity: Basic issues in objective scale development.
Clark, L. A., & Watson, D. (1995) · Psychological Assessment · doi:10.1037/1040-3590.7.3.309 · Cited by 6,567
On the use, the misuse, and the very limited usefulness of Cronbach’s alpha
Sijtsma, K. (2009) · Psychometrika · doi:10.1007/s11336-008-9101-0 · Cited by 2,814
The Scientific Use of Factor Analysis in Behavioral and Life Sciences
Cattell, R. B. (1978) · doi:10.1007/978-1-4684-2262-7 · Cited by 2,165
The Rediscovery of Bifactor Measurement Models
Reise, S. P. (2012) · Multivariate Behavioral Research · doi:10.1080/00273171.2012.715555 · Cited by 2,112 · cited 6×
“…on this equation, it is clear that, as the proportion of variance explained by the general factor increases, ECV will increase, and, as noted by Reise (2012), higher ECV values indicate a strong general factor, which could guide researchers in deciding whether to fit a unidimensional model…”
InfluentialSelf-citationBorsboom, D., Mellenbergh, G. J., & van Heerden, J. (2004) · Psychological Review · doi:10.1037/0033-295x.111.4.1061 · Cited by 1,733
Coefficients Alpha, Beta, Omega, and the glb: Comments on Sijtsma
Revelle, W., & Zinbarg, R. E. (2008) · Psychometrika · doi:10.1007/s11336-008-9102-z · Cited by 1,652
Zinbarg, R. E., Revelle, W., Yovel, I., & Li, W. (2005) · Psychometrika · doi:10.1007/s11336-003-0974-7 · Cited by 1,439 · cited 1×
“For more details, we refer readers to the original sources as well as to reviews (e.g., McDonald, 1999; Reise, Moore, & Haviland, 2013; Revelle & Zinbarg, 2009; Zinbarg et al., 2005; Zinbarg, Yovel, Revelle, & McDonald, 2006).”
MethodOsman, A., Wong, J. L., Bagge, C. L., Freedenthal, S. L., Gutierrez, P. M., & Lozano, G. (2012) · Journal of Clinical Psychology · doi:10.1002/jclp.21908 · Cited by 1,097
Scoring and Modeling Psychological Measures in the Presence of Multidimensionality
Reise, S. P., Bonifay, W. E., & Haviland, M. G. (2013) · Journal of Personality Assessment · doi:10.1080/00223891.2012.725437 · Cited by 1,069
Self-citationReise, S. P., Scheines, R., Widaman, K. F., & Haviland, M. G. (2013) · Educational and Psychological Measurement · doi:10.1177/0013164412449831 · Cited by 766
Self-citationThe role of the bifactor model in resolving dimensionality issues in health outcomes measures
Reise, S. P., Morizot, J., & Hays, R. D. (2007) · Quality of Life Research · doi:10.1007/s11136-007-9183-7 · Cited by 747
Self-citationComputing and evaluating factor scores.
Grice, J. W. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.4.430 · Cited by 653
Does item homogeneity indicate internal consistency or item redundancy in psychometric scales?
Boyle, G. J. (1991) · Personality and Individual Differences · doi:10.1016/0191-8869(91)90115-r · Cited by 495
The Hospital Anxiety and Depression Scale: A meta confirmatory factor analysis
Norton, S., Cosco, T. D., Doyle, F., Done, J., & Sacker, A. (2012) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2012.10.010 · Cited by 426
Zinbarg, R. E., Yovel, I., Revelle, W., & McDonald, R. P. (2006) · Applied Psychological Measurement · doi:10.1177/0146621605278814 · Cited by 422
Alpha, Dimension-Free, and Model-Based Internal Consistency Reliability
Bentler, P. M. (2008) · Psychometrika · doi:10.1007/s11336-008-9100-1 · Cited by 404
The Greatest Lower Bound to the Reliability of a Test and the Hypothesis of Unidimensionality
ten Berge, J. M. F., & Sočan, G. (2004) · Psychometrika · doi:10.1007/bf02289858 · Cited by 334
When Can Subscores Have Value?
Haberman, S. J. (2008) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998607302636 · Cited by 240
The State-Trait Anxiety Inventory, Trait Version: Does It Really Measure Anxiety?
Bados, A., Gómez‐Benito, J., & Fort, G. B. (2010) · Journal of Personality Assessment · doi:10.1080/00223891.2010.513295 · Cited by 215
Ciucci, E., Baroncelli, A., Franchi, M., Golmaryami, F. N., & Frick, P. J. (2013) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-013-9384-z · Cited by 181
Iani, L., Lauriola, M., & Costantini, M. (2014) · Health and Quality of Life Outcomes · doi:10.1186/1477-7525-12-84 · Cited by 158
Jonason, P. K., & Luévano, V. X. (2013) · Personality and Individual Differences · doi:10.1016/j.paid.2013.02.010 · Cited by 157
The Big Five Personality Factors as Predictors of Changes Across Time in Burnout and Its Facets
Armon, G., Shirom, A., & Melamed, S. (2011) · Journal of Personality · doi:10.1111/j.1467-6494.2011.00731.x · Cited by 150
Using Logistic Approximations of Marginal Trace Lines to Develop Short Assessments
Stucky, B. D., Thissen, D., & Edelen, M. O. (2012) · Applied Psychological Measurement · doi:10.1177/0146621612462759 · Cited by 142
Toplak, M. E., Pitch, A., Flora, D. B., Iwenofu, L., Ghelani, K., Jain, U., & Tannock, R. (2009) · Research on Child and Adolescent Psychopathology · doi:10.1007/s10802-009-9336-y · Cited by 141
Osman, A., Gutierrez, P. M., Smith, K. D., Fang, Q., Lozano, G., & Devine, A. (2010) · Journal of Personality Assessment · doi:10.1080/00223890903379332 · Cited by 139
Bonifay, W. E., Reise, S. P., Scheines, R., & Meijer, R. R. (2015) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2014.938596 · Cited by 139
Self-citationByrd, A., Kahn, R. E., & Pardini, D. (2012) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-012-9315-4 · Cited by 136
Testing the Psychopathology of Psychosis: Evidence for a General Psychosis Dimension
Reininghaus, U., Priebe, S., & Bentall, R. P. (2012) · Schizophrenia Bulletin · doi:10.1093/schbul/sbr182 · Cited by 129
Using Hierarchical IRT Models to Create Unidimensional Measures From Multidimensional Data
Stucky, B. D., & Edelen, M. O. (2014) · doi:10.4324/9781315736013.ch9 · Cited by 120
Unidimensionality and interpretability of psychological instruments.
Gustafsson, J., & Åberg‐Bengtsson, L. (2010) · American Psychological Association eBooks · doi:10.1037/12074-005 · Cited by 115
Gignac, G. E., Palmer, B. R., & Stough, C. (2007) · Journal of Personality Assessment · doi:10.1080/00223890701629730 · Cited by 104
On the dimensionality of the Dispositional Hope Scale.
Brouwer, D., Meijer, R. R., Weekers, A. M., & Baneke, J. J. (2008) · Psychological Assessment · doi:10.1037/1040-3590.20.3.310 · Cited by 101
Freedenthal, S. L., Lamis, D. A., Osman, A., Kahlo, D. M., & Gutierrez, P. M. (2011) · Journal of Clinical Psychology · doi:10.1002/jclp.20782 · Cited by 98
The Rosenberg Self-Esteem Scale: A Bifactor Answer to a Two-Factor Question?
McKay, M., Boduszek, D., & Harvey, S. A. (2014) · Journal of Personality Assessment · doi:10.1080/00223891.2014.923436 · Cited by 98
Psychometric Evaluation of the Perceived Stress Scale in Multiple Sclerosis
Wu, S. M., & Amtmann, D. (2013) · ISRN Rehabilitation · doi:10.1155/2013/608356 · Cited by 96
An NCME Instructional Module on Subscores
Sinharay, S., Puhan, G., & Haberman, S. J. (2011) · Educational Measurement Issues and Practice · doi:10.1111/j.1745-3992.2011.00208.x · Cited by 90
The hierarchical factor model of ADHD: invariant across age and national groupings?
Toplak, M. E., Sorge, G. B., Flora, D. B., Chen, W., Banaschewski, T., Buitelaar, J. K., Ebstein, R. P., Eisenberg, J., Franke, B., Gill, M., et al. (2011) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.2011.02500.x · Cited by 89
What Lies Beneath the Dark Triad Dirty Dozen: Varied Relations with the Big Five
Jonason, P. K., Kaufman, S. B., Webster, G. D., & Geher, G. (2013) · Individual Differences Research · doi:10.65030/idr.11008 · Cited by 89
Structural equation modeling: Present and future. A festschrift in honor of Karl Joreskog.
Bunting, B. P. (2001) · British Journal of Mathematical and Statistical Psychology · Cited by 89
Martel, M. M., von Eye, A., & Nigg, J. T. (2012) · International Journal of Behavioral Development · doi:10.1177/0165025412444077 · Cited by 77
A Bifactor Model of Anxiety Sensitivity: Analysis of the Anxiety Sensitivity Index-3
Ebesutani, C. K., McLeish, A. C., Luberto, C. M., Young, J. N., & Maack, D. J. (2013) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-013-9400-3 · Cited by 72
Evidence for a General Factor Model of ADHD in Adults
Gibbins, C., Toplak, M. E., Flora, D. B., Weiss, M. D., & Tannock, R. (2011) · Journal of Attention Disorders · doi:10.1177/1087054711416310 · Cited by 69
Reichenheim, M. E., de Moraes, C. L., de Oliveira, A. S. D., & Lobato, G. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-93 · Cited by 69
Reporting subscores for institutions
Haberman, S. J., Sinharay, S., & Puhan, G. (2008) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711007x248875 · Cited by 63
Luciano, J. V., Barrada, J. R., Aguado, J., Osma, J., & García-Campayo, J. (2014) · Psychological Assessment · doi:10.1037/a0035284 · Cited by 62
Reynolds Adolescent Depression Scale-second edition: a reliable and useful instrument
Osman, A., Gutierrez, P. M., Bagge, C. L., Fang, Q., & Emmerich, A. (2010) · Journal of Clinical Psychology · doi:10.1002/jclp.20727 · Cited by 62
An Item Response Theory Analysis of the Narcissistic Personality Inventory
Ackerman, R. A., Donnellan, M. B., & Robins, R. W. (2012) · Journal of Personality Assessment · doi:10.1080/00223891.2011.645934 · Cited by 62
Olatunji, B. O., Ebesutani, C. K., Haidt, J., & Sawchuk, C. N. (2014) · Behavior Therapy · doi:10.1016/j.beth.2014.02.006 · Cited by 58
Ebesutani, C. K., Smith, A. M., Bernstein, A., Chorpita, B. F., Higa-McMillan, C. K., & Nakamura, B. J. (2011) · Psychological Assessment · doi:10.1037/a0023234 · Cited by 55
Wiesner, M., & Schanding, G. T. (2013) · Journal of School Psychology · doi:10.1016/j.jsp.2013.09.001 · Cited by 54
Yap, S. C. Y., Donnellan, M. B., Schwartz, S. J., Kim, S. Y., Castillo, L. G., Zamboanga, B. L., Weisskirch, R. S., Lee, R. M., Park, I. J. K., Whitbourne, S. K., et al. (2014) · Journal of Counseling Psychology · doi:10.1037/a0036253 · Cited by 52
Xie, J., Bi, Q., li, W., Shang, W., Yan, M., Yang, Y., Miao, D., & Zhang, H. (2012) · PLoS ONE · doi:10.1371/journal.pone.0047577 · Cited by 50
A psychometric examination of the Interpersonal Sexual Objectification Scale among college men.
Davidson, M. M., Gervais, S. J., Canivez, G. L., & Cole, B. P. (2013) · Journal of Counseling Psychology · doi:10.1037/a0032075 · Cited by 42
Yang, F. M., Tommet, D., & Jones, R. N. (2009) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2008.12.007 · Cited by 38
Gomez, R., Vance, A. L. A., & Gomez, R. M. (2013) · Journal of Attention Disorders · doi:10.1177/1087054713480034 · Cited by 37
Yang, F. M., & Jones, R. N. (2008) · Psychosomatic Medicine · doi:10.1097/psy.0b013e31818ce4fa · Cited by 37
Applying unidimensional item response theory models to psychological data.
Reise, S. P., Moore, T. M., & Haviland, M. G. (2013) · American Psychological Association eBooks · doi:10.1037/14047-006 · Cited by 37
Self-citationIndeterminacy of Factor Score Estimates In Slightly Misspecified Confirmatory Factor Models
Beauducel, A. (2011) · Journal of Modern Applied Statistical Methods · doi:10.22237/jmasm/1320120900 · Cited by 33
Dimensionality Assessment Using the Full-Information Item Bifactor Analysis for Graded Response Data
Immekus, J. C., & Imbrie, P. K. (2008) · Educational and Psychological Measurement · doi:10.1177/0013164407313366 · Cited by 32
Stratton, K. J., Aggen, S. H., Richardson, L., Acierno, R. E., Kilpatrick, D. G., Gaboury, M. T., Tran, T. L., Trung, L. T., Tâm, N. T., Tran, T., et al. (2012) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2012.10.011 · Cited by 31
Gomez, R. (2012) · Personality and Individual Differences · doi:10.1016/j.paid.2012.11.025 · Cited by 31
A bifactor model of the Posttraumatic Growth Inventory
Thege, B. K., Kovács, É., & Balog, P. (2014) · Health Psychology and Behavioral Medicine · doi:10.1080/21642850.2014.905208 · Cited by 30
Further Evidence of the Reliability and Validity of the Multidimensional Anxiety
Osman, A., Williams, J. E., Espenschade, K., Gutierrez, P. M., Bailey, J., & Chowdhry, O. (2008) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-008-9095-z · Cited by 28
The Relationship Between Life Satisfaction and ADHD Symptoms in Middle School Students
Ogg, J. A., Bateman, L. P., Dedrick, R. F., & Suldo, S. M. (2014) · Journal of Attention Disorders · doi:10.1177/1087054714521292 · Cited by 26
The eACS: Attentional control in the presence of emotion
Barry, T. J., Hermans, D., Lenaert, B., Debeer, E., & Griffith, J. W. (2013) · Personality and Individual Differences · doi:10.1016/j.paid.2013.06.014 · Cited by 26
Psychometric Examination of the Interpersonal Sexual Objectification Scale Among College Men
Davidson, M. M., Gervais, S. J., Canivez, G. L., & Cole, B. P. (2013) · PsycEXTRA Dataset · doi:10.1037/e594022013-001 · Cited by 25
A Note on Assessing the Added Value of Subscores
Sinharay, S. (2013) · Educational Measurement Issues and Practice · doi:10.1111/emip.12021 · Cited by 23
Innamorati, M., Balsamo, M., Fairfield, B., Fabbricatore, M., Tamburello, A., & Saggino, A. (2014) · Depression Research and Treatment · doi:10.1155/2014/972424 · Cited by 21
A Bifactor Model of Disgust Proneness
Olatunji, B. O., Ebesutani, C. K., & Reise, S. P. (2014) · Assessment · doi:10.1177/1073191114541673 · Cited by 18
Self-citationCoco, A. L., Ingoglia, S., & Lundqvist, L. (2013) · Journal of Nonverbal Behavior · doi:10.1007/s10919-013-0166-9 · Cited by 17
Development and Validation of an Inventory for Measuring Student Attitudes Toward Calculus
Huang, Y., & Lin, S.-H. (2014) · Measurement and Evaluation in Counseling and Development · doi:10.1177/0748175614563314 · Cited by 16
Zhang, J., Cui, M., Wang, W., Lu, H., Wu, Q., Zhu, X., Miao, D., Zhang, Y., Feng, X., & Xiao, W. (2014) · PLoS ONE · doi:10.1371/journal.pone.0096451 · Cited by 13
ADHD bifactor model based on parent and teacher ratings of Malaysian children
Gomez, R. (2013) · Asian Journal of Psychiatry · doi:10.1016/j.ajp.2013.10.009 · Cited by 11
Assessing Usual Seasonal Depression Symptoms: The Seasonality Assessment Form
Young, M. A., Hutman, P., Enggasser, J. L., & Meesters, Y. (2014) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-014-9440-3 · Cited by 9
The factor structure and composite reliability of the Profile of Emotional Distress
Hyland, P., Shevlin, M. E., Adamson, G., & Boduszek, D. (2013) · The Cognitive Behaviour Therapist · doi:10.1017/s1754470x13000214 · Cited by 8
Research works printed but not linked: 5
- Gorsuch R. L., 1983, Factor analysis
- *Jonason P. K., 2013, Individual Differences Research, 11, 81
- McDonald R. P., 1999, Test theory: A unified approach
- Reise S. P., 2013, APA Handbook of Testing and Assessment in Psychology
- Stucky B. D., 2014, Handbook of item response theory modeling: Applications to typical performance assessment, 183
Cited by
Sorted by citations
Ferrando, P. J., & Lorenzo‐Seva, U. (2017) · Educational and Psychological Measurement · doi:10.1177/0013164417719308 · Cited by 630
Neff, K. D., Tóth-Király, I., Yarnell, L. M., Arimitsu, K., Castilho, P., Ghorbani, N., Guo, H. X., Hirsch, J. K., Hupfeld, J., Hutz, C. S., et al. (2019) · Psychological Assessment · doi:10.1037/pas0000629 · Cited by 397
Three Concerns With Applying a Bifactor Model as a Structure of Psychopathology
Bonifay, W. E., Lane, S. P., & Reise, S. P. (2016) · Clinical Psychological Science · doi:10.1177/2167702616657069 · Cited by 423
Self-citationZanon, C., Brenner, R. E., Baptista, M. N., Vogel, D. L., Rubin, M., Al-Darmaki, F. R., Gonçalves, M., Heath, P. J., Liao, H.-Y., Mackenzie, C. S., et al. (2020) · Assessment · doi:10.1177/1073191119887449 · Cited by 429
Twenty-five years with the 20-item Toronto Alexithymia Scale
Bagby, R. M., Parker, J. D. A., & Taylor, G. J. (2020) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2020.109940 · Cited by 334
Metadata
- Authors
- Rodriguez, A., Reise, S. P., & Haviland, M. G.
- Affiliations
- University of California, Los Angeles (US)
- Loma Linda University (US)
- Corresponding author
- Reise, Steven P.
- Venue
- Journal of Personality Assessment
- Publisher
- Informa UK Limited
- Published
- 2015-10-29
- Volume
- 98
- Issue
- 3
- Pages
- 223-237
- ISSN
- 0022-3891 (ISSN-L)
- 1532-7752
- OpenAlex
- W1881923603
- Cited by
- 990
- References
- 77
- Citations by year
- 2026: 105 · 2025: 97 · 2024: 104 · 2023: 134 · 2022: 127 · 2021: 124 · 2020: 102 · 2019: 75 · 2018: 59 · 2017: 49 · 2016: 13
- FWCI
- 50.26
- Type
- journal article
- Language
- en
- Update notices
- correction (2015-12-23)
- Open access
- closed
- Locations
- Journal of Personality Assessment (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Social Sciences › Psychology › Applied Psychology › Psychological Testing and Assessment
- Keywords
- bifactor model, multidimensional measure, personality assessment, psychopathology assessment
- MeSH terms
- Factor Analysis, Statistical, Humans, Models, Psychological, Psychometrics, Models, Statistical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review