The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index.
Measurement invariance of the patient health questionnaire‐9 (PHQ‐9) depression screener in U.S. adults across sex, race/ethnicity, and education level: NHANES 2005–2016
Open accessAbstract
BACKGROUND: Despite its popularity, little is known about the measurement invariance of the Patient Health Questionnaire-9 (PHQ-9) across U.S. sociodemographic groups. Use of a screener shown not to possess measurement invariance could result in under/over-detection of depression, potentially exacerbating sociodemographic disparities in depression. Therefore, we assessed the factor structure and measurement invariance of the PHQ-9 across major U.S. sociodemographic groups. METHODS: U.S. population representative data came from the 2005-2016 National Health and Nutrition Examination Survey (NHANES) cohorts. We conducted a measurement invariance analysis of 31,366 respondents across sociodemographic factors of sex, race/ethnicity, and education level. RESULTS: Considering results of single-group confirmatory factor analyses (CFAs), depression theory, and research utility, we justify a two-factor structure for the PHQ-9 consisting of a cognitive/affective factor and a somatic factor (RMSEA = 0.034, TLI = 0.985, CFI = 0.989). On the basis of multiple-group CFAs testing configural, scalar, and strict factorial invariance, we determined that invariance held for sex, race/ethnicity, and education level groups, as all models demonstrated close model fit (RMSEA = 0.025-0.025, TLI = 0.985-0.992, CFI = 0.986-0.991). Finally, for all steps ΔCFI was <-0.004, and ΔRMSEA was <0.01. CONCLUSIONS: We demonstrate that the PHQ-9 is acceptable to use in major U.S. sociodemographic groups and allows for meaningful comparisons in total, cognitive/affective, and somatic depressive symptoms across these groups, extending its use to the community. This knowledge is timely as medicine moves towards alternative payment models emphasizing high-quality and cost-efficient care, which will likely incentivize behavioral and population health efforts. We also provide a consistent, evidence-based approach for calculating PHQ-9 subscale scores.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The PHQ-8 has good reliability (α=.86-.89) and has shown validity across different racial and ethnic groups [69].”
From App-Based Mindfulness Meditation for People of Color Who Experience Race-Related Stress: Protocol for a Randomized Controlled Trial (2022) · cites this paper as [69]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910
Self-citationConfirmatory factor analysis for applied research
Brown, T. A. (2007) · Choice Reviews Online · doi:10.5860/choice.44-2769 · Cited by 15,619
Spitzer, R. L. (1999) · JAMA · doi:10.1001/jama.282.18.1737 · Cited by 9,802
Kessler, R. C., Berglund, P., Demler, O., Jin, R., Koretz, D., Merikangas, K. R., Rush, A. J., Walters, E. E., & Wang, P. S. (2003) · JAMA · doi:10.1001/jama.289.23.3095 · Cited by 7,988
Selective Cortical Undercutting As a Means of Modifying and Studying Frontal Lobe Function in Man
Scoville, W. B. (1949) · Journal of neurosurgery · doi:10.3171/jns.1949.6.1.0065 · Cited by 7,374
The PHQ-9: A New Depression Diagnostic and Severity Measure
Kroenke, K., & Spitzer, R. L. (2002) · Psychiatric Annals · doi:10.3928/0048-5713-20020901-06 · Cited by 5,898
Self-citationOverweight, Obesity, and Depression: A Systematic Review and Meta-analysis of Longitudinal Studies
Luppino, F. S., de Wit, L. M., Bouvy, P. F., Stijnen, T., Cuijpers, P., Penninx, B. W. J. H., & Zitman, F. G. (2010) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2010.2 · Cited by 4,322
Ferrari, A. J., Charlson, F. J., Norman, R. E., Patten, S. B., Freedman, G., Murray, C. J. L., Vos, T., & Whiteford, H. A. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001547 · Cited by 3,115
Rhemtulla, M., Brosseau-Liard, P. É., & Savalei, V. (2012) · Psychological Methods · doi:10.1037/a0029315 · Cited by 2,816
Utility of a new procedure for diagnosing mental disorders in primary care. The PRIME-MD 1000 study
Spitzer, R. L. (1994) · JAMA · doi:10.1001/jama.272.22.1749 · Cited by 2,648
Socioeconomic inequalities in depression: a meta-analysis
Lorant, V. (2003) · American Journal of Epidemiology · doi:10.1093/aje/kwf182 · Cited by 2,603
Utility of a New Procedure for Diagnosing Mental Disorders in Primary Care: The PRIME-MD 1000 Study
Spitzer, R. L. (1994) · JAMA · doi:10.1001/jama.1994.03520220043029 · Cited by 1,873
The Economic Burden of Adults With Major Depressive Disorder in the United States (2005 and 2010)
Greenberg, P. E., Fournier, A.-A., Sisitsky, T., Pike, C. T., & Kessler, R. C. (2015) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.14m09298 · Cited by 1,779
Reconsidering anhedonia in depression: Lessons from translational neuroscience
Treadway, M. T., & Zald, D. H. (2011) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2010.06.006 · Cited by 1,579
Screening for Depression in Adults: US Preventive Services Task Force Recommendation Statement
Siu, A. L., and the US Preventive Services Task Force (USPSTF), Bibbins-Domingo, K., Grossman, D. C., Baumann, L. C., Davidson, K. W., Ebell, M., García, F. A. R., Gillman, M., Herzstein, J., et al. (2016) · JAMA · doi:10.1001/jama.2015.18392 · Cited by 1,407
Diniz, B. S., Butters, M. A., Albert, S. M., Dew, M. A., & Reynolds, C. F. (2013) · British Journal of Psychiatry · doi:10.1192/bjp.bp.112.118307 · Cited by 1,290
The Economic Burden of Depression in the United States: How Did It Change Between 1990 and 2000?
Greenberg, P. E., Kessler, R. C., Birnbaum, H. G., Leong, S. A., Lowe, S. W., Berglund, P. A., & Corey-Lisle, P. K. (2003) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v64n1211 · Cited by 1,280
Drevets, W. C. (2001) · Current Opinion in Neurobiology · doi:10.1016/s0959-4388(00)00203-8 · Cited by 1,146
Setting Value-Based Payment Goals — HHS Efforts to Improve U.S. Health Care
Burwell, S. M. (2015) · New England Journal of Medicine · doi:10.1056/nejmp1500445 · Cited by 935
Do Self-Report Instruments Allow Meaningful Comparisons Across Diverse Population Groups?
Gregorich, S. E. (2006) · Medical Care · doi:10.1097/01.mlr.0000245454.12228.8f · Cited by 802
Kocalevent, R.-D., Hinz, A., & Brähler, E. (2013) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2013.04.006 · Cited by 748
Little, T. D., Jorgensen, T. D., Lang, K. M., & Moore, E. W. G. (2013) · Journal of Pediatric Psychology · doi:10.1093/jpepsy/jst048 · Cited by 745
Huang, F. Y., Chung, H., Kroenke, K., Delucchi, K., & Spitzer, R. L. (2006) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2006.00409.x · Cited by 688
Self-citationCuijpers, P., Vogelzangs, N., Twisk, J., Kleiboer, A., Li, J., & Penninx, B. W. (2014) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2013.13030325 · Cited by 677
Psychometric comparison of PHQ-9 and HADS for measuring depression severity in primary care
Cameron, I. M., Crawford, J. R., Lawton, K., & Reid, I. C. (2008) · British Journal of General Practice · doi:10.3399/bjgp08x263794 · Cited by 590
Depression and the risk of coronary heart disease: a meta-analysis of prospective cohort studies
Gan, Y., Gong, Y., Tong, X., Sun, H., Cong, Y., Dong, X., Wang, Y., Xu, X., Yin, X., Deng, J., et al. (2014) · BMC Psychiatry · doi:10.1186/s12888-014-0371-z · Cited by 502
Factor Analytic Models: Viewing the Structure of an Assessment Instrument From Three Perspectives
Byrne, B. M. (2005) · Journal of Personality Assessment · doi:10.1207/s15327752jpa8501_02 · Cited by 496
Prevalence and predictors of recurrence of major depressive disorder in the adult population
Hardeveld, F., Spijker, J., De Graaf, R., Nolen, W. A., & Beekman, A. T. F. (2010) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.2009.01519.x · Cited by 486
Sass, D. A., Schmitt, T. A., & Marsh, H. W. (2014) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2014.882658 · Cited by 456
Moriarty, A. S., Gilbody, S., McMillan, D., & Manea, L. (2015) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2015.06.012 · Cited by 372
Depression as a Risk Factor for Diabetes: A Meta-Analysis of Longitudinal Studies
Rotella, F., & Mannucci, E. (2013) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.12r07922 · Cited by 360
Mitchell, A. J., Yadegarfar, M., Gill, J., & Stubbs, B. (2016) · BJPsych Open · doi:10.1192/bjpo.bp.115.001685 · Cited by 354
Duivis, H. E., Vogelzangs, N., Kupper, N., de Jonge, P., & Penninx, B. W. J. H. (2013) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2013.01.002 · Cited by 282
Keum, B. T., Miller, M. J., & Inkelas, K. K. (2018) · Psychological Assessment · doi:10.1037/pas0000550 · Cited by 235
de Jonge, P., Mangano, D. T., & Whooley, M. A. (2007) · Psychosomatic Medicine · doi:10.1097/psy.0b013e31815743ca · Cited by 234
Kroenke, K., & Unutzer, J. (2017) · Journal of General Internal Medicine · doi:10.1007/s11606-016-3967-9 · Cited by 231
Self-citationMeasurement Invariance in International Surveys: Categorical Indicators and Fit Measure Performance
Rutkowski, L., & Svetina, D. (2017) · Applied Measurement in Education · doi:10.1080/08957347.2016.1243540 · Cited by 225
Zuithoff, N. P., Vergouwe, Y., King, M., Nazareth, I., van Wezep, M. J., Moons, K. G., & Geerlings, M. I. (2010) · BMC Family Practice · doi:10.1186/1471-2296-11-98 · Cited by 218
Merz, E. L., Malcarne, V. L., Roesch, S. C., Riley, N. E., & Sadler, G. R. (2011) · Cultural Diversity & Ethnic Minority Psychology · doi:10.1037/a0023883 · Cited by 194
Racial Disparities in Diagnosis and Treatment of Depression: A Literature Review
Simpson, S., Krishnan, L. L., Kunik, M. E., & Ruiz, P. (2006) · Psychiatric Quarterly · doi:10.1007/s11126-006-9022-y · Cited by 178
Smolderen, K. G. E., Spertus, J. A., Reid, K. J., Buchanan, D. M., Krumholz, H. M., Denollet, J., Vaccarino, V., & Chan, P. S. (2009) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.109.868588 · Cited by 177
Elhai, J. D., Contractor, A. A., Tamburrino, M., Fine, T. H., Prescott, M. R., Shirley, E. R., Chan, P. K., Slembarski, R., Liberzon, I., Galea, S., et al. (2012) · Psychiatry Research · doi:10.1016/j.psychres.2012.05.018 · Cited by 158
McIntyre, R. S., Xiao, H. X., Syeda, K., Vinberg, M., Carvalho, A. F., Mansur, R. B., Maruschak, N., & Cha, D. S. (2015) · CNS Drugs · doi:10.1007/s40263-015-0263-x · Cited by 145
Richardson, E. J., & Richards, J. S. (2008) · Rehabilitation Psychology · doi:10.1037/0090-5550.53.2.243 · Cited by 142
Comparing the BDI-II and the PHQ-9 with outpatient substance abusers
Dum, M., Pickren, J., Sobell, L. C., & Sobell, M. B. (2007) · Addictive Behaviors · doi:10.1016/j.addbeh.2007.09.017 · Cited by 141
Prina, M. A., Cosco, T. D., Dening, T., Beekman, A. T. F., Brayne, C., & Huisman, M. A. (2014) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2014.11.002 · Cited by 131
Luber, M. P., Hollenberg, J. P., Williams-Russo, P. G., DiDomenico, T. N., Meyers, B. S., Alexopoulos, G. S., & Charlson, M. E. (2000) · The International Journal of Psychiatry in Medicine · doi:10.2190/ytry-e86m-g1vc-lc79 · Cited by 126
Bock, J., Luppa, M., Brettschneider, C., Riedel-Heller, S. G., Bickel, H., Fuchs, Â., Gensichen, J. S., Maier, W., Mergenthal, K., Schäfer, I., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0091973 · Cited by 120
González‐Blanch, C., Medrano, L. A., Muñoz‐Navarro, R., Ruiz‐Rodríguez, P., Moriana, J. A., Limonero, J. T., Schmitz, F., & Vindel, A. C. (2018) · PLoS ONE · doi:10.1371/journal.pone.0193356 · Cited by 114
Hoen, P. W., Whooley, M. A., Martens, E. J., Na, B., van Melle, J. P., & de Jonge, P. (2010) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2010.03.080 · Cited by 113
The factor structure of the PHQ-9 in palliative care
Chilcot, J., Rayner, L., Lee, W. E., Price, A. C., Goodwin, L., Monroe, B., Sykes, N. P., Hansford, P., & Hotopf, M. H. (2013) · Journal of Psychosomatic Research · doi:10.1016/j.jpsychores.2012.12.012 · Cited by 111
Krause, J. S., Reed, K. S., & McArdle, J. J. (2010) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2010.04.015 · Cited by 104
Measurement invariance testing of the PHQ-9 in a multi-ethnic population in Europe: the HELIUS study
Galenkamp, H., Stronks, K., Snijder, M. B., & Derks, E. M. (2017) · BMC Psychiatry · doi:10.1186/s12888-017-1506-9 · Cited by 98
Krause, J. S., Bombardier, C. H., & Carter, R. E. (2008) · Rehabilitation Psychology · doi:10.1037/a0013354 · Cited by 95
Measurement invariance with respect to ethnicity of the Patient Health Questionnaire-9 (PHQ-9)
Baas, K. D., Cramer, A. O. J., Koeter, M. W. J., van de Lisdonk, E. H., van Weert, H. C. P. M., & Schene, A. H. (2010) · Journal of Affective Disorders · doi:10.1016/j.jad.2010.08.026 · Cited by 95
Petersen, J. J., Paulitsch, M., Hartig, J., Mergenthal, K., Gerlach, F. M., & Gensichen, J. S. (2014) · Journal of Affective Disorders · doi:10.1016/j.jad.2014.08.053 · Cited by 91
Financial Incentives and Vulnerable Populations — Will Alternative Payment Models Help or Hurt?
Maddox, K. E. J. (2018) · New England Journal of Medicine · doi:10.1056/nejmp1715455 · Cited by 88
Roest, A. M., Thombs, B. D., Grace, S. L., Stewart, D. E., Abbey, S. E., & de Jonge, P. (2010) · Journal of Affective Disorders · doi:10.1016/j.jad.2010.11.018 · Cited by 78
Crane, P. K., Gibbons, L. E., Willig, J. H., Mugavero, M. J., Lawrence, S. T., Schumacher, J. E., Saag, M. S., Kitahata, M. M., & Crane, H. M. (2010) · AIDS Care · doi:10.1080/09540120903483034 · Cited by 75
Kalpakjian, C. Z., Toussaint, L. L., Albright, K. J., Bombardier, C. H., Krause, J. K., & Tate, D. G. (2009) · Journal of Spinal Cord Medicine · doi:10.1080/10790268.2009.11760766 · Cited by 63
Case, S. M., & Stewart, J. C. (2014) · Brain Behavior and Immunity · doi:10.1016/j.bbi.2014.04.004 · Cited by 63
Hirschtritt, M. E., & Kroenke, K. (2017) · JAMA · doi:10.1001/jama.2017.9820 · Cited by 58
Self-citationDepression profile in patients with and without chronic heart failure
Holzapfel, N., Müller‐Tasch, T., Wild, B., Jünger, J., Zugck, C., Remppis, A. B., Herzog, W., & Löwe, B. (2007) · Journal of Affective Disorders · doi:10.1016/j.jad.2007.04.009 · Cited by 52
Racial/Ethnic Disparities in Depression and Its Theoretical Perspectives
Kim, M. (2013) · Psychiatric Quarterly · doi:10.1007/s11126-013-9265-3 · Cited by 48
Roest, A. M., Carney, R. M., Freedland, K. E., Martens, E. J., Denollet, J., & de Jonge, P. (2013) · Journal of Affective Disorders · doi:10.1016/j.jad.2013.02.008 · Cited by 44
Cameron, I. M., Crawford, J. R., Lawton, K., & Reid, I. (2012) · Journal of Affective Disorders · doi:10.1016/j.jad.2012.11.015 · Cited by 44
Vrany, E. A., Berntson, J., Khambaty, T., & Stewart, J. C. (2015) · Annals of Behavioral Medicine · doi:10.1007/s12160-015-9725-0 · Cited by 38
Self-citationUS Preventive Services Task Force Recommendation Statement on Screening for Depression in Adults
Reynolds, C. F., & Frank, E. (2016) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2015.3281 · Cited by 34
Recommendations for Screening for Depression in Adults
Thase, M. E. (2016) · JAMA · doi:10.1001/jama.2015.18406 · Cited by 27
Accountable Care at the Frontlines of a Health System
Ganguli, I., & Ferris, T. G. G. (2017) · JAMA · doi:10.1001/jama.2017.18995 · Cited by 10
Flagged first, then by citations
Cited by
Sorted by citations
Villarreal‐Zegarra, D., Copez-Lonzoy, A. J. E., Bernabé‐Ortiz, A., Meléndez‐Torres, G. J., & Bazo‐Alvarez, J. C. (2019) · PLoS ONE · doi:10.1371/journal.pone.0221717 · Cited by 192
Zeng, Y., Hu, C., Li, Y.-Z., Zhou, J., Wang, S., Liu, M.-D., Qiu, Z., Deng, C., Ma, F., Xia, C.-F., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02929-4 · Cited by 167
Lamela, D. J. P. D. V., Soreira, C. S. S., Matos, P., & Morais, A. (2020) · Journal of Affective Disorders · doi:10.1016/j.jad.2020.06.066 · Cited by 151
Is the PHQ-9 a unidimensional measure of depression? A 58,272-participant study.
Bianchi, R., Verkuilen, J., Toker, S., Schonfeld, I. S., Gerber, M., Brähler, E., & Kroenke, K. (2022) · Psychological Assessment · doi:10.1037/pas0001124 · Cited by 125
Self-citationRahman, M. A., Dhira, T. A., Sarker, A. R., & Mehareen, J. (2022) · PLoS ONE · doi:10.1371/journal.pone.0269634 · Cited by 113
Metadata
- Authors
- Patel, J. S., Oh, Y., Rand, K. L., Wu, W., Cyders, M. A., Kroenke, K., & Stewart, J. C.
- Affiliations
- Indiana University – Purdue University Indianapolis (US)
- Texas Tech University (US)
- Regenstrief Institute (US)
- Corresponding author
- Stewart, Jesse C.
- Venue
- Depression and Anxiety
- Publisher
- Wiley
- Published
- 2019-07-29
- Volume
- 36
- Issue
- 9
- Pages
- 813-823
- ISSN
- 1091-4269 (ISSN-L)
- 1520-6394
- PMC
- PMC6736700
- OpenAlex
- W2786060782
- Cited by
- 306
- References
- 71
- Citations by year
- 2026: 47 · 2025: 49 · 2024: 83 · 2023: 49 · 2022: 36 · 2021: 25 · 2020: 11 · 2019: 5
- FWCI
- 13.73
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- Locations
- Depression and Anxiety (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- IUScholarWorks (Indiana University) (submittedVersion)
- PMC
- Funders
- National Heart, Lung, and Blood Institute (R01HL122245)
- Topics
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Health Sciences › Medicine › Psychiatry and Mental health › Mental Health Treatment and Access
- Keywords
- PHQ-9, measurement invariance, depression screening, factor structure, somatic depressive symptoms, sociodemographic disparities, sex differences, race and ethnicity, educational attainment, NHANES
- MeSH terms
- Patient Health Questionnaire, Depression, Depressive Disorder, Educational Status, Ethnicity, Factor Analysis, Statistical, Female, Humans, Male, Mass Screening, Middle Aged, Nutrition Surveys, Psychometrics, Racial Groups
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
