Establishing reliability and validity for mental health screening instruments in resource-constrained settings: Systematic review of the PHQ-9 and key recommendations
Open accessThe DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Abstract
Mental illness is one of the largest contributors to the global disease burden. The importance of valid and reliable mental health measures is crucial in order to accurately measure said burden, to quantify symptom improvement, and to ensure that symptoms are appropriately identified and quantified. This is of particular importance in low and middle-income countries (LMICs), where burden of mental illness is relatively high, and there is heterogeneity in linguistic, racial, and ethnic groups. Using the PHQ-9 as an illustrative example, this systematic review aims to provide an overview of existing work and highlight common validation and reporting practices. A systematic review of published literature validating the use of the PHQ-9 in LMICs as indexed in the PubMed and PsychInfo databases was conducted. The review included n = 49 articles (reduced from n = 2,349). This manuscript summarizes these results in terms of the frequency of reporting on important procedures and the types of reliability and validity measured. Then, building off of the existing literature, we provide key recommendations for measure validation in LMICs, which can be generalized for any type of measure used in a setting in which it was not initially developed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910
Interrater reliability: The kappa statistic
McHugh, M. L. (2012) · Biochemia Medica · doi:10.11613/bm.2012.031 · Cited by 19,877
Making sense of cronbach’s alpha.
Tavakol, M., & Dennick, R. (2011) · International Journal of Medical Education · doi:10.5116/ijme.4dfb.8dfd · Cited by 15,126
Whiteford, H. A., Degenhardt, L., Rehm, J., Baxter, A. J., Ferrari, A. J., Erskine, H. E., Charlson, F. J., Norman, R. E., Flaxman, A. D., Johns, N., et al. (2013) · The Lancet · doi:10.1016/s0140-6736(13)61611-6 · Cited by 6,006
The PHQ-9: A New Depression Diagnostic and Severity Measure
Kroenke, K., & Spitzer, R. L. (2002) · Psychiatric Annals · doi:10.3928/0048-5713-20020901-06 · Cited by 5,898 · cited 1×
“In the United States (HIC), the PHQ-9 has been reported to improve the recognition of depression and has shown excellent psychometric properties (Kroenke et al., 2001; Kroenke & Spitzer, 2002, Kroenke et al., 2010).”
BackgroundKroenke, K., Spitzer, R. L., Williams, J. B. W., & Löwe, B. (2010) · General Hospital Psychiatry · doi:10.1016/j.genhosppsych.2010.03.006 · Cited by 4,057
On the use, the misuse, and the very limited usefulness of Cronbach’s alpha
Sijtsma, K. (2009) · Psychometrika · doi:10.1007/s11336-008-9101-0 · Cited by 2,814
Levis, B., Benedetti, A., & Thombs, B. D. (2019) · BMJ · doi:10.1136/bmj.l1476 · Cited by 1,718
Hasin, D. S., Goodwin, R. D., Stinson, F. S., & Grant, B. F. (2005) · Archives of General Psychiatry · doi:10.1001/archpsyc.62.10.1097 · Cited by 1,645 · cited 1×
“Past studies have established differences in epidemiology of depressive disorders across countries, racial groups, and ethnic groups (Ayuso-Mateos et al., 2001; de Wit et al., 2008; Gonzalez et al., 2010; Hasin et al., 2005), and evidence of differing symptom presentations of depressive disorders in varied cultural co…”
BackgroundLöwe, B., Kroenke, K., Herzog, W., & Gräfe, K. (2004) · Journal of Affective Disorders · doi:10.1016/s0165-0327(03)00198-8 · Cited by 1,558 · cited 1×
“The most frequently used screener for depression globally is the Patient Health Questionnaire depression scale (the nine item version is often referred to as the PHQ-9; Lowe et al., 2004a; Lowe et al., 2004b), which has been translated into over 70 different languages and dialects (Pfizer, 2013).”
MethodSize and burden of mental disorders in Europe—a critical review and appraisal of 27 studies
Wittchen, H.-U., & Jacobi, F. (2005) · European Neuropsychopharmacology · doi:10.1016/j.euroneuro.2005.04.012 · Cited by 1,503
Gilbody, S., Richards, D., Brealey, S., & Hewitt, C. (2007) · Journal of General Internal Medicine · doi:10.1007/s11606-007-0333-y · Cited by 1,290 · cited 1×
“Professional application of the PHQ-9 ranges widely from community mental health settings to primary care and personnel sites (Gilbody et al., 2007; Wennerstrom et al., 2011).”
BackgroundLOWE, B. (2004) · Journal of Affective Disorders · doi:10.1016/s0165-0327(02)00237-9 · Cited by 1,060
Storch, E. A., Roberti, J. W., & Roth, D. A. (2004) · Depression and Anxiety · doi:10.1002/da.20002 · Cited by 694
Depressive disorders in Europe: prevalence figures from the ODIN study
Ayuso-Mateos, J. L., Vázquez-Barquero, J. L., Dowrick, C., Lehtinen, V., Dalgard, O. S., Casey, P., Wilkinson, C., Lasa, L., Page, H., Dunn, G., et al. (2001) · British Journal of Psychiatry · doi:10.1192/bjp.179.4.308 · Cited by 564 · cited 1×
“Past studies have established differences in epidemiology of depressive disorders across countries, racial groups, and ethnic groups (Ayuso-Mateos et al., 2001; de Wit et al., 2008; Gonzalez et al., 2010; Hasin et al., 2005), and evidence of differing symptom presentations of depressive disorders in varied cultural co…”
BackgroundThe Link between Thyroid Function and Depression
Hage, M. P., & Azar, S. T. (2012) · Journal of Thyroid Research · doi:10.1155/2012/590648 · Cited by 549
Adewuya, A. O., Ola, B. A., & Afolabi, O. O. (2006) · Journal of Affective Disorders · doi:10.1016/j.jad.2006.05.021 · Cited by 474
Duff, K. (2012) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acr120 · Cited by 452
Ali, G.-C., Ryan, G., & De Silva, M. J. (2016) · PLOS ONE · doi:10.1371/journal.pone.0156939 · Cited by 358
Liu, S.-I., Yeh, Z.-T., Huang, H.-C., Sun, F.-J., Tjung, J.-J., Hwang, L.-C., Shih, Y.-H., & Yeh, A. W.-C. (2011) · Comprehensive Psychiatry · doi:10.1016/j.comppsych.2010.04.013 · Cited by 192
Becker, S., Al Zaid, K., & Al Faris, E. (2002) · The International Journal of Psychiatry in Medicine · doi:10.2190/xtdd-8l18-p9e0-jyrv · Cited by 190 · cited 3×
“A uthor M anuscript A uthor M anuscript A uthor M anuscript A uthor M anuscript widely embraced for cross-cultural use and translated into many languages including Arabic (Becker et al., 2002), Korean (Donnelly, 2007), Malay (Azah et al., 2005), Turkish (Corapcioglu & Ozer, 2004), Spanish (Huang et al., 2006), and man…”
BackgroundThe complex relationship between depression and dementia
Varghese, M., & Muliyala, K. (2010) · Annals of Indian Academy of Neurology · doi:10.4103/0972-2327.74248 · Cited by 119
(2004) · World Pumps · doi:10.1016/s0262-1762(04)00171-3 · Cited by 47
Gonzalez, J., & Trickett, E. J. (2014) · American Journal of Community Psychology · doi:10.1007/s10464-014-9655-1 · Cited by 47
Common mental disorders and primary health care: current practice in low-income countries
Abas, M., Baingana, F., Broadhead, J., Iacoponi, E., & Vanderpyl, J. (2003) · Harvard Review of Psychiatry · doi:10.1080/10673220303954 · Cited by 36
Cited by
Fu, Z., Burger, H., Arjadi, R., Nauta, M. H., & Bockting, C. L. H. (2021) · Clinical Psychology in Europe · doi:10.32872/cpe.5467 · Cited by 16
“The PHQ-9 has acceptable validity and reliability (Carroll et al., 2020), and the Cronbach’s alphas in the current study ranged from .”
MethodBah, A. J., Wurie, H. R., Samai, M., Horn, R., & Ager, A. (2025) · Journal of Affective Disorders Reports · doi:10.1016/j.jadr.2024.100852 · Cited by 7
Laurenzi, C. A., Menyatsoe, M., Shuaib, M., Edwards, A., Gemmell, K., Geffen, H., Tomlinson, M., Müller, A., Manda, S., & Mabaso, K. M. (2024) · Poverty & Public Policy · doi:10.1002/pop4.408 · Cited by 2
Metadata
- Authors
- Carroll, H. A., Hook, K., Perez, O. F. R., Denckla, C., Vince, C. C., Ghebrehiwet, S., Ando, K., Touma, M., Borba, C. P. C., Fricchione, G. L., & Henderson, D. C.
- Venue
- Psychiatry Research
- Publisher
- Elsevier BV
- Published
- 2020-09
- Volume
- 291
- Pages
- 113236
- ISSN
- 0165-1781 (ISSN-L)
- OpenAlex
- W3036325417
- Cited by
- 131
- References
- 25
- FWCI
- 5.11
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.