CoChat
Sign Up
Cited by 28Medical Coding and Health InformationMental Health Treatment and AccessAnxiety, Depression, Psychometrics, Treatment, Cognitive Processes

The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.

Validation of case definitions of depression derived from administrative data against the CIDI-SF as reference standard: results from the PROspective Québec (PROQ) study

Open access
Pena-Gralle, Ana Paula Bruno, Talbot, Denis, Trudel, Xavier, Aubé, Karine, Lesage, Alain, Lauzier, Sophie, Milot, Alain, Brisson, Chantal·2021·BMC Psychiatry
DOI 10.1186/s12888-021-03501-xPMC PMC8496029
Open this paper in CoChat — free accountOpen PDF

Abstract

Background: Administrative data have several advantages over questionnaire and interview data to identify cases of depression: they are usually inexpensive, available for a long period of time and are less subject to recall bias and differential classification errors. However, the validity of administrative data in the correct identification of depression has not yet been studied in general populations. The present study aimed to 1) evaluate the sensitivity and specificity of administrative cases of depression using the validated Composite International Diagnostic Interview – Short Form (CIDI-SF) as reference standard and 2) compare the known-groups validity between administrative and CIDI-SF cases of depression. Methods: The 5487 participants seen at the last wave (2015–2018) of the PROQ cohort had CIDI-SF questionnaire data linked to hospitalization and medical reimbursement data provided by the provincial universal healthcare provider and coded using the International Classification of Disease. We analyzed the sensitivity and specificity of several case definitions of depression from this administrative data. Their association with known predictors of depression was estimated using robust Poisson regression models. Results: Administrative cases of depression showed high specificity (≥ 96%), low sensitivity (19–32%), and rather low agreement (Cohen’s kappa of 0.21–0.25) compared with the CIDI-SF. These results were consistent over strata of sex, age and education level and with varying case definitions. In known-groups analysis, the administrative cases of depression were comparable to that of CIDI-SF cases (RR for sex: 1.80 vs 2.03 respectively, age: 1.53 vs 1.40, education: 1.52 vs 1.28, psychological distress: 2.21 vs 2.65). Conclusion: The results obtained in this large sample of a general population suggest that the dimensions of depression captured by administrative data and by the CIDI-SF are partially distinct. However, their known-groups validity in relation to risk factors for depression was similar to that of CIDI-SF cases. We suggest that neither of these data sources is superior to the other in the context of large epidemiological studies aiming to identify and quantify risk factors for depression.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12888-021-03501-x

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Depression was identified using ICD codes with positive predictive values of 92.0% for ICD-9 and 91.1% for ICD-10.

    “Depression was identified using International Classification of Disease (ICD) codes (ICD-9: 296.20-296.25, 296.30-296.35, 300.4, 311; positive predictive value [PPV] = 92.0%; ICD-10: F32.0-32.9, F33.0-33.3, F33.8, F33.9, F34.1 & F41.2; PPV = 91.1%) [18].”

    From Association between first-line antidepressant use and risk of dementia in older adults: a retrospective cohort study (2023) · cites this paper as [18]

    The Methods (Administrative cases of depression; page unavailable) use ICD-9 and ICD-10 codes to identify cases, but list different code ranges from those in the citing sentence. The Results (Table 2; page unavailable) report PPVs of 32.8% and 25.6% for the two administrative case definitions, not 92.0% for ICD-9 or 91.1% for ICD-10. (LLM, conf=0.90)

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • Dementia outcome was identified using ICD codes with positive predictive values ranging from 73.2% to 93.6%.

    “The PPV values of using these ICD codes to identify dementia ranged from 73.2 to 93.6% [18].”

    From Association between first-line antidepressant use and risk of dementia in older adults: a retrospective cohort study (2023) · cites this paper as [18]

    The Results (Table 2; page unavailable) report PPVs of 25.6%–32.8% for administrative definitions of depression, not dementia. A search for dementia and the claimed 73.2%–93.6% range found no passage evaluating ICD codes for dementia, so our copy cannot verify the cited claim. (LLM, conf=0.77)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 38

  1. WHO. Depression and other common mental disorders. Geneva: WHO; 2017.
  2. Kruijshaar ME, Barendregt J, Vos T, de Graaf R, Spijker J, Andrews G. Lifetime prevalence estimates of major depression: an indirect estimation method and a quantification of recall bias. Eur J Epidemiol. 2005;20(1):103–11. · DOI
  3. Kessing LV. Epidemiology of subtypes of depression. Acta Psychiatr Scand. 2007;115:85–9. · DOI
  4. GBD. Global, regional, and national incidence, prevalence, and years lived with disability for 354 diseases and injuries for 195 countries and territories, 1990–2017: a systematic analysis for the Global Burden of Disease Study 2017. Lancet (London, England). 2018;392(10159):1789–858. · DOI · Paper page
  5. Roy T, Lloyd CE. Epidemiology of depression and diabetes: a systematic review. J Affect Disord. 2012;142(Suppl):S8–21. · DOI · Paper page
  6. Green RC, Cupples LA, Kurz A, Auerbach S, Go R, Sadovnick D, et al. Depression as a risk factor for Alzheimer disease: the MIRAGE study. Arch Neurol. 2003;60(5):753–9. · DOI · Paper page
  7. Marrie RA, Walker JR, Graff LA, Lix LM, Bolton JM, Nugent Z, et al. Performance of administrative case definitions for depression and anxiety in inflammatory bowel disease. J Psychosom Res. 2016;89:107–13. · DOI
  8. Jette N, Quan H, Hemmelgarn B, Drosler S, Maass C, Moskal L, et al. The development, evolution, and modifications of ICD-10: challenges to the international comparability of morbidity data. Med Care. 2010;48(12):1105–10. · DOI
  9. Craven MA, Bland R. Depression in primary care: current and future challenges. Can J Psychiatry. 2013;58(8):442–8. · DOI
  10. Smolders M, Laurant M, Verhaak P, Prins M, van Marwijk H, Penninx B, et al. Adherence to evidence-based guidelines for depression and anxiety disorders is associated with recording of the diagnosis. Gen Hosp Psychiatry. 2009;31(5):460–9. · DOI
  11. Joling KJ, van Marwijk HW, Piek E, van der Horst HE, Penninx BW, Verhaak P, et al. Do GPs' medical records demonstrate a good recognition of depression? A new perspective on case extraction. J Affect Disord. 2011;133(3):522–7. · DOI
  12. John A, McGregor J, Fone D, Dunstan F, Cornish R, Lyons RA, et al. BMC Medical Informatics and Decision Making. 2016;16:35. · DOI
  13. Solberg LI, Engebretson KI, Sperl-Hillen JM, Hroscikoski MC, O'Connor PJ. Are claims data accurate enough to identify patients for performance measures or quality improvement? The case of diabetes, heart disease, and depression. Am J Med Qual. 2006;21(4):238–45. · DOI
  14. Doktorchik C, Patten S, Eastwood C, Peng MK, Chen GM, Beck CA, et al. Validation of a case definition for depression in administrative data against primary chart data as a reference standard. BMC Psychiatry. 2019;19:9. · DOI · Paper page
  15. Fiest KM, Jette N, Quan HD, St Germaine-Smith C, Metcalfe A, Patten SB, et al. Systematic review and assessment of validated case definitions for depression in administrative data. BMC Psychiatry. 2014;14:289. · DOI
  16. Rawson NS, Malcolm E, D'Arcy C. Reliability of the recording of schizophrenia and depressive disorder in the Saskatchewan health care datafiles. Soc Psychiatry Psychiatr Epidemiol. 1997;32(4):191–9. · DOI
  17. Trinh NH, Youn SJ, Sousa J, Regan S, Bedoya CA, Chang TE, et al. Using electronic medical records to determine the diagnosis of clinical depression. Int J Med Inform. 2011;80(7):533–40. · DOI
  18. Townsend L, Walkup JT, Crystal S, Olfson M. A systematic review of validated methods for identifying depression using administrative data. Pharmacoepidemiol Drug Saf. 2012;21(Suppl 1):163–73. · DOI
  19. Howren A, Aviña-Zubieta JA, Puyat JH, Esdaile JM, Da Costa D, De Vera MA. Defining depression and anxiety in individuals with rheumatic diseases using administrative health databases: a systematic review. Arthritis Care Res (Hoboken). 2020;72(2):243–55. · DOI
  20. Katon WJ, Simon G, Russo J, Von Korff M, Lin EH, Ludman E, et al. Quality of depression care in a population-based sample of patients with diabetes and major depression. Med Care. 2004;42(12):1222–9. · DOI
  21. Trudel X, Gilbert-Ouimet M, Milot A, Duchaine CS, Vezina M, Laurin D, et al. Cohort profile: the PROspective Quebec (PROQ) study on work and health. Int J Epidemiol. 2018;47(3):693-i. · DOI
  22. Muntaner C, Eaton WW, Diala C, Kessler RC, Sorlie PD. Social class, assets, organizational control and the prevalence of common groups of psychiatric disorders. Soc Sci Med. 1998;47(12):2043–53. · DOI
  23. Gravel R, Beland Y. The Canadian community health survey: mental health and well-being. Can J Psychiatry. 2005;50(10):573–9. · DOI
  24. Patten SB. Performance of the composite international diagnostic interview short form for major depression in community and clinical samples. Chronic Dis Can. 1997;18(3):109–12.
  25. Kessler RC, Berglund P, Demler O, Jin R, Koretz D, Merikangas KR, et al. The epidemiology of major depressive disorder: results from the National Comorbidity Survey Replication (NCS-R). Jama. 2003;289(23):3095–105. · DOI · Paper page
  26. Plante C, Goudreau S, Jacques L, Tessier F. Agreement between survey data and Régie de l'assurance maladie du Québec (RAMQ) data with respect to the diagnosis of asthma and medical services use for asthma in children. Chronic Dis Inj Can. 2014;34(4):256–62. · DOI
  27. Davidson M. Known-groups validity. In: Michalos AC, editor. Encyclopedia of quality of life and well-being research. Dordrecht: Springer Netherlands; 2014. p. 3481–2. · DOI
  28. McCoach DB, Gable RK, Madura JP. Evidence Based on Relations to Other Variables: Bolstering the Empirical Validity Arguments for Constructs. In: Instrument Development in the Affective Domain: School and Corporate Applications. New York, NY: Springer New York; 2013. p. 209–48. · DOI
  29. Ilfeld F. Further validation of a psychiatric symptom index in a normal population. Psychol Rep. 1976;39:12015–1228.
  30. Perreault C. Les Mesures de Santé Mentale. Possibilités et Limites de la Mesure Utilisée (mental health measurements: strengths and limitation of the instrument used in Quebec). Québec, QC, Canada: Gouvernement du Québec; 1987.
  31. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74. · DOI · Paper page
  32. Schomaker M, Heumann C. Bootstrap inference when using multiple imputation. Stat Med. 2018;37(14):2252–66. · DOI
  33. Chen W, Qian L, Shi J, Franklin M. Comparing performance between log-binomial and robust Poisson regression models for estimating risk ratios under model misspecification. BMC Med Res Methodol. 2018;18(1):63. · DOI
  34. Lasalvia A, Zoppei S, Van Bortel T, Bonetto C, Cristofalo D, Wahlbeck K, et al. Global pattern of experienced and anticipated discrimination reported by people with major depressive disorder: a cross-sectional survey. Lancet (London, England). 2013;381(9860):55–62. · DOI · Paper page
  35. Lesage AD, Fournier L, Cyr M, Toupin J, Fabian J, Gaudette G, et al. The reliability of the community version of the MRC needs for care assessment. Psychol Med. 1996;26(2):237–43. · DOI
  36. Patten SB, Wang JL, Beck CA, Maxwell CJ. Measurement issues related to the evaluation and monitoring of major depression prevalence in Canada. Chronic Dis Can. 2005;26(4):100–6.
  37. Mitchell AJ, Vaze A, Rao S. Clinical diagnosis of depression in primary care: a meta-analysis. Lancet (London, England). 2009;374(9690):609–19. · DOI · Paper page
  38. Cepoiu M, McCusker J, Cole MG, Sewitch M, Belzile E, Ciampi A. Recognition of depression by non-psychiatric physicians--a systematic literature review and meta-analysis. J Gen Intern Med. 2008;23(1):25–36. · DOI

Cited by

Sorted by citations

Metadata

Authors
Pena-Gralle, A. P. B., Talbot, D., Trudel, X., Aubé, K., Lesage, A., Lauzier, S., Milot, A., & Brisson, C.
Affiliations
  • Université Laval (CA)
  • Université de Montréal (CA)
Corresponding author
Pena-Gralle, Ana Paula Bruno
ORCID
Venue
BMC Psychiatry
Publisher
Springer Science and Business Media LLC
Published
2021-10-07
Volume
21
Issue
1
Pages
491-491
ISSN
1471-244X (ISSN-L)
DOI
10.1186/s12888-021-03501-x
PMC
PMC8496029
OpenAlex
W3203836101
Cited by
28
References
3
Citations by year
2026: 5 · 2025: 6 · 2024: 4 · 2023: 13
FWCI
2.49
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Psychiatry (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, other-oa)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Canadian Institutes of Health Research (MOP-113542)
Topics
  • Health Sciences › Health Professions › Health Information Management › Medical Coding and Health Information
  • Health Sciences › Medicine › Psychiatry and Mental health › Mental Health Treatment and Access
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
Keywords
depression, administrative data, case definition, diagnostic validity, sensitivity and specificity, known-groups validity, hospitalization data, ICD coding, risk factors for depression, psychological distress
MeSH terms
Depression, Humans, Infant, Interview, Psychological, Prospective Studies, Quebec, Reference Standards

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review