The DOI matches its Crossref record; 88 of the 88 references it cites were found in a registry or an index.
Meta‐analyses of agreement between diagnoses made from clinical evaluations and standardized diagnostic interviews
Open accessAbstract
Standardized diagnostic interviews (SDIs) have become de facto gold standards for clinical research. However, because clinical practitioners seldom use SDIs, it is essential to determine how well SDIs agree with clinical diagnoses. In meta‐analyses of 38 articles published from 1995 to 2006 (N = 15,967 probands), mean kappas (z‐transformed) between diagnoses from clinical evaluations versus SDIs were 0.27 for a broad category of all disorders, 0.29 for externalizing disorders, and 0.28 for internalizing disorders. Kappas for specific disorders ranged from 0.19 for generalized anxiety disorder to 0.86 for anorexia nervosa (median = 0.48). For diagnostic clusters (e.g. psychotic disorders), kappas ranged from 0.14 for affective disorders (including bipolar) to 0.70 for eating disorders (median = 0.43). Kappas were significantly higher for outpatients than inpatients and for children than adults. However, these effects were not significant in meta‐regressions. Conclusions: Diagnostic agreement between SDIs and clinical evaluations varied widely by disorder and was low to moderate for most disorders. Thus, findings from SDIs may not fully apply to diagnoses based on clinical evaluations of the sort used in the published studies. Rather than implying that SDIs or clinical evaluations are inferior, characteristics of both may limit agreement and generalizability from SDI findings to clinical practice. Copyright © 2009 John Wiley & Sons, Ltd.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398
Diagnostic and Statistical Manual of Mental Disorders
American Psychiatric Association (2022) · doi:10.1176/appi.books.9780890425787 · Cited by 23,476
(1998) · The Journal of Clinical Psychiatry · Cited by 19,139
Depression and quality of life in older persons: A review
Sivertsen, H., Bjørkløf, G. H., Engedal, K., Selbæk, G., & Helvik, A.-S. (2015) · Dementia and Geriatric Cognitive Disorders · doi:10.1159/000437299 · Cited by 14,152
Diagnostic and Statistical Manual of Mental Disorders
&NA; (1988) · Alzheimer Disease & Associated Disorders · doi:10.1097/00002093-198802020-00022 · Cited by 12,526
(1998) · PubMed · Cited by 10,216
KAUFMAN, J., BIRMAHER, B., BRENT, D., RAO, U., FLYNN, C., MORECI, P., WILLIAMSON, D., & RYAN, N. (1997) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-199707000-00021 · Cited by 10,148
Statistical Methods for Rates and Proportions.
Everitt, B. S., & Fleiss, J. L. (1981) · Biometrics · doi:10.2307/2530193 · Cited by 8,990
A Diagnostic Interview: The Schedule for Affective Disorders and Schizophrenia
Endicott, J. (1978) · Archives of General Psychiatry · doi:10.1001/archpsyc.1978.01770310043002 · Cited by 5,684
Robins, L. N. (1981) · Archives of General Psychiatry · doi:10.1001/archpsyc.1981.01780290015001 · Cited by 5,219
Meta-Analytic Procedures for Social Research
Rosenthal, R. (1991) · doi:10.4135/9781412984997 · Cited by 4,635
The Structured Clinical Interview for DSM-III-R (SCID): I: History, Rationale, and Description
Spitzer, R. L. (1992) · Archives of General Psychiatry · doi:10.1001/archpsyc.1992.01820080032005 · Cited by 4,336
SHAFFER, D., FISHER, P., LUCAS, C. P., DULCAN, M. K., & SCHWAB-STONE, M. E. (2000) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-200001000-00014 · Cited by 3,505
CONSORT statement: extension to cluster randomised trials
Campbell, M. K., Elbourne, D. R., & Altman, D. G. (2004) · BMJ · doi:10.1136/bmj.328.7441.702 · Cited by 2,557
Statistical Methods for Meta-Analysis
(1985) · Elsevier eBooks · doi:10.1016/c2009-0-03396-0 · Cited by 2,521
Wittchen, H.-U. (1994) · Journal of Psychiatric Research · doi:10.1016/0022-3956(94)90036-1 · Cited by 2,348
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings.
Raudenbush, S. W., Hunter, J. E., & Schmidt, F. L. (1991) · Journal of the American Statistical Association · doi:10.2307/2289738 · Cited by 2,327
Goodman, R., Ford, T., Richards, H., Gatward, R., & Meltzer, H. (2000) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.2000.tb02345.x · Cited by 1,755
The Structured Clinical Interview for DSM-III-R (SCID): II. Multisite Test-Retest Reliability
Williams, J. B. W. (1992) · Archives of General Psychiatry · doi:10.1001/archpsyc.1992.01820080038006 · Cited by 1,618
Best Estimate of Lifetime Psychiatric Diagnosis: A Methodological Study
Leckman, J. F. (1982) · Archives of General Psychiatry · doi:10.1001/archpsyc.1982.04290080001001 · Cited by 1,425
LOWE, B. (2004) · Journal of Affective Disorders · doi:10.1016/s0165-0327(02)00237-9 · Cited by 1,060
The Child and Adolescent Psychiatric Assessment (CAPA)
Angold, A., Prendergast, M., Cox, A., Harrington, R., Simonoff, E., & Rutter, M. (1995) · Psychological Medicine · doi:10.1017/s003329170003498x · Cited by 968
Diagnostic Interview for Children and Adolescents (DICA)
REICH, W. (2000) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-200001000-00017 · Cited by 823
Field, A. P. (2001) · Psychological Methods · doi:10.1037/1082-989x.6.2.161 · Cited by 725
Psychiatric diagnosis: Are clinicians still necessary?
Spitzer, R. L. (1983) · Comprehensive Psychiatry · doi:10.1016/0010-440x(83)90032-9 · Cited by 687
Statistical Methods for Research Workers
F., W. T., & Fisher, R. A. (1971) · Biometrics · doi:10.2307/2528855 · Cited by 657
Helzer, J. E. (1985) · Archives of General Psychiatry · doi:10.1001/archpsyc.1985.01790300019003 · Cited by 617
Krueger, R. F., Markon, K. E., Patrick, C. J., & Iacono, W. G. (2005) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.114.4.537 · Cited by 566
Anthony, J. C. (1985) · Archives of General Psychiatry · doi:10.1001/archpsyc.1985.01790300029004 · Cited by 558
Regier, D. A., Kaelber, C. T., Rae, D. S., Farmer, M. E., Knauper, B., Kessler, R. C., & Norquist, G. S. (1998) · Archives of General Psychiatry · doi:10.1001/archpsyc.55.2.109 · Cited by 549
WELNER, Z., REICH, W., HERJANIC, B., JUNG, K. G., & AMADO, H. (1987) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-198709000-00007 · Cited by 513
Reliability and validity of Japanese version of the Mini‐International Neuropsychiatric Interview
OTSUBO, T., TANAKA, K., KODA, R., SHINODA, J., SANO, N., TANAKA, S., AOYAMA, H., MIMURA, M., & KAMIJIMA, K. (2005) · Psychiatry and Clinical Neurosciences · doi:10.1111/j.1440-1819.2005.01408.x · Cited by 507
Assessment of Adult Psychopathology: Meta-Analyses and Implications of Cross-Informant Correlations.
Achenbach, T. M., Krukowski, R. A., Dumenci, L., & Ivanova, M. Y. (2005) · Psychological Bulletin · doi:10.1037/0033-2909.131.3.361 · Cited by 406
Self-citationMethods to Improve Diagnostic Accuracy in a Community Mental Health Setting
Basco, M. R., Bostic, J. Q., Davies, D., Rush, A. J., Witte, B., Hendrickse, W., & Barnett, V. (2000) · American Journal of Psychiatry · doi:10.1176/appi.ajp.157.10.1599 · Cited by 390
Kim, Y. S., Cheon, K. A., Kim, B. N., Chang, S. A., Yoo, H. J., Kim, J. W., Cho, S. C., Seo, D. H., Bae, M. O., So, Y. K., et al. (2004) · Yonsei Medical Journal · doi:10.3349/ymj.2004.45.1.81 · Cited by 354
Diagnosis of Nonpsychotic Patients in Community Clinics
Shear, M. K., Greeno, C., Kang, J., Ludewig, D., Frank, E., Swartz, H. A., & Hanekamp, M. (2000) · American Journal of Psychiatry · doi:10.1176/appi.ajp.157.4.581 · Cited by 316
Occurrence, recognition, and outcome of psychological disorders in primary care
(1996) · American Journal of Psychiatry · doi:10.1176/ajp.153.5.636 · Cited by 245
A Computer-Administered Telephone Interview to Identify Mental Disorders
Kobak, K. A. (1997) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1997.03550110043034 · Cited by 238
Mullick, M. S. I., & Goodman, R. (2005) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-005-0939-5 · Cited by 214
BRUGHA, T. S., BEBBINGTON, P. E., & JENKINS, R. (1999) · Psychological Medicine · doi:10.1017/s0033291799008880 · Cited by 191
Ghanizadeh, A., Mohammadi, M. R., & Yazdanshenas, A. (2006) · BMC Psychiatry · doi:10.1186/1471-244x-6-10 · Cited by 191
Jensen, A. L., & Weisz, J. R. (2002) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.70.1.158 · Cited by 189
BRUGHA, T. S., JENKINS, R., TAUB, N., MELTZER, H., & BEBBINGTON, P. E. (2001) · Psychological Medicine · doi:10.1017/s0033291701004184 · Cited by 171
Alyahri, A., & Goodman, R. (2006) · PubMed · Cited by 145
Kadri, N., Agoub, M., Gnaoui, S. E., Alami, K. M., Hergueta, T., & Moussaoui, D. (2005) · European Psychiatry · doi:10.1016/j.eurpsy.2004.11.007 · Cited by 142
A Comparison of the Structured Clinical Interview for DSM-III-R and Clinical Diagnoses
Steiner, J. L., Tebes, J. K., Sledge, W. H., & Walker, M. L. (1995) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-199506000-00003 · Cited by 138
Klinkman, M. S., Coyne, J. C., Gallo, S., & Schwenk, T. L. (1998) · Archives of Family Medicine · doi:10.1001/archfami.7.5.451 · Cited by 129
Jordanova, V., WICKRAMESINGHE, C., Gerada, C., & Prince, M. J. (2004) · Psychological Medicine · doi:10.1017/s0033291703001727 · Cited by 129
Ezpeleta, L., de la Osa, N., Domenech, J. M., Navarro, J. B., Losilla, J.-M., & Júdez, J. (1997) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.1997.tb01528.x · Cited by 127
Psychometric properties of the K-SADS-PL in an Israeli adolescent clinical population.
Shanee, N., Apter, A., & Weizman, A. (1997) · PubMed · Cited by 120
Child behavior analysis and therapy
Yuli, W. (1976) · Behaviour Research and Therapy · doi:10.1016/0005-7967(76)90034-6 · Cited by 114
Comparison of DISC and K-SADS-P Interviews of an Epidemiological Sample of Children
Cohen, P. R., O’Connor, P. A., Lewis, S. A., Vélez, C. N., & MALACHOWSKI, B. (1987) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-198709000-00009 · Cited by 108
The reliability of psychiatric and psychological diagnosis
Matarazzo, J. D. (1983) · Clinical Psychology Review · doi:10.1016/0272-7358(83)90008-9 · Cited by 98
Strakowski, S. M., Hawkins, J. M., Keck, P. E., McElroy, S. L., West, S. A., Bourne, M. L., Sax, K. W., & Tugrul, K. C. (1997) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v58n1010a · Cited by 96
Comparing DISC-IV and Clinician Diagnoses Among Youths Receiving Public Mental Health Services
Lewczyk, C. M., Garland, A. F., Hurlburt, M. S., Gearity, J., & Hough, R. L. (2003) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-200303000-00016 · Cited by 94
Beal, D., Corey, D. M., & Dunlap, W. (2002) · Journal of Applied Psychology · doi:10.1037/0021-9010.87.3.583 · Cited by 93
Prevalence and 12-month outcome of threshold and subthreshold mental disorders in primary care
Pini, S., Perkonnig, A., Tansella, M., & Wïttchen, H. (1999) · Journal of Affective Disorders · doi:10.1016/s0165-0327(99)00141-x · Cited by 90
McClellan, J., & WERRY, J. S. (2000) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-200001000-00013 · Cited by 88
Research on the Clinical Interview
Young, J. G., O'BRIEN, J. D., Gutterman, E. M., & Cohen, P. R. (1987) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-198709000-00002 · Cited by 79
Diagnosing mental disorders in the community. A difference that matters?
Wïttchen, H., Üstün, T. B., & Kessler, R. C. (1999) · Psychological Medicine · doi:10.1017/s0033291799008831 · Cited by 77
Depression in Primary Care—More like Asthma Than Appendicitis: The Michigan Depression Project
Klinkman, M. S., Schwenk, T. L., & Coyne, J. C. (1997) · The Canadian Journal of Psychiatry · doi:10.1177/070674379704200909 · Cited by 76
Structured Diagnostic Interviews for Children and Adolescents: Current Status and Future Directions
Gutterman, E. M., O'BRIEN, J. D., & Young, J. G. (1987) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-198709000-00003 · Cited by 75
Diagnosis of depression by primary care physicians versus a structured diagnostic interview
Tiemens, B. G., VonKorff, M., & Lin, E. H. B. (1999) · General Hospital Psychiatry · doi:10.1016/s0163-8343(98)00077-2 · Cited by 69
Recognition of psychiatric disorders in musculoskeletal and cardiovascular rehabilitation patients
Härter, M., Woll, S., Reuter, K., Wunsch, A., & Bengel, J. (2004) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2003.08.106 · Cited by 66
Komiti, A., Jackson, H. J., Judd, F. K., Cockram, A. M., Kyrios, M., Yeatman, R., Murray, G., Hordern, C., Wainwright, K., Allen, N. B., et al. (2001) · Australian & New Zealand Journal of Psychiatry · doi:10.1046/j.1440-1614.2001.00868.x · Cited by 61
Detecting Symptoms of Alcohol Abuse in Primary Care Settings
McQuade, W., Levy, S., Yanek, L. R., Davis, S. W., & Liepman, M. R. (2000) · Archives of Family Medicine · doi:10.1001/archfami.9.9.814 · Cited by 59
Comparison of DISC with Clinicians' DSM-III Diagnoses in Psychiatric Inpatients
Weinstein, S. R., Stone, K., Noam, G. G., GRIMES, K., & Schwab‐Stone, M. (1989) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/00004583-198901000-00010 · Cited by 58
Dimensional Approaches in Diagnostic Classification: Refining the Research Agenda for DSM-V
Harkavy‐Friedman, J. (2009) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2008.08060939 · Cited by 56
Identification of depression by medical and surgical general hospital physicians
Balestrieri, M., Bisoffi, G., Tansella, M., et al. (2002) · General Hospital Psychiatry · doi:10.1016/s0163-8343(01)00176-1 · Cited by 49
van Weel-Baumgarten, E. M., van den Bosch, W. J., van den Hoogen, H. J., & Zitman, F. G. (2000) · Europe PMC (PubMed Central) · Cited by 47
Tenney, N. H., Schotte, C., Denys, D. A. J. P., van Megen, H. J. G. M., & Westenberg, H. G. M. (2003) · Journal of Personality Disorders · doi:10.1521/pedi.17.6.550.25352 · Cited by 47
Computerised diagnosis in acute psychiatry: Validity of CIDI-Auto against routine clinical diagnosis
Rosenman, S. J., Korten, A. E., & Levings, C. T. (1997) · Journal of Psychiatric Research · doi:10.1016/s0022-3956(97)00032-0 · Cited by 43
Jewell, J. D., Handwerk, M. L., Almquist, J., & Lucas, C. P. (2004) · Journal of Clinical Child & Adolescent Psychology · doi:10.1207/s15374424jccp3303_11 · Cited by 43
A Comparison of Clinical and Structured Interview Diagnoses in a Homeless Mental Health Clinic
North, C. S., Pollio, D. E., Thompson, S. J., Ricci, D. A., Smith, E. M., & Spitznagel, E. L. (1997) · Community Mental Health Journal · doi:10.1023/a:1025052720325 · Cited by 41
Kranzler, H. R., Kadden, R. M., Babor, T. F., & Rounsaville, B. J. (1994) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-199405000-00005 · Cited by 40
Lörch, B., Szegedi, A., Kohnen, R., & Benkert, O. (2000) · Journal of Psychiatric Research · doi:10.1016/s0022-3956(00)00005-4 · Cited by 39
Does the Composite International Diagnostic Interview underdiagnose the eating disorders?
Thornton, C. E., Russell, J. D., & Hudson, J. L. (1998) · International Journal of Eating Disorders · doi:10.1002/(sici)1098-108x(199804)23:3<341::aid-eat11>3.0.co;2-0 · Cited by 35
Detection and Outcomes of Substance Use Disorders in Adolescents Seeking Mental Health Treatment
Kramer, T. L., Robbins, J. M., Phillips, S. D., Miller, T. L., & Burns, B. J. (2003) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/01.chi.0000084833.67701.44 · Cited by 35
The Role of Symptoms in the Recognition of Mental Health Disorders in Primary Care
Füredi, J., Rózsa, S., Zámbori, J., & Szádóczky, E. (2003) · Psychosomatics · doi:10.1176/appi.psy.44.5.402 · Cited by 35
Moya, T., Fleitlich‐Bilyk, B., Goodman, R., Nogueira, F. C., Focchi, P. S., Nicoletti, M., Pinzon, V., Córdas, T. A., & Neto, F. L. (2005) · Brazilian Journal of Psychiatry · doi:10.1590/s1516-44462005000100008 · Cited by 34
Characteristics, recognition and treatment of dysthymics in primary care
Lecrubier, Y., & Weiller, E. (1998) · European Psychiatry · doi:10.1016/s0924-9338(98)80004-7 · Cited by 23
Rosenman, S. J., Levings, C. T., & Korten, A. E. (1997) · Psychiatric Services · doi:10.1176/ps.48.6.815 · Cited by 22
Anxiety and mood disorders in primary care practice
Szádóczky, E., Rózsa, S., Zámbori, J., & Füredi, J. (2004) · International Journal of Psychiatry in Clinical Practice · doi:10.1080/13651500310004966 · Cited by 21
van Marwijk, H., de Bock, G. H., Hermans, J., Mulder, J. D., & Springer, M. P. (1996) · Scandinavian Journal of Primary Health Care · doi:10.3109/02813439609024168 · Cited by 20
Clinical Diagnosis and SCID-II Assessment of DSM-III-R Personality Disorders
Fridell, M., & Hesse, M. (2006) · European Journal of Psychological Assessment · doi:10.1027/1015-5759.22.2.104 · Cited by 19
Modelling covariate effects in observer agreement studies: The case of nominal scale agreement
Guggenmoss‐Holzmann, I. (1995) · Statistics in Medicine · doi:10.1002/sim.4780142012 · Cited by 19
Cognitive therapy and serious mental illness. An interacting cognitive subsystems approach
Clarke, I. (1999) · Clinical Psychology & Psychotherapy · doi:10.1002/(sici)1099-0879(199911)6:5<384::aid-cpp218>3.0.co;2-8 · Cited by 15
Cited by
Achenbach, T. M., Becker, A., Döpfner, M., Heiervang, E., Roessner, V., Steinhausen, H., & Rothenberger, A. (2008) · Journal of Child Psychology and Psychiatry · doi:10.1111/j.1469-7610.2007.01867.x · Cited by 679
Self-citationMargraf, J., Cwik, J. C., Pflug, V., & Schneider, S. (2017) · Zeitschrift für Klinische Psychologie und Psychotherapie · doi:10.1026/1616-3443/a000430 · Cited by 293
Jensen-Doss, A., & Hawley, K. M. (2010) · Journal of Clinical Child & Adolescent Psychology · doi:10.1080/15374416.2010.517169 · Cited by 238
Which instruments to support diagnosis of depression have sufficient accuracy? A systematic review
Pettersson, A., Boström, K. B., Gustavsson, P., & Ekselius, L. (2015) · Nordic Journal of Psychiatry · doi:10.3109/08039488.2015.1008568 · Cited by 187
Aebi, M., Kuhn, C., Banaschewski, T., Grimmer, Y., Poustka, L., Steinhausen, H.-C., & Goodman, R. (2017) · Child and Adolescent Psychiatry and Mental Health · doi:10.1186/s13034-017-0160-9 · Cited by 95
Metadata
- Authors
- Rettew, D. C., Lynch, A. D., Achenbach, T. M., Dumenci, L., & Ivanova, M. Y.
- Venue
- International Journal of Methods in Psychiatric Research
- Publisher
- Wiley
- Published
- 2009-08-21
- Volume
- 18
- Issue
- 3
- Pages
- 169-184
- ISSN
- 1049-8931 (ISSN-L)
- 1557-0657
- OpenAlex
- W2165744474
- Cited by
- 283
- References
- 88
- FWCI
- 15.13
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.