Progress toward achieving a common language in psychiatry, II: Results from the international field trials of the ICD-10 diagnostic criteria for research for mental and behavioral disorders
Abstract
OBJECTIVE: In preparing for the publication of ICD-10, the Division of Mental Health of the World Health Organization developed several versions of chapter V, which deals with mental and behavioral disorders. The version for research purposes is called the Diagnostic Criteria for Research (ICD-10 DCR) and gives operational criteria for the diagnosis of mental disorders. This article describes the results of international field trials undertaken to evaluate the draft criteria and refine them further. METHOD: Data were obtained to assess interrater agreement, the confidence with which diagnoses could be made, and the ease of use of the criteria. Additional substudies examined the concordance between ICD-10 DCR, ICD-10 Clinical Descriptions and Diagnostic Guidelines, and other national classification systems (e.g., DSM-IV). The field trials were carried out at 151 clinical centers in 32 countries by 942 clinician/researchers who conducted 11,491 individual assessments of patients. Results for cases assessed by at least two raters are reported here. RESULTS: Most clinician/researchers found the criteria to be explicit and easy to apply. Interrater agreement was high for most diagnostic categories. For some categories, such as those dealing with certain polymorphic psychotic disorders or milder forms of affective disorders, the criteria were rated as somewhat difficult to use, and reliability was lower. Comparison of the results of these field trials with those of the ICD-10 Clinical Descriptions and Diagnostic Guidelines demonstrated that there are increases in interrater agreement when the operational criteria are used. CONCLUSIONS: The use of internationally accepted research criteria enhances the reliability of diagnosis of mental disorders made in research settings worldwide.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the ICD-10 Diagnostic Criteria for Research field trial, acute and transient psychotic disorder with polymorphic presentation had reliability similar to that found in the current study.
“The reliability of acute and transient psychotic disorder with polymorphic presentation in ICD-10 Diagnostic Criteria for Research field trial 33 was similar to that in the present study.”
In the ICD-10 field trials, less severe disorders had lower reliability than more severe disorders such as schizophrenia and bipolar disorder.
“This is reassuring, because the reliability of milder disorders compared to more severe disorders (e.g., schizophrenia and bipolar disorder) was lower in ICD-10 field trials 16,33 .”
The ICD-10 Diagnostic Criteria for Research field trial involved a structured diagnostic instrument covering the diagnostic criteria for assessment.
“The ICD-10 Diagnostic Criteria for Research field trial involved the use of a structured diagnostic instrument that covered the diagnostic criteria for assessment 33 .”
Cites
Spitzer, R. L. (1978) · Archives of General Psychiatry · doi:10.1001/archpsyc.1978.01770300115013 · Cited by 6,946 · see citing works
Diagnostic criteria for use in psychiatric research.
Feighner, J. P., Robins, E., Guze, S. B., Woodruff, R. A., Winokur, G., & Munoz, R. A. (1972) · Archives of General Psychiatry · doi:10.1001/archpsyc.1972.01750190059011 · Cited by 5,491 · see citing works
Robins, L. N. (1988) · Archives of General Psychiatry · doi:10.1001/archpsyc.1988.01800360017003 · Cited by 2,909 · see citing works
SCAN: Schedules fonr Clinical Assessment in Neuropsychiatry
Wing, J. K. (1990) · Archives of General Psychiatry · doi:10.1001/archpsyc.1990.01810180089012 · Cited by 2,517 · see citing works
Loranger, A. W. (1994) · Archives of General Psychiatry · doi:10.1001/archpsyc.1994.03950030051005 · Cited by 1,069 · see citing works
Reliability Studies of Psychiatric Diagnosis
Grove, W. M. (1981) · Archives of General Psychiatry · doi:10.1001/archpsyc.1981.01780290042004 · Cited by 450 · see citing works
Sartorius, N. (1993) · Archives of General Psychiatry · doi:10.1001/archpsyc.1993.01820140037004 · Cited by 195 · see citing works
Self-citationModeling Agreement Among Raters
Tanner, M. A., & Young, M. A. (1985) · Journal of the American Statistical Association · doi:10.1080/01621459.1985.10477157 · Cited by 162 · see citing works
(1951) · Tubercle · doi:10.1016/s0041-3879(51)80130-2 · Cited by 125 · see citing works
Assessment of agreement among several raters formulating multiple diagnoses
Mezzich, J. E., KRAEMER, H. C., Worthington, D. R. L., & Coffman, G. A. (1981) · Journal of Psychiatric Research · doi:10.1016/0022-3956(81)90011-x · Cited by 120 · see citing works
The WHO/ADAMHA International Pilot Study of Personality Disorders: Background and Purpose
Loranger, A. W., Hirschfeld, R. M. A., Sartorius, N., & Regier, D. A. (1991) · Journal of Personality Disorders · doi:10.1521/pedi.1991.5.3.296 · Cited by 65 · see citing works
Self-citationLandis, J. R., & Koch, G. G. (1975) · Statistica Neerlandica · doi:10.1111/j.1467-9574.1975.tb00259.x · Cited by 40 · see citing works
The ICD‐9 classification of mental disorders| A REVIEW OF ITS DEVELOPMENT AND CONTENTS
Kramer, M. E., Sartorius, N., Jablensky, A. V., et al. (1979) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.1979.tb06965.x · Cited by 34 · see citing works
Self-citationClassification: An International Perspective
Sartorius, N. (1976) · Psychiatric Annals · doi:10.3928/0048-5713-19760801-06 · Cited by 32 · see citing works
Self-citationMellsop, G. W., Thomas, C. S., Ellis, P. M., Purdie, G., Crawshaw, J., & Mendis, N. (1991) · Acta Psychiatrica Scandinavica · doi:10.1111/j.1600-0447.1991.tb03155.x · Cited by 14 · see citing works
Kitamura, T., Nakagawa, Y., & Machizawa, S. (1993) · Comprehensive Psychiatry · doi:10.1016/0010-440x(93)90011-r · Cited by 7 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Regier, D. A., Narrow, W. E., Clarke, D. E., Kraemer, H. C., Kuramoto, S. J., Kuhl, E. A., & Kupfer, D. J. (2013) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2012.12070999 · Cited by 1,163 · see citing works
Classification in psychiatry: ICD–10 <i>versus</i> DSM–IV
Andrews, G., Slade, T., & Peters, L. (1999) · British Journal of Psychiatry · doi:10.1192/bjp.174.1.3 · Cited by 145 · see citing works
Reed, G. M., Sharan, P., Rebello, T. J., Keeley, J. W., Elena Medina‐Mora, M., Gureje, O., Luis Ayuso‐Mateos, J., Kanba, S., Khoury, B., Kogan, C. S., et al. (2018) · World Psychiatry · doi:10.1002/wps.20524 · Cited by 126 · see citing works
Schizoaffective disorder–an ongoing challenge for psychiatric nosology
Jäger, M., Haack, S., Becker, T., & Frasch, K. (2011) · European Psychiatry · doi:10.1016/j.eurpsy.2010.03.010 · Cited by 98 · see citing works
Reed, G. M., Keeley, J. W., Rebello, T. J., First, M. B., Gureje, O., Ayuso‐Mateos, J. L., Kanba, S., Khoury, B., Kogan, C. S., Krasnov, V. N., et al. (2018) · World Psychiatry · doi:10.1002/wps.20581 · Cited by 91 · see citing works
Dalsgaard, S., Hansen, N., Mortensen, P. B., Damm, D., & Thomsen, P. H. (2001) · European Child & Adolescent Psychiatry · doi:10.1007/s007870170012 · Cited by 32 · see citing works
Classification in psychiatry: ICD–10 <i>v.</i> DSM–IV: A response
First, M. B., & Pincus, H. A. (1999) · British Journal of Psychiatry · doi:10.1192/bjp.175.3.205 · Cited by 30 · see citing works
Beltran, R. O., Silove, D., & Llewellyn, G. M. (2009) · Psychopathology · doi:10.1159/000204761 · Cited by 28 · see citing works
Bertelsen, A. (1999) · Australian & New Zealand Journal of Psychiatry · doi:10.1046/j.1440-1614.1999.00525.x · Cited by 24 · see citing works
- Authors
- Sartorius, N., Üstün, T. B., Korten, A. E., Cooper, J. E., & Drimmelen, J. V.
- Affiliations
- University of Geneva (CH)
- ORCID
- Norman Sartorius
- Venue
- American Journal of Psychiatry
- Published
- 1995-10-01
- Volume
- 152
- Issue
- 10
- Pages
- 1427-1437
- ISSN
- 0002-953X (ISSN-L)
- 1535-7228
- OpenAlex
- W61915756
- Cited by
- 135
- References
- 16
- Citations by year
- 2026: 3 · 2023: 1 · 2022: 1 · 2021: 2 · 2020: 1 · 2019: 4 · 2018: 5 · 2017: 3 · 2016: 7 · 2015: 5 · 2013: 7 · 2012: 5
- FWCI
- 24.89
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- American Journal of Psychiatry (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Arts and Humanities › Philosophy › Mental Health and Psychiatry
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Psychology › Social Psychology › Mental Health Treatment and Access
- Keywords
- ICD-10, mental illness, behavioral disorders, diagnostic classification, interobserver agreement, diagnostic reliability, field trials, psychiatric classification
- MeSH terms
- Adolescent, Adult, Aged, Mental Disorders, Child, Evaluation Studies as Topic, Female, Humans, Male, Middle Aged, Terminology as Topic, Psychiatric Status Rating Scales, Psychiatry, World Health Organization, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
