Inter-observer reliability of DSM-5 substance use disorders
Open accessAbstract
Aims Although studies have examined the impact of changes made in DSM-5 on the estimated prevalence of substance use disorder (SUD) diagnoses, there is limited evidence concerning the reliability of DSM-5 SUDs. We evaluated the inter-observer reliability of four DSM-5 SUDs in a sample in which we had previously evaluated the reliability of DSM-IV diagnoses, allowing us to compare the two systems.Methods Two different interviewers each assessed 173 subjects over a 2-week period using the Semi-Structured Assessment for Drug Dependence and Alcoholism (SSADDA). Using the percent agreement and kappa (κ) coefficient, we examined the reliability of DSM-5 lifetime alcohol, opioid, cocaine, and cannabis use disorders, which we compared to that of SSADDA-derived DSM-IV SUD diagnoses. We also assessed the effect of additional lifetime SUD and lifetime mood or anxiety disorder diagnoses on the reliability of the DSM-5 SUD diagnoses.Results Reliability was good to excellent for the four disorders, with κ values ranging from 0.65 to 0.94. Agreement was consistently lower for SUDs of mild severity than for moderate or severe disorders. DSM-5 SUD diagnoses showed greater reliability than DSM-IV diagnoses of abuse or dependence or dependence only. Co-occurring SUD and lifetime mood or anxiety disorders exerted a modest effect on the reliability of the DSM-5 SUD diagnoses.Conclusions For alcohol, opioid, cocaine and cannabis use disorders, DSM-5 criteria and diagnoses are at least as reliable as those of DSM-IV.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
DSM-5 SUD diagnoses showed greater reliability than DSM-IV diagnoses of abuse or dependence.
“DSM-5 SUD diagnoses showed greater reliability than DSM-IV diagnoses of abuse or dependence [33,34].”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965 · see citing works
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398 · see citing works
Statistical Methods for Rates and Proportions
Fleiss, J. L., Levin, B., & Paik, M. C. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/0471445428 · Cited by 18,646 · see citing works
Beckett, A. H., Rowland, M., & TURNER, P. (1965) · The Lancet · doi:10.1016/s0140-6736(65)91037-8 · Cited by 16,917 · see citing works
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736 · see citing works
The kappa statistic in reliability studies: use, interpretation, and sample size requirements
Sim, J., & Wright, C. C. (2005) · Physical Therapy · doi:10.1093/ptj/85.3.257 · Cited by 4,093 · see citing works
Hasin, D. S., Stinson, F. S., Ogburn, E., & Grant, B. F. (2007) · Archives of General Psychiatry · doi:10.1001/archpsyc.64.7.830 · Cited by 2,908 · see citing works
(1981) · PubMed · Cited by 2,253 · see citing works
Bucholz, K. K., Cadoret, R. J., Cloninger, C. R., Dinwiddie, S. H., HESSELBROCK, V. M., Nürnberger, J. I., Reich, T., Schmidt, I., & Schuckit, M. A. (1994) · Journal of Studies on Alcohol · doi:10.15288/jsa.1994.55.149 · Cited by 1,758 · see citing works
DSM-5 criteria for substance use disorders: recommendations and rationale
Hasin, D. S., O’Brien, C. P., Auriacombe, M., Borges, G., Bucholz, K., Budney, A., Compton, W. M., Crowley, T., Ling, W., Petry, N. M., et al. (2013) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2013.12060782 · Cited by 1,622 · see citing works
Regier, D. A., Narrow, W. E., Clarke, D. E., Kraemer, H. C., Kuramoto, S. J., Kuhl, E. A., & Kupfer, D. J. (2013) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2012.12070999 · Cited by 1,163 · cited 6× · see citing works
“Consistent with the results of other studies, we found that SUDs commonly cooccurred with other SUDs (Regier et al., 2013; Stinson et al., 2005) and psychiatric disorders (Compton et al., 2007; Conway et al., 2006).”
BackgroundResultInfluentialCompton, W. M., Thomas, Y. F., Stinson, F. S., & Grant, B. F. (2007) · Archives of General Psychiatry · doi:10.1001/archpsyc.64.5.566 · Cited by 1,152 · see citing works
Conway, K. P., Compton, W., Stinson, F. S., & Grant, B. F. (2006) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v67n0211 · Cited by 934 · cited 2× · see citing works
“Consistent with the results of other studies, we found that SUDs commonly cooccurred with other SUDs (Regier et al., 2013; Stinson et al., 2005) and psychiatric disorders (Compton et al., 2007; Conway et al., 2006).”
ResultStatistical Methods for Rates and Proportions
(2004) · Technometrics · doi:10.1198/tech.2004.s812 · Cited by 689 · see citing works
Stinson, F. S., Grant, B. F., Dawson, D. A., Ruan, W. J., Huang, B., & Saha, T. D. (2005) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2005.03.009 · Cited by 505 · see citing works
A Proposed Solution to the Base Rate Problem in the Kappa Statistic
Spitznagel, E. L. (1985) · Archives of General Psychiatry · doi:10.1001/archpsyc.1985.01790300093012 · Cited by 488 · see citing works
Hasin, D., Carpenter, K. M., McCloud, S., Smith, M., & Grant, B. F. (1997) · Drug and Alcohol Dependence · doi:10.1016/s0376-8716(97)01332-x · Cited by 369 · see citing works
Grant, B. F., Goldstein, R. B., Smith, S. M., Jung, J., Zhang, H., Chou, S. P., Pickering, R. P., Ruan, W. J., Huang, B., Saha, T. D., et al. (2015) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2014.11.026 · Cited by 346 · see citing works
Hasin, D., Hatzenbuehler, M. L., Keyes, K., & Ogburn, E. (2006) · Addiction · doi:10.1111/j.1360-0443.2006.01584.x · Cited by 213 · cited 4× · see citing works
“The DSM-IV substance dependence diagnoses have been shown to be reliable, with substance abuse diagnoses less so (Hasin et al., 1997, 2006; Pierucci-Lagha et al., 2007, 2005).”
MethodResultInfluentialDiagnostic reliability of the Semi-structured Assessment for Drug Dependence and Alcoholism (SSADDA)
Pierucci‐Lagha, A., Gelernter, J. E., Feinn, R., Cubells, J. F., Pearson, D., Pollastri, A. R., Farrer, L. A., & Kranzler, H. R. (2005) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2005.04.005 · Cited by 196 · see citing works
DSM-5: How Reliable Is Reliable Enough?
KRAEMER, H. C., Kupfer, D. J., Clarke, D. E., Narrow, W. E., & Regier, D. A. (2012) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2011.11010050 · Cited by 180 · see citing works
Pierucci‐Lagha, A., Gelernter, J. E., Chan, G., Arias, A. J., Cubells, J. F., Farrer, L. A., & Kranzler, H. R. (2007) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2007.04.014 · Cited by 140 · see citing works
Hasin, D. S., Fenton, M. C., Beseler, C. L., Park, J. Y., & Wall, M. M. (2011) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2011.09.005 · Cited by 131 · see citing works
Peer, K., Rennert, L., Lynch, K. G., Farrer, L. A., Gelernter, J. E., & Kranzler, H. R. (2012) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2012.07.009 · Cited by 108 · see citing works
Bucholz, K. K., HESSELBROCK, V. M., Shayka, J. J., Nürnberger, J. I., Schuckit, M. A., Schmidt, I., & Reich, T. (1995) · Journal of Studies on Alcohol · doi:10.15288/jsa.1995.56.500 · Cited by 106 · see citing works
Alcohol craving and the dimensionality of alcohol disorders
Keyes, K. M., Krueger, R. F., Grant, B. F., & Hasin, D. S. (2010) · Psychological Medicine · doi:10.1017/s003329171000053x · Cited by 105 · see citing works
Testing the equality of two dependent kappa statistics
Donner, A. P., Shoukri, M. M., Klar, N. S., & Bartfay, E. (2000) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(20000215)19:3 · Cited by 90 · see citing works
The dimensionality of alcohol use disorders and alcohol consumption in a cross‐national perspective
Borges, G. L. G., Ye, Y., Bond, J. C., CHERPITEL, C. J. S., Cremonte, M., Moskalewicz, J., Świątkiewicz, G., & Rubio‐Stipec, M. (2010) · Addiction · doi:10.1111/j.1360-0443.2009.02778.x · Cited by 85 · see citing works
Saha, T., Compton, W., Chou, S., Smith, S. M., Ruan, W., Huang, B.-J., Pickering, R., & Grant, B. F. (2011) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2011.09.004 · Cited by 81 · see citing works
Kelly, S. M., Gryczynski, J., Mitchell, S. G., Kirk, A. S., O’Grady, K. E., & Schwartz, R. P. (2014) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2014.03.034 · Cited by 33 · see citing works
Mewton, L. R., Slade, T. N., & Teesson, M. (2013) · Journal of Studies on Alcohol and Drugs · doi:10.15288/jsad.2013.74.614 · Cited by 32 · see citing works
Compatibility of current DSM-IV and proposed DSM-5 diagnostic criteria for cocaine use disorders
Proctor, S. L., Kopak, A. M., & Hoffmann, N. G. (2012) · Addictive Behaviors · doi:10.1016/j.addbeh.2012.02.010 · Cited by 28 · see citing works
Preuss, U. W., Watzke, S., & Wurst, F. (2014) · Psychological Medicine · doi:10.1017/s0033291714000889 · Cited by 26 · see citing works
Kerridge, B. T., Saha, T. D., Smith, S. M. R., Chou, P. S., Pickering, R. P., Huang, B., Ruan, J., & Pulay, A. J. (2011) · Addictive Behaviors · doi:10.1016/j.addbeh.2011.04.006 · Cited by 21 · see citing works
Gilder, D. A., Gizer, I. R., Lau, P., & Ehlers, C. L. (2013) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2013.10.010 · Cited by 16 · see citing works
Sources of Unreliability in the Diagnosis of Substance Dependence
Feinn, R., Gelernter, J., Cubells, J. F., Farrer, L., & Kranzler, H. R. (2009) · Journal of Studies on Alcohol and Drugs · doi:10.15288/jsad.2009.70.475 · Cited by 10 · see citing works
Self-citationA Note on Issues in Meta-Analysis for Behavioral Genetic Studies Using Categorical Phenotypes
Ridenour, T. A., & Heath, A. C. (1999) · Behavior Genetics · doi:10.1023/a:1021635800770 · Cited by 8 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Prevalence and Predictors of Substance Use Disorders Among HIV Care Enrollees in the United States
Hartzler, B. J., Dombrowski, J. C., Crane, H. M., Eron, J. J., Geng, E. H., Mathews, W. C., Mayer, K. H., Moore, R. D., Mugavero, M. J., Napravnik, S., et al. (2016) · AIDS and Behavior · doi:10.1007/s10461-016-1584-6 · Cited by 218 · see citing works
Will This Hospitalized Patient Develop Severe Alcohol Withdrawal Syndrome?
Wood, E., Albarqouni, L., Tkachuk, S., Green, C. J., Ahamad, K., Nolan, S., McLean, M., & Klimas, J. (2018) · JAMA · doi:10.1001/jama.2018.10574 · Cited by 114 · see citing works
Drug Use Disorders and Violence: Associations With Individual Drug Categories
Zhong, S., Yu, R., & Fazel, S. (2020) · Epidemiologic Reviews · doi:10.1093/epirev/mxaa006 · Cited by 112 · see citing works
Lago, L., Bruno, R., & Degenhardt, L. J. (2016) · The Lancet Psychiatry · doi:10.1016/s2215-0366(16)00088-2 · Cited by 58 · see citing works
Getting “clean” from nonsuicidal self-injury: Experiences of addiction on the subreddit r/selfharm
Himelein-Wachowiak, M., Giorgi, S., Kwarteng, A., Schriefer, D., Smitterberg, C. W., Yadeta, K., Bragard, E., Devoto, A., Kulkarni, P. V., & Curtis, B. L. (2022) · Journal of Behavioral Addictions · doi:10.1556/2006.2022.00005 · Cited by 44 · see citing works
Jun, H., Webb-Morgan, M., Felner, J. K., Wisdom, J. P., Haley, S. J., Austin, S. B., Katuska, L. M., & Corliss, H. L. (2019) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2019.107619 · Cited by 42 · see citing works
Hasin, D., Shmulewitz, D., Stohl, M., Greenstein, E., Roncone, S., Aharonovich, E., & Wall, M. (2020) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2020.108294 · Cited by 31 · see citing works
Does This Patient Have Alcohol Use Disorder?
Wood, E., Pan, J., Cui, Z., Bach, P., Dennis, B. B., Nolan, S., & Socias, M. E. (2024) · JAMA · doi:10.1001/jama.2024.3101 · Cited by 22 · see citing works
Megherbi-moulay, O., Igier, V., Jullian, B., Franchitto, N., & Sordes, F. (2022) · International Journal of Mental Health and Addiction · doi:10.1007/s11469-022-00974-z · Cited by 18 · see citing works
Fink, D. S., Shmulewitz, D., Mannes, Z. L., Stohl, M., Livne, O., Wall, M. M., & Hasin, D. S. (2022) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2022.09.016 · Cited by 16 · see citing works
Severity of Substance Use Disorder: Utility as an Outcome in Clinical Settings
Dacosta‐Sánchez, D., Fernández‐Calderón, F., González‐Ponce, B. M., Díaz, C., & Lozano, O. M. (2019) · Alcoholism Clinical and Experimental Research · doi:10.1111/acer.14020 · Cited by 16 · see citing works
Identification of Opioid Use Disorder Using Electronic Health Records: Beyond Diagnostic Codes
Poulsen, M., Nordberg, C., Troiani, V., Berrettini, W., Asdell, P. B., & Schwartz, B. S. (2023) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2023.110950 · Cited by 16 · see citing works
The Effect of Health Belief Model-Based Training on Preventive Behaviors of Hepatitis B in Addicts
Khodaveisi, M., Khah, M. S., Bashirian, S., Karami, M., & Khodaveisi, M. (2018) · international journal high risk behaviors & addiction · doi:10.5812/ijhrba.58579 · Cited by 15 · see citing works
Butelman, E. R., Bacciardi, S., Maremmani, A. G. I., Darst-Campbell, M., da Rosa, J. C., & Kreek, M. J. (2017) · American Journal on Addictions · doi:10.1111/ajad.12578 · Cited by 12 · see citing works
Liu, M.-C., Chang, J., & Lee, C. (2021) · BMC Psychiatry · doi:10.1186/s12888-021-03157-7 · Cited by 9 · see citing works
Fitzgerald, N. D., Striley, C. W., Palamar, J. J., Copeland, J., Kurtz, S. P., & Cottler, L. B. (2021) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2021.109056 · Cited by 8 · see citing works
Cochran, G., Shen, J., Cox, N., Field, C. A., Carlston, K., Sengpraseut, B., White, A., Okifuji, A., Jackman, C., Haaland, B. A., et al. (2022) · Contemporary Clinical Trials · doi:10.1016/j.cct.2022.106759 · Cited by 8 · see citing works
Choi, N. G., & Marti, C. N. (2025) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph22040640 · Cited by 8 · see citing works
“DSM-5 SUD diagnoses showed greater reliability than DSM-IV diagnoses of abuse or dependence [33,34].”
Ma, Y., Wang, Y., Liu, M., Fang, T., Wei, Z.-R., Chen, S., Tan, H., Nuñez, Y. Z., Zhang, X., Hao, W., et al. (2021) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2021.109047 · Cited by 7 · see citing works
The Impact of BMI on Adult Blunt Trauma Outcomes
Barry, R. G., Modarresi, M., Aguilar, R., Sanabria, J. A., Wolbert, T. T., Denning, D. A., Thompson, E. C., & Sanabria, J. R. (2019) · The American Surgeon · doi:10.1177/000313481908501229 · Cited by 7 · see citing works
Yang, B., Wang, L., Huang, M., Wang, S., Tsai, M.-C., Huang, Y., Nuñez, Y. Z., Ng, M., Kranzler, H. R., Gelernter, J. E., et al. (2020) · Complex Psychiatry · doi:10.1159/000511606 · Cited by 5 · see citing works
Taste and Smell Alterations and Substance Use Disorders
Agarwal, K., McDuffie, C., Manza, P., & Joseph, P. V. (2021) · Springer eBooks · doi:10.1007/978-3-030-86282-4_8 · Cited by 4 · see citing works
Characterizing Anxiety Among Individuals Receiving Treatment for Alcohol and Substance Use Disorders
Domenico, L. H., Lewis, B., Hazarika, M., & Nixon, S. J. (2017) · Journal of the American Psychiatric Nurses Association · doi:10.1177/1078390317739106 · Cited by 3 · see citing works
Wang, Y., Zuo, J., Wang, L., Wang, Q., Wang, X., Yang, Q., Wu, H. E., Goodman, C., Wang, D., Liu, T., et al. (2021) · Frontiers in Public Health · doi:10.3389/fpubh.2021.739068 · Cited by 3 · see citing works
Hohmeier, K. C., Broussard, G., Tyszko, M., Barland, K. M., Carlston, K., Wei, Y., Field, C. A., Gordon, A. J., Cernasev, A., & Cochran, G. (2025) · JACCP JOURNAL OF THE AMERICAN COLLEGE OF CLINICAL PHARMACY · doi:10.1002/jac5.70113 · Cited by 2 · see citing works
Chapter 4. Substance Use Disorder Diagnoses in DSM-5
Hasin, D. S., & Larson, F. R. R. (2021) · American Psychiatric Association Publishing eBooks · doi:10.1176/appi.books.9781615373970.kb04 · Cited by 2 · see citing works
Wang, M., Qian, Y.-X., Ding, Z., Yuan, C., Yang, X., Liu, Y., & Sun, Y. (2022) · IntechOpen eBooks · doi:10.5772/intechopen.105582 · Cited by 2 · see citing works
Substance Use Disorder - New Research Perspectives in the Diagnosis, Treatment, and Prognosis
Sampedro-Piquero, P., Fernández, R. D. M., & Zancada-Menéndez, C. (2022) · IntechOpen eBooks · doi:10.5772/intechopen.100838
Ware, O. D., & Dunn, K. E. (2023) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780197618431.013.3
Identifying neurofunctional domains across substance use disorders
Hartwell, E. E., Schwandt, M. L., Nuñez, Y. Z., Wetherill, R. R., Kember, R. L., Wiers, C. E., Gelernter, J. E., & Kranzler, H. R. (2024) · The American Journal of Drug and Alcohol Abuse · doi:10.1080/00952990.2024.2368180
Magnerou, A. M., Eyoum, C., Bako, A. F. H., Sini, V., Gams, D. M., Lamou, E. G. B., Iyawa, H., Doumbe, J. N., Kuate-Tegueu, C., & Mapoure, Y. N. (2025) · BMC Psychiatry · doi:10.1186/s12888-025-07690-7
Liu, M.-C., Chang, J. C., & Lee, C.-S. (2021) · Figshare · doi:10.6084/m9.figshare.14266744
Cochran, G., Broussard, G., Wei, Y., Field, C. A., Gordon, A. J., & Hohmeier, K. C. (2026) · Drug and Alcohol Dependence Reports · doi:10.1016/j.dadr.2026.100418
Setzer, M. R., Kember, R. L., Hartwell, E. E., Piserchia, Z., Gelernter, J. E., Kranzler, H. R., & Gray, J. C. (2026) · Psychological Medicine · doi:10.1017/s0033291725103103
- Authors
- Denis, C. M., Gelernter, J., Hart, A. B., & Kranzler, H. R.
- Affiliations
- University of Pennsylvania (US)
- Yale University (US)
- Mental Illness Research, Education and Clinical Centers (US)
- Corresponding author
- Kranzler, Henry R.
- Venue
- Drug and Alcohol Dependence
- Publisher
- Elsevier BV
- Published
- 2015-08
- Volume
- 153
- Pages
- 229-235
- ISSN
- 0376-8716 (ISSN-L)
- PMC
- PMC4509850
- OpenAlex
- W1588220416
- Cited by
- 36
- References
- 37
- Citations by year
- 2026: 2 · 2025: 3 · 2024: 2 · 2023: 3 · 2022: 7 · 2021: 7 · 2020: 3 · 2019: 3 · 2018: 2 · 2017: 2 · 2016: 2
- FWCI
- 1.57
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Drug and Alcohol Dependence (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- Funders
- NIH (DA12422, DA12849, DA12690, DA028874, AA13736)
- Topics
- Health Sciences › Medicine › Epidemiology › Substance Abuse Treatment and Outcomes
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Arts and Humanities › Philosophy › Mental Health and Psychiatry
- Keywords
- DSM-5, substance use disorders, inter-rater reliability, alcoholism, opioid use disorder, cocaine dependence, cannabis use disorder, diagnostic criteria
- MeSH terms
- Adult, Anxiety Disorders, Female, Humans, Male, Observer Variation, Mood Disorders, Substance-Related Disorders, Diagnostic and Statistical Manual of Mental Disorders
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
