The Reliability of Clinical Diagnoses: State of the Art

Kraemer, Helena Chmura

doi:10.1146/annurev-clinpsy-032813-153739

Cited by 39Reliability and Agreement in MeasurementClinical Reasoning and Diagnostic SkillsPsychometric Methodologies and Testing

The Reliability of Clinical Diagnoses: State of the Art

Kraemer, Helena Chmura·2014·Annual Review of Clinical Psychology

DOI 10.1146/annurev-clinpsy-032813-153739

Open this paper in CoChat — free account

Abstract

Reliability of clinical diagnosis is essential for good clinical decision making as well as productive clinical research. The current review emphasizes the distinction between a disorder and a diagnosis and between validity and reliability of diagnoses, and the relationships that exist between them. What is crucial is that reliable diagnoses are essential to establishing valid diagnoses. The present review discusses the theoretical background underlying the evaluation of diagnoses, possible designs of reliability studies, estimation of the reliability coefficient, the standards for assessment of reliability, and strategies for improving reliability without compromising validity.

Integrity signals

Registry

Crossref · 10.1146/annurev-clinpsy-032813-153739

Cites

49 of 49 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The joint-rater concurrent reliability method, which gives both diagnosticians identical information, usually produces higher kappa values than separate interviews.
“First, it bears repeating that the joint-rater (concurrent) method of testing reliability, which constrains the information provided to the two diagnosticians to be identical, usually generates higher kappa values compared to those obtained when separate interviews are conducted 26,51 .”
From The ICD‐11 developmental field study of reliability of diagnoses of high‐burden mental disorders: results among adult patients in mental health settings of 13 countries (2018) · cites this paper as [51]

Diagnostic and Statistical Manual of Mental Disorders
American Psychiatric Association (2013) · doi:10.1176/appi.books.9780890425596 · Cited by 116,369 · see citing works
Diagnostic and Statistical Manual of Mental Disorders
Williams, J. B. W., & First, M. B. (2013) · Oxford University Press eBooks · doi:10.1093/acrefore/9780199975839.013.104 · Cited by 109,841 · see citing works
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148 · see citing works
Intraclass correlations: Uses in assessing rater reliability.
Shrout, P. E., & Fleiss, J. L. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.2.420 · Cited by 23,433 · see citing works
Statistical Methods for Rates and Proportions
Fleiss, J. L., Levin, B., & Paik, M. C. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/0471445428 · Cited by 18,646 · see citing works
Diagnostic and Statistical Manual of Mental Disorders
&NA; (1988) · Alzheimer Disease & Associated Disorders · doi:10.1097/00002093-198802020-00022 · Cited by 12,603 · see citing works
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736 · see citing works
Measuring nominal scale agreement among many raters.
Fleiss, J. L. (1971) · Psychological Bulletin · doi:10.1037/h0031619 · Cited by 8,733 · see citing works
Reviews: Lord, Frederic M., and Novick, Melvin R. (with contributions by Allan Birnbaum). Statistical Theories of Mental Test Scores. Reading, Mass.: Addison-Wesley, 1968. xii + 568 pp. $14.95.
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012 · see citing works
A leisurely look at the bootstrap, the jackknife, and cross-validation
Efron, B., & Gong, G. (1983) · The American Statistician · doi:10.1080/00031305.1983.10483087 · Cited by 3,229 · cited 1× · see citing works
“Once the study is done, bootstrap methods (Efron 1988, Efron & Gong 1983, Efron & Tibshirani 1995) can also be used both to estimate the SE and to compute the 95% confidence interval.”
Method
Limitations of the Application of Fourfold Table Analysis to Hospital Data
Berkson, J. (1946) · Biometrics Bulletin · doi:10.2307/3002000 · Cited by 1,810 · cited 1× · see citing works
“Frequently, the highrisk group is not representative of the likely stratum or the “low-risk” group is not representative of the less likely stratum of the population, a situation producing Berkson’s fallacy (Berkson 1946, 1955; Brown 1976).”
Background
Statistical Methods for Rates and Proportions.
Plewis, I., & Fleiss, J. L. (1982) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2982107 · Cited by 1,753 · see citing works
Large sample standard errors of kappa and weighted kappa.
Fleiss, J. L., Cohen, J., & Everitt, B. S. (1969) · Psychological Bulletin · doi:10.1037/h0028106 · Cited by 1,542 · see citing works
CORRELATION CALCULATED FROM FAULTY DATA
Spearman, C. (1910) · British Journal of Psychology 1904-1920 · doi:10.1111/j.2044-8295.1910.tb00206.x · Cited by 1,192 · see citing works
DSM-5 field trials in the United States and Canada, part II: test-retest reliability of selected categorical diagnoses
Regier, D. A., Narrow, W. E., Clarke, D. E., Kraemer, H. C., Kuramoto, S. J., Kuhl, E. A., & Kupfer, D. J. (2013) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2012.12070999 · Cited by 1,163 · cited 4× · see citing works
“For example, the proposed diagnosis of disruptive mood dysregulation disorder (excluded from DSM-5) was good at one site (k = 0.49) and unacceptable at two others (k = 0.06, 0.11) (Regier et al. 2013).”
BackgroundInfluentialSelf-citation
On various intraclass correlation reliability coefficients.
Bartko, J. J. (1976) · Psychological Bulletin · doi:10.1037/0033-2909.83.5.762 · Cited by 818 · see citing works
The Reliability of Clinical Methods, Data and Judgments
Koran, L. M. (1975) · New England Journal of Medicine · doi:10.1056/nejm197509252931307 · Cited by 654 · see citing works
Evaluating Medical Tests: Objective and Quantitative Guidelines.
Guthrie, D., & Kraemer, H. C. (1992) · Journal of the American Statistical Association · doi:10.2307/2290671 · Cited by 636 · see citing works
Self-citation
SOME EXPERIMENTAL RESULTS IN THE CORRELATION OF MENTAL ABILITIES 1
Brown, W. E. (1910) · British Journal of Psychology 1904-1920 · doi:10.1111/j.2044-8295.1910.tb00207.x · Cited by 580 · see citing works
A Proposed Solution to the Base Rate Problem in the Kappa Statistic
Spitznagel, E. L. (1985) · Archives of General Psychiatry · doi:10.1001/archpsyc.1985.01790300093012 · Cited by 488 · see citing works
Saving normal: an insider’s revolt against out-of-control psychiatric diagnosis, DSM-5, big pharma, and the medicalization of ordinary life
Lakeman, R. (2013) · Psychosis · doi:10.1080/17522439.2013.830642 · Cited by 473 · see citing works
Saving normal: an insider's revolt against out-of-control psychiatric diagnosis, DSM-5, big pharma, and the medicalization of ordinary life
(2014) · Choice Reviews Online · doi:10.5860/choice.51-3280 · Cited by 449 · see citing works
Observer agreement in evaluating coronary angiograms.
Detre, K. M., Wright, E., Murphy, M. L., & Takaro, T. (1975) · Circulation · doi:10.1161/01.cir.52.6.979 · Cited by 442 · see citing works
The Reliability of Clinical Methods, Data and Judgments
Koran, L. M. (1975) · New England Journal of Medicine · doi:10.1056/nejm197510022931405 · Cited by 402 · see citing works
DSM-III field trials: I. Initial interrater diagnostic reliability
(1979) · American Journal of Psychiatry · doi:10.1176/ajp.136.6.815 · Cited by 394 · see citing works
Ramifications of a Population Model for κ as a Coefficient of Reliability
KRAEMER, H. C. (1979) · Psychometrika · doi:10.1007/bf02296208 · Cited by 295 · see citing works
Self-citation
2 x 2 Kappa Coefficients: Measures of Agreement or Association
Bloch, D. A., & KRAEMER, H. C. (1989) · Biometrics · doi:10.2307/2532052 · Cited by 291 · see citing works
Self-citation
Extension of the Kappa Coefficient
KRAEMER, H. C. (1980) · Biometrics · doi:10.2307/2529972 · Cited by 237 · see citing works
Self-citation
DSM-5 field trials in the United States and Canada, part I: study design, sampling strategy, implementation, and analytic approaches
Clarke, D. E., Narrow, W. E., Regier, D. A., Kuramoto, S. J., Kupfer, D. J., Kuhl, E. A., Greiner, L., & Kraemer, H. C. (2013) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2012.12070998 · Cited by 201 · see citing works
Self-citation
DSM-5: How Reliable Is Reliable Enough?
KRAEMER, H. C., Kupfer, D. J., Clarke, D. E., Narrow, W. E., & Regier, D. A. (2012) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2011.11010050 · Cited by 180 · see citing works
Self-citation
The validity of psychiatric diagnosis
Fahy, T. A. (1990) · Behaviour Research and Therapy · doi:10.1016/0005-7967(90)90036-i · Cited by 175 · see citing works
Evaluating Medical Tests: Objective and Quantitative Guidelines
Traub, S. L. (1993) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/50.11.2440 · Cited by 163 · see citing works
Bootstrap confidence intervals: Good or bad?
Efron, B. (1988) · Psychological Bulletin · doi:10.1037/0033-2909.104.2.293 · Cited by 162 · see citing works
A Comparison of Confidence Interval Methods for the Intraclass Correlation Coefficient
Donner, A. P., & Wells, G. (1986) · Biometrics · doi:10.2307/2531060 · Cited by 147 · see citing works
The Validity of psychiatric diagnosis
Robins, L. N., & Barrett, J. E. (1989) · Raven Press eBooks · Cited by 143 · see citing works
Inference About Weighted Kappa in the Non-Null Case
Fleiss, J. L., & Cicchetti, D. V. (1978) · Applied Psychological Measurement · doi:10.1177/014662167800200111 · Cited by 142 · see citing works
The Book of Woe: The DSM and the Unmaking of Psychiatry
Greenberg, G. (2013) · Medical Entomology and Zoology · Cited by 133 · see citing works
The Book of Woe: The DSM and the Unmaking of Psychiatry
Bard, J. S. (2013) · Journal of Legal Medicine · doi:10.1080/01947648.2013.831304 · Cited by 131 · see citing works
THE STATISTICAL STUDY OF ASSOCIATION BETWEEN SMOKING AND LUNG CANCER
Berkson, J. (1955) · Mayo Clinic Proceedings · doi:10.1016/s0025-6196(26)01720-9 · Cited by 110 · see citing works
Psychological tests and clinical discriminations: Beginning to address the base rate problem
Elwood, R. W. (1993) · Clinical Psychology Review · doi:10.1016/0272-7358(93)90012-b · Cited by 94 · see citing works
A nonparametric measure of intraclass correlation
ROTHERY, P. (1979) · Biometrika · doi:10.1093/biomet/66.3.629 · Cited by 62 · see citing works
Inferences Concerning a Common Intraclass Correlation Coefficient
Donner, A. P., & Bull, S. B. (1983) · Biometrics · doi:10.2307/2531107 · Cited by 60 · see citing works
Berkson Fallacy Revisited
Brown, G. W. (1976) · American journal of diseases of children · doi:10.1001/archpedi.1976.02120020058009 · Cited by 38 · see citing works
Confidence interval construction for a difference between two dependent intraclass correlation coefficients
Ramasundarahettige, C. F., Donner, A. P., & Zou, G. (2009) · Statistics in Medicine · doi:10.1002/sim.3523 · Cited by 37 · see citing works
How many raters? toward the most reliable diagnostic consensus
KRAEMER, H. C. (1992) · Statistics in Medicine · doi:10.1002/sim.4780110305 · Cited by 35 · see citing works
Self-citation
Comment on Bartko's "On various intraclass correlation reliability coefficients."
Algina, J. J. (1978) · Psychological Bulletin · doi:10.1037/0033-2909.85.1.135 · Cited by 33 · see citing works
Base-rates, cut-points and interaction effects: the problem with dichotomized continuous variables
Veiel, H. O. F. (1988) · Psychological Medicine · doi:10.1017/s0033291700008394 · Cited by 27 · see citing works
Exact and Resampling Probability Values for Weighted Kappa
Berry, K. J., Johnston, J. E., & Mielke, P. W. (2005) · Psychological Reports · doi:10.2466/pr0.96.2.243-252 · Cited by 10 · see citing works
Computer‐Intensive Statistical Methods
Efron, B., & Tibshirani, R. J. (2004) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess0273 · Cited by 4 · see citing works

Flagged first, then by citations

Sorted by citations

Harnessing reliability for neuroscience research
Zuo, X.-N., Xu, T., & Milham, M. P. (2019) · Nature Human Behaviour · doi:10.1038/s41562-019-0655-x · Cited by 310 · see citing works
Method of adult diagnosis influences estimated persistence of childhood ADHD: a systematic review of longitudinal studies
Sibley, M. H., Mitchell, J. T., & Becker, S. P. (2016) · The Lancet Psychiatry · doi:10.1016/s2215-0366(16)30190-0 · Cited by 268 · see citing works
Pressure ulcer/injury classification today: An international perspective
Kottner, J., Cuddigan, J., Carville, K., Balzer, K., Berlowitz, D. R., Law, S., Litchford, M. D., Mitchell, P., Moore, Z. E. H., Pittman, J. A., et al. (2020) · Journal of Tissue Viability · doi:10.1016/j.jtv.2020.04.003 · Cited by 194 · see citing works
Assessing clinicopathological correlation in chronic traumatic encephalopathy: rationale and methods for the UNITE study
Mez, J., Solomon, T. M., Daneshvar, D. H., Murphy, L., Kiernan, P. T., Montenigro, P. H., Kriegel, J., Abdolmohammadi, B., Fry, B. T., Babcock, K. J., et al. (2015) · Alzheimer s Research & Therapy · doi:10.1186/s13195-015-0148-8 · Cited by 148 · see citing works
Individual Variability and Test-Retest Reliability Revealed by Ten Repeated Resting-State Brain Scans over One Month
Chen, B., Xu, T., Zhou, C., Wang, L., Yang, N., Wang, Z., Dong, H.-M., Yang, Z., Zang, Y., Zuo, X., et al. (2015) · PLoS ONE · doi:10.1371/journal.pone.0144963 · Cited by 136 · see citing works
The ICD‐11 developmental field study of reliability of diagnoses of high‐burden mental disorders: results among adult patients in mental health settings of 13 countries
Reed, G. M., Sharan, P., Rebello, T. J., Keeley, J. W., Elena Medina‐Mora, M., Gureje, O., Luis Ayuso‐Mateos, J., Kanba, S., Khoury, B., Kogan, C. S., et al. (2018) · World Psychiatry · doi:10.1002/wps.20524 · Cited by 126 · see citing works
A Comparison Among 5 Methods for the Clinical Diagnosis of Fetal Alcohol Spectrum Disorders
Coles, C. D., Gailey, A. R., Mulle, J. G., Kable, J. A., Lynch, M. E., & Jones, K. L. (2016) · Alcoholism Clinical and Experimental Research · doi:10.1111/acer.13032 · Cited by 126 · see citing works
Empirically-informed guidelines for first-time adult ADHD diagnosis
Sibley, M. (2021) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/13803395.2021.1923665 · Cited by 74 · see citing works
Moving beyond processing- and analysis-related variation in resting-state functional brain imaging
Li, X., Esper, N. B., Ai, L., Giavasis, S., Jin, H., Feczko, E., Xu, T., Clucas, J. C., Franco, A. R., Heinsfeld, A. S., et al. (2024) · Nature Human Behaviour · doi:10.1101/2021.12.01.470790 · Cited by 73 · see citing works
Effect sizes and test-retest reliability of the fMRI-based neurologic pain signature
Han, X., Ashar, Y. K., Kragel, P. A., Petre, B., Schelkun, V., Atlas, L. Y., Chang, L. J., Jepma, M., Koban, L., Losin, E. A. R., et al. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2021.118844 · Cited by 63 · see citing works
The anatomy of reliability: a must read for future human brain mapping
Xing, X.-X., & Zuo, X. (2018) · 中国科学通报：英文版 · doi:10.1016/j.scib.2018.12.010 · Cited by 57 · see citing works
Editorial: Reliability and Reproducibility in Functional Connectomics
Zuo, X., Biswal, B. B., & Poldrack, R. A. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.00117 · Cited by 57 · see citing works
Test–retest reliability of schizoaffective disorder compared with schizophrenia, bipolar disorder, and unipolar depression—a systematic review and meta‐analysis
Santelmann, H., Franklin, J., Bußhoff, J., & Baethge, C. (2015) · Bipolar Disorders · doi:10.1111/bdi.12340 · Cited by 53 · see citing works
Challenges and opportunities for the development of new antipsychotic drugs
Forray, C., & Buller, R. (2017) · Biochemical Pharmacology · doi:10.1016/j.bcp.2017.05.009 · Cited by 33 · see citing works
The elephant in the record: On the multiplicity of data recording work
Cabitza, F., Locoro, A., Alderighi, C., Rasoini, R., Compagnone, D., & Berjano, P. L. (2019) · Health Informatics Journal · doi:10.1177/1460458218824705 · Cited by 32 · see citing works
The future of diagnosis in clinical neurosciences: Comparing multiple sclerosis and schizophrenia
Misiak, B., Samochowiec, J., Kowalski, K., Gäebel, W., Bassetti, C. L., Chan, A., Gorwood, P. A. P. M., Papiol, S., Dom, G., Volpe, U., et al. (2023) · European Psychiatry · doi:10.1192/j.eurpsy.2023.2432 · Cited by 28 · see citing works
Optimizing network neuroscience computation of individual differences in human spontaneous brain activity for test-retest reliability
Jiang, C., He, Y., Betzel, R. F., Wang, Y., Xing, X.-X., & Zuo, X. (2023) · Network Neuroscience · doi:10.1162/netn_a_00315 · Cited by 22 · see citing works
Comparing reliabilities of strip and conventional patch testing
Dickel, H., Geier, J., Kreft, B., Pfützner, W., & Kuß, O. (2017) · Contact Dermatitis · doi:10.1111/cod.12758 · Cited by 21 · see citing works
Changes in the classification of personality disorders: Comparing the DSM–5 Section II personality disorder model to the alternative model for personality disorders using structured clinical interviews.
Weekers, L. C., Hutsebaut, J., Zimmermann, J., & Kamphuis, J. H. (2021) · Personality Disorders Theory Research and Treatment · doi:10.1037/per0000512 · Cited by 18 · see citing works
Process and Content of DSM-5
Widiger, T. A., & Crego, C. (2015) · Psychopathology Review · doi:10.5127/pr.035314 · Cited by 14 · see citing works
The Clinical Consistency and Utility of ICD-11 Diagnostic Guidelines for Gaming Disorder: A Field Study Among the Chinese Population
Ma, C., Wang, Z., Li, C., Lu, J., Long, J., Li, R., Wu, Q., Jiang, H., Du, J., Li, R., et al. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.781992 · Cited by 10 · see citing works
Taking Steps Toward a Consensus on Night Eating Syndrome Diagnostic Criteria
Weissman, R. S., Martin‐Wagar, C. A., Attaway, S. E., Penwell, T., Hogan, A., & Pruscino, I. (2024) · International Journal of Eating Disorders · doi:10.1002/eat.24309 · Cited by 9 · see citing works
Emotional Awareness and Decision-Making in the Context of Computer-Mediated Psychotherapy
Alvandi, E. O., Doorn, G. H. V., & Symmons, M. A. (2019) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-019-00050-7 · Cited by 9 · see citing works
Core considerations in the development of the world health organization's international classification of diseases, 11 th Revision
Reed, G. M., Rebello, T. J., & Saxena, S. (2018) · Indian Journal of Social Psychiatry · doi:10.4103/ijsp.ijsp_43_18 · Cited by 9 · see citing works
Reliability and Clinical Utility of ICD-11 Diagnostic Guidelines for Severe Mental Disorders in Nigeria
Onofa, L. E. U., Odunleye, M., Kola, L., & Gureje, O. (2019) · Archives of Medical Research · doi:10.1016/j.arcmed.2020.01.004 · Cited by 8 · see citing works
Diagnostic Test Score Validation With a Fallible Criterion
Jewsbury, P. A. (2018) · Applied Psychological Measurement · doi:10.1177/0146621618817785 · Cited by 7 · see citing works
A new perspective on individual reliability beyond group effect for event-related potentials: A multisensory investigation and computational modeling
Hu, Z., Zhang, Z., Liang, Z., Zhang, L., Li, L., & Huang, G. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.118937 · Cited by 7 · see citing works
Reliability of hysteroscopy-based diagnosis of septate, arcuate and normal uterus: estimate or guestimate?
Ludwin, A., & Ludwin, I. (2016) · Human Reproduction · doi:10.1093/humrep/dew086 · Cited by 7 · see citing works
Examining the psychometric properties of the flexible interview for ICD-11 (FLII-11) among adults in India
Jaisoorya, T. S., Rao, G. N., Reed, G. M., Binu, V. S. N., Thennarasu, K., Stein, D. J., Maré, K. T., Löchner, C., García‐Pacheco, J. Á., & Benegal, V. (2025) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2025.03.024 · Cited by 6 · see citing works
Refining Diagnostic Procedures for Adults with Symptoms of ADHD: A Response to “Psychometrically-Informed Approach to Integration of Multiple Informant Ratings in Adult ADHD Diagnosis in a Community-Recruited Sample”
Sibley, M., Coxe, S., & Molina, B. (2016) · Assessment (Odessa, Fla.) · doi:10.1177/1073191116676890 · Cited by 3 · see citing works
Richer than we thought: neurophysiological methods reveal rich-club network development is frequency- and sex-dependent
Vandewouw, M. M., Pang, E. W., Lai, M., Kelley, E. A., Ayub, M., Lerch, J. P., Taylor, M. J., & Anagnostou, E. A. (2023) · iScience · doi:10.1016/j.isci.2023.106384 · Cited by 5 · see citing works
Developmental cognitive neurosciecne: Functional connectomics agenda for human brain lifespan development
Tan, S., Liu, X., ZhiXiong, Y. A. N., Yang, Z., Tan, Y., Wei, G., & Zuo, X. (2015) · Chinese Science Bulletin (Chinese Version) · doi:10.1360/n972015-01146 · Cited by 4 · see citing works
Is postexercise hypotension a method‐dependent phenomenon in chronic stroke? A crossover randomized controlled trial
Fonseca, G. F., Michalski, A. C., Ferreira, A. D. S., Costa, V. A. B., Massaferri, R., Farinatti, P., & da Cunha, F. A. (2023) · Clinical Physiology and Functional Imaging · doi:10.1111/cpf.12812 · Cited by 2 · see citing works
Reliability of the ICD‐11 personality disorder severity ratings and diagnosis
Aydın‐Seyrek, T., Gandur, T., Turgut, N., Kunt, D. A., & Dereboy, F. (2024) · Personality and Mental Health · doi:10.1002/pmh.1629 · Cited by 2 · see citing works
Challenging Encounters and Within-Physician Practice Variability
Chodick, G., Goldstein, Y., Shurtz, I., & Zeltzer, D. (2022) · The Review of Economics and Statistics · doi:10.1162/rest_a_01314 · Cited by 2 · see citing works
Grandmother and grandchild reports of psychological difficulties among custodial Grandchildren: Whose view matters and why is it important?
Smith, G. C., & Hayslip, B. (2023) · Children and Youth Services Review · doi:10.1016/j.childyouth.2022.106793 · Cited by 1 · see citing works
Grand rounds in methodology: four key things to know about the reliability of measurement
Hofer, T. P. (2026) · BMJ Quality & Safety · doi:10.1136/bmjqs-2025-018924 · Cited by 1 · see citing works
Mental Health Emergency in Nigeria: The Interventionist and Preventive Roles of Clinical Psychologists and Allied Professionals
Ehusani, G. (2025) · International Journal of Research and Innovation in Social Science · doi:10.47772/ijriss.2025.91100313 · Cited by 1 · see citing works
relationship between specific psychiatric disorders and type of crimes committed in a forensic psychiatry population
de Jesús, M., Martins, V. E., Faria, I., & Quartilho, M. J. (2022) · Debates em Psiquiatria · doi:10.25118/2763-9037.2022.v12.405

Authors: Kraemer, H. C.
Affiliations: University of Pittsburgh (US)
Corresponding author: Kraemer, Helena Chmura
ORCID: Kraemer, Helena Chmura
Venue: Annual Review of Clinical Psychology
Publisher: Annual Reviews
Published: 2014-03-28
Volume: 10
Issue: 1
Pages: 111-130
ISSN: 1548-5943 (ISSN-L)
1548-5951
DOI: 10.1146/annurev-clinpsy-032813-153739
OpenAlex: W2146580987
Cited by: 39
References: 49
Citations by year: 2026: 1 · 2025: 1 · 2024: 3 · 2023: 5 · 2022: 3 · 2021: 5 · 2020: 1 · 2019: 5 · 2018: 4 · 2017: 2 · 2016: 4 · 2015: 5
FWCI: 1.01
Type: journal article
Language: en
Open access: closed
Locations: Annual Review of Clinical Psychology (publishedVersion)
PubMed (publishedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Keywords: diagnostic reliability, validity, diagnosis, reliability studies, reliability coefficients, clinical decision-making, psychodiagnosis
MeSH terms: Mental Disorders, Humans, Reproducibility of Results, Diagnostic and Statistical Manual of Mental Disorders

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

The Reliability of Clinical Diagnoses: State of the Art

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks