The meta-analysis of clinical judgment project: fifty-six years of accumulated research on clinical versus statistical prediction

Ægisdóttir, Stefanía; White, Michael J.; Spengler, Paul M.; Maugherman, Alan S.; Anderson, Linda A.; Cook, Robert S.; Nichols, Cassandra N.; Lampropoulos, Georgios K.; Walker, Blain S.; Cohen, Genna; Rush, Jeffrey D.

doi:10.1177/0011000005285875

Cited by 737Clinical Reasoning and Diagnostic SkillsPsychotherapy Techniques and ApplicationsPsychology Research and Bibliometrics

The meta-analysis of clinical judgment project: fifty-six years of accumulated research on clinical versus statistical prediction

Ægisdóttir, Stefanía, White, Michael J., Spengler, Paul M., Maugherman, Alan S., Anderson, Linda A., Cook, Robert S., Nichols, Cassandra N., Lampropoulos, Georgios K. +3 more·2006·The Counseling Psychologist

DOI 10.1177/0011000005285875

Open this paper in CoChat — free account

Abstract

Clinical predictions made by mental health practitioners are compared with those using statistical approaches. Sixty-seven studies were identified from a comprehensive search of 56 years of research; 92 effect sizes were derived from these studies. The overall effect of clinical versus statistical prediction showed a somewhat greater accuracy for statistical methods. The most stringent sample of studies, from which 48 effect sizes were extracted, indicated a 13% increase in accuracy using statistical versus clinical methods. Several variables influenced this overall effect. Clinical and statistical prediction accuracy varied by type of prediction, the setting in which predictor data were gathered, the type of statistical formula used, and the amount of information available to the clinicians and the formulas. Recommendations are provided about when and under what conditions counseling psychologists might use statistical formulas as well as when they can rely on clinical methods. Implications for clinical judgment research and training are discussed.

Integrity signals

Registry

Crossref · 10.1177/0011000005285875

Cites

153 of 153 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Statistical models have long been known to be superior to unstructured clinical judgments in predicting a wide range of clinical outcomes, although varying over settings and decisions.
“Statistical models have long been known to be superior to unstructured clinical judgments in predicting a wide range of clinical outcomes (107), although varying over settings and decisions (108).”
From Using Administrative Data to Predict Suicide After Psychiatric Hospitalization in the Veterans Health Administration System (2020) · cites this paper as [108]

Diagnostic and Statistical Manual of Mental Disorders
Williams, J. B. W., & First, M. B. (2013) · Oxford University Press eBooks · doi:10.1093/acrefore/9780199975839.013.104 · Cited by 109,841 · see citing works
Diagnostic and Statistical Manual of Mental Disorders.
Widiger, T. A. (2000) · Oxford University Press eBooks · doi:10.1037/10518-009 · Cited by 19,767 · see citing works
Book Reviews: Statistical Methods for Meta-Analysis Larry V. Hedges and Ingram Olkin New York: Academic Press, 1985. xxii + 369 pp
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,347 · see citing works
The Epidemiology of Major Depressive Disorder: Results From the National Comorbidity Survey Replication (NCS-R)
Kessler, R. C., Berglund, P., Demler, O., Jin, R., Koretz, D., Merikangas, K. R., Rush, A. J., Walters, E. E., & Wang, P. S. (2003) · JAMA · doi:10.1001/jama.289.23.3095 · Cited by 7,988 · see citing works
On the psychology of prediction.
Kahneman, D., & Tversky, A. (1973) · Psychological Review · doi:10.1037/h0034747 · Cited by 6,202 · see citing works
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings
Schmidt, F. L., & Hunter, J. E. (2015) · doi:10.4135/9781483398105 · Cited by 5,542 · see citing works
Statistical Methods for Meta-Analysis.
Freeman, P. R., Hedges, L. V., & Olkin, I. (1986) · Biometrics · doi:10.2307/2531069 · Cited by 4,832 · see citing works
Introduction to Classical and Modern Test Theory
Crocker, L. M., & Algina, J. J. (1986) · Cited by 4,803 · see citing works
Meta-Analytic Procedures for Social Research
Rosenthal, R. (1991) · doi:10.4135/9781412984997 · Cited by 4,635 · see citing works
Meta-Analytic Procedures for Social Research.
Metcalfe, A., & Rosenthal, R. (1994) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348600 · Cited by 4,249 · see citing works
Human Inference: Strategies and Shortcomings of Social Judgment.
Cherniak, C., Nisbett, R. E., & Ross, L. D. (1983) · The Philosophical Review · doi:10.2307/2184495 · Cited by 4,196 · see citing works
A leisurely look at the bootstrap, the jackknife, and cross-validation
Efron, B., & Gong, G. (1983) · The American Statistician · doi:10.1080/00031305.1983.10483087 · Cited by 3,229 · see citing works
Book Review Nisbett, R. , & Ross, L.Human inference: Strategies and shortcomings of social judgment.Englewood Cliffs, N.J.: Prentice-Hall, 1980.
Miller, G. A., & Cantor, N. E. (1982) · Social Cognition · doi:10.1521/soco.1982.1.1.83 · Cited by 2,862 · see citing works
Methods of Meta-Analysis: Correcting Error and Bias in Research Findings.
Raudenbush, S. W., Hunter, J. E., & Schmidt, F. L. (1991) · Journal of the American Statistical Association · doi:10.2307/2289738 · Cited by 2,363 · see citing works
“Clinical versus Actuarial Judgment.”
Dawes, R. M., Faust, D., & Meehl, P. E. (1989) · Science · doi:10.1126/science.2648573 · Cited by 2,298 · see citing works
Clinical versus statistical prediction: A theoretical analysis and a review of the evidence.
Meehl, P. E. (1954) · University of Minnesota Press eBooks · doi:10.1037/11281-000 · Cited by 2,187 · see citing works
EXPERT AND EXCEPTIONAL PERFORMANCE: Evidence of Maximal Adaptation to Task Constraints
Ericsson, K. A., & Lehmann, A. C. (1996) · Annual Review of Psychology · doi:10.1146/annurev.psych.47.1.273 · Cited by 2,139 · see citing works
Linear models in decision making.
Dawes, R. M., & Corrigan, B. (1974) · Psychological Bulletin · doi:10.1037/h0037613 · Cited by 1,898 · see citing works
Clinical versus mechanical prediction: a meta-analysis
Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000) · Psychological Assessment · doi:10.1037/1040-3590.12.1.19 · Cited by 1,640 · see citing works
Human Inference: Strategies and Shortcomings of Social Judgment.
Weiner, H. J., Nisbett, R. E., & Ross, L. D. (1981) · Journal of the American Statistical Association · doi:10.2307/2287875 · Cited by 1,468 · see citing works
Comparative efficiency of informal (subjective, impressionistic) and formal (mechanical, algorithmic) prediction procedures: the clinical -statistical controversy
Grove, W. M., & Meehl, P. E. (1996) · Psychology, Public Policy, and Law · doi:10.1037/1076-8971.2.2.293 · Cited by 1,457 · see citing works
Journal of Experimental Research in Personality
(1965) · Nature · doi:10.1038/2061296c0 · Cited by 1,426 · see citing works
Synthesizing Research: A Guide for Literature Reviews
Cooper, H. (1998) · Cited by 1,285 · see citing works
A simple, general purpose display of magnitude of experimental effect.
Rosenthal, R., & Rubin, D. B. (1982) · Journal of Educational Psychology · doi:10.1037//0022-0663.74.2.166 · Cited by 1,233 · see citing works
Violent offenders: Appraising and managing risk.
Quinsey, V. L., Harris, G. T., Rice, M. E., & Cormier, C. A. (1998) · American Psychological Association eBooks · doi:10.1037/10304-000 · Cited by 1,223 · see citing works
Violent Offenders: Appraising and Managing Risk
Heilbrun, K. (2000) · Psychiatric Services · doi:10.1176/appi.ps.51.3.395 · Cited by 1,210 · see citing works
Journal of Consulting and Clinical Psychology
Rice, T. K., & Wallace, D. (2008) · Wiley eBooks · doi:10.1002/9780470373699.speced1159 · Cited by 1,192 · see citing works
Consequences of prejudice against the null hypothesis.
Greenwald, A. G. (1975) · Psychological Bulletin · doi:10.1037/h0076157 · Cited by 1,022 · cited 1× · see citing works
“The direction of publication bias is that studies reporting significant results are more often published than studies presenting nonsignificant results (e.g., Greenwald, 1975; Rosnow & Rosenthal, 1989).”
Result
Statistical Procedures and the Justification of Knowledge in Psychological Science
Rosnow, R., & Rosenthal, R. (1989) · American Psychologist · doi:10.1037/0003-066x.44.10.1276 · Cited by 951 · see citing works
A simple, general purpose display of magnitude of experimental effect.
Rosenthal, R., & Rubin, D. B. (1982) · Journal of Educational Psychology · doi:10.1037/0022-0663.74.2.166 · Cited by 920 · see citing works
Human Judgment <i>and</i> Social Policy: Irreducible Uncertainty, Inevitable Error, Unavoidable Injustice
Hammond, K. R. (1996) · doi:10.1093/oso/9780195097344.001.0001 · Cited by 851 · see citing works
Costs and benefits of judgment errors: Implications for debiasing.
Arkes, H. R. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.110.3.486 · Cited by 764 · see citing works
House of cards: psychology and psychotherapy built on myth
(1994) · Choice Reviews Online · doi:10.5860/choice.32-1228 · Cited by 754 · see citing works
Human judgment and social policy: irreducible uncertainty, inevitable error, unavoidable injustice
(1997) · Choice Reviews Online · doi:10.5860/choice.34-4545 · Cited by 717 · see citing works
Statistical procedures and the justification of knowledge in psychological science.
Rosnow, R. L., & Rosenthal, R. (1992) · American Psychological Association eBooks · doi:10.1037/10109-027 · Cited by 653 · see citing works
The MMPI-2 : an interpretive manual
Greene, R. L. (1980) · Allyn and Bacon eBooks · Cited by 628 · see citing works
Man versus model of man: A rationale, plus some evidence, for a method of improving on clinical inferences.
Goldberg, L. R. (1970) · Psychological Bulletin · doi:10.1037/h0029230 · Cited by 622 · see citing works
Patient-focused research: Using patient outcome data to enhance treatment effects.
Lambert, M. J., Hansen, N. B., & Finch, A. E. (2001) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.69.2.159 · Cited by 602 · see citing works
Measurement and prediction, clinical and statistical.
Sawyer, J. (1966) · Psychological Bulletin · doi:10.1037/h0023624 · Cited by 594 · see citing works
The Annual Meeting of the American Psychological Association
(1936) · Science · doi:10.1126/science.84.2174.197 · Cited by 573 · see citing works
Evaluation of psychotherapy: Efficacy, effectiveness, and patient progress.
Howard, K. I., Moras, K., Brill, P. L., Martinovich, Z., & Lutz, W. (1996) · American Psychologist · doi:10.1037/0003-066x.51.10.1059 · Cited by 573 · see citing works
Measurement error in psychological research: Lessons from 26 research scenarios.
Schmidt, F. L., & Hunter, J. E. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.199 · Cited by 567 · see citing works
Proceedings of the 76th Annual Convention of the American Psychological Association
(1968) · PsycEXTRA Dataset · doi:10.1037/e473742008-359 · Cited by 549 · see citing works
Preliminary Report: Findings from the Aspirin Component of the Ongoing Physicians' Health Study
The Steering Committee of the Physicians' Health Study Research Group (1988) · New England Journal of Medicine · doi:10.1056/nejm198801283180431 · Cited by 522 · see citing works
DSTAT : software for the meta-analytic review of research literatures
Johnson, B. T. (1989) · Medical Entomology and Zoology · Cited by 494 · see citing works
Studying the clinician: judgment research and psychological assessment
Garb, H. N. (1999) · Choice Reviews Online · doi:10.5860/choice.36-3045 · Cited by 444 · see citing works
Why we still use our heads instead of formulas: Toward an integrative approach.
Kleinmuntz, B. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.107.3.296 · Cited by 437 · see citing works
How hard is hard science, how soft is soft science? The empirical cumulativeness of research.
Hedges, L. V. (1987) · American Psychologist · doi:10.1037/0003-066x.42.5.443 · Cited by 435 · see citing works
Causes and effects of my disturbing little book
Meehl, P. E. (1986) · Journal of Personality Assessment · doi:10.1207/s15327752jpa5003_6 · Cited by 410 · see citing works
A Brief Actuarial Assessment for the Prediction of Wife Assault Recidivism: The Ontario Domestic Assault Risk Assessment.
Hilton, N. Z., Harris, G. T., Rice, M. E., Lang, C., Cormier, C. A., & Lines, K. J. (2004) · Psychological Assessment · doi:10.1037/1040-3590.16.3.267 · Cited by 378 · see citing works
Studying the clinician: Judgment research and psychological assessment.
Garb, H. N. (1998) · American Psychological Association eBooks · doi:10.1037/10299-000 · Cited by 365 · see citing works
Diagnosticians vs. diagnostic signs: The diagnosis of psychosis vs. neurosis from the MMPI.
Goldberg, L. R. (1965) · The Psychological Monographs · doi:10.1037/h0093885 · Cited by 342 · see citing works
Impediments to accurate clinical judgment and possible ways to minimize their impact.
Arkes, H. R. (1981) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.49.3.323 · Cited by 341 · see citing works
Untitled work
(1998) · Cited by 326 · see citing works
When clinical description becomes statistical prediction.
Westen, D., & Weinberger, J. L. (2004) · American Psychologist · doi:10.1037/0003-066x.59.7.595 · Cited by 292 · see citing works
Clinical judgment, clinical training, and professional experience.
Garb, H. N. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.105.3.387 · Cited by 273 · see citing works
Diseases of the Nervous System.
(1975) · Annals of Internal Medicine · doi:10.7326/0003-4819-83-6-925_2 · Cited by 270 · see citing works
Impediments to accurate clinical judgment and possible ways to minimize their impact.
Arkes, H. R. (1981) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.49.3.323 · Cited by 246 · see citing works
The relationship of clinical experience and training methods to several criteria of clinical prediction.
Oskamp, S. (1962) · The Psychological Monographs · doi:10.1037/h0093849 · Cited by 243 · see citing works
Clinical versus actuarial predictions of violence in patients with mental illnesses.
Gardner, W. P., Lidz, C. W., Mulvey, E. P., & Shaw, E. C. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.64.3.602 · Cited by 197 · see citing works
Counseling theory and practice.
Pepinsky, H. B., & Pepinsky, P. N. (1954) · doi:10.1037/10631-000 · Cited by 196 · see citing works
Assessing outcome in clinical practice
Ogles, B. M., Lambert, M. J., & Masters, K. S. (1996) · Medical Entomology and Zoology · Cited by 189 · see citing works
Biological Psychiatry
Malitz, S. (1974) · JAMA · doi:10.1001/jama.1974.03230200062024 · Cited by 188 · see citing works
A comparison of clinicians with five statistical methods of identifying psychotic MMPI profiles.
Meehl, P. E. (1959) · Journal of Counseling Psychology · doi:10.1037/h0049190 · Cited by 186 · see citing works
The MMPI-2: An interpretive manual, 2nd ed.
Greene, R. L. (2000) · Cited by 183 · see citing works
Detecting Fake-Good and Fake-Bad MMPI-2 Profiles
Graham, J. R., Watts, D., & Timbrook, R. E. (1991) · Journal of Personality Assessment · doi:10.1207/s15327752jpa5702_6 · Cited by 178 · see citing works
Quantitative synthesis of social psychological research.
Johnson, B. T., & Eagly, A. H. (2000) · OpenCommons - UConn (University of Connecticut) · Cited by 168 · see citing works
Clinical and statistical prediction: A reformulation and some new data.
Holt, R. R. (1958) · Journal of Abnormal & Social Psychology · doi:10.1037/h0041045 · Cited by 168 · see citing works
Clinical neuropsychology
Purser, H. R. M. (1982) · doi:10.1007/978-1-349-16746-3_13 · Cited by 164 · see citing works
Methodological Problems in Identifying Efficacious Psychotherapies
Wampold, B. E. (1997) · Psychotherapy Research · doi:10.1080/10503309712331331853 · Cited by 149 · see citing works
The local clinical scientist: A bridge between science and practice.
Stricker, G., & Trierweiler, S. J. (1995) · American Psychologist · doi:10.1037/0003-066x.50.12.995 · Cited by 147 · see citing works
Clinical versus actuarial predictions of violence in patients with mental illnesses.
Gardner, W. P., Lidz, C. W., Mulvey, E. P., et al. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.64.3.602 · Cited by 139 · see citing works
Clinical and statistical prediction
Wiggins, J. S. (1981) · Clinical Psychology Review · doi:10.1016/0272-7358(81)90015-5 · Cited by 134 · see citing works
Research on human judgment and its application to clinical practice.
Faust, D. (1986) · Professional Psychology Research and Practice · doi:10.1037/0735-7028.17.5.420 · Cited by 134 · see citing works
A Contribution to the Study of Actuarial and Individual Methods of Prediction
Sarbin, T. R. (1943) · American Journal of Sociology · doi:10.1086/219248 · Cited by 117 · see citing works
Predicting Length of Hospitalization of Mental Patients
Dunham, H. W., & Meltzer, B. N. (1946) · American Journal of Sociology · doi:10.1086/219959 · Cited by 101 · see citing works
A Scientist-Practitioner Model of Psychological Assessment
Spengler, P. M., Strohmer, D. C., Dixon, D. N., & Shivy, V. A. (1995) · The Counseling Psychologist · doi:10.1177/0011000095233009 · Cited by 101 · see citing works
Self-citation
Yet another look at clinical and statistical prediction: Or, is clinical psychology worthwhile?
Holt, R. R. (1970) · American Psychologist · doi:10.1037/h0029481 · Cited by 100 · see citing works
A comparison of clinical and automated interpretation of the halstead-reitan battery
Heaton, R. K., Grant, I., Anthony, W. Z., & Lehman, R. A. W. (1981) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/01688638108403118 · Cited by 87 · see citing works
The detection of feigned uncoached and coached posttraumatic stress disorder with the MMPI-2 in a sample of workplace accident victims.
Bury, A. S., & Bagby, R. M. (2002) · Psychological Assessment · doi:10.1037//1040-3590.14.4.472 · Cited by 85 · see citing works
Man versus model of man revisited: The forecasting of graduate school success.
Wiggins, N., & Kolen, E. S. (1971) · Journal of Personality and Social Psychology · doi:10.1037/h0031147 · Cited by 82 · see citing works
Some test correlates of response to psychotherapy.
Barron, F. (1953) · Journal of Consulting Psychology · doi:10.1037/h0062081 · Cited by 76 · see citing works
Diagnostic classification through content analysis of patients' speech
Oxman, T. E., Rosenberg, S. D., Schnurr, P. P., & TUCKER, G. J. (1988) · American Journal of Psychiatry · doi:10.1176/ajp.145.4.464 · Cited by 76 · see citing works
The prediction of success in the VA training program in clinical psychology.
Kelly, E. L., & Fiske, D. W. (1950) · American Psychologist · doi:10.1037/h0062436 · Cited by 74 · see citing works
House of Cards: Psychology and Psychotherapy Built on Myth
Lewis, T. H. (1994) · JAMA · doi:10.1001/jama.1994.03520180095045 · Cited by 74 · see citing works
The detection of feigned uncoached and coached posttraumatic stress disorder with the MMPI-2 in a sample of workplace accident victims.
Bury, A. S., & Bagby, R. M. (2002) · Psychological Assessment · doi:10.1037/1040-3590.14.4.472 · Cited by 73 · see citing works
Statistical versus clinical lie detection.
Szucko, J. J., & Kleinmuntz, B. (1981) · American Psychologist · doi:10.1037/0003-066x.36.5.488 · Cited by 72 · see citing works
Use of the MMPI to identify malingering and exaggeration of psychiatric symptomatology in male prison inmates.
Walters, G. D., White, T. W., & Greene, R. L. (1988) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.56.1.111 · Cited by 70 · see citing works
Criminal behavior as a function of clinical and actuarial variables in a sexual offender population.
Hall, G. C. N. (1988) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.56.5.773 · Cited by 69 · see citing works
The professional counselor
Blocher, D. H. (1987) · Medical Entomology and Zoology · Cited by 63 · see citing works
Psychiatrists' judgments of dangerousness in patients on an acute care unit
Werner, P. D., Rose, T. L., Yesavage, J. A., & Seeman, K. (1984) · American Journal of Psychiatry · doi:10.1176/ajp.141.2.263 · Cited by 61 · see citing works
THE PREDICTION OF SUICIDE IN SCHIZOPHRENIA
Shaffer, J. W., Perlin, S., Schmidt, C. W., & Stephens, J. H. (1974) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-197411000-00005 · Cited by 61 · see citing works
Criminal behavior as a function of clinical and actuarial variables in a sexual offender population.
Hall, G. C. N. (1988) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.56.5.773 · Cited by 54 · see citing works
Expertise in Counseling Psychology: A Concept in Search of Support
Lichtenberg, J. W. (1997) · Educational Psychology Review · doi:10.1023/a:1024783107643 · Cited by 50 · see citing works
Actuarial versus Clinical Prediction in Psychopathology
Sines, J. O. (1970) · The British Journal of Psychiatry · doi:10.1192/bjp.116.531.129 · Cited by 49 · see citing works
Relative accuracy of actuarial prediction, experienced clinicians, and graduate students in a clinical judgment task.
Grebstein, L. C. (1963) · Journal of Consulting Psychology · doi:10.1037/h0040580 · Cited by 48 · see citing works
Effect of experience and amount of information on identification of cerebral impairment.
Goldstein, S. G., Deysach, R. E., & Kleinknecht, R. A. (1973) · Journal of Consulting and Clinical Psychology · doi:10.1037/h0035614 · Cited by 45 · see citing works
Effects of teaching a cognitive strategy on counselor trainee internal dialogue and clinical hypothesis formulation.
Kurpius, D. J., Benjamin, D., & Morran, D. K. (1985) · Journal of Counseling Psychology · doi:10.1037/0022-0167.32.2.263 · Cited by 45 · see citing works
PSYCHOLOGICAL TESTING
(1947) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.1947.tb63710.x · Cited by 43 · see citing works
The accuracy of automated and clinical detection of brain damage and lateralization in neuropsychology
Russell, E. W. (1995) · Neuropsychology Review · doi:10.1007/bf02214929 · Cited by 42 · see citing works
A probabilistic system for identifying suicide attemptors
Gustafson, D. H., Greist, J. H., Stauss, F. F., Erdman, H. P., & LAUGHREN, T. P. (1977) · Computers and Biomedical Research · doi:10.1016/0010-4809(77)90026-x · Cited by 41 · see citing works
Assessment of substance abuse in psychotherapy clients: The effectiveness of the MMPI-2 substance abuse scales.
Rouse, S. V., Butcher, J. N., & Miller, K. B. (1999) · Psychological Assessment · doi:10.1037/1040-3590.11.1.101 · Cited by 41 · see citing works
The local clinical scientist: A bridge between science and practice.
Stricker, G., & Trierweiler, S. J. (2006) · Training and Education in Professional Psychology · doi:10.1037/1931-3918.s.1.37 · Cited by 39 · see citing works
The study of clinical judgment: an ecological approach
Rock, D. L., Bransford, J. D., Maisto, S. A., et al. (1987) · Clinical Psychology Review · doi:10.1016/0272-7358(87)90011-0 · Cited by 38 · see citing works
Configurational analysis of MMPI profiles of psychiatric groups.
Taulbee, E. S., & Sisson, B. D. (1957) · Journal of Consulting Psychology · doi:10.1037/h0044567 · Cited by 37 · see citing works
The Mathematics of Risk Classification: Changing Data into Valid Instruments for Juvenile Courts. NCJ 209158.
Gottfredson, D. M., & Snyder, H. N. (2005) · Cited by 34 · see citing works
Comparison and combination of clinical and statistical predictions of recidivism among adult offenders.
Holland, T. R., Holt, N., Levi, M., et al. (1983) · Journal of Applied Psychology · doi:10.1037//0021-9010.68.2.203 · Cited by 32 · see citing works
Biodata in professional entry‐level selection: Statistical scoring of common format applications
Harvey‐Cook, J., & Taffler, R. (2000) · Journal of Occupational and Organizational Psychology · doi:10.1348/096317900166903 · Cited by 32 · see citing works
Actuarial, naive clinical, and sophisticated clinical prediction of pathology from figure drawings.
Stricker, G. (1967) · Journal of Consulting Psychology · doi:10.1037/h0025021 · Cited by 31 · see citing works
Seer versus sign.
Lindzey, G. E. (1965) · Cited by 31 · see citing works
Using the MMPI-2 to detect substance abuse in an outpatient mental health setting.
Stein, L. A. R., Graham, J. R., Ben‐Porath, Y. S., & McNulty, J. L. (1999) · Psychological Assessment · doi:10.1037/1040-3590.11.1.94 · Cited by 30 · see citing works
The Bender Gestalt: Use of clinical judgment versus recall scores in prediction of Huntington's disease.
Lyle, O. E., & Quast, W. (1976) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.44.2.229 · Cited by 29 · see citing works
The Ethics of Using or Not Using Statistical Prediction Rules in Psychological Practice and Related Consulting Activities
Dawes, R. M. (2002) · Philosophy of Science · doi:10.1086/341844 · Cited by 29 · see citing works
The relation of the children's manifest anxiety scale to the concept of anxiety as used in the clinic.
Wirt, R. D., & Broen, W. E. (1956) · Journal of Consulting Psychology · doi:10.1037/h0041678 · Cited by 27 · see citing works
Statistical versus clinical prediction: Length of neuropsychiatric hospital stay.
Johnston, R., & McNeal, B. F. (1967) · Journal of Abnormal Psychology · doi:10.1037/h0020105 · Cited by 25 · see citing works
Toward a second generation of statistical prediction rules in psychodiagnosis and personality assessment
Garb, H. N. (1994) · Computers in Human Behavior · doi:10.1016/0747-5632(94)90063-9 · Cited by 25 · see citing works
(Hu)man versus mean revisited: MMPI group data and psychiatric diagnosis.
Zalewski, C. E., & Gottesman, I. I. (1991) · Journal of Abnormal Psychology · doi:10.1037//0021-843x.100.4.562 · Cited by 23 · see citing works
Reliability and validity of standardized case management simulations.
Berven, N. L. (1985) · Journal of Counseling Psychology · doi:10.1037/0022-0167.32.3.397 · Cited by 23 · see citing works
Evaluation of clinical problem-solving skills through standardized case-management simulations.
Berven, N. L., & Scofield, M. E. (1980) · Journal of Counseling Psychology · doi:10.1037/0022-0167.27.2.199 · Cited by 23 · see citing works
Empirically Supported Treatments (ESTs) in Perspective
Waehler, C. A., Kalodner, C. R., Wampold, B. E., & Lichtenberg, J. W. (2000) · The Counseling Psychologist · doi:10.1177/0011000000285004 · Cited by 23 · see citing works
Comparison and combination of clinical and statistical predictions of recidivism among adult offenders.
Holland, T. R., Holt, N., Levi, M., et al. (1983) · Journal of Applied Psychology · doi:10.1037/0021-9010.68.2.203 · Cited by 23 · see citing works
Sign and seer: Another example.
Kleinmuntz, B. (1967) · Journal of Abnormal Psychology · doi:10.1037/h0024433 · Cited by 22 · see citing works
MMPI in actuarial diagnosis of psychological disturbance among college males.
Cooke, J. K. (1967) · Journal of Counseling Psychology · doi:10.1037/h0024878 · Cited by 19 · see citing works
Holistic Thinking is not the Whole Story
Faust, D. (2003) · Assessment · doi:10.1177/1073191103259534 · Cited by 19 · see citing works
(Hu)man versus mean revisited: MMPI group data and psychiatric diagnosis.
Zalewski, C. E., & Gottesman, I. I. (1991) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.100.4.562 · Cited by 19 · see citing works
Counselor variability in making accurate predictions.
Watley, D. J. (1966) · Journal of Counseling Psychology · doi:10.1037/h0023066 · Cited by 18 · see citing works
A Validation of the Glueck Social Prediction Scale for Proneness to Delinquency
Thompson, R. E. (1952) · The Journal of Criminal Law Criminology and Police Science · doi:10.2307/1139334 · Cited by 17 · see citing works
Multiple MMPI Profiles of Suicidal Persons
Devries, A. G., & Shneidman, E. S. (1967) · Psychological Reports · doi:10.2466/pr0.1967.21.2.401 · Cited by 17 · see citing works
Psychometric Study of the Clinical Treatment Planning Simulations (CTPS) for Assessing Clinical Judgment.
Falvey, J. E., & Hebert, D. J. (1992) · Journal of Mental Health Counseling · Cited by 16 · see citing works
Clinical intuition and test scores as a basis for diagnosis.
Klehr, H. (1949) · Journal of Consulting Psychology · doi:10.1037/h0057291 · Cited by 15 · see citing works
Of Science, Meta-Science, and Clinical Practice: The Generalization of a Generalization to a Particular
Faust, D. (1997) · Journal of Personality Assessment · doi:10.1207/s15327752jpa6802_6 · Cited by 14 · see citing works
Behavioral Analysis of Clinical Judgment
Perez, F. I. (1976) · Perceptual and Motor Skills · doi:10.2466/pms.1976.43.3.711 · Cited by 13 · see citing works
Methodology for the comparison of clinical with actuarial predictions.
McHugh, R. B., & Apostolakos, P. C. (1959) · Psychological Bulletin · doi:10.1037/h0044142 · Cited by 13 · see citing works
Prediction of mental illness in college students on the basis of "nonpsychiatric" MMPI profiles.
Danet, B. N. (1965) · Journal of Consulting Psychology · doi:10.1037/h0022752 · Cited by 13 · see citing works
Accuracy of actuarial and clinical predictions for length of stay and unauthorized absence.
Rc, E., Altman, H., Sletten, I. W., et al. (1975) · PubMed · Cited by 13 · see citing works
Intuitive versus objective prediction from a test
Hovey, H. B., & Stauffacher, J. C. (1953) · Journal of Clinical Psychology · doi:10.1002/1097-4679(195310)9:4 · Cited by 12 · see citing works
CLINICAL VERSUS ACTUARIAL PREDICTION OF COLLEGE ACHIEVEMENT AND LEADERSHIP ACTIVITY.
Vance, F. L., & Watley, D. J. (1964) · Cited by 11 · see citing works
Multicultural Assessment and a Scientist-Practitioner Model of Psychological Assessment
Spengler, P. M. (1998) · The Counseling Psychologist · doi:10.1177/0011000098266004 · Cited by 11 · see citing works
Self-citation
The discrimination of patient-generated and randomly generated MMPIs.
Carlin, A. S., & Hewitt, P. L. (1990) · PubMed · doi:10.1080/00223891.1990.9673970 · Cited by 10 · see citing works
Counselor vs. statistical predictions of job satisfaction in engineering.
Lewis, E. C., & MacKinney, A. C. (1961) · Journal of Counseling Psychology · doi:10.1037/h0043207 · Cited by 10 · see citing works
Using a battery of tests to predict suicide in a long-term hospital: A quantitative analysis
Smith, K. (1981) · Journal of Clinical Psychology · doi:10.1002/1097-4679(198107)37:3 · Cited by 9 · see citing works
Predicted and observed outcome in followed-up functional psychoses.
Astrup, C. (1975) · PubMed · Cited by 9 · see citing works
Predictive efficiency of two multivariate statistical techniques in comparison with clinical predictions.
Alexakos, C. E. (1966) · Journal of Educational Psychology · doi:10.1037/h0023729 · Cited by 8 · see citing works
SUICIDE PREDICTION FOR PSYCHIATRIC PATIENTS: A COMPARISON OF THE MMPI AND CLINICAL JUDGMENTS
Lemerond, J. N. (1977) · e-Publications@Marquette (Marquette University) · Cited by 7 · see citing works
Diagnosis of Schizophrenia by Rorschach Patterns
Klinger, E., & Roth, I. (1965) · Journal of Projective Techniques and Personality Assessment · doi:10.1080/0091651x.1965.10120216 · Cited by 7 · see citing works
Actuarial prediction.
Wirt, R. D. (1956) · Journal of Consulting Psychology · doi:10.1037/h0040407 · Cited by 7 · see citing works
The Discrimination of Patient-Generated and Randomly Generated MMPIs
Carlin, A. S., & Hewitt, P. L. (1990) · Journal of Personality Assessment · doi:10.1207/s15327752jpa5401&2_3 · Cited by 7 · see citing works
Predicting occupational choice by clinical and statistical methods.
Webb, S. C., Hultgen, D. D., & Craddick, R. A. (1977) · Journal of Counseling Psychology · doi:10.1037//0022-0167.24.2.98 · Cited by 6 · see citing works
Predictive Validities of Clinical and Actuarial Scores of the Gesell Incomplete Man Test
Popovics, A. J. (1983) · Perceptual and Motor Skills · doi:10.2466/pms.1983.56.3.864 · Cited by 6 · see citing works
Severe somnambulism: A case study
Edmonds, C. (1967) · Journal of Clinical Psychology · doi:10.1002/1097-4679(196704)23:2 · Cited by 5 · see citing works
Predicting occupational choice by clinical and statistical methods.
Webb, S. C., Hultgen, D. D., & Craddick, R. A. (1977) · Journal of Counseling Psychology · doi:10.1037/0022-0167.24.2.98 · Cited by 4 · see citing works
Factorial structure of the aftermath of suicide instrument
Spence, N. D., Goldney, R. D., & Moffitt, P. F. (1984) · Journal of Clinical Psychology · doi:10.1002/1097-4679(198411)40:6 · Cited by 2 · see citing works
The Search for Models of Clinical Judgment: Fast, Frugal, and Friendly in Paul Meehl's Spirit
Katsikopoulos, K. V., Machery, É., Pachur, T., & Wallin, A. (2004) · HAL · Cited by 2 · see citing works

Flagged first, then by citations

Sorted by citations

Rehabilitating criminal justice policy and practice
Andrews, D. A., & Bonta, J. (2010) · Psychology, Public Policy, and Law · doi:10.1037/a0018362 · Cited by 1,223 · see citing works
Enhancing treatment outcome of patients at risk of treatment failure: Meta-analytic and mega-analytic review of a psychotherapy quality assurance system.
Shimokawa, K., Lambert, M. J., & Smart, D. W. (2010) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0019247 · Cited by 657 · see citing works
The WPA-Lancet psychiatry commission on the future of psychiatry
Bhugra, D., Tasman, A., Pathare, S., Priebe, S., Smith, S., Torous, J., Arbuckle, M. R., Langford, A., Alarcón, R. D., Chiu, H. F. K., et al. (2017) · The Lancet Psychiatry · doi:10.1016/s2215-0366(17)30333-4 · Cited by 392 · see citing works
Algorithms in practice: Comparing web journalism and criminal justice
Christin, A. (2017) · Big Data & Society · doi:10.1177/2053951717718855 · Cited by 379 · see citing works
Why many clinical psychologists are resistant to evidence-based practice: Root causes and constructive remedies
Lilienfeld, S. O., Ritschel, L. A., Lynn, S. J., Cautin, R. L., & Latzman, R. D. (2013) · Clinical Psychology Review · doi:10.1016/j.cpr.2012.09.008 · Cited by 341 · see citing works
Suicide prediction models: a critical review of recent research with recommendations for the way forward
Kessler, R. C., Bossarte, R. M., Luedtke, A., Zaslavsky, A. M., & Zubizarreta, J. R. (2020) · Molecular Psychiatry · doi:10.1038/s41380-019-0531-0 · Cited by 234 · see citing works
A Case for Humans-in-the-Loop: Decisions in the Presence of Erroneous Algorithmic Scores
De-Arteaga, M., Fogliato, R., & Chouldechova, A. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376638 · Cited by 221 · see citing works
Risk assessment and decision making in child protective services: Predictive risk modeling in context
Cuccaro-Alamin, S., Foust, R., Vaithianathan, R., & Putnam-Hornstein, E. (2017) · Children and Youth Services Review · doi:10.1016/j.childyouth.2017.06.027 · Cited by 145 · see citing works
The therapeutic role of emotion regulation and coping strategies during a stand-alone DBT Skills training program for alcohol use disorder and concurrent substance use disorders
Cavicchioli, M., Movalli, M., Vassena, G., Ramella, P., Prudenziati, F., & Maffei, C. (2019) · Addictive Behaviors · doi:10.1016/j.addbeh.2019.106035 · Cited by 94 · see citing works
Predicting domestic violence: A meta-analysis on the predictive validity of risk assessment tools
van der Put, C. E., Gubbels, J., & Assink, M. (2019) · Aggression and Violent Behavior · doi:10.1016/j.avb.2019.03.008 · Cited by 83 · see citing works
Why Machine Learning May Lead to Unfairness: Evidence from Risk Assessment for Juvenile Justice in Catalonia
Tolan, S., Miron, M., Gómez, E., & Castillo, C. (2019) · Proceedings of the Seventeenth International Conference on Artificial Intelligence and Law · doi:10.1145/3322640.3326705 · Cited by 75 · see citing works
Racist Algorithms or Systemic Problems? Risk Assessments and Racial Disparities
Vincent, G. M., & Viljoen, J. L. (2020) · Criminal Justice and Behavior · doi:10.1177/0093854820954501 · Cited by 68 · see citing works
Using Administrative Data to Predict Suicide After Psychiatric Hospitalization in the Veterans Health Administration System
Kessler, R. C., Bauer, M. S., Bishop, T. M., Demler, O. V., Dobscha, S. K., Gildea, S. M., Goulet, J. L., Karras, E., Kreyenbuhl, J., Landes, S. J., et al. (2020) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.00390 · Cited by 52 · see citing works
The Five Assessment Issues You Meet When You Go to Heaven
Groth-Marnat, G. (2009) · Journal of Personality Assessment · doi:10.1080/00223890902935662 · Cited by 38 · see citing works
Case Mis‐Conceptualization in Psychological Treatment: An Enduring Clinical Problem
Ridley, C. R., Jeffrey, C. E., & Roberson, R. B. (2017) · Journal of Clinical Psychology · doi:10.1002/jclp.22354 · Cited by 25 · see citing works
What the future holds: Machine learning to predict success in psychotherapy
Taubitz, F.-S., Büdenbender, B., & Alpers, G. W. (2022) · Behaviour Research and Therapy · doi:10.1016/j.brat.2022.104116 · Cited by 25 · see citing works
StatRec - Performance, Validation and Preservability of a Static Risk Prediction Instrument
Tollenaar, N., Wartna, B. S. J., Van der Heijden, P. G. M., & Bogaerts, S. (2016) · Bulletin of Sociological Methodology/Bulletin de Méthodologie Sociologique · doi:10.1177/0759106315615504 · Cited by 7 · see citing works

Authors: Ægisdóttir, S., White, M. J., Spengler, P. M., Maugherman, A. S., Anderson, L. A., Cook, R. S., Nichols, C. N., Lampropoulos, G. K., Walker, B. S., Cohen, G., & Rush, J. D.
Affiliations: Ball State University (US)
Oregon State University (US)
Washington State University (US)
Tripler Army Medical Center (US)
ORCID: Spengler, Paul M.
Lampropoulos, Georgios K.
Venue: The Counseling Psychologist
Publisher: SAGE Publications
Published: 2006-05
Volume: 34
Issue: 3
Pages: 341-382
ISSN: 0011-0000 (ISSN-L)
1552-3861
DOI: 10.1177/0011000005285875
OpenAlex: W2150412137
Cited by: 737
References: 153
Citations by year: 2026: 18 · 2025: 35 · 2024: 32 · 2023: 50 · 2022: 42 · 2021: 60 · 2020: 84 · 2019: 38 · 2018: 39 · 2017: 53 · 2016: 47 · 2015: 45 · 2014: 38 · 2013: 34 · 2012: 29
FWCI: 1.93
Type: journal article
Language: en
Open access: closed
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: The Counseling Psychologist (publishedVersion)
Topics: Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Keywords: clinical judgment, statistical forecasting, prediction accuracy, counseling psychology, effect size, mental health professionals, prediction formula

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

The meta-analysis of clinical judgment project: fifty-six years of accumulated research on clinical versus statistical prediction

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks