Cited by 209Functional Brain Connectivity StudiesStatistical Methods in EpidemiologyArtificial Intelligence in Healthcare

When Optimism Hurts: Inflated Predictions in Psychiatric Neuroimaging

Whelan, Robert, Garavan, Hugh·2014·Biological Psychiatry

Abstract

The ability to predict outcomes from neuroimaging data has the potential to answer important clinical questions such as which depressed patients will respond to treatment, which abstinent drug users will relapse, or which patients will convert to dementia. However, many prediction analyses require methods and techniques, not typically required in neuroimaging, to accurately assess a model's predictive ability. Regression models will tend to fit to the idiosyncratic characteristics of a particular sample and consequently will perform worse on unseen data. Failure to account for this inherent optimism is especially pernicious when the number of possible predictors is high relative to the number of participants, a common scenario in psychiatric neuroimaging. We show via simulated data that models can appear predictive even when data and outcomes are random, and we note examples of optimistic prediction in the literature. We provide some recommendations for assessment of model performance.

Integrity signals

Registry

Crossref · 10.1016/j.biopsych.2013.05.014

Cites

40 of 40 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Whelan, R., & Garavan, H.
Affiliations
University of Vermont (US)
Corresponding author
Garavan, Hugh
Venue
Biological Psychiatry
Publisher
Elsevier BV
Published
2014-05
Volume
75
Issue
9
Pages
746-748
ISSN
0006-3223 (ISSN-L)
OpenAlex
W1979450493
Cited by
209
References
40
Citations by year
2026: 13 · 2025: 12 · 2024: 11 · 2023: 17 · 2022: 24 · 2021: 22 · 2020: 17 · 2019: 22 · 2018: 17 · 2017: 15 · 2016: 20 · 2015: 12 · 2014: 6 · 2013: 1
FWCI
7.09
Type
journal article
Language
en
Open access
closed
Locations
  • Biological Psychiatry (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
optimism bias, overfitting, model evaluation, psychiatric neuroimaging, outcome prediction, high-dimensional predictors, regression models, depression treatment response, recurrence prediction, conversion to dementia
MeSH terms
Brain, Computer Simulation, Humans, Regression Analysis, ROC Curve, Signal Processing, Computer-Assisted, Neuroimaging

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review