When Optimism Hurts: Inflated Predictions in Psychiatric Neuroimaging
Abstract
The ability to predict outcomes from neuroimaging data has the potential to answer important clinical questions such as which depressed patients will respond to treatment, which abstinent drug users will relapse, or which patients will convert to dementia. However, many prediction analyses require methods and techniques, not typically required in neuroimaging, to accurately assess a model's predictive ability. Regression models will tend to fit to the idiosyncratic characteristics of a particular sample and consequently will perform worse on unseen data. Failure to account for this inherent optimism is especially pernicious when the number of possible predictors is high relative to the number of participants, a common scenario in psychiatric neuroimaging. We show via simulated data that models can appear predictive even when data and outcomes are random, and we note examples of optimistic prediction in the literature. We provide some recommendations for assessment of model performance.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Non-nested feature selection using random data inflates performance.
“Whelan and Garavan64 previously demonstrated inflated performance via non-nested feature selection using random data, though we think it is also useful to demonstrate this with neuroimaging data.”
From Data leakage inflates prediction performance in connectome-based machine learning models (2024) · cites this paper as [64]
Cites
Regression shrinkage and selection via the lasso
Tibshirani, R. (1996) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1996.tb02080.x · Cited by 51,482 · see citing works
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500 · see citing works
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1993) · doi:10.1007/978-1-4899-4541-9 · Cited by 21,230 · see citing works
Regularization and Variable Selection Via the Elastic Net
Zou, H., & Hastie, T. (2005) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2005.00503.x · Cited by 20,957 · see citing works
A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection
Kohavi, R. (1995) · Zenodo · doi:10.5281/zenodo.19712698 · Cited by 10,909 · see citing works
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655 · see citing works
Ridge regression: Biased estimation for nonorthogonal problems
Hoerl, A. E., & Kennard, R. W. (1970) · Technometrics · doi:10.1080/00401706.1970.10488634 · Cited by 9,648 · see citing works
Ridge Regression: Biased Estimation for Nonorthogonal Problems
Hoerl, A. E., & Kennard, R. W. (2000) · Technometrics · doi:10.2307/1271436 · Cited by 6,553 · see citing works
Relaxing the Rule of Ten Events per Variable in Logistic and Cox Regression
Vittinghoff, E., & McCulloch, C. E. (2007) · American Journal of Epidemiology · doi:10.1093/aje/kwk052 · Cited by 4,359 · see citing works
Calculation of signal detection theory measures
Harold Stanislaw, A., & Todorov, N. (1999) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03207704 · Cited by 3,087 · see citing works
Improvements on Cross-Validation: The .632+ Bootstrap Method
Efron, B., & Tibshirani, R. J. (1997) · Journal of the American Statistical Association · doi:10.1080/01621459.1997.10474007 · Cited by 1,638 · see citing works
Selection bias in gene extraction on the basis of microarray gene-expression data
Ambroise, C., & McLachlan, G. J. (2002) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.102102699 · Cited by 1,487 · see citing works
Bayesian Statistics: An Introduction
Ziegel, E. R., & Lee, P. M. (1998) · Technometrics · doi:10.2307/1270551 · Cited by 1,191 · see citing works
Improvements on Cross-Validation: The .632+ Bootstrap Method
Efron, B., & Tibshirani, R. J. (1997) · Journal of the American Statistical Association · doi:10.2307/2965703 · Cited by 723 · see citing works
Devanand, D. P., Liu, X., Tabert, M. H., Pradhaban, G., Cuasay, K., Bell, K. L., de Leon, M. J., Doty, R. L., Stern, Y., & Pelton, G. H. (2008) · Biological Psychiatry · doi:10.1016/j.biopsych.2008.06.020 · Cited by 396 · see citing works
Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence - IJCAI-95
Bundy, A., & Lombart, V. (1995) · Cited by 361 · see citing works
Gomar, J. J. (2011) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.96 · Cited by 340 · see citing works
Evans, D. M., Visscher, P. M., & Wray, N. R. (2009) · Human Molecular Genetics · doi:10.1093/hmg/ddp295 · Cited by 297 · see citing works
Prichep, L. S., John, E. R., Ferris, S. H., Rausch, L., Fang, Z., Cancro, R., Torossian, C., & Reisberg, B. (2006) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2005.07.021 · Cited by 278 · see citing works
Bayesian statistics: an introduction
(1990) · Choice Reviews Online · doi:10.5860/choice.27-3925 · Cited by 278 · see citing works
Neuroanatomical Abnormalities That Predate the Onset of Psychosis
Mechelli, A., Riecher‐Rössler, A., Meisenzahl, E. M., Tognin, S., Wood, S. J., Borgwardt, S., Koutsouleris, N., Yung, A. R., Stone, J. M., Phillips, L., et al. (2011) · Archives of General Psychiatry · doi:10.1001/archgenpsychiatry.2011.42 · Cited by 240 · see citing works
Rando, K., Hong, K.-I., Bhagwagar, Z., Li, C.-S. R., Bergquist, K., Guarnaccia, J., & Sinha, R. (2011) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2010.10020233 · Cited by 189 · see citing works
Moons, K. G. M., Donders, A. R. T., Steyerberg, E. W., & Harrell, F. E. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.01.020 · Cited by 175 · see citing works
Cognitive impairment predicts conversion to multiple sclerosis in clinically isolated syndromes
Zipoli, V., Goretti, B., Hakiki, B., Siracusa, G., Sorbi, S., Portaccio, E., & Amato, M. P. (2009) · Multiple Sclerosis Journal · doi:10.1177/1352458509350311 · Cited by 169 · see citing works
Preventing Overfitting of Cross-Validation Data
Ng, A. Y. (1997) · International Conference on Machine Learning · Cited by 165 · see citing works
The general linear model and fMRI: Does love last forever?
Poline, J., & Brett, M. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.133 · Cited by 162 · see citing works
MRI hippocampal and entorhinal cortex mapping in predicting conversion to Alzheimer's disease
Devanand, D. P., Bansal, R., Liu, J., Hao, X., Pradhaban, G., & Peterson, B. S. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.075 · Cited by 151 · see citing works
Estimating effects and making predictions from genome-wide marker dataPreprint
Goddard, M. E., Wray, N. R., VERBYLA, K. L., & Visscher, P. M. (2009) · Cited by 140 · see citing works
Estimating Effects and Making Predictions from Genome-Wide Marker Data
Goddard, M. E., Wray, N. R., Verbyla, K., & Visscher, P. M. (2009) · Statistical Science · doi:10.1214/09-sts306 · Cited by 121 · see citing works
Error processing and gender-shared and -specific neural predictors of relapse in cocaine dependence
Luo, X., Zhang, S., Hu, S., Bednarski, S. R., Erdman, E., Farr, O. M., Hong, K., Sinha, R., Mazure, C. M., & Li, C.-S. R. (2013) · Brain · doi:10.1093/brain/awt040 · Cited by 105 · see citing works
Corpus callosum shape alterations in individuals prior to the onset of psychosis
Walterfang, M., Yung, A. R., WOOD, A. G., Reutens, D. C., Phillips, L., Wood, S. J., Chen, J., Velakoulis, D., McGorry, P. D., & Pantelis, C. (2008) · Schizophrenia Research · doi:10.1016/j.schres.2008.04.042 · Cited by 76 · see citing works
Lavretsky, H., Zheng, L., Weiner, M. W., Mungas, D., Reed, B., Kramer, J. H., Jagust, W. J., Chui, H. C., & Mack, W. J. (2010) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2009.09020280 · Cited by 69 · see citing works
Using J-pruning to reduce overfitting in classification trees
Bramer, M. (2002) · Knowledge-Based Systems · doi:10.1016/s0950-7051(01)00163-0 · Cited by 63 · see citing works
Reduced fMRI activity predicts relapse in patients recovering from stimulant dependence
Clark, V. P., Beatty, G. K., Anderson, R. E., Kodituwakku, P., Phillips, J., Lane, T. D. R., Kiehl, K. A., & Calhoun, V. D. (2012) · Human Brain Mapping · doi:10.1002/hbm.22184 · Cited by 60 · see citing works
Predicting memory decline in normal elderly: Genetics, MRI, and cognitive reserve
Tupler, L. A., Krishnan, K., Greenberg, D. L., Marcovina, S. M., Payne, M. E., MacFall, J. R., Charles, H. C., & Doraiswamy, P. M. (2006) · Neurobiology of Aging · doi:10.1016/j.neurobiolaging.2006.07.001 · Cited by 49 · see citing works
Lin, Y., Liu, C., Chiu, M., Liu, C., Chien, Y., Hwang, T., Jaw, F., Shan, J., Hsieh, M. H., & Hwu, H. (2012) · PLoS ONE · doi:10.1371/journal.pone.0034454 · Cited by 40 · see citing works
Predicting Sensation Seeking From Dopamine Genes
Powell, J. E., & Zietsch, B. P. (2011) · Psychological Science · doi:10.1177/0956797610397669 · Cited by 13 · see citing works
Braverman, E. R., Blum, K., Damle, U. J., Kerner, M. M., Dushaj, K., & Oscar‐Berman, M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0055398 · Cited by 11 · see citing works
Garner, B., Pariante, C. M., Wood, S. J., Velakoulis, D., Phillips, L., Soulsby, B., Brewer, W. J., Smith, D. J., Dazzan, P., Berger, G. E., et al. (2005) · Biological Psychiatry · doi:10.1016/j.biopsych.2005.04.018 · Cited by 8 · see citing works
A permutation approach to validation*
Magdon‐Ismail, M., & Mertsalov, K. (2010) · Statistical Analysis and Data Mining The ASA Data Science Journal · doi:10.1002/sam.10096 · Cited by 7 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Building better biomarkers: brain models in translational neuroimaging
Woo, C.-W., Chang, L. J., Lindquist, M. A., & Wager, T. D. (2017) · Nature Neuroscience · doi:10.1038/nn.4478 · Cited by 1,056 · see citing works
Leakage and the reproducibility crisis in machine-learning-based science
Kapoor, S., & Narayanan, A. (2023) · Patterns · doi:10.1016/j.patter.2023.100804 · Cited by 1,387 · see citing works
Computational psychiatry as a bridge from neuroscience to clinical applications
Huys, Q. J. M., Maia, T. V., & Frank, M. J. (2016) · Nature Neuroscience · doi:10.1038/nn.4238 · Cited by 1,002 · see citing works
Using connectome-based predictive modeling to predict individual behavior from brain connectivity
Shen, X.-L., Finn, E., Scheinost, D., Rosenberg, M., Chun, M. M., Papademetris, X., & Constable, R. T. (2017) · Nature Protocols · doi:10.1038/nprot.2016.178 · Cited by 1,088 · see citing works
Machine Learning Approaches for Clinical Psychology and Psychiatry
Dwyer, D. B., Falkai, P., & Koutsouleris, N. (2018) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-032816-045037 · Cited by 966 · see citing works
Building a Science of Individual Differences from fMRI
Dubois, J., & Adolphs, R. (2016) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2016.03.014 · Cited by 610 · see citing works
Prediction as a Humanitarian and Pragmatic Contribution from Human Cognitive Neuroscience
Gabrieli, J. D. E., Ghosh, S., & Whitfield, S. (2015) · Neuron · doi:10.1016/j.neuron.2014.10.047 · Cited by 526 · see citing works
Task-induced brain state manipulation improves prediction of individual traits
Greene, A., Gao, S., Scheinost, D., & Constable, R. T. (2018) · Nature Communications · doi:10.1038/s41467-018-04920-3 · Cited by 525 · see citing works
Neuropsychosocial profiles of current and future adolescent alcohol misusers
Whelan, R., Watts, R., Orr, C., Althoff, R., Artiges, E., Banaschewski, T., Barker, G., Bokde, A., Büchel, C., Carvalho, F. M., et al. (2014) · Nature · doi:10.1038/nature13402 · Cited by 438 · see citing works
Self-citationTen simple rules for predictive modeling of individual differences in neuroimaging
Scheinost, D., Noble, S., Horien, C., Greene, A. S., Lake, E. M., Salehi, M., Gao, S., Shen, X., O'Connor, D., Barron, D. S., et al. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.02.057 · Cited by 381 · see citing works
Can brain state be manipulated to emphasize individual differences in functional connectivity?
Finn, E., Scheinost, D., Finn, D., Shen, X.-L., Papademetris, X., & Constable, R. T. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.03.064 · Cited by 361 · see citing works
Sui, J., Jiang, R., Bustillo, J., & Calhoun, V. (2020) · Biological Psychiatry · doi:10.1016/j.biopsych.2020.02.016 · Cited by 309 · see citing works
A small number of abnormal brain connections predicts adult autism spectrum disorder
Yahata, N., Morimoto, J., Hashimoto, R., Lisi, G., Shibata, K., Kawakubo, Y., Kuwabara, H., Kuroda, M., Yamada, T., Megumi, F., et al. (2016) · Nature Communications · doi:10.1038/ncomms11254 · Cited by 305 · see citing works
Elliott, M. L., Knodt, A. R., Cooke, M. E., Kim, M. J., Melzer, T. R., Keenan, R. J., Ireland, D. R., Ramrakha, S., Poulton, R., Caspi, A., et al. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.01.068 · Cited by 299 · see citing works
Deep neural networks in psychiatry
Durstewitz, D., Koppe, G., & Meyer‐Lindenberg, A. (2019) · Molecular Psychiatry · doi:10.1038/s41380-019-0365-9 · Cited by 286 · see citing works
Advances in fMRI Real-Time Neurofeedback
Watanabe, T., Sasaki, Y., Shibata, K., & Kawato, M. (2017) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2017.09.010 · Cited by 242 · see citing works
Connectome-based prediction of cocaine abstinence
Yip, S. W., Scheinost, D., Potenza, M. N., & Carroll, K. M. (2019) · American Journal of Psychiatry · doi:10.1176/appi.ajp.2018.17101147 · Cited by 196 · see citing works
Data leakage inflates prediction performance in connectome-based machine learning models
Rosenblatt, M., Tejavibulya, L., Jiang, R., Noble, S., & Scheinost, D. (2024) · Nature Communications · doi:10.1038/s41467-024-46150-w · Cited by 236 · see citing works
Garbusow, M., Schad, D. J., Sebold, M., Friedel, E., Bernhardt, N. R., Koch, S., Steinacher, B., Kathmann, N., Geurts, D. E. M., Sommer, C., et al. (2015) · Addiction Biology · doi:10.1111/adb.12243 · Cited by 166 · see citing works
Resting‐state functional connectivity and nicotine addiction: prospects for biomarker development
Fedota, J. R., & Stein, E. A. (2015) · Annals of the New York Academy of Sciences · doi:10.1111/nyas.12882 · Cited by 165 · see citing works
A neuromarker for drug and food craving distinguishes drug users from non-users
Koban, L., Wager, T. D., & Hedy Kober, A. (2022) · Nature Neuroscience · doi:10.1038/s41593-022-01228-w · Cited by 150 · see citing works
Gupta, N. S., & Kumar, P. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107051 · Cited by 155 · see citing works
Biomarkers in Substance Use Disorders
Volkow, N. D., Koob, G., & Baler, R. (2015) · ACS Chemical Neuroscience · doi:10.1021/acschemneuro.5b00067 · Cited by 141 · see citing works
Pinaya, W. H. L., Mechelli, A., & Sato, J. R. (2018) · Human Brain Mapping · doi:10.1002/hbm.24423 · Cited by 137 · see citing works
Reward Region Responsivity Predicts Future Weight Gain and Moderating Effects of the TaqIA Allele
Stice, E. M., Burger, K. S., & Yokum, S. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.3607-14.2015 · Cited by 136 · see citing works
Combining Multiple Connectomes Improves Predictive Modeling of Phenotypic Measures
Gao, S., Greene, A., Constable, R. T., & Scheinost, D. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.116038 · Cited by 149 · see citing works
Widge, A. S., Ellard, K. K., Paulk, A. C., Basu, I., Yousefi, A., Zorowitz, S., Gilmour, A., Afzal, A., Deckersbach, T., Cash, S. S., et al. (2017) · Experimental Neurology · doi:10.1016/j.expneurol.2016.07.021 · Cited by 118 · see citing works
Ahn, W., & Vassileva, J. (2016) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2016.02.008 · Cited by 119 · see citing works
Yamada, T., Hashimoto, R., Yahata, N., Ichikawa, N., Yoshihara, Y., Okamoto, Y., Kato, N., Takahashi, H., & Kawato, M. (2017) · The International Journal of Neuropsychopharmacology · doi:10.1093/ijnp/pyx059 · Cited by 133 · see citing works
Working Memory-Related Neural Activity Predicts Future Smoking Relapse
Loughead, J., Wileyto, E. P., Ruparel, K., Falcone, M., Hopson, R. D., Gur, R. C., & Lerman, C. E. (2014) · Neuropsychopharmacology · doi:10.1038/npp.2014.318 · Cited by 145 · see citing works
Feng, G., Ingvalson, E. M., Grieco‐Calub, T. M., Roberts, M. Y., Ryan, M. E., Birmingham, P. K., Burrowes, D. M., Young, N. M., & Wong, P. C. M. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1717603115 · Cited by 113 · see citing works
Computational neuroscience approach to biomarkers and treatments for mental disorders
Yahata, N., Kasai, K., & Kawato, M. (2016) · Psychiatry and Clinical Neurosciences · doi:10.1111/pcn.12502 · Cited by 126 · see citing works
What big data can do for treatment in psychiatry
Gillan, C. M., & Whelan, R. (2017) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2017.07.003 · Cited by 106 · see citing works
Self-citationBoyle, R. T., Jollans, L., Rueda‐Delgado, L. M., Rizzo, R., Yener, G., McMorrow, J. P., Knight, S. P., Carey, D., Robertson, I. H., Emek‐Savaş, D. D., et al. (2020) · Brain Imaging and Behavior · doi:10.1007/s11682-020-00260-3 · Cited by 104 · see citing works
Self-citationQuaak, M., van de Mortel, L. A., Thomas, R. M., & van Wingen, G. A. (2021) · NeuroImage Clinical · doi:10.1016/j.nicl.2021.102584 · Cited by 103 · see citing works
Your evidence? Machine learning algorithms for medical diagnosis and prediction
Heinrichs, B., & Eickhoff, S. B. (2019) · Human Brain Mapping · doi:10.1002/hbm.24886 · Cited by 99 · see citing works
Feeling How Old I Am: Subjective Age Is Associated With Estimated Brain Age
Kwak, S., Kim, H., Chey, J., & Youm, Y. (2018) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2018.00168 · Cited by 93 · see citing works
Ahn, W., Ramesh, D., Moeller, F. G., & Vassileva, J. (2016) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2016.00034 · Cited by 90 · see citing works
Yamashita, M., Yoshihara, Y., Hashimoto, R., Yahata, N., Ichikawa, N., Sakai, Y., Yamada, T., Matsukawa, N., Okada, G., Tanaka, S., et al. (2018) · eLife · doi:10.7554/elife.38844 · Cited by 88 · see citing works
Khosla, A., Khandnor, P., & Chand, T. (2021) · Journal of Applied Biomedicine · doi:10.1016/j.bbe.2021.12.005 · Cited by 89 · see citing works
Kusuma, K., Larsen, M. E., Quiroz, J. C., Gillies, M. B., Burnett, A., Qian, J., & Torok, M. H. (2022) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2022.09.050 · Cited by 90 · see citing works
Generalizable brain network markers of major depressive disorder across multiple imaging sites
Yamashita, A., Sakai, Y., Yamada, T., Yahata, N., Kunimatsu, A., Okada, N., Itahashi, T., Hashimoto, R., Mizuta, H., Ichikawa, N., et al. (2020) · PLoS Biology · doi:10.1371/journal.pbio.3000966 · Cited by 82 · see citing works
EEG spectral power, but not theta/beta ratio, is a neuromarker for adult ADHD
Kiiski, H. S. M., Bennett, M., Rueda‐Delgado, L. M., Farina, F. R., Knight, R., Boyle, R. T., Roddy, D. W., Grogan, K., Bramham, J., Kelly, C. A. M., et al. (2019) · European Journal of Neuroscience · doi:10.1111/ejn.14645 · Cited by 81 · see citing works
Self-citationNeuroscience and the Criminal Justice System
Greely, H. T., & Farahany, N. A. (2018) · Annual Review of Criminology · doi:10.1146/annurev-criminol-011518-024433 · Cited by 77 · see citing works
Yip, S. W., Kiluk, B. D., & Scheinost, D. (2019) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2019.11.001 · Cited by 77 · see citing works
Translational machine learning for psychiatric neuroimaging
Walter, M., Alizadeh, S., Jamalabadi, H., Lueken, U., Dannlowski, U., Walter, H., Olbrich, S., Čolić, L., Kambeitz, J. P., Koutsouleris, N., et al. (2018) · Progress in Neuro-Psychopharmacology and Biological Psychiatry · doi:10.1016/j.pnpbp.2018.09.014 · Cited by 78 · see citing works
Transdiagnostic Connectome-Based Prediction of Craving
Garrison, K. A., Sinha, R., Potenza, M. N., Gao, S., Liang, Q., Lacadie, C., & Scheinost, D. (2023) · American Journal of Psychiatry · doi:10.1176/appi.ajp.21121207 · Cited by 68 · see citing works
A Neural Marker of Obsessive-Compulsive Disorder from Whole-Brain Functional Connectivity
Takagi, Y., Sakai, Y., Lisi, G., Yahata, N., Abe, Y., Nishida, S., Nakamae, T., Morimoto, J., Kawato, M., Narumoto, J., et al. (2017) · Scientific Reports · doi:10.1038/s41598-017-07792-7 · Cited by 67 · see citing works
Few-Shot Domain-Adaptive Anomaly Detection for Cross-Site Brain Images
Su, J., Shen, H., Peng, L., & Hu, D. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3125686 · Cited by 68 · see citing works
Gowin, J. L., Ball, T. M., Wittmann, M., Tapert, S. F., & Paulus, M. P. (2015) · Drug and Alcohol Dependence · doi:10.1016/j.drugalcdep.2015.04.018 · Cited by 64 · see citing works
SMRI Biomarkers Predict Electroconvulsive Treatment Outcomes: Accuracy with Independent Data Sets
Jiang, R., Abbott, C., Jiang, T., Du, Y., Espinoza, R. T., Narr, K. L., Wade, B., Yu, Q., Song, M., Lin, D., et al. (2017) · Neuropsychopharmacology · doi:10.1038/npp.2017.165 · Cited by 64 · see citing works
Predicting the knowledge–recklessness distinction in the human brain
Vilares, I., Wesley, M. J., Ahn, W., Bonnie, R. J., Hoffman, M. B., Jones, O. D., Morse, S. J., Yaffe, G., Lohrenz, T. M., & Montague, P. R. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1619385114 · Cited by 62 · see citing works
Heinrich, A., Müller, K. U., Banaschewski, T., Barker, G. J., Bokde, A. L. W., Bromberg, U., Büchel, C., Conrod, P., Fauth‐Bühler, M., Orfanos, D. P., et al. (2016) · Biological Psychology · doi:10.1016/j.biopsycho.2016.05.002 · Cited by 64 · see citing works
Self-citationMikolas, P., Melicher, T., Skoch, A., Matejka, M., Slovakova, A., Bakstein, E., Hajek, T., & Spaniel, F. (2016) · Psychological Medicine · doi:10.1017/s0033291716000878 · Cited by 58 · see citing works
Zhang, B., Qi, S., Pan, X., Li, C., Yao, Y., Qian, W., & Guan, Y. (2021) · Frontiers in Oncology · doi:10.3389/fonc.2020.598721 · Cited by 58 · see citing works
Neuromarkers for Mental Disorders: Harnessing Population Neuroscience
Jollans, L., & Whelan, R. (2018) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2018.00242 · Cited by 56 · see citing works
Self-citationPredicting the future of neuroimaging predictive models in mental health
Tejavibulya, L., Rolison, M. J., Gao, S., Liang, Q., Peterson, H., Dadashkarimi, J., Farruggia, M. C., Hahn, C. A., Noble, S., Lichenstein, S. D., et al. (2022) · Molecular Psychiatry · doi:10.1038/s41380-022-01635-2 · Cited by 57 · see citing works
Acuff, H. E., Versace, A., Bertocci, M. A., Ladouceur, C. D., Hanford, L. C., Manelis, A., Monk, K., Bonar, L. K., Mccaffrey, A., Goldstein, B. I., et al. (2018) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2018.2318 · Cited by 54 · see citing works
Acuff, H. E., Versace, A., Bertocci, M. A., Ladouceur, C. D., Hanford, L. C., Manelis, A., Monk, K., Bonar, L. K., Mccaffrey, A., Goldstein, B. I., et al. (2019) · Neuropsychopharmacology · doi:10.1038/s41386-019-0339-2 · Cited by 50 · see citing works
Toward Robust Anxiety Biomarkers: A Machine Learning Approach in a Large-Scale Sample
Boeke, E. A., Holmes, A. J., & Phelps, E. A. (2020) · Biological Psychiatry: Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2019.05.018 · Cited by 50 · see citing works
Psychotherapy and Neuroimaging
Fournier, J. C. J., & Price, R. B. (2014) · FOCUS The Journal of Lifelong Learning in Psychiatry · doi:10.1176/appi.focus.12.3.290 · Cited by 48 · see citing works
Čukić, M., López, V., & Pavón, J. (2020) · Journal of Medical Internet Research · doi:10.2196/19548 · Cited by 45 · see citing works
Kiiski, H. S. M., Jollans, L., Donnchadha, S. Ó., Nolan, H., Lonergan, R. M., Kelly, S. B., O’Brien, M. C., Kinsella, K., Bramham, J., Burke, T., et al. (2018) · Brain Topography · doi:10.1007/s10548-018-0620-4 · Cited by 45 · see citing works
Self-citationChallenges and promises for translating computational tools into clinical practice
Ahn, W., & Busemeyer, J. R. (2016) · Current Opinion in Behavioral Sciences · doi:10.1016/j.cobeha.2016.02.001 · Cited by 44 · see citing works
Benschop, L., Vanhollebeke, G., Li, J., Leahy, R. M., Vanderhasselt, M., & Baeken, C. (2022) · Scientific Reports · doi:10.1038/s41598-022-20274-9 · Cited by 41 · see citing works
Bridging the gap between the neurocognitive lab and the addiction clinic
Franken, I. H. A., & van de Wetering, B. J. M. (2014) · Addictive Behaviors · doi:10.1016/j.addbeh.2014.11.034 · Cited by 42 · see citing works
Spechler, P. A., Allgaier, N. A., Chaarani, B., Whelan, R., Watts, R., Orr, C., Albaugh, M. D., D’Alberto, N., Higgins, S. T., Hudson, K. E., et al. (2018) · European Journal of Neuroscience · doi:10.1111/ejn.13989 · Cited by 42 · see citing works
Self-citationIdentifying the signature of prospective motor control in children with autism
Cavallo, A., Romeo, L., Ansuini, C., Battaglia, F. M., Nobili, L., Pontil, M., Panzeri, S., & Becchio, C. (2021) · Scientific Reports · doi:10.1038/s41598-021-82374-2 · Cited by 40 · see citing works
Bertocci, M. A., Bebko, G. M., Versace, A., Fournier, J. C. J., Iyengar, S., Olino, T. M., Bonar, L. K., Almeida, J., Perlman, S. B., Schirda, C., et al. (2016) · Molecular Psychiatry · doi:10.1038/mp.2016.5 · Cited by 38 · see citing works
Psychiatric Neural Networks and Precision Therapeutics by Machine Learning
Komatsu, H., Watanabe, E., & Fukuchi, M. (2021) · Biomedicines · doi:10.3390/biomedicines9040403 · Cited by 37 · see citing works
Reporting details of neuroimaging studies on individual traits prediction: A literature survey
Yeung, A. W. K., More, S., Wu, J., & Eickhoff, S. B. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119275 · Cited by 36 · see citing works
Jiang, R., Woo, C., Qi, S., Wu, J., & Jing Sui (Beijing Normal University), my correct affiliation is beijing normal university, N. Q. U. O. S. A. T. (2022) · IEEE Signal Processing Magazine · doi:10.1109/msp.2022.3155951 · Cited by 36 · see citing works
The Clinical Added Value of Imaging: A Perspective From Outcome Prediction
Jollans, L., & Whelan, R. (2016) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2016.04.005 · Cited by 39 · see citing works
Self-citationDryburgh, E., McKenna, S. J., & Rekik, I. (2019) · Brain Imaging and Behavior · doi:10.1007/s11682-019-00111-w · Cited by 35 · see citing works
Antonucci, L. A., Penzel, N., Pergola, G., Kambeitz‐Ilankovic, L., Dwyer, D., Kambeitz, J. P., Haas, S. S., Passiatore, R., Fazio, L., Caforio, G., et al. (2019) · Neuropsychopharmacology · doi:10.1038/s41386-019-0532-3 · Cited by 33 · see citing works
Closed-Loop Control of a Neuroprosthetic Hand by Magnetoencephalographic Signals
Fukuma, R., Yanagisawa, T., Yorifuji, S., Kato, R., Yokoi, H., Hirata, M., Saitoh, Y., Kishima, H., Kamitani, Y., & Yoshimine, T. (2015) · PLoS ONE · doi:10.1371/journal.pone.0131547 · Cited by 32 · see citing works
O’Halloran, L., Nymberg, C., Jollans, L., Garavan, H., & Whelan, R. (2016) · Addiction · doi:10.1111/add.13629 · Cited by 32 · see citing works
Self-citationHawes, M., Schwartz, H. A., Son, Y., & Klein, D. (2022) · Psychological Medicine · doi:10.1017/s0033291722003452 · Cited by 27 · see citing works
Pergola, G., Carlo, P. D., Andriola, I., Gelao, B., Torretta, S., Attrotto, M. T., Fazio, L., Raio, A., Albergo, D., Masellis, R., et al. (2015) · Psychological Medicine · doi:10.1017/s0033291715002639 · Cited by 32 · see citing works
Ewing, S. W. F., Tapert, S. F., & Molina, B. S. G. (2015) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2015.12.011 · Cited by 31 · see citing works
Chopra, S., Dhamala, E., Lawhead, C., Ricard, J. A., Orchard, E. R., An, L., Chen, P., Wulan, N., Kumar, P., Rubenstein, A., et al. (2024) · Science Advances · doi:10.1126/sciadv.adn1862 · Cited by 37 · see citing works
Closed-Loop Stimulation in Emotional Circuits for Neuro-Psychiatric Disorders
Widge, A. S., & Moritz, C. T. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-802452-2.00017-2 · Cited by 29 · see citing works
Habets, P. C., Thomas, R. M., Milaneschi, Y., Jansen, R., Pool, R., Peyrot, W. J., Penninx, B. W. J. H., Meijer, O. C., van Wingen, G. A., & Vinkers, C. H. (2023) · Biological Psychiatry · doi:10.1016/j.biopsych.2023.05.024 · Cited by 30 · see citing works
Machine Learning and Prediction in Fetal, Infant, and Toddler Neuroimaging: A Review and Primer
Scheinost, D., Pollatou, A., Dufford, A. J., Jiang, R., Farruggia, M. C., Rosenblatt, M., Peterson, H., Rodriguez, R. X., Dadashkarimi, J., Liang, Q., et al. (2022) · Biological Psychiatry · doi:10.1016/j.biopsych.2022.10.014 · Cited by 29 · see citing works
Liu, M., Liu, X., Hildebrandt, A., & Zhou, C. (2020) · Cerebral Cortex Communications · doi:10.1093/texcom/tgaa015 · Cited by 28 · see citing works
Recent advances of deep learning in psychiatric disorders
Chen, L., Xia, C., & Sun, H. (2020) · Precision Clinical Medicine · doi:10.1093/pcmedi/pbaa029 · Cited by 28 · see citing works
Rutherford, H., Potenza, M., Mayes, L., & Scheinost, D. (2019) · Cerebral Cortex · doi:10.1093/cercor/bhz185 · Cited by 24 · see citing works
Connectome‐based predictive modelling of cognitive reserve using task‐based functional connectivity
Boyle, R. T., Connaughton, M., McGlinchey, E., Knight, S. P., Looze, C. D., Carey, D., Stern, Y., Robertson, I. H., Kenny, R. A., & Whelan, R. (2022) · European Journal of Neuroscience · doi:10.1111/ejn.15896 · Cited by 27 · see citing works
Self-citationBailey, N. W., Fulcher, B., Caldwell, B., Hill, A. T., Fitzgibbon, B. M., van Dijk, H., & Fitzgerald, P. B. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.12.007 · Cited by 27 · see citing works
Lee, S., Ahn, W., Seweryn, M. T., & Sadée, W. (2018) · BMC Genomics · doi:10.1186/s12864-018-5219-3 · Cited by 27 · see citing works
Widge, A. S., & Moritz, C. T. (2014) · Journal of Neural Engineering · doi:10.1088/1741-2560/11/2/024001 · Cited by 29 · see citing works
Schuckit, M. A., Smith, T. L., Paulus, M. P., Tapert, S. F., Simmons, A. N., Tolentino, N. J., & Shafir, A. (2016) · Alcoholism Clinical and Experimental Research · doi:10.1111/acer.12935 · Cited by 26 · see citing works
Bučková, B. R., Brunovský, M., Bareš, M., & Hlinka, J. (2020) · Frontiers in Neuroscience · doi:10.3389/fnins.2020.589303 · Cited by 26 · see citing works
Price, M., Albaugh, M. D., Hahn, S., Juliano, A., Fani, N., Brier, Z. M. F., Legrand, A. C., van Stolk‐Cooke, K., Chaarani, B., Potter, A., et al. (2021) · Neuropsychopharmacology · doi:10.1038/s41386-021-00987-7 · Cited by 23 · see citing works
Self-citationLiu, M., Wang, Y., Zhang, H., Yang, Q., Shi, F., Zhou, Y. F., & Shen, D. G. (2022) · Cerebral Cortex · doi:10.1093/cercor/bhab507 · Cited by 25 · see citing works
Classifying Conduct Disorder using a biopsychosocial model and machine learning method
Chan, L., Simmons, C., Tillem, S., Conley, M., Brazil, I., & Baskin-Sommers, A. (2022) · Biological Psychiatry: Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2022.02.004 · Cited by 24 · see citing works
Quality over quantity: powering neuroimaging samples in psychiatry
Makowski, C., Nichols, T. E., & Dale, A. M. (2024) · Neuropsychopharmacology · doi:10.1038/s41386-024-01893-4 · Cited by 23 · see citing works
Medaglia, J. D., Erickson, B. A., Zimmerman, J. P., & Kelkar, A. S. (2019) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2019.01.002 · Cited by 23 · see citing works
Brain event-related potentials predict individual differences in inhibitory control
Rueda‐Delgado, L. M., O’Halloran, L., Enz, N., Ruddy, K. L., Kiiski, H. S. M., Bennett, M., Farina, F. R., Jollans, L., Vahey, N. A., & Whelan, R. (2019) · International Journal of Psychophysiology · doi:10.1016/j.ijpsycho.2019.03.013 · Cited by 23 · see citing works
Self-citationIdentification of a brain fingerprint for overweight and obesity
Farruggia, M. C., van Kooten, M. J., Perszyk, E. E., Burke, M. V., Scheinost, D., Constable, R. T., & Small, D. M. (2020) · Physiology & Behavior · doi:10.1016/j.physbeh.2020.112940 · Cited by 25 · see citing works
- Authors
- Whelan, R., & Garavan, H.
- Affiliations
- University of Vermont (US)
- Corresponding author
- Garavan, Hugh
- Venue
- Biological Psychiatry
- Publisher
- Elsevier BV
- Published
- 2014-05
- Volume
- 75
- Issue
- 9
- Pages
- 746-748
- ISSN
- 0006-3223 (ISSN-L)
- OpenAlex
- W1979450493
- Cited by
- 209
- References
- 40
- Citations by year
- 2026: 13 · 2025: 12 · 2024: 11 · 2023: 17 · 2022: 24 · 2021: 22 · 2020: 17 · 2019: 22 · 2018: 17 · 2017: 15 · 2016: 20 · 2015: 12 · 2014: 6 · 2013: 1
- FWCI
- 7.09
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Biological Psychiatry (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- optimism bias, overfitting, model evaluation, psychiatric neuroimaging, outcome prediction, high-dimensional predictors, regression models, depression treatment response, recurrence prediction, conversion to dementia
- MeSH terms
- Brain, Computer Simulation, Humans, Regression Analysis, ROC Curve, Signal Processing, Computer-Assisted, Neuroimaging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review