How to remove or control confounds in predictive models, with applications to brain biomarkers
Open accessThe DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.
Abstract
Background: With increasing data sizes and more easily available computational methods, neurosciences rely more and more on predictive modeling with machine learning, e.g., to extract disease biomarkers. Yet, a successful prediction may capture a confounding effect correlated with the outcome instead of brain features specific to the outcome of interest. For instance, because patients tend to move more in the scanner than controls, imaging biomarkers of a disease condition may mostly reflect head motion, leading to inefficient use of resources and wrong interpretation of the biomarkers. Results: Here we study how to adapt statistical methods that control for confounds to predictive modeling settings. We review how to train predictors that are not driven by such spurious effects. We also show how to measure the unbiased predictive accuracy of these biomarkers, based on a confounded dataset. For this purpose, cross-validation must be modified to account for the nuisance effect. To guide understanding and practical recommendations, we apply various strategies to assess predictive models in the presence of confounds on simulated data and population brain imaging settings. Theoretical and empirical studies show that deconfounding should not be applied to the train and test data jointly: modeling the effect of confounds, on the training data only, should instead be decoupled from removing confounds. Conclusions: Cross-validation that isolates nuisance effects gives an additional piece of information: confound-free prediction accuracy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures












Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“For example, regressing the covariates/confounds out of the whole dataset in a non-cross-validated manner has been shown to negatively affect prediction performance40.”
From Data leakage inflates prediction performance in connectome-based machine learning models (2024) · cites this paper as [40]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, the pre-fusion exclusion of biased subpopulation creates an artificial construct that fails to capture the true population-level associations, while diminishing available sample size and data utility in neuroimaging research, thereby severely limiting the generalizability of findings, particularly in small-sample studies.”
From Confound Controlled Multimodal Neuroimaging Data Fusion and Its Application to Developmental Disorders (2025) · cites this paper as [13]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Our results corroborate previous works, suggesting that covariate regression must be performed within the cross-validation loop to avoid false deflation of effect sizes38–40.”
From Data leakage inflates prediction performance in connectome-based machine learning models (2024) · cites this paper as [40]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Cross-validated and non-cross-validated covariate regression have been compared in several previous studies38,40, and we build upon those previous works here by evaluating them in three phenotypes and four datasets through the view of leakage.”
From Data leakage inflates prediction performance in connectome-based machine learning models (2024) · cites this paper as [40]
The cited source says what the claim attributes to it.
In the citing paper
“In addition, we performed cross-validated covariate regression, where several covariates were regressed from the functional connectivity data within the cross-validation scheme38,40.”
From Data leakage inflates prediction performance in connectome-based machine learning models (2024) · cites this paper as [40]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Sudlow, C., Gallacher, J., Allen, N., Beral, V., Burton, P., Danesh, J., Downey, P., Elliott, P., Green, J., Landray, M., et al. (2015) · PLOS Medicine · doi:10.1371/journal.pmed.1001779 · Cited by 12,728 · cited 1×
“(cid:2) The UK Biobank (UKBB) project [51] is a prospective epidemiological study to elucidate the development of diseases of the UK population over the years.”
Statistical parametric maps in functional imaging: A general linear approach
Friston, K. J., Holmes, A. P., Worsley, K. J., Poline, J. ‐., Frith, C. D., & Frackowiak, R. S. J. (1994) · Human Brain Mapping · doi:10.1002/hbm.460020402 · Cited by 9,688 · cited 2×
“In the context of the GLM,an alternative implementation relies on removing the effect of variables that are correlated [25].”
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2009) · doi:10.1515/9781400829828 · Cited by 9,467
Power, J. D., Barnes, K. A., Snyder, A. Z., Schlaggar, B. L., & Petersen, S. E. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.10.018 · Cited by 7,429 · cited 2×
“For instance, brain imaging reflects age quite accurately and actually carries information about age-related diseases [8,10,11], yet [12] showed that participants’ in-scanner motion varies with age and it creates systematic differences in recorded brain imaging signals.”
Mostly Harmless Econometrics: An Empiricist's Companion
Angrist, J. D., & Pischke, J.-S. (2008) · doi:10.2307/j.ctvcm4j72 · Cited by 5,051
Statistical Modeling: The Two Cultures (with comments and a rejoinder by the author)
Breiman, L. (2001) · Statistical Science · doi:10.1214/ss/1009213726 · Cited by 4,340 · cited 1×
“Often, a predictive analysis does not build on a generative model of the signal but on optimizing discrimination [18].”
Non-Uniform Random Variate Generation
Devroye, L. (1986) · Springer eBooks · doi:10.1007/978-1-4613-8643-8 · Cited by 4,041
Estimating Causal Effects from Large Data Sets Using Propensity Scores
Rubin, D. B. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-8_part_2-199710151-00064 · Cited by 2,877 · cited 1×
“For in-sample analysis, propensity scores are a classic reweighting technique used to obtain reliable effect estimates from confounded datasets [68,69].”
Causality: models, reasoning, and inference
(2000) · Choice Reviews Online · doi:10.5860/choice.38-1494a · Cited by 2,873
Estimation of Average Treatment Effects Based on Propensity Scores
Becker, S. O., & Ichino, A. (2002) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0200200403 · Cited by 2,754 · cited 1×
“For in-sample analysis, propensity scores are a classic reweighting technique used to obtain reliable effect estimates from confounded datasets [68,69].”
Functional connectome fingerprinting: Identifying individuals using patterns of brain connectivity
Finn, E. S., Shen, X., Scheinost, D., Rosenberg, M. D., Huang, J., Chun, M. M., Papademetris, X., & Constable, R. T. (2015) · Nature Neuroscience · doi:10.1038/nn.4135 · Cited by 2,747 · cited 1×
“Various studies have predicted individual cognitive abilities from brain functional connectivity [42,43].”
Machine learning for neuroimaging with scikit-learn
Abraham, A., Pedregosa, F., Eickenberg, M., Gervais, P., Mueller, A., Kossaifi, J., Gramfort, A., Thirion, B., & Varoquaux, G. (2014) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2014.00014 · Cited by 2,384 · cited 1×
“We use Nilearn [58] for the whole predictive pipeline.”
Self-citationThe influence of head motion on intrinsic functional connectivity MRI
Van Dijk, K. R. A., Sabuncu, M. R., & Buckner, R. L. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.07.044 · Cited by 2,344 · cited 2×
“In addition, they severely affect measurements of functional connectivity [41].”
The interpretation of interaction in contingency tables
Simpson, E. H. (1951) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1951.tb00088.x · Cited by 2,301 · cited 1×
“Conditioning on the third variable z can then reverse the correlation between 2 variables X and y , a phenomenon known as the Berkson or Simpson statistical paradox [66,67].”
Beyond mind-reading: multi-voxel pattern analysis of fMRI data
Norman, K. A., Polyn, S. M., Detre, G. J., & Haxby, J. V. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2006.07.005 · Cited by 2,271 · cited 1×
“In cognitive neuroscience, they can be used for “decoding,” to make conclusions on mental processes given observed brain activity [1–3].”
Nipype: A Flexible, Lightweight and Extensible Neuroimaging Data Processing Framework in Python
Gorgolewski, K., Burns, C. D., Madison, C., Clark, D., Halchenko, Y. O., Waskom, M. L., & Ghosh, S. S. (2011) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2011.00013 · Cited by 2,186
Prediction of individual brain maturity using fMRI
Dosenbach, N. U. F., Nardos, B., Cohen, A. L., Fair, D. A., Power, J. D., Church, J. A., Nelson, S. M., Wig, G. S., Vogel, A. C., Lessov-Schlaggar, C. N., et al. (2010) · Science · doi:10.1126/science.1194144 · Cited by 2,183 · cited 2×
“Many recent studies report age prediction, e.g., from resting-state functional connectivity [7,32,34], from structural imaging [35], or combining multiple imaging modalities [8,10].”
Multimodal population brain imaging in the UK Biobank prospective epidemiological study
Miller, K. L., Alfaro-Almagro, F., Bangerter, N. K., Thomas, D. L., Yacoub, E., Xu, J., Bartsch, A. J., Jbabdi, S., Sotiropoulos, S. N., Andersson, J. L. R., et al. (2016) · Nature Neuroscience · doi:10.1038/nn.4393 · Cited by 2,020 · cited 1×
“To make matters worse, large cohorts developed in population imaging to answer epidemiological questions (e.g., UK Biobank [13]) are observational data: there is no controlled intervention or balanced case-control group; rather, individuals are recruited from diverse populations with various…”
Satterthwaite, T. D., Elliott, M. A., Gerraty, R. T., Ruparel, K., Loughead, J., Calkins, M. E., Eickhoff, S. B., Hakonarson, H., Gur, R. C., Gur, R. E., et al. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2012.08.052 · Cited by 1,881 · cited 2×
“Thus, age-related changes observed in brain images may be confounded by head motion [39] and image quality [40].”
Limitations of the Application of Fourfold Table Analysis to Hospital Data
Berkson, J. (1946) · Biometrics Bulletin · doi:10.2307/3002000 · Cited by 1,810 · cited 1×
“Conditioning on the third variable z can then reverse the correlation between 2 variables X and y , a phenomenon known as the Berkson or Simpson statistical paradox [66,67].”
An fMRI-Based Neurologic Signature of Physical Pain
Wager, T. D., Atlas, L. Y., Lindquist, M. A., Roy, M., Woo, C.-W., & Kross, E. (2013) · New England Journal of Medicine · doi:10.1056/nejmoa1204471 · Cited by 1,597
Image processing and Quality Control for the first 10,000 brain imaging datasets from UK Biobank
Alfaro-Almagro, F., Jenkinson, M., Bangerter, N. K., Andersson, J. L. R., Griffanti, L., Douaud, G., Sotiropoulos, S. N., Jbabdi, S., Hernandez-Fernandez, M., Vallee, E., et al. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2017.10.034 · Cited by 1,557 · cited 1×
“The data used here contain 9,302 participants from the first release of the UKBB ongoing cohort study with available resting-state fMRI scans and extensive health and lifestyle information [52,53].”
Yan, C.-G., Cheung, B., Kelly, C., Colcombe, S., Craddock, R. C., Di Martino, A., Li, Q., Zuo, X.-N., Castellanos, F. X., & Milham, M. P. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.03.004 · Cited by 1,461
Self-citationAbilities: Their Structure, Growth, and Action
Merrifield, P. R., & Cattell, R. B. (1975) · American Educational Research Journal · doi:10.3102/00028312012004516 · Cited by 1,401
Abilities: Their Structure, Growth, and Action
Merrifield, P. R., & Cattell, R. B. (1975) · American Educational Research Journal · doi:10.2307/1162752 · Cited by 1,396 · cited 1×
“Fluid intelligence quantifies the ability to solve novel problems independently from accumulated knowledge, as opposed to crystallized intelligence, which involves experience and previous knowledge [44].”
Age differences in fluid and crystallized intelligence
Horn, J. L., & Cattell, R. B. (1967) · Acta Psychologica · doi:10.1016/0001-6918(67)90011-x · Cited by 1,331 · cited 1×
“It is well known that cognitive abilities change with age [45– 48], in particular that fluid Intelligence progressively declines in middle age [49], while crystallized intelligence continues to grow with age.”
A General Statistical Analysis for fMRI Data
Worsley, K. J., Liao, C. H., Aston, J., Petre, V., Duncan, G. H., Morales, F., & Evans, A. C. (2002) · NeuroImage · doi:10.1006/nimg.2001.0933 · Cited by 1,163 · cited 2×
“The statistical literature on controlling confounding variables is well developed for classic statistical analysis, such as statistical testing in a linear model at the heart of the standard mass-univariate brain mapping [16,17].”
Satterthwaite, T. D., Wolf, D. H., Loughead, J., Ruparel, K., Elliott, M. A., Hakonarson, H., Gur, R. C., & Gur, R. E. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.12.063 · Cited by 1,125 · cited 1×
“2 [12,36– 38]).”
Building better biomarkers: brain models in translational neuroimaging
Woo, C.-W., Chang, L. J., Lindquist, M. A., & Wager, T. D. (2017) · Nature Neuroscience · doi:10.1038/nn.4478 · Cited by 1,056 · cited 1×
“A crucial aspect of these biomarkers is their ability to predict the outcome of interest, i.e., to generalize to new data [9].”
Arjovsky, M., Bottou, L., Gulrajani, I., & Lopez-Paz, D. (2019) · arXiv · doi:10.48550/arxiv.1907.02893 · Cited by 927 · cited 1×
“This can be addressed with techniques such as invariant risk minimization [28], but we do not further consider this approach here.”
Cole, J. H., Ritchie, S. J., Bastin, M. E., Valdés Hernández, M. C., Muñoz Maniega, S., Royle, N., Corley, J., Pattie, A., Harris, S. E., Zhang, Q., et al. (2018) · Molecular Psychiatry · doi:10.1038/mp.2017.62 · Cited by 831 · cited 2×
“Because brain aging is a risk factor of many diseases, the prediction of brain age from MRI is a promising biomarker [11].”
Inferring mental states from neuroimaging data: from reverse inference to large-scale decoding
Poldrack, R. A. (2011) · Neuron · doi:10.1016/j.neuron.2011.11.001 · Cited by 821
Quantifying Biases in Causal Models: Classical Confounding vs Collider-Stratification Bias
Greenland, S. (2003) · Epidemiology · doi:10.1097/01.ede.0000042804.12056.6c · Cited by 810 · cited 1×
“In such a situation conditioning on it can create a form of selection bias, sometimes known as “collider bias” [64,65].”
Illustrating bias due to conditioning on a collider
Cole, S. R., Platt, R. W., Schisterman, E. F., Chu, H., Westreich, D., Richardson, D., & Poole, C. (2010) · International Journal of Epidemiology · doi:10.1093/ije/dyp334 · Cited by 796 · cited 1×
“In such a situation conditioning on it can create a form of selection bias, sometimes known as “collider bias” [64,65].”
Franke, K., Ziegler, G., Klöppel, S., & Gaser, C. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.01.005 · Cited by 794 · cited 1×
“Many recent studies report age prediction, e.g., from resting-state functional connectivity [7,32,34], from structural imaging [35], or combining multiple imaging modalities [8,10].”
Assessing and tuning brain decoders: Cross-validation, caveats, and guidelines
Varoquaux, G., Raamana, P. R., Engemann, D. A., Hoyos-Idrobo, A., Schwartz, Y., & Thirion, B. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2016.10.038 · Cited by 792 · cited 3×
“Prediction accuracy is ideally measured on an independent validation set, but most often, no large independent validation set is available and a cross-validation procedure, which iteratively separates train and test sets [21],is used.”
InfluentialSelf-citationHartshorne, J. K., & Germine, L. T. (2015) · Psychological Science · doi:10.1177/0956797614567339 · Cited by 788 · cited 1×
“It is well known that cognitive abilities change with age [45– 48], in particular that fluid Intelligence progressively declines in middle age [49], while crystallized intelligence continues to grow with age.”
Taylor, J. R., Williams, N., Cusack, R., Auer, T., Shafto, M. A., Dixon, M., Tyler, L. K., Cam-CAN, & Henson, R. N. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2015.09.018 · Cited by 782 · cited 1×
“(cid:2) Cambridge Center for Ageing and Neuroscience (CamCan) data [50] study age-related changes in cognition and brain anatomy and function.”
Establishment of Best Practices for Evidence for Prediction: A Review
Poldrack, R. A., Huckins, G., & Varoquaux, G. (2020) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2019.3671 · Cited by 738 · cited 2×
“In addition, predictive models draw their purpose and validity from out-of-sample prediction rather than in-sample statistical testing [19].”
Self-citationBest practices in data analysis and sharing in neuroimaging using MRI
Nichols, T. E., Das, S., Eickhoff, S. B., Evans, A. C., Glatard, T., Hanke, M., Kriegeskorte, N., Milham, M. P., Poldrack, R. A., Poline, J.-B., et al. (2017) · Nature Neuroscience · doi:10.1038/nn.4500 · Cited by 714 · cited 1×
“We give detailed information on preprocessing steps for each dataset in Appendix 8 of Nichols et al., following COBIDAS recommendations [54].”
Self-citationDeriving reproducible biomarkers from multi-site resting-state data: An Autism-based example
Abraham, A., Milham, M. P., Di Martino, A., Craddock, R. C., Samaras, D., Thirion, B., & Varoquaux, G. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2016.10.045 · Cited by 701 · cited 3×
“This argument is used by Abraham et al.[6] to control for the effect of movement on autism diagnostics.”
Self-citationHandbook of Functional MRI Data Analysis
Poldrack, R. A., Mumford, J. A., & Nichols, T. E. (2011) · Cambridge University Press eBooks · doi:10.1017/cbo9780511895029 · Cited by 699 · cited 1×
“The statistical literature on controlling confounding variables is well developed for classic statistical analysis, such as statistical testing in a linear model at the heart of the standard mass-univariate brain mapping [16,17].”
Predicting brain-age from multimodal imaging data captures cognitive impairment
Liem, F., Varoquaux, G., Kynast, J., Beyer, F., Kharabian Masouleh, S., Huntenburg, J. M., Lampe, L., Rahim, M., Abraham, A., Craddock, R. C., et al. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2016.11.005 · Cited by 511 · cited 4×
“Many recent studies report age prediction, e.g., from resting-state functional connectivity [7,32,34], from structural imaging [35], or combining multiple imaging modalities [8,10].”
InfluentialSelf-citationp-Values for High-Dimensional Regression
Meinshausen, N., Meier, L., & Bühlmann, P. (2009) · Journal of the American Statistical Association · doi:10.1198/jasa.2009.tm08647 · Cited by 442 · cited 1×
“The figure also reports the P -value of predictive accuracy,from permutations (technically,there is one P -value per fold; to report only one number,we use P -value aggregation [59]).”
The need to approximate the use-case in clinical machine learning
Saeb, S., Lonini, L., Jayaraman, A., Mohr, D. C., & Kording, K. P. (2017) · GigaScience · doi:10.1093/gigascience/gix019 · Cited by 367 · cited 2×
“Similarly, in multi-participant prediction with repeated measurements from the same participant, participant-wise cross-validation can rule out that prediction is driven by participant identification [22,27].”
Automated detection of brain atrophy patterns based on MRI for the prediction of Alzheimer's disease
Plant, C., Teipel, S. J., Oswald, A., Böhm, C., Meindl, T., Mourao-Miranda, J., Bokde, A. W., Hampel, H., & Ewers, M. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2009.11.046 · Cited by 343 · cited 1×
“With the rise of large-scale brain-imaging cohorts,they can extract imaging biomarkers that predict across subjects phenotypes such as neuropsychiatric conditions [4–6] or individual traits [7,8].”
Multi-level bootstrap analysis of stable clusters in resting-state fMRI
Bellec, P., Rosa-Neto, P., Lyttelton, O. C., Benali, H., & Evans, A. C. (2010) · NeuroImage · doi:10.1016/j.neuroimage.2010.02.082 · Cited by 333
Benchmarking functional connectome-based predictive models for resting-state fMRI
Dadi, K., Rahim, M., Abraham, A., Chyzhyk, D., Milham, M., Thirion, B., & Varoquaux, G. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.02.062 · Cited by 333 · cited 1×
“To build predictive models from resting-state fMRI, we follow the recommendations of Dadi et al. [55].”
Self-citationStatistical Challenges in “Big Data” Human Neuroimaging
Smith, S. M., & Nichols, T. E. (2018) · Neuron · doi:10.1016/j.neuron.2017.12.018 · Cited by 330
Confound modelling in UK Biobank brain imaging
Alfaro-Almagro, F., McCarthy, P., Afyouni, S., Andersson, J. L. R., Bastiani, M., Miller, K. L., Nichols, T. E., & Smith, S. M. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2020.117002 · Cited by 249 · cited 1×
“Such choices are implicitly based on a model of which variables are causes or consequences of the fictional intervention and the outcome of interest (see [63] for guidelines in the case of UKBB).”
Brumback, B. A., Hernán, M. A., Haneuse, S. J. P. A., & Robins, J. M. (2004) · Statistics in Medicine · doi:10.1002/sim.1657 · Cited by 212 · cited 1×
“In that case, the proposed approach does not help, but such a case is very hard to handle with statistical meth-ods (see,e.g. [61]).”
Age differences in fluid intelligence: contributions of general slowing and frontal decline
Bugg, J. M., Zook, N. A., DeLosh, E. L., Davalos, D. B., & Davis, H. P. (2006) · Brain and Cognition · doi:10.1016/j.bandc.2006.02.006 · Cited by 209
Functional brain networks related to individual differences in human intelligence at rest
Hearne, L. J., Mattingley, J. B., & Cocchi, L. (2016) · Scientific Reports · doi:10.1038/srep32328 · Cited by 207 · cited 2×
“In particular, [43] used machine learning to predict fluid intelligence from rest fMRI.”
Geerligs, L., Tsvetanov, K. A., Cam‐CAN, & Henson, R. N. (2017) · Human Brain Mapping · doi:10.1002/hbm.23653 · Cited by 174 · cited 1×
“2 [12,36– 38]).”
Using and understanding cross-validation strategies. Perspectives on Saeb et al.
Little, M. A., Varoquaux, G., Saeb, S., Lonini, L., Jayaraman, A., Mohr, D. C., & Kording, K. P. (2017) · GigaScience · doi:10.1093/gigascience/gix020 · Cited by 158 · cited 2×
“Similarly, in multi-participant prediction with repeated measurements from the same participant, participant-wise cross-validation can rule out that prediction is driven by participant identification [22,27].”
Self-citationTraining confounder-free deep learning models for medical applications
Zhao, Q., Adeli, E., & Pohl, K. M. (2020) · Nature Communications · doi:10.1038/s41467-020-19784-9 · Cited by 148 · cited 1×
“As an attractive complementary approach, note that deep learning approaches for learning confound-free models have been proposed [60].”
How to control for confounds in decoding analyses of neuroimaging data
Snoek, L., Miletić, S., & Scholte, H. S. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2018.09.074 · Cited by 143 · cited 3×
“1), estimated on the train data with Equation 2 and then applied to the test [26].”
Engemann, D. A., Kozynets, O., Sabbagh, D., Lemaître, G., Varoquaux, G., Liem, F., & Gramfort, A. (2020) · eLife · doi:10.7554/elife.54055 · Cited by 131 · cited 2×
“Many recent studies report age prediction, e.g., from resting-state functional connectivity [7,32,34], from structural imaging [35], or combining multiple imaging modalities [8,10].”
Self-citationPredictive modelling using neuroimaging data in the presence of confounds
Rao, A., Monteiro, J. M., & Mourao-Miranda, J. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.01.066 · Cited by 114 · cited 3×
“From a prediction perspective, when the training population reflects the target population adequately, changing the training data to remove the effect of the confounder may not improve prediction accuracy [24].”
Varoquaux, G., Baronnet, F., Kleinschmidt, A., Fillard, P., & Thirion, B. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15705-9_25 · Cited by 107
Self-citationVariations in structural MRI quality significantly impact commonly used measures of brain anatomy
Gilmore, A. D., Buser, N. J., & Hanson, J. L. (2021) · Brain Informatics · doi:10.1186/s40708-021-00128-2 · Cited by 96 · cited 1×
“Thus, age-related changes observed in brain images may be confounded by head motion [39] and image quality [40].”
Samu, D., Campbell, K. L., Tsvetanov, K. A., Shafto, M. A., Brayne, C., Bullmore, E. T., Calder, A. J., Cusack, R., Dalgleish, T., Duncan, J., et al. (2017) · Nature Communications · doi:10.1038/ncomms14743 · Cited by 95
Li, H., Satterthwaite, T. D., & Fan, Y. (2018) · 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) · doi:10.1109/isbi.2018.8363532 · Cited by 87
Predictive models avoid excessive reductionism in cognitive neuroimaging
Varoquaux, G. P., & Poldrack, R. A. (2018) · Current Opinion in Neurobiology · doi:10.1016/j.conb.2018.11.002 · Cited by 76 · cited 1×
“In cognitive neuroscience, they can be used for “decoding,” to make conclusions on mental processes given observed brain activity [1–3].”
Self-citationAge-related functional brain changes in young children
Long, X., Benischek, A., Dewey, D., & Lebel, C. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.04.059 · Cited by 71 · cited 2×
“Many recent studies report age prediction, e.g., from resting-state functional connectivity [7,32,34], from structural imaging [35], or combining multiple imaging modalities [8,10].”
Addressing Confounding in Predictive Models with an Application to Neuroimaging
Linn, K. A., Gaonkar, B., Doshi, J., Davatzikos, C., & Shinohara, R. T. (2016) · The International Journal of Biostatistics · doi:10.1515/ijb-2015-0030 · Cited by 44 · cited 1×
“The problem can then be tackled as a “domain adaptation” problem [23,24].”
Kievit, R., Fuhrmann, D., Borgeest, G. S., Simpson-Kent, I. L., & Henson, R. N. (2018) · Wellcome Open Research · doi:10.12688/wellcomeopenres.14241.2 · Cited by 43 · cited 1×
“It is well known that cognitive abilities change with age [45– 48], in particular that fluid Intelligence progressively declines in middle age [49], while crystallized intelligence continues to grow with age.”
Zepf, F. D., Bubenzer‐Busch, S., Runions, K., Rao, P., Wong, J. W. Y., Mahfouda, S., Morandini, H. A. E., Stewart, R. M., Moore, J. K., Biskup, C. S., et al. (2017) · Brain and Cognition · doi:10.1016/j.bandc.2017.10.005 · Cited by 23 · cited 1×
“In childhood also, markers of functional brain development can help to reveal neurodevelopmental disorders [32,33].”
Controlling a confound in predictive models with a test set minimizing its effect
Chyzhyk, D., Varoquaux, G. P., Thirion, B., & Milham, M. P. (2018) · International Workshop on Pattern Recognition in NeuroImaging · doi:10.1109/prni.2018.8423961 · Cited by 17 · cited 1×
“A preliminary version of the work discussed here was presented at the PRNI conference [20].”
Self-citationChyzhyk, D., Varoquaux, G., Thirion, B., et al. (2022) · GigaDB · doi:10.5524/100980
Self-citation
Cited by
Data leakage inflates prediction performance in connectome-based machine learning models
Rosenblatt, M., Tejavibulya, L., Jiang, R., Noble, S., & Scheinost, D. (2024) · Nature Communications · doi:10.1038/s41467-024-46150-w · Cited by 236
“In addition, we performed cross-validated covariate regression, where several covariates were regressed from the functional connectivity data within the cross-validation scheme 38,40 .”
InfluentialWang, R., Chaudhari, P., & Davatzikos, C. (2023) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2211613120 · Cited by 78
Biological markers and psychosocial factors predict chronic pain conditions
Fillingim, M., Tanguay-Sabourin, C., Parisien, M., Zare, A., Guglietti, G. V., Norman, J., Petre, B., Bortsov, A. V., Ware, M. A., Pérez, J., et al. (2025) · Nature Human Behaviour · doi:10.1038/s41562-025-02156-y · Cited by 69
Wang, H.-T., Meisler, S. L., Sharmarke, H., Clarke, N., Gensollen, N., Markiewicz, C. J., Paugam, F., Thirion, B., & Bellec, P. (2024) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1011942 · Cited by 56
“One such key challenge is the effective correction of non-neuronal sources of fluctuations (called confounds), known as denoising, which is important to reduce bias when studying the association between connectomes and behavioral measures of interest [5].”
Self-citationChopra, S., Dhamala, E., Lawhead, C., Ricard, J. A., Orchard, E. R., An, L., Chen, P., Wulan, N., Kumar, P., Rubenstein, A., et al. (2024) · Science Advances · doi:10.1126/sciadv.adn1862 · Cited by 43
“For each of the 100 train-test splits, the variables were first regressed out of the global cognition training data, and the resulting regression coefficients were used to residualize the global cognition test data ( 88 ), after which the entire prediction modeling procedure was repeated.”
Metadata
- Authors
- Chyzhyk, D., Varoquaux, G., Milham, M., & Thirion, B.
- Affiliations
- Institut national de recherche en sciences et technologies du numérique (FR)
- Parietal project-team , INRIA Saclay-île de France,
- Nathan Kline Institute for Psychiatric Research (US)
- Corresponding author
- Chyzhyk, Darya
- Venue
- GigaScience
- Publisher
- Oxford University Press (OUP)
- Published
- 2022
- Volume
- 11
- ISSN
- 2047-217X (ISSN-L)
- PMC
- PMC8917515
- OpenAlex
- W4220876827
- Semantic Scholar
- c705dc0b28422c58e3d1a615da3427b760cfc49d
- Cited by
- 65
- References
- 68
- Citations by year
- 2026: 12 · 2025: 16 · 2024: 21 · 2023: 10 · 2022: 6
- FWCI
- 9.82
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- GigaScience (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- HAL (submittedVersion, other-oa)
- Funders
- Child Mind Institute
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning and Data Classification
- Keywords
- confounding, deconfounding, neuroimaging biomarkers, predictive modeling, head movement, cross-validation, prediction accuracy, machine learning
- MeSH terms
- Machine Learning, Brain, Humans, Biomarkers
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.