Cited by 37Functional Brain Connectivity StudiesAdvanced MRI Techniques and ApplicationsArtificial Intelligence in Healthcare

Confound Removal and Normalization in Practice: A Neuroimaging Based Sex Prediction Case Study

Open access
More, Shammi, Eickhoff, Simon B., Caspers, Julian, Patil, Kaustubh R.·2021·Lecture Notes in Computer Science

Abstract

Machine learning (ML) methods are increasingly being used to predict pathologies and biological traits using neuroimaging data. Here controlling for confounds is essential to get unbiased estimates of generalization performance and to identify the features driving predictions. However, a systematic evaluation of the advantages and disadvantages of available alternatives is lacking. This makes it difficult to compare results across studies and to build deployment quality models. Here, we evaluated two commonly used confound removal schemes–whole data confound regression (WDCR) and cross-validated confound regression (CVCR)–to understand their effectiveness and biases induced in generalization performance estimation. Additionally, we study the interaction of the confound removal schemes with Z-score normalization, a common practice in ML modelling. We applied eight combinations of confound removal schemes and normalization (pipelines) to decode sex from resting-state functional MRI (rfMRI) data while controlling for two confounds, brain size and age. We show that both schemes effectively remove linear univariate and multivariate confounding effects resulting in reduced model performance with CVCR providing better generalization estimates, i.e., closer to out-of-sample performance than WDCR. We found no effect of normalizing before or after confound removal. In the presence of dataset and confound shift, four tested confound removal procedures yielded mixed results, raising new questions. We conclude that CVCR is a better method to control for confounding effects in neuroimaging studies. We believe that our in-depth analyses shed light on choices associated with confound removal and hope that it generates more interest in this problem instrumental to numerous applications.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/978-3-030-67670-4_1

Cites

46 of 46 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Covariate regression must be performed within the cross-validation loop to avoid false deflation of effect sizes.

    “Our results corroborate previous works, suggesting that covariate regression must be performed within the cross-validation loop to avoid false deflation of effect sizes38–40.”

    From Data leakage inflates prediction performance in connectome-based machine learning models (2024) · cites this paper as [39]

    The Experimental Setup (Confound Regression; page unavailable) says that regression before cross-validation causes data leakage, whereas regression fitted within the loop avoids it. The Results (Generalization Performance Estimates; page unavailable) report lower performance estimates with whole-data regression and suggest the within-loop approach is better. The paper does not establish false deflation of effect sizes specifically, and its recommendation is less categorical than “must.” (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
More, S., Eickhoff, S. B., Caspers, J., & Patil, K. R.
Affiliations
  • Forschungszentrum Jülich (DE)
  • Düsseldorf University Hospital (DE)
Corresponding author
Patil, Kaustubh R.
Venue
Lecture Notes in Computer Science
Published
2021
Pages
3-18
ISSN
  • 0302-9743 (ISSN-L)
  • 1611-3349
OpenAlex
W3130801515
Cited by
37
References
46
Citations by year
2025: 5 · 2024: 10 · 2023: 8 · 2022: 10 · 2021: 3 · 2020: 1
FWCI
12.43
Type
conference paper
Language
en
Open access
Locations
  • Lecture notes in computer science (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
Funders
  • European Commission (HORIZON2020, 945539)
  • Deutsche Forschungsgemeinschaft (EI 816/21-1)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced MRI Techniques and Applications
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
Z-score normalization, sex prediction, resting-state fMRI, brain size, age confounding

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review