Cross-ethnicity/race generalization failure of behavioral prediction from resting-state functional connectivity
Open accessThe DOI matches its Crossref record; 35 of the 35 references it cites were found in a registry or an index.
Abstract
Algorithmic biases that favor majority populations pose a key challenge to the application of machine learning for precision medicine. Here, we assessed such bias in prediction models of behavioral phenotypes from brain functional magnetic resonance imaging. We examined the prediction bias using two independent datasets (preadolescent versus adult) of mixed ethnic/racial composition. When predictive models were trained on data dominated by white Americans (WA), out-of-sample prediction errors were generally higher for African Americans (AA) than for WA. This bias toward WA corresponds to more WA-like brain-behavior association patterns learned by the models. When models were trained on AA only, compared to training only on WA or an equal number of AA and WA participants, AA prediction accuracy improved but stayed below that for WA. Overall, the results point to the need for caution and further research regarding the application of current brain-behavior prediction models in minority populations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Hungarian method for the assignment problem
Kuhn, H. W. (1955) · Naval Research Logistics Quarterly · doi:10.1002/nav.3800020109 · Cited by 13,120
Sudlow, C., Gallacher, J., Allen, N., Beral, V., Burton, P., Danesh, J., Downey, P., Elliott, P., Green, J., Landray, M., et al. (2015) · PLOS Medicine · doi:10.1371/journal.pmed.1001779 · Cited by 12,728
Jenkinson, M., Bannister, P., Brady, M., & Smith, S. (2002) · NeuroImage · doi:10.1006/nimg.2002.1132 · Cited by 11,888
Fischl, B. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2012.01.021 · Cited by 9,526
The organization of the human cerebral cortex estimated by intrinsic functional connectivity
Thomas Yeo, B. T., Krienen, F. M., Sepulcre, J., Sabuncu, M. R., Lashkari, D., Hollinshead, M., Roffman, J. L., Smoller, J. W., Zöllei, L., Polimeni, J. R., et al. (2011) · Journal of Neurophysiology · doi:10.1152/jn.00338.2011 · Cited by 9,182
Fischl, B., Salat, D. H., Busa, E., Albert, M., Dieterich, M., Haselgrove, C., van der Kouwe, A., Killiany, R., Kennedy, D., Klaveness, S., et al. (2002) · Neuron · doi:10.1016/s0896-6273(02)00569-x · Cited by 8,610
Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697
Power, J. D., Barnes, K. A., Snyder, A. Z., Schlaggar, B. L., & Petersen, S. E. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.10.018 · Cited by 7,429 · cited 10×
“Age, gender, FD, DVARS, intracranial volume, years of education, and household income were selected as confounding variables in the behavioral prediction models.”
MethodInfluentialThe organization of the human cerebellum estimated by intrinsic functional connectivity
Buckner, R. L., Krienen, F. M., Castellanos, A., Diaz, J. C., & Yeo, B. T. T. (2011) · Journal of Neurophysiology · doi:10.1152/jn.00339.2011 · Cited by 7,304
Self-citationDissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137
Cortical Surface-Based Analysis
Fischl, B., Sereno, M. I., & Dale, A. M. (1999) · NeuroImage · doi:10.1006/nimg.1998.0396 · Cited by 6,147 · cited 1×
“The rs-fMRI data were processed by the minimal preprocessing pipeline (83) and the following steps: (i) removal of the first X volumes (Siemens and Philips: X = 8; GE DV25: X = 5; GE DV26: X = 16; following the ABCD release notes); (ii) alignment with the T1 images using boundary-based registration (87) with FsFast D…”
MethodThe WU-Minn Human Connectome Project: An overview
Van Essen, D. C., Smith, S. M., Barch, D. M., Behrens, T. E. J., Yacoub, E., & Ugurbil, K. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.05.041 · Cited by 5,651
Methods to detect, characterize, and remove motion artifact in resting state fMRI
Power, J. D., Mitra, A., Laumann, T. O., Snyder, A. Z., Schlaggar, B. L., & Petersen, S. E. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.08.048 · Cited by 3,748 · cited 1×
“, censoring; see below); (iv) nuisance regression of six motion parameters, global signal, mean white matter signal, mean ventricular signal, and their temporal derivatives (18 regressors in total); (v) interpolation of censored frames using Lomb-Scargle periodogram (88); (vi) bandpass filtering (0.”
MethodLocal-global parcellation of the human cerebral cortex from intrinsic functional connectivity MRI
Schaefer, A., Kong, R., Gordon, E. M., Laumann, T. O., Zuo, X.-N., Holmes, A. J., Eickhoff, S. B., & Yeo, B. T. T. (2018) · Cerebral Cortex · doi:10.1093/cercor/bhx179 · Cited by 3,657 · cited 2×
“Another possible source of bias could be functional atlas, as the cortical parcellation we used here was derived from a dataset that was also predominately composed of WA (56).”
MethodSelf-citationClinical use of current polygenic risk scores may exacerbate health disparities
Martin, A. R., Kanai, M., Kamatani, Y., Okada, Y., Neale, B. M., & Daly, M. J. (2019) · Nature Genetics · doi:10.1038/s41588-019-0379-x · Cited by 2,821
Functional connectome fingerprinting: Identifying individuals using patterns of brain connectivity
Finn, E. S., Shen, X., Scheinost, D., Rosenberg, M. D., Huang, J., Chun, M. M., Papademetris, X., & Constable, R. T. (2015) · Nature Neuroscience · doi:10.1038/nn.4135 · Cited by 2,747
Salimi-Khorshidi, G., Douaud, G., Beckmann, C. F., Glasser, M. F., Griffanti, L., & Smith, S. M. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.11.046 · Cited by 1,770
Cem: Coarsened Exact Matching in Stata
Blackwell, M., Iacus, S., King, G., & Porro, G. (2009) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0900900402 · Cited by 1,689
Function in the human connectome: task-fMRI and individual differences in behavior
Barch, D. M., Burgess, G. C., Harms, M. P., Petersen, S. E., Schlaggar, B. L., Corbetta, M., Glasser, M. F., Curtiss, S., Dixit, S., Feldt, C., et al. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.05.033 · Cited by 1,529
Resting-state fMRI in the human connectome project
Smith, S. M., Beckmann, C. F., Andersson, J., Auerbach, E. J., Bijsterbosch, J., Douaud, G., Duff, E., Feinberg, D. A., Griffanti, L., Harms, M. P., et al. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.05.039 · Cited by 1,502
Family income, parental education and brain structure in children and adolescents
Noble, K. G., Houston, S. M., Brito, N. H., Bartsch, H., Kan, E., Kuperman, J. M., Akshoomoff, N., Amaral, D. G., Bloss, C. S., Libiger, O., et al. (2015) · Nature Neuroscience · doi:10.1038/nn.3983 · Cited by 1,423
Recruiting the ABCD sample: Design considerations and procedures
Garavan, H., Bartsch, H., Conway, K., Decastro, A., Goldstein, R. Z., Heeringa, S., Jernigan, T., Potter, A., Thompson, W., & Zahs, D. (2018) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2018.04.004 · Cited by 1,393
Ensuring fairness in machine learning to advance health equity.
Rajkomar, A., Hardt, M., Howell, M. D., Corrado, G., & Chin, M. H. (2018) · Annals of Internal Medicine · doi:10.7326/m18-1990 · Cited by 1,341 · cited 2×
“, lower true-positive rate to detect a disease in a minority ethnic group), equalized sensitivity should be implemented to correct this bias (14, 15).”
BackgroundAhmed, Z., Mohamed, K., Zeeshan, S., & Dong, X. (2020) · Database · doi:10.1093/database/baaa010 · Cited by 1,035
The conception of the ABCD study: From substance use to a broad NIH collaboration
Volkow, N. D., Koob, G. F., Croyle, R. T., Bianchi, D. W., Gordon, J. A., Koroshetz, W. J., Pérez-Stable, E. J., Riley, W. T., Bloch, M. H., Conway, K., et al. (2018) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2017.10.002 · Cited by 970
Barch, D. M., Albaugh, M. D., Avenevoli, S., Chang, L., Clark, D. B., Glantz, M. D., Hudziak, J. J., Jernigan, T. L., Tapert, S. F., Yurgelun-Todd, D., et al. (2018) · Developmental Cognitive Neuroscience · doi:10.1016/j.dcn.2017.10.010 · Cited by 814
VanderWeele, T. J., & Robinson, W. R. (2014) · Epidemiology · doi:10.1097/ede.0000000000000105 · Cited by 567
Ten simple rules for predictive modeling of individual differences in neuroimaging
Scheinost, D., Noble, S., Horien, C., Greene, A. S., Lake, E. M., Salehi, M., Gao, S., Shen, X., O'Connor, D., Barron, D. S., et al. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.02.057 · Cited by 381
Li, J., Kong, R., Liégeois, R., Orban, C., Tan, Y., Sun, N., Holmes, A. J., Sabuncu, M. R., Ge, T., & Yeo, B. T. T. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.04.016 · Cited by 380
Self-citationHousing and wealth inequality: Racial-ethnic differences in home equity in the United States
Krivo, L. J., & Kaufman, R. L. (2004) · Demography · doi:10.1353/dem.2004.0023 · Cited by 274
He, T., Kong, R., Holmes, A. J., Nguyen, M., Sabuncu, M. R., Eickhoff, S. B., Bzdok, D., Feng, J., & Yeo, B. T. T. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2019.116276 · Cited by 271
Self-citationBurgess, G. C., Kandala, S., Nolan, D., Laumann, T. O., Power, J. D., Adeyemo, B., Harms, M. P., Petersen, S. E., & Barch, D. M. (2016) · Brain Connectivity · doi:10.1089/brain.2016.0435 · Cited by 216 · cited 1×
“To further reduce the global motion and respirational artifacts (78, 79) and to improve behavioral prediction performance (20, 74), we performed additional regression of global signal (the signal averaged across cortical vertices) and its first temporal derivative.”
MethodA Convex Framework for Fair Regression
Berk, R., Heidari, H., Jabbari, S., Joseph, M., Kearns, M., Morgenstern, J., Neel, S., & Roth, A. (2017) · arXiv · doi:10.48550/arxiv.1706.02409 · Cited by 198
Berry, C. M., Clark, M. A., & McClure, T. K. (2011) · Journal of Applied Psychology · doi:10.1037/a0023222 · Cited by 99
The Dangers of Counterfactual Causal Thinking about Detecting Racial Discrimination
Kohler-Hausmann, I. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3050650 · Cited by 9
Cited by
Data leakage inflates prediction performance in connectome-based machine learning models
Rosenblatt, M., Tejavibulya, L., Jiang, R., Noble, S., & Scheinost, D. (2024) · Nature Communications · doi:10.1038/s41467-024-46150-w · Cited by 236
Brain–phenotype models fail for individuals who defy sample stereotypes
Greene, A. S., Shen, X., Noble, S., Horien, C., Hahn, C. A., Arora, J., Tokoglu, F., Spann, M. N., Carrión, C. I., Barron, D. S., et al. (2022) · Nature · doi:10.1038/s41586-022-05118-w · Cited by 167
Dhamala, E., Yeo, B. T. T., & Holmes, A. J. (2023) · Biological Psychiatry · doi:10.1016/j.biopsych.2022.09.024 · Cited by 91
Self-citationChen, Z., Hu, B., Liu, X., Becker, B., Eickhoff, S. B., Miao, K., Gu, X., Tang, Y., Dai, X., Li, C., et al. (2023) · BMC Medicine · doi:10.1186/s12916-023-02941-4 · Cited by 35
Self-citationFeng, G., Wang, Y., Huang, W., Chen, H., Cheng, J., & Shu, N. (2024) · eLife · doi:10.7554/elife.93325 · Cited by 15
Metadata
- Authors
- Li, J., Bzdok, D., Chen, J., Tam, A., Ooi, L. Q. R., Holmes, A. J., Ge, T., Patil, K. R., Jabbi, M., Eickhoff, S. B., Yeo, B. T. T., & Genon, S.
- Venue
- Science Advances
- Published
- 2022-03-18
- Volume
- 8
- Issue
- 11
- Pages
- eabj1812-eabj1812
- ISSN
- 2375-2548 (ISSN-L)
- OpenAlex
- W4220992429
- Cited by
- 133
- References
- 35
- FWCI
- 13.90
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.