The DOI matches its Crossref record; 154 of the 154 references it cites were found in a registry or an index.
Multimodal Fusion of Brain Imaging Data: A Key to Finding the Missing Link(s) in Complex Mental Illness
Open accessAbstract
It is becoming increasingly clear that combining multi-modal brain imaging data is able to provide more information for individual subjects by exploiting the rich multimodal information that exists. However, the number of studies that do true multimodal fusion (i.e. capitalizing on joint information among modalities) is still remarkably small given the known benefits. In part, this is because multi-modal studies require broader expertise in collecting, analyzing, and interpreting the results than do unimodal studies. In this paper, we start by introducing the basic reasons why multimodal data fusion is important and what it can do, and importantly how it can help us avoid wrong conclusions and help compensate for imperfect brain imaging studies. We also discuss the challenges that need to be confronted for such approaches to be more widely applied by the community. We then provide a review of the diverse studies that have used multimodal data fusion (primarily focused on psychosis) as well as provide an introduction to some of the existing analytic approaches. Finally, we discuss some up-and-coming approaches to multi-modal fusion including deep learning and multimodal classification which show considerable promise. Our conclusion is that multimodal data fusion is rapidly growing, but it is still underutilized. The complexity of the human brain coupled with the incomplete measurement provided by existing imaging technology makes multimodal fusion essential in order to mitigate against misdirection and hopefully provide a key to finding the missing link(s) in complex mental illness.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Correspondence of the brain's functional architecture during activation and rest
Smith, S. M., Fox, P. T., Miller, K. L., Glahn, D. C., Fox, P. M., Mackay, C. E., Filippini, N., Watkins, K. E., Toro, R., Laird, A. R., et al. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0905267106 · Cited by 5,150
Common polygenic variation contributes to risk of schizophrenia and bipolar disorder
The International Schizophrenia Consortium (2009) · Nature · doi:10.1038/nature08185 · Cited by 4,821
Deep Learning: Methods and Applications
Deng, L., & Yu, D. (2014) · Foundations and Trends® in Signal Processing · doi:10.1561/2000000039 · Cited by 3,480
Selection of relevant features and examples in machine learning
Blum, A. L., & Langley, P. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(97)00063-5 · Cited by 3,298
The economy of brain network organization
Bullmore, E., & Sporns, O. (2012) · Nature Reviews Neuroscience · doi:10.1038/nrn3214 · Cited by 3,284
Dale, A. M., & Sereno, M. I. (1993) · Journal of Cognitive Neuroscience · doi:10.1162/jocn.1993.5.2.162 · Cited by 2,003
Deep Learning: Methods and Applications
Deng, L. (2014) · now publishers, Inc. eBooks · doi:10.1561/9781601988157 · Cited by 1,759
Imaging the developing brain: what have we learned about cognitive development?
CASEY, B., TOTTENHAM, N., LISTON, C., & DURSTON, S. (2005) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.01.011 · Cited by 1,556
Partial Least Squares (PLS) methods for neuroimaging: A tutorial and review
Krishnan, A., Williams, L. J., McIntosh, A. R., & Abdi, H. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2010.07.034 · Cited by 1,349
Multimodal classification of Alzheimer's disease and mild cognitive impairment
Zhang, D., Wang, Y., Zhou, L., Yuan, H., & Shen, D. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.01.008 · Cited by 1,314
Research works printed but not linked: 14
- Sui, 2009, A new joint blind source separation model for two datasets and its application to second-level FMRI group analysis, Presented at the Organization for Human Brain Mapping Annual Meeting, June 18--23, San Francisco, California.
- Liu, 2008, Extracting principle components for discriminant analysis of fMRI images, Proc ICASSP
- Miyata, 2007, Interfrontal commissural abnormality in schizophrenia: Tractography-assisted callosal parcellation, Schizophr Res, 97, 236, 10.1016/j.schres.2007.07.032 · doi:10.1016/j.schres.2007.07.032
- Correa, 2010, Multi-set canonical correlation analysis for the fusion of concurrent single trial ERP and functional MRI, Neuroimage, 50, 1438, 10.1016/j.neuroimage.2010.01.062 · DOI
- Sui, 2011, Discriminating schizophrenia and bipolar disorder by fusing fMRI and DTI in a multimodal CCA+ joint ICA model, Neuroimage, 57, 839, 10.1016/j.neuroimage.2011.05.055 · DOI · Paper page
- Plis SM, Hjelm RD, Salakhutdinov RR, Calhoun VD (2014): Deep learning models for brain imaging: Model depth enhances discovery power. Proceedings of the Organization of Human Brain Mapping Annual Meeting, June 8--12, Hamburg, Germany.
- Castro E, Hjelm D, Plis S, Dinh L, Turner J, Calhoun VD (2015): Independent component estimation of simulated structural magnetic resonance imaging data using deep learning. Presented at the IEEE Machine Learning for Signal Processing Workshop, August 25--29, Boston, Massachusetts.
- Srivastava N, Salakhutdinov RR (2012): Multimodal learning with deep Boltzmann machines. In Bartlett P, Pereira FCN, Burges CJC, Bottou L, Weinberger KQ, editors. Advances in Neural Information Processing Systems 25, 2231–2239.
- Amin, 2015, A deep-learning approach to translate between brain structure and brain function, Pattern Recognition in NeuroImaging Conference, June 10--12, Palo Alto, California.
- Sui, 2012, A selective review of multimodal fusion methods in schizophrenia, Front Hum Neurosci, 6, 27, 10.3389/fnhum.2012.00027 · DOI
- Segall, 2012, Correspondence between structure and function in the human brain at rest, Front Neuroinform, 6, 10, 10.3389/fninf.2012.00010 · DOI · Paper page
- Michael, 2010, Does function follow form? Methods to fuse structural and functional brain images show decreased linkage in schizophrenia, Hum Brain Mapp, 49, 2626
- Smieskova, 2011, Different duration of at-risk mental state associated with neurofunctional abnormalities. A multimodal imaging study, Hum Brain Mapp, 33, 2281, 10.1002/hbm.21360 · DOI
- Xu, 2009, Joint source based morphometry identifies linked gray and white matter group differences, Neuroimage, 44, 777, 10.1016/j.neuroimage.2008.09.051 · DOI
Cited by
Single subject prediction of brain disorders in neuroimaging: Promises and pitfalls
Arbabshirani, M. R., Plis, S., Sui, J., & Calhoun, V. D. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2016.02.079 · Cited by 912
Self-citationVieira, S., Pinaya, W. H. L., & Mechelli, A. (2017) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2017.01.002 · Cited by 656
A Comprehensive Survey of the Internet of Things (IoT) and AI-Based Smart Healthcare
Alshehri, F., & Muhammad, G. (2021) · IEEE Access · doi:10.1109/access.2020.3047960 · Cited by 348
Adolescent Tuning of Association Cortex in Human Structural Brain Networks
Váša, F., Seidlitz, J., Romero-Garcia, R., Whitaker, K. J., Rosenthal, G., Vértes, P. E., Shinn, M., Alexander-Bloch, A., Fonagy, P., Dolan, R. J., et al. (2018) · Cerebral Cortex · doi:10.1093/cercor/bhx249 · Cited by 340
Du, Y., Fu, Z., & Calhoun, V. D. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00525 · Cited by 332
Self-citation
Metadata
- Authors
- Calhoun, V. D., & Sui, J.
- Venue
- Biological Psychiatry: Cognitive Neuroscience and Neuroimaging
- Publisher
- Elsevier BV
- Published
- 2016-05
- Volume
- 1
- Issue
- 3
- Pages
- 230-244
- ISSN
- 2451-9022 (ISSN-L)
- OpenAlex
- W2252433728
- Cited by
- 461
- References
- 154
- FWCI
- 22.13
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Multimodal Fusion of Brain Imaging Data: A Key to Finding the Missing Link(s) in Complex Mental Illness. https://app.cochat.ai/papers/10.1016/j.bpsc.2015.12.005
- The paper
- Calhoun, V. D., & Sui, J. (2016). Multimodal Fusion of Brain Imaging Data: A Key to Finding the Missing Link(s) in Complex Mental Illness. Biological Psychiatry: Cognitive Neuroscience and Neuroimaging. https://doi.org/10.1016/j.bpsc.2015.12.005
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.