Brain Decoding from Functional MRI Using Long Short-Term Memory Recurrent Neural Networks
Open accessAbstract
Decoding brain functional states underlying different cognitive processes using multivariate pattern recognition techniques has attracted increasing interests in brain imaging studies. Promising performance has been achieved using brain functional connectivity or brain activation signatures for a variety of brain decoding tasks. However, most of existing studies have built decoding models upon features extracted from imaging data at individual time points or temporal windows with a fixed interval, which might not be optimal across different cognitive processes due to varying temporal durations and dependency of different cognitive processes. In this study, we develop a deep learning based framework for brain decoding by leveraging recent advances in sequence modeling using long short-term memory (LSTM) recurrent neural networks (RNNs). Particularly, functional profiles extracted from task functional imaging data based on their corresponding subject-specific intrinsic functional networks are used as features to build brain decoding models, and LSTM RNNs are adopted to learn decoding mappings between functional profiles and brain states. We evaluate the proposed method using task fMRI data from the HCP dataset, and experimental results have demonstrated that the proposed method could effectively distinguish brain states under different task events and obtain higher accuracy than conventional decoding models.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
RNNs may be better suited for characterizing longitudinal data.
“Recently, deep learning techniques built upon recurrent neural networks (RNNs) with a long short term memory (LSTM) [17] structure have achieved remarkable advances in sequence modeling, such as machine translation and functional MRI modeling [18, 19], indicating RNNs might be better tools for characterizing longitudinal data.”
LSTM-based RNNs have made notable advances in sequence modeling, including machine translation and functional MRI modeling.
“Recently, deep learning techniques built upon recurrent neural networks (RNNs) with a long short term memory (LSTM) [17] structure have achieved remarkable advances in sequence modeling, such as machine translation and functional MRI modeling [18, 19], indicating RNNs might be better tools for characterizing longitudinal data.”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Jenkinson, M., Beckmann, C. F., Behrens, T. E. J., Woolrich, M. W., & Smith, S. M. (2012) · NeuroImage · doi:10.1016/j.neuroimage.2011.09.015 · Cited by 11,314 · see citing works
The minimal preprocessing pipelines for the Human Connectome Project
Glasser, M. F., Sotiropoulos, S. N., Wilson, J. A., Coalson, T. S., Fischl, B., Andersson, J. L., Xu, J., Jbabdi, S., Webster, M., Polimeni, J. R., et al. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.04.127 · Cited by 5,253 · see citing works
Correspondence of the brain's functional architecture during activation and rest
Smith, S. M., Fox, P. T., Miller, K. L., Glahn, D. C., Fox, P. M., Mackay, C. E., Filippini, N., Watkins, K. E., Toro, R., Laird, A. R., et al. (2009) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0905267106 · Cited by 5,150 · see citing works
A Critical Review of Recurrent Neural Networks for Sequence LearningPreprint
Lipton, Z. C., Berkowitz, J. A., & Elkan, C. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.00019 · Cited by 2,711 · see citing works
Decoding subject-driven cognitive states with whole-brain connectivity patterns
Shirer, W. R., Ryali, S., Rykhlevskaia, E., Menon, V., & Greicius, M. D. (2012) · Cerebral Cortex · doi:10.1093/cercor/bhr099 · Cited by 1,843 · see citing works
Function in the human connectome: task-fMRI and individual differences in behavior
Barch, D. M., Burgess, G. C., Harms, M. P., Petersen, S. E., Schlaggar, B. L., Corbetta, M., Glasser, M. F., Curtiss, S., Dixit, S., Feldt, C., et al. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.05.033 · Cited by 1,529 · see citing works
Deconvolving BOLD activation in event-related designs for multivoxel pattern classification analyses
Mumford, J. A., Turner, B. O., Ashby, F. G., & Poldrack, R. A. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.08.076 · Cited by 729 · see citing works
Davatzikos, C. A., Ruparel, K., Yong Fan, A., Shen, D. G., Acharyya, M., Loughead, J., Gur, R. C., & Langleben, D. D. (2005) · NeuroImage · doi:10.1016/j.neuroimage.2005.08.009 · Cited by 470 · see citing works
Decoding the Semantic Content of Natural Movies from Human Brain Activity
Huth, A. G., Lee, T., Nishimoto, S., Bilenko, N. Y., Vu, A. T., & Gallant, J. L. (2016) · Frontiers in Systems Neuroscience · doi:10.3389/fnsys.2016.00081 · Cited by 165 · see citing works
Estimating the Delay of the fMRI Response
Liao, C. H., Worsley, K. J., Poline, J.-B., Aston, J. A. D., Duncan, G. H., & Evans, A. C. (2002) · NeuroImage · doi:10.1006/nimg.2002.1096 · Cited by 155 · see citing works
Jang, H., Plis, S., Calhoun, V. D., & Lee, J. (2016) · NeuroImage · doi:10.1016/j.neuroimage.2016.04.003 · Cited by 111 · see citing works
Large-scale sparse functional networks from resting state fMRI
Li, H.-M., Satterthwaite, T., & Fan, Y. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.05.004 · Cited by 87 · see citing works
Self-citationDetecting Cognitive States from fMRI Images by Machine Learning and Multivariate Classification
Yong Fan, A., Shen, D. G., & Davatzikos, C. A. (2006) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2006.64 · Cited by 40 · see citing works
Shen, G., Zhang, J., Wang, M., Lei, D., Yang, G., Zhang, S., & Du, X. (2014) · European Journal of Neuroscience · doi:10.1111/ejn.12547 · Cited by 27 · see citing works
Decoding fMRI activity in the time domain improves classification performance
Loula, J., Varoquaux, G., & Thirion, B. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.08.018 · Cited by 15 · see citing works
Li, H., Satterthwaite, T. D., & Yong Fan, A. (2016) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2016.7493430 · Cited by 9 · see citing works
Self-citationPrincipal Sensitivity Analysis
Koyamada, S., Koyama, M., Nakae, K., & Ishii, S. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-18038-0_48 · Cited by 8 · see citing works
Wang, X., Xiao, L., Zhou, Y., et al. (2018) · arXiv (Cornell University) · Cited by 6 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Federated learning for healthcare applications
Chaddad, A., Wu, Y., & Desrosiers, C. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3325822 · Cited by 136 · see citing works
Li, H., & Fan, Y. (2019) · 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) · doi:10.1109/isbi.2019.8759397 · Cited by 83 · see citing works
Self-citation
- Authors
- Li, H., & Fan, Y.
- Affiliations
- University of Pennsylvania (US)
- ORCID
- Li, Hongming
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer International Publishing
- Published
- 2018
- Volume
- 11072
- Pages
- 320-328
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- PMC
- PMC6180332
- OpenAlex
- W2889753628
- Cited by
- 45
- References
- 21
- Citations by year
- 2026: 2 · 2025: 4 · 2024: 4 · 2023: 8 · 2022: 9 · 2021: 2 · 2020: 7 · 2019: 9
- FWCI
- 7.74
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Lecture notes in computer science (publishedVersion)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Nvidia
- National Institutes of Health
- National Institute on Drug Abuse (R01DA039215, U54DA039002)
- National Cancer Institute (R21CA223358)
- National Institute of Diabetes and Digestive and Kidney Diseases (P50DK114786)
- National Institute of Biomedical Imaging and Bioengineering (R01EB022573)
- Topics
- Life Sciences › Neuroscience › Cognitive Neuroscience › Functional Brain Connectivity Studies
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Advanced MRI Techniques and Applications
- Keywords
- brain decoding, functional MRI, long short-term memory, recurrent neural networks, multivariate pattern classification, functional profile, task-based fMRI, brain states, temporal sequence modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
