Task-specific feature extraction and classification of fMRI volumes using a deep neural network initialized with a deep belief network: Evaluation using sensorimotor tasks
Open accessThe DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
Abstract
Feedforward deep neural networks (DNNs), artificial neural networks with multiple hidden layers, have recently demonstrated a record‐breaking performance in multiple areas of applications in computer vision and speech processing. Following the success, DNNs have been applied to neuroimaging modalities including functional/structural magnetic resonance imaging (MRI) and positron‐emission tomography data. However, no study has explicitly applied DNNs to 3D whole‐brain fMRI volumes and thereby extracted hidden volumetric representations of fMRI that are discriminative for a task performed as the fMRI volume was acquired. Our study applied fully connected feedforward DNN to fMRI volumes collected in four sensorimotor tasks (i.e., left‐hand clenching, right‐hand clenching, auditory attention, and visual stimulus) undertaken by 12 healthy participants. Using a leave‐one‐subject‐out cross‐validation scheme, a restricted Boltzmann machine‐based deep belief network was pretrained and used to initialize weights of the DNN. The pretrained DNN was fine‐tuned while systematically controlling weight‐sparsity levels across hidden layers. Optimal weight‐sparsity levels were determined from a minimum validation error rate of fMRI volume classification. Minimum error rates (mean ± standard deviation; %) of 6.9 (± 3.8) were obtained from the three‐layer DNN with the sparsest condition of weights across the three hidden layers. These error rates were even lower than the error rates from the single‐layer network (9.4 ± 4.6) and the two‐layer network (7.4 ± 4.1). The estimated DNN weights showed spatial patterns that are remarkably task‐specific, particularly in the higher layers. The output values of the third hidden layer represented distinct patterns/codes of the 3D whole‐brain fMRI volume and encoded the information of the tasks as evaluated from representational similarity analysis. Our reported findings show the ability of the DNN to classify a single fMRI volume based on the extraction of hidden representations of fMRI volumes associated with tasks across multiple hidden layers. Our study may be beneficial to the automatic classification/diagnosis of neuropsychiatric and neurological diseases and prediction of disease severity and recovery in (pre‐) clinical settings using fMRI volumes without requiring an estimation of activation patterns or ad hoc statistical evaluation. HighlightsDeep neural network (DNN) was proposed to classify fMRI volume of sensorimotor tasks.DNN weights were optimized via non‐zero value percentage in cross‐validation (CV) framework.Classification performance was superior from three‐layer DNN than one‐/two‐/four‐layer DNNs.Weight/hidden representations were highly task‐specific from higher than lower hidden layers.Sparsity of weights between the input and first hidden layer enhanced the performance.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Cortes, C., & Vapnik, V. (1995) · Machine Learning · doi:10.1007/bf00994018 · Cited by 59,421
Neural Networks for Pattern Recognition
Bishop, C. M. (1995) · doi:10.1093/oso/9780198538493.001.0001 · Cited by 12,287
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382 · cited 7×
“…Network, 1101 Yale Blvd NE, d Cognitive Engineering, Korea epublic of Korea. onghwan_lee@korea.ac.kr extraction and/or classification of multimedia data, such as image (Krizhevsky et al., 2012) and speech (Hinton et al., 2012a) data, and have shown unprecedented classification performance.”
BackgroundMethodInfluential3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,235
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370 · cited 2×
“This activation maximization method would be particularly useful for characterizing the learned features of deep convolutional neural networks which consist of several pooling layers (Simonyan et al., 2013)whichwould not be straightforward using the linear combination of weight matrices.”
MethodTraining Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · cited 10×
“The visible and hidden nodes of the RBM have undirected connections, and there is no connection either between the visible nodes or between the hidden nodes (Hinton, 2002).”
BackgroundMethodInfluentialGreedy layer-wise training of deep networks
Bengio, Y., Lamblin, P., Popovici, D., & Larochelle, H. (2007) · Advances in Neural Information Processing Systems 19 · doi:10.7551/mitpress/7503.003.0024 · Cited by 4,841 · cited 3×
“…belief network (DBN) using the restricted Boltzmann machine (RBM) (Hinton, 2002; Hinton and Salakhutdinov, 2006) and the stacked auto-encoder (SAE) (Bengio et al., 2007), artificial deep neural networks (DNNs) have been gainfully utilized for the feature h Network, 1101 Yale Blvd NE, d Cognitive…”
MethodDistributed and overlapping representations of faces and objects in ventral temporal cortex
Haxby, J. V., Gobbini, M. I., Furey, M. L., Ishai, A., Schouten, J. L., & Pietrini, P. (2001) · Science · doi:10.1126/science.1063736 · Cited by 3,940 · cited 3×
“…layer representations can be treated as patterns and that several maximally active hidden nodes to the task would not critically alter the structure of the patterns, as similarly shown by Haxby et al. (2001) for the multivoxel-pattern analysis of visual object recognition using BOLD fMRI activity.”
BackgroundMethodRepresentational similarity analysis-connecting the branches of systems neuroscience
Kriegeskorte, N. (2008) · Frontiers in Systems Neuroscience · doi:10.3389/neuro.06.004.2008 · Cited by 3,355
Cited by
Deep Learning for Spatio-Temporal Data Mining: A Survey
Wang, S., Cao, J., & Yu, P. S. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.3025580 · Cited by 822
Du, Y., Fu, Z., & Calhoun, V. D. (2018) · Frontiers in Neuroscience · doi:10.3389/fnins.2018.00525 · Cited by 332
Self-citationSchulz, M.-A., Yeo, B. T. T., Vogelstein, J. T., Mourao-Miranada, J., Kather, J. N., Kording, K., Richards, B., & Bzdok, D. (2020) · Nature Communications · doi:10.1038/s41467-020-18037-z · Cited by 315
Deep neural networks in psychiatry
Durstewitz, D., Koppe, G., & Meyer-Lindenberg, A. (2019) · Molecular Psychiatry · doi:10.1038/s41380-019-0365-9 · Cited by 278
Inference in the age of big data: Future perspectives on neuroscience
Bzdok, D., & Yeo, B. T. T. (2017) · NeuroImage · doi:10.1016/j.neuroimage.2017.04.061 · Cited by 189
“Therefore, despite the growing literature applying DNNs to neuroscience applications (e.g., Kim et al., 2014; Plis et al., 2014; Khaligh-Razavi et al., 2014; Yamins et al., 2014; Zhang et al., 2015; Güçlü et al., 2015; Eickenberg et al., 2016; Jang et al., 2017), exploiting DNNs in neuroscience may be hindered by the…”
Deep Learning Intervention for Health Care Challenges: Some Biomedical Domain Considerations
Tobore, I., Li, J., Yuhang, L., Al-Handarish, Y., Kandwal, A., Nie, Z., & Wang, L. (2019) · JMIR mHealth and uHealth · doi:10.2196/11966 · Cited by 180
Wen, D., Wei, Z., Zhou, Y., Li, G., Zhang, X., & Han, W. (2018) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2018.00023 · Cited by 93
“To reduce the number of features extracted, a deep belief network (DBN) was used to initialize the parameters and sparse pre-training was performed on weight (Jang et al., 2017).”
MethodAmini, M., Pedram, M. M., Moradi, A., & Ouchani, M. (2021) · Computational and Mathematical Methods in Medicine · doi:10.1155/2021/5514839 · Cited by 84
Brain Decoding from Functional MRI Using Long Short-Term Memory Recurrent Neural Networks
Li, H., & Fan, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00931-1_37 · Cited by 45
Wang, S., Duan, F., & Zhang, M. (2020) · Applied Sciences · doi:10.3390/app10217465 · Cited by 10
“Even so, for hundreds of thousands of voxel signals in multiple brain regions, manual feature extraction is a very complicated work, although various feature extraction methods have been proposed [12,13].”
Background
Metadata
- Authors
- Jang, H., Plis, S. M., Calhoun, V. D., & Lee, J.-H.
- Venue
- NeuroImage
- Publisher
- Elsevier BV
- Published
- 2017-01
- Volume
- 145
- Issue
- Pt B
- Pages
- 314-328
- ISSN
- 1053-8119 (ISSN-L)
- OpenAlex
- W2325276834
- Cited by
- 110
- References
- 21
- FWCI
- 7.44
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.