Sociodemographic data and APOE-ε4 augmentation for MRI-based detection of amnestic mild cognitive impairment using deep learning systems
Open accessAbstract
Detection and diagnosis of early and subclinical stages of Alzheimer's Disease (AD) play an essential role in the implementation of intervention and prevention strategies. Neuroimaging techniques predominantly provide insight into anatomic structure changes associated with AD. Deep learning methods have been extensively applied towards creating and evaluating models capable of differentiating between cognitively unimpaired, patients with Mild Cognitive Impairment (MCI) and AD dementia. Several published approaches apply information fusion techniques, providing ways of combining several input sources in the medical domain, which contributes to knowledge of broader and enriched quality. The aim of this paper is to fuse sociodemographic data such as age, marital status, education and gender, and genetic data (presence of an apolipoprotein E (APOE)-ε4 allele) with Magnetic Resonance Imaging (MRI) scans. This enables enriched multi-modal features, that adequately represent the MRI scan visually and is adopted for creating and modeling classification systems capable of detecting amnestic MCI (aMCI). To fully utilize the potential of deep convolutional neural networks, two extra color layers denoting contrast intensified and blurred image adaptations are virtually augmented to each MRI scan, completing the Red-Green-Blue (RGB) color channels. Deep convolutional activation features (DeCAF) are extracted from the average pooling layer of the deep learning system Inception_v3. These features from the fused MRI scans are used as visual representation for the Long Short-Term Memory (LSTM) based Recurrent Neural Network (RNN) classification model. The proposed approach is evaluated on a sub-study containing 120 participants (aMCI = 61 and cognitively unimpaired = 59) of the Heinz Nixdorf Recall (HNR) Study with a baseline model accuracy of 76%. Further evaluation was conducted on the ADNI Phase 1 dataset with 624 participants (aMCI = 397 and cognitively unimpaired = 227) with a baseline model accuracy of 66.27%. Experimental results show that the proposed approach achieves 90% accuracy and 0.90 F1-Score at classification of aMCI vs. cognitively unimpaired participants on the HNR Study dataset, and 77% accuracy and 0.83 F1-Score on the ADNI dataset.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1× · see citing works
“This pretrained deep convolutional neural network models were designed to extract among other features, color information in three separate channels (RGB) from the images [57, 58].”
MethodHochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 1× · see citing works
“RNNs are mostly used for modeling long-range dependencies, where future events are predicted with past events [67] and has proven to be successful for several research topics such as medical question and answering [68].”
BackgroundLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126 · see citing works
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · cited 1× · see citing works
“For comparison purposes, additional DeCAF features were extracted using a medical context pre-trained DenseNet-121 model [65] on the ChestX-Ray8 database [16].”
MethodLIBSVM: A library for support vector machines
Chang, C.-C., & Lin, C.-J. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961199 · Cited by 40,918 · cited 1× · see citing works
“[28] to fuse decision values from a multiclass linear kernel Support Vector Machine (SVM) [29] and Random Forest [30] classifiers to predict the modality of subfigures extracted from the PubMed Central (PMC) Open Access Subset [31].”
MethodImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 5× · see citing works
“Inception-v3 [39], which is pre-trained on the ImageNet [64].”
BackgroundMethodInfluentialRethinking the Inception Architecture for Computer Vision
Szegedy, C., Vanhoucke, V., Ioffe, S., Shlens, J., & Wojna, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.308 · Cited by 32,138 · cited 5× · see citing works
“Analyzing the ablation study with DeCAF representation extracted from Inception-v3 [39], the following findings can be taken:”
BackgroundMethodInfluentialGrad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513 · see citing works
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · cited 4× · see citing works
“• Transfer Learning is utilized for creating deep convolutional activation features.”
BackgroundMethodInfluentialMcKhann, G. M., Knopman, D. S., Chertkow, H., Hyman, B. T., Jack, C. R., Kawas, C. H., Klunk, W. E., Koroshetz, W. J., Manly, J. J., Mayeux, R., et al. (2011) · Alzheimer's & Dementia · doi:10.1016/j.jalz.2011.03.005 · Cited by 18,657 · see citing works
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · cited 1× · see citing works
“The effective characteristic of LSTM is the ability to accumulate state information, as information of every new input is accumulated onto previous input [69, 70].”
Background(2016) · Cited by 11,679 · see citing works
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · cited 1× · see citing works
“Deep learning techniques [10] have improved prediction accuracies in object detection [11], speech recognition [12] and in medical imaging [13, 14].”
BackgroundAlbert, M. S., DeKosky, S. T., Dickson, D., Dubois, B., Feldman, H. H., Fox, N. C., Gamst, A., Holtzman, D. M., Jagust, W. J., Petersen, R. C., et al. (2011) · Alzheimer's & Dementia · doi:10.1016/j.jalz.2011.03.008 · Cited by 10,152 · cited 1× · see citing works
“[51] were not yet published when the sub-study started, the Winblad et al.”
Mild cognitive impairment as a diagnostic entity
Petersen, R. C. (2004) · Journal of Internal Medicine · doi:10.1111/j.1365-2796.2004.01388.x · Cited by 7,864 · cited 1× · see citing works
“Mild cognitive impairment (MCI) is a clinical entity that describes the stage between cognitive changes of normal aging and dementia [3, 4].”
BackgroundA Non-Local Algorithm for Image Denoising
Buades, A., Coll, B., & Morel, J.-M. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.38 · Cited by 7,616 · cited 1× · see citing works
“This is a digital image denoising method, based on a non-local averaging of all present pixels in an image [63].”
MethodHow transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061 · see citing works
Winblad, B., Palmer, K., Kivipelto, M., Jelic, V., Fratiglioni, L., Wahlund, L. ‐., Nordberg, A., Bäckman, L., Albert, M., Almkvist, O., et al. (2004) · Journal of Internal Medicine · doi:10.1111/j.1365-2796.2004.01380.x · Cited by 4,946 · cited 1× · see citing works
“[52] MCI criteria were used to diagnose aMCI.”
MethodWang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.369 · Cited by 4,616 · see citing works
Visual categorization with bags of keypoints
Csurka, G. (2004) · Cited by 4,379 · see citing works
The unified medical language system (UMLS): integrating biomedical terminology
Bodenreider, O. (2004) · Nucleic Acids Research · doi:10.1093/nar/gkh061 · Cited by 4,356 · cited 1× · see citing works
“In [32], automatic generated semantic information from Unified Modeling Language System (UMLS) [33] concepts were combined with Bag-of-Keypoints representations [34] computed with Dense (dSIFT) [35] features and applyed for predicting image modality, body region examined, orientation of the image and biological system…”
MethodOn the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310 · cited 1× · see citing works
“The effective characteristic of LSTM is the ability to accumulate state information, as information of every new input is accumulated onto previous input [69, 70].”
BackgroundContrast Limited Adaptive Histogram Equalization
Zuiderveld, K. (1994) · Graphics Gems · doi:10.1016/b978-0-12-336156-1.50061-6 · Cited by 4,192 · see citing works
Adaptive histogram equalization and its variations
Pizer, S. M., Amburn, E. P., Austin, J. D., Cromartie, R., Geselowitz, A., Greer, T., Romeny, B. M. T. H., Zimmerman, J. B., & Zuiderveld, K. J. (1987) · Computer Vision Graphics and Image Processing · doi:10.1016/s0734-189x(87)80186-x · Cited by 4,050 · cited 1× · see citing works
“applicable and has demonstrated effectiveness, especially for medical images [62].”
BackgroundDeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019 · cited 2× · see citing works
“For visual representation, deep convolutional activation features (DeCAF) [37] were chosen.”
MethodA Bayesian Hierarchical Model for Learning Natural Scene Categories
Fei-Fei, L., & Perona, P. (2005) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2005.16 · Cited by 3,657 · see citing works
Wide & Deep Learning for Recommender Systems
Cheng, H.-T., Koç, L., Harmsen, J., Shaked, T., Chandra, T., Aradhye, H., Anderson, G., Corrado, G. S., Chai, W., Ispir, M., et al. (2016) · Workshop on Deep Learning for Recommender Systems (DLRS) · doi:10.1145/2988450.2988454 · Cited by 3,547 · cited 6× · see citing works
“For the k-fold cross validation samples with n = 99 participants on the HNR Study dataset, the Wide and Deep input method achieved a higher sensitivity rate.”
MethodInfluentialAlbert, M., DeKosky, S. T., Dickson, D. W., Dubois, B., Feldman, H., Fox, N. C., Gamst, A. C., Holtzman, D. M., Jagust, W. J., Petersen, R. C., et al. (2013) · FOCUS The Journal of Lifelong Learning in Psychiatry · doi:10.1176/appi.focus.11.1.96 · Cited by 2,630 · see citing works
Alzheimer's Disease Neuroimaging Initiative (ADNI)
Petersen, R. C., Aisen, P. S., Beckett, L. A., Donohue, M. C., Gamst, A. C., Harvey, D. J., Jack, C. R., Jagust, W. J., Shaw, L. M., Toga, A. W., et al. (2010) · Neurology · doi:10.1212/wnl.0b013e3181cb3e25 · Cited by 2,572 · see citing works
International Standard Classification of Education, ISCED 1997
Scientific (2003) · Springer eBooks · doi:10.1007/978-1-4419-9186-7_10 · Cited by 2,510 · see citing works
Peter, H., Penzel, T., Peter, J. H., & Peter, J. G. (2025) · Springer Reference Medizin · doi:10.1007/978-3-662-65186-5_302294 · Cited by 2,278 · see citing works
Reynolds, C. R., & Powel, J. (1988) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/3.4.397 · Cited by 2,115 · see citing works
An overview of deep learning in medical imaging focusing on MRI
Lundervold, A. S., & Lundervold, A. (2019) · Zeitschrift für Medizinische Physik · doi:10.1016/j.zemedi.2018.11.002 · Cited by 2,058 · cited 1× · see citing works
“The limitation of the usage of 2D slices instead of the 3D MRI scans will be experimented in further work, as positive results have been reported in the overview survey of deep learning techniques for MRI [56].”
Background2018 Alzheimer's disease facts and figures
Alzheimer's Association (2018) · Alzheimer's & Dementia · doi:10.1016/j.jalz.2018.02.001 · Cited by 1,912 · see citing works
Multimodal classification of Alzheimer's disease and mild cognitive impairment
Zhang, D., Wang, Y., Zhou, L., Yuan, H., & Shen, D. G. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.01.008 · Cited by 1,312 · see citing works
Multi-scale Orderless Pooling of Deep Convolutional Activation Features
Gong, Y., Wang, L., Guo, R., & Lazebnik, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10584-0_26 · Cited by 1,043 · see citing works
An overview of deep learning in medical imaging focusing on MRI
Lundervold, A. S., & Lundervold, A. (2018) · NORA - Norwegian Open Research Archives · Cited by 1,012 · see citing works
Chollet, F. (2017) · Cited by 742 · see citing works
Schmermund, A., Möhlenkamp, S., Stang, A., Grönemeyer, D. H. W., Seibel, R. M. M., Hirche, H., Mann, K., Siffert, W., Lauterbach, K. W., Siegrist, J., et al. (2002) · American Heart Journal · doi:10.1067/mhj.2002.123579 · Cited by 576 · see citing works
MISRA, C., FAN, Y., & DAVATZIKOS, C. (2009) · NeuroImage · doi:10.1016/j.neuroimage.2008.10.031 · Cited by 552 · see citing works
Abrão, M., Gonçalves, M. O. D. C., Dias, J., Podgaec, S., Chamié, L., & Blasbalg, R. (2007) · Human Reproduction · doi:10.1093/humrep/dem187 · Cited by 520 · cited 1× · see citing works
“Deep learning techniques [10] have improved prediction accuracies in object detection [11], speech recognition [12] and in medical imaging [13, 14].”
BackgroundHigher risk of progression to dementia in mild cognitive impairment cases who revert to normal
Roberts, R. O., Knopman, D. S., Mielke, M. M., Cha, R. H., Pankratz, V. S., Christianson, T. J. H., Geda, Y. E., Boeve, B. F., Ivnik, R. J., Tangalos, E. G., et al. (2014) · Neurology · doi:10.1212/wnl.0000000000000055 · Cited by 487 · cited 1× · see citing works
“Individuals with aMCI have a higher risk to develop AD dementia, but some individuals also revert to normal or stay stable without reaching the AD dementia stage [5].”
BackgroundSpasov, S., Passamonti, L., Duggento, A., Lió, P., & Toschi, N. (2019) · NeuroImage · doi:10.1016/j.neuroimage.2019.01.031 · Cited by 429 · cited 1× · see citing works
“[18] proposed a MCI to AD conversion and AD vs.”
BackgroundPredicting alzheimer’s disease progression using multi-modal deep learning approach
Lee, G., Nho, K., Kang, B., Sohn, K.-A., Kim, D., Initiative, F. A. D. N., Weiner, M. W., Aisen, P., Petersen, R., Jack, C. R., et al. (2019) · Scientific Reports · doi:10.1038/s41598-018-37769-z · Cited by 406 · see citing works
Multi-Modality Cascaded Convolutional Neural Networks for Alzheimer’s Disease Diagnosis
Initiative, T. A. D. N., Liu, M., Cheng, D., Wang, K., & Wang, Y. (2018) · Neuroinformatics · doi:10.1007/s12021-018-9370-4 · Cited by 404 · see citing works
Deep learning of feature representation with multiple instance learning for medical image analysis
Xu, Y. L., Mo, T., Feng, Q., Zhong, P., Lai, M., & Chang, E. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6853873 · Cited by 385 · cited 1× · see citing works
“Deep learning techniques [10] have improved prediction accuracies in object detection [11], speech recognition [12] and in medical imaging [13, 14].”
BackgroundToward open sharing of task-based fMRI data: the OpenfMRI project
Poldrack, R. A., Barch, D. M., Mitchell, J. P., Wager, T. D., Wagner, A. D., Devlin, J. T., Cumba, C., Koyejo, S., & Milham, M. P. (2013) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2013.00012 · Cited by 341 · cited 1× · see citing works
“However, a major concern in the medical domain is the insufficient number of large datasets such as ChestX-Ray8 database [16] and the OpenfMRI project [17].”
BackgroundDiagnosis of Alzheimer’s Disease via Multi-Modality 3D Convolutional Neural Network
Huang, Y., Xu, J., Zhou, Y., Tong, T., Zhuang, X., & (ADNI), T. A. D. N. I. (2019) · Frontiers in Neuroscience · doi:10.3389/fnins.2019.00509 · Cited by 303 · see citing works
[Paper] Visual Instance Retrieval with Deep Convolutional Networks
Razavian, A. S., Sullivan, J., Carlsson, S., & Maki, A. (2016) · ITE Transactions on Media Technology and Applications · doi:10.3169/mta.4.251 · Cited by 276 · see citing works
PubMed Central: The GenBank of the published literature.
Roberts, R. J. (2001) · Proceedings of the National Academy of Sciences of the United States of America · doi:10.1073/pnas.98.2.381 · Cited by 261 · cited 1× · see citing works
“[28] to fuse decision values from a multiclass linear kernel Support Vector Machine (SVM) [29] and Random Forest [30] classifiers to predict the modality of subfigures extracted from the PubMed Central (PMC) Open Access Subset [31].”
MethodSørensen, L., Igel, C., Pai, A., Balas, I., Anker, C., Lillholm, M., & Nielsen, M. (2017) · NeuroImage: Clinical · doi:10.1016/j.nicl.2016.11.025 · Cited by 179 · see citing works
Samper‐Gonzàlez, J., Burgos, N., Bottani, S., Fontanella, S., Lu, P., Marcoux, A., Routier, A., Guillon, J., Bacci, M., Wen, J., et al. (2018) · NeuroImage · doi:10.1016/j.neuroimage.2018.08.042 · Cited by 171 · see citing works
DeepHand: Robust Hand Pose Estimation by Completing a Matrix Imputed with Deep Features
Sinha, A., Choi, C., & Karthik Ramani, A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.450 · Cited by 161 · see citing works
[Alzheimer's disease Neuroimaging Initiative (ADNI)].
Iwatsubo, T. (2011) · PubMed · Cited by 141 · see citing works
Kalpathy–Cramer, J., de Herrera, A. G. S., Demner‐Fushman, D., Antani, S., Bedrick, S. D., & Müller, H. (2014) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2014.03.004 · Cited by 130 · cited 1× · see citing works
“[8] adopted a combination of visual representation with text information extracted from captions to classify and predict the image modality at the ImageCLEF2016 Medical Task [9].”
MethodTeare, P., Fishman, M. D. C., Benzaquen, O., Toledano, E., & Elnekave, E. (2017) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-017-9993-2 · Cited by 108 · cited 1× · see citing works
“Color input enhancement have aided to substantially improve prediction accuracy from 86% to 92% for the detection of malignancy in digital mammography images [59] and approximately 3% for structuring 2D x-rays according to imaging technique modality, anatomical region and biological systems examined, which is applied…”
BackgroundImproving Committee Diagnosis with Resampling Techniques
Parmanto, B., Munro, P., & Doyle, H. R. (1995) · Cited by 92 · see citing works
Visual Instance Retrieval with Deep Convolutional NetworksPreprint
Razavian, A. S., Sullivan, J., Carlsson, S., & Maki, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.6574 · Cited by 81 · see citing works
Disentangling factors that shape the gut microbiota in German Shepherd dogs
Vilson, Å., Ramadan, Z., Li, Q., Hedhammar, Å. A., Reynolds, A. J., Spears, J. K., Labuda, J., Pelker, R., Björkstén, B., Dicksved, J., et al. (2018) · PLoS ONE · doi:10.1371/journal.pone.0193507 · Cited by 50 · see citing works
Pelka, O., Nensa, F., & Friedrich, C. M. (2018) · PLoS ONE · doi:10.1371/journal.pone.0206229 · Cited by 32 · cited 1× · see citing works
“Color input enhancement have aided to substantially improve prediction accuracy from 86% to 92% for the detection of malignancy in digital mammography images [59] and approximately 3% for structuring 2D x-rays according to imaging technique modality, anatomical region and biological systems examined, which is applied…”
BackgroundWege, N., Jokisch, M., Siegrist, J., Dragano, N., Erbel, R., Jöckel, K., Moebus, S., Weimar, C., & on behalf of the Heinz Nixdorf Recall Study Investigative Group (2011) · Neuroepidemiology · doi:10.1159/000328262 · Cited by 30 · cited 3× · see citing works
“Five years after baseline (2005-2008, n = 4,145), the first follow-up of the HNR Study was conducted and included a short cognitive assessment (for details see [46]).”
BackgroundMethodJokisch, M., on behalf of the Heinz Nixdorf Recall Study Investigative Group, Gerwig, M., Wege, N., Siegrist, J., Mann, K., Bröcker‐Preuß, M., Dragano, N., Moebus, S., Jöckel, K., et al. (2012) · Journal of Alzheimer s Disease · doi:10.3233/jad-2011-111352 · Cited by 26 · see citing works
FHDO Biomedical Computer Science Group at Medical Classification Task of ImageCLEF 2015.
Pelka, O., & Friedrich, C. M. (2015) · CLEF (Working Notes) · Cited by 25 · see citing works
Classification of Alzheimer’s Disease using volumetric features of multiple MRI scans
Bloch, L., & Friedrich, C. M. (2019) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2019.8857188 · Cited by 16 · see citing works
Characterizing Alzheimer's Disease With Image and Genetic Biomarkers Using Supervised Topic Models
Yang, J., Feng, X., Laine, A. F., & Angelini, E. D. (2019) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2019.2928831 · Cited by 13 · cited 1× · see citing works
“[19] to characterize Alzheimer’s Disease using image and categorical genetic features, based on supervised topic modeling.”
Automated Medical Image Modality Recognition by Fusion of Visual and Text Information
Codella, N., Connell, J. H., Pankanti, S., Merler, M., & Smith, J. R. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10470-6_61 · Cited by 12 · cited 1× · see citing works
“[6], automated medical image modality recognition was achieved by fusing visual and text information.”
Modality prediction of biomedical literature images using multimodal feature representation
Pelka, O., & Friedrich, C. M. (2016) · German Medical Science (German Research Foundation) · doi:10.3205/mibe000166 · Cited by 11 · cited 1× · see citing works
“[8] adopted a combination of visual representation with text information extracted from captions to classify and predict the image modality at the ImageCLEF2016 Medical Task [9].”
MethodAdopting Semantic Information of Grayscale Radiographs for Image Classification and Retrieval
Pelka, O., Nensa, F., & Friedrich, C. M. (2018) · International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC) · doi:10.5220/0006732301790187 · Cited by 10 · cited 1× · see citing works
“In [32], automatic generated semantic information from Unified Modeling Language System (UMLS) [33] concepts were combined with Bag-of-Keypoints representations [34] computed with Dense (dSIFT) [35] features and applyed for predicting image modality, body region examined, orientation of the image and biological system…”
MethodVariations on Branding with Text Occurrence for Optimized Body Parts Classification
Pelka, O., Nensa, F., & Friedrich, C. M. (2019) · doi:10.1109/embc.2019.8857478 · Cited by 5 · cited 3× · see citing works
“[38], where automatic generated keywords were incorporated onto x-ray and biomedical images.”
BackgroundMethodInfluentialMönninghoff, C., Jokisch, M., Kraff, O., Geisel, M. H., Jöckel, K., Erbel, R., Weimar, C., & Wanke, I. (2015) · Academic Radiology · doi:10.1016/j.acra.2015.04.008 · Cited by 4 · see citing works
Optimizing Body Region Classification with Deep Convolutional Activation Features
Pelka, O., Nensa, F., & Friedrich, C. M. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11018-5_60 · Cited by 4 · cited 1× · see citing works
“[36] by using Deep convolutional activation features (DeCAF) [37] to obtain an optimized medical image body region classification.”
Method
Flagged first, then by citations
Research works printed but not linked: 5
- [7]Valavanis L, Stathopoulos S, Kalamboukis T. IPL at CLEF 2016 Medical Task. In: Working Notes of CLEF 2016—Conference and Labs of the Evaluation forum, Évora, Portugal, 5-8 September, 2016.; 2016. p. 413–420. Available from: http://ceur-ws.org/Vol-1609/16090413.pdf.
- [27]de Herrera AGS, Müller H, Bromuri S. Overview of the ImageCLEF 2015 Medical Classification Task. In: Working Notes of CLEF 2015—Conference and Labs of the Evaluation forum, Toulouse, France, September 8-11, 2015. vol. 1391; 2015. Available from: http://ceur-ws.org/Vol-1391/172-CR.pdf.
- [52]Winblad B, Palmer K, Kivipelto M, Jelic V, Fratiglioni L, Wahlund LO, et al Mild cognitive impairment—Beyond controversies, towards a consensus: Report of the International Working Group on Mild Cognitive Impairment. Journal of internal medicine. 2004; 256: 240– 6. 10.1111/j.1365-2796.2004.01380.x 15324367 · DOI · Paper page
- [65]Huang G, Liu Z, Van Der Maaten L, Weinberger KQ. Densely Connected Convolutional Networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, CVPR, Honolulu, USA, July 22-25, 2017; 2017. p. 2261–2269.
- [68]Ben Abacha A, Hasan SA, Datla VV, Liu J, Demner-Fushman D, Müller H. VQA-Med: Overview of the Medical Visual Question Answering Task at ImageCLEF 2019. In: Working Notes of CLEF 2019—Conference and Labs of the Evaluation Forum, Lugano, Switzerland, September 9-12, 2019.; 2019. Available from: http://ceur-ws.org/Vol-2380/paper_272.pdf.
Cited by
Sorted by citations
Multi-modality approaches for medical support systems: A systematic review of the last decade
Salvi, M., Loh, H. W., Seoni, S., Barua, P. D., García, S., Molinari, F., & Acharya, U. R. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102134 · Cited by 144 · see citing works
“Pelka et al. [26] fused sociodemographic data, APOE genotype, and MRI data using an LSTM model to improve the detection of amnestic mild cognitive impairment.”
MethodSchouten, D., Nicoletti, G., Dille, B., Chia, C., Vendittelli, P., Schuurmans, M., Litjens, G., & Khalili, N. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103621 · Cited by 108 · see citing works
Bloch, L., Friedrich, C. M., & Initiative, F. T. A. D. N. (2021) · Alzheimer's Research & Therapy · doi:10.1186/s13195-021-00879-4 · Cited by 68 · see citing works
Self-citationCombining 3D Image and Tabular Data via the Dynamic Affine Feature Map Transform
Pölsterl, S., Wolf, T. N., & Wachinger, C. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-87240-3_66 · Cited by 68 · see citing works
DAFT: A universal module to interweave tabular data and 3D images in CNNs
Wolf, T. N., Pölsterl, S., & Wachinger, C. (2022) · NeuroImage · doi:10.1016/j.neuroimage.2022.119505 · Cited by 52 · see citing works
“By design, CNNs excel at extracting task-specific descriptors of high-dimensional 3D image data.”
MethodInfluentialYousefi, M., Akhbari, M., Mohamadi, Z., Karami, S., Dasoomi, H., Atabi, A., Sarkeshikian, S. A., Dehaki, M. A., Bayati, H., Mashayekhi, N., et al. (2024) · Frontiers in Neurology · doi:10.3389/fneur.2024.1413071 · Cited by 51 · see citing works
Prediction of Alzheimer’s Disease Progression Based on Magnetic Resonance Imaging
Zhou, Y., Song, Z., Han, X., Li, H., & Tang, X. (2021) · ACS Chemical Neuroscience · doi:10.1021/acschemneuro.1c00472 · Cited by 46 · see citing works
Bloch, L., & Friedrich, C. M. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108029 · Cited by 45 · see citing works
Bloch, L., Friedrich, C. M., & Initiative, A. D. N. (2022) · SN Computer Science · doi:10.1007/s42979-022-01371-y · Cited by 33 · see citing works
Explainable MRI-Based Ensemble Learnable Architecture for Alzheimer’s Disease Detection
Adeniran, O., Ojeme, B., Ajibola, T. E., Peter, O. O. E., Ajala, A. O., Rahman, M. M., & Khalifa, F. (2025) · Algorithms · doi:10.3390/a18030163 · Cited by 20 · see citing works
Kim, H. H., Hillis, A. E., & Themistocleous, C. K. (2024) · Brain Sciences · doi:10.3390/brainsci14070652 · Cited by 15 · see citing works
“This highlights the clinical need for a robust, reliable system for classifying aMCI and naMCI [18].”
BackgroundO’Connell, S., Cannon, D. M., & Broin, P. Ó. (2023) · Human Brain Mapping · doi:10.1002/hbm.26521 · Cited by 10 · see citing works
“…model was trained on the whole dataset and the weights from that model were then used for transfer learning of another model with the same dataset, leading to potential leakage or overfitting (Ahmed et al., 2020; Folego et al., 2020; Lin et al., 2018; Mendoza-Léon et al., 2020; Pelka et al., 2020).”
BackgroundMethodBiswas, M., Mahbub, M. K., & Miah, M. A. M. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-07005-1_5 · Cited by 7 · see citing works
Almatrafi, S., Abbas, Q., & Ibrahim, M. E. A. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19104-z · Cited by 6 · see citing works
Rahman, M. M., Calhoun, V. D., & Plis, S. (2026) · Imaging Neuroscience · doi:10.1162/imag.a.1129 · Cited by 14 · see citing works
Classifying Alzheimer’s Disease Neuropathology Using Clinical and MRI Measurements
Zhuang, X., Cordes, D., Bender, A. R., Nandy, R., Oh, E. C., Kinney, J., Caldwell, J., Cummings, J. L., & Miller, J. B. (2024) · Journal of Alzheimer's Disease · doi:10.3233/jad-231321 · Cited by 5 · see citing works
Bloch, L., Borys, K., Nensa, F., & Friedrich, C. M. (2026) · NeuroImage · doi:10.1016/j.neuroimage.2026.121813 · Cited by 2 · see citing works
Pelka, O., Sigle, S., Werner, P., Schweizer, S. T., Iancu, A., Scherer, L., Kamzol, N. A., Eil, J. H., Apfelbacher, T., Seletkov, D., et al. (2025) · RöFo - Fortschritte auf dem Gebiet der Röntgenstrahlen und der bildgebenden Verfahren · doi:10.1055/a-2651-6653 · Cited by 1 · see citing works
Keine Angst vor Konkurrenz – KI ist auch nur ein Werkzeug und schon gar nicht intelligent
Nensa, F. (2022) · RöFo - Fortschritte auf dem Gebiet der Röntgenstrahlen und der bildgebenden Verfahren · doi:10.1055/a-1892-8286
Kim, H. H., Argye, H., & Themistocleous, C. K. (2024) · medRxiv · doi:10.1101/2024.02.16.24302965
Liang, C., Adalı, T., Jing Sui （Beijing Normal University）， my correct affiliation is beijing normal university, N. Q. U. O. S. A. T., Qi, S., & Calhoun, V. D. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3818859
Hybrid Deep Learning Framework for Mild Cognitive Impairment Stage Classification for Brain Health
Vinodhini, S., & Vimal, V. R. (2026) · International Conference on Intelligent Cyber Physical Systems and Internet of Things (ICoICI) · doi:10.1109/icoici69824.2026.11723601
“Pelka et al. proposed a hybrid framework for classifying amnestic MCI by combining MRI, sociodemographic data, and APOE genetic biomarkers, with an accuracy of around 90% [12].”
- Authors
- Pelka, O., Friedrich, C. M., Nensa, F., Mönninghoff, C., Bloch, L., Jöckel, K.-H., Schramm, S., Sanchez Hoffmann, S., Winkler, A., Weimar, C., Jokisch, M., & Initiative, F. T. A. D. N.
- Affiliations
- Dortmund University of Applied Sciences and Arts (DE)
- Essen University Hospital (DE)
- Clemenshospital Münster (DE)
- Institut für Medizinische Informatik, Biometrie und Epidemiologie (DE)
- Corresponding author
- Friedrich, Christoph M.
- ORCID
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2020-09-25
- Volume
- 15
- Issue
- 9
- Pages
- e0236868
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC7518632
- OpenAlex
- W3088432570
- Cited by
- 22
- References
- 72
- Citations by year
- 2026: 4 · 2025: 3 · 2024: 6 · 2023: 2 · 2022: 4 · 2021: 3
- FWCI
- 1.64
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- U.S. Department of Defense (U01 AG024904, W81XWH, W81XWH-12-2-, W81XWH1220012, AG024904)
- Foundation for the National Institutes of Health (W81XWH-12-2-0012, U01AG024904, Grant U01 AG024904, AG024904)
- Alzheimer's Association (U01 AG024904, W81XWH-12-2-0012)
- Bristol-Myers Squibb (U01 AG024904, AG024904, W81XWH-12-2-0012)
- Eli Lilly and Company (U01 AG024904, AG024904, W81XWH-12-2-0012)
- Pfizer (AG024904, W81XWH-12-2-0012, U01 AG024904)
- Biogen (W81XWH-12-2-0012, U01 AG024904, AG024904)
- University of Southern California (W81XWH-12-2-0012, U01 AG024904)
- Celgene
- F. Hoffmann-La Roche (W81XWH-12-2-0012, AG024904, U01 AG024904)
- Meso Scale Diagnostics (U01 AG024904, AG024904, W81XWH-12-2-0012)
- Alzheimer's Disease Neuroimaging Initiative (W81XWH-12-2–0012, U01AG024904, AG024904, W81XWH-12-2-, Department of Defense award number W81XWH-12-2-0012)
- BioClinica (U01 AG024904, W81XWH-12-2-0012, AG024904)
- Novartis Pharmaceuticals Corporation (W81XWH-12-2-0012, U01 AG024904, AG024904)
- Northern California Institute for Research and Education (U01 AG024904, W81XWH-12-2-0012)
- Kulturstiftung Essen
- Ministerium für Innovation, Wissenschaft und Forschung des Landes Nordrhein-Westfalen
- ResMed
- European Commission
- Deutsche Forschungsgemeinschaft (JO-170/8-1, SI 236/8–1, 236/8-1, ER 155/6-1;6-2, HO 3314/2-1, ER 155/6-1, EI 969/2-3, INST 58219/32-1, HO 3314/2-1;2-2;2-3;4-3, PE 2309/2-1, KN885/3-1, SI 236/8-1;9-1;10-1)
- Volkswagen Foundation
- Bundesministerium für Bildung und Forschung (01EG0401, 01ER1001D, 01GS0820_WB2-C, 01GI0860, 01GI0856, 01GI0205)
- Deutsche Gesetzliche Unfallversicherung (FF-FP295, FP 295)
- Eisai (W81XWH-12-2-0012, U01 AG024904, AG024904)
- Servier (AG024904, W81XWH-12-2-0012, U01 AG024904)
- H. Lundbeck A/S (U01 AG024904, AG024904, W81XWH-12-2-0012)
- IXICO (AG024904, W81XWH-12-2-0012, U01 AG024904)
- Heinz Nixdorf Stiftung (SI 236/8-1, ER 155/6-1)
- National Institutes of Health (3u01ag024904-10s2, W81XWH, W81XWH-12-2-0012, U01-AG024904, Grant U01 AG024904, AG024904)
- Genentech (W81XWH-12-2-0012, AG024904, U01 AG024904)
- Canadian Institutes of Health Research (U01 AG024904, W81XWH-12-2-0012)
- National Institute on Aging (U01-AG-024904, W81XWH-12-2–0012, AG 024904)
- National Institute of Biomedical Imaging and Bioengineering (W81XWH-12-2-0012, U01 AG024904, AG024904)
- Topics
- Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Health Sciences › Medicine › Physiology › Alzheimer's disease research and treatments
- Keywords
- amnestic mild cognitive impairment, Alzheimer's disease, APOE ε4, magnetic resonance imaging, multimodal fusion, convolutional neural networks, InceptionV3, LSTM, early diagnosis, Heinz Nixdorf Recall Study, ADNI
- MeSH terms
- Deep Learning, Aged, Aged, 80 and over, Apolipoproteins E, Female, Humans, Magnetic Resonance Imaging, Male, Middle Aged, Socioeconomic Factors, Disease Progression, Neuroimaging, Cognitive Dysfunction, Datasets as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
