Multimodal deep learning for biomedical data fusion: a review
Open accessThe DOI matches its Crossref record; 107 of the 117 references it cites were found in a registry or an index; 60 of the 96 sources its claims cite were checked for support.
Abstract
Biomedical data are becoming increasingly multimodal and thereby capture the underlying complex relationships among biological processes. Deep learning (DL)-based data fusion strategies are a popular approach for modeling these nonlinear relationships. Therefore, we review the current state-of-the-art of such methods and propose a detailed taxonomy that facilitates more informed choices of fusion strategies for biomedical applications, as well as research on novel methods. By doing so, we find that deep fusion strategies often outperform unimodal and shallow approaches. Additionally, the proposed subcategories of fusion strategies show different advantages and drawbacks. The review of current methods has shown that, especially for intermediate fusion strategies, joint representation learning is the preferred approach as it effectively models the complex interactions of different levels of biological organization. Finally, we note that gradual fusion, based on prior biological knowledge or on search strategies, is a promising future research path. Similarly, utilizing transfer learning might overcome sample size limitations of multimodal data sets. As these data sets become increasingly available, multimodal DL approaches present the opportunity to train holistic models that can learn the complex regulatory dynamics behind health and disease.
Overview
CoChat Summary
TL;DR
This review surveys deep learning-based multimodal biomedical data fusion and proposes a taxonomy of early, intermediate, and late fusion strategies. It finds that deep fusion often outperforms unimodal and shallow approaches, with intermediate joint representation learning preferred, while gradual fusion and transfer learning remain underexplored.
Key findings
- The paper proposes a taxonomy dividing fusion into early fusion (direct modeling, autoencoder subcategories), intermediate fusion (homogeneous/heterogeneous designs; marginal/joint representations), and late fusion (averaging equal/weighted, meta-learning).
- Deep fusion strategies frequently outperform unimodal and shallow methods; joint intermediate fusion is the preferred approach because it effectively models complex interactions across levels of biological organization.
- Early fusion is simple but may miss cross-modal relationships apparent only at higher abstraction; intermediate fusion offers flexibility for heterogeneity, missing modalities, and dimensionality imbalance; late fusion combines decisions but cannot learn feature-level cross-modal interactions.
- Gradual fusion guided by prior biological knowledge or search strategies and transfer learning are promising future paths; reviewed applications included sample sizes as low as 96 patients.
Methods
- Narrative literature review of deep learning-based data fusion strategies in the biomedical domain, taking a representation learning perspective.
- Proposes a detailed taxonomy of early, intermediate, and late fusion and their subcategories, exemplified with published biomedical applications.
- No primary dataset, systematic search protocol, or quantitative meta-analysis is described; synthesis is conceptual and qualitative.
Limitations
- The paper states the literature is most likely skewed toward reporting positive results, performance of fusion strategies is problem- and data-specific, and more theoretical knowledge is needed to specify when strategies excel.
- The paper states multimodal DL faces general challenges including data volume, quality, interpretability, temporality, missing entire modalities, and overfitting/generalizability due to small sample sizes and many parameters.
- Evident from the text: as a narrative review, it does not report a systematic search strategy or quantitative comparison across studies.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
![Figure 1 Development of technologies and multimodal deep learning (DL). ‘Omics’ and ‘multi-omics’ data become increasingly relevant in the scientific literature. To fully utilize the growing number of multimodal data sets, data fusion methods based on DL are evolving into an important approach in the biomedical field. This unprecedented generation of data has been made possible by high-throughtput technologies like microarrays and next-generation sequencing [7]. The development of bulk RNA-seq was followed by several related sequencing technologies, such as single-cell RNA-seq and ATAC-seq [8]. Currently, spatial transcriptomics [9] and single-cell multi-omics [10] are being increasingly used.](https://figures.cochat.ai/paper-figures/5c1e0ae86d784af8937add9f4a2f8906/multimodal-deep-learning-for-biomedical-data-fusion-a-review-figure-1.jpg)
![Figure 2 DL-based fusion strategies. Layers marked in blue are shared between modalities and learn joint representations. (a) Early fusion strategies take as input a concatenated vector. No marginal representations are learned. (b) Intermediate fusion strategies first learn marginal representations and fuse these later inside the network. This can occur in one layer or gradually. (c) Late fusion strategies combine decisions by sub-models for each modality. Figure adapted from [2].](https://figures.cochat.ai/paper-figures/73c64526f30e41a695c45c40cae87d27/multimodal-deep-learning-for-biomedical-data-fusion-a-review-figure-2.jpg)


Verification
Claims the paper makes
40 Valid · 5 Not valid · 23 Not checked · 87 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“Despite the aforementioned drawbacks, some reviewed papers have shown that early fusion AEs can perform on par with intermediate strategies [38, 41]”
§ Multimodal deep learning for biomedical data fusion: a review > Early fusion > Discussion of early fusion strategies
What the sources say
- Disputedcites [38] Islam, 2020, An integrative deep learning framework for classifying molecular subtypes of breast cancer, Comput Struct Biotechnol J sourceExperimental — accuracy not yet measured
The Methods section describes an early-fusion stacked autoencoder that takes concatenated CNA and gene-expression data as input, alongside a DCNN_Concat model that combines features learned separately from each source. The Results section (page unavailable) says DCNN_Concat performed better than the autoencoder-based DNN_SE, rather than reporting that they performed on par. (LLM, conf=0.86)
Checked by a reasoning model — may be wrong.
- Supportedcites [41] Simidjievski, 2019, Variational autoencoders for cancer data integration: design principles and computational practice, Front Genet source
The Methods section describes CNC-VAE as combining data at the input and the other architectures as integrating data in hidden layers. In Results > Quality of the Learned Representations (page unavailable), H-VAE and X-VAE mostly produced more accurate predictions, but the differences among the four architectures were not significant. This supports the claim that an early-fusion AE can perform on par with intermediate strategies in the tasks studied. (LLM, conf=0.89)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Every source the paper cites for this claim was checked and supports it.
In the paper
“Questioning this assumption is relevant as many methods have shown unequal contribution of different modalities to the predictive performance (see for example [36, 42]).”
§ Multimodal deep learning for biomedical data fusion: a review > Late fusion
What the sources say
- Supportedcites [36] Chai, 2021, Integrating multi-omics data through deep learning for accurate cancer prognosis prediction, Comput Biol Med source
The Conclusion and Discussion reports an ablation study in which mRNA performed best, followed by miRNA and methylation, while copy-number variation contributed least. Table 3 also shows different C-index values when individual omics types are used or excluded, supporting unequal contributions to predictive performance. (LLM, conf=0.92)
- Supportedcites [42] Albaradei, 2021, Meta cancer: a deep learning-based pan-cancer metastasis prediction model developed using multi-omics data, Comput Struct Biotechnol J source
In “Evaluating the contribution of different omics data types” (page not supplied), MetaCancer reports different predictive contributions from mRNA, DNA methylation and microRNA. The section also describes similar differences in other studies, supporting its use as an example of the claim. (LLM, conf=0.91)
boostedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“However, different modalities can add information to different degrees (e.g. [36, 44]).”
§ Multimodal deep learning for biomedical data fusion: a review > Early fusion > Discussion of early fusion strategies
What the sources say
- Supportedcites [36] Chai, 2021, Integrating multi-omics data through deep learning for accurate cancer prognosis prediction, Comput Biol Med source
The ‘Patients’ risks estimation by multi-omics data’ section compares the contributions of individual omics types: mRNA contributed most to prognosis prediction, while copy number variation contributed least. This supports the claim that different modalities add information to different degrees. (LLM, conf=0.93)
- Supportedcites [44] Lee, 2019, Alzheimer’s disease neuroimaging initiative. Predicting Alzheimer’s disease progression using multi-modal deep learning approach, Sci Rep source
The Results section, in its comparison of single-modality and multimodal predictions, reports that cognitive-performance data predicted conversion more accurately than the other individual modalities, while demographic data performed too poorly to include in the figure. Table 5 also shows different accuracies across modalities and higher accuracy for the combined model. The supplied passages have no page numbers. (LLM, conf=0.91)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Vale-Silva et al. [74] compared these methods, although they did not find large differences in performance for predicting long-term cancer survival.”
§ Multimodal deep learning for biomedical data fusion: a review > Intermediate fusion > Heterogeneous network design
What the sources say
- Partly supportedcites [74] Vale-Silva, 2021, Long-term cancer survival prediction using multimodal deep learning, Sci Rep sourceExperimental — accuracy not yet measured
The Methods section (“Multimodal data fusion layer”; page unavailable) says MultiSurv used a row-wise maximum to combine modality representations and tested alternative fusion schemes without improving performance. It does not identify those alternatives in our copy or establish that performance differences among them were small. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Particularly in cancer patient survival subtyping with multi-omics modalities, this sequence of steps has become popular [25–31, 34].”
§ Multimodal deep learning for biomedical data fusion: a review > Early fusion > Latent representation with multimodal AEs
What the sources say
- Supportedcites [25] Chaudhary, 2018, Deep learning-based multi-omics integration robustly predicts survival in liver cancer, Clin Cancer Res source
In Results, the paper describes integrating three omics data types with an autoencoder, selecting bottleneck features using univariate Cox-PH regression, and inferring survival-subtype labels through K-means clustering. It therefore exemplifies the sequence for which it is cited; the citation need not establish popularity on its own. (LLM, conf=0.92)
- Partly supportedcites [26] Lv, 2020, Survival prediction in patients with colon adenocarcinoma via multiomics data integration using a deep learning algorithm, Biosci Rep sourceExperimental — accuracy not yet measured
The Results section ('Identification of two survival risk groups in TCGA multiomics data'; page unavailable) describes the stated sequence: an autoencoder transformed multiomics features, univariable Cox-PH regression selected survival-associated features, and K-means clustering inferred two groups. This establishes that the paper uses the sequence, but not that it has become popular; the Introduction instead describes deep-learning applications in prognostic modeling as being in their infancy. (LLM, conf=0.85)
- Partly supportedcites [27] Zhang, 2018, Deep learning-based multi-omics data integration reveals two prognostic subtypes in high-risk neuroblastoma, Front Genet sourceExperimental — accuracy not yet measured
The Materials and methods sections (pages unavailable) describe multi-omics features generated by an autoencoder, subsequent Cox regression to select prognostic features, and K-means clustering to infer survival-risk labels. The paper does not establish that this full sequence has become popular, and the passages do not specify that the post-autoencoder Cox modeling was univariate. (LLM, conf=0.83)
- Partly supportedcites [28] Yu, 2020, A model for predicting prognosis in patients with esophageal squamous cell carcinoma based on joint representation learning, Oncol Lett sourceExperimental — accuracy not yet measured
The Materials and methods section (page unavailable) describes joint multimodal autoencoder features, univariate Cox-PH feature selection, and K-means clustering to infer survival-risk subgroup labels. The paper therefore exemplifies the sequence, but it does not establish that the full sequence had become popular in multi-omics cancer survival subtyping. (LLM, conf=0.83)
- Supportedcites [29] Asada, 2020, Uncovering prognosis-related genes and pathways by multi-omics analysis in lung cancer, Biomol Ther source
The Methods section (2.2) describes an autoencoder followed by univariate Cox-PH feature selection and K-means clustering to infer labels. The Discussion (section 4) identifies this as the paper’s multi-omics pipeline for lung-cancer survival subtyping, so the paper supports its use as an example of the cited sequence. (LLM, conf=0.90)
- Supportedcites [30] Zhang, 2021, Robust prognostic subtyping of muscle-invasive bladder cancer revealed by deep learning-based multi-omics data integration, Front Oncol source
The Results section, “The Identification of OS-Related Subtypes Based on TCGA Multi-Omics Data,” describes the claimed sequence: autoencoder-derived multi-omics features, univariate Cox selection of survival-related features, and K-means clustering to assign subtypes. The paper therefore serves as an example of the approach cited alongside other studies; it does not independently establish how popular the approach is. (LLM, conf=0.89)
- Not checked — no full text of the sourcecites [31] Lee, 2020, Incorporating deep learning and multi-omics autoencoding for analysis of lung adenocarcinoma prognostication, Comput Biol Chem source
- Supportedcites [34] Zhao, 2020, Deep learning-based model for predicting progression in patients with head and neck squamous cell carcinoma, Cancer Biomark source
Section 3.1 describes combining three omics modalities in an autoencoder, selecting survival-associated features by univariate Cox-PH regression, and using K-means clustering to identify patient subtypes. Section 2.3 explicitly lays out the autoencoder → Cox-PH selection → clustering sequence. The paper therefore exemplifies the approach for which it is cited. (LLM, conf=0.91)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
91 more claims · 26 with their cited sources checked
Create a free account to see all 155 claims and to verify the 4 references still open.
- It may state two separate claims as one. 19
- Probably not a substantial claim. 43
- The quote only partly states this claim. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“This method has consistently demonstrated superior performance compared to early and late fusion strategies across diverse clinical applications14–16.”
From MoMA: a mixture-of-multimodal-agents architecture for enhancing clinical prediction modelling (2025) · cites this paper as [15]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, each of these public datasets, as well as the private datasets used in included studies, contains no more than 280 subjects (with sample sizes ranging from 24 to 275), which hampers the ability to identify clinical biomarkers universally and reliably, especially with DL approaches132,133.”
From AI-assisted multi-modal information for the screening of depression: a systematic review and meta-analysis (2025) · cites this paper as [132]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“DL architectures are well-suited for intermediate fusion as they enable an easy connection of marginal representations to a shared layer, facilitating the correspondence of hierarchical representations to real-world phenomena (Stahlschmidt et al. 2022).”
From A review of multimodal deep learning methods for genomic-enabled prediction in plant breeding (2024) · cites this paper as Stahlschmidt et al. 2022
The cited source says what the claim attributes to it.
In the citing paper
“This approach assumes an equal contribution from each sub-model, as there is no weighting of their outputs (Stahlschmidt et al. 2022).”
From A review of multimodal deep learning methods for genomic-enabled prediction in plant breeding (2024) · cites this paper as Stahlschmidt et al. 2022
The cited source says what the claim attributes to it.
In the citing paper
“Stahlschmidt et al. summarized multimodal fusion methods in biomedical analysis [31]”
From Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis (2025) · cites this paper as [31]
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Regression Models and Life-Tables
Cox, D. R. (1972) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1972.tb00899.x · Cited by 39,793
A fast learning algorithm for deep belief nets
Hinton, G. E., Osindero, S., & Teh, Y.-W. (2006) · Neural Computation · doi:10.1162/neco.2006.18.7.1527 · Cited by 16,336 · cited 1×
“Alternatively, the representations can serve as a useful and computationally efficient initialization of a DNN, which is fine-tuned with more expensive supervised algorithms such as back propagation to learn p(y|x) [94].”
MethodRepresentation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001 · cited 3×
“Earlier layers learn simple abstractions of the data, whereas deeper layers combine these into more abstract representations that are informative for the learning task [13].”
BackgroundInfluentialA Comprehensive Survey on Graph Neural Networks
Wu, Z., Pan, S., Chen, F., Long, G., Zhang, C., & Yu, P. S. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2978386 · Cited by 11,000
Extracting and composing robust features with denoising autoencoders
Vincent, P., Larochelle, H., Bengio, Y., & Manzagol, P.-A. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390294 · Cited by 7,436 · cited 1×
“For the same clinical task, a similar workflow has been adapted by researchers, but applying denoising AEs (DAEs) [91] instead [35, 36].”
MethodGrad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01228-7 · Cited by 6,909 · cited 2×
“[79] enabled modality-specific interpretability by applying Grad-CAM [97] for WSI and integrated gradient [98] for cell graph and genomic modalities using convolutional, graph convolutional [14] and fully connected branches.”
MethodLarge-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354 · cited 1×
“Despite the aforementioned drawbacks, some reviewed papers have shown that early fusion AEs can perform on par with intermediate strategies [38, 41], though other structured investigations inside [28] and outside [93] the biomedical domain have shown superiority of intermediate fusion over early fusion strategies.”
BackgroundMultimodal Machine Learning: A Survey and Taxonomy
Baltrusaitis, T., Ahuja, C., & Morency, L.-P. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2798607 · Cited by 5,018 · cited 5×
“Thereby, coordinated representations [11] of each modality could be learned in an unsupervised way, which resulted in encodings of patterns between modalities.”
BackgroundInfluentialVisualization and analysis of gene expression in tissue sections by spatial transcriptomics
Ståhl, P. L., Salmén, F., Vickovic, S., Lundmark, A., Navarro, J. F., Magnusson, J., Giacomello, S., Asp, M., Westholm, J. O., Huss, M., et al. (2016) · Science · doi:10.1126/science.aaf2403 · Cited by 3,834 · cited 1×
“Currently, spatial transcriptomics [9] and single-cell multi-omics [10] are being increasingly used.”
Method
Research works printed but not linked: 6
- Springer Nature. Milestones in Genomic Sequencing
- Goodfellow, 2016, Deep Learning
- Ballard, 1987, Modular learning in neural networks. In: Proceedings of the Sixth National Conference on Artificial Intelligence, 279
- Kingma, 2014, Proceedings of the International Conference on Learning Representations (ICLR)
- Alemi, 2017, 5th International Conference on Learning Representations
- Sundararajan, 2017, 34th International Conference on Machine Learning, 5109
Cited by
Advance in peptide-based drug development: delivery platforms, therapeutics and vaccines
Xiao, W., Jiang, W., Chen, Z., Huang, Y., Mao, J., Zheng, W., Hu, Y., & Shi, J. (2025) · Signal Transduction and Targeted Therapy · doi:10.1038/s41392-024-02107-5 · Cited by 449
Multimodal data fusion for cancer biomarker discovery with deep learning
Steyaert, S., Pizurica, M., Nagaraj, D., Khandelwal, P., Hernandez-Boussard, T., Gentles, A. J., & Gevaert, O. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00633-5 · Cited by 402
Hybrid multimodal wearable sensors for comprehensive health monitoring
Mahato, K., Saha, T., Ding, S., Sandhu, S. S., Chang, A.-Y., & Wang, J. (2024) · Nature Electronics · doi:10.1038/s41928-024-01247-4 · Cited by 404
Deep learning based multimodal biomedical data fusion: an overview and comparative review
Duan, J., Xiong, J., Li, Y., & Ding, W. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102536 · Cited by 284
Artificial intelligence-based methods for fusion of electronic health records and imaging data
Mohsen, F., Ali, H., El Hajj, N., & Shah, Z. (2022) · Scientific Reports · doi:10.1038/s41598-022-22514-4 · Cited by 250
Review of multimodal machine learning approaches in healthcare
Krones, F., Marikkar, U., Parsons, G., Szmul, A., & Mahdi, A. (2025) · Information Fusion · doi:10.1016/j.inffus.2024.102690 · Cited by 235
Deep multimodal fusion of image and non-image data in disease diagnosis and prognosis: a review
Cui, C., Yang, H., Wang, Y., Zhao, S., Asad, Z., Coburn, L. A., Wilson, K. T., Landman, B. A., & Huo, Y. (2023) · Progress in Biomedical Engineering · doi:10.1088/2516-1091/acc2fe · Cited by 218
“Huang et al [12] and Stahlschmidt et al [15] categorized the fusion methods by the stages of fusion.”
Baião, A. R., Cai, Z., Poulos, R. C., Robinson, P. J., Reddel, R. R., Zhong, Q., Vinga, S., & Gonçalves, E. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf355 · Cited by 211
Effective Techniques for Multimodal Data Fusion: A Comparative Analysis
Pawłowski, M., Wróblewska, A., & Sysko-Romańczuk, S. (2023) · Sensors · doi:10.3390/s23052381 · Cited by 180
Multimodal Attention-Based Deep Learning for Alzheimer’s Disease Diagnosis
Golovanevsky, M., Eickhoff, C., & Singh, R. (2022) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocac168 · Cited by 165
“Even with DL’s advantage, single-modality input is often insufficient to support clinical decision-making [16].”
BackgroundNakach, F.-Z., Idri, A., & Göçeri, E. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10984-z · Cited by 168
Dentamaro, V., Impedovo, D., Musti, L., Pirlo, G., & Taurisano, P. (2024) · Scientific Reports · doi:10.1038/s41598-024-70165-4 · Cited by 116
“The main reason of adopting intermediate fusion over early and late fusion, is its better suitability for deep learning approaches and the ability to learn cross-modality interactions from the marginal representations 57 leading to an improvement in the overall classification task.”
Multimodal Deep Learning Approaches for Precision Oncology: A Comprehensive Review
Yang, H., Yang, M., Chen, J., Yao, G., Zou, Q., & Jia, L. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae699 · Cited by 115
A review on the application of molecular descriptors and machine learning in polymer design
Zhao, Y., Mulder, R. J., Houshyar, S., & Le, T. C. (2023) · Polymer Chemistry · doi:10.1039/d3py00395g · Cited by 131
Diagnosis of brain diseases in fusion of neuroimaging modalities using deep learning: A review
Shoeibi, A., Khodatars, M., Jafari, M., Ghassemi, N., Moridian, P., Alizadehsani, R., Ling, S. H., Khosravi, A., Alinejad‐Rokny, H., Lam, H., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.12.010 · Cited by 121
Elazab, A., Wang, C., Abdel-Aziz, M. H., Zhang, J., Gu, J. J., Górriz, J. M., Zhang, Y., & Chang, C. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124780 · Cited by 126
A systematic review of intermediate fusion in multimodal deep learning for biomedical applications
Guarrasi, V., Aksu, F., Caruso, C. M., Di Feola, F., Rofena, A., Ruffini, F., & Soda, P. (2025) · Image and Vision Computing · doi:10.1016/j.imavis.2025.105509 · Cited by 95
Multimodal deep learning approaches for single-cell multi-omics data integration
Athaya, T., Ripan, R. C., Li, X., & Hu, H. (2023) · Briefings in Bioinformatics · doi:10.1093/bib/bbad313 · Cited by 108
“Three types of fusion strategies have been identified: early fusion, intermediate fusion and late fusion [52] (as shown in Figure 2).”
BackgroundMethodInfluentialFayad, I., Ciais, P., Schwartz, M., Wigneron, J., Baghdadi, N. N., Truchis, A. D., d’Aspremont, A., Frappart, F., Saatchi, S. S., Sean, E., et al. (2023) · Remote Sensing of Environment · doi:10.1016/j.rse.2023.113945 · Cited by 107
“This approach can lead to a more focused representation of the data, as each encoder can specialize in processing its own modality (Stahlschmidt et al., 2022).”
Teoh, J. R., Dong, J., Zuo, X., Lai, K. W., Hasikin, K. B., & Wu, X. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.2298 · Cited by 139
“Early fusion merges modalities based on predictor information or independent variables, serving either as a preprocessing step or an unsupervised task to create features that capture underlying patterns ( Gaw, Yousefi & Gahrooei, 2022 ; Stahlschmidt, Ulfenborg & Synnergren, 2022 ).”
InfluentialSynthetic data in biomedicine via generative artificial intelligence
van Breugel, B., Liu, T., Oglić, D., & van der Schaar, M. (2024) · Nature Reviews Bioengineering · doi:10.1038/s44222-024-00245-7 · Cited by 149
A Comprehensive Review on Deep Learning-Based Data Fusion
Hussain, M., O’Nils, M., Lundgren, J., & Mousavirad, S. J. (2024) · IEEE Access · doi:10.1109/access.2024.3508271 · Cited by 137
A review of cancer data fusion methods based on deep learning
Zhao, Y., Li, X., Zhou, C., Peng, H., Zheng, Z., Chen, J., & Ding, W. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102361 · Cited by 95
Ranjan, R., Sahana, B. C., & Bhandari, A. K. (2024) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-023-10047-6 · Cited by 100
“DL offers a ubiquitous, adaptable, and highly flexible framework for complex biomedical data [59].”
Shen, Z., Hu, J., Wu, H., Chen, Z., Wu, W., Lin, J., Xu, Z., Kong, J., & Lin, T. (2022) · Journal of Translational Medicine · doi:10.1186/s12967-022-03615-0 · Cited by 93
“In addition, the combination of multiomics analysis [53–55] such as radiomics [56, 57] is the focus of future breakthrough in digital tumor pathology.”
BackgroundDeep learning–driven multi-omics analysis: enhancing cancer diagnostics and therapeutics
Zhang, J., Che, Y., Liu, R., Wang, Z., & Liu, W. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf440 · Cited by 112
“Lately, DL has emerged as one of the most efficacious methods for integrating multi-omics data analysis in cancer [14, 21].”
Smart closed-loop drug delivery systems
Paci, M. M., Saha, T., Djassemi, O., Wu, S. T., Chua, C. Y. X., Wang, J., & Grattoni, A. (2025) · Nature Reviews Bioengineering · doi:10.1038/s44222-025-00328-z · Cited by 111
Challenges in AI-driven Biomedical Multimodal Data Fusion and Analysis
Liu, J., Cen, X., Yi, C., Wang, F.-A., Ding, J., Cheng, J., Wu, Q., Gai, B., Zhou, Y., He, R., et al. (2025) · Genomics, Proteomics & Bioinformatics · doi:10.1093/gpbjnl/qzaf011 · Cited by 75
Oyelade, O. N., Irunokhai, E., & Wang, H. (2024) · Scientific Reports · doi:10.1038/s41598-024-51329-8 · Cited by 66
Muhsin, Z. J., Qahwaji, R., Ghanchi, F. D., & Al-Taee, M. A. (2023) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-023-00427-4 · Cited by 86
“[ 61 ].”
Multimodal Machine Learning Guides Low Carbon Aeration Strategies in Urban Wastewater Treatment
Wang, H., Wang, Y., Wang, X., Yin, W.-X., Yu, T., Xue, C.-H., & Wang, A. (2024) · Engineering · doi:10.1016/j.eng.2023.11.020 · Cited by 69
Neural multi-task learning in drug design
Allenspach, S., Hiss, J. A., & Schneider, G. (2024) · Nature Machine Intelligence · doi:10.1038/s42256-023-00785-4 · Cited by 85
“Another learning paradigm related to MTL, is multi-modal learning that has been applied in various domains such as computer vision 23 and biomedical science 24,25 .”
Bischof, J., Fletcher, G., Verkade, P., Kuntner, C., Fernández-Rodríguez, J., Chaabane, L., Rose, L. A., Walter, A., Vandenbosch, M., van Zandvoort, M. A. M. J., et al. (2024) · npj Imaging · doi:10.1038/s44303-024-00010-w · Cited by 73
“Effectivecross-modalitydatafusionrequirescommon fi le-formatstoenable consistentAPIsacrossmodalities 33,36 andcantakeplaceatdifferentstagesof theinformationextractedfromeachmodality,forexample, rawimagedata, image embeddings and/or image-derived features 37,38 .”
Liu, S., Wang, X., Hou, Y., Li, G., Wang, H., Xu, H., Xiang, Y., & Tang, B. (2022) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3217810 · Cited by 92
Neurosymbolic AI for Reasoning Over Knowledge Graphs: A Survey
DeLong, L. N., Mir, R. F., & Fleuriot, J. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3420218 · Cited by 134
“Existing approaches tend to either: 1) concatenate modalities into one, input vector, in which modalities are no longer distinguishable; 2) project modalities into representative embdeddings via an autoencoder; or 3) use entirely separate models for each modality [46], [137].”
Open challenges and opportunities in federated foundation models towards biomedical healthcare
Li, X., Peng, L., Wang, Y., & Zhang, W. (2025) · BioData Mining · doi:10.1186/s13040-024-00414-9 · Cited by 103
“A comprehensive examination of these strategies is presented in a review of multimodal deep learning applications in biomedical data fusion, which outlines the potential advancements and methodologies in the field [105].”
Multimodality Fusion Aspects of Medical Diagnosis: A Comprehensive Review
Kumar, S., Rani, S., Sharma, S., & Min, H. (2024) · Bioengineering · doi:10.3390/bioengineering11121233 · Cited by 85
Model Selection of Hybrid Feature Fusion for Coffee Leaf Disease Classification
Faisal, M. A., Leu, J., & Darmawan, J. T. (2023) · IEEE Access · doi:10.1109/access.2023.3286935 · Cited by 53
“However, the drawback of this technique is that multimodal relationships are not learned by the final model [37].”
BackgroundLu, X., Xie, L., Xu, L., Mao, R., Xu, X., & Chang, S. (2024) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2024.04.030 · Cited by 64
“Currently, multi-modal learning is a vibrant multidisciplinary field, which provides frameworks to process multiple sources of information [42,43].”
BackgroundDeep Learning-Based Multi-Modal Ensemble Classification Approach for Human Breast Cancer Prognosis
Jadoon, E. K., Khan, F. G., Shah, S., Khan, A., & ElAffendi, M. (2023) · IEEE Access · doi:10.1109/access.2023.3304242 · Cited by 61
“The multi-modal is becoming more prevalent in capturing the complication of biological process, as it can unveil the underlying interconnections [16].”
BackgroundA machine learning approach to leveraging electronic health records for enhanced omics analysis
Mataraso, S. J., Espinosa, C. A., Seong, D., Reincke, S. M., Berson, E., Reiss, J. D., Kim, Y., Ghanem, M., Shu, C.-H., James, T. K., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-024-00974-9 · Cited by 67
End-to-End Fusion of Hyperspectral and Chlorophyll Fluorescence Imaging to Identify Rice Stresses
Zhang, C., Zhou, L., Xiao, Q., et al. (2022) · Plant Phenomics · doi:10.34133/2022/9851096 · Cited by 47
“Recent studies have developed end-to-end deep fusion models for applications, combining feature extraction and modeling in one model [47, 48].”
BackgroundA review of multimodal deep learning methods for genomic-enabled prediction in plant breeding
Montesinos-López, O. A., Chavira-Flores, M., Kiasmiantini, Crespo-Herrera, L., Saint Piere, C., Li, H., Fritsche-Neto, R., Al-Nowibet, K., Montesinos-López, A., & Crossa, J. (2024) · GENETICS · doi:10.1093/genetics/iyae161 · Cited by 47
Spatial pharmacology using mass spectrometry imaging
Rajbhandari, P., Neelakantan, T. V., Hosny, N., & Stockwell, B. R. (2024) · Trends in Pharmacological Sciences · doi:10.1016/j.tips.2023.11.003 · Cited by 49
Zhu, F., Zhang, J., Dang, R., Hu, B., & Wang, Q. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106996 · Cited by 53
MXene-based nano(bio)sensors for the detection of biomarkers: A move towards intelligent sensors
Khorsandi, D., Yang, J., Ülker, Z., et al. (2023) · Microchemical Journal · doi:10.1016/j.microc.2023.109874 · Cited by 42
Deep learning methods improve genomic prediction of wheat breeding
Montesinos‐López, A., Crespo‐Herrera, L., Dreisigacker, S., Gerard, G., Vitale, P., Pierre, C. S., Govindan, V., Tadesse, Z., Flores, M. C., Pérez‐Rodríguez, P., et al. (2024) · Frontiers in Plant Science · doi:10.3389/fpls.2024.1324090 · Cited by 52
Multimodal deep learning methods enhance genomic prediction of wheat breeding
Montesinos-López, A., Rivera, C., Pinto, F., Piñera, F., Gonzalez, D., Reynolds, M., Pérez-Rodríguez, P., Li, H., Montesinos-López, O. A., & Crossa, J. (2023) · G3: Genes, Genomes, Genetics · doi:10.1093/g3journal/jkad045 · Cited by 41
Optimizing the OCTA layer fusion option for deep learning classification of diabetic retinopathy
Ebrahimi, B., Le, D., Abtahi, M., Dadzie, A. K., Lim, J. I., Chan, R. V. P., & Yao, X. (2023) · Biomedical Optics Express · doi:10.1364/boe.495999 · Cited by 50
An overview of methods and techniques in multimodal data fusion with application to healthcare
Chaabene, S., Boudaya, A., Bouaziz, B., & Chaâri, L. (2025) · International Journal of Data Science and Analytics · doi:10.1007/s41060-025-00715-0 · Cited by 68
“As discussed by [149], deep fusion strategies often tend to outperform unimodal and shallow approaches, especially on their intermediate fusion strategies that allow joint representation learning.”
Geometric Correspondence-Based Multimodal Learning for Ophthalmic Image Analysis
Wang, Y., Zhen, L., Tan, T., Fu, H., Feng, Y., Wang, Z., Xu, X., Goh, R. S. M., Ng, Y. X., Calhoun, C. T., et al. (2024) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2024.3352602 · Cited by 49
“Intermediate fusion is similar to early fusion, but the model of intermediate fusion strategy has an interaction among modalities during the training process, and the model attempts to discover within-modality correlations [23].”
BackgroundAutomated Ensemble Multimodal Machine Learning for Healthcare
Imrie, F., Denner, S., Brunschwig, L. S., Maier-Hein, K. H., & van der Schaar, M. (2025) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2025.3530156 · Cited by 57
A Comprehensive Review of Multimodal Deep Learning for Enhanced Medical Diagnostics
Al‐Zoghby, A. M., Ebada, A. I., Saleh, A. S., Abdelhay, M., & Awad, W. A. (2025) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2025.065571 · Cited by 87
A Review on Data Fusion of Multidimensional Medical and Biomedical Data
Azam, K. S. F., Ryabchykov, O., & Bocklitz, T. W. (2022) · Molecules · doi:10.3390/molecules27217448 · Cited by 45
“[66] reviewed the issues that generally arise when analyzing biological data using multimodal deep learning techniques.”
BackgroundEvaluating the potential of Distribution of Relaxation Times analysis for plant agriculture
Haeverbeke, M. V., Bernard De Baets, C., & Stock, M. (2023) · Computers and Electronics in Agriculture · doi:10.1016/j.compag.2023.108249 · Cited by 38
Advancing Chinese biomedical text mining with community challenges
Zong, H., Wu, R., Cha, J., Feng, W., Wu, E., Li, J., Shao, A., Tao, L., Li, Z., Tang, B., et al. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104716 · Cited by 44
Zhou, X., Chen, C., Zuo, E., Chen, C., & Lv, X. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.112204 · Cited by 41
Designing interpretable deep learning applications for functional genomics: a quantitative analysis
van Hilten, A., Katz, S., Saccenti, E., Niessen, W. J., & Roshchupkin, G. V. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae449 · Cited by 44
“Similarly, other types of data such as clinical data, imaging data, and patient records can be integrated in a single prediction model [142, 143].”
A review on multimodal machine learning in medical diagnostics
Yan, K., Li, T., Marques, J. A. L., Gao, J. T., & Fong, S. J. (2023) · Mathematical Biosciences & Engineering · doi:10.3934/mbe.2023382 · Cited by 42
Machine Learning in Petrology: State-of-the-Art and Future Perspectives
Petrelli, M. (2024) · Journal of Petrology · doi:10.1093/petrology/egae036 · Cited by 54
“This strategy takes the name of unimodal approach (Ramachandram and Taylor, 2017; Stahlschmidt et al., 2022).”
BackgroundInfluentialKumar, A., Nashte, A., Amit, R. P., & Choudhary, C. (2023) · International Conference on Image Information Processing (ICIIP) · doi:10.1109/iciip61524.2023.10537751 · Cited by 40
“An extensive survey outlines the profound impact deep learning has had on medical image interpretation, providing insights into improved patient outcomes and more precise diagnostics [4].”
BackgroundTowards verifiable cancer digital twins: tissue level modeling protocol for precision medicine
Kemkar, S., Tao, M., Ghosh, A., STAMATAKOS, G. S., Graf, N. M., Poorey, K., Balakrishnan, U., Trask, N., & Radhakrishnan, R. (2024) · Frontiers in Physiology · doi:10.3389/fphys.2024.1473125 · Cited by 47
A machine learning approach for multimodal data fusion for survival prediction in cancer patients
Νικολάου, Ν., Salazar, D., RaviPrakash, H., Gonçalves, M., Mulla, R., Burlutskiy, N., Markuzon, N., & Jacob, E. (2025) · npj Precision Oncology · doi:10.1038/s41698-025-00917-6 · Cited by 65
Zhou, X., Chen, C., Lv, X., Zuo, E., Li, M., Wu, L., Chen, X., Wu, X., & Chen, C. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103804 · Cited by 45
Supervised multiple kernel learning approaches for multi-omics data integration
Briscik, M., Tazza, G., Vidács, D. L., Dillies, M., & Dejean, S. (2024) · BioData Mining · doi:10.1186/s13040-024-00406-9 · Cited by 50
“As reviewed in Stahlschmidt et al. [2022], this kind of architecture is gaining popularity in the biomedical field, where data are becoming increasingly multi-modal.”
Guo, R., Wei, J., Sun, L., Yu, B., Chang, G., Liu, D., Zhang, S., Yao, Z., Xu, M., & Bu, L. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108709 · Cited by 69
“Unique Challenges for Image-Text Tasks The fusion of images and text within artificial intelligence applications, especially in the biomedical domain, introduces several unique challenges that require targeted solutions [363, 364].”
BackgroundPre-gating and contextual attention gate — A new fusion method for multi-modal data tasks
Zhang, D., Nayak, R., & Bashar, M. A. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106553 · Cited by 47
Computational Approaches to Drug Repurposing: Methods, Challenges, and Opportunities
Cousins, H. C., Nayar, G., & Altman, R. B. (2024) · Annual Review of Biomedical Data Science · doi:10.1146/annurev-biodatasci-110123-025333 · Cited by 48
Caruso, C. M., Guarrasi, V., Cordelli, E., Sicilia, R., Gentile, S., Messina, L., Fiore, M., Piccolo, C. L., Zobel, B. B., Iannello, G., et al. (2022) · Journal of Imaging · doi:10.3390/jimaging8110298 · Cited by 33
“The patients were then clinically followed until they were divided into two classes based on the median OS of all the patients: 95 dead and 96 alive.”
BackgroundDrouard, G., Mykkänen, J., Heiskanen, J. S., Pohjonen, J., Ruohonen, S., Pahkala, K., Lehtimäki, T. J., Wang, X., Ollikainen, M., Ripatti, S., et al. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02521-3 · Cited by 38
“These neural networks reduce the dimensions of omics data by linearly or non-linearly encoding them into lower-dimensional subspaces which can, in the case of several omics, be concatenated or pre-trained separately before integration [14, 15].”
BackgroundPerspectives on Advancing Multimodal Learning in Environmental Science and Engineering Studies
Liu, W., Chen, J., Wang, H., Fu, Z., Peijnenburg, W. J. G. M., & Hong, H. (2024) · Environmental Science & Technology · doi:10.1021/acs.est.4c03088 · Cited by 54
Hu, Y., Li, X., Yi, Y., Huang, Y., Wang, G., & Wang, D. (2025) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf121 · Cited by 39
“As medical data continue to accumulate, the evolution of precision medicine demands a more nuanced understanding,necessitating the integration of diverse data types, including genomics, histopathology, and clinical data [20, 21].”
Utilizing multimodal artificial intelligence to advance cardiovascular diseases
Yang, X.-Y., Li, Y., Wang, J., Jia, Y., Zhang, Y., & Chen, M. (2025) · Precision Clinical Medicine · doi:10.1093/pcmedi/pbaf016 · Cited by 49
“Advantages and disadvantages of commonly used multimodal fusion methods [ 7 , 9 , 17 , 112 , 113 ].”
Hierarchical multimodal self-attention-based graph neural network for DTI prediction
Bian, J., Lu, H., Dong, G., & Wang, G. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae293 · Cited by 31
“Multimodal learning is able to enhance feature representation by fusing features from multiple data sources to improve prediction performance [34].”
BackgroundTian, F., Zhang, L., Zhu, L., Zhao, M., Liu, J., Dong, Q., & Zhao, Q. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3420445 · Cited by 33
Integrating artificial intelligence with smartphone-based imaging for cancer detection in vivo
Song, B., & Liang, R. (2024) · Biosensors and Bioelectronics · doi:10.1016/j.bios.2024.116982 · Cited by 29
Guarrasi, V., & Soda, P. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.106625 · Cited by 33
“for CNNs working on image data, whereas Integrated Gradients should be used by both network typologies Sundararajan et al. (2017). The literature is well advanced in models which work on a single modality Arrieta et al.”
Wang, L., Wang, C., Li, C., Murai, T., Bai, Y., Song, Z., Zhang, S., Zhang, Q., Huang, Y., Bi, X., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01933-3 · Cited by 33
Getting the bugs out of AI : Advancing ecological research on arthropods through computer vision
Schneider, S., Taylor, G. W., Kremer, S. C., & Fryxell, J. M. (2023) · Ecology Letters · doi:10.1111/ele.14239 · Cited by 34
An artificial intelligence-driven revolution in orthopedic surgery and sports medicine
Guan, J., Li, Z., Sheng, S., Lin, Q., Wang, S., Wang, D., Chen, X., & Su, J. (2024) · International Journal of Surgery · doi:10.1097/js9.0000000000002187 · Cited by 33
Gao, Y., Wen, P., Liu, Y., Sun, Y., Qian, H., Zhang, X., Peng, H., Gao, Y., Li, C., Gu, Z., et al. (2025) · Journal of Translational Medicine · doi:10.1186/s12967-025-06428-z · Cited by 35
Cai, Y., Ke, L., Du, A., et al. (2025) · ACS Nano · doi:10.1021/acsnano.5c02822 · Cited by 27
Missing-modality enabled multi-modal fusion architecture for medical data
Wang, M., Fan, S., Li, Y., Xie, Z., & Chen, H. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104796 · Cited by 31
Integration of epigenetic regulatory mechanisms in heart failure
Sopić, M., Robinson, E. L., Emanueli, C., Srivastava, P. K., Angione, C., Gaetano, C., Condorelli, G., Martelli, F., Pedrazzini, T., Devaux, Y., et al. (2023) · Basic Research in Cardiology · doi:10.1007/s00395-023-00986-3 · Cited by 35
“It is now possible to explore inter-relationships between different layers of biological complexity through the construction of multi-modal machine learning systems, also taking into account mechanistically derived biological features [1, 78, 102].”
BackgroundSurveying the deep: A review of computer vision in the benthos
Trotter, C., Griffiths, H. J., & Whittle, R. J. (2025) · Ecological Informatics · doi:10.1016/j.ecoinf.2024.102989 · Cited by 29
Kwak, M. G., Mao, L., Zheng, Z., Su, Y., Lure, F. Y. M., Li, J., & Initiative, A. D. N. (2025) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2025.3556290 · Cited by 40
“The competing methods were as follows: • Multi-Late: a complete multimodal model that merges the decisions derived from individual modality models [62]; • Multi-Interm: a complete multimodal model that merges marginal representations from di ff erent modalities [62]; • Stagewise: a complete…”
Arco, J. E., Ortíz, A., Gallego-Molina, N. J., Górriz, J. M., & Ramírez, J. (2023) · International Journal of Neural Systems · doi:10.1142/s0129065723500193 · Cited by 31
Multimodal analysis methods in predictive biomedicine
Qoku, A., Katsaouni, N., Flinner, N., Buettner, F., & Schulz, M. H. (2023) · Computational and Structural Biotechnology Journal · doi:10.1016/j.csbj.2023.11.011 · Cited by 29
Surgical workflow recognition with temporal convolution and transformer for action segmentation
Zhang, B., Goel, B., Sarhan, M. H., Goel, V. K., Abukhalil, R., Kalesan, B., Stottler, N., & Petculescu, S. (2022) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-022-02811-z · Cited by 37
“Moreover, otherworks [41,42] introduce the idea of the model ensemble as well as different data fusion techniques [43].”
BackgroundHamed, S. K., Aziz, M. J. A., & Yaakub, M. R. (2024) · IEEE Access · doi:10.1109/access.2024.3443092 · Cited by 34
“However, it may fail to identify intermodal relationships evident at higher abstraction levels since it does not explicitly learn marginal representations [25].”
BackgroundMultimodal Data Hybrid Fusion and Natural Language Processing for Clinical Prediction Models
Ye, J., Hai, J., Song, J., & Wang, Z. (2023) · doi:10.1101/2023.08.24.23294597 · Cited by 24
Deep learning algorithms applied to computational chemistry
Pando, A. G., Ramírez-Alonso, G. M. D. J., Arzate-Quintana, C., & Camarillo‐Cisneros, J. (2023) · Molecular Diversity · doi:10.1007/s11030-023-10771-y · Cited by 29
Oei, C. W., Chan, Y. M., Zhang, X., Leo, K. H., Yong, E., Chong, L. R. C., Hong, Q., Zhang, L., Pan, Y., Tan, G. W. L., et al. (2024) · Journal of Diabetes Science and Technology · doi:10.1177/19322968241228606 · Cited by 31
Amador, K., Pinel, N., Winder, A. J., Fiehler, J., Wilms, M., & Forkert, N. D. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103381 · Cited by 28
Hypercomplex Multimodal Emotion Recognition from EEG and Peripheral Physiological Signals
Lopez, E., Chiarantano, E., Grassucci, E., & Comminiello, D. (2023) · IEEE International Conference on Acoustics, Speech, and Signal Processing Workshops (ICASSPW) · doi:10.1109/icasspw59220.2023.10193329 · Cited by 28
Li, Z., Wang, W., Liang, H., Li, Y., Zhang, Z., & Han, L. (2023) · Cancer Letters · doi:10.1016/j.canlet.2023.216369 · Cited by 25
Design of UAV Flight State Recognition System for Multisensor Data Fusion
Shi, Z., Shi, G., Zhang, J., et al. (2024) · IEEE Sensors Journal · doi:10.1109/jsen.2024.3394883 · Cited by 24
“In the field of machine learning, overlearning directly affects classification accuracy [29], [30], [31] and can increase the task size of machine learning.”
BackgroundApplications of Multimodal Artificial Intelligence in Non-Hodgkin Lymphoma B Cells
Isavand, P., Aghamiri, S. S., & Amin, R. (2024) · Biomedicines · doi:10.3390/biomedicines12081753 · Cited by 27
Botros, J., Mourad-Chehade, F., & Laplanche, D. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106997 · Cited by 27
Li, J., Lin, S., Zhang, L., Zhong, L., Ding, L., & Hu, Q. (2025) · Water Research · doi:10.1016/j.watres.2025.123405 · Cited by 30
Metadata
- Authors
- Stahlschmidt, S. R., Ulfenborg, B., & Synnergren, J.
- Affiliations
- University of Skövde (SE)
- Corresponding author
- Stahlschmidt, Sören Richard
- Venue
- Briefings in Bioinformatics
- Publisher
- Oxford University Press (OUP)
- Published
- 2022-01-28
- Volume
- 23
- Issue
- 2
- ISSN
- 1467-5463 (ISSN-L)
- 1477-4054
- PMC
- PMC8921642
- OpenAlex
- W4206042814
- Cited by
- 766
- References
- 117
- Citations by year
- 2026: 241 · 2025: 245 · 2024: 175 · 2023: 82 · 2022: 23
- FWCI
- 7.34
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Briefings in Bioinformatics (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- KTH Publication Database DiVA (KTH Royal Institute of Technology) (submittedVersion, other-oa)
- Funders
- University of Skövde, Sweden (20170302, 20200014)
- Topics
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Bioinformatics and Genomic Networks
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- multimodal fusion, biomedical data, deep learning, joint representation learning, intermediate fusion, transfer learning, biological knowledge integration
- MeSH terms
- Deep Learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.