Advanced Neural Networks for Multimodal Data Fusion in Interdisciplinary Research
Abstract
The chapter will delve into deep neural network architectures developed for interdisciplinary research, specifically aimed at combining multimodal data. Modern research, spanning multiple domains like health and environmental science, necessitates the integration of diverse data sources like images, text, and sensor data. This discussion will explore advanced techniques like convolutional neural networks, recurrent neural networks, and transformer models that effectively combine these modalities. This paper emphasizes the development of robust multimodal systems, primarily focusing on data alignment and feature extraction prior to model training. The text discusses the challenges of heterogeneous data integration, its solutions, and the role of transfer learning in improving model performance. The chapter explores the potential of this approach in medical diagnostics, climate modeling, and social sciences, offering insights into future research directions and applications.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Transfer learning allows models to apply knowledge acquired from one task to different yet related tasks, significantly reducing the need for large labeled datasets.
“Transfer learning emphasizes versatility and efficiency in AI training, allowing models to apply knowledge acquired from one task to different yet related tasks, which significantly reduces the need for large labeled datasets [186], [187].”
From From Google Gemini to OpenAI Q* (Q-Star): A Survey on Reshaping the Generative Artificial Intelligence (AI) Research Landscape (2025) · cites this paper as [187]
Cites
A review of affective computing: From unimodal analysis to multimodal fusion
Poria, S., Cambria, E., Bajpai, R., & Hussain, A. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.003 · Cited by 1,644 · see citing works
Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects
Lahat, D., Adali, T., & Jutten, C. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2460697 · Cited by 1,296 · see citing works
A survey on deep learning for multimodal data fusion
Gao, J., Li, P., Chen, Z., & Zhang, J. (2020) · Neural Computation · doi:10.1162/neco_a_01273 · Cited by 821 · see citing works
Harnessing multimodal data integration to advance precision oncology
Boehm, K. M., Khosravi, P., Vanguri, R., Gao, J., & Shah, S. P. (2022) · Nature Reviews Cancer · doi:10.1038/s41568-021-00408-3 · Cited by 662 · see citing works
Deep learning in multimodal remote sensing data fusion: A comprehensive review
Li, J., Hong, D., Gao, L., Yao, J., Zheng, K., Zhang, B., & Chanussot, J. (2022) · International Journal of Applied Earth Observation and Geoinformation · doi:10.1016/j.jag.2022.102926 · Cited by 649 · see citing works
Ghamisi, P., Gloaguen, R., Atkinson, P. M., Benediktsson, J. A., Rasti, B., Yokoya, N., Wang, Q., Höfle, B., Bruzzone, L., Bovolo, F., et al. (2019) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2018.2890023 · Cited by 638 · see citing works
Azam, M. A., Khan, K. B., Salahuddin, S., Rehman, E., Khan, S. A., Khan, M. A., Kadry, S., & Gandomi, A. H. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105253 · Cited by 449 · see citing works
Advances in multimodal data fusion in neuroimaging: Overview, challenges, and novel orientation
Zhang, Y.-D., Dong, Z.-C., Wang, S.-H., Yu, X., Yao, X., Zhou, Q., Hu, H., Li, M., Jiménez-Mesa, C., Ramírez, J., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.07.006 · Cited by 409 · see citing works
Multimodal medical image fusion review: Theoretical background and recent advances
Hermessi, H., Mourali, O., & Zagrouba, E. (2021) · Signal Processing · doi:10.1016/j.sigpro.2021.108036 · Cited by 306 · see citing works
A comparative review on multi-modal sensors fusion based on deep learning
Tang, Q., Liang, J., & Zhu, F. (2023) · Signal Processing · doi:10.1016/j.sigpro.2023.109165 · Cited by 292 · see citing works
Deep learning based multimodal biomedical data fusion: an overview and comparative review
Duan, J., Xiong, J., Li, Y., & Ding, W. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102536 · Cited by 284 · see citing works
L(1/2) Regularization for ISAR Imaging and Target Enhancement of Complex Image
Wu, A., Wu, Y., Jin, Y., Wang, Y., & Guo, Z. (2020) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.3040277 · Cited by 249 · see citing works
Deep multimodal fusion of image and non-image data in disease diagnosis and prognosis: a review
Cui, C., Yang, H., Wang, Y., Zhao, S., Asad, Z., Coburn, L. A., Wilson, K. T., Landman, B. A., & Huo, Y. (2023) · Progress in Biomedical Engineering · doi:10.1088/2516-1091/acc2fe · Cited by 239 · see citing works
Jiang, Y., Li, W., Hossain, M. S., Chen, M., Alelaiwi, A. A., & Al-Hammadi, M. H. (2019) · Information Fusion · doi:10.1016/j.inffus.2019.06.019 · Cited by 219 · see citing works
Multimodal data integration for oncology in the era of deep neural networks: a review
Waqas, A., Tripathi, A., Ramachandran, R. P., Stewart, P. A., & Rasool, G. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1408843 · Cited by 165 · see citing works
Multimodal Data Fusion in Learning Analytics: A Systematic Review
Mu, S., Cui, M., & Huang, X. (2020) · Sensors · doi:10.3390/s20236856 · Cited by 157 · see citing works
Multimodal Sensors and ML‐Based Data Fusion for Advanced Robots
Duan, S., Shi, Q., & Wu, J. L. (2022) · Advanced Intelligent Systems · doi:10.1002/aisy.202200213 · Cited by 87 · see citing works
RGB-D-Based Object Recognition Using Multimodal Convolutional Neural Networks: A Survey
Gao, M., Jiang, J., Zou, G., John, V., & Liu, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2907071 · Cited by 63 · see citing works
Multimodal Medical Image Fusion Techniques – A Review
Tirupal, T., Mohan, B. C., & Kumar, S. S. (2020) · Current Signal Transduction Therapy · doi:10.2174/1574362415666200226103116 · Cited by 61 · see citing works
Self-citationA Study on the Complex Nature of Higher Education Leadership
Prabhuswamy, M. B., Tripathi, R., Vijayakumar, M. D., Thulasimani, T., Sundharesalingam, P., & Boopathi, S. (2024) · Advances in library and information science (ALIS) book series · doi:10.4018/979-8-3693-1371-8.ch013 · Cited by 60 · see citing works
Bokade, R., Navato, A. P., Ouyang, R., Jin, X., Chou, C.-A., Ostadabbas, S., & Mueller, A. V. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113885 · Cited by 59 · see citing works
Tan, C. N. W., Ceballos, G. A., Kasabov, N. K., & Subramaniyam, N. P. (2020) · Sensors · doi:10.3390/s20185328 · Cited by 57 · see citing works
Green Supply Chain Management in Eco-Friendly Sustainable Manufacturing Industries
Vangeri, A., Bathrinath, S., Anand, M. C. J., Shanmugathai, M., Meenatchi, N., & Boopathi, S. (2024) · Advances in chemical and materials engineering book series · doi:10.4018/979-8-3693-3625-0.ch010 · Cited by 52 · see citing works
Psychological Impacts, Prevention Strategies, and Intervention Approaches Across Age Groups
Kushwah, J. S., Gupta, M., Shrivastava, S. R., Saxena, N., Saini, R., & Boopathi, S. (2024) · Advances in medical technologies and clinical practice book series · doi:10.4018/979-8-3693-3555-0.ch005 · Cited by 30 · see citing works
Future Developments of Higher Education on Social Psychology
Kalaiselvi, D., Ramaratnam, M. S., Kokila, S., Sarkar, R., Anandakumar, S., & Boopathi, S. (2024) · Advances in human and social aspects of technology book series · doi:10.4018/979-8-3693-2569-8.ch008 · Cited by 30 · see citing works
Multimodal Data Fusion for Systems Improvement
Gaw, N. B., Yousefi, S., & Gahrooei, M. R. (2022) · CRC Press eBooks · doi:10.1201/9781003220978-7 · Cited by 28 · see citing works
Study on Environmental and Social Impacts Through Electric Vehicles
Rajan, T. S., Rex, C. R. E. S., Shastri, D. S., Naidu, G., Chandrasekaran, S. K., & Boopathi, S. (2024) · Advances in mechatronics and mechanical engineering (AMME) book series · doi:10.4018/979-8-3693-4314-2.ch013 · Cited by 15 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
McIntosh, T. R., Susnjak, T., Liu, T., Watters, P., Xu, D., Liu, D., & Halgamuge, M. N. (2025) · Technologies · doi:10.3390/technologies13020051 · Cited by 102 · see citing works
Harmonizer: A Universal Signal Tokenization Framework for Multimodal Large Language Models
Amiri, A., Ghaffarnia, A., Nia, N. G., Wu, D., & Liang, Y. (2025) · Mathematics · doi:10.3390/math13111819 · Cited by 6 · see citing works
Kaur, S., & Sharma, H. (2025) · Discover Sustainability · doi:10.1007/s43621-025-02317-z · Cited by 6 · see citing works
Qutaishat, D., & Li, S. (2025) · ISPRS International Journal of Geo-Information · doi:10.3390/ijgi14110434 · Cited by 5 · see citing works
- Authors
- Dhivya, K., Kumar, S. N., Victoria, D. R. S., Sherly, S. I., & Durgadevi, G.
- Affiliations
- SRM Institute of Science and Technology (IN)
- St. Joseph’s College of Engineering (IN)
- Department of Computer Science and Engineering, Chennai Institute of Technology, Chennai, India
- Saveetha University (IN)
- Department of Electronics and Communication Engineering, New Prince Shri Bhavani College of Engineering and Technology, Chennai, India
- Venue
- Advances in IT Standards and Standardization Research
- Publisher
- IGI Global
- Published
- 2024-12-27
- Pages
- 201-232
- ISSN
- 1935-3391 (ISSN-L)
- 1935-3405
- OpenAlex
- W4405530131
- Cited by
- 4
- References
- 27
- Citations by year
- 2025: 4
- FWCI
- 3.11
- Type
- book chapter
- Language
- en
- Open access
- closed
- Locations
- Advances in IT standards and standardization research (AISSR) book series/Advances in IT standards and standardization research series (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Engineering › Media Technology › Advanced Image Fusion Techniques
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Multisensory perception and integration
- Keywords
- multimodal fusion, deep neural network, convolutional neural networks, recurrent neural networks, transformer models, data alignment, feature extraction, transfer learning, heterogeneous data integration, medical diagnostics, climate modeling
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
