The DOI matches its Crossref record; 117 of the 119 references it cites were found in a registry or an index.
Multimodal data fusion for systems improvement: A review
Abstract
In recent years, information available from multiple data modalities has become increasingly common for industrial engineering and operations research applications. There have been a number of research works combining these data in unsupervised, supervised, and semi-supervised fashions that have addressed various issues of combining heterogeneous data, as well as several existing open challenges that remain to be addressed. In this review paper, we provide an overview of some methods for the fusion of multimodal data. We provide detailed real-world examples in manufacturing and medicine, introduce early, late, and intermediate fusion, as well as discuss several approaches under decomposition-based and neural network fusion paradigms. We summarize the capabilities and limitations of these methods and conclude the review article by discussing the existing challenges and potential research opportunities.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“This method typically employs attention mechanisms, feature weighting, connectivity, or GNNs to combine modality specific features [41,42].”
From Multimodal fusion and vision–language models: A survey for robot vision (2026) · cites this paper as [41]
Citations
Cites
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.1993.341010 · Cited by 14,475
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Robust principal component analysis?
Candès, E. J., Li, X., Ma, Y., & Wright, J. N. (2011) · Journal of the ACM · doi:10.1145/1970392.1970395 · Cited by 7,048
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354 · cited 7×
“Karpathy et al. (2014) introduced a model that fuses video stream representations in a gradual manner, using multiple fusion layers, and were able to show the superiority of intermediate fusion to early and late fusion approaches.”
BackgroundInfluentialFrom few to many: illumination cone models for face recognition under variable lighting and pose
Georghiades, A. S., Belhumeur, P. N., & Kriegman, D. J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.927464 · Cited by 4,917
Sagi, O., & Rokach, L. (2018) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1249 · Cited by 3,103
PARAFAC. Tutorial and applications
Bro, R. (1997) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/s0169-7439(97)00032-4 · Cited by 2,824
Zhang, C., & Ma, Y. (2012) · Springer eBooks · doi:10.1007/978-1-4419-9326-7 · Cited by 2,314
Multisensor data fusion: A review of the state-of-the-art
Khaleghi, B., Khamis, A. M., Karray, F. O., & Razavi, S. (2011) · Information Fusion · doi:10.1016/j.inffus.2011.08.001 · Cited by 2,097
Deep learning for smart manufacturing: Methods and applications
Wang, J., Ma, Y., Zhang, L., Gao, R. X., & Wu, D. (2018) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2018.01.003 · Cited by 1,740
Research works printed but not linked: 25
- Elhamifar E., 2016, Advances in Neural Information Processing Systems, 73
- Gahrooei M.R., 2020, Technometrics
- Gao, H. (2020) Coordination and Privacy Preservation in Multi-Agent Systems (Doctoral dissertation, Clemson University).
- Ilarshman RA., 1970, UCLA Working Papers in Phonetics, 16, 1
- Joze H.R.V., 2020, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13289
- Klami A., 2013, Journal of Machine Learning Research, 14, 965
- Lahat D., 2014, 2014 22nd European Signal Processing Conference (EUSIPCO), 101
- Li Q., 2019, arXiv preprint arXiv:1911.04056 · arXiv
- Li Y., 2018, Briefings in Bioinformatics, 19, 325
- Liu D., 2020, arXiv preprint arXiv:2004.04019 · arXiv
- Liu X., 2020, medRxiv
- Mascheroni P., 2020, bioRxiv
- Ng A., 2011, CS294A Lecture Notes, 72, 1
- Rajalingam B., 2017, International Journal of Engineering and Manufacturing Science, 7, 22
- Seyyedi H., 2010, Comparing satellite derived rainfall with ground based radar for Northwestern Europe
- Si B., 2020, IISE Transactions, 1
- Simonyan K., 2014, Advances in Neural Information Processing Systems, 27, 568
- Şimşekli U., 2013, 21st European Signal Processing Conference (EUSIPCO 2013), 1
- Srivastava N., 2012, Proceedings of Neural Information and Processing Systems
- Van der Maaten L., 2008, Journal of Machine Learning Research, 9, 2579
- Vielzeuf V., 2018, Proceedings of the European Conference on Computer Vision (ECCV)
- White M., 2012, Artificial Intelligence and Statistics, 1305
- Zhang C., 2020, Technometrics, 1
- Zhang, C., Yan, H., Lee, S. and Shi, J. (2018b) Weakly correlated profile monitoring based on sparse multi-channel functional principal component analysis. IISE Transactions , 50 (10), 878–891. · DOI
- Zhang X., 2011, Twenty-Fifth AAAI Conference on Artificial Intelligence
Cited by
Sorted by citations
Shaik, T., Tao, X., Li, L., Xie, H., & Velasquez, J. D. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102040 · Cited by 228
Diagnosis of brain diseases in fusion of neuroimaging modalities using deep learning: A review
Shoeibi, A., Khodatars, M., Jafari, M., Ghassemi, N., Moridian, P., Alizadehsani, R., Ling, S. H., Khosravi, A., Alinejad‐Rokny, H., Lam, H., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.12.010 · Cited by 110
Teoh, J. R., Dong, J., Zuo, X., Lai, K. W., Hasikin, K. B., & Wu, X. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.2298 · Cited by 117
An image and text-based multimodal model for detecting fake news in OSN’s
Uppada, S. K., Patel, P. K., & Sivaselvan, B. (2022) · Journal of Intelligent Information Systems · doi:10.1007/s10844-022-00764-y · Cited by 75
Smart Contract Vulnerability Detection Based on Deep Learning and Multimodal Decision Fusion
Deng, W., Wei, H., Huang, T., Cao, C., Peng, Y., & Hu, X. (2023) · Sensors · doi:10.3390/s23167246 · Cited by 63
Metadata
- Authors
- Gaw, N., Yousefi, S., & Gahrooei, M. R.
- Affiliations
- Wright-Patterson Air Force Base (US)
- Microsoft (United States) (US)
- University of Florida (US)
- Corresponding author
- Gahrooei, Mostafa Reisi
- Venue
- IISE Transactions
- Publisher
- Informa UK Limited
- Published
- 2021-12-03
- Volume
- 54
- Issue
- 11
- Pages
- 1098-1116
- ISSN
- 2472-5854 (ISSN-L)
- 2472-5862
- OpenAlex
- W3210892971
- Cited by
- 101
- References
- 119
- Citations by year
- 2026: 20 · 2025: 28 · 2024: 33 · 2023: 14 · 2022: 5 · 2021: 1
- FWCI
- 4.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IISE Transactions (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Engineering › Media Technology › Advanced Image Fusion Techniques
- Physical Sciences › Computer Science › Information Systems › Data Mining and Machine Learning Applications
- Keywords
- multimodal fusion, data heterogeneity, feature-level fusion, decision fusion, intermediate fusion, neural network fusion, manufacturing industry, health data, industrial engineering, operations research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
