Cited by 101Multimodal Machine Learning ApplicationsAdvanced Image Fusion TechniquesData Mining and Machine Learning Applications

The DOI matches its Crossref record; 117 of the 119 references it cites were found in a registry or an index.

Multimodal data fusion for systems improvement: A review

Gaw, Nathan, Yousefi, Safoora, Gahrooei, Mostafa Reisi·2022·IISE Transactions

DOI 10.1080/24725854.2021.1987593

Open this paper in CoChat — free account

Abstract

In recent years, information available from multiple data modalities has become increasingly common for industrial engineering and operations research applications. There have been a number of research works combining these data in unsupervised, supervised, and semi-supervised fashions that have addressed various issues of combining heterogeneous data, as well as several existing open challenges that remain to be addressed. In this review paper, we provide an overview of some methods for the fusion of multimodal data. We provide detailed real-world examples in manufacturing and medicine, introduce early, late, and intermediate fusion, as well as discuss several approaches under decomposition-based and neural network fusion paradigms. We summarize the capabilities and limitations of these methods and conclude the review article by discussing the existing challenges and potential research opportunities.

Integrity signals

Registry

Crossref · 10.1080/24725854.2021.1987593

Cites

117 of 117 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.1993.341010 · Cited by 14,475
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
Robust principal component analysis?
Candès, E. J., Li, X., Ma, Y., & Wright, J. N. (2011) · Journal of the ACM · doi:10.1145/1970392.1970395 · Cited by 7,048
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354 · cited 7×
“Karpathy et al. (2014) introduced a model that fuses video stream representations in a gradual manner, using multiple fusion layers, and were able to show the superiority of intermediate fusion to early and late fusion approaches.”
BackgroundInfluential
From few to many: illumination cone models for face recognition under variable lighting and pose
Georghiades, A. S., Belhumeur, P. N., & Kriegman, D. J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.927464 · Cited by 4,917
Ensemble learning: A survey
Sagi, O., & Rokach, L. (2018) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1249 · Cited by 3,103
PARAFAC. Tutorial and applications
Bro, R. (1997) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/s0169-7439(97)00032-4 · Cited by 2,824
Ensemble Machine Learning
Zhang, C., & Ma, Y. (2012) · Springer eBooks · doi:10.1007/978-1-4419-9326-7 · Cited by 2,314
Multisensor data fusion: A review of the state-of-the-art
Khaleghi, B., Khamis, A. M., Karray, F. O., & Razavi, S. (2011) · Information Fusion · doi:10.1016/j.inffus.2011.08.001 · Cited by 2,097
Deep learning for smart manufacturing: Methods and applications
Wang, J., Ma, Y., Zhang, L., Gao, R. X., & Wu, D. (2018) · Journal of Manufacturing Systems · doi:10.1016/j.jmsy.2018.01.003 · Cited by 1,740
Relations Between Two Sets of Variates
Hotelling, H. (1936) · Biometrika · doi:10.2307/2333955 · Cited by 1,663
Multi-Omics Factor Analysis—a framework for unsupervised integration of multi-omics data sets
Argelaguet, R., Velten, B., Arnol, D., Dietrich, S., Zenz, T., Marioni, J. C., Buettner, F., Huber, W., & Stegle, O. (2018) · Molecular Systems Biology · doi:10.15252/msb.20178124 · Cited by 1,510
Multimodal Data Fusion: An Overview of Methods, Challenges, and Prospects
Lahat, D., Adali, T., & Jutten, C. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2460697 · Cited by 1,296
Multimodal fusion for multimedia analysis: a survey
Atrey, P. K., Hossain, M. A., El Saddik, A., & Kankanhalli, M. S. (2010) · Multimedia Systems · doi:10.1007/s00530-010-0182-0 · Cited by 1,279
Deep multimodal learning: a survey on recent advances and trends
Ramachandram, D., & Taylor, G. W. (2017) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2738401 · Cited by 1,139 · cited 6×
“Atrey et al. (2010), Khaleghi et al. (2013), Lahat et al. (2014) and Ramachandram and Taylor (2017) have performed comprehensive reviews of these challenges and we provide these as additional references for the reader.”
BackgroundInfluential
Artificial intelligence and statistics
Wiggins, L. L. (1989) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(89)90027-6 · Cited by 1,116
A survey on deep learning for multimodal data fusion
Gao, J., Li, P., Chen, Z., & Zhang, J. (2020) · Neural Computation · doi:10.1162/neco_a_01273 · Cited by 821
Canonical analysis of several sets of variables
Kettenring, J. R. (1971) · Biometrika · doi:10.1093/biomet/58.3.433 · Cited by 798
On exploratory factor analysis: A review of recent evidence, an assessment of current practice, and recommendations for future use
Gaskin, C. J., & Happell, B. (2013) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2013.10.005 · Cited by 619
Tensor Regression with Applications in Neuroimaging Data Analysis
Zhou, H., Li, L., & Hongtu Zhu, B. (2013) · Journal of the American Statistical Association · doi:10.1080/01621459.2013.776499 · Cited by 563
Deep Learning Based Imaging Data Completion for Improved Brain Disease Diagnosis
Li, R., Zhang, W., Suk, H., Wang, L., Jiang, L., Shen, D. G., & Ji, S. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10443-0_39 · Cited by 487
Multimodal Fusion of Brain Imaging Data: A Key to Finding the Missing Link(s) in Complex Mental Illness
Calhoun, V. D., & Jing Sui （Beijing Normal University）， my correct affiliation is beijing normal university, N. Q. U. O. S. A. T. (2016) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2015.12.005 · Cited by 472
Contributions to mathematical psychology
Frederiksen, N., & Gulliksen, H. O. (1964) · Medical Entomology and Zoology · Cited by 458
Deep Dynamic Neural Networks for Multimodal Gesture Segmentation and Recognition
Wu, D., Pigou, L., Kindermans, P.-J., Le, N., Shao, L., Dambre, J., & Odobez, J. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2537340 · Cited by 451
ModDrop: Adaptive Multi-Modal Gesture Recognition
Neverova, N., Wolf, C., Taylor, G., & Nebout, F. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2461544 · Cited by 420
EmoNets: Multimodal deep learning approaches for emotion recognition in video
Kahou, S. E., Bouthillier, X., Lamblin, P., Gülçehre, Ç., Michalski, V., Konda, K., Jean, S., Froumenty, P., Dauphin, Y., Boulanger-Lewandowski, N., et al. (2015) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-015-0195-2 · Cited by 413
A novel multi-modality image fusion method based on image decomposition and sparse representation
Zhu, Z., Yin, H., Chai, Y., Li, Y., & Qi, G. (2017) · Information Sciences · doi:10.1016/j.ins.2017.09.010 · Cited by 411
Multilinear operators for higher-order decompositions.
Kolda, T. G. (2006) · doi:10.2172/923081 · Cited by 375
Data Fusion and Remote Sensing: An ever-growing relationship
Schmitt, M., & Zhu, X. X. (2016) · IEEE Geoscience and Remote Sensing Magazine · doi:10.1109/mgrs.2016.2561021 · Cited by 374
Deep ensemble learning of sparse regression models for brain disease diagnosis
Suk, H., Lee, S., & Shen, D. G. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.01.008 · Cited by 352
An Inter-Battery Method of Factor Analysis
Tucker, L. R. (1958) · Psychometrika · doi:10.1007/bf02289009 · Cited by 338
A Data-Level Fusion Model for Developing Composite Health Indices for Degradation Modeling and Prognostic Analysis
Liu, K., Gebraeel, N., & Shi, J. (2013) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2013.2250282 · Cited by 332
Proceedings of the Genetic and Evolutionary Computation Conference 2016
(2016) · doi:10.1145/2908812 · Cited by 300
Multimedia content processing through cross-modal association
Li, D., Dimitrova, N., Li, M., & Sethi, I. K. (2003) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/957013.957143 · Cited by 277
Urban big data fusion based on deep learning: An overview
Liu, J., Li, T., Xie, P., Du, S., Teng, F., & Yang, X. (2019) · Information Fusion · doi:10.1016/j.inffus.2019.06.016 · Cited by 277
Frameworks for latent variable multivariate regression
Burnham, A. J., Viveros, R., & MacGregor, J. F. (1996) · Journal of Chemometrics · doi:10.1002/(sici)1099-128x(199601)10:1 · Cited by 275
Emotion Recognition through Multiple Modalities: Face, Body Gesture, Speech
Castellano, G., Kessous, L., & Caridakis, G. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-85099-1_8 · Cited by 267
Multimodal medical image fusion based on IHS and PCA
He, C., Liu, Q., Li, H., & Wang, H. (2010) · Procedia Engineering · doi:10.1016/j.proeng.2010.11.045 · Cited by 253
A hybrid model approach for forecasting future residential electricity consumption
Dong, B., Li, Z., Rahman, S. M. M., & Vega, R. (2016) · Energy and Buildings · doi:10.1016/j.enbuild.2015.09.033 · Cited by 229
CCL: Cross-modal Correlation Learning With Multigrained Fusion by Hierarchical Network
Peng, Y., Qi, J., Huang, X., & Yuan, Y. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2017.2742704 · Cited by 228
SD-CNN: A shallow-deep CNN for improved breast cancer diagnosis
Gao, F., Wu, T., Li, J., Zheng, B., Ruan, L.-X., Shang, D., & Patel, B. (2018) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2018.09.004 · Cited by 221
Application of t-SNE to human genetic data
Li, W., Cerise, J. E., Yang, Y., & Han, H. (2017) · Journal of Bioinformatics and Computational Biology · doi:10.1142/s0219720017500172 · Cited by 205
Hybrid regression model for near real-time urban water demand forecasting
Brentan, B., Luvizotto, E., Herrera, M., Izquierdo, J., & Pérez-García, R. (2016) · Journal of Computational and Applied Mathematics · doi:10.1016/j.cam.2016.02.009 · Cited by 201
Marble
Ho, J. C., Ghosh, J., & Sun, J. (2014) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2623330.2623658 · Cited by 198
Rubik
Wang, Y., Chen, R. F., Ghosh, J., Denny, J. C., Kho, A. N., Chen, Y., Malin, B., & Sun, J. (2015) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2783258.2783395 · Cited by 159
Integration of Data Fusion Methodology and Degradation Modeling Process to Improve Prognostics
Liu, K., & Huang, S. (2014) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2014.2349733 · Cited by 156
Kernel Cross-Modal Factor Analysis for Information Fusion With Application to Bimodal Emotion Recognition
Wang, Y., Guan, L., & Venetsanopoulos, A. N. (2012) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2012.2189550 · Cited by 156
Remaining useful life prediction based on a multi-sensor data fusion model
Li, N., Gebraeel, N., Lei, Y., Fang, X., Cai, X. W., & Yan, T. (2020) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2020.107249 · Cited by 149
Higher Order Partial Least Squares (HOPLS): A Generalized Multilinear Regression Method
Zhao, Q., Caiafa, C. F., Mandic, D. P., Chao, Z. C., Nagasaka, Y., Fujii, N., Zhang, L., & Cichocki, A. S. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.254 · Cited by 148
Optimize the Signal Quality of the Composite Health Index via Data Fusion for Degradation Modeling and Prognostic Analysis
Liu, K., Chehade, A., & Song, C. (2015) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2015.2446752 · Cited by 147
Image-Based Process Monitoring Using Low-Rank Tensor Decomposition
Yan, H., Paynabar, K., & Shi, J. (2014) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2014.2327029 · Cited by 139
Multi-source heterogeneous data fusion
Zhang, L., Xie, Y., Xidao, L., & Zhang, X. (2018) · International Conference on Artificial Intelligence and Big Data (ICAIBD) · doi:10.1109/icaibd.2018.8396165 · Cited by 134
An overview of data fusion techniques for Internet of Things enabled physical activity recognition and measure
Qi, J., Yang, P., Newcombe, L., Peng, X., Yang, Y., & Zhao, Z. (2019) · Information Fusion · doi:10.1016/j.inffus.2019.09.002 · Cited by 133
Structure-revealing data fusion
Acar, E., Papalexakis, E. E., Gürdeniz, G., Rasmussen, M. A., Lawaetz, A. J., Nilsson, M., & Bro, R. (2014) · BMC Bioinformatics · doi:10.1186/1471-2105-15-239 · Cited by 123
Low-rank tensor decomposition based anomaly detection for hyperspectral imagery
Li, S., Wang, W., Qi, H., Ayhan, B., Kwan, C., & Vance, S. D. (2015) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2015.7351663 · Cited by 122
An Efficient Surrogate Model for Emulation and Physics Extraction of Large Eddy Simulations
Mak, S., Sung, C., Wang, X., Yeh, S.-T., Chang, Y., Joseph, V. R., Yang, V., & Wu, C. F. J. (2018) · Journal of the American Statistical Association · doi:10.1080/01621459.2017.1409123 · Cited by 121
Data Fusion in Metabolomics Using Coupled Matrix and Tensor Factorizations
Acar, E., Bro, R., & Smilde, A. K. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2438719 · Cited by 119
Tensor-on-Tensor Regression
Lock, E. F. (2017) · Journal of Computational and Graphical Statistics · doi:10.1080/10618600.2017.1401544 · Cited by 118
Understanding data fusion within the framework of coupled matrix and tensor factorizations
Acar, E., Rasmussen, M. A., Savorani, F., Næs, T., & Bro, R. (2013) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2013.06.006 · Cited by 108
Knowledge discovery from observational data for process control using causal Bayesian networks
Li, J., & Shi, J. (2007) · IIE Transactions · doi:10.1080/07408170600899532 · Cited by 99
Multimodal sparse and low-rank subspace clustering
Abavisani, M., & Patel, V. M. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.05.002 · Cited by 95
2023 31st European Signal Processing Conference (EUSIPCO)
(2023) · doi:10.23919/eusipco58844.2023 · Cited by 95
Bi-level multi-source learning for heterogeneous block-wise missing data
Xiang, S., Yuan, L., Fan, W., Wang, Y., Thompson, P. M., & Ye, J. (2013) · NeuroImage · doi:10.1016/j.neuroimage.2013.08.015 · Cited by 93
32nd European Signal Processing Conference (EUSIPCO 2024)
(2024) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco63174.2024.10714946 · Cited by 91
Statistical degradation modeling and prognostics of multiple sensor signals via data fusion: A composite health index approach
Song, C., & Liu, K. (2018) · IISE Transactions · doi:10.1080/24725854.2018.1440673 · Cited by 87
Multimodal Deep Learning Approach for Joint EEG-EMG Data Compression and Classification
Ben Said, A., Mohamed, A., Elfouly, T., Harras, K., & Wang, Z. J. (2017) · 2017 IEEE Wireless Communications and Networking Conference (WCNC) · doi:10.1109/wcnc.2017.7925709 · Cited by 86
Integration of machine learning and mechanistic models accurately predicts variation in cell density of glioblastoma using multiparametric MRI
Gaw, N. B., Hawkins-Daarud, A. J., Hu, L., Yoon, H., Wang, L., Xu, Y., Jackson, P., Singleton, K., Baxter, L. C., Eschbacher, J. M., et al. (2019) · Scientific Reports · doi:10.1038/s41598-019-46296-4 · Cited by 85
Self-citation
A data-level fusion approach for degradation modeling and prognostic analysis under multiple failure modes
Chehade, A., Song, C., Liu, K., Saxena, A., & Zhang, X. (2018) · Journal of Quality Technology · doi:10.1080/00224065.2018.1436829 · Cited by 80
A Method for Judicious Fusion of Inconsistent Multiple Sensor Data
Kumar, M., Garg, D. P., & Zachery, R. A. (2007) · IEEE Sensors Journal · doi:10.1109/jsen.2007.894905 · Cited by 75
Hybrid modeling based on mechanistic and data-driven approaches for cane sugar crystallization
Meng, Y., Yu, S., Zhang, J., Qin, J., Dong, Z., Lu, G., & Pang, H. (2019) · Journal of Food Engineering · doi:10.1016/j.jfoodeng.2019.03.026 · Cited by 71
A generic linked-mode decomposition model for data fusion
Mechelen, I. V., & Smilde, A. K. (2010) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/j.chemolab.2010.04.012 · Cited by 70
Weakly-Paired Maximum Covariance Analysis for Multimodal Dimensionality Reduction and Transfer Learning
Lampert, C. H., & Krömer, O. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15552-9_41 · Cited by 68
Multiscale fusion of multimodal medical images using lifting scheme based biorthogonal wavelet transform
Prakash, O., Park, C. M., Khare, A., Jeon, M., & Gwak, J. (2019) · Optik · doi:10.1016/j.ijleo.2018.12.028 · Cited by 66
An adaptive functional regression-based prognostic model for applications with missing data
Fang, X., Zhou, R., & Gebraeel, N. (2014) · Reliability Engineering & System Safety · doi:10.1016/j.ress.2014.08.013 · Cited by 64
Secure consensus averaging in sensor networks using random offsets
Kefayati, M., Talebi, M. S., Khalaj, B. H., & Rabiee, H. R. (2007) · International Conference on Telecommunications · doi:10.1109/ictmicc.2007.4448699 · Cited by 64
Remote Sensing Image Fusion Using Hierarchical Multimodal Probabilistic Latent Semantic Analysis
Fernández-Beltrán, R., Haut, J. M., Paoletti, M. E., Plaza, J., Plaza, A., & Pla, F. (2018) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2018.2881342 · Cited by 63
Structured Point Cloud Data Analysis Via Regularized Tensor Regression for Process Modeling and Optimization
Yan, H., Paynabar, K., & Pacella, M. (2018) · Technometrics · doi:10.1080/00401706.2018.1529628 · Cited by 60
A Novel Positive Transfer Learning Approach for Telemonitoring of Parkinson’s Disease
Yoon, H., & Li, J. (2018) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2018.2874233 · Cited by 58
Missing data imputation of questionnaires by means of genetic algorithms with different fitness functions
Ordóñez, C., Lasheras, F. S., Juez, F. J. D. C., & Bernardo-Sánchez, A. (2016) · Journal of Computational and Applied Mathematics · doi:10.1016/j.cam.2016.08.012 · Cited by 56
Model-Driven Sparse CP Decomposition for Higher-Order Tensors
Li, J., Choi, J. W., Perros, I., Sun, J., & Vuduc, R. W. (2017) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps.2017.80 · Cited by 54
Weakly correlated profile monitoring based on sparse multi-channel functional principal component analysis
Zhang, C., Yan, H., Lee, S., & Shi, J. (2018) · IISE Transactions · doi:10.1080/24725854.2018.1451012 · Cited by 53
A hybrid approach to integrate machine learning and process mechanics for the prediction of specific cutting energy
Liu, Z., & Guo, Y. B. (2018) · CIRP Annals · doi:10.1016/j.cirp.2018.03.015 · Cited by 53
New algorithm for integration between wireless microwave sensor network and radar for improved rainfall measurement and mapping
Liberman, Y., Samuels, R., Alpert, P., & Messer, H. (2014) · Atmospheric measurement techniques · doi:10.5194/amt-7-3549-2014 · Cited by 45
Cancer gene prioritization by integrative analysis of mRNA expression and DNA copy number data: a comparative review
Lahti, L., Schäfer, M. K.-H., Klein, H., Bicciato, S., & Dugas, M. (2012) · Briefings in Bioinformatics · doi:10.1093/bib/bbs005 · Cited by 44
Scalable prognostic models for large-scale condition monitoring applications
Fang, X., Gebraeel, N., & Paynabar, K. (2016) · IISE Transactions · doi:10.1080/24725854.2016.1264646 · Cited by 44
Mechanistic machine learning: how data assimilation leverages physiologic knowledge using Bayesian inference to forecast the future, infer the present, and phenotype
Albers, D. J., Levine, M. E., Stuart, A. M., Mamykina, L., Gluckman, B. J., & Hripcsak, G. M. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy106 · Cited by 44
Multimodal Probabilistic Latent Semantic Analysis for Sentinel-1 and Sentinel-2 Image Fusion
Fernández-Beltrán, R., Haut, J. M., Paoletti, M. E., Plaza, J., Plaza, A., & Pla, F. (2018) · IEEE Geoscience and Remote Sensing Letters · doi:10.1109/lgrs.2018.2843886 · Cited by 40
Image-Based Prognostics Using Penalized Tensor Regression
Fang, X., Paynabar, K., & Gebraeel, N. (2018) · Technometrics · doi:10.1080/00401706.2018.1527727 · Cited by 39
A novel approach for multimodality medical image fusion over secure environment
Kumar, P., & Diwakar, M. (2020) · Transactions on Emerging Telecommunications Technologies · doi:10.1002/ett.3985 · Cited by 38
Identification of structural features in chemicals associated with cancer drug response: a systematic data-driven analysis
Khan, S. A., Virtanen, S., Kallioniemi, O., Wennerberg, K., Poso, A., & Samuel Kaski, A. (2014) · Bioinformatics · doi:10.1093/bioinformatics/btu456 · Cited by 37
Multimodal, multitemporal, and multisource global data fusion for local climate zones classification based on ensemble learning
Yokoya, N., Ghamisi, P., & Xia, J. (2017) · IEEE International Geoscience and Remote Sensing Symposium (IGARSS) · doi:10.1109/igarss.2017.8127172 · Cited by 34
Quantitative prediction of the aged state of Ni-base superalloys using PCA and tensor regression
Gorgannejad, S., Gahrooei, M. R., Paynabar, K., & Neu, R. W. (2018) · Acta Materialia · doi:10.1016/j.actamat.2018.11.047 · Cited by 31
Self-citation
Robust heart rate estimation from multimodal physiological signals using beat signal quality index based majority voting fusion method
Rankawat, S. A., & Dubey, R. (2016) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2016.12.004 · Cited by 29
A generic framework for multisensor degradation modeling based on supervised classification and failure surface
Song, C., Liu, K., & Zhang, X. (2019) · IISE Transactions · doi:10.1080/24725854.2018.1555384 · Cited by 29
Detect depression from communication: how computer vision, signal processing, and sentiment analysis join forces
Samareh, A., Jin, Y., Wang, Z., Chang, X., & Huang, S. (2018) · IISE Transactions on Healthcare Systems Engineering · doi:10.1080/24725579.2018.1496494 · Cited by 28
Bayesian optimization on graph-structured search spaces: Optimizing deep multimodal fusion architectures
Ramachandram, D., Lisicki, M., Shields, T. J., Amer, M. R., & Taylor, G. W. (2018) · Neurocomputing · doi:10.1016/j.neucom.2017.11.071 · Cited by 27
An ensemble learning method for asthma control level detection with leveraging medical knowledge-based classifier and supervised learning
Khasha, R., Sepehri, M. M., & Mahdaviani, S. A. (2019) · Journal of Medical Systems · doi:10.1007/s10916-019-1259-8 · Cited by 26
Informative Sensor and Feature Selection via Hierarchical Nonnegative Garrote
Paynabar, K., Jin, J., & Reed, M. P. (2015) · Technometrics · doi:10.1080/00401706.2014.947383 · Cited by 21
Weighted-PCA Based Multimodal Medical Image Fusion in Contourlet Domain
Moin, A., Bhateja, V., & Srivastava, A. (2016) · Advances in intelligent systems and computing · doi:10.1007/978-981-10-0755-2_63 · Cited by 21
Process Modeling and Prediction With Large Number of High-Dimensional Variables Using Functional Regression
Gahrooei, M. R., Paynabar, K., Pacella, M., & Shi, J. (2019) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2019.2941167 · Cited by 20
Self-citation
Deep fusion of heterogeneous sensor data
Liu, Z., Zhang, W., Quek, T. Q. S., & Lin, S. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7953301 · Cited by 19
Tensor Mixed Effects Model With Application to Nanomanufacturing Inspection
Yue, X., Park, J. G., Liang, Z., & Shi, J. (2019) · Technometrics · doi:10.1080/00401706.2019.1592783 · Cited by 19
Transcriptional response networks for elucidating mechanisms of action of multitargeted agents
Kibble, M., Khan, S. A., Saarinen, N., Iorio, F., Sáez-Rodríguez, J., Mäkelä, S., & Aittokallio, T. A. (2016) · Drug Discovery Today · doi:10.1016/j.drudis.2016.03.001 · Cited by 18
High‐dimensional longitudinal classification with the multinomial fused lasso
Adhikari, S., Lecci, F., Becker, J. T., Junker, B. W., Kuller, L. H., López, O. L., & Tibshirani, R. J. (2019) · Statistics in Medicine · doi:10.1002/sim.8100 · Cited by 17
Diagnosis of Alzheimer’s Disease Using View-Aligned Hypergraph Learning with Incomplete Multi-modality Data
Liu, M., Zhang, J., Yap, P., & Shen, D. G. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46720-7_36 · Cited by 17
Black-box Solar Performance Modeling
Chen, D., & Irwin, D. (2017) · ACM SIGMETRICS Performance Evaluation Review · doi:10.1145/3152042.3152067 · Cited by 16
A clinical decision support system using multi-modality imaging data for disease diagnosis
Gaw, N. B., Schwedt, T. J., Chong, C. D., Wu, T., & Li, J. (2017) · IISE Transactions on Healthcare Systems Engineering · doi:10.1080/24725579.2017.1403520 · Cited by 16
Self-citation
The Application of Heterogeneous Information Fusion in Misalignment Fault Diagnosis of Wind Turbines
Xiao, Y., Wang, Y., & Ding, Z. (2018) · Energies · doi:10.3390/en11071655 · Cited by 15
From drug response profiling to target addiction scoring in cancer cell models
Yadav, B., Gopalacharyulu, P. V., Pemovska, T., Khan, S. A., Szwajda, A., Tang, J. P., Wennerberg, K., & Aittokallio, T. A. (2015) · Disease Models & Mechanisms · doi:10.1242/dmm.021105 · Cited by 13
Hybrid Statistical and Mechanistic Mathematical Model Guides Mobile Health Intervention for Chronic Pain
Clifton, S. M., Kang, C., Li, J., Long, Q., Shah, N., & Abrams, D. M. (2017) · J. Comput. Biol. · doi:10.1089/cmb.2017.0059 · Cited by 11
Confidence fusion
Siegel, M., & Wu, H. (2004) · doi:10.1109/rose.2004.1317621 · Cited by 9
On Secure Consensus Information Fusion over Sensor Networks
Kefayati, M., Talebi, M. S., Rabiee, H. R., & Khalaj, B. H. (2007) · IEEE/ACS International Conference on Computer Systems and Applications (AICCSA) · doi:10.1109/aiccsa.2007.370871 · Cited by 9
An adaptive fused sampling approach of high-accuracy data in the presence of low-accuracy data
Gahrooei, M. R., Paynabar, K., Pacella, M., & Colosimo, B. M. (2018) · IISE Transactions · doi:10.1080/24725854.2018.1540901 · Cited by 9
Self-citation
Joint intermodal and intramodal correlation preservation for semi-paired learning
Guo, X., Wang, S., Tie, Y., Qi, L., & Guan, L. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.03.013 · Cited by 8
Impact of noise correlation on multimodality
Chlaily, S., Amblard, P., Michel, O., & Christian Jutten, B. (2016) · European Signal Processing Conference (EUSIPCO) · doi:10.1109/eusipco.2016.7760237 · Cited by 7
Dynamical reconfiguration strategy of a multi sensor data fusion algorithm based on Information Theory
Tmazirte, N. A., Najjar, M. E. B. E., Smaili, C., & Pomorski, D. (2013) · IEEE Intelligent Vehicles Symposium · doi:10.1109/ivs.2013.6629580 · Cited by 6
Study on missing data imputation and modeling for the leaching process
He, D., Wang, Z., Yang, L., & Dai, W. (2017) · Process Safety and Environmental Protection · doi:10.1016/j.cherd.2017.05.023 · Cited by 6

Research works printed but not linked: 25

Elhamifar E., 2016, Advances in Neural Information Processing Systems, 73
Gahrooei M.R., 2020, Technometrics
Gao, H. (2020) Coordination and Privacy Preservation in Multi-Agent Systems (Doctoral dissertation, Clemson University).
Ilarshman RA., 1970, UCLA Working Papers in Phonetics, 16, 1
Joze H.R.V., 2020, Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13289
Klami A., 2013, Journal of Machine Learning Research, 14, 965
Lahat D., 2014, 2014 22nd European Signal Processing Conference (EUSIPCO), 101
Li Q., 2019, arXiv preprint arXiv:1911.04056 · arXiv
Li Y., 2018, Briefings in Bioinformatics, 19, 325
Liu D., 2020, arXiv preprint arXiv:2004.04019 · arXiv
Liu X., 2020, medRxiv
Mascheroni P., 2020, bioRxiv
Ng A., 2011, CS294A Lecture Notes, 72, 1
Rajalingam B., 2017, International Journal of Engineering and Manufacturing Science, 7, 22
Seyyedi H., 2010, Comparing satellite derived rainfall with ground based radar for Northwestern Europe
Si B., 2020, IISE Transactions, 1
Simonyan K., 2014, Advances in Neural Information Processing Systems, 27, 568
Şimşekli U., 2013, 21st European Signal Processing Conference (EUSIPCO 2013), 1
Srivastava N., 2012, Proceedings of Neural Information and Processing Systems
Van der Maaten L., 2008, Journal of Machine Learning Research, 9, 2579
Vielzeuf V., 2018, Proceedings of the European Conference on Computer Vision (ECCV)
White M., 2012, Artificial Intelligence and Statistics, 1305
Zhang C., 2020, Technometrics, 1
Zhang, C., Yan, H., Lee, S. and Shi, J. (2018b) Weakly correlated profile monitoring based on sparse multi-channel functional principal component analysis. IISE Transactions , 50 (10), 878–891. · DOI
Zhang X., 2011, Twenty-Fifth AAAI Conference on Artificial Intelligence

Sorted by citations

A survey of multimodal information fusion for smart healthcare: Mapping the journey from data to wisdom
Shaik, T., Tao, X., Li, L., Xie, H., & Velasquez, J. D. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102040 · Cited by 228
Diagnosis of brain diseases in fusion of neuroimaging modalities using deep learning: A review
Shoeibi, A., Khodatars, M., Jafari, M., Ghassemi, N., Moridian, P., Alizadehsani, R., Ling, S. H., Khosravi, A., Alinejad‐Rokny, H., Lam, H., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.12.010 · Cited by 110
Advancing healthcare through multimodal data fusion: a comprehensive review of techniques and applications
Teoh, J. R., Dong, J., Zuo, X., Lai, K. W., Hasikin, K. B., & Wu, X. (2024) · PeerJ Computer Science · doi:10.7717/peerj-cs.2298 · Cited by 117
An image and text-based multimodal model for detecting fake news in OSN’s
Uppada, S. K., Patel, P. K., & Sivaselvan, B. (2022) · Journal of Intelligent Information Systems · doi:10.1007/s10844-022-00764-y · Cited by 75
Smart Contract Vulnerability Detection Based on Deep Learning and Multimodal Decision Fusion
Deng, W., Wei, H., Huang, T., Cao, C., Peng, Y., & Hu, X. (2023) · Sensors · doi:10.3390/s23167246 · Cited by 63

See all 101

Authors: Gaw, N., Yousefi, S., & Gahrooei, M. R.
Affiliations: Wright-Patterson Air Force Base (US)
Microsoft (United States) (US)
University of Florida (US)
Corresponding author: Gahrooei, Mostafa Reisi
ORCID: Gaw, Nathan
Yousefi, Safoora
Gahrooei, Mostafa Reisi
Venue: IISE Transactions
Publisher: Informa UK Limited
Published: 2021-12-03
Volume: 54
Issue: 11
Pages: 1098-1116
ISSN: 2472-5854 (ISSN-L)
2472-5862
DOI: 10.1080/24725854.2021.1987593
OpenAlex: W3210892971
Cited by: 101
References: 119
Citations by year: 2026: 20 · 2025: 28 · 2024: 33 · 2023: 14 · 2022: 5 · 2021: 1
FWCI: 4.77
Type: journal article
Language: en
Open access: closed
Locations: IISE Transactions (publishedVersion)
RePEc: Research Papers in Economics (submittedVersion)
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Physical Sciences › Engineering › Media Technology › Advanced Image Fusion Techniques
Physical Sciences › Computer Science › Information Systems › Data Mining and Machine Learning Applications
Keywords: multimodal fusion, data heterogeneity, feature-level fusion, decision fusion, intermediate fusion, neural network fusion, manufacturing industry, health data, industrial engineering, operations research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Multimodal data fusion for systems improvement: A review

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks