The DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.
MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis
Open accessAbstract
Multimodal Sentiment Analysis is an active area of research that leverages multimodal signals for affective understanding of user-generated videos. The predominant approach, addressing this task, has been to develop sophisticated fusion techniques. However, the heterogeneous nature of the signals creates distributional modality gaps that pose significant challenges. In this paper, we aim to learn effective modality representations to aid the process of fusion. We propose a novel framework, MISA, which projects each modality to two distinct subspaces. The first subspace is modality-invariant, where the representations across modalities learn their commonalities and reduce the modality gap. The second subspace is modality-specific, which is private to each modality and captures their characteristic features. These representations provide a holistic view of the multimodal data, which is used for fusion that leads to task predictions. Our experiments on popular sentiment analysis benchmarks, MOSI and MOSEI, demonstrate significant gains over state-of-the-art models. We also consider the task of Multimodal Humor Detection and experiment on the recently proposed UR_FUNNY dataset. Here too, our model fares better than strong baselines, establishing MISA as a useful multimodal framework.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“On CMU-MOSEI dataset, our proposed C-MIB yields 0.7% improvement on Acc2 and 1.2% improvement on F1 score compared to the current state-of-the-art MISA [53]. The performance on Acc7 and Corr is also improved compared to the state-of-the-art methods.”
From Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representations (2023) · cites this paper as [53]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · cited 8×
“Inter-utterance context. et al. proposed one of the first models, bc-LSTM , which utilized”
MethodInfluentialBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 1×
“However, following recent works [7], including the stateof-the-art ICCN [50], we utilize the pre-trained BERT [11] as the feature extractor for textual utterances.”
MethodGlove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 1×
“ndard low-level features that are provided by the respective benchmarks and utilized by the state-of-the-art methods. 4.3.1 Language Features. Traditionally, language modality features has been GloVe [38] embeddings for each token in the utterance. However, following recent works [7], including the stateof-the-art ICC…”
MethodInfluential(2008) · Journal of Machine Learning Research · Cited by 33,051
Multimodal Transformer for Unaligned Multimodal Language Sequences
Tsai, Y.-H. H., Bai, S., Liang, P. P., Kolter, J. Z., Morency, L.-P., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1656 · Cited by 2,259
Tensor Fusion Network for Multimodal Sentiment Analysis
Zadeh, A., Chen, M., Poria, S., Cambria, E., & Morency, L.-P. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1115 · Cited by 2,102 · cited 5×
“licly available models of [26, 51, 58]).”
BackgroundInfluentialSelf-citationEkman, P., & Rosenberg, E. L. (2005) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195179644.001.0001 · Cited by 1,971 · cited 1×
“Both MOSI and MOSEI use Facet 3 to extract facial expression features, which include facial action units and face pose based on the Facial Action Coding System (FACS) [14].”
MethodOpenFace: An open source facial behavior analysis toolkit
Baltrušaitis, T., Robinson, P., & Morency, L. (2016) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2016.7477553 · Cited by 1,794 · cited 1×
“For UR_FUNNY, OpenFace [4], a facial behavioral analysis tool, is used to extract features related to the facial expressions of the speaker.”
Method
Research works printed but not linked: 15
- Andrew Galen, 2013, Proceedings of the 30th International Conference on Machine Learning, ICML 2013 (JMLR Workshop and Conference Proceedings, 1255
- Bousmalis Konstantinos, Advances in Neural Information Processing Systems 29. Curran Associates
- 10.5555/2354409.2355082 · DOI
- Devlin Jacob, 2019, Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1, 19
- Drugman Thomas, 2011, Joint Robust Voicing Detection and Pitch Estimation Based on Residual Harmonics. In INTERSPEECH 2011, 12th Annual Conference of the International Speech Communication Association. ISCA
- Kiela Douwe, 2019, NeurIPS 2019 Workshop. Curran Associates, Inc.
- Lu Jiasen, Advances in Neural Information Processing Systems 32. Curran Associates
- 10.5555/2392963.2392965 · DOI
- Peng Minlong, 2018, Long Papers
- Rajagopalan Shyam Sundar, 2016, Proceedings, Part VII (Lecture Notes in Computer Science, 353
- Salzmann Mathieu, 2010, Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2010 (JMLR Proceedings, 708
- Hubert Tsai Yao-Hung, 2019, Learning Factorized Multimodal Representations. In 7th International Conference on Learning Representations, ICLR
- Vaswani Ashish, Advances in Neural Information Processing Systems 30. Curran Associates
- Zadeh Amir, 2018, Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, 2246
- Zellinger Werner, 2017, Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning. In 5th International Conference on Learning Representations, ICLR
Cited by
Sorted by citations
Items 901–1,000 of 1,001
Hierarchical Structure Consistency Learning for Multimodal Emotion Recognition
Yang, B., Shi, Q., Zong, X., & Ye, M. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3694526
“Recent studies have also demonstrated that additional regularization terms—such as contrastive and consistency losses—can further refine feature representations to capture subtle relationships that are not fully exploited by general pre-training [12], [16], [51].”
InfluentialLiu, L. (2026) · Progress in Artificial Intelligence · doi:10.1007/s13748-026-00462-8
徐, 晋. (2026) · Modeling and Simulation · doi:10.12677/mos.2026.155071
Bidirectional mamba enhanced multimodal fusion framework for emotion recognition in conversations
Zhang, B., Ding, Z., Ye, L., Jiang, X., & Qi, H. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.115133
Decoupled Cross-Modal Fusion and Knowledge Injection for Multimodal Sentiment Analysis
Zhu, A., Hu, M. Q., Wang, X. J., Yan, X., & Ren, F. (2026) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2026.3696459 · Cited by 1
“In addition to contrastive methods, decoupled learning has been explored [32], [33], [34] via autoregressive decomposition or subspace projection, which project each modality into invariant and specific subspaces.”
Evidential Mixture-of-Experts for Sentiment Regression
Yu, Y., Xie, Z., Li, X. L., Yang, Y., Peng, B., & He, Y. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774905.3795845
Fang, T., & Yang, G. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774905.3795843
Zhang, R., & Chang, Q. (2026) · Frontiers in Medicine · doi:10.3389/fmed.2026.1758840
Dai, Y., Li, Y., Li, J., & Lu, G. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3699029
“In alignment with the existing literature [12], [62], we utilize WAF as our performance metric.”
Tri-MCA fusion: cross-modal attention and dynamic gating for multimodal sentiment analysis
Alrayzah, A. (2026) · Scientific Reports · doi:10.1038/s41598-026-56224-y · Cited by 2
Multimodal Emotion Recognition Algorithm Based on Multimodal Bottleneck Fusion Network
Chen, Y., 扈进宗, Zhang, Q. P., Liu, C., Sun, Y., Long, Y., Chen, J., Qian, C., & Tian, X. (2026) · IEEE International Conference on Computing and Machine Intelligence (ICMI) · doi:10.1109/icmi68585.2026.11539906
Label-oriented Hierarchical Framework for Multimodal Multi-label Emotion RecognitionPreprint
Hu, G., Li, M., Zhao, Y., & Lu, G. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6876307
Luo, F., Kuang, H., Fang, J., Ma, X., & Liu, X. (2026) · International Conference on Intelligent Computing and Signal Processing (ICSP) · doi:10.1109/icsp69961.2026.11540610
“Contrastive learning methods like JOYFUL [8] and MISA [9] improve alignment but use simplistic graphs, limiting handling of noisy dynamics [11], [12].”
Ren, Y., Bao, J., He, K., Li, T., & Zhao, J. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6880625
Wang, W., Huang, J., & Xu, B. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3700026
“With the rapid growth of human-centered multimodal data (e.g., social media audio–visual content, dialogue recordings), text–audio-based Multimodal Sentiment Analysis (MSA) has emerged as a central focus in affective computing [2], [3].”
InfluentialHuang, S., & Thiengburanathum, P. (2026) · International Conference on Intelligent Computing and Signal Processing (ICSP) · doi:10.1109/icsp69961.2026.11540775
Duan, R., Li, Z., Yi, P., Liu, R., & Dong, J. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02398-4
Esmaeili, M., & Sabeti, V. (2025) · International Conference on Signal Processing and Intelligent Systems (ICSPIS) · doi:10.1109/icspis68676.2025.11551654
“For instance, recent multimodal representation learning approaches [6] aim to learn features that are less sensitive to demographic factors and achieve only modest reductions in gender gaps when evaluated on IEMOCAP [12], with reported improvements of ( ∆ F1 ≈ 2 – 5%).”
InfluentialShen, Q., Xin, J., Dai, B. T., Yao, Z., Liu, X., Huang, Z., & Wang, Z. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3702501
“We compare our MGCCF with ten state-of-the-art (SOTA) MSA methods, including eight conventional methods (i.e., MulT [20] MISA [2], Self-MM [14], UniMF [53], EMT-DLFR [7], DMD [22], MetaNa [54],and DLF [3])andtwoASR error-aware methods (i.e., SWRM [8], and ARF-MSA [10]).”
Hong, C., Li, X., Huang, D., Du, Y., Lee, Y.-L., Liu, J., Chen, X., & Fan, Y. (2026) · The Journal of Supercomputing · doi:10.1007/s11227-026-08609-z
“To this end, a representation similarity loss based on central moment discrepancy (CMD) is introduced to regularize the generated representation from a distributional perspective [41].”
Sun, S., An, W., Liu, Q., Nan, F., Nie, J., Zeng, Z., Hua, X., Wu, Y., & Tian, F. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116392
Mitigating Multimodal Inconsistency via Cognitive Dual-Pathway Reasoning for Intent Recognition
Wang, Y., Wang, P., Chi, Y., Gou, Z., & Gao, K. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810808
Structure Aware Distillation for Multimodal Intent Understanding Under Missing Modalities
Lu, L., Yang, Q., Liu, Y., Pei, X., & Song, J. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810618
Xu, F., Li, P., Li, D., Li, B., HUANG, L., Zhu, W., Xu, Z., & Xie, Y. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810760
CERA: Conflict-Explicit Reflective Agent for Multimodal Emotion Reasoning
Liu, K., Liu, Y., Wang, K., (4859203), J. Z., Xu, L., Tang, C., Chen, Z., & Zhan, Y. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810634
GS³-ICL: Graph-Structured Sparse Selection with Invariance-Consistent Learning for multimodal SER
Zhou, Y., Ma, Z., Wang, L. S., Ren, H., Zhong, Z., Wang, S., Zhang, C., & Yang, H. (2026) · International Conference on Multimedia Retrieval · doi:10.1145/3805622.3810766
Bhatia, T., Arora, C., Salvi, R., & Meena, S. (2025) · International Conference on Computational Engineering, Sensing Technology and Management (ICCETM) · doi:10.1109/iccetm66557.2025.11558026
FDC-MSA: Feature denoising and conflict decoupling for inconsistent multimodal sentiment analysis
Su, N., Liu, Y., Jiang, D., Li, Y., & Ji, S. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-00935-9
Gou, Z., Jia, M., Xue, X., & Wang, Y. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116472
Cheng, K., Lei, X., Wang, R., Li, H., & Chen, C. S. (2026) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg67764.2026.11556928
“…concatenation (ImageBind + WavLM) already reaches a salience accuracy of 0.1600, [11] 0.3256 0.1324 0.2290 RJCMA [22] 0.3336 0.1275 0.2306 MISA [10] 0.3505 0.1226 0.2365 LMF [20] 0.3499 0.1336 0.2418 MMIM [9] 0.3398 0.1438 0.2418 HRG-SSA [12] 0.3525 0.1323 0.2424 TFN [28] 0.3554 0.1410 0.2482…”
InfluentialLearning Dynamic Frame Semantics for Multimodal Sentiment Analysis
Zhuang, J., He, T., Hou, Y., Ong, Y.-S., & Zhang, Q. (2026) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2026.3706491
“To comprehensively evaluate the performance of the proposed model on the above datasets, we adopt a set of widely used evaluation metrics, and we follow the task settings established in previous studies [35], [36].”
U-TCHFM: A Unified Text-Centric Hierarchical Fusion Model*
Lu, F., Leng, Y., & Yang, H. (2026) · International Conference on Artificial Intelligence and Industrial Technology Applications (AIITA) · doi:10.1109/aiita69518.2026.11567092
Anjana Thampy, S., & Angelina, J. J. R. (2026) · Optical Memory and Neural Networks · doi:10.3103/s1060992x25602817
Dan, X., Gao, B., Chen, Y., Fu, Q., Li, Y., Liu, S., Liu, Z., & Liu, Z. (2026) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2026.3707891
“Compared with disentanglement-oriented methods such as MISA [23], DMD [44], and DLF [17], CADET further improves shared–specific representation learning through adversarial regularization and cycle-consistent semantic preservation.”
InfluentialMultimodal sentiment analysis based on noise injection and adaptive multi-view dynamic fusion
Zheng, W., Chen, Q., Zhang, Z., & Cui, C. (2026) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-026-03205-2
Co-Attention based Multi-Modal Sentimental Analysis
Dhilipkumar, V., Thejaswi, R., Anupama, & Babu, K. M. (2026) · International Conference on Inventive Computing and Informatics (ICICI) · doi:10.1109/icici68773.2026.11580563
PCAMP: Personality-Conditioned Anchored Multimodal Perception for Robust Sentiment AnalysisPreprint
Li, Y., Gao, B., Li, X., Li, L., Li, Y., Liu, S., & Liu, Z. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7037063
Improved evidence theory based on information fusion for multimodal emotion recognition
Xiao, K., Yang, W., Zhu, G., Qiu, J., Pang, S., & Zhao, C. (2026) · The Journal of Supercomputing · doi:10.1007/s11227-026-08685-1
“MISA [42]: Modality-invariant and specific representations that disentangles shared and unique information across modalities through adversarial training and reconstruction objectives.”
Disentangled Multimodal Tuning and Interaction for Human Perception Understanding
Sun, H., Yu, X. F., Niu, Z., Liu, J., Chen, Y., & Lin, L. (2026) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3828163
“To this end, we evaluate and compare the inference latency, computational resource usage (in TFlops), and parameter efficiency against representative baseline methods, including MISA [16], MulT [39], and CubeMLP [36].”
InfluentialWang, X., Yuan, Y., Yang, C., & Ke, Z. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11582575
DVSF: Disentanglement and Variational Semantic Fusion for Multimodal Sentiment Analysis
Zhang, D., Xiao, Q., He, X., & Zhang, J. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11582567
“[6] 33.20 82.50 82.40 0.917 0.695 MFM [23] 35.83 78.87 78.90 0.927 0.670 MuLT [10] 40.00 83.00 82.00 0.871 0.698 PMR [9] 40.60 83.60 83.60 - - MISA [17] 41.37 83.54 83.58 0.777 0.778 FDMER [18] 42.10 84.20 83.90 0.845 0.732 CLGSI [24] 47.26 83.40 83.20 0.810 0.728 DMD [15] 46.06 83.23 83.29 - 0.752…”
InfluentialAkhila, N., Ekbal, A., & Pomeroy, D. (2026) · medRxiv · doi:10.64898/2026.07.03.26357258
Ji, H., Zheng, X., Liu, C., Wei, N., 池景志, Lü, W., & Ren, X. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11582821
A Collaborative Framework for Small-Scale Public Opinion Analysis
Xu, Y. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11582495
Zhang, Y., Kuang, W., Wu, K., Huang, Z., Fan, C., & Zou, Y. (2026) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2026.3711501
“For example, Hazarika proposes MISA [38], which imposes orthogonality constraints andCentralMomentDiscrepancy(CMD)toguidetheseparation offactors.”
Zhang, K., Lu, H., Chen, J. Y., & Han, S. (2026) · IEEE Sensors Journal · doi:10.1109/jsen.2026.3709324
CAMS-AMSS: Causal-aware adaptive masked subnetworks for imbalanced multimodal sentiment analysis
Xiaosong, E., Li, K., Dong, X. Q., & Wang, R. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.114436
He, X., Liu, F., Li, Y., & Chen, X. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02469-6
Chen, R., Zhang, T., Zhang, X., Cheng, S., Li, Z., & Zhāng, W. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3447-9_30
Group Emotion Recognition via Multimodal Transformer with Scene-Derived Prior Bias
Ma, S., Long, J., & Li, C. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3432-5_44
Tang, Y., Zhang, H., Liu, H., & Hong, J. (2026) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2026.3712935
“…input data, that is, the shared features and private features can, respectively, learn the common information contained in multimodal sensors and the unique discriminative information contained in each individual sensor, we design a feature separation loss inspired by the works of [18] and [27].”
InfluentialACRM: Affective Calibration and Rewriting Model for Multimodal Emotional Dialogue
Wang, Z., Li, L., Lei, Q., & He, F. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3432-5_42
MGCF: A Multi-granular Complementary Fusion Framework for Multimodal Sentiment Analysis
Liu, S., Chen, X., Wang, L., Sun, F., & Yang, S. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3432-5_2
CARD: Conflict-Aware Routing and Disentanglement Network for Multimodal Sentiment Analysis
Liao, Y., Zhang, Q., Wang, J., Li, W., Feng, G., & Liang, X. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-3560-5_22
Incomplete Multimodal Probability Flow Recovery for Emotion Recognition
Wang, Y., Cui, Z., Liu, M., Zhang, X., & Li, Z. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3713383 · Cited by 1
“Consequently, numerous researchers have dedicated efforts to developing a variety of MER methods, which could be divided into two categories: fusion strategy-based paradigm [5], [9], [27], [28], [29] and cross-modal attention-based paradigm [3], [4], [30], [31], [32], [33], [34].”
Hariharan, R., & Gideon, D. A. M. (2025) · International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC) · doi:10.1109/i-smac65734.2025.11597858
AHSD: adaptive multimodal fusion with hierarchical semantic disentanglement for sentiment analysis
Shi, Z., & Huang, W. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02527-z
Complementarity and Adaptive Fusion for Robust Multimodal Sentiment Analysis
Zhu, A., Hu, M. Q., Xing, Y., Yang, J., & Ren, F. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3714315
“…RoCAF, we conduct fair comparisons with representative and recent state-of-the-art methods, including complete-modality models such as MISA [43], Self-MM [44], MMIM [45], CENET [46], TETFN [47], ALMT [25], Cube-MLP [48], and DMD [22], as well as modality-missing methods such as TFR-Net…”
Li, J. Y., Zhang, X., Wang, J., & Zhou, X. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3435-6_5
Robust Multimodal Sentiment Analysis for Real Missing Data Using Confidence-Aware Generation
Chen, H., Zeng, B., Zhang, Y., & Chen, C. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3417-2_17
T-FCA: Template-Guided Fusion Contrastive Alignment for Multimodal Sentiment Analysis
Tang, M., Zhang, S., geng, X., & Zhang, P. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3417-2_29
Lin, Y. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3394-6_6
Wang, Z., Zhao, S., & Liang, H. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3403-5_28
Sun, L., Li, J., Ma, J., Zhang, Y., Sun, H., & Wang, X. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3520-9_48
Dynamic Interaction-Aware and Causality-Disentangled Framework for Multimodal Sentiment Analysis
Tung, K., Hong, Z., Liu, S., Cl, W., Fang, Y., Zhang, T., Liu, J., Zhang, Y., Ding, H., Zhou, Z. D., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3403-5_29
TAUA: A Text-Anchored Uncertainty-Aware Fusion Framework for Multimodal Sentiment Analysis
Zhao, X. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3403-5_24
Multimodal Emotion Recognition Based on Sparse Attention and Emotion Shift Modeling
Si, W., Li, H., Meng, X., & Zhao, Q. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-3557-5_21
Uncertainty-Aware Multimodal Emotion Recognition Method for Underground Coal Mine Dispatch Scenarios
Zhu, P., Wang, S., Chen, W., Liu, J., Wang, Z., Xu, T., & Tian, Q. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-3450-9_46
SpEmoC: A Balanced Speaker-Segment Multimodal Emotion Benchmark
Bano, S., Ahmad, S., Vipparthi, S. K., Chanda, S., & Murala, S. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36846-1_2
Closed-Loop Self-Calibrated Reliability Control for Multimodal Sentiment AnalysisPreprint
Gao, J., & Zhao, W. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7169748
Label-knowledge guided heterogeneous-temporal graph network for multimodal intent understanding
Si, T., Zhang, S., Li, P., He, F., & Dong, Z. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116678
SEMSA: A Semantic-Enhanced Framework for Multimodal Sentiment Analysis
Huang, J., Chen, M., Zhu, X., Zhang, J., & Duan, Z. (2026) · International Conference on Computer and Communication Systems (ICCCS) · doi:10.1109/icccs69761.2026.11613348
CSG ‐Net: A Causal‐Aware Structural Graph Network for Multimodal Music Emotion Recognition
Chen, L., & Liu, Y. (2026) · Expert Systems · doi:10.1111/exsy.70384
“Despite their representational power, a critical vulnerability of these advanced fusion frameworks (e.g., MISA and Self-MM) is their fundamental reliance on statistical correlations.”
InfluentialAdaptive Multitask Learning With Uncertainty‐Aware Loss Balancing for Multimodal Emotion Recognition
Tong, L., Li, Y., Cai, H., & Yan, H. (2026) · Electronics Letters · doi:10.1049/ell2.70680
Emotion-Aware Intelligent Agents Using Multimodal Neuro-Symbolic Reasoning
Hussain, T., Azad, M. A., Nujumudeen, F., & Tabarej, M. S. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-28307-8_26
Chau, V. T., & Hong, V. N. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-30005-8_44
GraphMamba: A two-stage hybrid framework for multimodal sentiment analysis
Wang, R., Liu, X., Wu, X., Gao, J., Mao, R., & Cambria, E. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116765
Fatoki, I., OYEKUNLE, V. B., & Adediran, E. (2026) · Research Square · doi:10.21203/rs.3.rs-10514016/v1
From Signal to Insight: Explainable Low-Rank Fusion for Multimodal eWOM Sentiment AnalysisPreprint
Liu, D., 文科选, Zhang, Z., & Shen, R. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7238274
HAFT: Hierarchical Audio-Enhanced Fusion Transformer for Efficient Multimodal Sentiment Analysis
Dong, Q., Lu, T., Shi, X., Xiao, L., & Liu, G. (2026) · International Journal of Software Engineering and Knowledge Engineering · doi:10.1142/s0218194026500579
Zhang, Y., Li, Q., Li, X., Wang, Z., Li, Z., Luo, X., & Wu, X. (2026) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3770855.3818011
Gradient-Informed Mixture of Fusion Experts for Parameter-Efficient Multimodal Sentiment Analysis
Ren, Q., Bai, Y., Zhang, H., Ren, X., & Lei, B. (2026) · International Conference on Artificial Intelligence and Computing (AIC) · doi:10.1109/aic65131.2026.11633836
EEG-Guided Multimodal Sentiment Analysis under Uncertain Missing Modalities
Zhang, K., Liu, Z., Song, P., Yu, Z., Liu, M., & Du, Z. (2026) · IEEE International Conference on Computer Vision and Deep Learning (DLCV) · doi:10.1109/dlcv69906.2026.11635540
A multi-modal data fusion and meta-learning framework for few-shot scam call detection
Wu, C.-T., Chen, J., Chai, Z., Wang, X., & Li, M. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02573-7
Discriminative semantic learning for incomplete multimodal sentiment analysis
Shan, T., Chen, H., Liu, Y., Liu, Y., & Li, Z. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02555-9
EAMF: Evidence-Aware Multimodal Fusion for Conversational Emotion RecognitionPreprint
Ma, H., Wang, L., Yu, Y., & Tao, X. (2026) · Research Square · doi:10.21203/rs.3.rs-10287292/v1
TGHIN: text-guided hyper-modality interaction network for multimodal sentiment analysis
Wang, K., Ding, R., Ge, Y., & Shao, H. (2026) · Signal Image and Video Processing · doi:10.1007/s11760-026-05536-7
Zhang, Y., & Liu, T. (2026) · IEEE China Conference on System Simulation Technology and its Applications (CCSSTA) · doi:10.1109/ccssta69471.2026.11635288
Singh, P., Tamana, & Pawar, L. (2026) · International Conference on Sustainable Engineering and Technology Innovations (ICSETI) · doi:10.1109/icseti67678.2026.11636668
Context-Aware Multimodal Depression Detection via LLM-Derived PHQ-9 Personal Feature Injection
Yang, H., Kumar, P., & Li, X. (2026) · Proceedings - International Conference on Image Processing · doi:10.1109/icip61757.2026.11630190
Audio-Visual-Textual Fusion for Emotion Recognition: A Neurologically Inspired Attention Mechanism
Chandran, N. (2026) · Eduschool Journal of Artificial Intelligence Research (EJAIR) · doi:10.63090/ejair/3139.7336.0007
A Cross-Attention-Based Framework for Multimodal Sentiment Analysis
Shen, Y. (2026) · IEEE International Conference on Electronics, Energy Systems and Power Engineering (EESPE) · doi:10.1109/eespe68405.2026.11648747
“This allows models [14] not only to predict overall sentiment polarity but also to characterize the fine-grained composition of emotions, thereby supporting multi-task learning paradigms ranging from sentiment regression to multi-label emotion recognition [15].”
Wu, Y., Hu, R., Li, Y., Luo, Z., Pan, L., & Peng, L. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02592-4
Personal Behavior-Robust and Interpretable Multimodal Sentiment Analysis
Gong, W., Li, J., Shi, T., Li, C., Chen, C., & Álvarez, J. M. (2026) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3829080
Text-Guided Joint Interaction Network for Multimodal Sentiment Analysis
Zhou, J., & Xu, L. (2026) · IEEE International Conference on Electronics, Energy Systems and Power Engineering (EESPE) · doi:10.1109/eespe68405.2026.11648622
Duong, D.-D., Nguyen, C.-V. T., & Le, D.-T. (2026) · Neural Computing and Applications · doi:10.1007/s00521-026-12421-4
Exploring imbalanced multimodal learning from intra-modality information learning perspective
Yin, H., Xu, Y., Zhao, J., & Zhang, X. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116881
A Multi-Modal Framework for Fraud Detection and Weakly Supervised Risk Stratification
Peng, Y., & Noman, S. M. (2026) · Journal of Organizational and End User Computing · doi:10.4018/joeuc.419519
Multi-Channel Batch-Wise Dynamic Hypergraph Network for Multimodal Sentiment Analysis
Wang, Y., Xie, W., Yang, Z., Chen, Y., & Gao, Y. (2026) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit62367.2026.11654141
Frequency-aware multimodal fusion framework for emotion recognition in conversation
Wu, C., Tang, J., Zhao, C., Zuo, S., & Kong, W. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02630-1
Metadata
- Authors
- Hazarika, D., Zimmermann, R., & Poria, S.
- Affiliations
- National University of Singapore (SG)
- Singapore University of Technology and Design (SG)
- Venue
- Proceedings of the 28th ACM International Conference on Multimedia
- Publisher
- ACM
- Published
- 2020-10-12
- Pages
- 1122-1131
- OpenAlex
- W3093051361
- Cited by
- 1,073
- References
- 45
- Citations by year
- 2026: 280 · 2025: 315 · 2024: 238 · 2023: 136 · 2022: 78 · 2021: 23 · 2020: 3
- FWCI
- 22.92
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM International Conference on Multimedia (ACM MM) (publishedVersion, cc-by)
- Funders
- Ministry of Education - Singapore (MOE2018-T2-1-103)
- DSO National Laboratories - Singapore (RTDST190702)
- RIE 2020 Advanced Manufacturing and Engineering (AME) programmatic grant (A19E2b0098)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- multimodal sentiment analysis, modality gap, modality-invariant representations, modality-specific representations, subspace projection, multimodal fusion, user-generated video, humor detection, MOSI, MOSEI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
