The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Multimodal Sentiment Detection Based on Multi-channel Graph Neural Networks
Open accessAbstract
Xiaocui Yang, Shi Feng, Yifei Zhang, Daling Wang. Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“Finally, we obtain the object and scene memory banks with the pretrained ResNet (He et al., 2016).”
MethodVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 2×
“Motivated by the Transformer (Vaswani et al., 2017) prototype, we design a Multimodal Multi-head Attention Interaction (MMAI) module that can effectively learn the interaction between text modality and image modality by multiple channels, as shown in Fig.”
BackgroundMethodJakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 3×
“For text modality , we first encode words by GloVe (Pennington et al., 2014) to obtain the embedding vector and then obtain the text memory bank, M t , by BiGRU (Cho et al., 2014): (1) where T is a text sequence, L t is the maximum length of a padded text sequence, and d t is the dimension of hidden units in the BiGRU.”
MethodLearning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · cited 3×
“For text modality , we first encode words by GloVe (Pennington et al., 2014) to obtain the embedding vector and then obtain the text memory bank, M t , by BiGRU (Cho et al., 2014): (1) where T is a text sequence, L t is the maximum length of a padded text sequence, and d t is the dimension of hidden units in the BiGRU.”
MethodSemi-supervised classification with graph convolutional networks
Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853
YOLOv3: An Incremental Improvement
Redmon, J., & Farhadi, A. (2018) · arXiv · doi:10.48550/arxiv.1804.02767 · Cited by 17,141 · cited 1×
“To explicitly model this co-occurrence, we first extract objects O = { o 1 , ..., o l o } by YOLOv3 (Farhadi and Redmon, 2018), and extract scenes S = { s 1 , ..., s l s } by VGG-Place (Zhou et al., 2017).”
MethodConvolutional Neural Networks for Sentence Classification
Kim, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1181 · Cited by 14,831 · cited 2×
“Unimodal Baselines : For text modality, CNN (Kim, 2014) and Bi-LSTM (Zhou et al., 2016) are well-known models for text classification tasks, and BiACNN (Lai et al., 2015) incorporates the CNN and BiLSTM models with an attention mechanism for text sentiment analysis.”
MethodA Multi-Modal Distributed Real-Time IoT System for Urban Traffic Control (Invited Paper)
Khanam, Z., Achari, V. P. S., Boukhennoufa, I., et al. (2024) · Durham Research Online (Durham University) · doi:10.4230/oasics.ng-res.2024.2 · Cited by 14,048
Neural Message Passing for Quantum ChemistryPreprint
Gilmer, J., Schoenholz, S. S., Riley, P. F., Vinyals, O., & Dahl, G. E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.01212 · Cited by 5,678 · cited 2×
“Then, we update the node representation based on its original representations and neighboring nodes by the message passing mechanism (MPM) (Gilmer et al., 2017), which is defined as follows: where A tk ∈ R d is the aggregated information from neighboring nodes from node k − ws to k + ws , and max is the reduction funct…”
Method
Cited by
Sorted by citations
Multimodal learning with transformers: A survey
Xu, P., Zhu, X., & Clifton, D. A. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3275156 · Cited by 1,072
“text word learned embedding Vanilla Transformer [2] text word GNN embedding MGNNS [181]”
MethodMulti-Modal Sarcasm Detection via Cross-Modal Graph Convolutional Network
Liang, B., Lou, C., Li, X., Yang, M., Gui, L., He, Y., Pei, W., & Xu, R. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.124 · Cited by 178
“Many GCN-based approaches have demonstrated that the weights of the edges are crucial in graph information aggregation (Liang et al., 2021b; Yang et al., 2021; Lou et al., 2021).”
BackgroundVision-Language Pre-Training for Multimodal Aspect-Based Sentiment Analysis
Yan, L., Yu, J., & Xia, R. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.152 · Cited by 169
CLMLF:A Contrastive Learning and Multi-Layer Fusion Method for Multimodal Sentiment Detection
Li, Z., Xu, B. H., Zhu, C., & Zhao, T. (2022) · Findings of the Association for Computational Linguistics: NAACL · doi:10.18653/v1/2022.findings-naacl.175 · Cited by 157
Targeted Multimodal Sentiment Classification based on Coarse-to-Fine Grained Image-Target Matching
Yu, J., Wang, J., Xia, R., & Li, J. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/622 · Cited by 67
Wei, Y., Yuan, S., Yang, R., Shen, L., Li, Z., Wang, L., & Chen, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.287 · Cited by 62
“, 2018), employing a co-memory network to iteratively model the interactions between multiple modalities; MGNNS (Yang et al., 2021) utilizing a multi-channel graph neural networks with sentiment-awareness for image-text sentiment detection.”
BackgroundMethodA multimodal fusion network with attention mechanisms for visual–textual sentiment analysis
Gan, C., Fu, X., Feng, Q., Zhu, Q., Cao, Y., & Zhu, Y. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122731 · Cited by 63
Liang, B., Gui, L., He, Y., Cambria, E., & Xu, R. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3380375 · Cited by 66
“As previously discussed by [24], [36], [53], the weights of the edges are crucial in graph information aggregation.”
BackgroundJoint multimodal sentiment analysis based on information relevance
Chen, D., Su, W., Wu, P., & Hua, B. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103193 · Cited by 60
A Survey of Multimodal Fake News Detection: A Cross-Modal Interaction Perspective
Li, X., Qiao, J., Yin, S., Wu, L., Gao, C., Wang, Z., & Li, X. (2025) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2025.3543389 · Cited by 62
“Joint architectures have demonstrated superior performance in various multimodal classification and regression tasks, including video classification [45], sentiment analysis [46], visual question answering [47], and speech recognition [48].”
Multimodal sentiment analysis based on multiple attention
Wang, H., Ren, C., & Yu, Z. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109731 · Cited by 53
Unified Multi-modal Pre-training for Few-shot Sentiment Analysis with Prompt-based Learning
Yu, Y., Zhang, D., & Li, S. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548306 · Cited by 47
“The previous studies [10, 31, 35] normally explore the coarse-grained MSA, which aims to detect the whole sentiment of a text-image pair.”
BackgroundInfluentialFew-Shot Multi-Modal Sentiment Analysis with Prompt-Based Vision-Aware Language Modeling
Yu, Y., & Zhang, D. (2022) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme52920.2022.9859654 · Cited by 43
“In the literature, there are two common kinds of sub-tasks for MSA: aspect-level sentiment classification [7] based on the aspect terms, the corresponding sentence and the accompanying image, as well as sentence-level sentiment classification [8] based on a sentence and its accompanying image.”
BackgroundMethodAl-Tameemi, I. K. S., Feizi‐Derakhshi, M., Pashazadeh, S., & Asadpour, M. (2023) · IEEE Access · doi:10.1109/access.2023.3307716 · Cited by 47
“[19] created a model to identify sentiment in text and images using multi-channel graph neural networks.”
Multimodal fusion and vision–language models: A survey for robot vision
Han, X., Chen, S., Fu, Z., Feng, Z., Fan, L., An, D., Wang, C., Guo, L., Meng, W., Zhang, X., et al. (2026) · Information Fusion · doi:10.1016/j.inffus.2025.103652 · Cited by 105
“In addition, models such as MGA [74] and MGNNS [75] demonstrate that heterogeneous GNNs and multi-channel architectures can effectively bridge structural and semantic modality gaps by integrating multimodal graph representations.”
Few-shot Joint Multimodal Aspect-Sentiment Analysis Based on Generative Multimodal Prompt
Yang, X., Shi, F., Wang, D., Sun, Q., Wu, W., Zhang, Y., Hong, P., & Poria, S. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.735 · Cited by 44
Self-citationA comprehensive review of visual–textual sentiment analysis from social media networks
Al-Tameemi, I. K. S., Feizi‐Derakhshi, M., Pashazadeh, S., & Asadpour, M. (2024) · Journal of Computational Social Science · doi:10.1007/s42001-024-00326-y · Cited by 50
Wang, Q., Xu, H., Wen, Z., Liang, B., Yang, M., Qin, B., & Xu, R. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3333200 · Cited by 42
“Among them, most networks focus on building the interactions between different modalities based on fusion paradigms, such as early and late fusion [48], tensor fusion [49], attention mechanism [50], [51], and multimodal gated units [52], [53].”
BackgroundMultimodal sentiment analysis based on cross-instance graph neural networks
Wang, H., Ren, C., & Yu, Z. (2024) · Applied Intelligence · doi:10.1007/s10489-024-05309-0 · Cited by 38
“Thus, the reasons for two aforementioned situations are that (a) multi-channel graph neural networks [1] cannot handle the relationships between the same objects, thereby ignoring the true semantics of the images; and (b) simply stitching objects and scenes ignores the interaction of the two…”
InfluentialWang, H., Li, X., Ren, Z., Wang, M., & Ma, C. (2023) · Sensors · doi:10.3390/s23052679 · Cited by 37
Fact-sentiment incongruity combination network for multimodal sarcasm detection
Lu, Q., Long, Y., Sun, X., Feng, J. H., & Zhang, H. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102203 · Cited by 40
Li, Z., Zhang, L., Zhang, K., Zhang, Y., & Mao, Z. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3358411 · Cited by 40
“Finally, we can obtain the global textual embedding as : 2) Cross-Modal Graph: Many GCN-based approaches have demonstrated that the weights of edges are crucial in graph information aggregation [67], [68], [69], so the key challenge is how to accurately assign weights of cross-modal edges.”
MethodSelf-citationAffective Region Recognition and Fusion Network for Target-Level Multimodal Sentiment Classification
Jia, L., Ma, T., Rong, H., & Al-Nabhan, N. (2023) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2022.3231746 · Cited by 36
“In terms of multimodal data fusion, Yang et al. [13] use the graph neural network and the graph convolutional network modals separately to extract co-occurrence features of text and images for multimodal sentiment analysis.”
BackgroundMultimodal Emotion Classification With Multi-Level Semantic Reasoning Network
Zhu, T., Li, L., Yang, J., Zhao, S., & Xiao, X. (2022) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2022.3214989 · Cited by 35
“We compare our model with the following multimodal sentiment analysis models, including HSAN [56], MultiSen-tiNet [31], CoMN [32], MVAN [48] and MGNNS [57].”
BackgroundIntegrating color cues to improve multimodal sentiment analysis in social media
An, J., & Zainon, W. M. N. W. (2023) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2023.106874 · Cited by 28
Phukan, A., Pal, S., & Ekbal, A. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3388016 · Cited by 28
“A few noteworthy mentions include Yang et al.’s [18] multichannel graph neural network that encodes text and image modalities to extract hidden representations and then uses their framework to capture multi-modal representations.”
BackgroundCrossmodal Translation Based Meta Weight Adaption for Robust Image-Text Sentiment Analysis
Zhang, B., Yuan, Z., Xu, H., & Gao, K. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3405662 · Cited by 27
“MGNNS [23] utilizes multi-channel graph neural networks, which consist of the Text channel, the Image-Scene channel, and the Image-Object channel, to perform multimodal sentiment analysis based on the global characteristics of the dataset.”
BackgroundMethodA graph convolution-based heterogeneous fusion network for multimodal sentiment analysis
Zhao, T., Peng, J., Huang, Y., Wang, L., Zhang, H., & Cai, Z. (2023) · Applied Intelligence · doi:10.1007/s10489-023-05151-w · Cited by 23
“Yang et al. [22] noticed that multi-modality has specific global characteristics in the process of sentiment expression, so they designed multi-channel graph neural networks to capture the global feature representation under multi-channel and introduced multi-head attention mechanisms to achieve…”
Yang, X., Wu, W., Feng, S., Wang, M., Wang, D., Li, Y., Sun, Q., Zhang, Y., Fu, X., & Poria, S. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103204 · Cited by 29
Self-citationGPT-4V(ision) as A Social Media Analysis Engine
Lyu, H., Huang, J., Zhang, D., Yu, Y., Mou, X., Pan, J., Yang, Z., Wei, Z., & Luo, J. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3709005 · Cited by 58
Fine-Grained Multimodal DeepFake Classification via Heterogeneous Graphs
Yin, Q., Lu, W., Cao, X., Luo, X., Zhou, Y., & Huang, J. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02128-1 · Cited by 23
“Yang et al. (2021) used multi-channel graph neural networks to learn multimodal representations and then fused multimodal information to predict the sentiment of image-text pairs.”
Social Image–Text Sentiment Classification With Cross-Modal Consistency and Knowledge Distillation
Liu, H., Li, K., Jianping Fan, B., Yan, C., Qin, T., & Zheng, Q. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3220762 · Cited by 20
“Moreover, several studies such as OSDA [13] and DuIG [12] have been proposed to extract visual features from both object and scene views; specifically, DuIG employs an image graph convolutional neural network [12] with dual views.”
BackgroundMethodInfluentialDominant SIngle-Modal SUpplementary Fusion (SIMSUF) for Multimodal Sentiment Analysis
Huang, J., Ji, Y., Qin, Z., Yang, Y., & Shen, H. T. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3344358 · Cited by 21
“Different combinations of multimodal information were adopted as vertices to design graphs for the interaction and fusion of multimodal features [20], [30], [50], [54], [60].”
MethodImage–Text Sentiment Analysis Via Context Guided Adaptive Fine-Tuning Transformer
Xiao, X., Pu, Y., Zhao, Z., Nie, R., Xu, D., Qian, W., & Wu, H. (2022) · Neural Processing Letters · doi:10.1007/s11063-022-11124-w · Cited by 19
“[67] proposed amulti-channelGCN tomodel the interactions between image-object, image-scene, and words after extracting features via pre-trained CNNs and GloVe.”
Cross-modality Representation Interactive Learning for Multimodal Sentiment Analysis
Huang, J., Ji, Y., Yang, Y., & Shen, H. T. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612295 · Cited by 17
Wan, Y., Chen, Y., Lin, J., Zhong, J., & Dong, C. (2023) · Computer Speech & Language · doi:10.1016/j.csl.2023.101587 · Cited by 17
Multimodal sentiment analysis method based on image-text quantum transformer
Wu, H., Kong, D., Wang, L., Li, D., Zhang, J., & Han, Y. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130107 · Cited by 17
Guo, Q., Liao, Y., Li, Z., & Liang, S. (2023) · Entropy · doi:10.3390/e25101421 · Cited by 19
“MGNNS [50] integrates multi-channel graph neural networks with sentiment awareness for image-text sentiment detection.”
BackgroundLiu, Y., Ding, X., Chi, M., Wu, J., & Ma, L. (2023) · Information Technology & Tourism · doi:10.1007/s40558-023-00280-x · Cited by 16
“Many GCN-based approaches have demonstrated that the weights of the edges are crucial in information aggregation (Yang et al. 2021; Liang et al. 2022).”
Wang, H., Du, Q., & Xiang, Y. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.110262 · Cited by 15
Multi-Modal Sarcasm Detection via Knowledge-Aware Focused Graph Convolutional Networks
Zhuang, X., Zhou, F., & Li, Z. (2025) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3722115 · Cited by 16
“We then map each word 𝑤 𝑖 into a dimensional embedding using the pre-trained uncased RoBERTa model [19]: For knowledge-text generating, given a sequence of words as 𝑠 = { 𝒲 𝑖 } = 1 , the RoBERTa input corresponding to each token is expressed as e 𝑖 = RoBERTa-embedding ( 𝒲 𝑖 ) ∈ R 𝑤 ,…”
Modality-Dependent Sentiments Exploring for Multi-Modal Sentiment Classification
Li, J., Wang, C., Luo, Z., Wu, Y., & Jiang, X. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10445820 · Cited by 13
Multi-grained feature gating fusion network for multimodal sentiment analysis
Yu, B., Li, C., & Shi, Z. (2025) · Knowledge and Information Systems · doi:10.1007/s10115-025-02446-x · Cited by 13
Multi-layer cross-modality attention fusion network for multimodal sentiment analysis
Yin, Z., Du, Y., Liu, Y., & Wang, Y. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17685-9 · Cited by 13
“Yang et al. [13] construct a Multi-channel Graph Neural Networks with sentiment-awareness to overcome the inability of capturing the global co-occurrence features between text and image.”
Dong, S., Fan, X., & Ma, X. (2024) · Neural Processing Letters · doi:10.1007/s11063-024-11641-w · Cited by 12
Cross-Domain Sentiment Analysis via Disentangled Representation and Prototypical Learning
Wang, Q., Wen, Z., Ding, K., Liang, B., & Xu, R. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3431946 · Cited by 14
“Besides, a few works [32], [33] exploit attention mechanisms to build the interactions between different modalities for improving the accuracy of sentiment analysis.”
Survey of Multimodal Sentiment Analysis Based on Deep Learning
Zhang, J., Xue, S., Wang, X., & Liu, J. (2023) · IEEE International Conference on Cloud Computing and Intelligence Systems (CCIS) · doi:10.1109/ccis59572.2023.10263012 · Cited by 12
A Social Media Dataset and H-GNN-Based Contrastive Learning Scheme for Multimodal Sentiment Analysis
Peng, J., He, Y., Chang, Y., Lu, Y., Zhang, P., Ou, Z., & Yu, Q. (2025) · Applied Sciences · doi:10.3390/app15020636 · Cited by 11
“We can use this embedding vector for tasks like multimodal sentiment analysis [52,54].”
Two-stage Attention-based Fusion Neural Network for Image-Text Sentiment Classification
Hu, X., & Yamamura, M. (2022) · International Conference on Image, Video and Signal Processing (IVSP) · doi:10.1145/3531232.3531233 · Cited by 9
“• MGNNS (2021) [32]: employed pre-trained models to explore the scene and object features of image and built multichannel graph neural networks to learn multimodal representations for deep fusion and sentiment analysis.”
MethodHolistic Visual-Textual Sentiment Analysis with Prior Models
Chen, J., An, J., Lyu, H., Kanan, C., & Luo, J. (2024) · IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr62202.2024.00037 · Cited by 11
“Regarding the visual expert branch, our method has four different visual features while OCR and face features are not used by MGNNS [26].”
BackgroundMethodResultInfluentialMulti-granularity visual-textual jointly modeling for aspect-level multimodal sentiment analysis
Chen, Y., Shi, L., Lin, J., Chen, J., Zhong, J., & Dong, C. (2024) · The Journal of Supercomputing · doi:10.1007/s11227-024-06567-y · Cited by 9
Improving multimodal sentiment prediction through vision-language feature interaction
An, J., Ding, B., & Zainon, W. M. N. W. (2025) · Multimedia Systems · doi:10.1007/s00530-024-01659-4 · Cited by 11
“MGNNS [37]: Uses a multi-channel graph neural network to represent objects, scenes, and text by capturing global co-occurrence patterns.”
Multiple Contrastive Learning for Multimodal Sentiment Analysis
Yang, X., Feng, S., Wang, D., Hong, P., & Poria, S. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10096777 · Cited by 9
“Like [14], each layer has a multi-head attention layer ( MH ( · ) ) and a position-wise fully connected feed-forward network ( FFN ( · ) ), the specific compute process is similar to [6].”
BackgroundMethodInfluentialSelf-citationFeature-guided Multimodal Sentiment Analysis towards Industry 4.0
Yu, B., Wei, J., Yu, B., Cai, X., Wang, K., Sun, H., Bu, L., & Chen, X. (2022) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2022.107961 · Cited by 10
BIT: Improving Image-text Sentiment Analysis via Learning Bidirectional Image-text Interaction
Xiao, X., Pu, Y., Zhao, Z., Gu, J., & Xu, D. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191445 · Cited by 8
“Image-text matching with self-supervised learning was utilized by [24] to achieve image-text sentiment analysis.”
MethodSSMIM: Semi-supervised Multimodal Interaction Model for real-time conversational emotion recognition
Liu, H., & Zhong, K. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-07373-w · Cited by 9
Multi-Modal Emotion Recognition for Online Education Using Emoji Prompts
Qin, X., Zhou, Y., & Li, J. (2024) · Applied Sciences · doi:10.3390/app14125146 · Cited by 6
Transformer-based short-term memory attention for enhanced multimodal sentiment analysis
Shao, D., Tang, K., Li, J., Yi, S., & Ma, L. (2025) · The Visual Computer · doi:10.1007/s00371-025-03883-z · Cited by 6
“For a fair comparison, we adopted the same data and processing methods as Yang et al. [19] for the MVSA-* datasets.”
InfluentialSemantic Interaction Fusion Framework for Multimodal Sentiment Recognition
Yang, S., Cui, L., & Wang, T. (2023) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc53992.2023.10394433 · Cited by 6
“These methods focus on feature representation learning [5, 6, 7, 8] or multimodal interactive modeling [4, 9, 10, 11].”
MethodAdaptive multimodal prompt-tuning model for few-shot multimodal sentiment analysis
Xiang, Y., Zhang, A., Guo, J., & Huang, Y. (2025) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-025-02549-5 · Cited by 7
CLIP-driven attention network for multimodal sentiment analysis
Lv, J., Yang, Q., Tian, S., Liu, B., & Yu, L. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-07411-7 · Cited by 6
“…enhancing the cross-modal alignment of text and image features; Multi-channel Graph Neural Networks with Sentiment-awareness (MGNNS) (Yang et al. [39]): proposed a multi-channel graph neural network that captures local modality dependencies and models global feature interactions by constructing…”
Deep Learning Approaches on Multimodal Sentiment Analysis
Cai, Z., Gao, H., Li, J., & Wang, X. (2022) · IEEE International Conference on Electrical Engineering, Big Data and Algorithms (EEBDA) · doi:10.1109/eebda53927.2022.9745018 · Cited by 6
“To avoid this situation, researchers from Northeastern University in China [10] provide another method to learn the interaction between different modalities.”
Hou, S., Tuerhong, G., & Wushouer, M. (2023) · Sensors · doi:10.3390/s23104829 · Cited by 6
“In addition, Yang et al. [7] and Zhang et al. [8] propose multi-head attention models to define the correlation between image and text content.”
BackgroundA Sentimental Prompt Framework with Visual Text Encoder for Multimodal Sentiment Analysis
Huang, S., Xu, B., Li, C., Ye, J., & Lin, X. A. (2024) · International Conference on Multimedia Retrieval · doi:10.1145/3652583.3658115 · Cited by 7
Multimodal sarcasm detection based on sentiment-clue inconsistency global detection fusion network
Xi, Z., Yu, B., & Wang, H. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.127020 · Cited by 5
Yan, K., Lai, P., Zheng, X., Yang, Y., Ren, Y. C., Badarch, T., & Chen, Y. (2025) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2025.112923 · Cited by 8
Semantic enhancement and cross-modal interaction fusion for sentiment analysis in social media
Mu, G., Chen, Y., Li, X., Li, D., & Dai, J. (2025) · PLoS ONE · doi:10.1371/journal.pone.0321011 · Cited by 5
Momentum Distillation Improves Multimodal Sentiment Analysis
Li, S., Deng, W., & Hu, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-18907-4_33 · Cited by 5
Heterogeneous Graph Learning for Acoustic Event Classification
Shirian, A., Ahmadian, M., Somandepalli, K., & Guha, T. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095073 · Cited by 5
“These recent works have established that a multimodal graph approach promotes closer coupling between various events across the modalities resulting in a significant performance gain [13, 14].”
Video Games as a Corpus: Sentiment Analysis using Fallout New Vegas Dialog
Hämäläinen, M. K., Alnajjar, K., & Poibeau, T. (2022) · International Conference on the Foundations of Digital Games (FDG) · doi:10.1145/3555858.3555930 · Cited by 5
“Sentiment analysis is often conducted in a simplified positive-negative scale in the field of NLP with highly successful results [1, 2, 14, 15].”
BackgroundCross-Modality Fused Graph Convolutional Network for Image-Text Sentiment Analysis
Tan, Q., Shen, X., Bai, Z., & Sun, Y. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-46314-3_32 · Cited by 5
MOC: Multi-modal Sentiment Analysis via Optimal Transport and Contrastive Interactions
Li, Y., Zhu, Q., He, H., Gu, Z., & Zheng, C. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-8082-6_34 · Cited by 5
An, J., Ding, B., & Jiang, N. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.114058 · Cited by 5
Cross-Modal Hierarchical Knowledge Distillation for Image Aesthetics Assessment
Chen, H., Shao, F., Jing, W., Wang, H., & Jiang, Q. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2024.3521765 · Cited by 4
“Given that the text data contains numerous characters irrelevant to sentiment analysis, we preprocessed the text data according to [67].”
Visually-aware Acoustic Event Detection using Heterogeneous Graphs
Shirian, A., Somandepalli, K., Sánchez, V., & Guha, T. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10670 · Cited by 4
Multi-Task Supervised Alignment Pre-Training for Few-Shot Multimodal Sentiment Analysis
Yang, J., Cao, J., & Duan, C. (2025) · Applied Sciences · doi:10.3390/app15042095 · Cited by 4
Multimodal Alignment and Hierarchical Fusion Network for Multimodal Sentiment Analysis
Huang, J., Li, H., & Mo, X. (2025) · Electronics · doi:10.3390/electronics14193828 · Cited by 3
Chang, Y., Li, Z., Ruan, Y., & Yin, G. (2026) · Big Data and Cognitive Computing · doi:10.3390/bdcc10010023 · Cited by 4
“Recognizing dataset-level global features’ impact, Yang et al. [22] later introduced MGNNS, the first graph neural network application in multimodal sentiment detection, modeling global characteristics through GNNs while enabling deep multimodal interaction via specialized channels.”
MPNAS: Multimodal Sentiment Analysis Pruning via Neural Architecture Search
Zhang, B., Ren, A., Zhang, Z., Duan, M., Liu, D., Tan, Y., & Zhong, K. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887670 · Cited by 3
Gupta, A., Mittal, A., & Jain, R. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103266 · Cited by 3
Multi-Modal Graph-Based Sentiment Analysis via Hybrid Contrastive Learning
Zhang, J. (2024) · IEEE International Conference on Sensors, Electronics and Computer Engineering (ICSECE) · doi:10.1109/icsece61636.2024.10729439 · Cited by 3
“[26][27] split the multimodal representations to single modality, while [28][29] integrated the messages through the deep neural network and the multi-channel network.”
BackgroundQuantum-inspired multimodal fusion with Lindblad master equation for sentiment analysis
Yan, K., Lai, P., Yang, Y., Ren, Y., Badarch, T., Chen, Y., & Zheng, X. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130710 · Cited by 3
Multimodal Affect Perception With Large Language Model Enhancement Network
Yang, K., Luo, Y., Zhang, Z., Chen, C. L. P., & Zhang, T. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3607727 · Cited by 2
“Multimodal models such as EF-CaTrBERT [80] integrate visual content as auxiliary text for BERT, MVAN [78] leverages multiview attention to capture salient object-level and scene-level features, and MGNNS [6] uses multichannel graph neural networks for coarse-grained MSA.”
FADD-Net:Multimodal Sentiment Analysis Based on Feature Alignment and Differential Decoupling
Zhang, J., Zhang, S., Liu, J., Wen, H., Zhang, Y., & Chen, M. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228674 · Cited by 2
“(8):MGNNS [34]: The accuracy of sentiment detection is improved by effectively fusing image and text features through graph neural networks and multi-head attention mechanisms.”
Self-citationFew-Shot Multimodal Sentiment Analysis Based on Dynamic Adjustment and Contrastive Learning
Wang, H., Liu, S., & Xie, N. (2026) · Journal of Advanced Computational Intelligence and Intelligent Informatics · doi:10.20965/jaciii.2026.p0472 · Cited by 2
AutoAMS: Automated attention-based multi-modal graph learning architecture search
Al-Sabri, R., Gao, J., Chen, J., Olouladé, B. M., & Wu, Z. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106427 · Cited by 5
Kavitha., A., Aishwarya, B., & Durgagowri, G. (2025) · International Conference on Multi-Agent Systems for Collaborative Intelligence (ICMSCI) · doi:10.1109/icmsci62561.2025.10894282 · Cited by 2
“X. Yang and others, [5] introduced multimodal sentiment detection derived from multi-channel graph neural networks in 2021.”
A Multimodal Sentiment Analysis Model for Graphic Texts Based on Deep Feature Interaction Networks
Chang, W., & Zhang, D. (2024) · International Journal of Ambient Computing and Intelligence · doi:10.4018/ijaci.355192 · Cited by 1
A Novel MLLMs-Based Two-Stage Model for Zero-Shot Multimodal Sentiment Analysis
Lu, H., & Li, X.-F. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0119-6_41 · Cited by 1
An Image-Text Sentiment Analysis Method for Small Samples Based on Image Captioning
Chen, S. (2024) · Highlights in Science Engineering and Technology · doi:10.54097/xsrhp326 · Cited by 1
“Yang et al. later proposed the MGNNS model that by combining graph neural networks and multi-head attention mechanisms, so as to effectively capture and merge the features of image and text [4].”
MC-GNN: Multi-Channel Graph Neural Networks With Hilbert-Schmidt Independence Criterion
Cui, S., Li, D., & Zhang, J. (2024) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2024.3522817 · Cited by 2
“Someothermulti-channelarchitec-tures[26],[27],[28]aredesignedforcertainheterogeneous relationshipsornodetypesorattributes,suchasimages,texts, drugs,etc. Despitetheirsuccess,somelimitationsprohibitthem from being widely adopted.”
Two-Stage Multi-Modal Prompt Tuning for Few-Shot Sentiment Analysis
Gu, J., & Niu, H. (2024) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc54092.2024.10831782 · Cited by 1
“[7] utilize multi-channel graph neural networks to learn multi-modal representations based on the global characteristics of datasets.”
MSAF: Multimodal Sentiment Detection via Multiscale Adaptive Fusion
Guan, J., Shu, Y., Liu, W., Li, W., Zhou, S., & Zhang, Y. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3602409 · Cited by 1
“For example, Yang et al. [43] trained a multichannel graph neural network using a graph information propagation task to explore the associations between images and text, thereby learning better multimodal fusion representations.”
Self-citationCross-Hierarchical Multimodal Sentiment Analysis for Mental State Monitoring
Yuan, W., Liu, Y., Gao, D., & Shi, Y. (2025) · International Conference on Computer Information Science and Application Technology (CISAT) · doi:10.1109/cisat66811.2025.11181716 · Cited by 1
“In the visual domain, deep learning frameworks have been employed to decode emotional signals from images, where convolutional layers hierarchically extract low-level visual features and high-level semantic features that correlate with affective states [11], [12].”
Image-Text Sentiment Analysis Using Modality Adaption and Multi-Broad Learning
Deng, C., Yu, J., Peng, S., Huang, Q., & Cao, L. (2025) · Tsinghua Science & Technology · doi:10.26599/tst.2025.9010152 · Cited by 1
Sahoo, R. K., Kumar, V., & Dash, S. (2025) · Lecture notes in networks and systems · doi:10.1007/978-981-96-8639-1_5 · Cited by 1
A Review of Multimodal Sentiment Analysis in Online Public Opinion Monitoring
Liu, S., & Li, T. (2026) · Informatics · doi:10.3390/informatics13010010 · Cited by 1
Multimodal Cross-Attention Graph Network for Desire Detection
Gu, R., Wang, X., & Yang, Q. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44216-2_42 · Cited by 1
Multimodal Sentiment Analysis With Image-Text Correlation Modal
Li, Y., Jiang, S. L., & Chaomurilige, C. (2023) · IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData) · doi:10.1109/ithings-greencom-cpscom-smartdata-cybermatics60724.2023.00067 · Cited by 1
“Yang et al. [14] initially applied a Graph Neural Network to the image-text multimodal sentiment analysis, modeling the global characteristics of the dataset.”
MethodNews-Driven Price Movement Forecasting with Label-Prior Graph Attention
Liu, Y. F., Chen, C.-C., Huang, H., & Chen, H. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589335.3651539 · Cited by 1
Metadata
- Authors
- Yang, X., Feng, S., Zhang, Y., & Wang, D.
- Affiliations
- Northeastern University (CN)
- Venue
- Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)
- Published
- 2021
- Pages
- 328-339
- OpenAlex
- W3174517569
- Cited by
- 145
- References
- 39
- Citations by year
- 2026: 27 · 2025: 41 · 2024: 33 · 2023: 27 · 2022: 17
- FWCI
- 8.88
- Type
- conference paper
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Funders
- National Natural Science Foundation of China (U1811261, 61872074, 61772122)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Keywords
- sentiment analysis, multimodal fusion, graph neural networks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
