Multimodal Emotion Recognition With Temporal and Semantic Consistency
The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
Abstract
Automated multimodal emotion recognition has become an emerging but challenging research topic in the fields of affective learning and sentiment analysis. The existing works mainly focus on developing multimodal fusion strategies to incorporate different emotion-related features. However, they fail to explore the inherent contextual consistency to reconcile the emotional information across modalities. In this paper, we propose a novel Time and Semantic Interaction Network (TSIN), which concurrently incorporates the advantages of temporal and semantic consistency into the multimodal emotion recognition task. Specifically, a well-designed Speech and Text Embedding (STE) module is devoted to formulating the initial embedding spaces by respectively building the modality-specific representations of speech and text. Instead of separately learning or directly fusing the acoustic and textual features, we propose a well-defined Time and Semantic Interaction (TSI) module to conduct the emotional parsing and sentiment refining by performing the fine-grained temporal alignment and cross-modal semantic interaction. Benefitting from temporal and semantic consistency constraints, both speech-text embeddings can be interactively optimized and fine-tuned in the learning process. In this way, the learnt acoustics and textual features can jointly and efficiently predict the final emotional state. Extensive experiments on the IEMOCAP dataset demonstrate the superiorities of our TSIN framework in comparison with state-of-the-art baselines.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
(2008) · Journal of Machine Learning Research · Cited by 33,051
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869
Google news personalization: scalable online collaborative filtering
Das, A. S., Datar, M., Garg, A., & Rajaram, S. (2007) · Proceedings of the 16th international conference on World Wide Web · doi:10.1145/1242572.1242610 · Cited by 1,511
Affective Computing and Sentiment Analysis
Cambria, E. (2016) · IEEE Intelligent Systems · doi:10.1109/mis.2016.31 · Cited by 1,204
Context-Dependent Sentiment Analysis in User-Generated Videos
Poria, S., Cambria, E., Hazarika, D., Majumder, N., Zadeh, A., & Morency, L.-P. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1081 · Cited by 939
FastText.zip: Compressing text classification models
Joulin, A., Grave, E., Bojanowski, P., Douze, M., Jégou, H., & Mikolov, T. (2016) · arXiv · doi:10.48550/arxiv.1612.03651 · Cited by 887
Automatic speech emotion recognition using recurrent neural networks with local attention
Mirsamadi, S., Barsoum, E., & Zhang, C. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952552 · Cited by 770
Cited by
Zhang, S., Yang, Y., Chen, C., Zhang, X., Leng, Q., & Zhao, X. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121692 · Cited by 251
TMBL: Transformer-based multimodal binding learning model for multimodal sentiment analysis
Huang, J., Zhou, J. W., Tang, Z., Lin, J., & Chen, C. Y. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.111346 · Cited by 198
A survey on semantic communications: Technologies, solutions, applications and challenges
Liu, Y., Wang, X., Ning, Z., Zhou, M., Guo, L., & Jedari, B. (2023) · Digital Communications and Networks · doi:10.1016/j.dcan.2023.05.010 · Cited by 122
Multimodal Emotion Recognition Based on Facial Expressions, Speech, and EEG
Pan, J., Fang, W., Zhang, Z., Chen, B., Zhang, Z., & Wang, S. (2024) · IEEE Open Journal of Engineering in Medicine and Biology · doi:10.1109/ojemb.2023.3240280 · Cited by 121
Self-citationMasked Graph Learning With Recurrent Alignment for Multimodal Emotion Recognition in Conversation
Meng, T., Zhang, F., Shou, Y., Shao, H., Ai, W., & Li, K. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3434495 · Cited by 100
Cheng, C., Yu, Z., Zhang, Y., & Feng, L. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3319315 · Cited by 72
Semantic Alignment Network for Multi-Modal Emotion Recognition
Hou, M.-X., Zhang, Z., Liu, C., & Lu, G. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3247822 · Cited by 71
Self-citationTDFNet: Transformer-Based Deep-Scale Fusion Network for Multimodal Emotion Recognition
Zhao, Z., Wang, Y., shen, G. Z., Xu, Y., & Zhang, J. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3316458 · Cited by 63
Liu, F., Yang, P., Shu, Y., Liu, N., Sheng, J., Luo, J., Wang, X., & Liu, Y. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3336531 · Cited by 59
Chen, C., Song, M., Song, W., Guo, L., & Jian, M. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3203421 · Cited by 56
RBA-GCN: Relational Bilevel Aggregation Graph Convolutional Network for Emotion Recognition
Yuan, L., Huang, G., Li, F., Yuan, X., Pun, C., & Zhong, G. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3284509 · Cited by 40
Unified Cross-Modal Attention: Robust Audio-Visual Speech Recognition and Beyond
Li, J., Li, C., Wu, Y., & Qian, Y. (2024) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3375641 · Cited by 38
Tian, F., Zhang, L., Zhu, L., Zhao, M., Liu, J., Dong, Q., & Zhao, Q. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3420445 · Cited by 33
Wang, Y., Lin, Z., Teng, Y., Cheng, Y., Jiang, H., & Yang, Y. (2025) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2025.3542986 · Cited by 29
Multi-Modal Emotion Recognition with Self-Guided Modality Calibration
Hou, M.-X., Zhang, Z., & Lu, G. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747859 · Cited by 28
Self-citationMultitask Transformer for Cross-Corpus Speech Emotion Recognition
Ahn, C.-S., Rana, R. K., Busso, C., & Rajapakse, J. C. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3526592 · Cited by 26
The effect of emojis in travel experience sharing
Yan, H., Liao, Q., & Xiong, H. (2024) · Annals of Tourism Research · doi:10.1016/j.annals.2024.103798 · Cited by 26
Exploring Semantic Relations for Social Media Sentiment Analysis
Zeng, J., Zhou, J., & Huang, C. (2023) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2023.3285238 · Cited by 25
WavFusion: Towards Wav2vec 2.0 Multimodal Speech Emotion Recognition
Li, F., Luo, J., & Xia, W. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-2071-5_24 · Cited by 24
ReDDIT: Regret detection and domain identification from text
Balouchzahi, F., Butt, S., Sidorov, G., & Gelbukh, A. F. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120099 · Cited by 19
A Survey on Multimodal Emotion Recognition: Methods, Datasets, and Future Directions
Moon, A.-S., Kim, H., Park, Y.-C., & Lee, J. (2026) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2026.076411 · Cited by 17
An SQL query generator for cross-domain human language based questions based on NLP model
Naik, B. B., Reddy, T. J. V. R., karthik, K. R. V., & Kuila, P. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15731-0 · Cited by 17
Blind Multimodal Quality Assessment of Low-Light Images
Wang, M., Xu, Z., Xu, M., & Lin, W. (2024) · International Journal of Computer Vision · doi:10.1007/s11263-024-02239-9 · Cited by 16
Ouyang, X., Wu, J., Kimura, T., Lin, Y., Verma, G., Abdelzaher, T., & Srivastava, M. B. (2025) · ACM Conference on Embedded Networked Sensor Systems (SenSys) · doi:10.1145/3715014.3722053 · Cited by 16
Multi-Modal Emotion Recognition Using Multiple Acoustic Features and Dual Cross-Modal Transformer
Wu, Y., Yue, P., Qu, L., Li, T., & Ruan, Y.-P. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447830 · Cited by 15
MMATERIC: Multi-Task Learning and Multi-Fusion for AudioText Emotion Recognition in Conversation
Liang, X., Zou, Y., Zhuang, X., Yang, J., Niu, T., & Xu, R. (2023) · Electronics · doi:10.3390/electronics12071534 · Cited by 14
Multi-Modal Hierarchical Empathetic Framework for Social Robots With Affective Body Control
Gao, Y., Fu, Y., Sun, M., & Gao, F. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3356511 · Cited by 12
Hu, F., He, K., Qian, M., Liu, X., Qiao, Z., Zhang, L., & Xiong, J. (2024) · Frontiers in Neuroscience · doi:10.3389/fnins.2024.1519970 · Cited by 10
Wang, J. (2024) · Frontiers in Neuroscience · doi:10.3389/fnins.2024.1449527 · Cited by 10
Novel Multimodal Biometric Feature Extraction for Precise Human Identification
Vasavi, J., & Abirami, M. S. (2023) · Intelligent Automation & Soft Computing · doi:10.32604/iasc.2023.032604 · Cited by 10
Missingness-resilient Video-enhanced Multimodal Disfluency Detection
Mohapatra, P., Likhite, S., Biswas, S., Islam, B., & Zhu, Q. (2024) · Interspeech (USB) · doi:10.21437/interspeech.2024-1458 · Cited by 10
Cross-corpus speech emotion recognition using subspace learning and domain adaption
Cao, X., Jia, M., Ru, J., & Pai, T. (2022) · EURASIP Journal on Audio Speech and Music Processing · doi:10.1186/s13636-022-00264-5 · Cited by 9
A cross-modal attention model with contextual enhancements for speech emotion recognition
Qi, R., Zhao, C., Guo, X., Li, Z., Li, S., Chen, H., & Sun, Y. (2025) · Speech Communication · doi:10.1016/j.specom.2025.103285 · Cited by 8
Chien, G., Liu, J., Wang, J., & Chen, L. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.115654 · Cited by 7
Leveraging Label Information for Multimodal Emotion Recognition
Wang, P., Zeng, S., Chen, J., Fan, L., Chen, M., Wu, Y., & He, X. (2023) · Interspeech (USB) · doi:10.21437/interspeech.2023-1732 · Cited by 7
Li, F., Luo, J., & Xia, W. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.113915 · Cited by 6
An efficient brain-heart coupling learning system for emotion recognition
Gao, D., Deng, L., Peng, Z., Wang, Q., Zhang, H., Liu, S., Wu, D., & Li, P. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.108856 · Cited by 6
Handwritten Signature Verification via Multimodal Consistency Learning
Shi, Z., Li, F., Hao, D., & Sun, Q. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3557674 · Cited by 5
Text and Sound-Based Feature Extraction and Speech Emotion Classification for Korean
Jo, J., Kim, S. K., & Yoon, Y. C. (2024) · International Journal on Advanced Science Engineering and Information Technology · doi:10.18517/ijaseit.14.3.18544 · Cited by 5
Prompt-Based Zero- and Few-Shot Node Classification: A Multimodal ApproachPreprint
Li, Y., & Hooi, B. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2307.11572 · Cited by 5
Feature Evaluation and Joint Interaction for Audio-Visual Emotion Recognition
Li, S., Lu, C., Zong, Y., Lian, H., & Zheng, W. (2025) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2025.3619205 · Cited by 4
SCMI-Net: Semantic constraints and modal interaction network for multimodal emotion recognition
Lei, J., Zhang, T. Q., Wang, Y., & Wang, Y. (2026) · Journal of Intelligent Information Systems · doi:10.1007/s10844-026-01021-2 · Cited by 4
Taware, S., & Thakare, A. (2023) · International Journal on Recent and Innovation Trends in Computing and Communication · doi:10.17762/ijritcc.v11i9s.7464 · Cited by 4
Xlnet Parallel Hybrid Network Sentiment Analysis Based on Sentiment Word Augmentation
Xu, Z., Liao, Y., & Zhan, S. (2022) · International Conference on Computer and Communication Systems (ICCCS) · doi:10.1109/icccs55155.2022.9845940 · Cited by 4
Research on sentiment analysis of online public opinion based on multimodal big language modeling
Du, H., Yu, Q., & Chen, J. (2025) · Journal of Computational Methods in Sciences and Engineering · doi:10.1177/14727978251361417 · Cited by 4
Mathumitha, R., & Maryposonia, A. (2022) · International Conference on Intelligent Computing and Control Systems (ICICCS) · doi:10.1109/iciccs53718.2022.9788367 · Cited by 4
Zhang, W. (2022) · Wireless Communications and Mobile Computing · doi:10.1155/2022/7691586 · Cited by 4
Spatio-Temporal Fuzzy-oriented Multi-Modal Meta-Learning for Fine-grained Emotion RecognitionPreprint
Wang, J., Qiang, W., Zheng, C., & Sun, F. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2412.13541 · Cited by 4
Emotion Recognition by Facial Expressions and Speech Using Deep Learning
Vardhan, J. V., Yelavarti, K. C., & Chand, A. J. (2024) · International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies (ICAECT) · doi:10.1109/icaect60202.2024.10469626 · Cited by 3
Multi-Level Interaction for Emotion Recognition From Unaligned Speech and Text
Han, L., Chen, X., Jia, M., & Bao, C. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3630662 · Cited by 3
Multimodal Emotion Recognition Using Modality-Wise Knowledge Distillation
Lee, S., Ahn, Y., & Shin, J. W. (2025) · Sensors · doi:10.3390/s25206341 · Cited by 3
A Framework for Multi-Modal Fusion Using Dempster-Shafer Theory in Computer Vision Applications
Qiyanhui, L. (2024) · IEEE International Conference on Power, Intelligent Computing and Systems (ICPICS) · doi:10.1109/icpics62053.2024.10797105 · Cited by 2
Multi-source knowledge enhancement for multimodal semantic representation
Lai, P.-Y., Dai, Q.-Y., Liao, D.-Z., Cai, H.-T., Chen, M.-S., & Wang, C. (2026) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-025-02888-3 · Cited by 2
Wu, B., & Li, Y. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i3.37199 · Cited by 2
Multimodal emotion recognition based on multilevel acoustic and textual information
Zhou, Y., Xing, Y., Huang, G., Guo, Q., & Deng, N. (2023) · International Conference on Artificial Intelligence and Computer Science (AICS) · doi:10.1117/12.3009468 · Cited by 2
Semantic Enhancement Network Integrating Label Knowledge for Multi-modal Emotion Recognition
Zheng, H., Miao, S., Qian, Y., Mu, Y., Jin, X., & Yan, K. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-5675-9_40 · Cited by 1
Ma, C., Zhao, S., Pei, Y., Xie, L., & Yin, E. (2025) · International Conference on Multimedia Retrieval · doi:10.1145/3731715.3733264 · Cited by 1
Multi-Modal Semantic Communication Systems for Sustainability: A Comprehensive Survey
Baghel, S., Das, P., Bhowal, A., & Kshetrimayum, R. S. (2025) · IEEE India Council International Subsections Conference (INDISCON) · doi:10.1109/indiscon66021.2025.11251781 · Cited by 1
G-TBML: GAN-enhanced Transformer-based Multimodal Binding Learning Model
Qu, W., & Wei, B. (2025) · International Conference on Machine Learning and Computer Application (ICMLCA) · doi:10.1109/icmlca66850.2025.11336479 · Cited by 1
Multimodal emotion recognition with high-level feature fusion of audio and text via cross-attention
Lee, S., & Choi, Y.-S. (2026) · Multimedia Tools and Applications · doi:10.1007/s11042-026-21298-3 · Cited by 1
CMA-DF: A Physiological Signal Fusion Method Based on Cross-Modal Alignment and Dynamic Fusion
Li, X., Li, F., Wu, M., Chen, H., Li, F., & Lv, Z. (2026) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2026.3682466 · Cited by 1
When story meets landscape: Enhancing the communicative power of travel experience sharing
Liu, Y., Wang, M., & Li, M. (2026) · Tourism Management Perspectives · doi:10.1016/j.tmp.2026.101477 · Cited by 1
Temporal multimodal knowledge distillation for modality-missing RGBT tracking
Ruan, R., Kang, Y., Liu, L., Sun, J., Li, C., & Tang, J. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.113991 · Cited by 1
Multi-Branch Cross-Modal Cross-Attention for Image–Text Multimodal Sentiment Classification
Huang, X., Pei, Z., Tan, C., & Meng, Z. (2026) · Computers, materials & continua/Computers, materials & continua (Print) · doi:10.32604/cmc.2026.081626 · Cited by 1
MS²FL: Modality-shared and Modality-specific Feature Learning for Multimodal Emotion Recognition
Li, X., Chen, H., Wu, M., Song, Q., Li, F., Huang, J., Cui, F.-Q., & Lv, Z. (2026) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3836771 · Cited by 1
A hierarchical multistage semantic fusion framework for multimodal sentiment analysis
Mu, G., Dai, J., Sun, H.-B., & Li, J. (2026) · Aslib Journal of Information Management · doi:10.1108/ajim-01-2026-0042 · Cited by 1
Feature Fusion and Disfluency for Automated Anxiety Detection in Clinical Interviews
Dang, P. H. N., Quach, M., Zhao, W., Nguyen, M., & Madanian, S. (2026) · Proceedings - IEEE Symposium on Computer-Based Medical Systems · doi:10.1109/cbms69103.2026.00205 · Cited by 1
Multimodal Emotion Recognition Method Based on MAT-MER
Qu, H., Yang, X., Shangguan, F., & Yang, H. (2024) · International Conference on Intelligent Computing and Signal Processing (ICSP) · doi:10.1109/icsp62122.2024.10743748 · Cited by 1
Teleology-Driven Affective Computing: A Causal Framework for Sustained Well-BeingPreprint
Yin, B., Liu, C., Fu, L., & Zhang, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.17172 · Cited by 1
Emotion recognition using Deep Learning
Strizhkova, V. (2025) · theses.fr (ABES) · doi:10.70675/d18d9ed4z0046z4b52za5c1zaf409882e4ea · Cited by 1
Zhang, Y. (2024) · Journal of Electrical Systems · doi:10.52783/jes.3055
Setyawan, H., Wijaya, R., & Kosala, G. (2024) · JURNAL MEDIA INFORMATIKA BUDIDARMA · doi:10.30865/mib.v8i2.7634
AI- Powered Emotion Recognition: A Multimodal Approach to Analyzing Human Sentiment
Shalma, H., Mukherjee, S., Jainulabudeen, S. A. K., Reddy, B. S. T., Budhati, B., & Krishna, N. S. (2025) · International Conference on Visual Analytics and Data Visualization (ICVADV) · doi:10.1109/icvadv63329.2025.10961939
Gan, C., Zhou, D., Wang, K., Zhu, Q., Jain, D. K., & Štruc, V. (2025) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2025.104483
Weighted Late Fusion based Deep Attention Neural Network for Detecting Multi-Modal Emotion
S., S. P. V. V., Khamar, S. N., Borusu, N., Kota, M. G. R., Vuyyuru, H., & Sampath, P. (2026) · International Journal of Image Graphics and Signal Processing · doi:10.5815/ijigsp.2026.01.07
Ren, J., Chen, X., Cai, X., Guo, K., Chen, L., & Wei, J. (2026) · IEEE Signal Processing Letters · doi:10.1109/lsp.2026.3668129
Zhang, W., & Tian, C. (2026) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslpro.2026.3678799
A modality-aware contrastive learning framework for multimodal sentiment analysis
Zhu, L. (2026) · PeerJ Computer Science · doi:10.7717/peerj-cs.3887
Conversational AI in Smart Homes for Enhancing Personalization and Adaptability
Jain, R., Shah, K., & Upreti, K. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-3361-8.ch007
MorphoFusion: A Short-Utterance Speech Emotion Recognition Framework via Speech–Semantic Fusion
Li, C., & Xi, Y. (2026) · International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd68734.2026.11582210
Zhang, X., Li, Q., Gao, J., Liang, Y., & Shi, X. (2026) · Multimedia Systems · doi:10.1007/s00530-026-02509-1
Mamba-CrossMod: a multimodal affective analysis framework based on selective state space model
Tong, Y., Mu, J., Chang, W., & Zeng, H. (2026) · Pattern Analysis and Applications · doi:10.1007/s10044-026-01772-2
Deep learning Multimodal feature enhanced with cross-modal attention for emotion recognition
Taware, S., Thakare, A., & Kitukale, M. D. (2025) · Journal of Neonatal Surgery
Towards micro foundation models for robust multimodal IoT sensing
Kimura, T. (2024) · Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign)
Metadata
- Authors
- Chen, B., Cao, Q., Hou, M., Zhang, Z., Lu, G., & Zhang, D.
- Affiliations
- South China Normal University (CN)
- Harbin Institute of Technology (CN)
- Chinese University of Hong Kong, Shenzhen (CN)
- Venue
- IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Published
- 2021
- Volume
- 29
- Pages
- 3592-3603
- ISSN
- 2329-9290 (ISSN-L)
- 2329-9304
- OpenAlex
- W3214882941
- Cited by
- 71
- References
- 50
- Citations by year
- 2026: 18 · 2025: 16 · 2024: 16 · 2023: 15 · 2022: 6
- FWCI
- 8.93
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE/ACM Transactions on Audio Speech and Language Processing (publishedVersion)
- Funders
- National Key Research and Development Program of China (2018AAA0100100)
- National Natural Science Foundation of China (62176077, 62002085)
- Basic and Applied Basic Research Foundation of Guangdong Province (2019Bl515120055)
- Shenzhen Key Technical Project (2020N046)
- Shenzhen Fundamental Research (JCYJ20210324132210025, GXWD20201230155427003-20200824103320001)
- Open Project Fund (AC01202005018)
- Shenzhen Institute of Artificial Intelligence, and Robotics for Society
- Medical Biometrics Perception and Analysis Engineering Laboratory, Shenzhen, China
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- multimodal emotion recognition, temporal coherence, semantic consistency, cross-modal interaction, temporal alignment, IEMOCAP, affective computing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.