Deep Multimodal Emotion Recognition on Human Speech: A Review
Open accessThe DOI matches its Crossref record; 90 of the 90 references it cites were found in a registry or an index.
Abstract
This work reviews the state of the art in multimodal speech emotion recognition methodologies, focusing on audio, text and visual information. We provide a new, descriptive categorization of methods, based on the way they handle the inter-modality and intra-modality dynamics in the temporal dimension: (i) non-temporal architectures (NTA), which do not significantly model the temporal dimension in both unimodal and multimodal interaction; (ii) pseudo-temporal architectures (PTA), which also assume an oversimplification of the temporal dimension, although in one of the unimodal or multimodal interactions; and (iii) temporal architectures (TA), which try to capture both unimodal and cross-modal temporal dependencies. In addition, we review the basic feature representation methods for each modality, and we present aggregated evaluation results on the reported methodologies. Finally, we conclude this work with an in-depth analysis of the future challenges related to validation procedures, representation learning and method robustness.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures

Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“but failed to mention early fusion and late fusion techniques.”
From A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face (2023) · cites this paper as [22]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“With the advent of transformer networks [37], the architectures that produce word embeddings were further enriched with GPT (generative pre-trained) 1, 2 and 3 [38–40]; BERT (bidirectional encoder representations from transformers) [41]; ALBERT (a lite BERT) [42], etc.”
MethodBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · cited 2×
“GloVe (Global Vectors) [34] is one of the first models to produce robust word embeddings, followed by the FastText model [35], which also included sub-word information, and ELMo (Embeddings from Language Models) [36], where the notion of deep context-dependent word representations has been introduced.”
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“With the advent of transformer networks [37], the architectures that produce word embeddings were further enriched with GPT (generative pre-trained) 1, 2 and 3 [38–40]; BERT (bidirectional encoder representations from transformers) [41]; ALBERT (a lite BERT) [42], etc.”
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Enriching Word Vectors with Subword Information
Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00051 · Cited by 9,645 · cited 1×
“GloVe (Global Vectors) [34] is one of the first models to produce robust word embeddings, followed by the FastText model [35], which also included sub-word information, and ELMo (Embeddings from Language Models) [36], where the notion of deep context-dependent word representations has been…”
Ekman, P., & Friesen, W. V. (2019) · PsycTESTS Dataset · doi:10.1037/t27734-000 · Cited by 6,019
Graves, A., Fernández, S., Gomez, F., & Schmidhuber, J. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143891 · Cited by 5,370
ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
Lan, Z., Chen, M., Goodman, S., Gimpel, K., Sharma, P., & Soricut, R. (2019) · arXiv · doi:10.48550/arxiv.1909.11942 · Cited by 4,308 · cited 5×
“ALBERT [42] embeddings for the text modality.”
Influential
Research works printed but not linked: 2
- 39. Radford, A.; Wu, J.; Child, R.; Luan, D.; Amodei, D.; Sutskever, I. Language models are unsupervised multitask learners. OpenAI Blog 2019 , 1 , 9.
- 51. Poria, S.; Cambria, E.; Hazarika, D.; Majumder, N.; Zadeh, A.; Morency, L.P. Context-dependent sentiment analysis in usergenerated videos. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, BC, Canada, 30 July-4 August 2017; Volume 1, pp. 873-883.
Cited by
A Survey of Deep Learning-Based Multimodal Emotion Recognition: Speech, Text, and Face
Lian, H., Lu, C., Li, S., Zhao, Y., Tang, C., & Zong, Y. (2023) · Entropy · doi:10.3390/e25101440 · Cited by 224
“Koromilas et al. [22] exhaustively introduced an intermediate fusion method based on a deep multimodal learning architecture, but failed to mention early fusion and late fusion techniques.”
MethodKalateh, S., Estrada-Jimenez, L. A., Nikghadam-Hojjati, S., & Barata, J. (2024) · IEEE Access · doi:10.1109/access.2024.3430850 · Cited by 200
“Another review in MER (2021) [26] introduces a new categorization of methods based on temporal dynamics handling for speech, discusses feature representation meth-ods for each modality, and presents challenges related to validation procedures, representation learning, and method robustness.”
BackgroundAl-Saadawi, H. F. T., Daş, B., & Daş, R. (2024) · Expert Systems with Applications · doi:10.1016/j.eswa.2024.124852 · Cited by 103
Multimodal emotion recognition: A comprehensive review, trends, and challenges
Ramaswamy, M. P. A., & Palaniswamy, S. (2024) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1563 · Cited by 70
Data Augmentation for Audio-Visual Emotion Recognition with an Efficient Multimodal Conditional GAN
Ma, F., Li, Y., Ni, S., Huang, S., & Zhang, L. (2022) · Applied Sciences · doi:10.3390/app12010527 · Cited by 81
Deep Learning-Based Speech Emotion Recognition Using Multi-Level Fusion of Concurrent Features
Kakuba, S., Poulose, A., & Han, D. S. (2022) · IEEE Access · doi:10.1109/access.2022.3225684 · Cited by 66
“Moreover, early fusion and late fusion prevent models from learning intra and inter-modality interaction dynamics [33], [34] respectively.”
BackgroundMutawa, A. M., & Hassouneh, A. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2023.105942 · Cited by 61
Khan, U. A., Xu, Q., Liu, Y. M., Lagstedt, A., Alamäki, A. V., & Kauttonen, J. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01302-2 · Cited by 85
Kakuba, S., Poulose, A., & Han, D. S. (2023) · IEEE Access · doi:10.1109/access.2023.3325037 · Cited by 60
“It should be noted that early fusion and late fusion prevent models from learning intra and inter-modality interaction characteristics respectively [67].”
BackgroundHarris Hawks Sparse Auto-Encoder Networks for Automatic Speech Recognition System
Ali, M. H., Jaber, M. M., Khalil, S., Rehman, A. U., Awan, M. J., Vitkutė-Adžgauskienė, D., Damaševičius, R., & Bahaj, S. A. (2022) · Applied Sciences · doi:10.3390/app12031091 · Cited by 42
“In the context of human–computer or human–human interaction applications, the challenge of identifying emotions in human speech signals is critical and extremely difficult [30].”
BackgroundSurvey of deep emotion recognition in dynamic data using facial, speech and textual cues
Zhang, T., & Tan, Z. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-023-17944-9 · Cited by 46
“[28]and[29] focusonmultimodalitybutlackacomprehensivesurveyofunimodalemotionrecognition anddatasets.”
InfluentialPark, E. H., & Storey, V. C. (2023) · ACM Computing Surveys · doi:10.1145/3555719 · Cited by 43
“For instance, multimodal emotion recognition based on speech [49], visual gesture [50], and facial expression data [50, 51] has been studied to support sentiment analysis [52, 53].”
BackgroundPłaza, M., Trusz, S., Kęczkowska, J., Boksa, E., Sadowski, S., & Koruba, Z. (2022) · Sensors · doi:10.3390/s22145311 · Cited by 33
“…in many ways, including those based on: (cid:110) Voice parameters (timbre, raised voice, speaking rate, linguistic analysis and errors made) [7–10]; (cid:110) Characteristics of writing [11–15]; (cid:110) Changes in facial expressions in specific areas of the face [16–20]; (cid:110) Gestures…”
BackgroundA Framework to Evaluate Fusion Methods for Multimodal Emotion Recognition
Peña, D., Aguilera, A., Dongo, I., Heredia, J., & Cardinale, Y. (2023) · IEEE Access · doi:10.1109/access.2023.3240420 · Cited by 34
“more accurate reading of people’s emotions [5].”
BackgroundMethodInfluentialA Review of Multimodal Interaction in Remote Education: Technologies, Applications, and Challenges
Xie, Y., Yang, L., Zhang, M., Chen, S., & Li, J. (2025) · Applied Sciences · doi:10.3390/app15073937 · Cited by 38
Vision-audio multimodal object recognition using hybrid and tensor fusion techniques
Ahmed, M. R., Haque, R., Rahman, S., Reza, A. W., Siddique, N. H., & Wang, H. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103667 · Cited by 32
Multimodal Feature Fusion Method for Unbalanced Sample Data in Social Network Public Opinion
Zhao, J. M., Dong, W., Shi, L., Qiang, W., Kuang, Z., Xu, D., & An, T. (2022) · Sensors · doi:10.3390/s22155528 · Cited by 16
Multimodal Classification of Safety-Report Observations
Paraskevopoulos, G., Pistofidis, P., Banoutsos, G., Georgiou, E., & Katsouros, V. (2022) · Applied Sciences · doi:10.3390/app12125781 · Cited by 14
“Common examples of multimodal tasks involve image captioning [9], visual question answering [10,11] and sentiment analysis [12–14].”
BackgroundEmotion Recognition from Speech in a Subject-Independent Approach
Majkowski, A., & Kołodziej, M. (2025) · Applied Sciences · doi:10.3390/app15136958 · Cited by 13
State-of-the-art Multimodal Emotion Recognition: A comprehensive survey and taxonomy
Yazici, A., Kucukyilmaz, T., Dokeroglu, T., Sharipbay, A., Lee, M.-H., & Tyler, B. (2026) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2026.200642 · Cited by 11
Unsupervised Multimodal Language Representations using Convolutional AutoencodersPreprint
Koromilas, P., & Γιαννακόπουλος, Θ. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.03007 · Cited by 13
“The two core application-specific problems of this area are Multimodal Sentiment Analysis, where the goal is to determine whether a utterance contains positive or negative sentiment, and Multimodal Emotion Recognition in which the objective is to predict the underlying emotion of the utterance.”
Self-citationOverview of Multimodal Machine Learning
Al‐Zoghby, A. M., Al-Awadly, E. M. K., Ebada, A. I., & Awad, W. A. (2025) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3701031 · Cited by 18
“DCCA can be used for early fusion, where the representations are concatenated, or for late fusion, where the representations are combined after being learned by individual models [ 44 ].”
Urdu Speech Emotion Recognition: A Systematic Literature Review
Taj, S., Mujtaba, G., Daudpota, S. M., & Mughal, M. H. (2023) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3595377 · Cited by 11
“[43] 2021 State-of-the-art Review The objective of this study is to summarize current research on Deep Multi-modal Emotion Recognition on Human Speech.”
BackgroundTaware, S., & Thakare, A. (2025) · Circuits Systems and Signal Processing · doi:10.1007/s00034-025-03080-2 · Cited by 4
“The HCI systems need the interpretation and modeling of ER systems [3].”
InfluentialZwol, B. E. V., Langezaal, M. A., Arts, L. P. A., Gatt, A., & van den Broek, E. L. (2023) · Ambient intelligence and smart environments · doi:10.3233/aise230012 · Cited by 4
Sangeetha, S., Boddu, R. S. K., Bhaumik, A., Mathivanan, S. K., & Moorthy, U. (2025) · Scientific Reports · doi:10.1038/s41598-025-25813-8 · Cited by 5
“For the purpose of extracting informative audio features for child emotion classification, spectrogram-based models employing CNN have been utilized in many studies 21,22 .”
Muhammad, U., Ullah, K., Hussain, I., Almutairi, S. S., Syed, I., & Abohashrh, M. (2025) · PeerJ Computer Science · doi:10.7717/peerj-cs.2984 · Cited by 3
A multi-modal speech emotion recognition method based on graph neural networks
Li, Y., Wang, Y., Yang, X., Hoi, L. M., & Im, S. (2025) · Applied Intelligence · doi:10.1007/s10489-025-06953-w · Cited by 2
Chittepu, S., Martha, S., & Banik, D. (2026) · Cognitive Processing · doi:10.1007/s10339-026-01330-y · Cited by 4
Beyond a single perspective: A multi-agent debate framework for affective computing
Pan, Y., Shi, Y., Yu, C. L., Kong, X., Zhang, Y., & Xiao, N. (2026) · Pattern Recognition · doi:10.1016/j.patcog.2026.113445 · Cited by 2
VL-Few: Vision Language Alignment for Multimodal Few-Shot Meta Learning
Ma, H., Fan, B., Ng, B. K., & Lam, C. (2024) · Applied Sciences · doi:10.3390/app14031169 · Cited by 4
Toward Domain Adaptive Learning-Based Variation Autoencoder Emotional Analysis in English Teaching
Wang, X. (2024) · International Journal of Artificial Intelligence Tools · doi:10.1142/s0218213024400062 · Cited by 2
A Review of Emotion-Aware Voice Systems in Sustainable Human–Machine Interaction
Tarunima, V. S., Harish, R., Dhanaselvam, J., & Singh, A. B. (2025) · International Conference on Sustainable Computing and Data Communication Systems (ICSCDS) · doi:10.1109/icscds65426.2025.11167087 · Cited by 2
“…models such as Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Long Short-Term Memory [17], and Deep Belief Networks (DBN) became prominent due to their ability to automatically extract high-level features and capture temporal dependencies in speech signals [18], [19], [20].”
Enhancing Human-Computer Interaction with Attention-Based CNNs in Speech Emotion Recognition
Singh, R., Sagar, S., Kumar, A., & Sharma, V. (2024) · International Conference on Communication, Computing and Energy Efficient Technologies (I3CEET) · doi:10.1109/i3ceet61722.2024.10993660 · Cited by 5
Santhiya, P., Shanmugavadivel, K., Rajalakshmi, R., & Krishnamoorthy, N. (2025) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-025-00811-w · Cited by 2
Abnormal Emotion Recognition Based on Audio-Visual Modality Fusion
Jiang, Y., Hirota, K., Dai, Y., Ji, Y., & Shao, S. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-6483-3_15 · Cited by 1
Multi-Modal Based Fusion Emotion Identification Using Inception V3 Algorithm
P, S. V., Srivarshini., R., & Grace, R. K. (2023) · International Conference on Self Sustainable Artificial Intelligence Systems (ICSSAS) · doi:10.1109/icssas57918.2023.10331693 · Cited by 1
“The proposed system in [2] presents a deep multimodal approach for recognizing emotions in speech, combining speech, facial expressions, and physiological signals.”
BackgroundCG-MER: a card game-based multimodal dataset for emotion recognition
Farhat, N., Bohi, A., Letaifa, L. B., & Slama, R. (2024) · International Conference on Machine Vision · doi:10.1117/12.3023377 · Cited by 4
“For instance, Koromilas et al. [15] reviewed the state-of-the-art multimodal speech emotion recognition methodologies, with a particular emphasis on integrating audio, text, and visual information, another survey [16] has presented the basic research in the field and the recent advances into the…”
BackgroundCross-corpus bimodal speech emotion recognitionPreprint
Yunxiang, L., & Kexin, Z. (2022) · doi:10.22541/au.167043076.61971339/v1 · Cited by 1
Content recommendation based on recognised Emotion
Challa, S. S. R. K., Gowri, M. V., Roobini, M. S., Nandhini, P., & Clemency, C. A. D. D. (2023) · International Conference on Recent Advances in Electrical, Electronics, Ubiquitous Communication, and Computational Intelligence (RAEEUCCI) · doi:10.1109/raeeucci57140.2023.10134505
“Yet, the emotional qualities of media material have not received the attention they deserve, nor have they been fully used in realworld or commercial contexts, despite being an integral part of this data [8].”
BackgroundImproving information fusion in deep learning
Don, D. P. M. K. (2023) · Queensland University of Technology · doi:10.5204/thesis.eprints.241291
Recognition of Facial and Vocal Emotional Expressions by SOAR Model
Shahrestani, M. R., Motamed, S., & Yamaghani, M. (2023) · Journal of Information Systems and Telecommunication (JIST) · doi:10.61186/jist.39828.11.43.209
Deep Modelling Strategies for Human Confidence Classification using Audio-visual Data
Gudipalli, Y., Deshpande, G., Patel, S., & Schuller, B. W. (2023) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc40787.2023.10340488
“Among several behavioral parameters such as emotions [4], anxiety [5], and stress [6], human confidence has fewer studies using both audio and video as information sources.”
BackgroundA Hydra-Grid Fusion Method for Multimodal Emotion RecognitionPreprint
Gu, Y., Wang, Y., Wu, H., Li, C., Zhang, H., Lin, H. X., Han, Y., Wang, S., & Quan, D. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4673656
Advancing multimodal emotion analysis: integrating machine learning and deep learning approaches
Zhu, Y. (2024) · International Conference on Electronics, Electrical and Information Engineering (ICEEIE) · doi:10.1117/12.3052226
“Common decision level fusion methods include voting method, weighted sum method, cascaded classifier, etc 16 .”
A Logical Emotion Recognition System Based on Human Speech by Using Hybrid Deep Learning Procedures
Arulananth, T. S., Varalakshmi, K., V, Sugadev, M., Anguraj, K., Chadge, R., & Attar, H. H. (2024) · International Conference on Innovative Computing, Intelligent Communication and Smart Electrical Systems (ICSES) · doi:10.1109/icses63760.2024.10910299
“Since there could be both permanent and fleeting emotions, it's unclear which one the recognizer will pick up on [6][7][8].”
Influence of the surprisal power adjustment on spoken word duration in emotional speech in Serbian
Lazić, J., & Vujnović, S. (2025) · Computer Speech & Language · doi:10.1016/j.csl.2025.101803 · Cited by 4
Mahaseni, B., & Khan, N. (2025) · Empathic Computing · doi:10.70401/ec.2025.0010 · Cited by 4
Analyzis of Emotional Speech in Serbian from Surprisal Theory Perspective
Lazić, J., Vujnović, S., & Krstić, A. (2025) · International Conference on Electrical, Electronic and Computing Engineering (IcETRAN) · doi:10.1109/icetran66854.2025.11114098
Cui, J., & Meng, F. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228755
“…in multimodal learning have shown that combining multiple modalities can significantly improve performance in tasks like speech emotion recognition [18] and speaker identification [19], where different types of features (e.g., spectral, prosodic, and deep acoustic features) provide complementary…”
A multimodal reconstruction network via dynamic gradient modulation for emotion recognition
Li, A., Wang, Z., & Liu, Y. (2026) · Alexandria Engineering Journal · doi:10.1016/j.aej.2025.12.016
Multimodal Emotion Detection and Classification of Audio-Visual Using Deep Learning
Srivastava, M., Singh, P., & Sharma, M. K. (2025) · International Conference on Advanced Computing and Emerging Technologies (ACET) · doi:10.1109/acet67282.2025.11430197
“For example, Zhang et al. [17] studied a combine characteristic that combines auditory deep neural network model is trained using extensive picture data and tagged voice and facial data.”
Deep Learning-Based Multimodal System for Behavioral Analysis in Criminal Investigations
Baidya, S., Layek, A. S., Misra, B., Paul, S., Desai, L. R., & Dey, N. (2026) · International Conference on Emerging Trends and Innovations in ICT (ICEI) · doi:10.1109/icei65890.2026.11447701
“It may be possible to find optimal deep learning models [14-16] for each analysis modality; however, implementing the best normalization methods for audio data representation, performing speech marking so that it is time-aligned with segments of coherent sentences, and obtaining video data in…”
Wei, X. (2026) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001426570107
Chen, Y., Jiang, W., Zhang, J., Xiao, H., Zhao, Z., & Wu, T. (2026) · Signal Image and Video Processing · doi:10.1007/s11760-026-05443-x
“Early fusion and late fusion, to some extent, prevent the model from learning intra-modal and inter-modal interaction features, respectively [4], as early fusion collapses modality-specific structure prematurely while late fusion misses fine-grained cross-modal dependencies.”
Multiscale EEG feature fusion for recognizing 3D object shapes through active touch
Jiang, J., Ren, Z., Wang, J., Guo, X., Yi, G., Deng, B., Wang, J., Song, Z., & Gao, T. (2026) · Neural Networks · doi:10.1016/j.neunet.2026.109289
Schmitt, N. N., Schlicher, M., Triantafyllopoulos, A., Seizer, L., Gawrilow, C., Eickhoff, C., Schuller, B. W., & Löchner, J. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1863876
“Available datasets are small, predominantly in English, and lack continuous well-being labels or multimodal entries [35, 36].”
Speech–Text Fusion With ASR Reliability Metadata for Speech Emotion Recognition
Wan, K. (2026) · IEEE Access · doi:10.1109/access.2026.3737228
“Koromilas and Giannakopoulos [27] reviewed speech-based deep multi-modal emotion recognition methods, and Atmaja et al. [28] summarized bimodal SER studies that integrate acoustic and linguistic information.”
Metadata
- Authors
- Koromilas, P., & Giannakopoulos, T.
- Affiliations
- National Centre of Scientific Research "Demokritos" (GR)
- Institute of Informatics and Telecommunications, National Center for Scientific Research—Demokritos, 15310 Athens, Greece
- Corresponding author
- Koromilas, Panagiotis
- Venue
- Applied Sciences
- Publisher
- MDPI AG
- Published
- 2021-08-28
- Volume
- 11
- Issue
- 17
- Pages
- 7962
- ISSN
- 2076-3417 (ISSN-L)
- OpenAlex
- W3198872525
- Cited by
- 58
- References
- 81
- Citations by year
- 2026: 10 · 2025: 14 · 2024: 13 · 2023: 12 · 2022: 8 · 2021: 1
- FWCI
- 8.21
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Applied Sciences (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- speech emotion recognition, multimodal emotion recognition, temporal modeling, feature representation, representation learning, validation, robustness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.