Speech Emotion Recognition Using Deep Neural Network Considering Verbal and Nonverbal Speech Sounds
The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
Speech emotion recognition is becoming increasingly important for many applications. In real-life communication, non-verbal sounds within an utterance also play an important role for people to recognize emotion. In current studies, only few emotion recognition systems considered nonverbal sounds, such as laughter, cries or other emotion interjection, which naturally exists in our daily conversation. In this work, both verbal and nonverbal sounds within an utterance were thus considered for emotion recognition of real-life conversations. Firstly, an SVM-based verbal/nonverbal sound detector was developed. A Prosodic Phrase (PPh) auto-tagger was further employed to extract the verbal/nonverbal segments. For each segment, the emotion and sound features were respectively extracted based on convolutional neural networks (CNNs) and then concatenated to form a CNN-based generic feature vector. Finally, a sequence of CNN-based feature vectors for an entire dialog turn was fed to an attentive long short-term memory (LSTM)-based sequence-to-sequence model to output an emotional sequence as recognition result. Experimental results on the recognition of seven emotional states in the NNIME (The NTHU-NTUA Chinese interactive multimodal emotion corpus) showed that the proposed method achieved a detection accuracy of 52.00% outperforming the traditional methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Opensmile: the munich versatile and fast open-source audio feature extractor
Eyben, F., Wöllmer, M., & Schuller, B. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1874246 · Cited by 2,642 · cited 1×
“OpenSMILE [20] was used to extract low-level descriptors (LLDs) with functionals proposed in [21] with a frame size”
MethodAdieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network
Trigeorgis, G., Ringeval, F., Brueckner, R., Marchi, E., Nicolaou, M. A., Schuller, B., & Zafeiriou, S. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472669 · Cited by 904
Error weighted semi-coupled hidden Markov model for audio-visual emotion recognition
Lin, J.-C., Wu, C.-H., & Wei, W.-L. (2012) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2011.2171334 · Cited by 121
Self-citationNNIME: The NTHU-NTUA Chinese interactive multimodal emotion corpus
Chou, H.-C., Lin, W.-C., Chang, L.-C., Li, C.-C., Ma, H.-P., & Lee, C.-C. (2017) · 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII) · doi:10.1109/acii.2017.8273615 · Cited by 57 · cited 6×
“Different from other scripted emotion speech database or segmented audio from dramas, NTHU-NTUA Chinese Interactive Multimodal Emotion Corpus (NNIME) [18], one of the few large-scale Chinese affective dyadic interaction database which contains various emotional nonverbal sounds, such as laughter,”
BackgroundInfluential
Cited by
A survey of the recent architectures of deep convolutional neural networks
Khan, A., Sohail, A., Zahoora, U., & Qureshi, A. S. (2020) · Artificial Intelligence Review · doi:10.1007/s10462-020-09825-6 · Cited by 3,137
CNN Variants for Computer Vision: History, Architecture, Application, Challenges and Future Scope
Bhatt, D., Patel, C., Talsania, H., Patel, J., Vaghela, R., Pandya, S., Modi, K., & Ghayvat, H. (2021) · Electronics · doi:10.3390/electronics10202470 · Cited by 890
“Hamid, in 2012, disclosed a speaker-independent voice recognition system using CNN [51].”
BackgroundSpeech emotion recognition with deep convolutional neural networks
Issa, D., Fatih Demirci, M., & Yazici, A. (2020) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2020.101894 · Cited by 619
Real-Time Video Emotion Recognition Based on Reinforcement Learning and Domain Knowledge
Zhang, K., Li, Y., Wang, J., Cambria, E., & Li, X. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3072412 · Cited by 184
“Huang et al.[24] focuses on nonverbal sounds which naturally exists in our daily conversation.”
Background
Metadata
- Authors
- Huang, K.-Y., Wu, C.-H., Hong, Q.-B., Su, M.-H., & Chen, Y.-H.
- Venue
- ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
- Publisher
- IEEE
- Published
- 2019-05
- Pages
- 5866-5870
- ISSN
- 1520-6149 (ISSN-L)
- OpenAlex
- W2938004196
- Cited by
- 116
- References
- 4
- FWCI
- 22.97
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Speech Emotion Recognition Using Deep Neural Network Considering Verbal and Nonverbal Speech Sounds. https://app.cochat.ai/papers/10.1109/icassp.2019.8682283
- The paper
- Huang, K.-Y., Wu, C.-H., Hong, Q.-B., Su, M.-H., & Chen, Y.-H. (2019). Speech Emotion Recognition Using Deep Neural Network Considering Verbal and Nonverbal Speech Sounds. ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). https://doi.org/10.1109/icassp.2019.8682283
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.