CoChat
Sign Up
Cited by 770Emotion and Mood RecognitionSpeech Recognition and SynthesisSpeech and Audio Processing

Automatic speech emotion recognition using recurrent neural networks with local attention

Mirsamadi, Seyedmahdad, Barsoum, Emad, Zhang, Cha·2017·2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
DOI 10.1109/icassp.2017.7952552
Open this paper in CoChat — free account

Abstract

Automatic emotion recognition from speech is a challenging task which relies heavily on the effectiveness of the speech features used for classification. In this work, we study the use of deep learning to automatically discover emotionally relevant features from speech. It is shown that using a deep recurrent neural network, we can learn both the short-time frame-level acoustic features that are emotionally relevant, as well as an appropriate temporal aggregation of those features into a compact utterance-level representation. Moreover, we propose a novel strategy for feature pooling over time which uses local attention in order to focus on specific regions of a speech signal that are more emotionally salient. The proposed solution is evaluated on the IEMOCAP corpus, and is shown to provide more accurate predictions compared to existing emotion recognition algorithms.

Integrity signals

Registry

Crossref · 10.1109/icassp.2017.7952552

Cites

12 of 12 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Deep learning is used for learning temporal feature representations in human affect recognition.

    “Learning temporal feature representations. To learn representations of the temporal dynamics found in audio [63], sequences of images [64], and physiological measurements [55], DNNs and especially RNNs are successfully applied.”

    From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [63]

  • Mirsamadi et al. [63] pointed out that most commonly used frame-level LLDs in SER can be derived from spectral representations of the raw speech signal.

    “Mirsamadi et al. [63] pointed out that most commonly used frame-level LLDs in SER can be derived from spectral representations of the raw speech signal.”

    From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [63]

  • Mirsamadi et al. [63] found that model performance is slightly lower with joint learning of both short-term spatial features and temporal context on the IEMOCAP dataset, while both improve model performance when applied independently.

    “Mirsamadi et al. [63] found that model performance is slightly lower with joint learning of both short-term spatial features and temporal context on the IEMOCAP dataset [165], while both improve model performance when applied independently.”

    From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [63]

  • In end-to-end training for spatial and temporal feature learning, the problem of limited labeled data becomes especially noticeable.

    “however in this setting the problem of limited labeled data becomes especially noticeable [63], [139].”

    From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [63]

  • Mirsamadi et al. found that even a simple attention mechanism contributes to SER by learning to ignore silent frames, adding 1-2% to model accuracy.

    “Mirsamadi et al. [63] found that even a simple attention mechanism contributes to SER by, for example, learning to ignore silent frames-an improvement that is easily interpretable and adds 1-2% to model accuracy.”

    From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [63]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Mirsamadi, S., Barsoum, E., & Zhang, C.
Affiliations
  • The University of Texas at Dallas (US)
  • Microsoft (United States) (US)
ORCID
Venue
2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Publisher
IEEE
Published
2017-03
Pages
2227-2231
ISSN
1520-6149 (ISSN-L)
DOI
10.1109/icassp.2017.7952552
OpenAlex
W2625297138
Cited by
770
References
12
Citations by year
2026: 37 · 2025: 58 · 2024: 72 · 2023: 100 · 2022: 106 · 2021: 138 · 2020: 123 · 2019: 84 · 2018: 48 · 2017: 4
FWCI
81.47
Type
conference paper
Language
en
Open access
closed
Locations
IEEE International Conference on Acoustics Speech and Signal Processing (publishedVersion)
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
  • Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
  • Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
Keywords
speech emotion recognition, recurrent neural networks, local attention, feature pooling, temporal aggregation, acoustic features, IEMOCAP, deep learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review