The DOI matches its Crossref record; 44 of the 51 references it cites were found in a registry or an index.
Domain Adversarial for Acoustic Emotion Recognition
Open accessAbstract
The performance of speech emotion recognition is affected by the differences in data distributions between train (source domain) and test (target domain) sets used to build and evaluate the models. This is a common problem, as multiple studies have shown that the performance of emotional classifiers drops when they are exposed to data that do not match the distribution used to build the emotion classifiers. The difference in data distributions becomes very clear when the training and testing data come from different domains, causing a large performance gap between development and testing performance. Due to the high cost of annotating new data and the abundance of unlabeled data, it is crucial to extract as much useful information as possible from the available unlabeled data. This study looks into the use of adversarial multitask training to extract a common representation between train and test domains. The primary task is to predict emotional-attribute-based descriptors for arousal, valence, or dominance. The secondary task is to learn a common representation, where the train and test domains cannot be distinguished. By using a gradient reversal layer, the gradients coming from the domain classifier are used to bring the source and target domain representations closer. We show that exploiting unlabeled data consistently leads to better emotion recognition performance across all emotional dimensions. We visualize the effect of adversarial training on the feature representation across the proposed deep learning architecture. The analysis shows that the data representations for the train and test domains converge as the data are passed to deeper layers of the network. We also evaluate the difference in performance when we use a shallow neural network versus a deep neural network and the effect of the number of shared layers used by the task and domain classifiers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
TensorFlow: A system for large-scale machine learningPreprint
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.5555/3026877.3026899 · Cited by 16,548
Domain-Adversarial Training of Neural Networks
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-58347-1_10 · Cited by 8,318
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869
Self-citationOpensmile: the munich versatile and fast open-source audio feature extractor
Eyben, F., Wöllmer, M., & Schuller, B. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1874246 · Cited by 2,642
Accelerating t-SNE using tree-based algorithms
van der Maaten, L. J. P. (2014) · Journal of machine learning research · doi:10.5555/2627435.2697068 · Cited by 2,211
Incorporating Nesterov Momentum into Adam
Dozat, T. (2016) · Cited by 1,595
Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach
Glorot, X., Bordes, A., & Bengio, Y. (2011) · Cited by 1,569
MSP-IMPROV: An Acted Corpus of Dyadic Interactions to Study Emotion Perception
Busso, C., Parthasarathy, S., Burmania, A., AbdelWahab, M., Sadoughi, N., & Provost, E. M. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2515617 · Cited by 391
Self-citationCross-Corpus Acoustic Emotion Recognition: Variances and Strategies
Schuller, B. W., Vlasenko, B., Eyben, F., Wöllmer, M., Stuhlsatz, A., Wendemuth, A., & Rigoll, G. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.8 · Cited by 386
Sparse Autoencoder-Based Feature Transfer Learning for Speech Emotion Recognition
Deng, J., Zhang, Z., Marchi, E., & Schuller, B. (2013) · 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction · doi:10.1109/acii.2013.90 · Cited by 366
Lotfian, R., & Busso, C. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2736999 · Cited by 330
Self-citationopenSMILE - The Munich Versatile and Fast Open-Source Audio Feature Extractor
Eyben, F. ;. (2013) · mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) · Cited by 322
Autoencoder-based Unsupervised Domain Adaptation for Speech Emotion Recognition
Deng, J., Zhang, Z., Eyben, F., & Schuller, B. W. (2014) · IEEE Signal Processing Letters · doi:10.1109/lsp.2014.2324759 · Cited by 268
Comparing Feature Sets for Acted and Spontaneous Speech in View of Automatic Emotion Recognition
Vogt, T., & André, E. (2005) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2005.1521463 · Cited by 223
Adversarial Multi-Task Learning of Deep Neural Networks for Robust Speech Recognition
Shinohara, Y. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-879 · Cited by 174
Unsupervised Domain Adaptation via Domain Adversarial Training for Speaker Recognition
Wang, Q., Rao, W., Sun, S., Xie, L., Chng, E. S., & Li, H. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461423 · Cited by 152
Jointly Predicting Arousal, Valence and Dominance with Multi-Task Learning
Parthasarathy, S., & Busso, C. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-1494 · Cited by 136
Self-citationDomain Adversarial Training for Accented Speech Recognition
Sun, S., Yeh, C.-F., Hwang, M.-Y., Mari Ostendorf, E., & Xie, L. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462663 · Cited by 128
Universum Autoencoder-Based Domain Adaptation for Speech Emotion Recognition
Deng, J., Xu, X., Zhang, Z., Fruhholz, S., & Schuller, B. (2017) · IEEE Signal Processing Letters · doi:10.1109/lsp.2017.2672753 · Cited by 125
Unsupervised learning in cross-corpus acoustic emotion recognition
Zhang, Z., Weninger, F. J., Wöllmer, M., & Schuller, B. W. (2011) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2011.6163986 · Cited by 124
The ordinal nature of emotions
Yannakakis, G. N., Cowie, R., & Busso, C. (2017) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2017.8273608 · Cited by 115
Self-citationSpeaker-Invariant Training Via Adversarial Learning
Meng, Z., Li, J., Chen, Z., Zhao, Y., Mazalov, V., Gong, Y., & Juang, B.-H. F. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461932 · Cited by 115
On Acoustic Emotion Recognition: Compensating for Covariate Shift
Hassan, A., Damper, R. I., & Niranjan, M. (2013) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2013.2255278 · Cited by 112
Increasing the Reliability of Crowdsourcing Evaluations Using Online Quality Assessment
Burmania, A., Parthasarathy, S., & Busso, C. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2493525 · Cited by 108
Self-citationLearning representations of emotional speech with deep convolutional generative adversarial networks
Chang, J., & Scherer, S. J. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7952656 · Cited by 108
Automatic Speech Classification To Five Emotional States Based On Gender Information
Ververidis, D., & Kotropoulos, C. L. (2004) · doi:10.5281/zenodo.38468 · Cited by 105
Cross-Corpus Speech Emotion Recognition Based on Domain-Adaptive Least-Squares Regression
Zong, Y., Zheng, W., Zhang, T., & Huang, X. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2537926 · Cited by 96
Toward Effective Automatic Recognition Systems of Emotion in Speech
Busso, C., Bulut, M., & Narayanan, S. S. (2013) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195387643.003.0008 · Cited by 87
Self-citationSupervised domain adaptation for emotion recognition from speech
Abdelwahab, M., & Busso, C. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178934 · Cited by 80
Self-citationDeng, J., Xia, R., Zhang, Z., Liu, Y., & Schuller, B. W. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854517 · Cited by 72
Cross-corpus speech emotion recognition based on transfer non-negative matrix factorization
Song, P., Zheng, W., Ou, S., Zhang, X., Jin, Y., Liu, J., & Yu, Y. (2016) · Speech Communication · doi:10.1016/j.specom.2016.07.010 · Cited by 63
Zhang, Y., Liu, Y., Weninger, F., & Schuller, B. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7953106 · Cited by 55
Automatic Classification of Expressiveness in Speech: A Multi-corpus Study
Shami, M., & Verhelst, W. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74122-0_5 · Cited by 53
Noise Adaptive Speech Enhancement Using Domain Adversarial Training
Liao, C.-F., Tsao, Y., Lee, H.-Y., & Wang, H. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-1519 · Cited by 52
Improving Speaker-Independent Lipreading with Domain-Adversarial Training
Wand, M., & Schmidhuber, J. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-421 · Cited by 50
Lotfian, R., & Busso, C. (2017) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2017.8273633 · Cited by 44
Self-citationSagha, H., Deng, J., Gavryukova, M., Han, J., & Schuller, B. W. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472789 · Cited by 42
Emotion Recognition from Speech by Combining Databases and Fusion of Classifiers
Lefter, I., Rothkrantz, L. J. M., Wiggers, P., & van Leeuwen, D. A. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15760-8_45 · Cited by 39
Unveiling the acoustic properties that describe the valence dimension
Busso, C., & Rahman, T. (2012) · Interspeech (USB) · doi:10.21437/interspeech.2012-124 · Cited by 38
Self-citationA personalized emotion recognition system using an unsupervised feature adaptation scheme
Rahman, T., & Busso, C. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6289072 · Cited by 34
Self-citationMao, Q., Xue, W., Rao, Q., Zhang, F., & Zhan, Y. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472149 · Cited by 34
Fuzzy emotion recognition in natural speech dialogue
Austermann, A., Esau, N., Kleinjohann, L., & Kleinjohann, B. (2006) · IEEE RO-MAN · doi:10.1109/roman.2005.1513798 · Cited by 29
Ensemble feature selection for domain adaptation in speech emotion recognition
Abdelwahab, M., & Busso, C. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7953108 · Cited by 27
Self-citationIncremental adaptation using active learning for acoustic emotion recognition
Abdelwahab, M., & Busso, C. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7953140 · Cited by 20
Self-citation
Research works printed but not linked: 7
- goodfellow, 0, Generative adversarial nets, Proc Int Conf Neural Inf Process, 27, 2672
- kim, 0, Say cheese vs. smile: Reducing speech-related variability for facial emotion recognition, Proc ACM Int Conf Multimedia, 27
- vidrascu, 0, Anger detection performances based on prosodic and acoustic cues in several corpora, Proc 2nd Int Workshop Emotion Corpora Res Emotion Affect Int Conf Lang Resour Eval, 13
- devillers, 0, Real-life emotion-related states detection in call centers: A cross-corpora study, Proc INTERSPEECH, 2350
- parthasarathy, 0, Ladder networks for emotion recognition: Using unsupervised auxiliary tasks to improve predictions of emotional attributes, Proc INTERSPEECH, 3698
- busso, 0, Recording audio-visual emotional databases from actors: A closer look, Proc 2nd Int Workshop Emotion Corpora Res Emotion Affect/Int Conf Lang Resour Eval, 17
- schuller, 0, The INTERSPEECH 2013 computational paralinguistics challenge: Social signals, conflict, emotion, autism, Proc INTERSPEECH, 148
Cited by
Sorted by citations
Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review
Zhang, J., Yin, Z., Chen, P., & Nichele, S. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.01.011 · Cited by 806
Akçay, M. B., & Oğuz, K. (2019) · Speech Communication · doi:10.1016/j.specom.2019.12.001 · Cited by 743
A systematic review on affective computing: emotion models, databases, and recent advances
Wang, Y., Song, W., Tao, W., Liotta, A., Yang, D., Li, X., Gao, S., Sun, Y., Ge, W., Zhang, W., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.009 · Cited by 713
A Review on Speech Emotion Recognition Using Deep Learning and Attention Mechanism
Lieskovská, E., Jakubec, M., Jarina, R., & Chmulík, M. (2021) · Electronics · doi:10.3390/electronics10101163 · Cited by 246
Automated emotion recognition: Current trends and future perspectives
Maithri, M., Raghavendra, U., Gudigar, A., Samanth, J., Barua, P. D., Murugappan, M., Chakole, Y., & Acharya, U. R. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.106646 · Cited by 172
Metadata
- Authors
- Abdelwahab, M., & Busso, C.
- Affiliations
- The University of Texas at Dallas (US)
- Venue
- IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Published
- 2018-12
- Volume
- 26
- Issue
- 12
- Pages
- 2423-2435
- ISSN
- 2329-9290 (ISSN-L)
- 2329-9304
- OpenAlex
- W2798632413
- Cited by
- 178
- References
- 51
- Citations by year
- 2026: 4 · 2025: 16 · 2024: 20 · 2023: 22 · 2022: 22 · 2021: 40 · 2020: 29 · 2019: 23 · 2018: 2
- FWCI
- 30.78
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/ACM Transactions on Audio Speech and Language Processing (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (public-domain)
- Funders
- National Science Foundation (IIS-1453781)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- acoustic emotion recognition, domain adaptation, domain adversarial learning, gradient reversal layer, unlabeled data, shared representation learning, multi-task learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
