Domain Adversarial for Acoustic Emotion Recognition
Open accessAbstract
The performance of speech emotion recognition is affected by the differences in data distributions between train (source domain) and test (target domain) sets used to build and evaluate the models. This is a common problem, as multiple studies have shown that the performance of emotional classifiers drops when they are exposed to data that do not match the distribution used to build the emotion classifiers. The difference in data distributions becomes very clear when the training and testing data come from different domains, causing a large performance gap between development and testing performance. Due to the high cost of annotating new data and the abundance of unlabeled data, it is crucial to extract as much useful information as possible from the available unlabeled data. This study looks into the use of adversarial multitask training to extract a common representation between train and test domains. The primary task is to predict emotional-attribute-based descriptors for arousal, valence, or dominance. The secondary task is to learn a common representation, where the train and test domains cannot be distinguished. By using a gradient reversal layer, the gradients coming from the domain classifier are used to bring the source and target domain representations closer. We show that exploiting unlabeled data consistently leads to better emotion recognition performance across all emotional dimensions. We visualize the effect of adversarial training on the feature representation across the proposed deep learning architecture. The analysis shows that the data representations for the train and test domains converge as the data are passed to deeper layers of the network. We also evaluate the difference in performance when we use a shallow neural network versus a deep neural network and the effect of the number of shared layers used by the task and domain classifiers.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
TensorFlow: A system for large-scale machine learningPreprint
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.5555/3026877.3026899 · Cited by 16,369 · see citing works
Domain-Adversarial Training of Neural Networks
Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., & Lempitsky, V. (2017) · Advances in Computer Vision and Pattern Recognition · doi:10.1007/978-3-319-58347-1_10 · Cited by 8,318 · see citing works
IEMOCAP: Interactive emotional dyadic motion capture database
Busso, C., Bulut, M., Lee, C.-C., Kazemzadeh, A., Mower, E., Kim, S., Chang, J. N., Lee, S., & Narayanan, S. S. (2008) · Language Resources and Evaluation · doi:10.1007/s10579-008-9076-6 · Cited by 3,869 · see citing works
Self-citationOpensmile: the munich versatile and fast open-source audio feature extractor
Eyben, F., Wöllmer, M., & Schuller, B. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1874246 · Cited by 2,642 · see citing works
Accelerating t-SNE using tree-based algorithms
van der Maaten, L. (2014) · Journal of machine learning research · doi:10.5555/2627435.2697068 · Cited by 2,207 · see citing works
Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach
Glorot, X., Bordes, A., & Bengio, Y. (2011) · Cited by 1,569 · see citing works
Incorporating Nesterov Momentum into Adam
Dozat, T. (2016) · Cited by 1,293 · see citing works
MSP-IMPROV: An Acted Corpus of Dyadic Interactions to Study Emotion Perception
Busso, C., Parthasarathy, S., Burmania, A., AbdelWahab, M., Sadoughi, N., & Provost, E. M. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2515617 · Cited by 391 · see citing works
Self-citationCross-Corpus Acoustic Emotion Recognition: Variances and Strategies
Schuller, B. W., Vlasenko, B., Eyben, F., Wöllmer, M., Stuhlsatz, A., Wendemuth, A., & Rigoll, G. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.8 · Cited by 386 · see citing works
Sparse Autoencoder-Based Feature Transfer Learning for Speech Emotion Recognition
Deng, J., Zhang, Z., Marchi, E., & Schuller, B. (2013) · 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction · doi:10.1109/acii.2013.90 · Cited by 366 · see citing works
Lotfian, R., & Busso, C. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2736999 · Cited by 330 · see citing works
Self-citationopenSMILE - The Munich Versatile and Fast Open-Source Audio Feature Extractor
Eyben, F. ;. (2013) · mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) · Cited by 322 · see citing works
Autoencoder-based Unsupervised Domain Adaptation for Speech Emotion Recognition
Deng, J., Zhang, Z., Eyben, F., & Schuller, B. W. (2014) · IEEE Signal Processing Letters · doi:10.1109/lsp.2014.2324759 · Cited by 268 · see citing works
Comparing Feature Sets for Acted and Spontaneous Speech in View of Automatic Emotion Recognition
Vogt, T., & André, E. (2005) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2005.1521463 · Cited by 223 · see citing works
Adversarial Multi-Task Learning of Deep Neural Networks for Robust Speech Recognition
Shinohara, Y. (2016) · Interspeech (USB) · doi:10.21437/interspeech.2016-879 · Cited by 174 · see citing works
Unsupervised Domain Adaptation via Domain Adversarial Training for Speaker Recognition
Wang, Q., Rao, W., Sun, S., Xie, L., Chng, E. S., & Li, H. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461423 · Cited by 152 · see citing works
Jointly Predicting Arousal, Valence and Dominance with Multi-Task Learning
Parthasarathy, S., & Busso, C. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-1494 · Cited by 136 · see citing works
Self-citationDomain Adversarial Training for Accented Speech Recognition
Sun, S., Yeh, C.-F., Hwang, M.-Y., Mari Ostendorf, E., & Xie, L. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8462663 · Cited by 128 · see citing works
Universum Autoencoder-Based Domain Adaptation for Speech Emotion Recognition
Deng, J., Xu, X., Zhang, Z., Fruhholz, S., & Schuller, B. (2017) · IEEE Signal Processing Letters · doi:10.1109/lsp.2017.2672753 · Cited by 125 · see citing works
Unsupervised learning in cross-corpus acoustic emotion recognition
Zhang, Z., Weninger, F. J., Wöllmer, M., & Schuller, B. W. (2011) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2011.6163986 · Cited by 124 · see citing works
Speaker-Invariant Training Via Adversarial Learning
Meng, Z., Li, J., Chen, Z., Zhao, Y., Mazalov, V., Gong, Y., & Juang, B.-H. F. (2018) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2018.8461932 · Cited by 115 · see citing works
On Acoustic Emotion Recognition: Compensating for Covariate Shift
Hassan, A., Damper, R. I., & Niranjan, M. (2013) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2013.2255278 · Cited by 112 · see citing works
The ordinal nature of emotions
Yannakakis, G. N., Cowie, R., & Busso, C. (2017) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2017.8273608 · Cited by 112 · see citing works
Self-citationLearning representations of emotional speech with deep convolutional generative adversarial networks
Chang, J., & Scherer, S. J. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7952656 · Cited by 112 · see citing works
Increasing the Reliability of Crowdsourcing Evaluations Using Online Quality Assessment
Burmania, A., Parthasarathy, S., & Busso, C. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2493525 · Cited by 108 · see citing works
Self-citationAutomatic Speech Classification To Five Emotional States Based On Gender Information
Ververidis, D., & Kotropoulos, C. L. (2004) · doi:10.5281/zenodo.38468 · Cited by 105 · see citing works
Cross-Corpus Speech Emotion Recognition Based on Domain-Adaptive Least-Squares Regression
Zong, Y., Zheng, W., Zhang, T., & Huang, X. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2537926 · Cited by 96 · see citing works
Toward Effective Automatic Recognition Systems of Emotion in Speech
Busso, C., Bulut, M., & Narayanan, S. S. (2013) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195387643.003.0008 · Cited by 87 · see citing works
Self-citationSupervised domain adaptation for emotion recognition from speech
Abdelwahab, M., & Busso, C. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178934 · Cited by 80 · see citing works
Self-citationDeng, J., Xia, R., Zhang, Z., Liu, Y., & Schuller, B. W. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854517 · Cited by 72 · see citing works
Cross-corpus speech emotion recognition based on transfer non-negative matrix factorization
Song, P., Zheng, W., Ou, S., Zhang, X., Jin, Y., Liu, J., & Yu, Y. (2016) · Speech Communication · doi:10.1016/j.specom.2016.07.010 · Cited by 63 · see citing works
Zhang, Y., Liu, Y., Weninger, F., & Schuller, B. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7953106 · Cited by 55 · see citing works
Automatic Classification of Expressiveness in Speech: A Multi-corpus Study
Shami, M., & Verhelst, W. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74122-0_5 · Cited by 53 · see citing works
Noise Adaptive Speech Enhancement Using Domain Adversarial Training
Liao, C.-F., Tsao, Y., Lee, H.-Y., & Wang, H. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-1519 · Cited by 52 · see citing works
Improving Speaker-Independent Lipreading with Domain-Adversarial Training
Wand, M., & Schmidhuber, J. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-421 · Cited by 45 · see citing works
Sagha, H., Deng, J., Gavryukova, M., Han, J., & Schuller, B. W. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472789 · Cited by 42 · see citing works
Emotion Recognition from Speech by Combining Databases and Fusion of Classifiers
Lefter, I., Rothkrantz, L. J. M., Wiggers, P., & van Leeuwen, D. A. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15760-8_45 · Cited by 39 · see citing works
Lotfian, R., & Busso, C. (2017) · International Conference on Affective Computing and Intelligent Interaction (ACII) · doi:10.1109/acii.2017.8273633 · Cited by 38 · see citing works
Self-citationUnveiling the acoustic properties that describe the valence dimension
Busso, C., & Rahman, T. (2012) · Interspeech (USB) · doi:10.21437/interspeech.2012-124 · Cited by 38 · see citing works
Self-citationA personalized emotion recognition system using an unsupervised feature adaptation scheme
Rahman, T., & Busso, C. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6289072 · Cited by 34 · see citing works
Self-citationMao, Q., Xue, W., Rao, Q., Zhang, F., & Zhan, Y. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472149 · Cited by 34 · see citing works
Fuzzy emotion recognition in natural speech dialogue
Austermann, A., Esau, N., Kleinjohann, L., & Kleinjohann, B. (2006) · IEEE RO-MAN · doi:10.1109/roman.2005.1513798 · Cited by 29 · see citing works
Ensemble feature selection for domain adaptation in speech emotion recognition
Abdelwahab, M., & Busso, C. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7953108 · Cited by 27 · see citing works
Self-citationIncremental adaptation using active learning for acoustic emotion recognition
Abdelwahab, M., & Busso, C. (2017) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2017.7953140 · Cited by 20 · see citing works
Self-citation
Flagged first, then by citations
7 of the 51 references couldn’t be matched to a known paper.
Research works printed but not linked: 5
- [2][2] Y. Kim and E. Mower Provost, 'Say cheese vs. smile: Reducing speechrelated variability for facial emotion recognition,' in ACM International Conference on Multimedia (MM 2014) , Orlando, FL, USA, November 2014, pp. 27-36. -
- [18][18] L. Vidrascu and L. Devillers, 'Anger detection performances based on prosodic and acoustic cues in several corpora,' in Second International Workshop on Emotion: Corpora for Research on Emotion and Affect, International conference on Language Resources and Evaluation (LREC 2008) , Marrakech, Morocco, May 2008, pp. 13-16. -
- [29][29] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, 'Generative adversarial nets,' in Advances in neural information processing systems (NIPS 2014) , Montreal, Canada, December 2014, vol. 27, pp. 2672-2680. -
- [33][33] C. Busso and S.S. Narayanan, 'Recording audio-visual emotional databases from actors: a closer look,' in Second International Workshop on Emotion: Corpora for Research on Emotion and Affect, International conference on Language Resources and Evaluation (LREC 2008) , Marrakech, Morocco, May 2008, pp. 17-22. -
- [35][35] B. Schuller, S. Steidl, A. Batliner, A. Vinciarelli, K. Scherer, F. Ringeval, M. Chetouani, F. Weninger, F. Eyben, E. Marchi, M. Mortillaro, H. Salamin, A. Polychroniou, F. Valente, and S. Kim, 'The INTERSPEECH 2013 computational paralinguistics challenge: Social signals, conflict, emotion, autism,' in Interspeech 2013 , Lyon, France, August 2013, pp. 148-152. -
Cited by
Sorted by citations
Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review
Zhang, J., Yin, Z., Chen, P., & Nichele, S. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.01.011 · Cited by 806 · see citing works
Akçay, M. B., & Oğuz, K. (2019) · Speech Communication · doi:10.1016/j.specom.2019.12.001 · Cited by 743 · see citing works
A systematic review on affective computing: emotion models, databases, and recent advances
Wang, Y., Song, W., Tao, W., Liotta, A., Yang, D., Li, X., Gao, S., Sun, Y., Ge, W., Zhang, W., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.009 · Cited by 657 · see citing works
A Review on Speech Emotion Recognition Using Deep Learning and Attention Mechanism
Lieskovská, E., Jakubec, M., Jarina, R., & Chmulík, M. (2021) · Electronics · doi:10.3390/electronics10101163 · Cited by 239 · see citing works
Automated emotion recognition: Current trends and future perspectives
Maithri, M., Raghavendra, U., Gudigar, A., Samanth, J., Barua, P. D., Murugappan, M., Chakole, Y., & Acharya, U. R. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.106646 · Cited by 172 · see citing works
Survey of Deep Representation Learning for Speech Emotion Recognition
Latif, S., Rana, R. K., Khalifa, S., Jurdak, R., Qadir, J., & Schuller, B. W. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3114365 · Cited by 178 · see citing works
A survey of speech emotion recognition in natural environment
Fahad, M. S., Ranjan, A., Yadav, J., & Deepak, A. (2020) · Digital Signal Processing · doi:10.1016/j.dsp.2020.102951 · Cited by 162 · see citing works
A systematic literature review of speech emotion recognition approaches
Singh, Y. B., & Goel, S. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.04.028 · Cited by 150 · see citing works
Pan, B., Hirota, K., Jia, Z., & Dai, Y. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126866 · Cited by 142 · see citing works
Affective Computing: Recent Advances, Challenges, and Future Trends
Pei, G., Li, H., Lu, Y., Wang, Y., Hua, S., & Li, T. (2023) · Intelligent Computing · doi:10.34133/icomputing.0076 · Cited by 161 · see citing works
Improving Speech Emotion Recognition with Unsupervised Representation Learning on Unlabeled Speech
Neumann, M., & Vu, N. T. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682541 · Cited by 140 · see citing works
Gideon, J., McInnis, M. G., & Provost, E. M. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2916092 · Cited by 128 · see citing works
Semi-Supervised Speech Emotion Recognition With Ladder Networks
Parthasarathy, S., & Busso, C. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.3023632 · Cited by 119 · see citing works
Self-citationCurriculum Learning for Speech Emotion Recognition From Crowdsourced Labels
Lotfian, R., & Busso, C. (2019) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2019.2898816 · Cited by 97 · see citing works
Self-citationLong Dialogue Emotion Detection Based on Commonsense Knowledge Graph Guidance
Nie, W., Bao, Y., Zhao, Y., & Liu, A. (2024) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3267295 · Cited by 96 · see citing works
End-to-End Audiovisual Speech Recognition System With Multitask Learning
Tao, F., & Busso, C. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.2975922 · Cited by 95 · see citing works
Self-citationSpeech emotion recognition approaches: A systematic review
Hashem, A., Arif, M., & Alghamdi, M. (2023) · Speech Communication · doi:10.1016/j.specom.2023.102974 · Cited by 99 · see citing works
A Review of Generalizable Transfer Learning in Automatic Emotion Recognition
Feng, K., & Chaspari, T. (2020) · Frontiers in Computer Science · doi:10.3389/fcomp.2020.00009 · Cited by 82 · see citing works
Han, J., Zhang, Z., Cummins, N., & Schuller, B. W. (2019) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2019.2901088 · Cited by 86 · see citing works
Multimodal Emotion Recognition With Temporal and Semantic Consistency
Chen, B., Cao, Q., Hou, M., Zhang, Z., Lu, G., & Zhang, D. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3129331 · Cited by 71 · see citing works
Latif, S., Rana, R. K., Khalifa, S., Jurdak, R., & Schuller, B. W. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3167013 · Cited by 71 · see citing works
Deep Representation Learning in Speech Processing: Challenges, Recent Advances, and Future TrendsPreprint
Latif, S., Rana, R. K., Khalifa, S., Jurdak, R., Qadir, J., & Schuller, B. W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.00378 · Cited by 87 · see citing works
Domain Invariant Feature Learning for Speaker-Independent Speech Emotion Recognition
Lu, C., Zong, Y., Zheng, W., Li, Y., Tang, C., & Schuller, B. W. (2022) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2022.3178232 · Cited by 69 · see citing works
Unsupervised Adversarial Domain Adaptation for Cross-Lingual Speech Emotion Recognition
Latif, S., Qadir, J., & Bilal, M. (2019) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2019.8925513 · Cited by 64 · see citing works
EmoBed: Strengthening Monomodal Emotion Recognition via Training with Crossmodal Emotion Embeddings
Han, J., Zhang, Z., Ren, Z. X., & Schuller, B. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2928297 · Cited by 64 · see citing works
Toward Label-Efficient Emotion and Sentiment Analysis
Zhao, S., Hong, X., Yang, J., Zhao, Y., & Ding, G. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3309299 · Cited by 60 · see citing works
Speaker-Invariant Affective Representation Learning via Adversarial Training
Li, H., Tu, M., Huang, J., Narayanan, S. S., & Georgiou, P. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054580 · Cited by 57 · see citing works
Cross-Corpus Speech Emotion Recognition Based on Few-Shot Learning and Domain Adaptation
Ahn, Y., Lee, S. J., & Shin, J. W. (2021) · IEEE Signal Processing Letters · doi:10.1109/lsp.2021.3086395 · Cited by 56 · see citing works
Li, H., Chen, X., Wang, J., Wu, D., & Liu, X. (2021) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3494954 · Cited by 54 · see citing works
Transfer Sparse Discriminant Subspace Learning for Cross-Corpus Speech Emotion Recognition
Zhang, W., & Song, P. (2019) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2019.2955252 · Cited by 49 · see citing works
Active Learning for Speech Emotion Recognition Using Deep Neural Network
Abdelwahab, M., & Busso, C. (2019) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2019.8925524 · Cited by 49 · see citing works
Self-citationLuo, H., & Han, J. (2020) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2020.3006331 · Cited by 46 · see citing works
Atmaja, B. T., & Akagi, M. (2020) · Speech Communication · doi:10.1016/j.specom.2020.11.003 · Cited by 46 · see citing works
Atmaja, B. T., & Akagi, M. (2021) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1896/1/012004 · Cited by 48 · see citing works
LSSED: A Large-Scale Dataset and Benchmark for Speech Emotion Recognition
Fan, W., Xu, X., Xing, X., Chen, W., & Huang, D. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414542 · Cited by 44 · see citing works
Deep Cross-Corpus Speech Emotion Recognition: Recent Advances and Perspectives
Zhang, S., Liu, R., Tao, X., & Zhao, X. (2021) · Frontiers in Neurorobotics · doi:10.3389/fnbot.2021.784514 · Cited by 44 · see citing works
ADAPT : Awesome Domain Adaptation Python ToolboxPreprint
de Mathelin, A., Atiq, M., Richard, G., de la Concha, A., Yachouti, M., Deheeger, F., Mougeot, M., & Vayatis, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.03049 · Cited by 42 · see citing works
Jing, F., Zhang, S., & Zhang, S. (2020) · BMC Bioinformatics · doi:10.1186/s12859-020-03844-4 · Cited by 39 · see citing works
Recurrent synchronization network for emotion-cause pair extraction
Chen, F., Shi, Z., Yang, Z., & Huang, Y. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.107965 · Cited by 40 · see citing works
Context-Dependent Domain Adversarial Neural Network for Multimodal Emotion Recognition
Lian, Z., Tao, J., Liu, B., Huang, J., Yang, Z., & Li, R. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-1705 · Cited by 38 · see citing works
Adversarial Domain Generalized Transformer for Cross-Corpus Speech Emotion Recognition
Gao, Y., Wang, L., Liu, J., Dang, J., & Okada, S. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3290795 · Cited by 35 · see citing works
Naderi, N., & Nasersharif, B. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110814 · Cited by 35 · see citing works
Xiao, Y., Zhao, H., & Li, T. (2020) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2020.2972926 · Cited by 34 · see citing works
A Cross-Corpus Study on Speech Emotion Recognition
Milner, R., Jalal, M. A., Ng, R. W. M., & Hain, T. (2019) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru46091.2019.9003838 · Cited by 34 · see citing works
Adversarial Training in Affective Computing and Sentiment Analysis: Recent Advances and PerspectivesPreprint
Han, J., Zhang, Z.-X., Cummins, N., & Schuller, B. (2018) · arXiv.org · doi:10.48550/arxiv.1809.08927 · Cited by 66 · see citing works
Semi-supervised cross-lingual speech emotion recognition
Agarla, M., Bianco, S., Celona, L., Napoletano, P., Petrovsky, A., Piccoli, F., Schettini, R., & Shanin, I. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121368 · Cited by 33 · see citing works
Ocquaye, E. N. N., Mao, Q., Xue, Y., & Song, H. (2020) · International Journal of Intelligent Systems · doi:10.1002/int.22291 · Cited by 32 · see citing works
Feng, K., & Chaspari, T. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3109485 · Cited by 32 · see citing works
Hybrid deep models for parallel feature extraction and enhanced emotion state classification
Pichandi, S., Gomathy, B., & Venkatesh, C. (2024) · Scientific Reports · doi:10.1038/s41598-024-75850-y · Cited by 31 · see citing works
Wen, X.-C., Ye, J., Luo, Y., Xu, Y. N., Wang, X.-Z., Wu, C.-L., & Liu, K. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/320 · Cited by 29 · see citing works
Multi-label emotion classification based on adversarial multi-task learning
Lin, N., Fu, S., Lin, X., & Wang, L. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103097 · Cited by 28 · see citing works
Unsupervised Cross-Corpus Speech Emotion Recognition Using a Multi-Source Cycle-GAN
Su, B.-H., & Lee, C.-C. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3146325 · Cited by 26 · see citing works
Research on Speech Emotion Recognition Technology based on Deep and Shallow Neural Network
Wang, J., & Han, Z. (2019) · Chinese Control Conference (CCC) · doi:10.23919/chicc.2019.8866568 · Cited by 25 · see citing works
A Study on Cross-Corpus Speech Emotion Recognition and Data Augmentation
Braunschweiler, N., Doddipatla, R., Keizer, S., & Stoyanchev, S. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9687987 · Cited by 24 · see citing works
Multitask Transformer for Cross-Corpus Speech Emotion Recognition
Ahn, C.-S., Rana, R. K., Busso, C., & Rajapakse, J. C. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3526592 · Cited by 23 · see citing works
Self-citationLee, C.-C., Sridhar, K., Li, J.-L., Lin, W.-C., Su, B.-H., & Busso, C. (2021) · IEEE Signal Processing Magazine · doi:10.1109/msp.2021.3105939 · Cited by 23 · see citing works
Self-citationHan, J., Zhang, Z., Pantić, M., & Schuller, B. W. (2020) · Future Generation Computer Systems · doi:10.1016/j.future.2020.08.002 · Cited by 22 · see citing works
Zhang, S., Liu, R., Yang, Y., Zhao, X., & Yu, J. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548328 · Cited by 21 · see citing works
Multimodal speech emotion recognition based on multi-scale MFCCs and multi-view attention mechanism
Feng, L., Liu, L., Liu, S., Zhou, J., Yang, H., & Yang, J. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-14600-0 · Cited by 21 · see citing works
Naini, A. R., Kohler, M. A., Richerson, E., Robinson, D., & Busso, C. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447678 · Cited by 21 · see citing works
Self-citationMulti-scale discrepancy adversarial network for cross-corpus speech emotion recognition
Zheng, W., Zheng, W., & Zong, Y. (2021) · Virtual Reality & Intelligent Hardware · doi:10.1016/j.vrih.2020.11.006 · Cited by 20 · see citing works
Lexical Dependent Emotion Detection Using Synthetic Speech Reference
Lotfian, R., & Busso, C. (2019) · IEEE Access · doi:10.1109/access.2019.2898353 · Cited by 20 · see citing works
Self-citationDeep Implicit Distribution Alignment Networks for cross-Corpus Speech Emotion Recognition
Zhao, Y., Wang, J., Zong, Y., Zheng, W., Lian, H., & Zhao, L. (2023) · ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp49357.2023.10095388 · Cited by 19 · see citing works
A Conditional Cycle Emotion Gan for Cross Corpus Speech Emotion Recognition
Su, B.-H., & Lee, C.-C. (2021) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt48900.2021.9383512 · Cited by 19 · see citing works
Lee, C.-C., Chaspari, T., Provost, E. M., & Narayanan, S. S. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3276209 · Cited by 20 · see citing works
Towards adversarial learning of speaker-invariant representation for speech emotion recognitionPreprint
Tu, M., Tang, Y., Huang, J., He, X., & Zhou, B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.09606 · Cited by 18 · see citing works
Unsupervised Cross-Lingual Speech Emotion Recognition Using Domain Adversarial Neural Network
Cai, X., Wu, Z., Zhong, K., Su, B., Dai, D., & Meng, H. M. L. (2021) · International Symposium on Chinese Spoken Language Processing (ISCSLP) · doi:10.1109/iscslp49672.2021.9362058 · Cited by 17 · see citing works
Modeling Uncertainty in Predicting Emotional Attributes from Spontaneous Speech
Sridhar, K., & Busso, C. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054237 · Cited by 16 · see citing works
Self-citationDeepemocluster: a Semi-Supervised Framework for Latent Cluster Representation of Speech Emotions
Lin, W.-C., Sridhar, K., & Busso, C. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414035 · Cited by 16 · see citing works
Self-citationSpoken Language Identification in Unseen Target Domain Using Within-Sample Similarity Loss
Muralikrishna, H., Kapoor, S., Dinesh, D. A., & Rajan, P. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414090 · Cited by 15 · see citing works
Amjad, A., Khuntia, S., Chang, H.-T., & Tai, L. (2024) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2024.3498694 · Cited by 15 · see citing works
Disentanglement for Audio-Visual Emotion Recognition Using Multitask Setup
Peri, R., Parthasarathy, S., Bradshaw, C., & Sundaram, S. (2021) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp39728.2021.9414705 · Cited by 15 · see citing works
Adaptive Domain-Aware Representation Learning for Speech Emotion Recognition
Fan, W., Xu, X., Xing, X., & Huang, D. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-2572 · Cited by 14 · see citing works
Speech emotion recognition based on syllable-level feature extraction
Rehman, A., Liu, Z., Wu, M., Cao, W., & Jiang, C.-S. (2023) · Applied Acoustics · doi:10.1016/j.apacoust.2023.109444 · Cited by 16 · see citing works
Improving Valence Prediction in Dimensional Speech Emotion Recognition Using Linguistic Information
Atmaja, B. T., & Akagi, M. (2020) · International Conference on Speech Database and Assessments · doi:10.1109/o-cocosda50338.2020.9295032 · Cited by 14 · see citing works
Unsupervised Cross-Lingual Speech Emotion Recognition Using Pseudo Multilabel
Jin, L., Yan, N., & Wang, L. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9688171 · Cited by 13 · see citing works
Domain-Invariant Feature Learning for Cross Corpus Speech Emotion Recognition
Gao, Y., Okada, S., Wang, L., Liu, J., & Dang, J. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9747129 · Cited by 13 · see citing works
Vaaras, E., Ahlqvist‐Björkroth, S., Drossos, K., Lehtonen, L., & Räsänen, O. J. (2023) · Speech Communication · doi:10.1016/j.specom.2023.02.001 · Cited by 13 · see citing works
Controlling for Confounders in Multimodal Emotion Classification via Adversarial Learning
Jaiswal, M., Aldeneh, Z., & Provost, E. M. (2019) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3340555.3353731 · Cited by 13 · see citing works
Upadhyay, S. G., Martinez-Lucas, L., Su, B.-H., Lin, W.-C., Chien, W.-S., Wu, Y.-T., Katz, W. F., Busso, C., & Lee, C.-C. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095250 · Cited by 12 · see citing works
Self-citationDomain Generalization with Triplet Network for Cross-Corpus Speech Emotion Recognition
Lee, S.-W. (2021) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt48900.2021.9383534 · Cited by 12 · see citing works
Unsupervised Domain Adaptation for Preference Learning Based Speech Emotion Recognition
Naini, A. R., Kohler, M. A., & Busso, C. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10094301 · Cited by 12 · see citing works
Self-citationEmotion Recognition with Capsule Neural Network
Van, L. T., Nguyen, Q. H., & Thủy, Đ. T. (2021) · Computer Systems Science and Engineering · doi:10.32604/csse.2022.021635 · Cited by 12 · see citing works
Progressively Discriminative Transfer Network for Cross-Corpus Speech Emotion Recognition
Lu, C., Tang, C., Zhang, J., & Zong, Y. (2022) · Entropy · doi:10.3390/e24081046 · Cited by 11 · see citing works
Learning Domain Invariant Representations for Child-Adult Classification from Speech
Lahiri, R., Kumar, M., Bishop, S. L., & Narayanan, S. S. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054276 · Cited by 11 · see citing works
Investigation of the Effect of Increased Dimension Levels in Speech Emotion Recognition
Wang, H., Zhao, X., & Zhao, Y. (2022) · IEEE Access · doi:10.1109/access.2022.3194039 · Cited by 10 · see citing works
A review on EEG based Emotion Analysis using Machine Learning approaches
Sharma, T., Diwakar, M., Singh, P., Arya, C., Lamba, S., & Kumar, P. (2021) · IEEE Uttar Pradesh Section International Conference on Electrical, Electronics and Computer Engineering (UPCON) · doi:10.1109/upcon52273.2021.9667588 · Cited by 9 · see citing works
Multi-layer maximum mean discrepancy in auto-encoders for cross-corpus speech emotion recognition
Nasersharif, B., Ebrahimpour, M., & Naderi, N. (2023) · The Journal of Supercomputing · doi:10.1007/s11227-023-05161-y · Cited by 10 · see citing works
Jia, N., & Zheng, C. (2021) · Arabian Journal for Science and Engineering · doi:10.1007/s13369-021-06090-9 · Cited by 9 · see citing works
Chang, C.-M., Chao, G.-Y., & Lee, C.-C. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3111110 · Cited by 9 · see citing works
Chen, C., & Zhang, P. (2022) · Interspeech (USB) · doi:10.21437/interspeech.2022-10403 · Cited by 9 · see citing works
Unsupervised Domain Adaptation Schemes for Building ASR in Low-Resource Languages
Anoop, C. S., Prathosh, A. P., & Ramakrishnan, A. G. (2021) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru51503.2021.9688269 · Cited by 14 · see citing works
Cross-corpus speech emotion recognition using semi-supervised domain adaptation network
Zhang, Y., Jia, M., Cao, X., Ru, J., & Zhang, X. (2024) · Speech Communication · doi:10.1016/j.specom.2024.103185 · Cited by 8 · see citing works
Cross-corpus speech emotion recognition using subspace learning and domain adaption
Cao, X., Jia, M., Ru, J., & Pai, T. (2022) · EURASIP Journal on Audio Speech and Music Processing · doi:10.1186/s13636-022-00264-5 · Cited by 8 · see citing works
Exploring Transfer Learning between Scripted and Spontaneous Speech for Emotion Recognition
Li, Q., & Chaspari, T. (2019) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3340555.3353762 · Cited by 8 · see citing works
Towards Noise Robust Speech Emotion Recognition Using Dynamic Layer Customization
Wilf, A., & Provost, E. M. (2021) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii52823.2021.9597437 · Cited by 8 · see citing works
Yu, X. L., Xia, B., Yang, S., Yin, H., Wang, Y., & Liu, X. (2022) · Journal of Sensors · doi:10.1155/2022/8959185 · Cited by 8 · see citing works
Exploring the Intersection Between Speaker Verification and Emotion Recognition
Bancroft, M., Lotfian, R., Hansen, J. H. L., & Busso, C. (2019) · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw.2019.8925044 · Cited by 8 · see citing works
Self-citationAdversarial Multi-Task Learning for Speaker Normalization in Replay Detection
Suthokumar, G., Sethu, V., Sriskandaraja, K., & Eliathamby Ambikairajah, A. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054322 · Cited by 8 · see citing works
Layer-Adapted Implicit Distribution Alignment Networks for Cross-Corpus Speech Emotion Recognition
Zhao, Y. D., Zong, Y., Wang, J., Lian, H., Lu, C., Zhao, L., & Zheng, W. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3362690 · Cited by 7 · see citing works
- Authors
- Abdelwahab, M., & Busso, C.
- Affiliations
- The University of Texas at Dallas (US)
- Venue
- IEEE/ACM Transactions on Audio, Speech, and Language Processing
- Published
- 2018-12
- Volume
- 26
- Issue
- 12
- Pages
- 2423-2435
- ISSN
- 2329-9290 (ISSN-L)
- 2329-9304
- OpenAlex
- W2798632413
- Cited by
- 178
- References
- 51
- Citations by year
- 2026: 4 · 2025: 16 · 2024: 20 · 2023: 22 · 2022: 22 · 2021: 40 · 2020: 29 · 2019: 23 · 2018: 2
- FWCI
- 30.78
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE/ACM Transactions on Audio Speech and Language Processing (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University) (public-domain)
- Funders
- National Science Foundation (IIS-1453781)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Keywords
- acoustic emotion recognition, domain adaptation, domain adversarial learning, gradient reversal layer, unlabeled data, shared representation learning, multi-task learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
