The Geneva Minimalistic Acoustic Parameter Set (GeMAPS) for Voice Research and Affective Computing
Open accessAbstract
Work on voice sciences over recent decades has led to a proliferation of acoustic parameters that are used quite selectively and are not always extracted in a similar fashion. With many independent teams working in different research areas, shared standards become an essential safeguard to ensure compliance with state-of-the-art methods allowing appropriate comparison of results across studies and potential integration and combination of extraction and recognition systems. In this paper we propose a basic standard acoustic parameter set for various areas of automatic voice analysis, such as paralinguistic or clinical speech analysis. In contrast to a large brute-force parameter set, we present a minimalistic set of voice parameters here. These were selected based on a) their potential to index affective physiological changes in voice production, b) their proven value in former studies as well as their automatic extractability, and c) their theoretical significance. The set is intended to provide a common baseline for evaluation of future research and eliminate differences caused by varying parameter sets or even different implementations of the same parameters. Our implementation is publicly available with the openSMILE toolkit. Comparative evaluations of the proposed feature set and large baseline feature sets of INTERSPEECH challenges show a high performance of the proposed set in relation to its size.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Our model should capture certain acoustic and prosodic features such as mean fundamental frequency (F0), mean speech intensity, loudness, and pitch range.
“Some of these features, such as the mean of the fundamental frequency (F0), mean speech intensity, loudness, as well as pitch range [46], should thus be captured by our model.”
From End-to-End Multimodal Emotion Recognition using Deep Neural Networks (2017) · cites this paper as [46]
Short-term spectral, prosodic, and energy features of speech carry affective information.
“Research has shown that short-term spectral, prosodic, and energy features of speech carry affective information [121].”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [121]
Since there is no consensus in the literature over a 'universal' handcrafted feature set with superior performance, many recent studies have applied a 'bruteforce' approach, resulting in a large number of features per utterance.
“Since there is no consensus in the literature over a 'universal' handcrafted feature set with superior performance [121], many recent studies have applied a 'bruteforce' approach, resulting in a large number of features per utterance.”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [121]
Cites
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750 · see citing works
Praat, a system for doing phonetics by computer
Boersma, P. (2002) · UvA-DARE (University of Amsterdam) · Cited by 4,417 · see citing works
Linear prediction: A tutorial review
Makhoul, J. (1975) · Proceedings of the IEEE · doi:10.1109/proc.1975.9792 · Cited by 4,047 · see citing works
Perceptual linear predictive (PLP) analysis of speech
Heřmanský, H. (1990) · The Journal of the Acoustical Society of America · doi:10.1121/1.399423 · Cited by 2,550 · see citing works
Psychoacoustics: facts and models
Fastl, H., & Zwicker, E. (1991) · Choice Reviews Online · doi:10.5860/choice.28-5706 · Cited by 2,413 · see citing works
A database of German emotional speech
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W. F., & Weiss, B. (2005) · Interspeech 2005 · doi:10.21437/interspeech.2005-446 · Cited by 2,365 · see citing works
Acoustic profiles in vocal emotion expression.
Banse, R., & Scherer, K. R. (1996) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.70.3.614 · Cited by 2,253 · see citing works
Self-citationVocal communication of emotion: A review of research paradigms
Scherer, K. (2003) · Speech Communication · doi:10.1016/s0167-6393(02)00084-5 · Cited by 2,044 · see citing works
Self-citationCommunication of emotions in vocal expression and music performance: Different channels, same code?
Juslin, P. N., & Laukka, P. (2003) · Psychological Bulletin · doi:10.1037/0033-2909.129.5.770 · Cited by 1,826 · see citing works
Self-citationVocal affect expression: A review and a model for future research.
Scherer, K. R. (1986) · Psychological Bulletin · doi:10.1037/0033-2909.99.2.143 · Cited by 1,768 · see citing works
Self-citationRecent developments in openSMILE, the munich open-source multimedia feature extractor
Eyben, F., Weninger, F. J., Gross, F., & Schuller, B. W. (2013) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2502081.2502224 · Cited by 1,265 · see citing works
Self-citationSchlosberg, H. (1954) · Psychological Review · doi:10.1037/h0054570 · Cited by 1,072 · see citing works
The INTERSPEECH 2009 emotion challenge
Schuller, B., Steidl, S., & Batliner, A. (2009) · Interspeech 2009 · doi:10.21437/interspeech.2009-103 · Cited by 938 · see citing works
Self-citationEmotional speech recognition: Resources, features, and methods
Ververidis, D., & Kotropoulos, C. L. (2006) · Speech Communication · doi:10.1016/j.specom.2006.04.003 · Cited by 927 · see citing works
Schuller, B. W., Steidl, S., Batliner, A. M., Vinciarelli, A., Scherer, K. R., Ringeval, F., Chétouani, M., Weninger, F. J., Eyben, F., Marchi, E., et al. (2013) · Interspeech (USB) · doi:10.21437/interspeech.2013-56 · Cited by 732 · see citing works
Self-citationZwicker, E., & Fastl, H. (1999) · Springer series in information sciences · doi:10.1007/978-3-662-09562-1 · Cited by 609 · see citing works
The INTERSPEECH 2010 paralinguistic challenge
Schuller, B. W., Steidl, S., Batliner, A. M., Burkhardt, F., Devillers, L. Y., Müller, C. A., & Narayanan, S. S. (2010) · Interspeech (USB) · doi:10.21437/interspeech.2010-739 · Cited by 577 · see citing works
Self-citationIntroducing the Geneva Multimodal expression corpus for experimental research on emotion perception.
Bänziger, T., Mortillaro, M., & Scherer, K. R. (2011) · Emotion · doi:10.1037/a0025827 · Cited by 437 · see citing works
Self-citationThe Vera am Mittag German audio-visual emotional speech database
Grimm, M., Kroschel, K., & Narayanan, S. S. (2008) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2008.4607572 · Cited by 413 · see citing works
Self-citationCross-Corpus Acoustic Emotion Recognition: Variances and Strategies
Schuller, B. W., Vlasenko, B., Eyben, F., Wöllmer, M., Stuhlsatz, A., Wendemuth, A., & Rigoll, G. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.8 · Cited by 386 · see citing works
Self-citationPerceptual Cues in Nonverbal Vocal Expressions of Emotion
Sauter, D. A., Eisner, F., Calder, A. J., & Scott, S. K. (2010) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470211003721642 · Cited by 358 · see citing works
Perceptual and Acoustic Correlates of Abnormal Voice Qualities
Hammarberg, B., Fritzell, B., Gauffin, J., Sundberg, J., & Wedin, L. (1980) · Acta Oto-Laryngologica · doi:10.3109/00016488009131746 · Cited by 350 · see citing works
Self-citationMeasurement of pitch by subharmonic summation
Hermes, D. J. (1988) · The Journal of the Acoustical Society of America · doi:10.1121/1.396427 · Cited by 313 · see citing works
Juslin, P. N., & Laukka, P. (2001) · Emotion · doi:10.1037/1528-3542.1.4.381 · Cited by 300 · see citing works
Self-citationPrimitives-based evaluation and estimation of emotions in speech
Grimm, M., Kroschel, K., Mower, E., & Narayanan, S. S. (2007) · Speech Communication · doi:10.1016/j.specom.2007.01.010 · Cited by 291 · see citing works
Self-citationAnalysis of Emotionally Salient Aspects of Fundamental Frequency for Emotion Detection
Busso, C., Lee, S., & Narayanan, S. S. (2009) · IEEE Transactions on Audio Speech and Language Processing · doi:10.1109/tasl.2008.2009578 · Cited by 278 · see citing works
Self-citationOn the Acoustics of Emotion in Audio: What Speech, Music, and Sound have in Common
Weninger, F. J., Eyben, F., Schuller, B. W., Mortillaro, M., & Scherer, K. R. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00292 · Cited by 277 · see citing works
Self-citationAcoustic emotion recognition: A benchmark comparison of performances
Schuller, B. W., Vlasenko, B., Eyben, F., Rigoll, G., & Wendemuth, A. (2009) · IEEE Workshop on Automatic Speech Recognition and Understanding (ASRU) · doi:10.1109/asru.2009.5372886 · Cited by 275 · see citing works
Self-citationThe INTERSPEECH 2011 speaker state challenge
Schuller, B. W., Steidl, S., Batliner, A. M., Schiel, F., & Krajewski, J. (2011) · Interspeech (USB) · doi:10.21437/interspeech.2011-801 · Cited by 225 · see citing works
Self-citationMoore, E., Clements, M. A., Peifer, J. W., & Weisser, L. (2007) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2007.900562 · Cited by 220 · see citing works
The INTERSPEECH 2012 speaker trait challenge
Schuller, B., Steidl, S., Batliner, A., Nöth, E., Vinciarelli, A., Burkhardt, F., Son, R. V., Weninger, F., Eyben, F., Bocklet, T., et al. (2012) · Interspeech 2012 · doi:10.21437/interspeech.2012-86 · Cited by 211 · see citing works
Self-citationBeyond arousal: Valence and potency/control cues in the vocal expression of emotion
Goudbeek, M. B., & Scherer, K. R. (2010) · The Journal of the Acoustical Society of America · doi:10.1121/1.3466853 · Cited by 201 · see citing works
Self-citationSchuller, B. W., Müller, R., Eyben, F., Gast, J., Hörnler, B., Wöllmer, M., Rigoll, G., Hoethker, A., & Konosu, H. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2009.02.013 · Cited by 194 · see citing works
Self-citationAcoustical Correlates of Affective Prosody
Hammerschmidt, K., & Jürgens, U. (2006) · Journal of Voice · doi:10.1016/j.jvoice.2006.03.002 · Cited by 193 · see citing works
Schuller, B. W., Batliner, A. M., Seppi, D., Steidl, S., Vogt, T., Wagner, J., Devillers, L. Y., Vidrascu, L., Amir, N., Kessous, L., et al. (2007) · Interspeech (USB) · doi:10.21437/interspeech.2007-612 · Cited by 191 · see citing works
Self-citationVocal biomarkers of depression based on motor incoordination
Williamson, J. R., Quatieri, T. F., Helfer, B. S., Horwitz-Martin, R. L., Yu, B., & Mehta, D. D. (2013) · doi:10.1145/2512530.2512531 · Cited by 190 · see citing works
An investigation of depressed speech detection: features and normalization
Cummins, N., Epps, J., Breakspear, M. J., & Goecke, R. (2011) · Interspeech (USB) · doi:10.21437/interspeech.2011-750 · Cited by 188 · see citing works
Self-citationMapping emotions into acoustic space: the role of voice production.
Patel, S., Scherer, K. R., Björkner, E., & Sundberg, J. (2011) · Biological Psychology · doi:10.1016/j.biopsycho.2011.02.010 · Cited by 177 · see citing works
Self-citationAn acoustic study of emotions expressed in speech
Yıldırım, S., Bulut, M., Lee, C. M., Kazemzadeh, A., Deng, Z., Lee, S., Narayanan, S. S., & Busso, C. (2004) · Interspeech (USB) · doi:10.21437/interspeech.2004-242 · Cited by 172 · see citing works
Self-citationCombining Efforts for Improving Automatic Classification of Emotional User States
Batliner, A., Steidl, S., Schuller, B., Seppi, D., Laskowski, K., Vogt, T., Devillers, L., Vidrascu, L., Amir, N., Kessous, L., et al. (2006) · Jožef Stefan Institute, Ljubljana · doi:10.5281/zenodo.14167504 · Cited by 119 · see citing works
Self-citationLa vive voix: essais de psycho-phonétique
(1986) · Journal of Pragmatics · doi:10.1016/0378-2166(86)90080-9 · Cited by 107 · see citing works
Interdependencies among Voice Source Parameters in Emotional Speech
Sundberg, J., Patel, S. A., Björkner, E., & Scherer, K. R. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.14 · Cited by 90 · see citing works
Self-citationPhonologically-based biomarkers for major depressive disorder
Trevino, A. C., Quatieri, T. F., & Malyska, N. (2011) · EURASIP Journal on Advances in Signal Processing · doi:10.1186/1687-6180-2011-42 · Cited by 77 · see citing works
Investigation of spectral centroid features for cognitive load classification
Le, P. N., Eliathamby Ambikairajah, A., Epps, J., Sethu, V., & Choi, E. H. C. (2011) · Speech Communication · doi:10.1016/j.specom.2011.01.005 · Cited by 74 · see citing works
Self-citationComparing the acoustic expression of emotion in the speaking and the singing voice
Scherer, K. R., Sundberg, J., Tamarit, L., & Salomão, G. L. (2013) · Computer Speech & Language · doi:10.1016/j.csl.2013.10.002 · Cited by 73 · see citing works
Self-citationSchuller, B. W. (2013) · Signals and communication technology · doi:10.1007/978-3-642-36806-6 · Cited by 73 · see citing works
Self-citationRobust Unsupervised Arousal Rating:A Rule-Based Framework withKnowledge-Inspired Vocal Features
Bone, D., Lee, C.-C., & Narayanan, S. S. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2326393 · Cited by 67 · see citing works
Self-citationBrute-forcing hierarchical functionals for paralinguistics: A waste of feature space?
Schuller, B. W., Wimmer, M., Mösenlechner, L., Kern, C., Arsić, D., & Rigoll, G. (2008) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2008.4518656 · Cited by 60 · see citing works
Self-citationSpeaker Normalisation for Speech-Based Emotion Detection
Sethu, V., Ambikairajah, E., & Epps, J. (2007) · doi:10.1109/icdsp.2007.4288656 · Cited by 54 · see citing works
Self-citationLPCW: An LPC vocoder with linear predictive spectral warping
Makhoul, J., & Cosell, L. (2005) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1976.1170013 · Cited by 51 · see citing works
Emotion Appraisal Dimensions can be Inferred From Vocal Expressions
Laukka, P., & Elfenbein, H. A. (2011) · Social Psychological and Personality Science · doi:10.1177/1948550611428011 · Cited by 42 · see citing works
Self-citationA toolkit for voice inverse filtering and parametrisation
Airas, M., Pulakka, H., Bäckström, T., et al. (2005) · Interspeech (USB) · doi:10.21437/interspeech.2005-434 · Cited by 41 · see citing works
Recognising interest in conversational speech - comparing bag of frames and supra-segmental features
Schuller, B. W., & Rigoll, G. (2009) · Interspeech (USB) · doi:10.21437/interspeech.2009-484 · Cited by 38 · see citing works
Self-citationFónagy, I. (1983) · Cited by 38 · see citing works
Affect recognition in real-life acoustic conditions — a new perspective on feature selection
Eyben, F., Weninger, F. J., & Schuller, B. W. (2013) · Interspeech (USB) · doi:10.21437/interspeech.2013-484 · Cited by 36 · see citing works
Self-citationFormant Frequencies under Cognitive Load: Effects and Classification
Yap, T. F., Epps, J., Eliathamby Ambikairajah, A., & Choi, E. H. C. (2010) · EURASIP Journal on Advances in Signal Processing · doi:10.1155/2011/219253 · Cited by 32 · see citing works
Self-citationTowards a standard set of acoustic features for the processing of emotion in speech.
Eyben, F., Batliner, A. M., & Schuller, B. W. (2010) · Proceedings of meetings on acoustics · doi:10.1121/1.4739483 · Cited by 21 · see citing works
Self-citationVoice source features for cognitive load classification
Yap, T. F., Epps, J., Eliathamby Ambikairajah, A., & Choi, E. H. C. (2011) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2011.5947654 · Cited by 21 · see citing works
Self-citationMarchi, E., Batliner, A. M., Schuller, B. W., Fridenzon, S., Tal, S., & Golan, O. (2012) · International Conference on Social Computing · doi:10.1109/socialcom-passat.2012.97 · Cited by 20 · see citing works
Self-citationAcoustic markers of emotions based on voice physiology
Patel, S. A., Scherer, K. R., Sundberg, J., & Björkner, E. (2010) · Speech prosody · doi:10.21437/speechprosody.2010-239 · Cited by 14 · see citing works
Self-citationAcoustic measures characterizing anger across corpora collected in artificial or natural context
Tahon, M., & Devillers, L. Y. (2010) · Speech prosody · doi:10.21437/speechprosody.2010-134 · Cited by 14 · see citing works
Self-citationSchröder, Burkhardt, & Krstulovic (2010) · Cited by 11 · see citing works
On the use of speech parameter contours for emotion recognition
Sethu, V., Eliathamby Ambikairajah, A., & Epps, J. (2013) · EURASIP Journal on Audio Speech and Music Processing · doi:10.1186/1687-4722-2013-19 · Cited by 11 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Adieu features? End-to-end speech emotion recognition using a deep convolutional recurrent network
Trigeorgis, G., Ringeval, F., Brueckner, R., Marchi, E., Nicolaou, M. A., Schuller, B., & Zafeiriou, S. (2016) · 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2016.7472669 · Cited by 904 · see citing works
Self-citationEmotion recognition using multi-modal data and machine learning techniques: A tutorial and review
Zhang, J., Yin, Z., Chen, P., & Nichele, S. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.01.011 · Cited by 806 · see citing works
End-to-End Multimodal Emotion Recognition using Deep Neural Networks
Tzirakis, P., Trigeorgis, G., Nicolaou, M. A., Schuller, B., & Zafeiriou, S. (2017) · arXiv · doi:10.48550/arxiv.1704.08619 · Cited by 752 · see citing works
Self-citationEvaluating deep learning architectures for Speech Emotion Recognition
Fayek, H. M., Lech, M., & Cavedon, L. (2017) · Neural Networks · doi:10.1016/j.neunet.2017.02.013 · Cited by 554 · see citing works
Ringeval, F., Schuller, B., Valstar, M., Cummins, N., Cowie, R., Tavabi, L., Schmitt, M., Alisamir, S., Amiriparian, S., Messner, E.-M., et al. (2019) · Proceedings of the 9th International on Audio/Visual Emotion Challenge and Workshop · doi:10.1145/3347320.3357688 · Cited by 409 · see citing works
Self-citationEmotion Recognition from Speech Using wav2vec 2.0 Embeddings
Pepino, L., Riera, P., & Ferrer, L. (2021) · Interspeech 2021 · doi:10.21437/interspeech.2021-703 · Cited by 389 · see citing works
Alzheimer’s Dementia Recognition Through Spontaneous Speech: The ADReSS Challenge
Luz, S., Haider, F., Fuente, S. D. L., Fromm, D., & MacWhinney, B. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-2571 · Cited by 274 · see citing works
Multimodal assessment of Parkinson’s disease: a deep learning approach
Vasquez-Correa, J. C., Arias-Vergara, T., Orozco-Arroyave, J. R., Eskofier, B., Klucken, J., & Noth, E. (2019) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2018.2866873 · Cited by 262 · see citing works
Sewa db: A rich database for audio-visual emotion and sentiment research in the wild
Kossaifi, J., Walecki, R., Panagakis, Y., Shen, J., Schmitt, M., Ringeval, F., Han, J., Pandit, V., Toisoul, A., Schuller, B., et al. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2944808 · Cited by 243 · see citing works
Self-citationDeep learning for human affect recognition: Insights and new developments
Rouast, P. V., Adam, M. T. P., & Chiong, R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2890471 · Cited by 243 · see citing works
Audio-visual emotion fusion (AVEF): A deep efficient weighted approach
Ma, Y., Hao, Y., Chen, M., Chen, J., Lu, P., & Košir, A. (2019) · Information Fusion · doi:10.1016/j.inffus.2018.06.003 · Cited by 155 · see citing works
Predicting dementia from spontaneous speech using large language models
Agbavor, F., & Liang, H. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000168 · Cited by 147 · see citing works
Progressive Neural Networks for Transfer Learning in Emotion Recognition
Gideon, J., Khorram, S., Aldeneh, Z., Dimitriadis, D., & Provost, E. M. (2017) · Interspeech 2017 · doi:10.21437/interspeech.2017-1637 · Cited by 129 · see citing works
Sun, L., Lian, Z., Tao, J., Liu, B., & Niu, M. (2020) · Proceedings of the 1st International on Multimodal Sentiment Analysis in Real-life Media Challenge and Workshop · doi:10.1145/3423327.3423672 · Cited by 81 · see citing works
Hecker, P., Steckhan, N., Eyben, F., Schuller, B. W., & Arnrich, B. (2022) · Frontiers in Digital Health · doi:10.3389/fdgth.2022.842301 · Cited by 78 · see citing works
Self-citationDo Deepfakes Feel Emotions? A Semantic Approach to Detecting Deepfakes Via Emotional Inconsistencies
Hosler, B., Salvi, D., Murray, A., Antonacci, F., Bestagini, P., Tubaro, S., & Stamm, M. C. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00112 · Cited by 78 · see citing works
Pre-trained models for detection and severity level classification of dysarthria from speech
Javanmardi, F., Kadiri, S. R., & Alku, P. (2024) · Speech Communication · doi:10.1016/j.specom.2024.103047 · Cited by 74 · see citing works
Zhang, L., Duvvuri, R., Chandra, K. K. L., Nguyen, T., & Ghomi, R. H. (2020) · Depression and Anxiety · doi:10.1002/da.23020 · Cited by 67 · see citing works
Shahin, M., Zafar, U., & Ahmed, B. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2019.2959393 · Cited by 62 · see citing works
Make Acoustic and Visual Cues Matter: CH-SIMS v2.0 Dataset and AV-Mixup Consistent Module
Liu, Y., Yuan, Z., Mao, H., Liang, Z., Yang, W., Qiu, Y., Cheng, T., Li, X., Xu, H., & Gao, K. (2022) · Proceedings of the 2022 International Conference on Multimodal Interaction · doi:10.1145/3536221.3556630 · Cited by 64 · see citing works
Automatic Assessment of Speech Intelligibility for Individuals with Aphasia
Le, D., Licata, K., Persad, C., & Provost, E. M. (2016) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2016.2598428 · Cited by 59 · see citing works
DNN-based Emotion Recognition Based on Bottleneck Acoustic Features and Lexical Features
Kim, E., & Shin, J. W. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683077 · Cited by 54 · see citing works
Martinc, M., Haider, F., Pollak, S., & Luz, S. (2021) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2021.642647 · Cited by 55 · see citing works
“The eGeMAPS feature set corresponds to a basic set of acoustic features based on their potential to detect physiological changes in voice production, as well as theoretical significance and proven usefulness in previous studies (Eyben et al., 2016).”
BackgroundMethodContinuous Multimodal Emotion Prediction Based on Long Short Term Memory Recurrent Neural Network
Huang, J., Li, Y., Tao, J., Lian, Z., Wen, Z., Yang, M., & Yi, J. (2017) · Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge · doi:10.1145/3133944.3133946 · Cited by 54 · see citing works
Audio and face video emotion recognition in the wild using deep neural networks and small datasets
Ding, W., Xu, M., Huang, D., Lin, W., Dong, M., Yu, X., & Li, H. (2016) · Proceedings of the 18th ACM International Conference on Multimodal Interaction · doi:10.1145/2993148.2997637 · Cited by 51 · see citing works
Huang, Z., Epps, J., Joachim, D., & Chen, M. (2018) · Interspeech 2018 · doi:10.21437/interspeech.2018-1743 · Cited by 52 · see citing works
Self-citationExpressGesture: Expressive gesture generation from speech through database matching
Ferstl, Y., Neff, M., & McDonnell, R. (2021) · Computer Animation and Virtual Worlds · doi:10.1002/cav.2016 · Cited by 52 · see citing works
Agbavor, F., & Liang, H. (2022) · Brain Sciences · doi:10.3390/brainsci13010028 · Cited by 47 · see citing works
Multimodal Continuous Emotion Recognition with Data Augmentation Using Recurrent Neural Networks
Huang, J., Li, Y., Tao, J., Lian, Z., Niu, M., & Yang, M. (2018) · Proceedings of the 2018 on Audio/Visual Emotion Challenge and Workshop · doi:10.1145/3266302.3266304 · Cited by 40 · see citing works
Multimodal Emotion Recognition and Sentiment Analysis via Attention Enhanced Recurrent Model
Sun, L., Xu, M., Lian, Z., Liu, B., Tao, J., Wang, M., & Cheng, Y. (2021) · Proceedings of the 2nd on Multimodal Sentiment Analysis Challenge · doi:10.1145/3475957.3484456 · Cited by 36 · see citing works
Feng, T., & Narayanan, S. (2023) · 2023 11th International Conference on Affective Computing and Intelligent Interaction (ACII) · doi:10.1109/acii59096.2023.10388152 · Cited by 34 · see citing works
Self-citationCosic, K., Popovic, S., Sarlija, M., Mijic, I., Kokot, M., Kesedzic, I., Strangman, G., Ivkovic, V., & Zhang, Q. (2019) · IEEE Access · doi:10.1109/access.2019.2957357 · Cited by 29 · see citing works
Beccaluva, E. A., Catania, F., Arosio, F., & Garzotto, F. (2024) · Human–Computer Interaction · doi:10.1080/07370024.2023.2242837 · Cited by 24 · see citing works
An explainable machine learning model of cognitive decline derived from speech
Chandler, C., Diaz‐Asper, C., Turner, R. S., Reynolds, B., & Elvevåg, B. (2023) · Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring · doi:10.1002/dad2.12516 · Cited by 24 · see citing works
Chou, H.-C., Goncalves, L., Leem, S.-G., Salman, A. N., Lee, C.-C., & Busso, C. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3411290 · Cited by 19 · see citing works
Self-citationWang, W., Xu, W., Chander, A., Nepal, S., Buck, B., Pakhomov, S., Cohen, T., Ben-Zeev, D., & Campbell, A. (2023) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3610890 · Cited by 19 · see citing works
Adaptive Domain-Aware Representation Learning for Speech Emotion Recognition
Fan, W., Xu, X., Xing, X., & Huang, D. (2020) · Interspeech 2020 · doi:10.21437/interspeech.2020-2572 · Cited by 14 · see citing works
Low, D. M., Rao, V., Randolph, G., Song, P. C., & Ghosh, S. S. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000516 · Cited by 16 · see citing works
(2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-72038-8 · Cited by 11 · see citing works
An Improved Multimodal Dimension Emotion Recognition Based on Different Fusion Methods
Su, H., Liu, B., Tao, J., Dong, Y., Huang, J., Lian, Z., & Song, L. (2020) · 2020 15th IEEE International Conference on Signal Processing (ICSP) · doi:10.1109/icsp48669.2020.9321008 · Cited by 10 · see citing works
Kim, Y. E., Dobko, M., Li, H., Shao, T., Periyakoil, P., Tipton, C., Colasacco, C., Serpedin, A., Elemento, O., Sabuncu, M., et al. (2025) · The Laryngoscope · doi:10.1002/lary.32036 · Cited by 9 · see citing works
Sawhney, R., Goyal, M., Goel, P., Mathur, P., & Shah, R. R. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.526 · Cited by 8 · see citing works
de la Fuente Garcia, S., Ritchie, C. W., & Luz, S. (2020) · Journal of Alzheimer's Disease · doi:10.3233/jad-200888 · Cited by 5 · see citing works
Dadgar, M., Ennis, C., Mokgosi, K., & Ross, R. (2025) · DIGITAL HEALTH · doi:10.1177/20552076251376533 · Cited by 4 · see citing works
“…Shimmer, Loudness, Spectral Energies, fi rst four Mel-Frequency Cepstral Coef fi cients (MFCCs), and mean and standard deviation of voiced/unvoiced regions, respectively, etc. 51 While paper C also using eGeMAPS, they mention that computational paralinguistic challenge (ComParE) feature set —”
Federated learning for cognitive impairment detection using speech data
Blazquez-Folch, J., Limones Andrade, M., Calm, B., Auñón García, J. M., Alegret, M., Muñoz, N., Cano, A., Fernández, V., García-Gutiérrez, F., De Rojas, I., et al. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1662859 · Cited by 2 · see citing works
- Authors
- Eyben, F., Scherer, K. R., Schuller, B. W., Sundberg, J., Andre, E., Busso, C., Devillers, L. Y., Epps, J., Laukka, P., Narayanan, S. S., & Truong, K. P.
- Affiliations
- University of Geneva (CH)
- KTH Royal Institute of Technology (SE)
- The University of Texas at Dallas (US)
- Centre National de la Recherche Scientifique (FR)
- UNSW Sydney (AU)
- Stockholm University (SE)
- University of Southern California (US)
- University of Twente (NL)
- ORCID
- Venue
- IEEE Transactions on Affective Computing
- Published
- 2016-04-01
- Volume
- 7
- Issue
- 2
- Pages
- 190-202
- ISSN
- 1949-3045 (ISSN-L)
- OpenAlex
- W2239141610
- Cited by
- 1,818
- References
- 63
- Citations by year
- 2026: 229 · 2025: 235 · 2024: 202 · 2023: 218 · 2022: 214 · 2021: 192 · 2020: 161 · 2019: 149 · 2018: 99 · 2017: 74 · 2016: 32 · 2015: 12
- FWCI
- 108.94
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- IEEE Transactions on Affective Computing (publishedVersion, cc-by)
- University of Twente Research Information (publishedVersion, other-oa)
- mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) (submittedVersion, other-oa)
- OPUS (Augsburg University) (submittedVersion, cc-by)
- University of Twente Research Information (submittedVersion, other-oa)
- Funders
- European Community's seventh Framework Programme (230331-PROPEREMO)
- National Center of Competence in Research (NCCR) Affective Sciences
- Swiss National Science Foundation (51NF40-104897)
- University of Geneva
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Speech Recognition and Synthesis
- Health Sciences › Medicine › Physiology › Voice and Speech Disorders
- Keywords
- voice analysis, affective computing, paralinguistic features, openSMILE, feature standardization, automatic feature extraction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
