The DOI matches its Crossref record; 80 of the 80 references it cites were found in a registry or an index.
Emotion Recognition from Physiological Signal Analysis: A Review
Open accessAbstract
Human computer interaction is increasingly utilized in smart home, industry 4.0 and personal health. Communication between human and computer can benefit by a flawless exchange of emotions. As emotions have substantial influence on cognitive processes of the human brain such as learning, memory, perception and problem solving, emotional interactions benefit different applications. It can further be relevant in modern health care especially in interaction with patients suffering from stress or depression. Additionally rehabilitation applications, guiding patients through their rehabilitation training while adapting to the patients emotional state, would be highly motivating and might lead to a faster recovery. Depending on the application area, different systems for emotion recognition suit different purposes. The aim of this work is to give an overview of methods to recognize emotions and to compare their applicability based on existing studies. This review paper should enable practitioners, researchers and engineers to find a system most suitable for certain applications. An entirely contact-less method is to analyze facial features with the help of a video camera. This is useful when computers, smart-phones or tablets with integrated cameras are included in the task. Smart wearables provide contact with the skin and physiological parameters such as electro-dermal activity and heart related signals can be recorded unobtrusively also during dynamical tasks. Next to unimodal solutions, multimodal affective computing systems are analyzed since they promise higher classification accuracy. Accuracy varies based on the amount of detected emotions, extracted features, classification method and the quality of the database. Electroencephalography achieves 88.86 % accuracy for four emotions, multimodal measurements (Electrocardiography, Electromyography and bio-signals) 79.3 % for four emotive states, facial recognition 89 % for seven states and speech recognition 80.46 % for happiness and sadness. Looking forward, heart-related parameters might be an option to measure emotions accurately and unobtrusive with the help of smart wearables. This can be used in dynamic or outdoor tasks. Facial recognition on the other hand is a useful contact-less tool when it comes to emotion recognition during computer interaction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ANFIS: adaptive-network-based fuzzy inference system
Jang, J.-S. R. (1993) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/21.256541 · Cited by 16,254
The expression of the emotions in man and animals.
Darwin, C. (1872) · J. Murray eBooks · doi:10.1037/10001-000 · Cited by 11,657
Measuring emotion: The self-assessment manikin and the semantic differential
Bradley, M. M., & Lang, P. J. (1994) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/0005-7916(94)90063-9 · Cited by 9,538
Constants across cultures in the face and emotion.
Ekman, P., & Friesen, W. V. (1971) · Journal of Personality and Social Psychology · doi:10.1037/h0030377 · Cited by 5,509
(2005) · CTIT technical reports series · Cited by 5,307
(1997) · Choice Reviews Online · doi:10.5860/choice.34-3702 · Cited by 4,887
Ekman, P., & Friesen, W. V. (1975) · Cited by 4,604
JAMES, W. (1884) · Mind · doi:10.1093/mind/os-ix.34.188 · Cited by 3,446
Emotion recognition in human-computer interaction
Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., & Taylor, J. G. (2001) · IEEE Signal Processing Magazine · doi:10.1109/79.911197 · Cited by 2,603
POSNER, J., RUSSELL, J. A., & PETERSON, B. S. (2005) · Development and Psychopathology · doi:10.1017/s0954579405050340 · Cited by 2,602
Toward machine emotional intelligence: analysis of affective physiological state
Picard, R. W., Vyzas, E., & Healey, J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.954607 · Cited by 2,334
Universals and cultural differences in facial expressions of emotion.
Ekman, P. (1972) · Medical Entomology and Zoology · Cited by 2,273
James, W. (2005) · The emotions, Vol. 1. · doi:10.1037/10735-001 · Cited by 2,206
Survey on speech emotion recognition: Features, classification schemes, and databases
El Ayadi, M., Kamel, M. S., & Karray, F. (2011) · Pattern Recognition · doi:10.1016/j.patcog.2010.09.020 · Cited by 2,073
Joint action: bodies and minds moving together
SEBANZ, N., BEKKERING, H., & KNOBLICH, G. (2006) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2005.12.009 · Cited by 1,950
A continuous measure of phasic electrodermal activity
Benedek, M., & Kaernbach, C. (2010) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2010.04.028 · Cited by 1,671
Thoughts on the relations between emotion and cognition.
Lazarus, R. S. (1982) · American Psychologist · doi:10.1037//0003-066x.37.9.1019 · Cited by 1,618
Acoustic profiles in vocal emotion expression
Banse, R., & Scherer, K. R. (1996) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.70.3.614 · Cited by 1,546
Plutchik, R. (2001) · American Scientist · doi:10.1511/2001.4.344 · Cited by 1,419
Emotion and sociable humanoid robots
Breazeal, C. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00018-1 · Cited by 1,279
Emotion recognition based on physiological changes in music listening
Kim, J., & Andre, E. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.26 · Cited by 1,082
Facial signs of emotional experience.
Ekman, P., Freisen, W. V., & Ancoli, S. (1980) · Journal of Personality and Social Psychology · doi:10.1037/h0077722 · Cited by 886
Analysis of emotion recognition using facial expressions, speech and multimodal information
Busso, C., Deng, Z., Yildirim, S., Bulut, M., Lee, C. M., Kazemzadeh, A., Lee, S., Neumann, U., & Narayanan, S. (2004) · Proceedings of the 6th international conference on Multimodal interfaces · doi:10.1145/1027933.1027968 · Cited by 840
Toward an affect-sensitive multimodal human-computer interaction
Pantic, M., & Rothkrantz, L. J. M. (2003) · Proceedings of the IEEE · doi:10.1109/jproc.2003.817122 · Cited by 791
Emotions and speech: some acoustical correlates.
Williams, C. E., & Stevens, K. N. (1972) · The Journal of the Acoustical Society of America · doi:10.1121/1.1913238 · Cited by 735
Acoustical properties of speech as indicators of depression and suicidal risk
France, D. J., Shiavi, R. G., Silverman, S., Silverman, M., & Wilkes, M. (2000) · IEEE Transactions on Biomedical Engineering · doi:10.1109/10.846676 · Cited by 504
Ecg pattern analysis for emotion detection
Agrafioti, F., Hatzinakos, D., & Anderson, A. K. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.28 · Cited by 476
Emotion Recognition Using Bio-sensors: First Steps towards an Automatic System
Haag, A., Goronzy, S., Schaich, P., & Williams, J. (2004) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-24842-2_4 · Cited by 451
Schuller, B., Rigoll, G., & Lang, M. (2004) · 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing · doi:10.1109/icassp.2004.1326051 · Cited by 426
Emotion recognition in human–computer interaction
Fragopanagos, N., & Taylor, J. G. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.03.006 · Cited by 414
Emotion recognition from physiological signals using wireless sensors for presence technologies
Nasoz, F., Alvarez, K., Lisetti, C. L., & Finkelstein, N. (2004) · Cognition, Technology & Work · doi:10.1007/s10111-003-0143-x · Cited by 367
Emotion Recognition in Speech Using Neural Networks
Nicholson, J., Takahashi, K., & Nakatsu, R. (2000) · Neural Computing & Applications · doi:10.1007/s005210070006 · Cited by 344
Physiological signals based human emotion Recognition: a review
Jerritta, S., Murugappan, M., Nagarajan, R., & Wan, K. (2011) · 2011 IEEE 7th International Colloquium on Signal Processing and its Applications · doi:10.1109/cspa.2011.5759912 · Cited by 341
Toward Emotion Recognition in Car-Racing Drivers: A Biosignal Processing Approach
Katsis, C. D., Katertsidis, N., Ganiatsas, G., & Fotiadis, D. I. (2008) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/tsmca.2008.918624 · Cited by 336
Nonverbal Behavior and Communication
(2014) · Psychology Press eBooks · doi:10.4324/9781315799759 · Cited by 335
Age differences in recognition of emotion in lexical stimuli and facial expressions.
Isaacowitz, D. M., Löckenhoff, C. E., Lane, R. D., Wright, R., Sechrest, L., Riedel, R., & Costa, P. T. (2007) · Psychology and Aging · doi:10.1037/0882-7974.22.1.147 · Cited by 328
A Stress Sensor Based on Galvanic Skin Response (GSR) Controlled by ZigBee
Villarejo, M. V., Zapirain, B. G., & Zorrilla, A. M. (2012) · Sensors · doi:10.3390/s120506075 · Cited by 315
An empirical study of machine learning techniques for affect recognition in human–robot interaction
Rani, P., Liu, C., Sarkar, N., & Vanman, E. (2006) · Pattern Analysis and Applications · doi:10.1007/s10044-006-0025-y · Cited by 313
AFFDEX SDK: A Cross-Platform Real-Time Multi-Face Expression Recognition Toolkit
McDuff, D., Mahmoud, A., Mavadati, M., Amr, M., Turcot, J., & Kaliouby, R. E. (2016) · Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/2851581.2890247 · Cited by 280
Ingjaldsson, J. T., Laberg, J. C., & Thayer, J. F. (2003) · Biological Psychiatry · doi:10.1016/s0006-3223(02)01926-1 · Cited by 275
Kessous, L., Castellano, G., & Caridakis, G. (2010) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-009-0025-5 · Cited by 256
Principles of physiological psychology, 1873.
Wundt, W. (1948) · Readings in the history of psychology. · doi:10.1037/11304-029 · Cited by 250
Tratt, L. (2008) · Research Portal (King's College London) · doi:10.1007/978-3-540-88643-3-11 · Cited by 250
Affective state estimation for human–robot Interaction
Kulic, D., & Croft, E. A. (2007) · IEEE Transactions on Robotics · doi:10.1109/tro.2007.904899 · Cited by 230
Automated stress detection using keystroke and linguistic features: An exploratory study
Vizer, L. M., Zhou, L., & Sears, A. (2009) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2009.07.005 · Cited by 220
Cannon, W. B. (1987) · PubMed · Cited by 174
Can PPG be used for HRV analysis?
Pinheiro, N. C., Couceiro, R., Henriques, J. M. O., Muehlsteff, J., Quintal, I., Gonçalves, L. M. M., & Carvalho, P. (2016) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2016.7591347 · Cited by 172
Respiration-based emotion recognition with deep learning
Zhang, Q., Chen, X., Zhan, Q., Yang, T., & Xia, S. (2017) · Computers in Industry · doi:10.1016/j.compind.2017.04.005 · Cited by 171
ELECTROACTIVE FABRICS AND WEARABLE BIOMONITORING DEVICES
Rossi, D. D., Carpi, F., Lorussi, F., Mazzoldi, A., Paradiso, R., Pasquale Scilingo, E., & Tognetti, A. (2003) · AUTEX Research Journal · doi:10.1515/aut-2003-030404 · Cited by 160
Kemp, A. H., Quintana, D. S., Kuhnert, R.-L., Griffiths, K., Hickie, I. B., & Guastella, A. J. (2012) · PLoS ONE · doi:10.1371/journal.pone.0044014 · Cited by 158
Musical neurofeedback for treating depression in elderly people
Ramírez, R., Palencia-Lefler, M., Giraldo, S., & Vamvakousis, Z. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00354 · Cited by 147
Readings in the history of psychology.
(1948) · Appleton-Century-Crofts eBooks · doi:10.1037/11304-000 · Cited by 131
Guo, H.-W., Huang, Y.-S., Lin, C., Chien, J.-C., Haraikawa, K., & Shieh, J.-S. (2016) · IEEE International Conference on Bioinformatics and Bioengineering (BIBE) · doi:10.1109/bibe.2016.40 · Cited by 129
Digital processing of affective signals
Healey, J., & Picard, R. W. (2002) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1998.679699 · Cited by 122
The r FaceReader: Online facial expression recognition.
y l, D. U. (2005) · Cited by 115
Emotion Recognition through Physiological Signals for Human-Machine Communication
Maaoui, C., & Pruski, A. (2010) · Cutting Edge Robotics 2010 · doi:10.5772/10312 · Cited by 109
Mehmood, R. M., & Lee, H. J. (2016) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2016.04.009 · Cited by 99
The Analysis of Emotion Recognition from GSR Based on PSO
Wu, G., Liu, G., & Hao, M. (2010) · Information Processing and Trusted Computing · doi:10.1109/iptc.2010.60 · Cited by 95
A User Independent, Biosignal Based, Emotion Recognition Method
Rigas, G., Katsis, C. D., Ganiatsas, G., & Fotiadis, D. I. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-73078-1_36 · Cited by 90
Encyclopedia of Social Psychology
Hansen, N. (2007) · Cited by 83
Proceedings of the 5th international conference on Multimodal interfaces
(2003) · doi:10.1145/958432 · Cited by 79
Emotion recognition based on pressure sensor keyboards
Lv, H., Lin, Z., Yin, W., & Dong, J. S. (2008) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2008.4607628 · Cited by 78
Axisa, F., Dittmar, A., & Delhomme, G. (2004) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/iembs.2003.1280974 · Cited by 73
The Role of Affect and Emotion in HCI
Beale, R., & Peter, C. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-85099-1_1 · Cited by 53
Naji, M., Firoozabadi, M., & Azadfallah, P. (2013) · Cognitive Computation · doi:10.1007/s12559-013-9239-7 · Cited by 49
Towards a smart glove: Arousal recognition based on textile Electrodermal Response
Valenza, G., Lanatà, A., Scilingo, E. P., & Rossi, D. D. (2010) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/iembs.2010.5627453 · Cited by 48
Emotions Under the Skin: Autonomic Reactivity to Emotional Pictures in Insecure Attachment
Silva, C., Ferreira, A. C., Soares, I., & Esteves, F. G. (2015) · Journal of Psychophysiology · doi:10.1027/0269-8803/a000147 · Cited by 48
Prosody and speaker state: paralinguistics, pragmatics, and proficiency
Hirschberg, J., & Liscombe, J. (2007) · Cited by 45
Garment-Based Monitoring of Respiration Rate Using a Foam Pressure Sensor
Brady, S. A., Dunne, L. E., Tynan, R., et al. (2005) · Proceedings · doi:10.1109/iswc.2005.23 · Cited by 38
Park, M. W., Kim, C. J., Hwang, M.-C., & Lee, E. C. (2013) · World Congress on Software Engineering (WCSE) · doi:10.1109/wcse.2013.34 · Cited by 30
Neural processing of facial expressions of emotion in first onset psychosis
Brennan, A. M., Harris, A. W. F., & Williams, L. M. (2014) · Psychiatry Research · doi:10.1016/j.psychres.2014.06.017 · Cited by 23
Gradl, S., Cibis, T., Lauber, J., Richer, R., Rybalko, R., Pfeiffer, N., Leutheuser, H., Wirth, M., von Tscharner, V., & Eskofier, B. M. (2017) · Applied Sciences · doi:10.3390/app7121277 · Cited by 15
Assessing the validity of FaceReader using facial EMG
D’Arcey, J. T. (2013) · California State University ScholarWorks (system-wide DSpace) · Cited by 9
(1922) · Nature · doi:10.1038/110730b0 · Cited by 7
From Signals to Emotions: Applying Emotion Models to HM Affective Interactions
Ciceri, M. R., & Balzarotti, S. (2008) · doi:10.5772/6176 · Cited by 6
Cited by
A systematic review on affective computing: emotion models, databases, and recent advances
Wang, Y., Song, W., Tao, W., Liotta, A., Yang, D., Li, X., Gao, S., Sun, Y., Ge, W., Zhang, W., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.009 · Cited by 615
The Importance of Respiratory Rate Monitoring: From Healthcare to Sport and Exercise
Nicolò, A., Massaroni, C., Schena, E., & Sacchetti, M. (2020) · Sensors · doi:10.3390/s20216396 · Cited by 440
ECG Monitoring Systems: Review, Architecture, Processes, and Key Challenges
Serhani, M. A., T. El Kassabi, H., Ismail, H., & Nujum Navaz, A. (2020) · Sensors · doi:10.3390/s20061796 · Cited by 357
Emotion recognition in EEG signals using deep learning methods: A review
Jafari, M., Shoeibi, A., Khodatars, M., Bagherzadeh, S., Shalbaf, A., García, D. L., Gorriz, J. M., & Acharya, U. R. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107450 · Cited by 240
Molenaar, I., Mooij, S. D., Azevedo, R., Bannert, M., Järvelä, S., & Gašević, D. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107540 · Cited by 217
Metadata
- Authors
- Egger, M., Ley, M., & Hanke, S.
- Venue
- Electronic Notes in Theoretical Computer Science
- Publisher
- Elsevier BV
- Published
- 2019-05
- Volume
- 343
- Pages
- 35-55
- ISSN
- 1571-0661 (ISSN-L)
- OpenAlex
- W2946526173
- Cited by
- 561
- References
- 80
- FWCI
- 50.36
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.