The DOI matches its Crossref record; 59 of the 59 references it cites were found in a registry or an index.
Wearable-Based Affect Recognition—A Review
Open accessAbstract
Affect recognition is an interdisciplinary research field bringing together researchers from natural and social sciences. Affect recognition research aims to detect the affective state of a person based on observables, with the goal to, for example, provide reasoning for the person’s decision making or to support mental wellbeing (e.g., stress monitoring). Recently, beside of approaches based on audio, visual or text information, solutions relying on wearable sensors as observables, recording mainly physiological and inertial parameters, have received increasing attention. Wearable systems enable an ideal platform for long-term affect recognition applications due to their rich functionality and form factor, while providing valuable insights during everyday life through integrated sensors. However, existing literature surveys lack a comprehensive overview of state-of-the-art research in wearable-based affect recognition. Therefore, the aim of this paper is to provide a broad overview and in-depth understanding of the theoretical background, methods and best practices of wearable affect and stress recognition. Following a summary of different psychological models, we detail the influence of affective states on the human physiology and the sensors commonly employed to measure physiological changes. Then, we outline lab protocols eliciting affective states and provide guidelines for ground truth generation in field studies. We also describe the standard data processing chain and review common approaches related to the preprocessing, feature extraction and classification steps. By providing a comprehensive summary of the state-of-the-art and guidelines to various aspects, we would like to enable other researchers in the field to conduct and evaluate user studies and develop wearable systems.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910
The Pittsburgh Sleep Quality Index: a new instrument for psychiatric practice and research
Buysse, D. J., Reynolds, C. F., Monk, T. H., Berman, S. R., & Kupfer, D. J. (1989) · Psychiatry Research · doi:10.1016/0165-1781(89)90047-4 · Cited by 35,234
A Global Measure of Perceived Stress
Cohen, S., Kamarck, T., & Mermelstein, R. (1983) · Journal of Health and Social Behavior · doi:10.2307/2136404 · Cited by 32,807
Studies of interference in serial verbal reactions
Stroop, J. R. (1935) · Journal of Experimental Psychology · doi:10.1037/h0054651 · Cited by 18,622
Heart rate variability: Standards of measurement, physiological interpretation, and clinical use
Malik, M., Bigger, J. T., Camm, A. J., Kleiger, R. E., Malliani, A., Moss, A. J., & Schwartz, P. J. (1996) · European Heart Journal · doi:10.1093/oxfordjournals.eurheartj.a014868 · Cited by 15,289
An argument for basic emotions
Ekman, P. (1992) · Cognition and Emotion · doi:10.1080/02699939208411068 · Cited by 9,646 · cited 1×
“Similar to Darwin [24], Ekman [25] argues that basic emotions are shared across cultures and appear to be universally recognised.”
BackgroundA Real-Time QRS Detection Algorithm
Pan, J., & Tompkins, W. J. (1985) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.1985.325532 · Cited by 7,855
Friedman, J., Hastie, T., & Tibshirani, R. (2000) · The Annals of Statistics · doi:10.1214/aos/1016218223 · Cited by 6,884
Core affect and the psychological construction of emotion.
Russell, J. A. (2003) · Psychological Review · doi:10.1037/0033-295x.110.1.145 · Cited by 6,113 · cited 1×
“Russell [21] defines affect as a neurophysiological state.”
BackgroundKirschbaum, C., Pirke, K.-M., & Hellhammer, D. H. (2008) · Neuropsychobiology · doi:10.1159/000119004 · Cited by 6,093
Ekman, P., & Friesen, W. V. (2019) · PsycTESTS Dataset · doi:10.1037/t27734-000 · Cited by 6,019
UCLA loneliness scale (version 3): reliability, validity, and factor structure
Russell, D. W. (1996) · Journal of Personality Assessment · doi:10.1207/s15327752jpa6601_2 · Cited by 5,800
DEAP: A Database for Emotion Analysis ;Using Physiological Signals
Koelstra, S., Muhl, C., Soleymani, M., Lee, J.-S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., & Patras, I. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.15 · Cited by 5,093
Fully integrated wearable sensor arrays for multiplexed in situ perspiration analysis
Gao, W., Emaminejad, S., Nyein, H. Y. Y., Challa, S., Chen, K., Peck, A., Fahad, H. M., Ota, H., Shiraki, H., Kiriya, D., et al. (2016) · Nature · doi:10.1038/nature16521 · Cited by 5,087 · cited 1×
“Hence, integrating chemical-electrophysiological sensors [179,180] in affect recognition studies has the potential to create new insights into the physiology of affective states.”
BackgroundNew well-being measures: Short scales to assess flourishing and positive and negative feelings
Diener, E., Wirtz, D., Tov, W., Kim-Prieto, C., Choi, D.-W., Oishi, S., & Biswas-Diener, R. (2010) · Social Indicators Research · doi:10.1007/s11205-009-9493-y · Cited by 4,980
Bower, G. H. (1981) · American Psychologist · doi:10.1037/0003-066x.36.2.129 · Cited by 4,154 · cited 1×
“The experiments of Bower [2] indicate that decision making and memorisation of a person are strongly influenced by their affective states.”
BackgroundJAMES, W. (1884) · Mind · doi:10.1093/mind/os-ix.34.188 · Cited by 3,446
Autonomic nervous system activity in emotion: a review
Kreibig, S. D. (2010) · Biological Psychology · doi:10.1016/j.biopsycho.2010.03.010 · Cited by 3,188 · cited 6×
“For more details, we refer the reader to Kreibig [58].”
BackgroundMethodInfluentialDeep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition
Ordóñez, F., & Roggen, D. (2016) · Sensors · doi:10.3390/s16010115 · Cited by 2,761 · cited 1×
“From a methodical point of view CNN offer interesting approaches to transfer [187] or semi-supervised [188] learning.”
BackgroundThe Concepts of Stress and Stress System Disorders: Overview of Physical and Behavioral Homeostasis
Chrousos, G. P. (1992) · JAMA · doi:10.1001/jama.1992.03480090092034 · Cited by 2,689
Toward machine emotional intelligence: analysis of affective physiological state
Picard, R. W., Vyzas, E., & Healey, J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.954607 · Cited by 2,334
Detecting Stress During Real-World Driving Tasks Using Physiological Sensors
Healey, J. A., & Picard, R. W. (2005) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2005.848368 · Cited by 2,207
Emotion elicitation using films
Gross, J. J., & Levenson, R. W. (1995) · Cognition & Emotion · doi:10.1080/02699939508408966 · Cited by 2,179
A tutorial on human activity recognition using body-worn inertial sensors
Bulling, A., Blanke, U., & Schiele, B. (2014) · ACM Computing Surveys · doi:10.1145/2499621 · Cited by 1,668
A review of affective computing: From unimodal analysis to multimodal fusion
Poria, S., Cambria, E., Bajpai, R., & Hussain, A. (2017) · Information Fusion · doi:10.1016/j.inffus.2017.02.003 · Cited by 1,644
Lupien, S. J., Maheu, F., Tu, M., Fiocco, A., & Schramek, T. E. (2007) · Brain and Cognition · doi:10.1016/j.bandc.2007.02.007 · Cited by 1,474
Introducing a New Benchmarked Dataset for Activity Monitoring
Reiss, A., & Stricker, D. (2012) · 2012 16th International Symposium on Wearable Computers · doi:10.1109/iswc.2012.13 · Cited by 1,456 · cited 7×
“75 Hz and 0–5 Hz band), peak frequency, entropy of the normalised power spectral density References: [45,137,153,154]”
BackgroundMethodInfluentialSelf-citationPeterson, R. A. (2001) · Journal of Consumer Research · doi:10.1086/323732 · Cited by 1,258 · cited 1×
“However, most affect recognition studies target students or research staff, which are likely to represent a homogeneous group [181].”
BackgroundVoluntary Facial Action Generates Emotion‐Specific Autonomic Nervous System Activity
Levenson, R. W., Ekman, P., & Friesen, W. V. (1990) · Psychophysiology · doi:10.1111/j.1469-8986.1990.tb02330.x · Cited by 1,248 · cited 1×
“This theory is supported, for instance, by experiments conducted by Levenson et al. [53], who found evidence that performing voluntary facial muscle movements exhibit similar changes in peripheral physiology as if the corresponding emotion is experienced.”
ResultOn the Analysis of Fingertip Photoplethysmogram Signals
Elgendi, M. (2012) · Current Cardiology Reviews · doi:10.2174/157340312801215782 · Cited by 1,206
Wang, R., Chen, F., Chen, Z., Li, T., Harari, G., Tignor, S., Zhou, X., Ben-Zeev, D., & Campbell, A. T. (2014) · Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing · doi:10.1145/2632048.2632054 · Cited by 1,116
Affective interactions using virtual reality: the link between presence and emotions
Riva, G., Mantovani, F., Capideville, C. S., Preziosa, A., Morganti, F., Villani, D., Gaggioli, A., Botella, C., & Alcañiz, M. (2007) · CyberPsychology & Behavior · doi:10.1089/cpb.2006.9993 · Cited by 1,111 · cited 1×
“[114], who used a head mounted display, showing three different VR environments (virtual parks), as an affective stimuli, eliciting a neutral, anxious and a relaxed affective state.”
BackgroundMulticenter Collaborative Panic Disorder Severity Scale
Shear, M. K., Brown, T. A., Barlow, D. H., Money, R., Sholomskas, D. E., Woods, S. W., Gorman, J. M., & Papp, L. A. (1997) · American Journal of Psychiatry · doi:10.1176/ajp.154.11.1571 · Cited by 1,001
Emotion recognition system using short-term monitoring of physiological signals
Kim, K. H., Bang, S. W., & Kim, S. R. (2004) · Medical & Biological Engineering & Computing · doi:10.1007/bf02344719 · Cited by 944
Wearable photoplethysmographic sensors—past and present
Tamura, T., Maeda, Y., Sekine, M., & Yoshida, M. (2014) · Electronics · doi:10.3390/electronics3020282 · Cited by 912 · cited 1×
“Hence, the cardiac cycle is captured by the PPG signal, where the pulsatile part of the PPG signal reflects the pulsatile component in arterial blood flow [61].”
Imani, S., Bandodkar, A. J., Mohan, A. M. V., Kumar, R., Yu, S., Wang, J., & Mercier, P. P. (2016) · Nature Communications · doi:10.1038/ncomms11650 · Cited by 861
Russell, J. A. (1979) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.37.3.345 · Cited by 857 · cited 2×
“At the end of the 1970s, Russell [30] postulated a two-dimensional model, namely the circumplex model (see Figure 1a).”
BackgroundAutomatic speech emotion recognition using recurrent neural networks with local attention
Mirsamadi, S., Barsoum, E., & Zhang, C. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952552 · Cited by 770
End-to-End Multimodal Emotion Recognition using Deep Neural Networks
Tzirakis, P., Trigeorgis, G., Nicolaou, M. A., Schuller, B., & Zafeiriou, S. (2017) · arXiv · doi:10.48550/arxiv.1704.08619 · Cited by 752
Discriminating Stress From Cognitive Load Using a Wearable EDA Device
Setz, C., Arnrich, B., Schumm, J., La Marca, R., Troster, G., & Ehlert, U. (2010) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2009.2036164 · Cited by 742
Deep, Convolutional, and Recurrent Models for Human Activity Recognition Using Wearables
Hammerla, N. Y., Halloran, S., & Ploetz, T. (2016) · arXiv · doi:10.48550/arxiv.1604.08880 · Cited by 606
Preprocessing techniques for context recognition from accelerometer data
Figo, D., Diniz, P. C., Ferreira, D. R., & Cardoso, J. M. P. (2010) · Personal and Ubiquitous Computing · doi:10.1007/s00779-010-0293-9 · Cited by 578
Smartphone-Based Recognition of States and State Changes in Bipolar Disorder Patients
Grunerbl, A., Muaremi, A., Osmani, V., Bahle, G., Ohler, S., Troster, G., Mayora, O., Haring, C., & Lukowicz, P. (2015) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2014.2343154 · Cited by 384
Towards Measuring Stress with Smartphones and Wearable Devices During Workday and Sleep
Muaremi, A., Arnrich, B., & Tröster, G. (2013) · BioNanoScience · doi:10.1007/s12668-013-0089-2 · Cited by 302
Kanjo, E., Younis, E. M. G., & Ang, C. S. (2019) · Information Fusion · doi:10.1016/j.inffus.2018.09.001 · Cited by 293
Santamaria-Granados, L., Munoz-Organero, M., Ramirez-Gonzalez, G., Abdulhay, E., & Arunkumar, N. (2019) · IEEE Access · doi:10.1109/access.2018.2883213 · Cited by 289
LSTM-Modeling of continuous emotions in an audiovisual affect recognition framework
Wöllmer, M., Kaiser, M., Eyben, F., Schuller, B., & Rigoll, G. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.03.001 · Cited by 288 · cited 1×
“In human activity recognition [184], audio analysis [185] or stock return forecasting [186], which all deal with time series data, (deep) neural networks (NN) proved to be powerful classifiers.”
BackgroundLearning deep physiological models of affect
Martinez, H. P., Bengio, Y., & Yannakakis, G. N. (2013) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2013.2247823 · Cited by 287
Automatic identification of artifacts in electrodermal activity data
Taylor, S., Jaques, N., Chen, W., Fedor, S., Sano, A., & Picard, R. (2015) · 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2015.7318762 · Cited by 279
Dynamic time warping and machine learning for signal quality assessment of pulsatile signals
Li, Q., & Clifford, G. D. (2012) · Physiological Measurement · doi:10.1088/0967-3334/33/9/1491 · Cited by 274
Personalized Multitask Learning for Predicting Tomorrow's Mood, Stress, and Health
Taylor, S., Jaques, N., Nosakhare, E., Sano, A., & Picard, R. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2784832 · Cited by 251
Development and Evaluation of an Ambulatory Stress Monitor Based on Wearable Sensors
Choi, J., Ahmed, B., & Gutierrez-Osuna, R. (2012) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2011.2169804 · Cited by 221 · cited 10×
“This is due to the fact, that changes in EDA are governed by the SNS [56].”
BackgroundMethodInfluentialCall Center Stress Recognition with Person-Specific Models
Hernandez, J., Morris, R. R., & Picard, R. W. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-24600-5_16 · Cited by 210
Eliciting positive, negative and mixed emotional states: A film library for affective scientists
Samson, A. C., Kreibig, S. D., Soderstrom, B., Wade, A. A., & Gross, J. J. (2016) · Cognition and Emotion · doi:10.1080/02699931.2015.1031089 · Cited by 155
A multimodal dataset for various forms of distracted driving
Taamneh, S., Tsiamyrtzis, P., Dcosta, M., Buddharaju, P., Khatri, A., Manser, M., Ferris, T., Wunderlich, R., & Pavlidis, I. (2017) · Scientific Data · doi:10.1038/sdata.2017.110 · Cited by 129
Willigenburg, N. W., Daffertshofer, A., Kingma, I., & van Dieën, J. H. (2012) · Journal of Electromyography and Kinesiology · doi:10.1016/j.jelekin.2012.01.001 · Cited by 120
Mobile Tele-Mental Health: Increasing Applications and a Move to Hybrid Models of Care
Chan, S., Torous, J., Hinton, L., & Yellowlees, P. (2014) · Healthcare · doi:10.3390/healthcare2020220 · Cited by 108 · cited 1×
“Furthermore, these systems could facilitate the development of tele-mental [192] and tele-medical applications.”
BackgroundKukolja, D., Popović, S., Horvat, M., Kovač, B., & Ćosić, K. (2014) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2014.05.006 · Cited by 66
Research works printed but not linked: 123
- Kory, 2015, A Review and Meta-Analysis of Multimodal Affect Detection Systems, ACM Comput. Surv., 47, 43:1
- McEwen, 1993, Stress and the individual: Mechanisms leading to disease, Arch. Intern. Med., 153, 2093, 10.1001/archinte.1993.00410180039004 · DOI · Paper page
- Rosmond, 1998, Endocrine and metabolic aberrations in men with abdominal obesity in relation to anxio-depressive infirmity, Metabolism, 47, 1187, 10.1016/S0026-0495(98)90321-3 · DOI
- Gangemi, 2014, Frame-Based Detection of Opinion Holders and Topics: A Model and a Tool, IEEE Comput. Intell. Mag., 9, 20, 10.1109/MCI.2013.2291688 · DOI
- Gravina, 2019, Emotion-relevant activity recognition based on smart cushion using multi-sensor fusion, Inf. Fusion, 48, 1, 10.1016/j.inffus.2018.08.001 · DOI
- Gjoreski, 2017, Monitoring stress with a wrist device using context, J. Biomed. Inform., 73, 159, 10.1016/j.jbi.2017.08.006 · DOI · Paper page
- Costa, 2019, Emotions detection on an ambient intelligent system using wearable devices, Future Gener. Comput. Syst., 92, 479, 10.1016/j.future.2018.03.038 · DOI
- Miller, 2012, The Smartphone Psychology Manifesto, Perspect. Psychol. Sci., 7, 221, 10.1177/1745691612441215 · DOI · Paper page
- Rastafoo, 2018, A Critical Review of Proactive Detection of Driver Stress Levels Based on Multimodal Measurements, ACM Comput. Surv., 51, 88
- Kim, 2013, A review on the computational methods for emotional state estimation from the human EEG, Comput. Math. Methods Med., 2013, 573734, 10.1155/2013/573734 · DOI · Paper page
- Liu, B. (2017). Many Facets of Sentiment Analysis. A Practical Guide to Sentiment Analysis, Springer. · DOI
- Cicero, M. (2002). Cicero on the Emotions: Tusculan Disputations 3 and 4, University of Chicago Press. · doi:10.7208/chicago/9780226305196.001.0001
- Darwin, C. (1999). The Expression of the Emotions in Man and Animals, HarperCollins Publishers. [3rd ed.]. First published in 1872. · DOI · Paper page
- Ekman, 1976, Measuring facial movement, Environ. Psychol. Nonverbal Behav., 1, 56, 10.1007/BF01115465 · DOI · Paper page
- Zenonos, A., Khan, A., and Sooriyabandara, M. (2016, January 14–18). HealthyOffice: Mood recognition at work using smartphones and wearable sensors. Proceedings of the PerCom Workshops, Sydney, Australia. · DOI
- Soleymani, 2012, A Multimodal Database for Affect Recognition and Implicit Tagging, IEEE Trans. Affect. Comput., 3, 42, 10.1109/T-AFFC.2011.25 · DOI · Paper page
- Becker-Asano, C. (2008). WASABI: Affect Simulation for Agents with Believable Interactivity, IOS Press.
- Kim, 2008, Emotion recognition based on physiological changes in music listening, IEEE Trans. Pattern Anal. Mach. Intell., 30, 2067, 10.1109/TPAMI.2008.26 · DOI · Paper page
- Valenza, 2014, Revealing real-time emotional responses: A personalized assessment based on heartbeat dynamics, Sci. Rep., 4, 4998, 10.1038/srep04998 · DOI · Paper page
- Abadi, 2015, DECAF: MEG-based multimodal database for decoding affective physiological responses, IEEE Trans. Affect. Comput., 6, 209, 10.1109/TAFFC.2015.2392932 · DOI · Paper page
- Morris, 1995, Observations: SAM: The Self-Assessment Manikin; an efficient cross-cultural measurement of emotional response, J. Advert. Res., 35, 63
- Subramanian, 2017, ASCERTAIN: Emotion and Personality Recognition using Commercial Sensors, IEEE Trans. Affect. Comput., 9, 147, 10.1109/TAFFC.2016.2625250 · DOI · Paper page
- Jirayucharoensak, 2014, EEG-based emotion recognition using deep learning network with principal component based covariate shift adaptation, Sci. World J., 2014, 627892, 10.1155/2014/627892 · DOI · Paper page
- Cannon, W. (1929). Bodily Changes in Pain, Hunger, Fear and Rage, D Appleton & Company. · DOI · Paper page
- Selye, H. (1974). Stress without distress. Psychopathology of Human Adaptation, Springer.
- Goldstein, 2007, Evolution of concepts of stress, Stress, 10, 109, 10.1080/10253890701288935 · DOI · Paper page
- Lu, H., Frauendorfer, D., and Choudhury, T. (2012, January 5–8). StressSense: Detecting stress in unconstrained acoustic environments using smartphones. Proceedings of the 2012 ACM Conference on Ubiquitous Computing, Pittsburgh, PA, USA. · DOI · Paper page
- Mozos, 2017, Stress detection using wearable physiological and sociometric sensors, Int. J. Neural Syst., 27, 1650041, 10.1142/S0129065716500416 · DOI · Paper page
- Plarre, K., Raij, A., and Scott, M. (2011, January 12–14). Continuous inference of psychological stress from sensory measurements collected in the natural environment. Proceedings of the 10th International Conference on Information Processing in Sensor Networks (IPSN), Chicago, IL, USA.
- Schmidt, P., Reiss, A., Dürichen, R., Marberger, C., and Van Laerhoven, K. (2018, January 16–20). Introducing WESAD, a Multimodal Dataset for Wearable Stress and Affect Detection. Proceedings of the 20th ACM International Conference on Multimodal Interaction, Boulder, CO, USA. · DOI · Paper page
- Sanches, P., Höök, K., Vaara, E., Weymann, C., Bylund, M., Ferreira, P., Peira, N., and Sjölinder, M. (2010, January 16–20). Mind the Body!: Designing a Mobile Stress Management Application Encouraging Personal Reflection. Proceedings of the 8th ACM Conference on Designing Interactive Systems, Aarhus, Denmark. · DOI
- Thayer, R. (1990). The Biopsychology of Mood and Arousal, Oxford University Press. · DOI · Paper page
- Schimmack, 2002, Experiencing activation: Energetic arousal and tense arousal are not mixtures of valence and activation, Emotion, 2, 412, 10.1037/1528-3542.2.4.412 · DOI
- Mehrotra, 2017, MyTraces: Investigating correlation and causation between users’ emotional states and mobile phone interaction, PACM Interact. Mob. Wearable Ubiquitous Technol., 1, 83
- Friedman, 2010, Feelings and the body: The Jamesian perspective on autonomic specificity of emotion, Biol. Psychol., 84, 383, 10.1016/j.biopsycho.2009.10.006 · DOI · Paper page
- McCorry, 2007, Physiology of the autonomic nervous system, Am. J. Pharm. Educ., 71, 78, 10.5688/aj710478 · DOI · Paper page
- Dawson, M., Schell, A., and Filion, D. (2000). The electrodermal system. Handbook of Psychophysiology, Cambridge University Press. [2nd ed.].
- Broek, E., Lisy, V., Janssen, J., Westerink, J., Schut, M., Tuinenbreijer, K., Fred, A., Filipe, J., and Gamboa, H. (2009). Affective Man-machine Interface: Unveiling Human Emotions through Biosignals. International Joint Conference on Biomedical Engineering Systems and Technologies, Springer.
- Mahdiani, S., Jeyhani, V., Peltokangas, M., and Vehkaoja, A. (2015, January 25–29). Is 50 Hz high enough ECG sampling frequency for accurate HRV analysis?. Proceedings of the 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy. · DOI
- Lin, W., Wu, D., Li, C., Zhang, H., and Zhang, Y. (2014). Comparison of Heart Rate Variability from PPG with That from ECG, Springer. · DOI
- Schmidt, P., Reiss, A., Dürichen, R., and Van Laerhoven, K. (2018, January 8–12). Labelling Affective States “in the Wild”: Practical Guidelines and Lessons Learned. Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers, Singapore. · DOI
- Lykken, 1971, Direct measurement of skin conductance: A proposal for standardization, Psychophysiology, 8, 656, 10.1111/j.1469-8986.1971.tb00501.x · DOI · Paper page
- Di Lascio, E., Gashi, S., and Santini, S. (2019, January 20–23). Laughter Recognition Using Non-invasive Wearable Devices. Proceedings of the 13th EAI International Conference on Pervasive Computing Technologies for Healthcare, Trento, Italy. · DOI
- Heinisch, J.S., Anderson, C., and David, K. (2019, January 11–15). Angry or Climbing Stairs? Towards Physiological Emotion Recognition in the Wild. Proceedings of the 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), Kyoto, Japan. · DOI
- 2001, Optimal signal bandwidth for the recording of surface EMG activity of facial, jaw, oral, and neck muscles, Psychophysiology, 38, 22, 10.1111/1469-8986.3810022 · doi:10.1111/1469-8986.3810022 · Paper page
- Wijsman, J., Grundlehner, B., and Hermens, H. (2010, January 5–7). Trapezius muscle EMG as predictor of mental stress. Proceedings of the Wireless Health 2010, San Diego, CA, USA. · DOI
- Lisetti, 2004, Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals, EURASIP J. Appl. Signal Process., 2004, 1672
- Soleymani, 2012, Multimodal emotion recognition in response to videos, IEEE Trans. Affect. Comput., 3, 211, 10.1109/T-AFFC.2011.37 · DOI · Paper page
- Ramos, J., Hong, J., and Dey, A. (2014, January 7–9). Stress recognition: A step outside the lab. Proceedings of the International Conference on Physiological Computing Systems, Lisbon, Portugal.
- Birjandtalab, J., Cogan, D., Pouyan, M.B., and Nourani, M. (2016, January 26–28). A Non-EEG Biosignals Dataset for Assessment and Visualization of Neurological Status. Proceedings of the 2016 IEEE International Workshop on Signal Processing Systems (SiPS), Dallas, TX, USA. · doi:10.1109/sips.2016.27
- Haag, A., Goronzy, S., Schaich, P., and Williams, J. (2004). Emotion recognition using bio-sensors: First steps towards an automatic system. Tutorial and Research Workshop on Affective Dialogue Systems, Springer. · DOI · Paper page
- Liu, C., Rani, P., and Sarkar, N. (2005, January 2–6). An empirical study of machine learning techniques for affect recognition in human-robot interaction. Proceedings of the 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, Edmonton, AB, Canada.
- Wagner, J., Kim, J., and André, E. (2005, January 6). From physiological signals to emotions: Implementing and comparing selected methods for feature extraction and classification. Proceedings of the IEEE International Conference on Multimedia and Expo (ICME), Amsterdam, The Netherlands.
- Leon, 2007, A user-independent real-time emotion recognition system for software agents in domestic environments, Eng. Appl. Artif. Intell., 20, 337, 10.1016/j.engappai.2006.06.001 · DOI
- Zhai, 2006, Stress detection in computer users through non-invasive monitoring of physiological signals, Biomed. Sci. Instrum., 42, 495
- Kim, D., Seo, Y., Cho, J., and Cho, C. (2008, January 20–25). Detection of subjects with higher self-reporting stress scores using heart rate variability patterns during the day. Proceedings of the 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Vancouver, BC, Canada. · DOI
- Katsis, 2008, Toward Emotion Recognition in Car-Racing Drivers: A Biosignal Processing Approach, IEEE Trans. Syst. Man, Cybern., 38, 502, 10.1109/TSMCA.2008.918624 · DOI · Paper page
- Calvo, R., Brown, I., and Scheding, S. (2009). Effect of Experimental Factors on the Recognition of Affective Mental States through Physiological Measures. AI 2009: Advances in Artificial Intelligenc, Springer. · DOI
- Chanel, 2009, Short-term emotion assessment in a recall paradigm, Int. J. Hum. Comput. Stud., 67, 607, 10.1016/j.ijhcs.2009.03.005 · DOI · Paper page
- Khalili, Z., and Moradi, M. (2009, January 14–19). Emotion recognition system using brain and peripheral signals: Using correlation dimension to improve the results of EEG. Proceedings of the 2009 International Joint Conference on Neural Networks, Atlanta, GA, USA. · DOI
- Healey, J., Nachman, L., Subramanian, S., Shahabdeen, J., and Morris, M. (2010). Out of the Lab and into the Fray: Towards Modeling Emotion in Everyday Life. Pervasive Computing, Springer. · DOI
- Valenza, 2012, The Role of Nonlinear Dynamics in Affective Valence and Arousal Recognition, IEEE Trans. Affect. Comput., 3, 237, 10.1109/T-AFFC.2011.30 · DOI · Paper page
- Hamdi, H., Richard, P., and Allain, P. (2012, January 10–15). Emotion assessment for affective computing based on physiological responses. Proceedings of the 2012 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE), Brisbane, Australia. · DOI
- Agrafioti, 2012, ECG Pattern Analysis for Emotion Detection, IEEE Trans. Affect. Comput., 3, 102, 10.1109/T-AFFC.2011.28 · DOI · Paper page
- Sano, A., and Picard, R. (2013, January 2–5). Stress Recognition Using Wearable Sensors and Mobile Phones. Proceedings of the Humaine Association Conference on Affective Computing and Intelligent Interaction (ACII), Geneva, Switzerland. · DOI · Paper page
- Adams, P., Rabbi, M., Rahman, T., Matthews, M., Voida, A., Gay, G., Choudhury, T., and Voida, S. (2014, January 20–23). Towards personal stress informatics: Comparing minimally invasive techniques for measuring daily stress in the wild. Proceedings of the 8th International Conference on Pervasive Computing Technologies for Healthcare, Oldenburg, Germany. · DOI
- Hovsepian, K., al’Absi, M., and Kumar, S. (2015, January 7–11). cStress: Towards a gold standard for continuous stress assessment in the mobile environment. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Osaka, Japan. · DOI · Paper page
- Rubin, J., Abreu, R., and Ahern, S. (2016, January 16–19). Time, Frequency & Complexity Analysis for Recognizing Panic States from Physiologic Time-Series. Proceedings of the 10th EAI International Conference on Pervasive Computing Technologies for Healthcare, Cancun, Mexico. · DOI
- Jaques, N., Taylor, S., Nosakhare, E., Sano, A., and Picard, R. (2016, January 5–10). Multi-task Learning for Predicting Health, Stress, and Happiness. Proceedings of the NIPS Workshop on Machine Learning for Healthcare, Barcelona, Spain.
- Rathod, P., George, K., and Shinde, N. (2016, January 14–17). Bio-signal based emotion detection device. Proceedings of the 2016 IEEE 13th International Conference on Wearable and Implantable Body Sensor Networks (BSN), San Francisco, CA, USA. · DOI
- Zhu, 2016, Naturalistic Recognition of Activities and Mood Using Wearable Electronics, IEEE Trans. Affect. Comput., 7, 272, 10.1109/TAFFC.2015.2491927 · DOI
- Girardi, D., Lanubile, F., and Novielli, N. (2017, January 23–26). Emotion detection using noninvasive low cost sensors. Proceedings of the Seventh International Conference on Affective Computing and Intelligent Interaction, San Antonio, TX, USA. · DOI
- Zhao, B., Wang, Z., Yu, Z., and Guo, B. (2018, January 8–12). EmotionSense: Emotion Recognition Based on Wearable Wristband. Proceedings of the 2018 IEEE SmartWorld, Ubiquitous Intelligence Computing, Advanced Trusted Computing, Scalable Computing Communications, Cloud Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), Guangzhou, China. · DOI
- Greco, 2018, Affective computing in virtual reality: Emotion recognition from brain and heartbeat dynamics using wearable sensors, Sci. Rep., 8, 13657, 10.1038/s41598-018-32063-4 · DOI · Paper page
- Hassan, 2019, Human emotion recognition using deep belief network architecture, Inf. Fusion, 51, 10, 10.1016/j.inffus.2018.10.009 · DOI · Paper page
- Mikels, 2005, Emotional category data on images from the International Affective Picture System, Behav. Res. Methods, 37, 626, 10.3758/BF03192732 · DOI · Paper page
- Hanai, T., and Ghassemi, M. (2017, January 4–10). Predicting Latent Narrative Mood Using Audio and Physiologic Data. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence, San Francisco, CA, USA.
- Peter, C., and Beale, R. (2008). Emotion Recognition through Multiple Modalities: Face, Body Gesture, Speech. Affect and Emotion in Human-Computer Interaction: From Theory to Applications, Springer.
- Dobriek, 2013, Towards Efficient Multi-Modal Emotion Recognition, Int. J. Adv. Robot. Syst., 10, 53, 10.5772/54002 · DOI
- Taylor, B., Dey, A., Siewiorek, D., and Smailagic, A. (2015, January 7–11). Using Physiological Sensors to Detect Levels of User Frustration Induced by System Delays. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Osaka, Japan. · DOI
- Mason, 1968, A review of psychoendocrine research on the sympathetic-adrenal medullary system, Psychosom. Med., 30, 631, 10.1097/00006842-196809000-00022 · DOI · Paper page
- Gjoreski, M., Gjoreski, H., and Gams, M. (2016, January 12–16). Continuous stress detection using a wrist device: In laboratory and real life. Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, Heidelberg, Germany. · DOI · Paper page
- Wijsman, J., Grundlehner, B., Liu, H., and Hermens, H. (2013, January 2–5). Wearable Physiological Sensors Reflect Mental Stress State in Office-Like Situations. Proceedings of the 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction, Geneva, Switzerland. · DOI
- Rubin, J., Eldardiry, H., Abreu, R., Ahern, S., Du, H., Pattekar, A., and Bobrow, D. (2015, January 7–11). Towards a mobile and wearable system for predicting panic attacks. Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing, Osaka, Japan. · DOI
- Sano, A., Yu, A., McHill, A., Phillips, A., and Picard, R. (2015, January 25–29). Prediction of Happy-Sad mood from daily behaviors and previous sleep history. Proceedings of the 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, Italy. · DOI
- Koh, 2001, Development of the Stress Response Inventory and its application in clinical practice, Psychosom. Med., 63, 668, 10.1097/00006842-200107000-00020 · DOI
- John, O., and Srivastava, S. (1999). The Big Five trait taxonomy: History, measurement, and theoretical perspectives. Handbook of Personality: Theory and Research, Guilford Press.
- Morris, 2009, Mobile Heart Health: Project Highlight, IEEE Pervasive Comput., 8, 57, 10.1109/MPRV.2009.31 · DOI
- Pollak, J.P., Adams, P., and Gay, G. (2011, January 7–12). PAM: A Photographic Affect Meter for Frequent, in Situ Measurement of Affect. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Vancouver, BC, Canada.
- Horne, 1976, A self-assessment questionnaire to determine morningness-eveningness in human circadian rhythms, Int. J. Chronobiol., 4, 97
- 2010, Pattern classification with missing data: A review, Neural Comput. Appl., 19, 263, 10.1007/s00521-009-0295-6 · DOI · Paper page
- Behar, 2013, ECG Signal Quality During Arrhythmia and Its Application to False Alarm Reduction, IEEE Trans. Biomed. Eng., 60, 1660, 10.1109/TBME.2013.2240452 · DOI · Paper page
- Biswas, 2019, Heart Rate Estimation From Wrist-Worn Photoplethysmography: A Review, IEEE Sens. J., 19, 6560, 10.1109/JSEN.2019.2914166 · DOI · Paper page
- Lee, 2010, Improved elimination of motion artifacts from a photoplethysmographic signal using a Kalman smoother with simultaneous accelerometry, Physiol. Meas., 31, 1585, 10.1088/0967-3334/31/12/003 · DOI
- Ram, 2012, A Novel Approach for Motion Artifact Reduction in PPG Signals Based on AS-LMS Adaptive Filter, IEEE Trans. Instrum. Meas., 61, 1445, 10.1109/TIM.2011.2175832 · DOI · Paper page
- Reiss, A., Indlekofer, I., Schmidt, P., and Van Laerhoven, K. (2019). Deep PPG: Large-Scale Heart Rate Estimation with Convolutional Neural Networks. Sensors, 19. · DOI · Paper page
- Salehizadeh, S.M.A., Dao, D., Bolkhovsky, J., Cho, C., Mendelson, Y., and Chon, K.H. (2016). A Novel Time-Varying Spectral Filtering Algorithm for Reconstruction of Motion Artifact Corrupted Heart Rate Signals During Intense Physical Activities Using a Wearable Photoplethysmogram Sensor. Sensors, 16. · DOI · Paper page
- Greco, 2016, cvxEDA: A Convex Optimization Approach to Electrodermal Activity Processing, IEEE Trans. Biomed. Eng., 63, 797
- Benedek, 2010, Decomposition of skin conductance data by means of nonnegative deconvolution, Psychophysiology, 47, 647
- Huynh, T., and Schiele, B. (2005, January 12–14). Analyzing Features for Activity Recognition. Proceedings of the 2005 Joint Conference on Smart Objects and Ambient Intelligence: Innovative Context-aware Services: Usages and Technologies, Grenoble, France. · DOI · Paper page
- Parkka, J., Ermes, M., Antila, K., van Gils, M., Manttari, A., and Nieminen, H. (2007, January 22–26). Estimating Intensity of Physical Activity: A Comparison of Wearable Accelerometer and Gyro Sensors and 3 Sensor Locations. Proceedings of the 29th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, Lyon, France. · DOI
- Lim, 1997, Decomposing skin conductance into tonic and phasic components, Int. J. Psychophysiol., 25, 97, 10.1016/S0167-8760(96)00713-1 · DOI · Paper page
- Rainville, 2006, Basic emotions are associated with distinct patterns of cardiorespiratory activity, Int. J. Psychophysiol., 61, 5, 10.1016/j.ijpsycho.2005.10.024 · DOI · Paper page
- Kollia, V. (2016). Personalization Effect on Emotion Recognition from Physiological Data: An Investigation of Performance on Different Setups and Classifiers. ArXiv.
- Cernadas, 2014, Do We Need Hundreds of Classifiers to Solve Real World Classification Problems?, J. Mach. Learn. Res., 15, 3133
- Freund, 1999, A Short Introduction to Boosting Yoav, J. Jpn. Soc. Artif. Intell., 14, 1612
- Münzner, S., Schmidt, P., Reiss, A., Hanselmann, M., Stiefelhagen, R., and Dürichen, R. (2017, January 11–15). CNN-based Sensor Fusion Techniques for Multimodal Human Activity Recognition. Proceedings of the 2017 ACM International Symposium on Wearable Computers, Maui, HI, USA. · DOI · Paper page
- Guyon, 2003, An Introduction to Variable and Feature Selection, J. Mach. Learn. Res., 3, 1157
- Ertin, E., Stohs, N., Kumar, S., Raij, A., al’Absi, M., and Shah, S. (2011, January 1–4). AutoSense: Unobtrusively wearable sensor suite for inferring the onset, causality, and consequences of stress in the field. Proceedings of the 9th ACM Conference on Embedded Networked Sensor Systems, Seattle, WA, USA. · DOI · Paper page
- Sadri, 2018, Wearable and Implantable Epidermal Paper-Based Electronics, ACS Appl. Mater. Interfaces, 10, 31061, 10.1021/acsami.8b11020 · DOI
- Ameri, S.K., Ho, R., Jang, H., Wang, Y., Schnyer, D.M., Akinwande, D., and Lu, N. (2016, January 3–7). Thinnest transparent epidermal sensor system based on graphene. Proceedings of the 2016 IEEE International Electron Devices Meeting (IEDM), San Francisco, CA, USA. · DOI
- Reiss, A., and Amft, O. (2015). Design challenges of real wearable computers. Fundamentals of Wearable Computers and Augmented Reality, CRC Press.
- Lonini, L., Shawen, N., Ghaffari, R., Rogers, J., and Jayarman, A. (2017, January 11–15). Automatic Detection of Spasticity from Flexible Wearable Sensors. Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers, Maui, HI, USA. · DOI
- Vrijkotte, 2000, Effects of Work Stress on Ambulatory Blood Pressure, Heart Rate, and Heart Rate Variability, Hypertension, 35, 880, 10.1161/01.HYP.35.4.880 · DOI · Paper page
- Gesche, 2012, Continuous blood pressure measurement by using the pulse transit time: Comparison to a cuff-based method, Eur. J. Appl. Physiol., 112, 309, 10.1007/s00421-011-1983-3 · DOI · Paper page
- Pandia, K., Ravindran, S., Cole, R., Kovacs, G., and Giovangrandi, L. (2010, January 14–19). Motion artifact cancellation to obtain heart sounds from a single chest-worn accelerometer. Proceedings of the 2010 IEEE International Conference on Acoustics, Speech and Signal Processing, Dallas, TX, USA. · DOI
- Popoola, G.A., Graves, C.A., and Ford-Booker, P. (2018, January 6–8). Using Unsupervised Anomaly Detection to Analyze Physiological Signals for Emotion Recognition. Proceedings of the 2018 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), Louisville, KY, USA. · DOI
- Yang, J., Nguyen, M., San, P., Li, X., and Krishnaswamy, S. (2015, January 6–8). Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition. Proceedings of the 24th International Conference on Artificial Intelligence, Louisville, KY, USA.
- Enke, 2005, The use of data mining and neural networks for forecasting stock market returns, Expert Syst. Appl., 29, 927, 10.1016/j.eswa.2005.06.024 · DOI · Paper page
- Vincent, 2010, Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion, J. Mach. Learn. Res., 11, 3371
- Bhattacharya, S., and Lane, N.D. (2016, January 14–18). From smart to deep: Robust activity recognition on smartwatches using deep learning. Proceedings of the 2016 IEEE International Conference on Pervasive Computing and Communication Workshops (PerCom Workshops), Sydney, Australia. · DOI · Paper page
- Budner, P., Eirich, J., and Gloor, P. (2017). “Making you happy makes me happy”—Measuring Individual Mood with Smartwatches. arXiv.
- Kanjo, 2015, Emotions in context: Examining pervasive affective sensing systems, applications, and analyses, Pers. Ubiquitous Comput., 19, 1197, 10.1007/s00779-015-0842-3 · DOI
Cited by
A systematic review on affective computing: emotion models, databases, and recent advances
Wang, Y., Song, W., Tao, W., Liotta, A., Yang, D., Li, X., Gao, S., Sun, Y., Ge, W., Zhang, W., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.009 · Cited by 615
Górriz, J. M., Ramírez, J., Ortíz, A., Martínez-Murcia, F. J., Segovia, F., Suckling, J., Leming, M., Zhang, Y.-D., Álvarez-Sánchez, J. R., Bologna, G., et al. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.05.078 · Cited by 319
A Review of Wearable Multi-Wavelength Photoplethysmography
Ray, D., Collins, T., Woolley, S., & Ponnapalli, P. (2023) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2021.3121476 · Cited by 155
Emotion assessment using Machine Learning and low-cost wearable devices
Laureanti, R., Bilucaglia, M., Zito, M., Circi, R., Fici, A., Rivetti, F., Valesi, R., Oldrini, C., Mainardi, L. T., & Russo, V. (2020) · 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC) · doi:10.1109/embc44109.2020.9175221 · Cited by 33
Comparison of Wearable and Clinical Devices for Acquisition of Peripheral Nervous System Signals
Bizzego, A., Gabrieli, G., Furlanello, C., & Esposito, G. (2020) · Sensors · doi:10.3390/s20236778 · Cited by 25
Metadata
- Authors
- Schmidt, P., Reiss, A., Dürichen, R., & Laerhoven, K. V.
- Venue
- Sensors
- Publisher
- MDPI AG
- Published
- 2019-09-20
- Volume
- 19
- Issue
- 19
- Pages
- 4079
- ISSN
- 1424-8220 (ISSN-L)
- OpenAlex
- W2974457857
- Cited by
- 218
- References
- 59
- FWCI
- 24.06
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.