Fusion of heart rate variability and pulse rate variability for emotion recognition using lagged poincare plots
Abstract
Designing an efficient automatic emotion recognition system based on physiological signals has attracted great interests within the research of human-machine interactions. This study was aimed to classify emotional responses by means of a simple dynamic signal processing technique and fusion frameworks. The electrocardiogram and finger pulse activity of 35 participants were recorded during rest condition and when subjects were listening to music intended to stimulate certain emotions. Four emotion categories, including happiness, sadness, peacefulness, and fear were chosen. Estimating heart rate variability (HRV) and pulse rate variability (PRV), 4 Poincare indices in 10 lags were extracted. The support vector machine classifier was used for emotion classification. Both feature level (FL) and decision level (DL) fusion schemes were examined. Significant differences have been observed between lag 1 Poincare plot indices and the other lagged measures. The mean accuracies of 84.1, 82.9, 79.68, and 76.05% were obtained for PRV, DL, FL, and HRV measures, respectively. However, DL outperformed others in discriminating sadness and peacefulness, using SD 1 and total features, correspondingly. In both cases, the classification rates improved up to 92% (with the sensitivity of 95% and specificity of 83.33%). Totally, DL resulted in better performances compared to FL. In addition, the impact of the fusion rules on the classification performances has been confirmed.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Goshvarpour et al. found that using HRV and PRV with feature-level and decision-level fusion rules improved classification rates up to 92% (sensitivity 95%, specificity 83.33%).
“Research by Goshvarpour et al. using HRV and PRV, and fusion rules (feature level, decision level) showed classification rates improved up to 92% (sensitivity: 95%, specificity: 83.33%) [63].”
From Survey on AI-Based Multimodal Methods for Emotion Detection (2019) · cites this paper as [63]
Cites
Cortes, C., & Vapnik, V. N. (1995) · Machine Learning · doi:10.1023/a:1022627411411 · Cited by 34,565 · see citing works
Heart Rate Variability: Standards of Measurement, Physiological Interpretation, and Clinical Use
Electrophysiology, T. F. O. T. E. S. O. C. T. N. A. (1996) · Circulation · doi:10.1161/01.cir.93.5.1043 · Cited by 17,159 · see citing works
Heart rate variability: Standards of measurement, physiological interpretation, and clinical use
Malik, M., Bigger, J. T., Camm, A. J., Kleiger, R. E., Malliani, A., Moss, A. J., & Schwartz, P. J. (1996) · European Heart Journal · doi:10.1093/oxfordjournals.eurheartj.a014868 · Cited by 15,289 · see citing works
Cook, R. J., Dickens, B. M., & Fathalla, M. F. (2003) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199241323.003.0025 · Cited by 10,813 · see citing works
Kittler, J., Hatef, M., Duin, R. P. W., & Matas, J. (1998) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.667881 · Cited by 5,312 · see citing works
Toward machine emotional intelligence: analysis of affective physiological state
Picard, R. W., Vyzas, E., & Healey, J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.954607 · Cited by 2,334 · see citing works
Emotion recognition based on physiological changes in music listening
Kim, J., & André, E. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.26 · Cited by 1,079 · see citing works
Emotion recognition system using short-term monitoring of physiological signals
Kim, K. H., Bang, S. W., & Kim, S. R. (2004) · Medical & Biological Engineering & Computing · doi:10.1007/bf02344719 · Cited by 944 · see citing works
Analysis of emotion recognition using facial expressions, speech and multimodal information
Busso, C., Deng, Z., Yildirim, S., Bulut, M., Lee, C. M., Kazemzadeh, A., Lee, S., Neumann, U., & Narayanan, S. (2004) · Proceedings of the 6th international conference on Multimodal interfaces · doi:10.1145/1027933.1027968 · Cited by 840 · see citing works
Quantitative beat-to-beat analysis of heart rate dynamics during exercise
Tulppo, M. P., Mäkikallio, T. H., Takala, T., Seppänen, T., & Huikuri, H. V. (1996) · American Journal of Physiology-Heart and Circulatory Physiology · doi:10.1152/ajpheart.1996.271.1.h244 · Cited by 813 · see citing works
A systems approach to appraisal mechanisms in emotion
Sander, D., Grandjean, D., & Scherer, K. R. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.03.001 · Cited by 703 · see citing works
Methods derived from nonlinear dynamics for analysing heart rate variability
Voss, A., Schulz, S., Schroeder, R., Baumert, M., & Caminal, P. (2008) · Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences · doi:10.1098/rsta.2008.0232 · Cited by 595 · see citing works
Ecg pattern analysis for emotion detection
Agrafioti, F., Hatzinakos, D., & Anderson, A. K. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.28 · Cited by 476 · see citing works
2015 IEEE Symposium Series on Computational Intelligence
(2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2015.2479518 · Cited by 451 · see citing works
Recognition of emotions using multimodal physiological signals and an ensemble deep learning model
Yin, Z., Zhao, M., Wang, Y., Yang, J., & Zhang, J. (2017) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2016.12.005 · Cited by 416 · see citing works
Happy, sad, scary and peaceful musical excerpts for research on emotions
Vieillard, S., Peretz, I., Gosselin, N. H., Khalfa, S., Gagnon, L., & Bouchard, B. (2008) · Cognition & Emotion · doi:10.1080/02699930701503567 · Cited by 383 · see citing works
Emotion recognition using wireless signals
Zhao, M., Adib, F., & Katabi, D. (2016) · Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking · doi:10.1145/2973750.2973762 · Cited by 370 · see citing works
Verma, G. K., & Tiwary, U. S. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.11.007 · Cited by 354 · see citing works
The Role of Nonlinear Dynamics in Affective Valence and Arousal Recognition
Valenza, G., Lanatà, A., & Scilingo, E. P. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.30 · Cited by 250 · see citing works
Revealing Real-Time Emotional Responses: a Personalized Assessment based on Heartbeat Dynamics
Valenza, G., Citi, L., Lanatá, A., Scilingo, E., & Barbieri, R. (2014) · Scientific Reports · doi:10.1038/srep04998 · Cited by 245 · see citing works
Guzik, P., Piskorski, J., Krauze, T., Schneider, R., Wesseling, K. H., Wykretowicz, A., & Wysocki, H. (2007) · The Journal of Physiological Sciences · doi:10.2170/physiolsci.rp005506 · Cited by 232 · see citing works
Fusion of facial expressions and EEG for implicit affective tagging
Koelstra, S., & Patras, I. (2012) · Image and Vision Computing · doi:10.1016/j.imavis.2012.10.002 · Cited by 228 · see citing works
Recognizing Emotions Induced by Affective Sounds through Heart Rate Variability
Nardelli, M., Valenza, G., Greco, A., Lanatà, A., & Scilingo, E. P. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2432810 · Cited by 219 · see citing works
Khalili, Z., & Moradi, M. H. (2009) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2009.5178854 · Cited by 145 · see citing works
Detecting Naturalistic Expressions of Nonbasic Affect Using Physiological Signals
AlZoubi, O., D’Mello, S. K., & Calvo, R. A. (2012) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2012.4 · Cited by 131 · see citing works
Selvaraj, J., Murugappan, M., Khairunizam, W., & Yaacob, S. (2013) · BioMedical Engineering OnLine · doi:10.1186/1475-925x-12-44 · Cited by 130 · see citing works
Valenza, G., Allegrini, P., Lanatà, A., & Scilingo, E. P. (2012) · Frontiers in Neuroengineering · doi:10.3389/fneng.2012.00003 · Cited by 113 · see citing works
Hilbert-Huang transform based physiological signals analysis for emotion recognition
Zong, C., & Chétouani, M. (2009) · IEEE International Symposium on Signal Processing and Information Technology (ISSPIT) · doi:10.1109/isspit.2009.5407547 · Cited by 107 · see citing works
Khezri, M., Firoozabadi, M., & Sharafat, A. R. (2015) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2015.07.006 · Cited by 89 · see citing works
The effects of auditory stimulation with music on heart rate variability in healthy women
Roque, A. L., Valenti, V. E., Guida, H. L., de Campos, M. F., Knap, A., Vanderlei, L. C. M., Ferreira, L. L., Ferreira, C., & de Abreu, L. C. (2013) · Clinics · doi:10.6061/clinics/2013(07)12 · Cited by 78 · see citing works
Emotion recognition from the facial image and speech signal
Go, H.-J., Kwak, K.-C., Lee, D.-J., & Chun, M.-G. (2003) · Cited by 63 · see citing works
Recognizing Human Emotion from Audiovisual Informaiton
Wang, Y., & Guan, L. (2006) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2005.1415607 · Cited by 55 · see citing works
Jerritta, S., Murugappan, M., Khairunizam, W., & Yaacob, S. (2013) · Expert Systems · doi:10.1111/exsy.12014 · Cited by 50 · see citing works
Murugappan, M., Murugappan, S., & Zheng, B. S. (2013) · Journal of Physical Therapy Science · doi:10.1589/jpts.25.753 · Cited by 47 · see citing works
Analysis of ECG signal by chaos principle to help automatic diagnosis of myocardial infarction
Lahiri, T., Kumar, U., Mishra, H., et al. (2009) · Journal of Scientific & Industrial Research · Cited by 38 · see citing works
Goshvarpour, A., Abbasi, A., & Goshvarpour, A. (2017) · Australasian Physical & Engineering Sciences in Medicine · doi:10.1007/s13246-017-0530-x · Cited by 37 · see citing works
Self-citationAuditory stimulation with music influences the geometric indices of heart rate variability in men
da Silva, S., Guida, H. L., dos SantosAntônio, A. M., Vanderlei, L. C. M., Ferreira, L. L., de Abreu, L. C., Sousa, F. H., & Valenti, V. E. (2014) · International Archives of Medicine · doi:10.1186/1755-7682-7-27 · Cited by 27 · see citing works
Moharreri, S., Parvaneh, S., Dabanloo, N. J., et al. (2010) · Iranian Conference of Biomedical Engineering (ICBME) · doi:10.1109/icbme.2010.5704919 · Cited by 25 · see citing works
Lingenfelser, F., Wagner, J., Deng, J., Brueckner, R., Schuller, B., & Andre, E. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2635124 · Cited by 23 · see citing works
Extracting Emotional Features from ECG by Using Wavelet Transform
Long, Z., Liu, G., & Dai, X. (2010) · doi:10.1109/icbecs.2010.5462441 · Cited by 22 · see citing works
A NOVEL SIGNAL-BASED FUSION APPROACH FOR ACCURATE MUSIC EMOTION RECOGNITION
Goshvarpour, A., Abbasi, A., Goshvarpour, A., et al. (2016) · Biomedical Engineering Applications Basis and Communications · doi:10.4015/s101623721650040x · Cited by 19 · see citing works
Self-citationSTATISTICAL PREDICTION OF EMOTIONAL STATES BY PHYSIOLOGICAL SIGNALS WITH MANOVA AND MACHINE LEARNING
Chueh, T.-H., Chen, T., Lu, H. H., JU, S.-S., Tao, T.-H., & Shaw, J.-H. (2012) · International Journal of Pattern Recognition and Artificial Intelligence · doi:10.1142/s0218001412500085 · Cited by 17 · see citing works
Goshvarpour, A., Abbasi, A., & Goshvarpour, A. (2015) · PubMed · Cited by 17 · see citing works
Self-citationGhaffari, S., Asadzadeh, R., Tajlil, A., Mohammadalian, A., & Pourafkari, L. (2016) · Annals of Noninvasive Electrocardiology · doi:10.1111/anec.12370 · Cited by 15 · see citing works
Toward Recognizing Two Emotion States from ECG Signals
Defu, C., Guangyuan, L., & Jing, C. (2009) · International Conference on Computational Intelligence and Natural Computing (CINC) · doi:10.1109/cinc.2009.240 · Cited by 10 · see citing works
Fusion Mappings for Multimodal Affect Recognition
Kächele, M., Schels, M., Thiam, P., et al. (2015) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2015.53 · Cited by 10 · see citing works
A new information fusion approach for recognition of music-induced emotions
Naji, M., Firoozabadi, M., & Azadfallah, P. (2014) · IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI ...) · doi:10.1109/bhi.2014.6864340 · Cited by 9 · see citing works
Multi-Modal Fusion Emotion Recognition Based on HMM and ANN
Xu, C., Cao, T., Feng, Z., & Dong, C. (2012) · Communications in computer and information science · doi:10.1007/978-3-642-34447-3_48 · Cited by 6 · see citing works
Analysis of Ecg In Phase Space for the Prediction of Spontaneous Atrial Fibrillation Termination
Parvaneh, S., Golpaygani, M. R. H., Firoozabadi, M., et al. (2016) · Journal of Electrocardiology · doi:10.1016/j.jelectrocard.2016.09.039 · Cited by 5 · see citing works
Goshvarpour, A., Abbasi, A., Goshvarpour, A., et al. (2016) · DOAJ (DOAJ: Directory of Open Access Journals) · doi:10.22038/ijmp.2016.7960 · Cited by 5 · see citing works
Self-citationA simple way of distinguishing chaotic characteristics in ECG signals
Wang, K., Zhao, Y., Sun, X., et al. (2010) · International Conference on Biomedical Engineering and Informatics (BMEI) · doi:10.1109/bmei.2010.5640068 · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Survey on AI-Based Multimodal Methods for Emotion Detection
Marechal, C., Mikołajewski, D., Tyburek, K., Prokopowicz, P., Bougueroua, L., Ancourt, C., & Węgrzyn-Wolska, K. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-16272-6_11 · Cited by 137 · see citing works
- Authors
- Goshvarpour, A., Abbasi, A., & Goshvarpour, A.
- Affiliations
- Sahand University of Technology (IR)
- Corresponding author
- Abbasi, Ataollah
- Venue
- Australasian Physical & Engineering Sciences in Medicine
- Published
- 2017-07-17
- Volume
- 40
- Issue
- 3
- Pages
- 617-629
- ISSN
- 0158-9938 (ISSN-L)
- 1879-5447
- OpenAlex
- W2735842846
- Cited by
- 46
- References
- 52
- Citations by year
- 2025: 2 · 2024: 3 · 2023: 5 · 2022: 6 · 2021: 10 · 2020: 8 · 2019: 7 · 2018: 5
- FWCI
- 7.12
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Australasian Physical & Engineering Sciences in Medicine (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › Heart Rate Variability and Autonomic Control
- Physical Sciences › Engineering › Biomedical Engineering › Non-Invasive Vital Sign Monitoring
- Keywords
- emotion recognition, heart rate variability, pulse rate variability, Poincaré plot, feature-level fusion, decision fusion, electrocardiogram, physiological signals, emotional response to music, support vector machine
- MeSH terms
- Algorithms, Emotions, Female, Heart Rate, Humans, Male, Pulse, Young Adult
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review