CoChat
Sign Up
Cited by 13Emotion and Mood RecognitionGait Recognition and AnalysisHuman Pose and Action Recognition

The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.

Learning Gait Emotions Using Affective and Deep Features

Randhavane, Tanmay, Bhattacharya, Uttaran, Kabra, Pooja, Kapsaskis, Kyra, Gray, Kurt, Manocha, Dinesh, Bera, Aniket·2022·Proceedings of the 15th ACM SIGGRAPH Conference on Motion, Interaction and Games
DOI 10.1145/3561975.3562957
Open this paper in CoChat — free account

Abstract

We present a novel data-driven algorithm to learn the perceived emotions of individuals based on their walking motion or gaits. Given an RGB video of an individual walking, we extract their walking gait as a sequence of 3D poses. Our goal is to exploit the gait features to learn and model the emotional state of the individual into one of four categorical emotions: happy, sad, angry, or neutral. Our perceived emotion identification approach uses deep features learned using long short-term memory networks (LSTMs) on datasets with labeled emotive gaits. We combine these features with gait-based affective features consisting of posture and movement measures. Our algorithm identifies both the categorical emotions from the gaits and the corresponding values for the dimensional emotion components - valence and arousal. We also introduce and benchmark a dataset called Emotion Walk (EWalk), consisting of videos of gaits of individuals annotated with emotions. We show that our algorithm mapping the combined feature space to the perceived emotional state provides an accuracy of 80.07% on the EWalk dataset, outperforming the current baselines by an absolute 13–24%.

Overview

Integrity signals

Registry

Crossref · 10.1145/3561975.3562957

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • EMERSK's gait module exhibits improved performance compared to several other contemporary works for gait emotion recognition and closely matches the best performing works of Tanmay et al. [3].

    “Our approach exhibits improved performance in comparison to several other contemporary works for gait emotion recognition and closely matches the best performing works of Tanmay et al. [3].”

    From EMERSK -- Explainable Multimodal Emotion Recognition with Situational Knowledge (2023) · cites this paper as [3]

  • The EWalk dataset is composed of acted videos featuring walking individuals with gaits and annotated emotions.

    “The EWalk [3] dataset is composed of acted videos featuring walking individuals with gaits and annotated emotions.”

    From EMERSK -- Explainable Multimodal Emotion Recognition with Situational Knowledge (2023) · cites this paper as [3]

Citations

Cites

Flagged first, then by citations

Research works printed but not linked: 66

  1. Danilo Avola , Marco Cascio , Luigi Cinque , Gian Luca Foresti , Cristiano Massaroni , and Emanuele Rodolà . 2019. 2d skeleton-based action recognition via two-branch stacked lstm-rnns . IEEE Transactions on Multimedia( 2019 ). Danilo Avola, Marco Cascio, Luigi Cinque, Gian Luca Foresti, Cristiano Massaroni, and Emanuele Rodolà. 2019. 2d skeleton-based action recognition via two-branch stacked lstm-rnns. IEEE Transactions on Multimedia(2019).
  2. LF Barrett 2019. Emotional expressions reconsidered: challenges to inferring emotion from human facial movements. Psychological Science in the Public Interest ( 2019 ). LF Barrett 2019. Emotional expressions reconsidered: challenges to inferring emotion from human facial movements. Psychological Science in the Public Interest (2019).
  3. F C Benitez-Quiroz , R Srinivasan , 2016 . Emotionet: An accurate, real-time algorithm for the automatic annotation of a million facial expressions in the wild. In CVPR. F C Benitez-Quiroz, R Srinivasan, 2016. Emotionet: An accurate, real-time algorithm for the automatic annotation of a million facial expressions in the wild. In CVPR.
  4. 10.1007/978-3-540-74889-2_6 · DOI
  5. 10.1609/aaai.v34i02.5490 · DOI
  6. 10.1007/978-3-030-58607-2_9 · DOI
  7. 10.1109/CVPR.2017.173 · DOI
  8. 10.1016/S1071-5819(03)00050-8 · DOI
  9. Z Cao T Simon 2017. Realtime 2D Pose Estimation using Part Affinity Fields. In CVPR. Z Cao T Simon 2017. Realtime 2D Pose Estimation using Part Affinity Fields. In CVPR. · DOI · Paper page
  10. L Carli 1995. Nonverbal behavior gender and influence.Journal of personality and social psychology 68 6(1995) 1030. L Carli 1995. Nonverbal behavior gender and influence.Journal of personality and social psychology 68 6(1995) 1030. · DOI
  11. 10.1109/CVPRW53098.2021.00265 · DOI
  12. A Crenn A Khan 2016. Body expression recognition from anim. 3D skeleton. In IC3D. A Crenn A Khan 2016. Body expression recognition from anim. 3D skeleton. In IC3D. · DOI · Paper page
  13. A Crenn A Meyer 2017. Toward an efficient body expression recognition based on the synthesis of a neutral movement. In ICMI. A Crenn A Meyer 2017. Toward an efficient body expression recognition based on the synthesis of a neutral movement. In ICMI. · DOI
  14. R Dabral A Mundhada 2018. Learning 3d human pose from structure and motion. In ECCV. R Dabral A Mundhada 2018. Learning 3d human pose from structure and motion. In ECCV. · DOI
  15. M Daoudi. 2017. Emotion recognition by body movement representation on the manifold of symmetric positive definite matrices. In ICIAP. M Daoudi. 2017. Emotion recognition by body movement representation on the manifold of symmetric positive definite matrices. In ICIAP. · DOI
  16. 10.1037/0003-066X.48.4.384 · DOI · Paper page
  17. Melissa J Ferguson and John A Bargh . 2004. How social perception can automatically influence behavior. Trends in cognitive sciences 8, 1 ( 2004 ), 33–39. Melissa J Ferguson and John A Bargh. 2004. How social perception can automatically influence behavior. Trends in cognitive sciences 8, 1 (2004), 33–39.
  18. Joseph L Fleiss and Jacob Cohen . 1973. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and psychological measurement 33, 3 ( 1973 ), 613–619. Joseph L Fleiss and Jacob Cohen. 1973. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educational and psychological measurement 33, 3 (1973), 613–619.
  19. 10.1145/1314161.1314180 · DOI · Paper page
  20. I Goodfellow 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672–2680. I Goodfellow 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672–2680.
  21. K Greff , R K Srivastava , 2017 . LSTM: A search space odyssey . IEEE transactions on neural networks and learning systems (2017). K Greff, R K Srivastava, 2017. LSTM: A search space odyssey. IEEE transactions on neural networks and learning systems (2017).
  22. James J Gross . 2008. Emotion regulation. Handbook of emotions 3, 3 ( 2008 ), 497–513. James J Gross. 2008. Emotion regulation. Handbook of emotions 3, 3 (2008), 497–513.
  23. 10.1016/j.humov.2011.05.001 · DOI
  24. M Gross E Crane 2010. Methodology for assessing bodily expression of emotion. Journal of Nonverbal Behavior(2010). M Gross E Crane 2010. Methodology for assessing bodily expression of emotion. Journal of Nonverbal Behavior(2010). · DOI
  25. 10.1109/TSMCB.2008.927269 · DOI
  26. David J Hauser and Norbert Schwarz . 2016 . Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior research methods 48, 1 (2016), 400–407. David J Hauser and Norbert Schwarz. 2016. Attentive Turkers: MTurk participants perform better on online attention checks than do subject pool participants. Behavior research methods 48, 1 (2016), 400–407.
  27. C Ionescu , D Papava , 2014. Human3.6M : Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments. TPAMI ( 2014 ). C Ionescu, D Papava, 2014. Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments. TPAMI (2014).
  28. M Karg 2013. Body movements for affective expression: A survey of automatic recognition and generation . IEEE Transactions on Affective Computing( 2013 ). M Karg 2013. Body movements for affective expression: A survey of automatic recognition and generation. IEEE Transactions on Affective Computing(2013).
  29. 10.1109/TSMCB.2010.2044040 · DOI
  30. ZH Kilimci 2019. Mood detection from physical and neurophysical data using deep learning models. Complexity 2019(2019). ZH Kilimci 2019. Mood detection from physical and neurophysical data using deep learning models. Complexity 2019(2019). · DOI
  31. Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114(2013). Diederik P Kingma and Max Welling. 2013. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114(2013). · arXiv
  32. 10.1109/TSMCB.2010.2103557 · DOI
  33. 10.1109/T-AFFC.2012.16 · DOI
  34. 10.1016/j.compedu.2016.04.002 · DOI
  35. J Richard Landis and Gary G Koch . 1977. The measurement of observer agreement for categorical data. biometrics ( 1977 ), 159–174. J Richard Landis and Gary G Koch. 1977. The measurement of observer agreement for categorical data. biometrics (1977), 159–174.
  36. 10.1109/TAFFC.2016.2637343 · DOI
  37. T Lin M Maire 2014. Microsoft coco: Common objects in context. In ECCV. T Lin M Maire 2014. Microsoft coco: Common objects in context. In ECCV. · DOI · Paper page
  38. Y Luo J Ren 2018. LSTM Pose Machines. In CVPR. Y Luo J Ren 2018. LSTM Pose Machines. In CVPR. · DOI
  39. Y Ma 2006. A motion capture library for the study of identity gender and emotion perception from biological motion. Behavior research methods(2006). Y Ma 2006. A motion capture library for the study of identity gender and emotion perception from biological motion. Behavior research methods(2006). · DOI
  40. 10.1145/3072959.3073596 · DOI
  41. J Michalak N Troje 2009. Embodiment of sadness and depression-gait patterns associated with dysphoric mood. Psychosomatic Medicine(2009). J Michalak N Troje 2009. Embodiment of sadness and depression-gait patterns associated with dysphoric mood. Psychosomatic Medicine(2009). · DOI
  42. J Mikels B Fredrickson 2005. Emotional category data on images from the International Affective Picture System. Behavior research methods(2005). J Mikels B Fredrickson 2005. Emotional category data on images from the International Affective Picture System. Behavior research methods(2005). · DOI
  43. 10.1109/CVPR42600.2020.01424 · DOI
  44. J Montepare S Goldstein 1987. The identification of emotions from gait information. Journal of Nonverbal Behavior(1987). J Montepare S Goldstein 1987. The identification of emotions from gait information. Journal of Nonverbal Behavior(1987). · DOI
  45. S Narang A Best 2017. Motion recognition of self and others on realistic 3D avatars. Computer Animation & Virtual Worlds(2017). S Narang A Best 2017. Motion recognition of self and others on realistic 3D avatars. Computer Animation & Virtual Worlds(2017). · DOI
  46. K M Naugle 2011. Emotional state affects the initiation of forward gait.Emotion 11, 2 ( 2011 ), 267. K M Naugle 2011. Emotional state affects the initiation of forward gait.Emotion 11, 2 (2011), 267.
  47. F Noroozi 2018. Survey on emotional body gesture recognition . IEEE transactions on affective computing( 2018 ). F Noroozi 2018. Survey on emotional body gesture recognition. IEEE transactions on affective computing(2018).
  48. L Omlor , M Giese , 2007. Extraction of spatio-temporal primitives of emotional body expressions. Neurocomputing ( 2007 ). L Omlor, M Giese, 2007. Extraction of spatio-temporal primitives of emotional body expressions. Neurocomputing (2007).
  49. S Piana 2013 . A set of full-body movement features for emotion recognition to help children affected by autism spectrum condition . In IDGEI International Workshop. S Piana 2013. A set of full-body movement features for emotion recognition to help children affected by autism spectrum condition. In IDGEI International Workshop.
  50. S Piana and Others. 2014. Real-time automatic emotion recognition from body gestures. arXiv preprint arXiv:1402.5047(2014). S Piana and Others. 2014. Real-time automatic emotion recognition from body gestures. arXiv preprint arXiv:1402.5047(2014). · arXiv
  51. S Piana and Others. 2016 . Adaptive body gesture representation for automatic emotion recognition . ACM Transactions on Interactive Intelligent Systems (TiiS) 6 , 1(2016), 6 . S Piana and Others. 2016. Adaptive body gesture representation for automatic emotion recognition. ACM Transactions on Interactive Intelligent Systems (TiiS) 6, 1(2016), 6.
  52. T Randhavane 2019. Learning perceived emotion using affective and deep features for mental health applications. In 2019 IEEE ISMAR-Adjunct . IEEE , 395–399. T Randhavane 2019. Learning perceived emotion using affective and deep features for mental health applications. In 2019 IEEE ISMAR-Adjunct. IEEE, 395–399.
  53. S Ribet 2019. Survey on Style in 3D Human Body Motion: Taxonomy , Data, Recognition and its Applications . IEEE Transactions on Affective Computing( 2019 ). S Ribet 2019. Survey on Style in 3D Human Body Motion: Taxonomy, Data, Recognition and its Applications. IEEE Transactions on Affective Computing(2019).
  54. C Roether , L Omlor , 2009b. Critical features for the perception of emotion from gait. Vision ( 2009 ). C Roether, L Omlor, 2009b. Critical features for the perception of emotion from gait. Vision (2009).
  55. C L Roether 2009a. Features in the recognition of emotions from dynamic bodily expression . In Dynamics of Visual Motion Processing . Springer , 313–340. C L Roether 2009a. Features in the recognition of emotions from dynamic bodily expression. In Dynamics of Visual Motion Processing. Springer, 313–340.
  56. J Russell , J Bachorowski , 2003. Facial & vocal expressions of emotion. Rev . of Psychology ( 2003 ). J Russell, J Bachorowski, 2003. Facial & vocal expressions of emotion. Rev. of Psychology (2003).
  57. 10.3390/e21070646 · DOI
  58. 10.1109/TCIAIG.2012.2202663 · DOI
  59. J Tilmanne 2012. Stylistic gait synthesis based on hidden Markov models. EURASIP Journal on advances in signal processing ( 2012 ). J Tilmanne 2012. Stylistic gait synthesis based on hidden Markov models. EURASIP Journal on advances in signal processing (2012).
  60. Joëlle Tilmanne and Thierry Dutoit . 2010. Expressive gait synthesis using PCA and Gaussian modeling . In MiG. Springer , 363–374. Joëlle Tilmanne and Thierry Dutoit. 2010. Expressive gait synthesis using PCA and Gaussian modeling. In MiG. Springer, 363–374.
  61. Silvan S Tomkins . 1984. Affect theory. Approaches to emotion 163, 163-195 ( 1984 ). Silvan S Tomkins. 1984. Affect theory. Approaches to emotion 163, 163-195 (1984).
  62. G Venture H Kadone 2014. Recognizing emotions conveyed by human gait. International Journal of Social Robotics(2014). G Venture H Kadone 2014. Recognizing emotions conveyed by human gait. International Journal of Social Robotics(2014). · DOI
  63. C Wang 2019. Learning Temporal and Bodily Attention in Protective Movement Behavior Detection. In 2019 8th ACIIW . IEEE , 324–330. C Wang 2019. Learning Temporal and Bodily Attention in Protective Movement Behavior Detection. In 2019 8th ACIIW. IEEE, 324–330.
  64. S Xia , C Wang , 2015. Realtime style transfer unlabeled heterogeneous human motion. TOG ( 2015 ). S Xia, C Wang, 2015. Realtime style transfer unlabeled heterogeneous human motion. TOG (2015).
  65. 10.1609/aaai.v32i1.12328 · DOI · Paper page
  66. Ceyuan Yang Zhe Wang Xinge Zhu Chen Huang Jianping Shi and Dahua Lin. 2018. Pose guided human video generation. In ECCV. 201–216. Ceyuan Yang Zhe Wang Xinge Zhu Chen Huang Jianping Shi and Dahua Lin. 2018. Pose guided human video generation. In ECCV. 201–216. · DOI

Cited by

Sorted by citations

Metadata

Authors
Randhavane, T., Bhattacharya, U., Kabra, P., Kapsaskis, K., Gray, K., Manocha, D., & Bera, A.
Affiliations
  • University of North Carolina at Chapel Hill (US)
  • University of Maryland, College Park (US)
  • Purdue University West Lafayette (US)
ORCID
Venue
Proceedings of the 15th ACM SIGGRAPH Conference on Motion, Interaction and Games
Publisher
ACM
Published
2022-11-03
Pages
1-10
DOI
10.1145/3561975.3562957
OpenAlex
W4304617161
Cited by
13
References
1
Citations by year
2026: 1 · 2025: 2 · 2024: 4 · 2023: 6
FWCI
3.92
Type
conference paper
Language
en
Open access
closed
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
ACM SIGGRAPH Conference on Motion, Interaction and Games (MIG) (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
  • Physical Sciences › Engineering › Biomedical Engineering › Gait Recognition and Analysis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Human Pose and Action Recognition
Keywords
walking motion, affective features, valence-arousal estimation, long short-term memory networks, RGB video

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review