Cascade Attention Networks For Group Emotion Recognition with Face, Body and Image Cues
The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
This paper presents our approach for group-level emotion recognition sub-challenge in the EmotiW 2018. The task is to classify an image into one of the group emotions such as positive, negative, and neutral. Our approach mainly explores three cues, namely face, body and global image with recent deep networks. Our main contribution is two-fold. First, we introduce body based Convolutional Neural Networks (CNNs) into this task based on our previous winner method [18]. For body based CNNs, we crop all bodies in an image with the state-of-the-art human pose estimation method and train CNNs with the image-level label to capture. The body cue captures a full view of an individual. Second, we propose a cascade attention network for the face cue in images. This network exploits the importance of each face in an image to generates a global representation based on all faces. The cascade attention network is not only complementary with other models but also improves the naive average pooling method by about 2%. We finally achieve the second place in this sub-challenge with classification accuracies of 86.9% and 67.48% on the validation set and testing set, respectively.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields
Cao, Z., Simon, T., Wei, S.-E., & Sheikh, Y. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.143 · Cited by 7,539
Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks
Zhang, K., Zhang, Z., Li, Z., & Qiao, Y. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2603342 · Cited by 6,214
Self-citationDeep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · arXiv · doi:10.48550/arxiv.1512.03385 · Cited by 4,196
Wei, S.-E., Ramakrishna, V., Kanade, T., & Sheikh, Y. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.511 · Cited by 2,966
From individual to group-level emotion recognition: EmotiW 5.0
Dhall, A., Goecke, R., Ghosh, S., Joshi, J., Hoey, J., & Gedeon, T. (2017) · Proceedings of the 19th ACM International Conference on Multimodal Interaction · doi:10.1145/3136755.3143004 · Cited by 218
The more the merrier: Analysing the affect of a group of people in images
Dhall, A., Joshi, J., Sikka, K., Goecke, R., & Sebe, N. (2015) · 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2015.7163151 · Cited by 93 · cited 1×
“In their work [7], global features contain scene features related to factors external to group members’ characteristics while local features contain face expressions, face attributes which related to intrinsic characteristics of the individuals in the group.”
BackgroundYang, J., Wang, K., Peng, X., & Qiao, Y. (2018) · Proceedings of the 20th ACM International Conference on Multimodal Interaction · doi:10.1145/3242969.3264981 · Cited by 79 · cited 1×
“Inspire of some advanced applications of image [11]and video [22].”
BackgroundSelf-citationGroup emotion recognition with individual facial emotion CNNs and global image based CNNs
Tan, L., Zhang, K., Wang, K., Zeng, X., Peng, X., & Qiao, Y. (2017) · Proceedings of the 19th ACM International Conference on Multimodal Interaction · doi:10.1145/3136755.3143008 · Cited by 73 · cited 4×
“Inspire by [18], we keep all the faces for emotion recognition in the test step.”
BackgroundMethodInfluentialSelf-citationHappiness level prediction with sequential inputs via multiple regressions
Li, J., Roy, S., Feng, J., & Sim, T. (2016) · Proceedings of the 18th ACM International Conference on Multimodal Interaction · doi:10.1145/2993148.2997636 · Cited by 47
Cited by
Suppressing Uncertainties for Large-Scale Facial Expression Recognition
Wang, K., Peng, X., Yang, J., Lu, S., & Qiao, Y. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00693 · Cited by 772
Self-citationYang, J., Wang, K., Peng, X., & Qiao, Y. (2018) · Proceedings of the 20th ACM International Conference on Multimodal Interaction · doi:10.1145/3242969.3264981 · Cited by 79
“In MOOCs, automatic engagement prediction can be based on kinds of data modalities such as student response [10], facial [14, 16] or body movements in learning videos [6, 18], behavior in test quizzes [9] and even advanced physiological and neurological measures [3, 8].”
BackgroundSelf-citationEMERSK -- Explainable Multimodal Emotion Recognition with Situational Knowledge
Palash, M., & Bhargava, B. (2023) · arXiv · doi:10.48550/arxiv.2306.08657 · Cited by 25
“Researchers in [38]–[40] showed the interaction between individual and group emotions and proposed several methods to determine the group emotion from individual emotion using face, body and other cues.”
Metadata
- Authors
- Wang, K., Zeng, X., Yang, J., Meng, D., Zhang, K., Peng, X., & Qiao, Y.
- Venue
- Proceedings of the 20th ACM International Conference on Multimodal Interaction
- Publisher
- ACM
- Published
- 2018-10-02
- Pages
- 640-645
- OpenAlex
- W2895443688
- Cited by
- 44
- References
- 10
- FWCI
- 8.28
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.