The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Former-DFER: Dynamic Facial Expression Recognition Transformer
Abstract
This paper proposes a dynamic facial expression recognition transformer (Former-DFER) for the in-the-wild scenario. Specifically, the proposed Former-DFER mainly consists of a convolutional spatial transformer (CS-Former) and a temporal transformer (T-Former). The CS-Former consists of five convolution blocks and N spatial encoders, which is designed to guide the network to learn occlusion and pose-robust facial features from the spatial perspective. And the temporal transformer consists of M temporal encoders, which is designed to allow the network to learn contextual facial features from the temporal perspective. The heatmaps of the leaned facial features demonstrate that the proposed Former-DFER is capable of handling the issues such as occlusion, non-frontal pose, and head motion. And the visualization of the feature distribution shows that the proposed method can learn more discriminative facial features. Moreover, our Former-DFER also achieves state-of-the-art results on the DFEW and AFEW benchmarks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2010.11929 · Cited by 47,472
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58452-8_13 · Cited by 18,285
End-to-End Object Detection with Transformers
Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., & Zagoruyko, S. (2020) · arXiv · doi:10.48550/arxiv.2005.12872 · Cited by 18,253
Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling
Chung, J., Gulcehre, C., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.3555 · Cited by 11,008
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593
Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset
Carreira, J., & Zisserman, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.502 · Cited by 9,570
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303 · cited 1×
“Towards lab-controlled DFER, the datasets, such as CK+ [48], Oulu-CASIA [76], and MMI [53] are all recorded under laboratory conditions, in which all the facial images are frontal and without any occlusion.”
Method
Research works printed but not linked: 57
- Masih Aminbeidokhti Marco Pedersoli Patrick Cardinal and Eric Granger. 2019. Emotion recognition with spatial attention and temporal softmax pooling. In ICIAR. 323--331. Masih Aminbeidokhti Marco Pedersoli Patrick Cardinal and Eric Granger. 2019. Emotion recognition with spatial attention and temporal softmax pooling. In ICIAR. 323--331. · DOI
- 10.1007/s10639-019-10004-6 · DOI
- Ba Jimmy Lei, 2016, Jamie Ryan Kiros, and Geoffrey E Hinton
- Wissam J Baddar and Yong Man Ro. 2019. Mode variational lstm robust to unseen modes of variation: Application to facial expression recognition. In AAAI. 3215--3223. Wissam J Baddar and Yong Man Ro. 2019. Mode variational lstm robust to unseen modes of variation: Application to facial expression recognition. In AAAI. 3215--3223. · DOI
- Cai Jie, 2019, Zhiyuan Li, James O'Reilly, Shizhong Han, Ping Liu, Min Chen, and Yan Tong.
- Joyati Chattopadhyay Souvik Kundu Arpita Chakraborty and Jyoti Sekhar Banerjee. 2018. Facial expression recognition for human computer interaction. In ICCVBIC. 1181--1192. Joyati Chattopadhyay Souvik Kundu Arpita Chakraborty and Jyoti Sekhar Banerjee. 2018. Facial expression recognition for human computer interaction. In ICCVBIC. 1181--1192. · DOI
- 10.1145/2663204.2666277 · DOI
- 10.1109/TAFFC.2016.2593719 · DOI · Paper page
- Darwin Charles, The expression of the emotions in man and animals
- 10.1145/3340555.3355710 · DOI
- 10.1109/MMUL.2012.26 · DOI · Paper page
- 10.1145/2818346.2830596 · DOI · Paper page
- 10.1145/3242969.3264978 · DOI
- 10.1145/2993148.2997632 · DOI · Paper page
- Rohit Girdhar Joao Carreira Carl Doersch and Andrew Zisserman. 2019. Video action transformer network. In CVPR. 244--253. Rohit Girdhar Joao Carreira Carl Doersch and Andrew Zisserman. 2019. Video action transformer network. In CVPR. 244--253. · DOI
- Kensho Hara Hirokatsu Kataoka and Yutaka Satoh. 2018. Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet?. In CVPR. 6546--6555. Kensho Hara Hirokatsu Kataoka and Yutaka Satoh. 2018. Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet?. In CVPR. 6546--6555. · DOI · Paper page
- Hu Qiaoping, 2020, RFAU: A Database for Facial Action Unit Analysis in Real Classrooms
- 10.1145/2663204.2666278 · DOI
- 10.1145/3394171.3413620 · DOI · Paper page
- 10.1109/TIP.2021.3051462 · DOI · Paper page
- Khan Salman, 2021, Fahad Shahbaz Khan, and Mubarak Shah.
- Jean Kossaifi Antoine Toisoul Adrian Bulat Yannis Panagakis Timothy M Hospedales and Maja Pantic. 2020. Factorized higher-order CNNs with an application to spatio-temporal emotion estimation. In CVPR. 6060--6069. Jean Kossaifi Antoine Toisoul Adrian Bulat Yannis Panagakis Timothy M Hospedales and Maja Pantic. 2020. Factorized higher-order CNNs with an application to spatio-temporal emotion estimation. In CVPR. 6060--6069. · DOI
- Felix Kuhnke Lars Rumberg and Jörn Ostermann. 2020. Two-Stream Aural- Visual Affect Analysis in the Wild. In FG. 366--371. Felix Kuhnke Lars Rumberg and Jörn Ostermann. 2020. Two-Stream Aural- Visual Affect Analysis in the Wild. In FG. 366--371. · DOI
- Vikas Kumar Shivansh Rao and Li Yu. 2020. Noisy Student Training using Body Language Dataset Improves Facial Expression Recognition. In ECCV. 756--773. Vikas Kumar Shivansh Rao and Li Yu. 2020. Noisy Student Training using Body Language Dataset Improves Facial Expression Recognition. In ECCV. 756--773. · DOI
- 10.1109/TIP.2020.2996086 · DOI
- Lee Min Kyu, 2019, Dae Ha Kim, and Byung Cheol Song.
- Li Shan, 2020, Deep facial expression recognition: A survey
- 10.1145/3340555.3355719 · DOI
- Liu Yang, 2021, Graph-based Facial Affect Analysis: A Review of Methods, Applications and Challenges. arXiv preprint arXiv:2103.15599 · arXiv
- 10.1145/3242969.3264992 · DOI
- Debin Meng Xiaojiang Peng Kai Wang and Yu Qiao. 2019. Frame attention networks for facial expression recognition in videos. In ICIP. 3866--3870. Debin Meng Xiaojiang Peng Kai Wang and Yu Qiao. 2019. Frame attention networks for facial expression recognition in videos. In ICIP. 3866--3870. · DOI · Paper page
- Trisha Mittal Uttaran Bhattacharya Rohan Chandra Aniket Bera and Dinesh Manocha. 2020. M3er: Multiplicative multimodal emotion recognition using facial textual and speech cues. In AAAI. 1359--1367. Trisha Mittal Uttaran Bhattacharya Rohan Chandra Aniket Bera and Dinesh Manocha. 2020. M3er: Multiplicative multimodal emotion recognition using facial textual and speech cues. In AAAI. 1359--1367. · DOI · Paper page
- 10.1109/TAFFC.2017.2713783 · DOI · Paper page
- 10.1145/3136755.3143012 · DOI
- Park Jongchan, 2018, BAM: Bottleneck Attention Module. In BMCV.
- 10.5555/3454287.3455008 · doi:10.5555/3454287.3455008
- Zhaofan Qiu Ting Yao and Tao Mei. 2017. Learning spatio-temporal representation with pseudo-3d residual networks. In ICCV. 5533--5541. Zhaofan Qiu Ting Yao and Tao Mei. 2017. Learning spatio-temporal representation with pseudo-3d residual networks. In ICCV. 5533--5541.
- Simonyan Karen, 2014, Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 · arXiv
- Sun Chen, 2019, Carl Vondrick, Kevin Murphy, and Cordelia Schmid.
- Touvron Hugo, 2020, Training data-efficient image transformers & distillation through attention. arXiv preprint arXiv:2012.12877 · arXiv
- der Maaten Laurens Van, 2008, Visualizing data using t-SNE, JMLR, 9, 2579
- 10.5555/3295222.3295349 · doi:10.5555/3295222.3295349
- 10.1145/3136755.3143011 · DOI · Paper page
- 10.1109/TAFFC.2020.3007531 · DOI
- 10.1145/3340555.3355720 · DOI
- 10.1109/CVPR.2013.439 · DOI
- Torsten Wilhelm. 2019. Towards facial expression analysis in a driver assistance system. In FG. 1--4. Torsten Wilhelm. 2019. Towards facial expression analysis in a driver assistance system. In FG. 1--4. · DOI
- Fuzhi Yang Huan Yang Jianlong Fu Hongtao Lu and Baining Guo. 2020. Learning texture transformer network for image super-resolution. In CVPR. 5791--5800. Fuzhi Yang Huan Yang Jianlong Fu Hongtao Lu and Baining Guo. 2020. Learning texture transformer network for image super-resolution. In CVPR. 5791--5800. · DOI · Paper page
- Peng Yang Qingshan Liu Xinyi Cui and Dimitris N Metaxas. 2008. Facial expression recognition using encoded dynamic features. In CVPR. 1--8. Peng Yang Qingshan Liu Xinyi Cui and Dimitris N Metaxas. 2008. Facial expression recognition using encoded dynamic features. In CVPR. 1--8. · DOI
- 10.1016/j.patrec.2008.03.014 · DOI
- Stefanos Zafeiriou Dimitrios Kollias Mihalis A Nicolaou Athanasios Papaioannou Guoying Zhao and Irene Kotsia. 2017. Aff-Wild: valence and arousal 'in-the-wild' challenge. In CVPRW. 34--41. Stefanos Zafeiriou Dimitrios Kollias Mihalis A Nicolaou Athanasios Papaioannou Guoying Zhao and Irene Kotsia. 2017. Aff-Wild: valence and arousal 'in-the-wild' challenge. In CVPRW. 34--41. · DOI · Paper page
- Yuan-Hang Zhang Rulin Huang Jiabei Zeng and Shiguang Shan. 2020. M3F: Multi-Modal Continuous Valence-Arousal Estimation in the Wild. In FG. 617--621. Yuan-Hang Zhang Rulin Huang Jiabei Zeng and Shiguang Shan. 2020. M3F: Multi-Modal Continuous Valence-Arousal Estimation in the Wild. In FG. 617--621. · DOI
- 10.1016/j.imavis.2011.07.002 · DOI · Paper page
- 10.1109/TIP.2021.3093397 · DOI · Paper page
- Zhao Zengqun, 2021, Robust Lightweight Facial Expression Recognition Network with Label Distribution Training, AAAI, 35, 3510, 10.1609/aaai.v35i4.16465 · DOI · Paper page
- 10.1109/TPAMI.2013.173 · DOI
- 10.1109/CVPR.2011.5995345 · DOI · Paper page
Cited by
Zheng, W., Yu, J., Xia, R., & Wang, S. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.861 · Cited by 51
“Moreover, there are also several studies proposing new datasets or methods for facial expression recognition of characters in movies and TV shows (Jiang et al., 2020; Zhao and Liu, 2021; Toisoul et al., 2021; Wang et al., 2022b; Liu et al., 2022b).”
BackgroundChumachenko, K., Iosifidis, A., & Gabbouj, M. (2024) · 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw63382.2024.00470 · Cited by 34
Metadata
- Authors
- Zhao, Z., & Liu, Q.
- Venue
- Proceedings of the 29th ACM International Conference on Multimedia
- Publisher
- ACM
- Published
- 2021-10-17
- Pages
- 1553-1561
- OpenAlex
- W3206349670
- Cited by
- 186
- References
- 22
- FWCI
- 8.32
- Type
- conference paper
- Language
- en
- Open access
- closed
- Funders
- National Natural ScienceFoundation of China (61825601)
- Natural Science Foundation of Jiangsu Province (BK20192004B)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Former-DFER: Dynamic Facial Expression Recognition Transformer. https://app.cochat.ai/papers/10.1145/3474085.3475292
- The paper
- Zhao, Z., & Liu, Q. (2021). Former-DFER: Dynamic Facial Expression Recognition Transformer. Proceedings of the 29th ACM International Conference on Multimedia. https://doi.org/10.1145/3474085.3475292
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.