The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index.
Joint Pose and Expression Modeling for Facial Expression Recognition
Abstract
Facial expression recognition (FER) is a challenging task due to different expressions under arbitrary poses. Most conventional approaches either perform face frontalization on a non-frontal facial image or learn separate classifiers for each pose. Different from existing methods, in this paper, we propose an end-to-end deep learning model by exploiting different poses and expressions jointly for simultaneous facial image synthesis and pose-invariant facial expression recognition. The proposed model is based on generative adversarial network (GAN) and enjoys several merits. First, the encoder-decoder structure of the generator can learn a generative and discriminative identity representation for face images. Second, the identity representation is explicitly disentangled from both expression and pose variations through the expression and pose codes. Third, our model can automatically generate face images with different expressions under arbitrary poses to enlarge and enrich the training set for FER. Quantitative and qualitative evaluations on both controlled and in-the-wild datasets demonstrate that the proposed algorithm performs favorably against state-of-the-art methods.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Unsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation
Charles, R. Q., Su, H., Kaichun, M., & Guibas, L. J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.16 · Cited by 9,237
Conditional Generative Adversarial Nets
Mirza, M., & Osindero, S. (2014) · arXiv · doi:10.48550/arxiv.1411.1784 · Cited by 8,863
Ekman, P. (1976) · Medical Entomology and Zoology · Cited by 4,345
A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions
Zeng, Z., Pantic, M., Roisman, G. I., & Huang, T. S. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.52 · Cited by 2,797
Gross, R., Matthews, I., Cohn, J., Kanade, T., & Baker, S. (2010) · Image and Vision Computing · doi:10.1016/j.imavis.2009.08.002 · Cited by 1,621
Unsupervised Pixel-Level Domain Adaptation with Generative Adversarial Networks
Bousmalis, K., Silberman, N., Dohan, D., Erhan, D., & Krishnan, D. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.18 · Cited by 1,603
A 3D Facial Expression Database For Facial Behavior Research
Yin, L., Wei, X., Sun, Y., Wang, J., & Rosato, M. J. (2006) · 7th International Conference on Automatic Face and Gesture Recognition (FGR06) · doi:10.1109/fgr.2006.6 · Cited by 1,171
Age Progression/Regression by Conditional Adversarial Autoencoder
Zhang, Z., Song, Y., & Qi, H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.463 · Cited by 1,145
Disentangled Representation Learning GAN for Pose-Invariant Face Recognition
Tran, L., Yin, X., & Liu, X. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.141 · Cited by 1,005
Perceptual Generative Adversarial Networks for Small Object Detection
Li, J., Liang, X., Wei, Y., Xu, T., Feng, J., & Yan, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.211 · Cited by 953 · cited 1×
“Besides, during the last three years, several approaches [4, 65, 55, 25, 29] have been proposed to improve the original GAN from different perspectives.”
BackgroundEnergy-based Generative Adversarial Network
Zhao, J., Mathieu, M., & LeCun, Y. (2016) · arXiv · doi:10.48550/arxiv.1609.03126 · Cited by 912 · cited 1×
“In recent times, GAN-based approaches have been successfully used to generate impressively realistic faces [26, 20], house-numbers [60], bedrooms [35] and a variety of other image categories [15, 65] through a two-player game between a generator G and discriminator D.”
BackgroundTowards Accurate Multi-person Pose Estimation in the Wild
Papandreou, G., Zhu, T., Kanazawa, N., Toshev, A., Tompson, J., Bregler, C., & Murphy, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.395 · Cited by 860
Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition
Jung, H., Lee, S., Yim, J., Park, S., & Kim, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.341 · Cited by 781 · cited 2×
“, Support Vector Machines (SVMs) [16], softmax [18], and logistic regression [36], to train a facial expression recognizer for a target expression.”
BackgroundGeneralized Multiview Analysis: A discriminative latent space
Sharma, A., Kumar, A., Daume, H., & Jacobs, D. W. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247923 · Cited by 714
Facial Expression Recognition via a Boosted Deep Belief Network
Liu, P., Han, S., Meng, Z., & Tong, Y. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.233 · Cited by 650
Benitez-Quiroz, C. F., Srinivasan, R., & Martinez, A. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.600 · Cited by 607
Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark
Dhall, A., Goecke, R., Lucey, S., & Gedeon, T. (2011) · 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) · doi:10.1109/iccvw.2011.6130508 · Cited by 541 · cited 4×
“The Static Facial Expressions in the wild (SFEW) dataset [7] contains only 700 images (including both training and testing) while the Multi-PIE [13] has 7,655 images (5 poses and 6 expressions).”
BackgroundMethodInfluentialCorneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527
Facial Expression Recognition Based on Deep Evolutional Spatial-Temporal Networks
Zhang, K., Huang, Y., Du, Y., & Wang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2689999 · Cited by 487
Unsupervised Feature Learning and Deep Learning: A Review and New Perspectives
Bengio, Y., Courville, A. C., & Vincent, P. (2012) · Cited by 454
Learning active facial patches for expression analysis
Zhong, L., Liu, Q., Yang, P., Liu, B., Huang, J., & Metaxas, D. N. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247974 · Cited by 369
Local binary patterns for multi-view facial expression recognition
Moore, S., & Bowden, R. (2011) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2010.12.001 · Cited by 351
A Comprehensive Survey on Pose-Invariant Face Recognition
Ding, C., & Tao, D. (2016) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/2845089 · Cited by 329
A Deep Neural Network-Driven Feature Learning Method for Multi-view Facial Expression Recognition
Zhang, T., Zheng, W., Cui, Z., Zong, Y., Yan, J., & Yan, K. (2016) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2016.2598092 · Cited by 314
Self-citationThe Unreasonable Effectiveness of Noisy Data for Fine-Grained Recognition
Krause, J., Sapp, B., Howard, A., Zhou, H., Toshev, A., Duerig, T., Philbin, J., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_19 · Cited by 296
Do Deep Neural Networks Learn Facial Action Units When Doing Expression Recognition?
Khorrami, P., Paine, T. L., & Huang, T. S. (2015) · 2015 IEEE International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2015.12 · Cited by 285
DeLiGAN: Generative Adversarial Networks for Diverse and Limited Data
Gurumurthy, S., Sarvadevabhatla, R. K., & Babu, R. V. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.525 · Cited by 277
Tian, Y.-L., Kanade, T., & Cohn, J. F. (2003) · Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition · doi:10.1109/afgr.2002.1004159 · Cited by 223
Disentangling Factors of Variation for Facial Expression Recognition
Rifai, S., Bengio, Y., Courville, A., Vincent, P., & Mirza, M. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33783-3_58 · Cited by 215
Eleftheriadis, S., Rudovic, O., & Pantic, M. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2375634 · Cited by 206
Multi-view Discriminant Analysis
Kan, M., Shan, S., Zhang, H., Lao, S., & Chen, X. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-33718-5_58 · Cited by 196
Selective Transfer Machine for Personalized Facial Expression Analysis
Chu, W.-S., De la Torre, F., & Cohn, J. F. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2547397 · Cited by 192
Learning Multi-Task Correlation Particle Filters for Visual Tracking
Zhang, T., Xu, C., & Yang, M.-H. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2797062 · Cited by 173
Self-citationCoupled Gaussian processes for pose-invariant facial expression recognition
Rudovic, O., Pantic, M., & Patras, I. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.233 · Cited by 159
Multi-View Facial Expression Recognition Based on Group Sparse Reduced-Rank Regression
Zheng, W. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2304712 · Cited by 145
Dynamic Facial Analysis: From Bayesian Filtering to Recurrent Neural Network
Gu, J., Yang, X., Mello, S. D., & Kautz, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.167 · Cited by 132
Sangineto, E., Zen, G., Ricci, E., & Sebe, N. (2014) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2647868.2654916 · Cited by 128
Facial expression recognition via deep learning
Lv, Y., Feng, Z., & Xu, C. (2014) · 2014 International Conference on Smart Computing · doi:10.1109/smartcomp.2014.7043872 · Cited by 122
Self-citationFacial Expression Intensity Estimation Using Ordinal Information
Zhao, R., Gan, Q., Wang, S., & Ji, Q. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.377 · Cited by 113
Deep Future Gaze: Gaze Anticipation on Egocentric Videos Using Adversarial Networks
Zhang, M., Ma, K. T., Lim, J., Zhao, Q., & Feng, J. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.377 · Cited by 104
Robust Structural Sparse Tracking
Zhang, T., Xu, C., & Yang, M. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2797082 · Cited by 102
Self-citationGenerating Facial Expressions with Deep Belief Nets
Joshua, M., Geoffrey, E., Javier, R. A. V., & Adam, K. (2008) · doi:10.5772/6167 · Cited by 95
Generative Attribute Controller with Conditional Filtered Generative Adversarial Networks
Kaneko, T., Hiramatsu, K., & Kashino, K. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.741 · Cited by 91
A novel approach to expression recognition from non-frontal face images
Zheng, W., Tang, H., Lin, Z., & Huang, T. S. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459421 · Cited by 76
Hierarchical Bayesian Theme Models for Multipose Facial Expression Recognition
Mao, Q., Rao, Q., Yu, Y., & Dong, M. (2016) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2016.2629282 · Cited by 76
Self-citationLearning Bases of Activity for Facial Expression Recognition
Sariyanidi, E., Gunes, H., & Cavallaro, A. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2662237 · Cited by 73
Robust Real-Time Face Pose and Facial Expression Recovery
Zhu, Z., & Ji, Q. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.259 · Cited by 60
Non-frontal view facial expression recognition based on ergodic hidden Markov model supervectors
Tang, H., Hasegawa‐Johnson, M., & Huang, T. S. (2010) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2010.5582576 · Cited by 48
Regression-Based Multi-view Facial Expression Recognition
Rudovic, O. O., Patras, I., & Pantić, M. (2010) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2010.1001 · Cited by 41
Supervised super-vector encoding for facial expression recognition
Tariq, U., Yang, J., & Huang, T. S. (2014) · Pattern Recognition Letters · doi:10.1016/j.patrec.2014.05.011 · Cited by 32
Maximum margin GMM learning for facial expression recognition
Tariq, U., Yang, J., & Huang, T. S. (2013) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2013.6553794 · Cited by 32
Cited by
Deep Facial Expression Recognition: A Survey
Li, S., & Deng, W. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981446 · Cited by 1,753
RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild
Deng, J., Guo, J., Ververas, E., Kotsia, I., & Zafeiriou, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00525 · Cited by 1,681
RetinaFace: Single-stage Dense Face Localisation in the Wild
Deng, J., Guo, J., Zhou, Y., Yu, J., Kotsia, I., & Zafeiriou, S. P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.00641 · Cited by 391
Zhao, Z., Liu, Q., & Wang, S. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3093397 · Cited by 375
FSA-Net: Learning Fine-Grained Structure Aggregation for Head Pose Estimation From a Single Image
Yang, T.-Y., Chen, Y.-T., Lin, Y.-Y., & Chuang, Y.-Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00118 · Cited by 347
Metadata
- Authors
- Zhang, F., Zhang, T., Mao, Q., & Xu, C.
- Venue
- 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2018-06
- Pages
- 3359-3368
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2798553619
- Cited by
- 256
- References
- 61
- FWCI
- 12.15
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.