The DOI matches its Crossref record; 53 of the 53 references it cites were found in a registry or an index.
Going deeper in facial expression recognition using deep neural networks
Open accessAbstract
Automated Facial Expression Recognition (FER) has remained a challenging and interesting problem in computer vision. Despite efforts made in developing various methods for FER, existing approaches lack generalizability when applied to unseen images or those that are captured in wild setting (i.e. the results are not significant). Most of the existing approaches are based on engineered features (e.g. HOG, LBPH, and Gabor) where the classifier's hyper-parameters are tuned to give best recognition accuracies across a single database, or a small collection of similar databases. This paper proposes a deep neural network architecture to address the FER problem across multiple well-known standard face datasets. Specifically, our network consists of two convolutional layers each followed by max pooling and then four Inception layers. The network is a single component architecture that takes registered facial images as the input and classifies them into either of the six basic or the neutral expressions. We conducted comprehensive experiments on seven publicly available facial expression databases, viz. MultiPIE, MMI, CK+, DISFA, FERA, SFEW, and FER2013. The results of our proposed architecture are comparable to or better than the state-of-the-art methods and better than traditional convolutional neural networks in both accuracy and training time.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
(2014) · Cited by 33,419
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. (2014) · Proceedings of the 22nd ACM international conference on Multimedia · doi:10.1145/2647868.2654889 · Cited by 11,077
DeepFace: Closing the Gap to Human-Level Performance in Face Verification
Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.220 · Cited by 6,542
Ekman, P., & Friesen, W. V. (2019) · PsycTESTS Dataset · doi:10.1037/t27734-000 · Cited by 6,019
Constants across cultures in the face and emotion.
Ekman, P., & Friesen, W. V. (1971) · Journal of Personality and Social Psychology · doi:10.1037/h0030377 · Cited by 5,509
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303
Caffe: Convolutional Architecture for Fast Feature Embedding
Jia, Y., Shelhamer, E., Donahue, J., Karayev, S., Long, J., Girshick, R., Guadarrama, S., & Darrell, T. J. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1408.5093 · Cited by 3,752
DeepPose: Human Pose Estimation via Deep Neural Networks
Toshev, A., & Szegedy, C. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.214 · Cited by 3,185
Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions
Zhao, G., & Pietikainen, M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1110 · Cited by 2,846
Facial expression recognition based on Local Binary Patterns: A comprehensive study
Shan, C., Gong, S., & McOwan, P. W. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.08.005 · Cited by 2,300
Coding facial expressions with Gabor wavelets
Lyons, M., Akamatsu, S., Kamachi, M., & Gyoba, J. (2002) · Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.1998.670949 · Cited by 2,210
Supervised Descent Method and Its Applications to Face Alignment
Xiong, X., & De la Torre, F. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.75 · Cited by 1,896
Liu, C., & Wechsler, H. (2002) · IEEE Transactions on Image Processing · doi:10.1109/tip.2002.999679 · Cited by 1,886
Gross, R., Matthews, I., Cohn, J., Kanade, T., & Baker, S. (2010) · Image and Vision Computing · doi:10.1016/j.imavis.2009.08.002 · Cited by 1,621
Going Deeper with Convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2014) · arXiv · doi:10.48550/arxiv.1409.4842 · Cited by 1,461
Web-Based Database for Facial Expression Analysis
Pantic, M., Valstar, M., Rademaker, R., & Maat, L. (2005) · 2005 IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2005.1521424 · Cited by 1,039
DeepID3: Face Recognition with Very Deep Neural Networks
Sun, Y. J., Liang, D., Wang, X., & Tang, X. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1502.00873 · Cited by 842
DISFA: A Spontaneous Facial Action Intensity Database
Mavadati, S. M., Mahoor, M. H., Bartlett, K., Trinh, P., & Cohn, J. F. (2013) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2013.4 · Cited by 793
Self-citationDeep Learning using Linear Support Vector Machines
Tang, Y. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1306.0239 · Cited by 708
Bal, E., Harden, E., Lamb, D., Van Hecke, A. V., Denver, J. W., & Porges, S. W. (2010) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-009-0884-3 · Cited by 575
Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark
Dhall, A., Goecke, R., Lucey, S., & Gedeon, T. (2011) · 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) · doi:10.1109/iccvw.2011.6130508 · Cited by 541 · cited 10×
“We evaluate the proposed method on well-known publicly available facial expression databases: CMU MultiPIE [15], MMI [36], Denver Intensity of Spontaneous Facial Actions (DISFA) [29], extended CK+ [27], GEMEPFERA database [5], SFEW [10], and FER2013 [1].”
MethodInfluentialRecognition of Facial Expression from Optical Flow
Mase, K. (1991) · IEICE Transactions on Information and Systems · Cited by 481
Observer-Based Measurement of Facial Expression with the Facial Action Coding System
Cohn, J. F., Ambadar, Z., & Ekman, P. (2007) · Handbook of Emotion Elicitation and Assessment · doi:10.1093/oso/9780195169157.003.0014 · Cited by 393
Combining modality specific deep neural networks for emotion recognition in video
Kahou, S. E., Pal, C., Bouthillier, X., Froumenty, P., Gülçehre, Ç., Memisevic, R., Vincent, P., Courville, A., Bengio, Y., Ferrari, R. C., et al. (2013) · Proceedings of the 15th ACM on International conference on multimodal interaction · doi:10.1145/2522848.2531745 · Cited by 391
The first facial expression recognition and analysis challenge
Valstar, M. F., Jiang, B., Mehu, M., Pantic, M., & Scherer, K. (2011) · Face and Gesture 2011 · doi:10.1109/fg.2011.5771374 · Cited by 385
Deeply Learning Deformable Facial Action Parts Model for Dynamic Expression Analysis
Liu, M., Li, S., Shan, S., Wang, R., & Chen, X. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-16817-3_10 · Cited by 329
(2009) · Encyclopedia of Biometrics · doi:10.1007/978-0-387-73003-5_2171 · Cited by 265
Provable Bounds for Learning Some Deep Representations
Arora, S., Bhaskara, A., Ge, R., & Ma, T. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1310.6343 · Cited by 240
AU-aware Deep Networks for facial expression recognition
Liu, M., Li, S., Shan, S., & Chen, X. (2013) · 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2013.6553734 · Cited by 218
Introducing the Geneva Multimodal Emotion Portrayal (GEMEP) corpus
Bänziger, & Scherer, T. A. (2010) · Cited by 209
Eleftheriadis, S., Rudovic, O., & Pantic, M. (2015) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2375634 · Cited by 206
Combining Multiple Kernel Methods on Riemannian Manifold for Emotion Recognition in the Wild
Liu, M., Wang, R., Li, S., Shan, S., Huang, Z., & Chen, X. (2014) · Proceedings of the 16th International Conference on Multimodal Interaction · doi:10.1145/2663204.2666274 · Cited by 202
de la Torre, F. N., & Cohn, J. F. (2011) · Springer eBooks · doi:10.1007/978-0-85729-997-0_19 · Cited by 188
Emotion recognition in the wild challenge 2013
Dhall, A., Goecke, R., Joshi, J., Wagner, M., & Gedeon, T. (2013) · Proceedings of the 15th ACM on International conference on multimodal interaction · doi:10.1145/2522848.2531739 · Cited by 178
Lee, S. H., Plataniotis, K. N., & Ro, Y. M. (2014) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2346515 · Cited by 124
Facial interaction between animated 3D face robot and human beings
Kobayashi, H., & Hara, F. (2002) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.1997.633250 · Cited by 121
Motion history for facial action detection in video
Valstar, M., Pantić, M., & Patras, I. (2005) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2004.1398371 · Cited by 113
PCA-based dictionary building for accurate facial expression recognition via sparse representation
Mohammadi, M. R., Fatemizadeh, E., & Mahoor, M. H. (2014) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2014.03.006 · Cited by 109
Self-citationStructure-Preserving Sparse Decomposition for Facial Expression Analysis
Taheri, S., Qiu, Q., & Chellappa, R. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2331141 · Cited by 80
Facial expression recognition using $${l}_{p}$$ l p -norm MKL multiclass-SVM
Zhang, X., Mahoor, M. H., & Mavadati, S. M. (2015) · Machine Vision and Applications · doi:10.1007/s00138-015-0677-y · Cited by 80
Self-citationFacial Expression Recognition Based on Local Phase Quantization and Sparse Representation
Wang, Z., & Ying, Z. (2012) · International Conference on Natural Computation (ICNC) · doi:10.1109/icnc.2012.6234551 · Cited by 78
Cross-database evaluation for facial expression recognition
Mayer, C., Eggers, M., & Radig, B. (2014) · Pattern Recognition and Image Analysis · doi:10.1134/s1054661814010106 · Cited by 55
Cross-Domain Facial Expression Recognition Using Supervised Kernel Mean Matching
Miao, Y.-Q., Araujo, R., & Kamel, M. S. (2012) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2012.178 · Cited by 43
Intensity Estimation of Spontaneous Facial Action Units Based on Their Sparsity Properties
Mohammadi, M. R., Fatemizadeh, E., & Mahoor, M. H. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2416317 · Cited by 27
Self-citationFacial expression recognition using lp-norm MKL multiclass-SVM
Zhang, X., Mahoor, M. H., & Mavadati, S. M. (2015) · Cited by 27
Self-citationExpressionBot: An emotive lifelike robotic face for face-to-face communication
Mollahosseini, A., Graitzer, G., Borts, E., Conyers, S., Voyles, R. M., Cole, R., & Mahoor, M. H. (2014) · IEEE-RAS International Conference on Humanoid Robots (Humanoids) · doi:10.1109/humanoids.2014.7041505 · Cited by 18
Self-citationBidirectional Warping of Active Appearance Model
Mollahosseini, A., & Mahoor, M. H. (2013) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2013.129 · Cited by 9
Self-citationeBear: An expressive Bear-Like robot
Zhang, X., Mollahosseini, A., B., A. H. K., et al. (2014) · IEEE RO-MAN · doi:10.1109/roman.2014.6926378 · Cited by 7
Self-citationBidirectional warping of Active Appearance Model
Mollahosseini, A., Mahoor, M. H., & Shahbazkia, H. R. (2012) · IEEE International Conference on Development and Learning (ICDL) · doi:10.1109/devlrn.2012.6400816 · Cited by 3
Self-citation
Cited by
AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2740923 · Cited by 2,338
Self-citationDeep Facial Expression Recognition: A Survey
Li, S., & Deng, W. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981446 · Cited by 1,753
Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild
Li, S., Deng, W., & Du, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.277 · Cited by 1,470
Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism
Li, Y., Zeng, J., Shan, S., & Chen, X. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2886767 · Cited by 916
A Brief Review of Facial Emotion Recognition Based on Visual Information
Ko, B. (2018) · Sensors · doi:10.3390/s18020401 · Cited by 724
Metadata
- Authors
- Mollahosseini, A., Chan, D., & Mahoor, M. H.
- Venue
- 2016 IEEE Winter Conference on Applications of Computer Vision (WACV)
- Publisher
- IEEE
- Published
- 2016-03
- Pages
- 1-10
- ISSN
- 2472-6737 (ISSN-L)
- OpenAlex
- W2244142460
- Cited by
- 1,086
- References
- 53
- FWCI
- 114.96
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.