The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Facial emotion recognition using deep learning: review and insights
Open accessAbstract
Automatic emotion recognition based on facial expression is an interesting research field, which has presented and applied in several areas such as safety, health and in human machine interfaces. Researchers in this field are interested in developing techniques to interpret, code facial expressions and extract these features in order to have a better prediction by computer. With the remarkable success of deep learning, the different types of architectures of this technique are exploited to achieve a better performance. The purpose of this paper is to make a study on recent works on automatic facial emotion recognition FER via deep learning. We underline on these contributions treated, the architecture and the databases used and we present the progress made by comparing the proposed methods and the results obtained. The interest of this paper is to serve and guide researchers by review recent works and providing insights to make improvements to this field.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303
Presentation and validation of the Radboud Faces Database
Langner, O., Dotsch, R., Bijlstra, G., Wigboldus, D. H. J., Hawk, S. T., & van Knippenberg, A. (2010) · Cognition and Emotion · doi:10.1080/02699930903485076 · Cited by 2,420
AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2740923 · Cited by 2,338
Facial expression recognition based on Local Binary Patterns: A comprehensive study
Shan, C., Gong, S., & McOwan, P. W. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.08.005 · Cited by 2,300
Challenges in Representation Learning: A Report on Three Machine Learning Contests
Goodfellow, I. J., Erhan, D., Carrier, P. L., Courville, A., Mirza, M., Hamner, B., Cukierski, W., Tang, Y., Thaler, D., Lee, D.-H., et al. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-42051-1_16 · Cited by 1,719
Gross, R., Matthews, I., Cohn, J., Kanade, T., & Baker, S. (2010) · Image and Vision Computing · doi:10.1016/j.imavis.2009.08.002 · Cited by 1,621
Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild
Li, S., Deng, W., & Du, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.277 · Cited by 1,470
A 3D Facial Expression Database For Facial Behavior Research
Yin, L., Wei, X., Sun, Y., Wang, J., & Rosato, M. J. (2006) · 7th International Conference on Automatic Face and Gesture Recognition (FGR06) · doi:10.1109/fgr.2006.6 · Cited by 1,171
Going deeper in facial expression recognition using deep neural networks
Mollahosseini, A., Chan, D., & Mahoor, M. H. (2016) · 2016 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2016.7477450 · Cited by 1,086
Web-Based Database for Facial Expression Analysis
Pantic, M., Valstar, M., Rademaker, R., & Maat, L. (2005) · 2005 IEEE International Conference on Multimedia and Expo · doi:10.1109/icme.2005.1521424 · Cited by 1,039
CASME II: An Improved Spontaneous Micro-Expression Database and the Baseline Evaluation
Yan, W.-J., Li, X., Wang, S.-J., Zhao, G., Liu, Y.-J., Chen, Y.-H., & Fu, X. (2014) · PLoS ONE · doi:10.1371/journal.pone.0086041 · Cited by 993
A review of emotion recognition using physiological signals
Shu, L., Xie, J., Yang, M., Li, Z., Li, Z., Liao, D., Xu, X., & Yang, X. (2018) · Sensors · doi:10.3390/s18072074 · Cited by 923
Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism
Li, Y., Zeng, J., Shan, S., & Chen, X. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2886767 · Cited by 916
Lopes, A. T., de Aguiar, E., De Souza, A. F., & Oliveira-Santos, T. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2016.07.026 · Cited by 799
DISFA: A Spontaneous Facial Action Intensity Database
Mavadati, S. M., Mahoor, M. H., Bartlett, K., Trinh, P., & Cohn, J. F. (2013) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2013.4 · Cited by 793
Toward an affect-sensitive multimodal human-computer interaction
Pantic, M., & Rothkrantz, L. J. M. (2003) · Proceedings of the IEEE · doi:10.1109/jproc.2003.817122 · Cited by 791
Facial expression recognition from near-infrared videos
Zhao, G., Huang, X., Taini, M., Li, S. Z., & Pietikäinen, M. (2011) · Image and Vision Computing · doi:10.1016/j.imavis.2011.07.002 · Cited by 778
Automatic Analysis of Facial Affect: A Survey of Registration, Representation, and Recognition
Sariyanidi, E., Gunes, H., & Cavallaro, A. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2014.2366127 · Cited by 685
Static facial expression analysis in tough conditions: Data, evaluation protocol and benchmark
Dhall, A., Goecke, R., Lucey, S., & Gedeon, T. (2011) · 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops) · doi:10.1109/iccvw.2011.6130508 · Cited by 541
Features and classifiers for emotion recognition from speech: a survey from 2000 to 2011
Anagnostopoulos, C.-N., Iliou, T., & Giannoukos, I. (2015) · Artificial Intelligence Review · doi:10.1007/s10462-012-9368-5 · Cited by 428
Extended deep neural network for facial emotion recognition
Jain, D. K., Shamsolmoali, P., & Sehdev, P. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2019.01.008 · Cited by 400
The first facial expression recognition and analysis challenge
Valstar, M. F., Jiang, B., Mehu, M., Pantic, M., & Scherer, K. (2011) · Face and Gesture 2011 · doi:10.1109/fg.2011.5771374 · Cited by 385
Robust Facial Expression Recognition Based on Local Directional Pattern
Jabid, T. (2010) · ETRI Journal · doi:10.4218/etrij.10.1510.0132 · Cited by 356
Agrawal, A., & Mittal, N. (2020) · The Visual Computer · doi:10.1007/s00371-019-01630-9 · Cited by 293
Kim, D. H., Baddar, W. J., Jang, J., & Ro, Y. M. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2695999 · Cited by 257
FERA 2015 - second Facial Expression Recognition and Analysis challenge
Valstar, M. F., Almaev, T., Girard, J. M., McKeown, G., Mehu, M., Yin, L., Pantic, M., & Cohn, J. F. (2015) · 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2015.7284874 · Cited by 252
Deep learning for human affect recognition: Insights and new developments
Rouast, P. V., Adam, M. T. P., & Chiong, R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2890471 · Cited by 243
Prediction of asynchronous dimensional emotion ratings from audiovisual and physiological data
Ringeval, F., Eyben, F., Kroupi, E., Yuce, A., Thiran, J.-P., Ebrahimi, T., Lalanne, D., & Schuller, B. (2015) · Pattern Recognition Letters · doi:10.1016/j.patrec.2014.11.007 · Cited by 202
Spatio-temporal convolutional features with nested LSTM for facial expression recognition
Yu, Z., Liu, G., Liu, Q., & Deng, J. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.07.028 · Cited by 162
Survey on AI-Based Multimodal Methods for Emotion Detection
Marechal, C., Mikołajewski, D., Tyburek, K., Prokopowicz, P., Bougueroua, L., Ancourt, C., & Węgrzyn-Wolska, K. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-16272-6_11 · Cited by 137
JAFFE (Deprecated, use v.2 instead)
Lyons, M. J., Kamachi, M., & Gyoba, J. (1997) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.3451523 · Cited by 122
Yolcu, G., Öztel, İ., Kazan, S., Öz, C., Palaniappan, K., Lever, T. E., & Bunyak, F. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-019-07959-6 · Cited by 121
Multimodal Deep Convolutional Neural Network for Audio-Visual Emotion Recognition
Zhang, S., Zhang, S. S., Huang, T., & Gao, W. (2016) · International Conference on Multimedia Retrieval · doi:10.1145/2911996.2912051 · Cited by 110
Deep convolutional BiLSTM fusion network for facial expression recognition
Liang, D., Liang, H., Yu, Z., & Zhang, Y. (2019) · The Visual Computer · doi:10.1007/s00371-019-01636-3 · Cited by 105
Facial emotion recognition using deep convolutional networks
Mohammadpour, M., Khaliliardali, H., Hashemi, S. M. R., & AlyanNezhadi, M. M. (2017) · International Conference on Knowledge-Based Engineering and Innovation · doi:10.1109/kbei.2017.8324974 · Cited by 87
Facial Expression Recognition Method Based on Sparse Batch Normalization CNN
Cai, J., Chang, O., Tang, X., Xue, C., & Wei, C. (2018) · Chinese Control Conference (CCC) · doi:10.23919/chicc.2018.8483567 · Cited by 51
Blend Shape Interpolation and FACS for Realistic Avatar
Alkawaz, M. H., Mohamad, D., Basori, A. H., & Saba, T. (2015) · 3D Research · doi:10.1007/s13319-015-0038-7 · Cited by 50
Facial expression recognition based on Gabor wavelets and sparse representation
Zhang, S., Li, L., & Zhao, Z. (2012) · International Conference on Signal Processing · doi:10.1109/icosp.2012.6491706 · Cited by 36
Cited by
Multimodal Emotion Recognition using Deep Learning
Abdullah, S. M. S. A., Ameen, S. Y. A., M. Sadeeq, M. A., & Zeebaree, S. (2021) · Journal of Applied Science and Technology Trends · doi:10.38094/jastt20291 · Cited by 347
Emotion recognition in EEG signals using deep learning methods: A review
Jafari, M., Shoeibi, A., Khodatars, M., Bagherzadeh, S., Shalbaf, A., García, D. L., Gorriz, J. M., & Acharya, U. R. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.107450 · Cited by 240
Metadata
- Authors
- Mellouk, W., & Handouzi, W.
- Venue
- Procedia Computer Science
- Publisher
- Elsevier BV
- Published
- 2020
- Volume
- 175
- Pages
- 689-694
- ISSN
- 1877-0509 (ISSN-L)
- OpenAlex
- W3048297990
- Cited by
- 339
- References
- 38
- FWCI
- 40.43
- Type
- conference paper
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.