The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition
Abstract
Temporal information has useful features for recognizing facial expressions. However, to manually design useful features requires a lot of effort. In this paper, to reduce this effort, a deep learning technique, which is regarded as a tool to automatically extract useful features from raw data, is adopted. Our deep network is based on two different models. The first deep network extracts temporal appearance features from image sequences, while the other deep network extracts temporal geometry features from temporal facial landmark points. These two models are combined using a new integration method in order to boost the performance of the facial expression recognition. Through several experiments, we show that the two models cooperate with each other. As a result, we achieve superior performance to other state-of-the-art methods in the CK+ and Oulu-CASIA databases. Furthermore, we show that our new integration method gives more accurate results than traditional methods, such as a weighted summation and a feature concatenation method.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“In benchmarking experiments across various datasets, many studies found that features extracted with CNNs lead to higher recognition accuracies than handcrafted features (e.g., [87], [88], [89], [90], [91], [64], [92], [93], [94]).”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [88]
In the citing paper
“This way, researchers artificially increase the number of available examples or training epochs by a factor typically between 10 and 30 [88], [113], [114], and up to 300 [115].”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [88]
Citations
Cites
(2008) · Journal of Machine Learning Research · Cited by 33,051
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
Improving neural networks by preventing co-adaptation of feature detectors
Hinton, G. E., Srivastava, N., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. R. (2012) · arXiv · doi:10.48550/arxiv.1207.0580 · Cited by 6,382
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303
Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions
Zhao, G., & Pietikainen, M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1110 · Cited by 2,846
Face detection, pose estimation, and landmark localization in the wild
Zhu, X., & Ramanan, D. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6248014 · Cited by 2,171
Supervised Descent Method and Its Applications to Face Alignment
Xiong, X., & la Torre, F. D. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.75 · Cited by 1,926
A Spatio-Temporal Descriptor Based on 3D-Gradients
Klaeser, A., Marszałek, M., & Schmid, C. (2008) · British Machine Vision Conference (BMVC) · doi:10.5244/c.22.99 · Cited by 1,842
A 3-dimensional sift descriptor and its application to action recognition
Scovanner, P., Ali, S., & Shah, M. A. (2007) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1291233.1291311 · Cited by 1,605
Facial expression recognition from near-infrared videos
Zhao, G., Huang, X., Taini, M., Li, S. Z., & Pietikäinen, M. (2011) · Image and Vision Computing · doi:10.1016/j.imavis.2011.07.002 · Cited by 771
Research works printed but not linked: 4
- sanin, 2013, Spatiotemporal covariance descriptors for action and gesture recognition, WACV 2013, 103
- valstar, 2010, Induced disgust, happiness and surprise: an addition to the mmi facial expression database, Proceedings of Int'l Conf Language Resources and Evaluation Workshop on EMOTION, 65
- glorot, 2011, Deep sparse rectifier networks, Proceedings of the 14th International Conference on Artificial Intelligence and Statistics JMLR W&CP Volume, 15, 315
- girard, 2013, Ground truth facs action unit coding on the group formation task, Tech Rep
Cited by
Sorted by citations
Items 201–300 of 779
Facial expression recognition via region-based convolutional fusion network
Ye, Y., Zhang, X., Lin, Y., & Wang, H. (2019) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2019.04.009 · Cited by 37
Orthogonalization-Guided Feature Fusion Network for Multimodal 2D+3D Facial Expression Recognition
Lin, S., Bai, M., Liu, F., Shen, L., & Zhou, Y. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3001497 · Cited by 38
Attend to Where and When: Cascaded Attention Network for Facial Expression Recognition
Qu, X., Zou, Z., Su, X., Zhou, P., Wei, W., Wen, S., & Wu, D. O. (2021) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2021.3070713 · Cited by 37
Compound Facial Expression Recognition Based on Highway CNN
Slimani, K., Lekdioui, K., Messoussi, R., & Touahni, R. (2019) · Conference of the Moroccan Classification Society: New Challenges in Data Sciences · doi:10.1145/3314074.3314075 · Cited by 38
“Most recently, different large databases [18] [24] [26] are available which allow to train neural network in different fields like: multi-view face detection [27] [28] and FER [15] [17].”
BackgroundReal-time Facial Expression Recognition “In The Wild” by Disentangling 3D Expression from Identity
Koujan, M. R., Alharbawee, L., Giannakakis, G. A., Pugeault, N., & Roussos, A. (2020) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg47880.2020.00084 · Cited by 40
“[34] 92.”
Learning Better Representations for Audio-Visual Emotion Recognition with Common Information
Ma, F., Zhang, W., Li, Y., Huang, S., & Zhang, L. (2020) · Applied Sciences · doi:10.3390/app10207239 · Cited by 35
Song, K.-S., Nho, Y., Seo, J.-H., & Kwon, D. (2018) · International Conference on Ubiquitous Robots (UR) · doi:10.1109/urai.2018.8441795 · Cited by 35
“First, the recognition rate of the facial expression accuracy [12][13] of an algorithm that considers only particular types of facial images in which the facial pose is constrained to a frontal or near-frontal view without considering speaking effects, de-”
BackgroundManifold feature integration for micro-expression recognition
Takalkar, M. A., Xu, M. G., & Chaczko, Z. (2020) · Multimedia Systems · doi:10.1007/s00530-020-00663-8 · Cited by 35
“Recent studies of emotion identification technology that uses facial expression (FER) have been seriously researched for advancing human computer interaction (HCI) as social robots became popular [18, 22, 38].”
BackgroundRanjan, R., Partl, R., Erhart, R., Kurup, N. B. R., & Schnidar, H. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104952 · Cited by 35
Facial Expression Recognition Based on Vision Transformer with Hybrid Local Attention
Tian, Y., Zhu, J., Huang, Y., & Chen, D. (2024) · Applied Sciences · doi:10.3390/app14156471 · Cited by 43
Weakly Supervised Local-Global Attention Network for Facial Expression Recognition
Zhang, H., Su, W., & Wang, Z. (2020) · IEEE Access · doi:10.1109/access.2020.2975913 · Cited by 34
“Because of CK+ and Oulu-CASIA do not provide specified training and test sets, we employ the most popular 10-fold validation strategy as in the previous methods [11], [14], [21], [23], [32], [37], [38], [41], [46].”
MethodReview of micro-expression spotting and recognition in video sequences
Pan, H., Xie, L., Wang, Z., Liu, B., Yang, M., & Tao, J. (2021) · Virtual Reality & Intelligent Hardware · doi:10.1016/j.vrih.2020.10.003 · Cited by 37
Orthogonal channel attention-based multi-task learning for multi-view facial expression recognition
Chen, J., Lei, Y., Tan, L., & Xu, R. (2022) · Pattern Recognition · doi:10.1016/j.patcog.2022.108753 · Cited by 34
SG-DSN: A Semantic Graph-based Dual-Stream Network for facial expression recognition
Liu, Y. M., Zhang, X., Zhou, J., & Fu, L. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.017 · Cited by 34
Deep peak-neutral difference feature for facial expression recognition
Chen, J., Xu, R., & Liu, L. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-5909-5 · Cited by 32
“[16] trained a deep temporal-geometry network and a deep temporal-appearance network with facial landmarks and images.”
Hazourli, A. R., Djeghri, A., Salam, H., & Othmani, A. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-020-10332-7 · Cited by 44
“Similar to the PHRNN-MSCNN, the Deep Temporal Appearance-Geometry Network (DTAGN) [29] combines Deep Temporal Geometry Network (DTGN) which extracts temporal geometry features from temporal facial landmark points and Deep Temporal Appearance Network (DTAN) which extracts temporal appearance features from image sequenc…”
MethodGan, C., Xiao, J., Wang, Z., Zhang, Z., & Zhu, Q. (2021) · Image and Vision Computing · doi:10.1016/j.imavis.2021.104342 · Cited by 34
Debiasing Pre-Trained Language Models via Efficient Fine-Tuning
Gira, M., Zhang, R., & Lee, K. (2022) · Workshop on Language Technology for Equality, Diversity, Inclusion (LT-EDI) · doi:10.18653/v1/2022.ltedi-1.8 · Cited by 38
“Fine-tuning, one approach of transfer learning, has been widely used for neural network models (Ge and Yu, 2017; Jung et al., 2015; Maqsood et al., 2019; Shin et al., 2016).”
The limitations for expression recognition in computer vision introduced by facial masks
Abate, A. F., Cimmino, L., Mocanu, B.-C., Narducci, F., & Pop, F. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13559-8 · Cited by 32
“Two types of geometric features are extracted in the approach by Jung et al. [9].”
MethodFacial Expression Recognition in the Wild Using Face Graph and Attention
Kim, H.-J., Lee, J.-H., & Ko, B. C. (2023) · IEEE Access · doi:10.1109/access.2023.3286547 · Cited by 32
“As an example of a typical CNN-based method, Jung et al. [10] improved the FER performance using two different CNN models: CNNs for temporal appearance features and CNNs for temporal geometry features from temporal facial landmark points [12].”
MethodDeep Learning Based Real-Time Driver Emotion Monitoring
Verma, B., & Choudhary, A. (2018) · IEEE International Conference on Vehicular Electronics and Safety (ICVES) · doi:10.1109/icves.2018.8519595 · Cited by 31
“We have shown comparison with [14], [16], [17], [24]–[27]”
BackgroundResultReducing Videoconferencing Fatigue through Facial Emotion Recognition
Rößler, J., Sun, J., & Gloor, P. A. (2021) · Future Internet · doi:10.3390/fi13050126 · Cited by 32
κ-Same-Net: Neural-Network-Based Face Deidentification
Meden, B., Emeršič, Ž., Štruc, V., & Peer, P. (2017) · IEEE International Work Conference on Bioinspired Intelligence (IWOBI) · doi:10.1109/iwobi.2017.7985521 · Cited by 30
“Due to recent advancements in the areas of facial landmarking [15], [37], [33], [29], [13], [38], facial expression recognition [14], [3], [25], [1], [28], [6], age estimation [39], [32], [11], [27], [17], gender recognition [12], [22] and a like [19], [31], there are several techniques with open-source implementation…”
A Graph-Structured Representation with BRNN for Static-based Facial Expression Recognition
Zhong, L., Bai, C., Li, J., Chen, T., Li, S., & Liu, Y. (2019) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2019.8756615 · Cited by 29
Automatic facial expression recognition based on MobileNetV2 in Real-time
Hu, L., & Ge, Q. (2020) · Journal of Physics Conference Series · doi:10.1088/1742-6596/1549/2/022136 · Cited by 30
“Jung et al. [4] proposed a joint fine-tuning method for facial expression recognition to extract the appearance features from image sequences.”
MethodSpatially Coherent Feature Learning for Pose-Invariant Facial Expression Recognition
Zhang, F., Mao, Q., Shen, X., Zhan, Y., & Dong, M. (2018) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3176646 · Cited by 30
“Recently, many researchers combined appearance features and geometric features to learn geometric constraints and improved the ability of pose-invariant FER [20].”
BackgroundAutomatic Analysis of Facial Expressions Based on Deep Covariance Trajectories
Otberdout, N., Kacem, A., Daoudi, M., Ballihi, L., & Berretti, S. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2947244 · Cited by 30
“[30] used a CNN to extract temporal appearance features from face image sequences with an additional deep network that extracts”
MethodRetracted: Artificial Intelligence Based Smart Facial Expression Recognition Remote Control System
Ganesh, R. S., Jausmin, K. J., Srilatha, J., Indumathy, R., Naved, M. I., & Ashok, M. (2021) · International Conference on Computing Methodologies and Communication (ICCMC) · doi:10.1109/iccmc51019.2021.9418031 · Cited by 29
“Jung et al. [30] utilized two unique sorts of CNN: the primary concentrates fleeting manifestation highlights from the picture groupings, though the second concentrates worldly math highlights from transient facial milestone focuses.”
BackgroundRetractedNon-rigid registration based model-free 3D facial expression recognition
Savran, A., & Sankur, B. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.07.005 · Cited by 30
“…of deep learning has shifted the state-of-the-art in acial expression recognition ( Kahou et al., 2013; Ding et al., 2016; aiswal and Valstar, 2016; Jung et al., 2015; Khorrami et al., 2015; im et al., 2015; Levi, 2015; Ng et al., 2015; Zhao et al., 2016 ) from ngineered local feature extractors…”
Normalized face image generation with perceptron generative adversarial networks
Liu, X., Kumar, B. V. K. V., Ge, Y., Yang, C., You, J., & Jia, P. (2018) · IEEE International Conference on Identity, Security and Behavior Analysis (ISBA) · doi:10.1109/isba.2018.8311462 · Cited by 29
“CK+ 97.25% [33], 97.1% [2] 97.49% MMI 75.12% [34], 78.53% [2] 80.26%”
InfluentialFacial Expression Recognition with Data Augmentation and Compact Feature Learning
Lin, F., Hong, R., Zhou, W., & Li, H. (2018) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2018.8451039 · Cited by 29
“The current methods have demonstrated very effectiveness for recognition of facial expression in “lab-controlled” environments [2, 3, 4].”
MethodMing, Z., Chazalon, J., Luqman, M. M., Visani, M., & Burie, J.-C. (2018) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2018.8545274 · Cited by 32
“As well as face verification, the state-of-the-art performance of facial expression recognition has been significantly improved [21]–[23] owing to the deep CNNs.”
BackgroundFacial Expression Recognition Based on Auxiliary Models
Wang, Y., Li, Y., Song, Y. H., & Rong, X. (2019) · Algorithms · doi:10.3390/a12110227 · Cited by 30
“[27] proposed a new CNN method based on two different models.”
MethodDeep Neural Network Architecture: Application for Facial Expression Recognition
Villanueva, M. G., & Zavala, S. R. (2020) · IEEE Latin America Transactions · doi:10.1109/tla.2020.9099774 · Cited by 29
“Muchos de los trabajos que emplean aprendizaje profundo han adaptado una CNN directamente para la detección de los movimientos especı́ficos de los músculos faciales, denominados unidades de acción [13], [14], [15], [16].”
BackgroundDetecting Expressions with Multimodal Transformers
Parthasarathy, S., & Sundaram, S. (2021) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt48900.2021.9383573 · Cited by 29
“Detection methods also vary by task: static recognition on images [23, 24] or continuous prediction that look at sequences of images [25].”
BackgroundZhao, R., Liu, T., Xiao, J., Lun, D. P.-K., & Lam, K. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9413000 · Cited by 29
“37 DTAGN [25] 3 Image sequence 97.”
Zhou, Y., Jin, L., Ma, G., & Xu, X. (2021) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2021.3120513 · Cited by 29
Body expression recognition from animated 3D skeleton
Crenn, A., Khan, R. A., Meyer, A., & Bouakaz, S. (2016) · 2016 International Conference on 3D Imaging (IC3D) · doi:10.1109/ic3d.2016.7823448 · Cited by 34
“In recent years, researchers have considerably focused on the automatic facial expression recognition (FER) [5, 15, 12].”
Learning to Generate 3D Stylized Character Expressions from Humans
Aneja, D., Chaudhuri, B., Colburn, A., Faigin, G., Shapiro, L. G., & Mones, B. (2018) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2018.00024 · Cited by 28
“With recent advancement in deep learning research, Convolutional Neural Networks have shown great improvement in the FER tasks [30, 6, 23, 21, 13, 22] and there are a number of fusion algorithms to boost the recognition performance [42, 46, 38, 44].”
BackgroundShima, Y., & Omori, Y. (2018) · International Conference on Robotics, Control and Automation (ICRCA) · doi:10.1145/3265639.3265664 · Cited by 28
Few-shot fine-grained classification with Spatial Attentive Comparison
Ruan, X., Lin, G., Long, C., & Lu, S. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.106840 · Cited by 28
Region Attention Enhanced Unsupervised Cross-Domain Facial Emotion Recognition
Ji, Y., Hu, Y., Yang, Y., & Shen, H. T. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3136606 · Cited by 31
“Therefore, we propose to detect facial landmark points to depict local regions around five facial sense organs [44], [45], [46], [47], and to learn attention in these regions to enhance emotion features, where the effectiveness of attention learning had been certified in many existing frameworks [48], [49], [50], [51].”
BackgroundDynamic Multi-Task Learning for Face Recognition with Facial ExpressionPreprint
Ming, Z., Xia, J., Luqman, M. M., Burie, J.-C., & Zhao, K. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.03281 · Cited by 40
“The situation is same as facial expression recognition based on deep CNNs [14, 40, 22].”
BackgroundWhen deep learning meets watermarking: A survey of application, attacks and defenses
Chen, H., Liu, C., Zhu, T., & Zhou, W. (2024) · Computer Standards & Interfaces · doi:10.1016/j.csi.2023.103830 · Cited by 28
Two Stream Deep Convolutional Neural Network for Eye State Recognition and Blink Detection
Sanyal, R., & Chakrabarty, K. (2019) · International Conference on Electronics, Materials Engineering & Nano-Technology (IEMENTech) · doi:10.1109/iementech48150.2019.8981102 · Cited by 27
“Inspired by [21], [22] , at training time, we freeze the weight values of the convolutional layers of the pre-trained RGB-CNN and MCNN models and retrain their top layers along with the concatenated and fully connected layer of the joint model.”
MethodNi, X., & Huttunen, H. J. (2020) · Journal of Signal Processing Systems · doi:10.1007/s11265-020-01567-6 · Cited by 30
“A joint fine-tuning method [32] is employed to fine-tune all parameters rather than only the last dense layer.”
MethodMiyoshi, R., Nagata, N., & Hashimoto, M. (2021) · Neural Computing and Applications · doi:10.1007/s00521-020-05557-4 · Cited by 27
Huang, M., Zhang, X., Lan, X., Wang, H., & Tang, Y. D. (2021) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2021.3073558 · Cited by 27
Towards Unbiased Visual Emotion Recognition via Causal Intervention
Chen, Y., Yang, X., Cham, T., & Cai, J. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3547936 · Cited by 30
“, facial landmarks [19, 55, 56], local regions [9, 17, 48], action units [7, 8, 24], optical flows [39], etc.”
BackgroundAlphonse, A. S., & Dharma, D. (2017) · Expert Systems with Applications · doi:10.1016/j.eswa.2017.08.013 · Cited by 27
A discriminative deep association learning for facial expression recognition
Jin, X., Sun, W., & Jin, Z. (2019) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-019-01024-2 · Cited by 27
A review of research on micro-expression recognition algorithms based on deep learning
Zhang, F., & Chai, L. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10262-7 · Cited by 25
Facial Expression Recognition by Multi-Scale CNN with Regularized Center Loss
Li, Z., Wu, S., & Xiao, G. (2018) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2018.8545489 · Cited by 25
Li, X., Zhang, X., Yang, H., Duan, W., Dai, W., & Yin, L. (2020) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg47880.2020.00050 · Cited by 25
Late Fusion-Based Video Transformer for Facial Micro-Expression Recognition
Hong, J., Lee, C., & Jung, H. (2022) · Applied Sciences · doi:10.3390/app12031169 · Cited by 27
“We used optical flow as a motion feature to complement the lack of data [15].”
MethodSelf-citationMaeng, J., Kang, D.-H., & Kim, D. (2020) · Electronics · doi:10.3390/electronics9121988 · Cited by 26
“Emotions are categorized by recognizing changes in facial expressions based mainly on the features obtained from facial images [3,4].”
BackgroundImproved curriculum learning using SSM for facial expression recognition
Liu, X., & Zhou, F. (2019) · The Visual Computer · doi:10.1007/s00371-019-01759-7 · Cited by 24
Multi-Pose Facial Expression Recognition Based on Generative Adversarial Network
Li, D., Li, Z., Luo, R., Deng, J. H., & Sun, S. (2019) · IEEE Access · doi:10.1109/access.2019.2945423 · Cited by 24
“[48] jointly trained CNNs with facial landmarks and color images.”
MethodWang, Y., Li, Y., Song, Y. H., & Rong, X. (2019) · IEEE Access · doi:10.1109/access.2019.2961161 · Cited by 24
“[15] proposed a new CNN method based on two different models.”
MethodPyFER: A Facial Expression Recognizer Based on Convolutional Neural Networks
Kabakuş, A. T. (2020) · IEEE Access · doi:10.1109/access.2020.3012703 · Cited by 24
“[16] proposed a deep neural network that was based on two different models.”
BackgroundMethodIdentity-aware Facial Expression Recognition in Compressed Video
Liu, X., Jin, L., Han, X., Lu, J., You, J., & Kong, L. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412820 · Cited by 24
“Accuracy Landmarks Test speed STM-ExpLet (2014) [51] 94.19 × - LOMo (2016) [52] 95.10 - DTAGN (2015) [53] 97.25 - PHRNN-MSCNN (2017) [54] 98.50 - (N+M)-tuplet (2018) [55] 93.90 12fps C3D-GRU (2019) [56] 97.25 × - CTSLSTM (2019) [57] 93.9 - SC (2019) [58] 97.60 - G2-VER (2019) [59] 97.40 × - LBVCNN (2019) [47] 97.38 ×…”
InfluentialA Self-Fusion Network Based on Contrastive Learning for Group Emotion Recognition
Wang, X., Zhang, D., Tan, H., & Lee, D.-J. (2022) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2022.3202249 · Cited by 23
“Individual emotion recognition (IER) focuses on classifying the image [4], [5], video [6], [7], audio [8], [9], or biological signals [10] of an individual into typical emotions, e.”
BackgroundSudha, S., & Suganya, S. S. (2023) · Measurement Sensors · doi:10.1016/j.measen.2023.100711 · Cited by 24
Facial Expression Recognition on the High Aggregation Subgraphs
Liu, T., Li, J., Wu, J. X., Du, B., Chang, J., & Liu, Y. (2023) · IEEE Transactions on Image Processing · doi:10.1109/tip.2023.3290520 · Cited by 25
Cognitive facial expression recognition with constrained dimensionality reduction
Sun, Y., & Wen, G. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.12.043 · Cited by 24
Recognizing Fine Facial Micro-Expressions Using Two-Dimensional Landmark Feature
Choi, D. Y., Kim, D. H., & Song, B. C. (2018) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2018.8451359 · Cited by 24
“Because of this advantage, Jung [11] and Zhang [12] proposed deep learning algorithms that classify emotions based on LMs.”
BackgroundInfluentialGurnani, A., Shah, K., Gajjar, V., Mavani, V., & Khandhediya, Y. (2019) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2019.00094 · Cited by 26
“[31] have proposed a novel joint fine-tuning method for Facial Expression classification.”
MethodDeep Covariance Descriptors for Facial Expression RecognitionPreprint
Otberdout, N., Kacem, A., Daoudi, M., Ballihi, L., & Berretti, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.03869 · Cited by 30
Facial Expression Recognition Based on Attention Mechanism
Jiang, D., Hu, Y., Lei, D., & Jin, P. (2021) · Scientific Programming · doi:10.1155/2021/6624251 · Cited by 23
“In reference [11], the pairwise random forest method was used to process face pose change in order to solve the problem of face pose change in facial expression recognition.”
MethodJin, X., & Jin, Z. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.079 · Cited by 25
Graph Regularized Structured Output SVM for Early Expression Detection With Online Extension
Xie, L., Luo, Y., Su, S., & Wei, H. (2021) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2021.3108143 · Cited by 23
“For example, in [41], a two-deep-network-based model is applied to FER, in which DTAN extracts the appearance and DTGN exploits the temporal geometric features.”
MethodAhadit, A. B., & Jatoth, R. K. (2022) · Machine Vision and Applications · doi:10.1007/s00138-022-01304-y · Cited by 24
“[13] have used two different CNN models to improve the FER accuracy.”
MethodA joint local spatial and global temporal CNN-Transformer for dynamic facial expression recognition
Wang, L., Kang, X., Ding, F., Nakagawa, S., & Ren, F. (2024) · Applied Soft Computing · doi:10.1016/j.asoc.2024.111680 · Cited by 27
On line emotion detection using retrainable deep neural networks
Kollias, D., Tagaris, A., & Stafylopatis, A. (2016) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2016.7850049 · Cited by 22
“We adopted the architecture used in [35] and shown in Fig.”
MethodPooling Map Adaptation in Convolutional Neural Network for Facial Expression Recognition
Li, Z., Han, S., Khan, A. S., Cai, J., Meng, Z., O’Reilly, J. D., & Tong, Y. (2019) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2019.00194 · Cited by 22
“4 6 Dynamic DTAGN [17] 70.”
Hybrid neural networks based facial expression recognition for smart city
Yan, L., Sheng, M., Wang, C., Gao, R., & Yu, H. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-021-11530-7 · Cited by 22
“[11] proposed a facial expression recognition method based on depth network, which combines two different networks.”
MethodA Region Group Adaptive Attention Model For Subtle Expression Recognition
Chen, G., Peng, J., Zhang, W., Huang, K., Cheng, F., Yuan, H., & Huang, Y. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3133429 · Cited by 22
“[26], [27] encoded the crucial points of the face to obtain the features, aggregated the positions and angles of”
BackgroundFour-player GroupGAN for weak expression recognition via latent expression magnification
Niu, W., Zhang, K., Li, D., & Luo, W. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.109304 · Cited by 22
Joint spatial and scale attention network for multi-view facial expression recognition
Liu, Y., Peng, J., Dai, W., Zeng, J., & Shan, S. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109496 · Cited by 23
Improving Speech Related Facial Action Unit Recognition by Audiovisual Information Fusion
Meng, Z., Han, S., Liu, P., & Tong, Y. (2018) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2018.2840090 · Cited by 22
A Weakly Supervised learning technique for classifying facial expressions
Happy, S. L., Dantcheva, A., & Brémond, F. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2019.08.025 · Cited by 22
“Recently, FER approaches based on CNNs (Jung et al., 2015; Liu et al., 2014b; Zhao et al., 2016; Ding et al., 2017; Meng et al., 2017) have attempted to replace classical approaches based on handcrafted”
BackgroundMethodInfluentialVedantham, R., & Reddy, E. S. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-08901-x · Cited by 22
RetractedFacial Expression Recognition by Fusing Gabor and Local Binary Pattern Features
Sun, Y., & Yu, J. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-51814-5_18 · Cited by 21
Semi-Supervised Learning for Continuous Emotion Recognition Based on Metric Learning
Choi, D. Y., & Song, B. C. (2020) · IEEE Access · doi:10.1109/access.2020.3003125 · Cited by 21
“In general, deep learning-based FERmethods [4] perform very well on acted datasets [5], [6].”
BackgroundMethodThe devil is in the face: Exploiting harmonious representations for facial expression recognition
Han, J., Du, L., Ye, X., Zhang, L., & Feng, J. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.02.054 · Cited by 21
“[21] proposed a deep temporal appearance-geometry network that uses both image sequences and the corresponding landmarks for recognition.”
BackgroundLearning multi-level representations for affective image recognition
Zhang, H., Xu, D., Luo, G., & He, K. (2022) · Neural Computing and Applications · doi:10.1007/s00521-022-07139-y · Cited by 21
“As a transfer learning strategy, fine-tuning makes it possible to use deep learning for small datasets and often achieves good results [34, 35].”
BackgroundZhao, L., Wang, Z., Wang, X., Qi, Y., Liu, Q., & Zhang, G. (2016) · Journal of Electronic Imaging · doi:10.1117/1.jei.25.5.053024 · Cited by 21
Emotional facial expression transfer from a single image via generative adversarial nets
Qiao, F., Yao, N., Jiao, Z., Li, Z., Chen, H., & Wang, H. (2018) · Computer Animation and Virtual Worlds · doi:10.1002/cav.1819 · Cited by 21
“Following the widely adopted protocol introduced in the work of Jung et al., 32 the subjects are grouped into 10 subsets by ID in Multi-PIE and CK + , of which nine subsets are used for training while the remaining subset is used for test.”
MethodSurrogate network-based sparseness hyper-parameter optimization for deep expression recognition
Xie, W., Chen, W., Shen, L., Duan, J., & Yang, M. (2020) · Pattern Recognition · doi:10.1016/j.patcog.2020.107701 · Cited by 21
Linear Disentangled Representation Learning for Facial Actions
Xiang, X., & Tran, T. D. (2017) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2017.2771150 · Cited by 21
“9) except DTGAN [14], which mutually fine tune an appearance-based CNN and a shape-based fully connected network.”
BackgroundMethodIllumination invariants in deep video expression recognition
Gupta, O., Raviv, D., & Raskar, R. (2017) · Pattern Recognition · doi:10.1016/j.patcog.2017.10.017 · Cited by 20
PAU-Net: Privileged Action Unit Network for Facial Expression Recognition
Wang, X., Zhang, T., & Chen, C. L. P. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2022.3203822 · Cited by 20
Facial expression recognition using lightweight deep learning modeling
Ahmad, M., Saira, Alfandi, O., Khattak, A. M., Qadri, S. F., Saeed, I. A., Khan, S., Hayat, B., & Ahmad, A. (2023) · Mathematical Biosciences & Engineering · doi:10.3934/mbe.2023357 · Cited by 20
Video Based Emotion Recognition Using CNN and BRNN
Cai, Y., Zheng, W., Zhang, T., Li, Q., Cui, Z., & Ye, J. (2016) · Communications in Computer and Information Science · doi:10.1007/978-981-10-3005-5_56 · Cited by 20
Facial Expression Recognition in the Wild
Zhang, F., Zhang, T., Mao, Q., Duan, L., & Xu, C. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240574 · Cited by 20
“After feature extraction, in the second stage, the extracted features are fed into a supervised classifier[18, 22, 36] to train an expression recognizer for FER.”
MethodDynamic Facial Expression Recognition Based on Convolutional Neural Networks with Dense Connections
Dong, J., Zheng, H., & Lian, L. (2018) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2018.8545596 · Cited by 19
“In [9], the scheme of joint fine-tuning reported better results than the simple weighted sum.”
BackgroundMethodResultInfluentialExpression recognition with deep features extracted from holistic and part-based models
Happy, S. L., Dantcheva, A., & Brémond, F. (2020) · Image and Vision Computing · doi:10.1016/j.imavis.2020.104038 · Cited by 20
“Deep learning methods for FER CNNs have excelled hierarchical feature extraction and while hand-crafted features dominated FER previously, recently researchers have been focusing on deep learning methods for FER [16, 24, 25, 26, 27, 28, 29, 30].”
BackgroundMethodZhu, J., Luo, B., Zhao, S., Ying, S., Zhao, X., & Gao, Y. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413718 · Cited by 21
“To bridge the affective gap, various hand-crafted [6, 16, 63, 77] or learning-based features [24, 26, 71, 74] are designed.”
BackgroundMethodHuang, C., Jiang, F., Han, Z., Huang, X., Wang, S., Zhu, Y., Jiang, Y., & Hu, B. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3530973 · Cited by 21
On-the-Fly Facial Expression Prediction Using LSTM Encoded Appearance-Suppressed Dynamics
Baddar, W. J., Lee, S., & Ro, Y. M. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2957465 · Cited by 22
Self-citation
Metadata
- Authors
- Jung, H., Lee, S., Yim, J., Park, S., & Kim, J.
- Affiliations
- Korea Advanced Institute of Science and Technology (KR)
- Venue
- 2015 IEEE International Conference on Computer Vision (ICCV)
- Publisher
- IEEE
- Published
- 2015-12
- Pages
- 2983-2991
- OpenAlex
- W2217426128
- Cited by
- 781
- References
- 27
- Citations by year
- 2026: 9 · 2025: 29 · 2024: 45 · 2023: 82 · 2022: 87 · 2021: 127 · 2020: 115 · 2019: 125 · 2018: 92 · 2017: 55 · 2016: 15
- FWCI
- 72.14
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
- KAIST Institutional Repository (KAIST) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- facial expression recognition, joint fine-tuning, facial landmarks, feature integration, image sequences, deep neural network, CK+ database
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
