The DOI matches its Crossref record; 31 of the 32 references it cites were found in a registry or an index.
Deep Learning for Emotion Recognition on Small Datasets using Transfer Learning
Abstract
This paper presents the techniques employed in our team's submissions to the 2015 Emotion Recognition in the Wild contest, for the sub-challenge of Static Facial Expression Recognition in the Wild. The objective of this sub-challenge is to classify the emotions expressed by the primary human subject in static images extracted from movies. We follow a transfer learning approach for deep Convolutional Neural Network (CNN) architectures. Starting from a network pre-trained on the generic ImageNet dataset, we perform supervised fine-tuning on the network in a two-stage process, first on datasets relevant to facial expressions, followed by the contest's dataset. Experimental results show that this cascading fine-tuning approach achieves better results, compared to a single stage fine-tuning with the combined datasets. Our best submission exhibited an overall accuracy of 48.5% in the validation set and 55.6% in the test set, which compares favorably to the respective 35.96% and 39.13% of the challenge baseline.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Since the number of labeled examples for FER remains relatively limited, large 11 models are likely to overfit the data [106], [107], [89], [108].”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [107]
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
How transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019
Return of the Devil in the Details: Delving Deep into Convolutional Nets
Chatfield, K., Simonyan, K., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.6 · Cited by 3,335
Recognizing action units for facial expression analysis
Tian, Y.-I., Kanade, T., & Cohn, J. F. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.908962 · Cited by 1,745
Research works printed but not linked: 1
- J. R. S. Widen and A. Brooks . Anger and disgust: Discrete or overlapping categories ? In Proceedings of the 2004 APS Annual Convention , 2004 . J. R. S. Widen and A. Brooks. Anger and disgust: Discrete or overlapping categories? In Proceedings of the 2004 APS Annual Convention, 2004.
Cited by
Sorted by citations
Items 101–200 of 655
Zhang, R., Guo, Z., Sun, Y., Lu, Q., Xu, Z., Yao, Z., Duan, M., Liu, S., Ren, Y., Huang, L., et al. (2020) · Interdisciplinary Sciences Computational Life Sciences · doi:10.1007/s12539-020-00393-5 · Cited by 56
“Transfer learning is a supervised machine learning strategy to exert the pre-trained model using a small-scale dataset [24].”
MethodThe Pixels and Sounds of Emotion: General-Purpose Representations of Arousal in Games
Makantasis, K., Liapis, A., & Yannakakis, G. N. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3060877 · Cited by 60
“[37] used transfer learning and CNNs for emotion Fig.”
MethodSurvey on Face Expression Recognition using CNN
Vyas, A. S., Prajapati, H. B., & Dabhi, V. K. (2019) · International Conference on Advanced Computing and Communication Systems (ICACCS) · doi:10.1109/icaccs.2019.8728330 · Cited by 61
“Face Detection Viola-Jones [6] [7] [8] [9] [10] [11] Dlib [12]”
Lee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino, A. J. M., & Badia, S. B. I. (2022) · International Journal of Social Robotics · doi:10.1007/s12369-022-00883-0 · Cited by 59
“Even if significant recent work aims to make a fully automated machine learning (ML) model to understand the emotional states of a user with a complex algorithm, it is still challenging to achieve a decent performance [35].”
BackgroundHEU Emotion: a large-scale database for multimodal emotion recognition in the wild
Chen, J., Wang, C., Wang, K., Yin, C., Zhao, C., Xu, T., Zhang, X., Huang, Z., Liu, M., & Yang, T. (2021) · Neural Computing and Applications · doi:10.1007/s00521-020-05616-w · Cited by 56
“Considering that using pre-training models can speed up the convergence [25,38,39], we trained the model of the convolutional neural network in the popular non-laboratory static expression database FER2013 [15] to obtain the parameters.”
MethodFacial Expression Recognition Based on VGGNet Convolutional Neural Network
He, J. Y. D., Li, S., Jinming, S., Liu, Y., Wang, J., & Peng, J. (2018) · Chinese Automation Congress · doi:10.1109/cac.2018.8623238 · Cited by 56
Deep transfer learning for image classification: a survey
Plested, J., Phiri, M., & Gedeon, T. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11491-z · Cited by 107
“Multi-step fine-tuning pipelines that start with pre-training on a large, less related dataset, then fine-tune on one or more smaller, more closely related datasets, before finally fine-tuning on the target task, are often more effective than single-step fine-tuning procedures [191, 195].”
Emotion recognition in the wild using deep neural networks and Bayesian classifiers
Surace, L., Patacchiola, M., Sönmez, E. B., Spataro, W., & Cangelosi, A. (2017) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3136755.3143015 · Cited by 60
“Deep neural networks have been used in [26]. authors started from a network pre-trained on the generic ImageNet dataset, and performing supervised fine-tuning in a two-stage process.”
MethodMilošević, D., Milosavljević, A., Predić, B., Medeiros, A. S., Savić‐Zdravković, D., Piperac, M. S., Kostić, T., Spasić, F., & Leese, F. (2019) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2019.135160 · Cited by 55
Predicting Parkinson’s Disease using Latent Information extracted from Deep Neural Networks
Kollia, I., Stafylopatis, A., & Kollias, S. (2019) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8851995 · Cited by 53
“There are different methods developed in the framework of transfer learning [19], for training a new DNN on P 1 using the structure and weights of the above-described DNN.”
Dynamic Objectives Learning for Facial Expression Recognition
Wen, G., Chang, T., Li, H., & Jiang, L. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.2966858 · Cited by 53
“There some methods, which employ the model based on features from pre-trained CNN [34], [35] and the model based on the ensemble of deep learning networks [36], [37], have significantly improved the recognition performance.”
MethodAudio and face video emotion recognition in the wild using deep neural networks and small datasets
Ding, W., Xu, M., Huang, D., Lin, W., Dong, M., Yu, X., & Li, H. (2016) · Proceedings of the 18th ACM International Conference on Multimodal Interaction · doi:10.1145/2993148.2997637 · Cited by 51
“First, we applied the CNN fine-tuning on small dataset strategy [19] and found that the extracted CNN features are also effective for video facial emotion recognition.”
MethodExploring the Contextual Factors Affecting Multimodal Emotion Recognition in Videos
Bhattacharya, P., Gupta, R., & Yang, Y. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3071503 · Cited by 49
“process a set of image frames in videos using deep learning techniques (see details in [58], [59], [60], [61], [62]).”
Tan, C. N. W., Ceballos, G. A., Kasabov, N. K., & Subramaniyam, N. P. (2020) · Sensors · doi:10.3390/s20185328 · Cited by 57
“Several deep learning techniques have been used to build automatic facial emotion recognition (FER) system , including deep boltzmann machine (DBM), deep belief networks (DBNs) [21, 22, 23], convolutional neural networks (CNNs) [24, 25, 26, 27, 11], auto-encoders [28, 29, 30] and recurrent neural networks (RNNs)to men…”
MethodA systematic comparison of deep learning methods for EEG time series analysis
Walther, D., Viehweg, J., Haueisen, J., & Mäder, P. (2023) · Frontiers in Neuroinformatics · doi:10.3389/fninf.2023.1067095 · Cited by 60
“In recent years, deep learning neural network approaches have been applied to a wide range of neuroscientific problems like feedback on motor imagery tasks (MI) (Tabar and Halici, 2016), emotion recognition (Ng et al., 2015), seizure detection (Thodoroff et al.”
BackgroundReview of automated emotion-based quantification of facial expression in Parkinson’s patients
Sonawane, B., & Sharma, P. (2020) · The Visual Computer · doi:10.1007/s00371-020-01859-9 · Cited by 50
Morid, M. A., Sheng, O. R. L., Kawamoto, K., & Abdelrahman, S. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103565 · Cited by 49
Joint Deep Learning of Facial Expression Synthesis and Recognition
Yan, Y., Huang, Y., Chen, S., Shen, C., & Wang, H. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2962317 · Cited by 52
Transfer Learning-Based Automatic Hurricane Damage Detection Using Satellite Images
Kaur, S., Gupta, S., Singh, S., Hoang, V. T., Almakdi, S., Alelyani, T., & Shaikh, A. (2022) · Electronics · doi:10.3390/electronics11091448 · Cited by 53
An adaptive federated learning system for information sharing in supply chains
Ge, Z., Ivanov, D., & Brintrup, A. M. (2025) · International Journal of Production Research · doi:10.1080/00207543.2024.2432469 · Cited by 49
“It has been widely applied in areas such as image recognition (Ng et al. 2015), natural language processing (Ruder et al. 2019), speech recognition (Kunze et al. 2017) and autonomous driving (Akhauri et al. 2021), with significant promise.”
Sun, X. N., Xia, P., & Ren, F. (2020) · Neurocomputing · doi:10.1016/j.neucom.2019.11.127 · Cited by 49
A Deep Learning System for Recognizing Facial Expression in Real-Time
Yu, M., Dong, H., Jaam, J. M., & Saddik, A. E. (2019) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3311747 · Cited by 54
“As mentioned in Section 5.1.1, most of the related work that evaluated on the JAFFE dataset utilized conventional machine-learning techniques for customized feature extraction and classification, such as References [12, 14].”
MethodInfluentialEmpowering Qualitative Research Methods in Education with Artificial Intelligence
Longo, L. (2019) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-31787-4_1 · Cited by 59
Cai, J., Meng, Z., Khan, A. S., Li, Z., O’Reilly, J. D., & Tong, Y. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3156920 · Cited by 48
“In addition, most of the state-of-the-art methods [49], [50], [55], [56], [57], [58], [59], [70] utilized the FER-2013 dataset as their extra training data, which contains about 35,000 face images.”
A hybrid approach to building face shape classifier for hairstyle recommender system
Pasupa, K., Sunhem, W., & Loo, C. K. (2018) · Expert Systems with Applications · doi:10.1016/j.eswa.2018.11.011 · Cited by 49
Real-time facial expression recognition using smoothed deep neural network ensemble
Benamara, N. K., Val-Calvo, M., Álvarez, J. R., Díaz‐Morcillo, A., Vicente, J. M. F., Fernández-Jover, E., & Stambouli, T. B. (2020) · Integrated Computer-Aided Engineering · doi:10.3233/ica-200643 · Cited by 46
“…(2016) [28] 71.33% Benamara et al. (Ours – 2019) [23] 72.47% Proposed Method 72.72% Kim et al. (2016) [33] 73.73% Pramerdorfer et al. (2016) [34] 75.20% SFEW 2.0 (validation set) Ng et al. (2015) [48] 48.50% ) [31] 71.90% the ensembles outperform the single models in term of recognition accuracy.”
InfluentialPersonalized models for facial emotion recognition through transfer learning
Rescigno, M., Spezialetti, M., & Rossi, S. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-09405-4 · Cited by 47
“(2015) [48] FER-2013 EmotiW [8] CNN (transfer learning) 7 expressions Accuracy 55.”
BackgroundMethodGroup happiness assessment using geometric features and dataset balancing
Vonikakis, V., Yazıcı, Y., Nguyễn, V. D., & WINKLER, S. S. (2016) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2993148.2997633 · Cited by 45
“Most existing methods are based on appearance features, such as Local Binary Patterns [11], Histogram of Gradients (HoG) [13], Gabor features [19], or even raw pixel values in combination with deep learning methods [12].”
MethodSelf-citationFacial Expression Recognition with Neighborhood-Aware Edge Directional Pattern (NEDP)
Iqbal, M. T. B., Abdullah-Al-Wadud, M., Ryu, B., Makhmudkhujaev, F., & Chae, O. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2829707 · Cited by 45
“To combat this issue, recent methods adopt data augmentation strategies to artificially increase the number of training samples [37], [38], [42], [45], or pre-train the network using additional facial image samples from other domains [46], [47].”
BackgroundMethodPhysiological Inspired Deep Neural Networks for Emotion Recognition
Ferreira, P. M., Marques, F. M., Cardoso, J. S., & Rebelo, A. (2018) · IEEE Access · doi:10.1109/access.2018.2870063 · Cited by 45
“This is also observed in other works [34].”
BackgroundMethodResultGoldfarb, J. W., Craft, J., & Cao, J. J. (2019) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.26658 · Cited by 45
Knowledge Augmented Deep Neural Networks for Joint Facial Expression and Action Unit Recognition
Cui, Z., Song, T., Wang, Y., & Ji, Q. (2020) · neural information processing systems · Cited by 44
Edge-Assisted Vehicle Mobility Prediction to Support V2X Communications
Liu, W., & Shoji, Y. (2019) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2019.2937825 · Cited by 45
“It has been validated that this strategy can improve the performances of many tasks like emotion recognition [18] and disease identification [19].”
Siddiqui, M. F. H., Dhakal, P., Yang, X., & Javaid, A. Y. (2022) · Multimodal Technologies and Interaction · doi:10.3390/mti6060047 · Cited by 46
Facial Affect “In-the-Wild”: A Survey and a New Database
Zafeiriou, S. P., Papaioannou, A., Kotsia, I., Nicolaou, M. A., & Zhao, G. (2016) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.186 · Cited by 45
“An interesting observation arising from the work of [64], is that a DCNN trained on sufficiently large auxiliary facial expression datasets alone can be used to obtain much better than baseline results, without using any data from the EmotiW dataste.”
BackgroundMethodFault detection in flotation processes based on deep learning and support vector machine
Li, Z.-M., Gui, W., & Zhu, J. (2019) · Journal of Central South University · doi:10.1007/s11771-019-4190-8 · Cited by 43
Learning deep representations of enzyme thermal adaptation
Li, G., Buric, F., Zrimec, J., Viknander, S., Nielsen, J., Zelezniak, A., & Engqvist, M. K. M. (2022) · Protein Science · doi:10.1002/pro.4480 · Cited by 44
“The two TOPT and TM data sets (Figure 2a,b) are small and comprise only a few thousand samples, which is why they benefited from the transfer learning approach.(35,43) Transfer learning may not provide the same benefit for large data sets.”
BackgroundA Framework for Driver Emotion Recognition using Deep Learning and Grassmann Manifolds
Verma, B., & Choudhary, A. (2018) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc.2018.8569461 · Cited by 40
Multi-stream CNN for facial expression recognition in limited training data
Aghamaleki, J. A., & Chenarlogh, V. A. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-019-7530-7 · Cited by 40
“have drawn on this approach for emotion recognition on small databases [33].”
Semantic Neighborhood-Aware Deep Facial Expression Recognition
Fu, Y., Wu, X., Li, X., Pan, Z., & Luo, D. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2991510 · Cited by 40
“To solve the lack of facial samples, [31], [32] pre-train the model on a low-resolution dataset with relatively more images [33], which leads to a two-stage training procedure.”
BackgroundA ROI-guided deep architecture for robust facial expressions recognition
Sun, X. N., Xia, P., Zhang, L., & Shao, L. (2020) · Information Sciences · doi:10.1016/j.ins.2020.02.047 · Cited by 40
Multitask, Multilabel, and Multidomain Learning With Convolutional Networks for Emotion Recognition
Pons, G., & Masip, D. (2020) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2020.3036935 · Cited by 100
Aparicio, J. A. B., Olivares‐Gil, A., Rodríguez, J. J., García‐Osorio, C., & Díez-Pastor, J.-F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102035 · Cited by 41
Machine Learning in Petrology: State-of-the-Art and Future Perspectives
Petrelli, M. (2024) · Journal of Petrology · doi:10.1093/petrology/egae036 · Cited by 47
Curriculum Learning for Facial Expression Recognition
Gui, L., Baltrušaitis, T., & Morency, L. (2017) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2017.68 · Cited by 41
“[24] demonstrate how to fine-tune CNNs on increasingly more task relevant datasets for better emotion recognition results.”
BackgroundCooney, M. D., & Menezes, M. L. R. (2018) · Multimodal Technologies and Interaction · doi:10.3390/mti2030052 · Cited by 39
Pre-training Strategies and Datasets for Facial Representation Learning
Bulat, A., Cheng, S., Yang, J., Garbett, A., Sánchez, E., & Tzimiropoulos, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19778-9_7 · Cited by 38
Sharan, R. V., Xiong, H., & Berkovsky, S. (2021) · Sensors · doi:10.3390/s21103434 · Cited by 45
How Can Research on Artificial Empathy Be Enhanced by Applying Deepfakes?
Yang, H., Rahmanti, A. R., Huang, C., & Li, Y. (2021) · Journal of Medical Internet Research · doi:10.2196/29506 · Cited by 38
“Based on our results, the major shortcoming that we need to address is that FER technology relies on the quality of data training and the quantity of training data [26,32].”
BackgroundFang, B., Li, X., Han, G., & Ju-hou, H. (2023) · IEEE Access · doi:10.1109/access.2023.3274193 · Cited by 37
Smile, Be Happy :) Emoji Embedding for Visual Sentiment Analysis
Al-Halah, Z., Aitken, A. P., Shi, W., & Caballero, J. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00550 · Cited by 37
“timent classification on the small target dataset. This ap1 arXiv:1907.06160v2 [cs.CV] 26 Nov 2019 proach is unanimously adopted by recent visual sentiment models and has led to improved results, e.g. [43,7,33]. Nonetheless, object categories and sentiment labels are not aligned and rather orthogonal. Object labels are…”
BackgroundShan, Y., Li, S., & Chen, T. (2020) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-020-01074-x · Cited by 36
AutoFCL: automatically tuning fully connected layers for handling small dataset
Basha, S. H. S., Vinakota, S. K., Dubey, S. R., Pulabaigari, V., & Mukherjee, S. (2021) · Neural Computing and Applications · doi:10.1007/s00521-020-05549-4 · Cited by 42
“A common approach for classifying such limited images is re-using the pre-trained models to fine-tune over other datasets [19].”
BackgroundAdvances and Trends in Real Time Visual Crowd Analysis
Khan, K., Albattah, W., Khan, R. U., Qamar, A. M., & Nayab, D. (2020) · Sensors · doi:10.3390/s20185073 · Cited by 43
Proceedings of the Fourth Arabic Natural Language Processing Workshop
El-Hajj, W., Belguith, L. H., Bougares, F., Magdy, W., Zitouni, I., Tomeh, N., & El Hajj, M. (2019) · doi:10.18653/v1/w19-46 · Cited by 37
CNN based 3D facial expression recognition using masking and landmark features
Yang, H., & Yin, L. (2017) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2017.8273654 · Cited by 36
“[18] performed a supervised cascaded approach to fine-tune a pre-trained deep model, and showed comparable results on small datasets.”
ResultAdversarial Domain Generalized Transformer for Cross-Corpus Speech Emotion Recognition
Gao, Y., Wang, L., Liu, J., Dang, J., & Okada, S. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3290795 · Cited by 36
Mengiste, E., Mannem, K. R., Prieto, S. A., & Garcia de Soto, B. (2024) · Journal of Computing in Civil Engineering · doi:10.1061/jccee5.cpeng-5478 · Cited by 35
Manifold feature integration for micro-expression recognition
Takalkar, M. A., Xu, M. G., & Chaczko, Z. (2020) · Multimedia Systems · doi:10.1007/s00530-020-00663-8 · Cited by 35
“Moreover, Ng et al. [31] and Takalkar et al. [48] have pointed out that the uneven distribution within emotional classes in the classes with lesser samples can lead to low precision.”
BackgroundResearch on Automatic Music Recommendation Algorithm Based on Facial Micro-expression Recognition
Yu, Z., Zhao, M., Wu, Y., Liu, P., & Chen, H. (2020) · Chinese Control Conference (CCC) · doi:10.23919/ccc50068.2020.9189600 · Cited by 34
Fang, B., Li, X., Han, G., & Ju-hou, H. (2023) · IEEE Access · doi:10.1109/access.2023.3322454 · Cited by 43
“The transfer learning method is an effective model training method [88].”
MethodDeep peak-neutral difference feature for facial expression recognition
Chen, J., Xu, R., & Liu, L. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-5909-5 · Cited by 32
“In [32], the deep neural network for FER was first trained using a large”
Hazourli, A. R., Djeghri, A., Salam, H., & Othmani, A. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-020-10332-7 · Cited by 44
“Even without expanding the labeled training sets or using transfer learning [48, 68], the accuracy is sufficiently acceptable.”
BackgroundThe limitations for expression recognition in computer vision introduced by facial masks
Abate, A. F., Cimmino, L., Mocanu, B.-C., Narducci, F., & Pop, F. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13559-8 · Cited by 32
“[23] apply a transfer learning technique for deep CNN architectures with two-stage supervised fine-tuning.”
MethodResNet-Based Parkinson's Disease Classification
Ariss, O. E., & Hu, K. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3193651 · Cited by 31
“In [46], transfer learning’s ability to reduce the size of the training set is verified again as the authors successfully transferred a pretrained generic CNN to the task of emotion recognition using a small training set.”
MethodAdaptation and contextualization of deep neural network models
Kollias, D., Yu, M., Tagaris, A., Leontidis, G., Stafylopatis, A., & Kollias, S. (2017) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2017.8280975 · Cited by 31
“- Transfer Learning: Transfer learning [6] is the main approach to avoid learning failure due to overfitting, when training complex CNNs with small amounts of (image) data.”
BackgroundDeep Learning Based Real-Time Driver Emotion Monitoring
Verma, B., & Choudhary, A. (2018) · IEEE International Conference on Vehicular Electronics and Safety (ICVES) · doi:10.1109/icves.2018.8519595 · Cited by 31
Schiller, D., Huber, T., Dietz, M., & André, E. (2020) · Frontiers in Computer Science · doi:10.3389/fcomp.2020.00006 · Cited by 32
“For instance, Ng et al. (2015) pretrained a deep Convolutional Neural Network on the generic ImageNet dataset.”
BackgroundReducing Videoconferencing Fatigue through Facial Emotion Recognition
Rößler, J., Sun, J., & Gloor, P. A. (2021) · Future Internet · doi:10.3390/fi13050126 · Cited by 32
Crowd Counting Using End-to-End Semantic Image Segmentation
Khan, K., Khan, R. U., Albattah, W., Nayab, D., Qamar, A. M., Habib, S., & Islam, M. (2021) · Electronics · doi:10.3390/electronics10111293 · Cited by 31
Automatic Analysis of Facial Expressions Based on Deep Covariance Trajectories
Otberdout, N., Kacem, A., Daoudi, M., Ballihi, L., & Berretti, S. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2947244 · Cited by 30
Applying machine learning to optical metrology: a review
Xue, R., Hooshmand, H., Isa, M., Piano, S., & Leach, R. (2024) · Measurement Science and Technology · doi:10.1088/1361-6501/ad7878 · Cited by 29
“Ng et al [56] used transfer learning with deep CNNs for recognising emotions in facial expressions, specifically for a 2015 competition about emotion recognition.”
Non-rigid registration based model-free 3D facial expression recognition
Savran, A., & Sankur, B. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.07.005 · Cited by 30
“…in acial expression recognition ( Kahou et al., 2013; Ding et al., 2016; aiswal and Valstar, 2016; Jung et al., 2015; Khorrami et al., 2015; im et al., 2015; Levi, 2015; Ng et al., 2015; Zhao et al., 2016 ) from ngineered local feature extractors (Gabor, LBP, HoG, SIFT, etc.) as ell.”
Ming, Z., Chazalon, J., Luqman, M. M., Visani, M., & Burie, J.-C. (2018) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2018.8545274 · Cited by 32
“50 TL [41] 48.”
Öztel, İ., Yolcu, G., & Öz, C. (2019) · International Conference on Computer Science and Engineering (UBMK) · doi:10.1109/ubmk.2019.8907203 · Cited by 33
Deep Neural Network Architecture: Application for Facial Expression Recognition
Villanueva, M. G., & Zavala, S. R. (2020) · IEEE Latin America Transactions · doi:10.1109/tla.2020.9099774 · Cited by 29
“…temporales en las componentes faciales, un enfoque h´ıbrido reciente presenta una combinaci´on de una CNN para las caracter´ısticas espaciales en los fotogramas en una secuencia de video, con una red LSTM (del ingl´es long short-term memory) para las caracter´ısticas temporales [17], [18], [19].”
BackgroundGupta, V., Demirer, M., Bigelow, M. T., Little, K. J., Candemir, S., Prevedello, L. M. S., White, R., O’Donnell, T., Wels, M., & Erdal, B. S. (2019) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-019-00267-3 · Cited by 29
“Lastly, transfer learning (TL) has also been shown to improve algorithm performance in datasets with limited size [4]; TL is a deep learning method leveraging pre-trained weighted models from large datasets to accomplish an identified task by fine-tuning the final layers of a DNN while the weights…”
MethodCAKE: Compact and Accurate K-dimensional representation of EmotionPreprint
Kervadec, C., Vielzeuf, V., Pateux, S., Lechervy, A., & Jurie, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.11215 · Cited by 32
“In [18], the authors make use of transfer learning to counteract the smallness of the SFEW [4] dataset, by pre-training their model on ImageNet [2] and FER [9].”
MethodHierarchical Group-Level Emotion Recognition
Fujii, K., Sugimura, D., & Hamamoto, T. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3033125 · Cited by 28
Enhancing visual quality of spatial image steganography using SqueezeNet deep learning network
Hamid, N., Sumait, B. S., Bakri, B. I., & Al-Qershi, O. M. (2021) · Multimedia Tools and Applications · doi:10.1007/s11042-021-11315-y · Cited by 28
“To overcome the problem of overfitting, transfer learning is used where the weights of the CNN are initialized using the weight from a network trained for similar tasks [22].”
MethodDiscussions of Different Deep Transfer Learning Models for Emotion Recognitions
Yen, C.-T., & Li, K.-H. (2022) · IEEE Access · doi:10.1109/access.2022.3209813 · Cited by 29
“Nguyen et al. [18] used multiple datasets for the training of a pretrained model for transfer learning with multiple source domains.”
BackgroundEmotion recognition from facial expressions for 3D videos using siamese network
Lawrance, D., & Palaniswamy, S. (2021) · International Conference on Communication, Control and Information Sciences (ICCISc) · doi:10.1109/iccisc52257.2021.9484949 · Cited by 27
“Gideon et al. [14] proposed Progressive Neural Networks (ProgNet) which can be used for transferring knowledge and to avoid forgetting effect of conventional pre-training and fine-tuning.”
BackgroundTransfer Learning Facilitates the Prediction of Polymer–Surface Adhesion Strength
Shi, J., Albreiki, F., Colón, Y. J., Srivastava, S., & Whitmer, J. K. (2023) · Journal of Chemical Theory and Computation · doi:10.1021/acs.jctc.2c01314 · Cited by 27
“− 48, In this study, we demonstrate the ability of transfer learning to leverage the prediction performance of adhesive free energies between polymer chains with a defined sequence and patterned surfaces via fine-tuning a pretrained model.”
EDL-Det: A Robust TTS Synthesis Detector Using VGG19-Based YAMNet and Ensemble Learning Block
Mahum, R., Irtaza, A., & Javed, A. (2023) · IEEE Access · doi:10.1109/access.2023.3332561 · Cited by 29
A Review on Deep Learning Algorithms for Speech and Facial Emotion Recognition
Latha, C. P., & Priya, M. (2016) · APTIKOM Journal on Computer Science and Information Technologies · doi:10.11591/aptikom.j.csit.118 · Cited by 28
“Hong–Wei Ng et al [26] developed a model for emotion recognition on small datasets using Deep CNN”
MethodPriya, D. T., & Udayan, J. D. (2020) · International Journal of Speech Technology · doi:10.1007/s10772-020-09707-w · Cited by 26
“The author Ng et al. (2015) proposed a facial expression recognition model on transfer features from deep convolutional networks using transfer learning techniques.”
BackgroundMeta-transfer learning for emotion recognition
Nguyen, D., Nguyen, T. T., Sridharan, S., Denman, S., Nguyen, T. T., Dean, D., & Fookes, C. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08248-y · Cited by 28
“, 2018) in which the CNN architectures were pre-trained using the generic ImageNet dataset and fine-tuned on emotion datasets (Ng et al., 2015; Kaya et al., 2017).”
BackgroundMethodMulti-Modal Residual Perceptron Network for Audio–Video Emotion Recognition
Chang, X., & Skarbek, W. (2021) · Sensors · doi:10.3390/s21165452 · Cited by 28
“Moreover, for video frames, HW. Ng et al. [28] got 47.3% validation accuracy and 53.8% testing accuracy on EmotiW dataset [29] using transfer learning and averaged temporal deep features.”
MethodInfluentialAssessment of Parkinson’s Disease Based on Deep Neural Networks
Tagaris, A., Kollias, D., & Stafylopatis, A. (2017) · Communications in computer and information science · doi:10.1007/978-3-319-65172-9_33 · Cited by 26
Trujillo, C., & Garcia-Sucerquia, J. I. (2019) · Optics and Lasers in Engineering · doi:10.1016/j.optlaseng.2019.02.010 · Cited by 26
Speech Emotion Recognition among Elderly Individuals using Multimodal Fusion and Transfer Learning
Boateng, G., & Kowatsch, T. (2020) · Companion Publication of the International Conference on Multimodal Interaction (ICMI Companion) · doi:10.1145/3395035.3425255 · Cited by 26
“Transfer learning has also been used in emotion recognition tasks ([7, 14, 25]).”
MethodDeep fair models for complex data: Graphs labeling and explainable face recognition
Franco, D., Navarin, N., Donini, M., Anguita, D., & Oneto, L. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.05.109 · Cited by 26
“As a positive side effect, these representations can be reused, incrementally enriched, and fine-tuned for many different tasks [53–56].”
BackgroundObject-Based Augmentation for Building Semantic Segmentation: Ventura and Santa Rosa Case Study
Illarionova, S., Nesteruk, S., Shadrin, D., Ignatiev, V., Pukalchik, M., & Oseledets, I. V. (2021) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw54120.2021.00191 · Cited by 26
“There are different approaches to address dataset limitations: pseudo labeling, special architectures development, transfer learning [2, 3, 32, 50].”
MethodSMC faster R-CNN: Toward a scene-specialized multi-object detector
Mhalla, A., Château, T., Maâmatou, H., Gazzah, S., & Amara, N. E. B. (2017) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2017.06.008 · Cited by 25
“Transfer learning using deep models has been turned out to be e ff ective in some challenges [22][23] like tra ffi c-object detection [24][20], emotion recognition [25] and sentiment analysis [26].”
MethodMasarczyk, W., Głomb, P., Grabowski, B., & Ostaszewski, M. (2020) · Remote Sensing · doi:10.3390/rs12162653 · Cited by 27
“In [37], authors used transfer learning on CNNs to recognize emotions from the pictures of faces.”
BackgroundCai, W., Huang, J., Deng, A., & Wang, Q. (2021) · Aerospace Science and Technology · doi:10.1016/j.ast.2020.106487 · Cited by 24
Wadhawan, R., & Gandhi, T. K. (2022) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2022.3172272 · Cited by 28
“Further, the proposed technique is different from [8] and [22] as no pretraining has been performed on external data.”
MethodFacial action recognition using very deep networks for highly imbalanced class distribution
Ding, W., Huang, D., Chen, Z., Yu, X., & Lin, W. (2017) · Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) · doi:10.1109/apsipa.2017.8282246 · Cited by 24
“Thus, the approaches are likely to train deep network by manually collecting extra data [10], [27], [4] or transfer learning [19] to improve the performances for facial expression recognition.”
BackgroundDeep Covariance Descriptors for Facial Expression RecognitionPreprint
Otberdout, N., Kacem, A., Daoudi, M., Ballihi, L., & Berretti, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.03869 · Cited by 30
“…some recent works that use covariance descriptors in conjunction with DCNN. DCNN for Facial Expression Recognition: Motivated by the success of DCNN models in facial analysis tasks, several papers proposed to use them for both static and dynamic facial expression recognition [12, 18, 19, 24].”
BackgroundMethodA Personalized Affective Memory Model for Improving Emotion Recognition
Barros, P. V. A., Parisi, G. I., & Wermter, S. (2019) · International Conference on Machine Learning · Cited by 23
Metadata
- Authors
- Ng, H.-W., Nguyen, V. D., Vonikakis, V., & Winkler, S.
- Affiliations
- University of Illinois at Urbana-Champaign (Singapore), Singapore, Singapore
- University of Illinois Urbana-Champaign (US)
- Venue
- Proceedings of the 2015 ACM on International Conference on Multimodal Interaction
- Publisher
- ACM
- Published
- 2015-11-09
- Pages
- 443-449
- OpenAlex
- W2253728219
- Cited by
- 657
- References
- 32
- Citations by year
- 2026: 13 · 2025: 29 · 2024: 37 · 2023: 70 · 2022: 69 · 2021: 106 · 2020: 106 · 2019: 106 · 2018: 63 · 2017: 44 · 2016: 13 · 2015: 1
- FWCI
- 53.84
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (publishedVersion)
- Funders
- Agency for Science Technology and Research
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- emotion recognition, facial expression recognition, still images, transfer learning, convolutional neural networks, fine-tuning, ImageNet pre-training, small datasets
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
