Cited by 657Emotion and Mood RecognitionFace recognition and analysisDomain Adaptation and Few-Shot Learning

The DOI matches its Crossref record; 31 of the 32 references it cites were found in a registry or an index.

Deep Learning for Emotion Recognition on Small Datasets using Transfer Learning

Ng, Hong-Wei, Nguyen, Viet Dung, Vonikakis, Vassilios, Winkler, Stefan·2015·Proceedings of the 2015 ACM on International Conference on Multimodal Interaction

DOI 10.1145/2818346.2830593

Open this paper in CoChat — free account

Abstract

This paper presents the techniques employed in our team's submissions to the 2015 Emotion Recognition in the Wild contest, for the sub-challenge of Static Facial Expression Recognition in the Wild. The objective of this sub-challenge is to classify the emotions expressed by the primary human subject in static images extracted from movies. We follow a transfer learning approach for deep Convolutional Neural Network (CNN) architectures. Starting from a network pre-trained on the generic ImageNet dataset, we perform supervised fine-tuning on the network in a two-stage process, first on datasets relevant to facial expressions, followed by the contest's dataset. Experimental results show that this cascading fine-tuning approach achieves better results, compared to a single stage fine-tuning with the combined datasets. Our best submission exhibited an overall accuracy of 48.5% in the validation set and 55.6% in the test set, which compares favorably to the respective 35.96% and 39.13% of the challenge baseline.

Integrity signals

Registry

Crossref · 10.1145/2818346.2830593

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.81 · Cited by 32,684
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
How transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019
Return of the Devil in the Details: Delving Deep into Convolutional Nets
Chatfield, K., Simonyan, K., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.6 · Cited by 3,335
Recognizing action units for facial expression analysis
Tian, Y.-I., Kanade, T., & Cohn, J. F. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.908962 · Cited by 1,745

Flagged first, then by citations

Show all 32 references

1 of the 32 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

J. R. S. Widen and A. Brooks . Anger and disgust: Discrete or overlapping categories ? In Proceedings of the 2004 APS Annual Convention , 2004 . J. R. S. Widen and A. Brooks. Anger and disgust: Discrete or overlapping categories? In Proceedings of the 2004 APS Annual Convention, 2004.

Sorted by citations

Items 401–500 of 655

Human emotion identification based on features extracted using CNN
Nawaf, A. Y., & Jasim, W. M. (2022) · AIP conference proceedings · doi:10.1063/5.0112131 · Cited by 6
Convolutional neural networks and feature fusion for bimodal emotion recognition on the emotiW 2016 challenge
Yan, J., Yan, B., Lu, G., Xu, Q., Li, H., Cheng, X., & Cai, X. (2017) · International Congress on Image and Signal Processing (CISP) · doi:10.1109/cisp-bmei.2017.8301997 · Cited by 6
“According to [7], [16], [24], [25], Alex Net is regarded as one of the most powerful CNN modal and acquire good performance in recognition task.”
BackgroundMethodInfluential
A Data-Driven Deep Learning Model to Pattern Recognition for Medical Diagnosis, by using Model Aggregation and Model Selection
Djellali, C., Adda, M., & Moutacalli, M. T. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.10.052 · Cited by 6
Emotional Landscape Image Generation Using Generative Adversarial Networks
Park, C., & Lee, I. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-69538-5_35 · Cited by 6
Off-Line Tamil Handwritten Character Recognition Based on Convolutional Neural Network with VGG16 and VGG19 Model
Lincy, R. B., & Gayathri, R. (2021) · Lecture notes in electrical engineering · doi:10.1007/978-981-15-8221-9_180 · Cited by 6
Contactless Human Emotion Analysis Across Different Modalities
Nahid, N., Rahman, A., & Ahad, M. A. R. (2021) · Intelligent systems reference library · doi:10.1007/978-3-030-68590-4_9 · Cited by 6
Unconstrained Facial Expression Recogniton Based on Cascade Decision and Gabor Filters
Wu, Y., Zhang, L., Chen, G., & Michelini, P. N. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9411983 · Cited by 6
“Inspired by research in [27], we do not train models on SFEW2.”
Method
Multitask Deep Learning for Edge Intelligence Video Surveillance System
Li, J., Zheng, Z., Li, Y., Ma, R., & Xia, S. (2020) · IEEE International Conference on Industrial Informatics (INDIN) · doi:10.1109/indin45582.2020.9442166 · Cited by 6
“Previous studies [24], [25] usually train several subdivisions of natural language”
Background
A CNN Transfer Learning -Electrocardiogram (ECG) Signal Approach to Predict COVID-19
Irungu, J., Oladunni, T., Denis, M. F., Ososanya, E., & Muriithi, R. (2023) · International Conference on Computer and Automation Engineering · doi:10.1109/iccae56788.2023.10111114 · Cited by 6
“Shahin’s [8] also adopted a CNN-LTSM model for ECG Classification for predicting COVID-19 and achieved an accuracy of 81.”
MethodInfluential
LARa: A Robotic Framework for Human-Robot Interaction on Indoor Environments
Ranieri, C. M., Nardari, G. V., Pinto, A. H. M., Tozadore, D. C., & Romero, R. A. F. (2018) · Latin American Robotics Symposium, Brazilian Symposium on Robotics and Workshop on Robotics in Education (LARS/SBR/WRE) · doi:10.1109/lars/sbr/wre.2018.00074 · Cited by 6
“[16], making it almost impossible to build refined interaction strategies based on these particular emotions.”
Background
Extraction of Semantic Relations in Noisy User-Generated Law Enforcement Data
Schraagen, M., & Bex, F. (2019) · IEEE International Conference on Semantic Computing (ICSC) · doi:10.1109/icosc.2019.8665497 · Cited by 6
“Moreover, neural networks are assumed to require a relatively large training set for convergence compared to other machine learning algorithms [16], [17], [18], parameter settings for complex networks are non-trivial, and computational cost may be prohibitive.”
Background
Class-Conditional Data Augmentation Applied to Image Classification
Aguilar, E., & Radeva, P. I. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-29891-3_17 · Cited by 6
Two-Channel Feature Extraction Convolutional Neural Network for Facial Expression Recognition
Liu, C., Hirota, K., Wang, B., Dai, Y., & Jia, Z. (2020) · Journal of Advanced Computational Intelligence and Intelligent Informatics · doi:10.20965/jaciii.2020.p0792 · Cited by 6
Multimodal emotion recognition using deep learning techniques
Nguyen, T. D. (2020) · Queensland University of Technology · doi:10.5204/thesis.eprints.180753 · Cited by 6
Disentanglement For Discriminative Visual Recognition
Liu, X. (2021) · Wiley eBooks · doi:10.1002/9781119751991.ch5 · Cited by 6
A Systematic Evaluation of Domain Adaptation in Facial Expression Recognition.Preprint
Kong, Y., Suresh, V., Soh, J. M., & Ong, D. C. (2021) · arXiv (Cornell University) · Cited by 6
Cross-Domain Facial Expression Recognition by Combining Transfer Learning and Face-Cycle Generative Adversarial Network
Zhou, Y., Yang, B. Q., Liu, Z., Wang, Q., & Xiong, P. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-18713-y · Cited by 5
“[36] first pre-trained the model on the generic ImageNet dataset and then perform supervised fine-tuning on FER datasets.”
Multimodal Sentiment Analysis for the Malay Language: New Corpus using CNN-based Framework
Taylor, S., & Fauzi, F. (2024) · ACM Transactions on Asian and Low-Resource Language Information Processing · doi:10.1145/3703445 · Cited by 5
The Intersection of AI and Emotions: Deep Learning Applications for a Better Society
Yadav, H., & Maini, S. (2025) · Springer eBooks · doi:10.1007/978-3-031-86445-2_2 · Cited by 5
A Multimodal Driver Anger Recognition Method Based on Context-Awareness
Ding, T., Zhang, K., Gao, S., Miao, X., & Xi, J. (2024) · IEEE Access · doi:10.1109/access.2024.3422383 · Cited by 5
“…of the face, but also taking into account the change of expression over time, which improves the accuracy of emotion recognition; Ng et al. [17] proposed a CNN model that simultaneously learns facial emotion recognition and other face-related tasks (e.g., gender recognition, age…”
Background
AI-Vision Towards an Improved Social Inclusion
Alashkar, R., ElSabbahy, M., Sabha, A., Abdelghany, M., Tlili, B., & Mounsef, J. (2020) · IEEE/ITU International Conference on Artificial Intelligence for Good (AI4G) · doi:10.1109/ai4g50087.2020.9311049 · Cited by 5
Real-time Feedback based on Emotion Recognition for Improving Children’s Metacognitive Monitoring Skill
Ruan, X., Palansuriya, C., & Constantin, A. (2022) · ACM International Conference on Interaction Design and Children (IDC) · doi:10.1145/3501712.3538831 · Cited by 5
“To be aware of unique behaviours of facial expression with small training dataset, transfer learning algorithms have been applied on training Neural Networks (NN) to classify emotions [1, 20, 21].”
Method
Soft Label Mining and Average Expression Anchoring for Facial Expression Recognition
Ming, H., Lu, W., & Zhang, W. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-26316-3_43 · Cited by 5
Deep cross-domain transfer for emotion recognition via joint learning
Nguyen, D., Nguyen, T. T., Sridharan, S., Abdelrazek, M., Denman, S., Tran, S. N., Zeng, R., & Fookes, C. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-15441-7 · Cited by 5
“For example, in [20], an emotion recognition model was fine-tuned from a model pre-trained on ImageNet.”
A Recursive Ensemble Learning Approach With Noisy Labels or Unlabeled Data
Wang, Y., Yang, Y., Liu, Y., & Bharath, A. A. (2019) · IEEE Access · doi:10.1109/access.2019.2904403 · Cited by 5
“When the amount of data is insufficient, one of the solutions is to use a model that is pre-trained on the large dataset, then to fine-tune it on a new dataset [10]–[14].”
Method
Multi-task Cascaded and Densely Connected Convolutional Networks Applied to Human Face Detection and Facial Expression Recognition System
Chou, K.-Y., Cheng, Y.-W., Chen, W.-R., & Chen, Y. (2019) · International Automatic Control Conference · doi:10.1109/cacs47674.2019.9024357 · Cited by 5
“The experimental results and application are shown and discussed in Section III.”
BackgroundInfluential
Deep Neural Networks for Detecting Real Emotions Using Biofeedback and Voice
Aledhari, M., Razzak, R., Parizi, R. M., & Srivastava, G. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68799-1_21 · Cited by 5
On Training Sketch Recognizers for New Domains
Yesilbek, K. T., & Sezgin, T. M. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00243 · Cited by 5
“Transfer learning is widely adopted in deep learning ﬁeld [29] [2] [15].”
Cross-modality interaction between EEG signals and facial expression
Rayatdoost, S. (2017) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3136755.3137034 · Cited by 5
“Ng et al. [16] used transfer learning and fine-tuned a pretrained convolutional network to achieve better result for emotion recognition on small datasets.”
Background
End to End Deep Learning for Single Step Real-Time Facial Expression Recognition
Reddy, B., Kim, Y.-H., Yun, S., Jang, J., & Hong, S.-H. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-56687-0_8 · Cited by 5
Estimating Public Speaking Anxiety from Speech Signals Using Unsupervised Transfer Learning
Feng, K., Yadav, M., Sakib, M. N., Behzadan, A. H., & Chaspari, T. (2019) · IEEE Global Conference on Signal and Information Processing · doi:10.1109/globalsip45357.2019.8969502 · Cited by 5
“PREVIOUS WORK Transfer learning approaches, such as fine-tuning [17], and”
Background
A Personalized Affective Memory Neural Model for Improving Emotion RecognitionPreprint
Barros, P. V. A., Parisi, G. I., & Wermter, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.12632 · Cited by 5
“Recent approaches propose to address these problems by introducing transfer learning techniques (Ng et al., 2015; Kaya et al., 2017), neural activation and data distribution regularization (Ding et al., 2017b; Pons & Masip, 2018), and unsupervised learning of facial representations (Zen et al.,…”
Method
Facial Expression Recognition Using Human to Animated-Character Expression TranslationPreprint
Ali, K., Isler, I., & Hughes, C. E. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.05595 · Cited by 5
“ation and head position. With the recent success of deep CNN, automatic feature learning for FER has been extensively studied by computer vision researchers and various techniques such as [16], [36], [28] [21] [15] [10] [41] have been developed to increase the robustness of FER. However, the performance of these autom…”
Method
Context-Dependent Models for Predicting and Characterizing Facial ExpressivenessPreprint
Lin, V., Girard, J. M., & Morency, L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.04523 · Cited by 7
Multimodal Continuous Emotion Recognition using Deep Multi-Task Learning with Correlation LossPreprint
Köprü, B., & Erzin, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.00876 · Cited by 5
Non-binary deep transfer learning for image classificationPreprint
Plested, J., Shen, X., & Gedeon, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.08585 · Cited by 5
“The relationship between the source and the target dataset and how well the pre-trained weights ﬁt the target task has been shown to be important to transfer learning performance numerous times [28, 22, 6, 8, 33, 21, 20, 37, 27, 42, 18].”
Background
XAI-DSCSA: explainable-AI-based deep semi-supervised convolutional sparse autoencoder for facial expression recognition
Mohana, M., Subashini, P., & Ghinea, G. (2025) · Signal Image and Video Processing · doi:10.1007/s11760-025-03984-1 · Cited by 4
Continual Facial Features Transfer for Facial Expression Recognition
Maharjan, R. S., Bonicelli, L., Romeo, M., Calderara, S., Cangelosi, A., & Cucchiara, R. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3561139 · Cited by 6
Multitask EfficientNet affective computing for student engagement detection
Thiruthuvanathan, M. M., & Krishnan, B. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-19815-3 · Cited by 4
“Their ﬁndings revealed an overall accuracy of 48.5% for validation and 55.65% for testing [14].”
Influential
Varying the Context to Advance Affect Modelling: A Study on Game Engagement Prediction
Pinitas, K., Rašajski, N., Barthet, M., Kaselimi, M., Makantasis, K., Liapis, A., & Yannakakis, G. N. (2024) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii63134.2024.00027 · Cited by 4
“Breuer and Kimmer [12] employed Convolutional Neural Networks (CNNs) for diverse facial expression recognition tasks, while Ng et al. [13] fine-tuned CNNs to detect emotions on smaller datasets.”
Tracking Emotional State of a Person with Artificial Intelligence Methods and Its Application to Customer Services
Sebyakin, A., & Zolotaryuk, A. V. (2019) · International Conference Management of Large-Scale System Development (MLSD) · doi:10.1109/mlsd.2019.8911054 · Cited by 4
“With the increasing popularity of convolutional neural networks ( Convolutional Neural Networks, CNN ) methods began to appear, without using the signs described by man, the image was entirely processed by a neural network (for example, in [9], [10], [11], [12] ).”
Background
Facial Expression Recognition using Convolutional Neural Network and SoftMax function on Captured Images
Deopa, A., Sinha, A., Prakash, A., & Sinha, R. K. (2019) · International Conference on Communication and Electronics Systems (ICCES) · doi:10.1109/icces45898.2019.9002524 · Cited by 4
“Their results verified that this cascading of fine-tuning yielded better results [13].”
Result
RETRACTED ARTICLE: Influence of enterprise cloud computing database application and enterprise debt financing based on SVM algorithm: internal control quality analysis
Liu, B. (2021) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-021-03103-1 · Cited by 4
Retracted
Facial Emotion Detection Using Convolutional Neural Networks
Adil, M. A. (2021) · UVic’s Research and Learning Repository (University of Victoria) · Cited by 4
Deep Facial Expression Recognition Algorithm Combining Channel Attention
Peixiang, Z., Liu, Y., Hao, Y., & Liu, J. (2021) · 2021 4th International Conference on Artificial Intelligence and Pattern Recognition · doi:10.1145/3488933.3489006 · Cited by 4
Chili Plant Classification Using Transfer Learning Models Through Object Detection
Shapiee, M. N. A., Manan, A. A. A., Razman, M. A. M., Khairuddin, I. M., & Majeed, A. P. P. A. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-981-19-2095-0_46 · Cited by 4
On the Audio-Visual Emotion Recognition using Convolutional Neural Networks and Extreme Learning Machine
Ashraf, A., Gunawan, T. S., Arifin, F., Kartiwi, M., Sophian, A., & Habaebi, M. H. (2022) · Indonesian Journal of Electrical Engineering and Informatics (IJEEI) · doi:10.52549/ijeei.v10i3.3879 · Cited by 4
A pre-trained model vs dedicated convolution neural networks for emotion recognition
Nawaf, A. Y., & Jasim, W. M. (2022) · International Journal of Power Electronics and Drive Systems/International Journal of Electrical and Computer Engineering · doi:10.11591/ijece.v13i1.pp1123-1133 · Cited by 4
Malayalam Handwritten Character Recognition Using Transfer Learning
Jose, B., & Pushpalatha, K. P. (2023) · International Conference on Advances in Intelligent Computing and Applications (AICAPS) · doi:10.1109/aicaps57044.2023.10074586 · Cited by 4
“The classiﬁcation of human emotions was carried out using sample frames obtained from movies, as said by Ng et al. [9].”
Method
Classification of Facial Expression of Post-Surgical Pain in Children
Moreno, C. J., Aristizabal, J. K. N., & Giraldo-Salazar, O. L. (2021) · Visión electrónica · doi:10.14483/22484728.17425 · Cited by 4
Emotion Recognition from Video Frame Sequence using Face Mesh and Pre-Trained Models of Convolutional Neural Network
Adi, D. P., Yuniarno, E. M., & Wulandari, D. P. (2023) · International Seminar on Intelligent Technology and Its Applications · doi:10.1109/isitia59021.2023.10221117 · Cited by 4
“EmotiW [22] Angry, Disgust, Fear, Happy, Sad, Surprise, and Neutral ~1,000 static images”
Efficient Regularization Framework for Histopathological Image Classification Using Convolutional Neural Networks.
Dif, N., & Elberrichi, Z. E. (2020) · International Journal of Cognitive Informatics and Natural Intelligence · doi:10.4018/ijcini.2020100104 · Cited by 4
S2ML-TL Framework for Multi-label Food Recognition
Nagarajan, B., Aguilar, E., & Radeva, P. I. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68821-9_50 · Cited by 4
Relaxed Spatio-Temporal Deep Feature Aggregation for Real-Fake Expression Prediction
Özkan, S., & Akar, G. B. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.366 · Cited by 4
Investigating the Effect of Noise on Facial Expression Recognition
Alkaddour, M., & Tariq, U. (2019) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-17798-0_55 · Cited by 4
The Many Moods of EmotionPreprint
Vielzeuf, V., Kervadec, C., Pateux, S., & Jurie, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.13197 · Cited by 4
“Indeed, several works propose to analyze facial expressions from images [1], [24], from multimodal videos [6], [17], [28], [40] or from multi-view videos [2], [37].”
Background
Deep and Shallow Covariance Feature Quantization for 3D Facial Expression RecognitionPreprint
Hariri, W., Farah, N., & Vishwakarma, D. K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.05708 · Cited by 7
“As an example, a supervised ﬁne-tuning on a small dataset is applied in [25] for FER.”
Object-Based Augmentation Improves Quality of Remote Sensing Semantic SegmentationPreprint
Illarionova, S., Nesteruk, S., Shadrin, D., Ignatiev, V., Pukalchik, M., & Oseledets, I. V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.05516 · Cited by 4
“There are different approaches to address dataset limitations: pseudo labeling, special architectures development, transfer learning [46], [3], [2], [29].”
Enhancing Driver Emotion Recognition Through Deep Ensemble Classification
Zaman, F., Xu, Z., Hussain, A., Ullah, A., & Zaman, K. (2025) · Journal of Intelligent and Connected Vehicles · doi:10.26599/jicv.2025.9210055 · Cited by 3
Late Fusion Model for Emotion Recognition from Facial Expressions and Biosignals in a Dataset of Children with Autism Spectrum Disorder
Kiejdo, D., Prądzinska, M. D., & Zawadzka, T. (2025) · Sensors · doi:10.3390/s25247485 · Cited by 3
“While capable of detecting high-frequency classes such as Happy , it struggled to differentiate between less represented expressions, which is consistent with challenges reported in the literature regarding facial emotion analysis [38].”
Robust CNN for facial emotion recognition and real-time GUI
Ali, I., & Ghaffar, F. (2024) · AIMS Electronics and Electrical Engineering · doi:10.3934/electreng.2024010 · Cited by 4
Revolutionizing Student Engagement: Real-Time Emotion Detection and Interest Identification in Live Video Streams
Govind, M., Humaid, A. S., & Malu, G. (2025) · Lecture notes in electrical engineering · doi:10.1007/978-981-97-4711-5_29 · Cited by 3
Enhancing emotion classification on the ISEAR dataset using fine-tuning and data augmentation with hybrid transformer models
Muhammad, U., Ullah, K., Hussain, I., Almutairi, S. S., Syed, I., & Abohashrh, M. (2025) · PeerJ Computer Science · doi:10.7717/peerj-cs.2984 · Cited by 3
Combination of valence-sensitive loss with restrictive center loss for facial expression recognition
Wang, X., Peng, M., Hu, M. Q., Jin, C., & Ren, F. (2018) · International Conference on Advanced Computational Intelligence (ICACI) · doi:10.1109/icaci.2018.8377514 · Cited by 3
Semi-supervised Emotion Recognition using Inconsistently Annotated Data
Happy, S. L., Dantcheva, A., & Brémond, F. (2020) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg47880.2020.00075 · Cited by 3
“Transfer learning addresses the limited data problem in emotion recognition [6], [8] by using a pretrained network trained with large face data in the context of face recognition.”
BackgroundMethod
Rethinking Binary Hyperparameters for Deep Transfer Learning
Plested, J., Shen, X., & Gedeon, T. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-92270-2_40 · Cited by 4
“The relationship between the source and the target dataset and how well the pre-trained weights ﬁt the target task has been shown to be important to transfer learning performance numerous times [28, 22, 6, 8, 33, 21, 20, 37, 27, 42, 18].”
Background
Implementation of Transfer Learning for Covid-19 and Pneumonia Disease Detection Through Chest X-Rays Based on Web
Apsari, N. E., Sugiyanto, S., & Handajani, S. S. (2022) · Indonesian Journal of Applied Statistics · doi:10.13057/ijas.v5i1.59442 · Cited by 3
AI_deation: A Creative Knowledge Mining Method for Design Exploration
Palamas, G., Guerra, A. M., & Møsbæk, L.-D. (2022) · EAI Endorsed Transactions on Creative Technologies · doi:10.4108/eetct.v9i3.2685 · Cited by 2
“In response to this, studies have shown that transfer learning methods can be used to counteract the lack of data [20, 21].”
Analysis for Sequential Frame with Facial Emotion Recognition Based on CNN and LSTM
Hung, J. C.-S., & Xu, S.-L. (2022) · Lecture notes in electrical engineering · doi:10.1007/978-981-19-4132-0_10 · Cited by 3
Emotional Classification Method (ECW): A Methodology for Measuring Emotional Sustainability in a Work Environment Utilizing Artificial Intelligence
Roldan-Castellanos, F. A., Pérez-Olguín, I. J. C., Gutiérrez-Vázquez, A., Méndez‐González, L. C., & Rodríguez‐Picón, L. A. (2023) · Axioms · doi:10.3390/axioms12020097 · Cited by 7
Recognition of Masked Facial Expressions Based on Transfer Learning and Data Augmentation
Liu, Y., Zhao, K., & Fu, X. (2023) · International Annual Conference on Complex Systems and Intelligent Science (CSIS-IAC) · doi:10.1109/csis-iac60628.2023.10364081 · Cited by 3
“The pre-trained weights on the ImageNet dataset are transferred to the network, and then the network is fine-tuned in two stages, first on datasets related to facial expressions, and then on the target dataset [39].”
Method
Neural Network-Based Crowd Counting Systems: State of the Art, Challenges, and Perspectives
George, A., Vinothina, V., & Gnanadurai, J. B. (2023) · Journal of Advances in Information Technology · doi:10.12720/jait.14.6.1450-1460 · Cited by 3
“Safety of people is ensured with the help of crowd analysis [24–27].”
Background
Group Affect Prediction Using Emotion Heatmaps and Scene Information.
Shamsi, S., Rawat, B. P. S., & Wadhwa, M. (2017) · Cited by 3
Discriminative Feature Learning Using Two-Stage Training Strategy for Facial Expression Recognition
Gan, L., Zou, Y., & Zhang, C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-30508-6_32 · Cited by 3
“In [15], a two-stage ﬁne-tuning method follows a transfer learning approaches.”
Method
Deep Learning-Based Software Energy Consumption Profiling
Öztürk, M. M. (2020) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-030-36178-5_7 · Cited by 3
Acquiring New Categories by Self Data Gathering with Bayesian Attractor Model
Otoshi, T., Arakawa, S., Murata, M., Wang, K., Hosomi, T., & Kanoh, T. (2020) · IEEE International Conference on Human-Machine Systems (ICHMS) · doi:10.1109/ichms49158.2020.9209496 · Cited by 3
“This means the learning for the neural network is conducted by the transfer learning [6].”
Background
A Comprehensive Survey of Different Phases for Involuntary System for Face Emotion Recognition
Pandit, D., & Jadhav, S. (2021) · Communications in computer and information science · doi:10.1007/978-981-16-0507-9_15 · Cited by 3
Supervised Cross-Database Transfer Learning-Based Facial Expression Recognition
Gupta, A., & Balakrishnan, R. (2021) · Algorithms for intelligent systems · doi:10.1007/978-981-33-6691-6_30 · Cited by 3
Emotion Recognition and Drowsiness Detection using Digital Image Processing and Python
Pramoda, R., S, A. P., Athul, B. S. K., Bharath, B., & Reddy, B. N. (2021) · International Journal of Scientific Research in Science and Technology · doi:10.32628/ijsrst2183209 · Cited by 3
Facial Expression Features Analysis With Transfer Learning
Khine, W. S. S., Siritanawan, P., & Kotani, K. (2022) · International Conference on Knowledge and Systems Engineering/Proceedings · doi:10.1109/kse56063.2022.9953754 · Cited by 3
Semi-supervised clustering of unknown expressions
Jalal, A., & Tariq, U. (2019) · Pattern Recognition Letters · doi:10.1016/j.patrec.2019.01.004 · Cited by 4
Customer Feedback Through Facial Expression Recognition Using Deep Neural Network
Dandgawhal, T. P., & Momin, B. F. (2020) · Advances in intelligent systems and computing · doi:10.1007/978-981-15-4029-5_41 · Cited by 3
Visualization as Intermediate Representations (VLAIR) for Human Activity Recognition
Jiang, A., Nacenta, M. A., Terzić, K., & Ye, J. (2020) · International Conference on Pervasive Computing Technologies for Healthcare · doi:10.1145/3421937.3422015 · Cited by 3
“We choose hyperparameters for learning rate and the optimizer in line with the state-of-the-art vision-based approaches in deep learning [37].”
Method
DeepHBSP: A Deep Learning Framework for Predicting Human Blood-Secretory Proteins Using Transfer Learning
Du, W., Sun, Y., Bao, H.-M., Chen, L., Li, Y., & Liang, Y. (2021) · Journal of Computer Science and Technology · doi:10.1007/s11390-021-0851-9 · Cited by 3
“Moreover, transfer learning, which refers to improving learning by transferring knowledge learned from related tasks to new tasks [30] , has been successfully applied to deep learning frameworks for small-sample problems [31] .”
Background
A Deep Learning Approach with Data Augmentation to Recognize Facial Expressions in Real Time
Ahmed, T. U., Hossain, S., Hossain, M. S., Islam, R. U., & Andersson, K. (2022) · Lecture notes in networks and systems · doi:10.1007/978-981-16-7597-3_40 · Cited by 3
Gradient Privacy-Preserving In Federated Learning via Proxy Re-Encryption
Sun, Y., & Yang, Y. (2022) · International Conference on Information Science and Systems (ICISS) · doi:10.1145/3561877.3561893 · Cited by 3
“3561893 prediction accuracy based on training vast volumes of data, deep neural networks have been used in a variety of application situations, such as computer vision [7], sentiment analysis [8], and natural language processing [9].”
Background
Face mask detection in foggy weather from digital images using transfer learning
Kansal, I., Khullar, V., Popli, R., Verma, J., & Kumar, R. (2023) · The Imaging Science Journal · doi:10.1080/13682199.2023.2218222 · Cited by 3
Contemplating Visual Emotions: Understanding and Overcoming Dataset BiasPreprint
Panda, R., Zhang, J., Li, H., Lee, J., Lu, X., & Roy–Chowdhury, A. K. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.02212 · Cited by 3
“One such task is to recognize emotions expressed by images which has gained momentum in last couple of years in both academia and industries [63,30,40,43,62,4].”
Background
Transferable Cost-Aware Security Policy Implementation for Malware Detection Using Deep Reinforcement LearningPreprint
Birman, Y., Hindi, S., Katz, G., & Shabtai, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.10517 · Cited by 3
Disentanglement for Discriminative Visual RecognitionPreprint
Liu, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07810 · Cited by 4
Combining Deep Transfer Learning with Signal-image Encoding for Multi-Modal Mental Wellbeing ClassificationPreprint
Woodward, K., Kanjo, E., & Tsanas, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.03711 · Cited by 3
“When using pre-trained ImageNet models and transferring this knowledge to infer seven facial expression an accuracy of 55.6% was achieved compared with the baseline performance of 39.13% [41].”
MethodInfluential
Efficient Facial Expression Analysis For Dimensional Affect Recognition Using Geometric FeaturesPreprint
Vonikakis, V., & WINKLER, S. S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.07817 · Cited by 3
“Inspired initially by the seminal work of Ekman [1], it has made significant strides, with increasingly powerful methods like deep learning being used [2, 3].”
MethodSelf-citation
Development of Video-Based Emotion Recognition System using Transfer Learning
Gunawan, T. S., Muktaruddin, M. N., Kartiwi, M., Ahmad, Y. A., Rosahdi, T. D., & Ulfiah (2022) · IEEE International Conference on Smart Instrumentation, Measurement and Applications (ICSIMA) · doi:10.1109/icsima55652.2022.9928867 · Cited by 2
“Deep CNN architectures were utilized in [1, 2].”
TweetFeel: Analyzing Emotions in the Twittersphere
Adithya, Gokulan, S., Reddy, M., Varshith, M., & Venugopalan, M. (2024) · International Conference on Knowledge Engineering and Communication Systems (ICKECS) · doi:10.1109/ickecs61492.2024.10617413 · Cited by 2
“In one of the Articles by Hong-Wei Ng [18], the authors talk about Transfer learning algorithms, which are used in deep learning approaches to emotion recognition on small datasets, utilizing trained models to improve accuracy.”
Background
Facial Emotion Recognition: A Comparison of Classic and Novel Convolutional Neural Networks through Transfer Learning
Khan, W. A., Rahman, A., Zafar, A. S., Ashfaq, A., & Karamat, F. (2024) · International Conference on Innovative Engineering Sciences and Technological Research (ICIESTR) · doi:10.1109/iciestr60916.2024.10798406 · Cited by 2
“In their study, Nguyen et al. [17] used a transfer learning model that has previously been trained using diverse datasets from other domains.”
A Systematic Review of Transfer Learning Research From the Perspective of Knowledge Graph: Visual Analysis Based on the Bibliometric Method
An, Y., Hou, N., & Lv, Z. (2026) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70084 · Cited by 3
MoVE-CNNs: Model aVeraging Ensemble of Convolutional Neural Networks for Facial Expression Recognition
Yu, J., Lim, K. M., & Lee, C. P. (2021) · Siti Hasmah Digital Library-MMU Institutiona Repository (Multimedia University) · Cited by 2
Neural Network Based Transfer Learning for Robot Path Generation
Tang, H., & Notash, L. (2022) · Journal of Mechanisms and Robotics · doi:10.1115/1.4054272 · Cited by 2
Facial Expression Synthesis and Recognition with Pre-Trained StyleGAN
Zhai, X., Ma, X., Tian, H., & Li, Y. (2023) · Chinese Control Conference (CCC) · doi:10.23919/ccc58697.2023.10240584 · Cited by 2
Picture This: A Deep Learning Model for Operational Real Estate Emissions
Gloria, B., & Höhn, B. (2023) · Journal of Sustainable Real Estate · doi:10.1080/19498276.2023.2251982 · Cited by 2

Authors: Ng, H.-W., Nguyen, V. D., Vonikakis, V., & Winkler, S.
Affiliations: University of Illinois at Urbana-Champaign (Singapore), Singapore, Singapore
University of Illinois Urbana-Champaign (US)
ORCID: Vonikakis, Vassilios
Winkler, Stefan
Venue: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction
Publisher: ACM
Published: 2015-11-09
Pages: 443-449
DOI: 10.1145/2818346.2830593
OpenAlex: W2253728219
Cited by: 657
References: 32
Citations by year: 2026: 13 · 2025: 29 · 2024: 37 · 2023: 70 · 2022: 69 · 2021: 106 · 2020: 106 · 2019: 106 · 2018: 63 · 2017: 44 · 2016: 13 · 2015: 1
FWCI: 53.84
Type: conference paper
Language: en
Open access: closed
Licence: www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (publishedVersion)
Funders: Agency for Science Technology and Research
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords: emotion recognition, facial expression recognition, still images, transfer learning, convolutional neural networks, fine-tuning, ImageNet pre-training, small datasets

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Deep Learning for Emotion Recognition on Small Datasets using Transfer Learning

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks