Cited by 235Emotion and Mood RecognitionFace recognition and analysisFace and Expression Recognition

The DOI matches its Crossref record; 68 of the 73 references it cites were found in a registry or an index.

Video-based emotion recognition in the wild using deep transfer learning and score fusion

Kaya, Heysem, Gürpınar, Furkan, Salah, Albert Ali·2017·Image and Vision Computing

DOI 10.1016/j.imavis.2017.01.012

Open this paper in CoChat — free account

Integrity signals

Registry

Crossref · 10.1016/j.imavis.2017.01.012

Cites

68 of 68 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Distinctive Image Features from Scale-Invariant Keypoints
Lowe, D. G. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029664.99615.94 · Cited by 55,589
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Multiresolution gray-scale and rotation invariant texture classification with local binary patterns
Ojala, T., Pietikainen, M., & Maenpaa, T. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2002.1017623 · Cited by 15,398
Least Squares Support Vector Machine Classifiers
Suykens, J., & Vandewalle, J. (1999) · Neural Processing Letters · doi:10.1023/a:1018628609742 · Cited by 9,481
Extreme Learning Machine for Regression and Multiclass Classification
Huang, G.-B., Zhou, H., Ding, X., & Zhang, R. (2011) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2011.2168604 · Cited by 5,575
Deep Face Recognition
Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015) · Procedings of the British Machine Vision Conference 2015 · doi:10.5244/c.29.41 · Cited by 5,479
The Extended Cohn-Kanade Dataset (CK+): A complete dataset for action unit and emotion-specified expression
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303
Return of the Devil in the Details: Delving Deep into Convolutional Nets
Chatfield, K., Simonyan, K., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.6 · Cited by 3,335
Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions
Zhao, G., & Pietikainen, M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1110 · Cited by 2,846

Flagged first, then by citations

Show all 73 references

5 of the 73 references couldn’t be matched to a known paper.

Research works printed but not linked: 2

Rao, 1971, vol. 7
Rifkin, 2003, Regularized least-squares classification, NATO Sci. Ser. Sub Ser. III Comput. Syst. Sci., 190, 131

Sorted by citations

Deep Facial Expression Recognition: A Survey
Li, S., & Deng, W. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981446 · Cited by 2,175
“[116] suggested that VGG-Face, which was trained for FR, overwhelmed ImageNet, which was developed for objected recognition.”
Background
Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review
Zhang, J., Yin, Z., Chen, P., & Nichele, S. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.01.011 · Cited by 806
A Review of Deep Transfer Learning and Recent Advancements
Iman, M., Arabnia, H. R., & Rasheed, K. (2023) · Technologies · doi:10.3390/technologies11020040 · Cited by 789
“[32] 2017 Video-based emotion recognition in the wild using deep transfer learning and score fusion video (audio & visual) yes Finetuning yes VGG-Face Human science / psychology”
Method
Emotion recognition using deep learning approach from audio–visual emotional big data
Hossain, M. S., & Muhammad, G. (2018) · Information Fusion · doi:10.1016/j.inffus.2018.09.008 · Cited by 460
Local Learning With Deep and Handcrafted Features for Facial Expression Recognition
Georgescu, M.-I., Ionescu, R. T., & Popescu, M. (2019) · IEEE Access · doi:10.1109/access.2019.2917266 · Cited by 351
“[30] employed SIFT, HOG and Local Gabor Binary Patterns (LGBP).”
BackgroundMethodInfluential
Multi-Scale Metric Learning for Few-Shot Learning
Jiang, W. R., Huang, K., Geng, J., & Deng, X. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.2995754 · Cited by 301
“…last layer with replacing the new softmax layers that are relevant to our own task, the universal features extracted by the pre-trained network that are also relevant to the new tasks are reserved, and the network will focus on learning dataset-speciﬁc features in the subsequent layers [39], [40].”
Background
Integrating Machine Learning with Human Knowledge
Deng, C., Ji, X., Rainey, C., Zhang, J., & Lu, W. (2020) · iScience · doi:10.1016/j.isci.2020.101656 · Cited by 243
Deep learning for human affect recognition: Insights and new developments
Rouast, P. V., Adam, M. T. P., & Chiong, R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2890471 · Cited by 243
Multimodal Co-learning: Challenges, applications with datasets, recent advances and future directions
Rahate, A., Walambe, R., Ramanna, S., & Kotecha, K. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.12.003 · Cited by 214
“[118] Emotion Prediction Audio, Video GAN CREMA-D, RAVDEES Conditional GAN [121] Emotion Prediction Audio, Video Pretraining and Fusion EmotiW, ChaLearn Pre-training on VGG Face and finetuning on FER [122] Emotion Prediction, Cross-media Retrieval a) Audio, Video b) Image, Text Domain Adaptation IEMOCAP & AFEW, MSCOCO…”
BackgroundMethodInfluential
Deep learning-based late fusion of multimodal information for emotion classification of music video
Pandeya, Y. R., & Lee, J. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-08836-3 · Cited by 236
“Video is a sequence of correlated pictures information widely exploited in diversified domain such as classification, object detection and tracking [64], 3D object retrieval [16], human action recognition [15], human emotion prediction [27, 36], object segmentation [65],”
Background
Temporal multimodal fusion for video emotion classification in the wild
Vielzeuf, V., Pateux, S., & Jurie, F. (2017) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3136755.3143011 · Cited by 183
“, [18, 23]) for representing faces in the context of emotion recognition, especially in the EmotiW”
Background
Multimodal big data affective analytics: A comprehensive survey using text, audio, visual and physiological signals
Shoumy, N. J., Ang, L.-M., Seng, K. P., Rahaman, D. M. M., & Zia, T. (2020) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2019.102447 · Cited by 171
A comprehensive survey on deep facial expression recognition: challenges, applications, and future guidelines
Sajjad, M., Ullah, F. U. M., Ullah, M., Christodoulou, G., Cheikh, F. A., Hijji, M., Muhammad, K., & Rodrigues, J. J. P. C. (2023) · Alexandria Engineering Journal · doi:10.1016/j.aej.2023.01.017 · Cited by 214
LSTM-Based Emotion Detection Using Physiological Signals: IoT Framework for Healthcare and Distance Learning in COVID-19
Awais, M., Raza, M., Singh, N. K., Bashir, K., Manzoor, U., Islam, S. U., & Rodrigues, J. J. P. C. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3044031 · Cited by 166
“The use of an autoassociative neural network-based face emotion recognition from the video [9] and deep transfer learning and score fusion was also carried out [10].”
Method
Spontaneous Speech Emotion Recognition Using Multiscale Deep Convolutional LSTM
Zhang, S., Zhao, X., & Tian, Q. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2947464 · Cited by 156
Predicting CO and NOxemissions from gas turbines: novel data and abenchmark PEMS
Kaya, H., Tüfekçi, P., & Uzun, E. (2019) · TURKISH JOURNAL OF ELECTRICAL ENGINEERING & COMPUTER SCIENCES · doi:10.3906/elk-1807-87 · Cited by 119
“Extreme learning machines ELMs [16–18] are recently used in many machine learning applications for classification and regression, including wind-speed forecasting [19] and audio-visual emotion recognition [20, 21].”
Self-citation
Facial Expression Recognition in Videos Using Dynamic Kernels
Perveen, N., Roy, D., & Mohan, C. K. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.3011846 · Cited by 116
“In order to further improve the recognition performance, other modalities like audio features are also fused with video features for integrating complementary information [44].”
Background
A Survey of AI-Based Facial Emotion Recognition: Features, ML & DL Techniques, Age-Wise Datasets and Future Directions
Dalvi, C., Rathod, M., Patil, S., Gite, S., & Kotecha, K. V. (2021) · IEEE Access · doi:10.1109/access.2021.3131733 · Cited by 128
Modeling, Recognizing, and Explaining Apparent Personality From Videos
Escalante, H. J., Kaya, H., Salah, A. A., Escalera, S., Güçlütürk, Y., Guclu, U., Baró, X., Guyon, I., Jacques, J. C. S., Madadi, M., et al. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2973984 · Cited by 120
“Facial features are combined with the Local Gabor Binary Patterns from Three Orthogonal Planes (LGBP-TOP) video descriptor, shown to be effective in emotion recognition [1], [36].”
MethodSelf-citation
On-Road Driver Emotion Recognition Using Facial Expression
Xiao, H., Li, W., Zeng, G., Wu, Y., Xue, J., Zhang, J., Li, C., & Guo, G. (2022) · Applied Sciences · doi:10.3390/app12020807 · Cited by 112
“With the lack of large-scale datasets, fine-tuning has been widely investigated for facial expression recognition [44,45].”
Background
Emotion recognition using secure edge and cloud computing
Hossain, M. S., & Muhammad, G. (2019) · Information Sciences · doi:10.1016/j.ins.2019.07.040 · Cited by 104
A comprehensive review of facial expression recognition techniques
Rashmi Adyapady, R., & Annappa, B. (2022) · Multimedia Systems · doi:10.1007/s00530-022-00984-w · Cited by 101
“[38] has proposed a multimodal approach.”
Method
Efficient facial representations for age, gender and identity recognition in organizing photo albums using multi-output ConvNet
Savchenko, A. (2019) · PeerJ Computer Science · doi:10.7717/peerj-cs.197 · Cited by 91
Deep-Learning-Based Multimodal Emotion Classification for Music Videos
Pandeya, Y. R., Bhattarai, B., & Lee, J. (2021) · Sensors · doi:10.3390/s21144927 · Cited by 89
“Affective computing of movies was later studied in [48] by using a neural network and face and speech data.”
A Review of Generalizable Transfer Learning in Automatic Emotion Recognition
Feng, K., & Chaspari, T. (2020) · Frontiers in Computer Science · doi:10.3389/fcomp.2020.00009 · Cited by 82
“Various studies on deep transfer learning for automatic emotion recognition have yielded promising results on speech and image datasets (Gideon et al., 2017; Kaya et al., 2017; Abdelwahab and Busso, 2018; Li and Chaspari, 2019.”
BackgroundMethod
In search of a robust facial expressions recognition model: A large-scale visual cross-corpus study
Рюмина, Е. В., Dresvyanskiy, D., & Karpov, A. A. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.10.013 · Cited by 91
Video based Traffic Forecasting using Convolution Neural Network Model and Transfer Learning Techniques
Kumar, T. S. (2020) · Journal of Innovative Image Processing · doi:10.36548/jiip.2020.3.002 · Cited by 78
Recognizing Induced Emotions of Movie Audiences from Multimodal Information
Muszyński, M., Tian, L., Lai, C., Moore, J. D., Κωστούλας, Θ., Lombardo, P., Pun, T., & Chanel, G. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2902091 · Cited by 79
Group-level emotion recognition using transfer learning from face identification
Rassadin, A. G., Gruzdev, A., & Savchenko, A. (2017) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3136755.3143007 · Cited by 73
“or random forests are applied for these features to recognize facial expressions. In fact, the state-of-the-art performance on the EmotiW dataset has been achieved by such transfer learning approach [24]. However, in this paper we propose to use CNN trained to identify faces rather than recognize emotions. In fact, th…”
Method
Learning deep facial expression features from image and optical flow sequences using 3D CNN
Zhao, J., Mao, X., & Zhang, J. (2018) · The Visual Computer · doi:10.1007/s00371-018-1477-y · Cited by 75
“[54] Seven classes Transfer Learning & Score Fusion 0.”
Background
Multi-modal Score Fusion and Decision Trees for Explainable Automatic Job Candidate Screening from Video CVs
Kaya, H., Gürpınar, F., & Salah, A. A. (2017) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.210 · Cited by 67
“We compare and combine deep facial features with a spatio-temporal descriptor called Local Gabor Binary Patterns from Three Orthogonal Planes (LGBP-TOP) [3] that is shown to be effective in emotion recognition [17].”
MethodSelf-citation
Multimodal emotion recognition: A comprehensive review, trends, and challenges
Ramaswamy, M. P. A., & Palaniswamy, S. (2024) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1563 · Cited by 70
PRATIT: a CNN-based emotion recognition system using histogram equalization and data augmentation
Mungra, D., Agrawal, A., Sharma, P., Tanwar, S., & Obaidat, M. S. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-019-08397-0 · Cited by 62
“It is amongst the most essential features of emotion recognition [22], which helps us”
Background
Facial emotion recognition using temporal relational network: an application to E-learning
Pise, A. A., Vadapalli, H., & Sanders, I. D. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-10133-y · Cited by 58
Emotion, age, and gender classification in children’s speech by humans and machines
Kaya, H., Salah, A. A., Karpov, A. A., Frolova, O. V., Grigorev, A., & Lyakso, E. E. (2017) · Computer Speech & Language · doi:10.1016/j.csl.2017.06.002 · Cited by 57
Self-citation
DTL-I-ResNet18: facial emotion recognition based on deep transfer learning and improved ResNet18
Helaly, R., Messaoud, S., Bouaafia, S., Hajjaji, M. A., & Mtibaa, A. (2023) · Signal Image and Video Processing · doi:10.1007/s11760-023-02490-6 · Cited by 61
“Similarly, authors, in [63], proposed a technique that combines deep and handcrafted parameters by using a single CNN model and various handcrafted features.”
Background
The Turkish Audio-Visual Bipolar Disorder Corpus
Çiftçi, E., Kaya, H., Güleç, H., & Salah, A. A. (2018) · Asian Conference on Affective Computing and Intelligent Interaction (ACII Asia) · doi:10.1109/aciiasia.2018.8470362 · Cited by 55
“learning methods learn projection matrices that map input features to the target (dependent) variable via regularized least squares, and hence, they are both fast and accurate [25, 26], Each method has a parameter that regularizes the model complexity against the training set classification error.”
BackgroundMethodSelf-citation
Transfer Learning Based Fault Diagnosis with Missing Data Due to Multi-Rate Sampling
Chen, D., Yang, S., & Zhou, F. (2019) · Sensors · doi:10.3390/s19081826 · Cited by 56
Brain signal based human emotion analysis by circular back propagation and Deep Kohonen Neural Networks
Hemanth, D. J., Anitha, J., & Sơn, L. H. (2018) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2018.04.006 · Cited by 53
“Deep Convolutional Neural Network was used to recognize emotions in human beings [9].”
Method
Generative adversarial networks and their application to 3D face generation: A survey
Toshpulatov, M., Lee, W., & Lee, S. (2021) · Image and Vision Computing · doi:10.1016/j.imavis.2021.104119 · Cited by 56
Fusing dynamic deep learned features and handcrafted features for facial expression recognition
Fan, X., & Tjahjadi, T. (2019) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2019.102659 · Cited by 50
Affective Robot Story-Telling Human-Robot Interaction: Exploratory Real-Time Emotion Estimation Analysis Using Facial Expressions and Physiological Signals
Val-Calvo, M., Álvarez, J. R., Vicente, J. M. F., & Fernández, E. (2020) · IEEE Access · doi:10.1109/access.2020.3007109 · Cited by 51
Context-Aware Emotion Recognition Based on Visual Relationship Detection
Hoang, M.-H., Kim, S.-H., Yang, H.-J., & Lee, G.-S. (2021) · IEEE Access · doi:10.1109/access.2021.3091169 · Cited by 48
Multimodal emotion recognition based on a fusion of audiovisual information with temporal dynamics
Salas-Cáceres, J., Lorenzo-Navarro, J., Freire-Obregón, D., & Castrillón-Santana, M. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-20227-6 · Cited by 45
“UAR provides a balanced measure of the model’s ability to correctly identify each emotion class, regardless of class distribution in the dataset [19, 37].”
Modeling multimodal cues in a deep learning-based framework for emotion recognition in the wild
Pini, S., Ahmed, O. B., Cornia, M., Baraldi, L., Cucchiara, R., & Huet, B. (2017) · Proceedings of the 19th ACM International Conference on Multimodal Interaction · doi:10.1145/3136755.3143006 · Cited by 44
SSA-ICL: Multi-domain adaptive attention with intra-dataset continual learning for Facial expression recognition
Gao, H., Wu, M., Chen, Z., Li, Y., Wang, X., An, S., Li, J., & Liu, C. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.11.025 · Cited by 48
ICA-Evolution Based Data Augmentation with Ensemble Deep Neural Networks Using Time and Frequency Kernels for Emotion Recognition from EEG-Data
Kang, J.-S., Kavuri, S., & Lee, M. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2942587 · Cited by 44
Image-based soft drink type classification and dietary assessment system using deep convolutional neural network with transfer learning
Hafiz, R., Haque, M. R., Rakshit, A., & Uddin, M. S. (2020) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2020.08.015 · Cited by 42
“The transfer learning technique also successfully utilized in video-based emotion recognition (Kaya et al., 2017), iris recognition (Nguyen et al., 2018), end-to-end airplane detection (Chen et al., 2018), and poverty mapping (Xie et al., 2016).”
Feature-Level and Model-Level Audiovisual Fusion for Emotion Recognition in the Wild
Cai, J., Meng, Z., Khan, A. S., Li, Z., O’Reilly, J. D., Han, S., Liu, P., Chen, M., & Tong, Y. (2019) · IEEE International Conference on Multimedia Information Processing and Retrieval (MIPR) · doi:10.1109/mipr.2019.00089 · Cited by 42
“…visual – 57.20 Kim et. al [26] audio, visual 50.39 57.12 Yan et. al [48] audio, visual – 56.66 Wu et. al [45] audio, visual – 55.31 Kaya et. al [24] audio, visual 57.02 54.55 Ding et. al [11] audio, visual 51.20 53.96 Yao et. al [50] audio, visual 49.09 53.80 Kaya et. al [23] audio, visual…”
Influential
A novel enhanced convolution neural network with extreme learning machine: facial emotional recognition in psychology practices
Banskota, N., Alsadoon, A., Prasad, P. W. C., Dawoud, A., Rashid, T. A., & Alsadoon, O. H. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13567-8 · Cited by 41
“[8] examined the large variance in wild Facial Emotion Recognition that occurred due to uneven illumination, cluttered background as well as a sudden change in facial expression for multimodal audio-visual recognition resulting in audio lag.”
BackgroundMethodInfluential
Deep Continual Learning for Emerging Emotion Recognition
Thuseethan, S., Rajasegarar, S., & Yearwood, J. L. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3116434 · Cited by 41
“Inspiredby the ability to adapt new classes across tasks, the transfer learning approach has been utilized in a variety of applications [38], [39].”
Background
Applying Segment-Level Attention on Bi-Modal Transformer Encoder for Audio-Visual Emotion Recognition
Hsu, J.-H., & Wu, C. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3258900 · Cited by 39
“The VGGNet model has been used by many facial expression recognition systems and achieves good performance [26], [27], [28], [29].”
Method
Visual Scene-aware Hybrid Neural Network Architecture for Video-based Facial Expression Recognition
Lee, M. K., Choi, D. Y., Kim, D. H., & Song, B. C. (2019) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2019.8756551 · Cited by 39
“On the other hand, in [12, 13, 14], so-called transfer learning is used in which networks are initially learned for a relatively large dataset and then are re-learned for a target dataset.”
Pre-training Strategies and Datasets for Facial Representation Learning
Bulat, A., Cheng, S., Yang, J., Garbett, A., Sánchez, E., & Tzimiropoulos, G. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19778-9_7 · Cited by 38
Multi-modal Correlated Network for emotion recognition in speech
Ren, M., Nie, W., Liu, A.-A., & Su, Y. (2019) · Visual Informatics · doi:10.1016/j.visinf.2019.10.003 · Cited by 39
A Hybrid Driver Fatigue and Distraction Detection Model Using AlexNet Based on Facial Features
Anber, S., Alsaggaf, W., & Shalash, W. M. (2022) · Electronics · doi:10.3390/electronics11020285 · Cited by 37
Adversarial Domain Generalized Transformer for Cross-Corpus Speech Emotion Recognition
Gao, Y., Wang, L., Liu, J., Dang, J., & Okada, S. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3290795 · Cited by 36
“Kaya et al. [41] combined the pretrained visual geometry group (VGG) model with a common feature extractor to learn the visual features and then fused these features with the audio features at the decision level to realize multimodal emotion recognition.”
Background
Convolutional Neural Network Based Offline Signature Verification Application
Yapıcı, M. M., Tekerek, A., & Topaloğlu, N. (2018) · International Congress on Big Data, Deep Learning and Fighting Cyber Terrorism (IBIGDELFT) · doi:10.1109/ibigdelft.2018.8625290 · Cited by 37
End-to-End Modeling and Transfer Learning for Audiovisual Emotion Recognition in-the-Wild
Dresvyanskiy, D., Рюмина, Е. В., Kaya, H., Markitantov, M., Karpov, A. A., & Minker, W. (2022) · Multimodal Technologies and Interaction · doi:10.3390/mti6020011 · Cited by 35
“Unlike former studies that reported only the visual modality contributing to the recognition of disgust [40], in our study, we observed the contribution of the acoustic system to this class.”
BackgroundMethodResultInfluentialSelf-citation
Time efficient real time facial expression recognition with CNN and transfer learning
Podder, T., Bhattacharya, D., & Majumdar, A. (2022) · Sadhana · doi:10.1007/s12046-022-01943-x · Cited by 35
A Multimodal Emotion Recognition System Using Facial Landmark Analysis
Rahdari, F., Rashedi, E., & Eftekhari, M. (2018) · Iranian Journal of Science and Technology Transactions of Electrical Engineering · doi:10.1007/s40998-018-0142-9 · Cited by 32
Multimodal Emotion and Sentiment Modeling From Unstructured Big Data: Challenges, Architecture, & Techniques
Seng, J. K. P., & Ang, L.-M. (2019) · IEEE Access · doi:10.1109/access.2019.2926751 · Cited by 32
“[63] proposed a video based emotion recognition using deep transfer learning and score fusion.”
Background
Teacher-Student Training and Triplet Loss for Facial Expression Recognition under Occlusion
Georgescu, M.-I., & Ionescu, R. T. (2021) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr48806.2021.9412493 · Cited by 33
“We note that most works studied facial expression recognition from static images, but there are also some works designed for video [14], [33].”
Background
Transfer-based adaptive tree for multimodal sentiment analysis based on user latent aspects
Rahmani, S., Hosseini, S., Zall, R., Kangavari, M. R., Kamran, S., & Hua, W. (2022) · Knowledge-Based Systems · doi:10.1016/j.knosys.2022.110219 · Cited by 33
Hybrid Multi-Attention Network for Audio–Visual Emotion Recognition Through Multimodal Feature Fusion
Moorthy, S., & Moon, Y.-K. (2025) · Mathematics · doi:10.3390/math13071100 · Cited by 31
Recognizing Facial Expressions of Occluded Faces Using Convolutional Neural Networks
Georgescu, M.-I., & Ionescu, R. T. (2019) · Communications in computer and information science · doi:10.1007/978-3-030-36808-1_70 · Cited by 31
An Enhanced Facial Expression Recognition Model Using Local Feature Fusion of Gabor Wavelets and Local Directionality Patterns
Bellamkonda, S., & Gopalan, N. P. (2019) · International Journal of Ambient Computing and Intelligence · doi:10.4018/ijaci.2020010103 · Cited by 31
Reducing Videoconferencing Fatigue through Facial Emotion Recognition
Rößler, J., Sun, J., & Gloor, P. A. (2021) · Future Internet · doi:10.3390/fi13050126 · Cited by 32
“In addition, Choi and Song [7] showed that their framework—a combination of CNN and LSTM—is not only robust and effective on artificial facial expression data sets (actors playing various emotions), but is also robust and effective on data sets collected in the wild.”
MethodInfluential
A Graph-Structured Representation with BRNN for Static-based Facial Expression Recognition
Zhong, L., Bai, C., Li, J., Chen, T., Li, S., & Liu, Y. (2019) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2019.8756615 · Cited by 29
“In the field of facial expression recognition, many deep learning methods were proposed [5, 6, 7].”
Method
Towards Facial Expression Recognition for On-Farm Welfare Assessment in Pigs
Hansen, M., Baxter, E., Rutherford, K., Futro, A., Smith, M. L., & Smith, L. N. (2021) · Agriculture · doi:10.3390/agriculture11090847 · Cited by 30
Explaining First Impressions: Modeling, Recognizing, and Explaining Apparent Personality from VideosPreprint
Escalante, H. J., Kaya, H., Salah, A. A., Escalera, S., Güçlütürk, Y., Güçlü, U., Baró, X., Guyon, I., Jacques, J. C. S., Madadi, M., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.00745 · Cited by 44
“Then this network is ﬁne-tuned for emotion [47], using more than 30K training images of the FER-2013 dataset [34].”
MethodSelf-citation
MUMBAI: multi-person, multimodal board game affect and interaction analysis dataset
Doyran, M., Schimmel, A., Baki, P., Ergin, K., Türkmen, B., Salah, A. A. A., Bakkes, S. C. J., Kaya, H., Poppe, R. W., & Salah, A. A. (2021) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-021-00364-0 · Cited by 32
Self-citation
Boosting Face in Video Recognition via CNN Based Key Frame Extraction
Qi, X., Liu, C., & Schuckers, S. (2018) · International Conference on Biometrics · doi:10.1109/icb2018.2018.00030 · Cited by 28
“Gallery Settings: It is widely accepted that gallery picking can affect the FR performance, as being quantified in [12].”
Background
Digital trace data collection through data donationPreprint
Boeschoten, L., Ausloos, J., Möller, J. E., Araujo, T., & Oberski, D. L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.09851 · Cited by 28
Emotion assessment and application in human–computer interaction interface based on backpropagation neural network and artificial bee colony algorithm
Liu, J., Ang, M. C., Chaw, J. K., Kor, A.-L., & Ng, K. W. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.120857 · Cited by 27
A discriminative deep association learning for facial expression recognition
Jin, X., Sun, W., & Jin, Z. (2019) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-019-01024-2 · Cited by 27
“In [13, 14], deep network is pre-trained on large Face Recognition(FR) data, and further fine-tuned on facial expression dataset.”
Method
A Hybrid Model for Bipolar Disorder Classification from Visual Information
Abaei, N., & Osman, H. A. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054648 · Cited by 26
Meta-transfer learning for emotion recognition
Nguyen, D., Nguyen, T. T., Sridharan, S., Denman, S., Nguyen, T. T., Dean, D., & Fookes, C. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08248-y · Cited by 28
“Examples of this approach include [18, 20–22, 28, 46, 47].”
Background
Jointly Aligning and Predicting Continuous Emotion Annotations
Khorram, S., McInnis, M. G., & Provost, E. M. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2917047 · Cited by 25
“Another future plan is to evaluate the performance of the proposed network over other physiological and behavioral modalities such as: video [79], [80], body language [81], [82] and EEG [83], [84].”
Background
Application of combined transfer learning and convolutional neural networks to optimize plasma spraying
Zhu, J., Wang, X., Kou, L., Zheng, L., & Zhang, H. (2021) · Applied Surface Science · doi:10.1016/j.apsusc.2021.150098 · Cited by 24
ATL-BP: A Student Engagement Dataset and Model for Affect Transfer Learning for Behavior Prediction
Ruiz, N., Yu, H., Allessio, D., Jalal, M., Joshi, A., Murray, T., Magee, J., Delgado, K. A. M., Ablavsky, V., Sclaroff, S., et al. (2022) · IEEE Transactions on Biometrics Behavior and Identity Science · doi:10.1109/tbiom.2022.3210479 · Cited by 29
“…prediction system to speciﬁc individuals [8], [9], [10], [11], [12], [13], improving results on a benchmark by ﬁne-tuning neural networks that are pre-trained on external datasets for a similar prediction task [14], or improving results by pre-training on a related facial analysis task [15], [16].”
Background
Joint low rank embedded multiple features learning for audio–visual emotion recognition
Wang, Z., Wang, L., & Huang, H. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.01.017 · Cited by 23
Pain-attentive network: a deep spatio-temporal attention model for pain estimation
Huang, D. J., Xia, Z., Mwesigye, J., & Feng, X. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-09397-1 · Cited by 25
Dynamic Facial Models for Video-Based Dimensional Affect Estimation
Song, S., Sánchez, E., Tellamekala, M. K., Shen, L., Johnston, A., & Valstar, M. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00200 · Cited by 23
MobileEmotiFace: Efficient Facial Image Representations in Video-Based Emotion Recognition on Mobile Devices
Demochkina, P., & Savchenko, A. V. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68821-9_25 · Cited by 23
Text-based Interpretable Depression Severity Modeling via Symptom Predictions
Steijn, F. V., Soğancıoğlu, G., & Kaya, H. (2022) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3536221.3556579 · Cited by 24
“paralinguistic and afective computing tasks [4, 19, 21].”
Self-citation
LSTM Based Cross-corpus and Cross-task Acoustic Emotion Recognition
Kaya, H., Fedotov, D., Yeşilkanat, A., Verkholyak, O., Zhang, Y., & Karpov, A. A. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-2298 · Cited by 22
“In line with experience on former challenge corpora [10, 11, 20], we investigate feature level fusion and two variants of score level fusion.”
BackgroundMethodSelf-citation
Multi-view laplacian eigenmaps based on bag-of-neighbors for RGB-D human emotion recognition
Liu, S., Guo, S., Wang, W., Qiao, H., Wang, Y., & Luo, W. (2019) · Information Sciences · doi:10.1016/j.ins.2019.08.035 · Cited by 22
Teacher–student training and triplet loss to reduce the effect of drastic face occlusion
Georgescu, M.-I., Duţǎ, G.-E., & Ionescu, R. T. (2021) · Machine Vision and Applications · doi:10.1007/s00138-021-01270-x · Cited by 23
“Although the majority of works studied facial expression recognition from static images, there is a body of works focusing on video [27,39].”
Background
Internet of emotional people: Towards continual affective computing cross cultures via audiovisual signals
Han, J., Zhang, Z., Pantić, M., & Schuller, B. W. (2020) · Future Generation Computer Systems · doi:10.1016/j.future.2020.08.002 · Cited by 22
“For example, the studies [44,45] have shown that the models pre-trained on large-scale datasets for other classification tasks can be fine-tuned on emotion recognition tasks to learn acoustic and visual representations.”
A Systematic Literature Review on Machine Learning Algorithms for Human Status Detection
Sardar, S. K., Kumar, N., V, & Lee, S. C. (2022) · IEEE Access · doi:10.1109/access.2022.3190967 · Cited by 24
Incongruity-Aware Cross-Modal Attention for Audio-Visual Fusion in Dimensional Emotion Recognition
Praveen, R. G., & Alam, J. (2024) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2024.3422823 · Cited by 21
“Most of the existing fusion models fall into this category, where fusion often occurs through concatenating features [16], [17] or individual modal predictions [18].”
Background
Incipient Fault Diagnosis Based on DNN with Transfer Learning
Chen, D., Yang, S., & Zhou, F. (2018) · International Conference on Control, Automation and Information Sciences (ICCAIS) · doi:10.1109/iccais.2018.8570702 · Cited by 24
Meta transfer learning for facial emotion recognition
Nguyen, D., Nguyen, K., Sridharan, S., Abbasnejad, I., Dean, D., & Fookes, C. (2018) · QUT ePrints (Queensland University of Technology) · Cited by 21
Audio-Visual Emotion Recognition Using a Hybrid Deep Convolutional Neural Network based on Census Transform
Cornejo, J. Y. R., & Pedrini, H. (2019) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc.2019.8914193 · Cited by 21
“region, using texture filters, such as Gabor wavelets and Local Binary Patterns (LBP) [13], [20].”
Method
Multimodal affect analysis of psychodynamic play therapy
Halfon, S., Doyran, M., Türkmen, B., Oktay, E., & Salah, A. A. (2020) · Psychotherapy Research · doi:10.1080/10503307.2020.1839141 · Cited by 21
“Even for paralinguistic affect detection, analysis methods do not generalize well from one language to the other (Kaya et al., 2017b).”
Self-citation
ACMF: An Attention Collaborative Extended Matrix Factorization Based Model for MOOC course service via a heterogeneous view
Sheng, D., Yuan, J., Xie, Q., & Li, L. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2021.08.001 · Cited by 21
Subject independent emotion recognition using EEG and physiological signals – a comparative study
Ramaswamy, M. P. A., & Palaniswamy, S. (2022) · Applied Computing and Informatics · doi:10.1108/aci-03-2022-0080 · Cited by 19
RS-Xception: A Lightweight Network for Facial Expression Recognition
Liao, L., Wu, S., Song, C., & Fu, J. (2024) · Electronics · doi:10.3390/electronics13163217 · Cited by 19
Content-based multimedia recommendation systems: Definition and application domains
Deldjoo, Y., Schedl, M., Cremonesi, P., & Pasi, G. (2018) · BOA (University of Milano-Bicocca) · Cited by 19

Authors: Kaya, H., Gürpınar, F., & Salah, A. A.
Affiliations: Tekirdağ Namık Kemal University (TR)
Boğaziçi University (TR)
Corresponding author: Kaya, Heysem
ORCID: Kaya, Heysem
Gürpınar, Furkan
Salah, Albert Ali
Venue: Image and Vision Computing
Publisher: Elsevier BV
Published: 2017-09
Volume: 65
Pages: 66-75
ISSN: 0262-8856 (ISSN-L)
DOI: 10.1016/j.imavis.2017.01.012
OpenAlex: W2585658440
Cited by: 235
References: 73
Citations by year: 2026: 6 · 2025: 12 · 2024: 13 · 2023: 19 · 2022: 31 · 2021: 37 · 2020: 42 · 2019: 41 · 2018: 26 · 2017: 8
FWCI: 41.68
Type: journal article
Language: en
Open access: closed
Licence: www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Locations: Image and Vision Computing (publishedVersion)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
Keywords: video-based emotion recognition, in-the-wild emotion recognition, multimodal emotion recognition, facial expression recognition, transfer learning, score-level fusion, audiovisual integration, convolutional neural network features, fine-tuning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Video-based emotion recognition in the wild using deep transfer learning and score fusion

Overview

Integrity signals

Verification

Claims the paper makes

Citations

Cites

Cited by

Related

Metadata

Record

About these checks