Cited by 211Face recognition and analysisEmotion and Mood RecognitionFace and Expression Recognition

The DOI matches its Crossref record; 68 of the 76 references it cites were found in a registry or an index.

Multimodal 2D+3D Facial Expression Recognition With Deep Fusion Convolutional Neural Network

Li, Huibin, Sun, Jian, Xu, Zongben, Chen, Liming·2017·IEEE Transactions on Multimedia

DOI 10.1109/tmm.2017.2713408

Open this paper in CoChat — free account

Abstract

This paper presents a novel and efficient deep fusion convolutional neural network (DF-CNN) for multimodal 2D+3D facial expression recognition (FER). DF-CNN comprises a feature extraction subnet, a feature fusion subnet, and a softmax layer. In particular, each textured three-dimensional (3D) face scan is represented as six types of 2D facial attribute maps (i.e., geometry map, three normal maps, curvature map, and texture map), all of which are jointly fed into DF-CNN for feature learning and fusion learning, resulting in a highly concentrated facial representation (32-dimensional). Expression prediction is performed by two ways: 1) learning linear support vector machine classifiers using the 32-dimensional fused deep features, or 2) directly performing softmax prediction using the six-dimensional expression probability vectors. Different from existing 3D FER methods, DF-CNN combines feature learning and fusion learning into a single end-to-end training framework. To demonstrate the effectiveness of DF-CNN, we conducted comprehensive experiments to compare the performance of DFCNN with handcrafted features, pre-trained deep features, finetuned deep features, and state-of-the-art methods on three 3D face datasets (i.e., BU-3DFE Subset I, BU-3DFE Subset II, and Bosphorus Subset). In all cases, DF-CNN consistently achieved the best results. To the best of our knowledge, this is the first work of introducing deep CNN to 3D FER and deep learning-based featurelevel fusion for multimodal 2D+3D FER.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1109/tmm.2017.2713408

Cites

68 of 68 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Untitled work
(2008) · Journal of Machine Learning Research · Cited by 33,051
CNN Features Off-the-Shelf: An Astounding Baseline for Recognition
Razavian, A. S., Azizpour, H., Sullivan, J., & Carlsson, S. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops · doi:10.1109/cvprw.2014.131 · Cited by 4,451
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019
Return of the Devil in the Details: Delving Deep into Convolutional Nets
Chatfield, K., Simonyan, K., Vedaldi, A., & Zisserman, A. (2014) · Proceedings of the British Machine Vision Conference 2014 · doi:10.5244/c.28.6 · Cited by 3,335
Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions
Zhao, G., & Pietikainen, M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1110 · Cited by 2,846
Vlfeat
Vedaldi, A., & Fulkerson, B. (2010) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1873951.1874249 · Cited by 2,691
Automatic analysis of facial expressions: the state of the art
Pantie, M., & Rothkrantz, L. J. M. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895976 · Cited by 1,778 · cited 4×
“, [5], [6], [8], [18], [37], [49], [55], [56], [58], [61], [62]).”
BackgroundMethodInfluential
A 3D Facial Expression Database For Facial Behavior Research
Yin, L., Wei, X., Sun, Y., Wang, J., & Rosato, M. J. (2006) · 7th International Conference on Automatic Face and Gesture Recognition (FGR06) · doi:10.1109/fgr.2006.6 · Cited by 1,171
Bosphorus Database for 3D Face Analysis
Savran, A., Alyüz, N., Dibeklioğlu, H., Çeliktutan, O., Gökberk, B., Sankur, B., & Akarun, L. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-89991-4_6 · Cited by 812
Facial Expression Recognition via a Boosted Deep Belief Network
Liu, P., Han, S., Meng, Z., & Tong, Y. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.233 · Cited by 667

Show all 76 references

Research works printed but not linked: 1

li, 0, 3D facial expression recognition via multiple kernel learning of multi-scale local normal patterns, Proc 21st Int Conf Pattern Recog, 2577

Sorted by citations

Deep Facial Expression Recognition: A Survey
Li, S., & Deng, W. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981446 · Cited by 2,175
A systematic review on affective computing: emotion models, databases, and recent advances
Wang, Y., Song, W., Tao, W., Liotta, A., Yang, D., Li, X., Gao, S., Sun, Y., Ge, W., Zhang, W., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2022.03.009 · Cited by 713
A Survey on Human Face Expression Recognition Techniques
Revina, I. M., & Emmanuel, W. R. S. (2018) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2018.09.002 · Cited by 327
Spatiotemporal Recurrent Convolutional Networks for Recognizing Spontaneous Micro-Expressions
Xia, Z., Hong, X., Gao, X., Feng, X., & Zhao, G. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2931351 · Cited by 296
Deep learning for human affect recognition: Insights and new developments
Rouast, P. V., Adam, M. T. P., & Chiong, R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2890471 · Cited by 243
A snapshot research and implementation of multimodal information fusion for data-driven emotion recognition
Jiang, Y., Li, W., Hossain, M. S., Chen, M., Alelaiwi, A. A., & Al-Hammadi, M. H. (2019) · Information Fusion · doi:10.1016/j.inffus.2019.06.019 · Cited by 224
Shallow 3D CNN for Detecting Acute Brain Hemorrhage From Medical Imaging Sensors
SINGH, S. P., Wang, L., Gupta, S., Gulyás, B. Z., & Padmanabhan, P. (2020) · IEEE Sensors Journal · doi:10.1109/jsen.2020.3023471 · Cited by 133
Automatic detection of students’ affective states in classroom environment using hybrid convolutional neural networks
Ashwin, T. S., & Guddeti, R. M. R. (2019) · Education and Information Technologies · doi:10.1007/s10639-019-10004-6 · Cited by 139
Fast Facial emotion recognition Using Convolutional Neural Networks and Gabor Filters
Zadeh, M. M. T., Imani, M., & Majidi, B. (2019) · International Conference on Knowledge-Based Engineering and Innovation · doi:10.1109/kbei.2019.8734943 · Cited by 97
Survey and Synthesis of State of the Art in Driver Monitoring
Halin, A., Verly, J., & Droogenbroeck, M. V. (2021) · Sensors · doi:10.3390/s21165558 · Cited by 97
3D Approaches and Challenges in Facial Expression Recognition Algorithms—A Literature Review
Nonis, F., Dagnes, N., Marcolin, F., & Vezzetti, E. (2019) · Applied Sciences · doi:10.3390/app9183904 · Cited by 81
Enhancement of Patient Facial Recognition through Deep Learning Algorithm: ConvNet
Onyema, E. M., Shukla, P. K., Dalal, S., Mathur, M. N., Zakariah, M., & Tiwari, B. (2021) · Journal of Healthcare Engineering · doi:10.1155/2021/5196000 · Cited by 81
A survey of multidisciplinary domains contributing to affective computing
Arya, R., Singh, J., & Kumar, A. P. (2021) · Computer Science Review · doi:10.1016/j.cosrev.2021.100399 · Cited by 84
A fast and robust 3D face recognition approach based on deeply learned face representation
Cai, Y., Lei, Y., Yang, M., You, Z., & Shan, S. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.07.047 · Cited by 64
A comprehensive survey on 3D face recognition methods
Li, M., Bin, H., & Tian, G. (2022) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2022.104669 · Cited by 62
Deep Auto-Encoders With Sequential Learning for Multimodal Dimensional Emotion Recognition
Nguyen, D., Nguyen, T. T., Zeng, R., Nguyen, T. T., Tran, S. N., Nguyen, T., Sridharan, S., & Fookes, C. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3063612 · Cited by 67
Facial Expression Recognition Using Facial Landmarks and Random Forest Classifier
Munasinghe, N. (2018) · IEEE/ACIS International Conference on Computer and Information Science (ICIS) · doi:10.1109/icis.2018.8466510 · Cited by 59
Facial emotion recognition using temporal relational network: an application to E-learning
Pise, A. A., Vadapalli, H., & Sanders, I. D. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-10133-y · Cited by 58
Survey on Face Expression Recognition using CNN
Vyas, A. S., Prajapati, H. B., & Dabhi, V. K. (2019) · International Conference on Advanced Computing and Communication Systems (ICACCS) · doi:10.1109/icaccs.2019.8728330 · Cited by 61
Dynamic Emotion Modeling With Learnable Graphs and Graph Inception Network
Shirian, A., Tripathi, S., & Guha, T. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3059169 · Cited by 58
Multimodal Emotion Recognition Based on Cascaded Multichannel and Hierarchical Fusion
Liu, X., Xu, Z., & Huang, K. (2023) · Computational Intelligence and Neuroscience · doi:10.1155/2023/9645611 · Cited by 55
Self-citation
Predicting the Survival of Cancer Patients With Multimodal Graph Neural Network
Gao, J., Lyu, T., Xiong, F., Wang, J., Ke, W., & Li, Z. (2021) · IEEE Transactions on Computational Biology and Bioinformatics · doi:10.1109/tcbb.2021.3083566 · Cited by 55
Research Trends of Human–Computer Interaction Studies in Construction Hazard Recognition: A Bibliometric Review
Wang, J., Rui, C., Liu, M., & Liao, P. (2021) · Sensors · doi:10.3390/s21186172 · Cited by 54
Accurate Facial Parts Localization and Deep Learning for 3D Facial Expression Recognition
Jan, A., Ding, H., Meng, H., Chen, L., & Li, H. (2018) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2018.00075 · Cited by 53
Self-citation
Fine-grained facial expression analysis using dimensional emotion model
Zhou, F., Kong, S., Fowlkes, C. C., Chen, T., & Lei, B. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.01.067 · Cited by 55
Dynamic Difficulty Awareness Training for Continuous Emotion Prediction
Zhang, Z., Han, J., Coutinho, E., & Schuller, B. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2871949 · Cited by 53
CWI: A multimodal deep learning approach for named entity recognition from social media using character, word and image features
Asgari-Chenaghlu, M., Feizi‐Derakhshi, M., Farzinvash, L., Balafar, M. A., & Motamed, C. (2021) · Neural Computing and Applications · doi:10.1007/s00521-021-06488-4 · Cited by 54
Systematic review of 3D facial expression recognition methods
Alexandre, G. R., Soares, J. M., & Thé, G. A. P. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.107108 · Cited by 48
Unsupervised Domain Adaptation with Regularized Optimal Transport for Multimodal 2D+3D Facial Expression Recognition
Wei, X., Li, H., Sun, J. Z., & Chen, L. (2018) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2018.00015 · Cited by 44
Self-citation
BReG-NeXt: Facial Affect Computing Using Adaptive Residual Networks With Bounded Gradient
Hasani, B., Negi, P. S., & Mahoor, M. H. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2986440 · Cited by 46
Facial Expression Recognition Through Cross-Modality Attention Fusion
Ni, R., Yang, B., Zhou, X., Cangelosi, A., & Liu, X. (2022) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2022.3150019 · Cited by 48
An Improved Entropy-Weighted Topsis Method for Decision-Level Fusion Evaluation System of Multi-Source Data
Liu, L., Wan, X., Li, J., Wang, W., & Gao, Z. (2022) · Sensors · doi:10.3390/s22176391 · Cited by 49
A novel enhanced convolution neural network with extreme learning machine: facial emotional recognition in psychology practices
Banskota, N., Alsadoon, A., Prasad, P. W. C., Dawoud, A., Rashid, T. A., & Alsadoon, O. H. (2022) · Multimedia Tools and Applications · doi:10.1007/s11042-022-13567-8 · Cited by 41
Emotion Recognition Using Fused Physiological Signals
Fabiano, D., & Canavan, S. J. (2019) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2019.8925486 · Cited by 39
Discriminative Attention-based Convolutional Neural Network for 3D Facial Expression Recognition
Zhu, K., Du, Z., Li, W., Huang, D., Wang, Y., & Chen, L. (2019) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2019.8756524 · Cited by 41
Self-citation
A novel approach inspired by optic nerve characteristics for few-shot occluded face recognition
Zheng, W., Gou, C., & Wang, F. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.09.045 · Cited by 39
Facial expression recognition using three-stage support vector machines
Dagher, I. J., Elio, D., & Shakik, M. A. (2019) · Visual Computing for Industry Biomedicine and Art · doi:10.1186/s42492-019-0034-5 · Cited by 38
Orthogonalization-Guided Feature Fusion Network for Multimodal 2D+3D Facial Expression Recognition
Lin, S., Bai, M., Liu, F., Shen, L., & Zhou, Y. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3001497 · Cited by 38
Image Decolorization Combining Local Features and Exposure Features
Liu, S., & Zhang, X. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2903413 · Cited by 34
Graph-based multimodal fusion with metric learning for multimodal classification
Angelou, M., Solachidis, V., Vretos, N., & Daras, P. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2019.06.013 · Cited by 34
On the frontiers of pose invariant face recognition: a review
Ahmed, S. B., Ali, S. F., Ahmad, J., Adnan, M., & Fraz, M. M. (2019) · Artificial Intelligence Review · doi:10.1007/s10462-019-09742-3 · Cited by 36
Cross-dataset emotion recognition from facial expressions through convolutional neural networks
Dias, W., Andaló, F., Padilha, R., Bertocco, G. C., de Almeida, W. R., Costa, P. D. P., & Rocha, A. D. R. (2021) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2021.103395 · Cited by 36
Fast and Light Manifold CNN based 3D Facial Expression Recognition across Pose Variations
Chen, Z., Huang, D., Wang, Y., & Chen, L. (2018) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3240508.3240568 · Cited by 34
Self-citation
Facial Expression Recognition through person-wise regeneration of expressions using Auxiliary Classifier Generative Adversarial Network (AC-GAN) based model
Dharanya, V., Raj, A. N. J., & Gopi, V. P. (2021) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2021.103110 · Cited by 33
FERLrTc: 2D+3D facial expression recognition via low-rank tensor completion
Fu, Y., Ruan, Q., Luo, Z., Jin, Y., An, G., & Wan, J. (2019) · Signal Processing · doi:10.1016/j.sigpro.2019.03.015 · Cited by 31
An Enhanced Facial Expression Recognition Model Using Local Feature Fusion of Gabor Wavelets and Local Directionality Patterns
Bellamkonda, S., & Gopalan, N. P. (2019) · International Journal of Ambient Computing and Intelligence · doi:10.4018/ijaci.2020010103 · Cited by 31
Weighted Mixed-Norm Regularized Regression for Robust Face Identification
Zheng, J., Lou, K., Yang, X. C., Bai, C., & Tang, J. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2899073 · Cited by 30
High-Performance SAR Automatic Target Recognition Under Limited Data Condition Based on a Deep Feature Fusion Network
Yu, Q., Hu, H., Geng, X., Jiang, Y., & An, J. (2019) · IEEE Access · doi:10.1109/access.2019.2952928 · Cited by 30
Disentangled Spectrum Variations Networks for NIR–VIS Face Recognition
Hu, W., & Hu, H. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2938685 · Cited by 29
Facial Expression Recognition Based on Auxiliary Models
Wang, Y., Li, Y., Song, Y. H., & Rong, X. (2019) · Algorithms · doi:10.3390/a12110227 · Cited by 30
Quaternion Capsule Neural Network With Region Attention for Facial Expression Recognition in Color Images
Zhou, Y., Jin, L., Ma, G., & Xu, X. (2021) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2021.3120513 · Cited by 29
Secondary Information Aware Facial Expression Recognition
Tian, Y., Cheng, J., Li, Y., & Wang, S. (2019) · IEEE Signal Processing Letters · doi:10.1109/lsp.2019.2942138 · Cited by 27
Adaptive Multimodal Fusion for Facial Action Units Recognition
Yang, H., Wang, T., & Yin, L. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413538 · Cited by 29
Multimodal Emotion Recognition using Deep Learning Architectures
Hina, I., Shaukat, A., & Akram, M. U. (2022) · International Conference on Digital Futures and Transformative Technologies (ICoDT2) · doi:10.1109/icodt255437.2022.9787437 · Cited by 28
Learned 3D Shape Representations Using Fused Geometrically Augmented Images: Application to Facial Expression and Action Unit Detection
Taha, B., Hayat, M., Berretti, S., Hatzinakos, D., & Werghi, N. (2020) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2020.2984241 · Cited by 27
Human emotion recognition in the significance assessment of property attributes
Renigier‐Biłozor, M., Janowski, A., Walacik, M., & Chmielewska, A. (2021) · Journal of Housing and the Built Environment · doi:10.1007/s10901-021-09833-0 · Cited by 27
Combining Deep Convolutional Neural Networks With Stochastic Ensemble Weight Optimization for Facial Expression Recognition in the Wild
Choi, J. Y., & Lee, B. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3121547 · Cited by 26
Wild facial expression recognition based on incremental active learning
Ahmed, M. U., Woo, K. J., Hyeon, K. Y., Bashar, M. R., & Rhee, P. K. (2018) · Cognitive Systems Research · doi:10.1016/j.cogsys.2018.06.017 · Cited by 24
A novel 2D and 3D multimodal approach for in-the-wild facial expression recognition
Ly, T. S., Do, N.-T., Kim, S.-H., Yang, H.-J., & Lee, G.-S. (2019) · Image and Vision Computing · doi:10.1016/j.imavis.2019.10.003 · Cited by 24
Hybridization of valuation procedures as a medicine supporting the real estate market and sustainable land use development during the covid-19 pandemic and afterwards
Renigier‐Biłozor, M., Źróbek, S., Walacik, M., & Janowski, A. (2020) · Land Use Policy · doi:10.1016/j.landusepol.2020.105070 · Cited by 24
Feature Selection on 2D and 3D Geometric Features to Improve Facial Expression Recognition
Perez-Gomez, V., Ríos-Figueroa, H. V., Rechy-Ramirez, E. J., Mezura‐Montes, E., & Marı́n-Hernández, A. (2020) · Sensors · doi:10.3390/s20174847 · Cited by 23
Monocular 3D Facial Expression Features for Continuous Affect Recognition
Pei, E., Oveneke, M. C., Zhao, Y. H., Jiang, D., & Hichem Sahli, B. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3026894 · Cited by 22
Towards Reading Beyond Faces for Sparsity-aware 3D/4D Affect Recognition
Behzad, M., Vo, N., Li, X., & Zhao, G. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.06.023 · Cited by 21
3-D Facial Expression Recognition via Attention-Based Multichannel Data Fusion Network
Gu, Y., Yan, H., Zhang, X., Liu, Z., & Ren, F. (2021) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2021.3125972 · Cited by 20
3D facial expression modeling based on facial landmarks in single image
Lv, C., Wu, Z., Wang, X., & Zhou, M. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.04.050 · Cited by 20
3D Facial Expression Recognition Using Multi-channel Deep Learning Framework
Ramya, R., Mala, K., & Nidhyananthan, S. S. (2019) · Circuits Systems and Signal Processing · doi:10.1007/s00034-019-01144-8 · Cited by 20
Mesh Convolution: A Novel Feature Extraction Method for 3D Nonrigid Object Classification
Chen, Y., Zhao, J., Shi, C., & Yuan, D. (2020) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2020.3020693 · Cited by 19
Deep Temporal–Spatial Aggregation for Video-Based Facial Expression Recognition
Pan, X., Guo, W., Guo, X., Li, W., Xu, J., & Wu, J. (2019) · Symmetry · doi:10.3390/sym11010052 · Cited by 19
Deep Learning-Based 3D Face Recognition Using Derived Features from Point Cloud
Atik, M. E., & Duran, Z. (2021) · Lecture notes in networks and systems · doi:10.1007/978-3-030-66840-2_60 · Cited by 19
Facial image inpainting using attention-based multi-level generative network
Liu, J., & Jung, C. (2021) · Neurocomputing · doi:10.1016/j.neucom.2020.12.118 · Cited by 18
Weakly Supervised Dual Learning for Facial Action Unit Recognition
Wang, S., & Peng, G. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2916063 · Cited by 18
Intensity enhancement via GAN for multimodal face expression recognition
Yang, H., Zhu, K., Huang, D., Li, H., Wang, Y., & Chen, L. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.05.022 · Cited by 19
Self-citation
Understanding Abstraction in Deep CNN: An Application on Facial Emotion Recognition
Nonis, F., Barbiero, P., Cirrincione, G., Olivetti, E. C., Marcolin, F., & Vezzetti, E. (2020) · Smart innovation, systems and technologies · doi:10.1007/978-981-15-5093-5_26 · Cited by 14
Person-independent facial expression recognition method based on improved Wasserstein generative adversarial networks in combination with identity aware
Xu, C., Cui, Y., Zhang, Y., Gao, P., & Xu, J. (2019) · Multimedia Systems · doi:10.1007/s00530-019-00628-6 · Cited by 16
Expression-Aware Face Reconstruction via a Dual-Stream Network
Chai, X., Chen, J., Liang, C., Xu, D., & Lin, C. (2021) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2021.3068567 · Cited by 16
Disentangling 3D/4D Facial Affect Recognition With Faster Multi-View Transformer
Behzad, M., Li, X., & Zhao, G. (2021) · IEEE Signal Processing Letters · doi:10.1109/lsp.2021.3111576 · Cited by 16
Weighted-fusion feature of MB-LBPUH and HOG for facial expression recognition
Wang, Y., Li, M., Zhang, C., Chen, H., & Lu, Y. (2019) · Soft Computing · doi:10.1007/s00500-019-04380-x · Cited by 15
Can ADAS Distract Driver’s Attention? An RGB-D Camera and Deep Learning-Based Analysis
Ulrich, L., Nonis, F., Vezzetti, E., Moos, S., Caruso, G., Shi, Y., & Marcolin, F. (2021) · Applied Sciences · doi:10.3390/app112411587 · Cited by 15
Deformable Synthesis Model for Emotion Recognition
Fabiano, D., & Canavan, S. J. (2019) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2019.8756614 · Cited by 15
A Coarse-to-Fine Approach for 3D Facial Landmarking by Using Deep Feature Fusion
Wang, K., Zhao, X., Gao, W., & Zou, J. (2018) · Symmetry · doi:10.3390/sym10080308 · Cited by 14
Creating Historical Building Models by Deep Fusion of Multi-Source Heterogeneous Data Using Residual 3D Convolutional Neural Network
Hu, W., & Hu, R. (2023) · International Journal of Architectural Heritage · doi:10.1080/15583058.2023.2229253 · Cited by 13
3D Facial Similarity Measurement and Its Application in Facial Organization
Lv, C., Wu, Z., Wang, X., & Zhou, M. (2020) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3397765 · Cited by 13
High-throughput virtual screening of small molecule inhibitors for SARS-CoV-2 protein targets with deep fusion models
Stevenson, G. A., Jones, D., Kim, H., Bennett, W. F., Bennion, B. J., Borucki, M. K., Bourguet, F. A., Epstein, A., Franco, M., Harmon, B., et al. (2021) · International Conference for High Performance Computing, Networking, Storage and Analysis (SC) · doi:10.1145/3458817.3476193 · Cited by 15
Recognition of 3D emotional facial expression based on handcrafted and deep feature combination
Hariri, W., & Farah, N. (2021) · Pattern Recognition Letters · doi:10.1016/j.patrec.2021.04.030 · Cited by 13
FFNet-M: Feature Fusion Network with Masks for Multimodal Facial Expression Recognition
Sui, M., Zhu, Z., Zhao, F., & Wu, F. (2021) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme51207.2021.9428100 · Cited by 13
Multi-Modal Learning for AU Detection Based on Multi-Head Fused Transformers
Zhang, X., & Yin, L. (2021) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg52635.2021.9667030 · Cited by 13
Hybrid Convolutional Neural Network and Long Short-Term Memory Approach for Facial Expression Recognition
Kavitha, M. N., & RajivKannan, A. (2022) · Intelligent Automation & Soft Computing · doi:10.32604/iasc.2023.025437 · Cited by 13
Gender recognition from facial images using multichannel deep learning framework
Ramya, R. S., Anandh, A., Muthulakshmi, K., & Venkatesh, S. (2022) · Elsevier eBooks · doi:10.1016/b978-0-323-85209-8.00009-2 · Cited by 12
Multi-Task Learning and Multimodal Fusion for Road Segmentation
Cheng, B., Tian, M., Jiang, S., Liu, W., & Pang, Y. (2022) · IEEE Access · doi:10.1109/access.2022.3151372 · Cited by 12
Disagreement Matters: Exploring Internal Diversification for Redundant Attention in Generic Facial Action Analysis
Li, X., Zhang, Z., Zhang, X., Wang, T., Li, Z., Yang, H., Çiftçi, U. A., Ji, Q., Cohn, J. F., & Yin, L. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3286838 · Cited by 12
Multimodal 2D and 3D for In-The-Wild Facial Expression Recognition
Ly, S. T., Do, N.-T., Lee, G.-S., Kim, S.-H., & Yang, H.-J. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00353 · Cited by 12
3D Facial Expression Recognition Using Deep Feature Fusion CNN
Tian, K., Zeng, L., McGrath, S., Yin, Q., & Wang, W. (2019) · Irish Signals and Systems Conference · doi:10.1109/issc.2019.8904930 · Cited by 12
Facial Attention based Convolutional Neural Network for 2D+3D Facial Expression Recognition
Jiao, Y., Niu, Y., Zhang, Y., Li, F., Zou, C., & Shi, G. (2019) · IEEE International Conference on Visual Communications and Image Processing (VCIP) · doi:10.1109/vcip47243.2019.8965843 · Cited by 12
Action Unit Detection by Learning the Deformation Coefficients of a 3D Morphable Model
Ariano, L., Ferrari, C., Berretti, S., & Bimbo, A. D. (2021) · Sensors · doi:10.3390/s21020589 · Cited by 12
Trends of Multimodal Neural Engineering Study: A Bibliometric Review
Wang, J., Rui, C., & Liao, P. (2021) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-021-09557-y · Cited by 11
Emotional Health Detection in HAR: New Approach Using Ensemble SNN
Bibbò, L., Cotroneo, F., & Vellasco, M. M. B. R. (2023) · Applied Sciences · doi:10.3390/app13053259 · Cited by 11
Facial Action Unit Analysis through 3D Point Cloud Neural Networks
Reale, M. J., Klinghoffer, B., Church, M., Szmurlo, H., & Yin, L. (2019) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2019.8756610 · Cited by 11
Multimodal Channel-Mixing: Channel and Spatial Masked AutoEncoder on Facial Action Unit Detection
Zhang, X., Yang, H., Wang, T., Li, X., & Yin, L. (2024) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv57701.2024.00597 · Cited by 10
An Efficient Real time Product Recommendation using Facial Sentiment Analysis
Suguna, R., Devi, M. S., Kushwaha, A., & Gupta, P. (2019) · International Conference on Electrical, Computer and Communication Technologies (ICECCT) · doi:10.1109/icecct.2019.8869300 · Cited by 10
Unsupervised emotion recognition algorithm based on improved deep belief model in combination with probabilistic linear discriminant analysis
Xiao, Y., Wang, D.-Y., & Hou, L.-G. (2019) · Personal and Ubiquitous Computing · doi:10.1007/s00779-019-01235-y · Cited by 10

Authors: Li, H., Sun, J., Xu, Z., & Chen, L.
Affiliations: Xi'an Jiaotong University (CN)
École Centrale de Lyon (FR)
ORCID: Li, Huibin
Sun, Jian
Xu, Zongben
Chen, Liming
Venue: IEEE Transactions on Multimedia
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2017-12
Volume: 19
Issue: 12
Pages: 2816-2831
ISSN: 1520-9210 (ISSN-L)
1941-0077
DOI: 10.1109/tmm.2017.2713408
OpenAlex: W2621864722
Cited by: 211
References: 76
Citations by year: 2026: 5 · 2025: 9 · 2024: 13 · 2023: 16 · 2022: 27 · 2021: 46 · 2020: 35 · 2019: 49 · 2018: 11
FWCI: 10.78
Type: journal article
Language: en
Open access: closed
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/IEEE.html (vor)
Locations: IEEE Transactions on Multimedia (publishedVersion)
Funders: NSFC (11401464, 61472313, 11622106)
Chinese Postdoctoral Science Foundation (2014M560785)
International Exchange Foundation of China NSFC
United Kingdom RS (61711530242)
French Research Agency
l’Agence Nationale de Recherche (ANR-13-CORD-0004-02, ANR-13-INSE-0004-02)
Partner University Foundation
Topics: Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
Keywords: facial expression recognition, 3D facial scans, multimodal fusion, feature fusion, normal maps, curvature maps, end-to-end learning, BU-3DFE

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Multimodal 2D+3D Facial Expression Recognition With Deep Fusion Convolutional Neural Network

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks