MMA-DFER: MultiModal Adaptation of unimodal models for Dynamic Facial Expression Recognition in-the-wild
Open accessAbstract
Dynamic Facial Expression Recognition (DFER) has received significant interest in the recent years dictated by its pivotal role in enabling empathic and human-compatible technologies. Achieving robustness towards in-the-wild data in DFER is particularly important for real-world applications. One of the directions aimed at improving such models is multimodal emotion recognition based on audio and video data. Multimodal learning in DFER increases the model capabilities by leveraging richer, complementary data representations. Within the field of multimodal DFER, recent methods have focused on exploiting advances of self-supervised learning (SSL) for pre-training of strong multi-modal encoders [40]. Another line of research has focused on adapting pre-trained static models for DFER [8]. In this work, we propose a different perspective on the problem and investigate the advancement of multimodal DFER performance by adapting SSL-pre-trained disjoint unimodal encoders. We identify main challenges associated with this task, namely, intra-modality adaptation, cross-modal alignment, and temporal adaptation, and propose solutions to each of them. As a result, we demonstrate improvement over current state-of-the-art on two popular DFER benchmarks, namely DFEW [19] and MFAW [29].
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In MMA-DFER, temporal relationships between frames and integration of features with the image during Transformer processing are inadequately considered.
“As demonstrated in Figure 3, within the MMA-DFER sequence, there is inadequate consideration given to the temporal relationships between frames and the integration of features with the image during the Transformer's processing.”
From Multimodal Dynamic Face Emotion Verification Model based on Priori Prompt Words for Dynamic Facial Expression Recognition (2025) · cites this paper as [19]
Section 3.4 (Temporal fusion) says MMA-DFER addresses between-frame dependencies with a Multimodal Temporal Transformer that processes a joint audio–video sequence. It also says fusion modules implicitly embed temporal information in image features at intermediate stages. Section 4.4 reports that the proposed temporal Transformer performed best among the tested temporal-adaptation approaches, rather than describing temporal processing as inadequate. (LLM, conf=0.85)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe multimodal temporal module in MMA-DFER primarily focuses on classification of features derived from already fused video and audio data, after substantial feature extraction.
“However, the primary focus of this module is the classification of features derived from already fused video and audio data, where a substantial extraction of features has been conducted.”
From Multimodal Dynamic Face Emotion Verification Model based on Priori Prompt Words for Dynamic Facial Expression Recognition (2025) · cites this paper as [19]
Section 3.4 places the Multimodal Temporal Transformer after the video and audio representations are fused, so the claim is right about its position after feature extraction. But the section describes its purpose as modeling between-frame temporal dependencies; its output is then passed to a separate classifier. Calling classification the module’s primary focus overstates what the paper says. (LLM, conf=0.86)
Experimental — accuracy not yet measuredMMA-DFER uses an audiovisual self-supervised model with a Transformer structure, a 3% parameter reduction, and feature-level fine-tuning, achieving strong learning results and sentiment classification using categorical tokens.
“MMA-DFER [19] proposed a framework based on an audiovisual self-supervised model using a Transformer structure with a 3% parameter reduction and fine-tuning at the feature level. This approach achieved strong learning results and resulted in sentiment classification using categorical tokens.”
From Multimodal Dynamic Face Emotion Verification Model based on Priori Prompt Words for Dynamic Facial Expression Recognition (2025) · cites this paper as [19]
Sections 3 and 3.4 describe independently self-supervised-pretrained audio and visual encoders, feature adaptation, a Multimodal Temporal Transformer, and classification using a [CLS] token; Section 4.3 reports strong benchmark results. But the paper does not describe a 3% parameter reduction: Section 4.2 reports 7.5 million tunable parameters, versus 9 million for S2D. It uses a classification token, not “categorical tokens,” and does not start from a jointly pretrained audiovisual model. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667 · see citing works
Learning Spatiotemporal Features with 3D Convolutional Networks
Tran, D., Bourdev, L., Fergus, R., Torresani, L., & Paluri, M. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.510 · Cited by 9,593 · see citing works
Masked Autoencoders Are Scalable Vision Learners
He, K., Chen, X., Xie, S., Li, Y., Dollar, P., & Girshick, R. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01553 · Cited by 8,489 · cited 1× · see citing works
“Specifically, we choose among the models pre-trained with Masked Autoendoder (MAE) reconstruction objective, dictated by outstanding performance of MAEs on tasks requiring fine-grained features [14].”
Joint Face Detection and Alignment Using Multitask Cascaded Convolutional Networks
Zhang, K., Zhang, Z., Li, Z., & Qiao, Y. (2016) · IEEE Signal Processing Letters · doi:10.1109/lsp.2016.2603342 · Cited by 6,214 · see citing works
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2023) · ACM Computing Surveys · doi:10.1145/3560815 · Cited by 4,361 · see citing works
Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., & Neubig, G. (2021) · arXiv · doi:10.48550/arxiv.2107.13586 · Cited by 4,352 · see citing works
A Closer Look at Spatiotemporal Convolutions for Action Recognition
Tran, D., Wang, H., Torresani, L., Ray, J., LeCun, Y., & Paluri, M. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00675 · Cited by 3,672 · see citing works
Audio Set: An ontology and human-labeled dataset for audio events
Gemmeke, J. F., Ellis, D. P. W., Freedman, D., Jansen, A., Lawrence, W., Moore, R. C., Plakal, M., & Ritter, M. (2017) · 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2017.7952261 · Cited by 3,134 · see citing works
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
Hsu, W.-N., Bolte, B., Tsai, Y.-H. H., Lakhotia, K., Salakhutdinov, R., & Mohamed, A. (2021) · IEEE/ACM Transactions on Audio, Speech, and Language Processing · doi:10.1109/taslp.2021.3122291 · Cited by 2,983 · see citing works
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Li, X. L., & Liang, P. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.353 · Cited by 2,473 · see citing works
AffectNet: A Database for Facial Expression, Valence, and Arousal Computing in the Wild
Mollahosseini, A., Hasani, B., & Mahoor, M. H. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2740923 · Cited by 2,338 · see citing works
Multimodal Transformer for Unaligned Multimodal Language Sequences
Tsai, Y.-H. H., Bai, S., Liang, P. P., Kolter, J. Z., Morency, L.-P., & Salakhutdinov, R. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1656 · Cited by 2,259 · cited 1× · see citing works
“Specifically, we compare with: • Addition in original space, i.e., without compression and expansion (ADD); • Multimodal Transformer following [45], where we introduce 2 multi-head self-attention blocks, audio-to-vision, and vision-to-audio, outputs of which are added to corresponding branches…”
Jia, M., Tang, L., Chen, B.-C., Cardie, C., Belongie, S., Hariharan, B., & Lim, S.-N. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19827-4_41 · Cited by 1,965 · see citing works
WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing
Chen, S., Wang, C., Chen, Z., Wu, Y., Liu, S., Chen, Z., Li, J., Kanda, N., Yoshioka, T., Xiao, X., et al. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3188113 · Cited by 1,926 · see citing works
Flamingo: a Visual Language Model for Few-Shot Learning
Alayrac, J.-B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.14198 · Cited by 1,555 · see citing works
MaPLe: Multi-modal Prompt Learning
Khattak, M. U., Rasheed, H., Maaz, M., Khan, S., & Khan, F. S. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01832 · Cited by 1,011 · see citing works
Deep learning-based facial emotion recognition for human–computer interaction applications
Chowdary, M. K., Nguyen, T. N., & Hemanth, D. J. (2021) · Neural Computing and Applications · doi:10.1007/s00521-021-06012-8 · Cited by 417 · see citing works
Driver Emotion Recognition for Intelligent Vehicles
Zepf, S., Hernandez, J., Schmitt, A., Minker, W., & Picard, R. W. (2020) · ACM Computing Surveys · doi:10.1145/3388790 · Cited by 244 · see citing works
Jiang, X., Zong, Y., Zheng, W., Tang, C., Xia, W., Lu, C., & Liu, J. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413620 · Cited by 209 · see citing works
Masked Autoencoders that Listen
Huang, P.-Y., Xu, H., Li, J., Baevski, A., Auli, M., Galuba, W., Metze, F., & Feichtenhofer, C. (2022) · neural information processing systems · doi:10.52202/068431-2081 · Cited by 206 · see citing works
Masked Autoencoders that Listen
Huang, P.-Y., Xu, H., Li, J., Baevski, A., Auli, M., Galuba, W., Metze, F., & Feichtenhofer, C. (2022) · arXiv · doi:10.48550/arxiv.2207.06405 · Cited by 206 · see citing works
Former-DFER: Dynamic Facial Expression Recognition Transformer
Zhao, Z., & Liu, Q. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475292 · Cited by 201 · see citing works
A facial expression emotion recognition based human-robot interaction system
Liu, Z., Wu, M. H., Cao, W., Chen, L., Xu, J., Zhang, R., Zhou, M., & Mao, J.-W. (2017) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2017.7510622 · Cited by 185 · see citing works
Emotion Recognition from Multimodal Physiological Signals for Emotion Aware Healthcare Systems
Ayata, D., Yaslan, Y., & Kamasak, M. E. (2020) · Journal of Medical and Biological Engineering · doi:10.1007/s40846-019-00505-7 · Cited by 182 · see citing works
A systematic survey on multimodal emotion recognition using learning algorithms
Ahmed, N., Aghbari, Z. A., & Girija, S. (2023) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2022.200171 · Cited by 178 · see citing works
ReZero is All You Need: Fast Convergence at Large DepthPreprint
Bachlechner, T. C., Majumder, B. P., Mao, H. H., Cottrell, G. W., & McAuley, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.04887 · Cited by 167 · see citing works
Impact of Deep Learning Approaches on Facial Expression Recognition in Healthcare Industries
Bisogni, C., Castiglione, A., Hossain, S., Narducci, F., & Umer, S. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3141400 · Cited by 157 · see citing works
Pan, B., Hirota, K., Jia, Z., & Dai, Y. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126866 · Cited by 142 · cited 1× · see citing works
“In the field of multimodal facial expression recognition, prior works have to a large extent focused on joint cross-modal feature learning through complex fusion methods and/or multimodal pre-training [1, 10, 35, 37, 40, 41].”
Mocanu, B., Țapu, R., & Zaharia, T. B. (2023) · Image and Vision Computing · doi:10.1016/j.imavis.2023.104676 · Cited by 142 · see citing works
Expression snippet transformer for robust video-based facial expression recognition
Liu, Y., Wang, W., Feng, C., Zhang, H., Chen, Z., & Zhan, Y. (2023) · Pattern Recognition · doi:10.1016/j.patcog.2023.109368 · Cited by 119 · see citing works
Rethinking the Learning Paradigm for Dynamic Facial Expression Recognition
Wang, H., Li, B., Wu, S., Shen, S., Liu, F., Ding, S., & Zhou, A. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01722 · Cited by 115 · see citing works
Kollias, D., Tzirakis, P., Baird, A., Cowen, A., & Zafeiriou, S. P. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00626 · Cited by 102 · see citing works
Liu, Y., Dai, W., Feng, C., Wang, W., Yin, G., Zeng, J., & Shan, S. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548190 · Cited by 93 · see citing works
MAE-DFER: Efficient Masked Autoencoder for Self-supervised Dynamic Facial Expression Recognition
Sun, L., Lian, Z., Liu, B., & Tao, J. (2023) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3581783.3612365 · Cited by 89 · see citing works
Chen, Y., Li, J., Shan, S., Wang, M., & Hong, R. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3453443 · Cited by 89 · see citing works
Clip-aware expressive feature learning for video-based facial expression recognition
Liu, Y., Feng, C., Yuan, X., Zhou, L., Wang, W., Qin, J., & Luo, Z. (2022) · Information Sciences · doi:10.1016/j.ins.2022.03.062 · Cited by 83 · see citing works
Intensity-Aware Loss for Dynamic Facial Expression Recognition in the Wild
Li, H., Niu, H., Zhu, Z., & Zhao, F. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i1.25077 · Cited by 83 · see citing works
Emotion-Driven Analysis and Control of Human-Robot Interactions in Collaborative Applications
Eyam, A. T., Mohammed, W. M., & Lastra, J. L. M. (2021) · Sensors · doi:10.3390/s21144626 · Cited by 81 · see citing works
Zhang, X., Li, M., Lin, S., Xu, H., & Xiao, G. (2023) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2023.3312858 · Cited by 79 · see citing works
Sun, L., Lian, Z., Liu, B., & Tao, J. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102382 · Cited by 78 · see citing works
Self-attention fusion for audiovisual emotion recognition with incomplete data
Chumachenko, K., Iosifidis, A., & Gabbouj, M. (2022) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr56361.2022.9956592 · Cited by 77 · see citing works
Self-citationTellamekala, M. K., Amiriparian, S., Schuller, B. W., André, E., Giesbrecht, T., & Valstar, M. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3325770 · Cited by 75 · see citing works
Attentive Modality Hopping Mechanism for Speech Emotion Recognition
Yoon, S., Dey, S., Lee, H., & Jung, K. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054229 · Cited by 46 · see citing works
SVFAP: Self-Supervised Video Facial Affect Perceiver
Sun, L., Lian, Z., Wang, K., He, Y., Xu, M., Sun, H., Liu, B., & Tao, J. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3436913 · Cited by 31 · see citing works
A Unified Approach to Facial Affect Analysis: the MAE-Face Visual Representation
Ma, B., Zhang, W., Qiu, F., & Ding, Y. (2023) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00630 · Cited by 15 · see citing works
Flagged first, then by citations
Research works printed but not linked: 12
- Alexei Baevski, Yuhao Zhou, Abdelrahman Mohamed, and Michael Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations. Advances in neural infor- mation processing systems, 33:12449-12460, 2020. 6
- Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020. 3, 7 · arXiv
- Jie Hu, Li Shen, and Gang Sun. Squeeze-and-excitation net- works. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 7132-7141, 2018. 4
- Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Vi- sual prompt tuning. In European Conference on Computer Vision, pages 709-727. Springer, 2022. 3
- Xingxun Jiang, Yuan Zong, Wenming Zheng, Chuangao Tang, Wanchuang Xia, Cheng Lu, and Jiateng Liu. Dfew: A large-scale database for recognizing dynamic facial expres- sions in the wild. In Proceedings of the 28th ACM Interna- tional Conference on Multimedia, pages 2881-2889, 2020. 1, 2, 5
- Diederik P Kingma and Max Welling. Auto-encoding varia- tional bayes. arXiv preprint arXiv:1312.6114, 2013. 4 · arXiv
- Hanting Li, Mingzhe Sui, Zhaoqing Zhu, et al. Nr-dfernet: Noise-robust network for dynamic facial expression recogni- tion. arXiv preprint arXiv:2206.04975, 2022. 6 · arXiv
- Hanting Li, Hongjing Niu, Zhaoqing Zhu, and Feng Zhao. Cliper: A unified vision-language framework for in-the-wild facial expression recognition. arXiv preprint arXiv:2303.00193, 2023. 6 · arXiv
- Xiao Liu, Kaixuan Ji, Yicheng Fu, Weng Lam Tam, Zhengx- iao Du, Zhilin Yang, and Jie Tang. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. arXiv preprint arXiv:2110.07602, 2021. 3 · arXiv
- Fuyan Ma, Bin Sun, and Shutao Li. Spatio-temporal trans- former for dynamic facial expression recognition in the wild. arXiv preprint arXiv:2205.04749, 2022. 6 · arXiv
- Zengqun Zhao and Ioannis Patras. Prompting visual- language models for dynamic facial expression recognition. arXiv preprint arXiv:2308.13382, 2023. 6 · arXiv
- Zengqun Zhao and Ioannis Patras. Prompting visual- language models for dynamic facial expression recognition. arXiv preprint arXiv:2308.13382, 2023. 2, 3 · arXiv
Cited by
Sorted by citations
Multimodal Interaction, Interfaces, and Communication: A Survey
Dritsas, E., Τρίγκα, Μ., Troussas, C., & Phivos Mylonas, B. (2025) · Multimodal Technologies and Interaction · doi:10.3390/mti9010006 · Cited by 108 · see citing works
Facial Expression Analysis and Its Potentials in IoT Systems: A Contemporary Survey
Shangguan, Z., Dong, Y., Guo, S., Leung, V. C. M., Deen, M. J., & Hu, X. (2025) · ACM Computing Surveys · doi:10.1145/3737456 · Cited by 20 · see citing works
Wu, X., Sun, H., Wang, Y., Nie, J., Zhang, J., Wang, Y., Xue, J., & He, L. (2025) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52734.2025.00854 · Cited by 9 · see citing works
MMFformer: Multimodal Fusion Transformer Network for Depression Detection
Haque, M. R., Islam, M. M., Raju, S. M. T. U., Altaheri, H., Nassar, L., & Karray, F. (2025) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc58881.2025.11342694 · Cited by 7 · see citing works
DAT: Dialogue-Aware Transformer with Modality-Group Fusion for Human Engagement Estimation
Li, J., Yu, Y., Chen, Y., Zhang, Y., Jia, P., Xu, Y., Li, Z., Wang, M., & Hong, R. (2024) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3664647.3688988 · Cited by 7 · see citing works
Leveraging Eye Movement for Instructing Robust Video-Based Facial Expression Recognition
Liu, Y., Wei, L., Liu, K., Chen, Z., Chen, Z., Tang, C., Chen, J., & Shan, S. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3599859 · Cited by 4 · see citing works
Pallapothula, V. K. N., Anand, S., Bhattacharjee, S. D., & Yuan, J. (2025) · Smart Health · doi:10.1016/j.smhl.2025.100571 · Cited by 4 · see citing works
Han, T., Dou, S., Zhang, W., & Liu, R. (2025) · Digital Signal Processing · doi:10.1016/j.dsp.2025.105470 · Cited by 4 · see citing works
Multimodal road perception with illumination adaptation in autonomous vehicles
Wei, J., Mo, Y., Su, C., & Li, X. (2026) · Scientific Reports · doi:10.1038/s41598-025-31173-0 · Cited by 3 · see citing works
3D Micro-Expression Recognition Based on Adaptive Dynamic Vision
Kong, W., You, Z., & Lv, X. (2025) · Sensors · doi:10.3390/s25103175 · Cited by 3 · see citing works
Cross-modal Emotion-specific Attention model for Multimodal Emotion Recognition
Chen, J., John, V., & Kawanishi, Y. (2025) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg61629.2025.11099454 · Cited by 2 · see citing works
Yang, C., Yang, B., Li, J., Wang, X., Li, J., Zhang, Y., Chen, Y., Cangelosi, A., & Xiang, W. (2025) · Information Fusion · doi:10.1016/j.inffus.2025.103959 · Cited by 2 · see citing works
Scalable Audiovisual Masked Autoencoders for Efficient Affective Video Facial Analysis
Wu, X., Xue, J., Yin, X., Shi, Y., Fu, L., Huang, D., Wang, Y., Zhang, J., Nie, J., & Wang, J. (2025) · Intelligent Computing · doi:10.34133/icomputing.0246 · Cited by 2 · see citing works
Integrating gating and learned queries in audiovisual emotion recognition
Zhang, Z., Guo, Q., Lu, S., Su, J., & Tang, T. (2024) · Multimedia Systems · doi:10.1007/s00530-024-01551-1 · Cited by 2 · see citing works
A Survey on Graph Deep Representation Learning for Facial Expression Recognition
Gueuret, T., Sellami, A., & Djéraba, C. (2024) · International Workshop on Content-Based Multimedia Indexing (CBMI) · doi:10.1109/cbmi62980.2024.10858877 · Cited by 2 · see citing works
Soften the Mask: Adaptive Temporal Soft Mask for Efficient Dynamic Facial Expression Recognition
Li, M., Zha, Q., & Wu, H. (2025) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme59968.2025.11209787 · Cited by 1 · see citing works
Liu, K., Liu, Y., Wei, L., Tang, C., Zhan, Y., Chen, Z., & Chen, Z. (2026) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2026.3673532 · Cited by 2 · see citing works
FRU-Adapter: Frame Recalibration Unit Adapter for Dynamic Facial Expression Recognition
Her, M., Nabi, H. G., & Han, J.-H. (2025) · Electronics · doi:10.3390/electronics14050978 · Cited by 1 · see citing works
Xie, H. (2025) · Proceedings of the 2025 International Conference on Artificial Intelligence and Educational Systems · doi:10.1145/3744367.3744415
“The methodology employed for the backbone comprised the utilisation of identical processing and training methods as those implemented in MMA-DFER [19] for video and audio, as illustrated in Figure 1.”
PMTSeg: Prompt-Driven Multimodal Transformer for Task-Adapted Remote Sensing Image Segmentation
Liu, K., Yan, X., Liu, Y., Tang, C., Zhan, Y., Luo, W., Zhou, W., & Zhang, H. (2025) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2025.3586620
Emotion-Aware Text Generation for Instructing Audio-Visual Emotion Recognition
Wei, L., Liu, Y., Liu, K., Huang, Y., & Feng, S. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11227919
Kong, X., & Yu, W. (2025) · International Conference on Intelligent Informatics and Biomedical Sciences (ICIIBMS) · doi:10.1109/iciibms66230.2025.11316745
MOFA: Modality-Orthogonalized Fusion Architecture for Multimodal Emotion Recognition
Chen, H., Feng, R., Li, J., Wang, W., Li, J., & Xiang, W. (2026) · Lecture notes in computer science · doi:10.1007/978-981-95-5567-3_29
Ma, S., Liu, L., Cheng, M., Qin, P., Han, Z., Chen, C., Yang, S., & Wang, H. (2026) · Electronics · doi:10.3390/electronics15061230
D2M: Decoupling to Modulate via Emotion Trajectories for Dynamic Facial Expression Recognition
Chen, P., Zhou, Z., Du, J., Sun, Y., Zhang, R., Zhao, Y., & Mao, K. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11462009
Bimodal Fusion Framework for Dynamic Facial Expression Recognition In-The-Wild
Yang, B., Lilin, L., Wang, X. W., Yang, C., & Xiang, W. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11465074
Zhang, Y., Han, Y., Ding, J., & Chen, L. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792417
Shen, W., Mou, X., Yi, J., & Le, S. (2026) · Applied Sciences · doi:10.3390/app16094312
Context Effect Guided Multi-Source Awareness Framework for Dynamic Facial Expression Recognition
Bao, J., Qian, J., Yan, M., & Yang, J. (2026) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2026.3688789
Cheng, K., Lei, X., Wang, R., Li, H., & Chen, C. S. (2026) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg67764.2026.11556928
Nabi, H. G., Alowonou, K. C., & Han, J.-H. (2026) · Electronics · doi:10.3390/electronics15163555
Ning, K., Gao, S., Xia, Z., Huang, D. J., & Li, L. (2026) · Sensors · doi:10.3390/s26175384
Human–Object Interaction as Affective Evidence for Multimodal Emotion Recognition
Sun, W., Cheng, X., Attfield, R., Ge, Z., & Carreno‐Medrano, P. (2026) · IEEE RO-MAN · doi:10.1109/ro-man72286.2026.11714954
Luo, M., Chongyang, D., Li, W., Yang, Y., 谢洪泼, Lin, H., Liao, C., Li, L., Zhao, H., Li, C., et al. (2026) · BMJ Open · doi:10.1136/bmjopen-2026-126107
- Authors
- Chumachenko, K., Iosifidis, A., & Gabbouj, M.
- Affiliations
- Tampere University (FI)
- Aarhus University (DK)
- Venue
- 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
- Publisher
- IEEE
- Published
- 2024-06-17
- Pages
- 4673-4682
- OpenAlex
- W4402916290
- Cited by
- 34
- References
- 46
- Citations by year
- 2026: 14 · 2025: 17 · 2024: 3
- FWCI
- 21.25
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Tampere University Institutional Repository (Tampere University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Business Finland
- Horizon Europe
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- dynamic facial expression recognition, multimodal emotion recognition, in-the-wild data, self-supervised pre-training, cross-modal alignment, temporal adaptation, audio-video fusion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
