CoChat

Multimodal Dynamic Face Emotion Verification Model based on Priori Prompt Words for Dynamic Facial Expression Recognition

Open access
Xie, Huiling · 2025 · Proceedings of the 2025 International Conference on Artificial Intelligence and Educational Systems

The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index; 11 of the 11 sources its claims cite were checked for support.

Abstract

This paper proposes MPPW-DFER, a multimodal dynamic facial emotion verification model that adds language-like prior prompt words to dynamic facial videos and introduces Temporal Deformable Attention Embedding. It reports outperforming existing methods on DFER and MAFW.

Overview

CoChat Summary

TL;DR

The paper proposes MPPW-DFER, a multimodal dynamic facial expression recognition model that uses a large multimodal model to generate Priori Prompt Word descriptions from video and adds Temporal Deformable Attention Embedding for early temporal modeling. It reports best UAR/WAR on DFER and improved WAR on MAFW over compared methods.

Key findings

  • On DFER, MPPW-DFER achieved UAR 68.23 and WAR 77.94, outperforming HiCMAE (63.76 UAR, 75.0 WAR) and MMA-DFER (67.01 UAR, 77.51 WAR).
  • On MAFW, MPPW-DFER achieved UAR 43.87 and WAR 60.04; UAR was only 0.8% lower than MMA-DFER, while WAR improved by 2.6% over MMA-DFER’s 58.45.
  • Ablation showed that adding PPW alone to a visual-only backbone reached UAR 56.95 and WAR 60.32 on DFER, and UAR 55.68 and WAR 60.57 on MAFW; adding TDAE produced the full model results.
  • The audio-only baseline performed sub-optimally on both datasets, which the paper says underscores the importance of visual cues for emotion recognition.

Methods

  • The backbone follows MMA-DFER for video and audio, using Face MAE and Video MAE for visual/audio backbones, an AudioMAE pre-trained on AudioSet, six transformers, MLP dimension adjustment, and feature-fusion bottlenecks between Transformer layers.
  • Priori Prompt Words are generated with Qwen2-VL-7B using prompts focused on facial expressions, body language, and gestures, limited to concise descriptions of no more than 15 words and excluding background information; CLIP Text Encoder and CLIP Image Encoder are used with only their first two Transformer layers, frozen for feature extraction and fusion.
  • TDAE is introduced in the embedding layer: video is divided into 16 segments, keyframes are randomly sampled from each segment, and attention is shifted to the spatial dimension with global pooling over the channel dimension.
  • Experiments use DFER/DFEW (16,000 video segments, 7 emotion categories) and MAFW (10,045 segments, 11 emotion categories), with 5-fold cross-validation, UAR and WAR metrics, 16 images per video at 224x224, learning rate 1e-4, batch size 8, AdamW, 50 rounds, and two Nvidia 4090 24G GPUs for about 20 hours.

Limitations

  • The paper states that in domains such as education, merely recognizing emotions is insufficient; models must also provide interpretable explanations and richer contextual descriptions, which is left for future work.
  • The paper reports that MAFW UAR was slightly lower than MMA-DFER by 0.8%, even though WAR improved.
  • The text makes evident that PPW generation was initially time-consuming: generating a five-second video glossary took about eight to ten seconds on an Nvidia H800 40G GPU, so it was moved to a preliminary processing step.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1145/3744367.3744415

Cites

31 of 31 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

7 Valid · 5 Not valid · 0 Not checked · 25 Own results and interpretations

10 more claims

Create a free account to see all 37 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 18
  • Probably not a substantial claim. 12

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

4 of 4

Citations

Cites

Metadata

Authors
Xie, H.
Affiliations
Shenzhen Entropy Future Education Technology Co., Shenzhen, Guangdong, China
Corresponding author
Xie, Huiling
Venue
Proceedings of the 2025 International Conference on Artificial Intelligence and Educational Systems
Publisher
ACM
Published
2025-04-25
Pages
297-303
OpenAlex
W4412923181
Cited by
0
References
22
FWCI
0.00
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
International Conference on Artificial Intelligence and Educational Systems (publishedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
dynamic facial expression recognition, multimodal fusion, temporal attention, large multimodal models, video-based emotion recognition

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-29, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-29 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.