EAMM: One-Shot Emotional Talking Face via Audio-Based Emotion-Aware Motion Model
Open accessAbstract
Although significant progress has been made to audio-driven talking face generation, existing methods either neglect facial emotion or cannot be applied to arbitrary subjects. In this paper, we propose the Emotion-Aware Motion Model (EAMM) to generate one-shot emotional talking faces by involving an emotion source video. Specifically, we first propose an Audio2Facial-Dynamics module, which renders talking faces from audio-driven unsupervised zero- and first-order key-points motion. Then through exploring the motion model’s properties, we further propose an Implicit Emotion Displacement Learner to represent emotion-related facial dynamics as linearly additive displacements to the previously acquired motion representations. Comprehensive experiments demonstrate that by incorporating the results from both modules, our method can generate satisfactory talking face results on arbitrary subjects with realistic emotion patterns.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Several face reenactment strategies including EAMM, AVFR-GAN, DiffTalk, ADNeRF and ATVGnet are not well suited for zero-shot lip-synced generation of unseen speakers.
“We investigated several face reenactment strategies including EAMM [26], AVFR-GAN [2], DiffTalk [54], ADNeRF [20] and ATVGnet [10] and concluded that these methods are not well suited for zero-shot lip-synced generation of unseen speakers.”
From AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset (2024) · cites this paper as [26]
The citation lists haven’t been fetched yet
References 43 · Cited by 155
- Authors
- Ji, X., Zhou, H., Wang, K., Wu, Q., Wu, W., Xu, F., & Cao, X.
- Affiliations
- Nanjing University (CN)
- Chinese University of Hong Kong (HK)
- The University of Sydney (AU)
- Monash University (AU)
- Sensetime (China) (CN)
- Tsinghua University (CN)
- Venue
- Special Interest Group on Computer Graphics and Interactive Techniques Conference Proceedings
- Publisher
- ACM
- Published
- 2022-08-07
- Pages
- 1-10
- OpenAlex
- W4281730245
- Cited by
- 155
- References
- 43
- Citations by year
- 2026: 20 · 2025: 45 · 2024: 48 · 2023: 41 · 2022: 1
- FWCI
- 12.24
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- ACM SIGGRAPH Conference (SIGGRAPH) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-sa)
- arXiv (Cornell University)
- Funders
- Beijing Natural Science Foundation (JQ19015)
- NSFC (62025108,62021002, 61727808)
- National Key Research and Development Program of China (2018YFA0704000)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Keywords
- talking face generation, audio-driven face animation, facial emotion
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
