CoChat
Sign Up
Cited by 621Generative Adversarial Networks and Image SynthesisFace recognition and analysisHuman Motion and Animation

GANimation: Anatomically-Aware Facial Animation from a Single Image

Open access
Pumarola, Albert, Agudo, Antonio, Martinez, Aleix M., Sanfeliu, Alberto, Moreno-Noguer, Francesc·2018·Lecture Notes in Computer Science
DOI 10.1007/978-3-030-01249-6_50PMC PMC6240441
Open this paper in CoChat — free account

Abstract

Recent advances in Generative Adversarial Networks (GANs) have shown impressive results for task of facial expression synthesis. The most successful architecture is StarGAN [4], that conditions GANs' generation process with images of a specific domain, namely a set of images of persons sharing the same expression. While effective, this approach can only generate a discrete number of expressions, determined by the content of the dataset. To address this limitation, in this paper, we introduce a novel GAN conditioning scheme based on Action Units (AU) annotations, which describes in a continuous manifold the anatomical facial movements defining a human expression. Our approach allows controlling the magnitude of activation of each AU and combine several of them. Additionally, we propose a fully unsupervised strategy to train the model, that only requires images annotated with their activated AUs, and exploit attention mechanisms that make our network robust to changing backgrounds and lighting conditions. Extensive evaluation show that our approach goes beyond competing conditional generators both in the capability to synthesize a much wider range of expressions ruled by anatomically feasible muscle movements, as in the capacity of dealing with images in the wild.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/978-3-030-01249-6_50

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • GANimation has introduced a novel GAN conditioning method based on action units (AU) annotations.

    “GAN animation (GANimation) (Pumarola et al. 2018) has introduced a novel GANconditioning method based on action units (AU) annotations.”

    From Countering Malicious DeepFakes: Survey, Battleground, and Horizon (2022) · cites this paper as Pumarola et al. 2018

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 31

  1. [1]Arjovsky M, Chintala S, Bottou L: Wasserstein GAN. arXiv preprint arXiv:1701.07875 (2017) · arXiv
  2. [3]Benitez-Quiroz CF, Srinivasan R, Martinez AM,: Emotionet: An accurate, real-time algorithm for the automatic annotation of a million facial expressions in the wild. In: CVPR (2016)
  3. [5]Du S, Tao Y, Martinez AM: Compound facial expressions of emotion. Proceedings of the National Academy of Sciences p. 201322355 (2014) PMC3992629 24706770
  4. [6]Ekman P, Friesen W: Facial action coding system: A technique for the measurement of facial movement. Consulting Psychologists Press (1978)
  5. [7]Fischler MA, Elschlager RA: The representation and matching of pictorial structures. IEEE Transactions on Computers 22(1), 67– 92 (1973)
  6. [8]Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y: Generative adversarial nets. In: NIPS (2014)
  7. [10]Isola P, Zhu JY, Zhou T, Efros AA: Image-to-image translation with conditional adversarial networks. In: CVPR (2017)
  8. [11]Johnson J, Alahi A, Fei-Fei L: Perceptual losses for real-time style transfer and super-resolution. In: ECCV (2016)
  9. [12]Karras T, Aila T, Laine S, Lehtinen J: Progressive growing of GANs for improved quality, stability, and variation. In: ICLR (2018)
  10. [13]Kim T, Cha M, Kim H, Lee J, Kim J: Learning to discover cross-domain relations with generative adversarial networks. In: ICML (2017)
  11. [14]Kingma D, Ba J: ADAM: A method for stochastic optimization. In: ICLR (2015)
  12. [15]Kingma DP, Welling M: Auto-encoding variational bayes. In: ICLR (2014)
  13. [16]Langner O, Dotsch R, Bijlstra G, Wigboldus DH, Hawk ST, Van Knip-penberg A: Presentation and validation of the radboud faces database. Cognition and emotion 24(8), 1377– 1388 (2010)
  14. [17]Larsen ABL, Spnderby SK, Larochelle H, Winther O: Autoencoding beyond pixels using a learned similarity metric. In: ICML (2016)
  15. [19]Li C, Wand M: Precomputed real-time texture synthesis with markovian generative adversarial networks. In: ECCV (2016)
  16. [20]Li M, Zuo W, Zhang D: Deep identity-aware transfer of facial attributes. arXiv preprint arXiv:1610.05586 (2016) · arXiv
  17. [21]Liu MY, Breuel T, Kautz J: Unsupervised image-to-image translation networks. In: NIPS (2017)
  18. [22]Mathieu M, Couprie C, LeCun Y: Deep multi-scale video prediction beyond mean square error. In: ICLR (2016)
  19. [23]Mirza M, Osindero S: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014) · arXiv
  20. [24]Odena A, Olah C, Shlens J: Conditional image synthesis with auxiliary classifier GANs. In: ICML (2017)
  21. [25]Pathak D, Krahenbuhl P, Donahue J, Darrell T, Efros AA: Context encoders: Feature learning by inpainting. In: CVPR (2016)
  22. [26]Perarnau G, van de Weijer J, Raducanu B, Álvarez JM: Invertible conditional GANs for image editing. arXiv preprint arXiv:1611.06355 (2016) · arXiv
  23. [27]Pumarola A, Agudo A, Sanfeliu A, Moreno-Noguer F: Unsupervised person image synthesis in arbitrary poses. In: CVPR (2018)
  24. [28]Radford A, Metz L, Chintala S: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: ICLR (2016)
  25. [29]Reed S, Akata Z, Yan X, Logeswaran L, Schiele B, Lee H: Generative adversarial text to image synthesis. In: ICML (2016)
  26. [31]Shen W, Liu R: Learning residual images for face attribute manipulation. In: CVPR (2017)
  27. [32]Shrivastava A, Pfister T, Tuzel O, Susskind J, Wang W, Webb R: Learning from simulated and unsupervised images through adversarial training. In: CVPR (2017)
  28. [33]Wang X, Gupta A: Generative image modeling using style and structure adversarial networks. In: ECCV (2016)
  29. [34]Wang Z, Liu D, Yang J, Han W, Huang T: Deep networks for image superresolution with sparse prior. In: ICCV (2015)
  30. [37]Zhang H, Xu T, Li H, Zhang S, Huang X, Wang X, Metaxas D: Stack-gan: Text to photo-realistic image synthesis with stacked generative adversarial networks. In: ICCV (2017) 30010548
  31. [39]Zhu S, Fidler S, Urtasun R, Lin D, Loy CC: Be your own prada: Fashion synthesis with structural coherence. In: ICCV (2017)

Cited by

Sorted by citations

Authors
Pumarola, A., Agudo, A., Martinez, A. M., Sanfeliu, A., & Moreno-Noguer, F.
Affiliations
Consejo Superior de Investigaciones Científicas (ES)
Corresponding author
Pumarola, Albert
ORCID
Venue
Lecture Notes in Computer Science
Publisher
Springer International Publishing
Published
2018
Volume
11214
Pages
835-851
ISSN
  • 0302-9743 (ISSN-L)
  • 1611-3349
DOI
10.1007/978-3-030-01249-6_50
PMC
PMC6240441
OpenAlex
W2883861033
Cited by
621
References
4
Citations by year
2026: 17 · 2025: 46 · 2024: 71 · 2023: 84 · 2022: 83 · 2021: 97 · 2020: 132 · 2019: 83 · 2018: 8
FWCI
24.11
Type
conference paper
Language
en
Open access
Licence
Locations
  • Lecture notes in computer science (publishedVersion)
  • PubMed (publishedVersion)
  • Europe PMC (PubMed Central) (acceptedVersion)
  • PubMed Central (acceptedVersion)
  • UPCommons institutional repository (Universitat Politècnica de Catalunya) (submittedVersion, other-oa)
  • DIGITAL.CSIC (Spanish National Research Council (CSIC)) (submittedVersion)
Funders
  • Nvidia
  • Ministerio de Economía y Competitividad (644271, MDM-2016-0656)
  • National Eye Institute (R01EY020834)
  • National Institute on Deafness and Other Communication Disorders (R01DC014498)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
  • Physical Sciences › Engineering › Control and Systems Engineering › Human Motion and Animation
Keywords
facial expression synthesis, facial animation, facial action units, conditional generative adversarial networks, unsupervised training, attention mechanism, StarGAN, facial muscle movement

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review