Cited by 687Generative Adversarial Networks and Image SynthesisDomain Adaptation and Few-Shot LearningFace recognition and analysis

Few-Shot Adversarial Learning of Realistic Neural Talking Head Models

Open access
Zakharov, Egor, Shysheya, Aliaksandra, Burkov, Egor, Lempitsky, Victor·2019·2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Abstract

Several recent works have shown how highly realistic human head images can be obtained by training convolutional neural networks to generate them. In order to create a personalized talking head model, these works require training on a large dataset of images of a single person. However, in many practical scenarios, such personalized talking head models need to be learned from a few image views of a person, potentially even a single image. Here, we present a system with such few-shot capability. It performs lengthy meta-learning on a large dataset of videos, and after that is able to frame few- and one-shot learning of neural talking head models of previously unseen people as adversarial training problems with high capacity generators and discriminators. Crucially, the system is able to initialize the parameters of both the generator and the discriminator in a person-specific way, so that training can be based on just a few images and done quickly, despite the need to tune tens of millions of parameters. We show that such an approach is able to learn highly realistic and personalized talking head models of new people and even portrait paintings.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/iccv.2019.00955

Cites

46 of 46 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Recent works on adversarial perturbation attacks to fool DNN-based detectors make the deepfake detection task more difficult.

    “Recent works on adversarial perturbation attacks to fool DNN-based detectors make the deepfake detection task more difficult [157, 171–174].”

    From Deep learning for deepfakes creation and detection: A survey (2022) · cites this paper as [172]

    The Introduction discusses adversarially trained networks for generating realistic talking heads, and §3.2 describes a discriminator used during generator training. Neither passage addresses adversarial perturbation attacks against DNN-based deepfake detectors or their effect on detection difficulty (searched for those terms and paraphrases). (LLM, conf=0.76)

    Experimental — accuracy not yet measured
  • Zakharov et al. used the cosine similarity between embedding vectors of the ArcFace network for measuring identity mismatch.

    “Zakharov et al . [117] used the cosine similarity between embedding vectors of the ArcFace network [118] for measuring identity mismatch.”

    From Deep learning for visual speech analysis: A survey (2024) · cites this paper as [117]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 5

  1. [10]10. Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model- agnostic meta-learning for fast adaptation of deep networks. In Proc. ICML, pages 1126-1135, 2017. 2
  2. [12]12. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672-2680, 2014. 2
  3. [27]27. Koki Nagano, Jaewoo Seo, Jun Xing, Lingyu Wei, Zimo Li, Shunsuke Saito, Aviral Agarwal, Jens Fursund, Hao Li, Richard Roberts, et al. paGAN: real-time avatars using dy- namic textures. In SIGGRAPH Asia 2018 Technical Papers, page 258. ACM, 2018. 2
  4. [33]33. Karen Simonyan and Andrew Zisserman. Very deep convo- lutional networks for large-scale image recognition. In Proc. ICLR, 2015. 4
  5. [46]46. Ruixiang Zhang, Tong Che, Zoubin Ghahramani, Yoshua Bengio, and Yangqiu Song. Metagan: An adversarial ap- proach to few-shot learning. In NeurIPS, pages 2371-2380, 2018.

Cited by

Sorted by citations

Authors
Zakharov, E., Shysheya, A., Burkov, E., & Lempitsky, V.
Affiliations
  • Skolkovo Institute of Science and Technology (RU)
  • Samsung (South Korea) (KR)
Venue
2019 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher
IEEE
Published
2019-10
Pages
9458-9467
OpenAlex
W2945052639
Cited by
687
References
46
Citations by year
2026: 19 · 2025: 68 · 2024: 86 · 2023: 123 · 2022: 130 · 2021: 135 · 2020: 104 · 2019: 22
FWCI
31.89
Type
conference paper
Language
en
Open access
Locations
  • IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
Keywords
few-shot learning, one-shot learning, adversarial training, meta-learning, generative adversarial networks

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review