Student or educator? Sign up with your school email and get 6 months of Pro free.

The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.

Synthesizing Obama: learning lip sync from audio

Abstract

Given audio of President Barack Obama, we synthesize a high quality video of him speaking with accurate lip sync, composited into a target video clip. Trained on many hours of his weekly address footage, a recurrent neural network learns the mapping from raw audio features to mouth shapes. Given the mouth shape at each time instant, we synthesize high quality mouth texture, and composite it with proper 3D pose matching to change what he appears to be saying in a target video to match the input audio track. Our approach produces photorealistic results.

Overview

Integrity signals

Registry

Crossref · 10.1145/3072959.3073640

Cites

51 of 51 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

6 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 51 references

Research works printed but not linked: 6

  1. Fabrice Bellard , M Niedermayer , and others. 2012. FFmpeg. Availabel from: http://ffm.peg.org ( 2012 ). Fabrice Bellard, M Niedermayer, and others. 2012. FFmpeg. Availabel from: http://ffm.peg.org (2012).
  2. G. Bradski . 2000. Dr. Dobb's Journal of Software Tools ( 2000 ). G. Bradski. 2000. Dr. Dobb's Journal of Software Tools (2000).
  3. Yarin Gal . 2015. A theoretically grounded application of dropout in recurrent neural networks. arXiv preprint arXiv:1512.05287 ( 2015 ). Yarin Gal. 2015. A theoretically grounded application of dropout in recurrent neural networks. arXiv preprint arXiv:1512.05287 (2015). · arXiv
  4. Alex Graves . 2013. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 ( 2013 ). Alex Graves. 2013. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 (2013). · arXiv
  5. Shunsuke Saito , Tianye Li , and Hao Li. 2016. Real-Time Facial Segmentation and Performance Capture from RGB Input. arXiv preprint arXiv:1604.02647 ( 2016 ). Shunsuke Saito, Tianye Li, and Hao Li. 2016. Real-Time Facial Segmentation and Performance Capture from RGB Input. arXiv preprint arXiv:1604.02647 (2016). · arXiv
  6. Aäron van den Oord , Sander Dieleman , Heiga Zen , Karen Simonyan , Oriol Vinyals , Alex Graves , Nal Kalchbrenner , Andrew Senior , and Koray Kavukcuoglu . 2016 . Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499 (2016). Aäron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. 2016. Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499 (2016). · arXiv

Cited by

See all 1,057

Metadata

Authors
Suwajanakorn, S., Seitz, S. M., & Kemelmacher-Shlizerman, I.
Affiliations
University of Washington (US)
Venue
ACM Transactions on Graphics
Published
2017-07-20
Volume
36
Issue
4
Pages
1-13
ISSN
  • 0730-0301 (ISSN-L)
  • 1557-7368
OpenAlex
W2738406145
Cited by
1,057
References
51
Citations by year
2026: 46 · 2025: 101 · 2024: 135 · 2023: 169 · 2022: 139 · 2021: 159 · 2020: 137 · 2019: 97 · 2018: 67 · 2017: 6
FWCI
34.40
Type
journal article
Language
en
Open access
closed
Locations
ACM Transactions on Graphics (publishedVersion)
Funders
  • Samsung
  • Google
  • Intel
  • University of Washington Animation Research Labs
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
Keywords
lip synchronization, audiovisual speech, video compositing, recurrent neural networks, Barack Obama

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Synthesizing Obama: learning lip sync from audio. https://app.cochat.ai/papers/10.1145/3072959.3073640
The paper
Suwajanakorn, S., Seitz, S. M., & Kemelmacher-Shlizerman, I. (2017). Synthesizing Obama: learning lip sync from audio. ACM Transactions on Graphics. https://doi.org/10.1145/3072959.3073640

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.