The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Synthesizing Obama: learning lip sync from audio
Abstract
Given audio of President Barack Obama, we synthesize a high quality video of him speaking with accurate lip sync, composited into a target video clip. Trained on many hours of his weekly address footage, a recurrent neural network learns the mapping from raw audio features to mouth shapes. Given the mouth shape at each time instant, we synthesize high quality mouth texture, and composite it with proper 3D pose matching to change what he appears to be saying in a target video to match the input audio track. Our approach produces photorealistic results.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A2V(Suwajanakorn et al. 2017) has used a recurrent neural network to train a model that can map from raw audio features of Obama's weekly address footage to mouth shapes.”
From Countering Malicious DeepFakes: Survey, Battleground, and Horizon (2022) · cites this paper as Suwajanakorn et al. 2017
In the citing paper
“It is a cross-modal method that leverages the pronunciation features of the target person to synthesize the correct lip shapes for given audio content.”
From Countering Malicious DeepFakes: Survey, Battleground, and Horizon (2022) · cites this paper as Suwajanakorn et al. 2017
In the citing paper
“It doesn't need an original video as expression-driven material.”
From Countering Malicious DeepFakes: Survey, Battleground, and Horizon (2022) · cites this paper as Suwajanakorn et al. 2017
In the citing paper
“For example, A2V (Suwajanakorn et al. 2017) successfully synthesized fake videos of Obama (i.e., the 44th president of the United States) according to the given audio.”
From Countering Malicious DeepFakes: Survey, Battleground, and Horizon (2022) · cites this paper as Suwajanakorn et al. 2017
In the citing paper
“Then, they generate photorealistic mouth texture based on the generated lip fiducials.”
From Countering Malicious DeepFakes: Survey, Battleground, and Horizon (2022) · cites this paper as Suwajanakorn et al. 2017
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems
Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., Corrado, G. S., Davis, A., Dean, J., Devin, M., et al. (2016) · arXiv · doi:10.48550/arxiv.1603.04467 · Cited by 9,408
Framewise phoneme classification with bidirectional LSTM and other neural network architectures
Graves, A., & Schmidhuber, J. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.06.042 · Cited by 5,839
Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.927467 · Cited by 5,409
Dlib-ml: A Machine Learning Toolkit
King, D. E. (2009) · doi:10.5555/1577069.1755843 · Cited by 2,777
Recurrent Neural Network Regularization
Zaremba, W., Sutskever, I., & Vinyals, O. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.2329 · Cited by 2,123
Supervised Descent Method and Its Applications to Face Alignment
Xiong, X., & De la Torre, F. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.75 · Cited by 1,896
Face2Face: Real-Time Face Capture and Reenactment of RGB Videos
Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., & Niessner, M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.262 · Cited by 1,747
Hybrid speech recognition with Deep Bidirectional LSTM
Graves, A., Jaitly, N., & Mohamed, A.-R. (2013) · 2013 IEEE Workshop on Automatic Speech Recognition and Understanding · doi:10.1109/asru.2013.6707742 · Cited by 1,736
Research works printed but not linked: 6
- Fabrice Bellard , M Niedermayer , and others. 2012. FFmpeg. Availabel from: http://ffm.peg.org ( 2012 ). Fabrice Bellard, M Niedermayer, and others. 2012. FFmpeg. Availabel from: http://ffm.peg.org (2012).
- G. Bradski . 2000. Dr. Dobb's Journal of Software Tools ( 2000 ). G. Bradski. 2000. Dr. Dobb's Journal of Software Tools (2000).
- Yarin Gal . 2015. A theoretically grounded application of dropout in recurrent neural networks. arXiv preprint arXiv:1512.05287 ( 2015 ). Yarin Gal. 2015. A theoretically grounded application of dropout in recurrent neural networks. arXiv preprint arXiv:1512.05287 (2015). · arXiv
- Alex Graves . 2013. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 ( 2013 ). Alex Graves. 2013. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 (2013). · arXiv
- Shunsuke Saito , Tianye Li , and Hao Li. 2016. Real-Time Facial Segmentation and Performance Capture from RGB Input. arXiv preprint arXiv:1604.02647 ( 2016 ). Shunsuke Saito, Tianye Li, and Hao Li. 2016. Real-Time Facial Segmentation and Performance Capture from RGB Input. arXiv preprint arXiv:1604.02647 (2016). · arXiv
- Aäron van den Oord , Sander Dieleman , Heiga Zen , Karen Simonyan , Oriol Vinyals , Alex Graves , Nal Kalchbrenner , Andrew Senior , and Koray Kavukcuoglu . 2016 . Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499 (2016). Aäron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. 2016. Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499 (2016). · arXiv
Cited by
FaceForensics++: Learning to Detect Manipulated Facial Images
Rossler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., & Niessner, M. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00009 · Cited by 3,314
Fake News Detection on Social Media: A Data Mining Perspective
Shu, K., Sliva, A., Wang, S., Tang, J., & Liu, H. (2017) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3137597.3137600 · Cited by 3,304
Kaplan, A., & Haenlein, M. (2019) · Business Horizons · doi:10.1016/j.bushor.2018.08.004 · Cited by 3,140
Celeb-DF: A Large-Scale Challenging Dataset for DeepFake Forensics
Li, Y., Yang, X., Sun, P., Qi, H., & Lyu, S. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00327 · Cited by 1,846
Deepfakes and beyond: A Survey of face manipulation and fake detection
Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., & Ortega-Garcia, J. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.06.014 · Cited by 1,237
Metadata
- Authors
- Suwajanakorn, S., Seitz, S. M., & Kemelmacher-Shlizerman, I.
- Affiliations
- University of Washington (US)
- Venue
- ACM Transactions on Graphics
- Published
- 2017-07-20
- Volume
- 36
- Issue
- 4
- Pages
- 1-13
- ISSN
- 0730-0301 (ISSN-L)
- 1557-7368
- OpenAlex
- W2738406145
- Semantic Scholar
- 7aa88dafb5d5fd5645c0ada2539e9eaf5b2fe949
- Cited by
- 1,057
- References
- 51
- Citations by year
- 2026: 46 · 2025: 101 · 2024: 135 · 2023: 169 · 2022: 139 · 2021: 159 · 2020: 137 · 2019: 97 · 2018: 67 · 2017: 6
- FWCI
- 34.40
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Graphics (publishedVersion)
- Funders
- Samsung
- Intel
- University of Washington Animation Research Labs
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Signal Processing › Speech and Audio Processing
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Keywords
- lip synchronization, audiovisual speech, video compositing, recurrent neural networks, Barack Obama
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Synthesizing Obama: learning lip sync from audio. https://app.cochat.ai/papers/10.1145/3072959.3073640
- The paper
- Suwajanakorn, S., Seitz, S. M., & Kemelmacher-Shlizerman, I. (2017). Synthesizing Obama: learning lip sync from audio. ACM Transactions on Graphics. https://doi.org/10.1145/3072959.3073640
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.