Deep learning and synthetic media
Open accessThe DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
Deep learning algorithms are rapidly changing the way in which audiovisual media can be produced. Synthetic audiovisual media generated with deep learning—often subsumed colloquially under the label “deepfakes”—have a number of impressive characteristics; they are increasingly trivial to produce, and can be indistinguishable from real sounds and images recorded with a sensor. Much attention has been dedicated to ethical concerns raised by this technological development. Here, I focus instead on a set of issues related to the notion of synthetic audiovisual media, its place within a broader taxonomy of audiovisual media, and how deep learning techniques differ from more traditional approaches to media synthesis. After reviewing important etiological features of deep learning pipelines for media manipulation and generation, I argue that “deepfakes” and related synthetic media produced with such pipelines do not merely offer incremental improvements over previous methods, but challenge traditional taxonomical distinctions, and pave the way for genuinely novel kinds of audiovisual media.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Diffusion Models Beat GANs on Image Synthesis
Dhariwal, P., & Nichol, A. (2021) · arXiv · doi:10.48550/arxiv.2105.05233 · Cited by 3,362
Semantic Image Synthesis With Spatially-Adaptive Normalization
Park, T., Liu, M.-Y., Wang, T.-C., & Zhu, J.-Y. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00244 · Cited by 2,974
A Neural Algorithm of Artistic Style
Gatys, L., Ecker, A., & Bethge, M. (2016) · Journal of Vision · doi:10.1167/16.12.326 · Cited by 2,612
Taming Transformers for High-Resolution Image Synthesis
Esser, P., Rombach, R., & Ommer, B. (2020) · arXiv · doi:10.48550/arxiv.2012.09841 · Cited by 2,514
Taming Transformers for High-Resolution Image Synthesis
Esser, P., Rombach, R., & Ommer, B. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01268 · Cited by 2,514
Zero-shot text-to-image generation
Ramesh, A., Pavlov, M., Goh, G., Gray, S., Voss, C., Radford, A., Chen, M., & Sutskever, I. (2021) · arXiv · doi:10.48550/arxiv.2102.12092 · Cited by 2,417
Carlsson, G. (2009) · Bulletin of the American Mathematical Society · doi:10.1090/s0273-0979-09-01249-x · Cited by 2,302
Chen, C., Chen, Q., Xu, J., & Koltun, V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00347 · Cited by 1,376
Deepfakes and beyond: A Survey of face manipulation and fake detection
Tolosana, R., Vera-Rodriguez, R., Fierrez, J., Morales, A., & Ortega-Garcia, J. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.06.014 · Cited by 1,237
Interpreting the Latent Space of GANs for Semantic Face Editing
Shen, Y., Gu, J., Tang, X., & Zhou, B. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00926 · Cited by 1,063
A Lip Sync Expert Is All You Need for Speech to Lip Generation In the Wild
Prajwal, K. R., Mukhopadhyay, R., Namboodiri, V. P., & Jawahar, C. V. (2020) · Proceedings of the 28th ACM International Conference on Multimedia · doi:10.1145/3394171.3413532 · Cited by 897 · cited 1×
“, 2019), while lip syncing consists in synchronizing lip movements on a video to match a target speech segment (Prajwal et al., 2020).”
BackgroundFew-Shot Adversarial Learning of Realistic Neural Talking Head Models
Zakharov, E., Shysheya, A., Burkov, E., & Lempitsky, V. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00955 · Cited by 687
Testing the manifold hypothesis
Fefferman, C., Mitter, S., & Narayanan, H. (2016) · Journal of the American Mathematical Society · doi:10.1090/jams/852 · Cited by 430
The Epistemic Threat of Deepfakes
Fallis, D. (2021) · Philosophy & Technology · doi:10.1007/s13347-020-00419-2 · Cited by 266 · cited 1×
“While deepfakes have recently garnered attention in philosophy, the discussion has mostly focused on their potentially harmful uses, such as impersonating identities and disseminating false information (Floridi, 2018; de Ruiter, 2021), undermining the epistemic and testimonial value of photographic media and videos (F…”
BackgroundScore-based Generative Modeling in Latent Space
Vahdat, A., Kreis, K., & Kautz, J. (2021) · arXiv · doi:10.48550/arxiv.2106.05931 · Cited by 224
StyleGAN-V: A Continuous Video Generator with the Price, Image Quality and Perks of StyleGAN2
Skorokhodov, I., Tulyakov, S., & Elhoseiny, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00361 · Cited by 199
The Distinct Wrong of Deepfakes
de Ruiter, A. (2021) · Philosophy & Technology · doi:10.1007/s13347-021-00459-2 · Cited by 175
Introducing the pervert’s dilemma: a contribution to the critique of Deepfake Pornography
Öhman, C. (2020) · Ethics and Information Technology · doi:10.1007/s10676-019-09522-1 · Cited by 130 · cited 1×
“While deepfakes have recently garnered attention in philosophy, the discussion has mostly focused on their potentially harmful uses, such as impersonating identities and disseminating false information (Floridi, 2018; de Ruiter, 2021), undermining the epistemic and testimonial value of photographic media and videos (F…”
BackgroundA Good Image Generator Is What You Need for High-Resolution Video Synthesis
Tian, Y., Ren, J., Chai, M., Olszewski, K., Peng, X., Metaxas, D. N., & Tulyakov, S. (2021) · arXiv · doi:10.48550/arxiv.2104.15069 · Cited by 88
DriveGAN: Towards a Controllable High-Quality Neural Simulation
Kim, S. W., Philion, J., Torralba, A., & Fidler, S. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00576 · Cited by 72
Cited by
Typology of Risks of Generative Text-to-Image Models
Bird, C., Ungless, E., & Kasirzadeh, A. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604722 · Cited by 121
Twomey, J., Ching, D., Aylett, M. P., Quayle, M., Linehan, C., & Murphy, G. (2023) · PLOS ONE · doi:10.1371/journal.pone.0291668 · Cited by 87
Pedersen, K. T., Pepke, L., Stærmose, T., Papaioannou, M., Choudhary, G., & Dragoni, N. (2025) · Journal of Cybersecurity and Privacy · doi:10.3390/jcp5020018 · Cited by 15
Metadata
- Authors
- Millière, R.
- Venue
- Synthese
- Published
- 2022-05-20
- Volume
- 200
- Issue
- 3
- ISSN
- 0039-7857 (ISSN-L)
- 1573-0964
- OpenAlex
- W4280549751
- Cited by
- 44
- References
- 22
- FWCI
- 34.68
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.