CoChat
Sign Up
PreprintCited by 19

The DOI matches its DataCite record; 1 of the 1 references it cites were found in a registry or an index.

Extracting Training Data from Diffusion Models

Carlini, Nicholas, Hayes, Jamie, Nasr, Milad, Jagielski, Matthew, Sehwag, Vikash, Tramèr, Florian, Balle, Borja, Ippolito, Daphne, Wallace, Eric·2023·arXiv
DOI 10.48550/arxiv.2301.13188arXiv 2301.13188
Open this paper in CoChat — free account

Abstract

Image diffusion models such as DALL-E 2, Imagen, and Stable Diffusion have attracted significant attention due to their ability to generate high-quality synthetic images. In this work, we show that diffusion models memorize individual images from their training data and emit them at generation time. With a generate-and-filter pipeline, we extract over a thousand training examples from state-of-the-art models, ranging from photographs of individual people to trademarked company logos. We also train hundreds of diffusion models in various settings to analyze how different modeling and data decisions affect privacy. Overall, our results show that diffusion models are much less private than prior generative models such as GANs, and that mitigating these vulnerabilities may require new advances in privacy-preserving training.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2301.13188

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Figures

Caption: Living in the light with Ann Graham Lotz
Caption: Living in the light with Ann Graham LotzCC-BY-SA · From the PDF
Prompt: Ann Graham Lotz
Prompt: Ann Graham LotzCC-BY-SA · From the PDF
Figure 2 We do not count the generated image of Obama (at left) as memorized because it has a high ℓ 2 distance to every training image. The four nearest training images are shown at right, each has a distance above 0 . 3.
Figure 2 We do not count the generated image of Obama (at left) as memorized because it has a high ℓ 2 distance to every training image. The four nearest training images are shown at right, each has a distance above 0 . 3.CC-BY-SA · From the PDF
Figure 3 Examples of the images that we extract from Stable Diffusion v1.4 using random sampling and our membership inference procedure. The top row shows the original images and the bottom row shows our extracted images.
Figure 3 Examples of the images that we extract from Stable Diffusion v1.4 using random sampling and our membership inference procedure. The top row shows the original images and the bottom row shows our extracted images.CC-BY-SA · From the PDF
Figure 4 Our attack reliably separates novel generations from memorized training examples, under two definitions of memorization-either ( ℓ 2 , 0 . 15 ) -extraction or manual human inspection of generated images.
Figure 4 Our attack reliably separates novel generations from memorized training examples, under two definitions of memorization-either ( ℓ 2 , 0 . 15 ) -extraction or manual human inspection of generated images.CC-BY-SA · From the PDF
Figure 5 Our attack extracts images from Stable Diffusion most often when they have been duplicated at least k = 100 times; although this should be taken as an upper bound because our methodology explicitly searches for memorization of duplicated images.
Figure 5 Our attack extracts images from Stable Diffusion most often when they have been duplicated at least k = 100 times; although this should be taken as an upper bound because our methodology explicitly searches for memorization of duplicated images.CC-BY-SA · From the PDF
Figure 6 Direct 2-norm measurement fails to identify memorized CIFAR-10 examples. Each of the above images have a ℓ 2 distance of less than 0 . 05, yet only one (the car) is actually a memorized training example.
Figure 6 Direct 2-norm measurement fails to identify memorized CIFAR-10 examples. Each of the above images have a ℓ 2 distance of less than 0 . 05, yet only one (the car) is actually a memorized training example.CC-BY-SA · From the PDF
Figure 7 Per-image ℓ 2 thresholds are necessary to separate memorized images from novel generations on a CIFAR-10 model. Each plot shows the distribution of ℓ 2 distances from a generated image to all training images (along with the image and the nearest training image). Left shows a typical distribution for a non-memorized image. Right shows a memorized image distribution; while the most similar training image has high absolute ℓ 2 distance, it is abnormally low for this distribution. The dashed black line shows our adaptive ℓ 2 threshold.
Figure 7 Per-image ℓ 2 thresholds are necessary to separate memorized images from novel generations on a CIFAR-10 model. Each plot shows the distribution of ℓ 2 distances from a generated image to all training images (along with the image and the nearest training image). Left shows a typical distribution for a non-memorized image. Right shows a memorized image distribution; while the most similar training image has high absolute ℓ 2 distance, it is abnormally low for this distribution. The dashed black line shows our adaptive ℓ 2 threshold.CC-BY-SA · From the PDF
Image 9
CC-BY-SA · From the PDF
Figure 9 We run membership inference using LiRA and compute the diffusion model loss at different noise timesteps on CIFAR-10. Evaluating L ( · , t , · ) at t ∈ [ 50 , 300 ] produces the best results.
Figure 9 We run membership inference using LiRA and compute the diffusion model loss at different noise timesteps on CIFAR-10. Evaluating L ( · , t , · ) at t ∈ [ 50 , 300 ] produces the best results.CC-BY-SA · From the PDF
Figure 10 Membership inference ROC curve for a diffusion model trained on CIFAR-10 using the loss threshold attack, baseline LiRA, and 'Strong LiRA' with repeated queries and augmentation (§5.2.2).
Figure 10 Membership inference ROC curve for a diffusion model trained on CIFAR-10 using the loss threshold attack, baseline LiRA, and 'Strong LiRA' with repeated queries and augmentation (§5.2.2).CC-BY-SA · From the PDF
Figure 11 Better diffusion models are more vulnerable to membership inference attacks; evaluating with TPR at an FPR of 1%. As the FID decreases (corresponding to a quality increase) the membership inference attack success rate grows from 7% to nearly 100%.
Figure 11 Better diffusion models are more vulnerable to membership inference attacks; evaluating with TPR at an FPR of 1%. As the FID decreases (corresponding to a quality increase) the membership inference attack success rate grows from 7% to nearly 100%.CC-BY-SA · From the PDF
Figure 12 Evaluating inpainting attacks on 100 CIFAR10 examples, measuring the ℓ 2 distance between images and their inpainted reconstructions when we mask out the left half of the image for 100 randomly selected images. We also plot the ℓ 2 distances for the bird and cat examples shown in Figure 13. When an adversary has partial knowledge of an image, inpainting attacks work far better than typical data extraction.
Figure 12 Evaluating inpainting attacks on 100 CIFAR10 examples, measuring the ℓ 2 distance between images and their inpainted reconstructions when we mask out the left half of the image for 100 randomly selected images. We also plot the ℓ 2 distances for the bird and cat examples shown in Figure 13. When an adversary has partial knowledge of an image, inpainting attacks work far better than typical data extraction.CC-BY-SA · From the PDF
Figure 13 Inpainting-based reconstruction attack on CIFAR-10. Given an image from CIFAR-10 (first column), we randomly mask half of the image (second column), and then inpaint the image for a model which contained this image in the training set (third column) versus inpainting the image for a model which did not contain this image in the training set (fourth column).
Figure 13 Inpainting-based reconstruction attack on CIFAR-10. Given an image from CIFAR-10 (first column), we randomly mask half of the image (second column), and then inpaint the image for a model which contained this image in the training set (third column) versus inpainting the image for a model which did not contain this image in the training set (fourth column).CC-BY-SA · From the PDF
Figure 14 Membership inference results on GAN models using the loss threshold and LiRA attacks on the discriminator. Overall, GANs are significantly more private than diffusion models under default training configurations.
Figure 14 Membership inference results on GAN models using the loss threshold and LiRA attacks on the discriminator. Overall, GANs are significantly more private than diffusion models under default training configurations.CC-BY-SA · From the PDF
Figure 15 Selected training examples we extract from three GANs trained on CIFAR-10 for different architectures. Top row: generated output from a diffusion model. Bottom row: nearest ( ℓ 2 ) example from the training dataset. Figure 25 in the Appendix contains all unique extracted images.
Figure 15 Selected training examples we extract from three GANs trained on CIFAR-10 for different architectures. Top row: generated output from a diffusion model. Bottom row: nearest ( ℓ 2 ) example from the training dataset. Figure 25 in the Appendix contains all unique extracted images.CC-BY-SA · From the PDF
Figure 16 Canary exposure (a measure of non-privacy) as a function of duplicate count. Inserting a canary twice is sufficient to reach maximum exposure.
Figure 16 Canary exposure (a measure of non-privacy) as a function of duplicate count. Inserting a canary twice is sufficient to reach maximum exposure.CC-BY-SA · From the PDF
Figure 17 All 1280 images we extract from diffusion models trained on CIFAR-10, after 1 million generations from 16 diffusion models.
Figure 17 All 1280 images we extract from diffusion models trained on CIFAR-10, after 1 million generations from 16 diffusion models.CC-BY-SA · From the PDF
Image 19
CC-BY-SA · From the PDF
Image 20
CC-BY-SA · From the PDF
(c) ROC curve for the membership attack for different training steps.
(c) ROC curve for the membership attack for different training steps.CC-BY-SA · From the PDF
Figure 19 We can improve membership inference attack success rates on CIFAR-10 by reducing noise. In (a) , membership inference attacks are improved by averaging the loss over multiple noise samples in the diffusion process. In (b) , attacks are improved by querying on augmented versions of the candidate image.
Figure 19 We can improve membership inference attack success rates on CIFAR-10 by reducing noise. In (a) , membership inference attacks are improved by averaging the loss over multiple noise samples in the diffusion process. In (b) , attacks are improved by querying on augmented versions of the candidate image.CC-BY-SA · From the PDF
Image 23
CC-BY-SA · From the PDF
Figure 20 When performing our membership inference attack, the hardest-to-attack examples (left) are all duplicates in the CIFAR-10 training set, and the easiest-to-attack examples (right) are visually outliers from CIFAR-10 images.
Figure 20 When performing our membership inference attack, the hardest-to-attack examples (left) are all duplicates in the CIFAR-10 training set, and the easiest-to-attack examples (right) are visually outliers from CIFAR-10 images.CC-BY-SA · From the PDF
Figure 21 Membership attack against a conditional and unconditional diffusion model on CIFAR-10.
Figure 21 Membership attack against a conditional and unconditional diffusion model on CIFAR-10.CC-BY-SA · From the PDF
(a) Loss (using the main model at diffusion timestep 100) on all 5,000 inpainted examples Xrec .
(a) Loss (using the main model at diffusion timestep 100) on all 5,000 inpainted examples Xrec .CC-BY-SA · From the PDF
Image 27
CC-BY-SA · From the PDF
Image 28
CC-BY-SA · From the PDF
(b) Comparison of loss on main and support models (at diffusion timestep 100) on all 5,000 inpainted examples.
(b) Comparison of loss on main and support models (at diffusion timestep 100) on all 5,000 inpainted examples.CC-BY-SA · From the PDF
(d) 8 inpainted examples with the smallest loss. Leftmost is the original example, second to left is the masked example and the rest are inpainted examples.
(d) 8 inpainted examples with the smallest loss. Leftmost is the original example, second to left is the masked example and the rest are inpainted examples.CC-BY-SA · From the PDF
(a) Loss (using the main model at diffusion timestep 100) on all 5,000 inpainted examples Xrec .
(a) Loss (using the main model at diffusion timestep 100) on all 5,000 inpainted examples Xrec .CC-BY-SA · From the PDF
Image 32
CC-BY-SA · From the PDF
(b) Comparison of loss on main and support models (at diffusion timestep 100) on all 5,000 inpainted examples.
(b) Comparison of loss on main and support models (at diffusion timestep 100) on all 5,000 inpainted examples.CC-BY-SA · From the PDF
(c) Contrastive loss ( main model loss support model loss ) on all 5,000 inpainted examples Xrec .
(c) Contrastive loss ( main model loss support model loss ) on all 5,000 inpainted examples Xrec .CC-BY-SA · From the PDF
Image 35
CC-BY-SA · From the PDF
Figure 25 Training examples extracted from different publicly available pretrained GANs and diffusion (DDPM) models. We use normalized ℓ 2 distance in pixel space to find memorized training samples. In each pair of images, left and right image corresponds to real and it closely synthetic image. For StyleGAN-ADA and DDPM model we display 120 pairs with smallest normalized ℓ 2 distance. For others we display all memorized training images. 1M generations
Figure 25 Training examples extracted from different publicly available pretrained GANs and diffusion (DDPM) models. We use normalized ℓ 2 distance in pixel space to find memorized training samples. In each pair of images, left and right image corresponds to real and it closely synthetic image. For StyleGAN-ADA and DDPM model we display 120 pairs with smallest normalized ℓ 2 distance. For others we display all memorized training images. 1M generationsCC-BY-SA · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 1 references

Research works printed but not linked: 61

  1. Martín Abadi, Andy Chu, Ian Goodfellow, H Bren- dan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In ACM CCS, 2016.
  2. Hazrat Ali, Shafaq Murad, and Zubair Shah. Spot the fake lungs: Generating synthetic medical im- ages using neural diffusion models. arXiv preprint arXiv:2211.00902, 2022. · arXiv
  3. Sanjeev Arora, Andrej Risteski, and Yi Zhang. Do GANs learn the distribution? Some theory and em- pirics. In International Conference on Learning Representations, 2018.
  4. Yogesh Balaji, Hamed Hassani, Rama Chellappa, and Soheil Feizi. Entropic GANs meet VAEs: A statistical approach to compute sample likelihoods in GANs. In International Conference on Machine Learning, 2019.
  5. Borja Balle, Giovanni Cherubin, and Jamie Hayes. Reconstructing training data with informed adver- saries. In IEEE Symposium on Security and Pri- vacy, 2022.
  6. Hannah Brown, Katherine Lee, Fatemehsadat Mireshghallah, Reza Shokri, and Florian Tramèr. What does it mean for a language model to pre- serve privacy? In ACM Conference on Fairness, Accountability, and Transparency, 2022.
  7. Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramer. Mem- bership inference attacks from first principles. In IEEE Symposium on Security and Privacy. IEEE, 2022.
  8. Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramer, and Chiyuan Zhang. Quantifying memorization across neural language models. arXiv preprint arXiv:2202.07646, 2022. · arXiv
  9. Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. The secret sharer: Evaluating and testing unintended memorization in neural networks. In USENIX Security Symposium, 2019.
  10. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ul- far Erlingsson, et al. Extracting training data from large language models. In USENIX Security Sym- posium, 2021.
  11. Pierre Chambon, Christian Bluethgen, Jean-Benoit Delbrouck, Rogier Van der Sluijs, Małgorzata Połacin, Juan Manuel Zambrano Chaves, Tan- ishq Mathew Abraham, Shivanshu Purohit, Cur- tis P. Langlotz, and Akshay Chaudhari. RoentGen: Vision-language foundation model for chest X- ray generation. arXiv preprint arXiv:2211.12737, 2022. · arXiv
  12. Pierre Chambon, Christian Bluethgen, Curtis P. Langlotz, and Akshay Chaudhari. Adapting pretrained vision-language foundational models to medical imaging domains. arXiv preprint arXiv:2210.04133, 2022. · arXiv
  13. Prafulla Dhariwal and Alexander Nichol. Diffusion models beat GANs on image synthesis. Advances in Neural Information Processing Systems, 2021.
  14. Tim Dockhorn, Tianshi Cao, Arash Vahdat, and Karsten Kreis. Differentially private diffusion mod- els. arXiv preprint arXiv:2210.09929, 2022. · arXiv
  15. Vitaly Feldman. Does learning require memo- rization? A short tale about a long tail. In ACM SIGACT Symposium on Theory of Comput- ing, 2020.
  16. Vitaly Feldman and Chiyuan Zhang. What neural networks memorize and why: Discovering the long tail via influence estimation. Advances in Neural Information Processing Systems, 2020.
  17. Qianli Feng, Chenqi Guo, Fabian Benitez-Quiroz, and Aleix M Martinez. When do GANs replicate? on the choice of dataset size. In IEEE/CVF Inter- national Conference on Computer Vision, 2021.
  18. Matt Fredrikson, Somesh Jha, and Thomas Risten- part. Model inversion attacks that exploit confi- dence information and basic countermeasures. In ACM Conference on Computer and Communica- tions Security (CCS), 2015.
  19. Matthew Fredrikson, Eric Lantz, Somesh Jha, Si- mon Lin, David Page, and Thomas Ristenpart. Privacy in pharmacogenetics: An end-to-end case study of personalized warfarin dosing. In USENIX Security Symposium, 2014.
  20. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016.
  21. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Advances in Neural Informa- tion Processing Systems, 2014.
  22. Peter Henderson, Koustuv Sinha, Nicolas Angelard-Gontier, Nan Rosemary Ke, Genevieve Fried, Ryan Lowe, and Joelle Pineau. Ethical challenges in data-driven dialogue systems. In AAAI/ACM Conference on AI, Ethics, and Society, 2018.
  23. Jonathan Ho, Ajay Jain, and Pieter Abbeel. De- noising diffusion probabilistic models. Advances in Neural Information Processing Systems, 2020.
  24. Hailong Hu and Jun Pang. Membership in- ference of diffusion models. arXiv preprint arXiv:2301.09956, 2023. · arXiv
  25. Daphne Ippolito, Florian Tramèr, Milad Nasr, Chiyuan Zhang, Matthew Jagielski, Katherine Lee, Christopher A Choquette-Choo, and Nicholas Car- lini. Preventing verbatim memorization in lan- guage models gives a false sense of privacy. arXiv preprint arXiv:2210.17546, 2022. · arXiv
  26. Matthew Jagielski, Jonathan Ullman, and Alina Oprea. Auditing differentially private machine learning: How private is private SGD? Advances in Neural Information Processing Systems, 2020.
  27. Ali Jahanian, Xavier Puig, Yonglong Tian, and Phillip Isola. Generative models as a data source for multiview representation learning. International Conference on Learning Representations, 2021.
  28. Bargav Jayaraman and David Evans. Are attribute inference attacks just imputation? ACM Confer- ence on Computer and Communications Security (CCS), 2022.
  29. Bargav Jayaraman, Lingxiao Wang, Katherine Knipmeyer, Quanquan Gu, and David Evans. Re- visiting membership inference under realistic as- sumptions. Proceedings on Privacy Enhancing Technologies, 2020.
  30. Nikhil Kandpal, Haikang Deng, Adam Roberts, Eric Wallace, and Colin Raffel. Large language models struggle to learn long-tail knowledge. arXiv preprint arXiv:2211.08411, 2022. · arXiv
  31. Nikhil Kandpal, Eric Wallace, and Colin Raffel. Deduplicating training data mitigates privacy risks in language models. International Conference on Machine Learning, 2022.
  32. MinGuk Kang, Joonghyuk Shin, and Jaesik Park. StudioGAN: A Taxonomy and Benchmark of GANs for Image Synthesis. arXiv preprint arXiv:2206.09479, 2022. · arXiv
  33. Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. In Advances in Neural Information Processing Sys- tems, 2020.
  34. Tero Karras, Samuli Laine, and Timo Aila. A style- based generator architecture for generative adver- sarial networks. In IEEE/CVF conference on com- puter vision and pattern recognition, 2019.
  35. Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool. RePaint: Inpainting using denoising dif- fusion probabilistic models. In IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, 2022.
  36. Vaishnavh Nagarajan, Colin Raffel, and Ian J Goodfellow. Theoretical insights into memoriza- tion in GANs. In Neural Information Processing Systems Workshop, 2018.
  37. Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International Conference on Machine Learning, 2021.
  38. Walter H. L. Pinaya, Petru-Daniel Tudosiu, Jes- sica Dafflon, Pedro F da Costa, Virginia Fer- nandez, Parashkev Nachev, Sebastien Ourselin, and M. Jorge Cardoso. Brain imaging genera- tion with latent diffusion models. arXiv preprint arXi:2209.07162, 2022.
  39. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual mod- els from natural language supervision. In Interna- tional Conference on Machine Learning, 2021.
  40. Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep convolutional generative adversarial networks. In International Conference on Learning Representa- tions, 2016.
  41. Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with CLIP latents. arXiv preprint arXiv:2204.06125, 2022. · arXiv
  42. Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International Conference on Ma- chine Learning, 2021.
  43. Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, 2022.
  44. Pouria Rouzrokh, Bardia Khosravi, Shahriar Faghani, Mana Moassefi, Sanaz Vahdati, and Bradley J. Erickson. Multitask brain tumor inpaint- ing with diffusion models: A methodological re- port. arXiv preprint arXiv:2210.12113, 2022. · arXiv
  45. Chitwan Saharia, William Chan, Saurabh Sax- ena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S Sara Mahdavi, Rapha Gontijo Lopes, et al. Photorealistic text-to-image diffusion models with deep language understanding. arXiv preprint arXiv:2205.11487, 2022. · arXiv
  46. Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Im- proved techniques for training GANs. Advances in Neural Information Processing Systems, 2016.
  47. Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference at- tacks against machine learning models. In IEEE Symposium on Security and Privacy, 2017.
  48. Jascha Sohl-Dickstein, Eric Weiss, Niru Mah- eswaranathan, and Surya Ganguli. Deep unsu- pervised learning using nonequilibrium thermody- namics. In International Conference on Machine Learning, 2015.
  49. Gowthami Somepalli, Vasu Singla, Micah Gold- blum, Jonas Geiping, and Tom Goldstein. Dif- fusion art or digital forgery? Investigating data replication in diffusion models. arXiv preprint arXiv:2212.03860, 2022. · arXiv
  50. Yang Song and Stefano Ermon. Generative mod- eling by estimating gradients of the data distribu- tion. Advances in Neural Information Processing Systems, 2019.
  51. Dávid Terjék. Adversarial lipschitz regularization. In International Conference on Learning Represen- tations, 2019.
  52. Yuan Tian, Qin Wang, Zhiwu Huang, Wen Li, Dengxin Dai, Minghao Yang, Jun Wang, and Olga Fink. Off-policy reinforcement learning for effi- cient and effective gan architecture search. In Eu- ropean Conference on Computer Vision, 2020.
  53. Patrick Tinsley, Adam Czajka, and Patrick Flynn. This face does not exist... but it might be yours! Identity leakage in generative models. In IEEE/CVF Winter Conference on Applications of Computer Vision, 2021.
  54. Florian Tramèr, Gautam Kamath, and Nicholas Carlini. Considerations for differentially private learning with large-scale public pretraining. arXiv preprint arXiv:2212.06470, 2022. · arXiv
  55. Ryan Turner, Jane Hung, Eric Frank, Yunus Saatchi, and Jason Yosinski. Metropolis-hastings generative adversarial networks. In International Conference on Machine Learning, 2019.
  56. Gerrit van den Burg and Chris Williams. On mem- orization in probabilistic deep generative models. Advances in Neural Information Processing Sys- tems, 2021.
  57. Yixin Wu, Ning Yu, Zheng Li, Michael Backes, and Yang Zhang. Membership inference attacks against text-to-image generation models. arXiv preprint arXiv:2210.00968, 2022. · arXiv
  58. Jiayuan Ye, Aadyaa Maddi, Sasi Kumar Mu- rakonda, Vincent Bindschaedler, and Reza Shokri. Enhanced membership inference attacks against machine learning models. In ACM SIGSAC Con- ference on Computer and Communications Security (CCS), 2021.
  59. Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learning: Analyzing the connection to overfitting. In IEEE Computer Security Foundations Symposium (CSF), 2018.
  60. Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxiao Wang, Bo Li, and Dawn Song. The secret re- vealer: Generative model-inversion attacks against deep neural networks. In IEEE/CVF conference on computer vision and pattern recognition, 2020.
  61. Shengyu Zhao, Zhijian Liu, Ji Lin, Jun-Yan Zhu, and Song Han. Differentiable augmentation for data-efficient GAN training. Advances in Neural Information Processing Systems, 2020.

Cited by

See all 19

Metadata

Authors
Carlini, N., Hayes, J., Nasr, M., Jagielski, M., Sehwag, V., Tramèr, F., Balle, B., Ippolito, D., & Wallace, E.
Venue
arXiv
Publisher
arXiv
Published
2023
DOI
10.48550/arxiv.2301.13188
arXiv
2301.13188
References
1
Type
Preprint
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Extracting Training Data from Diffusion Models. https://app.cochat.ai/papers/10.48550/arxiv.2301.13188
The paper
Carlini, N., Hayes, J., Nasr, M., Jagielski, M., Sehwag, V., Tramèr, F., Balle, B., Ippolito, D., & Wallace, E. (2023). Extracting Training Data from Diffusion Models. arXiv. https://doi.org/10.48550/arxiv.2301.13188