CoChat
Sign Up
PreprintCited by 77Generative Adversarial Networks and Image SynthesisAdvanced Image Processing TechniquesAdvanced Neural Network Applications

Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation

Open access
Sauer, Axel, Boesel, Frederic, Dockhorn, Tim, Blattmann, Andreas, Esser, Patrick, Rombach, Robin·2024·arXiv
DOI 10.48550/arxiv.2403.12015arXiv 2403.12015
Open this paper in CoChat — free accountOpen PDF

Abstract

Diffusion models are the main driver of progress in image and video synthesis, but suffer from slow inference speed. Distillation methods, like the recently introduced adversarial diffusion distillation (ADD) aim to shift the model from many-shot to single-step inference, albeit at the cost of expensive and difficult optimization due to its reliance on a fixed pretrained DINOv2 discriminator. We introduce Latent Adversarial Diffusion Distillation (LADD), a novel distillation approach overcoming the limitations of ADD. In contrast to pixel-based ADD, LADD utilizes generative features from pretrained latent diffusion models. This approach simplifies training and enhances performance, enabling high-resolution multi-aspect ratio image synthesis. We apply LADD to Stable Diffusion 3 (8B) to obtain SD3-Turbo, a fast model that matches the performance of state-of-the-art text-to-image generators using only four unguided sampling steps. Moreover, we systematically investigate its scaling behavior and demonstrate LADD's effectiveness in various applications such as image editing and inpainting.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2403.12015

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Bias amplification is a recurring issue: evaluators inherit the training data biases of the underlying LLM, leading to uneven scoring across subdomains.

    “Bias amplification is a recurring issue: evaluators inherit the training data biases of the underlying LLM, leading to uneven scoring across subdomains [21].”

    From Evaluating Metrics for Safety with LLM-as-Judges (2025) · cites this paper as [21]

    The Experiments section (p. 6) describes CLIP-score evaluation of generated images, and Appendix A (p. 17) describes human evaluators and a precaution against bias. Neither addresses LLM-based evaluators inheriting training-data biases or uneven scoring across subdomains. Searches for those terms and for bias amplification found no supporting passage. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 16 references couldn’t be matched to a known paper.

Research works printed but not linked: 30

  1. [3][3] D. Berthelot, A. Autef, J. Lin, D. A. Yap, S. Zhai, S. Hu, D. Zheng, W. Talbott, and E. Gu. Tract: Denoising diffusion models with transitive closure time-distillation. arXiv preprint arXiv:2303.04248 , 2023. 4 - · arXiv
  2. [4][4] A. Blattmann, T. Dockhorn, S. Kulal, D. Mendelevitch, M. Kilian, D. Lorenz, Y. Levi, Z. English, V. Voleti, A. Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127 , 2023. 2 - · arXiv
  3. [14][14] R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, and W. Brendel. Imagenettrained cnns are biased towards texture; increasing shape bias improves accuracy and robustness. ICLR , 2018. 5 -
  4. [15][15] J. Heek, E. Hoogeboom, and T. Salimans. Multistep consistency models. arXiv preprint arXiv:2403.06807 , 2024. 4 - · arXiv
  5. [16][16] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems , 30, 2017. 17 -
  6. [17][17] J. Ho and T. Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598 , 2022. 6, 10 - · arXiv
  7. [21][21] A. Hyvärinen and P. Dayan. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research , 6(4), 2005. 4 -
  8. [22][22] P. Jaini, K. Clark, and R. Geirhos. Intriguing properties of generative classifiers. ICLR , 2023. 5 -
  9. [23][23] X. Ju, A. Zeng, Y. Bian, S. Liu, and Q. Xu. Direct inversion: Boosting diffusion-based editing with 3 lines of code. arXiv preprint arXiv:2310.01506 , 2023. 12 - · arXiv
  10. [26][26] T. Karras, M. Aittala, T. Aila, and S. Laine. Elucidating the design space of diffusion-based generative models. Advances in Neural Information Processing Systems , 35:26565-26577, 2022. 4, 5 -
  11. [30][30] Y. Lipman, R. T. Q. Chen, H. Ben-Hamu, M. Nickel, and M. Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations , 2023. URL https://openreview.net/forum?id=PqvMRDCJT9t. 2 -
  12. [32][32] G. Luo, L. Dunlap, D. H. Park, A. Holynski, and T. Darrell. Diffusion hyperfeatures: Searching through time and space for semantic correspondence. Advances in Neural Information Processing Systems , 36, 2024. 5 -
  13. [33][33] S. Luo, Y. Tan, L. Huang, J. Li, and H. Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378 , 2023. 4, 7, 13 - · arXiv
  14. [34][34] S. Luo, Y. Tan, S. Patil, D. Gu, P. von Platen, A. Passos, L. Huang, J. Li, and H. Zhao. Lcm-lora: A universal stable-diffusion acceleration module. arXiv preprint arXiv:2311.05556 , 2023. 2, 7 - · arXiv
  15. [36][36] M. Oquab, T. Darcet, T. Moutakanni, H. Vo, M. Szafraniec, V. Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193 , 2023. 2, 5 - · arXiv
  16. [38][38] D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. Müller, J. Penna, and R. Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952 , 2023. 2, 4, 9, 10, 17 - · arXiv
  17. [40][40] R. Rafailov, A. Sharma, E. Mitchell, S. Ermon, C. D. Manning, and C. Finn. Direct Preference Optimization: Your Language Model is Secretly a Reward Model. arXiv:2305.18290 , 2023. 8 - · arXiv
  18. [43][43] C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. L. Denton, K. Ghasemipour, R. Gontijo Lopes, B. Karagol Ayan, T. Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing systems , 2022. 2, 6 -
  19. [46][46] A. Sauer, K. Chitta, J. Müller, and A. Geiger. Projected gans converge faster. Advances in Neural Information Processing Systems , 34:17480-17492, 2021. 5 -
  20. [48][48] A. Sauer, T. Karras, S. Laine, A. Geiger, and T. Aila. Stylegan-t: Unlocking the power of gans for fast large-scale text-to-image synthesis. In International conference on machine learning , pages 30105-30118. PMLR, 2023. 2, 6 -
  21. [49][49] A. Sauer, D. Lorenz, A. Blattmann, and R. Rombach. Adversarial diffusion distillation. arXiv preprint arXiv:2311.17042 , 2023. 2, 4, 6 - · arXiv
  22. [54][54] J. N. Sohl-Dickstein, E. A. Weiss, N. Maheswaranathan, and S. Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. ArXiv , abs/1503.03585, 2015. URL https: //api.semanticscholar.org/CorpusID:14888175. 2 - · arXiv
  23. [56][56] Y. Song and P. Dhariwal. Improved techniques for training consistency models. arXiv preprint arXiv:2310.14189 , 2023. 4, 7 - · arXiv
  24. [57][57] Y. Song, J. N. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole. Score-based generative modeling through stochastic differential equations. ArXiv , abs/2011.13456, 2020. URL https://api.semanticscholar.org/CorpusID:227209335. 2, 4 - · arXiv
  25. [58][58] Y. Song, P. Dhariwal, M. Chen, and I. Sutskever. Consistency models. In International conference on machine learning , 2023. 2, 4, 7, 9 -
  26. [61][61] B. Wallace, M. Dang, R. Rafailov, L. Zhou, A. Lou, S. Purushwalkam, S. Ermon, C. Xiong, S. Joty, and N. Naik. Diffusion Model Alignment Using Direct Preference Optimization. arXiv:2311.12908 , 2023. 8 - · arXiv
  27. [64][64] J. Yu, Y. Xu, J. Y . Koh, T. Luong, G. Baid, Z. Wang, V. Vasudevan, A. Ku, Y. Yang, B. K. Ayan, et al. Scaling autoregressive models for content-rich text-to-image generation. arXiv preprint arXiv:2206.10789 , 2(3):5, 2022. 6, 17 - · arXiv
  28. [65][65] K. Zhang, L. Mo, W. Chen, H. Sun, and Y. Su. Magicbrush: A manually annotated dataset for instruction-guided image editing. Advances in Neural Information Processing Systems , 36, 2024. 11 -
  29. [67][67] S. Zhang, X. Yang, Y. Feng, C. Qin, C.-C. Chen, N. Yu, Z. Chen, H. Wang, S. Savarese, S. Ermon, et al. Hive: Harnessing human feedback for instructional visual editing. arXiv preprint arXiv:2303.09618 , 2023. 11 - · arXiv
  30. [68][68] J. Zheng, M. Hu, Z. Fan, C. Wang, C. Ding, D. Tao, and T.-J. Cham. Trajectory consistency distillation. arXiv preprint arXiv:2402.19159 , 2024. 4 · arXiv

Cited by

Sorted by citations

Related

Authors
Sauer, A., Boesel, F., Dockhorn, T., Blattmann, A., Esser, P., & Rombach, R.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2024
Pages
1-11
DOI
10.48550/arxiv.2403.12015
arXiv
2403.12015
OpenAlex
W4404965447
Cited by
76
References
16
Citations by year
2026: 26 · 2025: 42 · 2024: 8
FWCI
15.69
Type
conference paper
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • SIGGRAPH Asia Conference Papers (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image Processing Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
Keywords
image generation, high-resolution imagery, latent diffusion models, adversarial training, knowledge distillation, fast inference

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review