Cited by 3,684Generative Adversarial Networks and Image SynthesisAdvanced Neural Network ApplicationsImage and Signal Denoising Methods

The DOI matches its Crossref record; 61 of the 61 references it cites were found in a registry or an index; 1 of the 5 sources its claims cite were checked for support.

Scalable Diffusion Models with Transformers

Open access
Peebles, William, Xie, Saining·2023·2023 IEEE/CVF International Conference on Computer Vision (ICCV)

Abstract

We explore a new class of diffusion models based on the transformer architecture. We train latent diffusion models of images, replacing the commonly-used U-Net backbone with a transformer that operates on latent patches. We analyze the scalability of our Diffusion Transformers (DiTs) through the lens of forward pass complexity as measured by Gflops. We find that DiTs with higher Gflops—through increased transformer depth/width or increased number of input tokens—consistently have lower FID. In addition to possessing good scalability properties, our largest DiT-XL/2 models outperform all prior diffusion models on the class-conditional ImageNet 512×512 and 256×256 benchmarks, achieving a state-of-the-art FID of 2.27 on the latter.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

Diffusion Transformers (DiTs) replace the standard U-Net backbone in latent diffusion models with a transformer operating on latent patches. DiT-XL/2 achieves state-of-the-art class-conditional ImageNet generation, reaching FID 2.27 at 256×256 and 3.04 at 512×512.

Key findings

  • DiT block conditioning matters: adaLN-Zero outperforms cross-attention and in-context conditioning at all training stages while being the most compute-efficient; at 400K iterations its FID is nearly half that of the in-context model.
  • Scaling DiT Gflops improves sample quality: across 12 models, transformer Gflops and FID-50K at 400K steps have a correlation of −0.93, with gains from increasing depth/width or number of input tokens.
  • DiT-XL/2 sets state of the art: FID 2.27 on class-conditional ImageNet 256×256 with classifier-free guidance cfg=1.50, improving on LDM’s previous best 3.60; at 512×512 it reaches FID 3.04 versus ADM’s 3.85.
  • Larger DiT models are more compute-efficient, and extra sampling compute cannot compensate for smaller models: DiT-L/2 with 1000 sampling steps uses 80.7 Tflops and gets FID-10K 25.9, while DiT-XL/2 with 128 steps uses 15.2 Tflops and gets FID-10K 23.7.

Methods

  • Design: latent diffusion with a frozen off-the-shelf Stable Diffusion VAE (downsample factor 8); DiT patchifies latents, applies ViT-style positional embeddings, processes tokens through N transformer blocks, and linearly decodes to noise and covariance predictions. Design space includes patch sizes p=2,4,8; block variants in-context, cross-attention, adaLN, and adaLN-Zero; model sizes DiT-S/B/L/XL.
  • Data and training: class-conditional ImageNet at 256×256 and 512×512; AdamW, constant learning rate 1×10⁻⁴, batch size 256, horizontal flips, EMA decay 0.9999, no learning-rate warmup or regularization; diffusion hyperparameters retained from ADM; implemented in JAX on TPU-v3 pods.
  • Evaluation: FID-50K with 250 DDPM sampling steps using ADM’s TensorFlow evaluation suite; 12 DiT models swept over S/B/L/XL and p=8/4/2 at 400K steps; DiT-XL/2 trained for 7M steps at 256×256 and 3M steps at 512×512; additional comparisons of conditioning strategies and model versus sampling compute.

Limitations

  • The paper does not state an explicit limitations section.
  • Evident from the text: experiments are limited to class-conditional ImageNet at 256×256 and 512×512; text-to-image and other domains are left to future work.
  • Evident from the text: DiTs use a frozen off-the-shelf VAE, so generation quality depends on that VAE; training hyperparameters were retained from ADM and not tuned; pixel-space DiT is mentioned as possible but not tested.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/iccv51070.2023.00387

Cites

61 of 61 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

28 Own result · 5 Own interpretation · 4 No basis given · 3 Sources not found · 2 Own method · 1 Partly supported · 1 Not checked yet · accuracy not yet measured

Show all 21 claims

  • The quote only partly states this claim. 4
  • Probably not a substantial claim. 6
  • It may state two separate claims as one. 18
23 of 23
  • The paper relies on this without support.

    FID-50K values reported by prior work are directly comparable to the FID-50K values computed in this paper.

    “We show samples from the model in Figures 1, and we compare against state-of-the-art class-conditional generative models. We report results in Table 2.”
1 of 1

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 59 references

Research works printed but not linked: 3

  1. Chen, Generative pre-training from pixels
  2. Dosovitskiy, An image is worth 16x16 words: Transformers for image recognition at scale
  3. Goodfellow, Generative adversarial nets

Cited by

Sorted by citations

Items 601–700 of 3,684

Metadata

Authors
Peebles, W., & Xie, S.
Affiliations
  • University of California, Berkeley (US)
  • New York University (US)
Venue
2023 IEEE/CVF International Conference on Computer Vision (ICCV)
Publisher
IEEE
Published
2023-10-01
Pages
4172-4182
OpenAlex
W4390872297
Cited by
2,284
References
59
Citations by year
2026: 1007 · 2025: 925 · 2024: 315 · 2023: 37
FWCI
244.26
Type
conference paper
Language
en
Open access
Locations
  • IEEE/CVF International Conference on Computer Vision (ICCV) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Signal Denoising Methods
Keywords
diffusion models, latent diffusion models, diffusion transformer, scalability, image generation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-11-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-11-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review