Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 81 of the 85 references it cites were found in a registry or an index.

Image Super-Resolution Via Iterative Refinement

Open access
Saharia, Chitwan, Ho, Jonathan, Chan, William, Salimans, Tim, Fleet, David J., Norouzi, Mohammad · 2022 · IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract

We present SR3, an approach to image Super-Resolution via Repeated Refinement. SR3 adapts denoising diffusion probabilistic models (Ho et al. 2020), (Sohl-Dickstein et al. 2015) to image-to-image translation, and performs super-resolution through a stochastic iterative denoising process. Output images are initialized with pure Gaussian noise and iteratively refined using a U-Net architecture that is trained on denoising at various noise levels, conditioned on a low-resolution input image. SR3 exhibits strong performance on super-resolution tasks at different magnification factors, on faces and natural images. We conduct human evaluation on a standard 8× face super-resolution task on CelebA-HQ for which SR3 achieves a fool rate close to 50%, suggesting photo-realistic outputs, while GAN baselines do not exceed a fool rate of 34%. We evaluate SR3 on a 4× super-resolution task on ImageNet, where SR3 outperforms baselines in human evaluation and classification accuracy of a ResNet-50 classifier trained on high-resolution images. We further show the effectiveness of SR3 in cascaded image generation, where a generative model is chained with super-resolution models to synthesize high-resolution images with competitive FID scores on the class-conditional 256×256 ImageNet generation challenge.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tpami.2022.3204461

Cites

81 of 81 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig. 1 Two representative SR3 outputs: (top) 8 × face superresolution at 16 × 16 → 128 × 128 pixels (bottom) 4 × natural image super-resolution at 64 × 64 → 256 × 256 pixels.
Fig. 1 Two representative SR3 outputs: (top) 8 × face superresolution at 16 × 16 → 128 × 128 pixels (bottom) 4 × natural image super-resolution at 64 × 64 → 256 × 256 pixels.CC-BY · From the PDF
Fig. 2 The forward diffusion process q (left to right) gradually adds Gaussian noise to the target image. The reverse process p (right to left) iteratively denoises the target image, conditioned on a source image x . (Source x is not shown.)
Fig. 2 The forward diffusion process q (left to right) gradually adds Gaussian noise to the target image. The reverse process p (right to left) iteratively denoises the target image, conditioned on a source image x . (Source x is not shown.)CC-BY · From the PDF
Fig. 3 Depiction of U-Net architecture of SR3. The low resolution input image x is up-sampled to the target resolution using bicubic interpolation, and concatenated with the noisy high resolution output image y t . We show the activation dimensions for a 16 × 16 → 128 × 128 super resolution model. We perform self-attention on 16 × 16 feature maps.
Fig. 3 Depiction of U-Net architecture of SR3. The low resolution input image x is up-sampled to the target resolution using bicubic interpolation, and concatenated with the noisy high resolution output image y t . We show the activation dimensions for a 16 × 16 → 128 × 128 super resolution model. We perform self-attention on 16 × 16 feature maps.CC-BY · From the PDF
Fig. 4 Super-resolution results (64 × 64 → 256 × 256) for SR3 and Regression on ImageNet test images. Both models use the same architecture and training data. We display the full image and an enlarged patch to show fine-grained details.
Fig. 4 Super-resolution results (64 × 64 → 256 × 256) for SR3 and Regression on ImageNet test images. Both models use the same architecture and training data. We display the full image and an enlarged patch to show fine-grained details.CC-BY · From the PDF
Fig. 5 Three samples from SR3 applied to ImageNet test images (16 × 16 → 256 × 256), demonstrating SR3 diversity.
Fig. 5 Three samples from SR3 applied to ImageNet test images (16 × 16 → 256 × 256), demonstrating SR3 diversity.CC-BY · From the PDF
Fig. 7 SR3 and state-of-the-art methods on 4 × super-resolution (64 × 64 → 256 × 256) applied to ImageNet test images. The outputs of EnhanceNet and ESRGAN are sharp, but include artifacts especially when inspecting enlarged patches. We found that ESRGAN trained on ImageNet-1M produced similar artifacts. SR3 outputs seem to resemble the original images the most, but one can still find patches in the original images that contain more interesting texture than in SR3 outputs.
Fig. 7 SR3 and state-of-the-art methods on 4 × super-resolution (64 × 64 → 256 × 256) applied to ImageNet test images. The outputs of EnhanceNet and ESRGAN are sharp, but include artifacts especially when inspecting enlarged patches. We found that ESRGAN trained on ImageNet-1M produced similar artifacts. SR3 outputs seem to resemble the original images the most, but one can still find patches in the original images that contain more interesting texture than in SR3 outputs.CC-BY · From the PDF
Fig. 8 Comparison on 4 × face super-resolution (16 × 16 → 128 × 128). Reference images are removed for privacy concerns.
Fig. 8 Comparison on 4 × face super-resolution (16 × 16 → 128 × 128). Reference images are removed for privacy concerns.CC-BY · From the PDF
Fig. 9 Face super-resolution human fool rates (higher is better, for photo-realistic samples one would expect a fool rate close to 50%). Outputs of four models are compared to ground truth. (top) Task-1, subjects are shown lowresolution inputs. (bottom) Task-2, inputs are not shown.
Fig. 9 Face super-resolution human fool rates (higher is better, for photo-realistic samples one would expect a fool rate close to 50%). Outputs of four models are compared to ground truth. (top) Task-1, subjects are shown lowresolution inputs. (bottom) Task-2, inputs are not shown.CC-BY · From the PDF
Fig. 10 ImageNet super-resolution fool rates. Model outputs are compared to ground truth with pair of images shown for 6 seconds.
Fig. 10 ImageNet super-resolution fool rates. Model outputs are compared to ground truth with pair of images shown for 6 seconds.CC-BY · From the PDF
Worst Fool Rates for PULSE Input Pulse
Worst Fool Rates for PULSE Input PulseCC-BY · From the PDF
Fool Rate: 2.2%
Fool Rate: 2.2%CC-BY · From the PDF
Image 12
CC-BY · From the PDF
Best Fool Rates for PULSE Input Pulse
Best Fool Rates for PULSE Input PulseCC-BY · From the PDF
Fool Rate: 63.4%
Fool Rate: 63.4%CC-BY · From the PDF
Image 15
CC-BY · From the PDF
Fool Rate: 18.6%
Fool Rate: 18.6%CC-BY · From the PDF
Image 17
CC-BY · From the PDF
Image 18
CC-BY · From the PDF
Best Fool Rates for SR3 Input SR3
Best Fool Rates for SR3 Input SR3CC-BY · From the PDF
Fool Rate: 83.7%
Fool Rate: 83.7%CC-BY · From the PDF
Image 21
CC-BY · From the PDF
Fig. 13 Synthetic 1024 × 1024 faces, sampled from an unconditional 64 × 64 model, followed by two 4 × SR3 models.
Fig. 13 Synthetic 1024 × 1024 faces, sampled from an unconditional 64 × 64 model, followed by two 4 × SR3 models.CC-BY · From the PDF
(64 × 64)
(64 × 64)CC-BY · From the PDF
(1024 × 1024)
(1024 × 1024)CC-BY · From the PDF
Fig. 14 Cascaded generation with an unconditional model chained with two SR3 models.
Fig. 14 Cascaded generation with an unconditional model chained with two SR3 models.CC-BY · From the PDF
Fig. 15 Class-conditional 256 × 256 ImageNet samples. Each row represents samples from a specific ImageNet class, from top to bottom: Goldfish, Red Fox, Balloon, Monarch Butterfly, Church, Fire Truck. For a given label, we sample a 64 × 64 image from a class-conditional diffusion model, and then apply a 4 × SR3 model.
Fig. 15 Class-conditional 256 × 256 ImageNet samples. Each row represents samples from a specific ImageNet class, from top to bottom: Goldfish, Red Fox, Balloon, Monarch Butterfly, Church, Fire Truck. For a given label, we sample a 64 × 64 image from a class-conditional diffusion model, and then apply a 4 × SR3 model.CC-BY · From the PDF
Fig. 16 Synthetic 256 × 256 ImageNet images. We draw a label at random, sample a 64 × 64 image from the corresponding class-conditional diffusion model, and then apply a 4 × SR3 model.
Fig. 16 Synthetic 256 × 256 ImageNet images. We draw a label at random, sample a 64 × 64 image from the corresponding class-conditional diffusion model, and then apply a 4 × SR3 model.CC-BY · From the PDF
Image 28
CC-BY · From the PDF
Image 29
CC-BY · From the PDF
Image 30
CC-BY · From the PDF
Image 31
CC-BY · From the PDF
Image 32
CC-BY · From the PDF
Image 33
CC-BY · From the PDF

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 85 references

Cited by

See all 1,829

Metadata

Authors
Saharia, C., Ho, J., Chan, W., Salimans, T., Fleet, D. J., & Norouzi, M.
Affiliations
Google (United States) (US)
Venue
IEEE Transactions on Pattern Analysis and Machine Intelligence
Published
2022
Volume
45
Issue
4
Pages
1-14
ISSN
  • 0162-8828 (ISSN-L)
  • 2160-9292
  • 1939-3539
OpenAlex
W3155072588
Cited by
1,829
References
85
Citations by year
2026: 234 · 2025: 580 · 2024: 662 · 2023: 301 · 2022: 40
FWCI
129.48
Type
journal article
Language
en
Open access
Locations
  • IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image Processing Techniques
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Signal Denoising Methods
Keywords
image super-resolution, iterative refinement, denoising diffusion probabilistic models, image-to-image translation, U-Net, face super-resolution, high-resolution image generation, human evaluation, image classification accuracy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Image Super-Resolution Via Iterative Refinement. https://app.cochat.ai/papers/10.1109/tpami.2022.3204461
The paper
Saharia, C., Ho, J., Chan, W., Salimans, T., Fleet, D. J., & Norouzi, M. (2022). Image Super-Resolution Via Iterative Refinement. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/tpami.2022.3204461

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.