Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 4,143

The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.

Restormer: Efficient Transformer for High-Resolution Image Restoration

Open access
Zamir, Syed Waqas, Arora, Aditya, Khan, Salman, Hayat, Munawar, Khan, Fahad Shahbaz, Yang, Ming-Hsuan·2022·2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
DOI 10.1109/cvpr52688.2022.00564
Open this paper in CoChat — free account

Abstract

Since convolutional neural networks (CNNs) perform well at learning generalizable image priors from large-scale data, these models have been extensively applied to image restoration and related tasks. Recently, another class of neural architectures, Transformers, have shown significant performance gains on natural language and high-level vision tasks. While the Transformer model mitigates the shortcomings of CNNs (i.e., limited receptive field and inadaptability to input content), its computational complexity grows quadratically with the spatial resolution, therefore making it infeasible to apply to most image restoration tasks involving high-resolution images. In this work, we propose an efficient Transformer model by making several key designs in the building blocks (multi-head attention and feed-forward network) such that it can capture long-range pixel interactions, while still remaining applicable to large images. Our model, named Restoration Transformer (Restormer), achieves state-of-the-art results on several image restoration tasks, including image deraining, single-image motion deblurring, defocus deblurring (single-image and dual-pixel data), and image denoising (Gaussian grayscale/color denoising, and real image denoising). The source code and pre-trained models are available at https://github.com/swz30/Restormer.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/cvpr52688.2022.00564

Cites

16 of 16 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 16 references

Research works printed but not linked: 93

  1. 10.1109/ICCV.2019.00897 · DOI · Paper page
  2. 10.1109/CVPR.2018.00854 · DOI · Paper page
  3. karaali, 2017, Edge-based defocus blur estimation with adaptive scale selection, TIP
  4. 10.1109/CVPR42600.2020.00837 · DOI · Paper page
  5. 10.1109/CVPR.2019.00621 · DOI · Paper page
  6. 10.1109/CVPRW.2019.00275 · DOI
  7. kumar, 2021, Colorization transformer, ICLRE
  8. 10.1145/1409060.1409069 · DOI · Paper page
  9. 10.1109/CVPR42600.2020.00354 · DOI · Paper page
  10. khan, 2021, Transformers in vision: A survey, ACM Computing Surveys
  11. hoffer, 2019, Mix & match training convnets with mixed image sizes for improved accuracy speed and scale resiliency
  12. hendrycks, 2016, Gaussian error linear units (gelus)
  13. 10.1109/CVPR.2015.7299156 · DOI · Paper page
  14. franzen, 1999, Kodak Lossless True Color Image Suite
  15. 10.1109/CVPR.2017.186 · DOI · Paper page
  16. 10.1109/TIP.2017.2691802 · DOI · Paper page
  17. 10.1109/ICCV.2019.00260 · DOI · Paper page
  18. 10.1109/CVPR.2019.00397 · DOI · Paper page
  19. 10.1109/TIP.2017.2662206 · DOI · Paper page
  20. he, 2010, Single image haze removal using dark channel prior, TPAMI
  21. 10.1109/CVPR42600.2020.00281 · DOI · Paper page
  22. loshchilov, 2017, SGDR: Stochastic gradient descent with warm restarts, ICLRE
  23. 10.1016/j.neucom.2018.12.075 · DOI
  24. park, 2020, Multi-temporal recurrent neural networks for progressive non-uniform single image deblurring with incremental temporal training, ECCV
  25. 10.1109/CVPRW53098.2021.00025 · DOI
  26. 10.1109/CVPR.2017.35 · DOI · Paper page
  27. 10.1109/ICCV48922.2021.00429 · DOI
  28. sreyas, 2019, Robust and interpretable blind image denoising via bias-free convolutional neural networks, ICLRE
  29. 10.1109/ICCV.2013.121 · DOI · Paper page
  30. 10.1109/ICCV.2001.937655 · DOI · Paper page
  31. 10.1109/CVPR.2019.01250 · DOI · Paper page
  32. abdullah, 2021, Learning to reduce defocus blur by realistically modeling dual-pixel data, ICCV
  33. abuolaim, 2020, Defocus deblurring using dual-pixel data, ECCV
  34. 10.1109/ICCV.2019.00325 · DOI · Paper page
  35. 10.1109/CVPRW53098.2021.00070 · DOI
  36. anwar, 2019, A deep journey into super-resolution: A survey, ACM Computing Surveys
  37. anwar, 2020, Densely residual laplacian super-resolution, TPAMI
  38. lei ba, 2016, Layer normalization
  39. liu, 2018, Non-local recurrent network for image restoration, NeurIPS
  40. 10.1109/CVPRW.2017.151 · DOI · Paper page
  41. 10.1109/CVPR.2019.00717 · DOI · Paper page
  42. 10.1109/CVPRW.2018.00121 · DOI · Paper page
  43. 10.1109/CVPR.2019.00396 · DOI · Paper page
  44. 10.1109/CVPR46437.2021.00207 · DOI · Paper page
  45. 10.1109/ICCVW54120.2021.00210 · DOI · Paper page
  46. li, 2018, Recurrent squeeze-and-excitation context aggregation net for single image deraining, ECCV
  47. 10.1016/j.neunet.2020.07.025 · DOI · Paper page
  48. 10.1109/CVPR.2018.00853 · DOI · Paper page
  49. 10.1109/CVPR42600.2020.00366 · DOI · Paper page
  50. 10.1109/ICCV48922.2021.00264 · DOI
  51. touvron, 2021, Training data-efficient image transformers & distillation through attention, ICML
  52. ashish, 2017, Attention is all you need, NeurlPS
  53. 10.1016/j.neunet.2019.08.022 · DOI · Paper page
  54. 10.1109/ICCV.2013.241 · DOI · Paper page
  55. 10.1109/CVPR.2017.294 · DOI · Paper page
  56. radford, 2018, Improving language understanding by generative pre-training, Technical report OpenAI
  57. 10.1109/ICCV48922.2021.00231 · DOI
  58. 10.1109/CVPR46437.2021.00849 · DOI · Paper page
  59. 10.1109/CVPR.2019.00406 · DOI · Paper page
  60. rim, 2020, Real-world blur dataset for learning and benchmarking deblurring algorithms, ECCV
  61. 10.1109/CVPR.2015.7298665 · DOI · Paper page
  62. 10.1109/CVPRW.2019.00273 · DOI
  63. wenzhe, 2016, Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network, CVPR
  64. 10.1109/CVPR.2018.00182 · DOI · Paper page
  65. 10.1109/CVPR.2019.00613 · DOI · Paper page
  66. zhang, 2020, Residual dense network for image restoration, TPAMI
  67. 10.1109/CVPR46437.2021.01458 · DOI · Paper page
  68. zhang, 2019, Residual non-local attention networks for image restoration, ICLRE
  69. waqas zamir, 2020, Learning enriched features for real image restoration and enhancement, ECCV
  70. zhang, 2018, Image super-resolution using very deep residual channel attention networks, ECCV
  71. waqas zamir, 2020, CycleISP: Real image restoration via improved data synthesis, CVPR
  72. zhang, 2011, Color demosaicking by local directional interpolation and nonlocal adaptive thresholding, JEI
  73. yue, 2020, Dual adversarial network: Toward real-world noise removal and noise generation, ECCV
  74. 10.1109/TIP.2018.2839891 · DOI · Paper page
  75. jiawei, 2018, Dynamic scene deblurring using spatially variant recurrent neural networks, CVPR
  76. 10.1109/TPAMI.2021.3088914 · DOI · Paper page
  77. zhang, 2019, Image de-raining using a conditional generative adversarial network, TCSVT
  78. chang, 2020, Spatial-adaptive network for single image denoising, ECCV
  79. 10.1109/ICCV48922.2021.00460 · DOI · Paper page
  80. 10.1109/TIP.2007.901238 · DOI · Paper page
  81. yann, 2017, Language modeling with gated convolutional networks, ICML
  82. 10.1109/WACV45572.2020.9093586 · DOI
  83. 10.1109/CVPR.2019.00400 · DOI · Paper page
  84. 10.1109/CVPR52688.2022.00567 · DOI
  85. xu, 2013, Unnatural 10 sparse representation for natural image deblurring, CVPR
  86. fedus, 2021, Switch Transformers Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
  87. enze, 2021, Segformer Simple and efficient design for semantic segmentation with transformers
  88. wang, 2021, Uformer A general u-shaped transformer for image restoration
  89. yue, 2019, Variational denoising network: Toward blind noise modeling and removal, NeurlPS
  90. 10.1109/CVPR42600.2020.00583 · DOI · Paper page
  91. wenhan, 2017, Deep joint rain detection and removal from a single image, CVPR
  92. 10.1109/CVPR.2019.00860 · DOI · Paper page
  93. li, 2021, Tokens-to-Token ViT Training Vision Transformers from Scratch on ImageNet [J]

Cited by

See all 4,143

Metadata

Authors
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., & Yang, M.-H.
Venue
2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Publisher
IEEE
Published
2022-06
Pages
5718-5729
ISSN
1063-6919 (ISSN-L)
DOI
10.1109/cvpr52688.2022.00564
OpenAlex
W4225672218
Cited by
4,143
References
16
FWCI
303.73
Type
conference paper
Language
en
Open access
green
Licence

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Restormer: Efficient Transformer for High-Resolution Image Restoration. https://app.cochat.ai/papers/10.1109/cvpr52688.2022.00564
The paper
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., & Yang, M.-H. (2022). Restormer: Efficient Transformer for High-Resolution Image Restoration. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/cvpr52688.2022.00564

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.