The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
CycleISP: Real Image Restoration via Improved Data Synthesis
Open accessAbstract
The availability of large-scale datasets has helped unleash the true potential of deep convolutional neural networks (CNNs). However, for the single-image denoising problem, capturing a real dataset is an unacceptably expensive and cumbersome procedure. Consequently, image denoising algorithms are mostly developed and evaluated on synthetic data that is usually generated with a widespread assumption of additive white Gaussian noise (AWGN). While the CNNs achieve impressive results on these synthetic datasets, they do not perform well when applied on real camera images, as reported in recent benchmark datasets. This is mainly because the AWGN is not adequate for modeling the real camera noise which is signal-dependent and heavily transformed by the camera imaging pipeline. In this paper, we present a framework that models camera imaging pipeline in forward and reverse directions. It allows us to produce any number of realistic image pairs for denoising both in RAW and sRGB spaces. By training a new image denoising network on realistic synthetic data, we achieve the state-of-the-art performance on real camera benchmark datasets. The parameters in our models are ~5 times lesser than the previous best method for RAW denoising. Furthermore, we demonstrate that the proposed framework generalizes beyond image denoising problem e.g., for color matching in stereoscopic cinema. The source code and pre-trained models are available at https://github.com/swz30/CycleISP.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Microsoft COCO: Common Objects in Context
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., & Zitnick, C. L. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_48 · Cited by 45,460
Squeeze-and-Excitation Networks
Hu, J., Shen, L., & Sun, G. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00745 · Cited by 35,667
CBAM: Convolutional Block Attention Module
Woo, S., Park, J., Lee, J.-Y., & Kweon, I. S. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_1 · Cited by 25,853
Nonlinear total variation based noise removal algorithms
Rudin, L. I., Osher, S., & Fatemi, E. (1992) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(92)90242-f · Cited by 15,704
Scale-space and edge detection using anisotropic diffusion
Perona, P., & Malik, J. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.56205 · Cited by 12,119
De-noising by soft-thresholding
Donoho, D. L. (1995) · IEEE Transactions on Information Theory · doi:10.1109/18.382009 · Cited by 9,619
$rm K$-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation
Aharon, M., Elad, M., & Bruckstein, A. (2006) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2006.881199 · Cited by 9,451
Image denoising by sparse 3-D transform-domain collaborative filtering
Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.901238 · Cited by 9,283
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2662206 · Cited by 8,983
Bilateral Filtering for Gray and Color Images
Tomasi, C., & Manduchi, R. (2002) · Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271) · doi:10.1109/iccv.1998.710815 · Cited by 8,177 · cited 1×
“(2) Averaging neighborhood values: in all directions using Gaussian kernel, in all directions only if pixels have similar values [56, 58] and along contours [42, 51].”
MethodA Non-Local Algorithm for Image Denoising
Buades, A., Coll, B., & Morel, J.-M. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.38 · Cited by 7,616 · cited 3×
“[8] makes significant advances in denoising.”
BackgroundMethodShi, W., Caballero, J., Huszar, F., Totz, J., Aitken, A. P., Bishop, R., Rueckert, D., & Wang, Z. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.207 · Cited by 7,570
Image Super-Resolution Using Very Deep Residual Channel Attention Networks
Zhang, Y., Li, K., Li, K., Wang, L., Zhong, B., & Fu, Y. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_18 · Cited by 5,708
SUSAN—a new approach to low level image processing
Smith, S. M., & Brady, J. M. (1997) · International Journal of Computer Vision · doi:10.1023/a:1007963824710 · Cited by 3,333 · cited 1×
“(2) Averaging neighborhood values: in all directions using Gaussian kernel, in all directions only if pixels have similar values [56, 58] and along contours [42, 51].”
Texture synthesis by non-parametric sampling
Efros, A. A., & Leung, T. K. (1999) · Proceedings of the Seventh IEEE International Conference on Computer Vision · doi:10.1109/iccv.1999.790383 · Cited by 3,023
Reinhard, E., Adhikhmin, M., Gooch, B., & Shirley, P. (2001) · IEEE Computer Graphics and Applications · doi:10.1109/38.946629 · Cited by 2,937
FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising
Zhang, K., Zuo, W., & Zhang, L. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2839891 · Cited by 2,678
Weighted Nuclear Norm Minimization with Application to Image Denoising
Gu, S., Zhang, L., Zuo, W., & Feng, X. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.366 · Cited by 2,316
From learning models of natural image patches to whole image restoration
Zoran, D., & Weiss, Y. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126278 · Cited by 1,522
Nonlocally Centralized Sparse Representation for Image Restoration
Dong, W., Zhang, L., Shi, G., & Li, X. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2235847 · Cited by 1,505
Chen, C., Chen, Q., Xu, J., & Koltun, V. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00347 · Cited by 1,376
Image denoising: Can plain neural networks compete with BM3D?
Burger, H. C., Schuler, C. J., & Harmeling, S. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247952 · Cited by 1,325
Toward Convolutional Blind Denoising of Real Photographs
Guo, S., Yan, Z., Zhang, K., Zuo, W., & Zhang, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00181 · Cited by 1,221
Progressive Image Deraining Networks: A Better and Simpler Baseline
Ren, D., Zuo, W., Hu, Q., Zhu, P., & Meng, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00406 · Cited by 1,000
Practical Poissonian-Gaussian Noise Modeling and Fitting for Single-Image Raw-Data
Foi, A., Trimeche, M., Katkovnik, V., & Egiazarian, K. (2008) · IEEE Transactions on Image Processing · doi:10.1109/tip.2008.2001399 · Cited by 928
FFDNet: Toward a fast and flexible solution for CNN-Based image denoising
Zhang, K., Zuo, W., & Zhang, L. (2018) · TUScholarShare (Temple University) · Cited by 916
A High-Quality Denoising Dataset for Smartphone Cameras
Abdelhamed, A., Lin, S., & Brown, M. S. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00182 · Cited by 867
Roth, S., & Black, M. J. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-008-0197-6 · Cited by 847
Real Image Denoising With Feature Attention
Anwar, S., & Barnes, N. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00325 · Cited by 784
Benchmarking Denoising Algorithms with Real Photographs
Plotz, T., & Roth, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.294 · Cited by 655
Noise removal via Bayesian wavelet coring
Simoncelli, E. P., & Adelson, E. H. (2002) · Proceedings of 3rd IEEE International Conference on Image Processing · doi:10.1109/icip.1996.559512 · Cited by 612
HDR image reconstruction from a single exposure using deep CNNs
Eilertsen, G., Kronander, J., Denes, G., Mantiuk, R. K., & Unger, J. (2017) · ACM Transactions on Graphics · doi:10.1145/3130800.3130816 · Cited by 598
Learning photographic global tonal adjustment with a database of input/output image pairs
Bychkovsky, V., Paris, S., Chan, E., & Durand, F. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995413 · Cited by 536
Learning photographic global tonal adjustment with a database of input / output image pairs
Bychkovsky, V., Paris, S., Chan, E., & Durand, F. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995332 · Cited by 529
Deep joint demosaicking and denoising
Gharbi, M., Chaurasia, G., Paris, S., & Durand, F. (2016) · ACM Transactions on Graphics · doi:10.1145/2980179.2982399 · Cited by 473
Unprocessing Images for Learned Raw Denoising
Brooks, T., Mildenhall, B., Xue, T., Chen, J., Sharlet, D., & Barron, J. T. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01129 · Cited by 473
A Guide to Convolutional Neural Networks for Computer Vision
Khan, S. H., Rahmani, H., Shah, S. A. A., & Bennamoun, M. (2018) · Synthesis lectures on computer vision · doi:10.2200/s00822ed1v01y201712cov015 · Cited by 443
Self-citationChatterjee, P., & Milanfar, P. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2009.2037087 · Cited by 422
Automated colour grading using colour distribution transfer
Pitié, F., Kokaram, A. C., & Dahyot, R. (2007) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2006.11.011 · Cited by 395
Color image processing pipeline
Ramanath, R., Snyder, W. E., Yoo, Y., & Drew, M. S. (2005) · IEEE Signal Processing Magazine · doi:10.1109/msp.2005.1407713 · Cited by 369
Optimal Inversion of the Generalized Anscombe Transformation for Poisson-Gaussian Noise
Makitalo, M., & Foi, A. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2012.2202675 · Cited by 363
New trends and ideas in visual concept detection: the MIR flickr retrieval evaluation initiative
Huiskes, M. J., Thomee, B., & Lew, M. S. (2010) · Proceedings of the international conference on Multimedia information retrieval · doi:10.1145/1743384.1743475 · Cited by 336
On learning optimized reaction diffusion processes for effective image restoration
Chen, Y., Yu, W., & Pock, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299163 · Cited by 323
Multi-channel Weighted Nuclear Norm Minimization for Real Color Image Denoising
Xu, J., Zhang, L., Zhang, D., & Feng, X. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.125 · Cited by 311
A Trilateral Weighted Sparse Coding Scheme for Real-World Image Denoising
Xu, J., Zhang, L., & Zhang, D. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01237-3_2 · Cited by 278
DeepISP: Toward Learning an End-to-End Image Processing Pipeline
Schwartz, E., Giryes, R., & Bronstein, A. M. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2872858 · Cited by 265
Natural image denoising: Optimality and inherent bounds
Levin, A., & Nadler, B. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995309 · Cited by 235
Talebi, H., & Milanfar, P. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2293425 · Cited by 200
Secrets of image denoising cuisine
Lebrun, M., Colom, M., Buades, A., & Morel, J. M. (2012) · Acta Numerica · doi:10.1017/s0962492912000062 · Cited by 198
Multi-scale Single Image Dehazing Using Perceptual Pyramid Deep Network
Zhang, H., Sindagi, V., & Patel, V. M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00135 · Cited by 179
The Noise Clinic: a Blind Image Denoising Algorithm
Lebrun, M., Colom, M., & Morel, J.-M. (2015) · Image Processing On Line · doi:10.5201/ipol.2015.125 · Cited by 176
A Software Platform for Manipulating the Camera Imaging Pipeline
Karaimer, H. C., & Brown, M. S. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46448-0_26 · Cited by 166
Clipped noisy images: Heteroskedastic modeling and practical denoising
Foi, A. (2009) · Signal Processing · doi:10.1016/j.sigpro.2009.04.035 · Cited by 149
NTIRE 2019 Challenge on Real Image Denoising: Methods and Results
Abdelhamed, A., Timofte, R., Brown, M. S., Yu, S., Park, B., Jeong, J., Jung, S., Kim, D.-W., Chung, J. R., Liu, J., et al. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00273 · Cited by 121
Self-citationLearning Raw Image Denoising With Bayer Pattern Unification and Bayer Preserving Augmentation
Liu, J., Wu, C.-H., Wang, Y., Xu, Q., Zhou, Y., Huang, H., Wang, C., Cai, S., Ding, Y., Fan, H., et al. (2019) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2019.00259 · Cited by 83
Learning digital camera pipeline for extreme low-light imaging
Zamir, S. W., Arora, A., Khan, S., Khan, F. S., & Shao, L. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.04.076 · Cited by 41
Self-citationColor Matching Images With Unknown Non-Linear Encodings
Rodríguez, R. G., Vázquez-Corral, J., & Bertalmı́o, M. (2020) · IEEE Transactions on Image Processing · doi:10.1109/tip.2020.2968766 · Cited by 17
Ghimpeţeanu, G., Batard, T., Seybold, T., et al. (2016) · Electronic Imaging · doi:10.2352/issn.2470-1173.2016.18.dpmi-015 · Cited by 7
Cited by
Restormer: Efficient Transformer for High-Resolution Image Restoration
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., & Yang, M.-H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.00564 · Cited by 4,143
Self-citationUformer: A General U-Shaped Transformer for Image Restoration
Wang, Z., Cun, X., Bao, J., Zhou, W., Liu, J., & Li, H. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01716 · Cited by 2,274
Multi-Stage Progressive Image Restoration
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., & Shao, L. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01458 · Cited by 2,095
Self-citationPlug-and-Play Image Restoration With Deep Denoiser Prior
Zhang, K., Li, Y., Zuo, W., Zhang, L., Van Gool, L., & Timofte, R. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3088914 · Cited by 1,113
Learning Enriched Features for Real Image Restoration and Enhancement
Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., & Shao, L. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58595-2_30 · Cited by 946
Self-citation
Metadata
- Authors
- Zamir, S. W., Arora, A., Khan, S., Hayat, M., Khan, F. S., Yang, M.-H., & Shao, L.
- Venue
- 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2020-06
- Pages
- 2693-2702
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W3034504121
- Cited by
- 393
- References
- 60
- FWCI
- 18.09
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.