The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.
Deep Image Prior
Open accessAbstract
Deep convolutional networks have become a popular tool for image generation and restoration. Generally, their excellent performance is imputed to their ability to learn realistic image priors from a large number of example images. In this paper, we show that, on the contrary, the structure of a generator network is sufficient to capture a great deal of low-level image statistics prior to any learning. In order to do so, we show that a randomly-initialized neural network can be used as a handcrafted prior with excellent results in standard inverse problems such as denoising, super-resolution, and inpainting. Furthermore, the same prior can be used to invert deep neural representations to diagnose them, and to restore images based on flash-no flash input pairs. Apart from its diverse applications, our approach highlights the inductive bias captured by standard generator network architectures. It also bridges the gap between two very popular families of image restoration methods: learning-based methods using deep convolutional networks and learning-free methods based on handcrafted image priors such as self-similarity (Code and supplementary material are available at https://dmitryulyanov.github.io/deep_image_prior).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
A theory for multiresolution signal decomposition: the wavelet representation
Mallat, S. G. (1989) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.192463 · Cited by 21,372
Auto-Encoding Variational Bayes
Kingma, D. P., & Welling, M. (2013) · arXiv · doi:10.48550/arxiv.1312.6114 · Cited by 20,067
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
Labaca-Castro, R. (2023) · Machine Learning under Malware Attack · doi:10.1007/978-3-658-40442-0_9 · Cited by 19,272
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
Nonlinear total variation based noise removal algorithms
Rudin, L. I., Osher, S., & Fatemi, E. (1992) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(92)90242-f · Cited by 15,704
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696
Image denoising by sparse 3-D transform-domain collaborative filtering
Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.901238 · Cited by 9,283
Accurate Image Super-Resolution Using Very Deep Convolutional Networks
Kim, J., Lee, J. K., & Lee, K. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.182 · Cited by 7,870
A Non-Local Algorithm for Image Denoising
Buades, A., Coll, B., & Morel, J.-M. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.38 · Cited by 7,616 · cited 2×
“For reference, the scores for the two popular approaches CMB3D [6] and Non-local means [4] that do not require pretraining are 31.”
MethodHe, K., Sun, J., & Tang, X. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.213 · Cited by 5,546
Learning a Deep Convolutional Network for Image Super-Resolution
Dong, C., Loy, C. C., He, K., & Tang, X. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10593-2_13 · Cited by 5,494
Single image super-resolution from transformed self-exemplars
Huang, J.-B., Singh, A., & Ahuja, N. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299156 · Cited by 3,472
On Single Image Scale-Up Using Sparse-Representations
Zeyde, R., Elad, M., & Protter, M. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-27413-8_47 · Cited by 3,352
Relations between the statistics of natural images and the response properties of cortical cells
Field, D. J. (1987) · Journal of the Optical Society of America A · doi:10.1364/josaa.4.002379 · Cited by 3,241
Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution
Lai, W.-S., Huang, J.-B., Ahuja, N., & Yang, M.-H. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.618 · Cited by 2,976
Understanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838 · cited 1×
“For instance, the authors of [33] recently showed that the same image classification network that generalizes well when trained on genuine data can also overfit when presented with random labels.”
BackgroundLow-Complexity Single-Image Super-Resolution based on Nonnegative Neighbor Embedding
Bevilacqua, M., Roumy, A., Guillemot, C., & Morel, M.-L. A. (2012) · Procedings of the British Machine Vision Conference 2012 · doi:10.5244/c.26.135 · Cited by 2,701
Image Super-Resolution via Deep Recursive Residual Network
Tai, Y., Yang, J., & Liu, X. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.298 · Cited by 2,415
Online Learning for Matrix Factorization and Sparse Coding
Mairal, J., Bach, F., Ponce, J., & Sapiro, G. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0908.0050 · Cited by 2,258
Globally and locally consistent image completion
Iizuka, S., Simo-Serra, E., & Ishikawa, H. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073659 · Cited by 2,203 · cited 4×
“We compare to a learning-based method of [15] in fig.”
ResultInfluentialLempitsky, V., Vedaldi, A., & Ulyanov, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00984 · Cited by 1,916
Self-citationSuper-resolution from a single image
Glasner, D., Bagon, S., & Irani, M. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459271 · Cited by 1,889 · cited 2×
“te PSNRs using center crops of the generated images (tables1and2). While our method is still outperformed by learning-based approaches, it does considerably better than the non-trained ones (bicubic, [19], [27]). Visually, it seems to close most of the gap between non-trained methods and state-of-the-art trained ConvN…”
MethodUnderstanding deep image representations by inverting them
Mahendran, A., & Vedaldi, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299155 · Cited by 1,827
Self-citationZeiler, M. D., Krishnan, D., Taylor, G. W., & Fergus, R. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2010.5539957 · Cited by 1,666
Image denoising: Can plain neural networks compete with BM3D?
Burger, H. C., Schuler, C. J., & Harmeling, S. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247952 · Cited by 1,325
Generating Images with Perceptual Similarity Metrics based on Deep NetworksPreprint
Dosovitskiy, A., & Brox, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.02644 · Cited by 1,276
EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis
Sajjadi, M. S. M., Scholkopf, B., & Hirsch, M. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.481 · Cited by 1,058
Digital photography with flash and no-flash image pairs
Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., & Toyama, K. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015777 · Cited by 1,056
A Theory for Multiresolution Signal Decomposition: The Wavelet Representation
Mallat, S. G. (2009) · Fundamental Papers in Wavelet Theory · doi:10.1515/9781400827268.494 · Cited by 1,018
Zero-Shot Super-Resolution Using Deep Internal Learning
Shocher, A., Cohen, N., & Irani, M. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00329 · Cited by 936
Statistics of natural images: Scaling in the woods
Ruderman, D. L., & Bialek, W. (1994) · Physical Review Letters · doi:10.1103/physrevlett.73.814 · Cited by 920
Roth, S., & Black, M. J. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-008-0197-6 · Cited by 847
Benchmarking Denoising Algorithms with Real Photographs
Plotz, T., & Roth, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.294 · Cited by 655
Learning to generate chairs with convolutional neural networks
Dosovitskiy, A., Springenberg, J. T., & Brox, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298761 · Cited by 640
Noise removal via Bayesian wavelet coring
Simoncelli, E. P., & Adelson, E. H. (2002) · Proceedings of 3rd IEEE International Conference on Image Processing · doi:10.1109/icip.1996.559512 · Cited by 612
Inverting Visual Representations with Convolutional Networks
Dosovitskiy, A., & Brox, T. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.522 · Cited by 603
Visualizing Deep Convolutional Neural Networks Using Natural Pre-images
Mahendran, A., & Vedaldi, A. (2016) · International Journal of Computer Vision · doi:10.1007/s11263-016-0911-8 · Cited by 496
Self-citationStatistics of natural images and models
Huang, J., & Mumford, D. (2003) · Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149) · doi:10.1109/cvpr.1999.786990 · Cited by 495
Non-local Color Image Denoising with Convolutional Neural Networks
Lefkimmiatis, S. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.623 · Cited by 363
Convolutional Sparse Coding for Image Super-Resolution
Gu, S., Zuo, W., Xie, Q., Meng, D., Feng, X., & Zhang, L. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.212 · Cited by 347
Fast Convolutional Sparse Coding
Bristow, H., Eriksson, A., & Lucey, S. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.57 · Cited by 329
Prior learning and Gibbs reaction-diffusion
Zhu, S. C., & Mumford, D. (1997) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.632983 · Cited by 319
FILTERS FOR COMMON RESAMPLING TASKS
Turkowski, K. (1990) · Graphics Gems · doi:10.1016/b978-0-08-050753-8.50042-5 · Cited by 296
Fast and flexible convolutional sparse coding
Heide, F., Heidrich, W., & Wetzstein, G. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299149 · Cited by 245
Compressed Sensing with Deep Image Prior and Learned RegularizationPreprint
Veen, D. V., Jalal, A., Soltanolkotabi, M., Price, E., Vishwanath, S., & Dimakis, A. G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.06438 · Cited by 215
Convolutional Neural Networks Analyzed via Convolutional Sparse CodingPreprint
PapyanVardan, RomanoYaniv, & EladMichael (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1607.08194 · Cited by 200
Digital photography with flash and no-flash image pairs
Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., & Toyama, K. (2004) · ACM SIGGRAPH 2004 Papers · doi:10.1145/1186562.1015777 · Cited by 195
Optimizing the Latent Space of Generative Networks
Bojanowski, P., Joulin, A., López-Paz, D., & Szlam, A. D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.05776 · Cited by 185
Shift-invariant sparse coding for audio classification
Grosse, R., Raina, R., Kwong, H., & Ng, A. Y. (2007) · Uncertainty in Artificial Intelligence · Cited by 168
The Robust Manifold Defense: Adversarial Training using Generative ModelsPreprint
Jalal, A., Ilyas, A., Daskalakis, C., & Dimakis, A. G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.09196 · Cited by 166
Shepard convolutional neural networks
Ren, J., Xu, L., Yan, Q., & Sun, W. (2015) · neural information processing systems · Cited by 161
Convolutional Dictionary Learning via Local Processing
Papyan, V., Romano, Y., Elad, M., & Sulam, J. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.566 · Cited by 152
Non-uniform Blind Deblurring by Reblurring
Bahat, Y., Efrat, N., & Irani, M. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.356 · Cited by 138
Shift-Invariance Sparse Coding for Audio ClassificationPreprint
Grosse, R., Raina, R., Kwong, H., & Ng, A. Y. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.5241 · Cited by 114
Inverting Convolutional Networks with Convolutional Networks.Preprint
Dosovitskiy, A., & Brox, T. (2015) · arXiv (Cornell University) · Cited by 75
The MADP Toolbox: An Open Source Library for Planning and Learning in (Multi-)Agent Systems
Oliehoek, F. A., Spaan, M. T. J., Terwijn, B., Robbel, P., & Messias, J. V. (2015) · Journal of Machine Learning Research · Cited by 70
Inverse Scale Space Theory for Inverse Problems
Scherzer, O., & Groetsch, C. (2001) · Springer eBooks · doi:10.1007/3-540-47778-0_29 · Cited by 49
Nonlinear Inverse Scale Space Methods for Total Variation Blind Deconvolution
Marquina, A. (2009) · SIAM Journal on Imaging Sciences · doi:10.1137/080724289 · Cited by 34
Statistics of Natural Images: Scaling in the Woods
Ruderman, D., & Bialek, W. (1993) · neural information processing systems · Cited by 24
Cited by
RePaint: Inpainting using Denoising Diffusion Probabilistic Models
Lugmayr, A., Danelljan, M., Romero, A., Yu, F., Timofte, R., & Van Gool, L. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01117 · Cited by 1,778
CNN-Generated Images Are Surprisingly Easy to Spot… for Now
Wang, S.-Y., Wang, O., Zhang, R., Owens, A., & Efros, A. A. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00872 · Cited by 1,210
Zhu, Y., Zabaras, N., Koutsourelakis, P.-S., & Perdikaris, P. (2019) · Journal of Computational Physics · doi:10.1016/j.jcp.2019.05.024 · Cited by 1,030
SinGAN: Learning a Generative Model From a Single Natural Image
Shaham, T. R., Dekel, T., & Michaeli, T. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00467 · Cited by 890
A survey of human-in-the-loop for machine learning
Wu, X., Xiao, L., Sun, Y., Zhang, J., Ma, T., & He, L. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.05.014 · Cited by 800
Metadata
- Authors
- Ulyanov, D., Vedaldi, A., & Lempitsky, V.
- Venue
- International Journal of Computer Vision
- Published
- 2020-03-04
- Volume
- 128
- Issue
- 7
- Pages
- 1867-1888
- ISSN
- 0920-5691 (ISSN-L)
- 1573-1405
- OpenAlex
- W2771305881
- Cited by
- 719
- References
- 63
- FWCI
- 37.97
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.