The DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
What makes a good model of natural images?
Abstract
Many low-level vision algorithms assume a prior probability over images, and there has been great interest in trying to learn this prior from examples. Since images are very non Gaussian, high dimensional, continuous signals, learning their distribution presents a tremendous computational challenge. Perhaps the most successful recent algorithm is the Fields of Experts (FOE) [20] model which has shown impressive performance by modeling image statistics with a product of potentials defined on filter outputs. However, as in previous models of images based on filter outputs [30], calculating the probability of an image given the model requires evaluating an intractable partition function. This makes learning very slow (requires Monte-Carlo sampling at every step) and makes it virtually impossible to compare the likelihood of two different models. Given this computational difficulty, it is hard to say whether nonintu-itive features learned by such models represent a true property of natural images or an artifact of the approximations used during learning. In this paper we present (1) tractable lower and upper bounds on the partition function of models based on filter outputs and (2) efficient learning algorithms that do not require any sampling. Our results are based on recent results in machine learning that deal with Gaussian potentials. We extend these results to non-Gaussian potentials and derive a novel, basis rotation algorithm for approximating the maximum likelihood filters. Our results allow us to (1) rigorously compare the likelihood of different models and (2) calculate high likelihood models of natural image statistics in a matter of minutes. Applying our results to previous models shows that the nonintuitive features are not an artifact of the learning process but rather are capturing robust properties of natural images.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1977.tb01600.x · Cited by 50,444
Horn, B. K. P., & Schunck, B. G. (1981) · Artificial Intelligence · doi:10.1016/0004-3702(81)90024-2 · Cited by 10,312
Independent component analysis: algorithms and applications
Hyvärinen, A., & Oja, E. (2000) · Neural Networks · doi:10.1016/s0893-6080(00)00026-5 · Cited by 8,705
Horn, B. K., & Schunck, B. G. (1981) · SPIE Proceedings · doi:10.1117/12.965761 · Cited by 7,394
Emergence of simple-cell receptive field properties by learning a sparse code for natural images
Olshausen, B. A., & Field, D. J. (1996) · Nature · doi:10.1038/381607a0 · Cited by 5,694
Bayesian Learning for Neural Networks
Neal, R. M. (1996) · Lecture Notes in Statistics · doi:10.1007/978-1-4612-0745-0 · Cited by 4,276
An Introduction to Variational Methods for Graphical Models
Jordan, M. I., Ghahramani, Z., Jaakkola, T. S., & Saul, L. K. (1999) · Machine Learning · doi:10.1023/a:1007665907178 · Cited by 3,791
Image denoising using scale mixtures of gaussians in the wavelet domain
Portilla, J., Strela, V., Wainwright, M. J., & Simoncelli, E. P. (2003) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.818640 · Cited by 2,255
The “independent components” of natural scenes are edge filters
Bell, A. J., & Sejnowski, T. J. (1997) · Vision Research · doi:10.1016/s0042-6989(97)00121-1 · Cited by 2,201
Removing camera shake from a single photograph
Fergus, R., Singh, B., Hertzmann, A., Roweis, S. T., & Freeman, W. T. (2006) · ACM Transactions on Graphics · doi:10.1145/1141911.1141956 · Cited by 1,815
Self-citation
Cited by
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2662206 · Cited by 8,983
From learning models of natural image patches to whole image restoration
Zoran, D., & Weiss, Y. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126278 · Cited by 1,522
Self-citationHigh-quality motion deblurring from a single image
Shan, Q., Jia, J., & Agarwala, A. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360672 · Cited by 1,386
Image denoising: Can plain neural networks compete with BM3D?
Burger, H. C., Schuler, C. J., & Harmeling, S. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247952 · Cited by 1,325
Understanding and evaluating blind deconvolution algorithms
Levin, A., Weiss, Y., Durand, F., & Freeman, W. T. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206815 · Cited by 1,208
Self-citationBlind deconvolution using a normalized sparsity measure
Krishnan, D., Tay, T., & Fergus, R. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995521 · Cited by 1,119
Rain Streak Removal Using Layer Priors
Li, Y., Tan, R. T., Guo, X., Lu, J., & Brown, M. S. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.299 · Cited by 952
Roth, S., & Black, M. J. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-008-0197-6 · Cited by 847
(2008) · Cited by 838
Real Image Denoising With Feature Attention
Anwar, S., & Barnes, N. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00325 · Cited by 784
Discriminative learned dictionaries for local image analysis
Mairal, J., Bach, F., Ponce, J., Sapiro, G., & Zisserman, A. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587652 · Cited by 726
Statistics of real-world hyperspectral images
Chakrabarti, A., & Zickler, T. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995660 · Cited by 620
Duarte-Carvajalino, J. M., & Sapiro, G. (2009) · IEEE Transactions on Image Processing · doi:10.1109/tip.2009.2022459 · Cited by 571
Image denoising review: From classical to state-of-the-art approaches
Goyal, B., Dogra, A., Agrawal, S., Sohi, B. S., & Sharma, A. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.09.003 · Cited by 463
Learning Multiscale Sparse Representations for Image and Video Restoration
Mairal, J., Sapiro, G., & Elad, M. (2008) · Multiscale Modeling & Simulation · doi:10.1137/070697653 · Cited by 460
Edge-based blur kernel estimation using patch priors
Sun, L., Cho, S., Wang, J., & Hays, J. (2013) · IEEE International Conference on Computational Photography (ICCP) · doi:10.1109/iccphot.2013.6528301 · Cited by 440
Understanding Blind Deconvolution Algorithms
Levin, A., Weiss, Y., Durand, F., & Freeman, W. T. (2011) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2011.148 · Cited by 425
Self-citationChatterjee, P., & Milanfar, P. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2009.2037087 · Cited by 422
Patch Group Based Nonlocal Self-Similarity Prior Learning for Image Denoising
Xu, J., Zhang, L., Zuo, W., Zhang, D., & Feng, X. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.36 · Cited by 393
Internal statistics of a single natural image
Zontak, M., & Irani, M. (2011) · CVPR 2011 · doi:10.1109/cvpr.2011.5995401 · Cited by 375
Blind Image Quality Assessment via Deep Learning
Hou, W., Gao, X., Tao, D., & Li, X. (2015) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2014.2336852 · Cited by 368
Single Image Deblurring Using Motion Density Functions
Gupta, A., Joshi, N., Lawrence Zitnick, C., Cohen, M., & Curless, B. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15549-9_13 · Cited by 345
Pushing the Boundaries of View Extrapolation With Multiplane Images
Srinivasan, P. P., Tucker, R., Barron, J. T., Ramamoorthi, R., Ng, R., & Snavely, N. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00026 · Cited by 322
Learning generative texture models with extended Fields-of-Experts
Heess, N., Williams, C. K. I., & Hinton, G. E. (2009) · Procedings of the British Machine Vision Conference 2009 · doi:10.5244/c.23.115 · Cited by 275
How Well Do Filter-Based MRFs Model Natural Images?
Gao, Q., & Roth, S. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-32717-9_7 · Cited by 22
Metadata
- Authors
- Weiss, Y., & Freeman, W. T.
- Venue
- 2007 IEEE Conference on Computer Vision and Pattern Recognition
- Publisher
- IEEE
- Published
- 2007-06
- Pages
- 1-8
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W2097713019
- Cited by
- 300
- References
- 30
- FWCI
- 18.61
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.