The DOI matches its Crossref record; 46 of the 46 references it cites were found in a registry or an index.
Using Deep Neural Networks for Inverse Problems in Imaging: Beyond Analytical Methods
Abstract
Traditionally, analytical methods have been used to solve imaging problems such as image restoration, inpainting, and superresolution (SR). In recent years, the fields of machine and deep learning have gained a lot of momentum in solving such imaging problems, often surpassing the performance provided by analytical approaches. Unlike analytical methods for which the problem is explicitly defined and domain-knowledge carefully engineered into the solution, deep neural networks (DNNs) do not benefit from such prior knowledge and instead make use of large data sets to learn the unknown solution to the inverse problem. In this article, we review deep-learning techniques for solving such inverse problems in imaging. More specifically, we review the popular neural network architectures used for imaging tasks, offering some insight as to how these deep-learning tools can solve the inverse problem. Furthermore, we address some fundamental questions, such as how deeplearning and analytical methods can be combined to provide better solutions to the inverse problem in addition to providing a discussion on the current limitations and future directions of the use of deep learning for solving inverse problem in imaging.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.123 · Cited by 19,741
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
(2010) · International Conference on Machine Learning · Cited by 13,995
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283
Image Super-Resolution Using Deep Convolutional Networks
Dong, C., Loy, C. C., He, K., & Tang, X. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2439281 · Cited by 10,105
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2662206 · Cited by 8,983
Accurate Image Super-Resolution Using Very Deep Convolutional Networks
Kim, J., Lee, J. K., & Lee, K. M. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.182 · Cited by 7,870
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691
Flownet: Learning optical flow with convolutional networks
Dosovitskiy, A., Fischer, P., Ilg, E., Hausser, P., Hazirbas, C., Golkov, V., Smagt, P. V. D., Cremers, D., & Brox, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.316 · Cited by 4,313
(2010) · Journal of Machine Learning Research · doi:10.5555/1756006.1953039 · Cited by 4,051
Deep Convolutional Neural Network for Inverse Problems in Imaging
Jin, K. H., McCann, M. T., Froustey, E., & Unser, M. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2713099 · Cited by 2,529 · cited 3×
“, image denoising [30], image inpainting [31], optical flow [32], and computed tomography reconstruction [33].”
BackgroundMethodLearning Fast Approximations of Sparse Coding
Gregor, K., & LeCun, Y. (2010) · International Conference on Machine Learning · Cited by 2,400
Learning Deep CNN Denoiser Prior for Image Restoration
Zhang, K., Zuo, W., Gu, S., & Zhang, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.300 · Cited by 2,116 · cited 1×
“[52] (2017) Denoising, inpainting, and SR Ten-layer encoder-decoder CNN for projection operator in the alternating direction method of multipliers (ADMM) method; residual blocks; adversarial training”
MethodHornik, K., Stinchcombe, M., & White, H. (1990) · Neural Networks · doi:10.1016/0893-6080(90)90005-6 · Cited by 2,111
Chen, Y., & Pock, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2596743 · Cited by 1,344
Image denoising: Can plain neural networks compete with BM3D?
Burger, H. C., Schuler, C. J., & Harmeling, S. (2012) · 2012 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2012.6247952 · Cited by 1,325
Image Denoising and Inpainting with Deep Neural Networks
Xie, J., Xu, L., & Chen, E. (2012) · Cited by 1,237
Fast Image Recovery Using Variable Splitting and Constrained Optimization
Afonso, M. V., Bioucas-Dias, J. M., & Figueiredo, M. A. T. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2047910 · Cited by 1,223
EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesis
Sajjadi, M. S. M., Scholkopf, B., & Hirsch, M. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.481 · Cited by 1,058
Deep Convolutional Neural Network for Image Deconvolution
Li, X., Ren, J., Liu, C., & Jia, J. (2014) · neural information processing systems · Cited by 1,044
Deep ADMM-Net for compressive sensing MRI
Yang, Y., Sun, J. Z., Li, H., & Xu, Z. (2016) · neural information processing systems · Cited by 1,024
Energy-based Generative Adversarial Network
Zhao, J., Mathieu, M., & LeCun, Y. (2016) · arXiv · doi:10.48550/arxiv.1609.03126 · Cited by 912 · cited 2×
“However, the core idea of representation learning with autoencoders is still used today to learn relevant statistics of images for inverse problems and can be a central component in some of the generative models described in the section “Recent Developments: Increasing the Perceptual Quality of Images Predicted by Neu…”
Background(2008) · Cited by 838
Deep Networks for Image Super-Resolution with Sparse Prior
Wang, Z., Liu, D., Yang, J., Han, W., & Huang, T. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.50 · Cited by 784
Video Super-Resolution With Convolutional Neural Networks
Kappeler, A., Yoo, S., Dai, Q., & Katsaggelos, A. K. (2016) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2016.2532323 · Cited by 647
Self-citationSchuler, C. J., Hirsch, M., Harmeling, S., & Schölkopf, B. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2481418 · Cited by 560
Restoring an Image Taken through a Window Covered with Dirt or Rain
Eigen, D., Krishnan, D., & Fergus, R. (2013) · 2013 IEEE International Conference on Computer Vision · doi:10.1109/iccv.2013.84 · Cited by 454
Deep Unfolding: Model-Based Inspiration of Novel Deep Architectures
Hershey, J. R., Roux, J. L., & Weninger, F. (2014) · arXiv · doi:10.48550/arxiv.1409.2574 · Cited by 412 · cited 3×
“LISTA [47] represents a simple case of an unfolding algorithm [48] that aims to combine the advantages of both analytical approaches and neural networks.”
BackgroundMethodDR2-Net: Deep Residual Reconstruction Network for image compressive sensing
Yao, H., Dai, F., Zhang, S., Zhang, Y., Tian, Q., & Xu, C. (2019) · Neurocomputing · doi:10.1016/j.neucom.2019.05.006 · Cited by 356
One Network to Solve Them All — Solving Linear Inverse Problems Using Deep Projection Models
Chang, J. H. R., Li, C.-L., Poczos, B., Vijaya Kumar, B. V. K., & Sankaranarayanan, A. C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.627 · Cited by 327
A Machine Learning Approach for Non-blind Image Deconvolution
Schuler, C. J., Burger, H. C., Harmeling, S., & Scholkopf, B. (2013) · 2013 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2013.142 · Cited by 319
Amortised MAP Inference for Image Super-resolution
Sønderby, C. K., Caballero, J., Theis, L., Shi, W., & Huszár, F. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.04490 · Cited by 300
Compressed Sensing using Generative Models
Bora, A., Jalal, A., Price, E., & Dimakis, A. G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.03208 · Cited by 269
Deep Network Cascade for Image Super-resolution
Cui, Z., Chang, H., Shan, S., Zhong, B., & Chen, X. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10602-1_4 · Cited by 255
The Essential Guide to Image Processing
(2007) · Journal of Electronic Imaging · doi:10.1117/1.3430078 · Cited by 246
Convolutional Neural Networks for Direct Text Deblurring
Hradiš, M., Kotera, J., Zemčík, P., & Šroubek, F. (2015) · Procedings of the British Machine Vision Conference 2015 · doi:10.5244/c.29.6 · Cited by 237
Coupled Deep Autoencoder for Single Image Super-Resolution
Zeng, K., Yu, J., Wang, R., Li, C., & Tao, D. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2501373 · Cited by 214
Machine Learning Refined: Foundations, Algorithms, and Applications
Watt, J. M., Borhani, R., & Katsaggelos, A. K. (2016) · Cambridge University Press eBooks · doi:10.1017/cbo9781316402276 · Cited by 120
Self-citationDeepBinaryMask: Learning a binary mask for video compressive sensing
Iliadis, M., Spinoulas, L., & Katsaggelos, A. K. (2019) · Digital Signal Processing · doi:10.1016/j.dsp.2019.102591 · Cited by 98
Self-citationMao, X.-J., Shen, C., & Yang, Y.-B. (2016) · arXiv (Cornell University) · Cited by 54
Image Denoising using a Neural Network Based Non-Linear Filter in Wavelet Domain
Zhang, S., & Salari, E. (2006) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2005.1415573 · Cited by 36
Variational Bayesian Methods For Multimedia Problems
Chen, Z., Babacan, S. D., Molina, R., & Katsaggelos, A. K. (2014) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2014.2307692 · Cited by 29
Self-citation
Cited by
An overview of deep learning in medical imaging focusing on MRI
Lundervold, A. S., & Lundervold, A. (2019) · Zeitschrift für Medizinische Physik · doi:10.1016/j.zemedi.2018.11.002 · Cited by 2,058
Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processing
Monga, V., Li, Y., & Eldar, Y. C. (2021) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.3016905 · Cited by 1,414
Deep Learning on Image Denoising: An overview
Tian, C., Fei, L., Zheng, W., Xu, Y., Zuo, W., & Lin, C.-W. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.07.025 · Cited by 1,091
On the use of deep learning for computational imaging
Barbastathis, G., Ozcan, A., & Situ, G. (2019) · Optica · doi:10.1364/optica.6.000921 · Cited by 869
On instabilities of deep learning in image reconstruction and the potential costs of AI
Antun, V., Renna, F., Poon, C., Adcock, B., & Hansen, A. C. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1907377117 · Cited by 779
Metadata
- Authors
- Lucas, A., Iliadis, M., Molina, R., & Katsaggelos, A. K.
- Venue
- IEEE Signal Processing Magazine
- Published
- 2018-01
- Volume
- 35
- Issue
- 1
- Pages
- 20-36
- ISSN
- 1053-5888 (ISSN-L)
- 1558-0792
- OpenAlex
- W2782977076
- Cited by
- 586
- References
- 46
- FWCI
- 37.01
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.