The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Deep Convolutional Framelets: A General Deep Learning Framework for Inverse Problems
Open accessAbstract
Recently, deep learning approaches with various network architectures have achieved significant performance improvement over existing iterative reconstruction methods in various imaging problems. However, it is still unclear why these deep learning architectures work for specific inverse problems. Moreover, in contrast to the usual evolution of signal processing theory around the classical theories, the link between deep learning and the classical signal processing approaches, such as wavelets, nonlocal processing, and compressed sensing, are not yet well understood. To address these issues, here we show that the long-sought missing link is the convolution framelets for representing a signal by convolving local and nonlocal bases. The convolution framelets were originally developed to generalize the theory of low-rank Hankel matrix approaches for inverse problems, and this paper further extends this idea so as to obtain a deep neural network using multilayer convolution framelets with perfect reconstruction (PR) under rectified linear unit (ReLU) nonlinearity. Our analysis also shows that the popular deep network components such as residual blocks, redundant filter channels, and concatenated ReLU (CReLU) do indeed help to achieve PR, while the pooling and unpooling layers should be augmented with high-pass branches to meet the PR condition. Moreover, by changing the number of filter channels and bias, we can control the shrinkage behaviors of the neural network. This discovery reveals the limitations of many existing deep learning architectures for inverse problems, and leads us to propose a novel theory for a deep convolutional framelet neural network. Using numerical experiments with various inverse problems, we demonstrate that our deep convolutional framelets network shows consistent improvement over existing deep architectures. This discovery suggests that the success of deep learning stems not from a magical black box, but rather from the power of a novel signal representation using a nonlocal basis combined with a data-driven local basis, which is indeed a natural extension of classical signal processing theory.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892
Donoho, D. L. (2006) · IEEE Transactions on Information Theory · doi:10.1109/tit.2006.871582 · Cited by 23,994
Daubechies, I. (1992) · doi:10.1137/1.9781611970104 · Cited by 15,979
Candes, E. J., Romberg, J., & Tao, T. (2006) · IEEE Transactions on Information Theory · doi:10.1109/tit.2005.862083 · Cited by 15,639
Image denoising by sparse 3-D transform-domain collaborative filtering
Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.901238 · Cited by 9,283
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
Zhang, K., Zuo, W., Chen, Y., Meng, D., & Zhang, L. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2662206 · Cited by 8,983
A Non-Local Algorithm for Image Denoising
Buades, A., Coll, B., & Morel, J.-M. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.38 · Cited by 7,616 · cited 1×
“Non-local image processing techniques such as non-local means [8], BM3D [16], etc have also demonstrated impressive performance for many image processing applications.”
BackgroundContext Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691
Ahmed, N., Natarajan, T., & Rao, K. R. (1974) · IEEE Transactions on Computers · doi:10.1109/t-c.1974.223784 · Cited by 5,156
Cited by
Deep Learning on Image Denoising: An overview
Tian, C., Fei, L., Zheng, W., Xu, Y., Zuo, W., & Lin, C.-W. (2020) · Neural Networks · doi:10.1016/j.neunet.2020.07.025 · Cited by 1,091
Multi-level Wavelet-CNN for Image Restoration
Liu, P., Zhang, H., Zhang, K., Lin, L., & Zuo, W. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2018.00121 · Cited by 933
Solving inverse problems using data-driven models
Arridge, S., Maass, P., Öktem, O., & Schönlieb, C.-B. (2019) · Acta Numerica · doi:10.1017/s0962492919000059 · Cited by 686
Framing U-Net via Deep Convolutional Framelets: Application to Sparse-View CT
Han, Y., & Ye, J. C. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2823768 · Cited by 588
Self-citationDeep Learning Techniques for Inverse Problems in Imaging
Ongie, G., Jalal, A., Metzler, C. A., Baraniuk, R. G., Dimakis, A. G., & Willett, R. (2020) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2020.2991563 · Cited by 577
On Interpretability of Artificial Neural Networks: A Survey
Fan, F.-L., Xiong, J., Li, M., & Wang, G. (2021) · IEEE Transactions on Radiation and Plasma Medical Sciences · doi:10.1109/trpms.2021.3066428 · Cited by 561
Deep learning for tomographic image reconstruction
Wang, G., Ye, J. C., & De Man, B. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-00273-z · Cited by 531
Self-citationA gentle introduction to deep learning in medical image processing
Maier, A., Syben, C., Lasser, T., & Riess, C. (2019) · Zeitschrift für Medizinische Physik · doi:10.1016/j.zemedi.2018.12.003 · Cited by 515
Photorealistic Style Transfer via Wavelet Transforms
Yoo, J., Uh, Y., Chun, S., Kang, B., & Ha, J.-W. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00913 · Cited by 388
Deep Magnetic Resonance Image Reconstruction: Inverse Problems Meet Neural Networks
Liang, D., Cheng, J., Ke, Z., & Ying, L. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2019.2950557 · Cited by 370
${k}$ -Space Deep Learning for Accelerated MRI
Han, Y., Sunwoo, L., & Ye, J. C. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2927101 · Cited by 344
Self-citationDeep Residual Learning for Accelerated MRI Using Magnitude and Phase Networks
Lee, D., Yoo, J., Tak, S., & Ye, J. C. (2018) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2018.2821699 · Cited by 340
Self-citationDeep Convolutional Framelet Denosing for Low-Dose CT via Wavelet Residual Network
Kang, E., Chang, W., Yoo, J., & Ye, J. C. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2823756 · Cited by 336
Self-citationMulti-Level Wavelet Convolutional Neural Networks
Liu, P., Zhang, H., Lian, W., & Zuo, W. (2019) · IEEE Access · doi:10.1109/access.2019.2921451 · Cited by 326
Deep learning electromagnetic inversion with convolutional neural networks
Puzyrev, V. (2019) · Geophysical Journal International · doi:10.1093/gji/ggz204 · Cited by 299
Image Reconstruction: From Sparsity to Data-Adaptive Methods and Machine Learning
Ravishankar, S., Ye, J. C., & Fessler, J. A. (2020) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2936204 · Cited by 294
Self-citationKnoll, F., Murrell, T., Sriram, A., Yakubova, N., Zbontar, J., Rabbat, M., Defazio, A., Muckley, M. J., Sodickson, D. K., Zitnick, C. L., et al. (2020) · Magnetic Resonance in Medicine · doi:10.1002/mrm.28338 · Cited by 263
Shallow neural networks for fluid flow reconstruction with limited sensors
Erichson, N. B., Mathelin, L., Yao, Z., Brunton, S. L., Mahoney, M. W., & Kutz, J. N. (2020) · Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rspa.2020.0097 · Cited by 259
DOLCE: A Model-Based Probabilistic Diffusion Framework for Limited-Angle CT Reconstruction
Liu, J., Anirudh, R., Thiagarajan, J. J., He, S., Mohan, K. A., Kamilov, U. S., & Kim, H. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00963 · Cited by 106
“Deep Learning for CT: A traditional DL reconstruction involves training a convolutional neural network (CNN) architecture, such as U-Net [60], to directly perform a regularized inversion of the forward model by exploiting redundancies in the training data [2, 27, 29, 37, 40, 88, 91, 92].”
Method
Metadata
- Authors
- Ye, J. C., Han, Y., & Cha, E.
- Venue
- SIAM Journal on Imaging Sciences
- Published
- 2018-01
- Volume
- 11
- Issue
- 2
- Pages
- 991-1048
- ISSN
- 1936-4954 (ISSN-L)
- OpenAlex
- W2777802649
- Cited by
- 330
- References
- 38
- FWCI
- 25.42
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.