Plug-and-Play priors for model based reconstruction
The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
Model-based reconstruction is a powerful framework for solving a variety of inverse problems in imaging. In recent years, enormous progress has been made in the problem of denoising, a special case of an inverse problem where the forward model is an identity operator. Similarly, great progress has been made in improving model-based inversion when the forward model corresponds to complex physical measurements in applications such as X-ray CT, electron-microscopy, MRI, and ultrasound, to name just a few. However, combining state-of-the-art denoising algorithms (i.e., prior models) with state-of-the-art inversion methods (i.e., forward models) has been a challenge for many reasons. In this paper, we propose a flexible framework that allows state-of-the-art forward models of imaging systems to be matched with state-of-the-art priors or denoising models. This framework, which we term as Plug-and-Play priors, has the advantage that it dramatically simplifies software integration, and moreover, it allows state-of-the-art denoising methods that have no known formulation as an optimization problem to be used. We demonstrate with some simple examples how Plug-and-Play priors can be used to mix and match a wide variety of existing denoising models with a tomographic forward model, thus greatly expanding the range of possible problem solutions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
A Threshold Selection Method from Gray-Level Histograms
Otsu, N. (1979) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1979.4310076 · Cited by 43,398 · cited 1×
“In order to initialize the class labels b we use Otsu’s method [30] on a low pass filtered version of the noisy input y.”
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075
Nonlinear total variation based noise removal algorithms
Rudin, L. I., Osher, S., & Fatemi, E. (1992) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(92)90242-f · Cited by 15,704 · cited 1×
“We compare reconstructions using the Plug-andPlay priors framework by experimenting with six different denoising techniques/priors - K-SVD [4], BM3D [5], PLOW [7], Total Variation (TV) [21], q-GGMRF [22] and discrete reconstruction (DR) [23].”
Boyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,677
Image denoising by sparse 3-D transform-domain collaborative filtering
Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.901238 · Cited by 9,283 · cited 3×
“We compare reconstructions using the Plug-andPlay priors framework by experimenting with six different denoising techniques/priors - K-SVD [4], BM3D [5], PLOW [7], Total Variation (TV) [21], q-GGMRF [22] and discrete reconstruction (DR) [23].”
A Non-Local Algorithm for Image Denoising
Buades, A., Coll, B., & Morel, J.-M. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.38 · Cited by 7,616 · cited 1×
“Examples of new methods include a wide variety of patch based non-local means approaches [3], generalizations of bi-lateral filtering approaches [1], patch-based dictionary learning methods such as K-SVD [4], block-matching with transform-based denoising such as BM3D [5], and a variety of total-variation [2], and Mark…”
Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries
Elad, M., & Aharon, M. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2006.881969 · Cited by 5,321 · cited 3×
“We compare reconstructions using the Plug-and- Play priors framework by experimenting with six different denoising techniques/priors -K-SVD [4], BM3D [5], PLOW [7], Total Variation (TV) [21], q-GGMRF [22] and discrete reconstruction (DR) [23].”
The Split Bregman Method for L1-Regularized Problems
Goldstein, T., & Osher, S. (2009) · SIAM Journal on Imaging Sciences · doi:10.1137/080725891 · Cited by 4,282 · cited 3×
“Examples of new methods include a wide variety of patch based non-local means approaches [3], generalizations of bi-lateral filtering approaches [1], patch-based dictionary learning methods such as K-SVD [4], block-matching with transform-based denoising such as BM3D [5], and a variety of total-variation [2], and Mark…”
Fast Image Recovery Using Variable Splitting and Constrained Optimization
Afonso, M. V., Bioucas-Dias, J. M., & Figueiredo, M. A. T. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2047910 · Cited by 1,223 · cited 3×
“In order to design an algorithm for (3) that decouples the forward and prior terms, we first split the variable x into two new variables x and v, and reformulate equation (3) as the following constrained optimization problem [15].”
InfluentialMR Image Reconstruction From Highly Undersampled k-Space Data by Dictionary Learning
Ravishankar, S., & Bresler, Y. (2011) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2010.2090538 · Cited by 1,059 · cited 2×
“Model-based reconstruction is a powerful framework for solving a variety of inverse problems in imaging including denoising, deblurring, tomographic reconstruction, and MRI reconstruction.”
Cited by
Learning Deep CNN Denoiser Prior for Image Restoration
Zhang, K., Zuo, W., Gu, S., & Zhang, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.300 · Cited by 2,116
“Here we note that, prior to [61], a similar idea of plug-and-play is also mentioned in [66] where a half quadratic splitting (HQS) method was proposed for image denoising, deblurring and inpainting.”
Plug-and-Play Image Restoration With Deep Denoiser Prior
Zhang, K., Li, Y., Zuo, W., Zhang, L., Van Gool, L., & Timofte, R. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3088914 · Cited by 1,113
“The plug-and-play IR can be traced back to [4], [14], [16].”
Deep Learning for Single Image Super-Resolution: A Brief Review
Yang, W., Zhang, X., Tian, Y., Wang, W., Xue, J.-H., & Liao, Q. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2919431 · Cited by 1,154
“[94] split (3) into a data part and a prior part with variable splitting techniques and then replaced the prior part with efficient denoising algorithms.”
The Little Engine That Could: Regularization by Denoising (RED)
Romano, Y., Elad, M., & Milanfar, P. (2017) · SIAM Journal on Imaging Sciences · doi:10.1137/16m1102884 · Cited by 934
“In this respect the proposed algorithm is superior to the P 3 scheme in its most general form, which at best is known to get to a stable-8 point [30, 32].”
InfluentialPlug-and-Play ADMM for Image Restoration: Fixed-Point Convergence and Applications
Chan, S. H., Wang, X., & Elgendy, O. A. (2017) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2016.2629286 · Cited by 904
Solving inverse problems using data-driven models
Arridge, S., Maass, P., Öktem, O., & Schönlieb, C.-B. (2019) · Acta Numerica · doi:10.1017/s0962492919000059 · Cited by 686
Deep Unfolding Network for Image Super-Resolution
Zhang, K., Van Gool, L., & Timofte, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00328 · Cited by 638
Deep Learning Techniques for Inverse Problems in Imaging
Ongie, G., Jalal, A., Metzler, C. A., Baraniuk, R. G., Dimakis, A. G., & Willett, R. (2020) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2020.2991563 · Cited by 577
“iterative reconstruction algorithm, while the second utilizes the ground truth training images to learn a generative prior. Learning a proximal operator or denoiser from data. The plug-and-play (PnP) [89] approach is a powerful method for solving inverse problems using exising image denoising algorithms. The high leve…”
InfluentialDenoising Prior Driven Deep Neural Network for Image Restoration
Dong, W., Wang, P., Yin, W., Shi, G., Wu, F., & Lu, X. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2873610 · Cited by 515
Shlezinger, N., Whang, J., Eldar, Y. C., & Dimakis, A. G. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3247480 · Cited by 368
CNN-Based Projected Gradient Descent for Consistent CT Image Reconstruction
Gupta, H., Jin, K. H., Nguyen, H. Q., McCann, M. T., & Unser, M. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2832656 · Cited by 380
“On the other hand, [14]–[18] use plug-and-play ADMM that uses a denoiser which is learnt independently of the iterative scheme.”
Deep Plug-And-Play Super-Resolution for Arbitrary Blur Kernels
Zhang, K., Zuo, W., & Zhang, L. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00177 · Cited by 408
“The plug-and-play image restoration which was first introduced in [15, 57, 69] has attracted significant attentions due to its flexibility and effectiveness in handling various inverse problems.”
Learning Proximal Operators: Using Denoising Networks for Regularizing Inverse Imaging Problems
Meinhardt, T., Moeller, M., Hazirbas, C., & Cremers, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.198 · Cited by 257
Fast Hyperspectral Image Denoising and Inpainting Based on Low-Rank and Sparse Representations
Zhuang, L., & Bioucas-Dias, J. M. (2018) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2018.2796570 · Cited by 363
“In this paper, we resort to the plug-and-play (PnP) prior framework [38] to address the denoising problem (6).”
Plug-and-Play Priors for Bright Field Electron Tomography and Sparse Interpolation
Sreehari, S., Venkatakrishnan, S. V., Wohlberg, B., Buzzard, G. T., Drummy, L. F., Simmons, J. P., & Bouman, C. A. (2016) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2016.2599778 · Cited by 300
“In this paper, we extend our plug-and-play framework of [40], [48] to present a robust algorithm for tomographic reconstruction and sparse image interpolation that exploits the non-local redundancies in microscope images.”
Self-citationOne Network to Solve Them All — Solving Linear Inverse Problems Using Deep Projection Models
Chang, J. H. R., Li, C.-L., Poczos, B., Vijaya Kumar, B. V. K., & Sankaranarayanan, A. C. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.627 · Cited by 327
Regularizing Hyperspectral and Multispectral Image Fusion by CNN Denoiser
Dian, R., Li, S., & Kang, X. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2980398 · Cited by 328
FlexISP: a flexible camera image processing framework
Heide, F., Steinberger, M., Tsai, Y.-T., Rouf, M., Pająk, D., Reddy, D., Gallo, O., Liu, J., Heidrich, W., Egiazarian, K., et al. (2014) · ACM Transactions on Graphics · doi:10.1145/2661229.2661260 · Cited by 298
“2013] PD DC TV + cross-channel [Venkatakrishnan et al. 2013] ADMM Tomographic DN KSVD | BM3D | TV | .”
Denoising Diffusion Models for Plug-and-Play Image Restoration
Zhu, Y., Zhang, K., Liang, J., Cao, J., Wen, B., Timofte, R., & Gool, L. V. (2023) · 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw59228.2023.00129 · Cited by 228
“Recent studies have demonstrated that plug-and-play Image Restoration (IR) methods can effectively handle a variety of low-level vision tasks, such as image denoising [5], image Super-Resolution (SR) [16, 17, 37], image deblurring [12] and image inpainting [27], with excellent results [6, 11, 54, 57, 58].”
Kamilov, U. S., Bouman, C. A., Buzzard, G. T., & Wohlberg, B. (2023) · IEEE Signal Processing Magazine · doi:10.1109/msp.2022.3199595 · Cited by 244
“The first PnP algorithm was PnP-ADMM [3], which is implemented by iterating the steps in Algorithm 3 until convergence.”
Self-citationPlug-and-Play Methods for Magnetic Resonance Imaging: Using Denoisers for Image Recovery
Ahmad, R., Bouman, C. A., Buzzard, G. T., Chan, S., Liu, S., Reehorst, E. T., & Schniter, P. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2019.2949470 · Cited by 261
“The original PnP method [7] is based on the alternating direction method of multipliers (ADMM) [13].”
InfluentialSelf-citationA Plug-and-Play Priors Approach for Solving Nonlinear Imaging Inverse Problems
Kamilov, U. S., Mansour, H., & Wohlberg, B. (2017) · IEEE Signal Processing Letters · doi:10.1109/lsp.2017.2763583 · Cited by 264
“FISTA enjoys a modular structure similar to ADMM, where the prior model on the image is only imposed via the proximal operator [8].”
Self-citationLearned D-AMP: Principled Neural Network based Compressive Image Recovery
Metzler, C. A., Mousavi, A., & Baraniuk, R. G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.06625 · Cited by 178
“This method uses a NN as a proximal operator, and can be interpreted as a NN-based version of Plug and Play ADMM [25].”
Regularization by Denoising: Clarifications and New Interpretations
Reehorst, E. T., & Schniter, P. (2019) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2018.2880326 · Cited by 239
Real-time 3D reconstruction from single-photon lidar data using plug-and-play point cloud denoisers
Tachella, J., Altmann, Y., Mellado, N., McCarthy, A., Tobin, R., Buller, G. S., Tourneret, J.-Y., & McLaughlin, S. (2019) · Nature Communications · doi:10.1038/s41467-019-12943-7 · Cited by 245
“on algorithm works directly on the lidar waveforms, making use of off-the-shelf computer graphics point cloud denoisers for distributed surfaces. We extend and adapt the ideas of plug-and-play priors [26, 27, 28, 29] and regularisation by denoising [30, 31], which have recently appeared in the image processing communi…”
Hyperspectral Image Denoising and Anomaly Detection Based on Low-Rank and Sparse Representations
Zhuang, L., Gao, L., Zhang, B., Fu, X., & Bioucas-Dias, J. M. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.3040221 · Cited by 227
“The convergence of the plug-and-play iterative procedures is currently an active area of research [41].”
InfluentialPlug-and-Play Methods Provably Converge with Properly Trained DenoisersPreprint
Ryu, E. K., Liu, J., Wang, S., Chen, X., Wang, Z., & Yin, W. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.05406 · Cited by 564
“The first PnP method was the Plug-and-play ADMM proposed in (Venkatakrishnan et al., 2013).”
InfluentialGraph Spectral Image Processing
Cheung, G., Magli, E., Tanaka, Y., & Ng, M. K. (2018) · Proceedings of the IEEE · doi:10.1109/jproc.2018.2799702 · Cited by 251
“Therefore, the set of all possible eigenbases satisfying (2) for the Laplacian of a square grid graph can be represented as where u ( k,l ) are the eigenvectors corresponding to the basis vectors of the separable 2D-DCT.”
An Online Plug-and-Play Algorithm for Regularized Image Reconstruction
Sun, Y., Wohlberg, B., & Kamilov, U. S. (2019) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2019.2893568 · Cited by 207
“Our goal is not to justify the PnP framework, as its benefits have been well illustrated in prior work [14], [18], [26], but to focus on the aspects that relate to online processing of data.”
InfluentialSelf-citationBlind Image Super-Resolution: A Survey and Beyond
Liu, A., Liu, Y., Gu, J., Qiao, Y., & Dong, C. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3203009 · Cited by 187
Image Restoration by Iterative Denoising and Backward Projections
Tirer, T., & Giryes, R. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2875569 · Cited by 214
“for each type of matrix H , a general recovery strategy has been proposed in [12], denoted as the Plug-and-Play (P&P).”
InfluentialPlug-and-Play Unplugged: Optimization-Free Reconstruction Using Consensus Equilibrium
Buzzard, G. T., Chan, S. H., Sreehari, S., & Bouman, C. A. (2018) · SIAM Journal on Imaging Sciences · doi:10.1137/17m1122451 · Cited by 191
Self-citationCasFormer: Cascaded transformers for fusion-aware computational hyperspectral imaging
Li, C., Zhang, B., Hong, D., Zhou, J., Vivone, G., Li, S., & Chanussot, J. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102408 · Cited by 183
Compressed Sensing with Deep Image Prior and Learned RegularizationPreprint
Veen, D. V., Jalal, A., Soltanolkotabi, M., Price, E., Vishwanath, S., & Dimakis, A. G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.06438 · Cited by 216
Deep Equilibrium Architectures for Inverse Problems in Imaging
Gilton, D., Ongie, G., & Willett, R. (2021) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2021.3118944 · Cited by 163
Image Denoising: The Deep Learning Revolution and Beyond—A Survey Paper
Elad, M., Kawar, B., & Vaksman, G. (2023) · SIAM Journal on Imaging Sciences · doi:10.1137/23m1545859 · Cited by 241
Graph-Based Blind Image Deblurring From a Single Photograph
Bai, Y., Cheung, G., Liu, X., & Gao, W. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2874290 · Cited by 165
Optimizing a Parameterized Plug-and-Play ADMM for Iterative Low-Dose CT Reconstruction
He, J., Yang, Y., Wang, Y., Zeng, D., Bian, Z., Zhang, H., Sun, J., Xu, Z., & Ma, J. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2865202 · Cited by 178
“[10] proposed a plug-and-play ADMM in which the denoising step is replaced with an off-the-shelf image denoising algorithm and achieved promising performance.”
Noise2Inverse: Self-Supervised Deep Convolutional Denoising for Tomography
Hendriksen, A. A., Pelt, D. M., & Batenburg, K. J. (2020) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2020.3019647 · Cited by 226
“Notethatitisalsopossible toincludeBM3Dasapriorinsideiterativealgorithmsforinverse problemsusingaplug-and-playframework[34].”
MuLoG, or How to Apply Gaussian Denoisers to Multi-Channel SAR Speckle Reduction?
Deledalle, C.-A., Denis, L., Tabti, S., & Tupin, F. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2713946 · Cited by 176
Style Transfer Via Texture Synthesis
Elad, M., & Milanfar, P. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2678168 · Cited by 172
“This is the essence of the Plugand-Play Prior method as proposed in [42].”
Primal-Dual Plug-and-Play Image Restoration
Ono, S. (2017) · IEEE Signal Processing Letters · doi:10.1109/lsp.2017.2710233 · Cited by 178
“The key idea in plug-and-play image restoration [6] is that any”
InfluentialWINNet: Wavelet-Inspired Invertible Network for Image Denoising
Huang, J., & Dragotti, P. L. (2022) · IEEE Transactions on Image Processing · doi:10.1109/tip.2022.3184845 · Cited by 180
“technique [3]–[7], image denoising algorithms have become even more important as they can also be used to solve other image restoration problems by acting as a powerful prior.”
CoIL: Coordinate-Based Internal Learning for Tomographic Imaging
Sun, Y., Liu, J., Xie, M., Wohlberg, B. E., & Kamilov, U. S. (2021) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2021.3125564 · Cited by 194
Self-citationFastHyMix: Fast and Parameter-Free Hyperspectral Image Mixed Noise Removal
Zhuang, L., & Ng, M. K. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3112577 · Cited by 167
RARE: Image Reconstruction Using Deep Priors Learned Without Groundtruth
Liu, J., Sun, Y., Eldeniz, C., Gan, W., An, H., & Kamilov, U. S. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.2998402 · Cited by 182
“While the original formulation is based on ADMM, other PnP algorithms have been developed using PGM and primal-dual splitting [27], [28], [49].”
InfluentialDeep plug-and-play priors for spectral snapshot compressive imaging
Zheng, S., Liu, Y., Meng, Z., Qiao, M., Tong, Z., Yang, X., Han, S., & Yuan, X. (2021) · Photonics Research · doi:10.1364/prj.411745 · Cited by 171
“…and deep-learning-based algorithms each have their own pros and cons, it is desirable to develop a fast, flexible, and high accuracy algorithm for spectral SCI. Fortunately, the plug-and-play (PnP) framework [40,41] has been proposed for inverse problems with provable convergence [42,43].”
Computational spectrometers enabled by nanophotonics and deep learning
Gao, L., Qu, Y., Wang, L., & Yu, Z. (2022) · Nanophotonics · doi:10.1515/nanoph-2021-0636 · Cited by 158
“play (PnP) framework [112, 113] has been proposed for”
Hyperspectral Anomaly Detection via Deep Plug-and-Play Denoising CNN Regularization
Fu, X., Jia, S., Zhuang, L., Xu, M., Zhou, J. W., & Li, Q. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2021.3049224 · Cited by 158
GAP-net for Snapshot Compressive ImagingPreprint
Meng, Z., Jalali, S., & Yuan, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.08364 · Cited by 169
“, the underlying desired signal, as output; during testing, feeding the captured measurements into the trained E2E-CNN, it is expected to produce the desired signal instantaneously [27], [24], [5]; (c) Deep unfolding algorithms, where, instead of one CNN, a sequence of K CNNs are trained to map the measurements to the…”
Self-supervised Neural Networks for Spectral Snapshot Compressive Imaging
Meng, Z., Yu, Z., Xu, K., & Yuan, X. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00262 · Cited by 144
“Specifically, we develop a reconstruction framework that integrates the untrained neural networks as priors [39] into the plug-and-play (PnP) algorithms [38, 40].”
DeepSIC: Deep Soft Interference Cancellation for Multiuser MIMO Detection
Shlezinger, N., Fu, R., & Eldar, Y. C. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3032663 · Cited by 163
“4, is related to the concepts of deep mutual learning [52], as well as plug-and-play methods [53].”
Learning Deep Gradient Descent Optimization for Image Deconvolution
Gong, D., Zhang, Z., Shi, Q., van den Hengel, A., Shen, C., & Zhang, Y. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2968289 · Cited by 148
CSI-Fingerprinting Indoor Localization via Attention-Augmented Residual Convolutional Neural Network
Zhang, B., Sifaou, H., & Li, G. Y. (2023) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2023.3235449 · Cited by 141
“PLUG AND PLAY FOR IMU MEASUREMENTS In this section, we adopt the idea of PnP [27], [41] to incorporate the IMU measurements with different precisions into an existing tracking system.”
Poisson inverse problems by the Plug-and-Play scheme
Rond, A., Giryes, R., & Elad, M. (2016) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2016.09.009 · Cited by 147
Learning Maximally Monotone Operators for Image Recovery
Pesquet, J.-C., Repetti, A., Terris, M., & Wiaux, Y. (2021) · SIAM Journal on Imaging Sciences · doi:10.1137/20m1387961 · Cited by 185
“Since a MMO is fully characterized by its resolvent, our approach enters into the family of so-called plug-and-play (PnP) methods [57], where one replaces the proximity operator of an optimization algorithm with a denoiser, e.”
Robust real-time 3D imaging of moving scenes through atmospheric obscurant using single-photon LiDAR
Tobin, R., Halimi, A., McCarthy, A., Soan, P. J., & Buller, G. S. (2021) · Scientific Reports · doi:10.1038/s41598-021-90587-8 · Cited by 142
Deep Image Demosaicking Using a Cascade of Convolutional Residual Denoising Networks
Kokkinos, F., & Lefkimmiatis, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_19 · Cited by 148
Regularization by Denoising via Fixed-Point Projection (RED-PRO)
Cohen, R., Elad, M., & Milanfar, P. (2021) · SIAM Journal on Imaging Sciences · doi:10.1137/20m1337168 · Cited by 168
Plug-and-Play Algorithms for Video Snapshot Compressive Imaging
Yuan, X., Liu, Y., Suo, J., Durand, F., & Dai, Q. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3099035 · Cited by 147
“Though PnP can date back to 2013 [52], it is getting powerful in real inverse problems because the usage of advanced deep denoising networks [56], [57].”
An Adaptive and Robust Deep Learning Framework for THz Ultra-Massive MIMO Channel Estimation
Yu, W., Shen, Y., He, H., Yu, X., Song, S., Zhang, J., & Letaief, K. B. (2023) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2023.3282832 · Cited by 143
“In [41], the authors found that the proximal operators in PGD and ADMM could be understood as MAP estimators of Gaussian denoising problems.”
Liu, R., Sun, Y., Zhu, J., Tian, L., & Kamilov, U. S. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00530-3 · Cited by 149
“The work on plug-and-play priors has generalized the notion of image priors to implicit regularizers characterized by image denoisers [3, 4, 44, 53].”
Bayesian Imaging Using Plug & Play Priors: When Langevin Meets Tweedie
Laumont, R., Bortoli, V. D., Almansa, A., Delon, J., Durmus, A., & Pereyra, M. (2022) · SIAM Journal on Imaging Sciences · doi:10.1137/21m1406349 · Cited by 194
MAGIC: Manifold and Graph Integrative Convolutional Network for Low-Dose CT Reconstruction
Xia, W., Lu, Z., Huang, Y., Shi, Z., Liu, Y., Chen, H., Chen, Y., Zhou, J., & Zhang, Y. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3088344 · Cited by 133
Iterative Joint Image Demosaicking and Denoising Using a Residual Denoising Network
Kokkinos, F., & Lefkimmiatis, S. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2019.2905991 · Cited by 133
“Finally, while our demosaick network shares a similar philosophy with methods such as RED [29], P3 [30] and IRCNN [31], it exhibits some important and distinct dif-”
Iterative integration of deep learning in hybrid Earth surface system modelling
Chen, M., Qian, Z., Boers, N., Jakeman, A. J., Kettner, A. J., Brandt, M., Kwan, M., Batty, M., Li, W., Zhu, R., et al. (2023) · Nature Reviews Earth & Environment · doi:10.1038/s43017-023-00452-7 · Cited by 133
“The embedding mode enables process-based models and deep neural networks to function as plug-and-play components 117–119 .”
Fixed Point Strategies in Data Science
Combettes, P. L., & Pesquet, J.-C. (2021) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2021.3069677 · Cited by 142
“The principle of the so-called Plug-and-play (PnP) methods [39], [145], [159], [174], [184] is to replace a proximity operator appearing in some proximal minimization algorithm by another operator Q.”
J-MoDL: Joint Model-Based Deep Learning for Optimized Sampling and Reconstruction
Aggarwal, H. K., & Jacob, M. (2020) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2020.3004094 · Cited by 131
“These model-based methods can be thought of as learning based variants of earlier plug-and-play methods [19], [20], which used off-the-shelf denoisers as regularization penalties.”
A review on deep learning MRI reconstruction without fully sampled k-space
Zeng, G., Guo, Y., Zhan, J., Wang, Z., Lai, Z., Du, X., Qu, X., & Guo, D. (2021) · BMC Medical Imaging · doi:10.1186/s12880-021-00727-9 · Cited by 131
“Additionally, experiments have shown that converting advanced denoising devices into regularization items can achieve good results [63, 74, 75], regularization by denoising (RED) [74] and the plug-and-play-prior (PnP) [76] are two common skills.”
Deep plug-and-play prior for low-rank tensor completion
Zhao, X.-L., Xu, W., Jiang, T.-X., Wang, Y., & Ng, M. K. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.03.018 · Cited by 120
“Recent studies about PnP ADMM show that any off-the-shelf denoising method can be directly utilized as priors instead of the step in ADMM to calculate the proximal operator of the specified regularizer [26], [40], [41].”
Deep Unfolding for Snapshot Compressive Imaging
Meng, Z., Yuan, X., & Jalali, S. (2023) · International Journal of Computer Vision · doi:10.1007/s11263-023-01844-4 · Cited by 124
“(d) Plug-and-play (PnP) methods (Sreehari et al., 2016; Venkatakrishnan et al., 2013) in which a pre-trained denoising network is plugged into one of the iterative algorithms of class (a) (Qiao et al., 2020b; Yuan et al., 2020; Zheng et al., 2021a).”
2022 Review of Data-Driven Plasma Science
Anirudh, R., Archibald, R. K., Asif, M. S., Becker, M. M., Benkadda, S., Bremer, P., Budé, R. H. S., Chang, C. S., Chen, L., Churchill, R. M., et al. (2023) · IEEE Transactions on Plasma Science · doi:10.1109/tps.2023.3268170 · Cited by 129
Scalable Plug-and-Play ADMM With Convergence Guarantees
Sun, Y., Wu, Z., Xu, X., Wohlberg, B. E., & Kamilov, U. S. (2021) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2021.3094062 · Cited by 140
Self-citationTutorial on Amortized Optimization
Amos, B. (2023) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000102 · Cited by 207
DOLCE: A Model-Based Probabilistic Diffusion Framework for Limited-Angle CT Reconstruction
Liu, J., Anirudh, R., Thiagarajan, J. J., He, S., Mohan, K. A., Kamilov, U. S., & Kim, H. (2023) · 2023 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00963 · Cited by 106
“Popular MBDL frameworks include Plug-and-Play Priors (PnP) [59,74], which use pre-trained deep denoisers as image priors [53, 69, 90], and Deep Unfolding [1, 24, 25, 35, 48, 94], which interpret the iterations of a model-based algorithm as layers of a CNN to perform end-to-end supervised training.”
Score-Based Diffusion Models as Principled Priors for Inverse Imaging
Feng, B. T., Smith, J., Rubinstein, M., Chang, H., Bouman, K. L., & Freeman, W. T. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00965 · Cited by 182
“Other methods like Plug-and-Play (PnP) [76] and Regularization by Denoising (RED) [59] use an image denoiser as an implicit regular-izer to provide a point solution.”
Multi-scale Weighted Nuclear Norm Image Restoration
Yair, N., & Michaeli, T. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00334 · Cited by 110
Chang, X., Bian, L., & Zhang, J. (2021) · eLight · doi:10.1186/s43593-021-00004-w · Cited by 102
“The PNP framework has a theoretical guarantee of convergence for most real-domain tasks, such as denoising, deblurring [52, 53], etc.”
InfluentialClass-Aware Fully Convolutional Gaussian and Poisson Denoising
Remez, T., Litany, O., Giryes, R., & Bronstein, A. M. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2859044 · Cited by 101
“Another approach is to cast the Poisson denoising problem to a Gaussian one using the plug-and-play framework [5] as suggested in the P4IP method [47].”
Total Deep Variation for Linear Inverse Problems
Kobler, E., Effland, A., Kunisch, K. K., & Pock, T. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00757 · Cited by 122
Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser
Kadkhodaie, Z., & Simoncelli, E. P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.13640 · Cited by 13
“Nearly a decade ago, a strategy known as “plug and play” was proposed for using the prior embedded in a denoiser to solve other inverse problems [9], and a number of recent extensions [10–17] have used this concept to develop MAP solutions for linear inverse problems.”
Turning a denoiser into a super-resolver using plug and play priors
Brifman, A., Romano, Y., & Elad, M. (2016) · 2016 IEEE International Conference on Image Processing (ICIP) · doi:10.1109/icip.2016.7532589 · Cited by 91
“Inspired by the PPP scheme [1], in this section we describe how to translate Equation (3) to a sequence of denoising problems , which minimize a much simpler function – Equation (5).”
InfluentialA Convergent Image Fusion Algorithm Using Scene-Adapted Gaussian-Mixture-Based Denoising
Teodoro, A. M., Bioucas‐Dias, J. M., & Figueiredo, M. A. T. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2869727 · Cited by 102
Graph Spectral Image Processing
(2021) · Wiley eBooks · doi:10.1002/9781119850830 · Cited by 90
Hyperspectral Mixed Noise Removal By $\ell _1$-Norm-Based Subspace Representation
Zhuang, L., & Ng, M. K. (2020) · IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing · doi:10.1109/jstars.2020.2979801 · Cited by 95
OCMR (v1.0)--Open-Access Multi-Coil k-Space Dataset for Cardiovascular Magnetic Resonance ImagingPreprint
Chen, C., Liu, Y., Schniter, P., Tong, M., Zaręba, K. M., Simonetti, O. P., Potter, L. C., & Ahmad, R. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.03410 · Cited by 87
“Plug-and-play (PnP) methods (37) provide another avenue to perform MRI recovery (38).”
State‐of‐art analysis of image denoising methods using convolutional neural networks
Thakur, R. S., Yadav, R. N., & Gupta, L. (2019) · IET Image Processing · doi:10.1049/iet-ipr.2019.0157 · Cited by 87
Learned Low-Rank Priors in Dynamic MR Imaging
Ke, Z., Huang, W., Cui, Z., Cheng, J., Jia, S., Wang, H., Liu, X., Zheng, H., Ying, L. L., Zhu, Y., et al. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3096218 · Cited by 84
Learned Reconstruction Methods With Convergence Guarantees: A survey of concepts and applications
Mukherjee, S., Hauptmann, A., Öktem, O., Pereyra, M., & Schönlieb, C. (2023) · IEEE Signal Processing Magazine · doi:10.1109/msp.2022.3207451 · Cited by 114
“[46] pioneered the idea of replacing the proximal operator with an off-the-shelf denoiser.”
CaGAN: A Cycle-Consistent Generative Adversarial Network With Attention for Low-Dose CT Imaging
Huang, Z., Chen, Z., Zhang, Q., Quan, G., Ji, M., Zhang, C., Yang, Y., Liu, X., Liang, D., Zheng, H., et al. (2020) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2020.3012928 · Cited by 91
Xu, J., Zhao, Y., Li, H., & Zhang, P. (2019) · Inverse Problems · doi:10.1088/1361-6420/ab08f9 · Cited by 85
Iterative Residual Optimization Network for Limited-Angle Tomographic Reconstruction
Pan, J., Yu, H., Gao, Z., Wang, S., Zhang, H., & Wu, W. (2024) · IEEE Transactions on Image Processing · doi:10.1109/tip.2024.3351382 · Cited by 81
“Plug-and-Play (PnP) [50] and Regularization by Denoising [51] are efficient iterative algorithms to solve image inverse problems by minimizing the combination of both data-fidelity term and regularization term.”
Image restoration and reconstruction using variable splitting and class-adapted image priors
Teodoro, A. M., Bioucas‐Dias, J. M., & Figueiredo, M. A. T. (2016) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2016.7533014 · Cited by 87
Tuning-free Plug-and-Play Proximal Algorithm for Inverse Imaging ProblemsPreprint
Wei, K., Avilés-Rivero, A. I., Liang, J., Fu, Y., Schönlieb, C., & Huang, H. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.09611 · Cited by 125
“2011; Venkatakrishnan et al., 2013), which has attracted great attention owing to its effectiveness and flexibility to handle a wide range of inverse imaging problems.”
InfluentialGradient Step Denoiser for convergent Plug-and-PlayPreprint
Hurault, S., Leclaire, A., & Papadakis, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.03220 · Cited by 113
“Plug-and-Play (PnP) methods (Venkatakrishnan et al., 2013) build on proximal splitting algorithms by replacing the proximity operator of g with a generic denoiser, e.g. a pretrained deep network.”
Block Coordinate Regularization by Denoising
Sun, Y., Liu, J., & Kamilov, U. S. (2020) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2020.2996385 · Cited by 83
“Consider the following alternative to PnP that also relies on a denoising function [18,19] x (cid:0) (cid:1) Under some conditions on the denoiser, it is possible to relate H ( ) in (3) to some explicit regularization function h .”
InfluentialImage-Adaptive GAN based ReconstructionPreprint
Abu-Hussein, S., Tirer, T., & Giryes, R. (2019) · arXiv (Cornell University) · doi:10.1609/aaai.v34i04.5708 · Cited by 76
Chen, G., Hong, X., Ding, Q., Zhang, Y., Chen, H., Fu, S., Zhao, Y., Zhang, X., Ji, H., Wang, G., et al. (2020) · Medical Physics · doi:10.1002/mp.14170 · Cited by 74
Provable Convergence of Plug-and-Play Priors With MMSE Denoisers
Xu, X., Sun, Y., Liu, J., Wohlberg, B. E., & Kamilov, U. S. (2020) · IEEE Signal Processing Letters · doi:10.1109/lsp.2020.3006390 · Cited by 101
Self-citationSingle-image deblurring with neural networks: A comparative survey
Koh, J., Lee, J., & Yoon, S. (2020) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2020.103134 · Cited by 82
Metadata
- Authors
- Venkatakrishnan, S. V., Bouman, C. A., & Wohlberg, B.
- Affiliations
- Purdue University West Lafayette (US)
- Los Alamos National Laboratory (US)
- Venue
- 2013 IEEE Global Conference on Signal and Information Processing
- Publisher
- IEEE
- Published
- 2013-12
- Pages
- 945-948
- ISSN
- 2376-4066 (ISSN-L)
- OpenAlex
- W2087416986
- Cited by
- 1,178
- References
- 32
- Citations by year
- 2026: 99 · 2025: 172 · 2024: 147 · 2023: 177 · 2022: 121 · 2021: 182 · 2020: 102 · 2019: 75 · 2018: 52 · 2017: 24 · 2016: 16 · 2015: 8 · 2014: 2
- FWCI
- 5.53
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Global Conference on Signal and Information Processing (publishedVersion)
- Purdue e-Pubs (Purdue University System) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Signal Denoising Methods
- Physical Sciences › Mathematics › Mathematical Physics › Numerical methods in inverse problems
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Medical Imaging Techniques and Applications
- Keywords
- plug-and-play priors, model-based reconstruction, image denoising, imaging inverse problems, X-ray CT, electron microscopy, magnetic resonance imaging, ultrasonography, prior models, software integration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.