The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Plug-and-Play priors for model based reconstruction
Abstract
Model-based reconstruction is a powerful framework for solving a variety of inverse problems in imaging. In recent years, enormous progress has been made in the problem of denoising, a special case of an inverse problem where the forward model is an identity operator. Similarly, great progress has been made in improving model-based inversion when the forward model corresponds to complex physical measurements in applications such as X-ray CT, electron-microscopy, MRI, and ultrasound, to name just a few. However, combining state-of-the-art denoising algorithms (i.e., prior models) with state-of-the-art inversion methods (i.e., forward models) has been a challenge for many reasons. In this paper, we propose a flexible framework that allows state-of-the-art forward models of imaging systems to be matched with state-of-the-art priors or denoising models. This framework, which we term as Plug-and-Play priors, has the advantage that it dramatically simplifies software integration, and moreover, it allows state-of-the-art denoising methods that have no known formulation as an optimization problem to be used. We demonstrate with some simple examples how Plug-and-Play priors can be used to mix and match a wide variety of existing denoising models with a tomographic forward model, thus greatly expanding the range of possible problem solutions.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
A Threshold Selection Method from Gray-Level Histograms
Otsu, N. (1979) · IEEE Transactions on Systems, Man, and Cybernetics · doi:10.1109/tsmc.1979.4310076 · Cited by 43,398 · cited 1×
“In order to initialize the class labels b we use Otsu’s method [30] on a low pass filtered version of the noisy input y.”
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075
Nonlinear total variation based noise removal algorithms
Rudin, L. I., Osher, S., & Fatemi, E. (1992) · Physica D: Nonlinear Phenomena · doi:10.1016/0167-2789(92)90242-f · Cited by 15,704 · cited 1×
“We compare reconstructions using the Plug-andPlay priors framework by experimenting with six different denoising techniques/priors - K-SVD [4], BM3D [5], PLOW [7], Total Variation (TV) [21], q-GGMRF [22] and discrete reconstruction (DR) [23].”
Boyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,677
Image denoising by sparse 3-D transform-domain collaborative filtering
Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.901238 · Cited by 9,283 · cited 3×
“We compare reconstructions using the Plug-andPlay priors framework by experimenting with six different denoising techniques/priors - K-SVD [4], BM3D [5], PLOW [7], Total Variation (TV) [21], q-GGMRF [22] and discrete reconstruction (DR) [23].”
A Non-Local Algorithm for Image Denoising
Buades, A., Coll, B., & Morel, J.-M. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.38 · Cited by 7,616 · cited 1×
“Examples of new methods include a wide variety of patch based non-local means approaches [3], generalizations of bi-lateral filtering approaches [1], patch-based dictionary learning methods such as K-SVD [4], block-matching with transform-based denoising such as BM3D [5], and a variety of total-variation [2], and Mark…”
Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries
Elad, M., & Aharon, M. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2006.881969 · Cited by 5,321 · cited 3×
“We compare reconstructions using the Plug-and- Play priors framework by experimenting with six different denoising techniques/priors -K-SVD [4], BM3D [5], PLOW [7], Total Variation (TV) [21], q-GGMRF [22] and discrete reconstruction (DR) [23].”
The Split Bregman Method for L1-Regularized Problems
Goldstein, T., & Osher, S. (2009) · SIAM Journal on Imaging Sciences · doi:10.1137/080725891 · Cited by 4,282 · cited 3×
“Examples of new methods include a wide variety of patch based non-local means approaches [3], generalizations of bi-lateral filtering approaches [1], patch-based dictionary learning methods such as K-SVD [4], block-matching with transform-based denoising such as BM3D [5], and a variety of total-variation [2], and Mark…”
Fast Image Recovery Using Variable Splitting and Constrained Optimization
Afonso, M. V., Bioucas-Dias, J. M., & Figueiredo, M. A. T. (2010) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2047910 · Cited by 1,223 · cited 3×
“In order to design an algorithm for (3) that decouples the forward and prior terms, we first split the variable x into two new variables x and v, and reformulate equation (3) as the following constrained optimization problem [15].”
InfluentialMR Image Reconstruction From Highly Undersampled k-Space Data by Dictionary Learning
Ravishankar, S., & Bresler, Y. (2011) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2010.2090538 · Cited by 1,059 · cited 2×
“Model-based reconstruction is a powerful framework for solving a variety of inverse problems in imaging including denoising, deblurring, tomographic reconstruction, and MRI reconstruction.”
A three‐dimensional statistical approach to improved image quality for multislice helical CT
Thibault, J., Sauer, K. D., Bouman, C. A., & Hsieh, J. (2007) · Medical Physics · doi:10.1118/1.2789499 · Cited by 937 · cited 4×
“The results indicate that methods such as K-SVD [4], BM3D [5], PLOW [22], q-GGMRF [23], TV [2], and discrete reconstruction (DR) [24] can all be easily applied as priors through direct use of software implementations of the denoising algorithms.”
InfluentialSelf-citationAfonso, M. V., Bioucas‐Dias, J. M., & Figueiredo, M. A. T. (2011) · Cited by 783
Low-Dose X-ray CT Reconstruction via Dictionary Learning
Xu, Q., Yu, H., Mou, X., Zhang, L., Hsieh, J., & Wang, G. (2012) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2012.2195669 · Cited by 690 · cited 1×
“For example, patch based dictionary priors have been used in inverse imaging problems such as tomography [10], [11] and MRI [12].”
BM3D Frames and Variational Image Deblurring
Danielyan, A., Katkovnik, V., & Egiazarian, K. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2176954 · Cited by 611 · cited 2×
“Furthermore, the approach in [13] is not directly applicable to denoising algorithms/priors formulated using a nonparametric point estimation framework such as [1] and [7].”
A Tour of Modern Image Filtering: New Insights and Methods, Both Practical and Theoretical
Milanfar, P. (2013) · IEEE Signal Processing Magazine · doi:10.1109/msp.2011.2179329 · Cited by 605 · cited 3×
“In recent years, there have been enormous advances in the solution of a particular inverse problem generally referred to as image denoising [1], [2].”
A unified approach to statistical tomography using coordinate descent optimization
Bouman, C. A., & Sauer, K. (1996) · IEEE Transactions on Image Processing · doi:10.1109/83.491321 · Cited by 442 · cited 2×
“Instead of using the simplified reconstruction operator F in the ADMM loop, we use an approximate operator F̃, which lowers the value of the cost function corresponding to F using NIter number of iterations of iterative coordinate descent (ICD) [24] with random order updates [8].”
Self-citation1 BM3D frames and variational image deblurring
Danielyan, A., Katkovnik, V., Egiazarian, K., & Member, S. (2012) · Cited by 414
Patch-Based Near-Optimal Image Denoising
Chatterjee, P., & Milanfar, P. (2012) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2172799 · Cited by 296 · cited 7×
“In fact, a number of very novel and effective approaches have recently emerged for image denoising [1]–[7].”
InfluentialFast Model-Based X-Ray CT Reconstruction Using Spatially Nonhomogeneous ICD Optimization
Yu, Z., Thibault, J.-B., Bouman, C. A., Sauer, K. D., & Hsieh, J. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2010.2058811 · Cited by 290 · cited 3×
“The random order ICD with surrogate functions [7] is used to implement the cost optimization.”
Self-citationA Splitting-Based Iterative Algorithm for Accelerated Statistical X-Ray CT Reconstruction
Ramani, S., & Fessler, J. A. (2012) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2011.2175233 · Cited by 257 · cited 2×
“While the ADMM approach is slower than ICD for our implementation and requires greater memory to store the auxiliary variables [20], [21], it provides a more flexible framework for incorporating different priors.”
Parallel MR Image Reconstruction Using Augmented Lagrangian Methods
Ramani, S., & Fessler, J. A. (2011) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2010.2093536 · Cited by 227
A Fast Total Variation Minimization Method for Image Restoration
Huang, Y., Ng, M. K., & Wen, Y.-W. (2008) · Multiscale Modeling & Simulation · doi:10.1137/070703533 · Cited by 218 · cited 1×
“We note that the variable splitting approach discussed here has been exploited to solve a variety of inverse problems [15], [18], [20] .”
Husarik, D. B., Marin, D., Samei, E., Richard, S., Chen, B., Jaffe, T. A., Bashir, M. R., & Nelson, R. C. (2012) · Investigative Radiology · doi:10.1097/rli.0b013e318251eafd · Cited by 53 · cited 1×
“Research in this field has demonstrated that model-based inverse methods can greatly improve the quality of reconstructed images [10].”
Model-Based Iterative Reconstruction for Bright-Field Electron Tomography
Venkatakrishnan, S. V., Drummy, L. F., Jackson, M., Graef, M. D., Simmons, J. P., & Bouman, C. A. (2015) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2014.2371751 · Cited by 53
Self-citationSparse representations for limited data tomography
Liao, H. Y., & Sapiro, G. (2008) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2008.4541261 · Cited by 48 · cited 1×
“For example, patch based dictionary priors have been used in inverse problems such as tomography [11] and MRI [12].”
Model based iterative reconstruction for Bright Field electron tomography
Venkatakrishnan, S. V., Drummy, L. F., Graef, M. D., Simmons, J. P., & Bouman, C. A. (2013) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2013228 · Cited by 24 · cited 1×
“In parallel with these efforts, researchers have been pioneering ways to create forward models for a wide array of imaging and sensing systems from medical scanners [8] to microscopes [9].”
Self-citationMultiscale Bayesian Methods for Discrete Tomography
Frese, T., Bouman, C. A., & Sauer, K. D. (1999) · Applied and numerical harmonic analysis · doi:10.1007/978-1-4612-1568-4_10 · Cited by 23 · cited 1×
“We compare reconstructions using the Plug-andPlay priors framework by experimenting with six different denoising techniques/priors - K-SVD [4], BM3D [5], PLOW [7], Total Variation (TV) [21], q-GGMRF [22] and discrete reconstruction (DR) [23].”
Self-citationReduced memory augmented Lagrangian algorithm for 3D iterative x-ray CT image reconstruction
McGaffin, M. G., Ramani, S., & Fessler, J. A. (2012) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.910941 · Cited by 14 · cited 1×
“Our proposed Plug-and-Play framework is based on a direct application of the alternating directions method of multipliers (ADMM) [14] that has recently become popular for the solution of a variety of MAP estimation/regularized inverse problems [15]–[18].”
Nilchian, M., & Unser, M. A. (2012) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2012.6235491 · Cited by 8
Moreau’s Proximal Mappings and Convexity in Hamilton-Jacobi Theory
Rockafellar, R. T. (2006) · Kluwer Academic Publishers eBooks · doi:10.1007/0-387-29195-4_1 · Cited by 7
Venkatakrishnan, S. V., Drummy, L. F., Jackson, M., Graef, M. D., Simmons, J. P., & Bouman, C. A. (2012) · IEEE Statistical Signal Processing Workshop (SSP) · doi:10.1109/ssp.2012.6319793 · Cited by 7 · cited 2×
“In parallel with these efforts, researchers have been pioneering ways to create forward models for a wide array of imaging and sensing systems from medical scanners [7] to microscopes [8].”
Self-citationImplicit priors for model-based inversion
Haneda, E., & Bouman, C. A. (2012) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2012.6288774 · Cited by 6 · cited 1×
“Examples of new methods include a wide variety of patch based non-local means approaches [3], generalizations of bi-lateral filtering approaches [1], patch-based dictionary learning methods such as K-SVD [4], block-matching with transform-based denoising such as BM3D [5], and a variety of total-variation [2], and Mark…”
Self-citation
Cited by
Learning Deep CNN Denoiser Prior for Image Restoration
Zhang, K., Zuo, W., Gu, S., & Zhang, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.300 · Cited by 2,116
“Here we note that, prior to [61], a similar idea of plug-and-play is also mentioned in [66] where a half quadratic splitting (HQS) method was proposed for image denoising, deblurring and inpainting.”
Plug-and-Play Image Restoration With Deep Denoiser Prior
Zhang, K., Li, Y., Zuo, W., Zhang, L., Van Gool, L., & Timofte, R. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2021.3088914 · Cited by 1,113
“The plug-and-play IR can be traced back to [4], [14], [16].”
Deep Learning for Single Image Super-Resolution: A Brief Review
Yang, W., Zhang, X., Tian, Y., Wang, W., Xue, J.-H., & Liao, Q. (2019) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2019.2919431 · Cited by 1,154
“[94] split (3) into a data part and a prior part with variable splitting techniques and then replaced the prior part with efficient denoising algorithms.”
The Little Engine That Could: Regularization by Denoising (RED)
Romano, Y., Elad, M., & Milanfar, P. (2017) · SIAM Journal on Imaging Sciences · doi:10.1137/16m1102884 · Cited by 934
“In this respect the proposed algorithm is superior to the P 3 scheme in its most general form, which at best is known to get to a stable-8 point [30, 32].”
InfluentialPlug-and-Play ADMM for Image Restoration: Fixed-Point Convergence and Applications
Chan, S. H., Wang, X., & Elgendy, O. A. (2017) · IEEE Transactions on Computational Imaging · doi:10.1109/tci.2016.2629286 · Cited by 904
Metadata
- Authors
- Venkatakrishnan, S. V., Bouman, C. A., & Wohlberg, B.
- Affiliations
- Purdue University West Lafayette (US)
- Los Alamos National Laboratory (US)
- Venue
- 2013 IEEE Global Conference on Signal and Information Processing
- Publisher
- IEEE
- Published
- 2013-12
- Pages
- 945-948
- ISSN
- 2376-4066 (ISSN-L)
- OpenAlex
- W2087416986
- Cited by
- 1,178
- References
- 32
- Citations by year
- 2026: 99 · 2025: 172 · 2024: 147 · 2023: 177 · 2022: 121 · 2021: 182 · 2020: 102 · 2019: 75 · 2018: 52 · 2017: 24 · 2016: 16 · 2015: 8 · 2014: 2
- FWCI
- 5.53
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- IEEE Global Conference on Signal and Information Processing (publishedVersion)
- Purdue e-Pubs (Purdue University System) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Signal Denoising Methods
- Physical Sciences › Mathematics › Mathematical Physics › Numerical methods in inverse problems
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Medical Imaging Techniques and Applications
- Keywords
- plug-and-play priors, model-based reconstruction, image denoising, imaging inverse problems, X-ray CT, electron microscopy, magnetic resonance imaging, ultrasonography, prior models, software integration
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.