Training an Active Random Field for Real-Time Image Denoising
The DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Abstract
Many computer vision problems can be formulated in a Bayesian framework based on Markov random fields (MRF) or conditional random fields (CRF). Generally, the MRF/CRF model is learned independently of the inference algorithm that is used to obtain the final result. In this paper, we observe considerable gains in speed and accuracy by training the MRF/CRF model together with a fast and suboptimal inference algorithm. An active random field (ARF) is defined as a combination of a MRF/CRF based model and a fast inference algorithm for the MRF/CRF model. This combination is trained through an optimization of a loss function and a training set consisting of pairs of input images and desired outputs. We apply the ARF concept to image denoising, using the Fields of Experts MRF together with a 1-4 iteration gradient descent algorithm for inference. Experimental validation on unseen data shows that the ARF approach obtains an improved benchmark performance as well as a 1000-3000 times speedup compared to the Fields of Experts MRF. Using the ARF approach, image denoising can be performed in real-time, at 8fps on a single CPU for a 256 x 256 image sequence, with close to state-of-the-art accuracy.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · cited 3×
“Other more appropriate loss functions could be used instead of the PSNR, for example the Structural Similarity Index (SSIM) [40].”
MethodGenetic algorithms in search, optimization, and machine learning
(1989) · Choice Reviews Online · doi:10.5860/choice.27-0936 · Cited by 49,010 · cited 1×
“Other optimization methods such as genetic algorithms [14] or simulated annealing [16] could be more appropriate for avoiding local optima and are subject to further investigation.”
Optimization by Simulated Annealing
Kirkpatrick, S., Gelatt, C. D., & Vecchi, M. P. (1983) · Science · doi:10.1126/science.220.4598.671 · Cited by 45,134 · cited 1×
“Other optimization methods such as genetic algorithms [14] or simulated annealing [16] could be more appropriate for avoiding local optima and are subject to further investigation.”
A Simplex Method for Function Minimization
Nelder, J. A., & Mead, R. (1965) · The Computer Journal · doi:10.1093/comjnl/7.4.308 · Cited by 28,905 · cited 1×
“We also experimented with gradient ascent, conjugate gradient and the simplex method [24].”
MethodGenetic Algorithms in Search, Optimization and Machine Learning
Goldberg, D. E., & Robson, D. (1988) · Cited by 18,391
Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data
Lafferty, J., McCallum, A., & Pereira, F. C. N. (2001) · ScholarlyCommons (University of Pennsylvania) · Cited by 12,935
Image denoising by sparse 3-D transform-domain collaborative filtering
Dabov, K., Foi, A., Katkovnik, V., & Egiazarian, K. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.901238 · Cited by 9,283 · cited 10×
“Non-local image denoising methods include [6] and especially 3D collaborative filtering (BM3D) [9], th e latter obtaining very good results with low computational expense.”
BackgroundMethodInfluentialOn the limited memory BFGS method for large scale optimization
Liu, D. C., & Nocedal, J. (1989) · Mathematical Programming · doi:10.1007/bf01589116 · Cited by 8,560 · cited 2×
“The MRF model was based on 3-cliques and inference was obtained by limited memory BFGS [21].”
MethodMartin, D., Fowlkes, C., Tal, D., & Malik, J. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937655 · Cited by 8,216
A Non-Local Algorithm for Image Denoising
Buades, A., Coll, B., & Morel, J.-M. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.38 · Cited by 7,616 · cited 1×
“Nonlocal image denoising methods include [6] and especially 3-”
MethodFast approximate energy minimization via graph cuts
Boykov, Y., Veksler, O., & Zabih, R. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.969114 · Cited by 6,988
Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.927467 · Cited by 5,409
Image Denoising Via Sparse and Redundant Representations Over Learned Dictionaries
Elad, M., & Aharon, M. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2006.881969 · Cited by 5,321 · cited 11×
“One could also observe that the ARF is within 0.5 dB from the best method and it is outperformed by two methods: KSVD [10], BM3D [9] and for some noise levels by wavelet denoising [27] and overcomplete DCT [10].”
BackgroundMethodInfluentialTraining Products of Experts by Minimizing Contrastive Divergence
Hinton, G. E. (2002) · Neural Computation · doi:10.1162/089976602760128018 · Cited by 4,879 · cited 4×
“26 ARF w/data term, train withσ ∈ [15, 40] 1 31.”
MethodInfluentialOn the Statistical Analysis of Dirty Pictures
Besag, J. (1986) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1986.tb01412.x · Cited by 4,191 · cited 1×
“The optimization is one of the available MRF/CRF maximum a posteriori (MAP) inference algorithms such as gradient descent, belief propagation [44], Graph Cuts [5], iterated conditional modes [3], etc.”
MethodImage denoising using scale mixtures of gaussians in the wavelet domain
Portilla, J., Strela, V., Wainwright, M. J., & Simoncelli, E. P. (2003) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.818640 · Cited by 2,255
Some generalized order-disorder transformations
Potts, R. B. (1952) · Mathematical Proceedings of the Cambridge Philosophical Society · doi:10.1017/s0305004100027419 · Cited by 2,122 · cited 2×
“For example, finding the global minimum of the Potts model [ 28 ], used in stereo matching as a prior term [5], [31], [32], is NP hard [5].”
BackgroundMathematical Statistics: Basic Ideas and Selected Topics.
Wachter, K. W., Bickel, P. J., & Doksum, K. A. (1979) · Journal of the American Statistical Association · doi:10.2307/2286373 · Cited by 1,886
Constructing Free-Energy Approximations and Generalized Belief Propagation Algorithms
Yedidia, J. S., Freeman, W. T., & Weiss, Y. (2005) · IEEE Transactions on Information Theory · doi:10.1109/tit.2005.850085 · Cited by 1,617
Image super-resolution as sparse representation of raw image patches
Yang, J., Wright, J., Huang, T., & Ma, Y. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587647 · Cited by 1,475 · cited 1×
“This type of training can be used in other applications where fast MRF inference is desired on models with a large number of parameters, for example super-resolution [13], [43].”
MethodFreeman, W. T., Pasztor, E. C., & Carmichael, O. T. (2000) · International Journal of Computer Vision · doi:10.1023/a:1026501619075 · Cited by 1,459 · cited 1×
“This type of training can be used in other applications where fast MRF inference is desired on models with a large number of parameters, for example super-resolution [13], [43].”
MethodStereo matching using belief propagation
Sun, J., Zheng, N.-N., & Shum, H.-Y. (2003) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2003.1206509 · Cited by 1,271 · cited 1×
“For example, finding the globa minimum of the Potts model [28], used in Stereo Matching as a prior term [5], [31], [32], is NP hard [5].”
MethodGeneralized Belief Propagation
Yedidia, J. S., Freeman, W. T., & Weiss, Y. (2000) · Cited by 918
SATzilla: Portfolio-based Algorithm Selection for SAT
Xu, L., Hutter, F., Hoos, H. H., & Leyton-Brown, K. (2008) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2490 · Cited by 857
Learning Conditional Random Fields for Stereo
Scharstein, D., & Pal, C. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383191 · Cited by 851 · cited 1×
“In [30], an approximate posterior was maximized by gradient optimization for learning a pairwise MRF for stereo matching.”
MethodRoth, S., & Black, M. J. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-008-0197-6 · Cited by 847 · cited 5×
“…original image, image corrupted with additive Gaussian noise with = 25; PSNR = 20:17; our result, PSNR = 28:94, 0.6 s and Fields of Experts result [29], PSNR = 28:67, 2280 s. Bottom results, from left to right: wavelet denoising [27], PSNR = 29:05, 16 s; overcomplete DCT, PSNR = 28:81, 38 s;…”
BackgroundMethodInfluentialFaraway, J. J. (2025) · Chapman & Hall/CRC eBooks · doi:10.1201/9781003449973 · Cited by 702
Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling
Zhu, S. C., Wu, Y., & Mumford, D. (1998) · International Journal of Computer Vision · doi:10.1023/a:1007925832420 · Cited by 652 · cited 1×
“A convolutional approach is also taken in the FRAME model for texture modeling [47].”
MethodTurney, P. D. (1995) · Journal of Artificial Intelligence Research · doi:10.1613/jair.120 · Cited by 570
Rao-Blackwellisation of sampling schemes
Casella, G. (1996) · Biometrika · doi:10.1093/biomet/83.1.81 · Cited by 546 · cited 1×
“This technique is know as Rao-Blackwellization [4], [7].”
MethodZheng, Y., Barbu, A., Georgescu, B., Scheuering, M., & Comaniciu, D. (2008) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2008.2004421 · Cited by 546
Self-citationA review of nonlinear diffusion filtering
Weickert, J. (1997) · Lecture Notes in Computer Science · doi:10.1007/3-540-63167-4_37 · Cited by 513 · cited 1×
“We present results for 1: Wiener filter; 2: nonlinear diffusion [41] using the nonlinear diffusion Matlab toolbox provided by Frederico”
MethodHidden Markov support vector machines
Altün, Y., Tsochantaridis, I., & Hofmann, T. (2003) · Cited by 458
Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters
Tappen, & Freeman (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238444 · Cited by 451 · cited 1×
“For example, finding the globa minimum of the Potts model [28], used in Stereo Matching as a prior term [5], [31], [32], is NP hard [5].”
MethodKumar, S., & Hebert (2003) · Proceedings Ninth IEEE International Conference on Computer Vision · doi:10.1109/iccv.2003.1238478 · Cited by 427 · cited 1×
“Quite recently, Conditional Random Fields (CRF) [18], [17] were developed as an extension of the MRF so that the clique potentials depend on the observed data y.”
BackgroundPizurica, A., & Philips, W. (2006) · IEEE Transactions on Image Processing · doi:10.1109/tip.2005.863698 · Cited by 411 · cited 1×
“This problem has been addressed using wavelets in [27], [26] and by learning a MRF prior model known as Fields of Experts on 5 5 pixel cliques in [29].”
MethodContextual models for object detection using boosted random fields
Torralba, A. B., Murphy, K. P., & Freeman, W. T. (2004) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 338
Toward automatic phenotyping of developing embryos from videos
Ning, F., Delhomme, D., LeCun, Y., Piano, F., Bottou, L., & Barbano, P. E. (2005) · IEEE Transactions on Image Processing · doi:10.1109/tip.2005.852470 · Cited by 261 · cited 3×
“69 ARF w/data term, trained withσ ∈ [15, 25] 1 31.”
BackgroundMethodAuto-context and its application to high-level vision tasks
Tu, Z. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587436 · Cited by 234 · cited 1×
“O a similar note, [37] trains a sequence of classifiers for obje ct segmentation.”
MethodLoss Functions for Discriminative Training of Energy-Based Models
LeCun, Y., & Huang, F. J. (2005) · International Conference on Artificial Intelligence and Statistics · Cited by 168
Learning Gaussian Conditional Random Fields for Low-Level Vision
Tappen, M. F., Liu, C., Adelson, E. H., & Freeman, W. T. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.382979 · Cited by 138 · cited 4×
“(20) is exactly the energy of a Gaussian Conditional Random Field [33]”
BackgroundResultInfluentialEstimating the Wrong Graphical Model: Benefits in the Computation-Limited Setting
Wainwright, M. J. (2006) · Journal of Machine Learning Research · doi:10.5555/1248547.1248613 · Cited by 119
Meltzer, T., Yanover, C., & Weiss, Y. (2005) · Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1 · doi:10.1109/iccv.2005.110 · Cited by 105 · cited 1×
“For example, even though the Potts model has been widely used in Stereo, the energy level of the desired result is higher th an the energy obtained by different optimization algorithms [ 32], or the global minimum [23].”
MethodMutual Boosting for Contextual Inference
Fink, M., & Perona, P. (2003) · CaltechAUTHORS (California Institute of Technology) · Cited by 84
Zhou, S. K., & Comaniciu, D. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-73273-0_2 · Cited by 71 · cited 1×
“A more complete solution for object or shape detection is offered by the shape regression machine [46], where an image based regression algorithm is trained to find a vector toward the object of interest from any random location inside the image.”
MethodUtilizing Variational Optimization to Learn Markov Random Fields
Tappen, M. F. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383037 · Cited by 54 · cited 5×
“For example, even though the Potts model has been widely used in Stereo, the energy level of the desired result is higher than the energy obtained by different optimization algorithms [32], or the global minimum [23].”
BackgroundInfluentialStructured Prediction via the Extragradient Method
Taskar, B., Lacoste-Julien, S., & Jordan, M. I. (2005) · Cited by 47
Learning for Optical Flow Using Stochastic Optimization
Li, Y., & Huttenlocher, D. P. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-88688-4_28 · Cited by 42 · cited 2×
“Recent work [1], [19], [20], [34], [35] introduced methods for training the MRF parameters such that the MRF energy minimum is as close as possible to the desired output on a training set.”
MethodLearning real-time MRF inference for image denoising
Barbu, A. (2009) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2009.5206811 · Cited by 33 · cited 1×
“A shorter version of this paper appeared in CVPR [2].”
BackgroundSelf-citation
Cited by
FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising
Zhang, K., Zuo, W., & Zhang, L. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2839891 · Cited by 2,678
“the discriminative architecture is derived from optimization algorithms such as HQS and gradient descent [17], [21]–[24].”
MethodInfluentialDeep Convolutional Neural Network for Inverse Problems in Imaging
Jin, K. H., McCann, M. T., Froustey, E., & Unser, M. (2017) · IEEE Transactions on Image Processing · doi:10.1109/tip.2017.2713099 · Cited by 2,529
“[34] described the usage of iterative gradient descent inferences for maximum a posteriori (MAP) estimation, and the unrolling concept came out in the derivation.”
BackgroundConditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386
“[4] proposed a joint training of a MRF/CRF model together with an inference algorithm in their Active Random Field approach.”
MethodLearning Deep CNN Denoiser Prior for Image Restoration
Zhang, K., Zuo, W., Gu, S., & Zhang, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.300 · Cited by 2,116
“On the contrary, discriminative learning methods try to learn the prior parameters Θ and a compact inference through an optimization of a loss function on a training set containing degraded-clean image pairs [2, 13, 51, 55, 57].”
BackgroundChen, Y., & Pock, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2596743 · Cited by 1,344
Metadata
- Authors
- Barbu, A.
- Affiliations
- Florida State University (US)
- Corresponding author
- Barbu, A.
- ORCID
- Barbu, A.
- Venue
- IEEE Transactions on Image Processing
- Published
- 2009-11
- Volume
- 18
- Issue
- 11
- Pages
- 2451-2462
- ISSN
- 1057-7149 (ISSN-L)
- 1941-0042
- OpenAlex
- W2119290843
- Cited by
- 105
- References
- 49
- Citations by year
- 2026: 2 · 2025: 8 · 2024: 3 · 2023: 6 · 2022: 6 · 2021: 10 · 2020: 13 · 2019: 6 · 2018: 13 · 2017: 7 · 2016: 3 · 2015: 6 · 2014: 5 · 2013: 3 · 2012: 3
- FWCI
- 5.15
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- IEEE Transactions on Image Processing (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Signal Denoising Methods
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Enhancement Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image Processing Techniques
- Keywords
- image denoising, Markov random fields, conditional random fields, real-time image processing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.