Robust guided image filtering using Nonconvex potentials
Open accessAbstract
Filtering images using a guidance signal, a process called guided or joint image filtering, has been used in various tasks in computer vision and computational photography, particularly for noise reduction and joint upsampling. This uses an additional guidance signal as a structure prior, and transfers the structure of the guidance signal to an input image, restoring noisy or altered image structure. The main drawbacks of such a data-dependent framework are that it does not consider structural differences between guidance and input images, and that it is not robust to outliers. We propose a novel SD (for static/dynamic) filter to address these problems in a unified framework, and jointly leverage structural information from guidance and input images. Guided image filtering is formulated as a nonconvex optimization problem, which is solved by the majorize-minimization algorithm. The proposed algorithm converges quickly while guaranteeing a local minimum. The SD filter effectively controls the underlying image structure at different scales, and can handle a variety of types of data from different sensors. It is robust to outliers and other artifacts such as gradient reversal and global intensity shift, and has good edge-preserving smoothing properties. We demonstrate the flexibility and effectiveness of the proposed SD filter in a variety of applications, including depth upsampling, scale-space filtering, texture removal, flash/non-flash denoising, and RGB/NIR denoising.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Cites
Scale-space and edge detection using anisotropic diffusion
Perona, P., & Malik, J. (1990) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.56205 · Cited by 12,143 · see citing works
Martin, D., Fowlkes, C., Tal, D., & Malik, J. (2002) · Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001 · doi:10.1109/iccv.2001.937655 · Cited by 8,216 · see citing works
Bilateral Filtering for Gray and Color Images
Tomasi, C., & Manduchi, R. (2002) · Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271) · doi:10.1109/iccv.1998.710815 · Cited by 8,177 · cited 4× · see citing works
“They regularize images through a weighted averaging process [2], [22], a weighted median process [11], or a weighted mode process [25].”
InfluentialA Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms
Scharstein, D., & Szeliski, R. (2002) · International Journal of Computer Vision · doi:10.1023/a:1014573219977 · Cited by 6,740 · see citing works
He, K., Sun, J., & Tang, X. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.213 · Cited by 5,503 · see citing works
Contour Detection and Hierarchical Image Segmentation
Arbeláez, P., Maire, M., Fowlkes, C. C., & Malik, J. (2010) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2010.161 · Cited by 5,472 · see citing works
The EM Algorithm and Extensions
Kushary, D., McLachlan, G. J., & Krishnan, T. (1998) · Technometrics · doi:10.2307/1271189 · Cited by 5,419 · see citing works
Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2014) · arXiv · doi:10.48550/arxiv.1412.7062 · Cited by 4,213 · see citing works
Robust Statistics: The Approach Based on Influence Functions
Ruppert, D. (1987) · Technometrics · doi:10.2307/1269782 · Cited by 3,880 · see citing works
On the Convergence Properties of the EM Algorithm
Wu, C. (1983) · The Annals of Statistics · doi:10.1214/aos/1176346060 · Cited by 3,275 · see citing works
Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
Krähenbühl, P., & Koltun, V. (2012) · arXiv · doi:10.48550/arxiv.1210.5644 · Cited by 2,908 · see citing works
Conditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386 · cited 1× · see citing works
“, in semantic segmentation [34], [35].”
Robust regression using iteratively reweighted least-squares.
Holland, P. W., & Welsch, R. E. (1977) · Communications in Statistics - Theory and Methods · doi:10.1080/03610927708827533 · Cited by 2,115 · see citing works
Hunter, D. R., & Lange, K. L. (2004) · The American Statistician · doi:10.1198/0003130042836 · Cited by 1,789 · see citing works
Robust Statistics: The Approach Based on Influence Functions.
Jennison, C., Hampel, F. R., Ronchetti, E. M., Rousseeuw, P. J., & Stahel, W. A. (1987) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2981480 · Cited by 1,753 · see citing works
Correntropy: Properties and Applications in Non-Gaussian Signal Processing
Liu, W., Pokharel, P. P., & Príncipe, J. C. (2007) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2007.896065 · Cited by 1,720 · see citing works
Fast bilateral filtering for the display of high-dynamic-range images
Durand, F., & Dorsey, J. O. (2002) · ACM SIGGRAPH Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH) · doi:10.1145/566570.566574 · Cited by 1,557 · see citing works
Edge-preserving decompositions for multi-scale tone and detail manipulation
Farbman, Z., Fattal, R., Lischinski, D., et al. (2008) · ACM Transactions on Graphics · doi:10.1145/1360612.1360666 · Cited by 1,469 · see citing works
Colorization using optimization
Levin, A., Lischinski, D., & Weiss, Y. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015780 · Cited by 1,446 · see citing works
Deterministic edge-preserving regularization in computed imaging
Charbonnier, P., Blanc‐Féraud, L., Aubert, G., & Barlaud, M. (1997) · IEEE Transactions on Image Processing · doi:10.1109/83.551699 · Cited by 1,303 · see citing works
Iteratively reweighted least squares minimization for sparse recovery
Daubechies, I. C., DeVore, R., Fornasier, M., & Güntürk, C. S. (2009) · Communications on Pure and Applied Mathematics · doi:10.1002/cpa.20303 · Cited by 1,295 · see citing works
Black, M. J., Sapiro, G., Marimont, D. H., & Heeger, D. J. (1998) · IEEE Transactions on Image Processing · doi:10.1109/83.661192 · Cited by 1,294 · see citing works
Fast Random Walk with Restart and Its Applications
Tong, H., Faloutsos, C., & Pan, J.-Y. (2006) · Proceedings · doi:10.1109/icdm.2006.70 · Cited by 1,218 · see citing works
Adaptive support-weight approach for correspondence search
Yoon, K., & Kweon, I. S. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.70 · Cited by 1,209 · see citing works
Constrained restoration and the recovery of discontinuities
Geman, D., & Reynolds, G. (1992) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.120331 · Cited by 1,100 · see citing works
Digital photography with flash and no-flash image pairs
Petschnigg, G., Szeliski, R., Agrawala, M., Cohen, M., Hoppe, H., & Toyama, K. (2004) · ACM Transactions on Graphics · doi:10.1145/1015706.1015777 · Cited by 1,062 · see citing works
Fast bilateral filtering for the display of high-dynamic-range images
Durand, F., & Dorsey, J. O. (2002) · ACM Transactions on Graphics · doi:10.1145/566654.566574 · Cited by 1,008 · see citing works
Kopf, J., Cohen, M. F., Lischinski, D., & Uyttendaele, M. (2007) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/1275808.1276497 · Cited by 891 · see citing works
Image smoothing via L 0 gradient minimization
Xu, L., Lu, C., Xu, Y., & Jia, J. (2011) · ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive Techniques in Asia (SIGGRAPH Asia) · doi:10.1145/2024156.2024208 · Cited by 886 · see citing works
Structure extraction from texture via relative total variation
Xu, L., Yan, Q., Xia, Y., & Jia, J. (2012) · ACM Transactions on Graphics · doi:10.1145/2366145.2366158 · Cited by 868 · see citing works
Robust Higher Order Potentials for Enforcing Label Consistency
Kohli, P., Ladický, Ľ., & Torr, P. H. S. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-008-0202-0 · Cited by 858 · see citing works
Kopf, J., Cohen, M. F., Lischinski, D., & Uyttendaele, M. (2007) · ACM Transactions on Graphics · doi:10.1145/1276377.1276497 · Cited by 844 · see citing works
Image smoothing via L 0 gradient minimization
Li, X., Lu, C., Xu, Y., & Jia, J. (2011) · ACM Transactions on Graphics · doi:10.1145/2070781.2024208 · Cited by 786 · see citing works
EpicFlow: Edge-preserving interpolation of correspondences for optical flow
Revaud, J., Weinzaepfel, P., Harchaoui, Z., & Schmid, C. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298720 · Cited by 757 · see citing works
Chen, Y., Davis, T. A., Hager, W. W., & Rajamanickam, S. (2008) · ACM Transactions on Mathematical Software · doi:10.1145/1391989.1391995 · Cited by 754 · see citing works
Segmentation as selective search for object recognition
van de Sande, K. E. A., Uijlings, J. R. R., Gevers, T., & Smeulders, A. W. M. (2011) · 2011 International Conference on Computer Vision · doi:10.1109/iccv.2011.6126456 · Cited by 739 · see citing works
Fast Cost-Volume Filtering for Visual Correspondence and Beyond
Hosni, A., Rhemann, C., Bleyer, M., Rother, C., & Gelautz, M. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.156 · Cited by 676 · see citing works
Learning hatching for pen-and-ink illustration of surfaces
Kalogerakis, E., Nowrouzezahrai, D., Breslav, S., & Hertzmann, A. (2012) · ACM Transactions on Graphics · doi:10.1145/2077341.2077342 · Cited by 653 · see citing works
Zhang, Q., Shen, X., Li, X., & Jia, J. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-10578-9_53 · Cited by 619 · see citing works
Image Guided Depth Upsampling Using Anisotropic Total Generalized Variation
Ferstl, D., Reinbacher, C., Ranftl, R., Rüther, M., & Bischof, H. (2013) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.127 · Cited by 535 · see citing works
A review of nonlinear diffusion filtering
Weickert, J. (1997) · Lecture notes in computer science · doi:10.1007/3-540-63167-4_37 · Cited by 530 · see citing works
High quality depth map upsampling for 3D-TOF cameras
Park, J., Kim, H., Tai, Y., Brown, M. S., & Kweon, I.-S. (2011) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2011.6126423 · Cited by 508 · see citing works
Analysis of Half-Quadratic Minimization Methods for Signal and Image Recovery
Nikolova, M., & Ng, M. K. (2005) · SIAM Journal on Scientific Computing · doi:10.1137/030600862 · Cited by 488 · see citing works
Automatic multimedia cross-modal correlation discovery
Pan, J.-Y., Yang, H.-J., Faloutsos, C., & Duygulu, P. (2004) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1014052.1014135 · Cited by 484 · see citing works
Chen, L.-C., Barron, J. T., Papandreou, G., Murphy, K. P., & Yuille, A. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.492 · Cited by 377 · see citing works
Fast Global Image Smoothing Based on Weighted Least Squares
Min, D., Choi, S., Lu, J., Ham, B., Sohn, K., & Do, M. N. (2014) · IEEE Transactions on Image Processing · doi:10.1109/tip.2014.2366600 · Cited by 368 · see citing works
Self-citationBarron, J. T., & Poole, B. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46487-9_38 · Cited by 339 · see citing works
Constant Time Weighted Median Filtering for Stereo Matching and Beyond
Ma, Z., He, K., Wei, Y., et al. (2013) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.13 · Cited by 295 · see citing works
Li, Y., Huang, J., Ahuja, N., & Yang, M. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46493-0_10 · Cited by 282 · see citing works
Joint Geodesic Upsampling of Depth Images
Liu, M.-Y., Tuzel, O., & Taguchi, Y. (2013) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2013.29 · Cited by 248 · see citing works
A Noise‐aware Filter for Real‐time Depth Upsampling
Chan, D. Y., Buisman, H., Theobalt, C., et al. (2008) · Cited by 247 · see citing works
Depth Video Enhancement Based on Weighted Mode Filtering
Min, D., Lu, J., & Do, M. N. (2011) · IEEE Transactions on Image Processing · doi:10.1109/tip.2011.2163164 · Cited by 240 · see citing works
Efficient Nonlocal Means for Denoising of Textural Patterns
Brox, T., Kleinschmidt, O., & Cremers, D. (2008) · IEEE Transactions on Image Processing · doi:10.1109/tip.2008.924281 · Cited by 234 · see citing works
Incremental Majorization-Minimization Optimization with Application to Large-Scale Machine Learning
Mairal, J. (2015) · SIAM Journal on Optimization · doi:10.1137/140957639 · Cited by 232 · see citing works
Structure-preserving image smoothing via region covariances
Karacan, L., Erdem, E., & Erdem, A. (2013) · ACM Transactions on Graphics · doi:10.1145/2508363.2508403 · Cited by 216 · see citing works
Patch Based Synthesis for Single Depth Image Super-Resolution
Aodha, O. M., Campbell, N. D. F., Nair, A. A., & Brostow, G. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33712-3_6 · Cited by 208 · see citing works
Robust higher order potentials for enforcing label consistency
Kohli, P., Ladický, Ľ., & Torr, P. H. S. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587417 · Cited by 203 · see citing works
Crisp Boundary Detection Using Pointwise Mutual Information
Isola, P., Zoran, D., Krishnan, D., & Adelson, E. H. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10578-9_52 · Cited by 199 · see citing works
FlowWeb: Joint image set alignment by weaving consistent, pixel-wise correspondences
Zhou, T., Lee, Y. J., Yu, S. X., & Efros, A. A. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298723 · Cited by 155 · see citing works
Robust image filtering using joint static and dynamic guidance
Ham, B., Cho, M., & Ponce, J. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299115 · Cited by 148 · see citing works
Self-citationGenerative Image Segmentation Using Random Walks with Restart
Kim, T. H., Lee, K. M., & Lee, S. U. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-88690-7_20 · Cited by 143 · see citing works
Practical temporal consistency for image-based graphics applications
Lang, M., Wang, O., Aydın, T. O., Smolić, A., & Groß, M. H. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2185530 · Cited by 132 · see citing works
The Equivalence of Half-Quadratic Minimization and the Gradient Linearization Iteration
Nikolova, M., & Chan, R. H. (2007) · IEEE Transactions on Image Processing · doi:10.1109/tip.2007.896622 · Cited by 126 · see citing works
Cross-Field Joint Image Restoration via Scale Map
Yan, Q., Shen, X., Xu, L., et al. (2013) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2013.194 · Cited by 123 · see citing works
Mutual-Structure for Joint Filtering
Shen, X., Zhou, C., Li, X., et al. (2015) · IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.389 · Cited by 113 · see citing works
Ham, B., Cho, M., Schmid, C., & Ponce, J. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.378 · Cited by 106 · see citing works
Self-citationOpen questions concerning Weiszfeld's algorithm for the Fermat-Weber location problem
Chandrasekaran, R., & Tamir, A. (1989) · Mathematical Programming · doi:10.1007/bf01587094 · Cited by 103 · see citing works
Ham, B., Cho, M., Schmid, C., & Ponce, J. (2016) · Cited by 67 · see citing works
Self-citationPractical temporal consistency for image-based graphics applications
Lang, M., Wang, O., Aydın, T. O., et al. (2012) · ACM Transactions on Graphics · doi:10.1145/2185520.2335385 · Cited by 56 · see citing works
Surrogate maximization/minimization algorithms for AdaBoost and the logistic regression model
Zhang, Z., Kwok, J. T.-Y., & Yeung, D. (2004) · doi:10.1145/1015330.1015342 · Cited by 29 · see citing works
Ham, B., Min, D., & Sohn, K. (2013) · IEEE Transactions on Image Processing · doi:10.1109/tip.2013.2253479 · Cited by 27 · see citing works
Self-citationFast Edge-Aware Processing via First Order Proximal Approximation
Badri, H., Yahia, H. M., & Aboutajdine, D. (2015) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2015.2396064 · Cited by 27 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Asadpour, V., Puttock, E. J., Getahun, D., Fassett, M. J., & Xie, F. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e13577 · Cited by 11 · see citing works
- Authors
- Ham, B., Cho, M., & Ponce, J.
- Affiliations
- Yonsei University (KR)
- Pohang University of Science and Technology (KR)
- Centre National de la Recherche Scientifique (FR)
- Venue
- IEEE Transactions on Pattern Analysis and Machine Intelligence
- Published
- 2018-01-01
- Volume
- 40
- Issue
- 1
- Pages
- 192-207
- ISSN
- 0162-8828 (ISSN-L)
- 2160-9292
- OpenAlex
- W2291143146
- Cited by
- 219
- References
- 72
- Citations by year
- 2026: 8 · 2025: 21 · 2024: 28 · 2023: 33 · 2022: 16 · 2021: 31 · 2020: 25 · 2019: 26 · 2018: 26 · 2017: 5
- FWCI
- 9.31
- Type
- journal article
- Language
- en
- Open access
- green
- hal.science
- Locations
- IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
- PubMed (publishedVersion)
- HAL (submittedVersion, other-oa)
- Open Access System for Information Sharing (Pohang University of Science and Technology) (submittedVersion)
- Funders
- Institute for Information & communications Technology Promotion (IITP)
- Korea government (MSIP) (2016-0- 00197)
- Development of the high-precision natural 3D view generation technology using smart-car multi sensors and deep learning
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Signal Denoising Methods
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image Enhancement Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Image Processing Techniques
- Keywords
- guided filter, non-convex optimization, outlier robustness, image denoising, scale-space filtering, texture removal, edge-preserving smoothing, structure prior
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
