Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
Open accessAbstract
Most state-of-the-art techniques for multi-class image segmentation and labeling use conditional random fields defined over pixels or image regions. While region-level models often feature dense pairwise connectivity, pixel-level models are considerably larger and have only permitted sparse graph structures. In this paper, we consider fully connected CRF models defined on the complete set of pixels in an image. The resulting graphs have billions of edges, making traditional inference algorithms impractical. Our main contribution is a highly efficient approximate inference algorithm for fully connected CRF models in which the pairwise edge potentials are defined by a linear combination of Gaussian kernels. Our experiments demonstrate that dense connectivity at the pixel level substantially improves segmentation and labeling accuracy.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The permutohedral lattice implementation allows efficient computation of fully connected CRFs only when pairwise potentials are Gaussian functions.
“First, the permutohedral lattice implementation [45], [50] allows efficient computation of fully connected CRFs only when pairwise potentials are Gaussian functions.”
From DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation (2019) · cites this paper as [45]
Long-distance dependency helps to improve segmentation in most RGB images.
“Second, though long-distance dependency helps to improve segmentation in most RGB images [22], [45], [50]”
From DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation (2019) · cites this paper as [45]
Mean-field approximation is often used for efficient inference of the CRF while allowing learning parameters by back-propagation.
“Alternatively, mean-field approximation [45], [50], [52] is often used for efficient inference of the CRF while allowing learning parameters by back-propagation.”
From DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation (2019) · cites this paper as [45]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cited by
Sorted by citations
Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K., & Yuille, A. L. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2699184 · Cited by 22,392 · see citing works
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631 · see citing works
Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., & Adam, H. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01234-2_49 · Cited by 20,054 · see citing works
Wang, X., Girshick, R., Gupta, A., & He, K. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00813 · Cited by 11,596 · see citing works
Learning Deconvolution Network for Semantic Segmentation
Noh, H., Hong, S., & Han, B. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.178 · Cited by 4,422 · see citing works
Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation
Kamnitsas, K., Ledig, C., Newcombe, V. F. J., Simpson, J. P., Kane, A. D., Menon, D. K., Rueckert, D., & Glocker, B. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.10.004 · Cited by 3,498 · see citing works
“Additionally, we have extended the fully-connected Conditional Random Field (CRF) model by Kr¨ahenb¨uhl and Koltun (2011) to 3D which we use for final post-processing of the CNN’s soft segmentation maps.”
BackgroundMethodInfluentialRefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation
Lin, G., Milan, A., Shen, C., & Reid, I. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.549 · Cited by 3,287 · see citing works
“The atrous convolution based approach DeepLab-CRF in [5] directly output a middleresolution score map then applies the dense CRF method [27] to refine boundaries by leveraging color contrast information.”
MethodRegion-based convolutional networks for accurate object detection and segmentation
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2437384 · Cited by 2,912 · see citing works
Conditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386 · see citing works
“The complete system, therefore, unifies strengths of both CNNs and CRFs and is trainable end-to-end using the back-propagation algorithm [32] and the Stochastic Gradient Descent (SGD) procedure.”
BackgroundMethodInfluentialLarge Kernel Matters — Improve Semantic Segmentation by Global Convolutional Network
Peng, C., Zhang, X., Yu, G., Luo, G., & Sun, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.189 · Cited by 1,750 · see citing works
Unsupervised CNN for Single View Depth Estimation: Geometry to the Rescue
Garg, R., B.G., V. K., Carneiro, G., & Reid, I. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46484-8_45 · Cited by 1,632 · see citing works
Scene Graph Generation by Iterative Message Passing
Xu, D., Zhu, Y., Choy, C. B., & Fei-Fei, L. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.330 · Cited by 1,271 · see citing works
Structural-RNN: Deep Learning on Spatio-Temporal Graphs
Jain, A., Zamir, A. R., Savarese, S., & Saxena, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.573 · Cited by 1,133 · see citing works
Embracing imperfect datasets: A review of deep learning solutions for medical image segmentation
Tajbakhsh, N., Jeyaseelan, L., Li, Q., Chiang, J. N., Wu, Z., & Ding, X. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101693 · Cited by 1,095 · see citing works
“Krähenbühl and Koltun (2011) provides an efficient inference approach to Eq.”
InfluentialWu, B., Wan, A., Yue, X., & Keutzer, K. (2018) · 2018 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2018.8462926 · Cited by 967 · see citing works
Efficient Piecewise Training of Deep Structured Models for Semantic Segmentation
Lin, G., Shen, C., van den Hengel, A., & Reid, I. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.348 · Cited by 862 · see citing works
A deep learning model integrating FCNNs and CRFs for brain tumor segmentation
Zhao, X., Wu, Y., Song, G., Li, Z., Zhang, Y., & Fan, Y. (2018) · Medical Image Analysis · doi:10.1016/j.media.2017.10.002 · Cited by 813 · see citing works
Christ, P. F., Elshaer, M. E. A., Ettlinger, F., Tatavarty, S., Bickel, M., Bilic, P., Rempfler, M., Armbruster, M., Hofmann, F., D’Anastasi, M., et al. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46723-8_48 · Cited by 625 · see citing works
Self-supervised learning for medical image analysis using image context restoration
Chen, L., Bentley, P., Mori, K., Misawa, K., Fujiwara, M., & Rueckert, D. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.101539 · Cited by 575 · see citing works
Algorithms for semantic segmentation of multispectral remote sensing imagery using deep learning
Kemker, R., Salvaggio, C., & Kanan, C. (2018) · ISPRS Journal of Photogrammetry and Remote Sensing · doi:10.1016/j.isprsjprs.2018.04.014 · Cited by 558 · see citing works
DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation
Wang, G., Zuluaga, M. A., Li, W., Pratt, R., Patel, P. A., Aertsen, M., Doel, T., David, A. L., Deprest, J., Ourselin, S., et al. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2840695 · Cited by 502 · see citing works
Laplacian Pyramid Reconstruction and Refinement for Semantic Segmentation
Ghiasi, G., & Fowlkes, C. C. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46487-9_32 · Cited by 502 · see citing works
Online Adaptation of Convolutional Neural Networks for Video Object Segmentation
Voigtlaender, P., & Leibe, B. (2017) · Procedings of the British Machine Vision Conference 2017 · doi:10.5244/c.31.116 · Cited by 443 · see citing works
Pathology image analysis using segmentation deep learning algorithms
Wang, S., Yang, D. M., Rong, R., Zhan, X., & Xiao, G. (2019) · The American Journal of Pathology · doi:10.1016/j.ajpath.2019.05.007 · Cited by 382 · see citing works
Causal Intervention for Weakly-Supervised Semantic Segmentation
Zhang, D., Zhang, H., Tang, J., Hua, X., & Sun, Q. (2020) · arXiv · doi:10.48550/arxiv.2009.12547 · Cited by 353 · see citing works
Learning Depth with Convolutional Spatial Propagation Network
Cheng, X., Wang, P., & Yang, R. (2018) · arXiv · doi:10.48550/arxiv.1810.02695 · Cited by 345 · see citing works
3D Deeply Supervised Network for Automatic Liver Segmentation from CT Volumes
Dou, Q., Chen, H., Jin, Y., Yu, L., Qin, J., & Heng, P.-A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46723-8_18 · Cited by 344 · see citing works
Pixel-Adaptive Convolutional Neural Networks
Su, H., Jampani, V., Sun, D., Gallo, O., Learned-Miller, E., & Kautz, J. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01142 · Cited by 348 · see citing works
DeepMedic for Brain Tumor Segmentation
Kamnitsas, K., Ferrante, E., Parisot, S., Ledig, C., Nori, A. V., Criminisi, A., Rueckert, D., & Glocker, B. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-55524-9_14 · Cited by 346 · see citing works
Automatic breast ultrasound image segmentation: a survey
Xian, M., Zhang, Y., Cheng, H. D., Xu, F., Zhang, B., & Ding, J. (2018) · Pattern Recognition · doi:10.1016/j.patcog.2018.02.012 · Cited by 284 · see citing works
Iterative Visual Reasoning Beyond Convolutions
Chen, X., Li, L.-J., Fei-Fei, L., & Gupta, A. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00756 · Cited by 236 · see citing works
Joint Semantic Segmentation and 3D Reconstruction from Monocular Video
Kundu, A., Li, Y., Dellaert, F., Li, F., & Rehg, J. M. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10599-4_45 · Cited by 220 · see citing works
Robust guided image filtering using Nonconvex potentials
Ham, B., Cho, M., & Ponce, J. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2017.2669034 · Cited by 219 · see citing works
Incremental dense semantic stereo fusion for large-scale semantic scene reconstruction
Vineet, V., Miksik, O., Lidegaard, M., Niebner, M., Golodetz, S., Prisacariu, V. A., Kahler, O., Murray, D. W., Izadi, S., Peerez, P., et al. (2015) · 2015 IEEE International Conference on Robotics and Automation (ICRA) · doi:10.1109/icra.2015.7138983 · Cited by 200 · see citing works
Dual-force convolutional neural networks for accurate brain tumor segmentation
Chen, S., Ding, C., & Liu, M. (2019) · Pattern Recognition · doi:10.1016/j.patcog.2018.11.009 · Cited by 195 · see citing works
Hu, K., Gan, Q., Zhang, Y., Deng, S., Xiao, F., Huang, W., Cao, C., & Gao, X. (2019) · IEEE Access · doi:10.1109/access.2019.2927433 · Cited by 188 · see citing works
PiCIE: Unsupervised Semantic Segmentation using Invariance and Equivariance in Clustering
Hyun Cho, J., Mall, U., Bala, K., & Hariharan, B. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01652 · Cited by 159 · see citing works
Wang, X., Girshick, R., Gupta, A., & He, K. (2017) · arXiv · Cited by 11,689 · see citing works
Gaussian Conditional Random Field Network for Semantic Segmentation
Vemulapalli, R., Tuzel, O., Liu, M.-Y., & Chellappa, R. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.351 · Cited by 150 · see citing works
Iterative Instance Segmentation
Li, K., Hariharan, B., & Malik, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.398 · Cited by 152 · see citing works
Segmentation-Aware Convolutional Networks Using Local Attention Masks
Harley, A. W., Derpanis, K. G., & Kokkinos, I. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.539 · Cited by 142 · see citing works
Recurrent Scene Parsing with Perspective Understanding in the Loop
Kong, S., & Fowlkes, C. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00106 · Cited by 114 · see citing works
Towards automated semantic segmentation in prenatal volumetric ultrasound
Yang, X., Yu, L., Li, S., Wen, H., Luo, D., Bian, C., Qin, J., Ni, D., & Heng, P.-A. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2858779 · Cited by 115 · see citing works
Fast Fully Automatic Segmentation of the Human Placenta from Motion Corrupted MRI
Alansary, A., Kamnitsas, K., Davidson, A., Khlebnikov, R., Rajchl, M., Malamateniou, C., Rutherford, M., Hajnal, J. V., Glocker, B., Rueckert, D., et al. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46723-8_68 · Cited by 77 · see citing works
Visual Semantic Information Pursuit: A Survey
Liu, D., Bober, M., & Kittler, J. (2021) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2950025 · Cited by 39 · see citing works
Neural Batch Sampling with Reinforcement Learning for Semi-supervised Anomaly Detection
Chu, W.-H., & Kitani, K. M. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58574-7_45 · Cited by 34 · see citing works
Semantic labeling for prosthetic vision
Horne, L., Alvarez, J., McCarthy, C., Salzmann, M., & Barnes, N. (2016) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2016.02.015 · Cited by 30 · see citing works
Lee, D., Lee, W. H., Seo, H. G., Oh, B.-M., Lee, J. C., & Kim, H. C. (2020) · IEEE Access · doi:10.1109/access.2020.3019532 · Cited by 22 · see citing works
Recognizing object affordances in terms of spatio-temporal object-object relationships
Pieropan, A., Ek, C. H., & Kjellstrom, H. (2014) · 2014 IEEE-RAS International Conference on Humanoid Robots · doi:10.1109/humanoids.2014.7041337 · Cited by 16 · see citing works
rNCA: Self-Repairing Segmentation Masks
Silbernagel, M., Alonso, A., Petersen, J., Ibragimov, B., de Bruijne, M., & Wyburd, M. K. (2025) · arXiv · doi:10.48550/arxiv.2512.13397
“Wecompare rNCA to three popular segmentation refinement methods: Seg-Fix [37],SegRefiner [40] and DenserCRF [46].”
- Authors
- Krähenbühl, P., & Koltun, V.
- Affiliations
- Stanford University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2012
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1210.5644
- OpenAlex
- W2161236525
- Cited by
- 2,908
- References
- 0
- Citations by year
- 2026: 1 · 2025: 85 · 2024: 177 · 2023: 218 · 2022: 259 · 2021: 336 · 2020: 367 · 2019: 424 · 2018: 369 · 2017: 264 · 2016: 198 · 2015: 95 · 2014: 56 · 2013: 39 · 2012: 19
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Bayesian Modeling and Causal Inference
- Keywords
- approximate inference, pair potential, multi-class image segmentation, image annotation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
