The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
What is a good evaluation measure for semantic segmentation?
Abstract
In this work, we consider the evaluation of the semantic segmentation task. We discuss the strengths and limitations of the few existing measures, and propose new ways to evaluate semantic segmentation. First, we argue that a per-image score instead of one computed over the entire dataset brings a lot more insight. Second, we propose to take contours more carefully into account. Based on the conducted experiments, we suggest best practices for the evaluation. Finally, we present a user study we conducted to better understand how the quality of image segmentations is perceived by humans.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mean shift: a robust approach toward feature space analysis
Comaniciu, D., & Meer, P. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.1000236 · Cited by 11,305
The Pascal Visual Object Classes Challenge: A Retrospective
Everingham, M., Eslami, S. M. A., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0733-5 · Cited by 7,139
An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision
Boykov, Y., & Kolmogorov, V. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.60 · Cited by 4,577
What energy functions can be minimized via graph cuts?
Kolmogorov, V., & Zabih, R. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.1262177 · Cited by 3,127
Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
Krähenbühl, P., & Koltun, V. (2012) · arXiv · doi:10.48550/arxiv.1210.5644 · Cited by 2,908
Learning to detect natural image boundaries using local brightness, color, and texture cues
Martin, D. R., Fowlkes, C. C., & Malik, J. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.1273918 · Cited by 2,420
Fisher Kernels on Visual Vocabularies for Image Categorization
Perronnin, F., & Dance, C. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383266 · Cited by 1,641
Self-citationShotton, J., Winn, J., Rother, C., & Criminisi, A. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744023_1 · Cited by 1,132
Robust Higher Order Potentials for Enforcing Label Consistency
Kohli, P., Ladický, L., & Torr, P. H. S. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-008-0202-0 · Cited by 853
Toward Objective Evaluation of Image Segmentation Algorithms
Unnikrishnan, R., Pantofaru, C., & Hebert, M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1046 · Cited by 792
Associative hierarchical CRFs for object class image segmentation
Ladicky, L., Russell, C., Kohli, P., & Torr, P. H. S. (2009) · 2009 IEEE 12th International Conference on Computer Vision · doi:10.1109/iccv.2009.5459248 · Cited by 596
Yet Another Survey on Image Segmentation: Region and Boundary Information Integration
Freixenet, J., Muñoz, X., Raba, D., Martí, J., & Cufí, X. (2002) · Lecture Notes in Computer Science · doi:10.1007/3-540-47977-5_27 · Cited by 519
Li, L.-J., Socher, R., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206718 · Cited by 484
Edge detection with embedded confidence
Meer, P., & Georgescu, B. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.977560 · Cited by 377
What, Where and How Many? Combining Object Detectors and CRFs
Ladický, Ľ., Sturgess, P., Alahari, K., Russell, C., & Torr, P. H. S. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-15561-1_31 · Cited by 310
Efficient inference for fully-connected CRFs with stationarity
Zhang, Y., & Chen, T. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247724 · Cited by 105
Get Out of my Picture! Internet-based Inpainting
Whyte, O., Šivic, J., & Zisserman, A. (2009) · British Machine Vision Conference (BMVC) · doi:10.5244/c.23.116 · Cited by 102
An Efficient Approach to Semantic Segmentation
Csurka, G., & Perronnin, F. (2010) · International Journal of Computer Vision · doi:10.1007/s11263-010-0344-8 · Cited by 98
Self-citationLarlus, D., Verbeek, J. J., & Jurie, F. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-009-0245-x · Cited by 50
Self-citationSegmentation over Detection by Coupled Global and Local Sparse Representations
Xia, W., Zheng, S., Feng, J., Cheong, L., & Yan, S. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-33715-4_48 · Cited by 18
Segmentation Based Interest Points and Evaluation of Unsupervised Image Segmentation Methods
Koniusz, P., & Mikolajczyk, K. (2009) · British Machine Vision Conference (BMVC) · doi:10.5244/c.23.24 · Cited by 18
On the use of regions for semantic image segmentation
Hu, R. S., Larlus, D., & Csurka, G. (2012) · Indian Conference on Computer Vision, Graphics and Image Processing · doi:10.1145/2425333.2425384 · Cited by 5
Self-citation
Cited by
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation
Perazzi, F., Pont-Tuset, J., McWilliams, B., Van Gool, L., Gross, M., & Sorkine-Hornung, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.85 · Cited by 1,989
Enhanced-alignment Measure for Binary Foreground Map Evaluation
Fan, D.-P., Gong, C., Cao, Y., Ren, B., Cheng, M.-M., & Borji, A. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/97 · Cited by 1,607
Convolutional neural networks for large-scale remote-sensing image classification
Maggiori, E., Tarabalka, Y., Charpiat, G., & Alliez, P. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2016.2612821 · Cited by 1,114
Berman, M., Triki, A. R., & Blaschko, M. B. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00464 · Cited by 1,027
Metadata
- Authors
- Csurka, G., Larlus, D., & Perronnin, F.
- Venue
- Procedings of the British Machine Vision Conference 2013
- Publisher
- British Machine Vision Association
- Published
- 2013
- Pages
- 32.1-32.11
- OpenAlex
- W2091695913
- Cited by
- 500
- References
- 22
- FWCI
- 2.35
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.