The DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
What is a good evaluation measure for semantic segmentation?
Abstract
In this work, we consider the evaluation of the semantic segmentation task. We discuss the strengths and limitations of the few existing measures, and propose new ways to evaluate semantic segmentation. First, we argue that a per-image score instead of one computed over the entire dataset brings a lot more insight. Second, we propose to take contours more carefully into account. Based on the conducted experiments, we suggest best practices for the evaluation. Finally, we present a user study we conducted to better understand how the quality of image segmentations is perceived by humans.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Mean shift: a robust approach toward feature space analysis
Comaniciu, D., & Meer, P. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.1000236 · Cited by 11,305
The Pascal Visual Object Classes Challenge: A Retrospective
Everingham, M., Eslami, S. M. A., Van Gool, L., Williams, C. K. I., Winn, J., & Zisserman, A. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-014-0733-5 · Cited by 7,139
An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision
Boykov, Y., & Kolmogorov, V. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.60 · Cited by 4,577
What energy functions can be minimized via graph cuts?
Kolmogorov, V., & Zabih, R. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.1262177 · Cited by 3,127
Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials
Krähenbühl, P., & Koltun, V. (2012) · arXiv · doi:10.48550/arxiv.1210.5644 · Cited by 2,908
Learning to detect natural image boundaries using local brightness, color, and texture cues
Martin, D. R., Fowlkes, C. C., & Malik, J. (2004) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2004.1273918 · Cited by 2,420
Fisher Kernels on Visual Vocabularies for Image Categorization
Perronnin, F., & Dance, C. (2007) · 2007 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2007.383266 · Cited by 1,641
Self-citationShotton, J., Winn, J., Rother, C., & Criminisi, A. (2006) · Lecture Notes in Computer Science · doi:10.1007/11744023_1 · Cited by 1,132
Robust Higher Order Potentials for Enforcing Label Consistency
Kohli, P., Ladický, L., & Torr, P. H. S. (2009) · International Journal of Computer Vision · doi:10.1007/s11263-008-0202-0 · Cited by 853
Toward Objective Evaluation of Image Segmentation Algorithms
Unnikrishnan, R., Pantofaru, C., & Hebert, M. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1046 · Cited by 792
Cited by
SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2644615 · Cited by 19,631
A Benchmark Dataset and Evaluation Methodology for Video Object Segmentation
Perazzi, F., Pont-Tuset, J., McWilliams, B., Van Gool, L., Gross, M., & Sorkine-Hornung, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.85 · Cited by 1,989
Enhanced-alignment Measure for Binary Foreground Map Evaluation
Fan, D.-P., Gong, C., Cao, Y., Ren, B., Cheng, M.-M., & Borji, A. (2018) · Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2018/97 · Cited by 1,607
Convolutional neural networks for large-scale remote-sensing image classification
Maggiori, E., Tarabalka, Y., Charpiat, G., & Alliez, P. (2017) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2016.2612821 · Cited by 1,114
Berman, M., Triki, A. R., & Blaschko, M. B. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00464 · Cited by 1,027
Boundary IoU: Improving Object-Centric Image Segmentation Evaluation
Cheng, B., Girshick, R., Dollar, P., Berg, A. C., & Kirillov, A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01508 · Cited by 495
Deep learning approaches to biomedical image segmentation
Rizwan I Haque, I., & Neubert, J. (2020) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2020.100297 · Cited by 456
Liang, X. (2019) · Computer-Aided Civil and Infrastructure Engineering · doi:10.1111/mice.12425 · Cited by 398
A comprehensive survey of loss functions and metrics in deep learning
Terven, J., Cordova-Esparza, D.-M., Romero-González, J.-A., Ramírez-Pedraza, A., & Chávez-Urbiola, E. A. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11198-7 · Cited by 343
Sa, I., Popović, M., Khanna, R., Chen, Z., Lottes, P., Liebisch, F., Nieto, J., Stachniss, C., Walter, A., & Siegwart, R. (2018) · Remote Sensing · doi:10.3390/rs10091423 · Cited by 305
Incremental Learning Techniques for Semantic Segmentation
Michieli, U., & Zanuttigh, P. (2019) · 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW) · doi:10.1109/iccvw.2019.00400 · Cited by 256
Alleviating Over-segmentation Errors by Detecting Action Boundaries
Ishikawa, Y., Kasai, S., Aoki, Y., & Kataoka, H. (2021) · 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv48630.2021.00237 · Cited by 171
Federated learning for healthcare applications
Chaddad, A., Wu, Y., & Desrosiers, C. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3325822 · Cited by 136
Mahmoudi, T., Kouzahkanan, Z. M., Radmard, A. R., Kafieh, R., Salehnia, A., Davarpanah, A. H., Arabalibeik, H., & Ahmadian, A. (2022) · Scientific Reports · doi:10.1038/s41598-022-07111-9 · Cited by 81
Semantic labeling for prosthetic vision
Horne, L., Alvarez, J., McCarthy, C., Salzmann, M., & Barnes, N. (2016) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2016.02.015 · Cited by 30
Zhang, J., Wang, Y., Donarski, E. D., Toma, T. T., Miles, M. T., Acton, S. T., & Gahlmann, A. (2022) · npj Biofilms and Microbiomes · doi:10.1038/s41522-022-00362-4 · Cited by 22
Ding, Y., Dhawan, G., Jones, C., Ness, T., Nichols, E., Krasnogor, N., & Reynolds, N. J. (2023) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.18726 · Cited by 9
Metadata
- Authors
- Csurka, G., Larlus, D., & Perronnin, F.
- Venue
- Procedings of the British Machine Vision Conference 2013
- Publisher
- British Machine Vision Association
- Published
- 2013
- Pages
- 32.1-32.11
- OpenAlex
- W2091695913
- Cited by
- 500
- References
- 22
- FWCI
- 2.35
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.