Unsupervised Performance Evaluation of Image Segmentation
Open accessAbstract
We present in this paper a study of unsupervised evaluation criteria that enable the quantification of the quality of an image segmentation result. These evaluation criteria compute some statistics for each region or class in a segmentation result. Such an evaluation criterion can be useful for different applications: the comparison of segmentation results, the automatic choice of the best fitted parameters of a segmentation method for a given image, or the definition of new segmentation methods by optimization. We first present the state of art of unsupervised evaluation, and then, we compare six unsupervised evaluation criteria. For this comparative study, we use a database composed of 8400 synthetic gray-level images segmented in four different ways. Vinet's measure (correct classification rate) is used as an objective criterion to compare the behavior of the different criteria. Finally, we present the experimental results on the segmentation evaluation of a few gray-level natural images.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The application of unsupervised methods in medical settings is unclear.
“While unsupervised methods can be applied to scenarios where the main purpose of segmentation is to yield visually consistent results that are meaningful to a human observer, the application in medical settings is unclear.”
From Reverse Classification Accuracy: Predicting Segmentation Performance in the Absence of Ground Truth (2017) · cites this paper as [24]
The Introduction says supervised evaluation criteria are widely used in medical applications, while Section 2 describes unsupervised criteria in relation to human perception. The paper’s real-image examples are aerial and radar images (Section 4). None of these passages addresses whether unsupervised methods can be applied in medical settings or says that their application there is unclear. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Some Methods for Strengthening the Common χ 2 Tests
Cochran, W. G. (1954) · Biometrics · doi:10.2307/3001616 · Cited by 3,718 · see citing works
Flagged first, then by citations
Research works printed but not linked: 24
- J. Freixenet, X. Muñoz, D. Raba, J. Marti, and X. Cufi, "Yet another survey on image segmentation: region and boundary information integration," in Proceedings of the European Con- ference on Computer Vision (ECCV '02), pp. 408-422, Copen- hagen, Denmark, May 2002.
- R. M. Haralick and L. G. Shapiro, "Image segmentation techniques," Computer Vision, Graphics, & Image Processing, vol. 29, no. 1, pp. 100-132, 1985.
- Y. J. Zhang, "A survey on evaluation methods for image seg- mentation," Pattern Recognition, vol. 29, no. 8, pp. 1335-1346, 1996.
- N. M. Nasab, M. Analoui, and E. J. Delp, "Robust and efficient image segmentation approaches using Markov random field models," Journal of Electronic Imaging, vol. 12, no. 1, pp. 50- 58, 2003.
- A. J. Baddeley, "An error metric for binary images," in Robust Computer Vision, pp. 59-78, Wichmann, Karlsruhe, Germany, 1992.
- L. Vinet, Segmentation et mise en correspondance de régions de paires d'images stéréoscopiques, Ph.D. thesis, Université de Paris IX Dauphine, Paris, France, 1991.
- D. P. Huttenlocher and W. J. Rucklidge, "Multi-resolution technique for comparing images using the Hausdorff dis- tance," in Proceedings of IEEE Computer Vision and Pattern Recognition (CVPR '93), pp. 705-706, New York, NY, USA, June 1993.
- D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A database of hu- man segmented natural images and its application to evaluat- ing segmentation algorithms and measuring ecological statis- tics," in Proceedings of the IEEE International Conference on Computer Vision (ICCV '01), vol. 2, pp. 416-423, Vancouver, BC, Canada, July 2001.
- J. S. Weszka and A. Rosenfeld, "Threshold evaluation tech- niques," IEEE Transactions on Systems, Man and Cybernetics, vol. 8, no. 8, pp. 622-629, 1978.
- M. D. Levine and A. M. Nazif, "Dynamic measurement of computer generated image segmentations," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 7, no. 2, pp. 155-164, 1985.
- M. Sezgin and B. Sankur, "Survey over image thresholding techniques and quantitative performance evaluation," Journal of Electronic Imaging, vol. 13, no. 1, pp. 146-168, 2004.
- N. R. Pal and S. K. Pal, "Entropic thresholding," Signal Process- ing, vol. 16, no. 2, pp. 97-108, 1989.
- C. Rosenberger, Mise en oeuvre d'un système adaptatif de segmentation d'images, Ph.D. thesis, Université de Rennes 1, Rennes, France, 1999.
- M. Borsotti, P. Campadelli, and R. Schettini, "Quantita- tive evaluation of color image segmentation results," Pattern Recognition Letters, vol. 19, no. 8, pp. 741-747, 1998.
- J. Liu and Y.-H. Yang, "Multiresolution color image segmen- tation," IEEE Transactions on Pattern Analysis and Machine In- telligence, vol. 16, no. 7, pp. 689-700, 1994.
- R. Zeboudj, Filtrage, seuillage automatique, contraste et con- tours: du pré-traitement à l'analyse d'image, Ph.D. thesis, Uni- versité de Saint Etienne, Saint Etienne, France, 1988.
- S. Chabrier, C. Rosenberger, H. Laurent, B. Emile, and P. Marché, "Evaluating the segmentation result of a gray-level image," in Proceedings of 12th European Signal Processing Con- ference (EUSIPCO '04), pp. 953-956, Vienna, Austria, Septem- ber 2004.
- S. Chabrier, B. Emile, H. Laurent, C. Rosenberger, and P. Marché, "Unsupervised evaluation of image segmentation ap- plication to multi-spectral images," in Proceedings of Interna- tional Conference on Pattern Recognition (ICPR '04), vol. 1, pp. 576-579, Cambridge, UK, August 2004.
- R. Krishnapuram and J. M. Keller, "Possibilistic c-means algo- rithm: insights and recommendations," IEEE Transactions on Fuzzy Systems, vol. 4, no. 3, pp. 385-393, 1996.
- D. Comaniciu and P. Meer, "Mean shift: a robust approach toward feature space analysis," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603-619, 2002.
- H. A. Monawer, "Image vector quantization using a modified LBG algorithm with approximated centroids," Electronics Let- ters, vol. 31, no. 3, pp. 174-175, 1995.
- C ¸. E. Erdem, B. Sankur, and A. M. Tekalp, "Performance mea- sures for video object segmentation and tracking," IEEE Trans- actions on Image Processing, vol. 13, no. 7, pp. 937-951, 2004.
- A. M. Nazif and M. D. Levine, "Low level image segmentation: an expert system," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 6, no. 5, pp. 555-577, 1984.
- R. Krishnapuram and J. M. Keller, "Possibilistic approach to clustering," IEEE Transactions on Fuzzy Systems, vol. 1, no. 2, pp. 98-110, 1993.
Cited by
Sorted by citations
Reverse Classification Accuracy: Predicting Segmentation Performance in the Absence of Ground Truth
Valindria, V. V., Lavdas, I., Bai, W., Kamnitsas, K., Aboagye, E. O., Rockall, A. G., Rueckert, D., & Glocker, B. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2665165 · Cited by 152 · see citing works
Specktor-Fadida, B., Ben-Sira, L., Ben-Bashat, D., & Joskowicz, L. (2024) · arXiv · Cited by 9 · see citing works
- Authors
- Chabrier, S., Emile, B., Rosenberger, C., & Laurent, H.
- Affiliations
- Université d'Orléans (FR)
- Venue
- EURASIP Journal on Advances in Signal Processing
- Published
- 2006-03-27
- Volume
- 2006
- Issue
- 1
- Pages
- 096306
- ISSN
- 1110-8657 (ISSN-L)
- 1687-6180
- OpenAlex
- W2092572597
- Cited by
- 136
- References
- 1
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 5 · 2023: 3 · 2022: 8 · 2021: 8 · 2020: 4 · 2019: 7 · 2018: 10 · 2017: 13 · 2016: 4 · 2015: 12 · 2014: 7 · 2013: 11 · 2012: 4
- FWCI
- 4.33
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- EURASIP Journal on Advances in Signal Processing (publishedVersion, cc-by)
- HAL (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- European Commission
- Conseil Régional du Centre-Val de Loire
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Image and Video Quality Assessment
- Keywords
- image segmentation, unsupervised evaluation, segmentation quality assessment, synthetic images, grayscale images
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
