Mitosis Counting in Breast Cancer: Object-Level Interobserver Agreement and Comparison to an Automatic Method
Open accessAbstract
BACKGROUND: Tumor proliferation speed, most commonly assessed by counting of mitotic figures in histological slide preparations, is an important biomarker for breast cancer. Although mitosis counting is routinely performed by pathologists, it is a tedious and subjective task with poor reproducibility, particularly among non-experts. Inter- and intraobserver reproducibility of mitosis counting can be improved when a strict protocol is defined and followed. Previous studies have examined only the agreement in terms of the mitotic count or the mitotic activity score. Studies of the observer agreement at the level of individual objects, which can provide more insight into the procedure, have not been performed thus far. METHODS: The development of automatic mitosis detection methods has received large interest in recent years. Automatic image analysis is viewed as a solution for the problem of subjectivity of mitosis counting by pathologists. In this paper we describe the results from an interobserver agreement study between three human observers and an automatic method, and make two unique contributions. For the first time, we present an analysis of the object-level interobserver agreement on mitosis counting. Furthermore, we train an automatic mitosis detection method that is robust with respect to staining appearance variability and compare it with the performance of expert observers on an "external" dataset, i.e. on histopathology images that originate from pathology labs other than the pathology lab that provided the training data for the automatic method. RESULTS: The object-level interobserver study revealed that pathologists often do not agree on individual objects, even if this is not reflected in the mitotic count. The disagreement is larger for objects from smaller size, which suggests that adding a size constraint in the mitosis counting protocol can improve reproducibility. The automatic mitosis detection method can perform mitosis counting in an unbiased way, with substantial agreement with human experts.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In histopathology, variation between hospitals can arise from differences in the patient population or in slide staining and image acquisition.
“In histopathology applications-studying tissue slides under a microscope-this variation can arise from sources like differences in the patient population or in slide staining and image acquisition (Veta et al., 2016; Komura and Ishikawa, 2018; Tellez et al., 2019).”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as Veta et al., 2016
The Introduction (page unavailable) describes tissue-appearance differences between pathology labs, and the Discussion and Conclusions (page unavailable) explicitly identifies staining variability between labs. The paper does not identify differences in patient populations or image-acquisition practices as sources of that variation; the Dataset section describes its scanner but does not establish acquisition differences between hospitals. (LLM, conf=0.86)
Experimental — accuracy not yet measuredIn histopathology applications, variation can arise from sources like differences in the patient population or in slide staining and image acquisition.
“In histopathology applications-studying tissue slides under a microscope-this variation can arise from sources like differences in the patient population or in slide staining and image acquisition (Veta et al., 2016; Komura and Ishikawa, 2018; Tellez et al., 2019).”
From WILDS: A Benchmark of in-the-Wild Distribution Shifts (2020) · cites this paper as Veta et al., 2016
The Discussion and Conclusions (page not provided) explicitly identifies staining variability between pathology labs as a source of variation that adversely affects image-analysis performance. The copy does not identify differences in patient populations or image acquisition as sources of that variation; the Dataset section describes cases and scanning equipment but does not make that connection. (LLM, conf=0.87)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 16 · Cited by 127
Research works printed but not linked: 12
- [1]van Diest PJ, van der Wall E, Baak JPA. Prognostic value of proliferation in invasive breast cancer: a review. J Clin Pathol. 2004; 57: 675– 681. 15220356 PMC1770351
- [2]Elston CW, Ellis IO. Pathological prognostic factors in breast cancer. I. The value of histological grade in breast cancer: experience from a large study with long-term follow-up. Histopathology. 1991; 19: 403– 410. 1757079
- [3]Baak JPA, van Diest PJ, Voorhorst FJ, van der Wall E, Beex LVAM, Vermorken JB, et al Prospective multicenter validation of the independent prognostic value of the mitotic activity index in lymph node-negative breast cancer patients younger than 55 years. J Clin Oncol. 2005; 23: 5993– 6001. 10.1200/JCO.2005.05.511 16135467 · DOI
- [4]van Diest PJ, Baak JP, Matze-Cok P, Wisse-Brekelmans EC, van Galen CM, Kurver PH, et al Reproducibility of mitosis counting in 2,469 breast cancer specimens: results from the Multicenter Morphometric Mammary Carcinoma Project. Hum Pathol. 1992; 23: 603– 607. 1592381
- [5]Veta M, van Diest PJ, Willems SM, Wang H, Madabhushi A, Cruz-Roa A, et al Assessment of algorithms for mitosis detection in breast cancer histopathology images. Med Image Anal. 2015; 20: 237– 248. 10.1016/j.media.2014.11.010 25547073 · DOI
- [6]Cireşan DC, Giusti A, Gambardella LM, Schmidhuber J. Mitosis detection in breast cancer histology images with deep neural networks. Medical Image Computing and Computer-Assisted Intervention (MICCAI 2013). 2013 pp. 411– 418. 24579167
- [7]Veta M, van Diest PJ, Pluim JPW. Detecting mitotic figures in breast cancer histopathology images. SPIE Medical Imaging 2013. 2013. pp. 867607–867607–7.
- [8]Roux L, Racoceanu D, Loménie N, Kulikova M, Irshad H, Klossa J, et al Mitosis detection in breast cancer histological images An ICPR 2012 contest. J Pathol Inform. 2013; 4: 8 10.4103/2153-3539.112693 23858383 PMC3709417 · DOI
- [9]Al-Janabi S, van Slooten H-J, Visser M, van der Ploeg T, van Diest PJ, Jiwa M. Evaluation of mitotic activity index in breast cancer using whole slide digital images. PLoS ONE. 2013; 8: e82576 10.1371/journal.pone.0082576 24386102 PMC3875418 · DOI
- [10]LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015; 521: 436– 444. 10.1038/nature14539 26017442 · DOI · Paper page
- [12]Vink JP, Van Leeuwen MB, Van Deurzen CHM, De Haan G. Efficient nucleus detector in histopathology images. J Microsc. 2013; 249: 124– 135. 10.1111/jmi.12001 23252774 · DOI
- [14]Dice LR. Measures of the amount of ecologic association between species. Ecology. 1945; 26: 297– 302.
- Authors
- Veta, M., van Diest, P. J., Jiwa, M., Al-Janabi, S., & Pluim, J. P. W.
- Affiliations
- Eindhoven University of Technology (NL)
- University Medical Center Utrecht (NL)
- Corresponding author
- Veta, Mitko
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2016-08-16
- Volume
- 11
- Issue
- 8
- Pages
- e0161286
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC4987048
- OpenAlex
- W2516333753
- Cited by
- 127
- References
- 16
- Citations by year
- 2026: 6 · 2025: 7 · 2024: 22 · 2023: 8 · 2022: 14 · 2021: 19 · 2020: 15 · 2019: 15 · 2018: 6 · 2017: 15
- FWCI
- 9.88
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion)
- TU/e Research Portal (publishedVersion, other-oa)
- Pure Amsterdam UMC (submittedVersion, other-oa)
- TU/e Research Portal (submittedVersion, other-oa)
- TU/e Research Portal (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Figshare (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Biophysics › Cell Image Analysis Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Imaging for Blood Diseases
- Keywords
- breast cancer, interobserver agreement, tumor cell proliferation, histopathological images, staining variability, observer reproducibility
- MeSH terms
- Algorithms, Automation, Breast Neoplasms, Cytological Techniques, Mitosis, Observer Variation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
