The DOI matches its Crossref record; 38 of the 39 references it cites were found in a registry or an index.
Automated deep-learning system for Gleason grading of prostate cancer using biopsies: a diagnostic study
Open accessAbstract
Background The Gleason score is the strongest correlating predictor of recurrence for prostate cancer, but has substantial inter-observer variability, limiting its usefulness for individual patients. Specialised urological pathologists have greater concordance; however, such expertise is not widely available. Prostate cancer diagnostics could thus benefit from robust, reproducible Gleason grading. We aimed to investigate the potential of deep learning to perform automated Gleason grading of prostate biopsies.Methods In this retrospective study, we developed a deep-learning system to grade prostate biopsies following the Gleason grading standard. The system was developed using randomly selected biopsies, sampled by the biopsy Gleason score, from patients at the Radboud University Medical Center (pathology report dated between Jan 1, 2012, and Dec 31, 2017). A semi-automatic labelling technique was used to circumvent the need for manual annotations by pathologists, using pathologists' reports as the reference standard during training. The system was developed to delineate individual glands, assign Gleason growth patterns, and determine the biopsy-level grade. For validation of the method, a consensus reference standard was set by three expert urological pathologists on an independent test set of 550 biopsies. Of these 550, 100 were used in an observer experiment, in which the system, 13 pathologists, and two pathologists in training were compared with respect to the reference standard. The system was also compared to an external test dataset of 886 cores, which contained 245 cores from a different centre that were independently graded by two pathologists.Findings We collected 5759 biopsies from 1243 patients. The developed system achieved a high agreement with the reference standard (quadratic Cohen's kappa 0·918, 95% CI 0·891-0·941) and scored highly at clinical decision thresholds: benign versus malignant (area under the curve 0·990, 95% CI 0·982-0·996), grade group of 2 or more (0·978, 0·966-0·988), and grade group of 3 or more (0·974, 0·962-0·984). In an observer experiment, the deep-learning system scored higher (kappa 0·854) than the panel (median kappa 0·819), outperforming 10 of 15 pathologist observers. On the external test dataset, the system obtained a high agreement with the reference standard set independently by two pathologists (quadratic Cohen's kappa 0·723 and 0·707) and within inter-observer variability (kappa 0·71).Interpretation Our automated deep-learning system achieved a performance similar to pathologists for Gleason grading and could potentially contribute to prostate cancer diagnosis. The system could potentially assist pathologists by screening biopsies, providing second opinions on grade group, and presenting quantitative measurements of volume percentages.Funding Dutch Cancer Society.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“multiple groups have reported on Gleason scoring predictions from prostate cancer H&E images (62)(63)(64).”
From Artificial Intelligence in Oncology: Current Landscape, Challenges, and Future Directions (2024) · cites this paper as [63]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404
Bray, F., Ferlay, J., Soerjomataram, I., Siegel, R. L., Torre, L. A., & Jemal, A. (2018) · CA: A Cancer Journal for Clinicians · doi:10.3322/caac.21492 · Cited by 84,109
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332
Self-citationDermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697
Ehteshami Bejnordi, B., Veta, M., Johannes van Diest, P., van Ginneken, B., Karssemeijer, N., Litjens, G., van der Laak, J. A. W. M., and the CAMELYON16 Consortium, Hermsen, M., Manson, Q. F., et al. (2017) · JAMA · doi:10.1001/jama.2017.14585 · Cited by 3,293
Self-citationEpstein, J. I., Egevad, L., Amin, M. B., Delahunt, B., Srigley, J. R., & Humphrey, P. A. (2015) · The American Journal of Surgical Pathology · doi:10.1097/pas.0000000000000530 · Cited by 3,246
Clinical-grade computational pathology using weakly supervised deep learning on whole slide images
Campanella, G., Hanna, M. G., Geneslaw, L., Miraflor, A., Werneck Krauss Silva, V., Busam, K. J., Brogi, E., Reuter, V. E., Klimstra, D. S., & Fuchs, T. J. (2019) · Nature Medicine · doi:10.1038/s41591-019-0508-1 · Cited by 2,799
Epstein, J. I., Allsbrook, W. C., Amin, M. B., & Egevad, L. L. (2005) · The American Journal of Surgical Pathology · doi:10.1097/01.pas.0000173646.99337.b1 · Cited by 2,763
Clinically applicable deep learning for diagnosis and referral in retinal disease
De Fauw, J., Ledsam, J. R., Romera-Paredes, B., Nikolov, S., Tomasev, N., Blackwell, S., Askham, H., Glorot, X., O’Donoghue, B., Visentin, D., et al. (2018) · Nature Medicine · doi:10.1038/s41591-018-0107-6 · Cited by 2,639
Ardila, D., Kiraly, A. P., Bharadwaj, S., Choi, B., Reicher, J. J., Peng, L., Tse, D., Etemadi, M., Ye, W., Corrado, G., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0447-x · Cited by 2,167
A Contemporary Prostate Cancer Grading System: A Validated Alternative to the Gleason Score
Epstein, J. I., Zeléfsky, M. J., Sjoberg, D. D., Nelson, J. B., Egevad, L., Magi‐Galluzzi, C., Vickers, A. J., Parwani, A. V., Reuter, V. E., Fine, S. W., et al. (2015) · European Urology · doi:10.1016/j.eururo.2015.06.046 · Cited by 1,375
Prostate-Cancer Mortality at 11 Years of Follow-up
Schröder, F. H., Hugosson, J., Roobol, M. J., Tammela, T. L. J., Ciatto, S., Nelen, V., KWIATKOWSKI, M. K., Lujan, M., Lilja, H. G., Zappa, M., et al. (2012) · New England Journal of Medicine · doi:10.1056/nejmoa1113135 · Cited by 1,241
Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis
Litjens, G., Sánchez, C. I., Timofeeva, N., Hermsen, M., Nagtegaal, I., Kovacs, I., Hulsbergen - van de Kaa, C., Bult, P., van Ginneken, B., & van der Laak, J. (2016) · Scientific Reports · doi:10.1038/srep26286 · Cited by 1,069
Self-citationPlacebo-Controlled Trial of Tofacitinib Monotherapy in Rheumatoid Arthritis
Fleischmann, R., Kremer, J. M., Cush, J. J., Schulze‐Koops, H., Connell, C. A., Bradley, J. D., Gruben, D. C., Wallenstein, G. V., Zwillich, S. H., & Kanik, K. S. (2012) · New England Journal of Medicine · doi:10.1056/nejmoa1109071 · Cited by 1,028
Time Trends and Local Variation in Primary Treatment of Localized Prostate Cancer
Cooperberg, M. R., Broering, J. M., & Carroll, P. R. (2010) · Journal of Clinical Oncology · doi:10.1200/jco.2009.26.0133 · Cited by 1,006
An Update of the Gleason Grading System
Epstein, J. I. (2009) · The Journal of Urology · doi:10.1016/j.juro.2009.10.046 · Cited by 533
Nagpal, K., Foote, D., Liu, Y., Chen, P.-H. C., Wulczyn, E., Tan, F., Olson, N., Smith, J. L., Mohtashamian, A., Wren, J. H., et al. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0112-2 · Cited by 495
Automated Gleason grading of prostate cancer tissue microarrays via deep learning
Arvaniti, E., Fricker, K. S., Moret, M., Rupp, N. J., Hermanns, T., Fankhauser, C. D., Wey, N., Wild, P. J., Rüschoff, J. H., & Claassen, M. (2018) · Scientific Reports · doi:10.1038/s41598-018-30535-1 · Cited by 461
Deep Learning is Robust to Massive Label Noise
Rolnick, D., Veit, A., Belongie, S., & Shavit, N. (2017) · arXiv · doi:10.48550/arxiv.1705.10694 · Cited by 402
Mukhopadhyay, S., Feldman, M. D., Abels, E., Ashfaq, R., Beltaifa, S., Cacciabeve, N. G., Cathro, H. P., Cheng, L., Cooper, K., Dickey, G. E., et al. (2018) · American Journal of Surgical Pathology · doi:10.1097/pas.0000000000000948 · Cited by 399
Interobserver reproducibility of Gleason grading of prostatic carcinoma: General pathologist
Allsbrook, W. C., Mangold, K. A., Johnson, M. H., Lane, R. B., Lane, C. G., & Epstein, J. I. (2001) · Human Pathology · doi:10.1053/hupa.2001.21135 · Cited by 380
Interobserver reproducibility of Gleason grading of prostatic carcinoma: Urologic pathologists
Allsbrook, W. C., Mangold, K. A., Johnson, M. H., Lane, R. B., Lane, C. G., Amin, M. B., Bostwick, D. G., Humphrey, P. A., Jones, E. C., Reuter, V. E., et al. (2001) · Human Pathology · doi:10.1053/hupa.2001.21134 · Cited by 334
Kweldam, C. F., Kümmerlin, I. P. E. D., Nieboer, D., Verhoef, E. I., Steyerberg, E. W., van der Kwast, T., Roobol, M. J., & van Leenders, G. J. L. H. (2016) · Modern Pathology · doi:10.1038/modpathol.2016.49 · Cited by 235
Interobserver variability in Gleason histological grading of prostate cancer
Özkan, T. A., Eruyar, A. T., Cebeci, O. Ö., Memik, Ö., Özcan, L., & Kuskonmaz, I. (2016) · Scandinavian Journal of Urology · doi:10.1080/21681805.2016.1206619 · Cited by 199
Standardization of Gleason grading among 337 European pathologists
Egevad, L., Ahmad, A. S., Ferran Algaba, B., Berney, D. M., Boccon-Gibod, L. A., Compérat, É., Evans, A. J., Griffiths, D. J., Grobholz, R., Kristiansen, G., et al. (2012) · Histopathology · doi:10.1111/his.12008 · Cited by 187
Bulten, W., Bándi, P., Hoven, J., van de Loo, R., Lotz, J., Weiss, N., van der Laak, J. A. W. M., van Ginneken, B., de Kaa, C. H., & Litjens, G. J. S. (2019) · Scientific Reports · doi:10.1038/s41598-018-37257-4 · Cited by 161
Self-citationLucas, M., Jansen, I., Savci-Heijink, C. D., Meijer, S. L., de Boer, O. J., van Leeuwen, T. G. J. M., de Bruin, D. M., & Marquering, H. A. (2019) · Archiv für Pathologische Anatomie und Physiologie und für Klinische Medicin · doi:10.1007/s00428-019-02577-x · Cited by 144
Machine learning approaches to analyze histological images of tissues from radical prostatectomies
Gertych, A., Ing, N., Ma, Z., Fuchs, T. J., Salman, S., Mohanty, S. K., Bhele, S., Velásquez, A., Amin, M. B., & Knudsen, B. S. (2015) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2015.08.002 · Cited by 117
Bulten, W., Pinckaers, H., van Boven, H., Vink, R., de Bel, T., van Ginneken, B., van der Laak, J. A. W. M., van de Kaa, C. A. H. ‐., & Litjens, G. J. S. (2020) · arXiv (Cornell University) · Cited by 112
Self-citationAutomatic Gleason grading of prostate cancer using quantitative phase imaging and machine learning
Nguyen, T. H., Sridharan, S., Macias, V., Kajdacsy‐Balla, A., Melamed, J., Do, M. N., & Popescu, G. (2017) · Journal of Biomedical Optics · doi:10.1117/1.jbo.22.3.036015 · Cited by 98
Bjartell, A. S. (2006) · European Urology · doi:10.1016/j.eururo.2006.02.007 · Cited by 93
Terabyte-scale Deep Multiple Instance Learning for Classification and Localization in PathologyPreprint
Campanella, G. S. V., Silva, V. W. K., & Fuchs, T. J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.06983 · Cited by 88
Kato, M., Tsuzuki, T., Kimura, K., Hirakawa, A., Kinoshita, F., Sassa, N., Ishida, R., Fukatsu, A., Kimura, T., Funahashi, Y., et al. (2016) · Modern Pathology · doi:10.1038/modpathol.2015.146 · Cited by 79
de Bel, T., Hermsen, M., Kers, J., van der Laak, J. A. W. M., & Litjens, G. J. S. (2018) · Data Archiving and Networked Services (DANS) · Cited by 60
Self-citationComparison of different methods for tissue segmentation in histopathological whole-slide images
Bándi, P., van de Loo, R., Intezar, M., Geijs, D. J., Ciompi, F., van Ginneken, B., van der Laak, J. A. W. M., & Litjens, G. J. S. (2017) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2017.7950590 · Cited by 56
Self-citationZhong, Q., Guo, T., Rechsteiner, M., Rüschoff, J. H., Rupp, N. J., Fankhauser, C. D., Saba, K., Mortezavi, A., Poyet, C., Hermanns, T., et al. (2017) · Scientific Data · doi:10.1038/sdata.2017.14 · Cited by 31
Automated Gleason Grading of Prostate Biopsies using Deep LearningPreprint
Bulten, W., Pinckaers, H., van Boven, H., Vink, R., de Bel, T., van Ginneken, B., Laak, J. A. W. M. V. D., de Kaa, C. H., & Litjens, G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.07980 · Cited by 11
Self-citation
Research works printed but not linked: 1
- Naik, 2007, Gland segmentation and computerized gleason grading of prostate histology by integrating low-, high-level and domain specific information, Proceedings of 2nd Workshop on Microsopic Image Analysis with Applications in Biology
Cited by
Sorted by citations
Data-efficient and weakly supervised computational pathology on whole-slide images
Lu, M. Y., Williamson, D. F. K., Chen, T. Y., Chen, R. J., Barbieri, M., & Mahmood, F. (2021) · Nature Biomedical Engineering · doi:10.1038/s41551-020-00682-w · Cited by 2,282
Towards a general-purpose foundation model for computational pathology
Chen, R. J., Ding, T., Lu, M. Y., Williamson, D. F. K., Jaume, G., Song, A. H., Chen, B., Zhang, A., Shao, D., Shaban, M., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02857-3 · Cited by 1,560
Zhou, S. K., Greenspan, H. K., Davatzikos, C. A., Duncan, J. S., van Ginneken, B., Madabhushi, A., Prince, J. L., Rueckert, D., & Summers, R. M. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3054390 · Cited by 1,373
A visual-language foundation model for computational pathology
Lu, M. Y., Chen, B., Williamson, D. F. K., Chen, R. J., Liang, I., Ding, T., Jaume, G., Odintsov, I., Le, L. P., Gerber, G., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-02856-4 · Cited by 966
Deep learning in histopathology: the path to the clinic
van der Laak, J., Litjens, G., & Ciompi, F. (2021) · Nature Medicine · doi:10.1038/s41591-021-01343-4 · Cited by 902
Self-citation
Metadata
- Authors
- Bulten, W., Pinckaers, H., van Boven, H., Vink, R., de Bel, T., van Ginneken, B., van der Laak, J., Hulsbergen-van de Kaa, C., & Litjens, G.
- Affiliations
- Radboud University Medical Center (NL)
- Corresponding author
- Bulten, Wouter
- Venue
- The Lancet Oncology
- Publisher
- Elsevier BV
- Published
- 2020-02
- Volume
- 21
- Issue
- 2
- Pages
- 233-241
- ISSN
- 1470-2045 (ISSN-L)
- OpenAlex
- W2999091210
- Cited by
- 705
- References
- 39
- Citations by year
- 2026: 84 · 2025: 96 · 2024: 112 · 2023: 115 · 2022: 128 · 2021: 115 · 2020: 55
- FWCI
- 40.59
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- The Lancet Oncology (publishedVersion)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- Funders
- Department of Health and Social Care
- KWF Kankerbestrijding (KUN 2015-7970)
- Radboud Universitair Medisch Centrum
- AbbVie
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Health Sciences › Medicine › Urology › Prostate Cancer Diagnosis and Treatment
- Keywords
- Gleason score, prostate cancer, deep learning, prostate biopsy, interobserver variability, external validation, benign versus malignant differentiation, urologic pathology, computer-aided diagnosis, recurrence prediction
- MeSH terms
- Deep Learning, Biopsy, Diagnosis, Computer-Assisted, Humans, Image Interpretation, Computer-Assisted, Male, Predictive Value of Tests, Prostatic Neoplasms, Retrospective Studies, Reproducibility of Results, Observer Variation, Automation, Laboratory, Neoplasm Grading
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
