Quality control stress test for deep learning-based diagnostic model in digital pathology
Open accessAbstract
Digital pathology provides a possibility for computational analysis of histological slides and automatization of routine pathological tasks. Histological slides are very heterogeneous concerning staining, sections' thickness, and artifacts arising during tissue processing, cutting, staining, and digitization. In this study, we digitally reproduce major types of artifacts. Using six datasets from four different institutions digitized by different scanner systems, we systematically explore artifacts' influence on the accuracy of the pre-trained, validated, deep learning-based model for prostate cancer detection in histological slides. We provide evidence that any histological artifact dependent on severity can lead to a substantial loss in model performance. Strategies for the prevention of diagnostic model accuracy losses in the context of artifacts are warranted. Stress-testing of diagnostic models using synthetically generated artifacts might be an essential step during clinical validation of deep learning-based algorithms.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The limited availability and variable quality of public pathology data can hinder the performance of foundation models when applied to real-world clinical scenarios.
“The limited availability and variable quality of public pathology data can hinder the performance of these models when applied to real-world clinical scenarios20.”
From Benchmarking foundation models as feature extractors for weakly supervised computational pathology (2025) · cites this paper as [20]
Adversarial debiasing and fairness-aware ML can help correct imbalances while preserving diagnostic accuracy.
“When disparities are detected, techniques like adversarial debiasing and fairness-aware ML can help correct imbalances while preserving diagnostic accuracy [123].”
From Artificial Intelligence in Clinical Oncology: From Productivity Enhancement to Creative Discovery (2025) · cites this paper as [123]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01228-7 · Cited by 6,909 · see citing works
Ehteshami Bejnordi, B., Veta, M., Johannes van Diest, P., van Ginneken, B., Karssemeijer, N., Litjens, G., van der Laak, J. A. W. M., and the CAMELYON16 Consortium, Hermsen, M., Manson, Q. F., et al. (2017) · JAMA · doi:10.1001/jama.2017.14585 · Cited by 3,293 · see citing works
Coudray, N., Ocampo, P. S., Sakellaropoulos, T., Narula, N., Snuderl, M., Fenyö, D., Moreira, A. L., Razavian, N., & Tsirigos, A. (2018) · Nature Medicine · doi:10.1038/s41591-018-0177-5 · Cited by 2,930 · see citing works
Clinical-grade computational pathology using weakly supervised deep learning on whole slide images
Campanella, G., Hanna, M. G., Geneslaw, L., Miraflor, A., Werneck Krauss Silva, V., Busam, K. J., Brogi, E., Reuter, V. E., Klimstra, D. S., & Fuchs, T. J. (2019) · Nature Medicine · doi:10.1038/s41591-019-0508-1 · Cited by 2,799 · see citing works
Artificial intelligence in digital pathology — new tools for diagnosis and precision oncology
Bera, K., Schalper, K. A., Rimm, D. L., Velcheti, V., & Madabhushi, A. (2019) · Nature Reviews Clinical Oncology · doi:10.1038/s41571-019-0252-y · Cited by 1,698 · see citing works
Self-citationA method for normalizing histology slides for quantitative analysis
Macenko, M., Niethammer, M., Marron, J. S., Borland, D., Woosley, J. T., Guan, X., Schmitt, C., & Thomas, N. E. (2009) · 2009 IEEE International Symposium on Biomedical Imaging: From Nano to Macro · doi:10.1109/isbi.2009.5193250 · Cited by 1,482 · see citing works
Kather, J. N., Pearson, A. T., Halama, N., Jäger, D., Krause, J., Loosen, S. H., Marx, A., Boor, P., Tacke, F., Neumann, U. P., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0462-y · Cited by 1,458 · see citing works
Digital pathology and artificial intelligence
Niazi, M. K. K., Parwani, A. V., & Gurcan, M. N. (2019) · The Lancet Oncology · doi:10.1016/s1470-2045(19)30154-8 · Cited by 1,232 · see citing works
Image analysis and machine learning in digital pathology: challenges and opportunities
Madabhushi, A., & Lee, G. (2016) · Medical Image Analysis · doi:10.1016/j.media.2016.06.037 · Cited by 1,093 · see citing works
Self-citationPredicting cancer outcomes from histology and genomics using convolutional networks
Mobadersany, P., Yousefi, S., Amgad, M., Gutman, D. A., Barnholtz-Sloan, J. S., Velázquez Vega, J. E., Brat, D. J., & Cooper, L. A. D. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1717139115 · Cited by 1,084 · see citing works
Kather, J. N., Krisam, J., Charoentong, P., Luedde, T., Herpel, E., Weis, C.-A., Gaiser, T., Marx, A., Valous, N. A., Ferber, D., et al. (2019) · PLOS Medicine · doi:10.1371/journal.pmed.1002730 · Cited by 1,082 · see citing works
Structure-Preserving Color Normalization and Sparse Stain Separation for Histological Images
Vahadane, A., Peng, T., Sethi, A., Albarqouni, S., Wang, L., Baust, M., Steiger, K., Schlitter, A. M., Esposito, I., & Navab, N. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2529665 · Cited by 833 · see citing works
Bulten, W., Pinckaers, H., van Boven, H., Vink, R., de Bel, T., van Ginneken, B., van der Laak, J., Hulsbergen-van de Kaa, C., & Litjens, G. (2020) · The Lancet Oncology · doi:10.1016/s1470-2045(19)30739-9 · Cited by 705 · see citing works
Tellez, D., Litjens, G., Bándi, P., Bulten, W., Bokhorst, J.-M., Ciompi, F., & van der Laak, J. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.101544 · Cited by 649 · see citing works
Deep learning based tissue analysis predicts outcome in colorectal cancer
Bychkov, D., Linder, N., Turkki, R., Nordling, S., Kovanen, P. E., Verrill, C., Walliander, M., Lundin, M., Haglund, C., & Lundin, J. (2018) · Scientific Reports · doi:10.1038/s41598-018-21758-3 · Cited by 644 · see citing works
Pantanowitz, L., Sinard, J. H., Henricks, W. H., Fatheree, L. A., Carter, A. B., Contis, L. C., Beckwith, B. A., Evans, A. J., Lal, A., & Parwani, A. V. (2013) · Archives of Pathology & Laboratory Medicine · doi:10.5858/arpa.2013-0093-cp · Cited by 622 · see citing works
Ström, P., Kartasalo, K., Olsson, H., Solorzano, L., Delahunt, B., Berney, D. M., Bostwick, D. G., Evans, A. J., Grignon, D. J., Humphrey, P. A., et al. (2020) · The Lancet Oncology · doi:10.1016/s1470-2045(19)30738-7 · Cited by 620 · see citing works
Bandi, P., Geessink, O., Manson, Q., Van Dijk, M., Balkenhol, M., Hermsen, M., Ehteshami Bejnordi, B., Lee, B., Paeng, K., Zhong, A., et al. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2867350 · Cited by 569 · see citing works
Nagpal, K., Foote, D., Liu, Y., Chen, P.-H. C., Wulczyn, E., Tan, F., Olson, N., Smith, J. L., Mohtashamian, A., Wren, J. H., et al. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0112-2 · Cited by 495 · see citing works
Staingan: Stain Style Transfer for Digital Histological Images
Shaban, M., Baur, C., Navab, N., & Albarqouni, S. (2019) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2019.8759152 · Cited by 361 · see citing works
HistoQC: An Open-Source Quality Control Tool for Digital Pathology Slides
Janowczyk, A. R., Zuo, R., Gilmore, H. L., Feldman, M. D., & Madabhushi, A. (2019) · JCO Clinical Cancer Informatics · doi:10.1200/cci.18.00157 · Cited by 357 · see citing works
Self-citationCouture, H. D., Williams, L. A., Geradts, J., Nyante, S. J., Butler, E. N., Marron, J. S., Perou, C. M., Troester, M. A., & Niethammer, M. (2018) · npj Breast Cancer · doi:10.1038/s41523-018-0079-1 · Cited by 327 · see citing works
Stain Specific Standardization of Whole-Slide Histopathological Images
Bejnordi, B. E., Litjens, G. J. S., Timofeeva, N., Otte‐Höller, I., Homeyer, A., Karssemeijer, N., & van der Laak, J. A. W. M. (2015) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2015.2476509 · Cited by 291 · see citing works
Adversarial Stain Transfer for Histopathology Image Analysis
BenTaieb, A., & Hamarneh, G. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2781228 · Cited by 231 · see citing works
Stain Normalization using Sparse AutoEncoders (StaNoSA): Application to digital pathology
Janowczyk, A. R., Basavanhally, A. N., & Madabhushi, A. (2016) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2016.05.003 · Cited by 211 · see citing works
Self-citationHigh-accuracy prostate cancer pathology using deep learning
Tolkach, Y., Dohmgörgen, T., Toma, M. I., & Kristiansen, G. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0200-7 · Cited by 183 · see citing works
Self-citationImage Sharpness Assessment by Sparse Representation
Li, L., Dong, W. C., Wu, J., Li, H., Lin, W., & Kot, A. C. (2016) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2016.2545398 · Cited by 130 · see citing works
DeepFocus: Detection of out-of-focus regions in whole slide digital images using deep learning
Şenaras, Ç., Niazi, M. K. K., Lozanski, G., & Gürcan, M. N. (2018) · PLoS ONE · doi:10.1371/journal.pone.0205387 · Cited by 121 · see citing works
Adversarial Domain Adaptation for Classification of Prostate Histopathology Whole-Slide Images
Ren, J., Hacihaliloglu, I., Singer, E. A., Foran, D. J., & Qi, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00934-2_23 · Cited by 118 · see citing works
Ianni, J. D., Soans, R. E., Sankarapandian, S., Chamarthi, R. V., Ayyagari, D., Olsen, T. G., Bonham, M. J., Stavish, C. C., Motaparthi, K., Cockerell, C. J., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-59985-2 · Cited by 113 · see citing works
Bulten, W., Pinckaers, H., van Boven, H., Vink, R., de Bel, T., van Ginneken, B., van der Laak, J. A. W. M., van de Kaa, C. A. H. ‐., & Litjens, G. J. S. (2020) · arXiv (Cornell University) · Cited by 106 · see citing works
Pham, H. H. N., Futakuchi, M., Bychkov, A., Furukawa, T., Kuroda, K., & Fukuoka, J. (2019) · American Journal Of Pathology · doi:10.1016/j.ajpath.2019.08.014 · Cited by 98 · see citing works
Self-citationWhole-Slide Image Focus Quality: Automatic Assessment and Impact on AI Cancer Detection
Kohlberger, T., Liu, Y., Moran, M., Chen, P.-H. C., Brown, T., Hipp, J., Mermel, C. H., & Stumpe, M. C. (2019) · Journal of Pathology Informatics · doi:10.4103/jpi.jpi_11_19 · Cited by 94 · see citing works
Otálora, S., Atzori, M., Andrearczyk, V., Khan, A., & Müller, H. (2019) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2019.00198 · Cited by 82 · see citing works
Leo, P., Lee, G., Shih, N. N. C., Elliott, R. M., Feldman, M. D., & Madabhushi, A. (2016) · Journal of medical imaging · doi:10.1117/1.jmi.3.4.047502 · Cited by 74 · see citing works
Self-citationFocus Quality Assessment of High-Throughput Whole Slide Imaging in Digital Pathology
Hosseini, M. S., Brawley-Hayes, J. A. Z., Zhang, Y., et al. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2919722 · Cited by 74 · see citing works
Świderska-Chadaj, Ż., de Bel, T., Blanchet, L., Baidoshvili, A., Vossen, D. L. J., van der Laak, J. A. W. M., & Litjens, G. J. S. (2020) · Scientific Reports · doi:10.1038/s41598-020-71420-0 · Cited by 71 · see citing works
Eycke, Y. V., Balsat, C., Verset, L., Debeir, O., Salmon, I. J., & Decaestecker, C. (2018) · Medical Image Analysis · doi:10.1016/j.media.2018.07.004 · Cited by 65 · see citing works
Schmitt, M., Maron, R. C., Hekler, A., Stenzinger, A., Hauschild, A., Weichenthal, M., Tiemann, M., Krahl, D., Kutzner, H., Utikal, J. S., et al. (2021) · Journal of Medical Internet Research · doi:10.2196/23436 · Cited by 59 · see citing works
Campanella, G. S. V., Rajanna, A. R., Corsale, L., Schüffler, P. J., Yagi, Y., & Fuchs, T. J. (2017) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2017.09.001 · Cited by 56 · see citing works
Chen, Y. M., Janowczyk, A. R., & Madabhushi, A. (2020) · JCO Clinical Cancer Informatics · doi:10.1200/cci.19.00068 · Cited by 43 · see citing works
Self-citationSinganamalli, A., Rusu, M., Sparks, R. E., et al. (2015) · Journal of Magnetic Resonance Imaging · doi:10.1002/jmri.24975 · Cited by 39 · see citing works
Zanjani, F. G., Zinger, S., Piepers, B., Mahmoudpour, S., Schelkens, P., & de With, P. H. N. (2019) · Journal of medical imaging · doi:10.1117/1.jmi.6.2.027501 · Cited by 38 · see citing works
Context-Based Normalization of Histological Stains Using Deep Convolutional Features
Bug, D., Schneider, S., Grote, A., Oswald, E., Feuerhake, F., Schüler, J., & Merhof, D. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67558-9_16 · Cited by 37 · see citing works
Marble, H. D., Huang, R., Dudgeon, S. N., Lowe, A., Herrmann, M. D., Blakely, S., Leavitt, M. O., Isaacs, M., Hanna, M. G., Sharma, A., et al. (2020) · Journal of Pathology Informatics · doi:10.4103/jpi.jpi_27_20 · Cited by 35 · see citing works
Ameisen, D., Deroulers, C., Perrier, V., et al. (2014) · Diagnostic Pathology · doi:10.1186/1746-1596-9-s1-s3 · Cited by 34 · see citing works
Pathologist-Level Grading of Prostate Biopsies with Artificial IntelligencePreprint
Ström, P., Kartasalo, K., Olsson, H., Solorzano, L., Delahunt, B., Berney, D. M., Bostwick, D. G., Evans, A. J., Grignon, D. J., Humphrey, P. A., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.01368 · Cited by 28 · see citing works
Doyle, S. T., Monaco, J., Madabhushi, A., et al. (2010) · IEEE International Symposium on Biomedical Imaging: From Nano to Macro (ISBI) · doi:10.1109/isbi.2010.5490238 · Cited by 14 · see citing works
Self-citationAsterias: a parallelized web-based suite for the analysis of expression and aCGH data.
Alibés, A., Morrissey, E. R., Cañada, A., et al. (2007) · PubMed · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Digital pathology and artificial intelligence in translational medicine and clinical practice
Baxi, V., Edwards, R., Montalto, M., & Saha, S. (2022) · Modern Pathology · doi:10.1038/s41379-021-00919-2 · Cited by 592 · see citing works
Artificial intelligence in histopathology: enhancing cancer research and clinical oncology
Shmatko, A., Ghaffari Laleh, N., Gerstung, M., & Kather, J. N. (2022) · Nature Cancer · doi:10.1038/s43018-022-00436-4 · Cited by 446 · see citing works
Scaling Self-Supervised Learning for Histopathology with Masked Image Modeling
Filiot, A., Ghermi, R., Olivier, A., Jacob, P., Fidon, L., Camara, A., Mac Kain, A., Saillard, C., & Schiratti, J.-B. (2023) · doi:10.1101/2023.07.21.23292757 · Cited by 148 · see citing works
Echle, A., Ghaffari Laleh, N., Quirke, P., Grabsch, H. I., Muti, H. S., Saldanha, O. L., Brockmoeller, S. F., van den Brandt, P. A., Hutchins, G. G. A., Richman, S. D., et al. (2022) · ESMO Open · doi:10.1016/j.esmoop.2022.100400 · Cited by 111 · see citing works
Overcoming the challenges to implementation of artificial intelligence in pathology
Reis-Filho, J. S., & Kather, J. N. (2023) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/djad048 · Cited by 79 · see citing works
Predictive uncertainty estimation for out-of-distribution detection in digital pathology
Linmans, J., Elfwing, S., van der Laak, J., & Litjens, G. (2023) · Medical Image Analysis · doi:10.1016/j.media.2022.102655 · Cited by 75 · see citing works
Benchmarking foundation models as feature extractors for weakly supervised computational pathology
Neidlinger, P., El Nahhas, O. S. M., Muti, H. S., Lenz, T., Hoffmeister, M., Brenner, H., van Treeck, M., Langer, R., Dislich, B., Behrens, H. M., et al. (2025) · Nature Biomedical Engineering · doi:10.1038/s41551-025-01516-3 · Cited by 80 · see citing works
Artificial Intelligence for Dermatopathology: Current Trends and the Road Ahead
Chen, S. B., & Novoa, R. A. (2022) · Seminars in Diagnostic Pathology · doi:10.1053/j.semdp.2022.01.003 · Cited by 37 · see citing works
Artificial Intelligence in Clinical Oncology: From Productivity Enhancement to Creative Discovery
Kuno, M., Osumi, H., Udagawa, S., Yoshikawa, K., Ooki, A., Shinozaki, E., Ishikawa, T., Oba, J., Yamaguchi, K., & Sakurada, K. (2025) · Current Oncology · doi:10.3390/curroncol32110588 · Cited by 7 · see citing works
- Authors
- Schömig-Markiefka, B., Pryalukhin, A., Hulla, W., Bychkov, A., Fukuoka, J., Madabhushi, A., Achter, V., Nieroda, L., Büttner, R., Quaas, A., & Tolkach, Y.
- Affiliations
- Universitätsklinikum Wiener Neustadt (AT)
- Kameda Medical Center (JP)
- Louis Stokes Cleveland VA Medical Center (US)
- University of Cologne (DE)
- University Hospital Cologne (DE)
- Corresponding author
- Tolkach, Yuri
- ORCID
- Venue
- Modern Pathology
- Publisher
- Elsevier BV
- Published
- 2021-12
- Volume
- 34
- Issue
- 12
- Pages
- 2098-2108
- ISSN
- 0893-3952 (ISSN-L)
- PMC
- PMC8592835
- OpenAlex
- W3176446229
- Cited by
- 146
- References
- 49
- Citations by year
- 2026: 21 · 2025: 31 · 2024: 30 · 2023: 30 · 2022: 30 · 2021: 3 · 2014: 1
- FWCI
- 8.30
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Modern Pathology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Kölner Universitäts PublikationsServer (Universität zu Köln) (publishedVersion)
- Funders
- U.S. Department of Veterans Affairs (I01 BX004121)
- National Institutes of Health (5u54ca254566-04, UL1TR002548)
- National Heart, Lung, and Blood Institute (R01HL151277)
- National Cancer Institute (U01CA248226, U54CA254566, U24CA199374, R01CA216579, U01CA239055, R01CA208236, R01CA202752, R01CA220581, R01CA249992)
- National Institute of Biomedical Imaging and Bioengineering (R43EB028736)
- National Center for Research Resources (C06RR012463)
- National Center for Advancing Translational Sciences
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- digital pathology, histological artifacts, stress testing, prostate cancer detection, clinical validation of AI
- MeSH terms
- Deep Learning, Humans, Image Processing, Computer-Assisted, Male, Pathology, Clinical, Prostatic Neoplasms, Quality Control, Reproducibility of Results, Artifacts, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
