Predictive uncertainty estimation for out-of-distribution detection in digital pathology
Open accessAbstract
Machine learning model deployment in clinical practice demands real-time risk assessment to identify situations in which the model is uncertain. Once deployed, models should be accurate for classes seen during training while providing informative estimates of uncertainty to flag abnormalities and unseen classes for further analysis. Although recent developments in uncertainty estimation have resulted in an increasing number of methods, a rigorous empirical evaluation of their performance on large-scale digital pathology datasets is lacking. This work provides a benchmark for evaluating prevalent methods on multiple datasets by comparing the uncertainty estimates on both in-distribution and realistic near and far out-of-distribution (OOD) data on a whole-slide level. To this end, we aggregate uncertainty values from patch-based classifiers to whole-slide level uncertainty scores. We show that results found in classical computer vision benchmarks do not always translate to the medical imaging setting. Specifically, we demonstrate that deep ensembles perform best at detecting far-OOD data but can be outperformed on a more challenging near-OOD detection task by multi-head ensembles trained for optimal ensemble diversity. Furthermore, we demonstrate the harmful impact OOD data can have on the performance of deployed machine learning models. Overall, we show that uncertainty estimates can be used to discriminate in-distribution from OOD data with high AUC scores. Still, model deployment might require careful tuning based on prior knowledge of prospective OOD data.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Hotspot detection approaches can reduce review time and improve safety but face challenges such as spurious attention to artifacts, sensitivity-specificity trade-offs in small-lesion detection, and overconfidence on out-of-distribution detection.
“These approaches can reduce review time and improve safety but face challenges such as spurious attention to artifacts, sensitivity-specificity trade-offs in small-lesion detection, and overconfidence on out-of-distribution detection [33,35,36].”
From Artificial Intelligence in Dermatopathology: An Update and Review of the Current Literature (2026) · cites this paper as [36]
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566 · see citing works
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471 · see citing works
Reading digits in natural images with unsupervised feature learning
Netzer, Y. (2024) · doi:10.57702/e7wwo0zk · Cited by 4,261 · see citing works
Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles
Lakshminarayanan, B., Pritzel, A., & Blundell, C. (2016) · arXiv · doi:10.48550/arxiv.1612.01474 · Cited by 3,666 · see citing works
On calibration of modern neural networks
Guo, C., Pleiss, G., Sun, Y., & Weinberger, K. Q. (2017) · arXiv · doi:10.48550/arxiv.1706.04599 · Cited by 3,435 · see citing works
Ehteshami Bejnordi, B., Veta, M., Johannes van Diest, P., van Ginneken, B., Karssemeijer, N., Litjens, G., van der Laak, J. A. W. M., and the CAMELYON16 Consortium, Hermsen, M., Manson, Q. F., et al. (2017) · JAMA · doi:10.1001/jama.2017.14585 · Cited by 3,293 · see citing works
What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?
Kendall, A., & Gal, Y. (2017) · arXiv · doi:10.48550/arxiv.1703.04977 · Cited by 3,088 · see citing works
A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks
Hendrycks, D., & Gimpel, K. (2016) · arXiv · doi:10.48550/arxiv.1610.02136 · Cited by 1,968 · see citing works
Efficient object localization using Convolutional Networks
Tompson, J., Goroshin, R., Jain, A., LeCun, Y., & Bregler, C. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298664 · Cited by 1,365 · see citing works
Deep Learning for Identifying Metastatic Breast CancerPreprint
Wang, D., Khosla, A., Gargeya, R., Irshad, H., & Beck, A. H. (2016) · arXiv.org · doi:10.48550/arxiv.1606.05718 · Cited by 1,028 · see citing works
Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift
Ovadia, Y., Fertig, E., Ren, J., Nado, Z., Sculley, D., Nowozin, S., Dillon, J. V., Lakshminarayanan, B., & Snoek, J. (2019) · arXiv · doi:10.48550/arxiv.1906.02530 · Cited by 989 · see citing works
Deep Exploration via Bootstrapped DQNPreprint
Osband, I., Blundell, C., Pritzel, A., & Roy, B. V. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.04621 · Cited by 837 · see citing works
Deep Ensembles: A Loss Landscape PerspectivePreprint
Fort, S., Hu, H., & Lakshminarayanan, B. (2019) · arXiv.org · doi:10.48550/arxiv.1912.02757 · Cited by 820 · see citing works
Bulten, W., Pinckaers, H., van Boven, H., Vink, R., de Bel, T., van Ginneken, B., van der Laak, J., Hulsbergen-van de Kaa, C., & Litjens, G. (2020) · The Lancet Oncology · doi:10.1016/s1470-2045(19)30739-9 · Cited by 705 · see citing works
Self-citationBandi, P., Geessink, O., Manson, Q., Van Dijk, M., Balkenhol, M., Hermsen, M., Ehteshami Bejnordi, B., Lee, B., Paeng, K., Zhong, A., et al. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2867350 · Cited by 569 · see citing works
Artificial intelligence for diagnosis and Gleason grading of prostate cancer: the PANDA challenge
Bulten, W., Kartasalo, K., Chen, P.-H. C., Ström, P., Pinckaers, H., Nagpal, K., Cai, Y., Steiner, D. F., van Boven, H., Vink, R., et al. (2022) · Nature Medicine · doi:10.1038/s41591-021-01620-2 · Cited by 559 · see citing works
Second opinion needed: Communicating uncertainty in medical machine learning
Kompa, B., Snoek, J., & Beam, A. L. (2021) · npj Digital Medicine · doi:10.1038/s41746-020-00367-3 · Cited by 514 · see citing works
Leveraging uncertainty information from deep neural networks for disease detection
Leibig, C., Allken, V., Ayhan, M. S., Berens, P., & Wahl, S. (2017) · Scientific Reports · doi:10.1038/s41598-017-17876-z · Cited by 510 · see citing works
Detecting Cancer Metastases on Gigapixel Pathology Images
Liu, Y., Gadepalli, K., Norouzi, M., Dahl, G. E., Kohlberger, T., Boyko, A., Venugopalan, S., Timofeev, A., Nelson, P. Q., Corrado, G. S., et al. (2017) · arXiv · doi:10.48550/arxiv.1703.02442 · Cited by 506 · see citing works
Nair, T., Precup, D., Arnold, D. L., & Arbel, T. (2020) · Medical Image Analysis · doi:10.1016/j.media.2019.101557 · Cited by 451 · see citing works
1399 H&E-stained sentinel lymph node sections of breast cancer patients: the CAMELYON dataset
Litjens, G. J. S., Bándi, P., Bejnordi, B. E., Geessink, O. G. F., Balkenhol, M. C. A., Bult, P., Halilović, A., Hermsen, M., van de Loo, R., Vogels, R. J., et al. (2018) · GigaScience · doi:10.1093/gigascience/giy065 · Cited by 414 · see citing works
Artificial intelligence for diagnosis and Gleason grading of prostate cancer: the PANDA challenge
Wouter, B., Kimmo, K., Cameron, C. P.-H., Peter, S., Hans, P., Kunal, N., Yuannan, C., Steiner, D. F., van Boven, H., Robert, V., et al. (2022) · UTUPub (University of Turku) · Cited by 375 · see citing works
Scaling Out-of-Distribution Detection for Real-World SettingsPreprint
Hendrycks, D., Basart, S., Mazeika, M., Zou, A., Kwon, J., Mostajabi, M., Steinhardt, J., & Song, D. X. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.11132 · Cited by 320 · see citing works
Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision
Gustafsson, F., Danelljan, M., & Schön, T. B. (2020) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw50498.2020.00167 · Cited by 272 · see citing works
Why M Heads are Better than One: Training a Diverse Ensemble of Deep NetworksPreprint
Lee, S., Purushwalkam, S., Cogswell, M., Crandall, D., & Batra, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.06314 · Cited by 190 · see citing works
Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses
Rupprecht, C., Laina, I., DiPietro, R. S., Baust, M., Tombari, F., Navab, N., & Hager, G. D. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.388 · Cited by 185 · see citing works
Quality control stress test for deep learning-based diagnostic model in digital pathology
Schömig-Markiefka, B., Pryalukhin, A., Hulla, W., Bychkov, A., Fukuoka, J., Madabhushi, A., Achter, V., Nieroda, L., Büttner, R., Quaas, A., et al. (2021) · Modern Pathology · doi:10.1038/s41379-021-00859-x · Cited by 146 · see citing works
Karimi, D., Zeng, Q., Mathur, P., Avinash, A., Mahdavi, S. S., Spadinger, I., Abolmaesumi, P., & Salcudean, S. E. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.07.005 · Cited by 127 · see citing works
Contrastive Training for Improved Out-of-Distribution DetectionPreprint
Winkens, J., Bunel, R., Roy, A. G., Stanforth, R., Natarajan, V., Ledsam, J. R. E., MacWilliams, P., Kohli, P., Karthikesalingam, A., Köhl, S., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.05566 · Cited by 122 · see citing works
Ianni, J. D., Soans, R. E., Sankarapandian, S., Chamarthi, R. V., Ayyagari, D., Olsen, T. G., Bonham, M. J., Stavish, C. C., Motaparthi, K., Cockerell, C. J., et al. (2020) · Scientific Reports · doi:10.1038/s41598-020-59985-2 · Cited by 113 · see citing works
Guha Roy, A., Ren, J., Azizi, S., Loh, A., Natarajan, V., Mustafa, B., Pawlowski, N., Freyberg, J., Liu, Y., Beaver, Z., et al. (2022) · Medical Image Analysis · doi:10.1016/j.media.2021.102274 · Cited by 77 · see citing works
Bándi, P., Balkenhol, M. C. A., van Ginneken, B., van der Laak, J. A. W. M., & Litjens, G. J. S. (2019) · PeerJ · doi:10.7717/peerj.8242 · Cited by 61 · see citing works
Can You Trust Predictive Uncertainty Under Real Dataset Shifts in Digital Pathology?
Thagaard, J., Hauberg, S., van der Vegt, B., Ebstrup, T., Hansen, J. D., & Dahl, A. B. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59710-8_80 · Cited by 36 · see citing works
Transformer-based out-of-distribution detection for clinically safe segmentationPreprint
Graham, M., Tudosiu, P.-D. D., Wright, P., Pinaya, W. H. L., U-King-im, J., Mah, Y., Teo, J., Jäger, H., Werring, D., Nachev, P., et al. (2022) · International Conference on Medical Imaging with Deep Learning · doi:10.48550/arxiv.2205.10650 · Cited by 28 · see citing works
Linmans, J., van der Laak, J., & Litjens, G. J. S. (2020) · Medical Imaging with Deep Learning · Cited by 16 · see citing works
Incidental lymphoma during sentinel lymph node biopsy for breast cancer.
Fox, J. P., Grignol, V. P., Gustafson, J., Cheng, P., Weighall, R., Ouellette, J. R., Hellan, M., Dowdy, Y. G., & Termuhlen, P. H. (2010) · Journal of Clinical Oncology · doi:10.1200/jco.2010.28.15_suppl.e11083 · Cited by 3 · see citing works
Non-prostatic pathology on prostate needle-biopsy – colorectal carcinoid: a case report
van den Bergh, R. C. N., Wolters, T., Spaander, M. C. W., Schröder, F. H., & van Leenders, G. J. L. H. (2009) · Cases Journal · doi:10.1186/1757-1626-2-75 · Cited by 2 · see citing works
Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision
Gustafsson, F. K., Danelljan, M., & Schön, T. B. (2019) · arXiv · doi:10.48550/arxiv.1906.01620
Flagged first, then by citations
Cited by
Sorted by citations
Artificial intelligence for digital and computational pathology
Song, A. H., Jaume, G., Williamson, D. F. K., Lu, M. Y., Vaidya, A., Miller, T. R., & Mahmood, F. (2023) · Nature Reviews Bioengineering · doi:10.1038/s44222-023-00096-8 · Cited by 356 · see citing works
Lambert, B., Forbes, F., Doyle, S., Dehaene, H., & Dojat, M. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102830 · Cited by 318 · see citing works
A review of uncertainty estimation and its application in medical imaging
Zou, K., Chen, Z., Yuan, X., Shen, X., Wang, M., & Fu, H. (2023) · Meta-Radiology · doi:10.1016/j.metrad.2023.100003 · Cited by 173 · see citing works
A comprehensive survey on deep active learning in medical image analysis
Wang, H., Jin, Q., Li, S., Liu, S., Wang, M., & Song, Z. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103201 · Cited by 139 · see citing works
Huang, L., Ruan, S., Xing, Y.-C., & Feng, M.-L. (2024) · Medical Image Anal. · doi:10.1016/j.media.2024.103223 · Cited by 90 · see citing works
The importance of resource awareness in artificial intelligence for healthcare
Jia, Z., Chen, J., Xu, X., Kheir, J. N., Hu, J., Xiao, H., Peng, S., Hu, X. S., Chen, D., & Shi, Y. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00670-0 · Cited by 85 · see citing works
Domain Generalization in Computational Pathology: Survey and Guidelines
Jahanifar, M., Raza, M., Xu, K., Vuong, T. T. L., Jewsbury, R., Shephard, A., Tajeddin, N. Z., Kwak, J. T., Raza, S. E. A., Minhas, F., et al. (2025) · ACM Computing Surveys · doi:10.1145/3724391 · Cited by 67 · see citing works
“These transformations, when applied in CPath studies [2, 57, 88, 101, 114, 170, 256, 261], have shown promising improvements in model generalizability against prevalent variances in histology datasets, such as color variations and compression artifacts [170, 204, 221].”
AdvMIL: Adversarial multiple instance learning for the survival analysis on whole-slide images
Liu, P., Ji, L., Ye, F., & Fu, B. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.103020 · Cited by 55 · see citing works
Diffusion models for out-of-distribution detection in digital pathology
Linmans, J., Raya, G., van der Laak, J. A. W. M., & Litjens, G. J. S. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103088 · Cited by 40 · see citing works
Prediction Variability to Identify Reduced AI Performance in Cancer Diagnosis at MRI and CT
Alves, N., Bosma, J. S., Venkadesh, K. V., Jacobs, C., Saghir, Z., de Rooij, M., Hermans, J. J., & Huisman, H. J. (2023) · Radiology · doi:10.1148/radiol.230275 · Cited by 34 · see citing works
Probabilistic Modeling of Inter- and Intra-observer Variability in Medical Image Segmentation
Schmidt, A., Morales-Álvarez, P., & Molina, R. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.01929 · Cited by 32 · see citing works
“Especially in the medical domain, the careful modeling of uncertainties in its different forms has a high priority to minimize the risk of relying on incorrect predictions [11, 3, 12, 1, 13].”
Backgroundvan Eekelen, L., Litjens, G. J. S., & Hebeda, K. M. (2023) · Pathobiology · doi:10.1159/000529701 · Cited by 22 · see citing works
“Another relevant development for digital pathology from the ML field is out-of-distribution detection [50].”
BackgroundReliability engineering, risk management, and trustworthiness assurance for AI systems
Zhang, X., Wang, T., Ma, L., & Mahadevan, S. (2025) · Journal of Reliability Science and Engineering · doi:10.1088/3050-2454/adcef2 · Cited by 23 · see citing works
Kanwal, N., Khoraminia, F., Kiraz, U., Mosquera‐Zamudio, A., Monteagudo, C. C., Janssen, E. A. M., Zuiverloon, T. C. M., Rong, C., & Engan, K. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02676-z · Cited by 19 · see citing works
Shi, J., Li, C., Gong, T., & Fu, H. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103294 · Cited by 18 · see citing works
Uncertainty-aware ensemble of foundation models differentiates glioblastoma from its mimics
Zhao, J., Lin, S.-Y., Attias, R., Mathews, L., Engel, C., Larghero, G., Vremenko, D., Kao, T., Lee, T.-H., Wang, Y.-H., et al. (2025) · Nature Communications · doi:10.1038/s41467-025-64249-6 · Cited by 19 · see citing works
“Similar to conformal prediction frameworks 48 , our Bayesian method offers a mathematically principled approach for estimatinguncertaintyinpredictions,thereby enhancingthereliability of model output 44 .”
Mehrtens, H. A., Kurz, A., Bucher, T.-C., & Brinker, T. J. (2023) · Medical Image Analysis · doi:10.1016/j.media.2023.102914 · Cited by 18 · see citing works
“Independently and concurrent to our work, [Linmans et al., 2023] compared prominent methods for uncertainty estimation on histopathological slides, with a focus on out-of-distribution detection of foreign tissue in breast and prostate tissues.”
MethodResultMulti-Head Multi-Loss Model Calibration
Galdrán, A., Verjans, J. W. H., Carneiro, G., & Ballester, M. Á. G. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-43898-1_11 · Cited by 15 · see citing works
Data-Driven Quantitation of Movement Abnormality after Stroke
Parnandi, A., Kaku, A., Venkatesan, A., Pandit, N. G., Fokas, E. E., Yu, B., Kim, G. J., Nilsen, D. M., Fernandez‐Granda, C., & Schambra, H. M. (2023) · Bioengineering · doi:10.3390/bioengineering10060648 · Cited by 13 · see citing works
“OOD detection has been increasingly used in medical image processing, for example, to identify and score brain lesions in MRIs [31], to distinguish malignancies in histopathological samples [32], to detect skin cancer in images [33], and to detect malaria in blood smears [33].”
MethodPRISM: A Promptable and Robust Interactive Segmentation Model with Visual Prompts
Li, H., Liu, H., Hu, D., Wang, J., & Oguz, I. (2024) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-72384-1_37 · Cited by 12 · see citing works
Wei, J., Wang, G., & Zhang, S. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.111401 · Cited by 12 · see citing works
Managing the unknown in machine learning: Definitions, related areas, recent advances, and prospects
Barcina-Blanco, M., Lobo, J. L., Bringas, P. G., & Ser, J. D. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128073 · Cited by 11 · see citing works
de Vente, C., van Ginneken, B., Hoyng, C. B., Klaver, C. C. W., & Sánchez, C. I. (2024) · Medical Image Analysis · doi:10.1016/j.media.2024.103259 · Cited by 11 · see citing works
“Furthermore, [30, 31] used multi-head CNNs, an approach similar to deep ensembles, to detect images with lymphoma in histopathology as OOD samples.”
MethodPeeters, D., Alves, N., Venkadesh, K. V., Dinnessen, R., Saghir, Z., Scholten, E. T., Schaefer‐Prokop, C. M., Vliegenthart, R., Prokop, M., & Jacobs, C. (2024) · European Radiology · doi:10.1007/s00330-024-10714-7 · Cited by 10 · see citing works
Liu, T., Gu, Y., Chen, H., Zhang, Y., Zheng, L., Huang, X., Xu, Y., Wen, C., Chen, M.-S., Lin, J., et al. (2025) · Communications Medicine · doi:10.1038/s43856-025-01052-w · Cited by 8 · see citing works
A multimodal uncertainty-aware AI system optimizes ovarian cancer risk assessment workflow
Wang, X., Lv, X., Wang, J., Zou, L., Chen, Z., Zhao, R., Zhao, L., Zhao, M., Zhang, X., Zhang, B., et al. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01986-4 · Cited by 8 · see citing works
Toward confident prostate cancer detection using ultrasound: a multi-center study
Wilson, P., Harmanani, M., To, M. N. N., Gilany, M., Jamzad, A., Fooladgar, F., Wodlinger, B., Abolmaesumi, P., & Mousavi, P. (2024) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-024-03119-w · Cited by 8 · see citing works
StaDis: Stability distance to detecting out-of-distribution data in computational pathology
Zhang, D., Ge, J., Liu, J., Wang, C., Gong, T., Gao, Z., & Li, C. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103774 · Cited by 7 · see citing works
Pocevičiūtė, M., Ding, Y., Bromée, R., & Eilertsen, G. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109327 · Cited by 5 · see citing works
Assessing informative tract segmentation and nTMS for pre-operative planning
Lucena, O., Lavrador, J. P., Irzan, H., Semedo, C., Borges, P., Vergani, F., Granados, A., Sparks, R. E., Ashkan, K., & Sebastien Ourselin, A. (2023) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2023.109933 · Cited by 5 · see citing works
“Therefore, uncertainty from patients where tumors considerably distort the CST anatomy will be computed based on in-distribution healthy subject data, and further characterization is needed to flag out-of-distribution (OoD) data for patients with tumors not adequately captured by the current model limit models from fa…”
BackgroundAre We Ready for Out-of-Distribution Detection in Digital Pathology?
Oh, J.-H., Falahkheirkhah, K., & Bhargava, R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72117-5_8 · Cited by 4 · see citing works
He, Q., Xu, Y., Huang, Q., Wang, Y., Ye, J., He, Y., Li, J., Zhu, L., Wang, Z., & Guan, T. (2024) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2024.104745 · Cited by 4 · see citing works
Uncertainty Estimation for Out-of-Distribution Detection of Whole Slide Images
Gheshlaghi, S. H., Yahyasoltani, N., & Ganji, M. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10889349 · Cited by 4 · see citing works
“It is calculated as follows: where C is the number of classes and x ∗ is unseen data and p ( y c | x ∗ ) is the predicted probability for class c [12]–[15].”
C-UQ: Conflict-based uncertainty quantification—A case study in lung cancer classification
Zahari, R., Cox, J., & Obara, B. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.109825 · Cited by 3 · see citing works
Generalisation and reliability of deep learning for digital pathology in a clinical setting
Pocevičiūtė, M. (2023) · Linköping studies in science and technology. Dissertations · doi:10.3384/9789180753005 · Cited by 3 · see citing works
“There are several frameworks developed for OOD detection in digital pathology applications [11, 76].”
BackgroundTo, M. N. N., Fooladgar, F., Wilson, P., Harmanani, M., Gilany, M., Sojoudi, S., Jamzad, A., Chang, S. D., Black, P. C., Mousavi, P., et al. (2024) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-024-03104-3 · Cited by 3 · see citing works
Explainability-based graph augmentation for out-of-distribution robustness in digital pathology
Gheshlaghi, S. H., Aryal, M., Yahyasoltani, N., & Ganji, M. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.113640 · Cited by 3 · see citing works
Peeters, D., Venkadesh, K. V., Dinnessen, R., Saghir, Z., Scholten, E. T., Vliegenthart, R., Prokop, M., & Jacobs, C. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109633 · Cited by 2 · see citing works
D-EDL: Differential evidential deep learning for robust medical out-of-distribution detection
Fu, W., Chen, Y., Liu, Y., & Yue, X. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103888 · Cited by 2 · see citing works
A nursing perspective on human-AI collaboration in personalized breast cancer care pathways
Zhang, J., Zhao, X., Tao, J.-H., & Su, D.-C. (2026) · Frontiers in Oncology · doi:10.3389/fonc.2026.1784401 · Cited by 2 · see citing works
“…mandatory escalation protocols are required in three speci fi c scenarios: when the AI system indicates low con fi dence in its output, when it operates outside its validated scope, or when a disagreement between the clinician and the algorithm involves potentially high-risk outcomes (63 – 65).”
Hosseini, S. M. M., Hannetel, P., Cataldo, S. D., Descombes, X., Sibony, M., Decaussin‐Petrucci, M., Ponzio, F., & Ambrosetti, D. (2026) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2026.109373 · Cited by 2 · see citing works
Sliding Window Optimal Transport for Open World Artifact Detection in Histopathology
Fuchs, M., Konstantin, M., Schrade, N., Schweizer, L., Tolkach, Y., & Mukhopadhyay, A. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3383590 · Cited by 2 · see citing works
García-de-la-Puente, N. P., López-Pérez, M., Launet, L., & Naranjo, V. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-72083-3_6 · Cited by 2 · see citing works
Navigating the unknown: out-of-distribution detection for medical imaging
Fuchs, M., Angelopoulos, A. N., Paschali, M., Baumgartner, C. F., & Mukhopadhyay, A. (2025) · Elsevier eBooks · doi:10.1016/b978-0-44-323761-4.00013-4 · Cited by 2 · see citing works
Assessing the quality of whole slide images in cytology from nuclei features
Barthe, P., Brixtel, R., Caillot, Y., Lemoine, B., Renouf, A., Thurotte, V., Beniken, O., Bougleux, S., & Lézoray, O. (2025) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2025.100420 · Cited by 2 · see citing works
Artificial Intelligence in Dermatopathology: An Update and Review of the Current Literature
Abu-Dayeh, A., Cazzato, G., & Giubellino, A. (2026) · Diagnostics · doi:10.3390/diagnostics16111702 · Cited by 1 · see citing works
“These approaches can reduce review time and improve safety but face challenges such as spurious attention to artifacts, sensitivity-specificity trade-offs in small-lesion detection, and overconfidence on out-of-distribution detection [33,35,36].”
Borah, J., Singh, H. K., & Sarmah, K. (2024) · SN Computer Science · doi:10.1007/s42979-024-03164-x · Cited by 1 · see citing works
“However, only a few prior studies have presented structured evaluations of DNNs’ automatic perceptions when dealing with images in terms of distribution shifts in medical domain [5–12].”
A human-in-the-loop framework for refining deep learning models in pathology segmentation
Khalili, N., Spronck, J., Ciompi, F., van der Laak, J. A. W. M., & Litjens, G. J. S. (2025) · SPIE Medical Imaging: Digital and Computational Pathology · doi:10.1117/12.3046750 · Cited by 1 · see citing works
“6 These ensembles, designed as collections of multiple deep-learning model instances, use the differences in their predictions to assess uncertainty.”
Larsen, K. S., Chen, Z., He, Z., Keyak, J. H., Sha, Q., Páez, D., Zhang, X., Hung, G., Zou, J., Peix, A., et al. (2025) · Journal of Imaging Informatics in Medicine · doi:10.1007/s10278-025-01666-5 · Cited by 1 · see citing works
“Linmans et al. utilized ensemble uncertainty quantification for tumor detection [27].”
ADAptation: Reconstruction-Based Unsupervised Active Learning for Breast Ultrasound Diagnosis
Duan, Y., Huang, Y., Yang, X., Han, L., Xie, X., Zhu, Z., He, P., Chan, K., Cui, L., Im, S., et al. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05325-1_4 · Cited by 1 · see citing works
Artificial Intelligence Basics for Head and Neck Pathologists: A Simple Guide
Wang, R., & Gibson, T. M. (2026) · Head and Neck Pathology · doi:10.1007/s12105-026-01892-5 · Cited by 1 · see citing works
Cai, L., Meng, B., Huang, J., Ding, G., Ju, M., Wang, W.-W., Shi-jie, D., Lai, L., Wang, J., Yang, C.-X., et al. (2026) · LabMed discovery. · doi:10.1016/j.lmd.2026.100130 · Cited by 1 · see citing works
Marchevsky, A. M., & Walts, A. E. (2026) · Annals of Diagnostic Pathology · doi:10.1016/j.anndiagpath.2026.152677 · Cited by 1 · see citing works
Steinmetz, P., Frouin, F., Morard, V., & Buvat, I. (2026) · doi:10.64898/2026.05.04.26350496
Benefits of spatial uncertainty aggregation for segmentation in digital pathology
Pocevičiūtė, M., Eilertsen, G., & Lundström, C. F. (2024) · Journal of medical imaging · doi:10.1117/1.jmi.11.1.017501
“For example, previous works have examined the correlation between estimated uncertainty and mispredictions 14,20 or outliers 27 in classification, which could be used to reduce the error in performance.”
ResultKanwal, N., Khoraminia, F., Kiraz, U., Mosquera‐Zamudio, A., Monteagudo, C. C., Janssen, E. A. M., Zuiverloon, T. C. M., Rong, C., & Engan, K. (2024) · Research Square · doi:10.21203/rs.3.rs-4092609/v1
Exploring distribution-based approaches for out-of-distribution detection in deep learning models
Carvalho, T. M., Vellasco, M. M. B. R., & Amaral, J. F. (2024) · Neural Computing and Applications · doi:10.1007/s00521-024-10912-w
“In medical contexts, OOD detection methods must handle both near-OOD and far-OOD samples without compromising classi-fication accuracy [28], and deal with different types of images, such as 3D data from CT scans [43].”
Sáez-Maldonado, F. J., García, L., Cooper, L., Goldstein, J. A., Molina, R., & Katsaggelos, A. K. (2025) · IEEE Access · doi:10.1109/access.2025.3593420 · Cited by 2 · see citing works
“Our proposals are in fully agreement with [23]: OOD detection is a capability Computer Assisted Diagnosis ( CAD ) systems should be provided with.”
InfluentialUncertainty Quantification for Metaplasia Detection in Gastrointestinal Histopathology Images
Acosta-Velasquez, R., Cano, F., Romero, E., & Gonzalez, F. A. (2025) · International Symposium on Medical Information Processing and Analysis (SIPAIM) · doi:10.1109/sipaim67325.2025.11283194
Uncertainty Estimation for Graph-Based Learning in Digital Pathology
Gheshlaghi, S. H., Yahyasoltani, N., & Ganji, M. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3646956
“Unlike previous multihead architectures designed for convolutional networks [13], [15], which operate on independent image patches, our study extends the multihead concept to graph-structured WSI data.”
InfluentialSCULPT: Semantic-aware causal prompt tuning for out-of-distribution detection of whole slide images
Liang, D., Li, X., Liang, D., Liu, J., Zhu, Z., Li, X., Dong, S., Luo, G., Wang, W., Wang, K., et al. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.115432
High-sensitivity pan-cancer AI assessment of lymph node metastasis via uncertainty quantification
Wang, X., Chen, Y., Liu, X., Qiu, C., Tang, H., Huang, T., Guo, S., Ma, S., Cai, M., Sun, Q., et al. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02564-y
“Classical uncertainty estimation methods — such as uncertainty-prediction model co-training 28,29 , multiple model ensembles 30 , or repeated dropout strategies 28,31 — increase computational burden and often fl ag large unnecessary regions for manual reassessment.”
Zhang, A. W., Cecchini, M. J., Bashashati, A., Gilks, C. B., Kinloch, M. A., & Schaeffer, D. F. (2026) · Laboratory Investigation · doi:10.1016/j.labinv.2026.106131
SGPMIL: Sparse Gaussian Process Multiple Instance Learning
Lolos, A., Christodoulidis, S., Moustakas, A. L., Dolz, J., & Vakalopoulou, M. (2026) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61042.2026.00057
“…MIL in various forms, including Bayesian CNNs under full supervision [47], Bayesian instance-based MIL under data scarcity and label noise [19, 36], uncertainty-driven aggregation models that refine attention or instance selection scores [12, 42, 46, 52, 56] and OOD uncertainty estimation [29].”
Steiger, M., Rezapourian, M., Rak, M., & Hansen, C. (2026) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-026-03725-w
“Recent work has explored uncertainty quantification through Bayesian neural networks and deep ensembles, estimating both aleatoric and epistemic uncertainties [10, 11].”
Out-of-Distribution Detection in the PCA space of Vision Transformer
Riahi-Madvar, M., & Nasersharif, B. (2025) · International Conference on Signal Processing and Intelligent Systems (ICSPIS) · doi:10.1109/icspis68676.2025.11551699
“PViT (Prior-augmented Vision Transformer) [13] is a transformer-based framework designed to improve out-of-distribution detection by leveraging prior class logits from a pretrained model.”
Amin, A., & Zaman, W. (2026) · Food Reviews International · doi:10.1080/87559129.2026.2715571
OOD-aware Reliability Learning for open-world semi-supervised liver lesion recognition
Pu, Y., Xia, W., Deng, L., Xiao, Y., Liu, W., Shen, Z., Xia, J., Zou, X., Dong, S., & Li, R. (2026) · Computerized Medical Imaging and Graphics · doi:10.1016/j.compmedimag.2026.102810
Uncertainty-Aware Machine Learning for Reliable Decision-Making in Data-Driven Systems
Yulianingsih, E., Noche, E. B., Yadav, V., & Sharma, D. (2026) · Journal of Data Sciences · doi:10.61453/jods.v20260207
Li, A., Oloughlin, P., Yue, L., Zhang, W. E., Swart, B., & Chen, W. (2026) · Journal of Pathology Informatics · doi:10.1016/j.jpi.2026.100720
A universal visual foundation model for computational cytopathology
Zheng, X., Zheng, K., Wang, J., Zhang, X., Li, K., Zhao, Z., Li, C., Chen, H., Wang, R., Lai, J., et al. (2026) · Nature Cancer · doi:10.1038/s43018-026-01240-0
“To further assess real-world applicability, we evaluated CROWN on out-of-distribution data, specifically focusing on blood smear cytology, a domain entirely excluded from model training 49,50 .”
Stylianakis, D., Sánchez-Bayona, R., Denaro, N., & Solinas, C. (2026) · Frontiers in Pharmacology · doi:10.3389/fphar.2026.1953714
“In digital histopathology, dropout-based uncertainty estimation flags a substantial proportion of cases a model is likely to misclassify, allowing low-confidence predictions to be routed for expert review rather than acted upon automatically (Dolezal et al., 2022; Linmans et al., 2023).”
van Aalst, J. E., Simões, R., Maruccio, F. C., MacRae, D. C., der Hoek, L. V., Langendijk, J. A., van Ooijen, P. M. A., Janssen, T. M., & Brouwer, C. L. (2026) · Radiotherapy and Oncology · doi:10.1016/j.radonc.2026.111826
Luo, Y., Wang, M., Lin, T., Lin, A., Chen, Y., He, G., Lin, Z., Chen, L. J., Yam, J. C. S., Cheng, C., et al. (2026) · Cell Reports Medicine · doi:10.1016/j.xcrm.2026.103124
- Authors
- Linmans, J., Elfwing, S., van der Laak, J., & Litjens, G.
- Affiliations
- Radboud University Medical Center (NL)
- Corresponding author
- Linmans, Jasper
- Venue
- Medical Image Analysis
- Publisher
- Elsevier BV
- Published
- 2023-01
- Volume
- 83
- Pages
- 102655
- ISSN
- 1361-8415 (ISSN-L)
- OpenAlex
- W4306648046
- Cited by
- 75
- References
- 40
- Citations by year
- 2026: 20 · 2025: 20 · 2024: 22 · 2023: 13
- FWCI
- 4.52
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Medical Image Analysis (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DiVA (Linkoping University) (submittedVersion, other-oa)
- Radboud Repository (Radboud University) (submittedVersion)
- Radboud Repository (Radboud University) (submittedVersion, other-oa)
- Funders
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (91618152)
- Dutch Cancer Society (KUN 2015-7970)
- European Commission
- Innovative Medicines Initiative
- Knut and Alice Wallenberg Foundation
- European Federation of Pharmaceutical Industries and Associations
- Horizon 2020 Framework Programme
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- out-of-distribution detection, uncertainty estimation, digital pathology, whole slide images, ensemble deep learning, ensemble diversity
- MeSH terms
- Machine Learning, Humans, Pathology, Prospective Studies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
