The DOI matches its Crossref record; 65 of the 69 references it cites were found in a registry or an index, and 1 are not checked yet.
Leveraging uncertainty information from deep neural networks for disease detection
Open accessPublished version of a preprint
Abstract
Deep learning (DL) has revolutionized the field of computer vision and image processing. In medical imaging, algorithmic solutions based on DL have been shown to achieve high performance on tasks that previously required medical experts. However, DL-based solutions for disease detection have been proposed without methods to quantify and control their uncertainty in a decision. In contrast, a physician knows whether she is uncertain about a case and will consult more experienced colleagues if needed. Here we evaluate drop-out based Bayesian uncertainty measures for DL in diagnosing diabetic retinopathy (DR) from fundus images and show that it captures uncertainty better than straightforward alternatives. Furthermore, we show that uncertainty informed decision referral can improve diagnostic performance. Experiments across different networks, tasks and datasets show robust generalization. Depending on network capacity and task/dataset difficulty, we surpass 85% sensitivity and 80% specificity as recommended by the NHS when referring 0−20% of the most uncertain decisions for further inspection. We analyse causes of uncertainty by relating intuitions from 2D visualizations to the high-dimensional image space. While uncertainty is sensitive to clinically relevant cases, sensitivity to unfamiliar data samples is task dependent, but can be rendered more robust.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures








Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Leibig al. [50] | et | 2017 | CNN | Classification | Diabetic Retinopathy | 3 | MC dropout”
From A review of uncertainty quantification in deep learning: Techniques, applications and challenges (2021) · cites this paper as [50]
Experimental — accuracy not yet measuredIn the citing paper
“This human-in-the-loop decision-making pipeline is known as selective prediction and has been commonly found to improve performance in non–text-based applications [25,26].”
From Identifying Deprescribing Opportunities With Large Language Models in Older Adults: Retrospective Cohort Study (2025) · cites this paper as [25]
In the citing paper
“In addition to producing valuable measures that function as uncertainty measures [358, 359, 66], these techniques can also lessen deep neural networks susceptibility to adversarial attacks [357, 360].”
From An overview of deep learning in medical imaging focusing on MRI (2019) · cites this paper as [358]
In the citing paper
“However, the intraobserver agreement was far from perfect, which is likely due to multiple factors, including the inherent model uncertainty of the network structure (43,44), the stochastic nature of network training, and the variation between the training and testing image data sets (24,45).”
From Deep learning approach for evaluating knee MR images: achieving high diagnostic performance for cartilage lesion detection. (2018) · cites this paper as [45]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 8×
“960] Messidor Single best44 (0) vs (1, 2, 3) 0.”
BackgroundInfluentialLeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · cited 1×
“The space of images with content unknown to the disease detection networks was sampled by performing predictions with associated uncertainties on the 2012 Imagenet validation set (49101 coloured, non-fundus images from 1000 different categories) 58 (Fig.”
Method(2014) · Cited by 33,419
Advances in neural information processing systems 7
Cios, K., & Shields, M. (1997) · Neurocomputing · doi:10.1016/s0925-2312(97)90018-7 · Cited by 22,059
Deep learning in neural networks: an overview
Schmidhuber, J. (2015) · Neural Networks · doi:10.1016/j.neunet.2014.09.003 · Cited by 18,570 · cited 7×
“960] Messidor Single best44 (0) vs (1, 2, 3) 0.”
BackgroundMethodInfluentialDermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362 · cited 1×
“This paradigm can readily be applied to recently published high performance disease detection networks 14,15 as well as other medical tasks and datasets as initial work on image registration 60 and genome data 61 has already shown.”
Method(2010) · International Conference on Machine Learning · Cited by 13,995
Gaussian Processes for Machine Learning
Rasmussen, C. E., & Williams, C. K. I. (2005) · doi:10.7551/mitpress/3206.001.0001 · Cited by 11,705
Research works printed but not linked: 8
- Lipton, Z. C., Kale, D. C., Elkan, C. & Wetzell, R. Learning to Diagnose with LSTM Recurrent Neural Networks. International Conference on Learning Representations (ICLR) (2016).
- Kingma, D. P., Salimans, T. & Welling, M. Variational Dropout and the Local Reparameterization Trick. Advances in Neural Information Processing Systems (2015).
- Bishop, C. M. Pattern Recognition and Machine Learning, Springer-Verlag New York, Inc., Secaucus, NJ, USA, (2006).
- Gal, Y. Uncertainty in Deep Learning. Ph.D. thesis, University of Cambridge (2016).
- Angermueller, C. & Stegle, O. Multi-task deep neural network to predict CpG methylation profiles from low-coverage sequencing data. NIPS MLCB workshop (2015).
- Damianou, A. C. & Lawrence, N. D. Deep Gaussian Processes. International Conference on Artificial Intelligence and Statistics 31, 207–215 (2013).
- Cho, E. & Cho, M. J. Variance of sample variance with replacement. International Journal of Pure and Applied Mathematics 52, 43–47 (2009).
- Al-Rfou, R. et al. Theano: A Python framework for fast computation of mathematical expressions. arXiv:1605.02688 (2016). · arXiv
Cited by
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 3,111
“[45] 2017 CNN diabetic retinopathy √”
Opportunities and obstacles for deep learning in biology and medicine
Ching, T., Himmelstein, D. S., Beaulieu-Jones, B. K., Kalinin, A. A., Do, B. T., Way, G. P., Ferrero, E., Agapow, P.-M., Zietz, M., Hoffman, M. M., et al. (2018) · Journal of The Royal Society Interface · doi:10.1098/rsif.2017.0387 · Cited by 2,310
“[45] estimated the uncertainty of deep networks for diabetic retinopathy diagnosis by linking dropout networks with approximate Bayesian inference.”
BackgroundAn overview of deep learning in medical imaging focusing on MRI
Lundervold, A. S., & Lundervold, A. (2019) · Zeitschrift für Medizinische Physik · doi:10.1016/j.zemedi.2018.11.002 · Cited by 2,058
“In addition to producing valuable measures that function as uncertainty measures [65,355,356], these techniques can also lessen deep neural networks susceptibility to adversarial attacks [354,357].”
BackgroundA Unifying Review of Deep and Shallow Anomaly Detection
Ruff, L., Kauffmann, J. R., Vandermeulen, R. A., Montavon, G., Samek, W., Kloft, M., Dietterich, T. G., & Muller, K.-R. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3052449 · Cited by 914
A Survey of Deep Active Learning
Ren, P., Xiao, Y., Chang, X., Huang, P.-Y., Li, Z., Gupta, B. B., Chen, X., & Wang, X. (2022) · ACM Computing Surveys · doi:10.1145/3472291 · Cited by 1,169
“In addition, in practice, researchers often also use a powerful and low-cost Monte-Carlo dropout (MC-dropout) [156] stochastic regularization technique to obtain posterior samples, consequently attaining good performance on real-world datasets [80, 94].”
Method
Metadata
- Authors
- Leibig, C., Allken, V., Ayhan, M. S., Berens, P., & Wahl, S.
- Affiliations
- University of Tübingen (DE)
- Bernstein Center for Computational Neuroscience Tübingen (DE)
- Carl Zeiss (Germany) (DE)
- Corresponding author
- Leibig, Christian
- Venue
- Scientific Reports
- Published
- 2017-12-19
- Volume
- 7
- Issue
- 1
- Pages
- 17816-17816
- ISSN
- 2045-2322 (ISSN-L)
- PMC
- PMC5736701
- OpenAlex
- W2951965145
- Cited by
- 510
- References
- 69
- Citations by year
- 2026: 47 · 2025: 50 · 2024: 41 · 2023: 63 · 2022: 60 · 2021: 90 · 2020: 80 · 2019: 52 · 2018: 25 · 2017: 2
- FWCI
- 33.37
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Scientific Reports (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nc)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Deutsche Forschungsgemeinschaft (EXC307)
- Eberhard Karls Universität Tübingen (EXC 307)
- Bundesministerium für Bildung und Forschung (01GQ1601)
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- Bayesian uncertainty quantification, deep neural network, diabetic retinopathy detection, fundus images, diagnostic decision-making, uncertainty visualization, medical image analysis, generalization to unseen data
- MeSH terms
- Machine Learning, Deep Learning, Algorithms, Bayes Theorem, Diabetic Retinopathy, Female, Humans, Image Processing, Computer-Assisted, Retina, Sensitivity and Specificity, Neural Networks, Computer, Uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Leveraging uncertainty information from deep neural networks for disease detection. https://app.cochat.ai/papers/10.1038/s41598-017-17876-z
- The paper
- Leibig, C., Allken, V., Ayhan, M. S., Berens, P., & Wahl, S. (2017). Leveraging uncertainty information from deep neural networks for disease detection. Scientific Reports. https://doi.org/10.1038/s41598-017-17876-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.