Towards Trustworthy AI in Healthcare: Epistemic Uncertainty Estimation for Clinical Decision Support
Open accessThe DOI matches its Crossref record; 48 of the 49 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Introduction: Widespread adoption of AI for medical decision-making is still hindered due to ethical and safety-related concerns. For AI-based decision support systems in healthcare settings, it is paramount to be reliable and trustworthy. Common deep learning approaches, however, have the tendency towards overconfidence when faced with unfamiliar or changing conditions. Inappropriate extrapolation beyond well-supported scenarios may have dire consequences highlighting the importance of the reliable estimation of local knowledge uncertainty and its communication to the end user. Materials and Methods: While neural network ensembles (ENNs) have been heralded as a potential solution to these issues for many years, deep learning methods, specifically modeling the amount of knowledge, promise more principled and reliable behavior. This study compares their reliability in clinical applications. We centered our analysis on experiments with low-dimensional toy datasets and the exemplary case study of mortality prediction for intensive care unit hospitalizations using Electronic Health Records (EHRs) from the MIMIC3 study. For predictions on the EHR time series, Encoder-Only Transformer models were employed. Knowledge uncertainty estimation is achieved with both ensemble and Spectral Normalized Neural Gaussian Process (SNGP) variants of the common Transformer model. We designed two datasets to test their reliability in detecting token level and more subtle discrepancies both for toy datasets and an EHR dataset. Results: While both SNGP and ENN model variants achieve similar prediction performance (AUROC: ≈0.85, AUPRC: ≈0.52 for in-hospital mortality prediction from a selected MIMIC3 benchmark), the former demonstrates improved capabilities to quantify knowledge uncertainty for individual samples/patients. Discussion/Conclusions: Methods including a knowledge model, such as SNGP, offer superior uncertainty estimation compared to traditional stochastic deep learning, leading to more trustworthy and safe clinical decision support.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Additionally, stochastic neural networks have been found to inadequately represent epistemic uncertainty, creating a false impression of confidence and limiting their reliability for clinical decision support [6,7].”
From Enhancing Clinician Trust in AI Diagnostics: A Dynamic Framework for Confidence Calibration and Transparency (2025) · cites this paper as [7]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 3×
“While in the landmark paper [25], temporal information is given via temporal embedding and many others have relied on this approach, the studies [33,34] developed other explicit methods of including arbitrary temporal information.”
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Goldberger, A. L., Amaral, L. A. N., Glass, L., Hausdorff, J. M., Ivanov, P. C., Mark, R. G., Mietus, J. E., Moody, G. B., Peng, C.-K., & Stanley, H. E. (2000) · Circulation · doi:10.1161/01.cir.101.23.e215 · Cited by 14,809
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141
A review of uncertainty quantification in deep learning: Techniques, applications and challenges
Abdar, M., Pourpanah, F., Hussain, S., Rezazadegan, D., Liu, L., Ghavamzadeh, M., Fieguth, P., Cao, X., Khosravi, A., Acharya, U. R., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.05.008 · Cited by 3,111 · cited 4×
“However, with the goal of introducing AI into medical practice on a large scale, AI models are not only expected to be performant but also safe and trustworthy under real-world circumstances [7,8,13,15,16,24,31,32].”
InfluentialBig data analytics in healthcare: promise and potential
Raghupathi, W., & Raghupathi, V. (2014) · Health Information Science and Systems · doi:10.1186/2047-2501-2-3 · Cited by 3,034 · cited 1×
“Electronic Health Records (EHRs) encapsulate an immense volume of data, encompassing intricate patient histories, treatment pathways, diagnostic information, and clinical outcomes [1,2].”
Shickel, B., Tighe, P. J., Bihorac, A., & Rashidi, P. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2767063 · Cited by 1,688 · cited 4×
“Machine learning (ML) approaches have shown promising results in the analysis of EHR data in plenty of studies centered around a multitude of predictive clinical applications [9].”
InfluentialA survey of uncertainty in deep neural networks
Gawlikowski, J., Tassi, C. R. N., Ali, M., Lee, J., Humt, M., Feng, J., Kruspe, A., Triebel, R., Jung, P., Roscher, R., et al. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10562-9 · Cited by 1,406 · cited 3×
“Among these methods, model ensembles and implicit ensembling methods such as dropout or stochastic model parameters stand out as particularly prominent and are as of today widely used for uncertainty quantification in diverse fields [15].”
Uncertainty Quantification: Theory, Implementation, and Applications
Smith, R. C. (2013) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611973228 · Cited by 1,138 · cited 1×
“KU can thus be regarded as a measure of the extent to which the predictions are supported by the evidence [7,42,43] (i.e., high KU indicates little/no data supporting the prediction).”
RETAIN: an interpretable predictive model for healthcare using reverse time attention mechanism
Choi, E., Bahadori, M. T., Kulas, J. A., Schuetz, A., Stewart, W. F., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1608.05745 · Cited by 1,029 · cited 1×
“Building on prior work, they propose RETAIN, an architecture more specifically targeted and inspired by medical decision-making [28].”
Ledley, R. S., & Lusted, L. B. (1959) · Science · doi:10.1126/science.130.3366.9 · Cited by 991 · cited 1×
“Early analysis of medical reasoning processes and the wish for computerized support in the medical field has been around since 1959 [17] and uncountable progress has been made in the field ever since [9,18,19].”
Multitask learning and benchmarking with clinical time series data
Harutyunyan, H., Khachatrian, H., Kale, D. C., Ver Steeg, G., & Galstyan, A. (2019) · Scientific Data · doi:10.1038/s41597-019-0103-9 · Cited by 958 · cited 7×
“The authors of [22] apply a Transformer to the [29] benchmarks.”
InfluentialArtificial intelligence, bias and clinical safety
Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2018-008370 · Cited by 956
Benefits and drawbacks of electronic health record systems
Menachemi, N., & Collum (2011) · Risk Management and Healthcare Policy · doi:10.2147/rmhp.s12985 · Cited by 909
Addressing bias in big data and AI for health care: A call for open science
Norori, N., Hu, Q., Aellen, F. M., Faraci, F. D., & Tzovara, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100347 · Cited by 903
Rasmy, L., Xiang, Y., Xie, Z., Tao, C., & Zhi, D. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00455-y · Cited by 886 · cited 4×
“The to-ken nomenclature is borrowed from the NLP domain and indeed there are similarities between language information and EHR records that have been exploited by numerous works [9,18,22,34,38].”
InfluentialThe need for uncertainty quantification in machine-assisted medical decision making
Begoli, E., Bhattacharya, T., & Kusnezov, D. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-018-0004-1 · Cited by 597 · cited 1×
“In the healthcare domain, where decisions are safety-critical and ethically relevant, it is imperative that such automated methods employed in clinical decision support systems (CDSSs) are not only effective but also trustworthy [6–8, 10, 11].”
Explainable Prediction of Medical Codes from Clinical Text
Mullenbach, J., Wiegreffe, S., Duke, J., Sun, J., & Eisenstein, J. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1100 · Cited by 597
BEHRT: Transformer for Electronic Health Records
Li, Y., Rao, S., Solares, J. R. A., Hassaine, A., Ramakrishnan, R., Canoy, D., Zhu, Y., Rahimi, K., & Salimi-Khorshidi, G. (2020) · Scientific Reports · doi:10.1038/s41598-020-62922-y · Cited by 595 · cited 1×
“Multiple studies by different autors have extensively investigated the perfromance of Transformer-based models [18,35,36] on different large-scale real-world medical datasets, showcasing the performance beyond typical benchmark datasets and potential real-word applications.”
(2016) · PubMed · Cited by 576
Second opinion needed: Communicating uncertainty in medical machine learning
Kompa, B., Snoek, J., & Beam, A. L. (2021) · npj Digital Medicine · doi:10.1038/s41746-020-00367-3 · Cited by 514 · cited 4×
“Providing a measure of uncertainty for a given decision can help alleviate such distrust [7,8,31,32].”
InfluentialAttend and Diagnose: Clinical Time Series Analysis Using Attention Models
Song, H., Rajan, D., Thiagarajan, J., & Spanias, A. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11635 · Cited by 468
Machine learning for precision medicine
MacEachern, S. J., & Forkert, N. D. (2021) · Genome · doi:10.1139/gen-2020-0131 · Cited by 455 · cited 1×
“This limitation necessitates the development of automated methods capable of identify complex patterns, summarizing vast data [3–5], and indicating critical points that require human attention [6–8].”
Introduction to Uncertainty Quantification
Sullivan, T. J. (2015) · Texts in Applied Mathematics · doi:10.1007/978-3-319-23395-6 · Cited by 432
Big Data Analytics in Medicine and Healthcare
Ristevski, B., & Chen, M. (2018) · Journal of Integrative Bioinformatics · doi:10.1515/jib-2017-0030 · Cited by 373 · cited 1×
“This limitation necessitates the development of automated methods capable of identify complex patterns, summarizing vast data [3–5], and indicating critical points that require human attention [6–8].”
What Clinicians Want: Contextualizing Explainable Machine Learning for Clinical End Use
Tonekaboni, S., Joshi, S., McCradden, M. D., & Goldenberg, A. (2019) · arXiv · doi:10.48550/arxiv.1905.05134 · Cited by 318 · cited 2×
“Tapping into the patients timeline, these models access longitudinal information that is important for clinical decision-making [24].”
Pre-training of Graph Augmented Transformers for Medication Recommendation
Shang, J., Ma, T., Xiao, C., & Sun, J. (2019) · Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2019/825 · Cited by 311
–Omic and Electronic Health Record Big Data Analytics for Precision Medicine
Wu, P.-Y., Cheng, C.-W., Kaddi, C. D., Venugopalan, J., Hoffman, R., & Wang, M. D. (2017) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2016.2573285 · Cited by 303
Doctor AI: Predicting Clinical Events via Recurrent Neural Networks
Choi, E., Bahadori, M. T., Schuetz, A., Stewart, W. F., & Sun, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.05942 · Cited by 244 · cited 1×
“In [27], the authors developed DoctorAI that predicts next-visit diagnosis and medication codes utilizing a classical RNN-based architecture.”
Simple and principled uncertainty estimation with deterministic deep learning via distance awareness
Liu, J. Z., Lin, Z., Padhy, S., Tran, D., Bedrax-Weiss, T., & Lakshminarayanan, B. (2020) · arXiv · doi:10.48550/arxiv.2006.10108 · Cited by 237 · cited 4×
“In the scope of this paper, we focus on two, a classical, yet still widely used [15], model ensemble approach and the recently developed method of Spectral Normalized Neural Gaussian Processes [41].”
InfluentialSi, Y., Du, J., Li, Z., Jiang, X., Miller, T., Wang, F., Jim Zheng, W., & Roberts, K. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103671 · Cited by 235
Johnson, A., Pollard, T., & Mark, R. (2015) · PhysioNet · doi:10.13026/c2xw26 · Cited by 165
Hyperparameter ensembles for robustness and uncertainty quantification
Wenzel, F., Snoek, J., Tran, D., & Jenatton, R. (2020) · arXiv · doi:10.48550/arxiv.2006.13570 · Cited by 122 · cited 1×
“We utilize an approach similar to that of [44], making use of heterogeneous models in terms of overall structure, combining smaller and larger models varying the number of layers and layer sizes.”
Liang, J., Li, Y., Zhang, Z., Shen, D., Xu, J., Zheng, X., Wang, T., Tang, B., Lei, J., & Zhang, J. (2021) · Journal of Medical Internet Research · doi:10.2196/24813 · Cited by 100 · cited 1×
“…different use cases such as image detection have received a lot of attention even in the medical field, the recent large-scale collection of patient EHR data [20,21] and continued improvements in computational resources has lead to a whole new class of systems entering the medical field [9,22,23].”
Alanazi, A. (2023) · Cureus · doi:10.7759/cureus.45255 · Cited by 75 · cited 2×
“However, with the goal of introducing AI into medical practice on a large scale, AI models are not only expected to be performant but also safe and trustworthy under real-world circumstances [7,8,13,15,16,24,31,32].”
TAPER: Time-Aware Patient EHR Representation
Darabi, S., Kachuee, M., Fazeli, S., & Sarrafzadeh, M. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.2984931 · Cited by 72 · cited 1×
“While in [37] information contained in medical notes is utilized by training two separate models and combining their embedding for various downstream tasks, the authors of [38] focus specifically on semantical differences between in-and outpatients.”
Analyzing the role of model uncertainty for electronic health records
Dusenberry, M. W., Tran, D., Choi, E., Kemp, J., Nixon, J., Jerfel, G., Heller, K., & Dai, A. M. (2020) · Proceedings of the ACM Conference on Health, Inference, and Learning · doi:10.1145/3368555.3384457 · Cited by 58 · cited 5×
“Providing a measure of uncertainty for a given decision can help alleviate such distrust [7,8,31,32].”
InfluentialThe Lipschitz Constant of Self-Attention
Kim, H., Papamakarios, G., & Mnih, A. (2020) · arXiv · doi:10.48550/arxiv.2006.04710 · Cited by 44 · cited 2×
“However, recent advancements have introduced alternative formulations that uphold Lipschitz continuity [47–50], offering promising avenues for enhancing SNGP Transformers.”
Radial Bayesian Neural Networks: Beyond Discrete Support In Large-Scale Bayesian Deep Learning
Farquhar, S., Osborne, M., & Gal, Y. (2019) · arXiv · doi:10.48550/arxiv.1907.00865 · Cited by 43
Temporal Self-Attention Network for Medical Concept Embedding
Peng, X., Long, G., Shen, T., Wang, S., Jiang, J. B., & Blumenstein, M. (2019) · Proceedings · doi:10.1109/icdm.2019.00060 · Cited by 39 · cited 3×
“While in the landmark paper [25], temporal information is given via temporal embedding and many others have relied on this approach, the studies [33,34] developed other explicit methods of including arbitrary temporal information.”
Blinov, P., Avetisian, M., Kokh, V., Umerenkov, D., & Tuzhilin, A. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59137-3_11 · Cited by 39
CEHR-BERT: Incorporating temporal information from structured EHR data to improve prediction tasks
Pang, C., Jiang, X., Kalluri, K. S., Spotnitz, M., Chen, R., Perotte, A., & Natarajan, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.08585 · Cited by 30 · cited 3×
“The to-ken nomenclature is borrowed from the NLP domain and indeed there are similarities between language information and EHR records that have been exploited by numerous works [9,18,22,34,38].”
Building Trust in Deep Learning System towards Automated Disease Detection
Lim, Z. W., Lee, M. L., Hsu, W., & Wong, T. Y. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33019516 · Cited by 26 · cited 2×
“In the healthcare domain, where decisions are safety-critical and ethically relevant, it is imperative that such automated methods employed in clinical decision support systems (CDSSs) are not only effective but also trustworthy [6–8, 10, 11].”
Inpatient2Vec: Medical Representation Learning for Inpatients
Wang, Y., Xiao, X., Jin, T., Li, X., Xie, G., & Wang, J. (2019) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm47256.2019.8983281 · Cited by 22 · cited 2×
“The to-ken nomenclature is borrowed from the NLP domain and indeed there are similarities between language information and EHR records that have been exploited by numerous works [9,18,22,34,38].”
Popat, R., & Ive, J. (2023) · Frontiers in Digital Health · doi:10.3389/fdgth.2023.1188338 · Cited by 21 · cited 1×
“They tested ensembles of LSTMs as well as different configurations of Bayesian LSTMs. Work in [39] explores the role of uncertainty in enhancing collaborative decision-making in mental health care based on the MIMIC3 dataset.”
Clinical decision making under uncertainty: a bootstrapped counterfactual inference approach
Wu, H., Shi, W., Choudhary, A., & Wang, M. D. (2024) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-024-02606-z · Cited by 19 · cited 1×
“The study done in [40] shows how uncertainty impacts clinical decision-making processes by employing a bootstrapped counterfactual inference framework.”
Mitigating Transformer Overconfidence via Lipschitz RegularizationPreprint
Ye, W., Ma, Y., Cao, X., & Kun, T. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.06849 · Cited by 12
Measures of Information and Uncertainty
Sullivan, T. J. (2015) · Texts in applied mathematics · doi:10.1007/978-3-319-23395-6_5 · Cited by 1
Research works printed but not linked: 6
- Gal, Y. (2016). Uncertainty in Deep Learning. [Ph.D. Thesis, University of Cambridge].
- Henry, 2016, Adoption of Electronic Health Record Systems among US Non-Federal Acute Care Hospitals: 2008–2015, ONC Data Brief, 35, 2008
- Qiu, R. (2020). Modeling Uncertainty in Deep Learning Models of Electronic Health Records. [Ph.D. Thesis, University of North Carolina at Charlotte].
- Qiu, R., Jia, Y., Hadzikadic, M., Dulin, M., Niu, X., and Wang, X. (2019). Modeling the Uncertainty in Electronic Health Records: A Bayesian Deep Learning Approach. arXiv.
- Huang, Y., Zhang, H., Shi, Y., Kolter, J.Z., and Anandkumar, A. (2021). Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Bounds. arXiv.
- Qi, X., Wang, J., Chen, Y., Shi, Y., and Zhang, L. (2023). LipsFormer: Introducing Lipschitz Continuity to Vision Transformers. arXiv.
Cited by
Yu, Y., Gomez-Cabello, C. A., Haider, S. A., Genovese, A., Prabha, S., Trabilsy, M., Collaco, B. G., Wood, N. G., Bagaria, S., Tao, C., et al. (2025) · Diagnostics · doi:10.3390/diagnostics15172204 · Cited by 32
“Additionally, stochastic neural networks have been found to inadequately represent epistemic uncertainty, creating a false impression of confidence and limiting their reliability for clinical decision support [6,7].”
Blattmann, M., Lindenmeyer, A., Franke, S., Neumuth, T., & Schneider, D. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000801 · Cited by 16
“…compare neural network ensembles and spectral-normalized neural Gaussian processes (SNGP) for epistemic uncertainty estimation in delirium risk prediction and in-hospital mortality prediction, showing that SNGP offers improved OOD detection while preserving comparable predictive accuracy [54,55].”
Self-citationJena, T., Suryodai, R., Reddy, D. N., Kumar, K. V., Muniyandy, E., & Kumar, N. V. P. S. (2025) · Intelligence-Based Medicine · doi:10.1016/j.ibmed.2025.100302 · Cited by 22
Eşki, S., & Taşkan, H. (2026) · Health Policy and Technology · doi:10.1016/j.hlpt.2026.101183 · Cited by 3
Engineering framework for curiosity-driven and humble AI in clinical decision support
Arslan, J., Benke, K. K., Cajas, S. A., Castro, R., Celi, L. A., Cruz-Suarez, G. A., Reyes, R. D., Engelmann, J., Ercole, A., Hilel, A., et al. (2026) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2025-101877 · Cited by 2
Metadata
- Authors
- Lindenmeyer, A., Blattmann, M., Franke, S., Neumuth, T., & Schneider, D.
- Affiliations
- Center for Scalable Data Analytics and Artificial Intelligence (DE)
- Leipzig University (DE)
- Corresponding author
- Lindenmeyer, Adrian
- Venue
- Journal of Personalized Medicine
- Publisher
- MDPI AG
- Published
- 2025-01-31
- Volume
- 15
- Issue
- 2
- Pages
- 58
- ISSN
- 2075-4426 (ISSN-L)
- PMC
- PMC11856777
- OpenAlex
- W4407039475
- Cited by
- 17
- References
- 43
- Citations by year
- 2026: 14 · 2025: 3
- FWCI
- 6.79
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Journal of Personalized Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- Federal Ministry of Education and Research of Germany (ScaDS.AI)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- trustworthy AI, clinical decision support, epistemic uncertainty, neural network ensembles, encoder-only transformer, in-hospital mortality prediction, intensive care unit, electronic health records, MIMIC-III, overconfidence, uncertainty quantification
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.