The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index; none of the 20 sources its claims cite is checked for support yet.
The need for uncertainty quantification in machine-assisted medical decision making
Open accessOverview
AI snapshot
Written from the paper’s full text.
TL;DR
This perspective argues that machine-assisted medical decision making, especially deep learning, requires a principled and formal discipline of uncertainty quantification (UQ) comparable to that used in nuclear stockpile stewardship and risk management. It maps where uncertainty enters the DL pipeline, identifies four overlapping challenge areas, and sketches research directions for building UQ for DL.
Key findings
- The paper contends that predictions without UQ "are neither predictions nor actionable," and that automated clinical systems not analyzable via known causal connections will require formal UQ to be adopted, as happened in nuclear security.
- It identifies four overlapping challenge groups for UQ of data-driven approaches: (1) absence of theory (no "hard laws" in medicine), (2) absence of causal models (DL exploits correlations without causal links), (3) sensitivity to imperfect data (missing elements and errors with patterned confounds), and (4) computational expense of recomputation for uncertainty bounds.
- It proposes that uncertainty must be quantified across at least four pipeline steps: collection/selection of training data and its coverage of the medical phenomenon, accuracy and completeness of labelling, selection and understanding of the DL model and its performance bounds, and model performance against operational (clinical inference) data.
- It highlights three research directions: quantifying and limiting overfitting (noting classical information criteria fail given DL's parameter counts and capacity to memorize random noise), understanding DL internals, and training DL to provide its own uncertainty estimates—suggesting GANs for detecting out-of-sample cases as "particularly fruitful."
- Cited (not original) performance claims: DL transfer-learning applications reported performance comparable to human experts or better, and DL function estimators reported performance better than state-of-the-art classical predictive models.
Methods
- Design: a conceptual/perspective article (no empirical study, dataset, or experiment conducted by the authors).
- Content: narrative discussion of the role of DL in clinical decision making, the proposed place of UQ, anticipated challenges, and foreseen research directions, illustrated by a figure of uncertainty sources in the DL pipeline.
- Analysis: argumentative synthesis drawing analogies to UQ practice in nuclear security and risk management; no statistical analysis is performed.
Limitations
- Stated by the paper: the domains where DL is applied lack underlying mechanistic theory to ground uncertainties; DL lacks causal models, limiting conclusions; real-world data is imperfect with patterned errors that can confound prediction; and recomputation for uncertainty bounds may currently be prohibitively expensive.
- Evident in the text: the article is a position/perspective piece with no empirical validation, no concrete UQ framework or method proposed, and no quantitative results of its own; the authors explicitly note that "it is possible that the entire new field of UQ for DL might need to be developed" and that a review of ongoing research is beyond its scope.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
29 Own interpretation · 17 No basis given · 10 Not checked yet
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we are ultimately operating without the fundamental, underlying mathematical model which we could otherwise use to ground our uncertainties and to bound any assumptions we make.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 4 Anticipated Challenges
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Advancements in UQ now provide measures of confidence necessary to inform national or international security decisions.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > Abstract
What the sources say
- Not checked yetcites [1] Oberkampf, W. L. & Roy, C. J. Verfication and Validation in Scientific Computing (Cambridge Univ. Press, Cambridge, 2010). source
- Not checked yetcites [2] National Research Council Evaluation of Quantification of Margins and Uncertainties: Methodology for Assessing and Certifying the Reliability of the Nuclear Stockpile (National Academies Press, Washington DC, 2009).
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The absence of a causal connection, however, means garnering limited conclusions from DL models”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 2. Absence of causal models:
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“most partial theories in medicine still explain an extremely small fraction of the observed phenomena and variations 3,4 .”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 2 Machine-assisted Clinical Decision Making and Research
What the sources say
- Not checked yetcites [3] Zuk, O., Hechter, E., Sunyaev, S. R. & Lander, E. S. The mystery of missing heritability: Genetic interactions create phantom heritability. Proc. Natl Acad. Sci. USA 109, 1193–1198 (2012). source
- Not checked yetcites [4] Choi, J. D. & Lee, J.-S. Interplay between epigenetics and genetics in cancer. Genomics Inform. 11, 164–173 (2013). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“The applications of DL techniques based on transfer learning have reported performance comparable to that of the human experts 5 , , 6 or better 7 ., 8”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 2.1 The Current Roles and Applications of AI
What the sources say
- Not checked yetcites [5] Kermany, D. S. et al. Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172, 1122–1131 (2018). source
- Not checked yetcites [6] Esteva, A. et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature 542, 115–118 (2017). source
- Not checked yetcites [7] Mar, V. & Soyer, H. Artificial intelligence for melanoma diagnosis: How can we deliver on the promise? Ann. Oncol. 29, 1625–1628 (2018). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“uncertainty in generalization depends on the density of training points in an appropriately defined neighborhood of the prediction target.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 5.3 Training DL to provide its own uncertainty estimates
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“A particularly fruitful approach seems to be the use of Generative Adversarial Networks (GANs) for detecting out-of-sample cases.[24][25][26]”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 5.3 Training DL to provide its own uncertainty estimates
What the sources say
- Not checked yetcites [24] Brahma, P. P., Huang, Q. & Wu, D. O. Structured memory based deep model to detect as well as characterize novel inputs. Preprint at http://arxiv.org/abs/1801.09859 (2018).
- Not checked yetcites [25] Yu, Y., Qu, W., Li, N. & Guo, Z. Open-category classification by adversarial sample generation. Preprint at http://arxiv.org/abs/1705.08722 (2017). source
- Not checked yetcites [26] Ge, Z., Demyanov, S., Chen, Z. & Garnavi, R. Generative openmax for multi-class open set classification. Preprint at http://arxiv.org/abs/1707.07418 (2017). source
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“the capacity of DL networks to memorize random noise 18”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 5.1 Quantifying and limiting overfitting
What the sources say
- Not checked yetcites [18] Zhang, C., Bengio, S., Hardt, M., Recht, B. & Vinyals, O. Understanding deep learning requires rethinking generalization. Preprint at http://arxiv.org/abs/1611.03530 (2016).
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 44
- It may state two separate claims as one. 16
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“computing capacity in support of DL is growing exponentially”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 4. Computational Expense:
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“UQ today underpins many decision processes in nuclear security, our risk management and associated investments, which can be at the scale of billions of dollars.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we propose that all of these steps (illustrated in figure 1) would need to be quantified in order to arrive at even crude, overall measures of the uncertainty of the DL-based decision model.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 3 The Role for UQ in DL
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“as DL-structured neural networks are particularly suitable for recognition of visually manifested conditions such as changes in tissue, lesions and growth, etc.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 2.1 The Current Roles and Applications of AI
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Real world data is usually imperfect-typically containing missing elements and errors-and these imperfections have patterns that can confound prediction.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 2. Absence of causal models:
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“the domains where the DL is usually applied, such as medicine, do not have such”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 4 Anticipated Challenges
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We see at least four overlapping groups of challenges associated with the uncertainty quantification of the data-driven approaches such as DL:”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 4 Anticipated Challenges
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“To achieve this confidence, UQ extends the traditional discipline of statistical error analysis to also capture uncertainties due to possibly incomplete, inaccurate, and contradictory input data, missing and undetected mechanisms and dependencies, expert judgment, and variations between reasonable model forms and modeling strategies.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“use of scientific insights and attempts at a cohesive framework incorporating the major clinical predictors are likely to be increasingly useful as predictive models are able to efficiently summarize more complex correlations in the data.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 2 Machine-assisted Clinical Decision Making and Research
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Predictions without UQ are neither predictions nor actionable.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Although AI-based research has long played an important role in medicine, it has, nevertheless, been one of the more challenging areas for AI to see an effective adoption.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 2 Machine-assisted Clinical Decision Making and Research
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
- Rests on an assumption: The reluctance to delegate to machine intelligence a function of life-critical and patient-safety-related decision making is a primary reason for the slow adoption of AI in medicine.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In most of the presented cases, the applications of AI are based on the DL neural networks, trained on a very large number of labeled data sets, and their learning tuned with the large number of hyperparameters.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 2.1 The Current Roles and Applications of AI
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“A need to systematize a similar situation is what actually led us to develop the formal approach to UQ in US national security sciences.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 4. Computational Expense:
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“medical AI, especially in its modern data-rich Deep Learning guise, needs to develop a principled and formal Uncertainty Quantification (UQ) discipline, just as we have seen in fields such as nuclear stockpile stewardship or risk management.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
- Rests on an assumption: The reluctance to delegate to machine intelligence a function of life-critical and patient-safety-related decision making is a primary reason for the slow adoption of AI in medicine.
None of the sources cited for this claim have been checked yet.
In the paper
“A notable example is the US support of a nuclear test moratorium since 1992, whereby we annually provide detailed measures of confidence in the safety, security, and performance of the nuclear stockpile--guaranteed through virtual testing 1,2 .”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > Abstract
What the sources say
- Not checked yetcites [1] Oberkampf, W. L. & Roy, C. J. Verfication and Validation in Scientific Computing (Cambridge Univ. Press, Cambridge, 2010). source
- Not checked yetcites [2] National Research Council Evaluation of Quantification of Margins and Uncertainties: Methodology for Assessing and Certifying the Reliability of the Nuclear Stockpile (National Academies Press, Washington DC, 2009).
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The data-rich world of AI-based learning, and the frequent absence of a well-understood underlying theory poses its own unique challenges to straightforward adoption of UQ.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Obviously, there are many steps in such scenarios where there are uncertainties that would need to be quantified.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 3 The Role for UQ in DL
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ad hoc solutions, such as sensitivity analysis and study of model variability, have sometimes been employed to mitigate some the problems we outline.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 4. Computational Expense:
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“once a low-dimensional representation is arrived at describing certain correlations (e.g., the difference between cancerous and matched normal cells), it can raise hypotheses that can be tested.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 2. Absence of causal models:
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These are then”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 2.1 The Current Roles and Applications of AI
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This may not seem like a limitation since prediction does not need causal relation.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 2. Absence of causal models:
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Understanding prediction system structure and defensibly quantifying uncertainty is possible, and, if done, can significantly benefit both research and practical applications of AI in this critical domain.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > Abstract
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Automated systems adoption, especially systems not analyzable in terms of known causal connections, will require principled and formal UQ to play a transformative role, just as we have seen in the nuclear security domain.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 2 Machine-assisted Clinical Decision Making and Research
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
In the paper
“techniques are being developed 17 to approximate some of the UQ-relevant calculations.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 4. Computational Expense:
What the sources say
- Not checked yetcites [17] Gal, Y. & Ghahramani, Z. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. Preprint at https://arxiv.org/abs/1506.02142 (2015).
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“But numerical results, without a measure of their veracity do not provide the trust needed to inform decisions.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The training of the DL models is computationally expensive, and any further recomputation and re-evaluation of the models, aimed, for example, at the calculation of uncertainty bounds might currently be prohibitively expensive.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 4. Computational Expense:
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These challenges, while not trivial, also present significant new research opportunities for the development of new theoretical approaches, and for the practical applications of UQ in the area of machine-assisted medical decision making.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > Abstract
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“furthermore, it is imperative to understand how the training data must be similar to prediction data.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 2. Absence of causal models:
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Specific UQ methods, therefore, need to be developed to quantify the sensitivity of models to imperfect data.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 2. Absence of causal models:
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“DL is essentially exploiting correlations in the data, without paying attention to any causal link.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 2. Absence of causal models:
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“To develop consequential decision-support from”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Unlike statistical approaches where mathematical models are used to explain variations observed in data, and to propose the margin of errors on inferences, with these recent applications, different learning architectures are combined with a large number of DL network parameters to form universal approximators.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 2.1 The Current Roles and Applications of AI
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Data-driven methods are emerging as the foundations of evidence-based decision making, and the future of data-driven scientific discovery.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 6 Summary
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Some approaches, such as attempts to empirically learn generalizable patterns with insertion of random noise 19,20 , or the use of cross-validation to determine the progression of generalizable learning, move us forward in this problem space, while still carrying the problem of overfitting 21 .”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 5.1 Quantifying and limiting overfitting
What the sources say
- Not checked yetcites [19] Arpit, D. et al. A closer look at memorization in deep networks. Preprint at https://arxiv.org/abs/1706.05394 (2017).
- Not checked yetcites [20] Zhang, C., Vinyals, O., Munos, R. & Bengio, S. A study on overfitting in deep reinforcement learning. Preprint at http://arxiv.org/abs/1804.06893 (2018).
- Not checked yetcites [21] Cawley, G. C. & Talbot, N. L. C. On over-fitting in model selection and subsequent selection bias in performance evaluation. J. Mach. Learn. Res. 11, 2079–2107 (2010).
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The wider application of DL in the biomedical field now requires an extension of these methods to this emerging field.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 4. Computational Expense:
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“an effective way of addressing some of the mentioned UQ problems might be to re-shape the DL engine itself to provide an uncertainty estimate on its predictions.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 5.3 Training DL to provide its own uncertainty estimates
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Overfitting, or the problem of a model performing well on the training set, but generalizing poorly for unseen datasets, is one of the fundamental problems of all data-centric methods, and therefore DL.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 5.1 Quantifying and limiting overfitting
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The details of the medical applications and DL networks are, however, significantly different since the theoretical foundations are far less developed and there are deep psychological and sociological implications in delegating to machines decisions that affect the life or health of human beings.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 6 Summary
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“progress in understanding the structure of these predictive systems, merging model and data driven approaches with strongly defensible UQ, and the formalization of the UQ for DL discipline will be needed to make DL and other data-centric tools and methods practically useful.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 6 Summary
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These have been studied in other scientific disciplines and decision-making arena, and we can learn from those.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 6 Summary
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“UQ for DL will likely not be simply a set of tools or procedures to apply, but a more complex wrap-up of disparate methods that in total help bound the overall confidence in predictions.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 6 Summary
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In high-dimensional problems, typical to the medical setting (images, large numbers of phenotypes, etc.), every point, however, can be isolated in some other dimension, and a density of points makes sense only after irrelevant dimensions, are projected out”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 5.3 Training DL to provide its own uncertainty estimates
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“With DL, and the large number of model parameters involved as well as the capacity of DL networks to memorize random noise 18 , classical approaches do not work.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 5.1 Quantifying and limiting overfitting
What the sources say
- Not checked yetcites [18] Zhang, C., Bengio, S., Hardt, M., Recht, B. & Vinyals, O. Understanding deep learning requires rethinking generalization. Preprint at http://arxiv.org/abs/1611.03530 (2016).
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Advances in understanding of how DL works internally will allow for a more effective UQ of interpreting DL.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 5.2 Understanding DL
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“there is a unifying structure to this problem in its various guises of complex predictive correlations in large data sets and engineering black boxes.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 6 Summary
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“there are ongoing studies that attempt to understand what DL does, 22 and how it learns 23”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 5.2 Understanding DL
What the sources say
- Not checked yetcites [22] Brahma, P. P., Wu, D. & She, Y. Why deep learning works: A manifold disentanglement perspective. IEEE Trans. Neural Netw. Learn. Sys. 27, 1997–2008 (2016). source
- Not checked yetcites [23] Raghu, M., Gilmer, J., Yosinski, J. & Sohl-Dickstein, J. SVCCA: Singular vector canonical correlation analysis for deep learning dynamics and interpretability. Preprint at https://arxiv.org/abs/1706.05806 (2017).
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the network itself can be used to study this uncertainty empirically and provide the uncertainty bounds.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 5.3 Training DL to provide its own uncertainty estimates
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In the end, it is possible that the entire new field of UQ for DL might need to be developed.”
§ The Need for Uncertainty Quantification in Machine-assisted Medical Decision Making > 4. Computational Expense:
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper relies on this without support.
The reluctance to delegate to machine intelligence a function of life-critical and patient-safety-related decision making is a primary reason for the slow adoption of AI in medicine.
“There are many reasons for this, primarily the reluctance to delegate to machine intelligence a function of life-critical and patient-safety-related decision making.”
The authors state this as an assumption.
Fair sampling in the training data holds.
“assuming fair sampling in the training data”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Better decision-making: AI, ML, and DL can enable robots to make better decisions based on data analysis and pattern recognition, leading to improved performance and outcomes [52] .”
From Artificial intelligence, machine learning and deep learning in advanced robotics, a review (2023) · cites this paper as [52]
Experimental — accuracy not yet measuredIn the citing paper
“However, these models have black-box nature and lack underlying theoretical foundation behind their decisions [8]”
From Explainable, trustworthy, and ethical machine learning for healthcare: A survey (2022) · cites this paper as [8]
In the citing paper
“However, due to the unavailability of sound fundamental laws and models, the explanation of DL cannot be produced up-to the required scale.”
From Explainable, trustworthy, and ethical machine learning for healthcare: A survey (2022) · cites this paper as [8]
Citations
Cites
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Dropout as a bayesian approximation: Representing model uncertainty in deep learning
Gal, Y., & Ghahramani, Z. (2015) · arXiv · doi:10.48550/arxiv.1506.02142 · Cited by 6,471
Identifying Medical Diagnoses and Treatable Diseases by Image-Based Deep Learning
Kermany, D. S., Goldbaum, M., Cai, W., Valentim, C. C. S., Liang, H., Baxter, S. L., McKeown, A., Yang, G., Wu, X., Yan, F., et al. (2018) · Cell · doi:10.1016/j.cell.2018.02.010 · Cited by 4,842
Understanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838
Scalable and accurate deep learning with electronic health records
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., et al. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0029-1 · Cited by 2,648 · cited 1×
“…DL methods have been used in the predictive scenarios related to quality of care, and clinical outcomes where large neural networks were used as function estimators in place of classical predictive models, with reported performance better than the state-of-the-art, classical model approaches 9–11 .”
MethodScalable and accurate deep learning with electronic health records.
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., et al. (2018) · eScholarship (California Digital Library) · Cited by 2,242
(2010) · doi:10.5555/1756006.1859921 · Cited by 1,924
Verification and Validation in Scientific Computing
Oberkampf, W. L., & Roy, C. J. (2010) · doi:10.1017/cbo9780511760396 · Cited by 1,556
The mystery of missing heritability: Genetic interactions create phantom heritability
Zuk, O., Hechter, E., Sunyaev, S. R., & Lander, E. S. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1119675109 · Cited by 1,542
Can machine-learning improve cardiovascular risk prediction using routine clinical data?
Weng, S. F., Reps, J., Kai, J., Garibaldi, J. M., & Qureshi, N. (2017) · PLOS ONE · doi:10.1371/journal.pone.0174944 · Cited by 1,346
Cited by
Sorted by citations
Items 201–300 of 583
Pitfalls of Conformal Predictions for Medical Image Classification
Mehrtens, H. A., Bucher, T.-C., & Brinker, T. J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44336-7_20 · Cited by 13
“…adaptation into clinical practice remains low, as current machine learning approaches based on deep neural networks are not able to provide reliable uncertainty estimates [1, 2, 3], which however are a necessary condition for the safe and reliable operation in and admission to clinical practice.”
Explainable AI: Developing Interpretable Deep Learning Models for Medical Diagnosis
-, R. T. (2024) · International Journal For Multidisciplinary Research · doi:10.36948/ijfmr.2024.v06i04.25281 · Cited by 12
“Quantifying Uncertainty : Providing accurate measures of uncertainty alongside explanations is crucial in medical contexts but remains a significant challenge [22].”
BackgroundAccurate Hit Estimation for Iterative Screening Using Venn–ABERS Predictors
Buendía, R., Kogej, T., Engkvist, O., Carlsson, L., Linusson, H., Johansson, U., Toccaceli, P., & Ahlberg, E. (2019) · Journal of Chemical Information and Modeling · doi:10.1021/acs.jcim.8b00724 · Cited by 9
Flexible Modelling of Longitudinal Medical Data
Bellot, A., & van der Schaar, M. (2020) · ACM Transactions on Computing for Healthcare · doi:10.1145/3377164 · Cited by 9
“Moreover, the sparsity of recorded information will tend to lead to substantial uncertainty about model predictions which needs to be captured for reliable inference in practice [6].”
BackgroundTowards Knowledge Uncertainty Estimation for Open Set Recognition
Pires, C. T. A., Barandas, M., Fernandes, L., Folgado, D., & Gamboa, H. F. S. (2020) · Machine Learning and Knowledge Extraction · doi:10.3390/make2040028 · Cited by 9
“Along with the interest arises some concerns related to the trustworthiness and robustness of the models [1].”
BackgroundA comparative study of kernel-based vector machines with probabilistic outputs for medical diagnosis
Qian, X., Zhou, Z., Hu, J., Zhu, J., Huang, H., & Dai, Y. (2021) · Journal of Applied Biomedicine · doi:10.1016/j.bbe.2021.09.003 · Cited by 9
Safety Issues in Human-Machine Collaboration and Possible Countermeasures
Liang Ma, A., & Wang, C. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-05890-5_21 · Cited by 10
Using Artificial Intelligence for COVID-19 Detection in Blood Exams: A Comparative Analysis
Loddo, A., Meloni, G., & Pes, B. (2022) · IEEE Access · doi:10.1109/access.2022.3221750 · Cited by 9
“making [4], [5], [6], being able to help experienced doctors, speed up the analysis process, and improve the reliability of results [7], [8], [9], [10].”
BackgroundBehavioral Priors and Dynamics Models: Improving Performance and Domain Transfer in Offline RLPreprint
Cang, C., Rajeswaran, A., Abbeel, P., & Laskin, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09119 · Cited by 9
“However, uncertainty quantification with deep neural networks can pose challenges in many domains, such as those with high dimensional input-output spaces or multiple confounding factors [24, 25, 26, 27, 28].”
BackgroundRobustness via Uncertainty-aware Cycle ConsistencyPreprint
Upadhyay, U., Chen, Y., & Akata, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.12467 · Cited by 8
“Uncertainty estimates for the predictions would allow subsequent revision by clinicians [41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51].”
BackgroundInsight into deep learning for glioma IDH medical image analysis: A systematic review
Lv, Q., Liu, Y., Sun, Y., & Wu, M. (2024) · Medicine · doi:10.1097/md.0000000000037150 · Cited by 8
“[28] Artifacts, model misspecifications, intrinsic biases in the data, the accuracy and completeness of the data, and the data selection process can all contribute to a model’s level of uncertainty.”
BackgroundWang, X., Li, Y., & Xue, C. (2024) · Electronics · doi:10.3390/electronics13153004 · Cited by 8
RelAI: an automated approach to judge pointwise ML prediction reliability
Peracchio, L., Nicora, G., Parimbelli, E., Buonocore, T. M., Tavazzi, E., Bergamaschi, R., Dagliati, A., & Bellazzi, R. (2025) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2025.105857 · Cited by 8
UTrack: Multi-object Tracking with Uncertain Detections
Solano‐Carrillo, E., Sattler, F., Alex, A., Klein, A., Costa, B. P., Rodríguez, Á. B., & Stoppe, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-91585-7_14 · Cited by 11
MU-Diff: a mutual learning diffusion model for synthetic MRI with Application for brain lesions
Dayarathna, S., Wu, Y., Cai, J., Wong, T.-T., Law, M., Islam, K. T., Peiris, H., & Chen, Z. (2025) · npj Artificial Intelligence · doi:10.1038/s44387-025-00016-8 · Cited by 11
Artificial Intelligence Segmentation Errors in Implant Planning Software Programs: An Overview
Lawand, G., Gonzaga, L., Issa, J., Revilla‐León, M., Tohmé, H., Saleh, A., & Martin, W. (2025) · Clinical Implant Dentistry and Related Research · doi:10.1111/cid.70095 · Cited by 8
van Aalst, J. E., Maruccio, F. C., Simões, R., Janssen, T. M., Wolterink, J. M., van Ooijen, P. M. A., & Brouwer, C. L. (2025) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ae110c · Cited by 9
Ye, J. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-025-11487-1 · Cited by 9
“Current work on uncertainty quantification emphasizes technical metrics—such as ensemble confidence intervals, entropy, and calibration—while rarely addressing how these metrics function in real clinical practice [ 105 ].”
Jackson, N. J., Brown, K. E., Rn, M., Murrow, M., Cauley, M. R., Collins, B. X., Novak, L. L., Benda, N. C., & Ancker, J. S. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocag002 · Cited by 13
“14,18 19–21 Promising approaches include providing explanations for AI decisions 22 (explainable AI) or disclosing AI uncertainty to the decision-maker 23,24 (uncertainty quantification).”
Malarvizhi, A. S., Smith, K., & Yang, C. P. (2026) · Big Earth Data · doi:10.1080/20964471.2026.2629680 · Cited by 8
“…in AI/ML tools through UQ, thereby fostering trust in their application, particularly in domains with significant decision-making implications (Begoli et al., 2019); and (iii) reducing the risk of adverse outcomes in high-stake domains, such as disaster management and climate change…”
Investigating Unfavorable Factors That Impede MALDI-TOF-Based AI in Predicting Antibiotic Resistance
Wang, H., Liu, Y.-H., Tseng, Y., Chung, C., Lin, T., Yu, J.-R., Huang, Y., & Lu, J. (2022) · Diagnostics · doi:10.3390/diagnostics12020413 · Cited by 8
Integer Programming Games: A Gentle Computational Overview
Carvalho, M., Dragotto, G., Lodi, A., & Sankaranarayanan, S. (2023) · doi:10.1287/educ.2023.0260 · Cited by 9
Explainable Semantic Federated Learning Enabled Industrial Edge Network for Fire Surveillance
Dong, L., Peng, Y., Jiang, F., Wang, K., & Yang, K. (2024) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2024.3441626 · Cited by 9
“Finally, simulation results demonstrate the effectiveness of XSFL. Index Terms —Explainable AI, federated learning, fire surveillance, semantic communication (SC).”
Maximizing Uncertainty for Federated Learning via Bayesian Optimization-Based Model Poisoning
Aristodemou, M., Liu, X., Wang, Y., Kyriakopoulos, K. G., Lambotharan, S., & Wei, Q. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3531143 · Cited by 8
Risk of estimators for Sobol’ sensitivity indices based on metamodels
Panin, I. (2021) · Electronic Journal of Statistics · doi:10.1214/20-ejs1793 · Cited by 8
“In the context of AI safety, we have faced the interpretability problem [56]: how can we explain the decision made by the neural network? For instance, such methods of analysis are especially important in medical applications, where almost every decision is accompanied by the most serious consequences [3].”
MethodEnvironmental Adaptation and Differential Replication in Machine Learning
Unceta, I., Nin, J., & Pujol, O. (2020) · Entropy · doi:10.3390/e22101122 · Cited by 9
“This is the case, for example, of medical diagnosis [50].”
BackgroundJavadi, G., Bayat, S., Esfeh, M. M. K., Samadi, S., Sedghi, A., Sojoudi, S., Hurtado, A., Chang, S. D., Black, P. C., Mousavi, P., et al. (2021) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-021-02485-z · Cited by 8
“We then evaluate aleatoric and model uncertainties to identify regions in ultrasound images for which more reliable model-basedpredictions areavailableintheform of PCadistribution colormaps.”
BackgroundInfluentialUsing Explainable AI to Measure Feature Contribution to Uncertainty
Brown, K. E., & Talbert, D. A. (2022) · Proceedings of the ... International Florida Artificial Intelligence Research Society Conference · doi:10.32473/flairs.v35i.130662 · Cited by 8
“…is a growing field of study that provides information regarding the reliability of neural networks, important information in safety-critical tasks, (Begoli, Bhattacharya, and Kusnezov 2019) and can provide insight into the robustness of the model given its training distribution or the data input…”
BackgroundA View on Model Misspecification in Uncertainty Quantification
Kato, Y., Tax, D. M. J., & Loog, M. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-39144-6_5 · Cited by 8
Towards causal algorithmic recourse
Karimi, A.-H., von Kügelgen, J., Schölkopf, B., & Valera, I. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-04083-2_8 · Cited by 11
Holstein, J., Böcking, L., Spitzer, P., Kühl, N., Vössing, M., & Satzger, G. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3762813 · Cited by 11
“For example, in medical diagnosis systems, presenting only epistemic uncertainty might be preferable when the AI encounters a rare condition in which it has only limited training data on [7, 19, 59].”
Milstein-driven neural stochastic differential equation model with uncertainty estimates
Zhang, X., Wei, W., Zhang, Z., Lei Zhang, A., & Li, W. (2023) · Pattern Recognition Letters · doi:10.1016/j.patrec.2023.08.018 · Cited by 7
Molchanova, N., Raina, V., Malinin, A., Rosa, F. L., Depeursinge, A., Gales, M., Granziera, C., Müller, H., Graziani, M., & Cuadra, M. B. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.109336 · Cited by 12
“…of uncertainty quantification (UQ) offers a possibility to tackle this issue by estimating the “degree of untrustworthiness” of model predictions (Begoli et al., 2019), focusing on two main uncertainty sources (Gawlikowski et al., 2023): i) data noise, captured by data uncertainty, and ii)…”
Artificial Intelligence in PET
Sitek, A., Ahn, S., Asma, E., Chandler, A. G., Ihsani, A., Prevrhal, S., Rahmim, A., Saboury, B., & Thielemans, K. (2021) · PET Clinics · doi:10.1016/j.cpet.2021.06.006 · Cited by 7
Artificial Intelligence in Anatomic Pathology
Levy, J., & Vaickus, L. J. (2021) · Advances in Molecular Pathology · doi:10.1016/j.yamp.2021.07.005 · Cited by 6
Rong, Y., Jiang, Z., Wu, W., Chen, Q., Wei, C., Fan, Z., & Chen, H. (2022) · Journal of Clinical Medicine · doi:10.3390/jcm11113203 · Cited by 7
“A promising way to address this shortage is uncertainty quantification [32], which can provide a measure of trust in CNNs’ decisions.”
BackgroundStochastic Synapses Made of Magnetic Domain Walls
Wang, C., Wang, K., Wen, X., Luo, W., Liang, S., Zhang, Y., & He, Y. (2022) · Physical Review Applied · doi:10.1103/physrevapplied.18.064014 · Cited by 7
Prabhudesai, S., Hauth, J. M., Guo, D., Rao, A. U. K., Banović, N., & Huan, X. (2023) · Frontiers in Computer Science · doi:10.3389/fcomp.2023.1071174 · Cited by 9
“, 2022), little attention has been dedicated to address uncertainty effects in AI models for healthcare (Begoli et al., 2019; Tonekaboni et al., 2019).”
BackgroundA Practitioner’s Guide to Digital Twin Development
Biller, B., Yi, J., & Biller, S. R. (2023) · doi:10.1287/educ.2023.0263 · Cited by 7
Jin, W., Fan, J., Gromala, D., Pasquier, P., & Hamarneh, G. (2021) · arXiv (Cornell University) · Cited by 8
Friesacher, H. R., Engkvist, O., Mervin, L., Moreau, Y., & Arany, Á. (2025) · Journal of Cheminformatics · doi:10.1186/s13321-025-00964-y · Cited by 7
“In drug discovery, accurate uncertainty estimates can be lever-aged to improve decisions about which candidates to pursue across a candidate portfolio.”
Detecting label noise in longitudinal Alzheimer’s data with explainable artificial intelligence
Sorino, P., Lombardi, A., Lofù, D., Colafiglio, T., Ferrara, A., Narducci, F., Sciascio, E. D., & Noia, T. D. (2025) · Brain Informatics · doi:10.1186/s40708-025-00261-2 · Cited by 6
“This strategy aligns with recent efforts in ML to integrate uncertainty quantification and human-in-the-loop verification for enhanced model robustness [41].”
Uncertainty quantification of surrogate models using conformal prediction
Gopakumar, V., Gray, A., Oskarsson, J., Zanisi, L., Giles, D., Kusner, M. J., Pamela, S. J. P., & Deisenroth, M. P. (2025) · Machine Learning Science and Technology · doi:10.1088/2632-2153/ae2e7b · Cited by 21
“In safety-critical applications such as fusion reactor design, climate modelling, and engineering optimisation, surrogate models must provide credible uncertainty estimates alongside their predictions (Begoli et al 2019).”
Gazi, A. H., Gao, D., Ghosh, S., Xu, Z., Trella, A. L., Klasnja, P., & Murphy, S. A. (2025) · Journal of Medical Internet Research · doi:10.2196/72830 · Cited by 8
“Uncertainty quantification ensures that decisions informed by a JITAI-Twin are made with an understanding of the uncertainty in JITAI-Twin modeling and simulation results [77].”
Data‐efficient Bayesian learning for radial dynamic MR reconstruction
Brahma, S., Kolbitsch, C., Martin, J., Schaeffter, T., & Kofler, A. (2023) · Medical Physics · doi:10.1002/mp.16543 · Cited by 6
Exploring and Exploiting Multi-Modality Uncertainty for Tumor Segmentation on PET/CT
Kang, S., Kang, Y., & Tan, S. (2024) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2024.3397332 · Cited by 6
BPEN: Brain Posterior Evidential Network for trustworthy brain imaging analysis
Ye, K., Tang, H., Dai, S., Fortel, I., Thompson, P. M., Mackin, R. S., Leow, A., Huang, H., & Zhan, L. (2024) · Neural Networks · doi:10.1016/j.neunet.2024.106943 · Cited by 6
“However, the black-box nature of deep learning models and the critical nature of medical decisions necessitate careful consideration of model uncertainty and interpretability (Begoli et al., 2019).”
Zhuang, J., Zhu, C. G., Han, R., Steuer, A., Kolb, J. F., & Shi, F. (2022) · Molecules · doi:10.3390/molecules27185861 · Cited by 6
“Examining the influence of model input uncertainties on model output can, on the one hand, improve the accuracy of model prediction and, on the other hand, determine the parameter sensitivity in order to optimize the fitting computation [24].”
BackgroundArtificial Intelligence, Bioinformatics, and Pathology
Levy, J., Lu, Y., Montivero, M. K. S., Ramwala, O. A., McFadden, J. R., Miles, C., Diamond, A. G., Reddy, R., Reddy, R., Hudson, T. R., et al. (2022) · Advances in Molecular Pathology · doi:10.1016/j.yamp.2023.01.001 · Cited by 7
Nashwan, A. J. (2023) · Endocrine Practice · doi:10.1016/j.eprac.2023.06.003 · Cited by 7
River Ice Regime Recognition Based on Deep Learning: Ice Concentration, Area, and Velocity
Yang, Z., Zong, J., Zhu, Y., Liu, X., Tao, R., & Yu, Y. (2023) · Water · doi:10.3390/w16010058 · Cited by 7
Uncertainty-Aware Time-to-Event Prediction using Deep Kernel Accelerated Failure Time ModelsPreprint
Wu, Z., Yang, Y., Fasching, P. A., & Volker Tresp, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.12250 · Cited by 6
“In healthcare applications, we would argue that uncertainty-awareness of the model is as important as point estimate performance, since the uncertainty estimates would assist the physicians in better interpreting the results from a black-box model (Begoli et al., 2019).”
Confidence-based prediction of antibiotic resistance at the patient level
Inda-Díaz, J. S., Johnning, A., Hessel, M., Sjöberg, A., Lokrantz, A., Helldal, L., Jirstrand, M., Svensson, L., & Kristiansson, E. (2026) · mBio · doi:10.1128/mbio.03431-25 · Cited by 7
Overconfident AI: how artificial intelligence navigates risk and uncertainty
Hong, S., & Lee, S. (2025) · Journal of Psychology and AI · doi:10.1080/29974100.2025.2486974 · Cited by 7
Multiscale Frequency-Guided Image Analyses for Mixed-Modality Medical Image Segmentation
Lin, Z., Huang, G., Chen, Z., Li, Y., Yuan, X., Pun, C., & Cheng, L. (2025) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2025.3562983 · Cited by 5
“Begoli et al. [44] discussed the necessity of acknowledging uncertainty in machine-assisted medical decision-making and the challenges we may face in this context.”
Toh, S. Y., Cai, C., Wang, L. R., Bai, X., Ngeow, J. Y. Y., & Fan, X. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719791 · Cited by 5
“UQ is a technique enhancing the use of AI in medicine [37].”
Abdulai, A. S. B., Storm, J., & Ehrlich, M. (2025) · Int. J. Medical Informatics · doi:10.1016/j.ijmedinf.2025.105957 · Cited by 7
Explainability in context: calibrating appropriate trust and reliance in artificial intelligence
Davis, S. E., & Salwei, M. E. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocag082 · Cited by 8
“18 Algorithms can learn biased patterns or perform differently across patient subpopulations.”
Digital Staining of White Blood Cells With Confidence Estimation
Tomczak, A., Ilić, S., Marquardt, G., Engel, T., Navab, N., & Albarqouni, S. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2023.3314695 · Cited by 5
“Indeed, such measures would be of particular interest in the medical applications as suggested by [2].”
BackgroundYapay Zekâ Risklerinin Etik Yönünden Değerlendirilmesi
Efe, A. (2021) · DergiPark (Istanbul University) · Cited by 5
Pomarico, E., Schmidt, C., Chays, F., Nguyen, D., Planchette, A., Tissot, A., Roux, A., Pagès, S., Batti, L., Clausen, C., et al. (2022) · Scientific Reports · doi:10.1038/s41598-022-07445-4 · Cited by 5
“According to Begoli et al 19 , the lifecycle of an AI process from the physical sample to the AI-assisted decisions is affected by multiple sources of uncertainty.”
BackgroundEnsemble Multi-Quantiles: Adaptively Flexible Distribution Prediction for Uncertainty Quantification
Xing, Y., Su, Y., & Ma, W. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2023.3288028 · Cited by 5
“For the network structure, at the step t = 0 for F0, we use 3 hidden layers with layer sizes [8, 16, 4] × d where d is the input dimension.”
BackgroundMethodInfluentialDensity regression and uncertainty quantification with Bayesian deep noise neural networks
Zhang, D., Liu, T., & Kang, J. (2023) · Stat · doi:10.1002/sta4.604 · Cited by 6
“Despite overwhelming cases of success, a major drawback of standard DNNs is the lack of reliable uncertainty quantification (UQ) (Begoli et al., 2019).”
A Bayesian Approach to Constructing Probabilistic Models from Knowledge Graphs
Freedman, H., Metzger, J., Abolhassani, N., Tudor, A., Tomlinson, B., & Paul, S. (2023) · International Journal of Semantic Computing · doi:10.1142/s1793351x24410022 · Cited by 5
Disease Spread Control in Cruise Ships: Monitoring, Simulation, and Decision Making
Triantafyllou, G., Kalozoumis, P. G., Cholopoulou, E., & Iakovidis, D. K. (2024) · Springer eBooks · doi:10.1007/978-3-031-48831-3_8 · Cited by 5
Guo, X., Xiang, Y., Yang, Y., Ye, C., Yu, Y., & Ma, T. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108933 · Cited by 5
SDE-HNN: Accurate and Well-Calibrated Forecasting Using Stochastic Differential Equations
Cui, P., Deng, Z., Hu, W., & Zhu, J. (2024) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/3691346 · Cited by 8
“Reliable uncertainty quantification is crucial for high-stake applications, e.g., autonomous driving [39], hydrological forecast [30], medical diagnosis [5], and index forecasting [17, 63].”
Hossain, T., Shen, W., Antar, A. D., Prabhudesai, S., Inoue, S., Huan, X., & Banović, N. (2022) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3551388 · Cited by 5
“…computation of uncertainty and confidence for models and data in a principled statistical manner, and is used across a wide range of scientific research domains (e.g., engineering physics [70], nuclear weapons stockpile management [74], astronautical engineering [24], medicine and healthcare [9]).”
BackgroundMeta-free few-shot learning via representation learning with weight averaging
Chen, K., & Lee, C.-G. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892722 · Cited by 5
“The risk of making wrong decisions is significant when using uncalibrated models [21].”
BackgroundTourassi, G. D. (2023) · Frontiers in Medical Engineering · doi:10.3389/fmede.2022.1112763 · Cited by 5
“, explainability) for its prediction (Begoli et al., 2019; Benrimoh et al., 2021).”
Deep Variational Inverse Scattering : (Invited Paper)
Khorashadizadeh, A., Aghababaei, A., Vlašić, T., Nguyen, H. P. T., & Dokmanić, I. (2023) · European Conference on Antennas and Propagation · doi:10.23919/eucap57121.2023.10133365 · Cited by 5
“Estimating uncertainty is paramount in safety-critical tasks such as medical imaging [9], nuclear stockpile [10], [11], and more recently self-driving vehicles [12], [13].”
BackgroundUncertainty bounds for multivariate machine learning predictions on high-strain brittle fracture
García–Cardona, C., Fernández-Godino, M. G., O’Malley, D., & Tanmoy Bhattacharya, A. (2021) · Computational Materials Science · doi:10.1016/j.commatsci.2021.110883 · Cited by 5
Evaluating and Boosting Uncertainty Quantification in ClassificationPreprint
Huang, X., Yang, J., Li, L., Deng, H., Ni, B., & Xu, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06030 · Cited by 5
“, medical decision making [1].”
ResultInfluentialLinli, Z., Liang, X., Zhang, Z., Hu, K., & Guo, S. (2025) · NeuroImage · doi:10.1016/j.neuroimage.2025.121184 · Cited by 4
The Role of Initial Acceptance Attitudes Toward AI Decisions in Algorithmic Recourse
Tominaga, T., Yamashita, N., & Kurashima, T. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713573 · Cited by 4
Topological Interpretability for Deep Learning
Spannaus, A., Hanson, H. A., Tourassi, G. D., & Penberthy, L. T. (2024) · Platform for Advanced Scientific Computing Conference (PASC) · doi:10.1145/3659914.3659935 · Cited by 4
“Due in part to the black-box nature of many AI-based algorithms [4] the field of machineassisted medical decision-making has been slow to adopt this paradigm.”
Data-Driven Innovation in Modern Laboratories: Harnessing Big Data for Better Outcomes
Almutairi, N. G., Almutairi, R. G., Arab, N. F., AL-Muhaize, R. H., Albasri, A. S., Alfifi, J. Y., Alanazi, R. N., Mubarak, A. S., Ahmed, A. N., & Alyahya, M. K. (2025) · Journal of Ecohumanism · doi:10.62754/joe.v4i2.5978 · Cited by 4
Remote health monitoring using artificial intelligence
Jain, P., & Palei, S. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-13816-4.00008-5 · Cited by 4
Yamin, M., Basahel, S. B., Bajaba, S. M., Abusurrah, M. M., & Lydia, E. L. (2023) · Computer Systems Science and Engineering · doi:10.32604/csse.2023.036455 · Cited by 4
Uncertainty quantification in game theory
de Cursi, E. S. (2020) · Chaos Solitons & Fractals · doi:10.1016/j.chaos.2020.110558 · Cited by 4
Dropout Strikes Back: Improved Uncertainty Estimation via Diversity Sampling
Fedyanin, K., Tsymbalov, E., & Panov, M. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-15168-2_11 · Cited by 6
“This can be especially important for error-critical applications like medical diagnostics [4] or autonomous car driving [8].”
BackgroundUnderstanding the Under-Coverage Bias in Uncertainty EstimationPreprint
Bai, Y., Mei, S., Wang, H., & Xiong, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05515 · Cited by 4
“This capability is crucial for deploying machine learning in the real world, in particular in risk-sensitive domains such as medical AI [15, 29], self-driving cars [47], and so on.”
BackgroundEvaluation of Various Open-Set Medical Imaging Tasks with Deep Neural NetworksPreprint
Ge, Z., & Wang, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.10888 · Cited by 7
“The epistemic uncertainty [44] is addressed by the learning model,”
BackgroundTowards Reliable Prognostics: RUL Uncertainty Estimation with Transformers and Monte Carlo Dropout
Prete, A. D., & Mashaallah, Z. (2025) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-032-10344-4_23 · Cited by 3
Weaknesses of the ANEES and New Calibration Measures for Multivariate Predictions
Walker, M., Reith‐Braun, M., & Hanebeck, U. D. (2025) · IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI) · doi:10.1109/mfi67357.2025.11259249 · Cited by 3
Youssef, Y. B., Amdouni, A., Manita, G., & Hassine, M. (2025) · Network Modeling Analysis in Health Informatics and Bioinformatics · doi:10.1007/s13721-025-00653-8 · Cited by 4
“Uncertainty quantification enables deferral or escalation pathways, targeted confirmatory testing, and conservative antimicrobial policies when confidence is low, mitigating false-positive harm (Begoli et al. 2019 ).”
Emerging anomaly detection techniques for electronic health records: A survey
Bhanja, S. N., Niu, H., Chen, Y., Omitaomu, O. A., Laurio, A., Trickey, A. W., Sampath, V., & Nebeker, J. R. (2026) · Intelligence-Based Medicine · doi:10.1016/j.ibmed.2026.100349 · Cited by 4
Jonkers, J., Coopman, F., Duchateau, L., Wallendael, G. V., & Hoecke, S. V. (2026) · Medical Image Analysis · doi:10.1016/j.media.2026.103953 · Cited by 3
Yuebo, M. E. N. G., Yin, G., Ye, S., Wang, Q., Liu, G., Li, X., & Geng, X. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.114129 · Cited by 3
Defining operational safety in clinical artificial intelligence systems
Kim, Y.-T., Kim, H., Bahl, M., Lev, M. H., González, R. G., Gee, M. S., & Do, S. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02450-7 · Cited by 5
Artificial Intelligence's Essential Role in the Process of Drug Discovery
Begoli, E., & Kusnezov, D. (2019) · Future Drug Discovery · doi:10.4155/fdd-2019-0026 · Cited by 3
Self-citationMining Nonlinear Dynamics in Operational Data for Process Improvement
Yang, H. Y. (2023) · doi:10.1287/educ.2023.0261 · Cited by 3
Iwata, T., & Kumagai, A. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127441 · Cited by 5
A penalized integrative deep neural network for variable selection among multiple omics datasets
Li, Y., Ren, X., Yu, H., Sun, T., & Ma, S. (2024) · Quantitative Biology · doi:10.1002/qub2.51 · Cited by 3
Bayesian Neural Network to Predict Antibiotic Resistance
Vouriot, L., Rebaudet, S., Gaudart, J., & Ureña, R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-66538-7_2 · Cited by 3
Lin, Y., Zhang, Q., Gao, B., Tang, J., Zhao, H., Qin, Q., Wang, Z., Qian, H., & Wu, H. (2024) · IEEE Transactions on Electron Devices · doi:10.1109/ted.2024.3497917 · Cited by 3
Sağlık Hizmetlerinde Hasta Güvenliği ve Yapay Zekâ: Bibliyometrik Bir Analiz
ŞAHİN, D. (2024) · Elektronik Sosyal Bilimler Dergisi · doi:10.17755/esosder.1563082 · Cited by 3
Xu, Z., Zhang, J., Haroutounian, S., Liu, H., Cao, Z., Messner, G. R., Alaverdyan, H., Ahuja, S., Koshy, R., Hanns, J., et al. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3729488 · Cited by 3
“Particularly in high-stakes scenarios, conveying uncertainty enhances trustworthiness by enabling clinicians to calibrate their confidence in model predictions, can mitigate risks associated with flawed outputs, and reduce over-reliance on "black box" systems [1, 9].”
Heinrich, K., Janiesch, C., Krancher, O., Stahmann, P., Wanner, J., & Zschech, P. (2025) · PLoS ONE · doi:10.1371/journal.pone.0328411 · Cited by 4
“Similarly, users may be uncertain whether an AI system that performs well on one dataset will generalize to new data [64].”
Tailored Bayes: a risk modeling framework under unequal misclassification costs
Karapanagiotis, S., Benedetto, U., Mukherjee, S., Kirk, P., & Newcombe, P. J. (2021) · Biostatistics · doi:10.1093/biostatistics/kxab023 · Cited by 3
“Quantification of uncertainty is critically important, especially in healthcare applications (Begoli et al., 2019; Kompa et al., 2021).”
Background
Metadata
- Authors
- Begoli, E., Bhattacharya, T., & Kusnezov, D.
- Affiliations
- Oak Ridge National Laboratory (US)
- United States Department of Energy (US)
- Corresponding author
- Begoli, Edmon
- ORCID
- Begoli, Edmon
- Venue
- Nature Machine Intelligence
- Published
- 2019-01-07
- Volume
- 1
- Issue
- 1
- Pages
- 20-23
- ISSN
- 2522-5839 (ISSN-L)
- OpenAlex
- W2904422658
- Semantic Scholar
- a73eef4e0ec8e49be1c0e6abb04cc1967a681c50
- Cited by
- 597
- References
- 22
- Citations by year
- 2026: 158 · 2025: 102 · 2024: 73 · 2023: 94 · 2022: 72 · 2021: 47 · 2020: 33 · 2019: 16
- FWCI
- 6.47
- Type
- journal article
- Language
- en
- Open access
- green
- www.osti.gov
- Licence
- Locations
- Nature Machine Intelligence (publishedVersion)
- OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) (submittedVersion)
- Funders
- Battelle (DE-AC05- 00OR22725, DE-AC05)
- UT-Battelle (DE-AC05, DEAC05-00OR22725, AC05-00OR22725)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- uncertainty quantification, artificial intelligence in healthcare, deep learning, patient safety, trust in AI, forecast uncertainty
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
