Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
Open accessThe DOI matches its Crossref record; 224 of the 230 references it cites were found in a registry or an index, and 6 are not checked yet; 41 of the 135 sources its claims cite were checked for support.
Abstract
Algorithmic transparency entails exposing system properties to various stakeholders for purposes that include understanding, improving, and contesting predictions. Until now, most research into algorithmic transparency has predominantly focused on explainability. Explainability attempts to provide reasons for a machine learning model's behavior to stakeholders. However, understanding a model's specific behavior alone might not be enough for stakeholders to gauge whether the model is wrong or lacks sufficient knowledge to solve the task at hand. In this paper, we argue for considering a complementary form of transparency by estimating and communicating the uncertainty associated with model predictions. First, we discuss methods for assessing uncertainty. Then, we characterize how uncertainty can be used to mitigate model unfairness, augment decision-making, and build trustworthy systems. Finally, we outline methods for displaying uncertainty to stakeholders and recommend how to collect information required for incorporating uncertainty into existing ML pipelines. This work constitutes an interdisciplinary review drawn from literature spanning machine learning, visualization/HCI, design, decision-making, and fairness. We aim to encourage researchers and practitioners to measure, communicate, and use uncertainty as a form of transparency.
Verified 2026-11-22 · 6 references not checked yet
Overview
CoChat Summary
TL;DR
This paper argues that estimating and communicating uncertainty should be treated as a complementary form of algorithmic transparency alongside explainability. It reviews methods for measuring uncertainty and discusses how uncertainty can support fairness, decision-making, trust, and stakeholder communication.
Key findings
- Uncertainty is split into aleatoric uncertainty (irreducible noise/class overlap) and epistemic uncertainty (reducible with more data); calibration is a precursor to using uncertainty as transparency, and deep ensembles are described as one of the best-performing uncertainty quantification approaches for neural networks, retaining calibration under dataset shift.
- Uncertainty can help address fairness issues: representation bias is epistemic and may be detected via large epistemic uncertainty on validation/deployment data, while feature and label noise are aleatoric and can distort fairness assessments; even a small amount of label noise can greatly impact fairness evaluation.
- In decision-making, uncertainty supports reject options and decision-theoretic action selection; JDM findings indicate risk is considered together with expected outcome, asymmetrically, and tolerance for uncertainty decreases superlinearly as stakes increase.
- Communication is difficult: in a 2010 US/Germany survey, 20% could not identify 10% as the biggest disease risk among 1%, 5%, and 10%, and almost 30% could not answer whether 1 in 10, 1 in 100, or 1 in 1000 represents the largest risk; people interpreted the IPCC’s “very likely” as around 60% probability, whereas the guideline specifies greater than 90%. Quantile dot plots and hypothetical outcome plots (HOPs) are reported to improve some probability/uncertainty judgments.
Methods
- Interdisciplinary literature review, not an empirical study, drawn from machine learning, visualization/HCI, design, decision-making, and fairness.
- Uses a recurring breast cancer diagnostics scenario to illustrate uncertainty in fairness, decision-making, trust formation, and communication.
- Synthesizes uncertainty quantification approaches (Bayesian methods, frequentist methods such as ensembles), evaluation metrics (calibration, NLL, Brier score, ECE), and communication methods (categories, numerical ranges, summary statistics, full distributions, visualizations).
Limitations
- The paper states that its findings should not be treated as conclusive for predicting real-world usability and recommends testing with real users in concrete tasks and settings.
- The paper states that empirical understanding of how decision-makers use aleatoric versus epistemic uncertainty is limited, that JDM literature has mostly focused on discrete outcomes, and that there is not a good understanding of how people perceive uncertainty over continuous outcomes such as error bars.
- Evident from the text: as a review, it provides no primary empirical validation or systematic review protocol; several claims are speculative or drawn from other fields and are framed as open questions for future research.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




![(c) Cone of uncertainty, showing the uncertainty around the predicted path of the center of a hurricane. Taken from [93].](https://figures.cochat.ai/paper-figures/42e1c0dd14b540788474c4bee4ed993b/uncertainty-as-a-form-of-transparency-measuring-communicating-and-using-figure-5.png)

Verification
Claims the paper makes
28 Valid · 10 Not valid · 60 Not checked · 104 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“When information on the sensitive attribute is unavailable (e.g., information on gender was not collected), but can be predicted from an auxiliary dataset, disparity measures are generally unidentifiable [20,63].”
§ Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty > 3.1 Uncertainty and Fairness
What the sources say
- Partly supportedcites [20] 20. Jiahao Chen, Nathan Kallus, Xiaojie Mao, Geoffry Svacha, and Madeleine Udell. 2019. Fairness under unawareness: Assessing disparity when protected class is unobserved. In Proceedings of the conference on fairness, accountability, and transparency. 339-348. sourceExperimental — accuracy not yet measured
Section 1.1 (p. 2) describes assessing disparity when protected-class labels are unavailable and must be imputed from proxy variables. The paper analyzes bias in the resulting disparity estimates, but the cited passages do not establish the broader claim that disparity measures are generally unidentifiable; the Conclusions (p. 9) instead note that the optimal imputation threshold cannot be determined without ground-truth labels. (LLM, conf=0.82)
- Supportedcites [63] 63. Nathan Kallus, Xiaojie Mao, and Angela Zhou. 2019. Assessing algorithmic fairness with unobserved protected class using data combination. arXiv preprint arXiv:1906.00285 (2019). source
The paper’s problem formulation describes a primary dataset missing protected-class labels and an auxiliary dataset containing those labels and proxy variables. Section 4 states that disparity measures are generally unidentifiable from the two datasets because their marginals do not uniquely determine the full joint distribution. The paper notes an exception for perfectly predictive proxies, consistent with the claim’s qualifier ‘generally.’ (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The experimental results of Gupta et al. [44] have shown that enforcing fairness constraints on a noisy sensitive attribute, without assumptions on the structure of the noise, is not guaranteed to lead to any improvement in the fairness properties of our model.”
§ Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty > 3.1 Uncertainty and Fairness
What the sources say
- Partly supportedcites [44] 44. Maya Gupta, Andrew Cotter, Mahdi Milani Fard, and Serena Wang. 2018. Proxy fairness. arXiv preprint arXiv:1806.11212 (2018). sourceExperimental — accuracy not yet measured
Section 4 (pp. 3–4) says poorly chosen proxy groups can worsen fairness for the true protected groups, supporting the claim that improvement is not guaranteed. But the paper presents this as a theoretical possibility, while its reported experiments generally found improvements (Conclusions, p. 8). It does not frame the result as an experiment on noisy sensitive attributes without assumptions about the noise structure. (LLM, conf=0.84)
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Indeed, Fogliato et al. [35] show that even a small amount of label noise can greatly impact the assessment of the fairness properties of the model.”
§ Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty > 3.1 Uncertainty and Fairness
What the sources say
- Supportedcites [35] 35. Riccardo Fogliato, Max G'Sell, and Alexandra Chouldechova. 2020. Fairness Evaluation in Presence of Biased Noisy Labels. arXiv preprint arXiv:2003.13808 (2020). source
The Abstract (p. 1) reports that even small biases in observed labels may call into question conclusions drawn from evaluating a model against those labels. The Conclusion (p. 8) gives an example in which a small gap in arrest likelihood could fully account for observed miscalibration. This supports the claim that a small amount of biased label noise can substantially affect fairness assessment. (LLM, conf=0.89)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Here, measures of demographic parity and equalized odds computed on the observed data are equal to the true metrics up to a scaling factor, which is proportional to the value of the noise [71]; if the noise rates are known, then the true metrics can be directly estimated.”
§ Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty > 3.1 Uncertainty and Fairness
What the sources say
- Supportedcites [71] 71. Alex Lamy, Ziyuan Zhong, Aditya K Menon, and Nakul Verma. 2019. Noise- tolerant fair classification. In Advances in Neural Information Processing Systems. 294-306. source
Section 4.2 (p. 6) states that MC noise scales the fairness measure, giving the clean measure as (1 − α − β) times the corrupted-data measure. Section 3.3 (p. 4) says equalized odds can be captured by adding a constraint for Y = 0, and §4.3 (p. 6) explains how known noise parameters supply the scaling factor. The factor depends on the noise rates; more precisely, it is 1 − α − β, not directly proportional to their value. (LLM, conf=0.88)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Similarly to noise in the sensitive attribute, Blum and Stangl [11], Jiang and Nachum [59] show that fairness constraints are guaranteed to improve the properties of a predictor only under appropriate assumptions on the label noise.”
§ Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty > 3.1 Uncertainty and Fairness
What the sources say
- Partly supportedcites [11] 11. Avrim Blum and Kevin Stangl. 2019. Recovering from biased data: Can fairness constraints improve accuracy? arXiv preprint arXiv:1912.01094 (2019). sourceExperimental — accuracy not yet measured
Section 3.3 (p. 11) says Equal Opportunity-constrained ERM recovers the Bayes-optimal classifier when Group A has sufficient mass and the signal is not too noisy. Section 3.1 (p. 9) also gives a recovery range bounded by the noise parameter η. The claim captures the dependence on noise assumptions, but overgeneralizes the guarantee to fairness constraints generally and omits the paper’s other conditions. (LLM, conf=0.86)
- Partly supportedcites [59] 59. Heinrich Jiang and Ofir Nachum. 2020. Identifying and correcting label bias in machine learning. In International Conference on Artificial Intelligence and Statistics. 702-712. sourceExperimental — accuracy not yet measured
Section 2.1 (p. 2) says that recovering unbiased labels requires an assumption about their relationship to observed biased labels. The Introduction (p. 1) reports a theoretical guarantee for the paper’s re-weighted training method under its model. It does not establish the broader claim that fairness constraints improve a predictor only under appropriate assumptions about label noise. (LLM, conf=0.83)
boostedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
74 more claims · 11 with their cited sources checked
Create a free account to see all 202 claims and to verify the 24 references still open.
- It may state two separate claims as one. 40
- Probably not a substantial claim. 105
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Uncertainty is considered a critical form of transparency that can alert people about the limitations of AI [10].”
From Investigating Explainability of Generative AI for Code through Scenario-based Design (2022) · cites this paper as [10]
The cited source says what the claim attributes to it.
In the citing paper
“Similar metrics have been used to evaluate the calibration of uncertainty quantification [7,9,64]-the estimated uncertainty should reflect the observed error rate.”
From Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations (2023) · cites this paper as [7]
The cited source says what the claim attributes to it.
In the citing paper
“Uncertainty expression can also increase the system's perceived transparency [13], which is generally shown to enhance trust, whether or not appropriate [56].”
From "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust (2024) · cites this paper as [13]
The cited source says what the claim attributes to it.
In the citing paper
“Finally, indicating model uncertainty [19,174,198], for example, by means of color-coding [167], and an explanation for that uncertainty [5,190], can also help users disentangle the role of their prompting and output evaluation ability from that of models' capabilities, further supporting confidence adjustment [197].”
From The Metacognitive Demands and Opportunities of Generative AI (2024) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty > References
“137. Yunfeng Zhang, Rachel KE Bellamy, and Wendy A. Kellogg. 2012.”
§ Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty > References
“In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (2015).”
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,370
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Knight, F. H. (1921) · Cited by 16,656
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325
Advances in prospect theory: Cumulative representation of uncertainty
Tversky, A., & Kahneman, D. (1992) · Journal of Risk and Uncertainty · doi:10.1007/bf00122574 · Cited by 15,467 · cited 2×
“Research also found that this bias occurs when people are asked to make decisions under risk and that their decisions imply such distortions [125, 137].”
Background(2011) · Cited by 14,317
Research works printed but not linked: 10
- 6. Emily Badger, Claire Cain Miller, Adam Pearce, and Kevin Quealy. 2018. Income Mobility Charts for Girls, Asian-Americans and Other Groups. Or Make Your Own. (2018).
- 19. Shelly Chaiken. 1999. The heuristic-systematic. Dual-process theories social psychology 73 (1999).
- 23. Marvin S Cohen, Raja Parasuraman, and Jared T Freeman. 1998. Trust in decision aids: A model and its training implications. In in Proc. Command and Control Research and Technology Symp. Citeseer.
- 36. Yarin Gal. 2016. Uncertainty in deep learning. University of Cambridge 1, 3 (2016).
- 54. Carl Iver Hovland, Irving Lester Janis, and Harold H Kelley. 1953. Communica- tion and persuasion. (1953).
- 87. Malik Sajjad Ahmed Nadeem, Jean-Daniel Zucker, and Blaise Hanczar. 2009. Accuracy-Rejection Curves (ARCs) for Comparing Classification Methods with a Reject Option. In Proceedings of the third International Workshop on Machine Learning in Systems Biology (Proceedings of Machine Learning Research), Sašo Džeroski, Pierre Guerts, and Juho Rousu (Eds.), Vol. 8. PMLR, Ljubljana, Slovenia, 65-81. http://proceedings.mlr.press/v8/nadeem10a.html
- 91. Radford M Neal. 1995. Bayesian Learning for Neural Networks. PhD thesis, University of Toronto (1995).
- 103. Jennifer Preece, Helen Sharp, and Yvonne Rogers. 2015. Interaction design: beyond human-computer interaction. John Wiley & Sons.
- 118. S Shyam Sundar. 2008. The MAIN model: A heuristic approach to understanding technology effects on credibility. MacArthur Foundation Digital Media and Learning Initiative.
- 130. Claus O. Wilke. 2019. Fundamentals of Data Visualization: A Primer on Making Informative and Compelling Figures (1st edition ed.). O'Reilly Media.
Cited by
Items 101–200 of 248
Explainable AI in Healthcare: Enhancing Transparency and Trust in Predictive Models
Yekollu, R. K., Ghuge, T. B., Biradar, S. S., Haldikar, S. V., & Kader, O. F. M. A. (2024) · International Conference on Electronics and Sustainable Communication Systems (ICESC) · doi:10.1109/icesc60852.2024.10690121 · Cited by 12
Generative artificial intelligence and social media: insights for tobacco control
Kong, G., Ouellette, R. R., & Murthy, D. (2024) · Tobacco Control · doi:10.1136/tc-2024-058813 · Cited by 11
“To address these limitations, it is essential to be transparent regarding each model’s level of uncertainty and risks, 37 with explanations about how the model reached its outputs.”
Uncertainty Analysis of Deep Kernel Learning Methods on Diabetic Retinopathy Grading
Siebert, M., Graßhoff, J., & Rostalski, P. (2023) · IEEE Access · doi:10.1109/access.2023.3343642 · Cited by 15
“Both are important to enable transparent usage within the high-stakes medical domain [9], [31].”
BackgroundMa, S., Sun, M., & Ma, X. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3571813 · Cited by 19
“In this article, we adopt entropy [145], which is widely used to measure a model’s uncertainty [10], as our learning status indicator because it can reflect the stability of Q-Learning representing whether the underlying policy is well-trained.”
BackgroundMethodUsing AI Uncertainty Quantification to Improve Human Decision-MakingPreprint
Marusich, L. R., Bakdash, J. Z., Zhou, Y., & Kantarcıoğlu, M. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/6syfv · Cited by 21
Kalogiannidis, S., Kontsas, S., Kalfas, D., & Chatzitheodoridis, F. (2024) · Operational Research · doi:10.1007/s12351-024-00854-5 · Cited by 9
“Risk is the outcome of uncertain occurrences that may be predicted using probability distributions (Clemen and Winkler 1999; Bhatt et al. 2021; Hashem and Hashem 2023).”
Papantonis, I., & Belle, V. (2025) · Frontiers in Computer Science · doi:10.3389/fcomp.2025.1560448 · Cited by 14
Uncertainty-aware probabilistic travel demand prediction for mobility-on-demand services
Peng, T., Gao, J., & Cats, O. (2025) · Transportation Research Part C Emerging Technologies · doi:10.1016/j.trc.2025.105383 · Cited by 13
Optimization’s Neglected Normative Commitments
Laufer, B., Gilbert, T. K., & Nissenbaum, H. F. (2023) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3593013.3593976 · Cited by 18
“There is significant work on the elusive nature of measurement— here we highlight theoretical groundwork [8, 38, 57, 61, 84] as well as technical work on bias in measurement [15, 51, 79, 91].”
BackgroundExplaining Model Confidence Using Counterfactuals
Le, T. N., Miller, T., Singh, R., & Sonenberg, L. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i10.26399 · Cited by 23
Chen, Q. Z., & Zhang, A. X. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610074 · Cited by 22
“Due to these observations, in many areas, approaches and processes have been developed to measure uncertainty in data in order to discard unreliable judgments [6, 34, 67] or to create systems that can make use of disaggregated data [27, 51, 87].”
BackgroundFuzzy Linkography: Automatic Graphical Summarization of Creative Activity Traces
Smith, A., Anderson, B. R., Otto, J., Karth, I., Sun, Y., Chung, J. J. Y., Roemmele, M., & Kreminski, M. (2025) · Creativity and Cognition · doi:10.1145/3698061.3726915 · Cited by 24
““Fuzzy” lighter-colored links thereby serve to communicate model uncertainty about move association; visual “fuzziness” has been found to be an intuitive way to convey uncertainty [61], which can help to improve the trans-parency of analyses that rely on machine learning models [5, 6].”
Dahlke, J., & Ebersberger, B. (2025) · Journal of Evolutionary Economics · doi:10.1007/s00191-025-00909-6 · Cited by 9
“One crucial characteristic of AI technology that distinguishes it from other digital technologies is the inherent opaqueness of connectionist AI models, potentially leading to inexplicable outputs and creating significant uncertainty for practitioners (Bhatt et al. 2020; Rai 2020).”
Continual Active Adaptation to Evolving Distributional Shifts
Machireddy, A., Krishnan, R., Ahuja, N., & Tickoo, O. (2022) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00388 · Cited by 12
“We use the predictive entropy of the model to quantify the uncertainty [30, 31] which is defined as”
MethodSelf-citationChen, S., Liu, Y., Lu, R., Zhou, Y., Lee, Y., & Huang, Y. (2022) · Designing Interactive Systems Conference · doi:10.1145/3532106.3533500 · Cited by 12
“Prior works on explainable AI found result granularity should be small enough to inform users of its uncertainty without overwhelming them with too many details [8], and if the AI behaved unexpectedly or erroneously, users should be”
BackgroundTowards Reliable Parameter Extraction in MEMS Final Module Testing Using Bayesian Inference
Heringhaus, M. E., Zhang, Y., Zimmermann, A., & Mikelsons, L. (2022) · Sensors · doi:10.3390/s22145408 · Cited by 9
“An example for VI is the ensembling of multiple trained models [29,33,36,37].”
Self-citationBenchmarking Uncertainty Quantification on Biosignal Classification Tasks Under Dataset Shift
Xia, T., Han, J., & Mascolo, C. (2022) · Studies in computational intelligence · doi:10.1007/978-3-031-14771-5_25 · Cited by 16
Teikari, P., & Pietrusz, A. (2021) · SportRxiv (OSF Preprints) · doi:10.31236/osf.io/w734a · Cited by 17
Schmidt, S., Rolff, T., Voigt, H., Offe, M., & Steinicke, F. (2024) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3654777.3676454 · Cited by 12
“In addition, a variety of cognitive biases afect the interpretation of uncertainty estimates, such as tendencies to underestimate high probabilities while overestimating low probabilities, as well as preferences towards positive framing [8].”
InfluentialKeshavamurthy, R., Pazdernik, K., Ham, C., et al. (2024) · JMIR Public Health and Surveillance · doi:10.2196/59971 · Cited by 7
“Hence, model uncertainties are often overlooked even though these estimates are crucial for systematic and transparent decision-making [15].”
Investigating Unfavorable Factors That Impede MALDI-TOF-Based AI in Predicting Antibiotic Resistance
Wang, H., Liu, Y.-H., Tseng, Y., Chung, C., Lin, T., Yu, J.-R., Huang, Y., & Lu, J. (2022) · Diagnostics · doi:10.3390/diagnostics12020413 · Cited by 8
“A key to medical AI success is to calibrate human trust by providing a confidence score in the model on a case-by-case basis [5,6].”
BackgroundPhysical Computing & Materials Acceleration Platforms
Peterson, E., & Lavin, A. (2022) · Matter · doi:10.1016/j.matt.2022.09.022 · Cited by 8
Prabhudesai, S., Hauth, J. M., Guo, D., Rao, A. U. K., Banović, N., & Huan, X. (2023) · Frontiers in Computer Science · doi:10.3389/fcomp.2023.1071174 · Cited by 15
“, 2021) to quantify (Bhatt et al., 2021; Ghosh et al., 2022), report (Arnold et al.”
BackgroundFeedbackLogs: Recording and Incorporating Stakeholder Feedback into Machine Learning Pipelines
Barker, M., Kallina, E., Ashok, D., Collins, K. M., Casovan, A., Weller, A., Talwalkar, A., Chen, V., & Bhatt, U. (2023) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3617694.3623239 · Cited by 15
“The information should help the stakeholder develop an appropriate understanding of the current pipeline Approaches to communicate such information include socio-technical details [28, 47], performance metrics [34, 54], model explanations [14, 24], and confidence estimates [7, 37].”
BackgroundSelf-citationPalmeiro-Silva, Y. K., Oppenheimer, C. W., Henríquez-Roldán, C. F., & Bangdiwala, S. I. (2021) · Revista Panamericana de Salud Pública · doi:10.26633/rpsp.2021.108 · Cited by 12
Levene, M., & Wooldridge, J. (2023) · doi:10.47120/npl.ms45 · Cited by 9
“…for measuring uncertainty, such as Bayesian neural networks (BNNs), are designed to produce uncertainty estimates along with their predictions, while in frequentist methods, such as ensemble methods, uncertainty is measured by retraining the network several times and averaging the results [13].”
BackgroundHolstein, J., Böcking, L., Spitzer, P., Kühl, N., Vössing, M., & Satzger, G. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3762813 · Cited by 17
Schäfer, U., Sipos, L., & Müller-Birn, C. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757451 · Cited by 13
Meng, Y., Cavalaro, S., Dizaye, F., et al. (2025) · Smart and Sustainable Built Environment · doi:10.1108/sasbe-04-2025-0185 · Cited by 6
Passi, S., Dhanorkar, S., & Vorvoreanu, M. (2025) · Springer eBooks · doi:10.1007/978-981-95-3658-0_98 · Cited by 15
Quantization for Bayesian Deep Learning: Low-Precision Characterization and Robustness
Lin, J.-L., Krishnan, R., Ranipa, K., Subedar, M. M., Sanghavi, V., Arunachalam, M., Tickoo, O., Iyer, R., & Kandemir, M. (2023) · IEEE International Symposium on Workload Characterization · doi:10.1109/iiswc59245.2023.00020 · Cited by 12
“This issue is particularly critical for real-world applications, especially in terms of explainability, safety, and reliability [5, 13].”
BackgroundSelf-citationBott, A., Liu, B., Puchta, A., & Fleischer, J. (2024) · Journal of Machine Engineering · doi:10.36897/jme/192681 · Cited by 8
“While Bayesian methods define a priori a hypothesis space of plausible models, frequentist approaches consider how well the distribution over the observations implied by each hypothesis is consistent with the data [32].”
BackgroundMethodDesign Thinking Framework for Integration of Transparency Measures in Time-Critical Decision Support
Stone, P. B., Jessup, S. A., Ganapathy, S., & Harel, A. (2022) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2022.2068745 · Cited by 9
“Bhatt et al. (2020) discuss the idea of introducing uncertainty into HMT as a form of transparency.”
BackgroundBayesian Neural Networks to Analyze Hyperspectral Datasets Using Uncertainty Metrics
Alcolea, A., & Resano, J. (2022) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2022.3205119 · Cited by 9
“The value of H is given by 2, where at = 266 p(y = ck |x, wt) [13] 267”
BackgroundMethodInfluentialUncertainty in Natural Language Generation: From Theory to Applications
Baan, J., Daheim, N., Ilia, E., Ulmer, D., Li, H.-S., Fernández, R., Plank, B., Sennrich, R., Zerva, C., & Aziz, W. (2023) · arXiv · doi:10.48550/arxiv.2307.15703 · Cited by 6
He, W., Li, L., & Gao, X. (2024) · Remote Sensing · doi:10.3390/rs16111987 · Cited by 5
Increasing system transparency through confidence information in cooperative, automated driving
Peintner, J. B., Manger, C., & Riener, A. (2024) · Behaviour and Information Technology · doi:10.1080/0144929x.2024.2407018 · Cited by 7
“In order to ensure successful human-machine cooperation, these missing cues must, therefore, be compensated by showing confidence estimates of the machine’s decision (Bhatt et al. 2021).”
Evaluating the Need for Explanations in Blockchain Smart Contracts to Reconcile Surprises
Ghanmi, H. A., Ahmadjee, S., & Rami Bahsoon, A. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3721283 · Cited by 9
“Aleatory uncertainty, or stochastic uncertainty, measures the intrinsic variability or randomness of a system or process.”
Leveraging subclass learning for improving uncertainty estimation in deep learning
Spanos, D., Passalis, N., & Tefas, A. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130954 · Cited by 7
Position Paper: Integrating Explainability and Uncertainty Estimation in Medical AI
Fan, X. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228229 · Cited by 11
Arous, I., Chehbouni, K., Cheng, Z., & Dossou, B. F. P. (2025) · Springer eBooks · doi:10.1007/978-981-97-8440-0_85-1 · Cited by 6
Chen, S., Cheng, H., & Huang, Y. (2024) · Postdigital science and education · doi:10.1007/978-3-031-64487-0_9 · Cited by 18
Uncertainty-Aware and Explainable Human Error Detection in the Operation of Nuclear Power Plants
Reddy, B., Gursel, E., Daniels, K., Khojandi, A., Coble, J. B., Agarwal, V., Boring, R. L., Yadav, V., & Madadi, M. (2024) · Nuclear Technology · doi:10.1080/00295450.2024.2372217 · Cited by 8
“[19] Aleatory uncertainty, also known as stochastic uncertainty or data uncertainty, arises from inherent variability and randomness in the data being modeled.”
Do I know this? segmentation uncertainty under domain shift
Hoebel, K., Bridge, C. P., Lemay, A., Chang, K., Patel, J., Rosen, B. R., & Kalpathy–Cramer, J. (2022) · SPIE Medical Imaging: Image Processing · doi:10.1117/12.2611867 · Cited by 7
“As proposed by Bhatt et al., 12 by monitoring the epistemic uncertainty during deployment of DL classification models, changes in the uncertainty distribution can be indicative of domain shifts in the input data over time.”
MethodCoKnowledge: Supporting Assimilation of Time-synced Collective Knowledge in Online Science Videos
Zhang, Y., Wang, Y., Wang, X., He, C., Huang, C., & Ma, X. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713682 · Cited by 10
Self-citationAudrin, B., & Audrin, C. (2026) · Technology in Society · doi:10.1016/j.techsoc.2026.103289 · Cited by 4
García-Poyo, M. C., Bérail, S., Ronzani, A., Rello, L., García-González, E., Nakadi, F. V., Aramendía, M., Resano, J., Resano, M., & Pécheyran, C. (2022) · Journal of Analytical Atomic Spectrometry · doi:10.1039/d2ja00267a · Cited by 4
Learning to Explain Selectively: A Case Study on Question Answering
Feng, S., & Boyd-Graber, J. L. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.573 · Cited by 8
Communication of Uncertainty in AI Regulations
Phutane, A. S. (2023) · Community Change · doi:10.21061/cc.v4i2.a.50 · Cited by 5
“Several uncertainties surround the application of AI due to the wide-ranging implications and lack of transparency into the inner workings of AI models (Bhatt et al. 2021).”
Analysing the Predictability of Language Model Performance
Schellaert, W., Martínez‐Plumed, F., & Hernández‐Orallo, J. (2025) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3706118 · Cited by 19
“For each particular case, we can assess whether the LM will be worth our time and trust [5, 54], but with modern general-purpose systems, the region of input space where we can expect useful performance has become unmanageably large.”
Evaluating subgroup disparity using epistemic uncertainty in mammographyPreprint
Lu, C., Lemay, A., Hoebel, K., & Kalpathy–Cramer, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.02716 · Cited by 10
“Inspired by works, such as Bhatt et al. (2020), which advocates for the use of uncertainty as one approach to achieve more transparency in ML, and Dusenberry et al. (2020), which shows ML model exhibiting higher uncertainty for different demographics in electronic health records, we investigate how…”
MethodEfficient Online Bayesian Inference for Neural BanditsPreprint
Duràn-Martín, G., Kara, A., & Murphy, K. D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.00195 · Cited by 7
Chen, Y., Verbert, K., Gerling, K., Abeele, M.-E. V., & Abeele, V. V. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712139 · Cited by 7
Xu, Z., Zhang, J., Haroutounian, S., Liu, H., Cao, Z., Messner, G. R., Alaverdyan, H., Ahuja, S., Koshy, R., Hanns, J., et al. (2025) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3729488 · Cited by 5
“This capability is a cornerstone of fostering trust and improving decision-making in healthcare settings [9, 10, 39, 67, 80].”
Nnadi, L. C., Ding, D., Muepu, D. M., Amin, M. F. I., & Kabir, R. (2025) · Lecture notes in computer science · doi:10.1007/978-981-96-8892-0_7 · Cited by 5
Pedersen, T. M., Øvlisen, K., Connelly, L. C., Assent, I., & Petersen, M. G. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791457 · Cited by 3
Investigating AI-induced Technostress and Coping Strategies of Professionals
Kwon, H., Oh, J., Lee, S., Lee, S., & Lee, S. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791671 · Cited by 8
“The opacity of the AI model makes it inherently difficult to comprehend, thereby frequently generating additional overload, as users must verify information or assume accountability themselves [9, 18, 37].”
Uncertainty in Predictive Process Monitoring
Portolani, P., Brusaferri, A., Ballarino, A., & Matteucci, M. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-08974-9_44 · Cited by 3
Explaining the Uncertainty in AI-Assisted Decision Making
Le, T. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i13.26920 · Cited by 6
CROSS-INDUSTRY INSIGHTS: A COMPREHENSIVE REVIEW OF EFFECTIVE STAKEHOLDER MANAGEMENT BENEFITS
Daraojimba, C., Okogwu, C., Egbokhaebho, B. A., Raji, A., Kolade, A. O., & Olalere, B. I. (2023) · Materials and Corrosion Engineering Management · doi:10.26480/macem.01.2023.12.19 · Cited by 3
Moral Perception and Uncertainty Expression in LLM-Augmented Judicial PracticePreprint
Delacroix, S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4787044 · Cited by 3
Incorporating Artificial Intelligence into Design Criteria Considerations
Drzyzga, G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60611-3_10 · Cited by 4
Trustable Network Intrusion Detection System through Wisdomnet and Uncertainty Measures
Vij, A., Tran, H. A., & Tran, T. X. (2024) · Conference on Local Computer Networks · doi:10.1109/lcn60385.2024.10639737 · Cited by 2
A Conceptual Framework of AI Transparency From Sociotechnical Perspective
Huang, C., Wang, Z., Wen, X., Wang, X., & Yao, X. (2025) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2025.3608735 · Cited by 3
“Bhatt et al. [50] proposed considering uncertainty as a form of transparency by estimating and communicating the uncertainty associated with model predictions.”
AI in Corporate Governance: Strengthening Oversight
Kayyali, M. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-3626-8.ch008 · Cited by 2
Embodied Natural Language Interaction (NLI): Speech Input Patterns in Immersive Analytics
Song, H., Johnson, M., Whitley, K. N., Krokos, E., & Varshney, A. (2025) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2025.3634798 · Cited by 3
“Uncertainty refers to the state of lacking definite knowledge or clarity , which has been studied across various domains [3,7].”
Societal Frameworks Can Improve LLM Alignment
Stańczak, K., Meade, N., Bhatia, M., Zhou, H., Böttinger, K., Barnes, J., Stanley, J., Papernot, N., Chapados, N., Thérien, D., et al. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3806486 · Cited by 10
Self-citationBishof, Z., Scheuerman, J., & Michael, C. J. (2023) · Entropy · doi:10.3390/e25101443 · Cited by 2
Molladavoudi, S., & Yung, W. (2023) · AStA Wirtschafts- und Sozialstatistisches Archiv · doi:10.1007/s11943-023-00331-z · Cited by 4
“The idea of a “reject option” in clas-sification, as discussed in Bhatt et al. (2021), has been considered in human-in-the-loop application settings Kaiser et al. (2022), and can be linked to the assessment of uncertainty and confidence in the model’s predictions, namely the capability of abstaining…”
Task Delegation from AI to Humans: The Impact of AI Transparency on Human Performance
Wang, Y., Jiang, Y., Tang, J., & Zhou, X. (2024) · Lecture notes in business information processing · doi:10.1007/978-3-031-60324-2_24 · Cited by 5
Evaluating Search System Explainability with Psychometrics and Crowdsourcing
Chen, C., & Eickhoff, C. (2024) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3626772.3657796 · Cited by 8
Accelerating Bayesian Neural Networks on Low-Power Edge RISC-V Processors
Pedrajas, S. P., Resano, J., & Gracia, D. S. (2024) · IEEE International Conference on Nanotechnology (IEEE-NANO) · doi:10.1109/nano61778.2024.10628877 · Cited by 2
“In a classification setting, we can measure the uncertainty of a prediction using the MC samples [9].”
MethodVisualizing Uncertainty in AI for Accident Severity Classification
Roussel, C., Böhm, K., Jakobi, B., Vlasov, A., Schmidt, D., Braun, S., & Rolwes, A. (2024) · International Conference on Information Visualisation (IV) · doi:10.1109/iv64223.2024.00030 · Cited by 2
“Especially communicating the uncertainties of the model is a challenge when dealing with machine learning [4].”
BackgroundDadjouy, S., & Sajedi, H. (2024) · IET Image Processing · doi:10.1049/ipr2.13292 · Cited by 4
Luo, Y., Yu, X., Tran, T. T. M., & Hoggenmüeller, M. (2025) · International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI) · doi:10.1145/3744333.3747826 · Cited by 2
“The decision-making processes of autonomous systems inherently involve uncertainty, which can arise from sources such as noise in data input and imperfections in machine learning models [4, 27].”
InfluentialMoral Perception and Uncertainty Expression in LLM-Augmented Judicial Practice
Delacroix, S. (2025) · Minds and Machines · doi:10.1007/s11023-025-09744-x · Cited by 9
“However, these approaches primarily address what (Bhatt et al., 2021) term ‘referential uncertainty’—uncertainty regarding factual claims or textual interpreta - tion—while providing limited support for engaging with what we might call ‘norma - tive uncertainty’—such as navigating any relevant…”
Speech-Driven Immersive Analytics: GenAI-grounded Sensemaking Perspective
Song, H. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742414.3789232 · Cited by 1
Sihotang, H. T., Marsoit, P. M. F., & Marsoit, P. T. (2023) · International Journal of Enterprise Modelling · doi:10.35335/emod.v17i1.70 · Cited by 1
“This research will increase stochastic variational inequalities' applicability and effectiveness in uncertainty, improving decision-making in complicated real-world systems(Bhatt et al., 2021).”
BackgroundKnow What Not To Know: Users’ Perception of Abstaining Classifiers
Papenmeier, A., Hienert, D., Kammerer, Y., Seifert, C., & Kern, D. (2023) · Designing Interactive Systems Conference · doi:10.1145/3563703.3596622 · Cited by 5
“, competence and trustworthiness [5, 21]), which is an important aspect of users’ perception [5], might be impaired [2].”
BackgroundLeveraging Machine Learning for Effective Data Management
Sellami, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-662-70140-9_1 · Cited by 1
Obesity prediction from structural MRI using conformal deep learning with uncertainty quantification
Farzana, W., Temtam, A., Humud-Arboleda, B., Ma, L., Bean, M. K., Moeller, G., & Iftekharuddin, K. M. (2025) · SPIE Medical Imaging: Computer-Aided Diagnosis · doi:10.1117/12.3048977 · Cited by 2
Bridging the gap between machine confidence and human perceptions
Yin, M. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-01013-x · Cited by 2
“Effectively measuring and communicating the uncertainty associated with AI outputs has long been considered critical for enhancing AI transparency 2 .”
Means to an End: Assessing the Validity of Trust Scales in Artificial Intelligence Vignettes
Jensen, T., Haber, G., & Choong, Y. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93412-4_5 · Cited by 1
Chen, C., & Goh, D. H. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-94153-5_16 · Cited by 4
Löfström, T., Yapicioglu, F. R., Stramiglio, A., Löfström, H., & Vitali, F. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-08333-3_16 · Cited by 3
Understanding Machine-Learning-Based Urban Parking Predictions: A Dashboard Approach
Rolwes, A., Roussel, C., Müller, T., Böhm, K., Raßmann, G., Weiß, J.-N., Weichold, T., & Balzer, B. (2025) · International Conference on Information Visualisation (IV) · doi:10.1109/iv68685.2025.00051 · Cited by 1
Optimizing In-Context Learning for Efficient Full Conformal Prediction
Deng, W., Park, S., Li, M., & Simeone, O. (2025) · IEEE Signal Processing Letters · doi:10.1109/lsp.2025.3636762 · Cited by 3
“To elaborate, assume the availability of a dataset D = { ( x i , y i ) } ni =1 with the i -th input x i ∈ X and the corresponding discrete label y i ∈ Y = { 1 , . . . , K } .”
Human at the Center: A Framework for Human‐Driven AI Development
Hu, D., Acosta-Navas, D., Gaube, S., Mozannar, H., Taylor, M. E., Dvijotham, K., & Bondi, E. (2025) · AI Magazine · doi:10.1002/aaai.70043 · Cited by 4
If It’s Safe, Don’t Ask: Decreasing Frustration through User Involvement for Risky Robot Behaviors
Leusmann, J., Schömbs, S., Diedrich, M., & Müller, F. (2026) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/3757279.3785545 · Cited by 2
Presta, R., Simone, F. D., Bacchiani, L., & Girau, R. (2026) · Technologies · doi:10.3390/technologies14070386 · Cited by 1
Pendrill, L., & Fisher, W. P. (2026) · Foundations · doi:10.3390/foundations6030026 · Cited by 1
“There is no obvious one-to-one correlation between discrete and continuous ranges versus qualitative and quantitative scales [23].”
Effects of Uncertainty and Knowledge Graph on Perception of Fairness
Zhou, J. J., Zheng, B., & Chen, F. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581754.3584157 · Cited by 1
“Similar to prior research [4, 6], we measure the uncertainty by predictive entropy [16] in this study.”
ResultOn Uses of Noise Analysis for the Uncertainty Quantification of Line Edge Roughness Estimation
Akpabio, I. I., & Savari, S. A. (2023) · IEEE Transactions on Semiconductor Manufacturing · doi:10.1109/tsm.2023.3270230 · Cited by 1
“together consist of 11520 noise images corresponding to correlation lengths (ξ) in the set {10, 20, 30, 40} nm which are split into two sets of 5760 images for the calibration and test sets, respectively, as discussed in [6].”
BackgroundMethodInfluentialNumerical versus graphical aids for decision-making in a multi-cue signal identification task
Szulc, J., & Fletcher, K. I. (2024) · Applied Ergonomics · doi:10.1016/j.apergo.2024.104260 · Cited by 1
Source-Free Continual Adaptive Learning With Limited Labels on Evolving Data Drifts
Machireddy, A., Krishnan, R., Narayanan, A., & Tickoo, O. (2024) · Proceedings - International Conference on Image Processing · doi:10.1109/icip51287.2024.10648016 · Cited by 2
Self-citationOn an Application of Denoising to the Uncertainty Quantification of Line Edge Roughness Estimation
Akpabio, I. I., & Savari, S. A. (2022) · ASMC proceedings · doi:10.1109/asmc54647.2022.9792521 · Cited by 1
Improving Robustness and Efficiency in Active Learning with Contrastive Loss.Preprint
Krishnan, R., Ahuja, N., Sinha, A., Subedar, M. M., Tickoo, O., & Iyer, R. (2021) · arXiv (Cornell University) · Cited by 1
Self-citationCURE2026: Communicating Uncertainty to foster Realistic Expectations via Human-Centered Design
Gajcin, J., Jeromela, J., Wester, J., Schömbs, S., Kleanthous, S., Ramamurthy, K. N., Hauptmann, H., & Amin, R. M. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742414.3794961 · Cited by 2
“However, uncertainty communication is plagued by challenges such as cognitive biases [4, 25, 35], numeracy skills [11, 26], calibrating risk perception, [29] and increased cognitive load [13], with research fnding that lay users can struggle to interpret probabilities and uncertainty visualizations…”
Sensemaking in Multi-Agent LLM Interfaces: How Users Interpret Transparency and Trustworthiness Cues
Pareek, S., Govers, J., Kollerup, N. K., Wong, E., Velloso, E., & Gonçalves, J. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791157 · Cited by 6
“Even uncertainty information, which is often intended to serve as a proxy for (expected) AI performance, can have unintended effects: for instance, being interpreted as a signal of transparency and honesty by the AI designers, thereby boosting trust (even when unwarranted) [7, 36, 65].”
Keys, R., O’Kane, A. A., Marshall, P., & Stuart, G. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790692
“However, the taxonomy of uncertainty must be considered alongside risk: how uncertainty is perceived by clinicians directly influences perceptions of risk [16].”
Metadata
- Authors
- Bhatt, U., Antorán, J., Zhang, Y., Liao, Q. V., Sattigeri, P., Fogliato, R., Melançon, G., Krishnan, R., Stanley, J., Tickoo, O., Nachman, L., Chunara, R., Srikumar, M., Weller, A., & Xiang, A.
- Affiliations
- University of Cambridge (GB)
- IBM (United States) (US)
- Carnegie Mellon University (US)
- Concordia University (CA)
- Intel (United States) (US)
- New York University (US)
- Partnership on AI, San Francisco, CA, USA
- Sony AI, Seattle, WA, USA
- ORCID
- Venue
- Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2021-07-21
- Pages
- 401-413
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W3183398589
- Semantic Scholar
- 3973e0fab69a00f5ed6a81ca408f60c420fa6e61
- Cited by
- 300
- References
- 116
- Citations by year
- 2026: 59 · 2025: 62 · 2024: 65 · 2023: 46 · 2022: 48 · 2021: 18
- FWCI
- 16.64
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- arXiv (Cornell University)
- Funders
- Microsoft Research
- Northwestern University
- Alan Turing Institute (EP/N510129/1., TU/B/000074, N510129)
- DeepMind
- Leverhulme Trust
- Engineering and Physical Sciences Research Council (N510129, EP/N510129/, EP/N510129/1, TU/B/000074, 2275741, EPSRC grant EP/N510129/1, EP/V025279/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- algorithmic transparency, uncertainty estimation, uncertainty communication, explainability, model fairness, trustworthy systems, decision-making, uncertainty visualization, human-computer interaction, machine learning pipelines, stakeholder awareness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-24, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-24 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.