Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
Open accessThe DOI matches its Crossref record; 224 of the 230 references it cites were found in a registry or an index, and 6 are not checked yet; 41 of the 135 sources its claims cite were checked for support.
Abstract
Algorithmic transparency entails exposing system properties to various stakeholders for purposes that include understanding, improving, and contesting predictions. Until now, most research into algorithmic transparency has predominantly focused on explainability. Explainability attempts to provide reasons for a machine learning model's behavior to stakeholders. However, understanding a model's specific behavior alone might not be enough for stakeholders to gauge whether the model is wrong or lacks sufficient knowledge to solve the task at hand. In this paper, we argue for considering a complementary form of transparency by estimating and communicating the uncertainty associated with model predictions. First, we discuss methods for assessing uncertainty. Then, we characterize how uncertainty can be used to mitigate model unfairness, augment decision-making, and build trustworthy systems. Finally, we outline methods for displaying uncertainty to stakeholders and recommend how to collect information required for incorporating uncertainty into existing ML pipelines. This work constitutes an interdisciplinary review drawn from literature spanning machine learning, visualization/HCI, design, decision-making, and fairness. We aim to encourage researchers and practitioners to measure, communicate, and use uncertainty as a form of transparency.
Verified 2026-11-22 · 6 references not checked yet
Overview
CoChat Summary
TL;DR
This paper argues that estimating and communicating uncertainty should be treated as a complementary form of algorithmic transparency alongside explainability. It reviews methods for measuring uncertainty and discusses how uncertainty can support fairness, decision-making, trust, and stakeholder communication.
Key findings
- Uncertainty is split into aleatoric uncertainty (irreducible noise/class overlap) and epistemic uncertainty (reducible with more data); calibration is a precursor to using uncertainty as transparency, and deep ensembles are described as one of the best-performing uncertainty quantification approaches for neural networks, retaining calibration under dataset shift.
- Uncertainty can help address fairness issues: representation bias is epistemic and may be detected via large epistemic uncertainty on validation/deployment data, while feature and label noise are aleatoric and can distort fairness assessments; even a small amount of label noise can greatly impact fairness evaluation.
- In decision-making, uncertainty supports reject options and decision-theoretic action selection; JDM findings indicate risk is considered together with expected outcome, asymmetrically, and tolerance for uncertainty decreases superlinearly as stakes increase.
- Communication is difficult: in a 2010 US/Germany survey, 20% could not identify 10% as the biggest disease risk among 1%, 5%, and 10%, and almost 30% could not answer whether 1 in 10, 1 in 100, or 1 in 1000 represents the largest risk; people interpreted the IPCC’s “very likely” as around 60% probability, whereas the guideline specifies greater than 90%. Quantile dot plots and hypothetical outcome plots (HOPs) are reported to improve some probability/uncertainty judgments.
Methods
- Interdisciplinary literature review, not an empirical study, drawn from machine learning, visualization/HCI, design, decision-making, and fairness.
- Uses a recurring breast cancer diagnostics scenario to illustrate uncertainty in fairness, decision-making, trust formation, and communication.
- Synthesizes uncertainty quantification approaches (Bayesian methods, frequentist methods such as ensembles), evaluation metrics (calibration, NLL, Brier score, ECE), and communication methods (categories, numerical ranges, summary statistics, full distributions, visualizations).
Limitations
- The paper states that its findings should not be treated as conclusive for predicting real-world usability and recommends testing with real users in concrete tasks and settings.
- The paper states that empirical understanding of how decision-makers use aleatoric versus epistemic uncertainty is limited, that JDM literature has mostly focused on discrete outcomes, and that there is not a good understanding of how people perceive uncertainty over continuous outcomes such as error bars.
- Evident from the text: as a review, it provides no primary empirical validation or systematic review protocol; several claims are speculative or drawn from other fields and are framed as open questions for future research.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




![(c) Cone of uncertainty, showing the uncertainty around the predicted path of the center of a hurricane. Taken from [93].](https://figures.cochat.ai/paper-figures/42e1c0dd14b540788474c4bee4ed993b/uncertainty-as-a-form-of-transparency-measuring-communicating-and-using-figure-5.png)

Verification
Claims the paper makes
28 Valid · 10 Not valid · 60 Not checked · 104 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“When information on the sensitive attribute is unavailable (e.g., information on gender was not collected), but can be predicted from an auxiliary dataset, disparity measures are generally unidentifiable [20,63].”
§ Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty > 3.1 Uncertainty and Fairness
What the sources say
- Partly supportedcites [20] 20. Jiahao Chen, Nathan Kallus, Xiaojie Mao, Geoffry Svacha, and Madeleine Udell. 2019. Fairness under unawareness: Assessing disparity when protected class is unobserved. In Proceedings of the conference on fairness, accountability, and transparency. 339-348. sourceExperimental — accuracy not yet measured
Section 1.1 (p. 2) describes assessing disparity when protected-class labels are unavailable and must be imputed from proxy variables. The paper analyzes bias in the resulting disparity estimates, but the cited passages do not establish the broader claim that disparity measures are generally unidentifiable; the Conclusions (p. 9) instead note that the optimal imputation threshold cannot be determined without ground-truth labels. (LLM, conf=0.82)
- Supportedcites [63] 63. Nathan Kallus, Xiaojie Mao, and Angela Zhou. 2019. Assessing algorithmic fairness with unobserved protected class using data combination. arXiv preprint arXiv:1906.00285 (2019). source
The paper’s problem formulation describes a primary dataset missing protected-class labels and an auxiliary dataset containing those labels and proxy variables. Section 4 states that disparity measures are generally unidentifiable from the two datasets because their marginals do not uniquely determine the full joint distribution. The paper notes an exception for perfectly predictive proxies, consistent with the claim’s qualifier ‘generally.’ (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The experimental results of Gupta et al. [44] have shown that enforcing fairness constraints on a noisy sensitive attribute, without assumptions on the structure of the noise, is not guaranteed to lead to any improvement in the fairness properties of our model.”
§ Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty > 3.1 Uncertainty and Fairness
What the sources say
- Partly supportedcites [44] 44. Maya Gupta, Andrew Cotter, Mahdi Milani Fard, and Serena Wang. 2018. Proxy fairness. arXiv preprint arXiv:1806.11212 (2018). sourceExperimental — accuracy not yet measured
Section 4 (pp. 3–4) says poorly chosen proxy groups can worsen fairness for the true protected groups, supporting the claim that improvement is not guaranteed. But the paper presents this as a theoretical possibility, while its reported experiments generally found improvements (Conclusions, p. 8). It does not frame the result as an experiment on noisy sensitive attributes without assumptions about the noise structure. (LLM, conf=0.84)
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Indeed, Fogliato et al. [35] show that even a small amount of label noise can greatly impact the assessment of the fairness properties of the model.”
§ Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty > 3.1 Uncertainty and Fairness
What the sources say
- Supportedcites [35] 35. Riccardo Fogliato, Max G'Sell, and Alexandra Chouldechova. 2020. Fairness Evaluation in Presence of Biased Noisy Labels. arXiv preprint arXiv:2003.13808 (2020). source
The Abstract (p. 1) reports that even small biases in observed labels may call into question conclusions drawn from evaluating a model against those labels. The Conclusion (p. 8) gives an example in which a small gap in arrest likelihood could fully account for observed miscalibration. This supports the claim that a small amount of biased label noise can substantially affect fairness assessment. (LLM, conf=0.89)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Here, measures of demographic parity and equalized odds computed on the observed data are equal to the true metrics up to a scaling factor, which is proportional to the value of the noise [71]; if the noise rates are known, then the true metrics can be directly estimated.”
§ Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty > 3.1 Uncertainty and Fairness
What the sources say
- Supportedcites [71] 71. Alex Lamy, Ziyuan Zhong, Aditya K Menon, and Nakul Verma. 2019. Noise- tolerant fair classification. In Advances in Neural Information Processing Systems. 294-306. source
Section 4.2 (p. 6) states that MC noise scales the fairness measure, giving the clean measure as (1 − α − β) times the corrupted-data measure. Section 3.3 (p. 4) says equalized odds can be captured by adding a constraint for Y = 0, and §4.3 (p. 6) explains how known noise parameters supply the scaling factor. The factor depends on the noise rates; more precisely, it is 1 − α − β, not directly proportional to their value. (LLM, conf=0.88)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Similarly to noise in the sensitive attribute, Blum and Stangl [11], Jiang and Nachum [59] show that fairness constraints are guaranteed to improve the properties of a predictor only under appropriate assumptions on the label noise.”
§ Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty > 3.1 Uncertainty and Fairness
What the sources say
- Partly supportedcites [11] 11. Avrim Blum and Kevin Stangl. 2019. Recovering from biased data: Can fairness constraints improve accuracy? arXiv preprint arXiv:1912.01094 (2019). sourceExperimental — accuracy not yet measured
Section 3.3 (p. 11) says Equal Opportunity-constrained ERM recovers the Bayes-optimal classifier when Group A has sufficient mass and the signal is not too noisy. Section 3.1 (p. 9) also gives a recovery range bounded by the noise parameter η. The claim captures the dependence on noise assumptions, but overgeneralizes the guarantee to fairness constraints generally and omits the paper’s other conditions. (LLM, conf=0.86)
- Partly supportedcites [59] 59. Heinrich Jiang and Ofir Nachum. 2020. Identifying and correcting label bias in machine learning. In International Conference on Artificial Intelligence and Statistics. 702-712. sourceExperimental — accuracy not yet measured
Section 2.1 (p. 2) says that recovering unbiased labels requires an assumption about their relationship to observed biased labels. The Introduction (p. 1) reports a theoretical guarantee for the paper’s re-weighted training method under its model. It does not establish the broader claim that fairness constraints improve a predictor only under appropriate assumptions about label noise. (LLM, conf=0.83)
boostedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
74 more claims · 11 with their cited sources checked
Create a free account to see all 202 claims and to verify the 24 references still open.
- It may state two separate claims as one. 40
- Probably not a substantial claim. 105
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Uncertainty is considered a critical form of transparency that can alert people about the limitations of AI [10].”
From Investigating Explainability of Generative AI for Code through Scenario-based Design (2022) · cites this paper as [10]
The cited source says what the claim attributes to it.
In the citing paper
“Similar metrics have been used to evaluate the calibration of uncertainty quantification [7,9,64]-the estimated uncertainty should reflect the observed error rate.”
From Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations (2023) · cites this paper as [7]
The cited source says what the claim attributes to it.
In the citing paper
“Uncertainty expression can also increase the system's perceived transparency [13], which is generally shown to enhance trust, whether or not appropriate [56].”
From "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust (2024) · cites this paper as [13]
The cited source says what the claim attributes to it.
In the citing paper
“Finally, indicating model uncertainty [19,174,198], for example, by means of color-coding [167], and an explanation for that uncertainty [5,190], can also help users disentangle the role of their prompting and output evaluation ability from that of models' capabilities, further supporting confidence adjustment [197].”
From The Metacognitive Demands and Opportunities of Generative AI (2024) · cites this paper as [19]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty > References
“137. Yunfeng Zhang, Rachel KE Bellamy, and Wendy A. Kellogg. 2012.”
§ Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty > References
“In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (2015).”
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb01338.x · Cited by 88,486
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Tversky, A., & Kahneman, D. (1974) · Science · doi:10.1126/science.185.4157.1124 · Cited by 28,810
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,370
Judgment under Uncertainty: Heuristics and Biases
Tversky, A., & Kahneman, D. (1975) · Utility, Probability, and Human Decision Making · doi:10.1007/978-94-010-1834-0_8 · Cited by 22,650
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Knight, F. H. (1921) · Cited by 16,656
Breiman, L. (1996) · Machine Learning · doi:10.1007/bf00058655 · Cited by 16,325
Advances in prospect theory: Cumulative representation of uncertainty
Tversky, A., & Kahneman, D. (1992) · Journal of Risk and Uncertainty · doi:10.1007/bf00122574 · Cited by 15,467 · cited 2×
“Research also found that this bias occurs when people are asked to make decisions under risk and that their decisions imply such distortions [125, 137].”
Background(2011) · Cited by 14,317
Research works printed but not linked: 10
- 6. Emily Badger, Claire Cain Miller, Adam Pearce, and Kevin Quealy. 2018. Income Mobility Charts for Girls, Asian-Americans and Other Groups. Or Make Your Own. (2018).
- 19. Shelly Chaiken. 1999. The heuristic-systematic. Dual-process theories social psychology 73 (1999).
- 23. Marvin S Cohen, Raja Parasuraman, and Jared T Freeman. 1998. Trust in decision aids: A model and its training implications. In in Proc. Command and Control Research and Technology Symp. Citeseer.
- 36. Yarin Gal. 2016. Uncertainty in deep learning. University of Cambridge 1, 3 (2016).
- 54. Carl Iver Hovland, Irving Lester Janis, and Harold H Kelley. 1953. Communica- tion and persuasion. (1953).
- 87. Malik Sajjad Ahmed Nadeem, Jean-Daniel Zucker, and Blaise Hanczar. 2009. Accuracy-Rejection Curves (ARCs) for Comparing Classification Methods with a Reject Option. In Proceedings of the third International Workshop on Machine Learning in Systems Biology (Proceedings of Machine Learning Research), Sašo Džeroski, Pierre Guerts, and Juho Rousu (Eds.), Vol. 8. PMLR, Ljubljana, Slovenia, 65-81. http://proceedings.mlr.press/v8/nadeem10a.html
- 91. Radford M Neal. 1995. Bayesian Learning for Neural Networks. PhD thesis, University of Toronto (1995).
- 103. Jennifer Preece, Helen Sharp, and Yvonne Rogers. 2015. Interaction design: beyond human-computer interaction. John Wiley & Sons.
- 118. S Shyam Sundar. 2008. The MAIN model: A heuristic approach to understanding technology effects on credibility. MacArthur Foundation Digital Media and Learning Initiative.
- 130. Claus O. Wilke. 2019. Fundamentals of Data Visualization: A Primer on Making Informative and Compelling Figures (1st edition ed.). O'Reilly Media.
Cited by
Items 201–248 of 248
A11yExtensions: Accessibility Extensions to Augment Mobile AI Assistive Technology In-Situ
Herskovitz, J., Seehorn, E., Jammoa, A., Meddaugh, J., & Guo, A. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791559 · Cited by 2
“We also note that large amounts of HCI research have also addressed related challenges, in areas such as human–AI collaboration and explainability [11, 12, 23, 56], nonvisual photography [41, 48], and adaptive or customizable software [36, 43, 65].”
Pre-emptive learning-to-defer for sequential medical decision-making under uncertainty.Preprint
Joshi, S. D., Parbhoo, S., & Doshi‐Velez, F. (2021) · arXiv (Cornell University) · Cited by 2
Communication of Uncertainty in AI Regulations
Phutane, A. S. (2023) · Community Change · doi:10.21061/cc.v5i1.a.50
Bayes in action in deep learning and dictionary learning
Arbel, J., Dang, H.-P., Elvira, C., Herzet, C., Naulet, Z., & Vladimirova, M. (2023) · ESAIM Proceedings and Surveys · doi:10.1051/proc/202374090
“Uncertainty estimates can also serve as a means of transparency as they inform when the model does not know the correct prediction [15].”
Explaining Prediction Uncertainty in Text Classification: The DUX Approach
Andersen, J. S., & Zukunft, O. (2023) · International Conference on Natural Language Processing and Information Retrieval (NLPIR) · doi:10.1145/3639233.3639352
“However, understanding the behavior of a model alone may not be adequate to assess whether the model is mistaken or lacks information to resolve the given task [6].”
BackgroundTrust in Robot Benchmarking and Benchmarking for Trustworthy Robots
Thoduka, S., Nair, D., Caleb-Solly, P., Dragone, M., Cavallo, F., & Hochgeschwender, N. (2024) · Studies in computational intelligence · doi:10.1007/978-3-031-55817-7_3 · Cited by 1
Public Health Decision-Making Using Uncertainty Displays
Tessema, T., Vasquez, H. M., Rosella, L. C., et al. (2024) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/10711813241280938
“These tools need to be well-designed, taking into account human factors such as usability and workload, and should effectively convey uncertainty information to users (Bhatt et al., 2021; Eberhard, 2021; Friedman, 2009; Harrigan, 2021; Lee et al., 2017).”
Interpretability of Uncertainty: Exploring Cortical Lesion Segmentation in Multiple Sclerosis
Molchanova, N., Cagol, A., Gordaliza, P. M., Ocampo‐Pineda, M., Lu, P., Weigel, M., Chen, X., Depeursinge, A., Granziera, C., Müller, H., et al. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-77610-6_9 · Cited by 2
Input Uncertainty Attribution by Uncertainty Propagation
Kantz, B., Steger, S., Staudinger, C., Feilmayr, C., Wachlmayr, J., Haberl, A., Schuster, S., & Pernkopf, F. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10890257
“Such robust, uncertainty-aware systems are essential for implementing ML as the trust in these systems can be increased significantly by using explanations and uncertainty estimates presented to the end-users [5].”
Assessor Models for Predicting and Explaining Aleatoric Uncertainty in Classification
Lima, P. B. S., Prudêncio, R. B. C., Lorena, A. C., & Valeriano, M. G. (2024) · doi:10.5753/eniac.2024.245096
Scaling Analysis of Creative Activity Traces via Fuzzy Linkography
Smith, A., Anderson, B. R., Otto, J., Karth, I., Sun, Y., Chung, J. J. Y., Roemmele, M., & Kreminski, M. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3720184 · Cited by 2
Designing Embodied Agents for Impactful Human-Centred Risk Communication
Schömbs, S. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3707602 · Cited by 2
A Review of Uncertainty Quantification in Convolutional Neural Networks
Montazeri, S., & Tavallali, P. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-86623-4_4 · Cited by 1
Explainable AI in Ad Enforcement: Striking the Balance Between Transparency and Safety
Shah, B. M. (2025) · Journal of Information Systems Engineering & Management · doi:10.52783/jisem.v10i41s.7584
“● Appeal rates: Reduced number of unsuccessful and unnecessary appeals [109].”
Explainability of Language Models (XAI and LLM)
Cecco, A. D., & Gianfagna, L. (2025) · Springer eBooks · doi:10.1007/978-3-031-92229-9_8 · Cited by 1
Communicating Uncertainty Can Increase AI AdoptionPreprint
Foroughifar, M., Ghanavi, R., Avi Goldfarb, B., & Webb, R. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5468868
LA-GMF-Based Framework for Transparent Alzheimer’s Diagnosis
Maranhão, G. B. A., & Mello, C. A. B. (2025) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc58881.2025.11343059
“While algorithmic transparency in ML has primarily focused on explainability, defined as the attempt to make a model’s behavior understandable to stakeholders, this alone may be insufficient to detect incorrect predictions [4].”
InfluentialShank, D. B., Canfield, C., & Subramanian, H. V. (2024) · OSF Preprints (OSF Preprints)
Mining hidden layers in CNN for certainty prediction in medical imaging
Hirimutugoda, Y. M., Sliva, T. P., & Wagarachchi, N. M. (2026) · Neural Computing and Applications · doi:10.1007/s00521-025-11840-z
Drivers and influence of social conformity on decision making in human-AI teams
Zhong, H., McKinlay, J., Yoon, J., Bagri, R., O'Neill, E. J., & Hoffmȧnn, J. (2026) · Scientific Reports · doi:10.1038/s41598-026-43042-5 · Cited by 4
“Studies explicitly investigating social conformity to AI agents have predominantly considered interactions between one human and one AI rather than advice from mixed teams composed of multiple humans and AI 12,13,32 .”
Morais, L. R. D. A., & Ludermir, T. B. (2026) · International Journal of Data Science and Analytics · doi:10.1007/s41060-026-01098-6
Bertrand, A. (2024) · theses.fr (ABES) · doi:10.70675/216d3647z78bcz4c68z9ba9z2d0a029cf1b4 · Cited by 8
Inside the Newsroom: How Media Company AI Strategies Fall Short in Practice
Pedersen, T. R., Øvlisen, K., & Petersen, M. (2026) · CHI Extended Abstracts · doi:10.1145/3772363.3799296 · Cited by 1
Qin, W., Xiao, F., Fang, Y., Tang, Z., Kang, S., Huang, Z., & He, T. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791067 · Cited by 1
“This format enables learners to witness the immediate reactions of previous viewers, fostering a sense of co-presence, the feeling of learning alongside others, even in asynchronous learning settings [11, 68].”
Lu, Z., Lim, G., & Yin, M. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790442 · Cited by 3
“Prior research in uncertainty communication [14, 108] also offers design insights for presenting probabilistic estimates in ways that improve users’ understanding without overwhelming them.”
Counterfactual Explanations for Conformal Regression Intervals
Maalej, A., & Johansson, U. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-23833-7_11 · Cited by 1
Kim, N., Kim, H. C., Lee, S., & Nam, C. (2026) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2026.3686647
“Transparency reduces this uncertainty [21].”
Human-AI Partnership and Innovative Work Behaviour
Parashar, A., & Singh, A. (2026) · Future of business and finance · doi:10.1007/978-3-032-06604-6_6
Arous, I., Chehbouni, K., Cheng, Z., & Dossou, B. F. P. (2026) · Springer eBooks · doi:10.1007/978-981-95-3658-0_85
Building user-driven climate adaptation products
Chaudhry, N. A., Collins, W. D., Anthoff, D., & Jones, A. D. (2026) · Nature · doi:10.1038/s41586-026-10555-y
“Given its foundation in usability and user experience, user-centred design is widely applied in developing websites, software applications, and other digital or software-based products 32,75,76 .”
A Multidimensional Taxonomy of Generative Artificial Intelligence Approaches for Drug Discovery
Crozes, T., Sabando, V., Soto, A. J., & Ponzoni, I. (2026) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.70104
“Finally, the communication of AD metrics to the chemical expert introduces new challenges as it is important to convey the metric in a non-technical way, but also showing what this uncertainty represents and how the expert could potentially improve the generated molecules (Bhatt et al. 2021).”
Fairness under uncertainty in sequential decisions
Lee, M. S. A., Padh, K. S., Watson, D., Kilbertus, N., & Singh, J. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812272 · Cited by 3
“Bhatt et al. [8] breaks this down further into model uncertainty in reference to model parameters and model specification uncertainty in reference to the model type.”
InfluentialInformation science for materials science
Rajan, K. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-22256-6.00001-4
The impact of AI transparency and AI authorization on human performance in AI-to-human delegation
Tang, J., Wang, Y., Zhou, X., Zhao, Y., & Jiang, Y. (2026) · Electronic Commerce Research · doi:10.1007/s10660-026-10166-9
A Conceptual Framework for Human-in-the-Loop Multi-attribute Decision Making Under Uncertainty
Çakır, E., Demırcıoğlu, E., & Tolga, A. Ç. (2026) · Lecture notes in computer science · doi:10.1007/978-981-92-2497-5_12
Cognitive Readiness for Human-AI Collaboration
Tremblay, S., de Hemptinne, D., Teyssier‐Roberge, G., Gallant, A., Marois, A., & Lafond, D. (2026) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208261474345 · Cited by 5
Lubis, M., Saleh, R. Z., Fauzan, R. A., Wibowo, J. A., Rahman, K. B. A., & Salleh, N. M. B. M. (2026) · International Conference on Smart Mechatronics (ICSMech) · doi:10.1109/icsmech69390.2026.11647228
“More recent HCI studies show that uncertainty displays change overreliance, trust, and sensemaking depending on how the uncertainty is framed [10].”
Ontology-Guided Decoding as Local Semantic Priming: A Mechanistic Study and Deployment DiagnosticPreprint
Karamanlıoğlu, A., Yurtalan, G., & Demirel, B. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7327575
Communicating AI Uncertainty in Assistive Navigation for People with Visual Impairments
Shaffer, H., Shaikh, A., Zhang, H., & Xie, J. (2026) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/10711813261484587
AI-Driven Visual Analytics: Designing Intelligent and Explainable Decision Systems
Acevedo, M. A. A., Bolivar, J. J., & Yosifov, N. M. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-3930-4.ch004
Navigating uncertainty in digital twin ecosystems: A unified framework and key challenges
Tan, A. H., & Chuah, T. C. (2026) · Journal of Industrial Information Integration · doi:10.1016/j.jii.2026.101189
Conceptual Approaches to Uncertainty Communication in Materials AI: A Review Study
Nkosi, T., Molefe, L., Dlamini, S., Mokoena, A., & Ndlovu, K. (2025) · Journal of Artificial Intelligence for Materials Science · doi:10.68159/l397814019
Wang, K., Pan, Z., & Lu, Y.-B. (2026) · Computers in Human Behavior · doi:10.1016/j.chb.2026.109201
Acharya, N., & Kar, P. (2026) · Frontiers in Future Transportation · doi:10.3389/ffutr.2026.1951756
AI-Enabled Digital Health Tool Implementation in Clinical Practice Settings: Umbrella Review
Gilbert, A. S., Gacad, A., Parrish, L. J., Nguyen, O. T., Turner, K. L., Tarver, W. L., Jonnalagadda, P., Tsagli, D. D., Wu, B., Lu, C., et al. (2026) · JMIR AI · doi:10.2196/101509
User-Friendly AI-Based Decision Support Systems
Echeverry, N. (2026) · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw71585.2026.11712078
“The transparency of a system has been associated with its perceived fairness [5], [25].”
Smart Resume Analysis for IT Skill Profiling Using Natural Language Processing Techniques
Huzairi, M. B. A., Ahmad, S., Zulkifli, Z., Mohamad, M., Ariff, M. I. M., & Nasrudin, N. H. (2026) · International Conference on Artificial Intelligence and Data Sciences (AiDAS) · doi:10.1109/aidas71914.2026.11712779
“The main limitation observed was semantic overlap between roles that share 70% or more of the same keywords, such as Full Stack and Frontend [19],[20],[21].”
What Is the Best Word to Describe AI Uncertainty: Confidence, Certainty, Reliability, or Accuracy?
Newendorp, A. K., & Brucciani, L. D. R. (2026) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/10711813261495702
Metadata
- Authors
- Bhatt, U., Antorán, J., Zhang, Y., Liao, Q. V., Sattigeri, P., Fogliato, R., Melançon, G., Krishnan, R., Stanley, J., Tickoo, O., Nachman, L., Chunara, R., Srikumar, M., Weller, A., & Xiang, A.
- Affiliations
- University of Cambridge (GB)
- IBM (United States) (US)
- Carnegie Mellon University (US)
- Concordia University (CA)
- Intel (United States) (US)
- New York University (US)
- Partnership on AI, San Francisco, CA, USA
- Sony AI, Seattle, WA, USA
- ORCID
- Venue
- Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society
- Publisher
- ACM
- Published
- 2021-07-21
- Pages
- 401-413
- ISSN
- 3065-8365 (ISSN-L)
- OpenAlex
- W3183398589
- Semantic Scholar
- 3973e0fab69a00f5ed6a81ca408f60c420fa6e61
- Cited by
- 300
- References
- 116
- Citations by year
- 2026: 59 · 2025: 62 · 2024: 65 · 2023: 46 · 2022: 48 · 2021: 18
- FWCI
- 16.64
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Cambridge University Engineering Department Publications Database (submittedVersion)
- arXiv (Cornell University)
- Funders
- Microsoft Research
- Northwestern University
- Alan Turing Institute (EP/N510129/1., TU/B/000074, N510129)
- DeepMind
- Leverhulme Trust
- Engineering and Physical Sciences Research Council (N510129, EP/N510129/, EP/N510129/1, TU/B/000074, 2275741, EPSRC grant EP/N510129/1, EP/V025279/1)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- algorithmic transparency, uncertainty estimation, uncertainty communication, explainability, model fairness, trustworthy systems, decision-making, uncertainty visualization, human-computer interaction, machine learning pipelines, stakeholder awareness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-24, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-24 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.