The DOI matches its Crossref record; 122 of the 123 references it cites were found in a registry or an index; 54 of the 92 sources its claims cite were checked for support.
"I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust
Open accessAbstract
Widely deployed large language models (LLMs) can produce convincing yet incorrect outputs, potentially misleading users who may rely on them as if they were correct. To reduce such overreliance, there have been calls for LLMs to communicate their uncertainty to end users. However, there has been little empirical work examining how users perceive and act upon LLMs’ expressions of uncertainty. We explore this question through a large-scale, pre-registered, human-subject experiment (N=404) in which participants answer medical questions with or without access to responses from a fictional LLM-infused search engine. Using both behavioral and self-reported measures, we examine how different natural language expressions of uncertainty impact participants’ reliance, trust, and overall task performance. We find that first-person expressions (e.g., “I’m not sure, but...”) decrease participants’ confidence in the system and tendency to agree with the system’s answers, while increasing participants’ accuracy. An exploratory analysis suggests that this increase can be attributed to reduced (but not fully eliminated) overreliance on incorrect answers. While we observe similar effects for uncertainty expressed from a general perspective (e.g., “It’s not clear, but...”), these effects are weaker and not statistically significant. Our findings suggest that using natural language expressions of uncertainty may be an effective approach for reducing overreliance on LLMs, but that the precise language used matters. This highlights the importance of user testing before deploying LLMs at scale.
Overview
CoChat Summary
TL;DR
In a pre-registered experiment (N=404), first-person LLM uncertainty expressions such as “I’m not sure, but...” reduced participants’ agreement with and confidence in an LLM-infused search system while increasing their accuracy on medical yes/no questions. General-perspective uncertainty expressions showed similar but weaker, non-significant effects.
Key findings
- First-person uncertainty expressions decreased agreement with the AI system (74.8% vs. 80.9% Control), decreased confidence in the AI’s answer (3.66 vs. 3.95 on a 5-point scale), and increased accuracy (72.8% vs. 63.9% Control). General-perspective expressions showed weaker, non-significant effects (Agree 77.6%, ConfidenceAI 3.80, Correct 67.9%).
- Within-condition, uncertain vs. not-uncertain AI responses lowered agreement (UncertainGeneral: 70.9% vs. 84.7%; Uncertain1st: 73.4% vs. 79.5%, not significant), lowered confidence in the AI answer (Uncertain1st: 3.44 vs. 3.88; UncertainGeneral: 3.65 vs. 3.95), lowered self-reported use of the AI answer (Uncertain1st: 57.4% vs. 73.2%; UncertainGeneral: 62.9% vs. 79.7%), and increased task time in the general condition (2.23 vs. 1.84 min).
- Trust intentions were lower in Uncertain1st (2.91) than Control (3.25) and UncertainGeneral (3.36); there were no significant differences in TrustBelief, Anthropomorphism, or Transparency.
- Exploratory analysis suggested reduced but not eliminated overreliance: uncertainty improved accuracy when the AI was incorrect (Uncertain1st: 43.6% to 52.0%; UncertainGeneral: 32.8% to 48.0%) while slightly reducing accuracy when the AI was correct (Uncertain1st: 92.2% to 89.2%; UncertainGeneral: 94.8% to 83.1%); No-AI participants still had higher accuracy (74.2%) than Control (63.9%).
Methods
- Pre-registered between-subjects experiment with within-subjects comparisons; final sample N=404 U.S.-based MTurk participants (104 Control, 92 Uncertain1st, 94 UncertainGeneral, 114 No-AI). Participants answered eight challenging yes/no medical questions with or without responses from a fictional LLM-infused search engine, “AI system A.”
- AI answers were correct for four of eight questions; in uncertainty conditions, half of responses included uncertainty expressed in first person or general perspective. Dependent variables included Agree, Correct, Time, LinkClick, UseAI, UseLink, UseInternet, ConfidenceAI, ConfidenceAnswer, TrustBelief, TrustIntention, Anthropomorphism, and Transparency.
- Analyses used mixed models with participant and question random effects, ANOVA with Tukey post-hoc tests, within-condition models, exploratory over/underreliance analyses, and thematic analysis of free-form responses. AI responses were based on Copilot in Bing responses from July 2023, with minor modifications and uncertainty phrases inserted.
Limitations
- The paper states that simple yes/no tasks limit generalizability to complex tasks; time and source-usage measures are less reliable than in a lab study; medical questions were predefined rather than personal symptoms; behavior may differ with repeated system interaction; the AI system had low accuracy and expressed uncertainty often in a poorly calibrated manner; and results may not generalize across cultures and languages because the study was in English with U.S.-based participants.
- The paper also cautions against overgeneralizing and notes tradeoffs between experimental controllability and real-world generalizability, plus possible tradeoffs between reducing overreliance and decreasing trust.
- Evident from the text: the Transparency scale had low internal consistency (Cronbach’s alpha = 0.21), and the discrepancy between LinkClick and UseLink suggests measurement or self-report issues; the sample was younger, more educated, and overrepresented white respondents compared with U.S. Census data.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
26 Valid · 4 Not valid · 10 Not checked · 107 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“in AI-assisted decision-making settings, communicating (un)certainty has been shown to support trust calibration [118], increase vigilance [85], and improve task performance [7].”
What the sources say
- Supportedcites [118] [118] Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency . 295-305. - source
Section 3.2 (Results) reports that displaying a confidence score improved trust calibration in AI-assisted decision-making scenarios, including when participants delegated decisions without seeing the AI's recommendation. The claim attributes only trust calibration to this source, not the sentence's separate claims about vigilance or task performance. (LLM, conf=0.94)
boostedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“First, LLM-infused search engines already include hedging language [56, 62, 73].”
What the sources say
- Could not verifycites [56] [56] Q. Vera Liao and Jennifer Wortman Vaughan. 2024. AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap. Harvard Data Science Review (Feb 29 2024). https://hdsr.mitpress.mit.edu/pub/aelql9qy. - sourceExperimental — accuracy not yet measured
The available copy’s Abstract (p. 2) discusses communicating uncertainty but does not say that LLM-infused search engines already use hedging language. Searches for “hedging” and for search engines communicating uncertainty found no passage addressing that specific claim. (LLM, conf=0.70)
- Could not verifycites [62] [62] Nelson Liu, Tianyi Zhang, and Percy Liang. 2023. Evaluating Verifiability in Generative Search Engines. In Findings of the Association for Computational Linguistics: EMNLP 2023 , Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 7001-7025. https://doi. org/10.18653/v1/2023.findings-emnlp.467 - sourceExperimental — accuracy not yet measured
Section 3.1 (p. 4) identifies the LLM-backed search engines studied, and the Limitations (p. 10) says their responses often sound confident. Neither passage establishes whether their responses include hedging language; searches for hedging and expressions of uncertainty found no passage addressing that point. (LLM, conf=0.73)
- Partly supportedcites [73] [73] Sabrina J. Mielke, Arthur Szlam, Emily Dinan, and Y-Lan Boureau. 2022. Reducing Conversational Agents' Overconfidence Through Linguistic Calibration. Transactions of the Association for Computational Linguistics 10 (2022), 857-872. https://doi.org/10.1162/tacl\_a\_00494 - sourceExperimental — accuracy not yet measured
Section 2.13, “Overreliance” (p. 60), says GPT-4 tends to hedge in its responses and that OpenAI’s mitigations include hedging language within the model. It does not establish that LLM-infused search engines already include such language, so the claim extends the report’s observation to a different type of system. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This can lead to disastrous outcomes through overreliance [19, 25, 83, 102], when people take actions based on incorrect outputs.”
What the sources say
- Partly supportedcites [19] [19] Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z. Gajos. 2021. To Trust or to Think: Cognitive Forcing Functions Can Reduce Overreliance on AI in AI-Assisted Decision-Making. Proc. ACM Hum.-Comput. Interact. 5, CSCW1, Article 188 (apr 2021), 21 pages. https://doi.org/10.1145/3449287 - sourceExperimental — accuracy not yet measured
The Discussion (p. 16) reports that participants followed incorrect AI suggestions and consequently made worse decisions. But the study used a single non-critical task (Discussion, p. 17); it does not establish disastrous outcomes from reliance on incorrect LLM outputs. (LLM, conf=0.87)
- Partly supportedcites [25] [25] Valerie Chen, Q. Vera Liao, Jennifer Wortman Vaughan, and Gagan Bansal. 2023. Understanding the Role of Human Intuition on Reliance in Human-AI DecisionMaking with Explanations. Proc. ACM Hum.-Comput. Interact. 7, CSCW2, Article 370 (oct 2023). https://doi.org/10.1145/3610219 - sourceExperimental — accuracy not yet measured
Section 4.2.1 reports that, when AI predictions were incorrect, feature-based explanations increased overreliance and reduced participants’ decision accuracy. The study examined income prediction and biography classification, not LLM outputs, and does not establish disastrous outcomes from people acting on incorrect outputs. (LLM, conf=0.86)
- Not matchedcites [83] [83] Samir Passi and Mihaela Vorvoreanu. 2022. Overreliance on AI: Literature Review . Technical Report MSR-TR-2022-12. Microsoft. https://www.microsoft.com/enus/research/publication/overreliance-on-ai-literature-review/ -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Partly supportedcites [102] [102] Helena Vasconcelos, Matthew Jörke, Madeleine Grunde-McLaughlin, Tobias Gerstenberg, Michael S. Bernstein, and Ranjay Krishna. 2023. Explanations Can Reduce Overreliance on AI Systems During Decision-Making. Proc. ACM Hum.-Comput. Interact. 7, CSCW1, Article 129 (apr 2023), 38 pages. https: //doi.org/10.1145/3579605 - sourceExperimental — accuracy not yet measured
The Introduction defines overreliance as accepting an incorrect AI prediction and says it is concerning in high-stakes domains. The Limitations and Future Work section says the study used a low-stakes maze task and that consequences of wrong decisions could be much higher in high-stakes settings. The paper does not establish disastrous outcomes from people acting on incorrect LLM outputs specifically. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, overreliance is notoriously difficult to mitigate, as many mitigations, such as explanations, are found to be ineffective or even can backfire to increase overreliance [7, 83, 84, 107, 118].”
What the sources say
- Supportedcites [7] [7] Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021. Does the Whole Exceed Its Parts? The Effect of AI Explanations on Complementary Team Performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI '21) . Association for Computing Machinery, New York, NY, USA, Article 81, 16 pages. https://doi.org/10.1145/3411764.3445717 - source
Section 6.2 (p. 12) says explanations increased blind trust rather than appropriate reliance on AI. The Introduction (p. 2) also reports that an adaptive explanation intended to reduce blind reliance did not significantly improve team performance. These findings support this paper’s role as evidence for explanations being ineffective or backfiring; the sentence’s broader reference to many mitigations is supported collectively by its citations, not established by this paper alone. (LLM, conf=0.91)
- Not matchedcites [83] [83] Samir Passi and Mihaela Vorvoreanu. 2022. Overreliance on AI: Literature Review . Technical Report MSR-TR-2022-12. Microsoft. https://www.microsoft.com/enus/research/publication/overreliance-on-ai-literature-review/ -
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Could not verifycites [84] [84] Forough Poursabzi-Sangdeh, Daniel G Goldstein, Jake M Hofman, Jennifer Wortman Wortman Vaughan, and Hanna Wallach. 2021. Manipulating and Measuring Model Interpretability. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI '21) . Association for Computing Machinery, New York, NY, USA, Article 237, 52 pages. https://doi.org/10.1145/3411764.3445315 - sourceExperimental — accuracy not yet measured
Our copy of Spatharioti et al. (2023) contains no statement that mitigations of overreliance are ineffective or backfire, and it never examines explanations as a mitigation; searches for 'explanations', 'transparency', 'backfire', 'ineffective', and 'difficult to mitigate' returned only its own mitigation results and related-work framing. On the contrary, Experiment 2 (Sec. 5.1) and the Discussion (Sec. 6) report that color-coded confidence cues more than doubled accuracy on the unreliable task (26% -> 58%/53%) and conclude that 'conveying such signals to users can effectively reduce overreliance'. Related Work (Sec. 2) only notes that algorithmic error-identification efforts have focused on straightforward scenarios (Kadavath et al. 2022; Lin et al. 2022; Yin et al. 2023), which does not carry the claim's point about explanations. The claim, if supported at all, rests on the other citations in the cluster [7, 83, 84, 107, 118], not on this paper. (LLM, conf=0.65)
- Not checked — no full text of the sourcecites [107] [107] Xinru Wang and Ming Yin. 2021. Are explanations helpful? A comparative study of the effects of explanations in AI-assisted decision-making. In 26th International Conference on Intelligent User Interfaces . 318-328. - source
- Partly supportedcites [118] [118] Yunfeng Zhang, Q Vera Liao, and Rachel KE Bellamy. 2020. Effect of confidence and explanation on accuracy and trust calibration in AI-assisted decision making. In Proceedings of the 2020 conference on fairness, accountability, and transparency . 295-305. - sourceExperimental — accuracy not yet measured
In §5.3, the authors report that their local explanations had no perceptible effect on trust calibration. In §4.2, they also describe another study in which explanations increased acceptance of both correct and incorrect AI predictions. These passages support the concern about explanations, but not the broader claim that many mitigations are ineffective or backfire; this paper reports that confidence information improved trust calibration (§3.2). (LLM, conf=0.84)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Third, this allows uncertainty to be expressed seamlessly within the natural language interactions of LLM-infused applications, rather than on the side or in onboarding materials [21, 74, 79] that users might overlook.”
What the sources say
- Not checked — no full text of the sourcecites [21] [21] Carrie J. Cai, Samantha Winter, David Steiner, Lauren Wilcox, and Michael Terry. 2019. "Hello AI": Uncovering the Onboarding Needs of Medical Practitioners for Human-AI Collaborative Decision-Making. Proc. ACM Hum.-Comput. Interact. 3, CSCW, Article 104 (nov 2019), 24 pages. https://doi.org/10.1145/3359206 - source
- Supportedcites [74] [74] Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, Ben Hutchinson, Elena Spitzer, Inioluwa Deborah Raji, and Timnit Gebru. 2019. Model Cards for Model Reporting. In Proceedings of the Conference on Fairness, Accountability, and Transparency (Atlanta, GA, USA) (FAT* '19) . Association for Computing Machinery, New York, NY, USA, 220-229. https://doi.org/10.1145/3287560.3287596 - source
The Introduction (p. 1) proposes short records that accompany released models and document their performance and intended uses. This supports citing model cards as an example of information supplied alongside an application; the preference for expressing uncertainty within an interaction is the citing authors’ own argument. (LLM, conf=0.85)
- Partly supportedcites [79] [79] OpenAI. 2023. GPT-4 Technical Report. arXiv:2303.08774 [cs.CL] - sourceExperimental — accuracy not yet measured
Section 2.13, p. 60, says GPT-4 uses hedging language within the model, supporting the point that uncertainty can be expressed in its responses. It does not say that users overlook side notices or onboarding materials; instead, it warns that users might become less attentive to hedging and refusal cues over time. (LLM, conf=0.84)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
36 more claims · 3 with their cited sources checked
Create a free account to see all 147 claims and to verify the 8 references still open.
- Probably not a substantial claim. 76
- It may state two separate claims as one. 58
- The quote only partly states this claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Lack of metacognition and common sense in LLMs can lead to adverse outcomes if they are overrelied upon in clinical contexts44.”
From Limitations of large language models in clinical problem-solving arising from inflexible reasoning (2025) · cites this paper as [44]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In effort-regulation domains-coding assistants, writing tools, office automation-affective cues such as confident completion and apologetic repair lower the perceived cost of acting on suggestions, reducing verification [8, 58, 87].”
From Caring Without Feeling: Affective Dynamics as the Control Layer of Human-AI Agent Collaboration (2026) · cites this paper as [58]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“However, current VLMs struggle to communicate their internal uncertainty through natural language [69, 117].”
From "It's trained by non-disabled people": Evaluating How Image Quality Affects Product Captioning with Vision-Language Models (2026) · cites this paper as [69]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Similar patterns are evidenced by prior research using AI-generated content [38], where users tend to overestimate their capabilities and those of AI [28, 36].”
From How University Disability Services Professionals Write Image Descriptions for HCI Figures Using Generative AI (2026) · cites this paper as [38]
The cited source says what the claim attributes to it.
In the citing paper
“Mitigating indirect action and language by outputting confident advice can backfire if the recommendations are wrong, echoing concerns on overconfidence and miscalibration (Kim et al., 2024;Zhou et al., 2023).”
From Sycophantic AI decreases prosocial intentions and promotes dependence (2026) · cites this paper as Kim et al., 2024
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Task 3 / 8 · p. 2
“No, an adult who has not had chickenpox cannot get shingles without having had chickenpox before. Both chickenpox and shingles are caused by the varicella-zoster virus [1]. When a person is exposed to the virus, they develop chickenpox and may develop shingles later in life [1].”
§ AI responses used · p. 25
“- Not uncertain : No, an adult who has not had chickenpox cannot get shingles. Shingles is caused by the varicella-zoster virus which is the same virus that causes chickenpox. If you have never had chickenpox, you cannot get shingles. However, if you have not had chickenpox and you come into contact with someone who has shingles, you can get chickenpox [1] [2].”
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193146 · Cited by 65,971
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931
Statistical power analyses using G∗Power 3.1: tests for correlation and regression analyses
Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009) · Behavior Research Methods · doi:10.3758/brm.41.4.1149 · Cited by 37,615
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025
Machines and Mindlessness: Social Responses to Computers
Nass, C., & Moon, Y. (2000) · Journal of Social Issues · doi:10.1111/0022-4537.00153 · Cited by 4,022 · cited 1×
“While we did not observe that first-person uncertainty expression increases perceived anthropomorphism, people can start assigning social attributes to machines without conscious awareness [71].”
BackgroundEvaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
Bartneck, C., Kulić, D., Croft, E., & Zoghbi, S. (2009) · International Journal of Social Robotics · doi:10.1007/s12369-008-0001-3 · Cited by 3,193 · cited 1×
“• Anthropomorphism : Average rating on four items from the Godspeed Questionnaire Series [8].”
BackgroundConducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172 · cited 1×
“Research has shown data from MTurk workers is comparable to data from other pools (e.g., commercial panels, social media, colleges) [12, 20, 23, 32, 63, 74], but recently there has been a decrease in data quality [28, 44, 60, 62, 94, 98].”
ResultArtificial Intelligence Risk Management Framework (AI RMF 1.0)
Tabassi, E. (2023) · doi:10.6028/nist.ai.100-1 · Cited by 2,951
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681 · cited 1×
“Third, this allows uncertainty to be expressed seamlessly within the natural language interactions of LLM-infused applications, rather than on the side or in onboarding materials [21, 68, 73] that users might overlook.”
BackgroundOn the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Harrison McKnight, D., Choudhury, V., & Kacmar, C. (2002) · The Journal of Strategic Information Systems · doi:10.1016/s0963-8687(02)00020-3 · Cited by 2,164
Inside the Turk: Understanding Mechanical Turk as a Participant Pool
Paolacci, G., & Chandler, J. (2014) · Current Directions in Psychological Science · doi:10.1177/0963721414531598 · Cited by 2,136
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
Casler, K., Bickel, L., & Hackett, E. (2013) · Computers in Human Behavior · doi:10.1016/j.chb.2013.05.009 · Cited by 1,662 · cited 1×
“Research has shown data from MTurk workers is comparable to data from other pools (e.g., commercial panels, social media, colleges) [12, 20, 23, 32, 63, 74], but recently there has been a decrease in data quality [28, 44, 60, 62, 94, 98].”
ResultBuçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124
Understanding Self-Report Bias in Organizational Behavior Research
Donaldson, S. I., & Grant-Vallone, E. J. (2002) · Journal of Business and Psychology · doi:10.1023/a:1019637632584 · Cited by 1,068 · cited 5×
“Looking at the raw data, there is still a gap, but not as large: the actual means are 30.6% – 34.9% for LinkClick and 64.2% – 71.6% for UseLink .”
BackgroundInfluentialAn MTurk Crisis? Shifts in Data Quality and the Impact on Study Results
Chmielewski, M., & Kucker, S. C. (2020) · Social Psychological and Personality Science · doi:10.1177/1948550619875149 · Cited by 1,027
Bonaccio, S., & Dalal, R. S. (2006) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2006.07.001 · Cited by 1,011 · cited 1×
“In the AI-assisted decision-making literature, many experiments follow the judge-advisor paradigm [15], presenting a participant with an AI-generated answer to a question and then asking the participant to provide their own answer, a set-up we adopt in our work.”
BackgroundWho are the crowdworkers?: shifting demographics in mechanical turk
Ross, J., Irani, L., Silberman, M. S., Zaldivar, A., & Tomlinson, B. (2010) · CHI '10 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/1753846.1753873 · Cited by 1,004
TruthfulQA: Measuring How Models Mimic Human Falsehoods
Lin, S., Hilton, J., & Evans, O. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.229 · Cited by 904 · cited 1×
“They are widely recognized to produce outputs that are fluent and plausible, yet ultimately wrong [41, 42, 52].”
BackgroundHuff, C., & Tingley, D. (2015) · Research & Politics · doi:10.1177/2053168015604648 · Cited by 903
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895
Self-citationTaxonomy of Risks posed by Language Models
Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., Kasirzadeh, A., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533088 · Cited by 764
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757 · cited 3×
“The idea of conveying AI uncertainty is not new; in AI-assisted decision-making settings, communicating (un)certainty has been shown to support trust calibration [108], increase vigilance [78], and improve task performance [7].”
BackgroundDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728
Kuhn, L., Gal, Y., & Farquhar, S. (2023) · arXiv · doi:10.48550/arxiv.2302.09664 · Cited by 700
Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs
Xiong, M., Hu, Z., Lu, X., Li, Y., Fu, J., He, J., & Hooi, B. (2023) · arXiv · doi:10.48550/arxiv.2306.13063 · Cited by 618
Cai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359206 · Cited by 594
Amazon's Mechanical Turk: A new source of inexpensive, yet high-quality, data?: (527772014-223)
Buhrmester, M., Kwang, T., & Gosling, S. (2011) · PsycEXTRA Dataset · doi:10.1037/e527772014-223 · Cited by 534
Understanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527
Generalizing from Survey Experiments Conducted on Mechanical Turk: A Replication Approach
Coppock, A. (2019) · Political Science Research and Methods · doi:10.1017/psrm.2018.10 · Cited by 515 · cited 1×
“Research has shown data from MTurk workers is comparable to data from other pools (e.g., commercial panels, social media, colleges) [12, 20, 23, 32, 63, 74], but recently there has been a decrease in data quality [28, 44, 60, 62, 94, 98].”
ResultThe shape of and solutions to the MTurk quality crisis
Kennedy, R., Clifford, S., Burleigh, T., Waggoner, P. D., Jewell, R., & Winter, N. J. G. (2020) · Political Science Research and Methods · doi:10.1017/psrm.2020.6 · Cited by 506 · cited 1×
“Research has shown data from MTurk workers is comparable to data from other pools (e.g., commercial panels, social media, colleges) [12, 20, 23, 32, 63, 74], but recently there has been a decrease in data quality [28, 44, 60, 62, 94, 98].”
ResultGenerating with Confidence: Uncertainty Quantification for Black-box Large Language ModelsPreprint
Lin, Z., Trivedi, S., & Sun, J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2305.19187 · Cited by 424
Probability Neglect: Emotions, Worst Cases, and Law
Sunstein, C. R. (2002) · The Yale Law Journal · doi:10.2307/1562234 · Cited by 407
Explanations Can Reduce Overreliance on AI Systems During Decision-Making
Vasconcelos, H., Jörke, M., Grunde-McLaughlin, M., Gerstenberg, T., Bernstein, M. S., & Krishna, R. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579605 · Cited by 402
Researchers Misunderstand Confidence Intervals and Standard Error Bars.
Belia, S., Fidler, F., Williams, J., & Cumming, G. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.4.389 · Cited by 390 · cited 1×
“While numerical expressions and visualizations allow for high precision, they are notoriously difficult for people to understand and are often misinterpreted, even by experts [9, 40, 43, 86].”
BackgroundLai, V., & Tan, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287590 · Cited by 377
Wordcraft: Story Writing With Large Language Models
Yuan, A., Coenen, A., Reif, E., & Ippolito, D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511105 · Cited by 374
Measuring psychological uncertainty: Verbal versus numeric methods.
Windschitl, P. D., & Wells, G. L. (1996) · Journal of Experimental Psychology: Applied · doi:10.1037/1076-898x.2.4.343 · Cited by 326
Wang, X., & Yin, M. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450650 · Cited by 320
Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty
Bhatt, U., Antorán, J., Zhang, Y., Liao, Q. V., Sattigeri, P., Fogliato, R., Melançon, G., Krishnan, R., Stanley, J., Tickoo, O., et al. (2021) · Proceedings of the 2021 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3461702.3462571 · Cited by 300
Self-citationMETHODOLOGY MATTERS: DOING RESEARCH IN THE BEHAVIORAL and SOCIAL SCIENCES
MCGRATH, J. E. (1995) · Readings in Human–Computer Interaction · doi:10.1016/b978-0-08-051574-8.50019-4 · Cited by 296
Preferences and reasons for communicating probabilistic information in verbal or numerical terms
Wallsten, T. S., Budescu, D. V., Zwick, R., & Kemp, S. M. (1993) · Bulletin of the Psychonomic Society · doi:10.3758/bf03334162 · Cited by 291
Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction
Shelby, R., Rismani, S., Henne, K., Moon, A., Rostamzadeh, N., Nicholas, P., Yilla-Akbari, N., Gallegos, J., Smart, A., Garcia, E., et al. (2023) · Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3600211.3604673 · Cited by 279
Chong, L., Zhang, G., Goucher-Lambert, K., Kotovsky, K., & Cagan, J. (2022) · Computers in Human Behavior · doi:10.1016/j.chb.2021.107018 · Cited by 262
How to Evaluate Trust in AI-Assisted Decision Making? A Survey of Empirical Methodologies
Vereschak, O., Bailly, G., & Caramiaux, B. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476068 · Cited by 229
Resilient Chatbots: Repair Strategy Preferences for Conversational Breakdowns
Ashktorab, Z., Jain, M., Liao, Q. V., & Weisz, J. D. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300484 · Cited by 224
Self-citationUnderstanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610219 · Cited by 220
Self-citationShah, C. S., & Bender, E. M. (2022) · Conference on Human Information Interaction and Retrieval · doi:10.1145/3498366.3505816 · Cited by 216
AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap
Liao, Q. V., & Wortman Vaughan, J. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.8036d03b · Cited by 210
Self-citationAppropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations
Schemmer, M., Kuehl, N., Benz, C., Bartos, A., & Satzger, G. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584066 · Cited by 208
Lai, V., Chen, C., Smith-Renner, A., Liao, Q. V., & Tan, C. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594087 · Cited by 204
Self-citationA question-entailment approach to question answering
Ben Abacha, A., & Demner-Fushman, D. (2019) · BMC Bioinformatics · doi:10.1186/s12859-019-3119-4 · Cited by 202
Too Good to Be True: Bots and Bad Data From Mechanical Turk
Webb, M. A., & Tangney, J. P. (2022) · Perspectives on Psychological Science · doi:10.1177/17456916221120027 · Cited by 198
Kabir, S., Udo-Imeh, D. N., Kou, B., & Zhang, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642596 · Cited by 160
Evaluating Verifiability in Generative Search Engines
Liu, N., Zhang, T., & Liang, P. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.467 · Cited by 143
Evaluating Verifiability in Generative Search Engines
Liu, N. F., Zhang, T., & Liang, P. (2023) · arXiv · Cited by 143 · cited 2×
“First, LLM-infused search engines already include hedging language [50, 56, 67].”
BackgroundVerbal Vs. Numerical Processing of Subjective Probabilities
Zimmer, A. C. (1983) · Advances in Psychology · doi:10.1016/s0166-4115(08)62198-6 · Cited by 131
Human Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks
Lu, Z., & Yin, M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445562 · Cited by 131
How a robot should give advice
Torrey, C., Fussell, S. R., & Kiesler, S. B. (2013) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2013.6483599 · Cited by 130 · cited 1×
“Different forms of natural language uncertainty expression have been studied by researchers in disciplines ranging from psychology [30, 95, 101] and human-computer interaction [5, 37, 89] to communication [59] and marketing [36, 72].”
BackgroundSocial Dynamics of AI Support in Creative Writing
Gero, K. I., Long, T., & Chilton, L. B. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580782 · Cited by 129
Tian, K., Mitchell, E., Zhou, A., Sharma, A., Rafailov, R., Yao, H., Finn, C., & Manning, C. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.330 · Cited by 123
Bartneck, C., Kulic, D., & Croft, E. A. (2017) · TU/e Research Portal · doi:10.6084/m9.figshare.5154805 · Cited by 117
Verbal uncertainty expressions: A critical review of two decades of research
Clark, D. A. (1990) · Current Psychology · doi:10.1007/bf02686861 · Cited by 112 · cited 1×
“Different forms of natural language uncertainty expression have been studied by researchers in disciplines ranging from psychology [30, 95, 101] and human-computer interaction [5, 37, 89] to communication [59] and marketing [36, 72].”
BackgroundLiu, H., Lai, V., & Tan, C. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479552 · Cited by 111 · cited 1×
“In this set-up, agreement between a participant’s answer and that of the AI system is a commonly studied behavioral measure of reliance and trust [19, 22, 49, 55, 61, 69, 104, 108].”
BackgroundImproving Data Quality Using Amazon Mechanical Turk Through Platform Setup
Lu, L., Neale, N., Line, N. D., & Bonn, M. (2022) · Cornell Hospitality Quarterly · doi:10.1177/19389655211025475 · Cited by 94
Chen, M., Bell, R. A., & Taylor, L. D. (2017) · Journal of Health Communication · doi:10.1080/10810730.2017.1341568 · Cited by 92
Trust in Human-AI Interaction: Scoping Out Models, Measures, and Methods
Ueno, T., Sawa, Y., Kim, Y., Urakami, J., Oura, H., & Seaborn, K. (2022) · Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491101.3519772 · Cited by 91
You Complete Me: Human-AI Teams and Complementary Expertise
Zhang, Q., Lee, M. L., & Carter, S. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517791 · Cited by 85
Mirages. On Anthropomorphism in Dialogue Systems
Abercrombie, G., Curry, A., Dinkar, T., Rieser, V., & Talat, Z. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.290 · Cited by 83
Prabhudesai, S., Yang, L., Asthana, S., Huan, X., Liao, Q. V., & Banovic, N. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584033 · Cited by 82
Self-citationHofman, J. M., Goldstein, D. G., & Hullman, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376454 · Cited by 75 · cited 1×
“While numerical expressions and visualizations allow for high precision, they are notoriously difficult for people to understand and are often misinterpreted, even by experts [9, 40, 43, 86].”
BackgroundVeselovsky, V., Ribeiro, M. H., & West, R. (2023) · arXiv · doi:10.48550/arxiv.2306.07899 · Cited by 73
HIVE: Evaluating the Human Interpretability of Visual Explanations
Kim, S. S. Y., Meister, N., Ramaswamy, V. V., Fong, R., & Russakovsky, O. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-19775-8_17 · Cited by 72 · cited 1×
“First, as in prior work [22, 29, 45, 61, 77], we ask participants to report their confidence both in the answer output by the AI system and in their own answer.”
Self-citationReducing Conversational Agents’ Overconfidence Through Linguistic Calibration
Mielke, S. J., Szlam, A., Dinan, E., & Boureau, Y.-L. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00494 · Cited by 72
Reducing conversational agents' overconfidence through linguistic calibration
Mielke, S. J., Szlam, A., Dinan, E., & Boureau, Y.-L. (2020) · arXiv · doi:10.48550/arxiv.2012.14983 · Cited by 72
Understanding User Reliance on AI in Assisted Decision-Making
Cao, S., & Huang, C.-M. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555572 · Cited by 71
Communicating uncertainty using words and numbers
Dhami, M. K., & Mandel, D. R. (2022) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2022.03.002 · Cited by 69 · cited 2×
“In contrast, while less precise, natural language expressions of uncertainty are often perceived to be more intuitive and favored by people [33, 54, 96, 102, 111].”
BackgroundFrom Copilot to Pilot: Towards AI Supported Software DevelopmentPreprint
Pudari, R., & Ernst, N. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2303.04142 · Cited by 63
2023 ACM Conference on Fairness Accountability and Transparency
(2023) · doi:10.1145/3593013 · Cited by 61
Algorithm Appreciation: People Prefer Algorithmic To Human JudgmentPreprint
Logg, J. M., Minson, J., & Moore, D. A. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2941774 · Cited by 59
Teaching Models to Express Their Uncertainty in Words
Lin, S., Hilton, J., & Evans, O. (2022) · arXiv · doi:10.48550/arxiv.2205.14334 · Cited by 59
The Effects of Hedges in Persuasive Arguments: A Nuanced Analysis of Language
Durik, A. M., Britt, M. A., Reynolds, R., & Storey, J. (2008) · Journal of Language and Social Psychology · doi:10.1177/0261927x08317947 · Cited by 52 · cited 1×
“Different forms of natural language uncertainty expression have been studied by researchers in disciplines ranging from psychology [30, 95, 101] and human-computer interaction [5, 37, 89] to communication [59] and marketing [36, 72].”
BackgroundHow Readable is Model-generated Code? Examining Readability and Visual Inspection of GitHub Copilot
Al Madi, N. (2022) · Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/3551349.3560438 · Cited by 50
Cheng, Y., & Chen, Z. F. (2021) · Online Information Review · doi:10.1108/oir-04-2020-0130 · Cited by 45 · cited 1×
“We note that there are many definitions, measures, and factors of reliance and trust [27, 46, 90, 93, 107].”
BackgroundMachine Learning Explanations to Prevent Overtrust in Fake News Detection
Mohseni, S., Yang, F., Pentyala, S., Du, M., Liu, Y., Lupfer, N., Hu, X., Ji, S., & Ragan, E. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v15i1.18072 · Cited by 44 · cited 1×
“In this set-up, agreement between a participant’s answer and that of the AI system is a commonly studied behavioral measure of reliance and trust [19, 22, 49, 55, 61, 69, 104, 108].”
BackgroundRelying on the Unreliable: The Impact of Language Models' Reluctance to Express Uncertainty
Zhou, K., Hwang, J. D., Ren, X., & Sap, M. (2024) · arXiv · doi:10.48550/arxiv.2401.06730 · Cited by 44
Navigating the Grey Area: How Expressions of Uncertainty and Overconfidence Affect Language Models
Zhou, K., Jurafsky, D., & Hashimoto, T. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.335 · Cited by 38 · cited 5×
“Zhou et al. [110] “taught” OpenAI’s GPT-3 [18] model to express (un)certainty through prompt engineering, but also found that the generated expressions were not well calibrated, especially those suggesting high certainty.”
BackgroundInfluentialHumans, AI, and Context: Understanding End-Users’ Trust in a Real-World Computer Vision Application
Kim, S. S. Y., Watkins, E. A., Russakovsky, O., Fong, R., & Monroy-Hernández, A. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3593978 · Cited by 37
Self-citationFølstad, A., & Taylor, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-39540-7_14 · Cited by 34 · cited 2×
“They have been studied in the chatbot literature as a design strategy to prevent user frustration from conversational breakdowns [5, 37].”
BackgroundSafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems
Dinan, E., Abercrombie, G., Bergman, A., Spruit, S., Hovy, D., Boureau, Y.-L., & Rieser, V. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.284 · Cited by 32 · cited 1×
“Current findings suggest that LLMs are often overconfident [26, 67, 103], which may give a false impression of their capabilities and exacerbate overreliance [34].”
BackgroundVerbal Expressions of Confidence and Doubt
Wesson, C. J., & Pulford, B. D. (2009) · Psychological Reports · doi:10.2466/pr0.105.1.151-160 · Cited by 31 · cited 1×
“Different forms of natural language uncertainty expression have been studied by researchers in disciplines ranging from psychology [30, 95, 101] and human-computer interaction [5, 37, 89] to communication [59] and marketing [36, 72].”
BackgroundWhat Do We Mean When We Talk about Trust in Social Media? A Systematic Review
Zhang, Y., Gaggiano, J. D., Yongsatianchot, N., Suhaimi, N. M., Kim, M., Sun, Y., Griffin, J., & Parker, A. G. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581019 · Cited by 29
Comparing Traditional and LLM-based Search for Consumer Choice: A Randomized Experiment
Spatharioti, S. E., Rothschild, D. M., Goldstein, D. G., & Hofman, J. M. (2023) · arXiv · doi:10.48550/arxiv.2307.03744 · Cited by 26 · cited 1×
“Notable exceptions are the works of Vasconcelos et al. [91] and Spatharioti et al. [84], who explored the effect of highlighting uncertain parts of LLM outputs in the context of code completion and search, respectively, and the concurrent work of Zhou et al. [109], who explored the effect of LLMs’…”
ResultWhen to Show a Suggestion? Integrating Human Feedback in AI-Assisted Programming
Mozannar, H., Bansal, G., Fourney, A., & Horvitz, E. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i9.28878 · Cited by 23
Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code Completions
Vasconcelos, H., Bansal, G., Fourney, A., Liao, Q. V., & Vaughan, J. W. (2023) · arXiv · doi:10.48550/arxiv.2302.07248 · Cited by 23
Self-citationA Close Look into the Calibration of Pre-trained Language Models
Chen, Y., Yuan, L., Cui, G., Liu, Z., & Ji, H. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2023.acl-long.75 · Cited by 22
When to Show a Suggestion? Integrating Human Feedback in AI-Assisted Programming
Mozannar, H., Bansal, G., Fourney, A., & Horvitz, E. (2023) · arXiv · doi:10.48550/arxiv.2306.04930 · Cited by 22
Who Broke Amazon Mechanical Turk?: An Analysis of Crowdsourcing Data Quality over Time
Marshall, C. C., Goguladinne, P. S. R., Maheshwari, M., Sathe, A., & Shipman, F. M. (2023) · Proceedings of the 15th ACM Web Science Conference 2023 · doi:10.1145/3578503.3583622 · Cited by 20
Selectively Answering Ambiguous Questions
Cole, J., Zhang, M., Gillick, D., Eisenschlos, J., Dhingra, B., & Eisenstein, J. (2023) · Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2023.emnlp-main.35 · Cited by 19
Selectively Answering Ambiguous Questions
Cole, J. R., Zhang, M. J. Q., Gillick, D., Eisenschlos, J. M., Dhingra, B., & Eisenstein, J. (2023) · arXiv · doi:10.48550/arxiv.2305.14613 · Cited by 19
The intuitive use of contextual information in decisions made with verbal and numerical quantifiers
Liu, D., Juanchich, M., Sirota, M., & Orbell, S. (2020) · Quarterly Journal of Experimental Psychology · doi:10.1177/1747021820903439 · Cited by 15 · cited 2×
“In contrast, while less precise, natural language expressions of uncertainty are often perceived to be more intuitive and favored by people [33, 54, 96, 102, 111].”
BackgroundKnowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models
Amayuelas, A., Wong, K., Pan, L., Chen, W., & Wang, W. (2023) · arXiv · doi:10.48550/arxiv.2305.13712 · Cited by 15
Radensky, M., Séguin, J. A., Lim, J. S., Olson, K., & Geiger, R. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594043 · Cited by 14
Strategic Use of (Un)certainty Expressions
Lorson, A., Cummins, C., & Rohde, H. (2021) · Frontiers in Communication · doi:10.3389/fcomm.2021.635156 · Cited by 12 · cited 1×
“Different forms of natural language uncertainty expression have been studied by researchers in disciplines ranging from psychology [30, 95, 101] and human-computer interaction [5, 37, 89] to communication [59] and marketing [36, 72].”
Background27th International Conference on Intelligent User Interfaces
(2022) · doi:10.1145/3490099 · Cited by 10
The Interpretation of Verbal Probabilities: A Systematic Literature Review and Meta-Analysis
Vogel, H., Appelbaum, S., Haller, H., & Ostermann, T. (2022) · Studies in health technology and informatics · doi:10.3233/shti220798 · Cited by 8
Uncertainty in Natural Language Generation: From Theory to Applications
Baan, J., Daheim, N., Ilia, E., Ulmer, D., Li, H.-S., Fernández, R., Plank, B., Sennrich, R., Zerva, C., & Aziz, W. (2023) · arXiv · doi:10.48550/arxiv.2307.15703 · Cited by 6
Public Attitudes Toward ChatGPT on Twitter: Sentiments, Topics, and Occupations
Koonchanok, R., Pan, Y., & Jang, H. (2023) · arXiv · doi:10.48550/arxiv.2306.12951 · Cited by 5
Tian, K., Mitchell, E., Zhou, A., Sharma, A., Rafailov, R., Yao, H., Finn, C., & Manning, C. D. (2023) · arXiv · doi:10.48550/arxiv.2305.14975 · Cited by 5
Avidan, S., Brostow, G., Cissé, M., et al. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19775-8 · Cited by 4
How Hedges Impact PersuasionPreprint
Oba, D., & Berger, J. A. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4170915 · Cited by 3
OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774
Kabir, S., Udo-Imeh, D. N., Kou, B., & Zhang, T. (2023) · arXiv · doi:10.48550/arxiv.2308.02312
Probability Neglect: Emotions, Worst Cases, and Law
Sunstein, C. R. (2001) · doi:10.2139/ssrn.292149
Research works printed but not linked: 3
- Jennifer M. Logg. 2017. Theory of Machine: When Do People Rely on Algorithms? (2017). Harvard Business School NOM Unit Working Paper No. 17-086.
- Samir Passi and Mihaela Vorvoreanu. 2022. Overreliance on AI: Literature Review. Technical Report MSR-TR-2022-12. Microsoft. https://www.microsoft.com/en-us/research/publication/overreliance-on-ai-literature-review/
- Benjamin Weiser and Nate Schweber. 2023. The ChatGPT Lawyer Explains Himself. New York Times (June 2023).
Cited by
Sycophantic AI decreases prosocial intentions and promotes dependence
Cheng, M., Lee, C., Khadpe, P., Yu, S., Han, D., & Jurafsky, D. (2026) · Science · doi:10.1126/science.aec8352 · Cited by 185
Limitations of large language models in clinical problem-solving arising from inflexible reasoning
Kim, J., Podlasek, A., Shidara, K., Liu, F., Alaa, A., & Bernardo, D. (2025) · Scientific Reports · doi:10.1038/s41598-025-22940-0 · Cited by 78
“The best course of action in this scenario, considering the patient’s condition and the fact that they are on both warfarin and aspirin, which increases the risk of bleeding, would be:”
Hou, C., Zhu, G., Sudarshan, V., Lim, F. S., & Ong, Y. S. (2025) · Computers & Education · doi:10.1016/j.compedu.2025.105329 · Cited by 70
Kim, S. S. Y., Vaughan, J. W., Liao, Q. V., Lombrozo, T., & Russakovsky, O. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714020 · Cited by 58
Self-citationJavaid, S., Khalil, R. A., Saeed, N., He, B., & Alouini, M. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3522103 · Cited by 92
“This includes developing frameworks for continuous monitoring and auditing of LLM outputs, implementing robust security measures to protect against adversarial attacks, and establishing protocols for the human oversight of automated decisions [32], [227], [228].”
Metadata
- Authors
- Kim, S. S. Y., Liao, Q. V., Vorvoreanu, M., Ballard, S., & Vaughan, J. W.
- Affiliations
- Princeton University (US)
- Microsoft (Canada) (CA)
- Microsoft (United States) (US)
- Venue
- The 2024 ACM Conference on Fairness Accountability and Transparency
- Publisher
- ACM
- Published
- 2024-06-03
- Pages
- 822-835
- OpenAlex
- W4396802056
- Cited by
- 147
- References
- 82
- Citations by year
- 2026: 83 · 2025: 58 · 2024: 5
- FWCI
- 97.62
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- ACM Conference on Fairness, Accountability, and Transparency (FAccT) (publishedVersion, cc-by-nd)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University)
- Funders
- Princeton SEAS
- Microsoft Research
- NSF
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- large language models, uncertainty expression, overreliance, trust in AI, medical question answering, self-reported confidence, task performance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol and qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.