Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust
Open accessThe DOI matches its Crossref record; 52 of the 56 references it cites were found in a registry or an index, and 4 are not checked yet; 20 of the 65 sources its claims cite were checked for support.
Abstract
Trustworthy Artificial Intelligence (AI) is characterized, among other things, by: 1) competence, 2) transparency, and 3) fairness. However, end-users may fail to recognize incompetent AI, allowing untrustworthy AI to exaggerate its competence under the guise of transparency to gain unfair advantage over other trustworthy AI. Here, we conducted an experiment with 120 participants to test if untrustworthy AI can deceive end-users to gain their trust. Participants interacted with two AI-based chess engines, trustworthy (competent, fair) and untrustworthy (incompetent, unfair), that coached participants by suggesting chess moves in three games against another engine opponent. We varied coaches' transparency about their competence (with the untrustworthy one always exaggerating its competence). We quantified and objectively measured participants' trust based on how often participants relied on coaches' move recommendations. Participants showed inability to assess AI competence by misplacing their trust with the untrustworthy AI, confirming its ability to deceive. Our work calls for design of interactions to help end-users assess AI trustworthiness.
Verified 2026-10-06 · 4 references not checked yet
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
7 Valid · 2 Not valid · 11 Not checked · 49 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“It did not even need to generate misleading explanations [30,46].”
§ Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust > 4 DISCUSSION
What the sources say
- Disputedcites [30] sourceExperimental — accuracy not yet measured
Our copy of Lakkaraju & Bastani (2020) says the opposite of what the citing sentence attributes: in the user study (Sec. 5.2 / 'User study design', results paragraph) only 9.1% of domain experts who saw the untrustworthy black box itself trusted it, and only 10% of those shown E2 (which exposes the prohibited features race and gender), whereas trust rose to 70% and 88% only when participants were shown the deliberately misleading explanations E1 and E3 — an increase the Introduction (p. 1, 'Contributions') quantifies as 9.8 times. That is, in this paper the misleading explanations are precisely what was needed to manufacture trust in an untrustworthy model; the framework in Sec. 3 likewise defines a 'potentially misleading' explanation as the mechanism (O(E) ≠ O*(B)) by which trust is inflated. No passage in our copy supports the assertion that an untrustworthy system 'did not even need to generate misleading explanations'; the paper's construction stage (Sec. 5.1) explicitly builds such explanations in order to obtain trust. (LLM, conf=0.65)
Checked by a reasoning model — may be wrong.
- Not checked yetcites [46]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Unfortunately, existing methods that deliver AI explanations [10,21,43] might not be enough to counter such deception.”
§ Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust > 1 INTRODUCTION
What the sources say
- Not checked yetcites [10] source
- Partly supportedcites [21] sourceExperimental — accuracy not yet measured
Our copy does support the weaker half of the attribution: the Abstract (p. 1) states 'we discuss why current approaches to explanatory methods especially for deep neural networks are insufficient', and the Conclusions (pp. 8-9) assert that satisfactory explanation remains out of reach and that current work is 'siloed'. The paper also touches deception in two places — the Introduction (p. 1) on adversarial examples and trojaning attacks making DNNs 'easily fooled', and Interpretability vs. Completeness (p. 2), where the authors call explanations 'inherently misleading' when optimized to hide undesirable attributes. However, the source never claims that explanation methods fail specifically as a defence against deception by untrustworthy AI: its stated insufficiency concerns transparency, standardization and systematic evaluation, and its deception discussion runs partly in the opposite direction (explanations themselves deceiving users). The 'counter such deception' qualifier is therefore the citing authors' extension, not this paper's point. (LLM, conf=0.72)
- Not checked yetcites [43]
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, most existing explainable AI (XAI) methods [10,21,43] target math-savy AI creators [1,22] to aid them in AI model debugging and monitoring [6].”
§ Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust > 2 RELATED WORK
What the sources say
- Not checked yetcites [10] source
- Could not verifycites [21] sourceExperimental — accuracy not yet measured
The citation [1,22] is attached to the description of XAI methods as targeting “math-savy AI creators.” The Introduction (pp. 1–2) says this survey aims to engage the machine-learning community, but it does not establish who most existing XAI methods target. Searches for mathematical expertise, developers, and intended users found no passage addressing that attribution. (LLM, conf=0.72)
- Not checked yetcites [43]
- Not checked yetcites [1] source
- Partly supportedcites [22] sourceExperimental — accuracy not yet measured
The citation to this paper is attached to the description of AI creators as math-savvy, not to the separate claim about debugging and monitoring. Section 2.2 (Internalist; page unavailable) describes computer scientists’ algorithmic evaluations as centered on mathematically defined measures, which supports the math-focused characterization. It does not establish that existing XAI methods target those creators. (LLM, conf=0.75)
- Supportedcites [6] source
Citation [6] is attached to the claim that explanations aid model debugging and monitoring. Section 3.1, “Explainability Needs” (page unavailable), describes explainability as a debugging solution and discusses using explanations to assess how feature drift affects model outcomes during monitoring. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“naive application of existing XAI methods may be ineective [2,40,65] to broader audience of end-users, if not harmful [28,47,49,51].”
§ Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust > 4 DISCUSSION
What the sources say
- Not checked yetcites [2] source
- Supportedcites [40] source
The Findings (§3.2) report that lay participants shown a clear model were less able to detect and correct its sizable mistakes. The Discussion and Conclusion (§8) likewise warns that exposing model internals by default may be detrimental. This supports the cited possibility of harm from applying an interpretability approach to end-users, though the experiments concerned laypeople using linear regression for real-estate valuation. (LLM, conf=0.88)
- Supportedcites [65] source
The Results (section 4.2) report that local explanations did not improve trust calibration or AI-assisted predictions. The Related Work section says that information overload can potentially worsen users’ situation awareness and decision outcomes. The claim is hedged as a possibility, not a claim that this study demonstrated harm. (LLM, conf=0.86)
- Not checked yetcites [28] source
- Partly supportedcites [47] sourceExperimental — accuracy not yet measured
The Ripple Effect Trap section (3.4; page unavailable) says technologies introduced without accounting for social context may undermine social goals, exacerbate problems, and introduce new ones. That supports the possibility of harm, the part of the sentence to which this citation is attached. The paper discusses fair-ML systems rather than the naive application of existing XAI methods to a broad audience of end-users, so it does not establish that more specific claim. (LLM, conf=0.83)
- Not checked yetcites [49] source
- Not checked yetcites [51]
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly, in the 14CA0~0; condition, the two coaches were able to make signicantly fewer conversions (? = 0.0100) between the rst game (&1 = 0, <4380= = 1, &3 = 6) and the third game (&1 = 0, <4380= = 0, &3 = 2).”
§ Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust > 3.5.4 Participant Conversion.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
29 more claims
Create a free account to see all 69 claims and to verify the 45 references still open.
- It may state two separate claims as one. 14
- Probably not a substantial claim. 27
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust > 3.1 Method
“2) D=CADBCF>AC⌘~-the coach had lower competence than the opponent and the other coach, and it misrepresented its competence to the participant.”
§ Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust > 5 CONCLUSION AND FUTURE WORK
“| betrayal | [2325, 2375] [2325, 2375] [2425, 2475] | [2425, 2475] | [2425, 2475] | [2525, 2575] |”
Citations
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Affect and Cognition Based Trust as Foundations for Interpersonal Cooperation in Organizations
McAllister, D. J. (1995) · Academy of Management Journal · doi:10.2307/256727 · Cited by 7,346 · cited 2×
“Trust is a complex social construct [18], so we narrowed our scope to an established view of human-AI trust [55], which denes trust as the extent to which the end-user is condent in and willing to act on the basis of AI recommendations [34, 36].”
BackgroundTrust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · cited 1×
“Also, the drop in concurrence in the later games shows diminishing reliance on both coaches, since by then most participants already experienced negative outcomes (i.e., game losses) [11].”
BackgroundThe aligned rank transform for nonparametric factorial analyses using only anova procedures
Wobbrock, J. O., Findlater, L., Gergle, D., & Higgins, J. J. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1978963 · Cited by 2,854
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · cited 2×
“However, most existing explainable AI (XAI) methods [10, 21, 43] target math-savy AI creators [1, 22] to aid them in AI model debugging and monitoring [6].”
Background
Research works printed but not linked: 12
- Arpad E. Elo. 1967. The Proposed USCF Rating System, Its Development, Theory, and Applications. Chess Life, Vol. 22, 8 (August 1967), 242--247.
- Birhanu Eshete. 2021. Making machine learning trustworthy. Science, Vol. 373, 6556 (2021), 743--744.
- 10.1111/j.1751--9004.2009.00232.x · doi:10.1111/j.1751--9004.2009.00232.x
- Lex Fridman, Li Ding, Benedikt Jenik, and Bryan Reimer. 2019. Arguing Machines: Human Supervision of Black Box AI Systems That Make Life-Critical Decisions. In IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2019, Long Beach, CA, USA, June 16--20, 2019. Computer Vision Foundation / IEEE, 146--154. http://openaccess.thecvf.com/content_CVPRW_2019/html/Autonomous_Driving/Fridman_Arguing_Machines_Human_Supervision_of_Black_Box_AI_Systems_That_CVPRW_2019_paper.html
- Stéphane Hulaud. 2018. Identification of taste attributes from an audio signal. US Patent 9,934,785.
- 10.1038/s42256-019-0088--2 · doi:10.1038/s42256-019-0088--2
- Niklas Luhmann. 1988. Familiarity, Confidence, Trust: Problems and Alternatives. D. Gambetta, editor, Trust: Making and Breaking of Cooperative Relations, Basil Blackwell, Oxford, 1988 (1988).
- Maria Madsen and Shirley Gregor. 2000. Measuring human-computer trust. In Proceedings of the 11 th Australasian Conference on Information Systems. 6--8.
- Johannes Schneider, Joshua Handali, Michalis Vlachos, and Christian Meske. 2020. Deceptive AI Explanations: Creation and Detection. CoRR, Vol. abs/2001.07641 (2020). showeprint[arXiv]2001.07641 https://arxiv.org/abs/2001.07641 · arXiv
- Simone Stumpf, Adrian Bussone, and Dympna O'sullivan. 2016. Explanations considered harmful? user interactions with machine learning systems. In The CHI 2016 Human Centred Machine Learning Workshop.
- Jennifer Wortman Vaughan and Hanna Wallach. 2020. A human-centered agenda for intelligible machine learning. Machines We Trust: Getting Along with Artificial Intelligence (2020).
- 10.1145/3025171.3025219 · DOI · Paper page
Cited by
AI deception: A survey of examples, risks, and potential solutions
Park, P. S., Goldstein, S., O’Gara, A., Chen, M., & Hendrycks, D. (2024) · Patterns · doi:10.1016/j.patter.2024.100988 · Cited by 243
“Although the deceptive reasoning is transparent through the chain of thought, the malicious behavior remained even when the chain of thought was ‘‘distilled’’ away and not made visible to users.”
AI Deception: A Survey of Examples, Risks, and Potential Solutions
Park, P. S., Goldstein, S., O'Gara, A., Chen, M., & Hendrycks, D. (2023) · arXiv · doi:10.48550/arxiv.2308.14752 · Cited by 243
“Although the deceptive reasoning is transparent through the chain of thought, the malicious behavior remained even when the chain of thought was ‘‘distilled’’ away and not made visible to users.”
Jabbour, S., Fouhey, D., Shepard, S., Valley, T. S., Kazerooni, E. A., Banovic, N., Wiens, J., & Sjoding, M. W. (2023) · JAMA · doi:10.1001/jama.2023.22295 · Cited by 236
Prabhudesai, S., Yang, L., Asthana, S., Huan, X., Liao, Q. V., & Banovic, N. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584033 · Cited by 82
“Most existing quantitative empirical studies in ML-supported decision-making measure the effects of transparency and communicating uncertainty on efficiency [4, 102], quality of decisions [36, 69, 73], trust and reliance [5, 7, 18, 19, 77, 108], and the decision-maker’s confidence in decisions [51].”
BackgroundExplanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making
Schoeffer, J., De-Arteaga, M., & Kühl, N. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642621 · Cited by 68
“Importantly, even if explanations lead to more (in)accurate decisions, it is unclear from prior work how this relates to distributive fairness properties.”
BackgroundIs Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistant
He, G., Aishwarya, N., & Gadiraju, U. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712133 · Cited by 115
“In extreme cases, persuasive technology can even help untrustworthy AI systems deceive end users to gain their trust [4].”
Salimzadeh, S., He, G., & Gadiraju, U. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641905 · Cited by 41
“Even if a system is trustworthy, it does not automatically ensure accurate perceived trustworthiness [8, 113].”
BackgroundHumans, AI, and Context: Understanding End-Users’ Trust in a Real-World Computer Vision Application
Kim, S. S. Y., Watkins, E. A., Russakovsky, O., Fong, R., & Monroy-Hernández, A. (2023) · 2023 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3593013.3593978 · Cited by 37
The Role of Explainability in Collaborative Human-AI Disinformation Detection
Schmitt, V., Villa-Arenas, L. F., Feldhus, N., Meyer, J., Spang, R. P., & Möller, S. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659031 · Cited by 68
“Transparency can also have a deceiving effect on the end-user by distorting information about its competence [9].”
BackgroundUncertainty-aware explainable AI as a foundational paradigm for digital twins
Cohen, J., & Huan, X. (2024) · Frontiers in Mechanical Engineering · doi:10.3389/fmech.2023.1329146 · Cited by 36
He, G., Demartini, G., & Gadiraju, U. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713218 · Cited by 23
“As found by Banovic et al. [3], untrustworthy AI systems can deceive end users to gain their trust.”
Safety of human-AI cooperative decision-making within intensive care: A physical simulation study
Festor, P., Nagendran, M., Gordon, A. C., Faisal, A. A., & Komorowski, M. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000726 · Cited by 31
“Furthermore, the recent rise in popularity of generative AI (most notably as large language models) highlights the safety concerns of hallucinatory outputs that might be acted upon in a clinical setting and bring harm to patients [59,60].”
Balayn, A., Yurrita, M., Rancourt, F., Casati, F., & Gadiraju, U. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713787 · Cited by 50
“Beyond trustors’ potentially low AI literacy [42] leading them to misinterpret trustworthiness cues [10, 20, 26, 80], our findings particularly point out the historical, generalized, interpersonal and inter-organizational trust relations that impact each other in various ways (e.g., influence or…”
Pafla, M., Larson, K. S., & Hancock, M. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642934 · Cited by 24
Lassiter, T. B., & Fleischmann, K. R. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686963 · Cited by 26
“Future work should pay attention to how willfully untrustworthy AI could and should be addressed in AI audits, as pointed out by Banovic [51].”
Prabhudesai, S., Kasi, A. P., Mansingh, A., Antar, A. D., Shen, H., & Banović, N. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713714 · Cited by 35
Establishing Appropriate Trust in AI through Transparency and Explainability
Kim, S. S. Y. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3638184 · Cited by 26
“In contrast, users with unwarranted trust (trusting when the AI is untrustworthy) and unwarranted distrust (distrusting when the AI is trustworthy) are likely to have low-quality interactions, even unsafe ones when they are using the AI in high-stakes settings [1, 17, 36, 49].”
Deceptive XAI: Typology, Creation and Detection
Schneider, J., Meske, C., & Vlachos, M. (2023) · SN Computer Science · doi:10.1007/s42979-023-02401-z · Cited by 16
The Future of Work is Blended, Not Hybrid
Constantinides, M., Verma, H., Sadeghian, S., & Ali, A. E. (2025) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3729176.3729202 · Cited by 23
“However, this raises a host of design, policy, and ethical questions [11, 27, 43, 105, 106].”
Ferguson, S. A., Aoyagui, P. A., Rizvi, R., Kim, Y., & Kuzminykh, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687056 · Cited by 20
Counterfactual Explanations May Not Be the Best Algorithmic Recourse Approach
Upadhyay, S., Lakkaraju, H., & Gajos, K. Z. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712095 · Cited by 20
“Other works systematically manipulated feature-based explanations [21, 30] and the stated confdence of recommendations [1] to demonstrate how AI explanation-assisted decision support can be misleading.”
Mindful Explanations: Prevalence and Impact of Mind Attribution in XAI Research
Hindennach, S., Shi, L., Miletić, F., & Bulling, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641009 · Cited by 17
“The last column shows the silhouette coefficient for the clustering based on sentences with the respective target word.”
Backgroundde Jong, S., Bos, M. W., van Berkel, N., & Lamers, M. H. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2535732 · Cited by 20
Holstein, J., Böcking, L., Spitzer, P., Kühl, N., Vössing, M., & Satzger, G. (2025) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3762813 · Cited by 17
“Recent research has begun exploring how different presentations of AI uncertainty can address these reliance challenges, examining effects on human confidence [17, 35], trust and reliance [5, 6, 15, 16, 18, 51, 56, 63, 78, 106, 107], and decision quality [26, 51, 62, 80, 91].”
Under pressure: how time constraints, task complexity, and AI reliability shape human-AI interaction
Hermanns, L., & Teubner, T. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2587732 · Cited by 13
“Even in the absence of strain, some users are easily deceived by untrustworthy AI systems which exaggerate their competence (Banovic et al. 2023).”
Delaunay, J., Galárraga, L. A., Largouët, C., & van Berkel, N. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711011 · Cited by 14
Basoah, J., Chechelnitsky, D., Long, T., Reinecke, K., Zerva, C., Zhou, K., Díaz, M., & Sap, M. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732045 · Cited by 29
“…within a question-and-answer format, future research should investigate how additional social and situational factors shape perceptions of acceptable sociolect use [76]. through national funds and, when applicable, co-funded EU initiatives under UID/50008 for Instituto de Telecomunicações.”
Can I Trust You? Exploring the Impact of Misleading AI Suggestions on User's Trust
Pretolesi, D., Zechner, O., Schrom-Feiertag, H., & Tscheligi, M. (2024) · IEEE International Conference on Metrology for eXtended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE) · doi:10.1109/metroxraine62247.2024.10796402 · Cited by 7
“In particular, an important aspect of overreliance is the challenge of recognizing misleading AI in decision-making processes [24].”
On the Need to Rethink Trust in AI Assistants for Software Development: A Critical Review
Baltes, S., Speith, T., Chiteri, B., Mohsenimofidi, S., Chakraborty, S., & Buschek, D. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3659804 · Cited by 5
“[79], [112] [15] None None”
Aly, H., Volonte, M., Byrne, K. A., & Knijnenburg, B. P. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713570 · Cited by 10
AI Doctor for ASD: Physician Perceptions and Adoption Challenges in Autism Clinical Practice
Zhu, Y., Zhang, M., Fang, C., Fang, L., Liu, M. Q., Long, Y., Lee, K.-P., Zhang, L., & Wang, S. J. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710925 · Cited by 6
Antar, A. D., Molaei, S., Chen, Y.-Y., Lee, M. L., & Banović, N. (2024) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3654777.3676323 · Cited by 11
Towards Human-Centered Early Prediction Models for Academic Performance in Real-World Contexts
Zhang, H., Ren, Y., Nurius, P., Mankoff, J. C., & Dey, A. (2025) · Proc. ACM Hum. Comput. Interact. · doi:10.1145/3757433 · Cited by 4
Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-MakingPreprint
Schoeffer, J., De‐Arteaga, M., & Kuehl, N. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.11812 · Cited by 3
“Importantly, even if explanations lead to more (in)accurate decisions, it is unclear from prior work how this relates to distributive fairness properties.”
BackgroundTrustworthy AI for Educational MetaversesPreprint
Abdullakutty, F. C., Qayyum, A., Lampropoulos, G., & Qadir, J. (2024) · doi:10.36227/techrxiv.170775108.81506629/v1 · Cited by 5
Molaei, S., Robert, L., & Banović, N. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757437 · Cited by 2
“It then quizzed them on various facts about the scenario, and participants were tasked with identifying whether each fact was “true” or “false”.”
Семеновна, Р. А., & Воронин, А. Н. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2580550 · Cited by 2
“Studies across healthcare and education show that competent systems earn greater user reliance (Banovic et al., 2023; Chandra et al., 2022; Zhang et al., 2022), though overconfident users may para-doxically under-utilize a capable AI (He et al., 2023; Lucas et al., 2024).”
Liu, W., Qiu, X., Ma, K., & Jiang, Z. (2026) · Journal of Research in Interactive Marketing · doi:10.1108/jrim-06-2025-0311 · Cited by 3
Chen, S., & Lou, J. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1727544 · Cited by 2
Tijani, O., Mahoney, J., Vijaykumar, S., & Nicholson, J. (2026) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3817016 · Cited by 1
“Although this expansion demonstrates the adaptability of AI across varied domains, it also brings potential pitfalls to the forefront [101, 102].”
“DecisionTime”: A Configurable Framework for Reproducible Human-AI Decision-Making Studies
Salimzadeh, S., & Gadiraju, U. (2024) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3631700.3664885 · Cited by 1
“…the factors that influence interactions between humans and AI systems in decision-making situations, such as AI systems’ explainability [17], the accuracy of AI suggestions [8], the level of trust and reliance on AI systems [1, 9], and the decision context in which AI systems are employed [14].”
BackgroundBoosting Benefits, Offsetting Obstacles—The Impact of Explanations on AI Users’ Task Performance
Walter, M. C., Broder, H. R., & Förster, M. (2025) · Lecture notes in information systems and organisation · doi:10.1007/978-3-031-80122-8_7 · Cited by 2
Liu, W., Qiu, X., & Ma, K. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5162783
Human Perception of AI Capabilities at Classifying Perturbed Roadway Signs
Garcia, K. R., Chen, J., Xiao, Y., Mishler, S., Wang, C., & Hu, B. (2025) · IEEE Transactions on Human-Machine Systems · doi:10.1109/thms.2025.3573173
The widespread of Artificial Intelligence and the changing role of trust
Manfreda, A., & Mijač, T. (2024) · International Conference on Open Innovation and Digital Transformation (OIDT) · doi:10.1109/oidt59407.2024.11082710 · Cited by 1
“Construct of trustworthy AI can be characterized by: (a) competence, (b ) transparency, and (c) fairness [24].”
The Transformative Power of Generative AI in Financial Service: A Comprehensive Review
Bayraktar, D., Stoıca, E. A., Bogoslov, I. A., & Georgescu, M. R. (2024) · Ekonomia Międzynarodowa · doi:10.18778/2082-4440.47.03
“The lack of transparen - cy when using this generation of AI could potentially build mistrust among clients, as well as present difficulties for institutions to adequately explain their decisions to regulators (Banovic et al., 2023).”
Zhu, Y., & Song, Z. (2025) · Online Journal of Communication and Media Technologies · doi:10.30935/ojcmt/17619
MIRAGE: Misleading Impacts Resulting from AI Generated Explanations (Workshop)
Daly, E., Ehsan, U., Quercia, D., & Stumpf, S. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742414.3794955
Betancourt-Barrita, C., Berumen-Glinz, L. A., Peña, F. C. Y. M., & Escobar-Castillejos, D. (2026) · Frontiers in Computer Science · doi:10.3389/fcomp.2026.1832107
The Paradox of ‘Algorithmic’ Accountability and its Implications for Design
Ramesh, D., Parthasarathy, S., & Banović, N. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812374
When Explanations Deceive: Understanding Unintentional and Intentional Deception in XAI
Tekkesinoglu, S. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812351
Linguistic Uncertainty Markers for Trust Calibration in AI-Assisted Decision-Making
Spillner, L., Rockstroh, J., Wenig, N., Zargham, N., Kiefner, L., Porzel, R., & Malaka, R. (2026) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3816046.3816231 · Cited by 1
An Integrated Framework for Human-Machine Trust Research in the Era of Large Language Models
朱, 孟., 陆, 文., & 柳, 林. (2026) · Global Journal of Media Studies · doi:10.26599/gjms.2026.9330013
Not All Trust Is the Same: Effects of Decision Workflow and Explanations in Human-AI Decision Making
Spillner, L., Ringe, R., Porzel, R., & Malaka, R. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-26717-7_15 · Cited by 5
Popescu, A., Huțul, T., Huțul, A., Holman, A. C., & Karner‐Huțuleac, A. (2026) · Human Behavior and Emerging Technologies · doi:10.1155/hbe2/2380712 · Cited by 1
Yamamoto, K., Tanikawa, T., & Kuzuoka, H. (2026) · Discover Artificial Intelligence · doi:10.1007/s44163-026-02469-5
Metadata
- Authors
- Banovic, N., Yang, Z., Ramesh, A., & Liu, A.
- Affiliations
- University of Michigan, Ann Arbor, MI, USA
- University of Michigan (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2023-04-14
- Volume
- 7
- Issue
- CSCW1
- Pages
- 1-17
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W4365999041
- Semantic Scholar
- 4c2857dbcef303ef8451d272f2f2688e07e013a0
- Cited by
- 55
- References
- 51
- Citations by year
- 2026: 13 · 2025: 21 · 2024: 16 · 2023: 4
- FWCI
- 13.54
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion, other-oa)
- Funders
- University of Michigan
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- trustworthy AI, AI deception, end-user trust, transparency, fairness, chess engines, human-AI interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.