Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust
Open accessThe DOI matches its Crossref record; 52 of the 56 references it cites were found in a registry or an index, and 4 are not checked yet; 20 of the 65 sources its claims cite were checked for support.
Abstract
Trustworthy Artificial Intelligence (AI) is characterized, among other things, by: 1) competence, 2) transparency, and 3) fairness. However, end-users may fail to recognize incompetent AI, allowing untrustworthy AI to exaggerate its competence under the guise of transparency to gain unfair advantage over other trustworthy AI. Here, we conducted an experiment with 120 participants to test if untrustworthy AI can deceive end-users to gain their trust. Participants interacted with two AI-based chess engines, trustworthy (competent, fair) and untrustworthy (incompetent, unfair), that coached participants by suggesting chess moves in three games against another engine opponent. We varied coaches' transparency about their competence (with the untrustworthy one always exaggerating its competence). We quantified and objectively measured participants' trust based on how often participants relied on coaches' move recommendations. Participants showed inability to assess AI competence by misplacing their trust with the untrustworthy AI, confirming its ability to deceive. Our work calls for design of interactions to help end-users assess AI trustworthiness.
Verified 2026-10-06 · 4 references not checked yet
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
7 Valid · 2 Not valid · 11 Not checked · 49 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“It did not even need to generate misleading explanations [30,46].”
§ Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust > 4 DISCUSSION
What the sources say
- Disputedcites [30] sourceExperimental — accuracy not yet measured
Our copy of Lakkaraju & Bastani (2020) says the opposite of what the citing sentence attributes: in the user study (Sec. 5.2 / 'User study design', results paragraph) only 9.1% of domain experts who saw the untrustworthy black box itself trusted it, and only 10% of those shown E2 (which exposes the prohibited features race and gender), whereas trust rose to 70% and 88% only when participants were shown the deliberately misleading explanations E1 and E3 — an increase the Introduction (p. 1, 'Contributions') quantifies as 9.8 times. That is, in this paper the misleading explanations are precisely what was needed to manufacture trust in an untrustworthy model; the framework in Sec. 3 likewise defines a 'potentially misleading' explanation as the mechanism (O(E) ≠ O*(B)) by which trust is inflated. No passage in our copy supports the assertion that an untrustworthy system 'did not even need to generate misleading explanations'; the paper's construction stage (Sec. 5.1) explicitly builds such explanations in order to obtain trust. (LLM, conf=0.65)
Checked by a reasoning model — may be wrong.
- Not checked yetcites [46]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Unfortunately, existing methods that deliver AI explanations [10,21,43] might not be enough to counter such deception.”
§ Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust > 1 INTRODUCTION
What the sources say
- Not checked yetcites [10] source
- Partly supportedcites [21] sourceExperimental — accuracy not yet measured
Our copy does support the weaker half of the attribution: the Abstract (p. 1) states 'we discuss why current approaches to explanatory methods especially for deep neural networks are insufficient', and the Conclusions (pp. 8-9) assert that satisfactory explanation remains out of reach and that current work is 'siloed'. The paper also touches deception in two places — the Introduction (p. 1) on adversarial examples and trojaning attacks making DNNs 'easily fooled', and Interpretability vs. Completeness (p. 2), where the authors call explanations 'inherently misleading' when optimized to hide undesirable attributes. However, the source never claims that explanation methods fail specifically as a defence against deception by untrustworthy AI: its stated insufficiency concerns transparency, standardization and systematic evaluation, and its deception discussion runs partly in the opposite direction (explanations themselves deceiving users). The 'counter such deception' qualifier is therefore the citing authors' extension, not this paper's point. (LLM, conf=0.72)
- Not checked yetcites [43]
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“However, most existing explainable AI (XAI) methods [10,21,43] target math-savy AI creators [1,22] to aid them in AI model debugging and monitoring [6].”
§ Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust > 2 RELATED WORK
What the sources say
- Not checked yetcites [10] source
- Could not verifycites [21] sourceExperimental — accuracy not yet measured
The citation [1,22] is attached to the description of XAI methods as targeting “math-savy AI creators.” The Introduction (pp. 1–2) says this survey aims to engage the machine-learning community, but it does not establish who most existing XAI methods target. Searches for mathematical expertise, developers, and intended users found no passage addressing that attribution. (LLM, conf=0.72)
- Not checked yetcites [43]
- Not checked yetcites [1] source
- Partly supportedcites [22] sourceExperimental — accuracy not yet measured
The citation to this paper is attached to the description of AI creators as math-savvy, not to the separate claim about debugging and monitoring. Section 2.2 (Internalist; page unavailable) describes computer scientists’ algorithmic evaluations as centered on mathematically defined measures, which supports the math-focused characterization. It does not establish that existing XAI methods target those creators. (LLM, conf=0.75)
- Supportedcites [6] source
Citation [6] is attached to the claim that explanations aid model debugging and monitoring. Section 3.1, “Explainability Needs” (page unavailable), describes explainability as a debugging solution and discusses using explanations to assess how feature drift affects model outcomes during monitoring. (LLM, conf=0.90)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“naive application of existing XAI methods may be ineective [2,40,65] to broader audience of end-users, if not harmful [28,47,49,51].”
§ Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust > 4 DISCUSSION
What the sources say
- Not checked yetcites [2] source
- Supportedcites [40] source
The Findings (§3.2) report that lay participants shown a clear model were less able to detect and correct its sizable mistakes. The Discussion and Conclusion (§8) likewise warns that exposing model internals by default may be detrimental. This supports the cited possibility of harm from applying an interpretability approach to end-users, though the experiments concerned laypeople using linear regression for real-estate valuation. (LLM, conf=0.88)
- Supportedcites [65] source
The Results (section 4.2) report that local explanations did not improve trust calibration or AI-assisted predictions. The Related Work section says that information overload can potentially worsen users’ situation awareness and decision outcomes. The claim is hedged as a possibility, not a claim that this study demonstrated harm. (LLM, conf=0.86)
- Not checked yetcites [28] source
- Partly supportedcites [47] sourceExperimental — accuracy not yet measured
The Ripple Effect Trap section (3.4; page unavailable) says technologies introduced without accounting for social context may undermine social goals, exacerbate problems, and introduce new ones. That supports the possibility of harm, the part of the sentence to which this citation is attached. The paper discusses fair-ML systems rather than the naive application of existing XAI methods to a broad audience of end-users, so it does not establish that more specific claim. (LLM, conf=0.83)
- Not checked yetcites [49] source
- Not checked yetcites [51]
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly, in the 14CA0~0; condition, the two coaches were able to make signicantly fewer conversions (? = 0.0100) between the rst game (&1 = 0, <4380= = 1, &3 = 6) and the third game (&1 = 0, <4380= = 0, &3 = 2).”
§ Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust > 3.5.4 Participant Conversion.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
29 more claims
Create a free account to see all 69 claims and to verify the 45 references still open.
- It may state two separate claims as one. 14
- Probably not a substantial claim. 27
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust > 3.1 Method
“2) D=CADBCF>AC⌘~-the coach had lower competence than the opponent and the other coach, and it misrepresented its competence to the participant.”
§ Being Trustworthy is Not Enough: How Untrustworthy Artificial Intelligence (AI) Can Deceive the End-Users and Gain Their Trust > 5 CONCLUSION AND FUTURE WORK
“| betrayal | [2325, 2375] [2325, 2375] [2425, 2475] | [2425, 2475] | [2425, 2475] | [2525, 2575] |”
Citations
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423
Affect and Cognition Based Trust as Foundations for Interpersonal Cooperation in Organizations
McAllister, D. J. (1995) · Academy of Management Journal · doi:10.2307/256727 · Cited by 7,346 · cited 2×
“Trust is a complex social construct [18], so we narrowed our scope to an established view of human-AI trust [55], which denes trust as the extent to which the end-user is condent in and willing to act on the basis of AI recommendations [34, 36].”
BackgroundTrust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605
The global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220
Explanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · cited 1×
“Also, the drop in concurrence in the later games shows diminishing reliance on both coaches, since by then most participants already experienced negative outcomes (i.e., game losses) [11].”
BackgroundThe aligned rank transform for nonparametric factorial analyses using only anova procedures
Wobbrock, J. O., Findlater, L., Gergle, D., & Higgins, J. J. (2011) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/1978942.1978963 · Cited by 2,854
Explaining Explanations: An Overview of Interpretability of Machine Learning
Gilpin, L. H., Bau, D., Yuan, B. Z., Bajwa, A., Specter, M., & Kagal, L. (2018) · 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2018.00018 · Cited by 2,507 · cited 2×
“However, most existing explainable AI (XAI) methods [10, 21, 43] target math-savy AI creators [1, 22] to aid them in AI model debugging and monitoring [6].”
BackgroundGuidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 1×
“Creating trustworthy AI is one of the tenets of human-centered AI [3].”
BackgroundMachine Learning Interpretability: A Survey on Methods and Metrics
Carvalho, D. V., Pereira, E. M., & Cardoso, J. S. (2019) · Electronics · doi:10.3390/electronics8080832 · Cited by 1,842 · cited 2×
“However, most existing explainable AI (XAI) methods [10, 21, 43] target math-savy AI creators [1, 22] to aid them in AI model debugging and monitoring [6].”
BackgroundHuman-Centered Artificial Intelligence: Reliable, Safe & Trustworthy
Shneiderman, B. (2020) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1741118 · Cited by 1,743 · cited 5×
“The trustworthy coach was everything that the existing XAI literature says is needed to build trust in AI.”
MethodInfluentialFairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124
Questioning the AI: Informing Design Practices for Explainable AI User Experiences
Liao, Q. V., Gruen, D., & Miller, S. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376590 · Cited by 925
Designing Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · cited 2×
“Such explanations often do not match the end-users’ mental models [56], making the explanations ineective [2, 40, 65] and at times even harmful [13, 28, 30, 47, 51].”
BackgroundManipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Vaughan, J. W., & Wallach, H. (2018) · arXiv · doi:10.48550/arxiv.1802.07810 · Cited by 711
An Aligned Rank Transform Procedure for Multifactor Contrast Tests
Elkin, L. A., Kay, M., Higgins, J. J., & Wobbrock, J. O. (2021) · The 34th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3472749.3474784 · Cited by 664
Explainable machine learning in deployment
Bhatt, U., Xiang, A., Sharma, S., Weller, A., Taly, A., Jia, Y., Ghosh, J., Puri, R., Moura, J. M. F., & Eckersley, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3375624 · Cited by 633
Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., & Wortman Vaughan, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376219 · Cited by 564 · cited 2×
“Such explanations often do not match the end-users’ mental models [56], making the explanations ine�ective [2, 40, 65] and at times even harmful [13, 28, 30, 47, 51].”
BackgroundMethodExpanding Explainability: Towards Social Transparency in AI systems
Ehsan, U., Liao, Q. V., Muller, M., Riedl, M. O., & Weisz, J. D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445188 · Cited by 538 · cited 2×
“Transparency can range from allowing insights into competence of AI-based systems [26] all the way to AI creators’ motivations and their development process [12].”
BackgroundUnderstanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527
Proxy tasks and subjective measures can be misleading in evaluating explainable AI systems
Buçinca, Z., Lin, P., Gajos, K. Z., & Glassman, E. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377498 · Cited by 307
How transparency modulates trust in artificial intelligence
Zerilli, J., Bhatt, U., & Weller, A. (2022) · Patterns · doi:10.1016/j.patter.2022.100455 · Cited by 257
Explanation Methods in Deep Learning: Users, Values, Concerns and Challenges
Ras, G., van Gerven, M., & Haselager, P. (2018) · The Springer Series on Challenges in Machine Learning · doi:10.1007/978-3-319-98131-4_2 · Cited by 255
"How do I fool you?": Manipulating User Trust via Misleading Black Box Explanations
Lakkaraju, H., & Bastani, O. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375833 · Cited by 241
How to Evaluate Trust in AI-Assisted Decision Making? A Survey of Empirical Methodologies
Vereschak, O., Bailly, G., & Caramiaux, B. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476068 · Cited by 229
Passi, S., & Jackson, S. J. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274405 · Cited by 225
Human factors in model interpretability: Industry practices, challenges, and needs
Hong, S. R., Hullman, J., & Bertini, E. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392878 · Cited by 219 · cited 5×
“The trustworthy coach was everything that the existing XAI literature says is needed to build trust in AI.”
BackgroundMethodInfluentialEvaluating Effects of User Experience and System Transparency on Trust in Automation
Yang, X. J., Unhelkar, V. V., Li, K., & Shah, J. A. (2017) · Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/2909824.3020230 · Cited by 208 · cited 1×
“Such measures include, but are not limited to measuring end-users’ decision time [19, 62], how many times end-users requested AI recommendation [52], and how many times they agreed with AI recommendations [7, 23, 58].”
BackgroundThe Psychology (and Economics) of Trust
Evans, A. M., & Krueger, J. I. (2009) · Social and Personality Psychology Compass · doi:10.1111/j.1751-9004.2009.00232.x · Cited by 171
Evaluating saliency map explanations for convolutional neural networks: a user study
Alqaraawi, A., Schuessler, M., Weiß, P., Costanza, E., & Berthouze, N. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377519 · Cited by 163
Algorithmic realism: expanding the boundaries of algorithmic thought
Green, B., & Viljoen, S. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372840 · Cited by 161
Auditing Algorithms: Understanding Algorithmic Systems from the Outside In
Metaxa, D., Park, J. S., Robertson, R. E., Karahalios, K., Wilson, C., Hancock, J., & Sandvig, C. (2021) · Foundations and Trends® in Human–Computer Interaction · doi:10.1561/1100000083 · Cited by 140
Smith-Renner, A., Fan, R., Birchfield, M., Wu, T., Boyd-Graber, J., Weld, D. S., & Findlater, L. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376624 · Cited by 112 · cited 1×
“Unfortunately, naive application of existing XAI methods may be ineective [2, 40, 65] to broader audience of end-users, if not harmful [28, 47, 49, 51].”
BackgroundTaxonomy of Trust-Relevant Failures and Mitigation Strategies
Tolmeijer, S., Weiss, A., Hanheide, M., Lindner, F., Powers, T. M., Dixon, C., & Tielman, M. L. (2020) · Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/3319502.3374793 · Cited by 103 · cited 1×
“Also, creating AI with the ability to explain and justify its decision-making process [5, 38, 44] could increase the end-users willingness to accept the AI’s recommendations [53, 55].”
BackgroundExplainability Pitfalls: Beyond Dark Patterns in Explainable AI
Ehsan, U., & Riedl, M. O. (2021) · arXiv · doi:10.48550/arxiv.2109.12480 · Cited by 99
COGAM: Measuring and Moderating Cognitive Load in Machine Learning Model Explanations
Abdul, A., von der Weth, C., Kankanhalli, M., & Lim, B. Y. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376615 · Cited by 96 · cited 1×
“However, most existing explainable AI (XAI) methods [10, 21, 43] target math-savy AI creators [1, 22] to aid them in AI model debugging and monitoring [6].”
BackgroundWhat can AI do for me?: evaluating machine learning interpretations in cooperative play
Feng, S., & Boyd-Graber, J. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302265 · Cited by 92
User Trust Dynamics: An Investigation Driven by Differences in System Performance
Yu, K., Berkovsky, S., Taib, R., Conway, D., Zhou, J., & Chen, F. (2017) · Proceedings of the 22nd International Conference on Intelligent User Interfaces · doi:10.1145/3025171.3025219 · Cited by 90
Israelsen, B. W., & Ahmed, N. R. (2019) · ACM Computing Surveys · doi:10.1145/3267338 · Cited by 74
Explainability pitfalls: beyond dark patterns in explainable AI
Ehsan, U., & Riedl, M. O. (2024) · Patterns · doi:10.1016/j.patter.2024.100971 · Cited by 72
Brains or Beauty: How to Engender Trust in User-Agent Interactions
Yuksel, B. F., Collisson, P., & Czerwinski, M. (2017) · ACM Transactions on Internet Technology · doi:10.1145/2998572 · Cited by 63
Kuipers, B. (2018) · Communications of the ACM · doi:10.1145/3173087 · Cited by 54 · cited 1×
“Instead, it is also important to consider other factors, such as the end-user’s expertise [65], their ethical considerations [27, 29, 47], and the e�ects that the recommended actions and outcomes of trying to reach user goals could have on the end-user’s well-being and satisfaction [35].”
BackgroundRamesh, D., Kameswaran, V., Wang, D., & Sambasivan, N. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533237 · Cited by 47
Evaluating the Interpretability of Generative Models by Interactive Reconstruction
Ross, A., Chen, N., Hang, E. Z., Glassman, E. L., & Doshi-Velez, F. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445296 · Cited by 40
Escher, N., & Banović, N. (2020) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3392874 · Cited by 29
Zhang, E., & Banovic, N. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445714 · Cited by 28
Leveraging Human Routine Models to Detect and Generate Human Behaviors
Banović, N., Wang, A., Jin, Y., Chang, C., Ramos, J., Dey, A. K., & Mankoff, J. C. (2017) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025571 · Cited by 28
Examining Profiles for Robotic Risk Assessment: Does a Robot's Approach to Risk Affect User Trust?
Bridgwater, T., Giuliani, M., van Maris, A., Baker, G., Winfield, A., & Pipe, T. (2020) · Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction · doi:10.1145/3319502.3374804 · Cited by 14
Prabhudesai, S., Wang, N. C., Ahluwalia, V., Huan, X., Bapuraj, J. R., Banovic, N., & Rao, A. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.740353 · Cited by 13
The Role of Environmental Predictability and Costs in Relying on Automation
Sutherland, S. C., Harteveld, C., & Young, M. E. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702609 · Cited by 12
Can Semi-Anthropomorphism Influence Trust and Compliance?: Exploring Image Use in App Interfaces
Gruber, D., Aune, A., & Koutstaal, W. (2018) · Proceedings of the Technology, Mind, and Society · doi:10.1145/3183654.3183700 · Cited by 8
Research works printed but not linked: 12
- Arpad E. Elo. 1967. The Proposed USCF Rating System, Its Development, Theory, and Applications. Chess Life, Vol. 22, 8 (August 1967), 242--247.
- Birhanu Eshete. 2021. Making machine learning trustworthy. Science, Vol. 373, 6556 (2021), 743--744.
- 10.1111/j.1751--9004.2009.00232.x · doi:10.1111/j.1751--9004.2009.00232.x
- Lex Fridman, Li Ding, Benedikt Jenik, and Bryan Reimer. 2019. Arguing Machines: Human Supervision of Black Box AI Systems That Make Life-Critical Decisions. In IEEE Conference on Computer Vision and Pattern Recognition Workshops, CVPR Workshops 2019, Long Beach, CA, USA, June 16--20, 2019. Computer Vision Foundation / IEEE, 146--154. http://openaccess.thecvf.com/content_CVPRW_2019/html/Autonomous_Driving/Fridman_Arguing_Machines_Human_Supervision_of_Black_Box_AI_Systems_That_CVPRW_2019_paper.html
- Stéphane Hulaud. 2018. Identification of taste attributes from an audio signal. US Patent 9,934,785.
- 10.1038/s42256-019-0088--2 · doi:10.1038/s42256-019-0088--2
- Niklas Luhmann. 1988. Familiarity, Confidence, Trust: Problems and Alternatives. D. Gambetta, editor, Trust: Making and Breaking of Cooperative Relations, Basil Blackwell, Oxford, 1988 (1988).
- Maria Madsen and Shirley Gregor. 2000. Measuring human-computer trust. In Proceedings of the 11 th Australasian Conference on Information Systems. 6--8.
- Johannes Schneider, Joshua Handali, Michalis Vlachos, and Christian Meske. 2020. Deceptive AI Explanations: Creation and Detection. CoRR, Vol. abs/2001.07641 (2020). showeprint[arXiv]2001.07641 https://arxiv.org/abs/2001.07641 · arXiv
- Simone Stumpf, Adrian Bussone, and Dympna O'sullivan. 2016. Explanations considered harmful? user interactions with machine learning systems. In The CHI 2016 Human Centred Machine Learning Workshop.
- Jennifer Wortman Vaughan and Hanna Wallach. 2020. A human-centered agenda for intelligible machine learning. Machines We Trust: Getting Along with Artificial Intelligence (2020).
- 10.1145/3025171.3025219 · DOI · Paper page
Cited by
AI deception: A survey of examples, risks, and potential solutions
Park, P. S., Goldstein, S., O’Gara, A., Chen, M., & Hendrycks, D. (2024) · Patterns · doi:10.1016/j.patter.2024.100988 · Cited by 243
“Although the deceptive reasoning is transparent through the chain of thought, the malicious behavior remained even when the chain of thought was ‘‘distilled’’ away and not made visible to users.”
AI Deception: A Survey of Examples, Risks, and Potential Solutions
Park, P. S., Goldstein, S., O'Gara, A., Chen, M., & Hendrycks, D. (2023) · arXiv · doi:10.48550/arxiv.2308.14752 · Cited by 243
“Although the deceptive reasoning is transparent through the chain of thought, the malicious behavior remained even when the chain of thought was ‘‘distilled’’ away and not made visible to users.”
Jabbour, S., Fouhey, D., Shepard, S., Valley, T. S., Kazerooni, E. A., Banovic, N., Wiens, J., & Sjoding, M. W. (2023) · JAMA · doi:10.1001/jama.2023.22295 · Cited by 236
Prabhudesai, S., Yang, L., Asthana, S., Huan, X., Liao, Q. V., & Banovic, N. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584033 · Cited by 82
“Most existing quantitative empirical studies in ML-supported decision-making measure the effects of transparency and communicating uncertainty on efficiency [4, 102], quality of decisions [36, 69, 73], trust and reliance [5, 7, 18, 19, 77, 108], and the decision-maker’s confidence in decisions [51].”
BackgroundExplanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making
Schoeffer, J., De-Arteaga, M., & Kühl, N. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642621 · Cited by 68
“Importantly, even if explanations lead to more (in)accurate decisions, it is unclear from prior work how this relates to distributive fairness properties.”
Background
Metadata
- Authors
- Banovic, N., Yang, Z., Ramesh, A., & Liu, A.
- Affiliations
- University of Michigan, Ann Arbor, MI, USA
- University of Michigan (US)
- Venue
- Proceedings of the ACM on Human-Computer Interaction
- Published
- 2023-04-14
- Volume
- 7
- Issue
- CSCW1
- Pages
- 1-17
- ISSN
- 2573-0142 (ISSN-L)
- OpenAlex
- W4365999041
- Semantic Scholar
- 4c2857dbcef303ef8451d272f2f2688e07e013a0
- Cited by
- 55
- References
- 51
- Citations by year
- 2026: 13 · 2025: 21 · 2024: 16 · 2023: 4
- FWCI
- 13.54
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Proceedings of the ACM on Human-Computer Interaction (publishedVersion, other-oa)
- Funders
- University of Michigan
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- trustworthy AI, AI deception, end-user trust, transparency, fairness, chess engines, human-AI interaction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.