The DOI matches its Crossref record; 95 of the 95 references it cites were found in a registry or an index.
Knowing About Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systems
Open accessAbstract
The dazzling promises of AI systems to augment humans in various tasks hinge on whether humans can appropriately rely on them. Recent research has shown that appropriate reliance is the key to achieving complementary team performance in AI-assisted decision making. This paper addresses an under-explored problem of whether the Dunning-Kruger Effect (DKE) among people can hinder their appropriate reliance on AI systems. DKE is a metacognitive bias due to which less-competent individuals overestimate their own skill and performance. Through an empirical study (N = 249), we explored the impact of DKE on human reliance on an AI system, and whether such effects can be mitigated using a tutorial intervention that reveals the fallibility of AI advice, and exploiting logic units-based explanations to improve user understanding of AI advice. We found that participants who overestimate their performance tend to exhibit under-reliance on AI systems, which hinders optimal team performance. Logic units-based explanations did not help users in either improving the calibration of their competence or facilitating appropriate reliance. While the tutorial intervention was highly effective in helping users calibrate their self-assessment and facilitating appropriate reliance among participants with overestimated self-assessment, we found that it can potentially hurt the appropriate reliance of participants with underestimated self-assessment. Our work has broad implications on the design of methods to tackle user cognitive biases while facilitating appropriate reliance on AI systems. Our findings advance the current understanding of the role of self-assessment in shaping trust and reliance in human-AI decision making. This lays out promising future directions for relevant HCI research in this community.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Similar patterns are evidenced by prior research using AI-generated content [38], where users tend to overestimate their capabilities and those of AI [28, 36].”
From How University Disability Services Professionals Write Image Descriptions for HCI Figures Using Generative AI (2026) · cites this paper as [28]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“These parameters are commonly adapted in literature to capture the level of reliance within the human-AI interaction context.”
From Dealing with Uncertainty: Understanding the Impact of Prognostic Versus Diagnostic Tasks on Trust and Reliance in Human-AI Decision Making (2024) · cites this paper as [51]
The cited source says what the claim attributes to it.
In the citing paper
“This dovetails with He et al. [77] who find that poor performers in a logical reasoning task tend to be overconfident-the Dunning-Kruger effect-leading to under-reliance on AI”
From The Metacognitive Demands and Opportunities of Generative AI (2024) · cites this paper as [77]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical power analyses using G∗Power 3.1: tests for correlation and regression analyses
Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009) · Behavior Research Methods · doi:10.3758/brm.41.4.1149 · Cited by 37,615 · cited 1×
“Before recruiting participants, we computed the required sample size in a power analysis for the 2 × 2 factorial design using G*Power [25].”
MethodKruger, J., & Dunning, D. (1999) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.77.6.1121 · Cited by 7,731 · cited 11×
“also found that a training session to increase the skills of participants is highly successful in mitigating such effect [43].”
InfluentialTrust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 4×
“"vulnerability" discussed by Lee and See [47] as a contextual requirement to encourage appropriate system reliance.”
BackgroundMethodInfluentialLikert scales, levels of measurement, and the “laws” of statistics
Norman, G. (2010) · Advances in Health Sciences Education · doi:10.1007/s10459-010-9222-y · Cited by 4,850 · cited 1×
“To further analyze how other covariates shape user trust in AI system, we decided to conduct AN(C)OVAs despite the anticipation that our data may not be normally distributed because these analyses have been shown to be robust to Likert-type ordinal data [60].”
MethodInternational evaluation of an AI system for breast cancer screening
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1799-6 · Cited by 3,287 · cited 1×
“In the last decade, powerful AI systems (especially deep learning systems) have shown better performance than human experts on many tasks, sometimes outperforming humans by a large margin [58, 87].”
BackgroundAlgorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
Resistance to Medical Artificial Intelligence
Longoni, C., Bonezzi, A., & Morewedge, C. K. (2019) · Journal of Consumer Research · doi:10.1093/jcr/ucz013 · Cited by 2,015 · cited 1×
“phenomenon is called “Algorithm Aversion,” which has been observed across multiple domains, like moral decision making [31], economic bargains [23], medical diagnosis [54], and autonomous driving [14].”
BackgroundFlawed Self-Assessment: Implications for Health, Education, and the Workplace
Dunning, D., Heath, C., & Suls, J. M. (2004) · Psychological Science in the Public Interest · doi:10.1111/j.1529-1006.2004.00018.x · Cited by 1,865
Task-Dependent Algorithm Aversion
Castelo, N., Bos, M. W., & Lehmann, D. R. (2019) · Journal of Marketing Research · doi:10.1177/0022243719851788 · Cited by 1,500 · cited 1×
“Others revealed similar findings in contexts where tasks are perceived as being more objective [7], machines share rationale with humans [75] or with prior exposure to similar systems [42].”
BackgroundTask-Dependent Algorithm Aversion
Castelo, N., Bos, M. W., Bos, M. W., Lehmann, D. R., & Lehmann, D. R. (2019) · Springer eBooks · doi:10.1007/978-3-031-88067-4_21 · Cited by 1,441
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2018) · Management Science · doi:10.1287/mnsc.2016.2643 · Cited by 1,230 · cited 2×
“[13] on trust in two-stage decision making.”
BackgroundBuçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124
Designing Theory-Driven User-Centric Explainable AI
Wang, D., Yang, Q., Abdul, A., & Lim, B. Y. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300831 · Cited by 906
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895 · cited 5×
“Topics like performance feedback [2, 55], risk perception [27, 34], uncertainty [77] and confdence [10, 79, 87] of machine learning models, impact of explanations [3, 46] have been extensively studied in human-AI decision making.”
BackgroundInfluentialFranke, T., Attig, C., & Wessel, D. (2019) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2018.1456150 · Cited by 828 · cited 2×
“The covariates’ distribution is as follows: ATI (𝑀 = 3.73, 𝑆𝐷 = 0.99, 6-point Likert scale, and 1: low, 6: 6https://github.com/RichardHGL/CHI2023_DKE high), TiA-Propensity to Trust (𝑀 = 2.95, 𝑆𝐷 = 0.60, 5-point Likert scale, 1: tend to distrust, 5: tend to trust).”
MethodInfluentialA systematic review of algorithm aversion in augmented decision making
Burton, J. W., Stein, M., & Jensen, T. B. (2020) · Journal of Behavioral Decision Making · doi:10.1002/bdm.2155 · Cited by 814 · cited 3×
“[6] summarized the cause and solution of algorithm aversion with fve aspects: expectations and expertise, decision autonomy, incentivization, cognitive compatibility,”
BackgroundDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757 · cited 10×
“At the same time, explanations have been shown to improve the human understanding and interpretation of AI advice [3, 52, 79], which can also potentially contribute to appropriate reliance.”
BackgroundMethodResultInfluentialDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754
Why the unskilled are unaware: Further explorations of (absent) self-insight among the incompetent
Ehrlinger, J., Johnson, K., Banner, M., Dunning, D., & Kruger, J. (2008) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2007.05.002 · Cited by 707
Advice taking in decision making: egocentric discounting and reputation formation
Yaniv, I., & Kleinberger, E. (2000) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.2000.2909 · Cited by 701
Expanding Explainability: Towards Social Transparency in AI systems
Ehsan, U., Liao, Q. V., Muller, M., Riedl, M. O., & Weisz, J. D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445188 · Cited by 538 · cited 1×
“Meanwhile, fairness, accountability, and transparency of incorporating AI systems for collaborative decision making received more and more attention from a wide range of stakeholders [19, 21].”
BackgroundUnderstanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527 · cited 1×
“They are commonly used in the literature, for example in [83, 87].”
BackgroundMetrics for explainable AI: challenges and prospects
Hoffman, R. R., Mueller, S. T., Klein, G., & Litman, J. (2018) · arXiv · doi:10.48550/arxiv.1812.04608 · Cited by 518 · cited 1×
“an intervention can potentially help users improve their reliance on AI systems [36].”
BackgroundMeaningful information and the right to explanation
Selbst, A. D., & Powles, J. (2017) · International Data Privacy Law · doi:10.1093/idpl/ipx022 · Cited by 516 · cited 1×
“According to GDPR, the users of AI systems should have the right to access meaningful explanations of model predictions [68].”
BackgroundBeyond Accuracy: The Role of Mental Models in Human-AI Team Performance
Bansal, G., Nushi, B., Kamar, E., Lasecki, W. S., Weld, D. S., & Horvitz, E. (2019) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v7i1.5285 · Cited by 466 · cited 1×
“Topics like performance feedback [2, 55], risk perception [27, 34], uncertainty [77] and confidence [10, 79, 87] of machine learning models, impact of explanations [3, 46] have been extensively studied in human-AI decision making.”
BackgroundAdvances in Experimental Social Psychology, Vol. 19
Fine, G. A., & Berkowitz, L. (1987) · Contemporary Sociology · doi:10.2307/2070758 · Cited by 458
Why are we averse towards Algorithms? A comprehensive literature Review on Algorithm aversionPreprint
Jussupow, E., Benbasat, I., & Heinzl, A. (2020) · RePEc: Research Papers in Economics · Cited by 440
Baeza-Yates, R. (2018) · Communications of the ACM · doi:10.1145/3209581 · Cited by 432
The Principles and Limits of Algorithm-in-the-Loop Decision Making
Green, B., & Chen, Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359152 · Cited by 392
ReClor: A Reading Comprehension Dataset Requiring Logical ReasoningPreprint
Yu, W., Jiang, Z., Dong, Y., & Feng, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.04326 · Cited by 380 · cited 1×
“We use the publicly available Reclor1 [85] dataset to this end.”
MethodLai, V., & Tan, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287590 · Cited by 377
Algorithm Aversion: People Erroneously Avoid Algorithms after Seeing Them ErrPreprint
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2466040 · Cited by 356
Wang, X., & Yin, M. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450650 · Cited by 320 · cited 10×
“At the same time, explanations have been shown to improve the human understanding and interpretation of AI advice [3, 52, 79], which can also potentially contribute to appropriate reliance.”
BackgroundMethodInfluentialTheoretical Considerations and Development of a Questionnaire to Measure Trust in Automation
Körber, M. (2019) · Advances in Intelligent Systems and Computing · doi:10.1007/978-3-319-96074-6_2 · Cited by 317 · cited 1×
“• Trust in Automation (TiA) questionnaire [41], a validated instrument to measure (subjective) trust [76].”
BackgroundLipton, P. (1990) · Royal Institute of Philosophy Supplement · doi:10.1017/s1358246100005130 · Cited by 281
Disparate Interactions: An Algorithm-in-the-Loop Analysis of Fairness in Risk Assessments
Green, B., & Chen, Y. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287563 · Cited by 275
Human-Centered Explainable AI: Towards a Reflective Sociotechnical Approach
Ehsan, U., & Riedl, M. O. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-60117-1_33 · Cited by 274
CHI '86 Conference on Human Factors in Computing Systems
(1985) · Computer · doi:10.1109/mc.1985.1662775 · Cited by 272
Chong, L., Zhang, G., Goucher-Lambert, K., Kotovsky, K., & Cagan, J. (2022) · Computers in Human Behavior · doi:10.1016/j.chb.2021.107018 · Cited by 262 · cited 3×
“Topics like performance feedback [2, 55], risk perception [27, 34], uncertainty [77] and confidence [10, 79, 87] of machine learning models, impact of explanations [3, 46] have been extensively studied in human-AI decision making.”
BackgroundUnderstanding Malicious Behavior in Crowdsourcing Platforms: The Case of Online Surveys
Gadiraju, U., Kawase, R., Dietze, S., & Demartini, G. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702443 · Cited by 219
Self-citationSchramowski, P., Stammer, W., Teso, S., Brugger, A., Herbert, F., Shao, X., Luigs, H.-G., Mahlein, A.-K., & Kersting, K. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0212-3 · Cited by 207 · cited 1×
“For a deeper analysis of our results, a number of additional measures were considered based on observations from existing literature [49, 67, 76]:”
MethodYou, S., Yang, C., & Li, X. (2022) · Journal of Management Information Systems · doi:10.1080/07421222.2022.2063553 · Cited by 192 · cited 2×
“Besides contradicting attitudes towards the use of AI systems, prior work has shown how diferent human factors such as algorithmic literacy [72], expertise [53], and cognitive load [84] can afect users’ fnal adoption of algorithmic advice.”
BackgroundShin, D., Kee, K. F., & Shin, E. Y. (2022) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2022.102494 · Cited by 191 · cited 1×
“Besides contradicting attitudes towards the use of AI systems, prior work has shown how diferent human factors such as algorithmic literacy [72], expertise [53], and cognitive load [84] can afect users’ fnal adoption of algorithmic advice.”
BackgroundContrastive explanation: a structural-model approach
Miller, T. (2021) · The Knowledge Engineering Review · doi:10.1017/s0269888921000102 · Cited by 186 · cited 1×
“For example, contrastive explanations may work in the context of human-AI decision making [51, 59].”
BackgroundRapid Trust Calibration through Interpretable and Uncertainty-Aware AI
Tomsett, R., Preece, A., Braines, D., Cerutti, F., Chakraborty, S., Srivastava, M., Pearson, G., & Kaplan, L. (2020) · Patterns · doi:10.1016/j.patter.2020.100049 · Cited by 186 · cited 1×
“Topics like performance feedback [2, 55], risk perception [27, 34], uncertainty [77] and confidence [10, 79, 87] of machine learning models, impact of explanations [3, 46] have been extensively studied in human-AI decision making.”
BackgroundHow do people judge the credibility of algorithmic sources?
Shin, D. (2021) · AI & Society · doi:10.1007/s00146-021-01158-4 · Cited by 178
Hou, Y. T.-Y., & Jung, M. F. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479864 · Cited by 173 · cited 1×
“A recent work [37] found that the choice of framings of human agents and algorithmic agents may afect user perception of agent competence (i.”
BackgroundI Think I Get Your Point, AI! The Illusion of Explanatory Depth in Explainable AI
Chromik, M., Eiband, M., Buchner, F., Krüger, A., & Butz, A. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450644 · Cited by 144 · cited 1×
“[11] compared perceived understanding of XAI methods with their actual understanding to reveal users’ illusion of explanatory depth.”
BackgroundWhy am I seeing this? Deconstructing algorithm literacy through the lens of users
Shin, D., Rasul, A., & Fotiadis, A. (2021) · Internet Research · doi:10.1108/intr-02-2021-0087 · Cited by 142
A Human-AI Collaborative Approach for Clinical Decision Making on Rehabilitation Assessment
Lee, M. H., Siewiorek, D. P., Smailagic, A., Bernardino, A., & Bermúdez i Badia, S. B. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445472 · Cited by 137 · cited 1×
“, financial [33], medical domains [48]).”
Human Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks
Lu, Z., & Yin, M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445562 · Cited by 131 · cited 2×
“Topics like performance feedback [2, 55], risk perception [27, 34], uncertainty [77] and confidence [10, 79, 87] of machine learning models, impact of explanations [3, 46] have been extensively studied in human-AI decision making.”
BackgroundUnderstanding and Mitigating Worker Biases in the Crowdsourced Collection of Subjective Judgments
Hube, C., Fetahu, B., & Gadiraju, U. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300637 · Cited by 129 · cited 1×
“Inspired by existing work to mitigate different kinds of cognitive biases through revealing such biases to users [1, 38], we decided to adopt a tutorial to help users calibrate their skills through self-assessment on logical reasoning tasks.”
MethodSelf-citation"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans
Lai, V., Liu, H., & Tan, C. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376873 · Cited by 123
Operationalizing Human-Centered Perspectives in Explainable AI
Ehsan, U., Wintersberger, P., Liao, Q. V., Mara, M., Streit, M., Wachter, S., Riener, A., & Riedl, M. O. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3441342 · Cited by 123 · cited 1×
“Meanwhile, fairness, accountability, and transparency of incorporating AI systems for collaborative decision making received more and more attention from a wide range of stakeholders [19, 21].”
BackgroundHuman-Centered Explainable AI (XAI): From Algorithms to User Experiences
Liao, Q. V., & Varshney, K. R. (2021) · arXiv · doi:10.48550/arxiv.2110.10790 · Cited by 120
I can do better than your AI: expertise and explanations
Schaffer, J., O'Donovan, J., Michaelis, J., Raglin, A., & Höllerer, T. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302308 · Cited by 119
Dillen, N., Ilievski, M., Law, E., Nacke, L. E., Czarnecki, K., & Schneider, O. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376247 · Cited by 117 · cited 1×
“phenomenon is called “Algorithm Aversion,” which has been observed across multiple domains, like moral decision making [31], economic bargains [23], medical diagnosis [54], and autonomous driving [14].”
BackgroundHuman-Centered Explainable AI (HCXAI): Beyond Opening the Black-Box of AI
Ehsan, U., Wintersberger, P., Liao, Q. V., Watkins, E. A., Manger, C., Daumé III, H., Riener, A., & Riedl, M. O. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3503727 · Cited by 117
Rage against the machine: Automation in the moral domain
Gogoll, J., & Uhl, M. (2018) · Journal of Behavioral and Experimental Economics · doi:10.1016/j.socec.2018.04.003 · Cited by 115 · cited 1×
“phenomenon is called “Algorithm Aversion,” which has been observed across multiple domains, like moral decision making [31], economic bargains [23], medical diagnosis [54], and autonomous driving [14].”
BackgroundLiu, H., Lai, V., & Tan, C. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479552 · Cited by 111
A rational model of the Dunning–Kruger effect supports insensitivity to evidence in low performers
Jansen, R. A., Rafferty, A. N., & Griffiths, T. L. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01057-0 · Cited by 100 · cited 1×
“Evaluating one’s own performance on a task, typically known as “self-assessment”, is perceived as a fundamental skill, but people appear to calibrate their abilities [39] poorly.”
BackgroundRechkemmer, A., & Yin, M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501967 · Cited by 97 · cited 1×
“They found that factors like first impression [76], AI literacy [8], risk perception [33, 34], and performance feedback [55, 61] among others, play important roles in shaping human trust and reliance on AI systems.”
BackgroundExperiences surveying the crowd: reflections on methods, participation, and reliability
Marshall, C. C., & Shipman, F. M., III (2013) · ACM Web Science Conference (WebSci) · doi:10.1145/2464464.2464485 · Cited by 89
In Platforms We Trust?Unlocking the Black-Box of News Algorithms through Interpretable AI
Shin, D., Zaid, B., Biocca, F. A., & Rasul, A. (2022) · Journal of Broadcasting & Electronic Media · doi:10.1080/08838151.2022.2057984 · Cited by 89 · cited 1×
“For example, users’ algorithmic literacy [71–73] about fairness, accountability, transparency, and explainability is found to greatly afect their trust and privacy concern in adopting the advice from AI systems [70, 74].”
BackgroundA Checklist to Combat Cognitive Biases in Crowdsourcing
Draws, T., Rieger, A., Inel, O., Gadiraju, U., & Tintarev, N. (2021) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v9i1.18939 · Cited by 86 · cited 2×
“With the help of Cognitive Biases Checklist introduced [15], we analyzed potential bias in our study.”
BackgroundMethodSelf-citationWhen Do People Want AI to Make Decisions?
Kramer, M. F., Schaich Borg, J., Conitzer, V., & Sinnott-Armstrong, W. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278752 · Cited by 83 · cited 1×
“Others revealed similar findings in contexts where tasks are perceived as being more objective [7], machines share rationale with humans [75] or with prior exposure to similar systems [42].”
BackgroundNo Risk No Trust: Investigating Perceived Risk in Highly Automated Driving
Li, M., Holthausen, B. E., Stuck, R. E., & Walker, B. N. (2019) · Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications · doi:10.1145/3342197.3344525 · Cited by 80
Fereday, J. A., & Muir‐Cochrane, E. (2006) · Collegian Journal of the Royal College of Nursing Australia · doi:10.1016/s1322-7696(08)60511-9 · Cited by 77 · cited 1×
“According to previous work [26, 57], interventions that provide users with feedback on their performance may help improve their self-assessment.”
Towards a Science of Human-AI Decision Making: A Survey of Empirical Studies
Lai, V., Chen, C., Liao, Q. V., Smith-Renner, A., & Tan, C. (2021) · arXiv · doi:10.48550/arxiv.2112.11471 · Cited by 75 · cited 4×
“of incorporating XAI methods in human-AI collaboration are still limited [3, 44].”
BackgroundMethodInfluentialShin, D. (2020) · Journalism Practice · doi:10.1080/17512786.2020.1841018 · Cited by 73 · cited 1×
“For example, users’ algorithmic literacy [71–73] about fairness, accountability, transparency, and explainability is found to greatly affect their trust and privacy concern in adopting the advice from AI systems [70, 74].”
BackgroundTheoretical considerations and development of a questionnaire to measure trust in automation
Körber, M. (2018) · doi:10.31234/osf.io/nfc45 · Cited by 71
Using Worker Self-Assessments for Competence-Based Pre-Selection in Crowdsourcing Microtasks
Gadiraju, U., Fetahu, B., Kawase, R., Siehndel, P., & Dietze, S. (2017) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3119930 · Cited by 69 · cited 4×
“[29] used self-assessment for competence-based pre-selection in crowdsourcing marketplaces, Green et al.”
BackgroundMethodInfluentialSelf-citationGreen, B., & Chen, Y. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479562 · Cited by 69 · cited 2×
“Topics like performance feedback [2, 55], risk perception [27, 34], uncertainty [77] and confdence [10, 79, 87] of machine learning models, impact of explanations [3, 46] have been extensively studied in human-AI decision making.”
BackgroundSecond Chance for a First Impression? Trust Development in Intelligent System Interaction
Tolmeijer, S., Gadiraju, U., Ghantasala, R., Gupta, A., & Bernstein, A. (2021) · Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3450613.3456817 · Cited by 67 · cited 7×
“, dealing with an imperfect AI system) [5, 76].”
BackgroundMethodInfluentialSelf-citationImpact of Algorithmic Decision Making on Human Behavior: Evidence from Ultimatum Bargaining
Erlei, A., Nekdem, F., Meub, L., Anand, A., & Gadiraju, U. (2020) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v8i1.7462 · Cited by 62 · cited 1×
“Several prior works have carried out empirical studies within this context of human-AI decision making, to explore the effectiveness of different kinds of explanations and the role of human factors in shaping such collaboration [3, 8, 24, 33, 52, 62, 79, 87].”
BackgroundSelf-citationKnowledge-Intensive Language Understanding for Explainable AI
Sheth, A., Gaur, M., Roy, K., & Faldu, K. (2021) · IEEE Internet Computing · doi:10.1109/mic.2021.3101919 · Cited by 61 · cited 1×
“However, multiple pieces of research work point out that such token-level highlights are still hard to interpret [69, 81, 86].”
BackgroundBradley, C. S., Dreifuerst, K. T., Johnson, B. K., & Loomis, A. (2022) · Clinical Simulation in Nursing · doi:10.1016/j.ecns.2022.02.010 · Cited by 57 · cited 1×
“Further work also proved the effectiveness of such training in different domains like medicine [4] and economics [64].”
BackgroundUnderstanding the Role of Explanation Modality in AI-assisted Decision-making
Robbemond, V., Inel, O., & Gadiraju, U. (2022) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3503252.3531311 · Cited by 56 · cited 1×
“Several prior works have carried out empirical studies within this context of human-AI decision making, to explore the effectiveness of different kinds of explanations and the role of human factors in shaping such collaboration [3, 8, 24, 33, 52, 62, 79, 87].”
BackgroundSelf-citationYou’d Better Stop! Understanding Human Reliance on Machine Learning Models under Covariate Shift
Chiang, C.-W., & Yin, M. (2021) · 13th ACM Web Science Conference 2021 · doi:10.1145/3447535.3462487 · Cited by 53 · cited 5×
“Based on the effectiveness of tutorials as interventions in previous HCI literature [8, 9, 45], we designed a tutorial as a means to shed light on the fallibility of AI advice.”
BackgroundMethodInfluentialErlei, A., Das, R., Meub, L., Anand, A., & Gadiraju, U. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517734 · Cited by 49
Self-citationChiang, C.-W., & Yin, M. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511121 · Cited by 42 · cited 8×
““User tutorials, when presented in appropriate forms, can help some people rely on ML models more appropriately” [8].”
BackgroundMethodInfluentialMcKevitt, C. (2016) · Journal of University Teaching and Learning Practice · doi:10.53761/1.13.1.2 · Cited by 37
Should I Follow AI-based Advice? Measuring Appropriate Reliance in Human-AI Decision-Making
Schemmer, M., Hemmer, P., Kühl, N., Benz, C., & Satzger, G. (2022) · arXiv · doi:10.48550/arxiv.2204.06916 · Cited by 33 · cited 1×
“[66] presented four conditions of appropriate reliance patterns (see Table 1) when the disagreement exists and correct answer exists in human initial decision or AI advice.”
BackgroundHierarchical Interpretation of Neural Text Classification
Yan, H., Gui, L., & He, Y. (2022) · Computational Linguistics · doi:10.1162/coli_a_00459 · Cited by 29 · cited 1×
“However, multiple pieces of research work point out that such token-level highlights are still hard to interpret [69, 81, 86].”
BackgroundLogiformer: A Two-Branch Graph Transformer Network for Interpretable Logical Reasoning
Xu, F., Liu, J., Lin, Q., Pan, Y., & Zhang, L. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3532016 · Cited by 27
Fogliato, R., Chouldechova, A., & Lipton, Z. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479572 · Cited by 23 · cited 1×
“Topics like performance feedback [2, 55], risk perception [27, 34], uncertainty [77] and confidence [10, 79, 87] of machine learning models, impact of explanations [3, 46] have been extensively studied in human-AI decision making.”
BackgroundAlgorithm Appreciation: Algorithmic Performance, Developmental Processes, and User Interactions
Shin, D., Zaid, B., & Ibahrine, M. (2020) · International Conference on Communications, Computing, Cybersecurity, and Informatics (CCCI) · doi:10.1109/ccci49893.2020.9256470 · Cited by 22 · cited 1×
“Others revealed similar fndings in contexts where tasks are perceived as being more objective [7], machines share rationale with humans [75] or with prior exposure to similar systems [42].”
BackgroundHe, G., Balayn, A., Buijsman, S., Yang, J., & Gadiraju, U. (2022) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v10i1.21990 · Cited by 17
Self-citationSawler, J. (2020) · International Review of Economics Education · doi:10.1016/j.iree.2020.100208 · Cited by 12 · cited 1×
“Further work also proved the effectiveness of such training in different domains like medicine [4] and economics [64].”
BackgroundMore Than Words: Towards Better Quality Interpretations of Text ClassifiersPreprint
Zafar, M. B., Schmidt, P., Donini, M., Archambeau, C., Bießmann, F., Das, S. R., & Kenthapadi, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.12444 · Cited by 8
Why the Unskilled are Unaware: Further Explorations of (Absent) Self-Insight Among the IncompetentPreprint
Ehrlinger, J. M., Johnson, K. L., Dunning, D. A., et al. (2006) · SSRN Electronic Journal · doi:10.2139/ssrn.946242 · Cited by 7
Exploring the Music Perception Skills of Crowd Workers
Samiotis, I. P., Qiu, S., Lofi, C., Yang, J., Gadiraju, U., & Bozzon, A. (2021) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v9i1.18944 · Cited by 7 · cited 1×
“” These effects were replicated in a number of other tasks, like human-AI collaboration [65] and crowdsourcing [63, 76].”
BackgroundSelf-citationGreen, B., & Chen, Y. (2020) · arXiv · doi:10.48550/arxiv.2012.05370 · cited 2×
“Topics like performance feedback [2, 55], risk perception [27, 34], uncertainty [77] and confdence [10, 79, 87] of machine learning models, impact of explanations [3, 46] have been extensively studied in human-AI decision making.”
BackgroundHierarchical Interpretation of Neural Text Classification
Yan, H., Gui, L., & He, Y. (2022) · arXiv · doi:10.48550/arxiv.2202.09792 · cited 1×
“However, multiple pieces of research work point out that such token-level highlights are still hard to interpret [69, 81, 86].”
Background
Research works printed but not linked: 1
- 10.1145/3449287 · doi:10.1145/3449287 · Paper page
Cited by
The Metacognitive Demands and Opportunities of Generative AI
Tankelevitch, L., Kewenig, V., Simkute, A., Scott, A. E., Sarkar, A., Sellen, A., & Rintel, S. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642902 · Cited by 493
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610219 · Cited by 220
Enhancing AI-Assisted Group Decision Making through LLM-Powered Devil's Advocate
Chiang, C.-W., Lu, Z., Li, Z., & Yin, M. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645199 · Cited by 142
Ma, S., Wang, X., Lei, Y., Shi, C., Yin, M., & Ma, X. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642671 · Cited by 153
“The third approach aims to elucidate the rationale behind AI predictions through AI explanations [2, 29, 40, 41, 64, 78, 81].”
InfluentialMa, S., Chen, Q., Wang, X., Zheng, C., Peng, Z., Yin, M., & Ma, X. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713423 · Cited by 48
Metadata
- Authors
- He, G., Kuiper, L., & Gadiraju, U.
- Affiliations
- Delft University of Technology (NL)
- Venue
- Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2023-04-19
- Pages
- 1-18
- OpenAlex
- W4318620371
- Cited by
- 107
- References
- 88
- Citations by year
- 2026: 40 · 2025: 38 · 2024: 23 · 2023: 6
- FWCI
- 53.47
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, cc-by)
- Research Repository (Delft University of Technology) (submittedVersion)
- arXiv (Cornell University) (cc-by)
- Funders
- Technische Universiteit Delft
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- Dunning-Kruger effect, human-AI teaming, AI-assisted decision-making, metacognitive bias, tutorial action, trust in AI, human-AI decision-making, cognitive biases
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.