The DOI matches its Crossref record; 158 of the 158 references it cites were found in a registry or an index.
Do people appropriately rely on AI-advice? An analytical review of HCI research on human-AI decision-making.
Open accessAbstract
AI systems are increasingly being positioned to assist people in decision-making. However, recent empirical studies show critical concerns that people over-rely on AI advice without analytically engaging with it. While HCI research explores how people rely on AI advice, we argue that it largely overlooks an important aspect: replicating realistic decision-making scenarios. Human-AI interaction factors influence people’s reliance on AI advice. To understand human-AI interaction factors and their interplay, we conducted an analytical review of recent studies in human-AI reliance literature. We analyzed the decision-making tasks in research and their validity in application-grounded contexts. Our findings show that user engagement is a precious commodity for relying on AI advice; however, it comes at a cost. We also discuss factors contributing to “appropriate reliance”, existing research gaps, and recommendations for intervention design for human-AI reliance. Our work contributes to the critical body of research on building appropriate reliance on AI advice.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Educational work likewise models trust, reliance, resistance, and actual use as distinct constructs (Wang et al., 2025), while HCI reviews note heterogeneous definitions of appropriate reliance (Raees et al., 2026).”
From Beyond checking: verification quality, reliance calibration, and learning in generative AI-assisted higher education (2026) · cites this paper as Raees et al., 2026
Citations
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283
The Framing of Decisions and the Psychology of Choice
Tversky, A., & Kahneman, D. (1981) · Science · doi:10.1126/science.7455683 · Cited by 18,497 · cited 1×
“To understand trust, theoretical frameworks from psychology are used in various studies [4, 59, 87, 149, 150].”
Literature review as a research methodology: An overview and guidelines
Snyder, H. (2019) · Journal of Business Research · doi:10.1016/j.jbusres.2019.07.039 · Cited by 11,524 · cited 1×
“We developed this protocol following recent guidelines on literature review research [141], and according to the PRISMA [110] framework, which is outlined in Figure 2.”
Kruger, J., & Dunning, D. (1999) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.77.6.1121 · Cited by 7,731 · cited 1×
“Conversely, users can overestimate their confidence in decisions due to inaccurate interpretations [94], or cognitive biases (e.g., Dunning–Kruger effect [79]: people overestimate their abilities), leading to under-reliance on the ML models.”
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 3×
“To understand trust, theoretical frameworks from psychology are used in various studies [4, 59, 87, 149, 150].”
Cacioppo, J. T., & Petty, R. E. (1982) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.42.1.116 · Cited by 5,359 · cited 1×
“An important trait for improving the users’ mental models concerning AI is capturing their “Need for Cognition” [19], a stable personality trait that captures one’s motivation to engage in effortful mental activities [10].”
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529786 · Cited by 4,332 · cited 1×
“Inter-rater reliability using Cohen’s Kappa [84] indicated strong agreement ( k = 0.80 ).”
InfluentialWohlin, C. (2014) · Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2601248.2601268 · Cited by 4,234 · cited 1×
“To further expand our selection, we performed iterative back-ward and forward (using Google Scholar) snowballing [163] by analyzing references and citations of each paper.”
International evaluation of an AI system for breast cancer screening
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1799-6 · Cited by 3,287 · cited 1×
“Although in many cases, AI performance is superior to human performance [5, 99], it still suffers from systematic or random errors.”
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · cited 4×
“Conversely, users may refrain from using the AI system due to performance bias , perceptions of self-efficacy (the ability to make decisions without advice), or purposefully considering the system’s advice as inferior, demonstrating algorithmic aversion [42, 143].”
InfluentialTrust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817
Baier, A. (1986) · Ethics · doi:10.1086/292745 · Cited by 2,567 · cited 1×
“To understand trust, theoretical frameworks from psychology are used in various studies [4, 59, 87, 149, 150].”
Conditions for intuitive expertise: A failure to disagree.
Kahneman, D., & Klein, G. (2009) · American Psychologist · doi:10.1037/a0016755 · Cited by 2,443 · cited 1×
“For instance, users may over-rely on the system’s advice, demonstrating complacency or automation bias , due to their belief in the system’s superiority [73, 91, 111, 161].”
Algorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061 · cited 2×
“For instance, users may over-rely on the system’s advice, demonstrating complacency or automation bias , due to their belief in the system’s superiority [73, 91, 111, 161].”
Flawed Self-Assessment: Implications for Health, Education, and the Workplace
Dunning, D., Heath, C., & Suls, J. M. (2004) · Psychological Science in the Public Interest · doi:10.1111/j.1529-1006.2004.00018.x · Cited by 1,865
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.101805 · Cited by 1,762
Human-Centered Artificial Intelligence: Reliable, Safe & Trustworthy
Shneiderman, B. (2020) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2020.1741118 · Cited by 1,743 · cited 1×
“In a nutshell, our work makes the following contributions towards the goal of understanding research on achieving appropriate reliance on AI advice: formation [56, 121, 137, 138, 166].”
InfluentialComplacency and Bias in Human Use of Automation: An Attentional Integration
Parasuraman, R., & Manzey, D. H. (2010) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720810376055 · Cited by 1,710 · cited 3×
“For instance, users may over-rely on the system’s advice, demonstrating complacency or automation bias , due to their belief in the system’s superiority [73, 91, 111, 161].”
The role of trust in automation reliance
Dzindolet, M. T., Peterson, S. A., Pomranky, R. A., Pierce, L. G., & Beck, H. P. (2003) · International Journal of Human-Computer Studies · doi:10.1016/s1071-5819(03)00038-7 · Cited by 1,276 · cited 1×
“…these frameworks, AI-assisted decision-making operationalizes trust and reliance measures, where trust reflects the users’ subjective perception (i.e., affective or emotional attitude) towards others or systems [87, 100], while reliance pertains to their (observable) objective behavior [47, 130].”
Buçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124
Sengers, P., Boehner, K., David, S., & Kaye, J. '. (2005) · Proceedings of the 4th decennial conference on Critical computing: between sense and sensibility · doi:10.1145/1094562.1094569 · Cited by 1,030 · cited 2×
“Cox et al. [36] propose adding reflection design points [135] to create friction that helps users shift from system 1 to system 2 thinking.”
Trust in AI and Its Role in the Acceptance of AI Technologies
Choung, H., David, P., & Ross, A. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2050543 · Cited by 963 · cited 1×
“The concept of trust has evolved over the years, from trust in automation (TiA) [65, 87] to trust in modern-day AI assistance [34, 74, 89, 154].”
Zhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895 · cited 10×
“For instance, model-based explanations may be used if the model is sensitive to the dataset [170], explanations adapted to AI’s confidence may be used if the model is focused on prediction [5], or, if the model is focused on user performance, the explanations may be adapted in terms of when they…”
InfluentialDellermann, D., Ebel, P., Söllner, M., & Leimeister, J. M. (2019) · Business & Information Systems Engineering · doi:10.1007/s12599-019-00595-2 · Cited by 826 · cited 2×
“Research increasingly highlights the need to gain a foundational understanding of appropriate reliance on AI advice [13, 41, 81, 132] to improve human-AI decision-making scenarios.”
Does the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445717 · Cited by 757 · cited 10×
“Exploring mental models, AI can be delegated to solve easier problems, and users can be engaged for difficult ones, for instance, to recognize AI errors [5, 8].”
InfluentialDoes the Whole Exceed its Parts? The Effect of AI Explanations on Complementary Team Performance
Bansal, G., Wu, T., Zhou, J., Fok, R., Nushi, B., Kamar, E., Ribeiro, M. T., & Weld, D. S. (2020) · arXiv · doi:10.48550/arxiv.2006.14779 · Cited by 754 · cited 10×
“Exploring mental models, AI can be delegated to solve easier problems, and users can be engaged for difficult ones, for instance, to recognize AI errors [5, 8].”
InfluentialWizard of Oz studies: why and how
Dahlbäck, N., Jönsson, A., & Ahrenberg, L. (1993) · Proceedings of the 1st international conference on Intelligent user interfaces - IUI '93 · doi:10.1145/169891.169968 · Cited by 742 · cited 1×
“However, around 33% (19 studies) do not construct real AI interventions, and, instead, use simulated AI or Wizard-of-Oz [37] approaches, trying to replicate what users would believe or experience while interacting with AI-assisted decision-making.”
Manipulating and Measuring Model Interpretability
Poursabzi-Sangdeh, F., Goldstein, D. G., Hofman, J. M., Wortman Vaughan, J. W., & Wallach, H. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445315 · Cited by 728 · cited 10×
“These studies use (experiment) conditions to test the effect of feature-based [60, 153, 156, 164], example-based [17, 23, 108, 146], model-based [8, 33, 55, 114], or general explanations methods [102, 117, 127].”
InfluentialGood intentions are not enough: how informatics interventions can worsen inequality
Veinot, T. C., Mitchell, H., & Ancker, J. S. (2018) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocy052 · Cited by 704 · cited 1×
“Even when the given task itself is high-stakes (e.g., recidivism [32]) or resembles real-world, if the users involved bear no responsibility or accountability or lack expertise [53, 152], the task may still fail to extract user reliance appropriately.”
Slow Technology – Designing for Reflection
Hallnäs, L., & Redström, J. (2001) · Personal and Ubiquitous Computing · doi:10.1007/pl00000019 · Cited by 693 · cited 1×
“For example, interventions could include putting barriers on default functionalities, making them difficult to run, or slowing them down [58] to enhance deliberate thinking.”
A Survey of Methods for Explaining Black Box Models
Guidotti, R., Monreale, A., Ruggieri, S., Turini, F., Giannotti, F., & Pedreschi, D. (2019) · ACM Computing Surveys · doi:10.1145/3236009 · Cited by 629 · cited 3×
“The literature on human-centered explainable AI (XAI) either advocates for transparent models or enhancing the interpretations of opaque models [50, 56].”
InfluentialLongo, L., Brcic, M., Cabitza, F., Choi, J., Confalonieri, R., Ser, J. D., Guidotti, R., Hayashi, Y., Herrera, F., Holzinger, A., et al. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102301 · Cited by 626 · cited 1×
“Existing reviews in HCAI can be broadly categorized into XAI [2, 6, 92], trust [100, 162], interactions (i.e., interactive Machine Learning (ML)) [45, 121], or related domains [1, 7, 49, 81].”
Cai, C. J., Winter, S., Steiner, D., Wilcox, L., & Terry, M. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359206 · Cited by 594 · cited 1×
“Research [5, 60, 151] has explored methods to understand and enhance human-AI decision-making in diverse domains, including medical [17, 20, 71], finance [21, 28, 43], and justice [32, 52, 86].”
Sarker, S., Chatterjee, S., Xiao, X., & Elbanna, A. (2019) · MIS Quarterly · doi:10.25300/misq/2019/13747 · Cited by 576 · cited 1×
“In the following sections, we provide a background on human-AI reliance and its progression, along with automation [143] and socio-technical systems’ maturity [128].”
InfluentialWason, P. C., & Evans, J. S. B. T. (1974) · Cognition · doi:10.1016/0010-0277(74)90017-1 · Cited by 575 · cited 3×
“Most of the research on interaction factors focuses on adapting user interaction with the AI system, drawing from human psychology theories [36, 72, 159] and implementation design.”
Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., & Wortman Vaughan, J. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376219 · Cited by 564 · cited 1×
“Intervention design should include detailed limitations and risks, for example, by including qualitative narratives, and uncertainty quantification of complex statistical decisions in a digestible (visual) manner [75].”
Technology, Humanness, and Trust: Rethinking Trust in Technology
Marshall University, Lankton, N., McKnight, D. H., Michigan State University, Tripp, J., & Baylor University (2015) · Journal of the Association for Information Systems · doi:10.17705/1jais.00411 · Cited by 553 · cited 1×
“Building reliance on AI advice is rooted in building trust [85, 130, 139].”
Do as AI say: susceptibility in deployment of clinical decision-aids
Gaube, S., Suresh, H., Raue, M., Merritt, A., Berkowitz, S. J., Lermer, E., Coughlin, J. F., Guttag, J. V., Colak, E., & Ghassemi, M. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00385-9 · Cited by 543 · cited 1×
“Even when the given task itself is high-stakes (e.g., recidivism [32]) or resembles real-world, if the users involved bear no responsibility or accountability or lack expertise [53, 152], the task may still fail to extract user reliance appropriately.”
Westbrook, A., Kester, D., & Braver, T. S. (2013) · PLoS ONE · doi:10.1371/journal.pone.0068210 · Cited by 533 · cited 2×
“People weigh the potential benefits of cognitive effort against its perceived cost [78], a phenomenon also referred to as cognitive-effort-discounting [160].”
Understanding the Effect of Accuracy on Trust in Machine Learning Models
Yin, M., Wortman Vaughan, J., & Wallach, H. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300509 · Cited by 527 · cited 10×
“%) of the studies use AI performance metrics like model confidence and uncertainty [27, 28, 98, 118], as well as accuracy [168, 170] to understand user behavior.”
Influential23 Ways to Nudge: A Review of Technology-Mediated Nudging in Human-Computer Interaction
Caraban, A., Karapanos, E., Gonçalves, D., & Campos, P. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300733 · Cited by 475 · cited 1×
“Hence, studies [10, 13, 15, 40] recommend design interventions to enhance oversight and discourage automatic behavior, such as through responsible nudging [25, 105, 119].”
Toward human-centered AI: a perspective from human-computer interaction
Xu, W. (2019) · Interactions · doi:10.1145/3328485 · Cited by 454
The Role of Explanations on Trust and Reliance in Clinical Decision Support Systems
Bussone, A., Stumpf, S., & O'Sullivan, D. (2015) · 2015 International Conference on Healthcare Informatics · doi:10.1109/ichi.2015.26 · Cited by 441 · cited 2×
“However, legal and ethical concerns with high-stakes tasks necessitate AI to support, instead of automating human decision-making [12, 147].”
Cueing and Cognitive Conflict in Judge-Advisor Decision Making
Sniezek, J. A., & Buckley, T. (1995) · Organizational Behavior and Human Decision Processes · doi:10.1006/obhd.1995.1040 · Cited by 427 · cited 1×
“To study reliance, research uses an approach called Judge-Advisor-System (JAS) [140], where a human acts as judge and gets advice from AI.”
Human-Centered Artificial Intelligence: Three Fresh Ideas
University of Maryland, C. P., & Shneiderman, B. (2020) · AIS Transactions on Human-Computer Interaction · doi:10.17705/1thci.00131 · Cited by 421 · cited 1×
“In a nutshell, our work makes the following contributions towards the goal of understanding research on achieving appropriate reliance on AI advice: formation [56, 121, 137, 138, 166].”
InfluentialExplanations Can Reduce Overreliance on AI Systems During Decision-Making
Vasconcelos, H., Jörke, M., Grunde-McLaughlin, M., Gerstenberg, T., Bernstein, M. S., & Krishna, R. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579605 · Cited by 402
The Principles and Limits of Algorithm-in-the-Loop Decision Making
Green, B., & Chen, Y. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359152 · Cited by 392 · cited 6×
“…several approaches for studying human-AI reliance, mainly categorized under enhancing the AI systems (explanations, performance, uncertainty) [5, 54, 118, 127], understanding users (biases, expertise, self-assessment) [48, 108, 126, 144, 151], and designing interaction methods [9, 10, 15, 60].”
InfluentialCognitive Biases and Strategic Decision Processes: An Integrative Perspective
Das, T. K., & Teng, B. (1999) · Journal of Management Studies · doi:10.1111/1467-6486.00157 · Cited by 387 · cited 1×
“Studies in our corpus show that various biases could negatively influence the users’ decisions, undermining their self-efficacy and AI perceptions [18, 39, 115].”
Lai, V., & Tan, C. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287590 · Cited by 377 · cited 8×
“…[22], News Classification [112], Trip Planning [127], Logic Puzzles [144, 145], Exercise Recommendation [11] Hotel Review Classification [18, 61, 83, 132], Hand-written Digits Identification [27], General Image Recognition [146] Table 7: A summary of intervention mechanics (datasets, data…”
InfluentialAI in Finance: Challenges, Techniques, and Opportunities
Cao, L. (2023) · ACM Computing Surveys · doi:10.1145/3502289 · Cited by 366 · cited 1×
“Research [5, 60, 151] has explored methods to understand and enhance human-AI decision-making in diverse domains, including medical [17, 20, 71], finance [21, 28, 43], and justice [32, 52, 86].”
Xu, W., Dainoff, M. J., Ge, L., & Gao, Z. (2023) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2022.2041900 · Cited by 363 · cited 1×
“In a nutshell, our work makes the following contributions towards the goal of understanding research on achieving appropriate reliance on AI advice: formation [56, 121, 137, 138, 166].”
InfluentialA Review of User Interface Design for Interactive Machine Learning
Dudley, J. J., & Kristensson, P. O. (2018) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3185517 · Cited by 343 · cited 7×
“Understanding can also be improved through interactions [45, 121].”
InfluentialWang, X., & Yin, M. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450650 · Cited by 320 · cited 10×
“These studies use (experiment) conditions to test the effect of feature-based [60, 153, 156, 164], example-based [17, 23, 108, 146], model-based [8, 33, 55, 114], or general explanations methods [102, 117, 127].”
InfluentialProxy tasks and subjective measures can be misleading in evaluating explainable AI systems
Buçinca, Z., Lin, P., Gajos, K. Z., & Glassman, E. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377498 · Cited by 307 · cited 10×
“…three main directions for enhancing appropriate reliance on AI, namely 1) AI systems (e.g., by improving performance, uncertainty, explanations) [5, 9, 118, 127, 156, 168, 170], 2) user factors (e.g., focusing on expertise, cognitive biases, traits) [33, 63, 98, 108, 123, 144, 145, 151, 153], and…”
InfluentialKool, W., & Botvinick, M. (2018) · Nature Human Behaviour · doi:10.1038/s41562-018-0401-9 · Cited by 288 · cited 1×
“People weigh the potential benefits of cognitive effort against its perceived cost [78], a phenomenon also referred to as cognitive-effort-discounting [160].”
Kocielnik, R., Amershi, S., & Bennett, P. N. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300641 · Cited by 277 · cited 1×
“Artificial Intelligence (AI) systems continue to be fallible [77].”
Human-Centered Explainable AI: Towards a Reflective Sociotechnical Approach
Ehsan, U., & Riedl, M. O. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-60117-1_33 · Cited by 274
Mehrotra, S., Degachi, C., Vereschak, O., Jonker, C. M., & Tielman, M. L. (2024) · ACM Journal on Responsible Computing · doi:10.1145/3696449 · Cited by 254
Rational Choice and the Framing of Decisions
Tversky, A., & Kahneman, D. (1989) · Multiple Criteria Decision Making and Risk Analysis Using Microcomputers · doi:10.1007/978-3-642-74919-3_4 · Cited by 238
How to Evaluate Trust in AI-Assisted Decision Making? A Survey of Empirical Methodologies
Vereschak, O., Bailly, G., & Caramiaux, B. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3476068 · Cited by 229
Gamut: A Design Probe to Understand How Data Scientists Understand Machine Learning Models
Hohman, F., Head, A., Caruana, R., DeLine, R., & Drucker, S. M. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300809 · Cited by 225 · cited 1×
“One overlooked issue is the task-expertise mismatch, i.e., a misalignment between the complexity of the tasks and the end users’ expertise [68].”
AI based decision making: combining strategies to improve operational performance
Al-Surmi, A., Bashiri, M., & Koliousis, I. (2022) · International Journal of Production Research · doi:10.1080/00207543.2021.1966540 · Cited by 225
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610219 · Cited by 220
Design Frictions for Mindful Interactions: The Case for Microboundaries
Cox, A. L., Gould, S. J. J., Cecchinato, M. E., Iacovides, I., & Renfree, I. (2016) · Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/2851581.2892410 · Cited by 218 · cited 4×
“CFFs are used in frictional design [36, 105], which add delays or slow downs to force users to use (system 2) analytical thinking [13, 15, 71].”
InfluentialDeciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making
Rastogi, C., Zhang, Y., Wei, D., Varshney, K. R., Dhurandhar, A., & Tomsett, R. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512930 · Cited by 216 · cited 10×
“Research explores how users behave under certain conditions, such as by investigating their biases [107, 123] or (self) capabilities [11, 17].”
InfluentialAppropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations
Schemmer, M., Kuehl, N., Benz, C., Bartos, A., & Satzger, G. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584066 · Cited by 208 · cited 10×
“Research increasingly highlights the need to gain a foundational understanding of appropriate reliance on AI advice [13, 41, 81, 132] to improve human-AI decision-making scenarios.”
InfluentialHow and why humans trust: A meta-analysis and elaborated model
Hancock, P. A., Kessler, T. T., Kaplan, A. D., Stowers, K. L., Brill, J. C., Billings, D. R., Schaefer, K. E., & Szalma, J. L. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1081086 · Cited by 204 · cited 1×
“To understand trust, theoretical frameworks from psychology are used in various studies [4, 59, 87, 149, 150].”
Lai, V., Chen, C., Smith-Renner, A., Liao, Q. V., & Tan, C. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594087 · Cited by 204 · cited 10×
“Previously, Lai et al. [81] and Eckhardt et al. [49] explored empirical human-AI decision-making research, without an explicit focus on appropriate reliance.”
InfluentialLeichtmann, B., Humer, C., Hinterreiter, A., Streit, M., & Mara, M. (2023) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107539 · Cited by 200 · cited 1×
“The concept of trust has evolved over the years, from trust in automation (TiA) [65, 87] to trust in modern-day AI assistance [34, 74, 89, 154].”
How do visual explanations foster end users' appropriate trust in machine learning?
Yang, F., Huang, Z., Scholtz, J., & Arendt, D. L. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377480 · Cited by 200 · cited 10×
“…[22, 40, 60– 63, 112, 127], Affinity for Technology [60–63, 112, 127] Others Understanding [9, 24, 32, 33, 43, 60, 76, 88, 94, 96, 112, 156, 164, 167], Confidence Rating [15, 17, 23, 24, 27, 29, 32, 40, 55, 76, 93, 94, 96–98, 112, 114, 117, 132, 167], Usefulness/helpfulness [5, 9, 15, 17, 18,…”
InfluentialInteractive machine teaching: a human-centered approach to building machine-learned models
Ramos, G., Meek, C., Simard, P., Suh, J., & Ghorashi, S. (2020) · Human-Computer Interaction · doi:10.1080/07370024.2020.1734931 · Cited by 180 · cited 1×
“Existing work [45, 122] also highlights that running complex tasks with crowd workers may not be entirely representative.”
From explainable to interactive AI: A literature review on current trends in human-AI interaction
Raees, M., Meijerink, I., Lykourentzou, I., Khan, V.-J., & Papangelis, K. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103301 · Cited by 180 · cited 8×
“Understanding can also be improved through interactions [45, 121].”
InfluentialSelf-citationMiller, T. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594001 · Cited by 172 · cited 3×
“For instance, Cabitza et al. [15] posit providing possible decisions instead of direct advice for user engagement, resonating with the psychological aspects of reflective [109] and evaluative AI [101].”
Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-Making
Schoeffer, J., De‐Arteaga, M., & Kühl, N. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642621 · Cited by 164
Wysocki, O., Davies, J. K., Vigo, M., Armstrong, A. C., Landers, D., Lee, R., & Freitas, A. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2022.103839 · Cited by 161 · cited 10×
“These studies use (experiment) conditions to test the effect of feature-based [60, 153, 156, 164], example-based [17, 23, 108, 146], model-based [8, 33, 55, 114], or general explanations methods [102, 117, 127].”
InfluentialMa, S., Wang, X., Lei, Y., Shi, C., Yin, M., & Ma, X. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642671 · Cited by 153 · cited 10×
“%) of the studies use AI performance metrics like model confidence and uncertainty [27, 28, 98, 118], as well as accuracy [168, 170] to understand user behavior.”
InfluentialKim, S. S. Y., Liao, Q. V., Vorvoreanu, M., Ballard, S., & Vaughan, J. W. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658941 · Cited by 147
Wischnewski, M., Krämer, N., & Müller, E. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581197 · Cited by 146
Ma, S., Lei, Y., Wang, X., Zheng, C., Shi, C., Yin, M., & Ma, X. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581058 · Cited by 143 · cited 10×
“…33, 43, 60, 76, 88, 94, 96, 112, 156, 164, 167], Confidence Rating [15, 17, 23, 24, 27, 29, 32, 40, 55, 76, 93, 94, 96–98, 112, 114, 117, 132, 167], Usefulness/helpfulness [5, 9, 15, 17, 18, 22, 33, 62, 63, 88, 96, 97, 108, 112, 144, 164], Mental Demand (and/or Task Load) [9, 10, 29, 61, 88, 96–98,”
InfluentialInkpen, K., Chappidi, S., Mallari, K., Nushi, B., Ramesh, D., Michelucci, P., Mandava, V., Vepřek, L. H., & Quinn, G. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3534561 · Cited by 132 · cited 1×
“(2) Human-AI Decision-Making: Articles that studied human-AI assistance, but did not focus on or measure objective user behavior for reliance measures [69] (i.e., only capturing users’ trust perception/feedback [124, 158]) were excluded.”
Human Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks
Lu, Z., & Yin, M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445562 · Cited by 131 · cited 10×
“…[33, 63], Cognitive Cost (Effort) [151] Positive Task Difficulty [23, 127, 151], Negative First Impression [61, 108, 153], Poor Self Assessment [63, 94], Lack of Domain Knowledge [103, 156], Feature-based XAI [30, 170] Negative Example-based XAI [10, 29, 131, 156], Feature-based XAI [29, 133,…”
InfluentialHow Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review
Bertrand, A., Belloum, R., Eagan, J. R., & Maxwell, W. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534164 · Cited by 129 · cited 2×
“Existing reviews in HCAI can be broadly categorized into XAI [2, 6, 92], trust [100, 162], interactions (i.e., interactive Machine Learning (ML)) [45, 121], or related domains [1, 7, 49, 81].”
An Evaluation of the Human-Interpretability of Explanation
Lage, I., Chen, E., He, J., Narayanan, M., Kim, B., Gershman, S., & Doshi-Velez, F. (2019) · arXiv · doi:10.48550/arxiv.1902.00006 · Cited by 125 · cited 1×
“Still, designing XAI to reduce biases and lack of expertise continues to be a challenge in human-AI reliance research studies [80, 104, 107].”
Lustig, C., Pine, K., Nardi, B. A., Irani, L. C., Lee, M. K., Nafus, D., & Sandvig, C. (2016) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/2851581.2886426 · Cited by 124 · cited 1×
“Our review identifies that interaction should deter the users’ loss of skills (over-dependence) and blind faith in AI due to its alleged exertion [15, 95].”
"Why is 'Chicago' deceptive?" Towards Building Model-Driven Tutorials for Humans
Lai, V., Liu, H., & Tan, C. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376873 · Cited by 123
The Impact of Imperfect XAI on Human-AI Decision-Making
Morrison, K., Spitzer, P., Turri, V., Feng, M., Kühl, N., & Perer, A. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3641022 · Cited by 122
I can do better than your AI: expertise and explanations
Schaffer, J., O'Donovan, J., Michaelis, J., Raglin, A., & Höllerer, T. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302308 · Cited by 119 · cited 1×
“However, explanations of AI systems are often not enough to make users rely effectively on AI assistance [5, 129].”
Rams, hounds and white boxes: investigating human–AI collaboration protocols in medical diagnosis
Cabitza, F., Campagner, A., Ronzio, L., Cameli, M., Mandoli, G. E., Pastore, M. C., Sconfienza, L. M., Folgado, D., Barandas, M., & Gamboa, H. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102506 · Cited by 117 · cited 1×
“For instance, engaging users to present multiple and contrastive examples instead of AI’s advice on the decision implements cognitive friction while reducing disengagement [11, 14].”
Dikmen, M., & Burns, C. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102792 · Cited by 116 · cited 10×
“Nevertheless, the exact way that people engage with AI may still depend on situational factors (e.g., time pressure) [144], traits (e.g., expertise) [43, 120], or individuality (e.g., motivation [10]).”
InfluentialNourani, M., King, J., & Ragan, E. (2020) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v8i1.7469 · Cited by 116
Springer, A., & Whittaker, S. (2019) · Proceedings of the 24th International Conference on Intelligent User Interfaces · doi:10.1145/3301275.3302322 · Cited by 115 · cited 1×
“Reflections can include partial explanations, progressive disclosure, layering information, or thought-provoking questioning [142].”
Is Conversational XAI All You Need? Human-AI Decision Making With a Conversational XAI Assistant
He, G., Aishwarya, N., & Gadiraju, U. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712133 · Cited by 115 · cited 9×
“These studies use (experiment) conditions to test the effect of feature-based [60, 153, 156, 164], example-based [17, 23, 108, 146], model-based [8, 33, 55, 114], or general explanations methods [102, 117, 127].”
InfluentialHuman Decision Making with Machine Assistance: An Experiment on Bailing and Jailing
Grgić-Hlača, N., Engel, C., & Gummadi, K. P. (2019) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3359280 · Cited by 113 · cited 10×
“These studies use (experiment) conditions to test the effect of feature-based [60, 153, 156, 164], example-based [17, 23, 108, 146], model-based [8, 33, 55, 114], or general explanations methods [102, 117, 127].”
InfluentialThe effects of explanations on automation bias
Vered, M., Livni, T., Howe, P. D. L., Miller, T., & Sonenberg, L. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2023.103952 · Cited by 111 · cited 10×
“These studies use (experiment) conditions to test the effect of feature-based [60, 153, 156, 164], example-based [17, 23, 108, 146], model-based [8, 33, 55, 114], or general explanations methods [102, 117, 127].”
InfluentialKnowing About Knowing: An Illusion of Human Competence Can Hinder Appropriate Reliance on AI Systems
He, G., Kuiper, L., & Gadiraju, U. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581025 · Cited by 107 · cited 10×
“A few studies [33, 61, 63] looked at providing XAI tutorials to convey AI performance and enhance users’ abilities.”
InfluentialAnchoring Bias Affects Mental Model Formation and User Reliance in Explainable AI Systems
Nourani, M., Roy, C., Block, J. E., Honeycutt, D. R., Rahman, T., Ragan, E., & Gogate, V. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450639 · Cited by 107 · cited 10×
“These studies use (experiment) conditions to test the effect of feature-based [60, 153, 156, 164], example-based [17, 23, 108, 146], model-based [8, 33, 55, 114], or general explanations methods [102, 117, 127].”
InfluentialA rational model of the Dunning–Kruger effect supports insensitivity to evidence in low performers
Jansen, R. A., Rafferty, A. N., & Griffiths, T. L. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01057-0 · Cited by 100 · cited 1×
“Evaluating one’s performance is important as people tend to estimate their abilities poorly [46, 70].”
Rechkemmer, A., & Yin, M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501967 · Cited by 97
Chiang, C.-W., Lu, Z., Li, Z., & Yin, M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581015 · Cited by 97 · cited 10×
“…96–98, 102, 112, 114, 127, 168, 170], AI-effect on Accuracy [5, 10, 11, 13, 15, 40, 43, 114, 131, 164, 170], Accuracy [8, 9, 11, 17, 18, 23, 27–29, 32, 40, 43, 60–63, 71, 76, 83, 88, 93, 96, 98, 108, 112, 118, 123, 127, 132, 133, 144–146, 156], Over-reliance [10, 27, 29, 32, 93, 96, 98, 103, 133,…”
InfluentialImproving Human-AI Collaboration With Descriptions of AI Behavior
Cabrera, Á. A., Perer, A., & Hong, J. I. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579612 · Cited by 87 · cited 10×
“…33, 43, 60, 76, 88, 94, 96, 112, 156, 164, 167], Confidence Rating [15, 17, 23, 24, 27, 29, 32, 40, 55, 76, 93, 94, 96–98, 112, 114, 117, 132, 167], Usefulness/helpfulness [5, 9, 15, 17, 18, 22, 33, 62, 63, 88, 96, 97, 108, 112, 144, 164], Mental Demand (and/or Task Load) [9, 10, 29, 61, 88, 96–98,”
InfluentialCao, S., Liu, A., & Huang, C. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637318 · Cited by 87 · cited 10×
“…102, 151, 164, 167, 168, 170], Trust in Automation [22, 40, 60– 63, 112, 127], Affinity for Technology [60–63, 112, 127] Others Understanding [9, 24, 32, 33, 43, 60, 76, 88, 94, 96, 112, 156, 164, 167], Confidence Rating [15, 17, 23, 24, 27, 29, 32, 40, 55, 76, 93, 94, 96–98, 112, 114, 117,…”
InfluentialYou Complete Me: Human-AI Teams and Complementary Expertise
Zhang, Q., Lee, M. L., & Carter, S. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517791 · Cited by 85 · cited 1×
“Human-AI CTP is defined as the situation where humans and AI systems collaboratively achieve a performance superior to their individual performances [41, 134, 169].”
Prabhudesai, S., Yang, L., Asthana, S., Huan, X., Liao, Q. V., & Banovic, N. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584033 · Cited by 82
Lee, M. H., & Chew, C. J. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610218 · Cited by 78
Does More Advice Help? The Effects of Second Opinions in AI-Assisted Decision Making
Lu, Z., Wang, D., & Yin, M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3653708 · Cited by 76 · cited 10×
“Lu et al. [93] pointed out that active solicitation can also help reduce over-reliance, for instance, by getting a second opinion to form a consensus.”
InfluentialBuçinca, Z., Swaroop, S., Paluch, A. E., Doshi‐Velez, F., & Gajos, K. Z. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713229 · Cited by 75 · cited 9×
“For instance, engaging users to present multiple and contrastive examples instead of AI’s advice on the decision implements cognitive friction while reducing disengagement [11, 14].”
InfluentialEffects of Explanations in AI-Assisted Decision Making: Principles and Comparisons
Wang, X., & Yin, M. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3519266 · Cited by 74 · cited 1×
“Many studies [5, 10, 102, 156, 157, 170] report that explanations increase users’ over-reliance on AI advice.”
Explainability pitfalls: beyond dark patterns in explainable AI
Ehsan, U., & Riedl, M. O. (2024) · Patterns · doi:10.1016/j.patter.2024.100971 · Cited by 72 · cited 2×
“While nudging and frictions can induce behavior change, research emphasizes that they must be applied carefully to avoid manipulating user behavior through dark patterns or reducing disconnect from AI advice (e.g., due to algorithmic aversion) [16, 51].”
Understanding User Reliance on AI in Assisted Decision-Making
Cao, S., & Huang, C.-M. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555572 · Cited by 71 · cited 9×
“…Second Opinion [32, 93], Natural Language XAI [76, 103], Interactive Tutorials [33, 63], Cognitive Cost (Effort) [151] Positive Task Difficulty [23, 127, 151], Negative First Impression [61, 108, 153], Poor Self Assessment [63, 94], Lack of Domain Knowledge [103, 156], Feature-based XAI [30,…”
InfluentialHe, G., Buijsman, S., & Gadiraju, U. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610067 · Cited by 71 · cited 10×
“…[33], RAIR [13, 60–63, 103, 127, 131, 132, 145], RSR [13, 60–63, 103, 127, 131, 132], AI Safety [13], Deception of Reliance [103], Accuracy-wid [60, 62, 63, 98, 127] Trust Subjective Trust Rating [8–10, 18, 23, 43, 76, 83, 88, 94, 96, 97, 102, 151, 164, 167, 168, 170], Trust in Automation [22, 40,…”
InfluentialCognitive bias, decision styles, and risk attitudes in decision making and DSS
Phillips-Wren, G. E., Power, D., & Mora, M. (2019) · Journal of Decision System · doi:10.1080/12460125.2019.1646509 · Cited by 70 · cited 1×
“Studies in our corpus show that various biases could negatively influence the users’ decisions, undermining their self-efficacy and AI perceptions [18, 39, 115].”
Capable but Amoral? Comparing AI and Human Expert Collaboration in Ethical Decision Making
Tolmeijer, S., Christen, M., Kandul, S., Kneer, M., & Bernstein, A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517732 · Cited by 70
Second Chance for a First Impression? Trust Development in Intelligent System Interaction
Tolmeijer, S., Gadiraju, U., Ghantasala, R., Gupta, A., & Bernstein, A. (2021) · Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3450613.3456817 · Cited by 67 · cited 1×
“Hence, the combined study of human-related aspects (e.g., biases) [42, 148], attributes of the AI systems (e.g., XAI) [90, 94, 124], and characteristics of the decision-making tasks [54, 126] is essential to understand appropriate reliance.”
An extension of the theory of technology dominance: Capturing the underlying causal complexity
Sutton, S. G., Arnold, V., & Holt, M. (2023) · International Journal of Accounting Information Systems · doi:10.1016/j.accinf.2023.100626 · Cited by 63
Cabitza, F., Fregosi, C., Campagner, A., & Natali, C. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-63803-9_14 · Cited by 59
Peng, L., Li, D., Zhang, Z., Zhang, T., Huang, A., Yang, S., & Hu, Y. (2024) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2024.103629 · Cited by 59 · cited 9×
“These studies use (experiment) conditions to test the effect of feature-based [60, 153, 156, 164], example-based [17, 23, 108, 146], model-based [8, 33, 55, 114], or general explanations methods [102, 117, 127].”
InfluentialNever tell me the odds: Investigating pro-hoc explanations in medical decision making
Cabitza, F., Natali, C., Famiglini, L., Campagner, A., Caccavella, V. M., & Gallazzi, E. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102819 · Cited by 58 · cited 10×
“These studies use (experiment) conditions to test the effect of feature-based [60, 153, 156, 164], example-based [17, 23, 108, 146], model-based [8, 33, 55, 114], or general explanations methods [102, 117, 127].”
InfluentialExploring the effects of human-centered AI explanations on trust and reliance
Scharowski, N., Perrig, S. A. C., Svab, M., Opwis, K., & Brühlmann, F. (2023) · Frontiers in Computer Science · doi:10.3389/fcomp.2023.1151150 · Cited by 57 · cited 3×
“Initial works [65] mostly studied users’ trust in automation; however, trust measures do not necessarily enhance reliance [131], which is gaining attention in recent work [81, 130].”
AI Reliance and Decision Quality: Fundamentals, Interdependence, and the Effects of Interventions
Schoeffer, J., Jakubik, J., Vössing, M., Kühl, N., & Satzger, G. (2025) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15873 · Cited by 55 · cited 2×
“Finally, some studies [57, 134] measure reliance as merely the presence of an AI system (i.e., deciding with and without AI systems in separate conditions), which falls out of the scope of this work.”
AI-Assisted Decision-making: a Cognitive Modeling Approach to Infer Latent Reliance Strategies
Tejeda, H., Kumar, A., Smyth, P., & Steyvers, M. (2022) · Computational Brain & Behavior · doi:10.1007/s42113-022-00157-y · Cited by 53 · cited 10×
“These studies use (experiment) conditions to test the effect of feature-based [60, 153, 156, 164], example-based [17, 23, 108, 146], model-based [8, 33, 55, 114], or general explanations methods [102, 117, 127].”
InfluentialProviding information for decision making: Contrasting description and simulation.
Hogarth, R. M., & Soyer, E. (2014) · Journal of Applied Research in Memory and Cognition · doi:10.1016/j.jarmac.2014.01.005 · Cited by 52 · cited 1×
“” Hence, discriminating between correct and incorrect advice is not sufficient, but humans also need to adapt their decisions accordingly to enhance human-AI CTP [67].”
Leveraging rationales to improve human task performance
Das, D., & Chernova, S. (2020) · Proceedings of the 25th International Conference on Intelligent User Interfaces · doi:10.1145/3377325.3377512 · Cited by 52 · cited 1×
“However, when users lack expertise in a task, they tend to (inappropriately) rely heavily on AI advice, especially as the task becomes more complex [38, 127].”
Effects of AI and Logic-Style Explanations on Users’ Decisions Under Different Levels of Uncertainty
Cau, F. M., Hauptmann, H., Spano, L. D., & Tintarev, N. (2023) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3588320 · Cited by 52 · cited 10×
“…96, 98, 108, 112, 118, 123, 127, 132, 133, 144–146, 156], Over-reliance [10, 27, 29, 32, 93, 96, 98, 103, 133, 144, 145, 151, 167], Under-reliance [27, 32, 93, 96, 98, 103, 133, 167], Human Error [10], Weight of Advice [13, 33, 117, 146], Absolute Percentage Error [33], RAIR [13, 60–63, 103, 127,…”
InfluentialCabitza, F., Campagner, A., Angius, R., Natali, C., & Reverberi, C. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581095 · Cited by 51 · cited 10×
“Various methods use other measures, such as user agreeableness, or switching fractions with AI advice [13, 132, 151], as elaborated in Section 4.”
InfluentialKahr, P. K., Rooks, G., Willemsen, M. C., & Snijders, C. C. P. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3686164 · Cited by 49
Accuracy-Time Tradeoffs in AI-Assisted Decision Making under Time Pressure
Swaroop, S., Buçinca, Z., Gajos, K. Z., & Doshi-Velez, F. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645206 · Cited by 49 · cited 10×
“…several approaches for studying human-AI reliance, mainly categorized under enhancing the AI systems (explanations, performance, uncertainty) [5, 54, 118, 127], understanding users (biases, expertise, self-assessment) [48, 108, 126, 144, 151], and designing interaction methods [9, 10, 15, 60].”
InfluentialHow Time Pressure in Different Phases of Decision-Making Influences Human-AI Collaboration
Cao, S., Gomez, C., & Huang, C.-M. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610068 · Cited by 48
Ma, S., Chen, Q., Wang, X., Zheng, C., Peng, Z., Yin, M., & Ma, X. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713423 · Cited by 48 · cited 9×
“Partial explanations align with a larger body of research on deliberation that encourages users to reflect on their decisions [40, 96].”
InfluentialNudging through Friction: An Approach for Calibrating Trust in Explainable AI
Naiseh, M., Al-Mansoori, R. S., Al‐Thani, D., Jiang, N., & Ali, R. (2021) · International Conference on Behavioral, Economic, and Socio-Cultural Computing · doi:10.1109/besc53957.2021.9635271 · Cited by 46 · cited 2×
“CFFs are used in frictional design [36, 105], which add delays or slow downs to force users to use (system 2) analytical thinking [13, 15, 71].”
Supporting High-Uncertainty Decisions through AI and Logic-Style Explanations
Cau, F. M., Hauptmann, H., Spano, L. D., & Tintarev, N. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584080 · Cited by 43
Eckhardt, S., Kühl, N., Dolata, M., & Schwabe, G. (2025) · ACM Computing Surveys · doi:10.1145/3776528 · Cited by 43
Chiang, C.-W., & Yin, M. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511121 · Cited by 42
A Missing Piece in the Puzzle: Considering the Role of Task Complexity in Human-AI Decision Making
Salimzadeh, S., He, G., & Gadiraju, U. (2023) · Proceedings of the 31st ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3565472.3592959 · Cited by 42
Effect of Explanation Conceptualisations on Trust in AI-assisted Credibility Assessment
Pareek, S., van Berkel, N., Velloso, E., & Goncalves, J. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686922 · Cited by 42 · cited 10×
“Studies also showed that confirming decisions by expert or empirical validation can be useful, in the form of conceptual validations (CVs) [112].”
InfluentialSalimzadeh, S., He, G., & Gadiraju, U. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641905 · Cited by 41
How Do HCI Researchers Study Cognitive Biases? A Scoping Review
Boonprakong, N., Tag, B., Gonçalves, J., & Dingler, T. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713450 · Cited by 40 · cited 2×
“Boonprakong et al. [7] conducted a review that focuses on how researchers study various types of cognitive biases in automated/computing systems.”
Exploring a Behavioral Model of “Positive Friction” in Human-AI Interaction
Chen, Z., & Schmidt, R. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-61353-1_1 · Cited by 40
Wang, X., & Yin, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581366 · Cited by 38 · cited 1×
“(2) Human-AI Decision-Making: Articles that studied human-AI assistance, but did not focus on or measure objective user behavior for reliance measures [69] (i.e., only capturing users’ trust perception/feedback [124, 158]) were excluded.”
Evaluating the impact of human explanation strategies on human-AI visual decision-making
Morrison, K., Shin, D., Holstein, K., & Perer, A. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579481 · Cited by 37 · cited 9×
“These studies use (experiment) conditions to test the effect of feature-based [60, 153, 156, 164], example-based [17, 23, 108, 146], model-based [8, 33, 55, 114], or general explanations methods [102, 117, 127].”
Influentialde Jong, S., Paananen, V., Tag, B., & van Berkel, N. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710946 · Cited by 32 · cited 10×
“…used subjective metrics include the users’ perceived trust (Likert scales [102] (18 studies), or Trust in Automation [60] (8 studies)), confidence [40] (20 studies), usefulness [15] (16 studies), understanding of advice [112] (14 studies), the users’ mental demand [10] (9 studies), satisfaction…”
InfluentialAI-Moderated Decision-Making: Capturing and Balancing Anchoring Bias in Sequential Decision Tasks
Echterhoff, J. M., Yarmand, M., & McAuley, J. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517443 · Cited by 30 · cited 1×
“…several approaches for studying human-AI reliance, mainly categorized under enhancing the AI systems (explanations, performance, uncertainty) [5, 54, 118, 127], understanding users (biases, expertise, self-assessment) [48, 108, 126, 144, 151], and designing interaction methods [9, 10, 15, 60].”
A Decision Theoretic Framework for Measuring AI Reliance
Guo, Z., Wu, Y., Hartline, J. D., & Hullman, J. (2024) · The 2024 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3630106.3658901 · Cited by 27 · cited 1×
“Finally, some studies [57, 134] measure reliance as merely the presence of an AI system (i.e., deciding with and without AI systems in separate conditions), which falls out of the scope of this work.”
The effect of individual and task characteristics on decision aid reliance
Parkes, A. (2017) · Behaviour & Information Technology · doi:10.1080/0144929x.2016.1209242 · Cited by 26 · cited 1×
“Task complexity increases information overload pertaining to the user’s perception and capabilities [113].”
To Err Is AI! Debugging as an Intervention to Facilitate Appropriate Reliance on AI Systems
He, G., Bharos, A., & Gadiraju, U. (2024) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3648188.3675130 · Cited by 24 · cited 8×
“…XAI [76, 103], Interactive Tutorials [33, 63], Cognitive Cost (Effort) [151] Positive Task Difficulty [23, 127, 151], Negative First Impression [61, 108, 153], Poor Self Assessment [63, 94], Lack of Domain Knowledge [103, 156], Feature-based XAI [30, 170] Negative Example-based XAI [10, 29,…”
InfluentialReflection machines: increasing meaningful human control over Decision Support Systems
Cornelissen, N. A. J., van Eerdt, R. J. M., Schraffenberger, H., & Haselager, P. (2022) · Ethics and Information Technology · doi:10.1007/s10676-022-09645-y · Cited by 20
Personalising AI Assistance Based on Overreliance Rate in AI-Assisted Decision Making
Swaroop, S., Buçinca, Z., Gajos, K. Z., & Doshi-Velez, F. (2025) · Proceedings of the 30th International Conference on Intelligent User Interfaces · doi:10.1145/3708359.3712128 · Cited by 20 · cited 10×
“Most studies rely on fixed reliance metrics (i.e., for each interaction), and user reliance development over time is under-explored [22, 40, 145].”
InfluentialMuijlwijk, H., Willemsen, M. C., Smyth, B., & IJsselsteijn, W. A. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640543.3645205 · Cited by 19 · cited 2×
“In fact, AI performance is often different when using simulated versus realistic or representative tasks, superficial versus real-world situations, and simple versus complex tasks [104, 127].”
Understanding and Explaining Diagnostic Paths: Toward Augmented Decision Making
Plass, M., Kargl, M., Nitsche, P., Jungwirth, E., Holzinger, A., & Müller, H. (2022) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2022.3197957 · Cited by 16 · cited 1×
“Users are better at calibrating their reliance on AI by reasoning or building causality [116] rather than using statistical or associative explanations.”
The Influence of Curiosity Traits and On-Demand Explanations in AI-Assisted Decision-Making
Cau, F. M., & Spano, L. D. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712165 · Cited by 15 · cited 10×
“…33, 43, 60, 76, 88, 94, 96, 112, 156, 164, 167], Confidence Rating [15, 17, 23, 24, 27, 29, 32, 40, 55, 76, 93, 94, 96–98, 112, 114, 117, 132, 167], Usefulness/helpfulness [5, 9, 15, 17, 18, 22, 33, 62, 63, 88, 96, 97, 108, 112, 144, 164], Mental Demand (and/or Task Load) [9, 10, 29, 61, 88, 96–98,”
Influential“Reliance and automation for Human-AI collaborative data labeling conflict resolution,”
Brachman, M., Ashktorab, Z., Desmond, M., Duesterwald, E., Dugan, C., Joshi, N. N., Pan, Q., & Sharma, A. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3555212 · Cited by 14 · cited 10×
“These studies use (experiment) conditions to test the effect of feature-based [60, 153, 156, 164], example-based [17, 23, 108, 146], model-based [8, 33, 55, 114], or general explanations methods [102, 117, 127].”
InfluentialUnderstanding the Effects of Explanation Types and User Motivations on Recommender System Use
Li, Q., Chu, S. L., Rao, N., & Nourani, M. (2020) · Proceedings of the AAAI Conference on Human Computation and Crowdsourcing · doi:10.1609/hcomp.v8i1.7466 · Cited by 12 · cited 1×
“Hence, the combined study of human-related aspects (e.g., biases) [42, 148], attributes of the AI systems (e.g., XAI) [90, 94, 124], and characteristics of the decision-making tasks [54, 126] is essential to understand appropriate reliance.”
Cabitza, F., Famiglini, L., Fregosi, C., Pe, S., Parimbelli, E., La Maida, G. A., & Gallazzi, E. (2025) · Proceedings of the 30th International Conference on Intelligent User Interfaces · doi:10.1145/3708359.3712157 · Cited by 11 · cited 10×
“Cognitive forcing [10, 40] and frictional methods [13, 15] are reported to reduce over-reliance.”
InfluentialJeon, Y., Hwang, C., & Chen, X. '. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712161 · Cited by 10 · cited 10×
“Researchers also explore calibrating the users’ capabilities to curb under-reliance [24, 71, 97, 98] by making users aware of their biases.”
InfluentialRaees, M., Khan, V.-J., & Papangelis, K. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3699682.3728320 · Cited by 9 · cited 1×
“Hence, studies [10, 13, 15, 40] recommend design interventions to enhance oversight and discourage automatic behavior, such as through responsible nudging [25, 105, 119].”
Self-citationTowards Understanding Persuasive and Personalized Engagement for Human-AI Reliance
Raees, M., Khan, V.-J., & Papangelis, K. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3708319.3733643 · Cited by 9 · cited 1×
“Nevertheless, the exact way that people engage with AI may still depend on situational factors (e.g., time pressure) [144], traits (e.g., expertise) [43, 120], or individuality (e.g., motivation [10]).”
Self-citationWang, L., Jamieson, G. A., & Hollands, J. G. (2008) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/154193120805200420 · Cited by 5 · cited 1×
“Enhancing the explainability of systems supports users in understanding those systems in the first place [155, 165].”
Hinrichs, M., Bui, T. B. D., & Schneegaß, S. (2024) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3701571.3701585 · Cited by 5 · cited 1×
“With step-wise examination of the task, users can build explanations and rationale with the system, which can help them contest, take ownership of their decisions, and establish critical engagement or mindful usage [64].”
Towards Understanding Human-AI Reliance Patterns Through Explanation Styles
Casolin, E. R., & Salim, F. D. (2024) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3675094.3678996 · Cited by 2 · cited 1×
““Objective reliance” is defined as a behavioral aspect with three patterns/views [26, 131], which are expressed in Table 2.”
Research works printed but not linked: 10
- Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:https://arXiv.org/abs/1702.08608 (2017). · arXiv
- Christoph Engel, Andreas Glöckner, and Sinika Timme. 2017. Defendant should have the last word: Experimentally manipulating order and provisional assessment of the facts in criminal procedure. Technical Report. Preprints of the Max Planck Institute for Research on Collective Goods.
- Robert R Hoffman Shane T Mueller Gary Klein and Jordan Litman. 2018. Metrics for explainable AI: Challenges and prospects. arXiv preprint arXiv:https://arXiv.org/abs/1812.04608 (2018). · arXiv
- Daniel Kahneman. 2011. Thinking, fast and slow. macmillan.
- Don Norman. 2013. The design of everyday things: Revised and expanded edition. Basic books.
- Lee Ross and Andrew Ward. 2013. Naive realism in everyday life: Implications for social conflict and misunderstanding. In Values and knowledge. Psychology Press, 103–135.
- Max Schemmer, Andrea Bartos, Philipp Spitzer, Patrick Hemmer, Niklas Kühl, Jonas Liebschner, and Gerhard Satzger. 2023. Towards effective human-AI decision-making: The role of human learning in appropriate reliance on AI advice. In Forty-Fourth International Conference on Information Systems (Hyderabad, India) (ICIS 2023 Proceedings). AISEL, 1–17. https://aisel.aisnet.org/icis2023/hti/hti/14
- Keng Siau and Weiyu Wang. 2018. Building trust in artificial intelligence machine learning and robotics. Cutter business technology journal 31 2 (2018) 47–53.
- Steve G Sutton Vicky Arnold and Matthew Holt. 2022. An extension of the theory of technology dominance: understanding the underlying nature causes and effects. Causes and effects (2022). · DOI
- Earl L Wiener. 1981. Complacency: Is the term useful for air safety. In Proceedings of the 26th corporate aviation safety seminar , Vol. 117. 116–125.
Cited by
From Trust to Appropriate Reliance: Measurement Constructs in Human-AI Decision-Making
Raees, M., & Papangelis, K. (2026) · arXiv · doi:10.48550/arxiv.2604.23896 · Cited by 5
Self-citationTrust to Reliance: Measurement Constructs for Human-AI Appropriate Reliance
Raees, M., & Papagelis, K. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798835 · Cited by 8
Self-citationRaees, M., Iwamoto, Y., Papangelis, K., Heard, J., & Tigwell, G. W. (2026) · arXiv · doi:10.48550/arxiv.2602.08937 · Cited by 2
“Similar patterns are evidenced by prior research using AI-generated content [36], where users tend to overestimate the capabilities of AI and not critically analyze or verify what they produce with AI [59].”
Self-citationRaees, M., & Papagelis, K. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798834 · Cited by 1
Self-citationWei, J., & Shang, Y. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1965371 · Cited by 1
“Educational work likewise models trust, reliance, resistance, and actual use as distinct constructs (Wang et al., 2025), while HCI reviews note heterogeneous definitions of appropriate reliance (Raees et al., 2026).”
Metadata
- Authors
- Raees, M., Khan, V.-J., Lykourentzou, I., & Papangelis, K.
- Affiliations
- Rochester Institute of Technology (US)
- Independent Researcher, Brussels, Belgium
- Utrecht University (NL)
- Rochester Institute of Technology, Rochester, New York, USA
- Venue
- Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2026-04-13
- Pages
- 1-24
- OpenAlex
- W7153972336
- Cited by
- 5
- References
- 157
- Citations by year
- 2026: 5
- FWCI
- 67.17
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- AI advice, AI reliance, human-AI decision-making, human-AI interaction, user engagement, decision-making tasks, intervention design, trust in AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.