Appropriate Reliance on AI Advice: Conceptualization and the Effect of Explanations
Open accessThe DOI matches its Crossref record; 83 of the 83 references it cites were found in a registry or an index; 23 of the 39 sources its claims cite were checked for support.
Abstract
AI advice is becoming increasingly popular, e.g., in investment and medical treatment decisions. As this advice is typically imperfect, decision-makers have to exert discretion as to whether actually follow that advice: they have to “appropriately” rely on correct and turn down incorrect advice. However, current research on appropriate reliance still lacks a common definition as well as an operational measurement concept. Additionally, no in-depth behavioral experiments have been conducted that help understand the factors influencing this behavior. In this paper, we propose Appropriateness of Reliance (AoR) as an underlying, quantifiable two-dimensional measurement concept. We develop a research model that analyzes the effect of providing explanations for AI advice. In an experiment with 200 participants, we demonstrate how these explanations influence the AoR, and, thus, the effectiveness of AI advice. Our work contributes fundamental concepts for the analysis of reliance behavior and the purposeful design of AI advisors.
Overview
CoChat Summary
TL;DR
The paper proposes Appropriateness of Reliance (AoR) as a two-dimensional metric—relative AI reliance (RAIR) and relative self-reliance (RSR)—for measuring whether decision-makers appropriately follow correct AI advice and reject incorrect AI advice. In a 200-participant hotel-review experiment, feature-importance explanations significantly increased RAIR but not RSR, and the human-AI team did not reach complementary team performance, so appropriate reliance was not displayed.
Key findings
- AoR is defined as the tuple of RAIR and RSR; appropriate reliance (AR) is defined as any AoR tuple that leads to complementary team performance (CTP).
- In the control condition, RAIR was 29.59% and RSR was 71.87%; with feature-importance explanations, RAIR rose significantly to 38.87% (t = -1.95, p = 0.05), while RSR did not change significantly (69.45%; t = 0.61, p = 0.54).
- Explanations significantly increased the change in self-confidence (t = -2.33, p = 0.02), which partially mediated the effect on RAIR; explanations had no significant effect on trust (H4 not supported), but trust increased RAIR and decreased RSR (H5a and H5b supported).
- CTP was not reached: down-sampled AI performance was 50% in both conditions; human accuracy was 55.50% (control) and 53.85% (feature importance), and AI-assisted accuracy was 53.94% (control) and 56.30% (feature importance), with human-AI team performance not significantly different from human accuracy.
Methods
- Between-subject online experiment with 200 participants recruited via Prolific; one participant in the feature-importance condition was excluded for a failed manipulation check, and SEM analyses used a final sample of 199.
- Participants classified 16 deceptive or genuine hotel reviews sequentially: first without AI advice and with a confidence rating, then after receiving AI advice (SVM with 86% accuracy) either alone or with LIME feature-importance explanations, then revised their decision and confidence.
- Analysis used descriptive t-tests and structural equation modeling (SEM) with the MLR estimator in lavaan, measuring RAIR, RSR, trust, and change in self-confidence.
Limitations
- The paper states that deception detection is difficult for humans, that generalizability is limited by the choice of explanations, that the concept is limited to classification tasks, and that the sequential task setup may change the task, induce anchoring, reduce over-reliance, and may not always be possible or desired in real-world settings; crowd workers also limit generalizability.
- Evident from the text: the sample was relatively young (mean age 27.5, SD 8.5) and drawn from an online platform, and the experimental AI’s down-sampled performance was 50% in both conditions, which may constrain conclusions about CTP and appropriate reliance.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




![Fig. 5 Online experiment graphical user interface for the feature importance condition. The ground truth of the exemplarily shown hotel review is 'fake'. The design of the interface is adapted from Lai et al. [42].](https://figures.cochat.ai/paper-figures/aa69c3bc1d844572931891d4e00e8f4d/appropriate-reliance-on-ai-advice-conceptualization-and-the-effect-of-figure-5.png)


Verification
Claims the paper makes
17 Valid · 3 Not valid · 10 Not checked · 54 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“The sequential task setup could induce an anchoring effect which prevents the human to more actively take the AI into account [9].”
What the sources say
- Disputedcites [9] [9] Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z Gajos. 2021. To trust or to think: cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1-21. - sourceExperimental — accuracy not yet measured
Section 3.2.2 (p. 7) describes a sequence in which people decide before seeing the AI and hypothesizes that it will motivate them to engage with the AI explanation, not prevent engagement. Section 2.2 (p. 4) instead associates anchoring with showing the AI recommendation before the person makes an independent decision. (LLM, conf=0.84)
Checked by a reasoning model — may be wrong.
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Every source the paper cites for this claim was checked and supports it.
In the paper
“Understanding is crucial in building trust [23].”
What the sources say
- Supportedcites [23] [23] Leilani H Gilpin, Cecilia Testart, Nathaniel Fruchter, and Julius Adebayo. 2019. Explaining explanations to society. arXiv preprint arXiv:1901.06560 (2019). - source
The Discussion and Conclusion (p. 5) says that truly trusting AI systems requires people to feel confident they understand how decisions are made. It also says they need recourse, so understanding is crucial but not sufficient on its own. (LLM, conf=0.92)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Confidence in one's own decision is a key mechanism underlying advice acceptance [14, 73].”
What the sources say
- Supportedcites [14] [14] Leah Chong, Guanglu Zhang, Kosa Goucher-Lambert, Kenneth Kotovsky, and Jonathan Cagan. 2022. Human confidence in artificial intelligence and in themselves: The evolution and impact of confidence on adoption of AI advice. Computers in Human Behavior 127 (2022), 107018. - source
The Discussion reports that self-confidence significantly contributes to whether people accept AI suggestions and describes it as a driving factor. The Conclusion likewise reports a significant impact of self-confidence on accepting or rejecting those suggestions. (LLM, conf=0.90)
- Supportedcites [73] [73] Xiuxin Wang and Xiufang Du. 2018. Why does advice discounting occur? The combined roles of confidence and trust. Frontiers in Psychology 9 (2018), 2381. - source
The Introduction calls confidence in one’s own judgment a key mechanism underlying advice taking. The General Discussion reports that advice discounting decreased when confidence decreased and that confidence helped predict advice taking. Page numbers are not provided in our copy. (LLM, conf=0.94)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“We would like to emphasize that deception detection is a difficult task for humans [42, 43].”
What the sources say
- Supportedcites [42] [42] Vivian Lai, Han Liu, and Chenhao Tan. 2020. " Why is' Chicago'deceptive?" Towards Building Model-Driven Tutorials for Humans. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems . 1-13. - source
The Deception detection section (p. 3) says it is well recognized that humans struggle to detect deception. The Discussion (p. 10) likewise describes deception detection as potentially too challenging for humans. (LLM, conf=0.94)
- Supportedcites [43] [43] Vivian Lai and Chenhao Tan. 2019. On human predictions with explanations and predictions of machine learning models: A case study on deception detection. In Proceedings of the conference on fairness, accountability, and transparency . 29-38. - source
The paper explicitly calls deception detection “a challenging task for humans” in both its introductory discussion (p. 2) and Related Work (p. 3). This directly supports the citing sentence’s paraphrase. (LLM, conf=0.95)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“While this may sound disappointing at first, it also shows that the claim that explanations would reduce overreliance [6, 9] does not seem to hold for all kinds of tasks.”
What the sources say
- Supportedcites [6] [6] Gagan Bansal, Tongshuang Wu, Joyce Zhou, Raymond Fok, Besmira Nushi, Ece Kamar, Marco Tulio Ribeiro, and Daniel Weld. 2021. Does the whole exceed its parts? the effect of ai explanations on complementary team performance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems . 1-16. - source
The Introduction (p. 2) reports that explanations increased reliance on AI recommendations even when the AI was incorrect. Section 6.2 (p. 12) describes this as increased blind trust rather than appropriate reliance, supporting the qualified claim that explanations do not reduce overreliance in all tasks. (LLM, conf=0.93)
- Supportedcites [9] [9] Zana Buçinca, Maja Barbara Malaya, and Krzysztof Z Gajos. 2021. To trust or to think: cognitive forcing functions can reduce overreliance on AI in AI-assisted decision-making. Proceedings of the ACM on Human-Computer Interaction 5, CSCW1 (2021), 1-21. - source
The Introduction (p. 2) says explanations were intended to reduce overreliance but have not had substantial success. The Discussion (p. 16) reports that, in this paper’s decision-making task, participants using simple explainable AI still followed incorrect suggestions. This supports the qualified claim that explanations do not reduce overreliance in all tasks. (LLM, conf=0.93)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
29 more claims · 5 with their cited sources checked
Create a free account to see all 84 claims and to verify the 2 references still open.
- It may state two separate claims as one. 21
- Probably not a substantial claim. 33
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Research on human-AI decision making suggests that reliance is appropriate when people accept correct AI output and reject incorrect output; thus, the relevant outcome is not simply whether and how often students defer to AI, but how well their reliance tracks the quality of AI suggestions (Raees et al., 2026;Schemmer et al., 2023;Bansal et al., 2021).”
From Think Before You Accept: Can Written Justification Reduce Uncritical Uptake of AI Writing Suggestions? (2026) · cites this paper as Schemmer et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Research on human-AI decision making suggests that reliance is appropriate when people accept correct AI output and reject incorrect output; thus, the relevant outcome is not simply whether and how often students defer to AI, but how well their reliance tracks the quality of AI suggestions (Raees et al., 2026;Schemmer et al., 2023;Bansal et al., 2021).”
From Think Before You Accept: Can Written Justification Reduce Uncritical Uptake of AI Writing Suggestions? (2026) · cites this paper as Schemmer et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“in contrast to prior work which has evaluated reliance in action space only [3, 28, 35].”
From A Decision Theoretic Framework for Measuring AI Reliance (2024) · cites this paper as [28]
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788
lavaan : An R Package for Structural Equation Modeling
Rosseel, Y. (2012) · Journal of Statistical Software · doi:10.18637/jss.v048.i02 · Cited by 27,955
Comparative fit indexes in structural models.
Bentler, P. M. (1990) · Psychological Bulletin · doi:10.1037/0033-2909.107.2.238 · Cited by 25,266
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
The future of employment: How susceptible are jobs to computerisation?
Frey, C. B., & Osborne, M. A. (2017) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2016.08.019 · Cited by 8,727
Trust and TAM in Online Shopping: An Integrated Model1
Gefen, D., Karahanna, E., & Straub, D. W. (2003) · MIS Quarterly · doi:10.2307/30036519 · Cited by 8,635 · cited 1×
“We use four items based on [15, 17, 21, 22].”
An Examination of the Nature of Trust in Buyer-Seller Relationships
Doney, P. M., & Cannon, J. P. (1997) · Journal of Marketing · doi:10.2307/1251829 · Cited by 6,822 · cited 1×
“In our study, we define trust as a belief in the integrity, benevolence, trustworthiness, and predictability of the AI advisor following [15, 17, 21, 49].”
BackgroundDeterminants of Long-Term Orientation in Buyer-Seller Relationships
Ganesan, S. (1994) · Journal of Marketing · doi:10.1177/002224299405800201 · Cited by 6,741 · cited 1×
“In our study, we define trust as a belief in the integrity, benevolence, trustworthiness, and predictability of the AI advisor following [15, 17, 21, 49].”
BackgroundTrust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 10×
“Trust influences reliance, but does not fully determine it [45].”
BackgroundInfluentialPeeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294 · cited 3×
“Most researchers that studied AR so far have proposed to use explanations of AI as a means for AR [1, 43, 78].”
Background
Research works printed but not linked: 3
- [29] Patrick Hemmer, Max Schemmer, Michael Vössing, and Niklas Kühl. 2021. Human-AI Complementarity in Hybrid Intelligence Systems: A Structured Literature Review. PACIS 2021 Proceedings (2021). -
- [66] Keng Siau and Weiyu Wang. 2018. Building trust in artificial intelligence, machine learning, and robotics. Cutter business technology journal 31, 2 (2018), 47-53. -
- [68] Philipp Spitzer, Niklas Kühl, and Marc Goutier. 2022. Training Novices: The Role of Human-AI Collaboration and Knowledge Transfer. Workshop on Human-Machine Collaboration and Teaming (HM-CaT 2022). The 39th International Conference on Machine Learning, Baltimore, Maryland, USA (2022). -
Cited by
Items 101–200 of 200
Multidisciplinary perspectives on human‑AI team trust
Tielman, M. L., Bailey, M. E., Frattolillo, F., Jorge, C. C., Ulfert, A., & Meyer-Vitali, A. (2025) · Interaction Studies Social Behaviour and Communication in Biological and Artificial Systems · doi:10.1075/is.24048.tie · Cited by 5
Pang, R. Y., Feng, K. J. K., Feng, S., Li, C., Shi, W., Tsvetkov, Y., Heer, J., & Reinecke, K. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789091 · Cited by 40
“The concept of appropriate reliance can typically be defined as “relying on the AI when it’s correct, and relying on yourself when it’s not” [61].”
Building Human-AI Reliance Through Cognitive Engagement and Exploratory AI Assistance
Raees, M., Khan, V.-J., & Papangelis, K. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3708319.3734171 · Cited by 6
Imperfections of XAI: Phenomena Influencing AI-Assisted Decision-Making
Spitzer, P., Morrison, K., Turri, V., Feng, M., Perer, A., & Kühl, N. (2025) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3750052 · Cited by 17
Explanations, Fairness, and Appropriate Reliance in Human-AI Decision-MakingPreprint
Schoeffer, J., De‐Arteaga, M., & Kuehl, N. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2209.11812 · Cited by 3
“From a fairness perspective, the goal is to minimize such disparities, so as to equalize the burden of being misclassified and, as a result, being excluded from exposure to relevant opportunities between men and women.”
BackgroundMethodInfluentialSelf-citationHybrid Automation Experiences – Communication, Coordination, and Collaboration within Human-AI Teams
Spitzer, P., Baldauf, M., Palanque, P. A., Roto, V., Morrison, K., Zipperling, D., & Holstein, J. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3706738 · Cited by 2
The Enterprising and Elusive Prospects of Human-AI Collaboration
Gadiraju, U., & Balayn, A. (2025) · Springer eBooks · doi:10.1007/978-3-032-01940-0_7 · Cited by 5
An Empirical Examination of the Evaluative AI Framework
Kornowicz, J. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2581260 · Cited by 3
“However, as summarized by recent studies (Bertrand et al., 2023; Lai, Zhang, et al., 2023; Rogha, 2023; Schemmer et al., 2022; 2023; Vasconcelos et al., 2023), the results are mixed.”
Sun, Z., & Ding, J. (2025) · Human Behavior and Emerging Technologies · doi:10.1155/hbe2/6901297 · Cited by 2
Narayanan, R., & Feigh, K. M. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2618568 · Cited by 6
“We also want to understand how much of this reliance was calibrated or appropriate (Schemmer et al., 2023).”
Korosec-Serfaty, M., Léger, P., Parent‐Rocheleau, X., & Sénécal, S. (2026) · Information Systems Frontiers · doi:10.1007/s10796-025-10691-2 · Cited by 4
Trust to Reliance: Measurement Constructs for Human-AI Appropriate Reliance
Raees, M., & Papagelis, K. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798835 · Cited by 8
Farmer, T. S., & Ho, C.-J. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789089 · Cited by 3
“Because they capture how individuals process and engage with information [13], they may also influence how users calibrate their reliance on AI assistance [19, 45], ultimately shaping team performance.”
Adjust for Trust: Mitigating Trust-Induced Inappropriate Reliance on AI Assistance
Srinivasan, T., & Thomason, J. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742413.3789136 · Cited by 18
AI Reliance and Decision Quality: Fundamentals, Interdependence, and the Effects of InterventionsPreprint
Schoeffer, J., Jakubik, J., Vössing, M., Kühl, N., & Satzger, G. (2024) · doi:10.31219/osf.io/cekm9 · Cited by 2
Self-citationEmpathy Cuts Both Ways in Clinical AI (Preprint)Preprint
Sorin, V., & Klang, E. (2026) · doi:10.2196/preprints.108778 · Cited by 2
Johnson, D. S. (2025) · arXiv · doi:10.48550/arxiv.2503.03529 · Cited by 2
Johnson, D. S. (2026) · Proceedings of the 34th ACM Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3774935.3806657 · Cited by 2
Conceptualizing Multi-party AI Reliance for Design Research
Eckhardt, S., Dolata, M., Bauer-Hänsel, I., & Schwabe, G. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-61175-9_4 · Cited by 3
Towards Understanding Human-AI Reliance Patterns Through Explanation Styles
Casolin, E. R., & Salim, F. D. (2024) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3675094.3678996 · Cited by 2
Zijlstra, M., & Payne, S. J. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2478277 · Cited by 3
“HCI research in this domain is often concerned with maximising utility of the AI (acceptance of advice, whether good or bad) and improving judgement of the advice (Schemmer et al. 2023; Vorm and Combs 2022).”
Spatola, N. (2025) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2025.100169 · Cited by 7
Human at the Center: A Framework for Human‐Driven AI Development
Hu, D., Acosta-Navas, D., Gaube, S., Mozannar, H., Taylor, M. E., Dvijotham, K., & Bondi, E. (2025) · AI Magazine · doi:10.1002/aaai.70043 · Cited by 4
Augmenting Human Cognition Through Generative AI
Rogers, Y., Reicherts, L., Zhang, Z. T., & Hassib, M. (2026) · Springer eBooks · doi:10.1007/978-3-031-61050-9_10-1 · Cited by 1
Sun, B., & Hu, D. (2025) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3714394.3756343 · Cited by 4
“From a human-computer interaction (HCI) perspective, explainability and user-centeredness are critical in the design of health technologies [7, 19, 21, 24], especially in sensitive domains such as prenatal care.”
Raees, M., & Papagelis, K. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798834 · Cited by 1
“For effective decision-making, users need to understand AI systems to make correct decisions [34].”
Chen, E., Judicke, S., Beigh, K., Tang, X., Wang, Y., Yuan, M., Xiao, Z., Li, C., Li, S., Luttmer, R., et al. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29763-1_41 · Cited by 5
The Dependency Dilemma: How Machine Learning Decision Aids can Undermine Skill Growth
Bauer, K., Nofer, M., Henrich, B., Drachsler, H., & Hinz, O. (2026) · Business & Information Systems Engineering · doi:10.1007/s12599-026-01014-z · Cited by 2
“Schemmer et al. (2023) highlight the critical notion of appropriateness of reliance as a measurable, two-dimensional construct.”
Zhang, H., Zhang, P., Chen, Y., Guo, M., Gu, H., Lu, T., & Gu, N. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757703 · Cited by 3
Luo, Y., Gkouskos, D., & Russo, N. L. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790672
Sensing What Surveys Miss: Understanding and Personalizing Proactive LLM Support by User Modeling
Liu, A., Karoui, Y., Draxler, F., Kreuter, F., & Chiossi, F. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791191 · Cited by 9
Learning About Explainable AI with Very Little Programming
Dodge, J. E., Diawuo, A., Dai, E., & Das, R. K. (2024) · Springer eBooks · doi:10.1007/978-3-031-61290-9_10
Der Einfluss von Datenkompetenz, Feintuning und Transparenz auf Conversational Business Analytics
Alparslan, A. (2024) · Springer eBooks · doi:10.1007/978-3-658-46113-3_18
Being Pragmatic About Reliance and Trust in Artificial IntelligencePreprint
Ferrario, A. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5148249
Farmer, T. S., & Ho, C.-J. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719998 · Cited by 4
Schoeffer, J., De‐Arteaga, M., & Elmer, J. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732070 · Cited by 7
“This also carries implications for research and practice on human-AI collaboration [3, 26, 67].”
Genc, Y., Ahsen, M. E., & Zhang, Z. (2025) · PLoS ONE · doi:10.1371/journal.pone.0321342 · Cited by 1
“While we control for participants’ trust in our work, there is a growing emphasis on the importance of calibrating trust (i.e., avoiding over-or under-reliance [111]) to enhance AI-assisted decision-making [112].”
Cabitza, F., Fregosi, C., & Vicente, L. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-08317-3_11 · Cited by 3
Designing Human–AI Interactions
Urbach, N., Feulner, D., & Mayer, V. (2026) · Future of business and finance · doi:10.1007/978-3-032-13308-3_11
Kahr, P. K., & Bernstein, A. (2026) · doi:10.1145/3780045.3780056 · Cited by 3
Accuracy is not all you need! The Reasons to Require AI Explainability
Buijsman, S. (2026) · Minds and Machines · doi:10.1007/s11023-026-09768-x · Cited by 2
“For example, Schemmer et al. (2023 ) conducted a study where participants had to decide whether a shown hotel review was deceptive or not.”
Passing the Buck to AI: How Individuals’ Decision-Making Patterns Affect Reliance on AI
Mei, K. X., Pang, R. Y., Lyford, A., Wang, L. L., & Reinecke, K. (2026) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3786326 · Cited by 9
“To mitigate these risks and inform interventions, research in Human-AI interaction (HAI) has focused on factors related to AI systems that affect users’ reliance, such as accuracy and transparency [15, 41, 59, 60, 71, 72, 74].”
How Do We Measure Over-Reliance? A Unified Probabilistic View
Mikhaylova, D., Turchi, T., & Malizia, A. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742414.3794719 · Cited by 2
Understanding and Designing Human-Centered AI Interactions for Non-Expert Users
Raees, M. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3799192 · Cited by 1
“My research (a visual plan outlined in Figure 1) investigates understanding human factors, such as interaction [13], agency [21], and reliance [37] in human-AI collaboration, in three key dimensions.”
Lee, G., Lee, H., & Kim, D. S. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798529
“Unlike ‘virtual’ AI advisors or coaches [4, 17, 20, 23], or generic AI assistants [16, 21, 25], these AI clones are designed to communicate and provide advice based on real human experts’ professional knowledge and judgment.”
Schlonsak, R., Jetter, H.-C., & Matthies, D. J. C. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798760 · Cited by 2
Understanding, Mitigating, and Leveraging Cognitive Biases to Calibrate Trust in Evolving AI Systems
Pareek, S., Boonprakong, N., Kollerup, N. K., Chen, S., Hosio, S. J., Yatani, K., Lee, Y., Gadiraju, U., van Berkel, N., & Gonçalves, J. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3778722 · Cited by 1
“AI systems have evolved far beyond mere tools; they now act as collaborators, advisors, and decision-makers in critical domains such as healthcare, hiring, and transportation [9, 43], as well as social companions in everyday life [34, 45, 48].”
Branchat: A Tree-Structured Interface for Efficient Revisitation in Long-Horizon LLM Conversations
Li, J., Qin, P., & Lee, Y. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798792 · Cited by 1
“Without transparent and efficient context management, users have to repeatedly re-enter key information to ensure that the LLM is operating on the intended context, thereby affecting user experience [19, 35], trust in the tool [18, 33], and their confidence in performing complex tasks [15, 30].”
Agentic Automation Experiences—Rethinking the Interaction of Humans and AI Agents
Spitzer, P., Baldauf, M., Palanque, P. A., Roto, V., Morrison, K., Gomez-Beldarrain, G., Westphal, M., & Holstein, J. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3778732 · Cited by 1
Wischnewski, M., Scharmann, A., Ridder, A., & Krämer, N. C. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790983
“Hence, following Schemmer et al. [82], we calculated participants’ Relative AI Reliance (RAIR) and Relative self-reliance (RSR) to gauge participants’ appropriate reliance on AI advice.”
Liu, Y., & Rau, P. P. (2026) · ACM Transactions on Human-Robot Interaction · doi:10.1145/3811031
Analyzing Empirical Findings on User Reliance Behaviors in XAI-Assisted Decision-Making
Filho, J. C. D. S., Belloum, R., & de Oliveira, K. M. (2026) · Lecture notes in business information processing · doi:10.1007/978-3-032-26836-5_3
Nguyen, T. H., Truong, L. T., & Nguyen, N. H. T. (2026) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2026.100618 · Cited by 4
Toward trustworthy healthcare AI: designing academic research for translation readiness
Hekler, A., & Buettner, F. (2026) · Molecular Systems Biology · doi:10.1038/s44320-026-00219-4
Sun, X., Wei, S., Pan, T., Wang, Y., Bosch, J. A., Echizen, I., Ali, A. E., & Sugawara, S. (2026) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3774935.3806190 · Cited by 1
For Those About to Rely—A Taxonomy of Experimental Studies on AI Reliance: Completed Research Paper
Schaschek, M., Spatscheck, N., & Winkelmann, A. (2026) · Lecture notes in information systems and organisation · doi:10.1007/978-3-032-08480-4_2
Complex-Time Framework for Authenticity and Identity in Personalized AI
Iovane, G., Iovane, G., Rosa, A. D., & Barbato, F. (2026) · Algorithms · doi:10.3390/a19060458
Choudhary, Z. D., Gottsacker, M., Furuya, H., Erickson, A., Schubert, R., Bruder, G., Browne, M. P., & Welch, G. (2026) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2026.3703957
“…between these AR display factors, objective performance, and users’ subjective reliance and trust in this technology, as well as implications for potential compensatory behavioral adaptations, we decided to evaluate these interrelations by controlling for different behaviors in a visual…”
Arun, S. K., & Fischer, J. E. (2026) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3808045.3808049 · Cited by 2
Grosswieser, H., Seeber, I., & Mara, M. (2026) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3808045.3808073
Navigating maritime emergencies with large models: A lifecycle-oriented review
Xu, J., Yang, Q., Yang, Z., Gao, Y., Yang, K., & Jiang, J. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.115411 · Cited by 2
Reliance-Aware Design of Interactive Explainable User Interfaces
Filho, J. C. D. S., Belloum, R., Monteiro, I. T., & de Oliveira, K. M. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812231 · Cited by 2
“”[39] [39, 40, 58] Time pressure for decision-making (g:4, d:4) “In settings with time pressure, following the AI assistant’s recommendation when the AI assistant is correct can save valuable time.”
Warren, G., Sun, J., Shklovski, I., & Augenstein, I. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812358 · Cited by 4
“Common mitigation approaches for overreliance include encouraging people to evaluate AI input before coming to a final decision [8, 48] and providing explanations for AI decisions [59].”
Yang, M., Kagawa, R., & Baba, Y. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812288
“These measures were selected based on methods widely adopted in existing XAI and human–AI collaboration research [24, 51, 68].”
Human-AI Complementarity: A Goal for Amplified Oversight
Jain, R., Bridgers, S., Janzer, L., Greig, R., Teh, T. H., & Mikulik, V. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812308 · Cited by 9
Dhanorkar, S., Passi, S., & Vorvoreanu, M. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812402 · Cited by 35
“…tasks using proxy measures (unsurprising, given the importance attributed to the oversight work of reviewing AI outputs) [e.g. : 12, 44, 46, 63, 72, 107] or is largely conceptual, focusing on high-level arguments, frameworks, and theories of oversight [e.g. : 29, 41, 65, 68, 70, 112, 127, 132].”
How Explanation Framing Shapes Reliance on AI in Clinical Decision Support
Cabitza, F., Papale, A., Vicente, L., & Tomaiuolo, R. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30710-1_11
Supporting Bayesian Reasoning to Mitigate Over-Reliance in AI-Assisted Decision Making
Mikhaylova, D., Turchi, T., Cevolani, G., & Malizia, A. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30849-8_6
Human Agency and Skill in AI-Supported Work: Countering Cognitive Offloading
Fregosi, C., Fischer, S. W. S., Natali, C., Schraffenberger, H., & Cabitza, F. (2026) · Annual Symposium on Human-Computer Interaction for Work (CHIWORK) · doi:10.1145/3805029.3818273
Irons, J. L., Cooper, P. S., Bölücü, N., Duenser, A., Timmer, R. C., Yang, H., Lee, C., Jin, B., & Wan, S. (2026) · ACM SIGIR International Conference on Theory of Information Retrieval (ICTIR) · doi:10.1145/3805713.3820430
Do people rely on ChatGPT more than their peers to detect deepfake news?
Fu, Y., & Hanaki, N. (2026) · Journal of Economic Interaction and Coordination · doi:10.1007/s11403-026-00490-6 · Cited by 1
The impact of artificial intelligence on close protection and surveillance
van der Vegt, I. (2026) · Legal and Criminological Psychology · doi:10.1111/lcrp.70054
Arthur, F., Abraham, A. Y., & Inkoom, M. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-9484-8.ch015
Kumari, J., Yadav, P., & Bhargava, C. P. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-8988-2.ch001
Tajima, H. (2026) · JMIR Nursing · doi:10.2196/99590
“In addition, users frequently overrely on AI by following incorrect recommendations [13], and appropriate reliance requires accepting correct advice while rejecting incorrect advice [26].”
Human-Verifiable API Documentation Generation for Legacy Web Frameworks Using Multi-Agent LLMs
Zhao, P., Blake, J., & Pyshkin, E. (2026) · Zooming Innovation in Consumer Technologies Conference (ZINC) · doi:10.1109/zinc69910.2026.11655803 · Cited by 1
“…finds that making uncertainty explicit is a precondition for effective collaboration: displaying confidence information has been shown to promote appropriate reliance behaviors [21], and continuously updated system confidence improves the performance of the human-machine team [22], [23].”
Boyacı, T., Canyakmaz, C., & de Véricourt, F. (2026) · Management Science · doi:10.1287/mnsc.2024.07450 · Cited by 4
Christopoulos, A., Mystakidis, S., Perikos, I., Bourazeri, A., Michael, A., & Laakso, M. (2026) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2026.100378
Wójcik, S., Rulkiewicz, A., & Domienik‐Karłowicz, J. (2026) · doi:10.2196/preprints.110249
Wang, F., Huang, Y., Jiang, Q., & Chen, S. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2705666
Hyytiäinen, T., & Luukka, P. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2721889
Authors' Reply: Empathy Cuts Both Ways in Clinical AI (Preprint)Preprint
Bae, S. W., Zhang, T., Chung, T. A., & Dey, A. K. (2026) · doi:10.2196/preprints.110739
Data-Centric ML for Regulatory Compliance: A Survey and Taxonomy
Brajovic, D., & Huber, M. F. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-32732-1_43
Human preferences are susceptible to covertly misaligned AI advice
Sabour, S., Liu, J. M., Liu, S., Yao, C. Z., Cui, S., Zhang, W., Zhang, X., Cao, Y., Bhat, A., Guan, J., et al. (2026) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2600684123 · Cited by 4
Tuo, B., Miller, T., & Demartini, G. (2026) · ACM Conference on Human-AI Complementarity and Alignment · doi:10.1145/3834580.3838739
Li, Z., Ma, Z., He, S., Lu, Z., Xiao, Z., & Yin, M. (2026) · ACM Conference on Human-AI Complementarity and Alignment · doi:10.1145/3834580.3838738
Biases in Human and AI-Assisted Requirements Elicitation for Mental Health Wearables
Hadar, I., & Levy, M. (2026) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew72749.2026.00069
“The Automation Bias Paradox, grounded in Lin's F1 and F4 [11] and confirmed by GKS automation bias [6], means no AI informant can surface over-reliance risks for a system it will power [17, 18].”
Authors’ Reply: Empathy Cuts Both Ways in Clinical AI
Bae, S. W., Zhang, T., Chung, T. A., & Dey, A. K. (2026) · JMIR Human Factors · doi:10.2196/110739
Empathy Cuts Both Ways in Clinical AI
Sorin, V., & Klang, E. (2026) · JMIR Human Factors · doi:10.2196/108778 · Cited by 1
“Users should accept correct AI advice and reject wrong advice [3].”
Distributive Effects of AI Support: A Framework for Analyzing Performance Shifts Within Groups
Campagner, A., Natali, C., Fregosi, C., & Cabitza, F. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-37976-4_16
Wired to believe: source-authority cues shape perceived AI capability and neural persuasion
Ding, Y., Deng, Z., Zhang, W., Lyu, W., Duffy, B. M., & Zhang, D. (2026) · Ergonomics · doi:10.1080/00140139.2026.2734930 · Cited by 2
From AI use to AI-enabled loafing: a reliance-centred account in hybrid intelligence
Zhou, Y., Du, T., & Xiaoqian, L. (2026) · Behaviour and Information Technology · doi:10.1080/0144929x.2026.2734077
Frey, N., Agha-Mir-Salim, L., Renner, L., Haufe, S., Voge, L., & Balzer, F. (2026) · Research Square · doi:10.21203/rs.3.rs-10892043/v1 · Cited by 2
Daniele, F., Montini, E., Conti, L., Gambardella, L. M., & Pedrazzoli, P. (2026) · IFIP advances in information and communication technology · doi:10.1007/978-3-032-38610-6_3
Li, S., Tang, J., Qiang, Z., & Wang, Z. (2026) · Journal of Intelligence · doi:10.3390/jintelligence14100232
AI Dependence as a Longitudinal Governance Problem
Du, Y. (2026) · AI & Innovation · doi:10.1002/aiv2.70018
Yao, N., Yang, X., & Xiao, F. (2026) · Information · doi:10.3390/info17100981
Shi, M., Liang, M., & Hwang, B. (2026) · Journal of Management in Engineering · doi:10.1061/jmenea.meeng-7659
Limits of Interventions in AI-Assisted Decision Making
Shrestha, S., Ahn, E., & Vorobeychik, Y. (2026) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1609/aies.v9i4.43309
Tao, Y., Meyer, J., & Kizilcec, R. F. (2026) · arXiv · doi:10.48550/arxiv.2609.23936
Metadata
- Authors
- Schemmer, M., Kuehl, N., Benz, C., Bartos, A., & Satzger, G.
- Affiliations
- Karlsruhe Institute of Technology (DE)
- Venue
- Proceedings of the 28th International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2023-03-27
- Pages
- 410-422
- OpenAlex
- W4360991389
- Semantic Scholar
- 95f751e8535a6d7f8fe733ab4729f738ef0566c4
- Cited by
- 208
- References
- 43
- Citations by year
- 2026: 93 · 2025: 71 · 2024: 37 · 2023: 6
- FWCI
- 36.59
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by-nc-nd)
- Repository KITopen (Karlsruhe Institute of Technology) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University)
- KITopen
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- AI advice, AI explanations, human-AI decision-making, trust in AI, measurement concepts, investment decision-making, treatment decision-making
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.