The DOI matches its Crossref record; 158 of the 158 references it cites were found in a registry or an index.
Do people appropriately rely on AI-advice? An analytical review of HCI research on human-AI decision-making.
Open accessAbstract
AI systems are increasingly being positioned to assist people in decision-making. However, recent empirical studies show critical concerns that people over-rely on AI advice without analytically engaging with it. While HCI research explores how people rely on AI advice, we argue that it largely overlooks an important aspect: replicating realistic decision-making scenarios. Human-AI interaction factors influence people’s reliance on AI advice. To understand human-AI interaction factors and their interplay, we conducted an analytical review of recent studies in human-AI reliance literature. We analyzed the decision-making tasks in research and their validity in application-grounded contexts. Our findings show that user engagement is a precious commodity for relying on AI advice; however, it comes at a cost. We also discuss factors contributing to “appropriate reliance”, existing research gaps, and recommendations for intervention design for human-AI reliance. Our work contributes to the critical body of research on building appropriate reliance on AI advice.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Educational work likewise models trust, reliance, resistance, and actual use as distinct constructs (Wang et al., 2025), while HCI reviews note heterogeneous definitions of appropriate reliance (Raees et al., 2026).”
From Beyond checking: verification quality, reliance calibration, and learning in generative AI-assisted higher education (2026) · cites this paper as Raees et al., 2026
Citations
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283
The Framing of Decisions and the Psychology of Choice
Tversky, A., & Kahneman, D. (1981) · Science · doi:10.1126/science.7455683 · Cited by 18,497 · cited 1×
“To understand trust, theoretical frameworks from psychology are used in various studies [4, 59, 87, 149, 150].”
Literature review as a research methodology: An overview and guidelines
Snyder, H. (2019) · Journal of Business Research · doi:10.1016/j.jbusres.2019.07.039 · Cited by 11,524 · cited 1×
“We developed this protocol following recent guidelines on literature review research [141], and according to the PRISMA [110] framework, which is outlined in Figure 2.”
Kruger, J., & Dunning, D. (1999) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.77.6.1121 · Cited by 7,731 · cited 1×
“Conversely, users can overestimate their confidence in decisions due to inaccurate interpretations [94], or cognitive biases (e.g., Dunning–Kruger effect [79]: people overestimate their abilities), leading to under-reliance on the ML models.”
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 3×
“To understand trust, theoretical frameworks from psychology are used in various studies [4, 59, 87, 149, 150].”
Cacioppo, J. T., & Petty, R. E. (1982) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.42.1.116 · Cited by 5,359 · cited 1×
“An important trait for improving the users’ mental models concerning AI is capturing their “Need for Cognition” [19], a stable personality trait that captures one’s motivation to engage in effortful mental activities [10].”
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529786 · Cited by 4,332 · cited 1×
“Inter-rater reliability using Cohen’s Kappa [84] indicated strong agreement ( k = 0.80 ).”
InfluentialWohlin, C. (2014) · Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2601248.2601268 · Cited by 4,234 · cited 1×
“To further expand our selection, we performed iterative back-ward and forward (using Google Scholar) snowballing [163] by analyzing references and citations of each paper.”
International evaluation of an AI system for breast cancer screening
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1799-6 · Cited by 3,287 · cited 1×
“Although in many cases, AI performance is superior to human performance [5, 99], it still suffers from systematic or random errors.”
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · cited 4×
“Conversely, users may refrain from using the AI system due to performance bias , perceptions of self-efficacy (the ability to make decisions without advice), or purposefully considering the system’s advice as inferior, demonstrating algorithmic aversion [42, 143].”
Influential
Research works printed but not linked: 10
- Finale Doshi-Velez and Been Kim. 2017. Towards a rigorous science of interpretable machine learning. arXiv preprint arXiv:https://arXiv.org/abs/1702.08608 (2017). · arXiv
- Christoph Engel, Andreas Glöckner, and Sinika Timme. 2017. Defendant should have the last word: Experimentally manipulating order and provisional assessment of the facts in criminal procedure. Technical Report. Preprints of the Max Planck Institute for Research on Collective Goods.
- Robert R Hoffman Shane T Mueller Gary Klein and Jordan Litman. 2018. Metrics for explainable AI: Challenges and prospects. arXiv preprint arXiv:https://arXiv.org/abs/1812.04608 (2018). · arXiv
- Daniel Kahneman. 2011. Thinking, fast and slow. macmillan.
- Don Norman. 2013. The design of everyday things: Revised and expanded edition. Basic books.
- Lee Ross and Andrew Ward. 2013. Naive realism in everyday life: Implications for social conflict and misunderstanding. In Values and knowledge. Psychology Press, 103–135.
- Max Schemmer, Andrea Bartos, Philipp Spitzer, Patrick Hemmer, Niklas Kühl, Jonas Liebschner, and Gerhard Satzger. 2023. Towards effective human-AI decision-making: The role of human learning in appropriate reliance on AI advice. In Forty-Fourth International Conference on Information Systems (Hyderabad, India) (ICIS 2023 Proceedings). AISEL, 1–17. https://aisel.aisnet.org/icis2023/hti/hti/14
- Keng Siau and Weiyu Wang. 2018. Building trust in artificial intelligence machine learning and robotics. Cutter business technology journal 31 2 (2018) 47–53.
- Steve G Sutton Vicky Arnold and Matthew Holt. 2022. An extension of the theory of technology dominance: understanding the underlying nature causes and effects. Causes and effects (2022). · DOI
- Earl L Wiener. 1981. Complacency: Is the term useful for air safety. In Proceedings of the 26th corporate aviation safety seminar , Vol. 117. 116–125.
Cited by
From Trust to Appropriate Reliance: Measurement Constructs in Human-AI Decision-Making
Raees, M., & Papangelis, K. (2026) · arXiv · doi:10.48550/arxiv.2604.23896 · Cited by 5
Self-citationTrust to Reliance: Measurement Constructs for Human-AI Appropriate Reliance
Raees, M., & Papagelis, K. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798835 · Cited by 8
Self-citationRaees, M., Iwamoto, Y., Papangelis, K., Heard, J., & Tigwell, G. W. (2026) · arXiv · doi:10.48550/arxiv.2602.08937 · Cited by 2
“Similar patterns are evidenced by prior research using AI-generated content [36], where users tend to overestimate the capabilities of AI and not critically analyze or verify what they produce with AI [59].”
Self-citationRaees, M., & Papagelis, K. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798834 · Cited by 1
Self-citationWei, J., & Shang, Y. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1965371 · Cited by 1
“Educational work likewise models trust, reliance, resistance, and actual use as distinct constructs (Wang et al., 2025), while HCI reviews note heterogeneous definitions of appropriate reliance (Raees et al., 2026).”
Metadata
- Authors
- Raees, M., Khan, V.-J., Lykourentzou, I., & Papangelis, K.
- Affiliations
- Rochester Institute of Technology (US)
- Independent Researcher, Brussels, Belgium
- Utrecht University (NL)
- Rochester Institute of Technology, Rochester, New York, USA
- Venue
- Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2026-04-13
- Pages
- 1-24
- OpenAlex
- W7153972336
- Cited by
- 5
- References
- 157
- Citations by year
- 2026: 5
- FWCI
- 67.17
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion, cc-by)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Social Sciences › Psychology › Social Psychology › Social Robot Interaction and HRI
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- AI advice, AI reliance, human-AI decision-making, human-AI interaction, user engagement, decision-making tasks, intervention design, trust in AI
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Do people appropriately rely on AI-advice? An analytical review of HCI research on human-AI decision-making. https://app.cochat.ai/papers/10.1145/3772318.3791467
- The paper
- Raees, M., Khan, V.-J., Lykourentzou, I., & Papangelis, K. (2026). Do people appropriately rely on AI-advice? An analytical review of HCI research on human-AI decision-making. Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3772318.3791467
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.