Human Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks
The DOI matches its Crossref record; 64 of the 64 references it cites were found in a registry or an index.
Abstract
This paper addresses an under-explored problem of AI-assisted decision-making: when objective performance information of the machine learning model underlying a decision aid is absent or scarce, how do people decide their reliance on the model? Through three randomized experiments, we explore the heuristics people may use to adjust their reliance on machine learning models when performance feedback is limited. We find that the level of agreement between people and a model on decision-making tasks that people have high confidence in significantly affects reliance on the model if people receive no information about the model’s performance, but this impact will change after aggregate-level model performance information becomes available. Furthermore, the influence of high confidence human-model agreement on people’s reliance on a model is moderated by people’s confidence in cases where they disagree with the model. We discuss potential risks of these heuristics, and provide design implications on promoting appropriate reliance on AI.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For example, confirmation bias and related confirmatory heuristics also contribute to inappropriate reliance on technology.”
From Exploring automation bias in human–AI collaboration: a review and implications for explainable AI (2026) · cites this paper as Lu and Yin 2021
In the citing paper
“As a result, users may over-rely on models that reinforce their own biases or under-rely on models that might challenge or correct those biases (Lu and Yin 2021).”
From Exploring automation bias in human–AI collaboration: a review and implications for explainable AI (2026) · cites this paper as Lu and Yin 2021
In the citing paper
“Participants' comments also reflected the strength of their intuition about the outcome (or confidence [59]), which impacted how they engaged with the AI system.”
From Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations (2023) · cites this paper as [59]
In the citing paper
“In this set-up, agreement between a participant's answer and that of the AI system is a commonly studied behavioral measure of reliance and trust [19, 22, 55, 61, 67, 75, 114, 118].”
From "I'm Not Sure, But...": Examining the Impact of Large Language Models' Uncertainty Expression on User Reliance and Trust (2024) · cites this paper as [67]
In the citing paper
“Nevertheless, the field of humancomputer interaction (HCI) has so far considered metacognition mainly in the context of computer science education [107,142].”
From The Metacognitive Demands and Opportunities of Generative AI (2024) · cites this paper as [107]
4 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Kruger, J., & Dunning, D. (1999) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.77.6.1121 · Cited by 7,731
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 2×
“Theoretical models and frameworks have been proposed to explain human’s reliance on the automation in various situations [12, 19, 20, 31, 57].”
BackgroundHumans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554 · cited 1×
“on the automation [13, 43], including both over-reliance on the automation when it performs poorly (i.”
BackgroundWhat is AI Literacy? Competencies and Design Considerations
Long, D., & Magerko, B. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376727 · Cited by 3,241 · cited 1×
“For example, tutorials can be provided to end-users of ML models to increase their general knowledge of ML, including that ML models are often learned from data, and depending on the quality of the data, ML models may exhibit dierent performance on dierent data [34].”
BackgroundThe “false consensus effect”: An egocentric bias in social perception and attribution processes
Ross, L., Greene, D., & House, P. (1977) · Journal of Experimental Social Psychology · doi:10.1016/0022-1031(77)90049-x · Cited by 3,139 · cited 2×
“ation reveals some consistent behavior pattern [33, 39, 46, 50, 63]— for example, the phenomenon of “naive realism” [61] suggests that people often consider their own judgment to be objective reections of reality and tend to discount advice that is more dierent”
BackgroundModel Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 1×
“On the conceptual level, our ndings echos that of other work in human-AI collaboration [1, 3, 4, 26] and reiterates that the design and development of ML-based decision aids should not be isolated from the people who will use them.”
BackgroundLeshno, M., Lin, V. Y., Pinkus, A., & Schocken, S. (1993) · Neural Networks · doi:10.1016/s0893-6080(05)80131-5 · Cited by 2,121
Research works printed but not linked: 5
- Joy Buolamwini and Timnit Gebru. 2018. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency. 77–91.
- David Dunning. 2014. We are all confident idiots. Pacific Standard 7(2014), 46–54.
- SM Fleming and ND Daw. 2016. Self-evaluation of decision performance: A general Bayesian framework for metacognitive computation. Psychol Rev 124(2016), 1–59.
- Maria Madsen and Shirley Gregor. 2000. Measuring human-computer trust. In 11th australasian conference on information systems, Vol. 53. Citeseer, 6–8.
- Gary Marks and Norman Miller. 1987. Ten years of research on the false-consensus effect: An empirical and theoretical review.Psychological bulletin 102, 1 (1987), 72.
Cited by
Items 101–196 of 196
Chen, A., Kim, S. S. Y., Franyutti, A., Dharmasiri, A., Mukherjee, K., Russakovsky, O., & Fan, J. E. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791049 · Cited by 12
“Following prior work[14,17,65,74,86],weprimarilymeasuredrelianceas agreement between the participant’s submitted answer and Theta’s answer to the factual question ( TRUE if the answers matched and FALSE otherwise).”
Accounting for Human Engagement Behavior to Enhance AI-Assisted Decision Making
Yin, M. (2024) · Proceedings of the AAAI Symposium Series · doi:10.1609/aaaiss.v3i1.31184 · Cited by 12
“…and humans’ independent judgement on those decision making tasks where humans are highly confident about their own judgement also significantly impacts humans’ trust and reliance on AI—the higher the level of agreement, the more humans trust and rely on the AI recommendation (Lu and Yin 2021).”
BackgroundSelf-citationLearning to complement with multiple humans
Zhang, Z., Nguyen, C. C., Wells, K., Do, T.-T., & Carneiro, G. (2025) · Pattern Recognition · doi:10.1016/j.patcog.2025.112376 · Cited by 12
“Recently, Chiou and Lee [48] studied how AI decisions influence human experts, reaching the conclusion that the trustworthiness of AI depends on both model confidence [49, 50] and explainability [51, 52].”
BackgroundRogha, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2312.11507 · Cited by 13
“Other research defines Inappropriate reliance as under or over reliance [91].”
BackgroundQiao, H., Vermeulen, J., Fitzmaurice, G., & Matejka, J. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714103 · Cited by 11
“GenAI recommendations are applied to more open-ended tasks that could be completed with a variety of pathways, whereas AI-assisted decision making is applied to tasks that can usually be reduced to binary decision making [35, 51, 68].”
AI Pilot in the Cockpit: An Investigation of Public Acceptance
Gao, S., Lu, Z., Luan, H., Yin, M., & Wang, L. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2301856 · Cited by 12
“The uncertainty and inherent complexity of modern AI systems can lead to mixed attitudes and expectations from the public regarding autonomous systems embedded with AI (Chiang & Yin, 2021; Lu & Yin, 2021).”
Self-citationŁajewska, W., Balog, K., Spina, D., & Trippas, J. R. (2024) · Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region (SIGIR-AP) · doi:10.1145/3673791.3698409 · Cited by 11
“Evaluating CIS responses from a user perspective involves multiple dimensions [40], including trust and fairness [56], credibility [4], reliability [30, 36], verifiability [29], factual correctness, transparency (e.g., information sources, ranking, and consolidation process) [42], relevance,…”
BackgroundGemalmaz, M. A., & Yin, M. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534201 · Cited by 10
Self-citationOgawa, N., Okafuji, Y., Hatada, Y., & Baba, J. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757686 · Cited by 11
Trust and Reliance in Evolving Human-AI Workflows (TREW)
Ashktorab, Z., Bansal, G., Buçinca, Z., Holstein, K., Hullman, J. R., Smith, A., Wu, T., & Zhang, W. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3636319 · Cited by 10
“Indeed, recent years have seen renewed interest in human-AI trust within the AI and HCI community [3, 13, 15, 16, 22, 25] motivated by directions and approaches introduced in prior literature on human-automation interaction and human factors [14].”
BackgroundHe, G., Balayn, A., Buijsman, S., Yang, J., & Gadiraju, U. (2024) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.15118 · Cited by 10
“…factors ( e.g., AI literacy (Chiang & Yin, 2022) and cognitive bias (Bertrand et al., 2022)) and interaction with AI systems (tutorial intervention (Lai et al., 2020; He et al., 2023) and performance feedback (Lu & Yin, 2021)) are observed to affect subjective trust and reliance behaviors greatly.”
BackgroundBIASeD: Bringing Irrationality into Automated System DesignPreprint
Gulati, A., Lozano, M. Á., Lepri, B., & Oliver, N. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.01122 · Cited by 10
“While it has been reported that humans rely on heuristics when deciding if they should trust AI decisions (96), would the existence of cognitive biases in AI systems have an impact on their interpretability and trustworthiness?”
BackgroundZhou, Y., Chen, E., Pisipati, M., Xiong, A., & Rajtmajer, S. M. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757588 · Cited by 9
Workshop on Trust and Reliance in AI-Human Teams (TRAIT)
Bansal, G., Buçinca, Z., Holstein, K., Hullman, J., Smith-Renner, A. M., Stumpf, S., & Wu, S. (2023) · Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544549.3573831 · Cited by 8
“Indeed, recent years have seen renewed interest in human-AI trust within the AI and HCI community [12, 16, 18, 21, 23] motivated by directions and approaches introduced in prior literature on human-automation interaction and human factors [14].”
Trust and Dependence on Robotic Decision Support
Natarajan, M., & Gombolay, M. (2024) · IEEE Transactions on Robotics · doi:10.1109/tro.2024.3484628 · Cited by 8
“3) Investigating Delayed Decision Outcomes: Delays in evaluating decisions are often inevitable in critical scenarios [38], [39], [40], [41] For instance, during an aircraft engine failure, the crew, guided by an automated system, must rapidly make decisions to prevent casualties.”
Does GenAI Deplete Us? The Effects of Using GenAI for Writing on Identity and Ego Depletion
Lee, S., Kim, T., & Moon, W.-K. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719875 · Cited by 8
García‐Peñalvo, F. J., Vázquez‐Ingelmo, A., & Garc韆-Holgado, A. (2023) · Computer Modeling in Engineering & Sciences · doi:10.32604/cmes.2023.023897 · Cited by 8
Please Take Over: Xai, Delegation of Authority, and Domain KnowledgePreprint
Bauer, K., von Zahn, M., & Hinz, O. F. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4512594 · Cited by 10
RelEYEance: Gaze-based Assessment of Users' AI-reliance at Run-time
Wu, Z., Wang, Y., Langer, M., & Feit, A. M. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3725841 · Cited by 7
Under what influence: Measuring AI influence to fit user profiles in decision-making
Campagner, A., Fregosi, C., Natali, C., & Cabitza, F. (2026) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2026.103763 · Cited by 7
Narechania, A. A., Endert, A., & Sinha, A. R. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712166 · Cited by 7
Lu, Z., Li, P., Wang, W., & Yin, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713871 · Cited by 7
“In recent years, extensive empirical research has explored whether andhowAImodelrecommendationsimpacthuman decision-making in a wide range of scenarios [14, 15, 56, 59, 65, 66, 97, 135].”
Self-citationDesigning Shared Information Displays for Agents of Varying Strategic Sophistication
Zhang, D., Hartline, J. D., & Hullman, J. R. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637319 · Cited by 7
“Prior work [42] suggest users’ trust in predictive model outputs is affected by the observed accuracy of the model.”
BackgroundCan I Trust You? Exploring the Impact of Misleading AI Suggestions on User's Trust
Pretolesi, D., Zechner, O., Schrom-Feiertag, H., & Tscheligi, M. (2024) · IEEE International Conference on Metrology for eXtended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE) · doi:10.1109/metroxraine62247.2024.10796402 · Cited by 7
“These metrics were adapted from previous works by [22] and [31].”
A Human-centric AI-driven Framework for Exploring Large and Complex Datasets
Costabile, F., Desolda, G., Dimauro, G., Lanzilotti, R., Loiacono, D., Matera, M., & Zancanaro, M. (2022) · Iris (University of Trento) · Cited by 7
Holstein, J., Diener, M., & Spitzer, P. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2504.02780 · Cited by 7
“While prior research has extensively studied the success of human-AI collaboration in traditional AI systems for classification [11, 13, 17, 18, 25] and regression tasks [7, 8], initial work has started to investigate human-LLM collaboration in structured domains like code generation or math [2,…”
Zhang, Z. T., Buchner, F., Liu, Y., & Butz, A. (2024) · Proceedings of Mensch und Computer 2024 · doi:10.1145/3670653.3670660 · Cited by 6
“In contrast, we were interested in settings like those studied by Lu and Yin [25] or Papenmeier et al. [27].”
Human-AI Teaming: Following the IMOI Framework
Kleanthous, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60611-3_27 · Cited by 6
Sun, Y., Yue, W., Jin, X., Ma, S., Ma, X., & Wang, X. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757705 · Cited by 6
“Since AI technologies are operated as “black box”, it is often difficult for clinicians to trust the results provided by models fully [61].”
Raees, M., Khan, V.-J., Lykourentzou, I., & Papangelis, K. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791467 · Cited by 5
“…[33, 63], Cognitive Cost (Effort) [151] Positive Task Difficulty [23, 127, 151], Negative First Impression [61, 108, 153], Poor Self Assessment [63, 94], Lack of Domain Knowledge [103, 156], Feature-based XAI [30, 170] Negative Example-based XAI [10, 29, 131, 156], Feature-based XAI [29, 133,…”
InfluentialHinrichs, M., Bui, T. B. D., & Schneegaß, S. (2024) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3701571.3701585 · Cited by 5
Dynamic Trust Calibration Using Contextual Bandits
Henrique, B. M., & Santos, E. (2025) · arXiv · doi:10.48550/arxiv.2509.23497
“For instance, Lu and Yin (2021) study human reliance on AI under conditions of limited accuracy information, revealing that the level of agreement between humans and AI is very influential on reliance.”
InfluentialLi, A., Guo, C., Zhang, J., & Zhao, Q. (2025) · Computers & Industrial Engineering · doi:10.1016/j.cie.2025.111532 · Cited by 4
Effects of AI explanations on trust and reliance: a study in job shop scheduling
Saßmannshausen, T. M., Burggräf, P., Hassenzahl, M., & Sauer, C. R. (2026) · Ergonomics · doi:10.1080/00140139.2026.2634115 · Cited by 4
Wang, Y., Yu, H., Zhang, L., Wu, Y., & Luo, S. (2026) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2026.103798 · Cited by 5
More Isn't Always Better: Balancing Decision Accuracy and Conformity Pressures in Multi-AI Advice
Tsuchiya, Y., & Baba, Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791648 · Cited by 4
Proceedings of The ACM Collective Intelligence Conference
(2023) · doi:10.1145/3582269 · Cited by 4
Kaur, S., Salimzadeh, S., & Gadiraju, U. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2601.15064 · Cited by 4
“For example, [74] mentioned, "...we randomly selected one prediction task in the sequence to check whether the subject’s final prediction on that task was correct.”
Amin, H., Yin, M., & Khanna, R. (2026) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v40i21.38786 · Cited by 4
“…convenient, accumulating empirical evidence shows that human reliance on AI advice is often more nuanced and probabilistic, influenced significantly by human trust and observed AI behavior (Chong et al. 2022; Schemmer et al. 2023; Lu and Yin 2021; Chiang and Yin 2021; Narayanan et al. 2023).”
InfluentialSelf-citationJessup, S. A., Alarcon, G. M., Willis, S. M., & Lee, M. A. (2024) · Applied Ergonomics · doi:10.1016/j.apergo.2024.104363 · Cited by 4
On the Interplay of Transparency and Fairness in AI-Informed Decision-Making
Schoeffer, J. (2023) · Repository KITopen (Karlsruhe Institute of Technology) · doi:10.5445/ir/1000164741 · Cited by 3
Grounded and Transparent Response Generation for Conversational Information-Seeking Systems
Łajewska, W. (2024) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3616855.3635727 · Cited by 2
“By showing how confident the system is in its response, users can assess the reliability of the information provided [15, 22, 24].”
BackgroundMethodFélix, R., Correia, F., Pedroso-Roussado, C., & Nunes, N. (2026) · Discover Sustainability · doi:10.1007/s43621-025-02474-1 · Cited by 2
“This aligns with the need to carefully consider how AI systems are introduced to users to maximize their influence and acceptance [86, 90].”
InfluentialHow Do We Measure Over-Reliance? A Unified Probabilistic View
Mikhaylova, D., Turchi, T., & Malizia, A. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742414.3794719 · Cited by 2
“Recent studies focus on appropriate reliance [22], defined through suggestion correctness [7, 10, 24, 38].”
The Impact of AI Trustworthiness Labels on the Perception of AI Products
Pfeuffer, C. U. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790776 · Cited by 2
Wang, T. R., Yang, Y., & Zhou, J. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2686773 · Cited by 2
Parambath, A., Pulpo, G., & Hartman, V. (2026) · JMIR Human Factors · doi:10.2196/95644 · Cited by 2
“Without prior mental models, clinicians may struggle to distinguish rare but true details from model error and may find it difficult to trust these outputs [6,7].”
LLMs in text linguistics teaching: an exploratory study with genAI novices in higher education
Brocca, N., & Garassino, D. (2026) · Computers and Education Open · doi:10.1016/j.caeo.2026.100414 · Cited by 2
Kim, M. J., Jo, H., Park, H., & Cho, Y. (2024) · Interpretation · doi:10.1190/int-2024-0019.1 · Cited by 2
To Err Is AI! Debugging as an Intervention to Facilitate Appropriate Reliance on AI SystemsPreprint
He, G., Bharos, A., & Gadiraju, U. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.14377 · Cited by 2
“Such a reliance pattern has been defined as appropriate reliance [26, 33, 43, 51].”
Vereschak, O. (2022) · theses.fr (ABES) · doi:10.70675/7ecd53bfza7ccz4edfz8f1azdb8052ba1233 · Cited by 2
People-Centred Medical Image Analysis via Fairness-Aware Human-AI CooperationPreprint
Zhang, Z., Masroor, M., Nguyen, C., Hassan, T., Chen, Y., Rosewarne, D., Wells, K. D., Do, T.-T., & Carneiro, G. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2604.26991 · Cited by 2
“Recent research highlights the need for AI systems that not only assist but also interact effectively with human experts [25], with factors such as model confidence [26] and explainability [27] playing an important role in shaping that interaction.”
LLMs as Oracles: Reliance on LLMs for Subjective Personal QuestionsPreprint
Cheng, M., Ibrahim, L., Liu, G., Lam, M. S., Padmakumar, V., Madibekov, N., Yang, D., & Jurafsky, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.14849 · Cited by 2
Pfeuffer, C. U. (2025) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3765766.3765868 · Cited by 1
“They therefore (dis-)trust mainly based on heuristics [3, 16, 17] and strong, often unjustified AI endorsement [11], is coupled with low understanding of AI in the general public [11, 15].”
Human, I Need You: Comparison of Human-Robot Interaction Modalities for Search and Rescue Missions
Pretolesi, D., Gutiérrez, R., Regal, G., & Tscheligi, M. (2025) · International Conference on Mobile and Ubiquitous Multimedia (MUM) · doi:10.1145/3771882.3771895 · Cited by 1
Behzad, T., Gurney, N., Wang, N., & Pynadath, D. V. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-13187-4_8 · Cited by 1
AI as the Phantom Limb: The Asymmetry of Attribution in Human vs. AI Delegation
Chen, R. Y.-S., Hou, Y. T.-Y., Lin, Y., Liu, J. M.-E., Chen, W., & Chen, Y. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791075 · Cited by 1
“Delegation in Human–AI interaction refers to the process in which humans transfer task execution authority and, to some extent, outcome responsibility to AI agents [36, 66, 83].”
Lu, Z., Wang, W., Yu, Y., Wang, X., Li, Z., Liu, Z., & Ananiadou, S. (2026) · ACM Conference on Human-AI Complementarity and Alignment · doi:10.1145/3834580.3838736 · Cited by 1
Self-citationLindvall, M. (2021) · Linköping studies in science and technology. Dissertations · doi:10.3384/diss.diva-176117 · Cited by 1
Society Protect: Developing A Voting Mechanism for Blockchain-based AI Alignment
Neulinger, A., Sparer, L., Roshanaei, M., & Ramljak, D. (2024) · International Conference on Blockchain Computing and Applications (BCCA) · doi:10.1109/bcca62388.2024.10844386 · Cited by 1
“Artificial Intelligence (AI) developments are driving the adoption of the technology in business and daily life, e.g. using ChatGPT [1], which could result in humans making themselves dependent on AI decisions, even if feedback about the correctness of the generated decision is limited [2].”
A Complementarity-Enhanced Mixture of Human-AI Teams for Decision-Making
Liang, H., Liu, J., Guo, B., & Yu, Z. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05981-9_17 · Cited by 1
Designing human-AI alignment to improve collaborative decision-making
Ma, S. (2024) · doi:10.14711/thesis-991013340344203412 · Cited by 1
von Felten, N., Schöning, J., Opwis, K., & Scharowski, N. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.14591 · Cited by 1
Cau, F. M., & Spano, L. D. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.17509 · Cited by 2
Social aspects of Artificial Intelligence – selected issues
Kowalczyk, M. (2021) · Publisher House WSGE Alcide De Gasperi University of Euroregional Economy ul. Sienkiewicza 4 05-410 Józefów eBooks · doi:10.13166/wsge/psaz2564
Homophily and Incentive Effects in Use of AlgorithmsPreprint
Fogliato, R., Fazelpour, S., Gupta, S. K., Lipton, Z. C., & Danks, D. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2205.09701
“Prior work by Lu and Yin (2021) is the closest in approach to our study.”
BackgroundResultInfluentialFlexible Heuristics for Supporting Recommendations Within an AI Platform Aimed at Non-expert Users
Vázquez‐Ingelmo, A., García‐Holgado, A., García‐Peñalvo, F. J., Andrés-Fraile, E., Pérez-Sánchez, P., Muiños, P. J. A., Sánchez-Puente, A., Vicente-Palacios, V., Dorado-Díaz, P. I., Cruz‐González, I., et al. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-33023-0_30
Recognizing Selected Features in the News Text
Kaczmarek, A., Tobolewski, F., Ganzha, M., Paprzycki, M., & Wróblewska, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-86193-2_17
Parsonage, G. (2024) · CLOK (University of Central Lancashire) · doi:10.17030/uclan.thesis.00052603
Machine learning in forecasting and decision-making: Merging AI and human expertise
Ketola, J. (2024) · Aaltodoc (Aalto University)
Explaining Unintuitive Feature Importance Explanations
Qu, J. (2025) · Carolina Digital Repository (University of North Carolina at Chapel Hill) · doi:10.17615/pbr7-sj68
Designing for Organizational Intelligence with Machine Learning
Schaschek, M. (2025) · Online Publication Service of Würzburg University (Würzburg University) · doi:10.25972/opus-40162
The Effect of Social Information on Reliance and Efficacy in AI-assisted Prediction
Alsobay, M. (2024) · DSpace@MIT (Massachusetts Institute of Technology)
Evaluating Innovation with AI: A Behavioral Analysis of Decision-Making in Startup Assessments
Costa, A. S. G. V. S. (2025) · Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT)
Human in the loop: Integration of experts in MR data processing
Carneiro, G. (2025) · Advances in magnetic resonance technology and applications · doi:10.1016/b978-0-443-14109-6.00018-3
Believing vs. Achieving -- The Disconnect between Efficacy Beliefs and Collaborative OutcomesPreprint
Spitzer, P., & Holstein, J. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2603.10708
“…information on the underlying data [64], explanations of the AI’s reasoning [94], or performance metrics of the AI across different scenarios [39, 68] influence reliance decisions, our contribution lies in systematically distinguishing between general efficacy beliefs formed prior to interaction…”
Ryu, K., Ghang, H. (., Chon, S., Lee, K., & Kang, Y. (2026) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3772363.3798817
“ThisworkreframestransparencyinLLM-basedbiomedicalquestion-answeringsystemsasaninfrastructureforverificationratherthan a mechanism for increasing trust or acceptance [21, 23].”
InfluentialFor Those About to Rely—A Taxonomy of Experimental Studies on AI Reliance: Completed Research Paper
Schaschek, M., Spatscheck, N., & Winkelmann, A. (2026) · Lecture notes in information systems and organisation · doi:10.1007/978-3-032-08480-4_2
Put Ai to Use: AI Prediction Versus Human Judgement
Göndöcs, D. (2025) · Széchenyi István Egyetem · doi:10.15477/sze.rgdi.2025.004
VILLAVICENCIO, M., Pan, S., & Wang, Q. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812329 · Cited by 1
Human-AI Interaction in Healthcare - A Manifesto for Improved Usability Evaluation
Oliveira, M., Cruz, C. F., & Matela, N. (2026) · ACM Interactive Health Conference · doi:10.1145/3786579.3804937
“…the clinician accepts the AI’s output without interacting with the raw data (e.g., viewing the original OCT scan or patient notes), where a steady increase in agreement over a few weeks/months would suggest automation complacency (where the user stops performing independent verification) [20].”
Developing and Validating the GenAI Reliance Scale in Business Practices
Abu-Elsamen, A. A., & Elhajjar, S. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2689524
Hunter, R. (2026) · Warwick Research Archive Portal (University of Warwick) · doi:10.82444/warw.33915685
Tajima, H. (2026) · JMIR Nursing · doi:10.2196/99590
“The leave-one-study-out cross-validation produced recalibrated β A values of 0.094 (excluding Lu and Yin [15]), 0.246 (excluding experiment 3 by Yin et al [14]), and 0.199 (excluding experiment 1 phase 1 by Yin et al [14]).”
InfluentialBias in Algorithm‐Assisted Decision‐Making: A Theory‐Based Intervention
Li, A., & Fan, S. (2026) · Journal of Applied Social Psychology · doi:10.1111/jasp.70083 · Cited by 1
The Belief Update Gate: Separating Inertia from Learning in Human–AI Interaction
Biswas, S., Erlei, A., & Gadiraju, U. (2026) · ACM Conference on Human-AI Complementarity and Alignment · doi:10.1145/3834580.3838740
Li, Z. "., Freeman, B., Raman, A., Rajkumar, K. A., Fang, X., Srivastava, R., Feiner, S., & Thakoor, K. A. (2026) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3830398.3830722
“Yet human–AI teams frequently fail to outperform the stronger agent alone due to miscali-brated trust and poor interaction design [69]: participants may over-accept AI advice or be distracted without accuracy gains [4, 11, 38, 48, 55, 71, 83].”
Available but Unclaimed: An Empirical Study of Human-AI SynergyPreprint
Welsch, R., Rausch, M., Knierim, P., Kosch, T., Kuhn, J., Schmidt, A., & Fernandes, D. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.16793
“Without corrective feedback, reliance hardens into coarse heuristics about whether to follow the system at all [47], and impressions of its competence persist despite repeated disconfirmation [16].”
Confident, Not Wiser: The Dunning-Kruger Effect in Human-AI InteractionPreprint
Fernandes, D., Rausch, M., Kloft, A. M., Buschek, D., & Welsch, R. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2609.31095
Kang, E. J., Peter, Duan, & Mishra, S. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.07800
“Existing behavioral measures such as Agreement Score [6, 27] and Switch Fraction [38] have been used to study reliance across a range of factors including explanation usability [44], cognitive load [1], and prior beliefs [45].”
Macca, L. S., Sestino, A., & Bresciani, S. (2026) · Competitiveness Review An International Business Journal incorporating Journal of Global Competitiveness · doi:10.1108/cr-02-2026-0105
When Trust Bends but Does Not Break: Trust During Algorithm-Supported Decision-Making
Sandia National Laboratories (SNL-NM), A., Sanchez, D. N., (NNSA), U. N. N. S. A., Winokur, E. J., Lancaster, C., Villarreal, R. T., Bradley, S., Schutte, P., Ingram, J. B., Bruner, T., et al. (2026) · OSTI OAI (U.S. Department of Energy Office of Scientific and Technical Information) · doi:10.2172/17315896
Meng, Y., Zhan, J., Lyu, Y., Lu, Z., & Zang, C. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2610.23276
“Automation misuse and bias describe users following outputs beyond their reliability or failing to seek contradictory information [33, 60, 87, 107].”
Self-citationAI Reliance in the Fantasy American Football Lineup-Setting Task
Nicholson, M. B., Keegan, B., & Semaan, B. (2026) · doi:10.1145/3838843.3838850
Schömbs, S. V. (2026) · Minerva Access (University of Melbourne)
Do People Trust AI Assistants Even When They Are Consistently Wrong?Preprint
Jones, C. R., Wong, J., Cohen, A. S., Woolley, J., & Bergen, B. K. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2611.14833
Metadata
- Authors
- Lu, Z., & Yin, M.
- Affiliations
- Purdue University West Lafayette (US)
- ORCID
- Venue
- Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2021-05-06
- Pages
- 1-16
- OpenAlex
- W3163667721
- Cited by
- 131
- References
- 60
- Citations by year
- 2026: 16 · 2025: 30 · 2024: 35 · 2023: 24 · 2022: 22 · 2021: 4
- FWCI
- 36.25
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- machine learning models, performance feedback, trust in AI, human-AI agreement, heuristics, AI-assisted decision-making, decision aids
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.