Advancing Human-AI Complementarity: The Impact of User Expertise and Algorithmic Tuning on Joint Decision Making
Open accessThe DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index.
Abstract
Human-AI collaboration for decision-making strives to achieve team performance that exceeds the performance of humans or AI alone. However, many factors can impact success of Human-AI teams, including a user’s domain expertise, mental models of an AI system, trust in recommendations, and more. This article reports on a study that examines users’ interactions with three simulated algorithmic models, all with equivalent accuracy rates but each tuned differently in terms of true positive and true negative rates. Our study examined user performance in a non-trivial blood vessel labeling task where participants indicated whether a given blood vessel was flowing or stalled. Users completed 140 trials across multiple stages, first without an AI and then with recommendations from an AI-Assistant. Although all users had prior experience with the task, their levels of proficiency varied widely. Our results demonstrated that while recommendations from an AI-Assistant can aid in users’ decision making, several underlying factors, including user base expertise and complementary human-AI tuning, significantly impact the overall team performance. First, users’ base performance matters, particularly in comparison to the performance level of the AI. Novice users improved, but not to the accuracy level of the AI. Highly proficient users were generally able to discern when they should follow the AI recommendation and typically maintained or improved their performance. Mid-performers, who had a similar level of accuracy to the AI, were most variable in terms of whether the AI recommendations helped or hurt their performance. Second, tuning an AI algorithm to complement users’ strengths and weaknesses also significantly impacted users’ performance. For example, users in our study were better at detecting flowing blood vessels, so when the AI was tuned to reduce false negatives (at the expense of increasing false positives), users were able to reject those recommendations more easily and improve in accuracy. Finally, users’ perception of the AI’s performance relative to their own performance had an impact on whether users’ accuracy improved when given recommendations from the AI. Overall, this work reveals important insights on the complex interplay of factors influencing Human-AI collaboration and provides recommendations on how to design and tune AI algorithms to complement users in decision-making tasks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“As clinicians gain experience, they develop a deeper understanding of clinical contexts and decision-making processes, which may influence both trust in and reliance on CDSSs [ 8 , 15 - 17 ].”
From Psychological Factors Influencing Appropriate Reliance on AI-enabled Clinical Decision Support Systems: Experimental Web-Based Study Among Dermatologists (2025) · cites this paper as [17]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Collaborative systems have shown that humans can correct model predictions in specific instances1, and that humans, in collaboration with a Machine Learning (ML) model, can achieve higher performance than either the expert or the model alone3.”
From A benchmarking framework and dataset for learning to defer in human-AI decision-making (2025) · cites this paper as [3]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Theoretical Extension of the Technology Acceptance Model: Four Longitudinal Field Studies
Venkatesh, V., & Davis, F. D. (2000) · Management Science · doi:10.1287/mnsc.46.2.186.11926 · Cited by 23,066
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605
Borderline-SMOTE : A New Over-Sampling Method in Imbalanced Data Sets Learning
Han, H., Wang, W.-Y., & Mao, B.-H. (2005) · Lecture notes in computer science · doi:10.1007/11538059_91 · Cited by 4,331 · cited 1×
“Such adjustments can be implemented in practice either by oversampling positive or negative examples [26], assigning diferent costs to diferent examples during learning [20], or simply by assigning diferent conidence thresholds for detection depending on whether the goal is to reduce the number of false positives or f…”
BackgroundAlgorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232
Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817
Proceedings of the International Joint Conference on Artificial Intelligence 2007
Bundy, A. (2007) · International Joint Conference on Artificial Intelligence · Cited by 2,761
Proceedings of the 16th international joint conference on Artificial Intelligence - IJCAI '99
Colton, S., Bundy, A., & Walsh, T. (1999) · Cited by 2,500
Guidelines for Human-AI Interaction
Amershi, S., Weld, D., Vorvoreanu, M., Fourney, A., Nushi, B., Collisson, P., Suh, J., Iqbal, S., Bennett, P. N., Inkpen, K., et al. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300233 · Cited by 2,439 · cited 1×
“Previous work has emphasized the importance of providing such information in order to set expectations on performance [3, 33, 61].”
BackgroundSelf-citationConfirmation, disconfirmation, and information in hypothesis testing
Klayman, J., & Ha, Y.-W. (1987) · Psychological Review · doi:10.1037/0033-295x.94.2.211 · Cited by 2,094 · cited 1×
“Klayman & Ha also find that the most critical way to test hypotheses is a "positive test strategy," where participants examine cases with the target characteristic and retrieve knowledge from their memory related to the assumption provided by an anchor [32].”
BackgroundAlgorithm appreciation: People prefer algorithmic to human judgment
Logg, J. M., Minson, J. A., & Moore, D. A. (2019) · Organizational Behavior and Human Decision Processes · doi:10.1016/j.obhdp.2018.12.005 · Cited by 2,061
Research works printed but not linked: 2
- 10.1145/3449287 · doi:10.1145/3449287 · Paper page
- 10.1518/hfes.46.1.50.30392 · doi:10.1518/hfes.46.1.50.30392 · Paper page
Cited by
From explainable to interactive AI: A literature review on current trends in human-AI interaction
Raees, M., Meijerink, I., Lykourentzou, I., Khan, V.-J., & Papangelis, K. (2024) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2024.103301 · Cited by 180
Understanding the Role of Human Intuition on Reliance in Human-AI Decision-Making with Explanations
Chen, V., Liao, Q. V., Wortman Vaughan, J., & Bansal, G. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610219 · Cited by 220
Complementarity in human-AI collaboration: concept, sources, and evidence
Hemmer, P., Schemmer, M., Kühl, N., Vössing, M., & Satzger, G. (2025) · European Journal of Information Systems · doi:10.1080/0960085x.2025.2475962 · Cited by 238
“…the story: On the one hand, various studies have demonstrated that human-AI teams are able to outperform human individuals (Alufaisan et al., 2021; Inkpen et al., 2023; Liu et al., 2021; Sarkar et al., 2023)—often not analyzing, though, whether they also surpass the AI model’s individual…”
AI makes you smarter but none the wiser: The disconnect between performance and metacognition
Fernandes, D., Villa, S., Nicholls, S., Haavisto, O., Buschek, D., Schmidt, A., Kosch, T., Shen, C., & Welsch, R. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2025.108779 · Cited by 185
“Fundamental biases, such as overtrust and overreliance, impair performance [46] up to the point that the interaction decreases overall performance as compared to having no AI at all [7, 85].”
A cognitive approach to human–AI complementarity in dynamic decision-making
González, C., & Heidari, H. (2025) · Nature Reviews Psychology · doi:10.1038/s44159-025-00499-x · Cited by 93
“Data-driven AI systems play an essential part in supporting dynamic decision-making in many domains 4,5 .”
Peng, L., Li, D., Zhang, Z., Zhang, T., Huang, A., Yang, S., & Hu, Y. (2024) · International Journal of Industrial Ergonomics · doi:10.1016/j.ergon.2024.103629 · Cited by 59
Identity, crimes, and law enforcement in the Metaverse
Qin, H.-X., Wang, Y.-Y., & Hui, P. (2022) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-04266-w · Cited by 65
“We discuss whether fantasy crimes should be criminalized in “Law Enforcement”.”
Psychological Traits and Appropriate Reliance: Factors Shaping Trust in AI
Küper, A., & Krämer, N. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2024.2348216 · Cited by 49
“Especially when investigating over-reliance on automation, task expertise has been listed as one of the most critical factors for following AI advice within the expertise area (Cabitza et al., 2023; Gaube et al., 2022; Inkpen et al., 2023).”
To Rely or Not to Rely? Evaluating Interventions for Appropriate Reliance on Large Language Models
Bo, J. Y., Wan, S., & Anderson, A. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714097 · Cited by 36
“In these tasks, the LLM can provide guidance and information for the user to assist in their decision-making, learning, and creations [89].”
RealtimeGen: An Intervenable AI Image Generation System for Commercial Digital Art Asset Creators
Li, Z., Zhang, Y., Zhou, S., Liu, Q., Zhang, J., Xu, H., Chen, S., Chen, X., & Sun, L. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2382508 · Cited by 44
“Two predominant viewpoints exist: one advocating AI’s role in replacing and reducing human labor, while the other champions a human-centric approach, where humans engage with AI to exert their subjective initiative (Inkpen et al., 2023).”
Trust Development and Repair in AI-Assisted Decision-Making during Complementary Expertise
Pareek, S., Velloso, E., & Gonçalves, J. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3658924 · Cited by 54
Advice from artificial intelligence: a review and practical implications
Baines, J. I., Dalal, R. S., Ponce, L. P., & Tsai, H.-C. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1390182 · Cited by 28
An Evaluation of Situational Autonomy for Human-AI Collaboration in a Shared Workspace Setting
Salikutluk, V., Schöpper, J., Herbert, F., Scheuermann, K., Frodl, E., Balfanz, D., Jäkel, F., & Koert, D. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642564 · Cited by 55
Göndöcs, D., Horváth, S., & Dörfler, V. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103622 · Cited by 52
Küper, A., Lodde, G. C., Livingstone, E., Schadendorf, D., & Krämer, N. (2025) · Journal of Medical Internet Research · doi:10.2196/58660 · Cited by 31
“As clinicians gain experience, they develop a deeper understanding of clinical contexts and decision-making processes, which may influence both trust in and reliance on CDSSs [8,15-17].”
Luo, Y. (2025) · Advances in Human-Computer Interaction · doi:10.1155/ahci/3869207 · Cited by 40
Generative AI in Knowledge Work: Design Implications for Data Navigation and Decision-Making
Yun, B., Feng, D., Chen, A. S., Nikzad, A., & Salehi, N. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713337 · Cited by 38
“This literature emphasizes the nuances of joint decision-making, as user background knowledge can affect the problem-solving process and results [28], with trust and explainability playing key roles in the adoption of generative AI applications [1].”
Chaturvedi, A., Yadav, N., & Dasgupta, M. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2456534 · Cited by 35
“This dynamic reflects a growing need for human-AI collaboration, where human expertise and AI’s computational power work to create more effective strategic decisions (Di Vaio et al., 2022; Haesevoets et al., 2021; Inkpen et al., 2023).”
Toward a science of human–AI teaming for decision making: a complementarity framework
Gonzalez, C., Donahue, K., Goldstein, D. G., Heidari, H., Jalali, M. S., Schelble, B., Singh, A., & Woolley, A. W. (2026) · PNAS Nexus · doi:10.1093/pnasnexus/pgag030 · Cited by 33
“Novices and experts may benefit from different collaboration modes (eg more guidance and guardrails for novices; “sparring partner” use for experts) (101).”
Greiner, D., & Lemoine, J.-F. (2025) · Journal of Services Marketing · doi:10.1108/jsm-02-2024-0056 · Cited by 18
Khac, L. T. D., & Leyer, M. (2025) · Technology in Society · doi:10.1016/j.techsoc.2025.103064 · Cited by 25
Machine Learning-Assisted Polymer and Polymer Composite Design for Additive Manufacturing
Gyabaah, K. Y., Mahoney, B., Martey, A. K., Yan, C., Mensah, P. F., & Li, G. (2026) · AI Materials · doi:10.3390/aimater1010002 · Cited by 15
Qin, H. X., Zhu, G., Fan, M., & Hui, P. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713569 · Cited by 31
“To prior LLM work on story generation based on a moral, we add insights on the complementary roles of AI and humans [21, 64], with AIs suggesting general views of a greater variety of audience groups and humans adding “individ-ualized” nuances (Section 7.1.4 and [51, 105]).”
InfluentialDon’t Be Fooled: The Misinformation Effect of Explanations in Human–AI Collaboration
Spitzer, P., Holstein, J., Morrison, K., Holstein, K., Satzger, G., & Kühl, N. (2026) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2025.2574511 · Cited by 12
Envisioning AI Support during Semi-Structured Interviews Across the Expertise Spectrum
Liu, Z., Dai, J., Conati, C., & McGrenere, J. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710909 · Cited by 25
“Expertise may also be considered a factor influencing collaboration effectiveness, such as “novice users may not utilize AI assistance as effectively as those with medium and high proficiency” [34].”
A benchmarking framework and dataset for learning to defer in human-AI decision-making
Alves, J. V., Leitão, D., Jesus, S., Sampaio, M. O. P., Liébana, J., Saleiro, P., Figueiredo, M. A. T., & Bizarro, P. (2025) · Scientific Data · doi:10.1038/s41597-025-04664-y · Cited by 10
“Collaborative systems have shown that humans can correct model predictions in specific instances 1 , and that humans, in collaboration with a Machine Learning (ML) model, can achieve higher performance than either the expert or the model alone 3 .”
Alexandropoulou, C., Panagiotopoulos, I. E., Kleanthous, S., Dimitrakopoulos, G. J., Constantinou, I. P., Politi, E., Ntalaperas, D., Papageorgiou, X. S., Stylianides, C., Ioannides, N., et al. (2023) · IEEE International Conference on eScience · doi:10.1109/e-science58273.2023.10254863 · Cited by 14
Breckner, K., Neumayr, T., Mara, M., Streit, M., & Augstein, M. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2482742 · Cited by 16
Semantic interlinking of Immigration Data using LLMs for Knowledge Graph Construction
Venkatakrishnan, R., Tanyildizi, E., & Canbaz, M. A. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589335.3651557 · Cited by 10
Ashktorab, Z., Desmond, M., Pan, Q., Johnson, J., Brachman, M., Dugan, C., Danilevsky, M., & Geyer, W. (2025) · Proceedings of the 4th Annual Symposium on Human-Computer Interaction for Work · doi:10.1145/3729176.3729179 · Cited by 10
Raees, M., Khan, V.-J., & Papangelis, K. (2025) · ACM Conference on User Modeling, Adaptation and Personalization (UMAP) · doi:10.1145/3699682.3728320 · Cited by 9
How Managers Perceive AI-Assisted Conversational Training for Workplace Communication
Wilhelm, L. A., Ding, X., Knutsen, K. M., Carik, B., & Rho, E. H. R. (2025) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3719160.3736639 · Cited by 15
Workshop on Understanding and Mitigating Cognitive Biases in Human-AI Collaboration
Boonprakong, N., He, G., Gadiraju, U., van Berkel, N., Wang, D., Chen, S., Liu, J., Tag, B., Gonçalves, J., & Dingler, T. (2023) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3584931.3611284 · Cited by 13
“Moreover, different individuals, such as those with varying levels of expertise, possess different mental models of interacting with and understanding AI systems [25, 53].”
BackgroundHuman–AI collaboration in operations and supply chain management: a systematic literature review
Kumar, N., & Kumar, R. (2025) · Management Review Quarterly · doi:10.1007/s11301-025-00575-9 · Cited by 19
Yao, X., Saha, A., Saravanan, S., Low, A., & SUSSMAN, J. S. (2025) · Cancer Innovation · doi:10.1002/cai2.70021 · Cited by 8
“Furthermore, user proficiency affects human – AI collaborations in general, so the effect of user proficiency on performance could be another direction of research in AI ‐ assisted systematic reviews [27, 28].”
Berretta, S., Tausch, A., Bülow, F., Kuhlenkötter, B., Topp, M., Els, C., Peifer, C., & Kluge, A. (2025) · Applied Ergonomics · doi:10.1016/j.apergo.2025.104669 · Cited by 9
Bridging Expertise Gaps: The Role of LLMs in Human-AI Collaboration for CybersecurityPreprint
Tariq, S., Singh, R., Chhetri, M. B., Nepal, ., & Paris, C. L. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5462737 · Cited by 14
Raees, M., Khan, V.-J., Lykourentzou, I., & Papangelis, K. (2026) · Proceedings of the 2026 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791467 · Cited by 5
“(2) Human-AI Decision-Making: Articles that studied human-AI assistance, but did not focus on or measure objective user behavior for reliance measures [69] (i.e., only capturing users’ trust perception/feedback [124, 158]) were excluded.”
Patole, S. R. (2024) · Performance Improvement Quarterly · doi:10.56811/piq-24-0039 · Cited by 7
AI constructs gendered struggle narratives: Implications for self-concept and systems design.
Fitzsimons, A., Gerber, E. M., & Long, D. (2025) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3715275.3732156 · Cited by 10
Schrills, T., Franke, T., Hoesterey, S., & Roesler, E. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2553153 · Cited by 5
“While it was essential for our research to have a task where users could not differ in pre-existing knowledge, collaborative human-AI tasks involve experienced humans, and expertise can change the result of the interaction (Inkpen et al. 2023).”
Mondrus, O. V., & SimanTov‐Nachlieli, I. (2025) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2025.100857 · Cited by 6
Liyanage, K. P. T. L. W., & Balalle, H. (2024) · Open Journal of Applied Sciences · doi:10.4236/ojapps.2024.149175 · Cited by 6
This One Weird Trick Gets Users to Stop Clicking on Clickbait
Shrestha, A., Behfar, A., Seng, S., Wright, M., & Al-Ameen, M. N. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2464890 · Cited by 4
“In that regard, mental models that refer to users’ understanding of a concept can play an important role in personalization (Dumaru et al., 2024; Inkpen et al., 2023; Kaptein et al., 2015; Liu et al., 2016; Paudel et al., 2023; Young, 2008).”
Cognitive Spillover in Human–AI Teams
Riedl, C., Savage, S., & Zvelebilova, J. (2026) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3805039 · Cited by 28
“While recent work has made significant advances in understanding human-AI interaction [8, 13, 30, 61, 100, 105, 130, 131], prior work focuses largely on effects within the human-AI dyad during active use [e.g., 76, 118, 133], without testing whether such effects produce spillover into human-human…”
Lancaster, C., Flathmann, C., Hsu, J., McNeese, N. J., O’Neill, T., & Salas, E. (2025) · Journal of Organizational Behavior · doi:10.1002/job.70036 · Cited by 4
Sharfaei, S., & Rompho, N. (2026) · Journal of Economic Surveys · doi:10.1111/joes.70072 · Cited by 3
Artificial Intelligence in Human Collaboration Skills Invocation, Recovery and Enhancement
Ridley, D., Llaugel, F., & Garcia, C. A. (2026) · Theoretical Economics Letters · doi:10.4236/tel.2026.161017 · Cited by 3
Human-AI Teaming: Following the IMOI Framework
Kleanthous, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60611-3_27 · Cited by 6
CISepsis: a causal inference framework for early sepsis detection
Li, Q., Li, D., He, J., Wu, Z., & Nie, W. (2024) · Frontiers in Cellular and Infection Microbiology · doi:10.3389/fcimb.2024.1488130 · Cited by 9
Erschließung handschriftlicher Dokumente zwischen Fachwissen, Citizen Science und KI
Rehbein, M., & Ernst, M. (2023) · BIBLIOTHEK Forschung und Praxis · doi:10.1515/bfp-2023-0043 · Cited by 3
Anik, A. I., & Bunt, A. (2024) · IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) · doi:10.1109/vl/hcc60511.2024.00024 · Cited by 6
“We included AI literacy as a covariate given prior work suggesting that people with differing levels of AI expertise might perceive the explanations differently [19], [60], [91] .”
BackgroundVepřek, L. H. (2024) · Science studies · doi:10.14361/9783839472286 · Cited by 4
Self-citationValidation of an acute kidney injury prediction model as a clinical decision support system
Yun, G., Yi, J., Han, S., Seong, J., Menadjiev, E., Han, H., Choi, J., Kim, J. H., & Kim, S. (2025) · Kidney Research and Clinical Practice · doi:10.23876/j.krcp.24.163 · Cited by 5
“These findings align with research on human-AI collaboration, revealing that the benefits of AI support depend on users’ expertise [20].”
Molaei, S., Robert, L., & Banović, N. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757437 · Cited by 2
“We extracted the facts from the human-annotated metadata for each driving scenario.”
Guo, C., Fang, C., Zhang, W., & Troyer, J. (2025) · Informatics · doi:10.3390/informatics12040120 · Cited by 4
“These questions further complicate the growing interest in human–AI complementarity and positions AI to support rather than replace healthcare practitioners in the integration of geriatric and palliative care [15–19].”
The Enterprising and Elusive Prospects of Human-AI Collaboration
Gadiraju, U., & Balayn, A. (2025) · Springer eBooks · doi:10.1007/978-3-032-01940-0_7 · Cited by 5
Generative AI for autonomous data analytics
Fikardos, M., Lepenioti, K., Bousdekis, A., Apostolou, D., & Mentzas, G. (2026) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2026.200626 · Cited by 3
Chen, S., & Lou, J. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1727544 · Cited by 2
“Inkpen et al. (2023) further elaborated on how user expertise moderates human-AI collaboration.”
InfluentialUsers’ mental models within human–AI interaction: a systematic scoping review
Bodamer, W. G., Schoenmakers, S., Castellar, E. N., & IJsselsteijn, W. A. (2026) · AI & Society · doi:10.1007/s00146-026-03038-1 · Cited by 5
Ding, H., Shen, Y., Chen, J., & Wang, P. (2026) · Information Processing & Management · doi:10.1016/j.ipm.2026.105046 · Cited by 5
Human Guidance Approaches for the Genetic Improvement of Software
Craine, B., Rainford, P. F., & Porter, B. (2024) · doi:10.1145/3643692.3648263 · Cited by 2
Action vs. Attention Signals for Human-AI Collaboration: Evidence from ChessPreprint
Poulidis, S., Ge, H., Bastani, H., & Bastani, O. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5128584 · Cited by 8
A Systematic Literature Review on Human-AI Collaboration in the Information Systems Research
Li, C., Li, X., & Long, T. (2025) · Lecture notes in business information processing · doi:10.1007/978-3-031-94184-9_6 · Cited by 5
Attorneys and AI: How Lawyers Use Artificial Intelligence and Analyze Its Impacts
Schieber, E. A. G., Kite, N., Hall, M. I., Turner, C., & Schlesinger, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757554 · Cited by 3
“In response to growing concerns over AI’s role in work, a growing body of work has dedicated itself to bringing a broad worker’s perspective of the possible transformation of knowledge work [126, 136] and collaborative work [67, 71, 92].”
Xu, S. X., Ye, Q., Wang, H., Ren, Z., Jiang, G., Gao, C., Xia, H., Xie, H., Hu, D., Chang, J., et al. (2025) · 中国科学基金 · doi:10.3724/bnsfc-2025.06.07.0002 · Cited by 3
Understanding the influence of design-related factors on human–AI teaming in a face matching task
Kokje, E., Lermer, E., Donkin, C., & Gaube, S. (2026) · Cognitive Research Principles and Implications · doi:10.1186/s41235-025-00701-x · Cited by 3
“Individually, both humans and AI aids are susceptible to errors, but together they may achieve higher performance, as each may be capable of overcoming the shortcomings of the other (Inkpen et al., 2022; Jarrahi, 2018; Johnson & Vera, 2019; Kamar, 2016).”
Who listens to ChatGPT and when should they? A two-study examination of AI-assisted decision making
Grawitch, M. J., Winton, S. L., & Mudigonda, S. P. (2026) · Computers in Human Behavior Artificial Humans · doi:10.1016/j.chbah.2026.100283 · Cited by 3
How Do We Measure Over-Reliance? A Unified Probabilistic View
Mikhaylova, D., Turchi, T., & Malizia, A. (2026) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3742414.3794719 · Cited by 2
Who Does What? Archetypes of Roles Assigned to LLMs During Human-AI Decision-Making
Chappidi, S., Singh, J., & Krauze, A. V. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791428 · Cited by 8
Self-citationNanda, P. K. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2697344 · Cited by 1
Absent Presence: The Human Influence in AI-Generated Content in the Age of Technoculture
Vosevich, K., & Hutson, J. (2024) · International Journal of Emerging and Disruptive Innovation in Education VISIONARIUM · doi:10.62608/2831-3550.1025 · Cited by 1
“Day by day, these GAI algorithms and models are advancing in areas once thought to be the exclusive domain of human expertise (Inkpen et al., 2023).”
A Crossroads for Hybrid Human-Machine Decision-Making
Wilson, B., Lakshmanan, K., Dix, A. J., Rahat, A., & Roach, M. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-74627-7_24 · Cited by 2
Counselor-AI Collaborative Transcription and Editing System for Child Counseling Analysis
Lee, H., Lee, J., Yang, M., Lee, D., Song, H., Han, Y., & Han, J. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712081 · Cited by 3
“In this context, human intervention in adjusting AI-generated outcomes can help mitigate biases in expert decision-making [73], and human-AI collaboration demonstrates improved performance by balancing the limitations of both AI and human expertise [39].”
Liu, Y., Shen, R., Zhou, L., Xiao, Q., Yuan, J., & Li, Y. (2024) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2024.05.19.594895 · Cited by 1
Visionarium, June 2024, Full Issue
(2024) · International Journal of Emerging and Disruptive Innovation in Education VISIONARIUM · doi:10.62608/2831-3550.1023
A metacognitive approach to learning and performance in human-AI interaction
Fan, Y., Yan, L., Molenaar, I., Greiff, S., & Gašević, D. (2026) · Communications Psychology · doi:10.1038/s44271-026-00548-y · Cited by 1
6 Intraversions: Human-Technology Relations in Flux
Veprek, L. H. (2024) · Science studies · doi:10.14361/9783839472286-009
Self-citation5 Multiple Meanings and Everyday Negotiations: Play/Science Entanglements
Veprek, L. H. (2024) · Science studies · doi:10.14361/9783839472286-008
Self-citationVeprek, L. H. (2024) · Science studies · doi:10.14361/9783839472286-011
Self-citationVepřek, L. H. (2024) · Science studies · doi:10.14361/9783839472286-003
Self-citation7 Building Trust in and With Human Computation
Veprek, L. H. (2024) · Science studies · doi:10.14361/9783839472286-010
Self-citation2 Approaching Human Computation-Based Citizen Science Analytically
Veprek, L. H. (2024) · Science studies · doi:10.14361/9783839472286-005
Self-citationVepřek, L. H. (2024) · Science studies · doi:10.14361/9783839472286-001
Self-citation3 Methodology: Encountering Human Computation Ethnographically
Veprek, L. H. (2024) · Science studies · doi:10.14361/9783839472286-006
Self-citationVepřek, L. H. (2024) · Science studies · doi:10.14361/9783839472286-toc
Self-citationVepřek, L. H. (2024) · Science studies · doi:10.14361/9783839472286-fm
Self-citationVepřek, L. H. (2024) · Science studies · doi:10.14361/9783839472286-012
Self-citation1 Introduction: "We're Doing Something Completely New"
Veprek, L. H. (2024) · Science studies · doi:10.14361/9783839472286-004
Self-citation4 Envisioning and Designing the Future
Veprek, L. H. (2024) · Science studies · doi:10.14361/9783839472286-007
Self-citationVepřek, L. H. (2024) · Science studies · doi:10.14361/9783839472286-002
Self-citationVepřek, L. H. (2024) · Science studies · doi:10.14361/9783839472286-013
Self-citationInclusive AI for radiology: Optimising ChatGPT-4 with advanced prompt engineering
Yasmeen, J., Qamar, M. T., & Yasmeen, S. (2024) · Clinical Imaging · doi:10.1016/j.clinimag.2024.110385
Daish, P., Roach, M., & Dix, A. J. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-74627-7_25
Luo, Y., Gkouskos, D., & Russo, N. L. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93236-6_2
Exploring Flow in IT Professionals’ Use of AI-Integrated Tools: Insights from Interviews
Lange, E. M., Cajander, Å., & Normark, M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93429-2_3 · Cited by 1
AI-Guided Professional Development
Hernandez-Ledezma, L. J., & Janecek, U. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-1122-7.ch009
Liang, B., Zhang, L., & Huo, W. (2025) · International Conference on Management Science and Computer Engineering (MSCE) · doi:10.1145/3760023.3760044
“High-ability individuals often rely on peer collaborators to compare ability, which supports motivation and performance expectations.AI lacks social status and reference attributes, so high-ability individuals have no clear benchmarks for engagement and flow [ 8 ] .”
Deng, N., Hu, Y., & Jiang, X. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5766511
Cruz, A. (2026) · JPAIR Multidisciplinary Research · doi:10.7719/jpair.v63i1.975
Metadata
- Authors
- Inkpen, K., Chappidi, S., Mallari, K., Nushi, B., Ramesh, D., Michelucci, P., Mandava, V., Vepřek, L. H., & Quinn, G.
- Affiliations
- Microsoft (United States) (US)
- University of Virginia (US)
- University of Washington (US)
- University of Michigan (US)
- Human Computation Institute (US)
- Ludwig-Maximilians-Universität München (DE)
- Western Washington University (US)
- ORCID
- Venue
- ACM Transactions on Computer-Human Interaction
- Published
- 2023-09-23
- Volume
- 30
- Issue
- 5
- Pages
- 1-29
- ISSN
- 1073-0516 (ISSN-L)
- 1557-7325
- OpenAlex
- W4323841539
- Cited by
- 132
- References
- 58
- Citations by year
- 2026: 48 · 2025: 44 · 2024: 36 · 2023: 4
- FWCI
- 39.07
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- ACM Transactions on Computer-Human Interaction (publishedVersion)
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Keywords
- human-AI collaboration, shared decision-making, user expertise, false positive and false negative rates, user proficiency, trust in AI recommendations, team performance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.