Exploring automation bias in human–AI collaboration: a review and implications for explainable AI
Open accessThe DOI matches its Crossref record; 75 of the 75 references it cites were found in a registry or an index; 17 of the 264 sources its claims cite were checked for support.
Abstract
Systematic review of automation bias in high-stakes human-AI collaboration, examining cognitive mechanisms, expertise, trust dynamics, verification demands, and the limits of explainable AI for mitigating overreliance.
Overview
CoChat Summary
## TL;DR
This paper is a PRISMA-based review of automation bias in human-AI collaboration, synthesizing 35 peer-reviewed experimental studies from 2015–April 2025. It concludes that trust, task complexity, AI literacy, expertise, and individual differences shape overreliance, while explainability/transparency alone often do not prevent automation bias and can sometimes worsen it.
## Key findings
- The review included 35 studies, published across 21 journals, with a combined 19,774 participants; all included studies used quantitative, experimental methods.
- Trust and task complexity emerged as the main drivers of automation bias; the paper reports that trust accounted for up to 24.1% of the variance in reliant behavior in one study (Küper et al. 2025).
- Explainability was mixed or limited: CFFs reduced reliance on incorrect AI recommendations compared with simple XAI, but were seen as more cognitively demanding; multiple explanation types did not improve detection of incorrect AI recommendations, and trust calibration feedback did not help.
- The review reports several quantitative examples of bias effects, including Bond et al. (2018) where incorrect automated diagnoses reduced ECG interpretation accuracy from 84.86% to 41.66% for fellows and from 86.38% to 27.43% for non-fellows, and Dratsch et al. (2023) where mammography accuracy dropped from 79.7% to 19.8%, 81.3% to 24.8%, and 82.3% to 45.5% across experience groups when AI predictions were incorrect.
## Methods
- Systematic review conducted using PRISMA 2020; the authors defined five research questions on causal/mediator factors, co-occurring biases, explainability, AI literacy/mental models, and individual differences.
- Searches were run in SCOPUS, ScienceDirect, PubMed, and Google Scholar in May 2025, using peer-reviewed English-language studies from January 2015 to April 2025 that were quantitative, experimental, and focused on automation bias or over-reliance.
- Screening proceeded from title/abstract to full text, with 35 studies included; the two reviewers reported 0.95 Cohen’s kappa agreement.
## Limitations
- The paper states that restricting inclusion to peer-reviewed journal articles from 2015–2025, English-language studies, and experimental designs may have excluded earlier work, non-English studies, preprints, qualitative studies, and mixed-methods research.
- The paper states its search strategy lacked sensitivity and missed some relevant studies due to indexing gaps and terminology variation.
- The paper notes limited evidence on cultural and socio-demographic factors because of the current lack of cross-cultural comparison studies.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
17 Valid · 3 Not valid · 203 Not checked · 48 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“To the best of our knowledge, existing reviews have focused on AB in relation to traditional computer-based aids rather than AI-driven systems (see Goddard etal. 2012; Lyell and Coiera 2017).”
What the sources say
- Not matchedcites Goddard etal. 2012
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Partly supportedcites Lyell and Coiera 2017 sourceExperimental — accuracy not yet measured
The Introduction and Method describe a review of automation bias involving clinical decision support and other automated aids, which supports the characterization of this review's focus. Our copy does not distinguish those aids from AI-driven systems or establish that existing reviews generally excluded AI-driven systems. (LLM, conf=0.82)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Not matched
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Cognitive overload is considered the most significant factor inhibiting users from adequately verifying automation accuracy, thereby increasing AB:”
What the sources say
- Partly supportedcites Lyell and Coiera 2017 sourceExperimental — accuracy not yet measured
The Discussion hypothesizes that cognitive overload can prevent users from adequately verifying automation and lead to automation-bias errors. It does not establish cognitive overload as the most significant factor; the Conclusion says further research is needed to test its impact. The supplied passages do not include page numbers. (LLM, conf=0.88)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Users often fail to critically evaluate automation outputs when they perceive the system to be highly reliable (Lyell and Coiera 2017).”
What the sources say
- Supportedcites Lyell and Coiera 2017 source
The Introduction (page unavailable) says high system accuracy may foster trust and that users with greater trust in automation are less likely to detect its failures. It also describes automation bias as replacing vigilant information seeking with reliance on automated cues, supporting the paraphrase. (LLM, conf=0.90)
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“the harder the verification complexity, that is the complexity of verifying that the automation is performing correctly, the greater the likelihood of undue trust in the system (Lyell and Coiera 2017).”
What the sources say
- Partly supportedcites Lyell and Coiera 2017 sourceExperimental — accuracy not yet measured
The Discussion (page unavailable) says high verification complexity appears to increase the risk of automation bias, and the Method defines verification complexity as the difficulty of checking whether automation performs correctly. The claim presents that tentative relationship as a definite one; the paper does not establish that likelihood rises consistently as complexity increases. (LLM, conf=0.83)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“In line with these results, Dikmen and Burns (2022) found that providing domain-specific knowledge helped non-expert users to critically evaluate AI outputs, relying less on incorrect AI predictions.”
What the sources say
- Supportedcites Dikmen and Burns 2022 source
Section 6.1 (page unavailable) reports that participants given domain-relevant guidance made fewer investments when the AI underpredicted loan-default risk. It says they may have noticed discrepancies and questioned the AI’s guidance, and concludes that conveying domain knowledge reduced over-reliance on incorrect AI predictions. The finding concerns the peer-to-peer lending task studied. (LLM, conf=0.91)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
132 more claims · 9 with their cited sources checked
Create a free account to see all 271 claims and every source verdict.
- It may state two separate claims as one. 96
- Probably not a substantial claim. 55
- Borderline: one of the tests for a substantial claim is unclear. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A comparable dynamic may plausibly extend to generative AI used as an information source, where fluent and confident outputs can be accepted uncritically (Romeo and Conti 2026), although this transfer across system classes should be treated with caution.”
From Epistemic Trust in Generative AI as an Information Source: Development and Validation of the Trust in AI-Epistemic Scale (TAI-E) (2026) · cites this paper as Romeo and Conti 2026
In the citing paper
“This configuration does not fit either of two simplified narratives that frequently appear in the surrounding literature: the assumption that analytic engagement necessarily reduces trust (Alon-Barkat and Busuioc 2023; Romeo and Conti 2026;Stanovich et al. 2016), or the assumption that trust primarily reflects uncritical acceptance (Davis 1989;Hoff and Bashir 2015).”
From Epistemic Trust in Generative AI as an Information Source: Development and Validation of the Trust in AI-Epistemic Scale (TAI-E) (2026) · cites this paper as Romeo and Conti 2026
Citations
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J., Bossuyt, P., Boutron, I., Hoffmann, T., mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S. E., et al. (2020) · doi:10.31222/osf.io/v7gm2_v1 · Cited by 8,090
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 2×
“In the automation context, trust is defined as «the attitude that an (automated) agent will help achieve an individual’s goals in a situation characterized by uncertainty and vulnerability» (Lee and See 2004: p.51).”
The PRISMA 2020 statement: an updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J., Bossuyt, P., Boutron, I., Hoffmann, T., mulrow, C., Shamseer, L., Tetzlaff, J., Akl, E., Brennan, S. E., et al. (2020) · doi:10.31222/osf.io/v7gm2 · Cited by 5,177
Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., Koffel, J. B., PRISMA-S Group, Blunt, H., Brigham, T., et al. (2021) · Systematic Reviews · doi:10.1186/s13643-020-01542-z · Cited by 3,714 · cited 2×
“In accordance with PRISMA-S guidelines (Rethlefsen et al. 2021), we also included records identified through personal files.”
Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., & Koffel, J. (2019) · doi:10.31219/osf.io/sfc38 · Cited by 3,696
What is AI Literacy? Competencies and Design Considerations
Long, D., & Magerko, B. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376727 · Cited by 3,241 · cited 1×
“Key challenges lie in AI literacy, whose lack can limit user’s ability to effectively collaborate with systems (Long and Magerko 2020), and explanation design itself.”
Complacency and Bias in Human Use of Automation: An Attentional Integration
Parasuraman, R., & Manzey, D. H. (2010) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720810376055 · Cited by 1,710 · cited 1×
“Some researchers emphasize differences in their cognitive and situational antecedents (Wickens et al. 2015), while others highlight substantial overlap, viewing both as manifestations of shared underlying attentional and decision-making mechanisms (Parasuraman and Manzey 2010).”
Automation bias: a systematic review of frequency, effect mediators, and mitigators
Goddard, K., Roudsari, A., & Wyatt, J. C. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000089 · Cited by 1,162 · cited 7×
“Additionally, accountability, which has been proposed as an intervention to mitigate AB (Goddard et al. 2012), lacks empirical support.”
InfluentialBuçinca, Z., Malaya, M. B., & Gajos, K. Z. (2021) · arXiv · doi:10.48550/arxiv.2102.09692 · Cited by 1,124 · cited 12×
“In an effort to reduce over-reliance, Buçinca et al. (2021) adopted CFFs, a metacognitive debiasing technique aimed at promoting analytical thinking.”
InfluentialZhang, Y., Liao, Q. V., & Bellamy, R. K. E. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372852 · Cited by 895 · cited 1×
“Merely making AI explainable or transparent is not enough: explanations must actually help users recognize uncertainty or errors (Zhang et al. 2020).”
Engineering Psychology and Human Performance
Wickens, C. D., Helton, W. S., Hollands, J. G., & Banbury, S. (2021) · Routledge eBooks · doi:10.4324/9781003177616 · Cited by 684 · cited 5×
“Like other decision heuristics and biases, AB reflects the tendency of humans to minimize cognitive effort, a tendency described by the “cognitive miser” hypothesis (Wickens et al. 2021).”
InfluentialDeveloping the Review Question and Inclusion Criteria
Stern, C., Jordan, Z., & McArthur, A. (2014) · AJN, American Journal of Nursing · doi:10.1097/01.naj.0000445689.67800.86 · Cited by 629 · cited 1×
“Applying the PICo (Population, phenomenon of Inter-est, Context) search strategy (Stern et al. 2014), the search strategy was constructed using Boolean operators to connect key terms and focus on the scope of the inquiry.”
Do as AI say: susceptibility in deployment of clinical decision-aids
Gaube, S., Suresh, H., Raue, M., Merritt, A., Berkowitz, S. J., Lermer, E., Coughlin, J. F., Guttag, J. V., Colak, E., & Ghassemi, M. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00385-9 · Cited by 543 · cited 9×
“Transparency mechanisms, such as uncertainty estimates, have been proposed to counteract overreliance by highlighting system limitations (Bond et al. 2018; Gaube et al. 2021; Dratsch et al. 2023).”
InfluentialAlon-Barkat, S., & Busuioc, M. (2023) · Journal of Public Administration Research and Theory · doi:10.1093/jopart/muac007 · Cited by 469
The future of human-AI collaboration: a taxonomy of design knowledge for hybrid intelligence systems
Dellermann, D., Calma, A., Lipusch, N., Weber, T., Weigel, S., & Ebel, P. (2021) · arXiv · doi:10.48550/arxiv.2105.03354 · Cited by 456
Automation bias and verification complexity: a systematic review
Lyell, D., & Coiera, E. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw105 · Cited by 451 · cited 7×
“Lyell and Coiera (2017) further emphasized that task complexity and verification demands contribute to cognitive overload, increasing susceptibility to AB.”
InfluentialTrust and reliance on AI — An experimental study on the extent and costs of overreliance on AI
Klingbeil, A., Grützner, C., & Schreck, P. (2024) · Computers in Human Behavior · doi:10.1016/j.chb.2024.108352 · Cited by 450 · cited 6×
“Yet, trust alone does not fully account for AB, as some samples tended to over-rely (e.g., Klingbeil et al. 2024), while others under-rely (e.g., Küper et al. 2025 ), underscoring the need to consider its interaction with other factors, explored in the sections that follow.”
InfluentialRethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., Koffel, J. B., & Group, P.-S. (2021) · Journal of the Medical Library Association · doi:10.5195/jmla.2021.962 · Cited by 409
Explanations Can Reduce Overreliance on AI Systems During Decision-Making
Vasconcelos, H., Jörke, M., Grunde-McLaughlin, M., Gerstenberg, T., Bernstein, M. S., & Krishna, R. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3579605 · Cited by 402 · cited 4×
“To support this, additional incentives for accurate decision-making may help compensate for low intrinsic motivation among users (Vasconcelos et al. 2023).”
InfluentialDratsch, T., Chen, X., Rezazade Mehrizi, M., Kloeckner, R., Mähringer-Kunz, A., Püsken, M., Baeßler, B., Sauer, S., Maintz, D., & Pinto dos Santos, D. (2023) · Radiology · doi:10.1148/radiol.222176 · Cited by 369 · cited 6×
“Meanwhile, Dratsch et al. (2023) further illustrated that confidence and experience reduced only errors of commission.”
InfluentialThe Future of Human-AI Collaboration: A Taxonomy of Design Knowledge for Hybrid Intelligence Systems
Dellermann, D., Calma, A., Lipusch, N., Weber, T., Weigel, S., & Ebel, P. (2019) · Proceedings of the Annual Hawaii International Conference on System Sciences · doi:10.24251/hicss.2019.034 · Cited by 279
Automation Bias in Intelligent Time Critical Decision Support Systems
Cummings, M. (2004) · AIAA 1st Intelligent Systems Technical Conference · doi:10.2514/6.2004-6313 · Cited by 269
Jacobs, M., Pradier, M. F., McCoy, T. H., Perlis, R. H., Doshi-Velez, F., & Gajos, K. Z. (2021) · Translational Psychiatry · doi:10.1038/s41398-021-01224-x · Cited by 247 · cited 5×
“Jacobs et al. (2021) identified no significant change in confidence across conditions involving correct or incorrect AI recommendations, suggesting that confidence may not protect against AB.”
InfluentialAdaptive trust calibration for human-AI collaboration
Okamura, K., & Yamada, S. (2020) · PLOS ONE · doi:10.1371/journal.pone.0229132 · Cited by 239
Do Recommender Systems Manipulate Consumer Preferences? A Study of Anchoring Effects
Adomavicius, G., Bockstedt, J. C., Curley, S. P., & Zhang, J. (2013) · Information Systems Research · doi:10.1287/isre.2013.0497 · Cited by 236 · cited 1×
“Lay users (e.g., consumers) have likewise shown a tendency to over-rely on algorithms even for low-stakes tasks, such as personal preference decisions (Banker and Khetani 2019), as evidenced by the extensive literature on recommender systems (see Adomavicius et al. 2013).”
Three Challenges for AI-Assisted Decision-Making
Steyvers, M., & Kumar, A. (2024) · Perspectives on Psychological Science · doi:10.1177/17456916231181102 · Cited by 217 · cited 5×
“These proposals seek to solve the three main challenges in hybrid decision-making (cf. 1.1.1; Steyvers and Kumar 2024).”
InfluentialDeciding Fast and Slow: The Role of Cognitive Biases in AI-assisted Decision-making
Rastogi, C., Zhang, Y., Wei, D., Varshney, K. R., Dhurandhar, A., & Tomsett, R. (2022) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3512930 · Cited by 216 · cited 1×
“Consistent with this, Rastogi et al. (2022) demonstrated that time is a useful resource for de-anchoring.”
Lai, V., Chen, C., Smith-Renner, A., Liao, Q. V., & Tan, C. (2023) · 2023 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3593013.3594087 · Cited by 204
Jones-Jang, S. M., & Park, Y. J. (2022) · Journal of Computer-Mediated Communication · doi:10.1093/jcmc/zmac029 · Cited by 178
Complacency and Automation Bias in the Use of Imperfect Automation
Wickens, C. D., Clegg, B. A., Vieane, A. Z., & Sebok, A. L. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720815581940 · Cited by 176 · cited 1×
“Some researchers emphasize differences in their cognitive and situational antecedents (Wickens et al. 2015), while others highlight substantial overlap, viewing both as manifestations of shared underlying attentional and decision-making mechanisms (Parasuraman and Manzey 2010).”
Naiseh, M., Al-Thani, D., Jiang, N., & Ali, R. (2023) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102941 · Cited by 170 · cited 11×
“Attempts to address this issue through multiple types of explanation have proven ineffective, as offering varied explanation formats did not significantly improve users’ ability to detect incorrect AI recommendations (Naiseh et al. 2023).”
InfluentialKeding, C., & Meissner, P. (2021) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2021.120970 · Cited by 168 · cited 2×
“Contrasting with the view of experts as more averse to algorithmic output, the study by Keding and Meissner (2021) examined AI-augmented decision-making among 150 senior executives making R&D investment decisions.”
Selten, F., Robeer, M., & Grimmelikhuijsen, S. (2023) · Public Administration Review · doi:10.1111/puar.13602 · Cited by 153 · cited 2×
“In like manner, the studies by Selten et al. (2023) and Bashkirova and Krpan (2024) found no evidence for AB, emphasizing confirmation bias as a more dominant cognitive mechanism in AI-assisted decision-making.”
Human Reliance on Machine Learning Models When Performance Feedback is Limited: Heuristics and Risks
Lu, Z., & Yin, M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445562 · Cited by 131 · cited 1×
“As a result, users may over-rely on models that reinforce their own biases or under-rely on models that might challenge or correct those biases (Lu and Yin 2021).”
How Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review
Bertrand, A., Belloum, R., Eagan, J. R., & Maxwell, W. (2022) · Proceedings of the 2022 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3514094.3534164 · Cited by 129 · cited 4×
“While their intuition is more refined than non-experts’ one, it also makes experts more susceptible to belief perseverance and algorithmic aversion (Bertrand et al. 2022).”
InfluentialBond, R. R., Novotny, T., Andrsova, I., Koc, L., Sisakova, M., Finlay, D., Guldenring, D., McLaughlin, J., Peace, A., McGilligan, V., et al. (2018) · Journal of Electrocardiology · doi:10.1016/j.jelectrocard.2018.08.007 · Cited by 127 · cited 7×
“Transparency mechanisms, such as uncertainty estimates, have been proposed to counteract overreliance by highlighting system limitations (Bond et al. 2018; Gaube et al. 2021; Dratsch et al. 2023).”
InfluentialBanker, S., & Khetani, S. (2019) · Journal of Public Policy & Marketing · doi:10.1177/0743915619858057 · Cited by 120 · cited 1×
“Lay users (e.g., consumers) have likewise shown a tendency to over-rely on algorithms even for low-stakes tasks, such as personal preference decisions (Banker and Khetani 2019), as evidenced by the extensive literature on recommender systems (see Adomavicius et al. 2013).”
Rams, hounds and white boxes: investigating human–AI collaboration protocols in medical diagnosis
Cabitza, F., Campagner, A., Ronzio, L., Cameli, M., Mandoli, G. E., Pastore, M. C., Sconfienza, L. M., Folgado, D., Barandas, M., & Gamboa, H. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2023.102506 · Cited by 117 · cited 9×
“Specifically, Cabitza et al. (2023) examined human–AI collaboration in medical diagnostics, comparing AI-first and human-first decision protocols.”
InfluentialDikmen, M., & Burns, C. (2022) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2022.102792 · Cited by 116 · cited 2×
“At the same time, several moderator factors have been identified that mitigate the likelihood of AB: professional experience (Rezazade Mehrizi et al. 2023; Kücking et al. 2024; Kim et al. 2025) and domain expertise (Nourani et al. 2022; Dikmen and Burns 2022) are the most protective.”
The effects of explanations on automation bias
Vered, M., Livni, T., Howe, P. D. L., Miller, T., & Sonenberg, L. (2023) · Artificial Intelligence · doi:10.1016/j.artint.2023.103952 · Cited by 114 · cited 9×
“…including perceived benefit (Kücking et al. 2024), usefulness (Jacobs et al. 2021; Cao et al. 2023), competence (Naiseh et al. 2023), transparency (Vered et al. 2023), absence of uncertainty (Gaube et al. 2021), and the apparent objectivity of AI recommendations (Keding & Meissner 2021).”
InfluentialHow Culture Shapes What People Want From AI
Ge, X., Xu, C., Misaki, D., Markus, H. R., & Tsai, J. L. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642660 · Cited by 86 · cited 3×
“Emerging evidence shows that cultural, gender, socioeconomic, and regional factors significantly influence how humans rely on AI, whereas many design assumptions continue to reflect dominant Western perspectives (Ge et al. 2024).”
Bao, Y., Gong, W., & Yang, K. (2023) · Systems · doi:10.3390/systems11090442 · Cited by 80 · cited 5×
“Human–AI collaboration models emphasize the importance of users’ mental models of AI in decision-making processes (Bao et al. 2023; Steyvers and Kumar 2024).”
InfluentialTowards a Science of Human-AI Decision Making: A Survey of Empirical Studies
Lai, V., Chen, C., Liao, Q. V., Smith-Renner, A., & Tan, C. (2021) · arXiv · doi:10.48550/arxiv.2112.11471 · Cited by 75
Horowitz, M. C., & Kahn, L. (2023) · International Studies Quarterly · doi:10.48550/arxiv.2306.16507 · Cited by 74
Horowitz, M. C., & Kahn, L. (2024) · International Studies Quarterly · doi:10.1093/isq/sqae020 · Cited by 68
The impact of AI errors in a human-in-the-loop process
Agudo, U., Liberal, K. G., Arrese, M., & Matute, H. (2024) · Cognitive Research: Principles and Implications · doi:10.1186/s41235-023-00529-3 · Cited by 66 · cited 4×
“This bias leads decision-makers to seek evidence that confirms the AI initial suggestion (Gaube et al. 2021; Nourani et al. 2022), especially when incorrect AI guidance is provided before they form their own judgment (Agudo et al. 2024).”
InfluentialCheck the box! How to deal with automation bias in AI-based personnel selection
Kupfer, C., Prassl, R., Fleiß, J., Malin, C., Thalmann, S., & Kubicek, B. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1118723 · Cited by 59 · cited 10×
“Lastly, Kupfer et al. (2023) found that when decision-makers were made aware of potential system errors, they exhibited greater verification intensity.”
InfluentialAutomation bias in public administration – an interdisciplinary perspective from law and psychology
Ruschemeier, H., & Hondrich, L. J. (2024) · Government Information Quarterly · doi:10.1016/j.giq.2024.101953 · Cited by 59
Automation trust increases under high-workload multitasking scenarios involving risk
Sato, T., Yamani, Y., Liechty, M., & Chancey, E. T. (2019) · Cognition Technology & Work · doi:10.1007/s10111-019-00580-5 · Cited by 57
Bashkirova, A., & Krpan, D. (2024) · Computers in Human Behavior: Artificial Humans · doi:10.1016/j.chbah.2024.100066 · Cited by 55 · cited 3×
“In like manner, the studies by Selten et al. (2023) and Bashkirova and Krpan (2024) found no evidence for AB, emphasizing confirmation bias as a more dominant cognitive mechanism in AI-assisted decision-making.”
Rezazade Mehrizi, M. H., Mol, F., Peter, M., Ranschaert, E., Dos Santos, D. P., Shahidi, R., Fatehi, M., & Dratsch, T. (2023) · Scientific Reports · doi:10.1038/s41598-023-36435-3 · Cited by 54 · cited 6×
“On this point, Rezazade Mehrizi et al. (2023) investigated how AI explainability tools (like heatmaps) and attitudinal priming (positive or negative information about AI) influence radiologists’ diagnostic decisions when interpreting mammograms.”
InfluentialAutomation bias in public administration - an interdisciplinary perspective from law and psychology
Ruschemeier, H., & Hondrich, L. (2024) · Government Information Quarterly · doi:10.2139/ssrn.4736646 · Cited by 52
Automation Bias in AI-Decision Support: Results from an Empirical Study
Kücking, F., Hübner, U., Przysucha, M., Hannemann, N., Kutza, J.-O., Moelleken, M., Erfurt-Berge, C., Dissemond, J., Babitsch, B., & Busch, D. (2024) · Studies in Health Technology and Informatics · doi:10.3233/shti240871 · Cited by 51 · cited 5×
“Kücking et al. (2024) analyzed wound assessment in 210 professionals (63.3% nurses, 36.7% physicians) using AI-enabled Clinical Decision Support Systems.”
InfluentialGlick, A., Clayton, M., Angelov, N., & Chang, J. T. (2022) · JAMIA Open · doi:10.1093/jamiaopen/ooac031 · Cited by 50 · cited 1×
“However, it is also possible that confidence is influenced by the clinician’s level of experience (Glick et al. 2022), which has been shown to be more predictive of diagnostic accuracy.”
How Time Pressure in Different Phases of Decision-Making Influences Human-AI Collaboration
Cao, S., Gomez, C., & Huang, C.-M. (2023) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3610068 · Cited by 48 · cited 6×
“Overreliance is further reinforced by high workload (Vered et al. 2023; Zhang et al. 2024; Alami et al. 2025), time pressure (Cao et al. 2023), and task complexity (Bond et al. 2018; Kupfer et al. 2023; Cecil et al. 2024).”
InfluentialNourani, M., Roy, C., Block, J. E., Honeycutt, D. R., Rahman, T., Ragan, E. D., & Gogate, V. (2022) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3531066 · Cited by 47 · cited 25×
“In contrast to all these findings, Alon-Barkat and Busuioc (2023) found selective adherence to AI suggestions, rather than AB, in employment decisions.”
InfluentialAutomation bias in AI-assisted detection of cerebral aneurysms on time-of-flight MR angiography
Kim, S. H., Schramm, S., Riedel, E. O., Schmitzer, L., Rosenkranz, E., Kertels, O., Bodden, J. H., Paprottka, K. J., Sepp, D., Renz, M., et al. (2025) · La radiologia medica · doi:10.1007/s11547-025-01964-6 · Cited by 45 · cited 2×
“At the same time, several moderator factors have been identified that mitigate the likelihood of AB: professional experience (Rezazade Mehrizi et al. 2023; Kücking et al. 2024; Kim et al. 2025) and domain expertise (Nourani et al. 2022; Dikmen and Burns 2022) are the most protective.”
Understanding the Evolvement of Trust Over Time within Human-AI Teams
Duan, W., Zhou, S., Scalia, M. J., Yin, X., Weng, N., Zhang, R., Freeman, G., McNeese, N., Gorman, J., & Tolston, M. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3687060 · Cited by 45
Cecil, J., Lermer, E., Hudecek, M. F. C., Sauer, J., & Gaube, S. (2024) · Scientific Reports · doi:10.1038/s41598-024-60220-5 · Cited by 37 · cited 7×
“Both cognitive load and task complexity have been confirmed to influence susceptibility to AB (Cecil et al. 2024; Zhang et al. 2024).”
InfluentialNever tell me the odds: Investigating pro-hoc explanations in medical decision making
Cabitza, F., Natali, C., Famiglini, L., Campagner, A., Caccavella, V., & Gallazzi, E. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102819 · Cited by 35 · cited 4×
“Building on this direction, Cabitza et al. (2024) introduce the concept of Frictional AI , which involves the use of “cog-nitively non-invasive” tools that encourage oversight, caution, and responsible commitment in decision-making.”
InfluentialKüper, A., Lodde, G. C., Livingstone, E., Schadendorf, D., & Krämer, N. (2025) · Journal of Medical Internet Research · doi:10.2196/58660 · Cited by 31 · cited 9×
“…biases, which can distort perceptions of reliability (Naiseh et al. 2023), Across the studies, trust consistently emerged as a central determinant of overreliance: Küper et al. (2025) reported that trust accounted for up to 24.1% of the variance of reliant behavior, whether appropriate or not.”
Influential(2024) · doi:10.2196/preprints.58660 · Cited by 31
Do Recommender Systems Manipulate Consumer Preferences? A Study of Anchoring EffectsPreprint
Adomavičius, G., Bockstedt, J. C., Curley, S. P., et al. (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2285042 · Cited by 25
Measuring the effect of mental workload and explanations on appropriate AI reliance using EEG
Zhang, Z. T., Argın, S. K., Bilen, M. B., Urgun, D., Deniz, S. M., Liu, Y., & Hassib, M. E. (2024) · Behaviour and Information Technology · doi:10.1080/0144929x.2024.2431055 · Cited by 24 · cited 5×
“Both cognitive load and task complexity have been confirmed to influence susceptibility to AB (Cecil et al. 2024; Zhang et al. 2024).”
InfluentialMitigating Gender Stereotypes Toward AI Agents Through an eXplainable AI (XAI) Approach
Duan, W., McNeese, N. J., Freeman, G., & Li, L. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686969 · Cited by 23 · cited 6×
“Expanding on this distinction between dispositional, situational, and learned trust, Duan et al. (2024) outlined a developmental trajectory of trust in AI.”
InfluentialCalibrating Reliance on Automated Advice: Transparency and Trust Calibration Feedback
Tatasciore, M., & Loft, S. (2025) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2025.2487861 · Cited by 20 · cited 2×
“Multiple explanation types (Naiseh et al. 2023) and trust calibration feedback (Tatasciore and Loft 2025) are proven not to help users recognize incorrect AI recommendations as well.”
Gegoff, I., Tatasciore, M., Bowden, V. K., & Loft, S. D. (2025) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208251318465 · Cited by 14 · cited 3×
“Multiple explanation types (Naiseh et al. 2023) and trust calibration feedback (Tatasciore and Loft 2025) are proven not to help users recognize incorrect AI recommendations as well.”
Alami, J., El Iskandarani, M., & Riggs, S. L. (2025) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208251323478 · Cited by 9 · cited 4×
“Occurrences of overreliance may also be understood as strategies to reduce cognitive load, rather than as mere manifestations of misplaced trust (Alami et al. 2025).”
InfluentialMatzen, L. E., Gastelum, Z., Howell, B. C., Divis, K., & Stites, M. C. (2024) · Cognitive Research Principles and Implications · doi:10.1186/s41235-024-00586-2 · Cited by 8 · cited 1×
“In the same vein, Matzen et al. (2024) found that users had more difficulty detecting errors where AI missed a target rather than false alarms.”
Wickens, C. D., Helton, W. S., Hollands, J. G., & Banbury, S. P. (2021) · Routledge eBooks · doi:10.4324/9781003177616-13 · Cited by 8
Policy-makers’ Processing of AI Algorithmic Advice: ‘Automation Bias’ versus Selective Adherence
Alon Barkat, S., & Busuioc, M. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3794660 · Cited by 6
How Different Explanations Impact Trust Calibration: The Case of Clinical Decision Support SystemsPreprint
Naiseh, M., Al‐Thani, D., Jiang, N., & Ali, R. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4098528 · Cited by 2
Alon‐Barkat, S., & Busuioc, M. (2022) · Harvard Dataverse · doi:10.7910/dvn/tqyjnf
Automation Bias in AI-Assisted Detection of Cerebral Aneurysms on Time-of-Flight MR-Angiography
Kim, S. H., Schramm, S., Riedel, E. O., et al. (2024) · doi:10.1101/2024.05.31.24308021
Research works printed but not linked: 2
- Hemmer P, Schemmer M, Vössing M, Kühl N (2021) Human–AI complementarity in hybrid intelligence systems: a structured literature review. In: Pacific Asia conference on information systems, vol 78. ISBN 978-1-7336325-7-7
- Romeo G, Conti D (2024) Beyond automation: reshaping Human-artificial intelligence interaction. Sistemi Intelligenti XXVI(3):641–648. https://doi.org/10.1422/115336 · doi:10.1422/115336
Cited by
Toward a science of human–AI teaming for decision making: a complementarity framework
Gonzalez, C., Donahue, K., Goldstein, D. G., Heidari, H., Jalali, M. S., Schelble, B., Singh, A., & Woolley, A. W. (2026) · PNAS Nexus · doi:10.1093/pnasnexus/pgag030 · Cited by 33
“Teams should establish escalation and disagreement protocols (eg confidence thresholds for deferring to humans; concurrence requirements for critical actions) and treat errors as learning opportunities through after-action reviews that incorporate both human and AI system outputs (109).”
Cognitive Risks of AI: Literacy, Trust, and Critical Thinking
Kulal, A. (2025) · Journal of Computer Information Systems · doi:10.1080/08874417.2025.2582050 · Cited by 46
Watts, K. J. (2025) · Education Sciences · doi:10.3390/educsci15101304 · Cited by 18
Cognitive Atrophy Paradox of AI–Human Interaction: From Cognitive Growth and Atrophy to Balance
Kabashkin, I. (2025) · Information · doi:10.3390/info16111009 · Cited by 39
“Human–AI collaboration studies extend these insights, showing that overconfidence in AI may increase even among moderately experienced users, especially when transparency is low, stakes are ambiguous, or monitoring demands are reduced [22].”
Examining human reliance on artificial intelligence in decision making
Pearson, J., Dror, I. E., Jayes, E., Whordley, G.-R., Mason, G., & Nightingale, S. J. (2026) · Scientific Reports · doi:10.1038/s41598-026-34983-y · Cited by 32
“much work has focused on the aviation sector, concerns about automation bias extend to other domains such as healthcare and military, especially with the relatively recent move to embed AI systems within these areas to assist human decision making 12,13 .”
Metadata
- Authors
- Romeo, G., & Conti, D.
- Affiliations
- University of Catania (IT)
- Corresponding author
- Romeo, Giuseppe
- Conti, Daniela
- Venue
- AI & SOCIETY
- Published
- 2025-07-03
- Volume
- 41
- Issue
- 1
- Pages
- 259-278
- ISSN
- 0951-5666 (ISSN-L)
- 1435-5655
- OpenAlex
- W4412013201
- Cited by
- 209
- References
- 60
- Citations by year
- 2026: 194 · 2025: 15
- FWCI
- 228.31
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- AI & Society (publishedVersion, cc-by)
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- automation bias, human-AI collaboration, explainable AI, trust in AI, AI literacy, explanation complexity, professional expertise, explanation design, user engagement, cognitive mechanisms
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.