The DOI matches its Crossref record; 55 of the 56 references it cites were found in a registry or an index.
Automation bias and verification complexity: a systematic review
Open accessAbstract
Introduction: While potentially reducing decision errors, decision support systems can introduce new types of errors. Automation bias (AB) happens when users become overreliant on decision support, which reduces vigilance in information seeking and processing. Most research originates from the human factors literature, where the prevailing view is that AB occurs only in multitasking environments. Objectives: This review seeks to compare the human factors and health care literature, focusing on the apparent association of AB with multitasking and task complexity. Data sources: EMBASE, Medline, Compendex, Inspec, IEEE Xplore, Scopus, Web of Science, PsycINFO, and Business Source Premiere from 1983 to 2015. Study selection: Evaluation studies where task execution was assisted by automation and resulted in errors were included. Participants needed to be able to verify automation correctness and perform the task manually. Methods: Tasks were identified and grouped. Task and automation type and presence of multitasking were noted. Each task was rated for its verification complexity. Results: Of 890 papers identified, 40 met the inclusion criteria; 6 were in health care. Contrary to the prevailing human factors view, AB was found in single tasks, typically involving diagnosis rather than monitoring, and with high verification complexity. Limitations: The literature is fragmented, with large discrepancies in how AB is reported. Few studies reported the statistical significance of AB compared to a control condition. Conclusion: AB appears to be associated with the degree of cognitive load experienced in decision tasks, and appears to not be uniquely associated with multitasking. Strategies to minimize AB might focus on cognitive load reduction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“To the best of our knowledge, existing reviews have focused on AB in relation to traditional computer-based aids rather than AI-driven systems (see Goddard etal. 2012; Lyell and Coiera 2017).”
From Exploring automation bias in human–AI collaboration: a review and implications for explainable AI (2026) · cites this paper as Lyell and Coiera 2017
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Therefore, cognitive overload and trust remain the most crucial factors since, paradoxically, higher automation accuracy can further reinforce user trust, leading to an increased likelihood of AB.”
From Exploring automation bias in human–AI collaboration: a review and implications for explainable AI (2026) · cites this paper as Lyell and Coiera 2017
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the harder the verification complexity, that is the complexity of verifying that the automation is performing correctly, the greater the likelihood of undue trust in the system (Lyell and Coiera 2017).”
From Exploring automation bias in human–AI collaboration: a review and implications for explainable AI (2026) · cites this paper as Lyell and Coiera 2017
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Cognitive overload is considered the most significant factor inhibiting users from adequately verifying automation accuracy, thereby increasing AB:”
From Exploring automation bias in human–AI collaboration: a review and implications for explainable AI (2026) · cites this paper as Lyell and Coiera 2017
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This finding aligns with Lyell and Coiera's (2017) consideration that accountability alone is an ineffective intervention.”
From Exploring automation bias in human–AI collaboration: a review and implications for explainable AI (2026) · cites this paper as Lyell and Coiera's (2017)
Experimental — accuracy not yet measured
11 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Dermyshi, E., Wang, Y., Yan, C., Hong, W., Qiu, G., Gong, X., & Zhang, T. (2017) · Neonatology · doi:10.1159/000454668 · Cited by 24,451
Sweller, J., Ayres, P., & Kalyuga, S. (2011) · doi:10.1007/978-1-4419-8126-4 · Cited by 5,531
Sweller, J. (2012) · Encyclopedia of the Sciences of Learning · doi:10.1007/978-1-4419-1428-6_446 · Cited by 4,405
Garg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., Sam, J., & Haynes, R. B. (2005) · JAMA · doi:10.1001/jama.293.10.1223 · Cited by 2,936
Complacency and Bias in Human Use of Automation: An Attentional Integration
Parasuraman, R., & Manzey, D. H. (2010) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720810376055 · Cited by 1,710
Automation bias: a systematic review of frequency, effect mediators, and mitigators
Goddard, K., Roudsari, A., & Wyatt, J. C. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000089 · Cited by 1,162
Research works printed but not linked: 7
- Mosier, 1996, Human decision makers and automated decision aids: made for each other, 201
- Parasuraman, Performance consequences of automation-induced “complacency.” Int J Aviation Psychol, 1993, 3, 1
- Prinzel, 2005, Individual differences in complacency and monitoring for automation failures, Individual Differences Res, 3, 27
- Singh, 2009, Effects of automation reliability and training on automation-induced complacency and perceived mental workload, J Indian Acad Appl Psychol, 35, 9
- Bagheri, 2004, Considering Subjective Trust and Monitoring Behavior in Assessing Automation-induced “Complacency.”
- Singh, 2010, Performance feedback, mental workload and monitoring efficiency, J Indian Acad Appl Psychol, 36, 151
- Singh, 2001, Effects of manual training and automation reliability on automation induced complacency in flight simulation task, Psychol Stud, 46, 21
Cited by
Rajkomar, A., Dean, J., & Kohane, I. (2019) · New England Journal of Medicine · doi:10.1056/nejmra1814259 · Cited by 4,579
The false hope of current approaches to explainable artificial intelligence in health care
Ghassemi, M., Oakden-Rayner, L., & Beam, A. L. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00208-9 · Cited by 1,724
Ensuring fairness in machine learning to advance health equity.
Rajkomar, A., Hardt, M., Howell, M. D., Corrado, G., & Chin, M. H. (2018) · Annals of Internal Medicine · doi:10.7326/m18-1990 · Cited by 1,341
“African American patients might be harmed if clinical teams started relying on alerts to identify at-risk patients without realizing that the prediction system underdetects patients in that group (automation bias) (16).”
BackgroundEvaluation and mitigation of the limitations of large language models in clinical decision-making
Hager, P., Jungmann, F., Holland, R., Bhagat, K., Hubrecht, I., Knauer, M., Vielhauer, J., Makowski, M., Braren, R., Kaissis, G., et al. (2024) · Nature Medicine · doi:10.1038/s41591-024-03097-1 · Cited by 798
“Importantly, studies as to the impact of automation bias 53,54 and human– AI interaction biases could also be explored in such a context 55–57 .”
Accountable Artificial Intelligence: Holding Algorithms to Account
Busuioc, M. (2021) · Public Administration Review · doi:10.1111/puar.13293 · Cited by 652
“…algorithmic outputs could conceivably lead to unjustified deference of the decision-maker to algorithm predictions: “Automation bias” (Cummings 2006; Lyell and Coiera 2017; Skitka, Mosier, and Burdick 1999) is a well-documented phenomenon reported in psychological studies manifest in human…”
Background
Metadata
- Authors
- Lyell, D., & Coiera, E.
- Affiliations
- Macquarie University (AU)
- Corresponding author
- Lyell, David
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2016-08-11
- Volume
- 24
- Issue
- 2
- Pages
- 423-431
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC7651899
- OpenAlex
- W2512902140
- Cited by
- 451
- References
- 56
- Citations by year
- 2026: 193 · 2025: 60 · 2024: 55 · 2023: 41 · 2022: 29 · 2021: 27 · 2020: 21 · 2019: 13 · 2018: 7 · 2017: 5
- FWCI
- 1.34
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Funders
- HCF Research Foundation
- Topics
- Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- automation bias, decision support system, verification complexity, multitasking, cognitive load, task complexity, over-reliance on automation, sustained attention, diagnostic errors, healthcare decision-making, human factors
- MeSH terms
- Attitude to Computers, Automation, Humans, Bias, Decision Support Systems, Clinical
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Automation bias and verification complexity: a systematic review. https://app.cochat.ai/papers/10.1093/jamia/ocw105
- The paper
- Lyell, D., & Coiera, E. (2017). Automation bias and verification complexity: a systematic review. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocw105
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.