CoChat
Sign Up
Cited by 11Human-Automation Interaction and SafetyEthics and Social Impacts of AISafety Systems Engineering in Autonomy

The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.

AI Safety and Automation Bias

Open access
Kahn, Lauren, Probasco, Emelia, Kinoshita, Ronnie·2024
DOI 10.51593/20230057
Open this paper in CoChat — free accountOpen PDF

Abstract

Automation bias is a critical issue for artificial intelligence deployment. It can cause otherwise knowledgeable users to make crucial and even obvious errors. Organizational, technical, and educational leaders can mitigate these biases through training, design, and processes. This paper explores automation bias and ways to mitigate it through three case studies: Tesla’s autopilot incidents, aviation incidents at Boeing and Airbus, and Army and Navy air defense incidents.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.51593/20230057

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Research works printed but not linked: 27

  1. Kate Goddard, Abdul Roudsari, Jeremy C Wyatt, "Automation bias: a systematic review of frequency, effect mediators, and mitigators," Journal of the American Medical Informatics Association, Volume 19, Issue 1 (2012): 121-27, https://doi.org/10.1136/amiajnl-2011-000089. · DOI · Paper page
  2. Mary Cummings, "Automation Bias in Intelligence Time Critical Decision Support Systems," AIAA 1st Intelligent Systems Technical Conference (AIAA), Chicago, IL (2012), https://doi.org/10.2514/6.2004- 6313. · DOI · Paper page
  3. Grace Augustine, Jan Lodge, Mislav Radic, "Mr. Bates vs The Post Office depicts one of the UK's worst miscarriages of justice: here's why so many victims didn't speak out," The Conversation, January 4, 2024, https://theconversation.com/mr-bates-vs-the-post-office-depicts-one-of-the-uks-worst-miscarriages- of-justice-heres-why-so-many-victims-didnt-speak-out-220513. errors," Theoretical Issues in Ergonomics Science, Volume 15, Issue 6 (2013): 545-577, https://doi.org/10.1080/1463922X.2013.817625, and Kate Goddard, Abdul Roudsari, Jeremy C. Wyatt, "Automation bias: a systematic review of frequency, effect mediators, and mitigators," Journal of the American Medical Informatics Association, Volume 19, Issue 1 (2012): 121-127, https://doi.org/10.1136/amiajnl-2011-000089. · DOI · Paper page
  4. Kate Goddard, Abdul Roudsari, Jeremy C. Wyatt, "Automation bias: Empirical results assessing influencing factors," International Journal of Medical Informatics, Volume 83, Issue 5 (2014): 368-375, https://doi.org/10.1016/j.ijmedinf.2014.01.001. · DOI · Paper page
  5. David M. Sanbonmatsu, David L. Strayer, Zhenghui Yu, Francesco Biondi, Joel M. Cooper, "Cognitive underpinnings of beliefs and confidence in beliefs about fully automated vehicles," Transportation Research Part F: Traffic Psychology and Behaviour, Volume 55 (2018): 114-122, https://psycnet.apa.org/doi/10.1016/j.trf.2018.02.029; · DOI
  6. Michael C. Horowitz, Lauren Kahn, Julia Macdonald, Jacquelyn Schneider, "Adopting AI: how familiarity breeds both trust and contempt," AI & Society, Volume 39 (2023): 1721-1735, https://doi.org/10.1007/s00146-023-01666-5; · DOI · Paper page
  7. Robert E. Burnkrant, Alain Cousineau, "Informational and Normative Social Influence in Buyer Behavior," Journal of Consumer Research, Volume 2, Issue 3 (1975): 206-215, https://psycnet.apa.org/doi/10.1086/208633. · DOI
  8. Michael C. Horowitz, Lauren Kahn, "Bending the Automation Bias Curve: A Study of Human and AI- Based Decision Making in National Security Contexts," International Studies Quarterly, Volume 68, Issue 2 (2024), https://doi.org/10.1093/isq/sqae020; · DOI · Paper page
  9. Justin Kruger, David Dunning, "Unskilled and unaware of it: How difficulties in recognizing one's own incompetence lead to inflated self-assessments," Journal of Personality and Social Psychology, Volume 77, no. 6 (1999): 1121-1134, https://psycnet.apa.org/doi/10.1037/0022-3514.77.6.1121; · DOI · Paper page
  10. Moritz Körber, Eva Baseler, Klaus Bengler, "Introduction matters: Manipulating trust in automation and reliance in automated driving," Applied Ergonomics, Volume 66 (2018): 18-31. https://doi.org/10.1016/j.apergo.2017.07.006. · DOI · Paper page
  11. Chris Schwarz, John Gaspar, Timothy Brown, "The effect of reliability on drivers' trust and behavior in conditional automation," Cognition, Technology & Work, Volume 21 (2019): 41-54. https://link.springer.com/article/10.1007/s10111-018-0522-y. · DOI
  12. Apoorva P. Hungund, Ganesh Pai, Anuj K. Pradhan, "Systematic Review of Research on Driver Distraction in the Context of Advanced Driver Assistance Systems," Transportation Research Record, Volume 2675, Issue 9 (2021): 756-765, https://doi.org/10.1177/03611981211004129; · DOI
  13. Moritz Körber, Eva Baseler, Klaus Bengler, "Introduction matters: Manipulating trust in automation and reliance in automated driving," Applied Ergonomics, Volume 66 (2018): 18-31. https://doi.org/10.1016/j.apergo.2017.07.006. · DOI · Paper page
  14. Moritz Körber, Eva Baseler, Klaus Bengler "Introduction matters: Manipulating trust in automation and reliance in automated driving," Applied Ergonomics, Volume 66 (2018): 18-31. https://doi.org/10.1016/j.apergo.2017.07.006. · DOI · Paper page
  15. Qiyuan Zhang, Christopher D. Wallbridge, Dylan M. Jones, Phillip L. Morgan, "Public perception of autonomous vehicle capability determines judgment of blame and trust in road traffic accidents," Transportation Research Part A: Policy and Practice, Volume 179 (2024), https://doi.org/10.1016/j.tra.2023.103887. · DOI
  16. Kathleen L. Mosier, Linda J. Skitka, Susan Heers, Mark Burdick, "Automation Bias: Decision Making and Performance in High-Tech Cockpits," The International Journal of Aviation Psychology, Volume 8, Issue 1 (1998): 47-63, https://doi.org/10.1207/s15327108ijap0801_3. · DOI · Paper page
  17. John Dowell, John Long, "Towards a conception for an engineering discipline of human factors," Ergonomics, Volume 32, no. 11 (1989): 1513-1535, https://doi.org/10.1080/00140138908966921. · DOI
  18. Alexander Z. Ibsen, "The politics of airplane production: The emergence of two technological frames in the competition between Boeing and Airbus," Technology in Society, Volume 31, Issue 4 (2009): 342-349, https://doi.org/10.1016/j.techsoc.2009.10.006. · DOI
  19. Kathy Abbott, "Human Factors Engineering and Flight Deck Design," in The Avionics Handbook, edited by Cary Spitzer, CRC Press LLC, 2001.
  20. Paulus A.J.M. de Wit, Roberto Moraes Cruz, "Learning from AF447: Human-machine interaction," Safety Science, Volume 112, (2019): 48-56, https://doi.org/10.1016/j.ssci.2018.10.009. · DOI
  21. N.A. "Final Report on the accident on 1st June 2009 to the Airbus A330-203 registered F-GZCP operated by Air France flight AF 447 Rio de Janeiro -Paris," (Le Bourget Cedex, France: Bureau d'Enquêtes et d'Analyses pour la sécurité de l'aviation civile, 2012). Critical Socio-technical System," IFIP Advances in Information and Communication Technology, (Berling: Springer, 2013): 183-197, https://doi.org/10.1007/978-3-642-41145-8_16. · DOI
  22. K. H. Roberts, "New challenges in organizational research: high reliability organizations," Industrial Crisis Quarterly, 1989.
  23. James Reason, "Human error: models and management," The BMJ (Clinical research ed.), Volume 320 (2000): 768-770, https://doi.org/10.1136/bmj.320.7237.768. · DOI · Paper page
  24. Sharif H. Calfee, "Autonomous Agent-Based Simulation of an AEGIS Cruiser Combat Information Center Performing Battle Group Air-Defense Commander Operations," Naval Postgraduate School, March, 2003, https://faculty.nps.edu/ncrowe/oldstudents/calfee-thesis.htm;
  25. Mary Cummings, "Automation Bias in Intelligence Time Critical Decision Support Systems," AIAA 1st Intelligent Systems Technical Conference (AIAA), Chicago, IL (2012), https://doi.org/10.2514/6.2004- 6313. · DOI · Paper page
  26. Anthony Tingle, "Human-Machine Team Failed Vincennes," Proceedings, Volume 144, no. 7 (2018), https://www.usni.org/magazines/proceedings/2018/july/human-machine-team-failed-vincennes.
  27. Study of the USS Vincennes incident under the Navy's Tactical Decision Making Under Stress program point to other sources of error besides automation bias, such as the design of the AEGIS user interface and its relationship to other human cognitive biases like framing errors and confirmation bias. For more see, Jeffrey G. Morrison, Richard T. Kelly, Ronald A. Moore, Susan G. Hutchins, "Tactical decision making under stress (TADMUS) decision support system," Calhoun: The NPS Institutional Archive (1996), https://hdl.handle.net/10945/41225.

Cited by

Sorted by citations

See all 11

Metadata

Authors
Kahn, L., Probasco, E., & Kinoshita, R.
Affiliations
Center for Security and Emerging Technology (US)
ORCID
Publisher
Center for Security and Emerging Technology
Published
2024-11
DOI
10.51593/20230057
OpenAlex
W7160376673
Cited by
11
References
1
Citations by year
2026: 10 · 2025: 1
Type
report
Language
en
Open access
Licence
cc-by-nc
Topics
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
Keywords
automation bias, AI safety, human-AI interaction, human oversight, human error, autonomous driving, Tesla Autopilot, aviation accidents, air defense, military automation, bias mitigation, human training

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review