Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 73 of the 73 references it cites were found in a registry or an index; 17 of the 264 sources its claims cite were checked for support.

Exploring automation bias in human–AI collaboration: a review and implications for explainable AI

Open access
Romeo, Giuseppe, Conti, Daniela · 2026 · AI & SOCIETY

Abstract

Systematic review of automation bias in high-stakes human-AI collaboration, examining cognitive mechanisms, expertise, trust dynamics, verification demands, and the limits of explainable AI for mitigating overreliance.

Overview

CoChat Summary

## TL;DR

This paper is a PRISMA-based review of automation bias in human-AI collaboration, synthesizing 35 peer-reviewed experimental studies from 2015–April 2025. It concludes that trust, task complexity, AI literacy, expertise, and individual differences shape overreliance, while explainability/transparency alone often do not prevent automation bias and can sometimes worsen it.

## Key findings

  • The review included 35 studies, published across 21 journals, with a combined 19,774 participants; all included studies used quantitative, experimental methods.
  • Trust and task complexity emerged as the main drivers of automation bias; the paper reports that trust accounted for up to 24.1% of the variance in reliant behavior in one study (Küper et al. 2025).
  • Explainability was mixed or limited: CFFs reduced reliance on incorrect AI recommendations compared with simple XAI, but were seen as more cognitively demanding; multiple explanation types did not improve detection of incorrect AI recommendations, and trust calibration feedback did not help.
  • The review reports several quantitative examples of bias effects, including Bond et al. (2018) where incorrect automated diagnoses reduced ECG interpretation accuracy from 84.86% to 41.66% for fellows and from 86.38% to 27.43% for non-fellows, and Dratsch et al. (2023) where mammography accuracy dropped from 79.7% to 19.8%, 81.3% to 24.8%, and 82.3% to 45.5% across experience groups when AI predictions were incorrect.

## Methods

  • Systematic review conducted using PRISMA 2020; the authors defined five research questions on causal/mediator factors, co-occurring biases, explainability, AI literacy/mental models, and individual differences.
  • Searches were run in SCOPUS, ScienceDirect, PubMed, and Google Scholar in May 2025, using peer-reviewed English-language studies from January 2015 to April 2025 that were quantitative, experimental, and focused on automation bias or over-reliance.
  • Screening proceeded from title/abstract to full text, with 35 studies included; the two reviewers reported 0.95 Cohen’s kappa agreement.

## Limitations

  • The paper states that restricting inclusion to peer-reviewed journal articles from 2015–2025, English-language studies, and experimental designs may have excluded earlier work, non-English studies, preprints, qualitative studies, and mixed-methods research.
  • The paper states its search strategy lacked sensitivity and missed some relevant studies due to indexing gaps and terminology variation.
  • The paper notes limited evidence on cultural and socio-demographic factors because of the current lack of cross-cultural comparison studies.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s00146-025-02422-7

Cites

73 of 73 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

17 Valid · 3 Not valid · 203 Not checked · 48 Own results and interpretations

132 more claims · 9 with their cited sources checked

Create a free account to see all 271 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 96
  • Probably not a substantial claim. 55
  • Borderline: one of the tests for a substantial claim is unclear. 2

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 60 references

Research works printed but not linked: 2

  1. Hemmer P, Schemmer M, Vössing M, Kühl N (2021) Human–AI complementarity in hybrid intelligence systems: a structured literature review. In: Pacific Asia conference on information systems, vol 78. ISBN 978-1-7336325-7-7
  2. Romeo G, Conti D (2024) Beyond automation: reshaping Human-artificial intelligence interaction. Sistemi Intelligenti XXVI(3):641–648. https://doi.org/10.1422/115336 · doi:10.1422/115336

Cited by

See all 209

Metadata

Authors
Romeo, G., & Conti, D.
Affiliations
University of Catania (IT)
Corresponding author
  • Romeo, Giuseppe
  • Conti, Daniela
Venue
AI & SOCIETY
Published
2025-07-03
Volume
41
Issue
1
Pages
259-278
ISSN
  • 0951-5666 (ISSN-L)
  • 1435-5655
OpenAlex
W4412013201
Cited by
209
References
60
Citations by year
2026: 194 · 2025: 15
FWCI
228.31
Type
journal article
Language
en
Open access
Licence
Locations
AI & Society (publishedVersion, cc-by)
Topics
  • Social Sciences › Human Factors and Ergonomics › Human-Automation Interaction and Safety
  • Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
Keywords
automation bias, human-AI collaboration, explainable AI, trust in AI, AI literacy, explanation complexity, professional expertise, explanation design, user engagement, cognitive mechanisms

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-01, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Exploring automation bias in human–AI collaboration: a review and implications for explainable AI. https://app.cochat.ai/papers/10.1007/s00146-025-02422-7
The paper
Romeo, G., & Conti, D. (2026). Exploring automation bias in human–AI collaboration: a review and implications for explainable AI. AI & SOCIETY. https://doi.org/10.1007/s00146-025-02422-7

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.