Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index; 10 of the 28 sources its claims cite were checked for support.

Managing a 'responsibility vacuum' in AI monitoring and governance in healthcare: a qualitative study

Open access
Owens, Kellie, Griffen, Zachary, Damaraju, Lasya · 2025 · BMC Health Services Research

Abstract

BACKGROUND: Despite the increasing implementation of artificial intelligence (AI) and machine learning (ML) technologies in healthcare, their long-term safety, effectiveness, and equity remain compromised by a lack of sustained oversight. This study explores the phenomenon of a "responsibility vacuum" in AI governance, wherein maintenance and monitoring tasks are poorly defined, inconsistently performed, and undervalued across healthcare systems. METHODS: We conducted semi-structured interviews with 21 experts involved in AI implementation in healthcare, including clinicians, clinical informaticists, computer scientists, and legal/policy professionals. Participants were recruited through purposive and snowball sampling. Interviews were transcribed, coded, and analyzed using abductive qualitative methods to identify themes related to maintenance practices, institutional incentives, and responsibility attribution. RESULTS: Participants widely recognized that AI models degrade over time due to factors such as data drift, changes in clinical practice, and poor generalizability. However, monitoring practices remain ad hoc and fragmented, with few institutions investing in structured oversight infrastructure. This "responsibility vacuum" is perpetuated by institutional incentives favoring rapid innovation and strategic ignorance of AI failures. Despite these challenges, some participants described grassroots efforts to monitor and maintain AI systems, drawing inspiration from fields such as radiology, laboratory medicine, and transportation safety. CONCLUSIONS: Our findings suggest that institutional and cultural forces in healthcare deprioritize the maintenance of AI tools, creating a governance gap that may lead to patient harm and inequitable outcomes. Addressing this responsibility vacuum will require formalized accountability structures, interdisciplinary collaboration, and policy reforms that center long-term safety and equity. Without such changes, AI/ML technologies designed to improve patient health may introduce new forms of harm, ultimately eroding trust in AI and machine learning for healthcare.

Overview

CoChat Summary

TL;DR

This qualitative study argues that healthcare AI/ML suffers from a “responsibility vacuum”: models are known to degrade, but monitoring and maintenance tasks are unclaimed, undervalued, and inconsistently performed. The authors find that institutional incentives favor rapid innovation and strategic ignorance, while grassroots actors creatively improvise maintenance practices drawn from other safety-critical fields.

Key findings

  • Participants widely recognized that AI/ML models degrade over time due to data drift, changes in clinical practice, poor generalizability, and related factors, but monitoring remained ad hoc and fragmented, with few institutions investing in structured oversight infrastructure.
  • The “responsibility vacuum” is perpetuated by institutional incentives favoring rapid innovation and strategic ignorance of AI failures; participants described a culture of “move fast and break things” and fear of being “left behind.”
  • Responsibility for monitoring was widely diffused and deflected: machine learning researchers, hospital IT teams, and clinicians were each described as poorly suited or lacking capacity to take ownership, and no existing actor was seen as a natural fit.
  • Despite the lack of formal structures, some participants described grassroots efforts to monitor and maintain AI systems, drawing analogies from radiology, laboratory medicine, and transportation safety, including aviation, car service manuals, and autonomous vehicles.

Methods

  • Semi-structured interviews with 21 experts involved in AI implementation in healthcare, including clinical informaticists, clinicians, computer scientists, and legal/policy professionals, recruited through purposive and snowball sampling.
  • Interviews lasted 30 min to 1 h, were professionally transcribed, coded by 2 researchers using ATLAS.ti 24, and analyzed using abductive qualitative methods; interviews stopped at thematic saturation.
  • Participants were asked about how AI/ML factors into their work, promising and challenging applications, and issues including safety, governance, maintenance, and bias.

Limitations

  • The paper states that this is a small, qualitative interview study with limited size and scope, so perspectives may not be generalizable outside the sample; it relies on self-reported experiences that may be shaped by participants’ organizational positions and willingness to speak openly about failures or institutional shortcomings.
  • Evident from the text: all participants were engaged in academic research in some capacity, many were clinicians or legal/regulatory practitioners, and the sample was predominantly U.S.-based, which may narrow the range of institutional and regulatory perspectives captured.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s12913-025-13388-z

Cites

21 of 21 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

10 Valid · 0 Not valid · 17 Not checked · 80 Own results and interpretations

49 more claims · 2 with their cited sources checked

Create a free account to see all 107 claims and to verify the 5 references still open.

See all claims — free account
  • It may state two separate claims as one. 38
  • Probably not a substantial claim. 24
  • The quote only partly states this claim. 1

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

1 of 1

Citations

Cites

Show all 20 references

Research works printed but not linked: 1

  1. Vaughan D. Analogy, Cases, and Comparative Social Organization. In: R Swedberg, ed. Theorizing in Social Science: The Context of Discovery. Redwood City, CA: Stanford University Press; 2014;61–84.

Cited by

See all 19

Metadata

Authors
Owens, K., Griffen, Z., & Damaraju, L.
Affiliations
  • NYU Langone Health (US)
  • New York University (US)
Corresponding author
Owens, Kellie
Venue
BMC Health Services Research
Published
2025-09-29
Volume
25
Issue
1
Pages
1217-1217
ISSN
1472-6963 (ISSN-L)
OpenAlex
W4414628640
Cited by
19
References
20
Citations by year
2026: 19
FWCI
4.53
Type
journal article
Language
en
Open access
Licence
Locations
  • BMC Health Services Research (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Physical Sciences › Engineering › Safety, Risk, Reliability and Quality › Safety Systems Engineering in Autonomy
Keywords
AI governance, AI monitoring, artificial intelligence in healthcare, data drift, model degradation, institutional incentives, accountability, AI safety, AI equity, patient harm, trust in AI
MeSH terms
Machine Learning, Artificial Intelligence, Delivery of Health Care, Humans, Interviews as Topic, Social Responsibility, Qualitative Research

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Managing a 'responsibility vacuum' in AI monitoring and governance in healthcare: a qualitative study. https://app.cochat.ai/papers/10.1186/s12913-025-13388-z
The paper
Owens, K., Griffen, Z., & Damaraju, L. (2025). Managing a 'responsibility vacuum' in AI monitoring and governance in healthcare: a qualitative study. BMC Health Services Research. https://doi.org/10.1186/s12913-025-13388-z

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.