A governance model for the application of AI in health care
Open accessThe DOI matches its Crossref record; 33 of the 36 references it cites were found in a registry or an index.
Abstract
As the efficacy of artificial intelligence (AI) in improving aspects of healthcare delivery is increasingly becoming evident, it becomes likely that AI will be incorporated in routine clinical care in the near future. This promise has led to growing focus and investment in AI medical applications both from governmental organizations and technological companies. However, concern has been expressed about the ethical and regulatory aspects of the application of AI in health care. These concerns include the possibility of biases, lack of transparency with certain AI algorithms, privacy concerns with the data used for training AI models, and safety and liability issues with AI application in clinical environments. While there has been extensive discussion about the ethics of AI in health care, there has been little dialogue or recommendations as to how to practically address these concerns in health care. In this article, we propose a governance model that aims to not only address the ethical and regulatory issues that arise out of the application of AI in health care, but also stimulate further discussion about governance of AI in health care.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Generative AI models can be complex and difficult to interpret, making it challenging for clinicians to understand how the model arrived at its conclusions [13, 67].”
From Generative AI in healthcare: an implementation science informed translational path on application, integration and governance (2024) · cites this paper as [67]
In the citing paper
“This divergence was encouraging and suggested a gradual shift toward patient-centered, high-quality evidence capable of building clinical trust25,26 and satisfying regulatory expectations27,28.”
From A quantitative analysis of global AI medical studies: gaps in randomized controlled trials (2026) · cites this paper as [28]
In the citing paper
“Chen et al. and Reddy et al. proposed a structure of subpanels, overseen by an executive-level overarching committee, each with specific foci to spearhead different aspects of evaluation throughout the AI lifecycle8,17.”
From Advancing healthcare AI governance through a comprehensive maturity model based on systematic review (2026) · cites this paper as [17]
In the citing paper
“A recurring theme across multiple articles was the necessity for an overarching governance body7–10,17. This entity would not only assist in selecting AI tools and software for implementation but also guide the validation, implementation, and monitoring processes.”
From Advancing healthcare AI governance through a comprehensive maturity model based on systematic review (2026) · cites this paper as [17]
In the citing paper
“Existing hospital governance models, model-documentation tools, patient-safety approaches, and reporting standards also provide important components for institutional oversight [15][16][17][18][19][20][21][22][23][24].”
From The AI Hospital Formulary: A Practical Governance Framework for Prescribing, Monitoring, and Deprescribing Artificial Intelligence in Hospitals (2026) · cites this paper as [16]
40 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
From local explanations to global understanding with explainable AI for trees
Lundberg, S. M., Erion, G., Chen, H., DeGrave, A., Prutkin, J. M., Nair, B., Katz, R., Himmelfarb, J., Bansal, N., & Lee, S.-I. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-019-0138-9 · Cited by 10,551
Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI)
Adadi, A., & Berrada, M. (2018) · IEEE Access · doi:10.1109/access.2018.2870052 · Cited by 6,294
Health Literacy: A Prescription to End Confusion
Committee on Health Literacy, Board on Neuroscience and Behavioral Health, & Institute of Medicine (2004) · doi:10.17226/10883 · Cited by 3,444
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Cancer · doi:10.1038/bjc.2014.639 · Cited by 3,293
Causability and explainability of artificial intelligence in medicine
Holzinger, A., Langs, G., Denk, H., Zatloukal, K., & Müller, H. (2019) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1312 · Cited by 1,885
Implementing Machine Learning in Health Care — Addressing Ethical Challenges
Char, D. S., Shah, N. H., & Magnus, D. (2018) · New England Journal of Medicine · doi:10.1056/nejmp1714229 · Cited by 1,857
Health literacy: a prescription to end confusion
(2005) · Choice Reviews Online · doi:10.5860/choice.42-4059 · Cited by 1,438
Medical ethics: four principles plus attention to scope
Gillon, R. (1994) · BMJ · doi:10.1136/bmj.309.6948.184 · Cited by 1,107
Artificial intelligence, bias and clinical safety
Challen, R., Denny, J., Pitt, M., Gompels, L., Edwards, T., & Tsaneva-Atanasova, K. (2019) · BMJ Quality & Safety · doi:10.1136/bmjqs-2018-008370 · Cited by 956 · cited 2×
“1,19 Also, recent episodes of hospitals sharing patient data with AI developers without the patients’ informed consent has added to the problem of trusting AI developers and AI itself.”
BackgroundMachine learning in medicine: Addressing ethical challenges
Vayena, E., Blasimme, A., & Cohen, I. G. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002689 · Cited by 912 · cited 10×
“8 Unrepresentative data can occur due to, for example, societal discrimination (eg, poor access to health care) and relatively small samples (eg, minority groups).”
BackgroundInfluentialGoverning artificial intelligence: ethical, legal and technical opportunities and challenges
Cath, C. (2018) · Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences · doi:10.1098/rsta.2018.0080 · Cited by 745 · cited 3×
“As discussed, inappropriate and poorly representative training datasets for AI models can lead to biases, inaccurate predictions, medical errors, and even large scale discrimination.(3,5,8) Therefore, we recommend a data governance panel constituted by the AI developers that includes patient and target group represent…”
BackgroundArtificial intelligence-enabled healthcare delivery
Reddy, S., Fox, J., & Purohit, M. P. (2019) · Journal of the Royal Society of Medicine · doi:10.1177/0141076818815510 · Cited by 635
Self-citationGoogle DeepMind and healthcare in an age of algorithms
Powles, J., & Hodson, H. (2017) · Health and Technology · doi:10.1007/s12553-017-0179-1 · Cited by 511 · cited 2×
“…episodes like Cambridge Ana-lytica using personal data collected by Facebook for political advertising 11 and the Royal Free London NHS Foundation trust sharing patient data for the development of a clinical application without explicit patient consent 12 present concerns about privacy breaches.”
BackgroundMethodLee, H., Yune, S., Mansouri, M., Kim, M., Tajmir, S. H., Guerrier, C. E., Ebert, S. A., Pomerantz, S. R., Romero, J. M., Kamalian, S., et al. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0324-9 · Cited by 441
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · European Urology · doi:10.1016/j.eururo.2014.11.025 · Cited by 439
IBM Watson, heal thyself: How IBM overpromised and underdelivered on AI health care
Strickland, E. (2019) · IEEE Spectrum · doi:10.1109/mspec.2019.8678513 · Cited by 402
Machine learning and medical education
Kolachalama, V. B., & Garg, P. S. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0061-1 · Cited by 320 · cited 1×
“31,32 We believe these initiatives are a vital element in building trust for AI among healthcare professionals.”
BackgroundExplainable artificial intelligence for breast cancer: A visual case-based reasoning approach
Lamy, J.-B., Sekar, B., Guezennec, G., Bouaud, J., & Séroussi, B. (2019) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2019.01.001 · Cited by 282
Regulation of predictive analytics in medicine
Parikh, R. B., Obermeyer, Z., & Navathe, A. S. (2019) · Science · doi:10.1126/science.aaw0029 · Cited by 223 · cited 1×
“14 Regulatory standards to assess AI algorithmic safety and impact are yet to be formalized in many countries.”
BackgroundImplementing clinical governance: turning vision into reality
Halligan, A., & Donaldson, L. (2001) · BMJ · doi:10.1136/bmj.322.7299.1413 · Cited by 185
Health Literacy: A Prescription to End Confusion
Nielsen-Bohlman, L., Panzer, A. M., Kindig, D. A., & Institute of Medicine (U.S.) (2004) · Cited by 173
Confronting the risks of artificial intelligence
Cheatham, B., Javanmardian, K., & Samandari, H. (2019) · Cited by 123
Bowens, F. M., Frye, P. A., & Jones, W. A. (2010) · PubMed · Cited by 119
(2019) · AMA Journal of Ethics · doi:10.1001/amajethics.2019.138 · Cited by 89 · cited 1×
“34 Where patient data may be shared with AI developers, there must be a process to seek fully informed consent from patients and if it is unrealistic to seek approval, data must be anonymized to that extent individual patient details cannot be recognized by the developers.”
BackgroundRegulation of medical devices outside the European Union
Lamph, S. (2012) · Journal of the Royal Society of Medicine · doi:10.1258/jrsm.2012.120037 · Cited by 88
Patient involvement 2.0: Experience-based co-design supported by action research
Gustavsson, S., & Andersson, T. (2017) · Action Research · doi:10.1177/1476750317723965 · Cited by 73
Focus Group on Artificial Intelligence for HealthPreprint
Salathé, M., Wiegand, T., & Wenzel, M. A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1809.04797 · Cited by 69
Froomkin, A. M., Kerr, I., & Pineau, J. (2025) · Robot Law: Volume II · doi:10.4337/9781800887305.00012 · Cited by 59
What Is Trust? Ethics and Risk Governance in Precision Medicine and Predictive Analytics
Adjekum, A., Ienca, M., & Vayena, E. (2017) · OMICS: A Journal of Integrative Biology · doi:10.1089/omi.2017.0156 · Cited by 56
AI and the Ethics of Automating Consent
Jones, M. L., Kaufman, E., & Edenberg, E. (2018) · IEEE Security & Privacy · doi:10.1109/msp.2018.2701155 · Cited by 50
Lupton, M. (2018) · Trends in Medicine · doi:10.15761/tim.1000147 · Cited by 40
Should Watson Be Consulted for a Second Opinion?
(2019) · AMA Journal of Ethics · doi:10.1001/amajethics.2019.131 · Cited by 33 · cited 3×
“16 What the implications will be for medical malpractice in the context of dominant AI-driven diagnostics is yet to be seen.”
BackgroundMethodScott, J., Heavey, E., Waring, J. J., Jones, D. E., & Dawson, P. (2016) · BMJ Open · doi:10.1136/bmjopen-2016-011222 · Cited by 31
Research works printed but not linked: 4
- 2018, Accelerating AI in Health and Care: Results from a State of the Nation Survey
- Angwin, 2016, Machine Bias, ProPublica
- 2011, Global Observatory for eHealth series, 5
- Wakabayashi, 2019, Google and the University of Chicago are sued over data sharing, The New York Times
Cited by
Ethical and regulatory challenges of AI technologies in healthcare: A narrative review
Mennella, C., Maniscalco, U., De Pietro, G., & Esposito, M. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e26297 · Cited by 1,229
Ethical principles for artificial intelligence in education
Nguyen, A., Ngo, H. N., Hong, Y., Dang, B., & Nguyen, B.-P. T. (2023) · Education and Information Technologies · doi:10.1007/s10639-022-11316-w · Cited by 1,228
“…regarding ethics of data exploitation in decision making and interventions occur cross-disciplines (Jalal et al., 2021 ; Farris, 2021 ; medical care as in Reddy et al., 2020 ) or human resources management, as in Tambe et al., (2019); sports per - formance analysis as in Araújo et al., ( 2021 ).”
BackgroundA Review of the Role of Artificial Intelligence in Healthcare
Al Kuwaiti, A., Nazer, K., Al-Reedy, A., Al-Shehri, S., Al-Muhanna, A., Subbarayalu, A. V., Al Muhanna, D., & Al-Muhanna, F. A. (2023) · Journal of Personalized Medicine · doi:10.3390/jpm13060951 · Cited by 915
Shneiderman, B. (2020) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3419764 · Cited by 768
“Implementations of safety cultures in HCAI-based organizations is emerging with initial efforts to support AI governance in medical care [19, 93].”
BackgroundWilliamson, S. M., & Prybutok, V. (2024) · Applied Sciences · doi:10.3390/app14020675 · Cited by 567
“Blockchain offers opportunities for enhancing data security and patient privacy but also challenges aligning with GDPR requirements.”
BackgroundInfluential
Metadata
- Authors
- Reddy, S., Allan, S., Coghlan, S., & Cooper, P.
- Affiliations
- Deakin University (AU)
- The University of Melbourne (AU)
- Corresponding author
- Reddy, Sandeep
- Venue
- Journal of the American Medical Informatics Association
- Publisher
- Oxford University Press (OUP)
- Published
- 2019-11-04
- Volume
- 27
- Issue
- 3
- Pages
- 491-497
- ISSN
- 1067-5027 (ISSN-L)
- 1527-974X
- PMC
- PMC7647243
- OpenAlex
- W2989512989
- Cited by
- 979
- References
- 36
- Citations by year
- 2026: 219 · 2025: 232 · 2024: 201 · 2023: 168 · 2022: 77 · 2021: 54 · 2020: 28
- FWCI
- 12.64
- Type
- journal article
- Language
- en
- Open access
- green
- www.ncbi.nlm.nih.gov
- Licence
- academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model (vor)
- Locations
- Journal of the American Medical Informatics Association (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Figshare (submittedVersion, cc-by)
- Minerva Access (University of Melbourne) (submittedVersion)
- RUNE (Research UNE) (submittedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Law › Digitalization, Law, and Regulation
- Keywords
- artificial intelligence in healthcare, AI governance models, AI ethics, regulatory challenges, algorithmic bias, algorithmic transparency, patient data privacy
- MeSH terms
- Artificial Intelligence, Delivery of Health Care, Ethics, Medical, Humans, Models, Theoretical, Organizational Policy, Government Regulation, Workflow
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.