CoChat
Sign Up
Cited by 7Ethics and Social Impacts of AIArtificial Intelligence in Healthcare and EducationDigitalization, Law, and Regulation

When AI is wrong: addressing liability challenges in women's healthcare

Open access
Marotta, Angelica·2022·Journal of Computer Information Systems
DOI 10.1080/08874417.2022.2089773
Open this paper in CoChat — free account

Abstract

Healthcare professionals can leverage Artificial intelligence (AI) to provide better care for their patients. However, it is also necessary to consider that AI algorithms operate according to historical diagnostic data, which often include evidence gathered from men. The biases of prior practices and the perpetuation of exclusionary processes toward women can lead to inaccurate medical decisions. The ramifications of such errors show that the incorrect use of AI raises several critical questions regarding who should be responsible for potential incidents. This study aims to provide an analysis of the role of AI in affecting women’s healthcare and an overview of the liability implications caused by AI mistakes. Finally, this work presents a framework for algorithmic auditing to ensure that AI data are collected and stored according to secure, legal, and fair practices.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1080/08874417.2022.2089773

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Demographic imbalances in datasets-such as overrepresentation of younger, male participants-can marginalize women, older adults, and individuals with disabilities in AI-driven healthcare.

    “Furthermore, demographic imbalances in datasets—such as the overrepresentation of younger, male participants—can marginalize women, older adults, and individuals with disabilities in AI-driven healthcare [14].”

    From Frameworks encompassing intersectional perspective of artificial intelligence in healthcare. Scoping review (2026) · cites this paper as [14]

    The Introduction (page unavailable) says women have been excluded from some clinical trials, that some treatments were tested primarily on males, and that such biases in AI can harm women. It does not establish overrepresentation of younger participants or extend the resulting marginalization to older adults or people with disabilities. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • A recent study found that robots trained on large datasets exhibited stereotypical and biased behaviors in terms of gender and race.

    “A recent study found that robots trained on large datasets exhibited stereotypical and biased behaviors in terms of gender and race [14,15].”

    From Frameworks encompassing intersectional perspective of artificial intelligence in healthcare. Scoping review (2026) · cites this paper as [14]

    The Introduction says biased AI algorithms can perpetuate gender inequities, and the Conclusions explain how bias in training data can alter AI decisions. The paper does not report a study of robots trained on large datasets or findings about racial stereotypes, so those specifics are unsupported. (LLM, conf=0.78)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 42

  1. Shaheen MY. Applications of Artificial Intelligence (AI) in healthcare: a review. Sci Prepr. 2021. Published online doi: 10.14293/S2199-1006.1.SOR-.PPVRY8K.v1. · DOI
  2. Sim KM. Bilattices and reasoning in artificial intelligence: concepts and foundations. Artif Intell Rev. 2001;15(3):219–40. doi:10.1023/A:1011049617655. · DOI
  3. Davenport T, Kalakota R. The potential for artificial intelligence in healthcare. Futur Healthc J. 2019;6(2):94–98. doi:10.7861/futurehosp.6-2-94. · DOI · Paper page
  4. Rong G, Mendez A, Bou Assi E, Zhao B, Sawan M. Artificial intelligence in healthcare: review and prediction case studies. Engineering. 2020;6(3):291–301. doi:10.1016/j.eng.2019.08.015. · DOI
  5. Tulk Jesso S, Kelliher A, Sanghavi H, Martin T, Henrickson Parker S. Inclusion of clinicians in the development and evaluation of clinical artificial intelligence tools: a systematic literature review. Front Psychol. 2022:13. doi:10.3389/fpsyg.2022.830345. · DOI
  6. Bates DW, Auerbach A, Schulam P, Wright A, Saria S. Reporting and implementing interventions involving machine learning and artificial intelligence. Ann Intern Med. 2020;172(11_Supplement):S137–S144. doi:10.7326/M19-0872. · DOI
  7. Schwendicke F, Krois J. Better reporting of studies on artificial intelligence: CONSORT-AI and beyond. J Dent Res. 2021;100(7):677–80. doi:10.1177/0022034521998337. · DOI
  8. Mun SK, Wong KH, Lo SCB, Li Y, Bayarsaikhan S. Artificial intelligence for the future radiology diagnostic service. Front Mol Biosci. 2021;7:512. doi:10.3389/fmolb.2020.614258. · DOI
  9. Shah NR. Health care in 2030: will artificial intelligence replace physicians? Ann Intern Med. 2019;170(6):407–08. doi:10.7326/M19-0344. · DOI
  10. Sanford T, Harmon SA, Turkbey EB, Kesani D, Tuncer S, Madariaga M, Yang C, Sackett J, Mehralivand S, Yan P, et al. Deep-learning-based artificial intelligence for PI-RADS classification to assist multiparametric prostate MRI interpretation: a development study. J Magn Reson Imaging. 2020;52(5):1499–507. doi:10.1002/jmri.27204. · DOI
  11. Schriger DL, Elder JW, Cooper RJ. Structured clinical decision aids are seldom compared with subjective physician judgment, and are seldom superior. Ann Emerg Med. 2017;70(3):338–344.e3. doi:10.1016/j.annemergmed.2016.12.004. · DOI
  12. Hu Y, Jacob J, Parker GJM, Hawkes DJ, Hurst JR, Stoyanov D. The challenges of deploying artificial intelligence models in a rapidly evolving pandemic. Nat Mach Intell. 2020;2(6):298–300. doi:10.1038/s42256-020-0185-2. · DOI
  13. Panch T, Mattie H, Celi LA. The “inconvenient truth” about AI in healthcare. Npj Digit Med. 2019;2(1):1–3. doi:10.1038/s41746-019-0155-4. · DOI · Paper page
  14. Doyen S, Dadario NB. 12 plagues of AI in healthcare: a practical guide to current issues with using machine learning in a medical context. Front Digit Heal. 2022;74. doi:10.3389/FDGTH.2022.765406. · DOI
  15. Wynants L, Van Calster B, Collins GS, Riley RD, Heinze G, Schuit E, Bonten MMJ, Dahly DL, Damen JAA, Debray TPA, et al. Prediction models for diagnosis and prognosis of covid-19: systematic review and critical appraisal. BMJ. 2020;369. doi:10.1136/bmj.m1328. · DOI · Paper page
  16. Bentley AR, Callier S, Rotimi CN. Diversity and inclusion in genomic research: why the uneven progress? J Community Genet. 2017;8(4):255–66. doi:10.1007/s12687-017-0316-6. · DOI · Paper page
  17. Chen IY, Szolovits P, Ghassemi M. Can AI help reduce disparities in general medical and mental health care? AMA J Ethics. 2019;21(2):167–79. doi:10.1001/amajethics.2019.167. · DOI · Paper page
  18. Cirillo D, Catuara-Solarz S, Morey C, Guney E, Subirats L, Mellino S, Gigante A, Valencia A, Rementeria MJ, Chadha AS, et al. Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare. Npj Digit Med. 2020;3(1):1–11. doi:10.1038/s41746-020-0288-5. · DOI · Paper page
  19. Miller IN, Cronin-Golomb A. Gender differences in Parkinson’s disease: clinical characteristics and cognition. Mov Disord. 2010;25(16):2695–703. doi:10.1002/MDS.23388. · DOI · Paper page
  20. Ferretti MT, Iulita MF, Cavedo E, Chiesa PA, Schumacher Dimech A, Santuccione Chadha A, Baracchi F, Girouard H, Misoch S, Giacobini E, et al. Sex differences in Alzheimer disease — the gateway to precision medicine. Nat Rev Neurol. 2018;14(8):457–69. doi:10.1038/s41582-018-0032-9. · DOI · Paper page
  21. Reddy S, Allan S, Coghlan S, Cooper P. A governance model for the application of AI in health care. J Am Med Informatics Assoc. 2020;27(3):491–97. doi:10.1093/jamia/ocz192. · DOI · Paper page
  22. Asan O, Bayrak AE, Artificial Intelligence CA. Human trust in healthcare: focus on clinicians. J Med Internet Res. 2020;22(6):e15154. doi:10.2196/15154. · DOI · Paper page
  23. Wiens J, Saria S, Sendak M, Ghassemi M, Liu VX, Doshi-Velez F, Jung K, Heller K, Kale D, Saeed M, et al. Do no harm: a roadmap for responsible machine learning for health care. Nat Med. 2019;25(9):1337–40. doi:10.1038/s41591-019-0548-6. · DOI · Paper page
  24. Janowitz MF, Solow D. How to read and do proofs. Am Math Mon. 1993;100(2):197. doi:10.2307/2323794. · DOI
  25. DeSanctis G, Gallupe RB. Decision support systems: concepts and resources for managers. Manage Sci. 1987; 20(4):80–81. [Accessed May 16, 2022]. https://books.google.com/books?hl=en‎&id=9NA6QMcte3cC&oi=fnd&pg=PR9&dq=+%22Decision+Support+Systems%22&ots=DPpsrwMvBa&sig=NgG5F6ya69eWvn6SsgY3SVG9pgw(open in a new window)
  26. Charu C, Aggarwal JH. Data mining: the textbook. Springer Int Publ. Published online 2015:746. Accessed May 16, 2022. https://books.google.com/books/about/Data_Mining.html?hl=it&id=cfNICAAAQBAJ(open in a new window)
  27. Bonettini S, Porta F, Ruggiero V. A variable metric forward-backward method with extrapolation. SIAM J Sci Comput. 2016;38(4):A2558–A2584. doi:10.1137/15M1025098. · DOI
  28. Dixit A, Sahu DR, Gautam P, Som T, Yao JC. An accelerated forward-backward splitting algorithm for solving inclusion problems with applications to regression and link prediction problems. J Nonlinear Var Anal. 2021;5(1):79–101. doi:10.23952/JNVA.5.2021.1.06. · DOI
  29. He B, Yuan X. Forward–backward-based descent methods for composite variational inequalities. Optimization Methods and Software . 2013;28(4):706–24. doi:10.1080/10556788.2011.645033. · DOI
  30. Mullen M, Brennan C, Downes T. A hybridized forward backward method applied to electromagnetic wave scattering problems. IEEE Trans Antennas Propag. 2009;57(6):1846–50. doi:10.1109/TAP.2009.2019994. · DOI
  31. Holliday D, Deraad LL, St-Cyr GJ. Forward-backward method for scattering from imperfect conductors. IEEE Trans Antennas Propag. 1998;46(1):101–07. doi:10.1109/8.655456. · DOI
  32. Phillips-Wren G. AI tools in decision making support systems: a review. Int J Artif Intell Tools. 2012;21(2):2. doi:10.1142/S0218213012400052. · DOI
  33. Bleher H, Braun M. Diffused responsibility: attributions of responsibility in the use of AI-driven clinical decision support systems. AI Ethics. 2022;1:1. doi:10.1007/s43681-022-00135-x. · DOI
  34. Maliha G, Gerke S, Cohen IG, Parikh RB. Artificial intelligence and liability in medicine: balancing safety and innovation. Milbank Q. 2021 April 6;99(3):629–47. Published online. doi:10.1111/1468-0009.12504. · DOI
  35. Lysaght T, Lim HY, Xafis V, Ngiam KY. AI-assisted decision-making in healthcare: the application of an ethics framework for big data in health and research. Asian Bioeth Rev. 2019;11(3):299–314. doi:10.1007/s41649-019-00096-0. · DOI · Paper page
  36. DePaul KS. Journal of health care law DePaul journal of health care law part of the health law and policy commons recommended citation recommended citation Sarah kamensky, artificial intelligence and technology in health care: overview and possible legal implications. 21 DePaul J Heal Care L. 2020. Published online doi: 10.1098/rsta.2017.0360. · DOI · Paper page
  37. Gerke S, Minssen T, Cohen G. Ethical and legal challenges of artificial intelligence-driven healthcare. Artificial intelligence in healthcare 1st Bohr, A, Memarzadeh, K. Elsevier; 2020. 295–336. doi: 10.1016/b978-0-12-818438-7.00012-5. · DOI · Paper page
  38. Yu R, Alì GS. What’s inside the black box? AI Challenges for lawyers and researchers. In: Legal information management. Vol. 19. Cambridge, UK: Cambridge University Press (CUP); 2019. p. 2–13. doi:10.1017/s1472669619000021. · DOI
  39. Zhou M, Guo J, Chen N, Ma M, Dong S, Li Y, Fang J, Zhang Y, Zhang Y, Bao J, et al. Effects of message framing and time discounting on health communication for optimum cardiovascular disease and stroke prevention (EMT-OCSP): a protocol for a pragmatic, multicentre, observer-blinded, 12-month randomised controlled study. BMJ Open. 2021;11(3):3. doi:10.1136/bmjopen-2020-043450. · DOI
  40. Garbin C, Marques MO. Tools to improve reporting, increaseTransparency, and reduce failures in machine learning applications in healthCare. Radiol Artif Intell. 2022;4(2):2. doi:10.1148/RYAI.210127. · DOI
  41. Johnston AC, Warkentin M. Information privacy compliance in the healthcare industry. Inf Manag & Comput Secur. 2008;16(1):5–19. doi:10.1108/09685220810862715. · DOI
  42. Mukherjee D, Yurochkin M, Banerjee M, Sun Y. Two simple ways to learn individual fairness metrics from data. 37th Int Conf Mach Learn ICML 2020. 2020;PartF16814: 7054-7064. Virtual Event. Accessed June 3, 2022. https://github.com/(open in a new window)

Cited by

Sorted by citations

Authors
Marotta, A.
Affiliations
Massachusetts Institute of Technology (US)
Corresponding author
Marotta, Angelica
ORCID
Marotta, Angelica
Venue
Journal of Computer Information Systems
Publisher
Informa UK Limited
Published
2022-06-20
Volume
62
Issue
6
Pages
1310-1319
ISSN
  • 0887-4417 (ISSN-L)
  • 2380-2057
DOI
10.1080/08874417.2022.2089773
OpenAlex
W4283163496
Cited by
7
References
9
Citations by year
2026: 2 · 2025: 1 · 2024: 4
FWCI
1.45
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Computer Information Systems (publishedVersion, cc-by-nc-nd)
  • DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc-nd)
Topics
  • Social Sciences › Safety Research › Ethics and Social Impacts of AI
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Law › Digitalization, Law, and Regulation
Keywords
artificial intelligence in healthcare, women's healthcare, algorithmic bias, liability, medical errors, algorithmic auditing, data privacy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review