Making decisions: Bias in artificial intelligence and data‑driven diagnostic tools
Open accessAbstract
BACKGROUND: Although numerous studies have shown the potential of artificial intelligence (AI) systems in drastically improving clinical practice, there are concerns that these AI systems could replicate existing biases. OBJECTIVE: This paper provides a brief overview of 'algorithmic bias', which refers to the tendency of some AI systems to perform poorly for disadvantaged or marginalised groups. DISCUSSION: AI relies on data generated, collected, recorded and labelled by humans. If AI systems remain unchecked, whatever biases that exist in the real world that are embedded in data will be incorporated into the AI algorithms. Algorithmic bias can be considered as an extension, if not a new manifestation, of existing social biases, understood as negative attitudes towards or the discriminatory treatment of some groups. In medicine, algorithmic bias can compromise patient safety and risks perpetuating disparities in care and outcome. Thus, clinicians should consider the risk of bias when deploying AI-enabled tools in their practice.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
If a healthcare provider relies solely on patient ratings and reviews to select a specialist, they may unknowingly favor providers who have received more positive feedback, potentially overlooking highly competent specialists who have fewer online ratings.
“For instance, if a healthcare provider relies solely on patient ratings and reviews to select a specialist, they may unknowingly favor providers who have received more positive feedback, potentially overlooking highly competent specialists who have fewer online ratings [88], [89].”
From Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond (2024) · cites this paper as [89]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Yaşlı Kadınlarda Üreme Sağlığı
Kokanalı, D., & Engin Üstün, Y. (n.d.) · DergiPark (Istanbul University) · Cited by 170,090 · see citing works
Racial bias in pulse oximetry measurement
Sjoding, M. W., Dickson, R. P., Iwashyna, T. J., Gay, S. E., & Valley, T. S. (2020) · New England Journal of Medicine · doi:10.1056/nejmc2029240 · Cited by 1,114 · see citing works
Addressing bias in big data and AI for health care: A call for open science
Norori, N., Hu, Q., Aellen, F. M., Faraci, F. D., & Tzovara, A. (2021) · Patterns · doi:10.1016/j.patter.2021.100347 · Cited by 903 · see citing works
Larrazabal, A. J., Nieto, N., Peterson, V., Milone, D. H., & Ferrante, E. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1919012117 · Cited by 685 · see citing works
Machine learning: applications of artificial intelligence to imaging and diagnosis
Nichols, J., Chan, H. H., & Baker, M. A. B. (2018) · Biophysical Reviews · doi:10.1007/s12551-018-0449-9 · Cited by 458 · see citing works
Deep neural networks are superior to dermatologists in melanoma image classification
Brinker, T. J., Hekler, A., Enk, A. H., Berking, C., Haferkamp, S., Hauschild, A., Weichenthal, M., Klode, J., Schadendorf, D., Holland-Letz, T., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2019.05.023 · Cited by 415 · see citing works
Deep Learning applications for COVID-19
Shorten, C., Khoshgoftaar, T. M., & Furht, B. (2021) · Journal of Big Data · doi:10.1186/s40537-020-00392-9 · Cited by 350 · see citing works
Cheung, C. Y.-L., Xu, D., Cheng, C., Sabanayagam, C., Tham, Y. C., Yu, M., Rim, T. H., Chai, C. Y., Gopinath, B., Mitchell, P. L. R., et al. (2020) · Nature Biomedical Engineering · doi:10.1038/s41551-020-00626-4 · Cited by 306 · see citing works
Ten Ways Artificial Intelligence Will Transform Primary Care
Lin, S. Y., Mahoney, M. R., & Sinsky, C. A. (2019) · Journal of General Internal Medicine · doi:10.1007/s11606-019-05035-1 · Cited by 248 · see citing works
Royce, C. S., Hayes, M. M., & Schwartzstein, R. M. (2019) · Academic Medicine · doi:10.1097/acm.0000000000002518 · Cited by 242 · see citing works
Latent bias and the implementation of artificial intelligence in medicine
DeCamp, M., & Lindvall, C. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa094 · Cited by 198 · see citing works
Critical appraisal of artificial intelligence-based prediction models for cardiovascular disease
van Smeden, M., Heinze, G., CALSTER, B. V., Asselbergs, F. W., Vardas, P. E., Bruining, N., de Jaegere, P. P. T., Moore, J. H., Denaxas, S., Boulesteix, A., et al. (2022) · European Heart Journal · doi:10.1093/eurheartj/ehac238 · Cited by 147 · see citing works
Aquino, Y. S. J., Rogers, W. A., Braunack-Mayer, A., Frazer, H., Win, K. T., Houssami, N., Degeling, C., Semsarian, C., & Carter, S. M. (2023) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2022.104903 · Cited by 131 · see citing works
Self-citationFDA-regulated AI Algorithms: Trends, Strengths, and Gaps of Validation Studies
Ebrahimian, S., Kalra, M. K., Agarwal, S., Bizzo, B. C., Elkholy, M., Wald, C., Allen, B., & Dreyer, K. J. (2022) · Academic Radiology · doi:10.1016/j.acra.2021.09.002 · Cited by 120 · see citing works
Valbuena, V. S. M., Barbaro, R. P., Claar, D. D., Valley, T. S., Dickson, R. P., Gay, S. E., Sjoding, M. W., & Iwashyna, T. J. (2021) · CHEST Journal · doi:10.1016/j.chest.2021.09.025 · Cited by 114 · see citing works
Kamulegeya, L. H., Okello, M., Bwanika, J. M., Musinguzi, D., Lubega, W., Rusoke, D., Nassiwa, F., & Börve, A. (2019) · doi:10.1101/826057 · Cited by 111 · see citing works
Achieving fairness in medical devices
Kadambi, A. (2021) · Science · doi:10.1126/science.abe9195 · Cited by 51 · see citing works
Strategies to Mitigate Clinician Implicit Bias Against Sexual and Gender Minority Patients.
McDowell, M., Goldhammer, H., Potter, J., & Keuroghlian, A. (2020) · Psychosomatics · doi:10.1016/j.psym.2020.04.021 · Cited by 34 · see citing works
O’Brien, P., Bunzli, S., Lin, I. B., Bessarab, D. C., Coffin, J., Dowsey, M. M., & Choong, P. (2021) · ANZ Journal of Surgery · doi:10.1111/ans.16557 · Cited by 31 · see citing works
Quality use of artificial intelligence in medical imaging: What do radiologists need to know?
Goergen, S., Frazer, H. M. L., & Reddy, S. (2022) · Journal of Medical Imaging and Radiation Oncology · doi:10.1111/1754-9485.13379 · Cited by 7 · see citing works
Assembled Bias: Beyond Transparent Algorithmic Bias
Waller, R. L., & Waller, R. L. (2022) · Minds and Machines · doi:10.1007/s11023-022-09605-x · Cited by 7 · see citing works
Schlender, T., & Spanakis, G. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-76640-5_9 · Cited by 5 · see citing works
Regulation of AI in Health Care: A Cautionary Tale Considering Horses and Zebras.
Richards, B. J., Jacobson, S. S., & Aquino, Y. S. J. (2021) · PubMed · Cited by 4 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 4
- Ntoutsi E, Fafalios P, Gadiraju U, et al. Bias in data-driven artificial intelligence systems -an introductory survey. WIREs Data Mining Knowl Discov 2020;10(3):e1356.
- Kordzadeh N, Ghasemaghaei M. Algorithmic bias: Review, synthesis, and future research directions. Eur J Inf Syst 2022;31(3):388-409.
- Dovidio JF, Hewstone M, Glick P, Esses VM. Prejudice, stereotyping and discrimination: Theoretical and empirical overview. In: Dovidio JF, Hewstone M, Glick P, Esses VM, editors. The SAGE Handbook of Prejudice, Stereotyping and Discrimination. Thousand Oaks: SAGE Publications, 2010; p. 3-28.
- Yuen T, Derenge D, Kalman N. Cognitive bias: Its influence on clinical diagnosis. J Fam Pract 2018;67(6):366;368;370;372.
Cited by
Sorted by citations
Ethical Framework for Harnessing the Power of AI in Healthcare and Beyond
Nasir, S., Khan, R. A., & Bai, S. (2024) · IEEE Access · doi:10.1109/access.2024.3369912 · Cited by 152 · see citing works
Busch, F., Hoffmann, L., Truhn, D., Palaian, S., Alomar, M., Shpati, K., Makowski, M. R., Bressem, K. K., & Adams, L. C. (2024) · British Journal of Clinical Pharmacology · doi:10.1111/bcp.15911 · Cited by 80 · see citing works
“17,18 21 However, it is critical to explore these perspectives to inform curricula and prepare future pharmacists for the increasing emergence of AI in healthcare.”
Background
- Authors
- Aquino, Y. S. J.
- Affiliations
- University of Wollongong (AU)
- Corresponding author
- Aquino, Yves Saint James
- Venue
- Australian Journal of General Practice
- Published
- 2023-07-01
- Volume
- 52
- Issue
- 7
- Pages
- 439-442
- ISSN
- 2208-794X (ISSN-L)
- OpenAlex
- W4383710950
- Cited by
- 58
- References
- 23
- Citations by year
- 2026: 13 · 2025: 25 · 2024: 17 · 2023: 2
- FWCI
- 14.29
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Australian Journal of General Practice (publishedVersion)
- PubMed (publishedVersion)
- Research Online (University of Wollongong) (submittedVersion)
- Funders
- Medical Research Council
- National Health and Medical Research Council (1181960)
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- algorithmic bias, artificial intelligence in healthcare, health disparities, social bias, patient safety, clinical decision support, marginalised groups
- MeSH terms
- Artificial Intelligence, Decision Making, Humans, Medicine, Bias, Patient Safety
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
