The DOI matches its Crossref record; 96 of the 96 references it cites were found in a registry or an index.
The Sociodemographic Biases in Machine Learning Algorithms: A Biomedical Informatics Perspective
Open accessAbstract
Artificial intelligence models represented in machine learning algorithms are promising tools for risk assessment used to guide clinical and other health care decisions. Machine learning algorithms, however, may house biases that propagate stereotypes, inequities, and discrimination that contribute to socioeconomic health care disparities. The biases include those related to some sociodemographic characteristics such as race, ethnicity, gender, age, insurance, and socioeconomic status from the use of erroneous electronic health record data. Additionally, there is concern that training data and algorithmic biases in large language models pose potential drawbacks. These biases affect the lives and livelihoods of a significant percentage of the population in the United States and globally. The social and economic consequences of the associated backlash cannot be underestimated. Here, we outline some of the sociodemographic, training data, and algorithmic biases that undermine sound health care risk assessment and medical decision-making that should be addressed in the health care system. We present a perspective and overview of these biases by gender, race, ethnicity, age, historically marginalized communities, algorithmic bias, biased evaluations, implicit bias, selection/sampling bias, socioeconomic status biases, biased data distributions, cultural biases and insurance status bias, conformation bias, information bias and anchoring biases and make recommendations to improve large language model training data, including de-biasing techniques such as counterfactual role-reversed sentences during knowledge distillation, fine-tuning, prefix attachment at training time, the use of toxicity classifiers, retrieval augmented generation and algorithmic modification to mitigate the biases moving forward.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · Advances in Neural Information Processing Systems 14 · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,223
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 4×
“With the advent of big data analytics, EHRs are often the source of training data for AIMs and, subsequently, MLAGs, thus creating a feedback loop of bias (Figure 1) where existing inequities and disparities can be exacerbated through positive reinforcement methods [12,50].”
BackgroundInfluentialOn the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812 · cited 2×
“The fact is that MLAGs born of AIMs may not perform as well on women as on men when they are under-represented in the training data [16,74].”
BackgroundExplanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Implicit bias in healthcare professionals: a systematic review
FitzGerald, C., & Hurst, S. (2017) · BMC Medical Ethics · doi:10.1186/s12910-017-0179-8 · Cited by 2,556 · cited 2×
“Implicit bias imposed by clinicians has an impact on the overall quality of patients’ health and health care and can lead to unequal treatment based on certain demographics [17,23,48].”
BackgroundHidden in Plain Sight — Reconsidering the Use of Race Correction in Clinical Algorithms
Vyas, D. A., Eisenstein, L. G., & Jones, D. S. (2020) · New England Journal of Medicine · doi:10.1056/nejmms2004740 · Cited by 1,731 · cited 3×
“While recommendations of race-based adjustment have been considered to mitigate these issues, these algorithms can also be negatively biased toward Black individuals, further perpetuating racial bias in health care [40].”
BackgroundAlgorithmic Decision Making and the Cost of Fairness
Corbett-Davies, S., Pierson, E., Feller, A., Goel, S., & Huq, A. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098095 · Cited by 1,149
Measuring Diagnoses: ICD Code Accuracy
O'Malley, K. J., Cook, K. F., Price, M. D., Wildes, K. R., Hurdle, J. F., & Ashton, C. M. (2005) · Health Services Research · doi:10.1111/j.1475-6773.2005.00444.x · Cited by 1,116
Research works printed but not linked: 2
- Hellström, T., Dignum, V., and Bensch, S. (2020). Bias in Machine Learning—What Is It Good For?. arXiv.
- Liyanage, 2023, Ethical considerations and potential risks in the deployment of large Language Models in diverse societal contexts, J. Comput. Soc. Dyn., 8, 15
Cited by
Mohammed, S. A. A. Q., Osman, Y. M. M., Ibrahim, A. M., & Shaban, M. (2025) · International Nursing Review · doi:10.1111/inr.70010 · Cited by 105
Alhumaidi, N. H., Dermawan, D., Kamaruzaman, H. F., & Alotaiq, N. (2025) · JMIR Medical Informatics · doi:10.2196/68898 · Cited by 109
“By incorporating fairness-aware ML models and building representative datasets, health care practitioners can ensure that ML applications benefit all patient groups, regardless of demographics [20].”
Diet Quality and Caloric Accuracy in AI-Generated Diet Plans: A Comparative Study Across Chatbots
Kaçar, H. K., Kaçar, Ö. F., & Avery, A. (2025) · Nutrients · doi:10.3390/nu17020206 · Cited by 49
“This disparity could arise from algorithmic biases informed by societal norms or training datasets that emphasise greater dietary diversity for women [56,57], possibly due to their unique nutritional requirements during reproductive years or broader dietary trends [58].”
Advancing clinical biochemistry: addressing gaps and driving future innovations
Cao, H., Oghenemaro, E. F., latypova, A., Abosaoda, M. K., Zaman, G. S., & Devi, A. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1521126 · Cited by 47
“Explainability holds great importance in the context of clinical responsibility (192, 193).”
Data Quality in the Age of AI: A Review of Governance, Ethics, and the FAIR Principles
Guillen-Aguinaga, M., Aguinaga-Ontoso, E., Guillen-Aguinaga, L., Guillen-Grima, F., & Aguinaga-Ontoso, I. (2025) · Data · doi:10.3390/data10120201 · Cited by 37
Metadata
- Authors
- Franklin, G., Stephens, R., Piracha, M., Tiosano, S., Lehouillier, F., Koppel, R., & Elkin, P.
- Affiliations
- United States Department of Veterans Affairs (US)
- University at Buffalo, State University of New York (US)
- Corresponding author
- Elkin, Peter
- Venue
- Life
- Publisher
- MDPI AG
- Published
- 2024-05-21
- Volume
- 14
- Issue
- 6
- Pages
- 652
- ISSN
- 2075-1729 (ISSN-L)
- PMC
- PMC11204917
- OpenAlex
- W4398169659
- Cited by
- 76
- References
- 95
- Citations by year
- 2026: 29 · 2025: 43 · 2024: 4
- FWCI
- 31.39
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Life (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- NIH NLM (T15LM012495, R25LM014213, NIAAA R21AA026954, R33AA0226954, NCATS UL1TR001412)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- algorithmic bias, sociodemographic bias, health care disparities, large language models, training data bias, gender bias, racial and ethnic bias, socioeconomic status, insurance status, implicit bias, sampling bias, anchoring effect, clinical decision-making, knowledge distillation, retrieval-augmented generation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Sociodemographic Biases in Machine Learning Algorithms: A Biomedical Informatics Perspective. https://app.cochat.ai/papers/10.3390/life14060652
- The paper
- Franklin, G., Stephens, R., Piracha, M., Tiosano, S., Lehouillier, F., Koppel, R., & Elkin, P. (2024). The Sociodemographic Biases in Machine Learning Algorithms: A Biomedical Informatics Perspective. Life. https://doi.org/10.3390/life14060652
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
![Figure 1 A taxonomy of sociodemographic bias in the clinical machine learning pipeline [13,14,15,16,17,18,19,20].](https://figures.cochat.ai/paper-figures/48764eb2a07844ccaefd605135a6e2d0/the-sociodemographic-biases-in-machine-learning-algorithms-a-biomedical-figure-1.jpg)