CoChat

The DOI matches its Crossref record; 96 of the 96 references it cites were found in a registry or an index.

The Sociodemographic Biases in Machine Learning Algorithms: A Biomedical Informatics Perspective

Open access

Abstract

Artificial intelligence models represented in machine learning algorithms are promising tools for risk assessment used to guide clinical and other health care decisions. Machine learning algorithms, however, may house biases that propagate stereotypes, inequities, and discrimination that contribute to socioeconomic health care disparities. The biases include those related to some sociodemographic characteristics such as race, ethnicity, gender, age, insurance, and socioeconomic status from the use of erroneous electronic health record data. Additionally, there is concern that training data and algorithmic biases in large language models pose potential drawbacks. These biases affect the lives and livelihoods of a significant percentage of the population in the United States and globally. The social and economic consequences of the associated backlash cannot be underestimated. Here, we outline some of the sociodemographic, training data, and algorithmic biases that undermine sound health care risk assessment and medical decision-making that should be addressed in the health care system. We present a perspective and overview of these biases by gender, race, ethnicity, age, historically marginalized communities, algorithmic bias, biased evaluations, implicit bias, selection/sampling bias, socioeconomic status biases, biased data distributions, cultural biases and insurance status bias, conformation bias, information bias and anchoring biases and make recommendations to improve large language model training data, including de-biasing techniques such as counterfactual role-reversed sentences during knowledge distillation, fine-tuning, prefix attachment at training time, the use of toxicity classifiers, retrieval augmented generation and algorithmic modification to mitigate the biases moving forward.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/life14060652

Cites

96 of 96 references verified0 citation errors · 0 not checkable

Statistics

Figures

Figure 1 A taxonomy of sociodemographic bias in the clinical machine learning pipeline [13,14,15,16,17,18,19,20].
Figure 1 A taxonomy of sociodemographic bias in the clinical machine learning pipeline [13,14,15,16,17,18,19,20].CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 96 references

Research works printed but not linked: 2

  1. Hellström, T., Dignum, V., and Bensch, S. (2020). Bias in Machine Learning—What Is It Good For?. arXiv.
  2. Liyanage, 2023, Ethical considerations and potential risks in the deployment of large Language Models in diverse societal contexts, J. Comput. Soc. Dyn., 8, 15

Cited by

See all 83

Metadata

Authors
Franklin, G., Stephens, R., Piracha, M., Tiosano, S., Lehouillier, F., Koppel, R., & Elkin, P.
Affiliations
  • United States Department of Veterans Affairs (US)
  • University at Buffalo, State University of New York (US)
Corresponding author
Elkin, Peter
Venue
Life
Publisher
MDPI AG
Published
2024-05-21
Volume
14
Issue
6
Pages
652
ISSN
2075-1729 (ISSN-L)
OpenAlex
W4398169659
Cited by
76
References
95
Citations by year
2026: 29 · 2025: 43 · 2024: 4
FWCI
31.39
Type
journal article
Language
en
Open access
Locations
  • Life (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
NIH NLM (T15LM012495, R25LM014213, NIAAA R21AA026954, R33AA0226954, NCATS UL1TR001412)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
algorithmic bias, sociodemographic bias, health care disparities, large language models, training data bias, gender bias, racial and ethnic bias, socioeconomic status, insurance status, implicit bias, sampling bias, anchoring effect, clinical decision-making, knowledge distillation, retrieval-augmented generation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The Sociodemographic Biases in Machine Learning Algorithms: A Biomedical Informatics Perspective. https://app.cochat.ai/papers/10.3390/life14060652
The paper
Franklin, G., Stephens, R., Piracha, M., Tiosano, S., Lehouillier, F., Koppel, R., & Elkin, P. (2024). The Sociodemographic Biases in Machine Learning Algorithms: A Biomedical Informatics Perspective. Life. https://doi.org/10.3390/life14060652

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.