The DOI matches its Crossref record; 13 of the 14 references it cites were found in a registry or an index; 10 of the 20 sources its claims cite were checked for support.
Mitigating bias in machine learning for medicine
Open accessAbstract
Several sources of bias can affect the performance of machine learning systems used in medicine and potentially impact clinical care. Here, we discuss solutions to mitigate bias across the different development steps of machine learning-based systems for medical applications. Vokinger et al. discuss potential sources of bias in machine learning systems used in medicine. The authors propose solutions to mitigate bias across the different stages of model development, from data collection and preparation to model evaluation and application.
Overview
CoChat Summary
TL;DR
This commentary argues that bias in machine-learning systems for medicine can be introduced at each development stage—data collection and preparation, model development, model evaluation, and post-authorization deployment—and proposes mitigation strategies for each. It calls for these strategies to become best practice to prevent health-care inequality and ensure safe function for all patients.
Key findings
No original data are reported; the paper synthesizes proposed solutions across four development steps.
- Sampling bias in training data can be replicated clinically: a system trained on melanoma images from people with white skin "might misinterpret images from patients with a darker skin tone, and might fail to diagnose melanoma," which matters because melanoma causes most skin-cancer deaths and early diagnosis is critical for cure.
- The paper cites prior evidence of racial bias in a commercial prediction algorithm: Black patients assigned the same risk level were sicker than white patients, because the algorithm used health costs as a proxy for health needs and less money is spent on Black patients with equal need.
- Proposed mitigations include transparency about training-data demographics and baseline characteristics, compiling datasets "as diverse and large as possible," monitoring error rates across cohorts, reporting checklists such as PROBAST, mathematical de-biasing (adversarial de-biasing, oversampling), continual learning, interpretability/explainability tools (SHAP, LIME), and post-authorization monitoring for domain shift and feedback loops.
- The author recommends that for high-risk ML-based systems in medicine, models "might be better" limited to those that are interpretable, given that explainability can be inaccurate or non-meaningful.
Methods
This is a narrative commentary/perspective, not an empirical study; no dataset, sample, or statistical analysis is reported. The argument is organized around a four-step framework of ML system development (data collection and preparation, model development, model evaluation, deployment/post-authorization), with proposed solutions presented per step and summarized in Fig. 1. Evidence is drawn from cited prior literature and FDA policy documents (e.g., the January 2021 FDA Action Plan).
Limitations
The paper states no formal limitations section. Limitations evident in the text: the author notes that de-biasing techniques "have only recently emerged in computer science and more research is still needed to demonstrate proof-of-principle and show that de-biasing reliably achieves its intended purpose," and that ML explainability "can be inaccurate or non-meaningful due to the underlying mathematical assumptions." The recommendations are prescriptive proposals rather than validated interventions, and no quantitative evaluation of their effectiveness is provided.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures

Verification
Claims the paper makes
8 Valid · 0 Not valid · 5 Not checked · 14 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These errors can be introduced across the various stages of development.”
§ Mitigating bias in machine learning for medicine > Abstract
What the sources say
- Not checked yetcites [4] Parikh, R. B., Teeple, S. & Navathe, A. S. Addressing bias in artificial intelligence in health care. J. Am. Med. Assoc. 322, 2377 (2019). source
- Partly supportedcites [5] Obermeyer, Z., Powers, B., Vogeli, C. & Mullainathan, S. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 447–453 (2019). sourceExperimental — accuracy not yet measured
Our copy supports only in part: the introduction (p. 2) says algorithms 'may reproduce racial and gender disparities via the people building them or through the data used to train them', but that general concern is credited to references 1-3 and is not a stage-by-stage account. The paper's own argument locates the bias in a single development decision - label choice/problem formulation - stating that 'the choice of label is perhaps the single most important decision made in the development of a prediction algorithm' (p. 6) and that 'labels are the key determinant of both predictive quality and predictive bias' (Discussion, p. 8). It explicitly denies that the modelling/fitting stage is the culprit: 'Changing the procedures by which we fit algorithms ... is not required. Rather, we must change the data we feed the algorithm - specifically, the labels we give it' (Discussion, p. 8). So attributing to this source the claim that errors arise 'across the various stages of development' overstates it. (LLM, conf=0.68)
- Not checked yetcites [6] FDA. Artificial intelligence/machine learning (AI/ML)-based software as a medical device (SaMD) action plan. https://www.fda.gov/media/145022/download (2021).
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Every source the paper cites for this claim was checked and supports it.
In the paper
“This bias occurred because the algorithm used health costs as a proxy for health needs.”
§ Mitigating bias in machine learning for medicine > Abstract
What the sources say
- Supportedcites [5] Obermeyer, Z., Powers, B., Vogeli, C. & Mullainathan, S. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 447–453 (2019). source
The abstract (p. 2) states directly that 'The bias arises because the algorithm predicts health care costs rather than illness,' and that cost 'appear[ed] to be an effective proxy for health.' The Mechanism of bias section (pp. 4-5) confirms the label was total medical expenditures, so 'the algorithm's prediction on health needs is, in fact, a prediction on health costs,' and the Discussion (p. 8) reiterates that cost was used as a proxy for health needs. The claim's causal attribution is fully stated in our copy. (LLM, conf=0.95)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Beyond creating diverse datasets for model development, there are mathematical approaches for de-biasing that mitigate the risk of bias at this step, such as adversarial de-biasing9 or oversampling10.”
§ Mitigating bias in machine learning for medicine > Model development
What the sources say
- Supportedcites [9] Zhang, B. H., Lemoine, B. & Mitchell, M. Mitigating unwanted biases with adversarial learning. in Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. (ed. Furman, Jason, Marchant, Gary, Price, Huw, Rossi, Francesca) 335–340 (ACM, 2018). source
The citation is attached to 'adversarial de-biasing' as a mathematical approach that mitigates bias at the model-development/training step, and our copy of Zhang et al. (2018) is exactly that method: Section 3 'ADVERSARIAL DEBIASING' (p. n/a) describes jointly learning a predictor and an adversary whose gradient is folded into the predictor's weight update rule to reduce information about the protected variable Z transmitted through Y-hat, and the Abstract states the framework 'mitigat[es] such biases' with results on analogy completion and the UCI Adult dataset. The Additional information section adds that they 'demonstrate a general and powerful method for training unbiased machine learning models' yielding models 'demonstrably less biased than the original one' while still performing well. The 'oversampling' element carries a separate citation (ref 10) and is not attributed to this source; our copy does not discuss oversampling, which does not affect this judgement. (LLM, conf=0.90)
- Supportedcites [10] Kamiran, F. & Calders, T. Data preprocessing techniques for classification without discrimination. Knowl. Inf. Syst. 33, 1–33 (2012). source
Sect. 5 'Solutions: data preprocessing techniques' (p. 14) and Sect. 5.3 'Sampling' (p. 18) present an oversampling/undersampling (plus Uniform and Preferential Sampling, Intro p. 2) preprocessing approach that removes discrimination from the training dataset before a classifier is learned, i.e. at the model-development step. Sect. 6.6 (p. 28) and the Conclusion (p. 30) report that these preprocessing methods consistently reduce discrimination versus baselines, with an accuracy trade-off, supporting 'mitigate the risk of bias'. The adversarial de-biasing part of the citing sentence carries a separate citation (9) and is not attributed to this source, so it is not judged here. (LLM, conf=0.85)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Black patients assigned the same level of risk by the algorithm were sicker than white patients.”
§ Mitigating bias in machine learning for medicine > Abstract
What the sources say
- Supportedcites [5] Obermeyer, Z., Powers, B., Vogeli, C. & Mullainathan, S. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 447–453 (2019). source
The abstract (p. 2) states that 'At a given risk score, Black patients are considerably sicker than White patients, as evidenced by signs of uncontrolled illnesses.' The Results section 'Health disparities conditional on risk score' (p. 3) quantifies this: at the same level of algorithm-predicted risk, Blacks have significantly more illness burden than Whites (e.g., 26.3% more chronic illnesses at the 97th percentile, 4.8 versus 3.8 conditions, P < 0.001), and biomarker evidence at p. 4 shows the same pattern across disease severity measures. The paraphrase matches the source exactly, including the conditioning on equal risk score. (LLM, conf=0.95)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Systematic errors might also be reduced through continual learning11, whereby an ML-based system is continuously updated through new data while retaining previously learned knowledge12.”
§ Mitigating bias in machine learning for medicine > Model development
What the sources say
- Supportedcites [11] Vokinger, K. N., Feuerriegel, S. & Kesselheim, A. S. Continual learning in medical devices: FDA’s action plan and beyond. Lancet Digit. Health 3, e337–e338 (2021). source
The paper says continual learning could remove errors and improve accuracy (p. 2), and describes identifying and addressing specific errors through updates (p. 3). It also defines continual learning as updating a model with new data while retaining previously learned knowledge (p. 2). (LLM, conf=0.88)
- Supportedcites [12] Lee, C. S. & Lee, A. Y. Clinical applications of continual learning machine learning. Lancet Digital Health 2, e279–e281 (2020). source
The citation to this source is attached to the definition of continual learning, not the preceding claim about reducing systematic errors. In “Applications of continual learning machine learning in clinical practice,” the paper defines it as models learning from increasing amounts of data while retaining previously learned knowledge. (LLM, conf=0.95)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
13 more claims · 3 with their cited sources checked
Create a free account to see all 27 claims and to verify the 10 references still open.
- It may state two separate claims as one. 6
- Probably not a substantial claim. 4
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“To reduce the risk of such biases, mathematical de-biasing approaches such as adversarial de-biasing or oversampling have been introduced [26].”
From Bias in artificial intelligence algorithms and recommendations for mitigation (2023) · cites this paper as [26]
The cited source says what the claim attributes to it.
In the citing paper
“With such techniques, the model is forced to account for underrepresented groups to achieve better performance.”
From Bias in artificial intelligence algorithms and recommendations for mitigation (2023) · cites this paper as [26]
The cited source says what the claim attributes to it.
In the citing paper
“Model interpretability is another important step in both validation and bias mitigation, offering insights into how decisions are generated (e.g., which features are most involved) or explaining the behavior of “black box” models (e.g., deep learning models with millions of parameters) [11].”
From Bias in medical AI: Implications for clinical decision-making (2024) · cites this paper as [11]
The cited source says what the claim attributes to it.
In the citing paper
“Bias can be generated across AI model development steps, including data collection/preparation, model development, model evaluation, and deployment in clinical settings9.”
From Bias in AI-based models for medical applications: challenges and mitigation strategies (2023) · cites this paper as [9]
The cited source says what the claim attributes to it.
In the citing paper
“Bias and unequal performance across demographic groups remain significant concerns [21,22].”
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
A Unified Approach to Interpreting Model Predictions
Lundberg, S., & Lee, S.-I. (2017) · arXiv · doi:10.48550/arxiv.1705.07874 · Cited by 23,707
"Why Should I Trust You?": Explaining the Predictions of Any Classifier
Ribeiro, M. T., Singh, S., & Guestrin, C. (2016) · Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/2939672.2939778 · Cited by 17,555
Rudin, C. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0048-x · Cited by 10,561
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137
PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group† (2019) · Annals of Internal Medicine · doi:10.7326/m18-1376 · Cited by 2,984
Mitigating Unwanted Biases with Adversarial Learning
Zhang, B. H., Lemoine, B., & Mitchell, M. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278779 · Cited by 1,387
Data preprocessing techniques for classification without discrimination
Kamiran, F., & Calders, T. (2012) · Knowledge and Information Systems · doi:10.1007/s10115-011-0463-8 · Cited by 1,305
Addressing bias in artificial intelligence in health care
Parikh, R. B., Teeple, S., & Navathe, A. S. (2019) · JAMA · doi:10.1001/jama.2019.18058 · Cited by 764
Muehlematter, U. J., Daniore, P., & Vokinger, K. N. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30292-2 · Cited by 665
Self-citationMachine Learning and Health Care Disparities in Dermatology
Adamson, A. S., & Smith, A. (2018) · JAMA Dermatology · doi:10.1001/jamadermatol.2018.2348 · Cited by 639
Cited by
Acosta, J. N., Falcone, G. J., Rajpurkar, P., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-022-01981-2 · Cited by 1,365
Lekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., Weicken, E., Asselbergs, F. W., Prior, F., Collins, G. S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081554 · Cited by 957
“31 35 To correct for any identified biases, mitigation measures should be tested, such as data resampling, bias free representations, and equalised odds postprocessing, 36-40 to verify their impact on both the tool’s fairness and the model’s accuracy.”
Bias in artificial intelligence algorithms and recommendations for mitigation
Nazer, L. H., Zatarah, R., Waldrip, S., Ke, J. X. C., Moukheiber, M., Khanna, A. K., Hicklen, R. S., Moukheiber, L., Moukheiber, D., Ma, H., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000278 · Cited by 621
Bias in medical AI: Implications for clinical decision-making
Cross, J. L., Choma, M. A., & Onofrey, J. A. (2024) · PLOS Digital Health · doi:10.1371/journal.pdig.0000651 · Cited by 607
Bias in AI-based models for medical applications: challenges and mitigation strategies
Mittermaier, M., Raza, M. M., & Kvedar, J. C. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00858-z · Cited by 372
“…challenges due to bias in AI and ML algorithms and released an action plan ( “ Arti fi cial Intelligence/Machine Learning (AI/ML)- Based Software as a Medical Device (SaMD) Action Plan ” ) in January 2021 9,19 , emphasizing the importance of identifying and mitigating bias in AI systems 9 .”
Metadata
- Authors
- Vokinger, K. N., Feuerriegel, S., & Kesselheim, A. S.
- Affiliations
- Brigham and Women's Hospital (US)
- ETH Zurich (CH)
- Corresponding author
- Vokinger, Kerstin N.
- Venue
- Communications Medicine
- Published
- 2021-08-23
- Volume
- 1
- Issue
- 1
- Pages
- 25-25
- ISSN
- 2730-664X (ISSN-L)
- PMC
- PMC7611652
- OpenAlex
- W3194979994
- Semantic Scholar
- 995781580ebeef01050ef234c14e46ba8aaca5d9
- Cited by
- 269
- References
- 14
- Citations by year
- 2026: 43 · 2025: 54 · 2024: 71 · 2023: 65 · 2022: 32 · 2021: 4
- FWCI
- 53.77
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Communications Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Repository for Publications and Research Data (ETH Zurich) (publishedVersion, cc-by)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, cc-by)
- Repository for Publications and Research Data (ETH Zurich) (submittedVersion, cc-by)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, cc-by)
- ETH Zürich Research Collection (cc-by)
- Zurich Open Repository and Archive (University of Zurich)
- Funders
- Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- algorithmic bias, bias mitigation, machine learning in medicine, clinical decision support, model evaluation, fairness in healthcare
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Mitigating bias in machine learning for medicine. https://app.cochat.ai/papers/10.1038/s43856-021-00028-w
- The paper
- Vokinger, K. N., Feuerriegel, S., & Kesselheim, A. S. (2021). Mitigating bias in machine learning for medicine. Communications Medicine. https://doi.org/10.1038/s43856-021-00028-w
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.