The DOI matches its Crossref record; 17 of the 17 references it cites were found in a registry or an index; 1 of the 29 sources its claims cite were checked for support.
Bias in AI-based models for medical applications: challenges and mitigation strategies
Open accessAbstract
Artificial intelligence systems are increasingly being applied to healthcare. In surgery, AI applications hold promise as tools to predict surgical outcomes, assess technical skills, or guide surgeons intraoperatively via computer vision. On the other hand, AI systems can also suffer from bias, compounding existing inequities in socioeconomic status, race, ethnicity, religion, gender, disability, or sexual orientation. Bias particularly impacts disadvantaged populations, which can be subject to algorithmic predictions that are less accurate or underestimate the need for care. Thus, strategies for detecting and mitigating bias are pivotal for creating AI technology that is generalizable and fair. Here, we discuss a recent study that developed a new strategy to mitigate bias in surgical AI systems.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This commentary discusses bias in AI-based medical applications, focusing on surgical AI systems (SAIS) that assess surgeon skill from robotic surgery videos. It highlights the TWIX add-on strategy developed by Kiyasseh et al., which mitigated SAIS bias, and surveys future challenges and regulatory efforts around bias detection and mitigation.
Key findings
- Kiyasseh et al. deployed SAIS on videos of robotic surgeries from three hospitals to assess surgeon skill (e.g., needle handling, needle driving); the model reliably assessed performance but exhibited bias, showing underskilling (erroneously downgrading performance) and overskilling (erroneously upgrading performance) at different rates across surgeon sub-cohorts, measured via negative and positive predictive values respectively.
- The TWIX add-on taught SAIS to predict the importance of video clips used for skill assessment, hypothesizing that bias arose from the system latching onto unreliable frames; TWIX mitigated SAIS bias, improving performance both for disadvantaged surgeon sub-cohorts and for surgical skill assessments overall.
- Bias is generally defined as a difference in performance between subgroups for a predictive task, and can arise across data collection/preparation, model development, evaluation, and clinical deployment.
- The paper notes open questions: what level of bias is acceptable, whether models should be more generalizable (larger, more diverse datasets) or more localized, and the risk that continuously learning/adaptive models may increase or add new bias if new data are biased.
Methods
- This is a commentary/perspective article, not a primary study; it discusses a recent study (Kiyasseh et al.) that developed TWIX to mitigate bias in surgical AI systems.
- It reviews bias mitigation categories (pre-processing via sampling, in-processing via mathematical approaches, post-processing, and "keeping the human in the loop") and regulatory developments, including the STANDING Together initiative (launched September 2022) and the FDA's AI/ML-Based SaMD Action Plan (January 2021) and draft guidance on predetermined change control plans.
- No primary data, sample, or statistical analysis are reported by this paper.
Limitations
- The paper does not state formal limitations for itself; it is a commentary without primary data or quantitative results of its own.
- Evident from the text: all empirical claims about SAIS and TWIX are attributed to the Kiyasseh et al. study rather than independently tested here, and the discussion of acceptable bias thresholds, generalizability versus localization, and adaptive-model risks is framed as unresolved questions rather than settled findings.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
19 Not checked yet · 3 Own interpretation · 1 Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Bias can be generated across AI model development steps, including data collection/preparation, model development, model evaluation, and deployment in clinical settings9.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > Strategies to mitigate bias
What the sources say
- Supportedcites [9] Vokinger, K. N., Feuerriegel, S. & Kesselheim, A. S. Mitigating bias in machine learning for medicine. Commun. Med. 1, 25 (2021). source
The Conclusion states that bias can occur across the listed steps, from data collection and preparation through post-authorization deployment in clinical practice. This directly supports the citing sentence. (LLM, conf=0.95)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
In the paper
“Further, to meet regulatory challenges that come with continuously adopting AI models, the FDA recently released a draft guidance to develop a less burdensome regulatory approach supporting the iterative improvement of, e.g., AI models while continuing to assure their safety and effectiveness20.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > A look into the future—challenges with continuously learning AI models
What the sources say
- Not checked yetcites [20] FDA. Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence/Machine Learning (AI/ML)-Enabled Device Software Functions. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/marketing-submission-recommendations-predetermined-change-control-plan-artificial (2023).
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Additionally, there are likely underlying social inequalities in healthcare access and expenditures that impact how a model might be trained to predict risk6,10.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > Strategies to mitigate bias
What the sources say
- Not checked yetcites [6] Seyyed-Kalantari, L., Zhang, H., McDermott, M. B. A., Chen, I. Y. & Ghassemi, M. Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nat. Med. 27, 2176–2182 (2021). source
- Not checked yetcites [10] Panch, T., Mattie, H. & Atun, R. Artificial intelligence and algorithmic bias: implications for health systems. J. Glob. Health 9, 010318 (2019). source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“On the other hand, AI systems can also suffer from bias, compounding existing inequities in socioeconomic status, race, ethnicity, religion, gender, disability, or sexual orientation.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > Abstract
What the sources say
- Not checked yetcites [6] Seyyed-Kalantari, L., Zhang, H., McDermott, M. B. A., Chen, I. Y. & Ghassemi, M. Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nat. Med. 27, 2176–2182 (2021). source
- Not checked yetcites [7] Yang, J., Soltan, A. A. S., Eyre, D. W., Yang, Y. & Clifton, D. A. An adversarial training framework for mitigating algorithmic biases in clinical machine learning. NPJ Digital Med. 6, 55 (2023). source
- Not checked yetcites [8] Obermeyer, Z., Powers, B., Vogeli, C. & Mullainathan, S. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 447–453 (2019). source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: AI systems are increasingly being applied to healthcare.
- Rests on an assumption: The issue of bias being exhibited, perpetuated, or even amplified by AI algorithms is an increasing concern within healthcare.
None of the sources cited for this claim have been checked yet.
In the paper
“Bias particularly impacts disadvantaged populations, which can be subject to algorithmic predictions that are less accurate or underestimate the need for care.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > Abstract
What the sources say
- Not checked yetcites [6] Seyyed-Kalantari, L., Zhang, H., McDermott, M. B. A., Chen, I. Y. & Ghassemi, M. Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nat. Med. 27, 2176–2182 (2021). source
- Not checked yetcites [7] Yang, J., Soltan, A. A. S., Eyre, D. W., Yang, Y. & Clifton, D. A. An adversarial training framework for mitigating algorithmic biases in clinical machine learning. NPJ Digital Med. 6, 55 (2023). source
- Not checked yetcites [8] Obermeyer, Z., Powers, B., Vogeli, C. & Mullainathan, S. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 447–453 (2019). source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: AI systems are increasingly being applied to healthcare.
In the paper
“TWIX is an add-on application that taught the SAIS model to add a prediction of the importance of video clips that was used to assess surgical skill.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > Strategies to mitigate bias
What the sources say
- Not checked yetcites [4] Kiyasseh, D. et al. Human visual explanations mitigate bias in AI-based assessment of surgeon skills. NPJ Digital Med. 6, 54 (2023). source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Currently, there is significant heterogeneity in terms of the datasets AI algorithms are trained with within algorithm types themselves13,14.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > A look into the future—challenges with continuously learning AI models
What the sources say
- Not checked yetcites [13] Gubatan, J. et al. Artificial intelligence applications in inflammatory bowel disease: emerging technologies and future directions. World J. Gastroenterol. 27, 1920–1935 (2021). source
- Not checked yetcites [14] Moglia, A., Georgiou, K., Georgiou, E., Satava, R. M. & Cuschieri, A. A systematic review on artificial intelligence in robot-assisted surgery. Int. J. Surg. 95, 106151 (2021). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Kiyasseh et al.4 highlight this potential application in their work deploying surgical AI systems (SAIS) on videos of robotic surgeries from three hospitals.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > Bias in medical AI algorithms
What the sources say
- Not checked yetcites [4] Kiyasseh, D. et al. Human visual explanations mitigate bias in AI-based assessment of surgeon skills. NPJ Digital Med. 6, 54 (2023). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“With their SAIS model, Kiyasseh et al.4 developed a strategy called TWIX to mitigate bias.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > Strategies to mitigate bias
What the sources say
- Not checked yetcites [4] Kiyasseh, D. et al. Human visual explanations mitigate bias in AI-based assessment of surgeon skills. NPJ Digital Med. 6, 54 (2023). source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“The SAIS model showed an underskilling or overskilling bias at different rates across surgeon sub-cohort.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > Bias in medical AI algorithms
What the sources say
- Not checked yetcites [4] Kiyasseh, D. et al. Human visual explanations mitigate bias in AI-based assessment of surgeon skills. NPJ Digital Med. 6, 54 (2023). source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 6
- It may state two separate claims as one. 6
In the paper
“Similar cases have already been seen in healthcare systems8.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > Strategies to mitigate bias
What the sources say
- Not checked yetcites [8] Obermeyer, Z., Powers, B., Vogeli, C. & Mullainathan, S. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 447–453 (2019). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In applying this AI model, Kiyasseh et al.4 found that it could reliably assess surgical performance but exhibited bias.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > Bias in medical AI algorithms
What the sources say
- Not checked yetcites [4] Kiyasseh, D. et al. Human visual explanations mitigate bias in AI-based assessment of surgeon skills. NPJ Digital Med. 6, 54 (2023). source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Regardless of the cause, the impact of an algorithm disproportionately assigning false negatives would include fewer follow-up scans, and potentially more undiagnosed/untreated cancer cases, worsening health inequity for an already disadvantaged population.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > Strategies to mitigate bias
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This accomplishment is beneficial not only for this particular use case but also implies that this type of bias mitigation strategy could be used to continue to improve AI applications in the future.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > Strategies to mitigate bias
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
In the paper
“The STANDING Together initiative (standards for data diversity, inclusivity, and generalizability), launched in September 2022, aims to develop recommendations for the composition (who is represented) and reporting (how they are represented) of datasets underpinning medical AI systems18.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > A look into the future—challenges with continuously learning AI models
What the sources say
- Not checked yetcites [18] Ganapathi, S. et al. Tackling bias in AI health datasets through the STANDING Together initiative. Nat. Med. 28, 2232–2233 (2022). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Bias mitigation strategies may involve interventions such as pre-processing data through sampling before a model is built, in-processing by implementing mathematical approaches to incentivize a model to learn balanced predictions, and post-processing11.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > Strategies to mitigate bias
What the sources say
- Not checked yetcites [11] Xu, J. et al. Algorithmic fairness in computational medicine. EBioMedicine 84, 104250 (2022). source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“In surgery, AI applications hold promise as tools to predict surgical outcomes, assess technical skills, or guide surgeons intraoperatively via computer vision.”
- Also stated in: § Bias in medical AI algorithms · plain
“Specifically, within the field of surgery, AI applications hold promise as tools to predict surgical outcomes1, aid surgeons via computer vision for intraoperative surgical navigation2, and even as algorithms to assess technical skills and surgical performance1,3–5.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > Abstract
What the sources say
- Not checked yetcites [1] Ma, R. et al. Surgical gestures as a method to quantify surgical performance and predict patient outcomes. NPJ Digital Med. 5, 187 (2022). source
- Not checked yetcites [2] Chadebecq, F., Vasconcelos, F., Mazomenos, E. & Stoyanov, D. Computer vision in the surgical operating room. Visc. Med. 36, 456–462 (2020). source
- Not checked yetcites [3] Kiyasseh, D. et al. A multi-institutional study using artificial intelligence to provide reliable and fair feedback to surgeons. Commun. Med. 3, 42 (2023). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: AI systems are increasingly being applied to healthcare.
- Also stated in: § Bias in medical AI algorithms · plain
In the paper
“As part of the FDA Action Plan, the FDA intends to support the piloting of real-world performance monitoring19, allowing for the detection of bias after deployment.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > A look into the future—challenges with continuously learning AI models
What the sources say
- Not checked yetcites [19] FDA. Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan. www.fda.gov/media/145022/download (2021).
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Kiyasseh et al.4 found that TWIX mitigated SAIS model bias, improving model performance both for the disadvantaged surgeon sub-cohorts and for surgical skill assessments overall.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > Strategies to mitigate bias
What the sources say
- Not checked yetcites [4] Kiyasseh, D. et al. Human visual explanations mitigate bias in AI-based assessment of surgeon skills. NPJ Digital Med. 6, 54 (2023). source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Further, the FDA has recognized challenges due to bias in AI and ML algorithms and released an action plan (“Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan”) in January 20219,19, emphasizing the importance of identifying and mitigating bias in AI systems9.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > A look into the future—challenges with continuously learning AI models
What the sources say
- Not checked yetcites [9] Vokinger, K. N., Feuerriegel, S. & Kesselheim, A. S. Mitigating bias in machine learning for medicine. Commun. Med. 1, 25 (2021). source
- Not checked yetcites [19] FDA. Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan. www.fda.gov/media/145022/download (2021).
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Previous groups suggested that any performance discrepancy is indicative of algorithmic bias, but expecting completely bias-free systems before implementation is unrealistic12.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > A look into the future—challenges with continuously learning AI models
What the sources say
- Not checked yetcites [12] Townson, S. Manage AI Bias Instead of Trying to Eliminate It. https://sloanreview.mit.edu/article/manage-ai-bias-instead-of-trying-to-eliminate-it/2023 (MIT Sloan Management Review, 2023).
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“TWIX requiring model predictions of video clip importance served a similar role to human assessors explaining the rationale for assessments.”
§ Bias in AI-based models for medical applications: challenges and mitigation strategies > Strategies to mitigate bias
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper relies on this without support.
AI systems are increasingly being applied to healthcare.
“Artificial intelligence systems are increasingly being applied to healthcare.”
“Artificial intelligence (AI) technology is increasingly applied to healthcare, from AI-augmented clinical research to algorithms for image analysis or disease prediction.”
The paper relies on this without support.
The integration of AI into medical technology and healthcare systems is only going to increase in the coming years.
“The integration of AI into medical technology and healthcare systems is only going to increase in the coming years.”
The paper relies on this without support.
Performance discrepancy may differ based on the data and population an AI algorithm is trained on and then subsequently applied to.
“Performance discrepancy may also differ based on the data and population an AI algorithm is trained on and then subsequently applied to.”
The paper relies on this without support.
The issue of bias being exhibited, perpetuated, or even amplified by AI algorithms is an increasing concern within healthcare.
“The issue of bias being exhibited, perpetuated, or even amplified by AI algorithms is an increasing concern within healthcare.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Studies have shown that racial and ethnic biases impact the use of AI/ML in medical imaging, diagnosis, and prognosis due to biases in healthcare databases [78, 154].”
From The ethics of using artificial intelligence in scientific research: new guidance needed for a new tool (2025) · cites this paper as [154]
In the citing paper
“Nevertheless, scientists can (and should) take steps to understand sources of bias and control them, especially those that can lead to discrimination, stigmatization, harm, or injustice [89, 154, 188].”
From The ethics of using artificial intelligence in scientific research: new guidance needed for a new tool (2025) · cites this paper as [154]
In the citing paper
“These systems can amplify and perpetuate biases present in the training data, leading to skewed predictions and potentially harmful implications for patient care.6,7”
From Balancing innovation and integrity: the role of AI in research and scientific writing (2023) · cites this paper as [7]
In the citing paper
“This is concerning, as biases in AI models can emerge during various stages, from data collection to model evaluation.7”
From Balancing innovation and integrity: the role of AI in research and scientific writing (2023) · cites this paper as [7]
Citations
Cites
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1×
“Similar cases have already been seen in healthcare systems 8 .”
Benjamens, S., Dhunnoo, P., & Meskó, B. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00324-0 · Cited by 1,215 · cited 1×
“An AI algorithm can either be locked — once the algorithm is trained, the model provides the same result when the same input is applied — or adaptive 16 .”
Seyyed-Kalantari, L., Zhang, H., McDermott, M. B. A., Chen, I. Y., & Ghassemi, M. (2021) · Nature Medicine · doi:10.1038/s41591-021-01595-0 · Cited by 928 · cited 2×
“Bias is usually de fi ned as a difference in performance between sub-groups for a predictive task 6,7 .”
Artificial intelligence and algorithmic bias: implications for health systems
Panch, T., Mattie, H., & Atun, R. (2019) · Journal of Global Health · doi:10.7189/jogh.09.020318 · Cited by 533 · cited 1×
“Additionally, there are likely underlying social inequalities in healthcare access and expenditures that impact how a model might be trained to predict risk 6,10 .”
Mitigating bias in machine learning for medicine
Vokinger, K. N., Feuerriegel, S., & Kesselheim, A. S. (2021) · Communications Medicine · doi:10.1038/s43856-021-00028-w · Cited by 269 · cited 2×
“…challenges due to bias in AI and ML algorithms and released an action plan ( “ Arti fi cial Intelligence/Machine Learning (AI/ML)- Based Software as a Medical Device (SaMD) Action Plan ” ) in January 2021 9,19 , emphasizing the importance of identifying and mitigating bias in AI systems 9 .”
A systematic review on artificial intelligence in robot-assisted surgery
Moglia, A., Georgiou, K., Georgiou, E., Satava, R. M., & Cuschieri, A. (2021) · International Journal of Surgery · doi:10.1016/j.ijsu.2021.106151 · Cited by 236
Latent bias and the implementation of artificial intelligence in medicine
DeCamp, M., & Lindvall, C. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa094 · Cited by 198 · cited 1×
“However, continuous learning also possesses the risk of increasing or adding new bias if the new data are biased 17 .”
Algorithmic fairness in computational medicine
Xu, J., Xiao, Y., Wang, W. H., Ning, Y., Shenkman, E. A., Bian, J., & Wang, F. (2022) · eBioMedicine · doi:10.1016/j.ebiom.2022.104250 · Cited by 170 · cited 1×
“Bias mitigation strategies may involve interventions such as pre-processing data through sampling before a model is built, in-processing by implementing mathematical approaches to incentivize a model to learn balanced predictions, and post-processing 11 .”
An adversarial training framework for mitigating algorithmic biases in clinical machine learning
Yang, J., Soltan, A. A. S., Eyre, D. W., Yang, Y., & Clifton, D. A. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00805-y · Cited by 159 · cited 1×
“Bias is usually de fi ned as a difference in performance between sub-groups for a predictive task 6,7 .”
A vision transformer for decoding surgeon activity from surgical videos
Kiyasseh, D., Ma, R., Haque, T. F., Miles, B. J., Wagner, C., Donoho, D. A., Anandkumar, A., & Hung, A. J. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01010-8 · Cited by 151
Cited by
Sorted by citations
Ethical and Bias Considerations in Artificial Intelligence/Machine Learning
Hanna, M. G., Pantanowitz, L., Jackson, B., Palmer, O., Visweswaran, S., Pantanowitz, J., Deebajah, M., & Rashidi, H. H. (2025) · Modern Pathology · doi:10.1016/j.modpat.2024.100686 · Cited by 535
A Scoping Review of Reporting Gaps in FDA-Approved AI Medical Devices
Muralidharan, V., Adewale, B. A., Huang, C. J., Nta, M. T., Ademiju, P. O., Pathmarajah, P., Hang, M. K., Adesanya, O., Abdullateef, R. O., Babatunde, A. O., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01270-x · Cited by 201
“However, despite these efforts, our study highlights reporting inconsistencies that may continue to propagate racial health disparities 21 .”
BackgroundResnik, D. B., & Hosseini, M. (2025) · AI and Ethics · doi:10.1007/s43681-024-00493-8 · Cited by 228
“Nevertheless, scientists can (and should) take steps to understand sources of bias and control them, especially those that can lead to discrimination, stigmatization, harm, or injustice [89, 154, 188].”
Fountzilas, E., Pearce, T. E., Baysal, M. A., Chakraborty, A., & Tsimberidou, A. M. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01471-y · Cited by 233
Saeidnia, H. R., Fotami, S. G. H., Lund, B. D., & Ghiasi, N. (2024) · Social Sciences · doi:10.3390/socsci13070381 · Cited by 227
“Practical strategies for mitigating bias in AI algorithms include diverse data collection, algorithmic audits, and ongoing evaluation to detect and correct biases that may arise during deployment (F. Chen et al. 2024; Ferrara 2023; Mensah 2023; Mittermaier et al. 2023).”
Influential
Metadata
- Authors
- Mittermaier, M., Raza, M. M., & Kvedar, J. C.
- Affiliations
- Humboldt-Universität zu Berlin (DE)
- Harvard University (US)
- Corresponding author
- Mittermaier, Mirja
- Venue
- npj Digital Medicine
- Published
- 2023-06-14
- Volume
- 6
- Issue
- 1
- Pages
- 113-113
- ISSN
- 2398-6352 (ISSN-L)
- PMC
- PMC10264403
- OpenAlex
- W4380538374
- Semantic Scholar
- 3e2fd4f099046ad3b61d8cad107d5883dc52370a
- Cited by
- 372
- References
- 17
- Citations by year
- 2026: 99 · 2025: 172 · 2024: 87 · 2023: 14
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- npj Digital Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Deutsche Forschungsgemeinschaft
- Berlin Institute of Health
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- algorithmic bias, bias mitigation, health care equity, algorithmic fairness, surgical outcome prediction, generalizability, disadvantaged groups
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
