Bias in AI-based models for medical applications: challenges and mitigation strategies
Open accessAbstract
Artificial intelligence systems are increasingly being applied to healthcare. In surgery, AI applications hold promise as tools to predict surgical outcomes, assess technical skills, or guide surgeons intraoperatively via computer vision. On the other hand, AI systems can also suffer from bias, compounding existing inequities in socioeconomic status, race, ethnicity, religion, gender, disability, or sexual orientation. Bias particularly impacts disadvantaged populations, which can be subject to algorithmic predictions that are less accurate or underestimate the need for care. Thus, strategies for detecting and mitigating bias are pivotal for creating AI technology that is generalizable and fair. Here, we discuss a recent study that developed a new strategy to mitigate bias in surgical AI systems.
AI snapshot
Written from the paper’s full text.
TL;DR
This commentary discusses bias in AI-based medical applications, focusing on surgical AI systems (SAIS) that assess surgeon skill from robotic surgery videos. It highlights the TWIX add-on strategy developed by Kiyasseh et al., which mitigated SAIS bias, and surveys future challenges and regulatory efforts around bias detection and mitigation.
Key findings
- Kiyasseh et al. deployed SAIS on videos of robotic surgeries from three hospitals to assess surgeon skill (e.g., needle handling, needle driving); the model reliably assessed performance but exhibited bias, showing underskilling (erroneously downgrading performance) and overskilling (erroneously upgrading performance) at different rates across surgeon sub-cohorts, measured via negative and positive predictive values respectively.
- The TWIX add-on taught SAIS to predict the importance of video clips used for skill assessment, hypothesizing that bias arose from the system latching onto unreliable frames; TWIX mitigated SAIS bias, improving performance both for disadvantaged surgeon sub-cohorts and for surgical skill assessments overall.
- Bias is generally defined as a difference in performance between subgroups for a predictive task, and can arise across data collection/preparation, model development, evaluation, and clinical deployment.
- The paper notes open questions: what level of bias is acceptable, whether models should be more generalizable (larger, more diverse datasets) or more localized, and the risk that continuously learning/adaptive models may increase or add new bias if new data are biased.
Methods
- This is a commentary/perspective article, not a primary study; it discusses a recent study (Kiyasseh et al.) that developed TWIX to mitigate bias in surgical AI systems.
- It reviews bias mitigation categories (pre-processing via sampling, in-processing via mathematical approaches, post-processing, and "keeping the human in the loop") and regulatory developments, including the STANDING Together initiative (launched September 2022) and the FDA's AI/ML-Based SaMD Action Plan (January 2021) and draft guidance on predetermined change control plans.
- No primary data, sample, or statistical analysis are reported by this paper.
Limitations
- The paper does not state formal limitations for itself; it is a commentary without primary data or quantitative results of its own.
- Evident from the text: all empirical claims about SAIS and TWIX are attributed to the Kiyasseh et al. study rather than independently tested here, and the discussion of acceptable bias thresholds, generalizability versus localization, and adaptive-model risks is framed as unresolved questions rather than settled findings.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 29 cited sources checked · 0 supported · 0 flagged
- Premisecitation
The FDA recently released a draft guidance to develop a less burdensome regulatory approach supporting the iterative improvement of AI models while continuing to assure their safety and effectiveness.
“Further, to meet regulatory challenges that come with continuously adopting AI models, the FDA recently released a draft guidance to develop a less burdensome regulatory approach supporting the iterative improvement of, e.g., AI models while continuing to assure their safety and effectiveness20.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [20] FDA. Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence/Machine Learning (AI/ML)-Enabled Device Software Functions. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/marketing-submission-recommendations-predetermined-change-control-plan-artificial (2023).
Not checked in the free run.
- Premisecitation
There are likely underlying social inequalities in healthcare access and expenditures that impact how a model might be trained to predict risk.
“Additionally, there are likely underlying social inequalities in healthcare access and expenditures that impact how a model might be trained to predict risk6,10.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [6] Seyyed-Kalantari, L., Zhang, H., McDermott, M. B. A., Chen, I. Y. & Ghassemi, M. Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nat. Med. 27, 2176–2182 (2021). source
Not checked in the free run.
- cites [10] Panch, T., Mattie, H. & Atun, R. Artificial intelligence and algorithmic bias: implications for health systems. J. Glob. Health 9, 010318 (2019). source
Not checked in the free run.
- Premisecitation
AI systems can suffer from bias, compounding existing inequities in socioeconomic status, race, ethnicity, religion, gender, disability, or sexual orientation.
“On the other hand, AI systems can also suffer from bias, compounding existing inequities in socioeconomic status, race, ethnicity, religion, gender, disability, or sexual orientation.”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [6] Seyyed-Kalantari, L., Zhang, H., McDermott, M. B. A., Chen, I. Y. & Ghassemi, M. Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nat. Med. 27, 2176–2182 (2021). source
Not checked in the free run.
- cites [7] Yang, J., Soltan, A. A. S., Eyre, D. W., Yang, Y. & Clifton, D. A. An adversarial training framework for mitigating algorithmic biases in clinical machine learning. NPJ Digital Med. 6, 55 (2023). source
Not checked in the free run.
- cites [8] Obermeyer, Z., Powers, B., Vogeli, C. & Mullainathan, S. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 447–453 (2019). source
Not checked in the free run.
- Rests on an assumption: AI systems are increasingly being applied to healthcare.
- Rests on an assumption: The issue of bias being exhibited, perpetuated, or even amplified by AI algorithms is an increasing concern within healthcare.
- Premisecitation
Bias particularly impacts disadvantaged populations, which can be subject to algorithmic predictions that are less accurate or underestimate the need for care.
“Bias particularly impacts disadvantaged populations, which can be subject to algorithmic predictions that are less accurate or underestimate the need for care.”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [6] Seyyed-Kalantari, L., Zhang, H., McDermott, M. B. A., Chen, I. Y. & Ghassemi, M. Underdiagnosis bias of artificial intelligence algorithms applied to chest radiographs in under-served patient populations. Nat. Med. 27, 2176–2182 (2021). source
Not checked in the free run.
- cites [7] Yang, J., Soltan, A. A. S., Eyre, D. W., Yang, Y. & Clifton, D. A. An adversarial training framework for mitigating algorithmic biases in clinical machine learning. NPJ Digital Med. 6, 55 (2023). source
Not checked in the free run.
- cites [8] Obermeyer, Z., Powers, B., Vogeli, C. & Mullainathan, S. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 447–453 (2019). source
Not checked in the free run.
- Rests on an assumption: AI systems are increasingly being applied to healthcare.
- Premisecitation
TWIX is an add-on application that taught the SAIS model to add a prediction of the importance of video clips used to assess surgical skill.
“TWIX is an add-on application that taught the SAIS model to add a prediction of the importance of video clips that was used to assess surgical skill.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [4] Kiyasseh, D. et al. Human visual explanations mitigate bias in AI-based assessment of surgeon skills. NPJ Digital Med. 6, 54 (2023). source
Not checked in the free run.
- Premisecitation
There is significant heterogeneity in the datasets AI algorithms are trained with within algorithm types themselves.
“Currently, there is significant heterogeneity in terms of the datasets AI algorithms are trained with within algorithm types themselves13,14.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [13] Gubatan, J. et al. Artificial intelligence applications in inflammatory bowel disease: emerging technologies and future directions. World J. Gastroenterol. 27, 1920–1935 (2021). source
Not checked in the free run.
- cites [14] Moglia, A., Georgiou, K., Georgiou, E., Satava, R. M. & Cuschieri, A. A systematic review on artificial intelligence in robot-assisted surgery. Int. J. Surg. 95, 106151 (2021). source
Not checked in the free run.
- Premisecitation
Kiyasseh et al. deployed surgical AI systems (SAIS) on videos of robotic surgeries from three hospitals.
“Kiyasseh et al.4 highlight this potential application in their work deploying surgical AI systems (SAIS) on videos of robotic surgeries from three hospitals.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [4] Kiyasseh, D. et al. Human visual explanations mitigate bias in AI-based assessment of surgeon skills. NPJ Digital Med. 6, 54 (2023). source
Not checked in the free run.
- Premisecitation
Kiyasseh et al. developed a strategy called TWIX to mitigate bias.
“With their SAIS model, Kiyasseh et al.4 developed a strategy called TWIX to mitigate bias.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [4] Kiyasseh, D. et al. Human visual explanations mitigate bias in AI-based assessment of surgeon skills. NPJ Digital Med. 6, 54 (2023). source
Not checked in the free run.
- Premisecitation
The SAIS model showed an underskilling or overskilling bias at different rates across surgeon sub-cohorts.
“The SAIS model showed an underskilling or overskilling bias at different rates across surgeon sub-cohort.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [4] Kiyasseh, D. et al. Human visual explanations mitigate bias in AI-based assessment of surgeon skills. NPJ Digital Med. 6, 54 (2023). source
Not checked in the free run.
- Premisecitation
Bias can be generated across AI model development steps, including data collection/preparation, model development, model evaluation, and deployment in clinical settings.
“Bias can be generated across AI model development steps, including data collection/preparation, model development, model evaluation, and deployment in clinical settings9.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [9] Vokinger, K. N., Feuerriegel, S. & Kesselheim, A. S. Mitigating bias in machine learning for medicine. Commun. Med. 1, 25 (2021). source
Not checked in the free run.
- Premisecitation
Continuous learning possesses the risk of increasing or adding new bias if the new data are biased.
“However, continuous learning also possesses the risk of increasing or adding new bias if the new data are biased17.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [17] DeCamp, M. & Lindvall, C. Latent bias and the implementation of artificial intelligence in medicine. J. Am. Med. Inform. Assoc. 27, 2020–2023 (2020). source
Not checked in the free run.
- Classified as probably not a substantial claim. 6
- May combine two claims. 6
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationSimilar cases of bias have already been seen in healthcare systems.
“Similar cases have already been seen in healthcare systems8.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [8] Obermeyer, Z., Powers, B., Vogeli, C. & Mullainathan, S. Dissecting racial bias in an algorithm used to manage the health of populations. Science 366, 447–453 (2019). source
Not checked in the free run.
May combine two claims.
PremisecitationSAIS could reliably assess surgical performance but exhibited bias.
“In applying this AI model, Kiyasseh et al.4 found that it could reliably assess surgical performance but exhibited bias.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [4] Kiyasseh, D. et al. Human visual explanations mitigate bias in AI-based assessment of surgeon skills. NPJ Digital Med. 6, 54 (2023). source
Not checked in the free run.
May combine two claims.
InterpretationreasoningAn algorithm disproportionately assigning false negatives would lead to fewer follow-up scans and potentially more undiagnosed/untreated cancer cases, worsening health inequity for an already disadvantaged population.
“Regardless of the cause, the impact of an algorithm disproportionately assigning false negatives would include fewer follow-up scans, and potentially more undiagnosed/untreated cancer cases, worsening health inequity for an already disadvantaged population.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable).
InterpretationreasoningThis type of bias mitigation strategy could be used to continue to improve AI applications in the future.
“This accomplishment is beneficial not only for this particular use case but also implies that this type of bias mitigation strategy could be used to continue to improve AI applications in the future.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe STANDING Together initiative, launched in September 2022, aims to develop recommendations for the composition and reporting of datasets underpinning medical AI systems.
“The STANDING Together initiative (standards for data diversity, inclusivity, and generalizability), launched in September 2022, aims to develop recommendations for the composition (who is represented) and reporting (how they are represented) of datasets underpinning medical AI systems18.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [18] Ganapathi, S. et al. Tackling bias in AI health datasets through the STANDING Together initiative. Nat. Med. 28, 2232–2233 (2022). source
Not checked in the free run.
May combine two claims.
PremisecitationBias mitigation strategies may involve pre-processing data through sampling, in-processing by mathematical approaches, and post-processing.
“Bias mitigation strategies may involve interventions such as pre-processing data through sampling before a model is built, in-processing by implementing mathematical approaches to incentivize a model to learn balanced predictions, and post-processing11.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [11] Xu, J. et al. Algorithmic fairness in computational medicine. EBioMedicine 84, 104250 (2022). source
Not checked in the free run.
May combine two claims.
PremisecitationAI applications in surgery hold promise as tools to predict surgical outcomes, assess technical skills, or guide surgeons intraoperatively via computer vision.
“In surgery, AI applications hold promise as tools to predict surgical outcomes, assess technical skills, or guide surgeons intraoperatively via computer vision.”
- Also stated in: § Bias in medical AI algorithms · plain
“Specifically, within the field of surgery, AI applications hold promise as tools to predict surgical outcomes1, aid surgeons via computer vision for intraoperative surgical navigation2, and even as algorithms to assess technical skills and surgical performance1,3–5.”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites [1] Ma, R. et al. Surgical gestures as a method to quantify surgical performance and predict patient outcomes. NPJ Digital Med. 5, 187 (2022). source
Not checked in the free run.
- cites [2] Chadebecq, F., Vasconcelos, F., Mazomenos, E. & Stoyanov, D. Computer vision in the surgical operating room. Visc. Med. 36, 456–462 (2020). source
Not checked in the free run.
- cites [3] Kiyasseh, D. et al. A multi-institutional study using artificial intelligence to provide reliable and fair feedback to surgeons. Commun. Med. 3, 42 (2023). source
Not checked in the free run.
- Rests on an assumption: AI systems are increasingly being applied to healthcare.
- Also stated in: § Bias in medical AI algorithms · plain
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationAs part of the FDA Action Plan, the FDA intends to support the piloting of real-world performance monitoring, allowing for the detection of bias after deployment.
“As part of the FDA Action Plan, the FDA intends to support the piloting of real-world performance monitoring19, allowing for the detection of bias after deployment.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [19] FDA. Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan. www.fda.gov/media/145022/download (2021).
Not checked in the free run.
May combine two claims.
PremisecitationTWIX mitigated SAIS model bias, improving model performance both for the disadvantaged surgeon sub-cohorts and for surgical skill assessments overall.
“Kiyasseh et al.4 found that TWIX mitigated SAIS model bias, improving model performance both for the disadvantaged surgeon sub-cohorts and for surgical skill assessments overall.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [4] Kiyasseh, D. et al. Human visual explanations mitigate bias in AI-based assessment of surgeon skills. NPJ Digital Med. 6, 54 (2023). source
Not checked in the free run.
May combine two claims.
PremisecitationThe FDA has recognized challenges due to bias in AI and ML algorithms and released an action plan in January 2021, emphasizing the importance of identifying and mitigating bias in AI systems.
“Further, the FDA has recognized challenges due to bias in AI and ML algorithms and released an action plan (“Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan”) in January 20219,19, emphasizing the importance of identifying and mitigating bias in AI systems9.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [9] Vokinger, K. N., Feuerriegel, S. & Kesselheim, A. S. Mitigating bias in machine learning for medicine. Commun. Med. 1, 25 (2021). source
Not checked in the free run.
- cites [19] FDA. Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD) Action Plan. www.fda.gov/media/145022/download (2021).
Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
PremisecitationExpecting completely bias-free AI systems before implementation is unrealistic.
“Previous groups suggested that any performance discrepancy is indicative of algorithmic bias, but expecting completely bias-free systems before implementation is unrealistic12.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [12] Townson, S. Manage AI Bias Instead of Trying to Eliminate It. https://sloanreview.mit.edu/article/manage-ai-bias-instead-of-trying-to-eliminate-it/2023 (MIT Sloan Management Review, 2023).
Not checked in the free run.
Classified as probably not a substantial claim.
InterpretationreasoningTWIX requiring model predictions of video clip importance served a similar role to human assessors explaining the rationale for assessments.
“TWIX requiring model predictions of video clip importance served a similar role to human assessors explaining the rationale for assessments.”
Quote in the paperSubstantial claim
The paper relies on this without support.
AI systems are increasingly being applied to healthcare.
“Artificial intelligence systems are increasingly being applied to healthcare.”
“Artificial intelligence (AI) technology is increasingly applied to healthcare, from AI-augmented clinical research to algorithms for image analysis or disease prediction.”
The paper relies on this without support.
The integration of AI into medical technology and healthcare systems is only going to increase in the coming years.
“The integration of AI into medical technology and healthcare systems is only going to increase in the coming years.”
The paper relies on this without support.
Performance discrepancy may differ based on the data and population an AI algorithm is trained on and then subsequently applied to.
“Performance discrepancy may also differ based on the data and population an AI algorithm is trained on and then subsequently applied to.”
The paper relies on this without support.
The issue of bias being exhibited, perpetuated, or even amplified by AI algorithms is an increasing concern within healthcare.
“The issue of bias being exhibited, perpetuated, or even amplified by AI algorithms is an increasing concern within healthcare.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Racial and ethnic biases impact the use of AI/ML in medical imaging, diagnosis, and prognosis due to biases in healthcare databases.
“Studies have shown that racial and ethnic biases impact the use of AI/ML in medical imaging, diagnosis, and prognosis due to biases in healthcare databases [78, 154].”
From The ethics of using artificial intelligence in scientific research: new guidance needed for a new tool (2025) · cites this paper as [154]
Scientists can take steps to understand sources of bias and control them, especially those that can lead to discrimination, stigmatization, harm, or injustice.
“Nevertheless, scientists can (and should) take steps to understand sources of bias and control them, especially those that can lead to discrimination, stigmatization, harm, or injustice [89, 154, 188].”
From The ethics of using artificial intelligence in scientific research: new guidance needed for a new tool (2025) · cites this paper as [154]
AI systems can amplify and perpetuate biases present in the training data, leading to skewed predictions and potentially harmful implications for patient care.
“These systems can amplify and perpetuate biases present in the training data, leading to skewed predictions and potentially harmful implications for patient care.6,7”
From Balancing innovation and integrity: the role of AI in research and scientific writing (2023) · cites this paper as [7]
Biases in AI models can emerge during various stages, from data collection to model evaluation.
“This is concerning, as biases in AI models can emerge during various stages, from data collection to model evaluation.7”
From Balancing innovation and integrity: the role of AI in research and scientific writing (2023) · cites this paper as [7]
Cites
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1× · see citing works
“Similar cases have already been seen in healthcare systems 8 .”
Benjamens, S., Dhunnoo, P., & Meskó, B. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-00324-0 · Cited by 1,215 · cited 1× · see citing works
“An AI algorithm can either be locked — once the algorithm is trained, the model provides the same result when the same input is applied — or adaptive 16 .”
Seyyed-Kalantari, L., Zhang, H., McDermott, M. B. A., Chen, I. Y., & Ghassemi, M. (2021) · Nature Medicine · doi:10.1038/s41591-021-01595-0 · Cited by 928 · cited 2× · see citing works
“Bias is usually de fi ned as a difference in performance between sub-groups for a predictive task 6,7 .”
Artificial intelligence and algorithmic bias: implications for health systems
Panch, T., Mattie, H., & Atun, R. (2019) · Journal of Global Health · doi:10.7189/jogh.09.020318 · Cited by 533 · cited 1× · see citing works
“Additionally, there are likely underlying social inequalities in healthcare access and expenditures that impact how a model might be trained to predict risk 6,10 .”
Mitigating bias in machine learning for medicine
Vokinger, K. N., Feuerriegel, S., & Kesselheim, A. S. (2021) · Communications Medicine · doi:10.1038/s43856-021-00028-w · Cited by 269 · cited 2× · see citing works
“…challenges due to bias in AI and ML algorithms and released an action plan ( “ Arti fi cial Intelligence/Machine Learning (AI/ML)- Based Software as a Medical Device (SaMD) Action Plan ” ) in January 2021 9,19 , emphasizing the importance of identifying and mitigating bias in AI systems 9 .”
A systematic review on artificial intelligence in robot-assisted surgery
Moglia, A., Georgiou, K., Georgiou, E., Satava, R. M., & Cuschieri, A. (2021) · International Journal of Surgery · doi:10.1016/j.ijsu.2021.106151 · Cited by 236 · see citing works
Latent bias and the implementation of artificial intelligence in medicine
DeCamp, M., & Lindvall, C. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa094 · Cited by 198 · cited 1× · see citing works
“However, continuous learning also possesses the risk of increasing or adding new bias if the new data are biased 17 .”
Algorithmic fairness in computational medicine
Xu, J., Xiao, Y., Wang, W. H., Ning, Y., Shenkman, E. A., Bian, J., & Wang, F. (2022) · eBioMedicine · doi:10.1016/j.ebiom.2022.104250 · Cited by 170 · cited 1× · see citing works
“Bias mitigation strategies may involve interventions such as pre-processing data through sampling before a model is built, in-processing by implementing mathematical approaches to incentivize a model to learn balanced predictions, and post-processing 11 .”
An adversarial training framework for mitigating algorithmic biases in clinical machine learning
Yang, J., Soltan, A. A. S., Eyre, D. W., Yang, Y., & Clifton, D. A. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00805-y · Cited by 159 · cited 1× · see citing works
“Bias is usually de fi ned as a difference in performance between sub-groups for a predictive task 6,7 .”
A vision transformer for decoding surgeon activity from surgical videos
Kiyasseh, D., Ma, R., Haque, T. F., Miles, B. J., Wagner, C., Donoho, D. A., Anandkumar, A., & Hung, A. J. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01010-8 · Cited by 151 · see citing works
Gubatan, J., Levitte, S., Patel, A., Balabanis, T., Wei, M. T., & Sinha, S. R. (2021) · World Journal of Gastroenterology · doi:10.3748/wjg.v27.i17.1920 · Cited by 121 · see citing works
Tackling bias in AI health datasets through the STANDING TOGETHER initiative
Ganapathi, S., Palmer, J., Alderman, J. E., Calvert, M., Espinoza, C., Gath, J., Ghassemi, M., Heller, K., Mckay, F., Karthikesalingam, A., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01987-w · Cited by 82 · cited 1× · see citing works
“The STANDING Together initiative (standards for data diversity, inclusivity, and generalizability), launched in September 2022, aims to develop recommendations for the composition (who is represented) and reporting (how they are represented) of datasets under-pinning medical AI systems 18 .”
Computer Vision in the Surgical Operating Room
Chadebecq, F., Vasconcelos, F., Mazomenos, E. B., & Stoyanov, D. V. (2020) · Visceral Medicine · doi:10.1159/000511934 · Cited by 73 · see citing works
Kiyasseh, D., Laca, J., Haque, T. F., Miles, B. J., Wagner, C., Donoho, D. A., Anandkumar, A., & Hung, A. J. (2023) · Communications Medicine · doi:10.1038/s43856-023-00263-3 · Cited by 69 · see citing works
Surgical gestures as a method to quantify surgical performance and predict patient outcomes
Ma, R., Ramaswamy, A., Xu, J., Trinh, L., Kiyasseh, D., Chu, T. N., Wong, E. Y., Lee, R. S., Rodriguez, I., DeMeo, G., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00738-y · Cited by 65 · see citing works
Human visual explanations mitigate bias in AI-based assessment of surgeon skills
Kiyasseh, D., Laca, J., Haque, T. F., Otiato, M., Miles, B. J., Wagner, C., Donoho, D. A., Trinh, Q.-D., Anandkumar, A., & Hung, A. J. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00766-2 · Cited by 53 · cited 3× · see citing works
“With their SAIS model, Kiyasseh et al. 4 developed a strategy called TWIX to mitigate bias.”
InfluentialPutting explainable AI in context: institutional explanations for medical AI
Theunissen, M., & Browning, J. (2022) · Ethics and Information Technology · doi:10.1007/s10676-022-09649-8 · Cited by 40 · cited 1× · see citing works
“In any case, AI models will have to be explainable 15 with transparent methodologies so that these questions can be studied and debated in the coming years.”
Flagged first, then by citations
Cited by
Sorted by citations
Ethical and Bias Considerations in Artificial Intelligence/Machine Learning
Hanna, M. G., Pantanowitz, L., Jackson, B., Palmer, O., Visweswaran, S., Pantanowitz, J., Deebajah, M., & Rashidi, H. H. (2025) · Modern Pathology · doi:10.1016/j.modpat.2024.100686 · Cited by 535 · see citing works
A Scoping Review of Reporting Gaps in FDA-Approved AI Medical Devices
Muralidharan, V., Adewale, B. A., Huang, C. J., Nta, M. T., Ademiju, P. O., Pathmarajah, P., Hang, M. K., Adesanya, O., Abdullateef, R. O., Babatunde, A. O., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01270-x · Cited by 201 · see citing works
“However, despite these efforts, our study highlights reporting inconsistencies that may continue to propagate racial health disparities 21 .”
BackgroundResnik, D. B., & Hosseini, M. (2025) · AI and Ethics · doi:10.1007/s43681-024-00493-8 · Cited by 228 · see citing works
“Nevertheless, scientists can (and should) take steps to understand sources of bias and control them, especially those that can lead to discrimination, stigmatization, harm, or injustice [89, 154, 188].”
Fountzilas, E., Pearce, T. E., Baysal, M. A., Chakraborty, A., & Tsimberidou, A. M. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01471-y · Cited by 219 · see citing works
Saeidnia, H. R., Fotami, S. G. H., Lund, B. D., & Ghiasi, N. (2024) · Social Sciences · doi:10.3390/socsci13070381 · Cited by 202 · see citing works
“Practical strategies for mitigating bias in AI algorithms include diverse data collection, algorithmic audits, and ongoing evaluation to detect and correct biases that may arise during deployment (F. Chen et al. 2024; Ferrara 2023; Mensah 2023; Mittermaier et al. 2023).”
InfluentialAI-driven healthcare: A review on ensuring fairness and mitigating bias
Chinta, S. V., Wang, Z., Palikhe, A., Zhang, X., Kashif, A., Smith, M. A., Liu, J., & Zhang, W. (2025) · PLOS Digital Health · doi:10.1371/journal.pdig.0000864 · Cited by 241 · see citing works
“Bias Type References Description Pre-processing Selection Bias [65,66] Arises from how the data is chosen or sampled for training, leading to unrepresentative datasets.”
InfluentialLevels of autonomy in FDA-cleared surgical robots: a systematic review
Lee, A., Baker, T. S., Bederson, J. B., & Rapoport, B. I. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01102-y · Cited by 157 · see citing works
“Machine learning algorithms may propagate biases that exist within training data 20 .”
BackgroundMultimodal generative AI for medical image interpretation
Rao, V. M., Hla, M., Moor, M., Adithan, S., Kwak, S., Topol, E. J., & Rajpurkar, P. (2025) · Nature · doi:10.1038/s41586-025-08675-y · Cited by 128 · see citing works
“Deep learning models, particularly LLMs, are prone to biases that are inherent in the data on which they are trained 36–39 , 126 .”
Chen, Z., Chen, Y., Sun, Y., Tang, L., Zhang, L., Hu, Y., He, M., Li, Z., Cheng, S., Yuan, J., et al. (2024) · Signal Transduction and Targeted Therapy · doi:10.1038/s41392-024-01932-y · Cited by 116 · see citing works
“24,25 In realistic anti-HER2 therapy or anti-HER2 combined immunotherapy scenarios, the patient ’ s imaging history may only be partially available.”
BackgroundArtificial intelligence and explanation: How, why, and when to explain black boxes
Marcus, E., & Teuwen, J. (2024) · European Journal of Radiology · doi:10.1016/j.ejrad.2024.111393 · Cited by 114 · see citing works
Frontiers in pancreatic cancer on biomarkers, microenvironment, and immunotherapy
Yu, B., Shao, S., & Ma, W. (2024) · Cancer Letters · doi:10.1016/j.canlet.2024.217350 · Cited by 112 · see citing works
Foundation models for fast, label-free detection of glioma infiltration
Kondepudi, A., Pekmezci, M., Hou, X., Scotford, K., Jiang, C., Rao, A. T., Harake, E. S., Chowdury, A., Al‐Holou, W. N., Wang, L., et al. (2024) · Nature · doi:10.1038/s41586-024-08169-3 · Cited by 119 · see citing works
“Large and diverse medical datasets combined with high-quality self-supervised training results in better performance and less bias across a wide range of complex clinical and diagnostic tasks 18,47 .”
Generalizability of FDA-Approved AI-Enabled Medical Devices for Clinical Use
Windecker, D., Baj, G., Shiri, I., Kazaj, P. M., Kaesmacher, J., Gräni, C., & Siontis, G. C. M. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.8052 · Cited by 123 · see citing works
Derraz, B., Breda, G., Kaempf, C., Baenke, F., Cotte, F., Reiche, K., Köhl, U., Kather, J. N., Eskenazy, D., & Gilbert, S. (2024) · npj Precision Oncology · doi:10.1038/s41698-024-00517-w · Cited by 94 · see citing works
“…to drug therapy, bring with it pressing ethical challenges, the foremost of which are: (i) informed patient consent for AI use in clinical decision making and patient care 106 ; (ii) the threats to the physician-patient relationship; and (iii) the ampli fi cation of biases by AI systems 107,108 .”
BackgroundAlzheimer's disease outlook: controversies and future directions
Frisoni, G. B., Aho, E., Brayne, C., Ciccarelli, O., Dubois, B., Fox, N. C., Frederiksen, K. S., Gabay, C., Garibotto, V., Hofmarcher, T., et al. (2025) · The Lancet · doi:10.1016/s0140-6736(25)01389-3 · Cited by 112 · see citing works
Pennisi, F., Pinto, A., Ricciardi, G. E., Signorelli, C., & Gianfredi, V. (2025) · Antibiotics · doi:10.3390/antibiotics14020134 · Cited by 113 · see citing works
“Ad-ditionally, the presence of publication bias in some studies highlights the need for the rigorous, independent validation of AI models to ensure their reliability and generalizability [77].”
Generative artificial intelligence in mental health care: potential benefits and current challenges
Torous, J., & Blease, C. (2024) · World Psychiatry · doi:10.1002/wps.21148 · Cited by 90 · see citing works
Exploring the potential and limitations of artificial intelligence in animal anatomy
Choudhary, O. P., Infant, S. S., As, V., Chopra, H., & Manuta, N. (2024) · Annals of Anatomy - Anatomischer Anzeiger · doi:10.1016/j.aanat.2024.152366 · Cited by 103 · see citing works
Denecke, K., May, R., & Rivera-Romero, O. (2024) · Journal of Medical Systems · doi:10.1007/s10916-024-02043-5 · Cited by 90 · see citing works
Machine Learning and Bias in Medical Imaging: Opportunities and Challenges
Vrudhula, A., Kwan, A. C., Ouyang, D., & Cheng, S. C. (2024) · Circulation Cardiovascular Imaging · doi:10.1161/circimaging.123.015495 · Cited by 85 · see citing works
“91 ,92,93 In addition to routinely applied and updated evaluations of overall performance and fairness metrics, these same tools are likely to add value for ongoing assessments of how an algorithm operates in real-world settings. will also for interdisciplinary teams to iterate on and further…”
BackgroundFrom Recruitment to Retention: AI Tools for Human Resource Decision-Making
Madanchian, M. (2024) · Applied Sciences · doi:10.3390/app142411750 · Cited by 123 · see citing works
Addressing fairness issues in deep learning-based medical image analysis: a systematic review
Xu, Z., Li, J., Yao, Q., Li, H., Zhao, M., & Zhou, S. K. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01276-5 · Cited by 82 · see citing works
“Recently, several studies in medical areas have urged the assessment of fairness in MedIA, including surgery [30], nuclear medicine [31], and dental care [32], which shed light on this area.”
BackgroundMitigating the risk of artificial intelligence bias in cardiovascular care
Mihan, A., Pandey, A., & Spall, H. G. C. V. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00155-9 · Cited by 75 · see citing works
Prasanthi, S., Kumarachari, R. K., Bukke, S. P. N., Rajasekhar, E. S. K., Ashokkumar, J. A., & Eftekhari, Z. (2025) · Discover Oncology · doi:10.1007/s12672-025-02469-4 · Cited by 78 · see citing works
“Furthermore, AI algorithms may reflect biases present in their training data, leading to disparities in surgical outcomes [162].”
Evaluation and mitigation of cognitive biases in medical language models
Schmidgall, S., Harris, C., Essien, I., Olshvang, D., Rahman, T., Kim, J. W., Ziaei, R., Eshraghian, J., Abadir, P., & Chellappa, R. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01283-6 · Cited by 68 · see citing works
“Unchecked cognitive biases in these models may result in reduced quality of care, increased medical errors, and erosion of patient trust in AI-assisted healthcare 24 .”
Advancing AI Data Ethics in Nursing: Future Directions for Nursing Practice, Research, and Education
Dunlap, P. A. B., & Michalowski, M. (2024) · JMIR Nursing · doi:10.2196/62678 · Cited by 70 · see citing works
“For instance, large language models and other machine learning algorithms have been reported to generate incorrect medical results [3-5], exhibit bias [5,6], and fail to identify chronically ill Black American patients who require high-risk care management [5,7].”
BackgroundBragazzi, N. L., Crapanzano, A., Converti, M., Zerbetto, R., & Khamisy‐Farah, R. (2023) · Journal of Medical Internet Research · doi:10.2196/52091 · Cited by 60 · see citing works
Balancing innovation and integrity: the role of AI in research and scientific writing
BaHammam, A. (2023) · Nature and Science of Sleep · doi:10.2147/nss.s455765 · Cited by 50 · see citing works
Khude, H., & Shende, P. K. (2025) · Advances in Integrative Medicine · doi:10.1016/j.aimed.2025.100529 · Cited by 57 · see citing works
Artificial intelligence bias in the prediction and detection of cardiovascular disease
Mihan, A., Pandey, A., & Van Spall, H. G. C. (2024) · npj Cardiovascular Health · doi:10.1038/s44325-024-00031-9 · Cited by 51 · see citing works
“These biases can be ampli fi ed in subsequent cycles of AI learning and cause clinical harm, further exacerbating inequities amongst those already facing them 15 .”
European Resuscitation Council Guidelines 2025 Ethics in Resuscitation
Raffay, V. I., Wittig, J., Bossaert, L. L., Djakow, J., Djärv, T., Estella, Á., Lulić, I., Mentzelopoulos, S. D., Monsieurs, K. G., de Voorde, P. V., et al. (2025) · Resuscitation · doi:10.1016/j.resuscitation.2025.110734 · Cited by 51 · see citing works
Harnessing artificial intelligence for enhanced public health surveillance: a narrative review
Mendes, V. I. S., Mendes, B. M. F., Moura, R. P., Lourenço, I., Oliveira, M. F. A., Ng, K. L., & Pinto, C. S. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1601151 · Cited by 60 · see citing works
Morone, G., De Angelis, L., Martino Cinnera, A., Carbonetti, R., Bisirri, A., Ciancarelli, I., Iosa, M., Negrini, S., Kiekens, C., & Negrini, F. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1550731 · Cited by 41 · see citing works
Open Science at the generative AI turn: An exploratory analysis of challenges and opportunities
Hosseini, M., Horbach, S. P. J. M., Holmes, K., & Ross-Hellauer, T. (2025) · Quantitative Science Studies · doi:10.1162/qss_a_00337 · Cited by 40 · see citing works
“Arguments in favor of using GenAI mostly highlight efficiency gains, while critics are concerned with issues such as systemic errors and biases (Mittermaier, Raza, & Kvedar, 2023); the lack of moral and legal agency and the resulting diffusion of responsibilities and account-abilities (Bro ż ek &…”
Regulating the AI-enabled ecosystem for human therapeutics
Singh, R., Paxton, M., & Auclair, J. R. (2025) · Communications Medicine · doi:10.1038/s43856-025-00910-x · Cited by 39 · see citing works
“‘ AI bias ’ describes systematic errors causing unfair outcomes due to skewed data, fl awed algorithms, or societal prejudices 18 .”
Sarantopoulos, A., Mastori Kourmpani, C., Yokarasa, A. L., Makamanzi, C., Antoniou, P., Spernovasilis, N., & Tsioutis, C. (2024) · Tropical Medicine and Infectious Disease · doi:10.3390/tropicalmed9100228 · Cited by 40 · see citing works
“…detail, if the research question is formulated by the AI software, there could be potential selection bias, such as a gender bias or racial bias, that occurs since specific population sub-groups may be “favored” by the algorithm based on disease prevalence and the demographic data utilized [45,46].”
BackgroundRoche, S. D., Ekwunife, O. I., Mendonca, R., Kwach, B., Omollo, V. O., Zhang, S., Ong’wen, P. A., Hattery, D. W., Smedinghoff, S., Morris, S., et al. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1334881 · Cited by 40 · see citing works
Tan, M. J. T., Kasireddy, H. R., Satriya, A. B., Karim, H. B. A., & AlDahoul, N. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1522673 · Cited by 44 · see citing works
Artificial intelligence across the cancer care continuum
Riaz, I. B., Khan, M. A., & Osterman, T. J. (2025) · Cancer · doi:10.1002/cncr.70050 · Cited by 40 · see citing works
Parmanto, B., Aryoyudanta, B., Soekinto, T. W., Setiawan, I. M. A., Wang, Y., Hu, H., Saptono, A., & Choi, Y. K. (2024) · JMIR Formative Research · doi:10.2196/54633 · Cited by 35 · see citing works
“The AI system must employ stringent data privacy measures and transparency in its decision-making processes [59-61].”
BackgroundBačanin, N., Jovanović, L., Stoean, R., Stoean, C., Zivkovic, M. Z., Antonijević, M., & Dobrojević, M. (2024) · Axioms · doi:10.3390/axioms13050335 · Cited by 36 · see citing works
Artificial intelligence in risk prediction and diagnosis of vertebral fractures
Namireddy, S. R., Gill, S. S., Peerbhai, A., Kamath, A. G., Ramsay, D. S. C., Ponniah, H. S., Salih, A. M., Janković, D., Kalasauskas, D., Neuhoff, J., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-75628-2 · Cited by 35 · see citing works
Yao, I. Z., Dong, M., & Hwang, W. Y. K. (2025) · Mayo Clinic Proceedings Digital Health · doi:10.1016/j.mcpdig.2025.100253 · Cited by 39 · see citing works
“58 Furthermore, removing or masking personally identifiable information from data sets effectively anonymizes the data, complying with regulatory standards.”
Evaluating transparency in AI/ML model characteristics for FDA-reviewed medical devices
Mehta, V., Komanduri, A., Bhadouriya, R. S., Mehta, V., Johnson, M. D., Shrestha, P., Nikolov, M. C., Jain, B., Shah, N. H., & Schulman, K. A. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02052-9 · Cited by 45 · see citing works
Salvagno, M., Cassai, A. D., Zorzi, S., Zaccarelli, M., Pasetto, M., Sterchele, E. D., Chumachenko, D., Gerli, A. G., Azamfirei, R., & Taccone, F. S. (2024) · PLOS ONE · doi:10.1371/journal.pone.0309208 · Cited by 32 · see citing works
“This is particularly concerning in social sciences and medicine, where biased conclusions can have far-reaching implications [27].”
BackgroundOdone, A., Barbati, C., Amadasi, S., Schultz, T., & Resnik, D. B. (2025) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(25)00412-8 · Cited by 39 · see citing works
How to mitigate the risks of deployment of artificial intelligence in medicine?
İlikhan, S. U., Özer, M., Tanberkan, H., & Bozkurt, V. (2024) · TURKISH JOURNAL OF MEDICAL SCIENCES · doi:10.55730/1300-0144.5814 · Cited by 38 · see citing works
“A similar situation applies to AI systems used in the healthcare field as well [16,41].”
BackgroundResultInfluentialJeong, S.-M., Kim, S., Lee, E. C., & Kim, H. (2024) · Sensors · doi:10.3390/s24144625 · Cited by 32 · see citing works
A validated framework for responsible AI in healthcare autonomous systems
Alelyani, T. (2025) · Scientific Reports · doi:10.1038/s41598-025-25266-z · Cited by 48 · see citing works
“Numerous studies have shown that AI models trained on imbalanced datasets can perpetuate health disparities, particularly along racial, gender, and socioeconomic lines 31,41 .”
Veetil, I. K., Chowdary, D. E., Chowdary, P. N., Sowmya, V., & Gopalakrishnan, E. A. (2024) · Digital Signal Processing · doi:10.1016/j.dsp.2024.104407 · Cited by 31 · see citing works
Abdollahi, M., Jafarizadeh, A., Asbagh, A. G., Sobhi, N., Pourmoghtader, K., Pedrammehr, S., Asadi, H., Tan, R., Alizadehsani, R., & Acharya, U. R. (2024) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1560 · Cited by 32 · see citing works
“These strategies aim to mitigate biases, ensure fairness and equity, improve transparency, and promote trust among stakeholders (Mittermaier et al., 2023).”
Urbina, J. T., Vu, P. D., & Nguyen, M. V. (2024) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2024.08.014 · Cited by 31 · see citing works
Computer Vision in Clinical Neurology
Friedrich, M. U., Relton, S. D., Wong, D., & Alty, J. (2025) · JAMA Neurology · doi:10.1001/jamaneurol.2024.5326 · Cited by 30 · see citing works
Artificial intelligence and robotic surgical education
Brian, R., Murillo, A. D., Gomes, C., & Alseidi, A. A. (2024) · Global Surgical Education - Journal of the Association for Surgical Education · doi:10.1007/s44186-024-00262-5 · Cited by 28 · see citing works
“While a goal of AI-based feedback and assessment is standardization and bias reduction, algorithmic bias remains a troubling barrier [56].”
Riley, B. K., & Dixon, A. L. (2024) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2024.101833 · Cited by 28 · see citing works
Artificial Intelligence Applications in Cardio-Oncology: A Comprehensive Review
Guha, A., Shah, V., Nahle, T., Singh, S., Kunhiraman, H. H., Shehnaz, F., Nain, P., Makram, O. M., Mahmoudi, M., Al-Kindi, S. G., et al. (2025) · Current Cardiology Reports · doi:10.1007/s11886-025-02215-w · Cited by 28 · see citing works
Hussain, M. S., Ramalingam, P. S., Chellasamy, G., Yun, K., Bisht, A. S., & Gupta, G. (2025) · Clinical Breast Cancer · doi:10.1016/j.clbc.2025.03.006 · Cited by 28 · see citing works
Stevenson, E., Walsh, C., & Hibberd, L. (2023) · Annals of Clinical Biochemistry International Journal of Laboratory Medicine · doi:10.1177/00045632231203473 · Cited by 28 · see citing works
“24,25 The impact of such bias on the safety and accuracy of any advice provided could be an area for future study.”
BackgroundNavigating Bias and Fairness in Digital AI Systems
Tariq, M. U. (2024) · Advances in human and social aspects of technology book series · doi:10.4018/979-8-3693-4147-6.ch006 · Cited by 26 · see citing works
The Way Forward to Embrace Artificial Intelligence in Public Health
Hattab, G., Irrgang, C., Körber, N., Kühnert, D., & Ladewig, K. (2024) · American Journal of Public Health · doi:10.2105/ajph.2024.307888 · Cited by 26 · see citing works
Integrating wearable health devices with AI and edge computing for personalized rehabilitation
Xi, L., Li, C., Anari, M. S., & Rezaee, K. (2025) · Journal of Cloud Computing Advances Systems and Applications · doi:10.1186/s13677-025-00795-0 · Cited by 34 · see citing works
Principles Governing Ethical Development and Deployment of AI
Shukla, S. (2024) · International Journal of Engineering Business and Management · doi:10.22161/ijebm.8.2.5 · Cited by 26 · see citing works
“…in the surgical domain show potential as instruments for prognosticating surgical results, as computer vision tools to assist surgeons during intraoperative surgical navigation, and even as algorithms to evaluate technical proficiency and surgical performance(Mittermaier, Raza and Kvedar, 2023).”
BackgroundChatGPT’s adherence to otolaryngology clinical practice guidelines
Tessler, I., Wolfovitz, A., Alon, E., Gecel, N., Livneh, N., Zimlichman, E., & Klang, E. (2024) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-024-08634-9 · Cited by 23 · see citing works
Role of artificial intelligence in critical care nutrition support and research
Kittrell, H. D., Shaikh, A., Adintori, P. A., McCarthy, P. J., Kohli-Seth, R. D., Nadkarni, G. N., & Sakhuja, A. (2024) · Nutrition in Clinical Practice · doi:10.1002/ncp.11194 · Cited by 25 · see citing works
Gbagbo, F. Y., Ameyaw, E. K., & Yaya, S. (2024) · Reproductive Health · doi:10.1186/s12978-024-01924-9 · Cited by 39 · see citing works
Revolutionising Acute Cardiac Care With Artificial Intelligence: Opportunities and Challenges
Doolub, G., Khurshid, S. S., Thériault-Lauzier, P., Lapalme, A. N., Tastet, O., So, D. Y. F., Langlais, É. L., Corbin, D., & Avram, R. (2024) · Canadian Journal of Cardiology · doi:10.1016/j.cjca.2024.06.011 · Cited by 57 · see citing works
“104 Bias mitigation 105 will occupy a pivotal role in the design of AI-based models, and strategies will encompass steps such as preprocessing data before developing the model, encouraging the model to learn balanced predictions with the use of mathematical integrations, and postprocessing.”
A Comprehensive Review of Multimodal Large Language Models for Medical Imaging and Omics Data
Vavekanand, R. (2026) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-026-10504-y · Cited by 29 · see citing works
Preiksaitis, C. M., Nash, C. J., Gottlieb, M. A., Chan, T. M., Alvarez, A., & Landry, A. I. (2023) · AEM Education and Training · doi:10.1002/aet2.10924 · Cited by 22 · see citing works
Feng, Y., & Xia, P. (2025) · Plant Cell & Environment · doi:10.1111/pce.70185 · Cited by 25 · see citing works
Cheungpasitporn, W., Athavale, A. M., Ghazi, L., KASHANI, K. B., Colicchio, T. K., Koyner, J. L., Chen, J., Ix, J. H., Nadkarni, G. N., & Neyra, J. A. (2026) · Clinical Kidney Journal · doi:10.1093/ckj/sfag004 · Cited by 31 · see citing works
“AI-based decision support may also introduce automation bias, whereby clinicians over-rely on model outputs despite contextual misalignment, particularly when models are trained on population-specific or suboptimal datasets [94 , 95 ].”
InfluentialBeyond the stereotypes: Artificial Intelligence image generation and diversity in anesthesiology
Gisselbaek, M., Minsart, L., Köselerli, E., Suppan, M., Meço, B. C., Seidel, L., Albert, A., Chang, O. L. B., Saxena, S., & Berger‐Estilita, J. (2024) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2024.1462819 · Cited by 21 · see citing works
Resnik, D. B., & Hosseini, M. (2023) · doi:10.31235/osf.io/rbg9z · Cited by 20 · see citing works
Artificial intelligence and its application in clinical microbiology
Mairi, A., Hamza, L., & Touati, A. (2025) · Expert Review of Anti-infective Therapy · doi:10.1080/14787210.2025.2484284 · Cited by 30 · see citing works
“To mitigate these challenges, practical methods such as cross-validation and external validation across multiple cohorts are recommended to rigorously assess and enhance model generalizability [57,58].”
Tan, R. E. S., Teo, W. Z. W., & Puhaindran, M. E. (2024) · The Journal of Hand Surgery (Asian-Pacific Volume) · doi:10.1142/s2424835524300019 · Cited by 22 · see citing works
Mixed methods assessment of the influence of demographics on medical advice of ChatGPT
Andreadis, K., Newman, D. R., Twan, C., Shunk, A., Mann, D. M., & Stevens, E. R. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae086 · Cited by 20 · see citing works
Biomarkers in high grade serous ovarian cancer
Bates, M., Mohamed, B. M., Lewis, F., O’Toole, S. A., & O’Leary, J. J. (2024) · Biochimica et Biophysica Acta (BBA) - Reviews on Cancer · doi:10.1016/j.bbcan.2024.189224 · Cited by 24 · see citing works
Artificial intelligence in corporate governance
Ustahaliloğlu, M. K. (2025) · Corporate Law & Governance Review · doi:10.22495/clgrv7i1p11 · Cited by 19 · see citing works
Biases in Artificial Intelligence Application in Pain Medicine
Jumreornvong, O., Perez, A., Malave, B., Fatimah, M., Kia, A., & Nwaneshiudu, C. A. (2025) · Journal of Pain Research · doi:10.2147/jpr.s495934 · Cited by 21 · see citing works
Huang, Y., Wang, W., Zhou, J., Zhang, L., Lin, J., Liu, H., Hu, X., Zhou, Z., & Dong, W. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104152 · Cited by 20 · see citing works
Navigating the complexities of AI and digital governance: the 5W1H framework
Liao, S. M., Haykel, I., Cheung, K., & Matalon, T. (2025) · Journal of Responsible Technology · doi:10.1016/j.jrt.2025.100127 · Cited by 19 · see citing works
Lin, Y.-Y., Ding, J., Shen, H., Lin, Y.-M., Adhidjaja, E. C., & Chang, S.-C. (2025) · Clinical reviews in allergy and immunology · doi:10.1007/s12016-025-09105-7 · Cited by 20 · see citing works
Palma, G. D., Scendoni, R., Ferorelli, D., Benedictis, A. D., Tambone, V., & Micco, F. D. (2025) · Risk Management and Healthcare Policy · doi:10.2147/rmhp.s544523 · Cited by 28 · see citing works
“42 These risks include the exposure and potential misuse of sensitive data, which could lead to violations of individual rights and non-medical uses of personal information.”
Lonergan, R. M., Curry, J., Dhas, K., & Simmons, B. I. (2023) · Cureus · doi:10.7759/cureus.48788 · Cited by 19 · see citing works
“Equally, LLM training data, as is common across datasets in general, undoubtedly contain hidden biases [19, 20].”
BackgroundYang, H., Zhu, D., He, S., Xu, Z., Liu, Z., Zhang, W., & Cai, J. (2024) · Psychiatry Research · doi:10.1016/j.psychres.2024.115896 · Cited by 19 · see citing works
Lonsdale, H., Burns, M. L., Epstein, R. H., Hofer, I. S., Tighe, P. J., Delgado, J. A. G., Kor, D. J., MacKay, E. J., Rashidi, P., Wanderer, J. P., et al. (2025) · Anesthesiology · doi:10.1097/aln.0000000000005326 · Cited by 19 · see citing works
“63 Discussions of ethical principles for artificial intelligence in anesthesia remain theoretical, as few applications are ready for clinical use.”
Du, Q., Wei, X., Li, J., Kuang, E., Hao, J., Weng, D., & Fan, M. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710924 · Cited by 20 · see citing works
Wenderott, K., Krups, J., Weigl, M., & Wooldridge, A. R. (2025) · Journal of Medical Internet Research · doi:10.2196/63649 · Cited by 25 · see citing works
“Algorithmic bias holds the potential for patient harm, especially for populations considered disadvantaged [132].”
Toward governance of artificial intelligence in pediatric healthcare
Richter, F., Holmes, E., Richter, F., Guttmann, K., Duong, S. Q., Gangadharan, S., Schadt, E., Salmasian, H., Gelb, B. D., & Glicksberg, B. S. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-02000-7 · Cited by 24 · see citing works
“The harm of biased datasets is thoroughly described in the healthcare AI literature 18,19 .”
Mitigating bias in AI mortality predictions for minority populations: a transfer learning approach
Gu, T., Pan, W., Yu, J., Ji, G., Meng, X., Wang, Y., & Li, M. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-02862-7 · Cited by 21 · see citing works
“Furthermore, the integration of demographic considerations into AI models remains a challenge, often due to the lack of representative data and the complexity of bias mitigation techniques [14, 15].”
Era of Generalist Conversational Artificial Intelligence to Support Public Health Communications
Sezgın, E., & Kocaballi, A. B. (2024) · Journal of Medical Internet Research · doi:10.2196/69007 · Cited by 17 · see citing works
“These biases risk distorting health communications and exacerbating health disparities [31,32].”
Palma, G. D., Scendoni, R., Tambone, V., Alloni, R., & Micco, F. D. (2025) · Journal of Healthcare Risk Management · doi:10.1002/jhrm.70000 · Cited by 20 · see citing works
“25 At the same time, health care data may be incomplete, erroneous, or even manipulated, introducing vulnerabilities into a system.”
Exploring the ethical landscape of AI in academic writing
Ateriya, N., Sonwani, N. S., Thakur, K., Kumar, A., & Verma, S. (2025) · Egyptian Journal of Forensic Sciences · doi:10.1186/s41935-025-00453-1 · Cited by 22 · see citing works
Zeng, Q., Huang, C., & Zhu, J. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1656603 · Cited by 20 · see citing works
Artificial Intelligence for the Detection of Patient–Ventilator Asynchrony
Tlimat, A., Fowler, C., Safadi, S., et al. (2025) · Respiratory Care · doi:10.1089/respcare.12540 · Cited by 14 · see citing works
Regehr, C., Regehr, K., Goel, V., Sato, C., Lyons, K. A., & Rudzicz, F. (2023) · BMJ Public Health · doi:10.1136/bmjph-2023-000444 · Cited by 14 · see citing works
“This results in concerns that deploying these models in relation to healthcare may exacerbate racial, gender, socioeconomic and other inequities 81 or even be the source of misinformation through the use of chatbots.”
Ma, S., Zhang, M., Sun, W., Gao, Y., Jing, M., Gao, L., & Wu, Z. (2025) · Healthcare and Rehabilitation · doi:10.1016/j.hcr.2024.100006 · Cited by 17 · see citing works
“22 Moreover, upcoming studies should prioritize addressing the most pressing clinical needs.”
Multimodal large language models for medical image diagnosis: Challenges and opportunities
Zhang, A., Zhao, E., Wang, R., Zhang, X., Wang, J., & Chen, E. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104895 · Cited by 14 · see citing works
Li, M., Xiong, X., Xu, B., & Dickson, C. (2024) · JMIR Formative Research · doi:10.2196/53918 · Cited by 15 · see citing works
“These biases might also exacerbate existing inequalities [31,32].”
BackgroundLarge Language Models in Otolaryngology Residency Admissions: A Random Sampling Analysis
Halagur, A. S., Balakrishnan, K., & Ayoub, N. (2024) · The Laryngoscope · doi:10.1002/lary.31705 · Cited by 13 · see citing works
“21,26 The LLM is ultimately limited by the quality of its training data, and it is this bias that impacts LLM decision-making.”
(2024) · doi:10.1177/08404704241257144 · Cited by 12 · see citing works
“14,15 For example, algorithms employed by prominent health systems have exhibited racial bias for marginalized populations, including Black, 16 LGBTQ2S+, Asian, 17 and Indigenous populations.”
Background
- Authors
- Mittermaier, M., Raza, M. M., & Kvedar, J. C.
- Affiliations
- Humboldt-Universität zu Berlin (DE)
- Harvard University (US)
- Corresponding author
- Mittermaier, Mirja
- Venue
- npj Digital Medicine
- Published
- 2023-06-14
- Volume
- 6
- Issue
- 1
- Pages
- 113-113
- ISSN
- 2398-6352 (ISSN-L)
- PMC
- PMC10264403
- OpenAlex
- W4380538374
- Semantic Scholar
- 3e2fd4f099046ad3b61d8cad107d5883dc52370a
- Cited by
- 372
- References
- 17
- Citations by year
- 2026: 99 · 2025: 172 · 2024: 87 · 2023: 14
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- npj Digital Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Deutsche Forschungsgemeinschaft
- Berlin Institute of Health
- Topics
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- algorithmic bias, bias mitigation, health care equity, algorithmic fairness, surgical outcome prediction, generalizability, disadvantaged groups
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
