The DOI matches its Crossref record; 56 of the 66 references it cites were found in a registry or an index; none of the 86 sources its claims cite is checked for support yet.
AI pitfalls and what not to do: mitigating bias in AI
Open accessAbstract
Various forms of artificial intelligence (AI) applications are being deployed and used in many healthcare systems. As the use of these applications increases, we are learning the failures of these models and how they can perpetuate bias. With these new lessons, we need to prioritize bias evaluation and mitigation for radiology applications; all the while not ignoring the impact of changes in the larger enterprise AI deployment which may have downstream impact on performance of AI models. In this paper, we provide an updated review of known pitfalls causing AI bias and discuss strategies for mitigating these biases within the context of AI deployment in the larger healthcare enterprise. We describe these pitfalls by framing them in the larger AI lifecycle from problem definition, data set selection and curation, model training and deployment emphasizing that bias exists across a spectrum and is a sequela of a combination of both human and machine factors.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This review argues that bias in radiology AI arises across the entire lifecycle—problem definition, data acquisition, model development, validation, and deployment—from human, machine, and systems factors. It recommends mitigation at every stage, including diverse teams and data, rigorous validation, ongoing monitoring, and caution with purely mathematical fairness approaches.
Key findings
- Bias can persist even when race is excluded from model inputs: a commercial algorithm referred fewer Black patients with similar disease burden than White patients because healthcare costs were used as a proxy for need, learning from a system where Black patients are less likely to seek or receive care.
- Imaging AI data are geographically and demographically narrow: a review of published AI articles from 2015–2019 found data largely from California, Massachusetts, and New York, with none from 34 states; among 23 chest X-ray datasets, only 8.7% reported race or ethnicity and 4.3% reported insurance status; the UK Biobank has 500,000 patients but only 6% of non-European ancestry, and only 8% has been used for research.
- Validation gaps are common: among 151 FDA-cleared imaging AI products by November 2021, only 64% used clinical data for validation, 4% had patient demographics, 5% reported machine specifications, and 34% had multi-institutional validation; a systematic review of external validation found 86 studies with a median of 240 cases, about 47% positive.
- Deployment failures and human–machine interactions matter: the Epic sepsis model at Michigan Medicine identified only 7% of patients whose sepsis diagnosis was missed by a clinician, failed to flag 67% of patients with sepsis, and generated alerts on 18% of all hospitalized patients; in another example, pathologist accuracy worsened when a model’s prediction was incorrect.
Methods
- Narrative/updated review, not a systematic review; it frames AI bias pitfalls across the AI lifecycle: problem definition, dataset selection and curation, model training and deployment, and post-deployment evaluation and monitoring.
- Uses cited literature and examples—such as Obermeyer et al., the Epic sepsis model, COVID-19 imaging prediction models, RadImageNet, and federated learning—to illustrate bias sources and mitigation strategies.
- No new primary data, sample, or statistical analysis are reported; a formal methods section is not stated in the paper.
Limitations
- The paper does not include a dedicated limitations section; as a narrative review, it lacks systematic search or selection criteria and does not quantitatively synthesize evidence (evident from the text).
- It relies on selected published examples and author experience, so its coverage may not be exhaustive or generalizable (evident from the text).
- It states field-level limitations: lack of a unified definition of bias, limited model explainability, proprietary “black box inside a black box” models, and difficulty detecting harmful hidden signals in imaging datasets.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
68 Not checked yet · 32 Own interpretation · 5 No basis given · 1 Own result
In the paper
“In our own experience, 53 we observed that pre-processing of images is important and affects the federated model performance.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in model development
What the sources say
- Not checked yetcites [53] Peng, 2022, Evaluation of Federated learning variations for COVID-19 diagnosis using chest Radiographs from 42 US and European hospitals, J Am Med Inform Assoc source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our preliminary work found that signals that identify race are also contained on synthetic data sets.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in data acquisition and collection
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“For example, COVID-19 prediction models have been demonstrated to learn which institution the images were obtained from rather than features pertaining to the underlying disease. 15”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in data acquisition and collection
What the sources say
- Not checked yetcites [15] Roberts, 2021, Common pitfalls and recommendations for using machine learning to detect and Prognosticate for COVID-19 using chest Radiographs and CT scans, Nat Mach Intell source
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Despite the promise and hope of artificial intelligence (AI) to improve patient care, several real-world failures of AI systems have been documented.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction
What the sources say
- Not checked yetcites [1] Obermeyer, 2019, Dissecting racial bias in an algorithm used to manage the health of populations, Science source
- Not checked yetcites [2] Seyyed-Kalantari, 2021, Underdiagnosis bias of artificial intelligence Algorithms applied to chest Radiographs in under-served patient populations, Nat Med source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Before the subsequent model overhaul as a result of the publication (rather than some planned post-deployment evaluation and monitoring), it was observed that one input variable to the model was antibiotic orders by a provider—a type of data leakage since an infection is already being considered at that point.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction
What the sources say
- Not checked yetcites [13] Wong, 2021, External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients, JAMA Intern Med source
- Not checked yetcites [14] Habib, 2021, The epic sepsis model falls short-the importance of external validation, JAMA Intern Med source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“However, by using healthcare costs as proxy for the need for complex care, the model learned from a system where Black patients are less likely to seek and/or receive care compared to matched White patients. 1”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > AI pitfalls when defining a task
What the sources say
- Not checked yetcites [1] Obermeyer, 2019, Dissecting racial bias in an algorithm used to manage the health of populations, Science source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“A review of published AI articles between 2015 and 2019 in six disciplines found that data used for imaging AI models largely come from three states—California, Massachusetts, and New York—with none from 34 states. 22”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in data acquisition and collection
What the sources say
- Not checked yetcites [22] Kaushal, 2020, Geographic distribution of US cohorts used to train deep learning Algorithms, JAMA source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“The methodological flaws likely is a reflection of the lack of diversity in the development team to allow clinicians, data analysts and patient advocacy groups to work side-by-side throughout the AI life cycle. 15”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction
What the sources say
- Not checked yetcites [15] Roberts, 2021, Common pitfalls and recommendations for using machine learning to detect and Prognosticate for COVID-19 using chest Radiographs and CT scans, Nat Mach Intell source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Zech et al demonstrated a similar finding where models predicted pneumonia by learning where the CXRs was obtained from. 20”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in data acquisition and collection
What the sources say
- Not checked yetcites [20] Zech, 2018, Variable generalization performance of a deep learning model to detect pneumonia in chest Radiographs: A cross-sectional study, PLoS Med source
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“A similar result is summarized in a systematic review of COVID-19 prediction models on CT scans and Chest X-rays (CXRs) that found limited clinical utility due to methodological flaws and underlying biases stemming from small training data sets, data set variability, and limited integration of non-imaging data, among others.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction
What the sources say
- Not checked yetcites [15] Roberts, 2021, Common pitfalls and recommendations for using machine learning to detect and Prognosticate for COVID-19 using chest Radiographs and CT scans, Nat Mach Intell source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 29
- Probably not a substantial claim. 21
In the paper
“Moreover, FedAvg is biased to research sites contributing larger data sets with poor performance reported for smaller data sets.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in model development
What the sources say
- Not checked yetcites [53] Peng, 2022, Evaluation of Federated learning variations for COVID-19 diagnosis using chest Radiographs from 42 US and European hospitals, J Am Med Inform Assoc source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“In our experimental evaluation of the reading race paper where we demonstrated superhuman ability for AI to predict self-reported race across multiple modalities, we found that some transfer learning tasks are better in race detection. 28”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in data acquisition and collection
What the sources say
- Not checked yetcites [28] Gichoya, 2022, AI recognition of patient race in medical imaging: a Modelling study, Lancet Digit Health source
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“When we used the FedAvg algorithm, the model improved its generalizability on external data sets but at the expense of some degradation in internal validation.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in model development
What the sources say
- Not checked yetcites [53] Peng, 2022, Evaluation of Federated learning variations for COVID-19 diagnosis using chest Radiographs from 42 US and European hospitals, J Am Med Inform Assoc source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Existing AI data sets are acquired from limited geographical regions; most lack information necessary to evaluate subgroup performance of models.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in data acquisition and collection
What the sources say
- Not checked yetcites [22] Kaushal, 2020, Geographic distribution of US cohorts used to train deep learning Algorithms, JAMA source
- Not checked yetcites [23] Yi, 2022, Demographic reporting in publicly available chest radiograph data SETS: opportunities for mitigating sex and racial disparities in deep learning models, J Am Coll Radiol source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“In addition to observing overall performance decline secondary to differences in data distribution after model deployment, subgroup evaluations have shown varying levels of performance with poor performance especially for historically underserved patients. 1,2”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction
What the sources say
- Not checked yetcites [1] Obermeyer, 2019, Dissecting racial bias in an algorithm used to manage the health of populations, Science source
- Not checked yetcites [2] Seyyed-Kalantari, 2021, Underdiagnosis bias of artificial intelligence Algorithms applied to chest Radiographs in under-served patient populations, Nat Med source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The algorithm (which was abandoned before deployment) learnt that patients from less affluent zip codes are likely to have longer hospital stays and may not benefit from case management facilitation of early discharge.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > AI pitfalls when defining a task
What the sources say
- Not checked yetcites [16] Nordling, 2019, A fairer way forward for AI in health care, Nature source
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Problems in data set development include limited patient diversity, limited resolution and missing clinical confounders, limited or suboptimal quality annotations, and lack of data standards and best practice to ensure quality and reproducibility.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in data acquisition and collection
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“The failure and subsequent withdrawal of the Epic (TM) sepsis model used by 180 customers representing hundreds of hospitals provides a template to study AI failure across a spectrum. 13,14”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction
What the sources say
- Not checked yetcites [13] Wong, 2021, External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients, JAMA Intern Med source
- Not checked yetcites [14] Habib, 2021, The epic sepsis model falls short-the importance of external validation, JAMA Intern Med source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We emphasize that bias permeates every step of the lifecycle and is a sequela of human, machine, and systems factors.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“A review of 23 CXR data sets found that although the majority reported age and sex, only 8.7% reported race or ethnicity and 4.3% reported insurance status. 23”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in data acquisition and collection
What the sources say
- Not checked yetcites [23] Yi, 2022, Demographic reporting in publicly available chest radiograph data SETS: opportunities for mitigating sex and racial disparities in deep learning models, J Am Coll Radiol source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Efforts to mitigate these biases are challenging due to—lack of a unified definition of bias 10 ; focus on a statistical definition of bias and a technocentric definition of bias without involving patients and communities; and post-hoc reflection on bias rather than as a deliverable by design. 11”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction
What the sources say
- Not checked yetcites [10] Narayanan A . Tutorial: 21 fairness definitions and their politics . 2018 . Available from : https://www.youtube.com/watch?v=jIXIuYdnyyk
- Not checked yetcites [11] Wawira Gichoya, 2021, Equity in essence: a call for Operationalising fairness in machine learning for Healthcare, BMJ Health Care Inform source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Upon testing at Michigan Medicine, the algorithm only identified 7% of patients whose sepsis diagnosis was missed by a clinician, failed to flag 67% of patients with sepsis despite generating alerts on 18% of all hospitalized patients resulting in alert fatigue.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction
What the sources say
- Not checked yetcites [13] Wong, 2021, External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients, JAMA Intern Med source
- Not checked yetcites [14] Habib, 2021, The epic sepsis model falls short-the importance of external validation, JAMA Intern Med source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It should be noted that model auditing and bias detection, which benefit from diverse perspectives during training and validation, are more difficult in federated learning where only the weights are shared across the groups of modelers.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in model development
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Some groups have suggested datasheets 39,40 —a checklist to guide model developers on how to use data—to address this problem.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in data acquisition and collection
What the sources say
- Not checked yetcites [39] Gebru T , Morgenstern J , Vecchione B , Vaughan JW , Wallach H , Daumé III H . Datasheets for Datasets . arXiv 2018 .
- Not checked yetcites [40] Garbin C , Rajpurkar P , Irvin J , Lungren MP , Marques O . Structured dataset documentation: a datasheet for CheXpert . arXiv 2021 .
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The availability of a radiology specific data set to facilitate transfer learning is a step towards efficiency in model development.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in data acquisition and collection
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“CXRs have also been used to predict biological age, cardiac disease outcomes and healthcare cost at 1, 3 and 5 years. 30”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in data acquisition and collection
What the sources say
- Not checked yetcites [30] Sohn, 2022, Prediction of future Healthcare expenses of patients from chest Radiographs using deep learning: a pilot study, Sci Rep source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Overall, when local model performance matters—then personalized federated learning models (such as FedBN) should be prioritized, while for external performance, FedAvg should be used. 53”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in model development
What the sources say
- Not checked yetcites [53] Peng, 2022, Evaluation of Federated learning variations for COVID-19 diagnosis using chest Radiographs from 42 US and European hospitals, J Am Med Inform Assoc source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Federated learning—whereby models are trained across multiple institutions without the data being shared except for the model weights—has been proposed as a solution to improve model performance across diverse patient groups.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in model development
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Such data set will also discourage the use of Frankenstein data sets—assembled from other data sets and redistributed under a new name—as pretraining can be done on RadImageNet and fine-tuned on a smaller data set.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in data acquisition and collection
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Beyond individual images, class imbalance also occurs due to distribution of positive versus negative cases (since most studies are normal), and across various demographic distributions within data sets.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in model development
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Class imbalance occurs frequently in radiology.
In the paper
“In fact, only 8% of the UK Biobank has been used for research, with most of this research focused on patients of European ancestry.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in data acquisition and collection
What the sources say
- Not checked yetcites [24] Privé, 2022, Using the UK Biobank as a global reference of worldwide populations: application to measuring ancestry diversity from GWAS summary Statistics, Bioinformatics source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These models have a larger feature space used for training, which require feature selection early in the training process.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in model development
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Radiology AI performance is affected by other underlying practice patterns which can appear as hidden signals in AI data sets.”
§ AI pitfalls and what not to do: mitigating bias in AI > Conclusion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
In the paper
“Bias risk assessment tools can facilitate this process. 65”
§ AI pitfalls and what not to do: mitigating bias in AI > Discussion
What the sources say
- Not checked yetcites [65] Moons, 2019, PROBAST: A tool to assess risk of bias and applicability of prediction model studies: explanation and elaboration, Ann Intern Med source
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As highlighted in Figure 1, bias at each stage of model development can occur due to machine factors, human factors or a combination of both.”
§ AI pitfalls and what not to do: mitigating bias in AI > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
In the paper
“This test data must be created by the local IT team and then curated carefully by the local clinicians who may not be familiar with the requisite subgroup model performance evaluation and other unintended consequences of AI bias. 61”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in model implementation and deployment
What the sources say
- Not checked yetcites [61] Ahluwalia, 2023, The subgroup imperative: chest X-ray Classifier generalization gaps in patient, setting, and pathology subgroups, Radiology: Artificial Intelligence
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
- Rests on an assumption: In many cases, models do not automatically perform well out of the box and require some fine tuning and calibration to the local site data.
None of the sources cited for this claim have been checked yet.
In the paper
“Following guidelines for research documentation and complete reporting increases transparency. 63,64”
§ AI pitfalls and what not to do: mitigating bias in AI > Discussion
What the sources say
- Not checked yetcites [63] Mongan, 2020, Checklist for artificial intelligence in medical imaging (CLAIM): A guide for authors and reviewers, Radiol Artif Intell source
- Not checked yetcites [64] Bossuyt, 2015, STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies, Radiology source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, bias at one stage can be propagated to other stages, hence efforts to detect and mitigate bias should be at each stage.”
§ AI pitfalls and what not to do: mitigating bias in AI > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
In the paper
“For example, the use of a machine learning algorithm by pathologists to differentiate different liver cancer types found that the pathologist accuracy did not improve when the model’s prediction was correct; however, the pathologist performance worsened when the model’s prediction was incorrect. 62”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in model implementation and deployment
What the sources say
- Not checked yetcites [62] Kiani, 2020, Impact of a deep learning assistant on the histopathologic classification of liver cancer, NPJ Digit Med source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Neglecting continents and ethnically diverse locations perpetuates health inequities and hinders the growth and applicability of AI models.”
§ AI pitfalls and what not to do: mitigating bias in AI > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“In fact, a review of the use of clinical trials for machine learning in healthcare found only 41 randomized clinical trials, and most did not adhere to accepted reporting guidelines and had limited participation from minority groups. 58”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in model evaluation and validation
What the sources say
- Not checked yetcites [58] Plana, 2022, Randomized clinical trials of machine learning interventions in health care: A systematic review, JAMA Netw Open source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“A review of 151 imaging AI products cleared by the FDA by November 2021 showed that only 64% of these products used clinical data for their validation; and these clinical data had limited information on demographic and technical confounders (only 4% had patient demographics and 5% reported machine specifications). 56”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in model evaluation and validation
What the sources say
- Not checked yetcites [56] Khunte, 2023, Trends in clinical validation and usage of US food and Drug Administration-cleared artificial intelligence Algorithms for medical imaging, Clin Radiol source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly, relying on data sets with narrow geographic and other dimensions of diversity can further biases and limits generalizability of AI technologies.”
§ AI pitfalls and what not to do: mitigating bias in AI > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“The regulatory landscape is changing as exemplified by the actions of four federal US agencies who recently issued a joint statement against biased models and their use including non-medical use like employment discrimination. 59”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in model evaluation and validation
What the sources say
- Not checked yetcites [59] Chopra R , Khan LM . Joint statement on enforcement efforts against discrimination and bias in automated systems . Available from : https://www.ftc.gov/system/files/ftc_gov/pdf/EEOC-CRT-FTC-CFPB-AI-Joint-Statement%28final%29.pdf
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Failure to provide transparent documentation and detailed methodologies especially for commercial algorithms has potential to perpetuate biases and prevent critical analysis.”
§ AI pitfalls and what not to do: mitigating bias in AI > Discussion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Ensuring that AI works for all will require diverse teams in addition to diverse data sets.”
§ AI pitfalls and what not to do: mitigating bias in AI > Conclusion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
In the paper
“Despite infrequent to even non-existent external validation for both FDA approved and peer reviewed articles, when external validation is performed, it is typically limited in scope.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in model evaluation and validation
What the sources say
- Not checked yetcites [57] Yu, 2022, External validation of deep learning Algorithms for Radiologic diagnosis: A systematic review, Radiol Artif Intell source
plainn = 240- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The most challenging issue for bias in medical imaging arises from techniques to address class imbalance and how to evaluate and mitigate harmful “hidden signals” in imaging data sets, as they can lead to shortcut learning and reinforce existing biases.”
§ AI pitfalls and what not to do: mitigating bias in AI > Discussion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Bias occurs across the AI life cycle, and hence must be tackled at each step from problem definition all the way through post-deployment monitoring.”
§ AI pitfalls and what not to do: mitigating bias in AI > Conclusion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“For example, these organizations issue statements like “consider evaluation of bias”, without clear instructions on how the evaluation should be performed, and what consequences result from not performing such evaluation. 11”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in model evaluation and validation
What the sources say
- Not checked yetcites [11] Wawira Gichoya, 2021, Equity in essence: a call for Operationalising fairness in machine learning for Healthcare, BMJ Health Care Inform source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Due to limited AI explainability, these hidden signals can be difficult to detect and have led to algorithmic bias that encrypts health inequities into practice.”
§ AI pitfalls and what not to do: mitigating bias in AI > Conclusion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Even on what would be considered a small sample size for external validation, nearly half of the 86 studies reported a modest increase in external performance with nearly a quarter reporting a substantial decrease, reiterating on the challenge of validation. 57”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in model evaluation and validation
What the sources say
- Not checked yetcites [57] Yu, 2022, External validation of deep learning Algorithms for Radiologic diagnosis: A systematic review, Radiol Artif Intell source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“A recent systematic review found that 86 studies have a median of 240 cases used for external validation, with about 47% of which are positive cases. 57”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in model evaluation and validation
What the sources say
- Not checked yetcites [57] Yu, 2022, External validation of deep learning Algorithms for Radiologic diagnosis: A systematic review, Radiol Artif Intell source
plainn = 240- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“because the approval process require resubmission when models are updated, there are no incentives to address algorithmic bias during the initial application.”
§ AI pitfalls and what not to do: mitigating bias in AI > Introduction > Pitfalls in model evaluation and validation
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
The paper relies on this without support.
Bias principles are largely considered too late, if at all, when designing AI.
“Bias principles are largely considered too late, if at all, when designing AI.”
The paper relies on this without support.
Comprehensive AI data sets are limited and expensive to curate.
“Comprehensive AI data sets are limited and expensive to curate.”
The paper relies on this without support.
Presently, AI models are trained on data collected from tertiary care academic institutions.
“While there have been initiatives to build data sets from “data deserts”, presently, AI models are trained on data collected from tertiary care academic institutions.”
The paper relies on this without support.
Class imbalance occurs frequently in radiology.
“Class imbalance occurs frequently in radiology.”
The paper relies on this without support.
Radiology images tend to have high pixel dimensions and resolutions, yet pathology is represented on a small percentage of the pixels.
“Radiology images tend to have high pixel dimensions and resolutions, yet pathology is represented on a small percentage of the pixels.”
The paper relies on this without support.
Model validation remains a barrier that limits incorporation of AI into clinical care.
“Model validation remains a barrier that limits incorporation of AI into clinical care.”
The paper relies on this without support.
Many factors affect AI validation, including high costs and lack of consensus on proper validation protocol.
“Many factors affect AI validation—including high costs of validation (technical pipeline for data set preparation and curation, diverse team of machine learning and clinical experts, regulatory approval burden), and lack of consensus on what is the proper protocol of validation.”
The paper relies on this without support.
AI models 'fail silently' i.e. they do not provide an output to the user when they do not recognize the model input.
“In these cases, the AI models “fail silently” i.e. they do not provide an output to the user when they do not recognize the model input.”
The paper relies on this without support.
AI models will make a guess estimate of the prediction for wrong input data without providing the level of uncertainty.
“These AI models will make a guess estimate of the prediction for wrong input data without providing the level of uncertainty.”
The paper relies on this without support.
The greatest challenge of bias detection and mitigation remains the lack of a consensus definition of algorithmic bias despite many toolkits developed for this purpose.
“The greatest challenge of bias detection and mitigation remains the lack of a consensus definition of algorithmic bias despite many toolkits developed for this purpose.”
The paper relies on this without support.
External validation can ensure models generalize well across populations.
“External validation can ensure models generalize well across populations.”
The paper relies on this without support.
Bias evaluation and mitigation frequently occurs post-hoc.
“Bias evaluation and mitigation frequently occurs post-hoc”
The paper relies on this without support.
A review of AI reporting guidelines noted that references to bias are either absent, made in passing or included in the guideline supplement.
“A review of AI reporting guidelines noted that references to bias are either absent, made in passing or included in the guideline supplement.”
The paper relies on this without support.
Clinical trials have limitations including high costs, most trials never meet their enrollment proportions, and bias still exists in clinical trials despite their rigor.
“While some people have called for more clinical trials in validating AI, we must acknowledge the limitations of such trials including high costs, the fact that most trials never meet their enrollment proportions, and that bias still exists in clinical trials despite their rigor.”
The paper relies on this without support.
Proprietary models are usually trained and validated on private data sets that are not accessible for other researchers to evaluate the model.
“Proprietary models present an additional layer of complexity because they are usually trained and validated on private data sets that are not accessible for other researchers to evaluate the model.”
The paper relies on this without support.
Evaluation of proprietary algorithms, when performed, involves simulations to try and replicate model performance.
“Evaluation of proprietary algorithms, when performed, involves simulations to try and replicate model performance.”
The paper relies on this without support.
Vendors must navigate the convoluted process of integrating with current hospital IT systems, and engaging clinicians and patients.
“Vendors must navigate the convoluted process of integrating with current hospital IT systems, and engaging clinicians and patients.”
The paper relies on this without support.
In many cases, models do not automatically perform well out of the box and require some fine tuning and calibration to the local site data.
“Moreover, in many cases, the models do not automatically perform well out of the box and require some fine tuning and calibration to the local site data.”
The paper relies on this without support.
Due to lack of IT infrastructure and process for continuous model monitoring, changes in software or image acquisition equipment or protocols can result in model failure that may not be immediately detected, if at all.
“Due to lack of IT infrastructure and process for continuous model monitoring, changes in software or image acquisition equipment or protocols can result in model failure that may not be immediately detected, if at all.”
The paper relies on this without support.
Human–machine collaboration is now at the forefront of improving model performance, and to prevent unintended consequences including automation bias.
“Human–machine collaboration is now at the forefront of improving model performance, and to prevent unintended consequences including automation bias.”
The paper relies on this without support.
Broad grouping of underrepresented populations into the “Other” category is common when researchers have small sample sizes.
“We also caution against broad grouping of underrepresented populations into the “Other” category which is common when researchers have small sample sizes.”
The paper relies on this without support.
Mathematical approaches of fairness evaluation (e.g., fairness through unawareness, demographic parity, equalized odds or opportunity) may overlook nuanced biases and fail to address systemic issues.
“We urge caution on relying on only mathematical approaches of fairness evaluation (e.g. relying solely on fairness through unawareness, demographic parity, or equalized odds or opportunity), as these approaches may overlook nuanced biases and fail to address systemic issues.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“AI systems may also produce outputs that are skewed by biases related to race, ethnicity, sex, religion, politics, and other factors. [5, 23, 53–56]”
From Autonomous artificial intelligence, scientific research, and human values (2026) · cites this paper as [53]
In the citing paper
“One of the first strategies that can be used to identify bias related to the dataset is dataset evaluation against a set of predefined criteria (searching for exclusion bias, selection bias, recall bias, observer bias, and prejudice bias) and comprehensive data analysis.47”
From Bias in artificial intelligence for medical imaging: fundamentals, detection, avoidance, mitigation, challenges, ethics, and prospects (2024) · cites this paper as [47]
In the citing paper
“Statistical comparison of model output according to different patient groups or confounders that may exist in the training dataset, such as the gender or age of patients. Potential discrepancies in group results could indicate a source of bias that can affect the final results.48”
From Bias in artificial intelligence for medical imaging: fundamentals, detection, avoidance, mitigation, challenges, ethics, and prospects (2024) · cites this paper as [48]
Citations
Cites
Dissecting racial bias in an algorithm used to manage the health of populations
Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019) · Science · doi:10.1126/science.aax2342 · Cited by 7,137 · cited 1×
“BMJ Health Care Inform 2021; 28(1): e100289.”
A Survey on Bias and Fairness in Machine Learning
Mehrabi, N., Morstatter, F., Saxena, N., Lerman, K., & Galstyan, A. (2022) · ACM Computing Surveys · doi:10.1145/3457607 · Cited by 5,812 · cited 1×
“18,19 Comprehensive AI data sets are limited and expensive to curate.”
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · cited 1×
“For example, a developer will have to decide between aiming for group fairness (Black vs white patients, patients on government healthcare programs, or a specific age group) vs individual fairness 49 (where two individuals with similar characteristics are treated in the same manner).”
Equality of Opportunity in Supervised Learning
Hardt, M., Price, E., & Srebro, N. (2016) · arXiv · doi:10.48550/arxiv.1610.02413 · Cited by 2,536 · cited 2×
“50 Attempts to be race agnostic or race neutral are challenging because it remains difficult to remove race and race proxies like pattern of health service utilization because of patient and/or provider factors.”
Zech, J. R., Badgeley, M. A., Liu, M., Costa, A. B., Titano, J. J., & Oermann, E. K. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002683 · Cited by 1,719 · cited 1×
“20 To overcome this confounding, progress must be made to improve model training, evaluation, and explainability.”
Moons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S. (2019) · Annals of Internal Medicine · doi:10.7326/m18-1377 · Cited by 1,669
Checklist for Artificial Intelligence in Medical Imaging (CLAIM): a guide for authors and reviewers
Mongan, J., Moy, L., & Kahn, C. E. (2020) · Radiology: Artificial Intelligence · doi:10.1148/ryai.2020200029 · Cited by 1,443 · cited 1×
“63,64 We encourage researchers to ensure that their work is reproducible, allowing for independent validation and bias detection.”
STARD 2015: An Updated List of Essential Items for Reporting Diagnostic Accuracy Studies
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C., Glasziou, P., Irwig, L., Lijmer, J. G., Moher, D., Rennie, D., de Vet, H. C. W., et al. (2015) · Radiology · doi:10.1148/radiol.2015151516 · Cited by 1,360
Racial bias in pulse oximetry measurement
Sjoding, M. W., Dickson, R. P., Iwashyna, T. J., Gay, S. E., & Valley, T. S. (2020) · New England Journal of Medicine · doi:10.1056/nejmc2029240 · Cited by 1,114 · cited 1×
“…in clinical parameters such as glomerular filtration rate, pulse oximetry (with undiagnosed hidden hypoxemia for historically underserved patients), 35 differences in ICU severity scoring 36 shape radiology data sets for machine learning through a cascade effect as they affect ordering patterns…”
Identifying and Avoiding Bias in Research
Pannucci, C. J., & Wilkins, E. G. (2010) · Plastic and Reconstructive Surgery · doi:10.1097/prs.0b013e3181de24bc · Cited by 1,079 · cited 1×
“Radiol Artif Intell 2022; 4(3): e210064.”
Research works printed but not linked: 5
- Gebru T , Morgenstern J , Vecchione B , Vaughan JW , Wallach H , Daumé III H . Datasheets for Datasets . arXiv 2018 .
- Garbin C , Rajpurkar P , Irvin J , Lungren MP , Marques O . Structured dataset documentation: a datasheet for CheXpert . arXiv 2021 .
- Chambon P et al . RoentGen: Vision-Language Foundation Model for Chest X-ray Generation . arXiv 2022 .
- Tsipras D , Santurkar S , Engstrom L , Ilyas A , Madry A . From ImageNet to Image Classification: Contextualizing Progress on Benchmarks . arXiv 2020 .
- Yang K , Yau J , Fei-Fei L , Deng J , Russakovsky O . A Study of Face Obfuscation in ImageNet . arXiv 2021 .
Cited by
Sorted by citations
Items 201–279 of 279
Invited Commentary: Unlocking the Secrets of Aging: The Promise of CT Imaging Biomarkers
Pyrros, A. T., & Siddiqui, N. (2025) · Radiographics · doi:10.1148/rg.250090
Progress in fully automated abdominal CT interpretation-an update over the past decade
Batheja, V., & Summers, R. (2025) · Abdominal Radiology · doi:10.1007/s00261-025-05094-5 · Cited by 1
Standards, Policies, Ethical Guidelines and Governance in Artificial Intelligence
Purohit, S., & Arora, R. (2025) · Wiley eBooks · doi:10.1002/9781394275311.ch1
Significance of Artificial Intelligence in Animal Disease Recognition
Kumar, R., Sharma, T., & Veer, K. (2025) · Current Signal Transduction Therapy · doi:10.2174/0115743624362164250630194443
Toward Healthcare (Social) Justice: On the Value of Biases in Healthcare AI
Tiribelli, S. (2025) · The International Library of Bioethics · doi:10.1007/978-3-031-94690-5_3
Taylor, S. A., & Nisbet, A. (2025) · British Journal of Radiology · doi:10.1093/bjr/tqaf185
Ujager, F. S., Hamacha, S. R., Imam, N. M., & Farhan, R. B. M. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-1414-3.ch007
The role of artificial intelligence in early detection and intervention of coronary artery disease
Butt, F. R., Chu, M. W. A., & Nedadur, R. (2025) · Current Opinion in Cardiology · doi:10.1097/hco.0000000000001237
Insertion of hepatic lesions into clinical photon-counting-detector CT projection data
Gong, H., Kharat, S., Wellinghoff, J., et al. (2025) · Physics in Medicine and Biology · doi:10.1088/1361-6560/ae0975
Analysis of AI-Bias in Modern Healthcare Systems
Kumar, A., & Singh, D. (2025) · Transactions on computer systems and networks · doi:10.1007/978-981-96-6703-1_14
Kane, O., & Wu, A. W. (2025) · Journal of Patient Safety and Risk Management · doi:10.1177/25160435251386506
From One-Size-Fits-All to Individualized Support
Kayyali, M. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-2948-2.ch007
A State of the Art on Applications, Recent Challenges, and Future Trends of Artificial Intelligence
Kumari, S. S., & Priyadharsini, S. S. (2025) · International Conference on Power, Instrumentation, Control and Computing (PICC) · doi:10.1109/picc67314.2025.11291239 · Cited by 1
Investigating Gender Bias Using Artificial Intelligence Classification Models on RAVDESS Dataset
Sefara, T. J., Khosa, M., & Kisten, M. (2026) · Lecture notes in networks and systems · doi:10.1007/978-3-032-11521-8_17
Ethical Considerations in AI-Enabled Healthcare
Aldemir, E., Bloom, C., Honeyman, S. I., & Vardanyan, R. (2026) · Biomedical visualization · doi:10.1007/978-3-032-06520-9_5
Design guidelines for the therapeutic use of AI in grief
Franco, M. I. B. (2026) · Journal of Responsible Technology · doi:10.1016/j.jrt.2026.100148 · Cited by 1
ChatGPT as a Catalyst for AI Literacy and Attitude Shifts in Diverse Educational Contexts
Grassini, S., Dietrich, L. K., & Cipriani, E. (2026) · Springer eBooks · doi:10.1007/978-981-95-4871-2_11 · Cited by 1
Ibrahim, B. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbaf729
Stella, L., Thompson, E., Margetts, L., et al. (2026) · Journal of Ethics in Entrepreneurship and Technology · doi:10.1108/jeet-06-2025-0043
Audisio, A., ZHU, T. Y., Joeris, A., Sermon, A., IJpma, F. F. A., Giordano, V., Desai, D., Giannoudis, P. V., & Aprato, A. (2026) · Injury · doi:10.1016/j.injury.2026.113056 · Cited by 2
From automation to authenticity: blockchain as a remedy to AI-enhanced social media activism
Kassen, M. (2026) · AI & Society · doi:10.1007/s00146-026-02867-4 · Cited by 1
Equitable AI in Healthcare: Navigating Sex, Gender, and Intersectional Biases in Diagnostics
Lee, H., Yang, S., & Chu, Y. (2026) · Springer eBooks · doi:10.1007/978-981-95-2070-1_10
Shah, N., Sanford, E. L., Busch, D. R., Singh, R., Mathur, S., Sharma, J., Reeder, P., Natarajan, S., & Raman, L. (2026) · Information · doi:10.3390/info17020135
Mandalapu, S. A., & Stawicki, S. P. (2026) · Artificial intelligence · doi:10.5772/intechopen.1013390 · Cited by 1
Byrd, A. R., & Geheran, M. P. (2026) · Journal of Computing in Civil Engineering · doi:10.1061/jccee5.cpeng-6649 · Cited by 3
Dubey, A., Gibson, J. G., Chapala, S., & Botchu, R. (2026) · Springer eBooks · doi:10.1007/978-981-96-7448-0_34-1
Ojo, A., Osborne, A., Soladoye, A. A., Matope, E. M., Bamidele, D. B., & Olawade, D. B. (2026) · Intelligent Hospital · doi:10.1016/j.inhs.2026.100053
Revisiting overlapped fingerprint separation methods and their key requirements for forensic systems
May, R., Filax, M., Theuerkauf, F., et al. (2026) · Forensic Science International Reports · doi:10.1016/j.fsir.2026.100464
Consensus on the application of generative artificial intelligence in medical manuscript writing
Huang, G., Zhong, R., Orsini, G., Ogawa, R., Kong, J., Zhang, J., Lei, M., Wang, S., Zhan, Y., Sjöberg, F., et al. (2026) · Regenesis repair rehabilitation. · doi:10.1016/j.rerere.2026.02.001
The Role of Artificial Intelligence in Cardiothoracic Surgery
Copeland, J., Farooq, T., Kuang, X., Fayazi, S., Asham, F., & Blehm, T. (2026) · Artificial intelligence · doi:10.5772/intechopen.1010693
KI in der Innovation – von der Effizienzsteigerung zur Transformation
Mandl, T. V., & Cao, H. Y. (2025) · Springer eBooks · doi:10.1007/978-3-658-46399-1_11-1
Chew, B., & Ngiam, K. Y. (2026) · JMIR AI · doi:10.2196/80340
Cox, J. A., Ivory, B. E., Salmon, P. M., & Read, G. J. M. (2026) · AI & Society · doi:10.1007/s00146-026-02949-3
Vijayasimha, M., & Srikanth, M. (2026) · International Journal of Legal Medicine · doi:10.1007/s00414-026-03769-2
Leveraging artificial intelligence for equitable women’s health outcomes through imaging
Ndirangu, B., Newsome, J. M., Chavoshi, M., Trivedi, H., & Gichoya, J. W. (2026) · British Journal of Radiology · doi:10.1093/bjr/tqag035 · Cited by 10
Self-citationDirksen, V., & Marella, V. (2026) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-981-95-4423-3_7
KI und Gerechtigkeit in der Zivilgesellschaft
Hassler, C., Nikolajevic, N., Obendiek, A. S., Pfleiderer, O., & Stettner, J. (2026) · Springer eBooks · doi:10.1007/978-3-658-50335-2_4
Four Sensitive Areas of AI Applications
Nayak, M., Patro, M., Choudhury, O. P., & Satapathy, S. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/406028
From AI Bias to AI by Us: A Case Study from MIT Critical Data
Bielick, C. G., Gameiro, R. R., & Celi, L. A. (2026) · Springer eBooks · doi:10.1007/978-3-032-07830-8_17
Self-citationArtificial Intelligence in Musculoskeletal Imaging: from Diagnostic to Interventions
Dubey, A., Gibson, J. G., Chapala, S., & Botchu, R. (2026) · Springer eBooks · doi:10.1007/978-981-96-7448-0_34-2
Leveraging clinical epidemiology concepts to strengthen machine learning fairness evaluations
Guo, L. L., Arciniegas, S. E., Yan, A. P., TOMLINSON, G. A., Beauchemin, M., Pfohl, S. R., & Sung, L. (2026) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocag041 · Cited by 1
Multi-site distributed training with data protections for PET-based synthetic CT
Jørgensen, K., Partin, L., Ashok, R., et al. (2026) · EJNMMI Physics · doi:10.1186/s40658-026-00868-2
Beyond the algorithm: embedding ethics for trustworthy AI in radiology and oncology
Abadía, M. C., & Goisauf, M. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1756256
Shift happens: a fairness-oriented framework for medical classification under hidden bias
To, M. N. N., Kim, D., Harmanani, M., Wilson, P., Fooladgar, F., Sojoudi, S., Jamzad, A., Abdalla, S., Tsang, T. S. M., Luong, C. L., et al. (2026) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-026-03624-0
Perlmutter, J. W., & Joseph, M. (2025) · Education and human development · doi:10.5772/intechopen.1014025 · Cited by 3
KI in der Innovation – von der Effizienzsteigerung zur Transformation
Mandl, T. V., & Cao, H. Y. (2026) · Springer eBooks · doi:10.1007/978-3-658-49581-7_11
Harzand, A., Ross, E., Weissler, E. H., Zheng, Y., Shah, N. H., Alabi, O., Attia, Z. I., & Beckman, J. A. (2026) · Circulation Population Health and Outcomes · doi:10.1161/hcq.0000000000000146 · Cited by 1
Adopting AI in medical ethics review: A configurational fsQCA study of practitioners’ willingness
Zhang, A., Sheng, A., Chen, H., Hu, N., Li, R., & Yu, Z. (2026) · Digital Health · doi:10.1177/20552076261455240
Responsible AI in financial identity verification and risk mitigation
Bello, A. A., Uzoma, C., Atadoga, S., Ayanrinno, P. D., & Akintola, A. S. (2026) · Discover Sustainability · doi:10.1007/s43621-026-03208-7 · Cited by 2
Joshi, U., & Lanzas, C. (2026) · Antimicrobial Stewardship & Healthcare Epidemiology · doi:10.1017/ash.2026.10747
Seven barriers to the ethical governance of artificial intelligence in defence
Toffert, A.-F., Klotz, K., Roberts, H., Birks, C., & Taddeo, M. (2026) · AI & Society · doi:10.1007/s00146-026-03118-2 · Cited by 1
Gandhi, C. R., Reshmma, V., Priyadarshini, B., Darshana, B., Tarani, V., & Jayanthy, A. K. (2026) · doi:10.1002/9781394358212.ch35
Guo, J., Luo, H., Liu, H., & Chen, H. (2026) · Journal of Broadcasting & Electronic Media · doi:10.1080/08838151.2026.2675497
Artificial intelligence in medical education: promise, caution, and the need for restraint
Han, C. T. Y., Marshall, D., & Burke, E. (2026) · Academic Medicine · doi:10.1093/acamed/wvag175
Söderholm, M., Moberg, A., & Falk, M. (2026) · Melanoma Research · doi:10.1097/cmr.0000000000001110 · Cited by 1
Zeka, B., & Johns, R. (2026) · Financial Planning Review · doi:10.1002/cfp2.70040 · Cited by 1
Regulating digital surgery to balance safety and innovation: a SAGES white paper
Guzman, N. M., Marwaha, J. S., Keller, D. S., Feldman, L. S., Hashimoto, D. A., Madani, A., & Telem, D. A. (2026) · Surgical Endoscopy · doi:10.1007/s00464-026-12875-6 · Cited by 2
Pilla, B., Mohan, A., Elanga, A. E., Dean, L., & McCulloch, H. (2026) · Frontiers in Communication · doi:10.3389/fcomm.2026.1879185
Kurpicz-Briki, M., Ikae, C., Puttick, A. R., Rafnsdóttir, G. L., Haraldsdóttir, R. K., Rigotti, C., Fosch‐Villaronga, E., Sangiuliano, M., Kharas, M. W., & Søraa, R. A. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1795293
Falling short? Mitigating equity gaps and demographic bias in AI-driven molecular diagnostics
Forkert, N. D. (2026) · Expert Review of Molecular Diagnostics · doi:10.1080/14737159.2026.2701359
Beyond the Hype: Risks and Limitations of Blockchain-Driven Social Interaction
Kassen, M. (2026) · Information Technology and Global Governance · doi:10.1007/978-3-032-20770-8_5
A Systematic Review of Bias Detection and Dataset Improvement Methods for Fair AI Systems
Leça, M. D. M., Bento, M., & Santos, R. D. S. (2026) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3803437.3806705
ASDGM: Agriculture Framework for Crops Data Generation and Recommendation
Hossny, E., Awad, A. A., & Omara, F. A. (2026) · doi:10.1109/3scea68071.2026.11603000
Subcategorisation of Data for AI Models in Healthcare: A Case Study in Mammography
Goldring, J. E., Cooke, E. A., van Engen, R. E., Mackenzie, A., Venton, J., Roozemond, C., Thomas, S. A., & Smith, N. A. S. (2026) · Diagnostics · doi:10.3390/diagnostics16152367
Fu, E., Qin, H., Tian, Z., Dong, H., Donghua, Z., Yu, X., & Liu, N. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1752043
The Role of AI in Shaping Organizational Culture and Values
Makinde, O. F. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-5213-8.ch016
Large language models and neoliberal hegemony: an analysis of AI outputs on diversity
Thielicke, V., Weiß, A.-N., & Miltzow, H. (2026) · AI & Society · doi:10.1007/s00146-026-03218-z
Blixt, F. F., & Bjällmark, A. (2026) · Cardiovascular Ultrasound · doi:10.1186/s12947-026-00380-8 · Cited by 1
Strategic Role of Performance Management Systems in Enhancing Employee Growth and Retention
Vishwakarma, K., Saxena, V., Khanchandani, A., et al. (2026) · Contributions to management science · doi:10.1007/978-3-032-23399-8_29
Prompting generative artificial intelligence to create plain language patient education materials
Frey, S. M., Wenderlich, A. M., Craig, C. N., Yin, H. S., & Montes, G. (2026) · Patient Education and Counseling · doi:10.1016/j.pec.2026.109832
Cabrera-Salcedo, S. C., Parra‐Lara, L. G., Fernandes-Pineda, M., Toro-Pedroza, A., Hincapie, J. S. V., Ríos-Serna, L. J., Loukanov, A. R., Ortiz‐Guzman, J., García-Robledo, J. E., Cañas, C. A., et al. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1875553
Rafnsdóttir, G. L., Marrero, Á. S., Rigotti, C., Haraldsdóttir, R. K., & Fosch‐Villaronga, E. (2026) · Gender Work and Organization · doi:10.1111/gwao.70239
Kowalska, A., Nowak, P., Zielinski, T., & Mazur, K. (2023) · Journal of Health Informatics and Digital Systems · doi:10.68159/u679026846
FORETHICS: From Principles to Operational Decisions in Forensic Research
Nioi, M., Chighine, A., & d’Aloja, E. (2026) · Forensic Sciences · doi:10.3390/forensicsci6030077
Artificial intelligence in biomedical visualization and education: a narrative review
Wang, X., Song, L., Liu, B., Chen, J., & Zhao, H. (2026) · Frontiers in Public Health · doi:10.3389/fpubh.2026.1956708
GENERATIVE AI IN CENTRAL ASIAN HIGHER EDUCATION: INSIGHTS FROM CHATGPT
Kamenova, S. (2026) · Bulletin of Zhetysu University named after I Zhansugurov · doi:10.53355/zhu.2026.120.3.004
Advancing Health Equity through Multi-Level Fairness in Health Informatics
Souligne, N., & Subbian, V. (2026) · arXiv · doi:10.48550/arxiv.2608.16902
Metadata
- Authors
- Gichoya, J. W., Thomas, K., Celi, L. A., Safdar, N., Banerjee, I., Banja, J. D., Seyyed-Kalantari, L., Trivedi, H., & Purkayastha, S.
- Affiliations
- Emory University (US)
- Beth Israel Deaconess Medical Center (US)
- Arizona State University (US)
- York University (CA)
- Indiana University – Purdue University Indianapolis (US)
- Corresponding author
- Gichoya, Judy Wawira
- Purkayastha, Saptarshi
- ORCID
- Venue
- The British Journal of Radiology
- Publisher
- Oxford University Press (OUP)
- Published
- 2023-09-25
- Volume
- 96
- Issue
- 1150
- Pages
- 20230023-20230023
- ISSN
- 0007-1285 (ISSN-L)
- 1748-880X
- PMC
- PMC10546443
- OpenAlex
- W4386624058
- Semantic Scholar
- ec14f2eda005b47d84f2be953a70ed5981fcc1aa
- Cited by
- 277
- References
- 66
- Citations by year
- 2026: 88 · 2025: 118 · 2024: 67 · 2023: 4
- FWCI
- 3.25
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc
- Locations
- British Journal of Radiology (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Funders
- National Science Foundation (1928481)
- Radiological Society of North America (#EIHD2204)
- National Institutes of Health (75N92020C00021, 5r01eb017205-07, 75N92020C00008, R01-EB017205, EB017205)
- National Heart, Lung, and Blood Institute (75N92020C00008, 75N92020C00021)
- National Institute of Biomedical Imaging and Bioengineering (75N92020C00021, R01-EB017205, EB017205, 75N92020C00008)
- National Institute on Minority Health and Health Disparities
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- algorithmic bias, bias mitigation, artificial intelligence in radiology, AI lifecycle, model training, model deployment, algorithmic fairness
- MeSH terms
- Artificial Intelligence, Humans, Learning, Radiology, Bias, Disease Progression
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
