The medical algorithmic audit
Open accessThe DOI matches its Crossref record; 44 of the 45 references it cites were found in a registry or an index; 13 of the 30 sources its claims cite were checked for support.
Abstract
Artificial intelligence systems for health care, like any other medical device, have the potential to fail. However, specific qualities of artificial intelligence systems, such as the tendency to learn spurious correlates in training data, poor generalisability to new deployment settings, and a paucity of reliable explainability mechanisms, mean they can yield unpredictable errors that might be entirely missed without proactive investigation. We propose a medical algorithmic audit framework that guides the auditor through a process of considering potential algorithmic errors in the context of a clinical task, mapping the components that might contribute to the occurrence of errors, and anticipating their potential consequences. We suggest several approaches for testing algorithmic errors, including exploratory error analysis, subgroup testing, and adversarial testing, and provide examples from our own work and previous studies. The medical algorithmic audit is a tool that can be used to better understand the weaknesses of an artificial intelligence system and put in place mechanisms to mitigate their impact. We propose that safety monitoring and medical algorithmic auditing should be a joint responsibility between users and developers, and encourage the use of feedback mechanisms between these groups to promote learning and maintain safe deployment of artificial intelligence systems.
Overview
CoChat Summary
TL;DR
This Viewpoint proposes a "medical algorithmic audit" framework, adapting Raji and colleagues' SMACTR internal audit process (scoping, mapping, artifact collection, testing, reflection) and failure modes and effects analysis (FMEA) for clinical AI systems. It argues that AI errors differ from human errors and that proactive, structured error investigation is needed before and during deployment.
Key findings
- The authors define algorithmic errors as any inaccurate outputs, including outputs that are correct but produced via a flawed decision-making process, and define a failure mode as a systematic tendency to malfunction under certain conditions.
- In the illustrative hip fracture detection audit, the overall error rate was 2.5%, but the error rate for the subset of cases with abnormal bones or joints (Paget's disease and femoral head deformity) was 50%.
- In the pneumothorax detection example, the model achieved an overall AUC of 0.87, but AUC was 0.94 for cases with chest drains versus 0.77 for cases without, indicating the model learned artifacts of clinical care rather than the pathology.
- The paper reports that 60–70% of human errors in medical imaging are related to perceptual failure, whereas AI is not susceptible to incomplete searches or satisfaction syndrome, so equal-performance AI systems can be expected to produce different errors and clinical outcomes.
Methods
- Design: a conceptual Viewpoint article proposing a framework, not an empirical study; it adapts the SMACTR framework (scoping, mapping, artifact collection, testing, reflection) and applies FMEA principles for risk assessment, prioritisation, and mitigation.
- The framework is presented from two perspectives—developers (who can modify the system) and users (who cannot modify it but can implement deployment-specific risk mitigation)—with a table of actions for each audit stage.
- Testing is described as comprising exploratory error analysis, subgroup testing (patient-specific, task-specific, and exploratory-error-discovered subgroups), and adversarial testing; illustrative examples are drawn from published studies (hip fracture audit, pneumothorax detection, diabetic retinopathy evaluation by Ting and colleagues).
Limitations
- The paper states that FMEA "must be interpreted with care due to issues around reproducibility and incompleteness," and that the risk priority number is not a measure of safety and should not be used with arbitrary acceptability thresholds.
- The paper states that subgroup analysis is not performed to test hypotheses, and that reduced power in subsets plus inflated type 1 error from multiple testing mean results "should not be considered reliable or definitive" and should be confirmed in a sufficiently powered sample.
- The paper states that who should conduct the medical algorithmic audit remains an unanswered question, that the required skills are not taught in combination in standard medical curricula, and that appropriately trained clinicians are rare in both the UK and Australia.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
6 Valid · 7 Not valid · 12 Not checked · 54 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“For example, an artificial intelligence system might learn to detect surgical skin markings to diagnose skin cancer, rather than looking for features related to the lesion itself.9”
§ The medical algorithmic audit > Introduction
What the sources say
- Supportedcites [9] matched to “Association between surgical skin markings in dermoscopic images and diagnostic performance of a deep learning convolutional neural network for melanoma recognition” (2019) source
The Discussion (page unavailable) says a CNN may associate surgical skin markings with a melanoma diagnosis, and the study found that markings increased false-positive melanoma classifications. The claim presents this as a possibility, not a proven account of how the CNN made every decision; the paper says the underlying mechanism remains unclear. (LLM, conf=0.88)
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Furthermore, this drop in performance might not be obvious at the aggregate level of typical artificial intelligence testing, but rather in subsets of the target cohort or specific strata within the input data, a concept that has been described as hidden stratification.13”
§ The medical algorithmic audit > Introduction
What the sources say
- Supportedcites [13] matched to “Hidden stratification causes clinically meaningful failures in machine learning for medical imaging” (2020) source
Split into 2 atoms — 2 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Indeed, the analysis of error cases is a minimum reporting requirement of the SPIRIT-AI (Standard Protocol Items: Recommendations for Interventional Trials-Artificial Intelligence) and CONSORT-AI (Consolidated Standards of Reporting Trials-Artificial Intelligence) guidelines for clinical trial protocols and reports of artificial intelligence interventions.7,8”
§ The medical algorithmic audit > Introduction
What the sources say
- Partly supportedcites [7] matched to “Reporting guidelines for clinical trial reports for interventions involving artificial intelligence: the CONSORT-AI extension” (2020) sourceExperimental — accuracy not yet measured
The Results section’s CONSORT-AI item 19 requires reporting the results of any performance-error analysis where applicable, or explaining why none was planned or done. The Discussion describes CONSORT-AI as minimum reporting guidance, but this paper does not establish the corresponding SPIRIT-AI error-analysis requirement. It also does not require investigators to conduct an error analysis in every trial. (LLM, conf=0.86)
- Partly supportedcites [8] matched to “Guidelines for clinical trial protocols for interventions involving artificial intelligence: the SPIRIT-AI extension” (2020) sourceExperimental — accuracy not yet measured
The Abstract and Discussion describe SPIRIT-AI as minimum reporting guidance that includes error cases. But the Monitoring section (SPIRIT-AI item 22) requires protocols to specify plans to identify and analyze performance errors—or explain why there are no plans—not to report a completed analysis. This paper identifies CONSORT-AI as the companion guideline for trial reports but does not establish its error-analysis requirement. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“In the hip fracture detection audit, 20 the discovery of algorithmic errors in cases with abnormal bones prompted an additional labelling exercise of all hips with abnormal bones and joints to find that the error rate in those cases was 50%, compared with 2•5% in the overall dataset (figure 2).”
- Also stated in: § Conclusions · plain
“The overall error rate was 2•5%, but the error rate for this subset was 50%.”
§ The medical algorithmic audit > Subgroup testing
What the sources say
- Could not verifycites [20] matched to “Causality matters in medical imaging” (2020) sourceExperimental — accuracy not yet measured
The Data mismatch sections discuss dataset shift and selection bias, but do not report a hip-fracture detection audit or error rates for abnormal bones and joints. Searches for the audit, the labelling exercise, and the 50% and 2.5% figures found no passage addressing those results; page numbers were not available in our copy. (LLM, conf=0.76)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Conclusions · plain
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“Almost all studies report demographic and non-demographic characteristics of the study population, such as age, sex, ethnicity, socioeconomic status, disease severity, and comorbidities, and some studies also include information on geographical location and study site.”
§ The medical algorithmic audit > Subgroup testing
What the sources say
- Not checked — no full text of the sourcecites [31] matched to “A deep learning system for differential diagnosis of skin diseases” (2020) source
- Could not verifycites [32] matched to “AI for radiographic COVID-19 detection selects shortcuts over signal” (2021) sourceExperimental — accuracy not yet measured
In “Evaluation of models on new hospital systems,” the paper recommends routine reporting of metadata on patient and hospital-system confounds; it does not assess how often other studies report population characteristics. “Shortcuts have a variable effect on generalization” discusses patient sex in the authors’ own datasets. Searches for age, ethnicity, socioeconomic status, disease severity, comorbidities, and demographic reporting found no passage supporting the claim that almost all studies report these characteristics. (LLM, conf=0.75)
- Not matchedcites [33]
The reference list we hold has no entry [33].
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
18 more claims
Create a free account to see all 79 claims and every source verdict.
- It may state two separate claims as one. 23
- Probably not a substantial claim. 52
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Research on AI auditing has focused on evaluating whether specific applications meet predefined industry requirements.”
From Is it all hype? ChatGPT’s performance and disruptive potential in the accounting and auditing industries (2024) · cites this paper as Liu et al. 2022
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“For instance, researchers have created procedures for auditing AI systems used in recruitment (Kazim et al. 2021), online search (Robertson et al. 2018), and medical diagnostics (Liu et al. 2022).”
From Is it all hype? ChatGPT’s performance and disruptive potential in the accounting and auditing industries (2024) · cites this paper as Liu et al. 2022
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ The medical algorithmic audit > Viewpoint Reflection
“These data are taken from a study by Ting and colleagues 31 and show the performance of their artificial intelligence model stratified by the clinical origin of the data.Reproduced with permission from JAMA 2016; 316: 2402-10.”
§ The medical algorithmic audit > References
“32. Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 2016; 316: 2402-10.”
Citations
Cites
Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697
International evaluation of an AI system for breast cancer screening
McKinney, S. M., Sieniek, M., Godbole, V., Godwin, J., Antropova, N., Ashrafian, H., Back, T., Chesus, M., Corrado, G. S., Darzi, A., et al. (2020) · Nature · doi:10.1038/s41586-019-1799-6 · Cited by 3,287
Model Cards for Model Reporting
Mitchell, M., Wu, S., Zaldivar, A., Barnes, P., Vasserman, L., Hutchinson, B., Spitzer, E., Raji, I. D., & Gebru, T. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287596 · Cited by 2,681
Clinically applicable deep learning for diagnosis and referral in retinal disease
De Fauw, J., Ledsam, J. R., Romera-Paredes, B., Nikolov, S., Tomasev, N., Blackwell, S., Askham, H., Glorot, X., O’Donoghue, B., Visentin, D., et al. (2018) · Nature Medicine · doi:10.1038/s41591-018-0107-6 · Cited by 2,639
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Gebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., III, H. D., & Crawford, K. (2021) · Communications of the ACM · doi:10.1145/3458723 · Cited by 2,469
Ting, D. S. W., Cheung, C. Y.-L., Lim, G., Tan, G. S. W., Quang, N. D., Gan, A., Hamzah, H., Garcia-Franco, R., San Yeo, I. Y., Lee, S. Y., et al. (2017) · JAMA · doi:10.1001/jama.2017.18152 · Cited by 2,402
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020) · Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency · doi:10.1145/3351095.3372873 · Cited by 2,245
Liu, X., Faes, L., Kale, A. U., Wagner, S. K., Fu, D. J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30123-2 · Cited by 1,883
Self-citation2013 IEEE 10th International Symposium on Biomedical Imaging
(2013) · doi:10.1109/isbi20048.2013 · Cited by 1,625
Research works printed but not linked: 1
- Topol E. The Topol review: preparing the healthcare workforce to deliver the digital future. National Health Service Health Education England, 2019.
Cited by
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083
“Key principles include the use of participatory methods to design contextualized evaluations that reflect the values of patients that may benefit or be harmed, grounding the evaluation in one or more specific downstream clinical use cases [54, 71], and the use of dataset and model documentation frameworks for transpar…”
BackgroundLarge Language Models Encode Clinical Knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.13138 · Cited by 4,083
Lekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., Weicken, E., Asselbergs, F. W., Prior, F., Collins, G. S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081554 · Cited by 957
Self-citationEthical Considerations of Using ChatGPT in Health Care
Wang, C., Liu, S., Yang, H., Guo, J., Wu, Y., & Liu, J. (2023) · Journal of Medical Internet Research · doi:10.2196/48009 · Cited by 541
“Rigorous testing and regulation of AI algorithms are necessary to protect human health [59].”
Ethical and regulatory challenges of large language models in medicine
Ong, J. C. L., Chang, S. Y.-H., William, W., Butte, A. J., Shah, N. H., Chew, L. S. T., Liu, N., Doshi-Velez, F., Lu, W., Savulescu, J., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00061-x · Cited by 342
Metadata
- Authors
- Liu, X., Glocker, B., McCradden, M. M., Ghassemi, M., Denniston, A. K., & Oakden-Rayner, L.
- Affiliations
- University Hospitals Birmingham NHS Foundation Trust (GB)
- Imperial College London (GB)
- Massachusetts Institute of Technology (US)
- Venue
- The Lancet Digital Health
- Publisher
- Elsevier BV
- Published
- 2022-05
- Volume
- 4
- Issue
- 5
- Pages
- e384-e397
- ISSN
- 2589-7500 (ISSN-L)
- OpenAlex
- W4223506650
- Cited by
- 266
- References
- 44
- Citations by year
- 2026: 67 · 2025: 54 · 2024: 63 · 2023: 61 · 2022: 21
- FWCI
- 3.25
- Type
- journal article
- Language
- en
- Update notices
- erratum (2022-06-01)
- Open access
- gold
- doi.org
- Licence
- cc-by
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by/4.0/ (vor)
- Locations
- The Lancet Digital Health (publishedVersion, cc-by)
- PubMed (publishedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion, cc-by)
- Spiral (Imperial College London) (submittedVersion, cc-by)
- Funders
- Wellcome Trust (200141/Z/15/Z, /Z/15/Z)
- Alan Turing Institute (TU/ASG/R-SPEH-113)
- H2020 European Research Council (757173)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- algorithmic auditing, artificial intelligence in healthcare, algorithmic errors, medical device safety, spurious correlations, generalizability, explainability, adversarial testing, safety monitoring, user-developer collaboration, feedback mechanisms, algorithmic bias
- MeSH terms
- Artificial Intelligence, Delivery of Health Care, Health Facilities
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The medical algorithmic audit. https://app.cochat.ai/papers/10.1016/s2589-7500(22)00003-6
- The paper
- Liu, X., Glocker, B., McCradden, M. M., Ghassemi, M., Denniston, A. K., & Oakden-Rayner, L. (2022). The medical algorithmic audit. The Lancet Digital Health. https://doi.org/10.1016/s2589-7500(22)00003-6
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.