Discovery of an Artificial Intelligence Label Feedback Loop: How the Success of a Clinically Implemented Artificial Intelligence Algorithm Has Created an Unforeseen Challenge to Algorithm Retraining
Abstract
Background Artificial intelligence (AI) augmented laboratory tests can improve quality, efficiency, cost-effectiveness, and staff satisfaction. However, the clinical success of these tests can introduce unforeseen challenges for model retraining. This study describes the discovery of an "AI label feedback loop" in a clinically implemented AI-augmented kidney stone composition test. Methods An AI-augmented kidney stone composition test (V1) has been previously deployed for clinical kidney stone characterization. After several years of clinical use, a retrained model (V2) was developed using 6 times more data. Model performance of both V1 and V2 were evaluated across 3 datasets: a recent production validation (hold-out) set (mostly AI-influenced labels), the original V1 validation set (pre-AI, entirely human-labeled), and a subset of recent cases with exclusively human-generated or human-corrected labels. Results V2 demonstrated a 10% lower concordance rate than V1 when evaluated on the recent production hold-out set, despite a much larger training dataset. Performance between V1 and V2 was similar when applied to the pre-AI validation set. Notably, V2 outperformed V1 on the recent subset of cases with human-only or human-corrected labels, particularly for less-common stone types. These findings revealed an AI label feedback loop, confounding retraining and evaluation. Conclusions The integration of AI into clinical practice can potentially influence reported test results, comp…
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Degradation may be linked to how data are produced within evolving workflows and information systems, not only to who the patients are.
“This supports a workflow-aware interpretation of post-deployment fragility: degradation may be linked to how data are produced within evolving workflows and information systems, not only to who the patients are [8,18,33,35].”
From Validation is not enough: Longitudinal evidence of post-deployment fragility in clinical AI systems (2026) · cites this paper as [35]
Cites
AI models collapse when trained on recursively generated data
Shumailov, I., Shumaylov, Z., Zhao, Y., Papernot, N., Anderson, R., & Gal, Y. (2024) · Nature · doi:10.1038/s41586-024-07566-y · Cited by 793 · see citing works
Feng, J., Phillips, R. V., Malenica, I., Bishara, A., Hubbard, A. E., Celi, L. A., & Pirracchio, R. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00611-y · Cited by 423 · see citing works
Data drift in medical machine learning: implications and potential remedies
Sahiner, B., Chen, W., Samala, R. K., & Petrick, N. (2023) · The British Journal of Radiology · doi:10.1259/bjr.20220878 · Cited by 279 · see citing works
Continual learning in medical devices: FDA's action plan and beyond
Vokinger, K. N., Feuerriegel, S., & Kesselheim, A. S. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00076-5 · Cited by 95 · see citing works
Oduoye, M. O., Fatima, E., Muzammil, M. A., Dave, T., Irfan, H., Fariha, F., Marbell, A., Ubechu, S. C., Scott, G. Y., & Elebesunu, E. E. (2024) · Health Science Reports · doi:10.1002/hsr2.1794 · Cited by 76 · see citing works
Rise of the Machines: Artificial Intelligence and the Clinical Laboratory
Haymond, S., & McCudden, C. R. (2021) · The Journal of Applied Laboratory Medicine · doi:10.1093/jalm/jfab075 · Cited by 59 · see citing works
Day, P. L., Erdahl, S. A., Rokke, D. L., Wieczorek, M. A., Johnson, P. W., Jannetto, P. J., Bornhorst, J. A., & Carter, R. E. (2023) · Mayo Clinic Proceedings Digital Health · doi:10.1016/j.mcpdig.2023.01.001 · Cited by 16 · see citing works
Self-citationModel drift: When it can be a sign of success and when it can be an occult problem
Carter, R. E., Anand, V., Harmon, D., & Pellikka, P. A. (2022) · Intelligence-Based Medicine · doi:10.1016/j.ibmed.2022.100058 · Cited by 14 · see citing works
Self-citationSeheult, J. N., Otteson, G. E., Weybright, M. J., Timm, M., Han, W., Jevremović, D., Horna, P., Olteanu, H., & Shi, M. (2025) · Cancers · doi:10.3390/cancers17101688 · Cited by 8 · see citing works
Day, P. L., Rokke, D. L., Schneider, L., Abbott, J., Holmen, B., Johnson, P. W., Wieczorek, M. A., Kunze, K. L., Carter, R. E., Bornhorst, J. A., et al. (2024) · The Journal of Applied Laboratory Medicine · doi:10.1093/jalm/jfae146 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Validation is not enough: Longitudinal evidence of post-deployment fragility in clinical AI systems
Kopanitsa, G. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001534 · Cited by 3 · see citing works
“This supports a workflow-aware interpretation of post-deployment fragility: degradation may be linked to how data are produced within evolving workflows and information systems, not only to who the patients are [8,18,33,35].”
Ahmad, A. A. B., & Bani‐Ahmad, S. (2026) · Molecules · doi:10.3390/molecules31203617
- Authors
- Day, P. L., Wieczorek, M. A., Rokke, D., Lantz, E., Bornhorst, J. A., Jannetto, P. J., & Carter, R. E.
- Affiliations
- Mayo Clinic (US)
- Mayo Clinic in Florida (US)
- Corresponding author
- Day, Patrick L
- Venue
- Clinical Chemistry
- Publisher
- Oxford University Press (OUP)
- Published
- 2026-02-02
- Volume
- 72
- Issue
- 4
- Pages
- 461-466
- ISSN
- 0009-9147 (ISSN-L)
- 1530-8561
- OpenAlex
- W7127382906
- Cited by
- 1
- References
- 10
- Citations by year
- 2026: 1
- FWCI
- 2.01
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Clinical Chemistry (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Innovation Accelerator Awards
- Mayo Clinic Office of Translation to Practice
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Pulmonary and Respiratory Medicine › Kidney Stones and Urolithiasis Treatments
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Keywords
- model retraining, kidney stone composition, model drift, manual annotation, model evaluation, clinical implementation of AI
- MeSH terms
- Intelligent Systems, Algorithms, Artificial Intelligence, Feedback, Humans, Kidney Calculi
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
