Algorithms on regulatory lockdown in medicine
The DOI matches its Crossref record; 8 of the 9 references it cites were found in a registry or an index.
Abstract
Prioritize risk monitoring to address the “update problem”
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“First, the lack of comprehensive oversight and accountability mechanisms for low-risk health-related AI systems may result in the wide emergence of ineffective, unproven, and potentially harmful AI systems [11].”
From The EU Artificial Intelligence Act (2024): Implications for healthcare (2024) · cites this paper as [11]
In the citing paper
“This means that all current AI applications are static models that can no longer adapt, and therefore, the approved product does not change over time.”
From Introduction to artificial intelligence in ultrasound imaging in obstetrics and gynecology (2020) · cites this paper as [57]
In the citing paper
“Regulatory bodies, including the FDA, currently approve only AI applications with models that have ‘locked’ parameters 56 , 57 .”
From Introduction to artificial intelligence in ultrasound imaging in obstetrics and gynecology (2020) · cites this paper as [57]
Citations
Cites
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501
Adversarial attacks on medical machine learning
Finlayson, S. G., Bowers, J. D., Ito, J., Zittrain, J. L., Beam, A. L., & Kohane, I. S. (2019) · Science · doi:10.1126/science.aaw4399 · Cited by 1,088 · cited 1×
“Further, describing how the distribution of patients may change is not something a maker may be able to do ex-ante because they usually do not know the distribution of the data that the algorithm will be applied to.”
BackgroundA Deep Learning Mammography-based Model for Improved Breast Cancer Risk Prediction
Yala, A., Lehman, C., Schuster, T., Portnoi, T., & Barzilay, R. (2019) · Radiology · doi:10.1148/radiol.2019182716 · Cited by 758
Towards Deep Neural Network Architectures Robust to Adversarial Examples
Gu, S., & Rigazio, L. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1412.5068 · Cited by 622
Discriminative learning for differing training and test distributions
Bickel, S., Brückner, M., & Scheffer, T. (2007) · Proceedings of the 24th international conference on Machine learning · doi:10.1145/1273496.1273507 · Cited by 387
Immune Activation with HIV Vaccines
Fauci, A. S., Marovich, M. A., Dieffenbach, C. W., Hunter, E., & Buchbinder, S. P. (2014) · Science · doi:10.1126/science.1250672 · Cited by 79
Cited by
Items 101–123 of 123
Gerke, S. (2022) · Cambridge University Press eBooks · doi:10.1017/9781108975452.012 · Cited by 2
Self-citationEthical and legal issues in artificial intelligence-based cardiology
Gerke, S. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-90534-3.00034-2 · Cited by 2
Self-citationMinssen, T., & Gerke, S. (2021) · Springer eBooks · doi:10.1007/978-3-662-63530-8_12 · Cited by 1
Self-citationDevelopment of medical applications based on AI models and register data – regulatory considerations
Lähteenmäki, J., Pajula, J., & Antikainen, E. (2022) · Linköping electronic conference proceedings · doi:10.3384/ecp187024 · Cited by 1
“ML based applications are problematic from the regulatory perspective as they may need to be updated when new data comes available [8, 14, 15].”
BackgroundAdaptive Machine Learning Systems in Medicine – More Learner, Less Machine
Weiss, A. P. (2024) · The American Journal of Bioethics · doi:10.1080/15265161.2024.2388740 · Cited by 1
“Citing the work of Babic et al. (2019), they describe the challenge of the ‘update problem’ and the concern that post-deployment learning may indeed create a machine quite different from the one initially approved for safe use.”
Untangling Data in Precision Oncology – A Model for Chronic Diseases?
Fernández, X. M. (2020) · Yearbook of Medical Informatics · doi:10.1055/s-0040-1701985
“Algorithms should be able to detect biases and therefore, they require robust and complete data [24, 25].”
BackgroundAgomoni, G.-M. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108620024.034
Correcting machine learning models using calibrated ensembles with ‘mlensemble’Preprint
Konopka, T. (2021) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/2021.07.26.453832 · Cited by 1
“Other aspects that can build confidence are rigorous testing in appropriate translational settings (Wiens et al. 2019) and the possibility to update imperfect models (Babic et al. 2019; Wu et al. 2021).”
BackgroundAnalysis of Effects of Covid-19 Pandemic on Students’ Academic Performance at Post Graduate Level
Batool, Z., Munir, H., Hashmi, R., Khatoon, N., Saeed, S., & Kashif, M. (2022) · Pakistan Journal of Medical and Health Sciences · doi:10.53350/pjmhs221611431 · Cited by 1
When Is a Change Significant? The Update Problem of Apps in Medical and Behavioral Research
Shachar, C., Gerke, S., Morrell, W., Kirby, A., Cohen, I. G., & Bierer, B. E. (2022) · Ethics & Human Research · doi:10.1002/eahr.500118 · Cited by 1
Self-citationMuhaibes, A. M. H. A. M. (2022) · Journal of Pharmaceutical Negative Results · doi:10.47750/pnr.2022.13.s08.209
Bächtiger, P., Kelshiker, M. A., Moe, M. E. G., Kramer, D. B., & Peters, N. S. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009373234.008
Adaptive Machine Learning as Research: Does the Cure Fit the Disease?
DeCamp, M. W., & Kao, D. P. (2024) · The American Journal of Bioethics · doi:10.1080/15265161.2024.2388742
“The challenges of adaptive machine learning (ML) algorithms have been long recognized (Babic et al. 2019).”
On the Legal Aspects of Responsible AI: Adaptive Change, Human Oversight, and Societal Outcomes
Onitiu, D., Yazdanpanah, V., Chapman, A. P., Gerding, E., Middleton, S. E., & Williams, J. A. (2024) · Lecture notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering · doi:10.1007/978-3-031-71304-0_6 · Cited by 2
A Historic FDA Clearance: Open Source Software and the Making of Tidepool Loop
Edmiston, P. (2024) · Contemporary Endocrinology · doi:10.1007/978-3-031-75352-7_9
Kazemian, P., Korth, H., & Malhotra, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5332131
Artificial intelligence/machine learning in the management of diabetes mellitus
Sonia, T. A., & Sharma, C. P. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-30272-5.00010-x · Cited by 1
Gupta, S. (2026) · Elsevier eBooks · doi:10.1016/b978-0-443-45252-9.00013-x · Cited by 4
Predicting Parkinson’s Disease Machine Learning Approach with Optimized Classifiers
Thayumanavan, T., Divya, A., Divya, A., & Harishkaruppiah, R. (2026) · International Conference on Inventive Computation Technologies (ICICT) · doi:10.1109/icict68280.2026.11510948
“Machine learning (ML) techniques are being frequently employed to analysed biomedical data due to their strength in recognizing complicated relationships in high dimensional data[5].”
AI agents in drug discovery: A review of evolution, applications, and future directions
Das, S., Ferdaus, M. M., & Dam, T. (2026) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2026.109539 · Cited by 2
Al-Hinai, S., Al-Balushi, A., & Al-Maskari, M. (2026) · International Journal of Pharmaceutical and Phytopharmacological Research · doi:10.51847/ni2b8sajli
Ionescu, C. M., Yumuk, E., Schiavo, M., Visioli, A., Paltenghi, M., Latronico, N., Aubouin–Pairault, B., Fiacchini, M., Moussa, K., Muresan, C. I., et al. (2026) · Communications Medicine · doi:10.1038/s43856-026-01833-x
“Traditionaldistinctionsbetweenlockedandadaptive algorithms may not adequately address concept drift, changing patient populations, and post-deployment model updates 47 .”
Hu, X., Liu, X., Wang, T., Liu, X., Yang, G., Huang, P., & Xu, H. (2026) · Biotechnology Advances · doi:10.1016/j.biotechadv.2026.109028 · Cited by 1
Metadata
- Authors
- Babic, B., Gerke, S., Evgeniou, T., & Cohen, I. G.
- Affiliations
- INSEAD (FR)
- Harvard University (US)
- Venue
- Science
- Published
- 2019-12-06
- Volume
- 366
- Issue
- 6470
- Pages
- 1202-1204
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W2991775327
- Cited by
- 123
- References
- 9
- Citations by year
- 2026: 13 · 2025: 18 · 2024: 27 · 2023: 6 · 2022: 19 · 2021: 23 · 2020: 17
- FWCI
- 16.08
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- PhilSci-Archive (University of Pittsburgh) (acceptedVersion)
- bepress Legal Repository (submittedVersion)
- eYLS (Yale Law School) (submittedVersion)
- Funders
- Novo Nordisk Fonden (NNF17SA0027784)
- Topics
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Healthcare Regulation
- Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Toxicology › Pharmacovigilance and Adverse Drug Reactions
- Health Sciences › Nursing › Issues, ethics and legal aspects › Biomedical Ethics and Regulation
- Keywords
- medical regulation, risk monitoring, algorithmic decision-making, medical device regulation
- MeSH terms
- Algorithms, Artificial Intelligence, Risk Management, Biomedical Technology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.