Open questions and research gaps for monitoring and updating AI-enabled tools in clinical settings
Open accessAbstract
As the implementation of artificial intelligence (AI)-enabled tools is realized across diverse clinical environments, there is a growing understanding of the need for ongoing monitoring and updating of prediction models. Dataset shift—temporal changes in clinical practice, patient populations, and information systems—is now well-documented as a source of deteriorating model accuracy and a challenge to the sustainability of AI-enabled tools in clinical care. While best practices are well-established for training and validating new models, there has been limited work developing best practices for prospective validation and model maintenance. In this paper, we highlight the need for updating clinical prediction models and discuss open questions regarding this critical aspect of the AI modeling lifecycle in three focus areas: model maintenance policies, performance monitoring perspectives, and model updating strategies. With the increasing adoption of AI-enabled tools, the need for such best practices must be addressed and incorporated into new and existing implementations. This commentary aims to encourage conversation and motivate additional research across clinical and data science stakeholders.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Ongoing monitoring is essential because the performance and effect(s) of AI models can change after deployment.
“Ongoing monitoring is essential because the performance and effect(s) of AI models can change after deployment 27,28 .”
From Standing on FURM Ground: A Framework for Evaluating Fair, Useful, and Reliable AI Models in Health Care Systems (2024) · cites this paper as [28]
AI models must be monitored after deployment because their performance and impact are likely to change over time.
“AI models must be monitored after deployment because their performance and impact are likely to change over time 28 .”
From Standing on FURM Ground: A Framework for Evaluating Fair, Useful, and Reliable AI Models in Health Care Systems (2024) · cites this paper as [28]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures
Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., Pencina, M. J., & Kattan, M. W. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181c30fb2 · Cited by 5,170 · see citing works
A survey on concept drift adaptation
Gama, J., Žliobaitė, I., Bifet, A., Pechenizkiy, M., & Bouchachia, A. (2014) · ACM Computing Surveys · doi:10.1145/2523813 · Cited by 3,735 · see citing works
Potential Biases in Machine Learning Algorithms Using Electronic Health Record Data
Gianfrancesco, M. A., Tamang, S., Yazdany, J., & Schmajuk, G. (2018) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2018.3763 · Cited by 1,395 · see citing works
Statistical process control as a tool for research and healthcare improvement
Benneyan, J. C. (2003) · BMJ Quality & Safety · doi:10.1136/qhc.12.6.458 · Cited by 1,052 · see citing works
Luo, W., Phung, D., Tran, T., Gupta, S., Rana, S., Karmakar, C., Shilton, A., Yearwood, J., Dimitrova, N., Ho, T. B., et al. (2016) · Journal of Medical Internet Research · doi:10.2196/jmir.5870 · Cited by 988 · see citing works
A calibration hierarchy for risk models was defined: from utopia to empirical data
Van Calster, B., Nieboer, D., Vergouwe, Y., De Cock, B., Pencina, M. J., & Steyerberg, E. W. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.12.005 · Cited by 910 · see citing works
The Clinician and Dataset Shift in Artificial Intelligence
Finlayson, S. G., Subbaswamy, A., Singh, K., Bowers, J., Kupke, A., Zittrain, J., Kohane, I. S., & Saria, S. (2021) · New England Journal of Medicine · doi:10.1056/nejmc2104626 · Cited by 855 · see citing works
Dataset Shift in Machine Learning
Adams, N. M. (2009) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2009.00624_10.x · Cited by 716 · see citing works
Validation, updating and impact of clinical prediction rules: A review
Toll, D. B., Janssen, K. J. M., Vergouwe, Y., & Moons, K. G. M. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.04.008 · Cited by 584 · see citing works
Predicting Risk of Suicide Attempts Over Time Through Machine Learning
Walsh, C. G., Ribeiro, J. D., & Franklin, J. C. (2017) · Clinical Psychological Science · doi:10.1177/2167702617691560 · Cited by 571 · see citing works
Self-citationSteyerberg, E. W., Borsboom, G. J. J. M., van Houwelingen, H. C., Eijkemans, M. J. C., & Habbema, J. D. F. (2004) · Statistics in Medicine · doi:10.1002/sim.1844 · Cited by 552 · see citing works
Minimum sample size for external validation of a clinical prediction model with a binary outcome
Riley, R. D., Debray, T. P. A., Collins, G. S., Archer, L., Ensor, J., van Smeden, M., & Snell, K. I. E. (2021) · Statistics in Medicine · doi:10.1002/sim.9025 · Cited by 400 · see citing works
Calibration of Risk Prediction Models: Impact on Decision-Analytic Performance
Van Calster, B., & Vickers, A. J. (2015) · Medical Decision Making · doi:10.1177/0272989x14547233 · Cited by 371 · see citing works
Calibration drift in regression and machine learning models for acute kidney injury
Davis, S. E., Lasko, T. A., Chen, G., Siew, E. D., & Matheny, M. E. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocx030 · Cited by 312 · see citing works
Self-citationPaulus, J. K., & Kent, D. M. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0304-9 · Cited by 274 · see citing works
Artificial Intelligence in Health Care: The Hope, the Hype, the Promise, the Peril
Series, T. L. H. S., & National Academy of Medicine (2019) · doi:10.17226/27111 · Cited by 227 · see citing works
Calibrating predictive model estimates to support personalized medicine
Jiang, X., Osl, M., Kim, J., & Lucila Ohno-machado, A. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000291 · Cited by 209 · see citing works
Minimum sample size for external validation of a clinical prediction model with a continuous outcome
Archer, L., Snell, K. I. E., Ensor, J., Hudda, M. T., Collins, G. S., & Riley, R. D. (2021) · Statistics in Medicine · doi:10.1002/sim.8766 · Cited by 198 · see citing works
Implementing Electronic Health Care Predictive Analytics: Considerations And Challenges
Amarasingham, R., Patzer, R. E., Huesch, M. D., Nguyen, N. Q., & Xie, B. (2014) · Health Affairs · doi:10.1377/hlthaff.2014.0352 · Cited by 180 · see citing works
An empirical characterization of fair machine learning for clinical risk prediction
Pfohl, S. R., Foryciarz, A., & Shah, N. H. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103621 · Cited by 164 · see citing works
Detection of calibration drift in clinical prediction models to inform model updating
Davis, S. E., Greevy, R. A., Lasko, T. A., Walsh, C. G., & Matheny, M. E. (2020) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103611 · Cited by 160 · see citing works
Self-citationA closed testing procedure to select an appropriate method for updating prediction models
Vergouwe, Y., Nieboer, D., Oostenbrink, R., Debray, T. P. A., Murray, G. D., Kattan, M. W., Koffijberg, H., Moons, K. G. M., & Steyerberg, E. W. (2016) · Statistics in Medicine · doi:10.1002/sim.7179 · Cited by 159 · see citing works
Hickey, G. L., Grant, S. W., Murphy, G. J., Bhabra, M., Pagano, D., McAllister, K., Buchan, I., & Bridgewater, B. (2013) · European Journal of Cardio-Thoracic Surgery · doi:10.1093/ejcts/ezs584 · Cited by 154 · see citing works
A review of statistical updating methods for clinical prediction models
Su, T.-L., Jaki, T., Hickey, G. L., Buchan, I., & Sperrin, M. (2018) · Statistical Methods in Medical Research · doi:10.1177/0962280215626466 · Cited by 147 · see citing works
Jenkins, D. A., Martin, G. P., Sperrin, M., Riley, R. D., Debray, T. P. A., Collins, G. S., & Peek, N. (2021) · Diagnostic and Prognostic Research · doi:10.1186/s41512-020-00090-3 · Cited by 131 · see citing works
van Walraven, C., Wong, J., & Forster, A. J. (2012) · PubMed · Cited by 117 · see citing works
Riley, R. D., Collins, P. G. S., Ensor, J., Archer, L., Booth, S., Mozumder, S. I., Rutherford, M. J., van Smeden, M., Lambert, P. C., & Snell, K. I. E. (2021) · Statistics in Medicine · doi:10.1002/sim.9275 · Cited by 116 · see citing works
A nonparametric updating method to correct clinical prediction model drift
Davis, S. E., Greevy, R., Fonnesbeck, C., Lasko, T., Walsh, C. G., & Matheny, M. E. (2019) · J. Am. Medical Informatics Assoc. · doi:10.1093/jamia/ocz127 · Cited by 112 · see citing works
Self-citationDynamic models to predict health outcomes: current status and methodological challenges
Jenkins, D. A., Sperrin, M., Martin, G., & Peek, N. (2018) · Diagnostic and Prognostic Research · doi:10.1186/s41512-018-0045-2 · Cited by 96 · see citing works
Guo, L. L., Pfohl, S. R., Fries, J., Posada, J., Fleming, S. L., Aftandilian, C., Shah, N., & Sung, L. (2021) · Applied Clinical Informatics · doi:10.1055/s-0041-1735184 · Cited by 96 · see citing works
Prognostic models will be victims of their own success, unless…
Lenert, M. C., Matheny, M. E., & Walsh, C. G. (2019) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocz145 · Cited by 90 · see citing works
Self-citationA framework for making predictive models useful in practice
Jung, K., Kashyap, S., Avati, A., Harman, S. M., Shaw, H., Li, R., Smith, M. A., Shum, K., Javitz, J., Vetteth, Y., et al. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa318 · Cited by 82 · see citing works
Prospective Validation of an Electronic Health Record–Based, Real-Time Suicide Risk Model
Walsh, C. G., Johnson, K. B., Ripperger, M., Sperry, S., Harris, J., Clark, N., Fielstein, E., Novak, L., Robinson, K., & Stead, W. W. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.1428 · Cited by 78 · see citing works
Self-citationMinne, L., Eslami, S., de Keizer, N., de Jonge, E., de Rooij, S. E., & Abu-Hanna, A. (2012) · Intensive Care Medicine · doi:10.1007/s00134-011-2390-2 · Cited by 69 · see citing works
Petersen, C. C., Smith, J. R. L., Freimuth, R. R., Goodman, K. W., Jackson, G. P., Kannry, J. L., Liu, H., Madhavan, S., Sittig, D. F., & Wright, A. T. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa319 · Cited by 66 · see citing works
Ross, E. L., Zuromski, K. L., Reis, B. Y., Nock, M. K., Kessler, R. C., & Smoller, J. W. (2021) · JAMA Psychiatry · doi:10.1001/jamapsychiatry.2021.0089 · Cited by 65 · see citing works
Quantification of Sepsis Model Alerts in 24 US Hospitals Before and During the COVID-19 Pandemic
Wong, A., Cao, J., Lyons, P. G., Dutta, S., Major, V. J., Ötleş, E., & Singh, K. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.35286 · Cited by 57 · see citing works
Luijken, K., Wynants, L., van Smeden, M., CALSTER, B. V., Steyerberg, E. W., Groenwold, R. H. H., Timmerman, D., Bourne, T., & Ukaegbu, C. I. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.11.001 · Cited by 53 · see citing works
Lu, J., Callahan, A., Patel, B., Morse, K., Dash, D., Pfeffer, M. A., & Shah, N. H. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.27779 · Cited by 52 · see citing works
Calibration Drift Among Regression and Machine Learning Models for Hospital Mortality.
Davis, S. E., Lasko, T. A., Chen, G., & Matheny, M. E. (2017) · PubMed Central · Cited by 50 · see citing works
Self-citationDavis, S. E., Greevy, R. A., Lasko, T. A., Walsh, C. G., & Matheny, M. E. (2019) · PubMed Central · Cited by 24 · see citing works
Self-citationEslami, S., de Keizer, N. F., de Jonge, E., de Rooij, S. E., Abu–Hanna, A., & Minne, L. (2012) · Methods of Information in Medicine · doi:10.3414/me11-02-0044 · Cited by 21 · see citing works
Monitoring Approaches for a Pediatric Chronic Kidney Disease Machine Learning Model
Morse, K., Brown, C., Fleming, S. L., Todd, I., Powell, A., Russell, W. A., Scheinker, D., Sutherland, S. M., Lu, J., Watkins, B., et al. (2022) · Applied Clinical Informatics · doi:10.1055/s-0042-1746168 · Cited by 7 · see citing works
Setting the agenda: an informatics-led policy framework for adaptive CDS
Smith, J. R. L. (2020) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa239 · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 4
- [22]Otles E Oh J Li B Bochinski M Joo H Ortwine J Mind the performance gap: examining dataset shift during prospective validation. Proceedings of the 6th Machine Learning for Healthcare Conference Proceedings of Machine Learning Research (2021). pp. 506– 34. PMC13213727 42211431
- [29]Selbst A Boyd D Friedler S Venkatasubramanian S Vertesi J., Fairness and abstraction in sociotechnical systems. ACM Conference of Fairness, Accountability, and Transparency New York, NY: Association for Computing Machinery (2019). pp. 59–68.
- [31]Beutel A Chen J Doshi T Qian H Woodruff A Luu C Putting fairness principles into practice: challenges, metrics, and improvements. 2019 AAAI/ACM Conference on AI, Ethics, and Society New York, NY: Association for Computing Machinery (2019). pp. 453– 9.
- [42]Bifet A Gavalda R. Learning from time-changing data with adaptive windowing. Proceedings of the 2007 SIAM International Conference on Data Mining. Philadephia, PA: Society for Industrial and Applied Mathematics (2007). pp. 443–8.
Cited by
Sorted by citations
Izadi, S., & Forouzanfar, M. (2024) · AI · doi:10.3390/ai5020041 · Cited by 102 · see citing works
Monitoring performance of clinical artificial intelligence in health care: a scoping review
Andersen, E. S., Birk-Korch, J. B., Hansen, R. S., Fly, L. H., Röttger, R., Arcani, D. M. C., Brasen, C. L., Brandslund, I., & Madsen, J. S. (2024) · JBI Evidence Synthesis · doi:10.11124/jbies-24-00042 · Cited by 73 · see citing works
Callahan, A., McElfresh, D., Banda, J. M., Bunney, G., Char, D., Chen, J., Corbin, C. K., Dash, D., Downing, N. L., Jain, S. S., et al. (2024) · NEJM Catalyst · doi:10.1056/cat.24.0131 · Cited by 61 · see citing works
Brown, K. E., Yan, C., Li, Z., Zhang, X., Collins, B. X., Chen, Y., Clayton, E. W., Kantarcıoğlu, M., Vorobeychik, Y., & Malin, B. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf038 · Cited by 64 · see citing works
Sustainable deployment of clinical prediction tools—a 360° approach to model maintenance
Davis, S. E., Embí, P. J., & Matheny, M. E. (2024) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocae036 · Cited by 46 · see citing works
Self-citationNot All Clinical AI Monitoring Systems Are Created Equal: Review and Recommendations
Feng, J., Xia, F., Singh, K., & Pirracchio, R. (2025) · NEJM AI · doi:10.1056/aira2400657 · Cited by 28 · see citing works
Artificial Intelligence (AI): A Potential Game Changer in Regenerative Orthopedics—A Scoping Review
Vaishya, R., Dhall, S., & Vaish, A. (2024) · Indian Journal of Orthopaedics · doi:10.1007/s43465-024-01189-1 · Cited by 21 · see citing works
Differences in technical and clinical perspectives on AI validation in cancer imaging: mind the gap!
Chouvarda, I., Colantonio, S., Verde, A. S. C., Jiménez-Pastor, A., Cerdá-Alberich, L., Metz, Y., Zacharias, L., Nabhani‐Gebara, S., Bobowicz, M., Tsakou, G., et al. (2025) · European Radiology Experimental · doi:10.1186/s41747-024-00543-0 · Cited by 18 · see citing works
Loaiza‐Bonilla, A., Thaker, N. G., Chung, C. D., Parikh, R. B., Stapleton, S., & Borkowski, P. (2025) · American Society of Clinical Oncology Educational Book · doi:10.1200/edbk-25-100048 · Cited by 16 · see citing works
Li, J., Tseng, Y., Chen, S.-H., & Chen, K. (2025) · Biomedical Journal · doi:10.1016/j.bj.2025.100929 · Cited by 14 · see citing works
The Promise of Artificial Intelligence-Assisted Point-of-Care Ultrasonography in Perioperative Care
Serrano, R. A., & Smeltz, A. M. (2024) · Journal of Cardiothoracic and Vascular Anesthesia · doi:10.1053/j.jvca.2024.01.034 · Cited by 11 · see citing works
Ethics framework for predictive clinical AI model updating
Pruski, M. (2023) · Ethics and Information Technology · doi:10.1007/s10676-023-09721-x · Cited by 10 · see citing works
Salwei, M. E., Davis, S. E., Reale, C., Novak, L. L., Walsh, C. G., Beebe, R., Nelson, S. D., Sundrani, S. C., Rose, S. L., Wright, A. T., et al. (2025) · JAMIA Open · doi:10.1093/jamiaopen/ooaf136 · Cited by 10 · see citing works
Self-citationNot the Models You Are Looking For: Traditional ML Outperforms LLMs in Clinical Prediction TasksPreprint
Brown, K. E., Yan, C., Li, Z., Zhang, X., Collins, B. X., Chen, Y., Clayton, E. W., Kantarcıoğlu, M., Vorobeychik, Y., & Malin, B. (2024) · medRxiv · doi:10.1101/2024.12.03.24318400 · Cited by 6 · see citing works
Saad, M., Almasri, W., Hye, T., Roni, M. A., & Mohiyeddini, C. (2024) · Algorithms · doi:10.3390/a17100469 · Cited by 6 · see citing works
Kusztos, V. E., & Simonetto, D. A. (2025) · Journal of Clinical and Experimental Hepatology · doi:10.1016/j.jceh.2025.102579 · Cited by 5 · see citing works
Monitoring performance of clinical artificial intelligence: a scoping review protocol
Andersen, E. S., Birk‐Korch, J. B., Röttger, R., Brasen, C. L., Brandslund, I., & Madsen, J. S. (2024) · JBI Evidence Synthesis · doi:10.11124/jbies-23-00390 · Cited by 5 · see citing works
Almasri, W., Saad, M., & Mohiyeddini, C. (2025) · Algorithms · doi:10.3390/a18100612 · Cited by 4 · see citing works
AI in Healthcare: Bridging the Gap between Research and Clinical Implementation
Lanka, S., & Madala, P. (2024) · International Journal of Innovative Science and Research Technology (IJISRT) · doi:10.38124/ijisrt/ijisrt24oct089 · Cited by 4 · see citing works
Marketing and AI-Based Image Generation: A Responsible AI Perspective
Rivas, P., & Liang, Z. (2024) · Smart innovation, systems and technologies · doi:10.1007/978-981-97-5810-4_13 · Cited by 3 · see citing works
Saad, M., Almasri, W., Hye, T., Roni, M. A., & Mohiyeddini, C. (2024) · Research Square · doi:10.21203/rs.3.rs-4676367/v1 · Cited by 2 · see citing works
RetractedXian, X., Wu, S., Fu, Y., Fan, X., Cheng, Y., Zeng, L., Hou, Z., & Chen, Y. (2025) · Frontiers in Public Health · doi:10.3389/fpubh.2025.1644729 · Cited by 1 · see citing works
Wang, T., Chen, R., Liang, M., Ke, H., Wang, B., Ma, Z., Wang, A., Fan, N., Yuan, S. Y., & Zang, L. (2026) · npj Digital Medicine · doi:10.1038/s41746-026-02588-4 · Cited by 1 · see citing works
Brosula, R. J., Corbin, C. K., & Chen, J. H. (2024) · PubMed Central · Cited by 1 · see citing works
Regulação da inteligência artificial na saúde
Dourado, D. D. A. (2024) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/t.5.2024.tde-23042024-111255 · Cited by 2 · see citing works
Salwei, M. E., Davis, S. E., Reale, C., Novak, L. L., Walsh, C. G., Beebe, R., Nelson, S., Sundrani, S. C., Rose, S. L., Wright, A. T., et al. (2025) · medRxiv · doi:10.1101/2025.06.05.25329034 · Cited by 1 · see citing works
Self-citationPersistent Homology as a Diagnostic Lens in Vision-Language Medical Systems
Aguilar-Ortíz, J., Díaz-Parra, O., Ruíz-Vanoye, J. A., Trejo-Macotela, F. R., Vera-Jiménez, M. A., Zamudio-García, V. M., & Domínguez-Mayorga, C. R. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-7315-7.ch005
From bedside to bench: towards clinical predictive AI research that achieves real-world impact
Si, S., Collins, P. G. S., Alderman, J. E., CALSTER, B. V., Wynants, L., Broome, M. R., Daw, P., Denniston, A. K., Ensor, J., & Perry, B. I. (2026) · BMJ Digital Health & AI · doi:10.1136/bmjdh-2026-000044
Augmenting individualized treatment planning via data-driven clinical role model selection
Liao, C.-Y., Keyvanshokooh, E., Pasquel, F. J., & Garcia, G. P. (2026) · Production and Operations Management · doi:10.1177/10591478261470667
Srirujee, S., & Chokchaipermpoonphol, P. (2026) · Journal of Clinical Laboratory Analysis · doi:10.1002/jcla.70335
Johnson, E., Smith, R. P., Brown, L., & Miller, K. (2024) · Journal of Artificial Intelligence for Healthcare Systems · doi:10.68159/h674523636
Post-Deployment Update Triggers for Clinical AI: An Error-Taxonomy Framework for Safe Model Revision
Herrera, L., Rojas, D., & Castro, A. (2024) · Journal of Health Informatics and Digital Systems · doi:10.68159/b329996013
Narvaez, R. A. (2026) · Environmental Disease · doi:10.4103/ed.ed_16_26
- Authors
- Davis, S. E., Walsh, C. G., & Matheny, M. E.
- Affiliations
- Vanderbilt University Medical Center (US)
- Veterans Health Administration (US)
- Corresponding author
- Davis, Sharon E.
- Venue
- Frontiers in Digital Health
- Publisher
- Frontiers Media SA
- Published
- 2022-09-02
- Volume
- 4
- Pages
- 958284-958284
- ISSN
- 2673-253X (ISSN-L)
- PMC
- PMC9478183
- OpenAlex
- W4294167842
- Cited by
- 33
- References
- 44
- Citations by year
- 2026: 6 · 2025: 10 · 2024: 16 · 2023: 1
- FWCI
- 2.03
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Digital Health (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- CW and the Vanderbilt Suicide Attempt and Ideation Likelihood model (VSAIL) were supported by funding from the National Institutes of Health (R01 MH121455; R01 MH120122; R01 MH116269)
- Vanderbilt University Medical Center’
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- clinical prediction model, model updating, model maintenance, distribution shift, performance monitoring, prospective validation, model accuracy degradation, model life cycle
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
