The DOI matches its Crossref record; 76 of the 76 references it cites were found in a registry or an index; 108 of the 519 sources its claims cite were checked for support.
Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review
Open accessAbstract
While the opportunities of ML and AI in healthcare are promising, the growth of complex data-driven prediction models requires careful quality and applicability assessment before they are applied and disseminated in daily practice. This scoping review aimed to identify actionable guidance for those closely involved in AI-based prediction model (AIPM) development, evaluation and implementation including software engineers, data scientists, and healthcare professionals and to identify potential gaps in this guidance. We performed a scoping review of the relevant literature providing guidance or quality criteria regarding the development, evaluation, and implementation of AIPMs using a comprehensive multi-stage screening strategy. PubMed, Web of Science, and the ACM Digital Library were searched, and AI experts were consulted. Topics were extracted from the identified literature and summarized across the six phases at the core of this review: (1) data preparation, (2) AIPM development, (3) AIPM validation, (4) software development, (5) AIPM impact assessment, and (6) AIPM implementation into daily healthcare practice. From 2683 unique hits, 72 relevant guidance documents were identified. Substantial guidance was found for data preparation, AIPM development and AIPM validation (phases 1–3), while later phases clearly have received less attention (software development, impact assessment and implementation) in the scientific literature. The six phases of the AIPM development, evaluation and implementation cycle provide a framework for responsible introduction of AI-based prediction models in healthcare. Additional domain and technology specific research may be necessary and more practical experience with implementing AIPMs is needed to support further guidance.
Overview
CoChat Summary
TL;DR
This scoping review identified 72 guidance documents and mapped their recommendations to six phases of the AI-based prediction model lifecycle, from data preparation through implementation. It found substantial guidance for data preparation, development, and validation but less for software development, impact assessment, and implementation, and highlights remaining gaps.
Key findings
- From 2683 unique hits, 72 relevant guidance documents were included; data extraction yielded 138 keywords mapped to 27 phase-specific topics and 6 phase-overarching topics.
- Substantial guidance was found for phases 1–3 (data preparation, AIPM development, AIPM validation), while later phases—software development, impact assessment, and implementation—received clearly less attention in the scientific literature.
- The review provides a six-phase framework for responsible AIPM introduction and identifies gaps including missing a priori sample size guidance for semi-supervised learning, decision trees/random forests, and convolutional neural networks; limited guidance on combining data sources/modalities, non-binary outcomes, AIPM-specific human–AI interaction, workflow integration, cost-effectiveness, accountability, and long-term maintenance/dataset shift.
- Phase-overarching topics included algorithmic bias, transparency and openness, interpretability, team/stakeholders, security, and risks.
Methods
- Scoping review with a multi-stage screening strategy; searched PubMed, Web of Science, and ACM Digital Library for literature published January 2000 to January 2021, supplemented by snowballing reference lists and consultation with a convenience sample of 14 AI experts.
- Inclusion criteria: documents aimed at individuals directly involved in AIPM development, evaluation, and implementation and providing actionable guidance; exclusions included guidance limited to one medical domain or one AI technique, guidance aimed at governing institutions, pre-2000 documents, prerequisites-only guidance, and non-English documents.
- Two reviewers performed title/abstract and full-text screening, resolving conflicts by consensus; keywords extracted from included documents were mapped to phase-specific and phase-overarching topics and validated with three co-authors.
Limitations
- Stated: the definitions of “actionable” guidance and the six phases are somewhat arbitrary and mainly informed by experience; individual AIPM applications may deviate from the presented structure.
- Stated: the databases and sources mostly contain scientific literature and only English documents were included, potentially biasing results toward academic sources and English-speaking countries; this was mitigated by identifying additional gray literature through expert consultation and citation screening.
- Stated: the consulted expert group was a convenience sample predominantly working in the Netherlands, though it included different occupations, healthcare domains, and academia/industry. Evident: no formal quality appraisal or risk-of-bias assessment of the included guidance documents is reported, and the synthesis is narrative/scoping rather than quantitative.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
42 Valid · 1 Not valid · 103 Not checked · 57 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“These phases are based on the predominant phases in clinical prediction model research4,5.”
§ Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review > Introduction
What the sources say
- Supportedcites [4] Steyerberg, E. W. et al. Prognosis Research Strategy (PROGRESS) 3: prognostic model research. PLoS Med. 10, e1001381 (2013). source
The Summary Points of PROGRESS 3 state that 'Prognostic model research has three main phases: model development (including internal validation), external validation, and investigations of impact in clinical practice', and that most publications describe model development with few external validation and very few impact studies; the Conclusion repeats the three-phase framing. The paper also explicitly equates prognostic models with 'risk (or clinical) prediction model' in its Introduction, so it does treat these as the predominant phases of clinical prediction model research that the citing authors' AIPM phases are said to build on. The number of citing phases (six) is the citing authors' own construction, not something attributed to this source, so no mismatch arises. (LLM, conf=0.85)
- Not checked yetcites [5] Moons, K. G., Altman, D. G., Vergouwe, Y. & Royston, P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ 338, b606 (2009). source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“It should be noted that when the sample size is simply too small, even penalization methods have been shown ineffective to mitigate overfitting83,84.”
§ Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review > Results > Phase 2. Development of the AIPM
What the sources say
- Partly supportedcites [83] Van Calster, B., van Smeden, M., De Cock, B. & Steyerberg, E. W. Regression shrinkage methods for clinical prediction models do not guarantee improved performance: Simulation study. Stat. Methods Med. Res. 29, 3166–3178 (2020). sourceExperimental — accuracy not yet measured
In the section labeled 3.4 (page not available), the paper says shrinkage is particularly variable at low sample sizes and does not justify developing a model with a smaller sample. But it also says shrinkage works on average and Firth’s penalized likelihood almost surely improved performance over maximum likelihood. The claim overstates this as penalization methods being ineffective to mitigate overfitting. (LLM, conf=0.84)
- Partly supportedcites [84] Riley, R. D. et al. Penalization and shrinkage methods produced unreliable clinical prediction models especially when sample size was small. J. Clin. Epidemiol. 132, 88–96 (2021). sourceExperimental — accuracy not yet measured
The Summary says penalization and shrinkage are not a solution to small effective sample sizes, and the Discussion says they may be unreliable when overfitting is greatest. But the Introduction also says penalization may be effective on average; the paper does not establish that it is categorically ineffective at mitigating overfitting in small samples. (LLM, conf=0.86)
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Unlike discrimination and calibration, DCA derives the clinical utility from the predictive performance5,6,17,49,68,72.”
§ Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review > Results > Phase 2. Development of the AIPM
What the sources say
- Not checked yetcites [5] Moons, K. G., Altman, D. G., Vergouwe, Y. & Royston, P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ 338, b606 (2009). source
- Partly supportedcites [6] Luo, W. et al. Guidelines for developing and reporting machine learning predictive models in biomedical research: a multidisciplinary view. J. Med. Internet Res. 18, e323 (2016). sourceExperimental — accuracy not yet measured
The Results section (Table 5, item 10; page unavailable) recommends reporting clinical implications derived from predictive performance. It does not identify decision curve analysis (DCA) as the method for doing so or contrast DCA with discrimination and calibration. (LLM, conf=0.78)
- Supportedcites [17] Steyerberg, E. W. & Vergouwe, Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur. Heart J. 35, 1925–1931 (2014). source
The Introduction’s “Decision-curve analysis” section says calibration and discrimination do not assess clinical usefulness, whereas decision thresholds and net benefit can be used to assess whether a model leads to better decisions. Table 2 identifies decision-curve analysis as the clinical-usefulness measure for model performance. (LLM, conf=0.91)
- Not checked yetcites [49] Steyerberg, E. W. Clinical Prediction Models (Springer Nature, 2009). source
- Supportedcites [68] Kappen, T. H. et al. Evaluating the impact of prediction models: lessons learned, challenges, and recommendations. Diagn. Progn. Res. 2, 11 (2018). source
The Background says discrimination and calibration can indicate predictive performance but do not guarantee improved clinical decisions or outcomes. In “Design of the impact study” (page unavailable), the paper explains that decision-analytic techniques estimate a model’s potential impact using predictive accuracy together with decision thresholds and other assumptions; its cited examples include decision curve analysis. This supports the distinction, provided “clinical utility” means estimated rather than demonstrated benefit. (LLM, conf=0.83)
- Supportedcites [72] Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G. & King, D. Key challenges for delivering clinical impact with artificial intelligence. BMC Med. 17, 195 (2019). source
In “Metrics often do not reflect clinical applicability” (page unavailable), the paper contrasts conventional predictive-performance measures with whether a model benefits patient care. It identifies decision curve analysis as an approach that aims to quantify the net benefit of using a model to guide actions, supporting the cited distinction. (LLM, conf=0.87)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“One commonly referred strategy is feature selection13,15,27,47,76, for which it is explicitly recommended that selection should work independently of model training (unlike in methods as forward and backward selection) and is best informed - a priori - by medical expert knowledge or existing literature13,17,76.”
§ Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review > Results > Phase 2. Development of the AIPM
What the sources say
- Supportedcites [13] Stevens, L. M., Mortazavi, B. J., Deo, R. C., Curtis, L. & Kao, D. P. Recommendations for reporting machine learning analyses in clinical research. Circ. Cardiovasc Qual. Outcomes 13, e006556 (2020). source
The Feature Engineering, Selection, and Transformation section (page not provided) says that excessive variables increase spurious findings, models perform best with less redundancy, and feature selection can eliminate that redundancy. This supports the paraphrase that feature selection is a strategy for reducing overfitting risk. (LLM, conf=0.82)
- Not checked yetcites [15] Food and Drug Administration. Computer-assisted detection devices applied to radiology images and radiology device data—premarket notification [510(k)] submissions. Retrieved from https://www.fda.gov/regulatory-information/search-fda-guidance-documents/computer-assisted-detection-devices-applied-radiology-images-and-radiology-device-data-premarket (2012).
- Partly supportedcites [27] Cearns, M., Hahn, T. & Baune, B. T. Recommendations and future directions for supervised machine learning in psychiatry. Transl. Psychiatry 9, 271 (2019). sourceExperimental — accuracy not yet measured
The Model training and evaluation section recommends careful feature selection to avoid data leakage. The Future goals section explains that high-dimensional data and too many model parameters can lead to overfitting, but our copy does not explicitly describe feature selection as a commonly used strategy to prevent overfitting. (LLM, conf=0.78)
- Not checked yetcites [47] Bhaskar, H., Hoyle, D. C. & Singh, S. Machine learning in bioinformatics: a brief survey and recommendations for practitioners. Comput Biol. Med. 36, 1104–1125 (2006). source
- Not checked yetcites [76] Aliferis, C. F., Statnikov, A. & Tsamardinos, I. Challenges in the analysis of mass-throughput data: a technical commentary from the statistical machine learning perspective. Cancer Inf. 2, 133–162 (2007).
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Similar to internal validation of the AIPM, external AIPM model validation can be based on discrimination (area under the receiver operating characteristic curve, sensitivity, specificity, positive and negative predictive values), calibration (calibration plot)5,6,13,17,49,56,78, and Decision Curve Analysis5,6,17,49,68,72.”
§ Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review > Results > Phase 3. Validation of the AIPM
What the sources say
- Not checked yetcites [5] Moons, K. G., Altman, D. G., Vergouwe, Y. & Royston, P. Prognosis and prognostic research: application and impact of prognostic models in clinical practice. BMJ 338, b606 (2009). source
- Partly supportedcites [6] Luo, W. et al. Guidelines for developing and reporting machine learning predictive models in biomedical research: a multidisciplinary view. J. Med. Internet Res. 18, e323 (2016). sourceExperimental — accuracy not yet measured
The Results checklist recommends external validation when possible and lists sensitivity, specificity, predictive values, area under the ROC curve, and a calibration plot as classification validation metrics. It does not identify Decision Curve Analysis as a validation method, so that part of the claim is unsupported by this source. Page numbers are unavailable in our copy. (LLM, conf=0.87)
- Partly supportedcites [13] Stevens, L. M., Mortazavi, B. J., Deo, R. C., Curtis, L. & Kao, D. P. Recommendations for reporting machine learning analyses in clinical research. Circ. Cardiovasc Qual. Outcomes 13, e006556 (2020). sourceExperimental — accuracy not yet measured
The Study Design section (Determining Measures for Evaluation; page not provided) lists ROC AUC, sensitivity and positive predictive value as supervised-model metrics and discusses calibration in external validation. The External Model Evaluation section also recommends reporting calibration where applicable. Neither passage identifies Decision Curve Analysis or a calibration plot as an external-validation method. (LLM, conf=0.86)
- Supportedcites [17] Steyerberg, E. W. & Vergouwe, Y. Towards better clinical prediction models: seven steps for development and an ABCD for validation. Eur. Heart J. 35, 1925–1931 (2014). source
The Introduction’s “An ABCD for model validation” section proposes measures of calibration, discrimination, and clinical usefulness and illustrates them using external validation. Table 2 identifies a calibration plot, ROC-based discrimination, and decision-curve analysis as corresponding measures. (LLM, conf=0.94)
- Not checked yetcites [49] Steyerberg, E. W. Clinical Prediction Models (Springer Nature, 2009). source
- Not checked yetcites [56] Park, S. H. & Han, K. Methodologic guide for evaluating clinical performance and effect of artificial intelligence technology for medical diagnosis and prediction. Radiology 286, 800–809 (2018). source
- Not checked yetcites [78] Altman, D. G., Vergouwe, Y., Royston, P. & Moons, K. G. M. Prognosis and prognostic research: validating a prognostic model. BMJ 338, b605 (2009). source
- Partly supportedcites [68] Kappen, T. H. et al. Evaluating the impact of prediction models: lessons learned, challenges, and recommendations. Diagn. Progn. Res. 2, 11 (2018). sourceExperimental — accuracy not yet measured
In the Background section, the paper treats discrimination and calibration as measures of predictive performance in a new setting. In “Design of the impact study,” it discusses decision-analytic studies as a way to estimate the potential impact of an already validated model, not as a measure of external validation. The claim therefore folds a separate assessment into external validation. (LLM, conf=0.86)
- Partly supportedcites [72] Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G. & King, D. Key challenges for delivering clinical impact with artificial intelligence. BMC Med. 17, 195 (2019). sourceExperimental — accuracy not yet measured
The 'Metrics often do not reflect clinical applicability' section discusses ROC area, sensitivity, specificity, predictive values, and decision curve analysis as ways to assess model performance or clinical benefit. The 'Challenges in generalisation to new populations and settings' section calls for external validation. Our copy does not identify calibration or calibration plots as a basis for that validation, so it supports only part of the attributed list. (LLM, conf=0.87)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
30 more claims · 3 with their cited sources checked
Create a free account to see all 203 claims and to verify the 411 references still open.
- Probably not a substantial claim. 161
- It may state two separate claims as one. 50
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“the clinical translation of multimodal fusion models requires rigorous validation through real-world trials, covering experimental design, data processing, modeling, testing, and safety assessment (de Hond et al., 2022).”
From Multimodal neuroimaging and AI integration in cognitive disorders: advances, challenges, and future directions for precision medicine (2026) · cites this paper as de Hond et al., 2022
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“As AI solutions have emerged, a wide variety of governance best practices in the form of frameworks, governance structures, checklists, and guidelines have been proposed7–23.”
From Advancing healthcare AI governance through a comprehensive maturity model based on systematic review (2026) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Most articles briefly addressed the importance of clear objectives for AI implementation in healthcare, with only a few offering actionable guidance9,11,14,17,21.”
From Advancing healthcare AI governance through a comprehensive maturity model based on systematic review (2026) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“AI tools that learn from their outputs through feedback loops must be closely monitored, as unchecked adjustments can amplify errors or biases over time14,19.”
From Advancing healthcare AI governance through a comprehensive maturity model based on systematic review (2026) · cites this paper as [14]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Existing strategies for testing and evaluating AI models in the context of the workflows they are part of consist primarily of reporting guidelines and checklists that focus on the properties of the model, including performance metrics, how it will be deployed, and potential outcomes 6,7 .”
From Standing on FURM Ground: A Framework for Evaluating Fair, Useful, and Reliable AI Models in Health Care Systems (2024) · cites this paper as [7]
Experimental — accuracy not yet measured
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
International Organization for Standardization
(2011) · SpringerReference · doi:10.1007/springerreference_16847 · Cited by 12,534
Arrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · cited 8×
“…with a possibility to discuss whether the associations on which the AIPM relies are likely to remain stable 8,27,61 , help to identify algorithmic bias 7,22,24,27,40,43 , provide information on where the AIPM could be most easily attacked 27 , or how the AIPM may behave under dataset shift 7 .”
BackgroundMethodInfluentialThe global landscape of AI ethics guidelines
Jobin, A., Ienca, M., & Vayena, E. (2019) · Nature Machine Intelligence · doi:10.1038/s42256-019-0088-2 · Cited by 6,220 · cited 1×
“This sets our review apart from other work that was previously undertaken (e.g., 32,123,124 ).”
ResultExplanation in artificial intelligence: Insights from the social sciences
Miller, T. (2019) · Artificial Intelligence · doi:10.1016/j.artint.2018.07.007 · Cited by 5,302
Key challenges for delivering clinical impact with artificial intelligence
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1426-2 · Cited by 3,136
Calculating the sample size required for developing a clinical prediction model
Riley, R. D., Ensor, J., Snell, K. I. E., Harrell, F. E., Martin, G. P., Reitsma, J. B., Moons, K. G. M., Collins, G., & van Smeden, M. (2020) · BMJ · doi:10.1136/bmj.m441 · Cited by 2,917 · cited 3×
“For regression-based methods 48 , and a selection of machine-learning-based methods 47 , technique-speci fi c a priori sample size calculations are available, although for many model architectures and settings (e.g., semi-supervised learning, decision trees, or convolutional neural networks) no…”
BackgroundMethodSelf-citationAntlanger, M., Aschauer, S., Kopecky, C., Hecking, M., Kovarik, J. J., Werzowa, J., Mascherbauer, J., Genser, B., Säemann, M. D., & Bonderman, D. (2017) · Kidney and Blood Pressure Research · doi:10.1159/000473868 · Cited by 2,808
Calibration: the Achilles heel of predictive analytics
On behalf of Topic Group ‘Evaluating diagnostic tests and prediction models’ of the STRATOS initiative, Van Calster, B., McLernon, D. J., van Smeden, M., Wynants, L., & Steyerberg, E. W. (2019) · BMC Medicine · doi:10.1186/s12916-019-1466-7 · Cited by 2,650
Self-citationGebru, T., Morgenstern, J., Vecchione, B., Vaughan, J. W., Wallach, H., Daumé, H., & Crawford, K. (2018) · arXiv · doi:10.48550/arxiv.1803.09010 · Cited by 2,474
Research works printed but not linked: 6
- Alonso, O. Challenges with label quality for supervised learning. ACM J. Data Inform. Quality 6, 2 (2015).
- Eggensperger, K., Lindauer, M. & Hutter, F. Pitfalls and best practices in algorithm configuration. J. Artif. Int. Res. 64, 861–893 (2019).
- Kaur, H., Pannu, H. S. & Malhi, A. K. A systematic review on imbalanced data challenges in machine learning: applications and solutions. ACM Comput. Surv. 52, 1–36 (2019).
- Institute of Electrical and Electronics Engineers. IEEE approved draft model process for addressing ethical concerns during system design (IEEE 7000-2021). Retrieved from https://standards.ieee.org/standard/7000-2021.html (2021).
- International Organization for Standardization. Health informatics—Personal health device communication—Part 10418: Device specialization—International Normalized Ratio (INR) monitor (ISO/IEEE 11073-10418:2014). Retrieved from https://www.iso.org/standard/61897.html (2014).
- Food and Drug Administration. Response to NIST workshop and call for position papers on standards and guidelines to enhance software supply chain security. Retrieved from https://www.fda.gov/media/149954/download (2021).
Cited by
Collins, G. S., Moons, K. G. M., Dhiman, P., Riley, R. D., Beam, A. L., Van Calster, B., Ghassemi, M., Liu, X., Reitsma, J. B., van Smeden, M., et al. (2024) · BMJ · doi:10.1136/bmj-2023-078378 · Cited by 4,718
Self-citationLekadir, K., Frangi, A. F., Porras, A. R., Glocker, B., Cintas, C., Langlotz, C. P., Weicken, E., Asselbergs, F. W., Prior, F., Collins, G. S., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081554 · Cited by 957
SKCV: Stratified K-fold cross-validation on ML classifiers for predicting cervical cancer
Prusty, S., Patnaik, S., & Dash, S. K. (2022) · Frontiers in Nanotechnology · doi:10.3389/fnano.2022.972421 · Cited by 248
Binuya, M. A. E., Engelhardt, E. G., Schats, W., Schmidt, M. K., & Steyerberg, E. W. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01801-8 · Cited by 246
Self-citation
Metadata
- Authors
- de Hond, A. A. H., Leeuwenberg, A. M., Hooft, L., Kant, I. M. J., Nijman, S. W. J., van Os, H. J. A., Aardoom, J. J., Debray, T. P. A., Schuit, E., van Smeden, M., Reitsma, J. B., Steyerberg, E. W., Chavannes, N. H., & Moons, K. G. M.
- Affiliations
- Utrecht University (NL)
- Leiden University Medical Center (NL)
- Corresponding author
- Leeuwenberg, Artuur M.
- ORCID
- Venue
- npj Digital Medicine
- Published
- 2022-01-10
- Volume
- 5
- Issue
- 1
- Pages
- 2-2
- ISSN
- 2398-6352 (ISSN-L)
- PMC
- PMC8748878
- OpenAlex
- W4205805845
- Cited by
- 499
- References
- 71
- Citations by year
- 2026: 93 · 2025: 127 · 2024: 136 · 2023: 105 · 2022: 38
- FWCI
- 6.97
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- npj Digital Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Leiden Repository (Leiden University) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Funders
- This research was funded by the Ministry of Health, Welfare and Sport.
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Keywords
- machine learning in healthcare, quality standards, clinical implementation, model validation, data preparation, impact assessment, responsible AI adoption, clinical decision support, software development
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review. https://app.cochat.ai/papers/10.1038/s41746-021-00549-7
- The paper
- de Hond, A. A. H., Leeuwenberg, A. M., Hooft, L., Kant, I. M. J., Nijman, S. W. J., van Os, H. J. A., Aardoom, J. J., Debray, T. P. A., Schuit, E., van Smeden, M., Reitsma, J. B., Steyerberg, E. W., Chavannes, N. H., & Moons, K. G. M. (2022). Guidelines and quality criteria for artificial intelligence-based prediction models in healthcare: a scoping review. npj Digital Medicine. https://doi.org/10.1038/s41746-021-00549-7
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
