Cited by 122Artificial Intelligence in HealthcareArtificial Intelligence in Healthcare and EducationMachine Learning in Healthcare

Artificial intelligence in the risk prediction models of cardiovascular disease and development of an independent validation screening tool: a systematic review

Open access
Cai, Yue, Cai, Yu-Qing, Tang, Li-Ying, Wang, Yi-Han, Gong, Mengchun, Jing, Tian-Ci, Li, Hui-Jun, Li-Ling, Jesse +4 more·2024·BMC Medicine

Abstract

Background: A comprehensive overview of artificial intelligence (AI) for cardiovascular disease (CVD) prediction and a screening tool of AI models (AI-Ms) for independent external validation are lacking. This systematic review aims to identify, describe, and appraise AI-Ms of CVD prediction in the general and special populations and develop a new independent validation score (IVS) for AI-Ms replicability evaluation. Methods: PubMed, Web of Science, Embase, and IEEE library were searched up to July 2021. Data extraction and analysis were performed for the populations, distribution, predictors, algorithms, etc. The risk of bias was evaluated with the prediction risk of bias assessment tool (PROBAST). Subsequently, we designed IVS for model replicability evaluation with five steps in five items, including transparency of algorithms, performance of models, feasibility of reproduction, risk of reproduction, and clinical implication, respectively. The review is registered in PROSPERO (No. CRD42021271789). Results: In 20,887 screened references, 79 articles (82.5% in 2017–2021) were included, which contained 114 datasets (67 in Europe and North America, but 0 in Africa). We identified 486 AI-Ms, of which the majority were in development ( n = 380), but none of them had undergone independent external validation. A total of 66 idiographic algorithms were found; however, 36.4% were used only once and only 39.4% over three times. A large number of different predictors (range 5–52,000, median 21) and large-span sample size (range 80–3,660,000, median 4466) were observed. All models were at high risk of bias according to PROBAST, primarily due to the incorrect use of statistical methods. IVS analysis confirmed only 10 models as “recommended”; however, 281 and 187 were “not recommended” and “warning,” respectively. Conclusion: AI has led the digital revolution in the field of CVD prediction, but is still in the early stage of development as the defects of research design, report, and evaluation systems. The IVS we developed may contribute to independent external validation and the development of this field.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/s12916-024-03273-7

Cites

191 of 191 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Neural networks and gradient boosting machines exhibit superior discriminative ability over traditional methods in predicting cardiovascular complications in diabetes.

    “Numerous studies have demonstrated that algorithms such as neural networks and gradient boosting machines exhibit superior discriminative ability over traditional methods in predicting cardiovascular complications in diabetes (6–8).”

    From AI-driven cardiovascular risk prediction in patients with diabetes: bridging algorithmic innovation to equitable clinical application (2026) · cites this paper as [7]

    The Background (page not available) says AI-driven cardiovascular risk prediction has shown better performance than traditional models, and the Results identify studies involving type 2 diabetes and neural networks. But our copy does not establish superior discriminative ability for both neural networks and gradient boosting machines specifically when predicting cardiovascular complications in diabetes. (LLM, conf=0.82)

    Experimental — accuracy not yet measured
  • Inadequate representation of Asian and other non-Western groups directly compromises clinical applicability in high-burden regions.

    “A large proportion of studies rely on populations from Europe and North America, with inadequate representation of Asian and other non-Western groups, directly compromising clinical applicability in high-burden regions (6, 7).”

    From AI-driven cardiovascular risk prediction in patients with diabetes: bridging algorithmic innovation to equitable clinical application (2026) · cites this paper as [7]

    The Discussion’s Principal findings say most studies were conducted in Europe and North America, with few in developing countries in Asia and South America and none in Africa. They also note the high CVD burden in low- and middle-income countries and recommend locally tailored models because ethnicity can affect prediction. The paper does not establish that this imbalance directly compromises clinical applicability in those regions; that causal wording is stronger than its conclusion. (LLM, conf=0.83)

    Experimental — accuracy not yet measured
  • A large proportion of diabetes-related cardiovascular risk prediction studies rely on populations from Europe and North America, with inadequate representation of Asian and other non-Western groups.

    “A large proportion of studies rely on populations from Europe and North America, with inadequate representation of Asian and other non-Western groups, directly compromising clinical applicability in high-burden regions (6, 7).”

    From AI-driven cardiovascular risk prediction in patients with diabetes: bridging algorithmic innovation to equitable clinical application (2026) · cites this paper as [7]

    The Discussion (Principal findings) says most included studies were conducted in Europe and North America, with few in developing countries in Asia or South America and none in Africa. It recommends locally tailored models because ethnic heterogeneity can affect prediction. The geographic finding concerns cardiovascular risk models overall, not specifically diabetes-related studies; the paper also does not directly establish that the imbalance compromises clinical applicability. (LLM, conf=0.84)

    Experimental — accuracy not yet measured
  • Existing machine learning models for cardiovascular risk prediction generally carry a high risk of bias.

    “Despite excellent algorithmic results in many studies, systematic evaluations have revealed a high prevalence of risk of bias, primarily stemming from deficiencies in the participant selection domain, outcome measurement domain, and analysis domain (6, 7).”

    From AI-driven cardiovascular risk prediction in patients with diabetes: bridging algorithmic innovation to equitable clinical application (2026) · cites this paper as [7]

    The Results (“Risk of bias”) report that all 486 assessed models were at high risk of bias, supporting the quoted claim. The citing sentence’s explanation of the main deficiencies is incomplete: the paper reports problems in the predictor domain for 83% of models, compared with 33% in the participant domain and 11% in the outcome domain; analysis-domain problems affected 100%. (LLM, conf=0.88)

    Experimental — accuracy not yet measured
  • Even models that perform exceptionally well on development datasets often show substantially lower or uncertain performance in external settings.

    “As a consequence, even models that perform exceptionally well on development datasets often show substantially lower or uncertain performance in external settings, highlighting a critical limitation in generalizability (7, 8).”

    From AI-driven cardiovascular risk prediction in patients with diabetes: bridging algorithmic innovation to equitable clinical application (2026) · cites this paper as [7]

    The Results section (‘Development models and external validation models’; page unavailable) reports that no independent external validations were found. The Discussion (‘Principal findings’; page unavailable) describes uncertain expected performance in independent external settings. These passages support uncertainty about generalizability, but do not show that exceptionally strong development models often perform substantially worse when externally validated. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. [140]Vollmer S, Mateen BA, Bohner G, Király FJ, Ghani R, Jonsson P, Cumbers S, Jonas A, McAllister KSL, Myles P, et al. Machine learning and artificial intelligence research for patient benefit: 20 critical questions on transparency, replicability, ethics, and effectiveness. BMJ. 2020;368:l6927. 10.1136/bmj.l6927. PMC11515850 32198138 · DOI
  2. [183]Directorate-General for Communications Networks, Content and Technology (2019) Ethics guidelines for trustworthy AI https://www.data.europa.eu/doi/10.2759/177365. · DOI

Cited by

Sorted by citations

Authors
Cai, Y., Cai, Y.-Q., Tang, L.-Y., Wang, Y.-H., Gong, M., Jing, T.-C., Li, H.-J., Li-Ling, J., Hu, W., Yin, Z., Gong, D.-X., & Zhang, G.-W.
Affiliations
  • China Medical University (CN)
  • Digital China Health (China) (CN)
  • First Hospital of China Medical University (CN)
Corresponding author
  • Yin, Zhihua
  • Gong, Da-Xin
Venue
BMC Medicine
Published
2024-02-05
Volume
22
Issue
1
Pages
56-56
ISSN
1741-7015 (ISSN-L)
OpenAlex
W4391531696
Cited by
122
References
191
Citations by year
2026: 62 · 2025: 45 · 2024: 15
FWCI
9.58
Type
journal article
Language
en
Open access
Locations
  • BMC Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
National Key Research and Development Program of China (2020YFC2006406)
Topics
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
artificial intelligence, cardiovascular disease, risk prediction model, algorithmic transparency, risk of bias, PROBAST, clinical prediction tool, digital health
MeSH terms
Heart Disease Risk Factors, Prediction Methods, Machine, Africa, Algorithms, Artificial Intelligence, Cardiovascular Diseases, Europe, Humans, Mass Screening, Reproducibility of Results, Risk Assessment, Datasets as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review