CoChat
Sign Up
Cited by 12Artificial Intelligence in HealthcareAcute Ischemic Stroke ManagementMachine Learning in Healthcare

The Value of Applying Machine Learning in Predicting the Time of Symptom Onset in Stroke Patients: Systematic Review and Meta-Analysis

Open access
Feng, Jing, Zhang, Qizhi, Wu, Feng, Peng, Jinxiang, Li, Ziwei, Chen, Zhuang·2023·Journal of Medical Internet Research
DOI 10.2196/44895PMC PMC10603565

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Background: Machine learning is a potentially effective method for identifying and predicting the time of the onset of stroke. However, the value of applying machine learning in this field remains controversial and debatable. Objective: We aimed to assess the value of applying machine learning in predicting the time of stroke onset. Methods: PubMed, Web of Science, Embase, and Cochrane were comprehensively searched. The C index and sensitivity with 95% CI were used as effect sizes. The risk of bias was evaluated using PROBAST (Prediction Model Risk of Bias Assessment Tool), and meta-analysis was conducted using R (version 4.2.0; R Core Team). Results: Thirteen eligible studies were included in the meta-analysis involving 55 machine learning models with 41 models in the training set and 14 in the validation set. The overall C index was 0.800 (95% CI 0.773-0.826) in the training set and 0.781 (95% CI 0.709-0.852) in the validation set. The sensitivity and specificity were 0.76 (95% CI 0.73-0.80) and 0.79 (95% CI 0.74-0.82) in the training set and 0.81 (95% CI 0.68-0.90) and 0.83 (95% CI 0.73-0.89) in the validation set, respectively. Subgroup analysis revealed that the accuracy of machine learning in predicting the time of stroke onset within 4.5 hours was optimal (training: 0.80, 95% CI 0.77-0.83; validation: 0.79, 95% CI 0.71-0.86). Conclusions: Machine learning has ideal performance in identifying the time of stroke onset. More reasonable image segmentation and texture extraction methods in radiomics should be used to promote the value of applying machine learning in diverse ethnic backgrounds. Trial Registration: PROSPERO CRD42022358898; https://www.crd.york.ac.uk/Prospero/display_record.php?RecordID=358898

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/44895

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 49 · Cited by 12

Research works printed but not linked: 44

  1. [1]GBD 2016 Neurology Collaborators Global, regional, and national burden of neurological disorders, 1990-2016: a systematic analysis for the Global Burden of Disease Study 2016 Lancet Neurol 2019 05 18 5 459 480 10.1016/S1474-4422(18)30499-X 30879893 S1474-4422(18)30499-X 30879893 PMC6459001 · DOI · Paper page
  2. [2]Chamorro Lo EH Renú Arturo van Leyen K Lyden PD The future of neuroprotection in stroke J Neurol Neurosurg Psychiatry 2021 02 92 2 129 135 10.1136/jnnp-2020-324283 33148815 jnnp-2020-324283 33148815 · DOI
  3. [3]Xiong Y Wakhloo AK Fisher M Advances in acute ischemic stroke therapy Circ Res 2022 04 15 130 8 1230 1251 10.1161/CIRCRESAHA.121.319948 35420919 35420919 · DOI
  4. [4]Saini V Guada L Yavagal DR Global epidemiology of stroke and access to acute ischemic stroke interventions Neurology 2021 11 16 97 20 Suppl 2 S6 S16 10.1212/WNL.0000000000012781 34785599 97/20_Supplement_2/S6 34785599 · DOI
  5. [5]Gariel F Lapergue B Bourcier R Berge J Barreau X Mazighi M Kyheng M Labreuche J Fahed R Blanc R Gory B Duhamel A Saleme S Costalat V Bracard S Desal H Detraz L Consoli A Piotin M Marnat G ASTER Trial Investigators Mechanical thrombectomy outcomes with or without intravenous thrombolysis Stroke 2018 10 49 10 2383 2390 10.1161/STROKEAHA.118.021500 30355117 30355117 · DOI
  6. [6]Phipps MS Cronin CA Management of acute ischemic stroke BMJ 2020 02 13 368 l6983 10.1136/bmj.l6983 32054610 32054610 · DOI
  7. [7]Simonsen CZ Leslie-Mazwi TM Thomalla G Which imaging approach should be used for stroke of unknown time of onset? Stroke 2021 01 52 1 373 380 10.1161/STROKEAHA.120.032020 33302796 33302796 · DOI
  8. [8]Mistry EA Mistry AM Nakawah MO Chitale RV James RF Volpi JJ Fusco MR Mechanical thrombectomy outcomes with and without intravenous thrombolysis in stroke patients: A meta-analysis Stroke 2017 09 48 9 2450 2456 10.1161/STROKEAHA.117.017320 28747462 STROKEAHA.117.017320 28747462 · DOI
  9. [9]Kim Y Kim BJ Kwon SU Kim JS Kang D Unclear-onset stroke: Daytime-unwitnessed stroke vs. wake-up stroke Int J Stroke 2016 02 11 2 212 20 10.1177/1747493015616513 26783313 11/2/212 26783313 · DOI
  10. [10]Campbell BCV Ma H Ringleb PA Parsons MW Churilov L Bendszus M Levi CR Hsu C Kleinig TJ Fatar M Leys D Molina C Wijeratne T Curtze S Dewey HM Barber PA Butcher KS De Silva DA Bladin CF Yassi N Pfaff JAR Sharma G Bivard A Desmond PM Schwab S Schellinger PD Yan B Mitchell PJ Serena J Toni D Thijs V Hacke W Davis SM Donnan GA EXTEND‚ ECASS-4‚EPITHET Investigators Extending thrombolysis to 4·5-9 h and wake-up stroke using perfusion imaging: a systematic review and meta-analysis of individual patient data Lancet 2019 07 13 394 10193 139 147 10.1016/S0140-6736(19)31053-0 31128925 S0140-6736(19)31053-0 31128925 · DOI
  11. [11]McGarry B Kauppinen R Timing the Ischemic Stroke by Multiparametric Quantitative Magnetic Resonance Imaging 2021 Brisbane, Australia Exon Publications 34279887
  12. [12]Rogers HJ McGarry BL Knight MJ Jokivarsi KT Gröhn Olli H J Kauppinen RA Timing the ischaemic stroke by 1H-MRI: improved accuracy using absolute relaxation times over signal intensities Neuroreport 2014 10 22 25 15 1180 5 10.1097/WNR.0000000000000238 25116145 25116145 · DOI
  13. [13]Li H Liu J Dong Z Chen X Zhou C Huang C Li Y Liu Q Su X Cheng X Lu G Identification of high-risk intracranial plaques with 3D high-resolution magnetic resonance imaging-based radiomics and machine learning J Neurol 2022 12 269 12 6494 6503 10.1007/s00415-022-11315-4 35951103 10.1007/s00415-022-11315-4 35951103 · DOI
  14. [14]Hoyer UCI Lennartz S Abdullayev N Fichter F Jünger Stephanie T Goertz L Laukamp KR Gertz RJ Grunz J Hohmann C Maintz D Persigehl T Kabbasch C Borggrefe J Weiss K Pennig L Imaging of the extracranial internal carotid artery in acute ischemic stroke: assessment of stenosis, plaques, and image quality using relaxation-enhanced angiography without contrast and triggering (REACT) Quant Imaging Med Surg 2022 07 12 7 3640 3654 10.21037/qims-21-1122 35782261 qims-12-07-3640 35782261 PMC9246733 · DOI
  15. [15]Jordan MI Mitchell TM Machine learning: Trends, perspectives, and prospects Science 2015 07 17 349 6245 255 60 10.1126/science.aaa8415 26185243 349/6245/255 26185243 · DOI · Paper page
  16. [16]Zhang Y Zhu D Li T Wang X Zhao L Yang X Dang M Li Y Wu Y Lu Z Lu J Jian Y Wang H Zhang L Lu X Shen Z Fan H Cai W Zhang G Detection of acute ischemic stroke and backtracking stroke onset time via machine learning analysis of metabolomics Biomed Pharmacother 2022 11 155 113641 10.1016/j.biopha.2022.113641 36088854 S0753-3322(22)01030-7 36088854 · DOI
  17. [17]Zhang Y Liu A Man F Zhang Y Li C Liu Y Zhou J Zhang A Zhang Y Lv J Jiang W MRI radiomic features-based machine learning approach to classify ischemic stroke onset time J Neurol 2022 01 269 1 350 360 10.1007/s00415-021-10638-y 34218292 10.1007/s00415-021-10638-y 34218292 · DOI
  18. [18]Lee H Lee E Ham S Lee H Lee JS Kwon SU Kim JS Kim N Kang D Machine learning approach to identify stroke within 4.5 Hours Stroke 2020 03 51 3 860 866 10.1161/STROKEAHA.119.027611 31987014 31987014 · DOI
  19. [19]Patel UK Anwar A Saleem S Malik P Rasul B Patel K Yao R Seshadri A Yousufuddin M Arumaithurai K Artificial intelligence as an emerging technology in the current care of neurological disorders J Neurol 2021 05 268 5 1623 1642 10.1007/s00415-019-09518-3 31451912 10.1007/s00415-019-09518-3 31451912 · DOI
  20. [20]Jiang F Jiang Y Zhi H Dong Y Li H Ma S Wang Y Dong Q Shen H Wang Y Artificial intelligence in healthcare: past, present and future Stroke Vasc Neurol 2017 12 2 4 230 243 10.1136/svn-2017-000101 29507784 svn-2017-000101 29507784 PMC5829945 · DOI · Paper page
  21. [21]Heo J Yoon JG Park H Kim YD Nam HS Heo JH Machine learning-based model for prediction of outcomes in acute stroke Stroke 2019 05 50 5 1263 1265 10.1161/STROKEAHA.118.024293 30890116 30890116 · DOI
  22. [24]Wouters A Dupont P Norrving B Laage R Thomalla G Albers GW Thijs V Lemmens R Prediction of stroke onset is improved by relative fluid-attenuated inversion recovery and perfusion imaging compared to the visual diffusion-weighted imaging/fluid-attenuated inversion recovery mismatch Stroke 2016 10 47 10 2559 64 10.1161/STROKEAHA.116.013903 27601375 STROKEAHA.116.013903 27601375 · DOI
  23. [25]Zhu H Jiang L Zhang H Luo L Chen Y Chen Y An automatic machine learning approach for ischemic stroke onset time identification based on DWI and FLAIR imaging Neuroimage Clin 2021 31 102744 10.1016/j.nicl.2021.102744 34245995 S2213-1582(21)00188-1 34245995 PMC8271155 · DOI
  24. [26]Zhang H Polson JS Nael K Salamon N Yoo B El-Saden S Scalzo F Speier W Arnold CW Intra-domain task-adaptive transfer learning to determine acute ischemic stroke onset time Comput Med Imaging Graph 2021 06 90 101926 10.1016/j.compmedimag.2021.101926 33934065 S0895-6111(21)00075-6 33934065 PMC13529409 · DOI
  25. [27]Yao X Mao L Lv S Ren Z Li W Ren K CT radiomics features as a diagnostic tool for classifying basal ganglia infarction onset time J Neurol Sci 2020 05 15 412 116730 10.1016/j.jns.2020.116730 32092485 S0022-510X(20)30066-6 32092485 · DOI
  26. [28]Wouters A Dupont P Christensen S Norrving B Laage R Thomalla G Kemp S Lansberg M Thijs V Albers GW Lemmens R Multimodal magnetic resonance imaging to identify stroke onset within 6 h in patients with large vessel occlusions Eur Stroke J 2018 06 3 2 185 192 10.1177/2396987317753486 31008349 10.1177_2396987317753486 31008349 PMC6460412 · DOI
  27. [29]Wouters A Cheng B Christensen S Dupont P Robben D Norrving B Laage R Thijs VN Albers GW Thomalla G Lemmens R Automated DWI analysis can identify patients within the thrombolysis time window of 4.5 hours Neurology 2018 05 01 90 18 e1570 e1577 10.1212/WNL.0000000000005413 29618622 WNL.0000000000005413 29618622 · DOI
  28. [30]Wen X Shu Z Li Y Hu X Gong X Developing a model for estimating infarction onset time based on computed tomography radiomics in patients with acute middle cerebral artery occlusion BMC Med Imaging 2021 10 11 21 1 147 10.1186/s12880-021-00678-1 34635087 10.1186/s12880-021-00678-1 34635087 PMC8507216 · DOI
  29. [31]Polson J Zhang H Nael K Salamon N Yoo B Kim N Kang D Speier W Arnold CW A semi-supervised learning framework to leverage proxy information for stroke MRI analysis Annu Int Conf IEEE Eng Med Biol Soc 2021 11 2021 2258 2261 10.1109/EMBC46164.2021.9631098 34891736 34891736 · DOI
  30. [32]Jiang L Wang S Ai Z Shen T Zhang H Duan S Chen Y Yin X Sun J Development and external validation of a stability machine learning model to identify wake-up stroke onset time from MRI Eur Radiol 2022 06 32 6 3661 3669 10.1007/s00330-021-08493-6 35037969 10.1007/s00330-021-08493-6 35037969 · DOI
  31. [33]Ho KC Speier W Zhang H Scalzo F El-Saden S Arnold CW A machine learning approach for classifying ischemic stroke onset time from imaging IEEE Trans Med Imaging 2019 07 38 7 1666 1676 10.1109/TMI.2019.2901445 30802855 30802855 PMC6661120 · DOI
  32. [34]Ho KC Speier W El-Saden S Arnold CW Classifying acute ischemic stroke onset time using deep imaging features AMIA Annu Symp Proc 2017 2017 892 901 29854156 29854156 PMC5977679
  33. [35]Campagnini S Arienti C Patrini M Liuzzi P Mannini A Carrozza MC Machine learning methods for functional recovery prediction and prognosis in post-stroke rehabilitation: a systematic review J Neuroeng Rehabil 2022 06 03 19 1 54 10.1186/s12984-022-01032-4 35659246 10.1186/s12984-022-01032-4 35659246 PMC9166382 · DOI
  34. [36]Vodencarevic A Weingärtner Michael Caro JJ Ukalovic D Zimmermann-Rittereiser M Schwab S Kolominsky-Rabas P Prediction of recurrent ischemic stroke using registry data and machine learning methods: The Erlangen Stroke Registry Stroke 2022 07 53 7 2299 2306 10.1161/STROKEAHA.121.036557 35360927 35360927 · DOI
  35. [37]Chun M Clarke R Cairns BJ Clifton D Bennett D Chen Y Guo Y Pei P Lv J Yu C Yang L Li L Chen Z Zhu T China Kadoorie Biobank Collaborative Group Stroke risk prediction using machine learning: a prospective cohort study of 0.5 million Chinese adults J Am Med Inform Assoc 2021 07 30 28 8 1719 1727 10.1093/jamia/ocab068 33969418 6272889 33969418 PMC8324240 · DOI
  36. [38]Koga M Yamamoto H Inoue M Asakura K Aoki J Hamasaki T Kanzawa T Kondo R Ohtaki M Itabashi R Kamiyama K Iwama T Nakase T Yakushiji Y Igarashi S Nagakane Y Takizawa S Okada Y Doijiri R Tsujino A Ito Y Ohnishi H Inoue T Takagi Y Hasegawa Y Shiokawa Y Sakai N Osaki M Uesaka Y Yoshimura S Urabe T Ueda T Ihara M Kitazono T Sasaki M Oita A Yoshimura S Fukuda-Doi M Miwa K Kimura K Minematsu K Toyoda K THAWS Trial Investigators Thrombolysis with alteplase at 0.6 Mg/kg for stroke with unknown time of onset: A randomized controlled trial Stroke 2020 05 51 5 1530 1538 10.1161/STROKEAHA.119.028127 32248771 32248771 PMC7185058 · DOI
  37. [39]Scheldeman L Wouters A Boutitie F Dupont P Christensen S Cheng B Ebinger M Endres M Fiebach JB Gerloff C Muir KW Nighoghossian N Pedraza S Simonsen CZ Thijs V Thomalla G Lemmens R WAKE-UP Investigators Different mismatch concepts for magnetic resonance imaging-guided thrombolysis in unknown onset stroke Ann Neurol 2020 06 87 6 931 938 10.1002/ana.25730 32227638 32227638 · DOI
  38. [40]Ebinger M Galinovic I Rozanski M Brunecker P Endres M Fiebach JB Fluid-attenuated inversion recovery evolution within 12 hours from stroke onset: a reliable tissue clock? Stroke 2010 02 41 2 250 5 10.1161/STROKEAHA.109.568410 20035068 STROKEAHA.109.568410 20035068 · DOI
  39. [41]Thomalla G Cheng B Ebinger M Hao Q Tourdias T Wu O Kim JS Breuer L Singer OC Warach S Christensen S Treszl A Forkert ND Galinovic I Rosenkranz M Engelhorn T Köhrmann Martin Endres M Kang D Dousset V Sorensen AG Liebeskind DS Fiebach JB Fiehler J Gerloff C STIRVISTA Imaging Investigators DWI-FLAIR mismatch for the identification of patients with acute ischaemic stroke within 4·5 h of symptom onset (PRE-FLAIR): a multicentre observational study Lancet Neurol 2011 11 10 11 978 86 10.1016/S1474-4422(11)70192-2 21978972 S1474-4422(11)70192-2 21978972 · DOI
  40. [42]Emeriau S Serre I Toubas O Pombourcq F Oppenheim C Pierot L Stroke 2013 06 44 6 1647 51 10.1161/STROKEAHA.113.001001 23640823 STROKEAHA.113.001001 23640823 · DOI
  41. [43]Lee E Kim Y Kim N Kang D Deep into the brain: Artificial intelligence in stroke imaging J Stroke 2017 09 19 3 277 285 10.5853/jos.2017.02054 29037014 jos-2017-02054 29037014 PMC5647643 · DOI
  42. [44]Hiremath A Shiradkar R Fu P Mahran A Rastinehad AR Tewari A Tirumani SH Purysko A Ponsky L Madabhushi A An integrated nomogram combining deep learning, Prostate Imaging-Reporting and Data System (PI-RADS) scoring, and clinical variables for identification of clinically significant prostate cancer on biparametric MRI: a retrospective multicentre study Lancet Digit Health 2021 07 3 7 e445 e454 10.1016/S2589-7500(21)00082-0 34167765 S2589-7500(21)00082-0 34167765 PMC8261599 · DOI
  43. [45]Liu A Wang Z Yang Y Wang J Dai X Wang L Lu Y Xue F Preoperative diagnosis of malignant pulmonary nodules in lung cancer screening with a radiomics nomogram Cancer Commun (Lond) 2020 01 40 1 16 24 10.1002/cac2.12002 32125097 32125097 PMC7163925 · DOI
  44. [46]Huang J Xie X Wu H Zhang X Zheng Y Xie X Wang Y Xu M Development and validation of a combined nomogram model based on deep learning contrast-enhanced ultrasound and clinical factors to predict preoperative aggressiveness in pancreatic neuroendocrine neoplasms Eur Radiol 2022 11 32 11 7965 7975 10.1007/s00330-022-08703-9 35389050 10.1007/s00330-022-08703-9 35389050 · DOI

Authors
Feng, J., Zhang, Q., Wu, F., Peng, J., Li, Z., & Chen, Z.
Affiliations
  • The Central Hospital of Enshi Tujia and Miao Autonomous Prefecture (CN)
  • Hubei Enshi College (CN)
  • Shandong University of Traditional Chinese Medicine (CN)
Corresponding author
Chen, Zhuang
ORCID
Venue
Journal of Medical Internet Research
Publisher
JMIR Publications Inc.
Published
2023-10-12
Volume
25
Pages
e44895
ISSN
1438-8871 (ISSN-L)
DOI
10.2196/44895
PMC
PMC10603565
OpenAlex
W4387564183
Cited by
12
References
49
Citations by year
2026: 4 · 2025: 7 · 2024: 1
FWCI
0.47
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
  • Health Sciences › Medicine › Epidemiology › Acute Ischemic Stroke Management
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
stroke, time of symptom onset, machine learning, prediction model, radiomics, image segmentation, texture feature extraction, concordance index, sensitivity and specificity, ethnic diversity
MeSH terms
Machine Learning, Ethnicity, Humans, Patients, Stroke, PubMed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review