CoChat
Sign Up
Cited by 45Artificial Intelligence in Healthcare and EducationAI in cancer detectionRadiomics and Machine Learning in Medical Imaging

Prospective evaluation of artificial intelligence (AI) applications for use in cancer pathways following diagnosis: a systematic review

Open access
Macheka, Sheba, Ng, Peng Yun, Ginsburg, Ophira, Hope, Andrew, Sullivan, Richard, Aggarwal, Ajay·2024·BMJ Oncology
DOI 10.1136/bmjonc-2023-000255PMC PMC11235004
Open this paper in CoChat — free accountOpen PDF

Abstract

Objectives: To assess the clinical readiness and deployability of artificial intelligence (AI) through evaluation of prospective studies of AI in cancer care following diagnosis. Design: We undertook a systematic review to determine the types of AI involved and their respective outcomes with a PubMed and Web of Science search between 1 January 2013 and 1 May 2023.15 articles detailing prospective evaluation of AI in postdiagnostic cancer pathway were identified. Setting: The role of AI in cancer care has evolved in the face of ageing population, workforce shortages and technological advancement. Despite recent uptake in AI research and adoption, the extent to which it improves quality, efficiency and equity of care beyond cancer diagnostics is uncertain to date. Interventions: We appraised all studies using Risk of Bias Assessment of Randomised Controlled Trials (ROB­2) and Risk of Bias In Non­randomised Studies- of Interventions (ROBIN­I) quality assessment tools, as well as implementational analysis concerning time, cost and resource, to ascertain the quality of clinical evidence and real-world feasibility of AI. Results: The results revealed that the majority of AI oncological research remained experimental without prospective clinical validation or deployment. Most studies failed to establish clinical validity and to translate measured AI efficacy into beneficial clinical outcomes. AI research is limited by lack of research standardisation and health system interoperability. Furthermore, implementational analysis and equity considerations of AI were largely missing. Conclusion: To overcome the triad of low-level clinical evidence, efficacy-outcome gap and incompatible research ecosystem for AI, future work should focus on multi-collaborative AI implementation research designed and conducted in accordance with up-to-date research standards and local health systems.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1136/bmjonc-2023-000255

Cites

6 of 6 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Emerging evidence suggests that AI-enabled applications can improve operational efficiency, clinical decision-making, patient triage, scheduling, and treatment planning when supported by appropriate governance, infrastructure, and workforce competencies.

    “Similarly, emerging evidence suggests that AI-enabled applications can improve operational efficiency, clinical decision-making, patient triage, scheduling, and treatment planning when supported by appropriate governance, infrastructure, and workforce competencies”

    From A Scoping Review of Implemented Innovations in Cancer Care: Implications for Pan-Canadian Scaling (2026) · cites this paper as [53]

    The Results (Types of AI investigated and clinical pathways; page unavailable) report shorter radiotherapy-planning times, better prediction of surgical case duration, identification of high-risk patients for additional reviews, and AI-influenced treatment decisions. The Recommendations (page unavailable) call for secure, interoperable systems, regulatory standards, and workforce training. But the review concerns postdiagnosis cancer care, does not establish that changed clinical decisions were better, and says broader improvements in care efficiency remain uncertain. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Clinical AI is increasingly moving beyond retrospective model development into prospective evaluation, implementation, and routine clinical care.

    “Clinical artificial intelligence (AI) is increasingly moving beyond retrospective model development into prospective evaluation, implementation, and routine clinical care [1,2].”

    From Beyond Model Development in Healthcare AI: Post-Development Robustness, Post-Deployment Monitoring, and Lifecycle Governance—A Scoping Review of Reviews (2026) · cites this paper as [2]

    The Conclusion (page unavailable) reports rapid growth in postdiagnostic cancer AI research and acknowledges that some tools have undergone prospective clinical evaluation. But it says only a small fraction have been evaluated prospectively and calls for future implementation research; WHAT THIS STUDY ADDS (page unavailable) says a lack of such research hinders clinical deployment. The review therefore does not establish an increasing move into implementation and routine care across clinical AI. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 42

  1. [1]Chua IS Gaziel-Yablowitz M Korach ZT et al Artificial intelligence in oncology: path to implementation Cancer Med 2021 10 4138 49 10.1002/cam4.3935 33960708 PMC8209596 · DOI · Paper page
  2. [2]Somashekhar SP Sepúlveda M-J Puglielli S et al Watson for oncology and breast cancer treatment recommendations: agreement with an expert Multidisciplinary tumor board Ann Oncol 2018 29 418 23 10.1093/annonc/mdx781 29324970 · DOI
  3. [3]Mak RH Endres MG Paik JH et al Use of crowd innovation to develop an artificial intelligence–based solution for radiation therapy targeting JAMA Oncol 2019 5 654 61 10.1001/jamaoncol.2019.0159 30998808 PMC6512265 · DOI
  4. [5]McKinney SM Sieniek M Godbole V et al International evaluation of an AI system for breast cancer screening Nature 2020 577 89 94 10.1038/s41586-019-1799-6 31894144 · DOI · Paper page
  5. [6]Eloy C Marques A Pinto J et al Artificial intelligence–assisted cancer diagnosis improves the efficiency of Pathologists in Prostatic biopsies Virchows Arch 2023 482 595 604 10.1007/s00428-023-03518-5 36809483 PMC10033575 · DOI
  6. [7]Senior K NHS embraces AI-assisted radiotherapy technology Lancet Oncol 2023 24 10.1016/S1470-2045(23)00353-4 37482068 · DOI
  7. [8]Cristiano S Leal A Phallen J et al Genome-wide cell-free DNA fragmentation in patients with cancer Nature 2019 570 385 9 10.1038/s41586-019-1272-6 31142840 PMC6774252 · DOI · Paper page
  8. [9]Cabral BP Braga LAM Syed-Abdul S et al Future of artificial intelligence applications in cancer care: A global cross-sectional survey of researchers Curr Oncol 2023 30 3432 46 10.3390/curroncol30030260 36975473 PMC10047823 · DOI
  9. [10]Kelly CJ Karthikesalingam A Suleyman M et al Key challenges for delivering clinical impact with artificial intelligence BMC Med 2019 17 10.1186/s12916-019-1426-2 PMC6821018 31665002 · DOI · Paper page
  10. [11]Crawford K Calo R There is a blind spot in AI research Nature 2016 538 311 3 10.1038/538311a 27762391 · DOI · Paper page
  11. [12]Chen I Frederik J Sontag D Why is my Classifier discriminatory? Adv Neural Inf Process Syst 2018
  12. [13]Topol EJ High-performance medicine: the convergence of human and artificial intelligence Nat Med 2019 25 44 56 10.1038/s41591-018-0300-7 30617339 · DOI · Paper page
  13. [14]Zhou N Zhang C-T Lv H-Y et al Concordance study between IBM Watson for oncology and clinical practice for patients with cancer in China Oncologist 2019 24 812 9 10.1634/theoncologist.2018-0255 30181315 PMC6656482 · DOI
  14. [15]Hutson M Artificial intelligence faces reproducibility crisis Science 2018 359 725 6 10.1126/science.359.6377.725 29449469 · DOI
  15. [16]Aristidou A Jena R Topol EJ Bridging the chasm between AI and clinical implementation The Lancet 2022 399 620 10.1016/S0140-6736(22)00235-5 35151388 · DOI · Paper page
  16. [17]Jandoo T WHO guidance for Digital health: what it means for researchers Digit Health 2020 6 10.1177/2055207619898984 PMC6952850 31949918 · DOI
  17. [18]Manz CR Parikh RB Small DS et al Effect of integrating machine learning mortality estimates with behavioral Nudges to Clinicians on serious illness conversations among patients with cancer: A stepped-wedge cluster randomized clinical trial JAMA Oncol 2020 6 e204759 10.1001/jamaoncol.2020.4759 33057696 PMC7563672 · DOI
  18. [19]Manz CR Zhang Y Chen K et al Long-term effect of machine learning–triggered behavioral Nudges on serious illness conversations and end-of-life outcomes among patients with cancer: A randomized clinical trial JAMA Oncol 2023 9 414 8 10.1001/jamaoncol.2022.6303 36633868 PMC9857721 · DOI
  19. [20]Hassoon A Baig Y Naiman DQ et al Randomized trial of two artificial intelligence coaching interventions to increase physical activity in cancer survivors NPJ Digit Med 2021 4 10.1038/s41746-021-00539-9 PMC8660785 34887491 · DOI
  20. [21]Strömblad CT Baxter-King RG Meisami A et al Effect of a predictive model on planned surgical duration accuracy, patient wait time, and use of Presurgical resources: A randomized clinical trial JAMA Surg 2021 156 315 21 10.1001/jamasurg.2020.6361 33502448 PMC7841577 · DOI
  21. [22]Kao J Zucker A Urso M et al Improving survival prognostication in patients with metastatic cancer through clinical judgment Anticancer Res 2022 42 1397 401 10.21873/anticanres.15609 35220232 · DOI
  22. [23]Manz CR Chen J Liu M et al Validation of a machine learning algorithm to predict 180-day mortality for outpatients with cancer JAMA Oncol 2020 6 1723 30 10.1001/jamaoncol.2020.4331 32970131 PMC7516810 · DOI
  23. [24]Hong JC Eclov NCW Dalal NH et al System for high-intensity evaluation during radiation therapy (SHIELD-RT): A prospective randomized study of machine learning–directed clinical evaluations during radiation and Chemoradiation J Clin Oncol 2020 38 3652 61 10.1200/JCO.20.01688 32886536 · DOI
  24. [25]Wong J Huang V Wells D et al Implementation of deep learning-based auto Segmentation for radiotherapy planning structures: a Workflow study at two cancer centers Radiat Oncol 2021 16 10.1186/s13014-021-01831-4 PMC8186196 34103062 · DOI
  25. [26]Nelson SH Natarajan L Patterson RE et al Physical activity change in an RCT: comparison of measurement methods Am J Health Behav 2019 43 543 55 10.5993/AJHB.43.3.9 31046885 PMC6812571 · DOI
  26. [27]Xu F Sepúlveda M-J Jiang Z et al Effect of an artificial intelligence clinical decision support system on treatment decisions for complex breast cancer JCO Clin Cancer Inform 2020 4 824 38 10.1200/CCI.20.00018 32970484 PMC7529515 · DOI
  27. [28]Feng L Liu Z Li C et al Development and validation of a Radiopathomics model to predict pathological complete response to Neoadjuvant Chemoradiotherapy in locally advanced Rectal cancer: a Multicentre observational study Lancet Digit Health 2022 4 e8 17 10.1016/S2589-7500(21)00215-6 34952679 · DOI · Paper page
  28. [29]Nicolae A Semple M Lu L et al Conventional vs machine learning–based treatment planning in prostate Brachytherapy: results of a phase I randomized controlled trial Brachytherapy 2020 19 470 6 10.1016/j.brachy.2020.03.004 32317241 · DOI
  29. [30]McIntosh C Conroy L Tjong MC et al Clinical integration of machine learning for curative-intent radiation treatment of patients with prostate cancer Nat Med 2021 27 999 1005 10.1038/s41591-021-01359-w 34083812 · DOI
  30. [31]Seok J Yoon S Ryu CH et al A personalized 3d-printed model for obtaining informed consent process for thyroid surgery: A randomized clinical study using a deep learning approach with mesh-type 3d modeling J Pers Med 2021 11 10.3390/jpm11060574 PMC8234549 34207419 · DOI
  31. [32]Hosny A Bitterman DS Guthier CV et al Clinical validation of deep learning Algorithms for radiotherapy targeting of non-small-cell lung cancer: an observational study Lancet Digit Health 2022 4 e657 66 10.1016/S2589-7500(22)00129-7 36028289 PMC9435511 · DOI
  32. [35]Hamamoto R Suvarna K Yamada M et al Application of artificial intelligence technology in oncology: towards the establishment of precision medicine Cancers (Basel) 2020 12 10.3390/cancers12123532 PMC7760590 33256107 · DOI · Paper page
  33. [36]Kann BH Hosny A Aerts H Artificial intelligence for clinical oncology Cancer Cell 2021 39 916 27 10.1016/j.ccell.2021.04.002 33930310 PMC8282694 · DOI
  34. [37]Mandel JC Kreda DA Mandl KD et al SMART on FHIR: a standards-based, Interoperable Apps platform for electronic health records J Am Med Inform Assoc 2016 23 899 908 10.1093/jamia/ocv189 26911829 PMC4997036 · DOI · Paper page
  35. [39]Mayo CS Feng MU Brock KK et al Operational Ontology for oncology (O3): A professional society-based, Multistakeholder, consensus-driven Informatics standard supporting clinical and research use of real-world data from patients treated for cancer International Journal of Radiation Oncology*Biology*Physics 2023 117 533 50 10.1016/j.ijrobp.2023.05.033 PMC10741247 37244628 · DOI
  36. [40]Dembrower K Crippa A Colón E et al Artificial intelligence for breast cancer detection in screening Mammography in Sweden: a prospective, population-based, paired-reader, non-inferiority study Lancet Digit Health 2023 5 e703 11 10.1016/S2589-7500(23)00153-X 37690911 · DOI
  37. [42]Keane PA Topol EJ With an eye to AI and autonomous diagnosis NPJ Digit Med 2018 1 10.1038/s41746-018-0048-y PMC6550235 31304321 · DOI · Paper page
  38. [43]Geary R Knight H Carroll F et al A Step‐Wise approach to developing indicators to compare the performance of maternity units using hospital administrative data BJOG 2018 125 857 65 10.1111/1471-0528.15013 29105913 PMC6001534 · DOI
  39. [44]Avery KNL Chalmers KA Brookes ST et al Development of a core outcome set for clinical effectiveness trials in Esophageal cancer resection surgery Ann Surg 2018 267 700 10 10.1097/SLA.0000000000002204 28288055 PMC5865486 · DOI
  40. [45]Aggarwal A Court LE Hoskin P et al ARCHERY: a prospective observational study of artificial intelligence-based radiotherapy treatment planning for Cervical, head and neck and prostate cancer – study protocol BMJ Open 2023 13 e077253 10.1136/bmjopen-2023-077253 PMC10711912 38149419 · DOI
  41. [48]Collins GS Reitsma JB Altman DG et al Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement Circulation 2015 131 211 9 10.1161/CIRCULATIONAHA.114.014508 25561516 PMC4297220 · DOI · Paper page
  42. [49]Sounderajah V Ashrafian H Aggarwal R et al Developing specific reporting guidelines for diagnostic accuracy studies assessing AI interventions: the STARD-AI steering group Nat Med 2020 26 807 8 10.1038/s41591-020-0941-1 32514173 · DOI · Paper page

Cited by

Sorted by citations

Authors
Macheka, S., Ng, P. Y., Ginsburg, O., Hope, A., Sullivan, R., & Aggarwal, A.
Affiliations
  • King's College London (GB)
  • National Cancer Institute (US)
  • Princess Margaret Cancer Centre (CA)
Corresponding author
Sullivan, Richard
ORCID
Venue
BMJ Oncology
Publisher
BMJ
Published
2024-05
Volume
3
Issue
1
Pages
e000255
ISSN
2752-7948 (ISSN-L)
DOI
10.1136/bmjonc-2023-000255
PMC
PMC11235004
OpenAlex
W4396815160
Cited by
45
References
6
Citations by year
2026: 24 · 2025: 18 · 2024: 3
FWCI
1.03
Type
journal article
Language
en
Update notices
correction (2025-05-15)
Open access
Licence
Locations
  • BMJ Oncology (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • LSHTM Research Online (London School of Hygiene and Tropical Medicine) (acceptedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • BASO/Rosetrees Research Grant in Cancer Surgery (N/A)
  • Medical Research Council (GACD-025)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
Keywords
artificial intelligence, cancer care, cancer pathways, clinical implementation, clinical validation, health care equity, implementation research, clinical evidence quality

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review