Cited by 35AI in cancer detectionRadiomics and Machine Learning in Medical ImagingProstate Cancer Diagnosis and Treatment

External validation of a digital pathology-based multimodal artificial intelligence architecture in the NRG/RTOG 9902 phase 3 trial

Open access
Ross, Ashley E., Zhang, Jingbin, Huang, Huei-Chung, Yamashita, Rikiya, Keim-Malpass, Jessica, Simko, Jeffry P., DeVries, Sandy, Morgan, Todd M. +22 more·2024·European Urology Oncology

Abstract

Background Accurate risk stratification is critical to guide management decisions in localized prostate cancer (PCa). Previously, we had developed and validated a multimodal artificial intelligence (MMAI) model generated from digital histopathology and clinical features. Here, we externally validate this model on men with high-risk or locally advanced PCa treated and followed as part of a phase 3 randomized control trial.Objective To externally validate the MMAI model on men with high-risk or locally advanced PCa treated and followed as part of a phase 3 randomized control trial.Design, setting, and participants Our validation cohort included 318 localized high-risk PCa patients from NRG/RTOG 9902 with available histopathology (337 [85%] of the 397 patients enrolled into the trial had available slides, of which 19 [5.6%] failed due to poor image quality).Outcome measurements and statistical analysis Two previously locked prognostic MMAI models were validated for their intended endpoint: distant metastasis (DM) and PCa-specific mortality (PCSM). Individual clinical factors and the number of National Comprehensive Cancer Network (NCCN) high-risk features served as comparators. Subdistribution hazard ratio (sHR) was reported per standard deviation increase of the score with corresponding 95% confidence interval (CI) using Fine-Gray or Cox proportional hazards models.Results and limitations The DM and PCSM MMAI algorithms were significantly and independently associated with the risk of DM (sHR [95% CI] = 2.33 [1.60-3.38], p < 0.001) and PCSM, respectively (sHR [95% CI] = 3.54 [2.38-5.28], p < 0.001) when compared against other prognostic clinical factors and NCCN high-risk features. The lower 75% of patients by DM MMAI had estimated 5- and 10-yr DM rates of 4% and 7%, and the highest quartile had average 5- and 10-yr DM rates of 19% and 32%, respectively (p < 0.001). Similar results were observed for the PCSM MMAI algorithm.Conclusions We externally validated the prognostic ability of MMAI models previously developed among men with localized high-risk disease. MMAI prognostic models further risk stratify beyond the clinical and pathological variables for DM and PCSM in a population of men already at a high risk for disease progression. This study provides evidence for consistent validation of our deep learning MMAI models to improve prognostication and enable more informed decision-making for patient care.Patient summary This paper presents a novel approach using images from pathology slides along with clinical variables to validate artificial intelligence (computer-generated) prognostic models. When implemented, clinicians can offer a more personalized and tailored prognostic discussion for men with localized prostate cancer.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.euo.2024.01.004

Cites

22 of 22 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The Esteva et al. (2022) model was externally validated in a phase 3 trial (NRG/RTOG 9902) and demonstrated significant and independent association with prognostic factors compared to current methods such as NCCN high-risk features.

    “In addition, this model was externally validated in a phase 3 trial (NRG/RTOG 9902) and demonstrated significant and independent association with prognostic factors compared to current methods such as the National Comprehensive Cancer Network (NCCN) high-risk features (Ross et al., 2024).”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. [17]Mucherino A, Papajorgji PJ, Pardalos PM. k-Nearest neighbor classification. Data mining in agriculture. New York, NY: Springer New York; 2009. p. 83– 106.

Cited by

Sorted by citations

Authors
Ross, A. E., Zhang, J., Huang, H.-C., Yamashita, R., Keim-Malpass, J., Simko, J. P., DeVries, S., Morgan, T. M., Souhami, L., Dobelbower, M. C., McGinnis, L. S., Jones, C. U., Dess, R. T., Zeitzer, K. L., Choi, K., Hartford, A. C., Michalski, J. M., Raben, A., Gomella, L. G., Sartor, A. O., Rosenthal, S. A., Sandler, H. M., Spratt, D. E., Pugh, S. L., Mohamad, O., Esteva, A., Chen, E., Schaeffer, E. M., Tran, P. T., & Feng, F. Y.
Affiliations
  • Northwestern Medicine (US)
  • University of California, San Francisco (US)
  • University of Michigan (US)
  • University of Alabama at Birmingham (US)
  • Novant Health Presbyterian Medical Center (US)
  • Sutter Medical Center (US)
  • Jefferson Einstein Philadelphia Hospital (US)
  • SUNY Downstate Health Sciences University (US)
  • Dartmouth–Hitchcock Medical Center (US)
  • Washington University in St. Louis (US)
  • Christiana Care Health System (US)
  • Thomas Jefferson University Hospital (US)
  • Cedars-Sinai Medical Center (US)
  • University Hospitals Seidman Cancer Center (US)
  • American College of Radiology (US)
  • Northwestern University (US)
  • University of Maryland, Baltimore (US)
Corresponding author
Ross, Ashley E.
Venue
European Urology Oncology
Publisher
Elsevier BV
Published
2024-10
Volume
7
Issue
5
Pages
1024-1033
ISSN
2588-9311 (ISSN-L)
OpenAlex
W4391426559
Cited by
35
References
22
Citations by year
2026: 14 · 2025: 15 · 2024: 4 · 2023: 2
FWCI
4.85
Type
journal article
Language
en
Open access
Locations
  • European Urology Oncology (acceptedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion)
  • Europe PMC (PubMed Central) (acceptedVersion)
Funders
  • Amgen
  • Bristol-Myers Squibb
  • Pfizer
  • AstraZeneca
  • Sanofi
  • Varian Medical Systems
  • Endocyte
  • NRG Oncology (U24CA196067, U10 CA180822, UG1CA189867, U10CA180868)
  • Advanced Accelerator Applications
  • Invitae
  • Astellas Pharma
  • EMD Serono
  • University of California, San Francisco
  • Foundation Medicine
  • National Cancer Institute (UG1CA233160, UG1CA189867, U10CA180868, U54CA273956, U24CA196067, U10CA180822)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
  • Health Sciences › Medicine › Pulmonary and Respiratory Medicine › Prostate Cancer Diagnosis and Treatment
Keywords
localized prostate cancer, high-risk prostate cancer, locally advanced prostate cancer, multimodal AI, digital histopathology, external validation, risk stratification, distant metastasis, prostate cancer-specific mortality, prognostic model
MeSH terms
Aged, Artificial Intelligence, Humans, Male, Middle Aged, Prognosis, Prostatic Neoplasms, Randomized Controlled Trials as Topic, Clinical Trials, Phase III as Topic, Risk Assessment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review