Cited by 217Radiomics and Machine Learning in Medical ImagingAI in cancer detectionProstate Cancer Diagnosis and Treatment

Prostate cancer therapy personalization via multi-modal deep learning on randomized phase III clinical trials

Open access
Esteva, Andre, Feng, Jean, van der Wal, Douwe, Huang, Shih-Cheng, Simko, Jeffry P., DeVries, Sandy, Chen, Emmalyn, Schaeffer, Edward M. +38 more·2022·npj Digital Medicine

Published version of a preprint

Abstract

Prostate cancer is the most frequent cancer in men and a leading cause of cancer death. Determining a patient's optimal therapy is a challenge, where oncologists must select a therapy with the highest likelihood of success and the lowest likelihood of toxicity. International standards for prognostication rely on non-specific and semi-quantitative tools, commonly leading to over- and under-treatment. Tissue-based molecular biomarkers have attempted to address this, but most have limited validation in prospective randomized trials and expensive processing costs, posing substantial barriers to widespread adoption. There remains a significant need for accurate and scalable tools to support therapy personalization. Here we demonstrate prostate cancer therapy personalization by predicting long-term, clinically relevant outcomes using a multimodal deep learning architecture and train models using clinical data and digital histopathology from prostate biopsies. We train and validate models using five phase III randomized trials conducted across hundreds of clinical centers. Histopathological data was available for 5654 of 7764 randomized patients (71%) with a median follow-up of 11.4 years. Compared to the most common risk-stratification tool-risk groups developed by the National Cancer Center Network (NCCN)-our models have superior discriminatory performance across all endpoints, ranging from 9.2% to 14.6% relative improvement in a held-out validation set. This artificial intelligence-based tool improves prognostication over standard tools and allows oncologists to computationally predict the likeliest outcomes of specific patients to determine optimal treatment. Outfitted with digital scanners and internet access, any clinic could offer such capabilities, enabling global access to therapy personalization.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1038/s41746-022-00613-w

Cites

39 of 39 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Compared to the unimodal imaging model, the Esteva et al. (2022) multimodal model obtained a notably higher AUC of 0.837 vs. 0.779 for predicting distant metastasis after 5 years.

    “Compared to the unimodal imaging model, the multimodal obtained a notably higher AUC of 0.837 vs. 0.779 for predicting distant metastasis after 5 years.”

    From Navigating the landscape of multimodal AI in medicine: A scoping review on technical challenges and clinical applications (2025) · cites this paper as Esteva et al., 2022

    The Results section’s ablation study says the multimodal model performed better than the pathology-images-only model for five-year distant metastasis. Our copy does not give the claimed 0.837-versus-0.779 comparison in text; its reported 0.84-versus-0.74 comparison is against NCCN risk groups, not the imaging-only model. No page number is available in our copy. (LLM, conf=0.84)

    Experimental — accuracy not yet measured
  • Esteva et al. (2022) describes a multimodal model that integrates digital pathology slides and clinical variables for risk stratification in prostate cancer.

    “First, Esteva et al. (2022) describes a multimodal model that integrates digital pathology slides and clinical variables for risk stratification in prostate cancer.”
  • The Esteva et al. (2022) model employs a ResNet50 pretrained in a self-supervised manner to extract a feature vector from pathology slides and concatenates this image feature vector with a clinical variable vector for final prediction.

    “Specifically, the model employs a ResNet50 pretrained in a self-supervised manner to extract a feature vector from the pathology slides and subsequently concatenates this image feature vector with a clinical variable vector for the final prediction.”
  • ArteraAI developed and tested a multimodal AI approach using histology images and clinical data to predict prostate cancer prognosis.

    “AI company ArteraAI has developed and tested a multimodal AI approach (113, 114) that uses histology images and clinical data to predict prostate cancer prognosis.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Esteva, A., Feng, J., van der Wal, D., Huang, S.-C., Simko, J. P., DeVries, S., Chen, E., Schaeffer, E. M., Morgan, T. M., Sun, Y., Ghorbani, A., Naik, N., Nathawani, D., Socher, R., Michalski, J. M., Roach, M., Pisansky, T. M., Monson, J. M., Naz, F., Wallace, J., Ferguson, M. J., Bahary, J.-P., Zou, J., Lungren, M., Yeung, S., Ross, A. E., NRG Prostate Cancer AI Consortium, Kucharczyk, M., Souhami, L., Ballas, L., Peters, C. A., Liu, S., Balogh, A. G., Randolph-Jackson, P. D., Schwartz, D. L., Girvigian, M. R., Saito, N. G., Raben, A., Rabinovitch, R. A., Katato, K., Sandler, H. M., Tran, P. T., Spratt, D. E., Pugh, S., Feng, F. Y., & Mohamad, O.
Affiliations
  • University of California, San Francisco (US)
  • Stanford University (US)
  • NRG Oncology (US)
  • Northwestern University (US)
  • University of Michigan (US)
  • Case Western Reserve University (US)
  • Salesforce (United States) (US)
  • Washington University in St. Louis (US)
  • Mayo Clinic (US)
  • California Cancer Associates for Research and Excellence (US)
  • Saint John Regional Hospital (CA)
  • Ingalls Memorial Hospital (US)
  • Centre Hospitalier de l’Université de Montréal (CA)
  • Nova Scotia Cancer Centre (CA)
  • University of Southern California (US)
  • Northeast Radiation Oncology Center (US)
  • Ronald Reagan UCLA Medical Center (US)
  • VA NY Harbor Healthcare System (US)
  • Kaiser Permanente (US)
  • Indiana University School of Medicine (US)
  • Christiana Care Health System (US)
  • University of Colorado Hospital (US)
  • Hurley Medical Center (US)
  • Cedars-Sinai Medical Center (US)
  • University of Maryland, Baltimore (US)
  • University Hospitals Seidman Cancer Center (US)
Corresponding author
Esteva, Andre
Venue
npj Digital Medicine
Published
2022-06-08
Volume
5
Issue
1
Pages
71-71
ISSN
2398-6352 (ISSN-L)
OpenAlex
W4282942659
Cited by
217
References
39
Citations by year
2026: 58 · 2025: 85 · 2024: 44 · 2023: 26 · 2022: 4
FWCI
22.55
Type
journal article
Language
en
Open access
Locations
  • npj Digital Medicine (publishedVersion, cc-by)
  • Research Square (acceptedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Digital Commons@Becker (Washington University School of Medicine) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
  • eScholarship (California Digital Library) (submittedVersion, cc-by)
Funders
  • NRG Oncology (U24CA196067, U10 CA180822, UG1CA189867, U10CA180868)
  • National Cancer Institute (UG1CA189867, U10CA180868, U24CA196067, U10CA180822)
Topics
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Health Sciences › Medicine › Pulmonary and Respiratory Medicine › Prostate Cancer Diagnosis and Treatment
Keywords
prostate cancer, multimodal deep learning, digital histopathology, prostate biopsy, outcome prediction, risk stratification, NCCN risk classification, long-term clinical outcomes, treatment selection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review