Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 37 of the 39 references it cites were found in a registry or an index; 16 of the 36 sources its claims cite were checked for support.

A multimodal and fully automated system for prediction of pathological complete response to neoadjuvant chemotherapy in breast cancer

Open access
Mao, Ning, Dai, Yi, Zhou, Heng, Lin, Fan, Zheng, Tiantian, Li, Ziyin, Yang, Ping, Zhao, Feng, Li, Qin, Wang, Weiwei, Liang, Yun, Xie, Haizhu, Ma, Heng, Zhang, Lina, Guo, Yuan, Song, Xicheng, Zhang, Haicheng, Lu, Jie · 2025 · Science Advances

Abstract

Accurately predicting pathological complete response (pCR) before neoadjuvant chemotherapy (NAC) is crucial for patients with breast cancer. In this study, we developed a multimodal integrated fully automated pipeline system (MIFAPS) in forecasting pCR to NAC, using a multicenter and prospective dataset of 1004 patients with locally advanced breast cancer, incorporating pretreatment magnetic resonance imaging, whole slide image, and clinical risk factors. The results demonstrated that MIFAPS offered a favorable predictive performance in both the pooled external test set [area under the curve (AUC) = 0.882] and the prospective test set (AUC = 0.909). In addition, MIFAPS significantly outperformed single-modality models ( P < 0.05). Furthermore, the high deep learning scores were associated with immune-related pathways and the promotion of antitumor cells in the microenvironment during biological basis exploration. Overall, our study demonstrates a promising approach for improving the prediction of pCR to NAC in patients with breast cancer through the integration of multimodal data.

Overview

CoChat Summary

## TL;DR

The paper develops MIFAPS, a fully automated multimodal deep learning pipeline that predicts pathological complete response (pCR) to neoadjuvant chemotherapy in locally advanced breast cancer from pretreatment MRI, biopsy whole-slide images (WSI), and clinical factors. It reports strong discrimination in external and prospective testing and links higher model scores to immune-response-related biology in the tumor microenvironment.

## Key findings

  • In 1004 enrolled patients, MIFAPS achieved AUCs of 0.932 in the internal test set, 0.882 in the pooled external test set, and 0.909 in the prospective test set.
  • In the pooled external and prospective test sets, MIFAPS outperformed the MRI-only model (0.813 vs 0.882; 0.822 vs 0.909), the WSI-only model (0.797–0.816 vs 0.882–0.932 reported across tests), and the clinical model (0.653–0.689 for clinical only in internal/external sets; 0.745 in the prospective set).
  • The model’s sensitivity and specificity across the three test sets ranged from 0.815 to 0.947 and 0.766 to 0.854, respectively; AUPRC was 0.781 in the pooled external test set and 0.833 in the prospective test set.
  • Biological exploration in 67 patients found that high MIFAPS scores were associated with immune response pathways, including positive regulation of interferon-γ production and T cell proliferation, with higher CD4 memory T cells (P = 0.010) and more M1 macrophages in the high-score group.

## Methods

  • Multicenter study with 1004 patients with locally advanced breast cancer: 478 retrospective development/internal validation cases from one hospital, 417 pooled external test cases from multiple centers, and 109 prospectively enrolled cases from one hospital.
  • The model integrated three pretreatment inputs: MRI (T2WI, DWI, CE-T1WI) with automated segmentation and classification, WSI with attention-based multi-instance learning, and clinical risk factors selected by logistic regression.
  • Main analysis was logistic-regression-based multimodal fusion; performance was evaluated with AUC, AUPRC, sensitivity, specificity, accuracy, and DeLong tests, with subgroup analyses and visualization using Grad-CAM and WSI attention heatmaps.

## Limitations

  • The paper states that, despite multicenter validation, the model “did not achieve satisfactory sensitivity in the prospective and external test sets,” suggesting limited generalization across institutions.
  • The authors note several model constraints: the MRI model used only a specific MRI phase, the WSI model used single-scale images, and the biological analysis relied on bulk RNA sequencing, which may lose gene-expression heterogeneity.
  • Evident from the text: the deep learning and biological-basis analyses were performed on a limited subset for transcriptomics (67 patients), which may constrain biological inference.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1126/sciadv.adr1576

Cites

37 of 37 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1. Patient enrollment workflow. MRI, magnetic resonance imaging.
Fig. 1. Patient enrollment workflow. MRI, magnetic resonance imaging.CC BY-NC · PubMed Central
Fig. 2. Study workflow. (A) MRI-based deep learning signature development. (B) WSI-based deep learning signature development. (C) Clinical signature development. (D) MIFAPS integrated clinical signature, pretreatment MRI-based deep learning signature, and WSI-based deep learning signature. (E) The biological basis of deep learning was explored using RNA sequencing data from 67 patients. (F) The predictive efficiency of MIFAPS was assessed in the internal/external/prospective test sets, and its underlying predictive mechanism was investigated with visual analysis. (G) Potential clinical impact of the MIFAPS. WSI, whole slide image; MRI, magnetic resonance imaging; MIFAPS, multimodal integrated fully automated pipeline system; pCR, pathological complete response; ROC, receiver operating characteristics.
Fig. 2. Study workflow. (A) MRI-based deep learning signature development. (B) WSI-based deep learning signature development. (C) Clinical signature development. (D) MIFAPS integrated clinical signature, pretreatment MRI-based deep learning signature, and WSI-based deep learning signature. (E) The biological basis of deep learning was explored using RNA sequencing data from 67 patients. (F) The predictive efficiency of MIFAPS was assessed in the internal/external/prospective test sets, and its underlying predictive mechanism was investigated with visual analysis. (G) Potential clinical impact of the MIFAPS. WSI, whole slide image; MRI, magnetic resonance imaging; MIFAPS, multimodal integrated fully automated pipeline system; pCR, pathological complete response; ROC, receiver operating characteristics.CC BY-NC · PubMed Central
Fig. 3. Performance of different models for predicting the pCR to NAC. ROC curves of different models in the training and validation sets (A), internal test set (B), pooled external test set (C), and prospective test set (D). PR curves of different models in the training and validation sets (E), internal test set (F), pooled external test set (G), and prospective test set (H). ROC curves of MIFAPS in different molecular subtypes on the pooled external test set (I) and prospective test set (J) and different lesion size subgroups on the pooled external test set (K) and prospective test set (L). Scatter graphs describing the MIFAPS correct cases falsely predicted by the MRI (M), WSI (N), and clinical (O) models in the pooled external test set. Scatter graphs describing the MIFAPS correct cases falsely predicted by the MRI (P), WSI (Q), and clinical (R) models in the prospective test set. pCR, pathological complete response; ROC, receiver operating characteristic; AUC, area under the ROC curve; WSI, whole slide image; MRI, magnetic resonance imaging; MIFAPS, multimodal integrated fully automated pipeline system; HER2, human epidermal growth factor receptor 2.
Fig. 3. Performance of different models for predicting the pCR to NAC. ROC curves of different models in the training and validation sets (A), internal test set (B), pooled external test set (C), and prospective test set (D). PR curves of different models in the training and validation sets (E), internal test set (F), pooled external test set (G), and prospective test set (H). ROC curves of MIFAPS in different molecular subtypes on the pooled external test set (I) and prospective test set (J) and different lesion size subgroups on the pooled external test set (K) and prospective test set (L). Scatter graphs describing the MIFAPS correct cases falsely predicted by the MRI (M), WSI (N), and clinical (O) models in the pooled external test set. Scatter graphs describing the MIFAPS correct cases falsely predicted by the MRI (P), WSI (Q), and clinical (R) models in the prospective test set. pCR, pathological complete response; ROC, receiver operating characteristic; AUC, area under the ROC curve; WSI, whole slide image; MRI, magnetic resonance imaging; MIFAPS, multimodal integrated fully automated pipeline system; HER2, human epidermal growth factor receptor 2.CC BY-NC · PubMed Central
Fig. 4. MIFAPS-assisted clinical decision-making. Confusion matrix of the MIFAPS in the pooled external test set (A) and prospective test set (B). Recommendation for therapies according to the MIFAPS in the pooled external test set (C) and prospective test set (D).
Fig. 4. MIFAPS-assisted clinical decision-making. Confusion matrix of the MIFAPS in the pooled external test set (A) and prospective test set (B). Recommendation for therapies according to the MIFAPS in the pooled external test set (C) and prospective test set (D).CC BY-NC · PubMed Central
Fig. 5. Multimodal model visualization. (A) Heatmaps obtained based on Grad-CAM from the MRI. The red and yellow regions have higher predictive significance than the green and blue regions. (B) Histopathology WSIs: the original pathological images of breast cancer in patients with high scores (top) and low scores (bottom). (C) The attention heatmap visualizes the heatmap used to assign different attention scores. The red represents areas of high attention, whereas the blue represents areas of low attention. (D) High-attention tiles. (E) Segmented and classified cell types of the highest attention tiles. (F) Quantification of cell types in high-attention tiles. * indicates significant difference; n.s., not significant; WSI, whole slide image; MRI, magnetic resonance imaging.
Fig. 5. Multimodal model visualization. (A) Heatmaps obtained based on Grad-CAM from the MRI. The red and yellow regions have higher predictive significance than the green and blue regions. (B) Histopathology WSIs: the original pathological images of breast cancer in patients with high scores (top) and low scores (bottom). (C) The attention heatmap visualizes the heatmap used to assign different attention scores. The red represents areas of high attention, whereas the blue represents areas of low attention. (D) High-attention tiles. (E) Segmented and classified cell types of the highest attention tiles. (F) Quantification of cell types in high-attention tiles. * indicates significant difference; n.s., not significant; WSI, whole slide image; MRI, magnetic resonance imaging.CC BY-NC · PubMed Central
Fig. 6. Biologic basis of the MIFAPS. (A) Volcano diagram of gene expression profiles in samples separated by low and high scores. (B) Top 5 up-regulated pathways in high scores via GSEA based on the GO database. (C) Box plot representing the estimation of the abundances of member cell types in a mixed cell population. (D) IHC staining of CD4 memory T cells, M1 macrophage, and M2 macrophage from a breast cancer patient with a high score and another one with a low score. A 51-year-old woman with breast cancer with a high score (left). A 70-year-old woman with breast cancer with a low score (right). CD4 memory T cells and M2 macrophages appear brown. M1 macrophages appear red and green. MIFAPS, multimodal integrated fully automated pipeline system; GSEA, gene set enrichment analysis; GO, Gene Ontology.
Fig. 6. Biologic basis of the MIFAPS. (A) Volcano diagram of gene expression profiles in samples separated by low and high scores. (B) Top 5 up-regulated pathways in high scores via GSEA based on the GO database. (C) Box plot representing the estimation of the abundances of member cell types in a mixed cell population. (D) IHC staining of CD4 memory T cells, M1 macrophage, and M2 macrophage from a breast cancer patient with a high score and another one with a low score. A 51-year-old woman with breast cancer with a high score (left). A 70-year-old woman with breast cancer with a low score (right). CD4 memory T cells and M2 macrophages appear brown. M1 macrophages appear red and green. MIFAPS, multimodal integrated fully automated pipeline system; GSEA, gene set enrichment analysis; GO, Gene Ontology.CC BY-NC · PubMed Central
Fig. 7. Misclassified cases analysis. (A) Heatmap visualization of the MRI-based deep learning model. (a) The pretreatment image of a 50-year-old woman with non-pCR to NAC in breast cancer was misclassified as pCR. The MRI shows a 5.8-cm lesion. (b) The pretreatment image of a 30-year-old woman with pCR to NAC in breast cancer was misclassified as non-pCR. The MRI shows a 2.3-cm lesion. (B) Segmented and classified cell types of the tile with the highest attention in WSI. Images (c) and (a) belong to the same patient. Images (d) and (b) belong to the same patient. (C) Quantification of cell types in high-attention tiles in pCR and non-pCR WSIs. (D) Top 5 enriched pathways via GSEA based on the GO database. * indicates significant difference; WSI, whole slide image; MRI, magnetic resonance imaging; pCR, pathological complete response; NAC, neoadjuvant chemotherapy; GSEA, gene set enrichment analysis; GO, Gene Ontology.
Fig. 7. Misclassified cases analysis. (A) Heatmap visualization of the MRI-based deep learning model. (a) The pretreatment image of a 50-year-old woman with non-pCR to NAC in breast cancer was misclassified as pCR. The MRI shows a 5.8-cm lesion. (b) The pretreatment image of a 30-year-old woman with pCR to NAC in breast cancer was misclassified as non-pCR. The MRI shows a 2.3-cm lesion. (B) Segmented and classified cell types of the tile with the highest attention in WSI. Images (c) and (a) belong to the same patient. Images (d) and (b) belong to the same patient. (C) Quantification of cell types in high-attention tiles in pCR and non-pCR WSIs. (D) Top 5 enriched pathways via GSEA based on the GO database. * indicates significant difference; WSI, whole slide image; MRI, magnetic resonance imaging; pCR, pathological complete response; NAC, neoadjuvant chemotherapy; GSEA, gene set enrichment analysis; GO, Gene Ontology.CC BY-NC · PubMed Central

Verification

Claims the paper makes

14 Valid · 1 Not valid · 6 Not checked · 66 Own results and interpretations

39 more claims

Create a free account to see all 87 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 23
  • It may state two separate claims as one. 24

Create a free account to see these claims.

Citations

Cites

Show all 39 references

Cited by

See all 51

Metadata

Authors
Mao, N., Dai, Y., Zhou, H., Lin, F., Zheng, T., Li, Z., Yang, P., Zhao, F., Li, Q., Wang, W., Liang, Y., Xie, H., Ma, H., Zhang, L., Guo, Y., Song, X., Zhang, H., & Lu, J.
Affiliations
  • Qingdao University (CN)
  • Peking University Shenzhen Hospital (CN)
  • Shandong Institute of Business and Technology (CN)
  • Fudan University Shanghai Cancer Center (CN)
  • Affiliated Hospital of Jining Medical University (CN)
  • Department of Medical Imaging, Guilin Municipal Hospital of Traditional Chinese Medicine, Guilin, Guangxi 541002, P. R. China.
  • Department of Radiology, The First Hospital of China Medical University, Shenyang, Liaoning 110001, P. R. China.
  • Guangzhou First People's Hospital (CN)
  • Yuhuangding Hospital (CN)
  • Capital Medical University (CN)
Corresponding author
  • Mao, Ning
  • Dai, Yi
  • Zhou, Heng
  • Song, Xicheng
  • Zhang, Haicheng
  • Lu, Jie
Venue
Science Advances
Published
2025-05-02
Volume
11
Issue
18
Pages
eadr1576-eadr1576
ISSN
2375-2548 (ISSN-L)
OpenAlex
W4409982803
Cited by
51
References
39
Citations by year
2026: 35 · 2025: 15
FWCI
19.90
Type
journal article
Language
en
Open access
Licence
cc-by-nc
Locations
  • Science Advances (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Cancer Research › Breast Cancer Treatment Studies
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › MRI in cancer diagnosis
Keywords
pathological complete response, neoadjuvant chemotherapy, breast cancer, locally advanced breast cancer, multimodal prediction, pre-treatment MRI, whole slide images, clinical risk factors, deep learning, tumor immune microenvironment, immune pathways
MeSH terms
Deep Learning, Adult, Aged, Breast Neoplasms, Female, Humans, Magnetic Resonance Imaging, Middle Aged, Prognosis, Prospective Studies, Treatment Outcome, Neoadjuvant Therapy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A multimodal and fully automated system for prediction of pathological complete response to neoadjuvant chemotherapy in breast cancer. https://app.cochat.ai/papers/10.1126/sciadv.adr1576
The paper
Mao, N., Dai, Y., Zhou, H., Lin, F., Zheng, T., Li, Z., Yang, P., Zhao, F., Li, Q., Wang, W., Liang, Y., Xie, H., Ma, H., Zhang, L., Guo, Y., Song, X., Zhang, H., & Lu, J. (2025). A multimodal and fully automated system for prediction of pathological complete response to neoadjuvant chemotherapy in breast cancer. Science Advances. https://doi.org/10.1126/sciadv.adr1576

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.