Cited by 267AI in cancer detectionPrivacy-Preserving Technologies in DataRadiomics and Machine Learning in Medical Imaging

The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index; none of the 19 sources its claims cite is checked for support yet.

Federated learning for computational pathology on gigapixel whole slide images

Open access
Lu, Ming Y., Chen, Richard J., Kong, Dehan, Lipkova, Jana, Singh, Rajendra, Williamson, Drew F.K., Chen, Tiffany Y., Mahmood, Faisal·2022·Medical Image Analysis

Abstract

Deep Learning-based computational pathology algorithms have demonstrated profound ability to excel in a wide array of tasks that range from characterization of well known morphological phenotypes to predicting non human-identifiable features from histology such as molecular alterations. However, the development of robust, adaptable and accurate deep learning-based models often rely on the collection and time-costly curation large high-quality annotated training data that should ideally come from diverse sources and patient populations to cater for the heterogeneity that exists in such datasets. Multi-centric and collaborative integration of medical data across multiple institutions can naturally help overcome this challenge and boost the model performance but is limited by privacy concerns among other difficulties that may arise in the complex data sharing process as models scale towards using hundreds of thousands of gigapixel whole slide images. In this paper, we introduce privacy-preserving federated learning for gigapixel whole slide images in computational pathology using weakly-supervised attention multiple instance learning and differential privacy. We evaluated our approach on two different diagnostic problems using thousands of histology whole slide images with only slide-level labels. Additionally, we present a weakly-supervised learning framework for survival prediction and patient stratification from whole slide images and demonstrate its effectiveness in a federated setting. Our results show that using federated learning, we can effectively develop accurate weakly-supervised deep learning models from distributed data silos without direct data sharing and its associated complexities, while also preserving differential privacy using randomized noise generation. We also make available an easy-to-use federated learning for computational pathology software package: http://github.com/mahmoodlab/HistoFL.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

This paper presents a privacy-preserving federated learning framework for gigapixel whole slide images in computational pathology, combining weakly-supervised attention multiple instance learning with differential privacy via Gaussian noise. It evaluates the approach on breast and renal cancer subtyping and clear cell renal cell carcinoma survival prediction, showing federated models can be competitive with centralized training while avoiding direct data sharing.

Key findings

  • Federated learning benefited from multi-institutional data compared with single-institution training and was generally competitive with centralized training. For BRCA subtyping, mean test AUC ranged from 0.833 to 0.862 for noise levels α ∈ {0, 0.001, 0.01, 0.1}; for RCC subtyping, macro-averaged test AUC ranged from 0.974 to 0.976.
  • Model performance significantly deteriorated when α was set too high (e.g., α = 1), indicating a trade-off between model performance and privacy protection.
  • For CCRCC survival prediction, all federated models except α = 1 achieved statistical significance (p < 0.05) and reasonable c-Index and average cumulative/dynamic AUC; when trained on single-institution data, only 1 of 3 institutions yielded a model that could stratify patients into distinct survival groups.
  • For survival prediction, R = 8 performed best among R ∈ {2, 4, 6, 8}; a Cox proportional hazard model based on nuclear grade lagged behind the weakly-supervised deep learning approach and only matched its performance when combined with age and gender. Less frequent communication decreased survival c-indices by around 2–3% for E = 4 and E = 8, while weighted versus simple federated averaging showed minimal differences.

Methods

  • The framework used a master-server federated averaging architecture: each participating institution trained a local attention-based MIL model, uploaded trainable parameters each epoch, and the central server averaged them and sent updated weights back. Differential privacy was implemented by adding Gaussian noise to shared weights, with α varied over {0, 0.001, 0.01, 0.1, 1.0}; communication frequency E was varied over {1, 2, 4, 8}, and weighted versus uniform averaging was compared.
  • Data included TCGA and Brigham and Women’s Hospital in-house WSIs. BRCA subtyping used 2126 breast WSIs (369 lobular, 1757 ductal); RCC subtyping used 1184 kidney WSIs (703 CCRCC, 337 PRCC, 144 CHRCC); CCRCC survival prediction used 511 patient cases from TCGA-KIRC. Classification used four simulated institutional sites (BWH plus three TCGA tissue-source-site partitions); survival prediction used three simulated sites. Data were split 70/15/15% for training/validation/test per site, repeated with 5 random seeds, with stratified sampling.
  • WSIs were processed at 20× magnification using the CLAM pipeline to segment tissue and extract 256×256 patches, embedded as 1024-dimensional feature vectors via a pretrained ResNet50 encoder truncated after the third residual block. Models used gated attention MIL with a projection to 512 dimensions, 256-dimensional attention hidden layers, dropout p = 0.25, and either cross-entropy for classification or a discrete-time hazard loss for survival prediction with R bins and β = 0.15.

Limitations

  • The paper does not include a dedicated limitations section. Stated caveats include a trade-off between model performance and privacy protection when α is too high, and dataset-size imbalance among participating institutions, where single sites with insufficient data may not yield meaningful generalizable models.
  • The text makes evident that federated learning was simulated by partitioning TCGA and in-house data into institutional sites rather than deploying across actual independent institutions.
  • The differential privacy analysis assumes Δ₂(f) = 1 because computing sensitivity for a multi-layer neural network is intractable, and the privacy guarantee is linked to an adjustable noise parameter rather than a formal end-to-end accounting in the experiments.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.media.2021.102298

Cites

68 of 68 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1. Overview of the weakly-supervised multiple instance learning in a federated learning framework. At each client site, for each WSI, the tissue regions are first automatically segmented and image patches are extracted from the segmented foreground regions. Then all patches are embedded into a low-dimension feature representation using a pretrained CNN as the encoder. Each client site trains a model using weakly-supervised learning on local data (requires only the slide-level or patient-level labels) and sends the model weights each epoch to a central server. Random noise can be added to the weight parameters before communicating with the central hub for differential privacy preservation. On the central server, the global model is updated by averaging the model weights retrieved from all client sites. After the federated averaging, the updated weights of the global model is then sent to each client model for synchronization prior to starting the next federated round.
Fig. 1. Overview of the weakly-supervised multiple instance learning in a federated learning framework. At each client site, for each WSI, the tissue regions are first automatically segmented and image patches are extracted from the segmented foreground regions. Then all patches are embedded into a low-dimension feature representation using a pretrained CNN as the encoder. Each client site trains a model using weakly-supervised learning on local data (requires only the slide-level or patient-level labels) and sends the model weights each epoch to a central server. Random noise can be added to the weight parameters before communicating with the central hub for differential privacy preservation. On the central server, the global model is updated by averaging the model weights retrieved from all client sites. After the federated averaging, the updated weights of the global model is then sent to each client model for synchronization prior to starting the next federated round.CC BY-NC-ND · PubMed Central
Fig. 2. Performance, comparative analysis and loss curves. a-c, d-f The classification performance and loss curves of BRCA histologic subtyping and RCC histological subtyping, respectively. Top: ROC curves are generated on the test sets for models trained using a centralized database, federated learning (with different levels of Gaussian random noise added during federated weight averaging) and using training data local to each institution individually. The AUC score (averaged over 5-fold cross-validation, s.d.) is reported for each experiment; macro-averaging is used for the multi-class classification of RCC subytping. Using multiinstitutional data and federated learning, we achieved a mean test AUC between 0.833 and 0.862 on BRCA histologic subtyping and an AUC of between 0.974 and 0.976 on RCC histologic subtyping respectively. Middle: Balanced accuracy score and the sensitivity (recall) for each class (IDC: Invasive Ductal Carcinoma, ILC: Invasive Lobular Carcinoma for BRCA subtyping; CHRCC: Chromophobe Renal Cell Carcinoma, CCRCC: Clear Cell Renal Cell Carcinoma, PRCC: Papillary Renal Cell Carcinoma for RCC subtyping) is plotted for all experiments to assess model performance when accounting for class-imbalance in the respective test set. Error bars show s.d. from 5-fold cross-validation. Bottom: For each experiment, the training loss and validation loss is monitored over each epoch before early stopping is triggered (see Section 3.2). Loss curves are shown for a single cross-validation fold from each task. Federated learning is observed to converge to a higher training and validation loss value in both tasks.
Fig. 2. Performance, comparative analysis and loss curves. a-c, d-f The classification performance and loss curves of BRCA histologic subtyping and RCC histological subtyping, respectively. Top: ROC curves are generated on the test sets for models trained using a centralized database, federated learning (with different levels of Gaussian random noise added during federated weight averaging) and using training data local to each institution individually. The AUC score (averaged over 5-fold cross-validation, s.d.) is reported for each experiment; macro-averaging is used for the multi-class classification of RCC subytping. Using multiinstitutional data and federated learning, we achieved a mean test AUC between 0.833 and 0.862 on BRCA histologic subtyping and an AUC of between 0.974 and 0.976 on RCC histologic subtyping respectively. Middle: Balanced accuracy score and the sensitivity (recall) for each class (IDC: Invasive Ductal Carcinoma, ILC: Invasive Lobular Carcinoma for BRCA subtyping; CHRCC: Chromophobe Renal Cell Carcinoma, CCRCC: Clear Cell Renal Cell Carcinoma, PRCC: Papillary Renal Cell Carcinoma for RCC subtyping) is plotted for all experiments to assess model performance when accounting for class-imbalance in the respective test set. Error bars show s.d. from 5-fold cross-validation. Bottom: For each experiment, the training loss and validation loss is monitored over each epoch before early stopping is triggered (see Section 3.2). Loss curves are shown for a single cross-validation fold from each task. Federated learning is observed to converge to a higher training and validation loss value in both tasks.CC BY-NC-ND · PubMed Central
Fig. 3. Interpretability and visualization for weakly-supervised federated classification. In order to interpret and validate the morphological features learned by the model for RCC and BRCA histologic subtype classification, for randomly selected WSIs in the respective test set, the model trained with privacy-preserving federated learning (α= 0.01) is used to generate attention heatmaps using 256 × 256 sized tissue patches tiled at the 20 × magnification with a 90% spatial overlap. For each WSI, the attention scores predicted for all patches in the slide are normalized to the range of [0, 1] by converting them to percentiles. For subtype classification, patches with high attention refers to image regions of high diagnostic relevance used for class prediction. The normalized scores are then mapped to their respective spatial location in the slide. Finally, an RGB colormap is applied (red: high attention, blue: low attention), and the heatmap is overlaid on top of the original H&E image for display. For BRCA, patches of the most highly attended regions (red border) exhibited well-known tumor morphology of invasive ductal carcinoma (round cells with varying degrees of polymorphism arranged in tubules, nests, or papillae) and invasive lobular carcinoma (round and signet-ring cells with intracellular lumina and targetoid cytoplasmic mucin arranged in a single-file or trabecular pattern). For RCC, highly attended regions exhibited well-known tumor morphology of chromophobe RCC (large, round to polygonal cells with abundant, finely-reticulated to granular cytoplasm and perinuclear halos), clear cell RCC (large, round to polygonal cells with clear cytoplasm and distinct, but delicate cell borders), and papillary RCC (round to cuboidal cells with prominent papillary or tubulopapillary architecture with fibrovascular cores). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)
Fig. 3. Interpretability and visualization for weakly-supervised federated classification. In order to interpret and validate the morphological features learned by the model for RCC and BRCA histologic subtype classification, for randomly selected WSIs in the respective test set, the model trained with privacy-preserving federated learning (α= 0.01) is used to generate attention heatmaps using 256 × 256 sized tissue patches tiled at the 20 × magnification with a 90% spatial overlap. For each WSI, the attention scores predicted for all patches in the slide are normalized to the range of [0, 1] by converting them to percentiles. For subtype classification, patches with high attention refers to image regions of high diagnostic relevance used for class prediction. The normalized scores are then mapped to their respective spatial location in the slide. Finally, an RGB colormap is applied (red: high attention, blue: low attention), and the heatmap is overlaid on top of the original H&E image for display. For BRCA, patches of the most highly attended regions (red border) exhibited well-known tumor morphology of invasive ductal carcinoma (round cells with varying degrees of polymorphism arranged in tubules, nests, or papillae) and invasive lobular carcinoma (round and signet-ring cells with intracellular lumina and targetoid cytoplasmic mucin arranged in a single-file or trabecular pattern). For RCC, highly attended regions exhibited well-known tumor morphology of chromophobe RCC (large, round to polygonal cells with abundant, finely-reticulated to granular cytoplasm and perinuclear halos), clear cell RCC (large, round to polygonal cells with clear cytoplasm and distinct, but delicate cell borders), and papillary RCC (round to cuboidal cells with prominent papillary or tubulopapillary architecture with fibrovascular cores). (For interpretation of the references to colour in this figure legend, the reader is referred to the web version of this article.)CC BY-NC-ND · PubMed Central
Fig. 4. Patient stratification and interpretability for weakly-supervised federated survival prediction. Patients in the test set were stratified into high risk and low risk groups using the median (50% percentile) of the model’s predicted risk score distribution as the cutoff and the log-rank test was used to assess the statistical significance between survival distributions of the resulting risk groups. Top: increasing α by over two orders of magnitude for stronger guarantees on differential privacy did not eliminate the model’s ability to stratify patients into statistically significantly (p -value < 0.05) different risk groups. Bottom: exemplars of Clear Cell Renal Cell Carcinoma patients predicted as high-risk and low-risk respectively by the model, the original H&E (left), attention-based heatmap (center), and highest-attention patches (right). As compared to the subtyping classification problem, since survival analysis is an ordinal regression problem, the high attention patches correspond to regions with high prognostic relevance in stratifying patients into low versus high risk groups. The highest attention patches for the high-risk case focus predominantly on the tumor cells themselves, while the highest attention patches for the low risk case focus predominantly on lymphocytes within the stroma and directly interfacing with tumor cells, which corroborates with the known prognostic relevance of tumor-immune co-localization in pathology.
Fig. 4. Patient stratification and interpretability for weakly-supervised federated survival prediction. Patients in the test set were stratified into high risk and low risk groups using the median (50% percentile) of the model’s predicted risk score distribution as the cutoff and the log-rank test was used to assess the statistical significance between survival distributions of the resulting risk groups. Top: increasing α by over two orders of magnitude for stronger guarantees on differential privacy did not eliminate the model’s ability to stratify patients into statistically significantly (p -value < 0.05) different risk groups. Bottom: exemplars of Clear Cell Renal Cell Carcinoma patients predicted as high-risk and low-risk respectively by the model, the original H&E (left), attention-based heatmap (center), and highest-attention patches (right). As compared to the subtyping classification problem, since survival analysis is an ordinal regression problem, the high attention patches correspond to regions with high prognostic relevance in stratifying patients into low versus high risk groups. The highest attention patches for the high-risk case focus predominantly on the tumor cells themselves, while the highest attention patches for the low risk case focus predominantly on lymphocytes within the stroma and directly interfacing with tumor cells, which corroborates with the known prognostic relevance of tumor-immune co-localization in pathology.CC BY-NC-ND · PubMed Central
Fig. 5. Performance comparison between simple averaging vs. weighted aggregation. Performance in terms of AUC ROC for classification and c-index for survival prediction is shown for federated averaging across different levels of α. Error bars show s.d. from 5-fold cross-validation.
Fig. 5. Performance comparison between simple averaging vs. weighted aggregation. Performance in terms of AUC ROC for classification and c-index for survival prediction is shown for federated averaging across different levels of α. Error bars show s.d. from 5-fold cross-validation.CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

14 Own result · 13 Own interpretation · 8 Not checked yet · 5 No basis given

Verify all 19 references — free account

Show all 26 claims

  • It may state two separate claims as one. 7
  • Probably not a substantial claim. 7
14 of 14
  • The authors state this as an assumption.

    The l2 sensitivity of the neural network function, Δ2(f), is 1.

    “However, without loss of generality, if we assume Δ2(f)=1, we see that for a given level of δ, increasing σ will enable a smaller ε to be satisfied.”
  • The paper relies on this without support.

    Each medical institution will not share data with other parties due to institutional policies, incompatible data sharing protocols, technical difficulties, or fear of privacy loss.

    “Since each medical institution will not share data with other parties due to various issues (e.g. institutional policies, incompatible data sharing protocols, technical difficulties associated with sharing large amount of data or fear of privacy loss), we cannot pool together their data and train a single deep learning model fcentralized for solving the desired task.”
  • The authors state this as an assumption.

    The global model is hosted on a central server hub.

    “In this paper, we adapted a master-server architecture, where each client node, representing each medical institution, locally utilizes the same deep learning architecture as one another and the global model, which we assume to be hosted on a central server hub.”
  • The paper relies on this without support.

    Model accuracy and robustness can be significantly increased by incorporating diverse data reflecting variations in underlying patient populations, as well as data collection and preparation protocols.

    “As in all machine learning affairs, the model’s accuracy and robustness can be significantly increased by incorporating diverse data reflecting variations in underlying patient populations, as well as data collection and preparation protocols.”
  • The paper relies on this without support.

    Whole slide images used for computational analysis exhibit immense heterogeneity arising from patient group, tissue preparation, fixation and staining protocols, and scanner hardware.

    “Specifically, in pathology, whole slide images (WSIs) used for computational analysis can exhibit immense heterogeneity. Such diversities arise from not only the patient group corresponding to the histology specimens and variations in the tissue preparation, fixation and staining protocols, but also different scanner hardware that are used for digitization.”
  • The paper relies on this without support.

    Histology data might allow indirect patient identification, e.g., in the case of rare diseases.

    “Even though histology data do not hold such a direct link with patient identity, it might still allow an indirect patient identification e.g. in the case of rare diseases.”
  • The paper relies on this without support.

    The development and validation of AI algorithms in pathology have been mainly limited to single-institutional datasets.

    “Despite these breakthroughs, the development and validation in AI algorithms in pathology have been mainly limited to single-institutional datasets”
  • The paper relies on this without support.

    Single-institutional datasets may not generalize at deployment time due to variations in the underlying patient population, staining protocols, and scanner hardware.

    “which may not generalize at deployment time due to variations in the underlying patient population, staining protocols, and scanner hardware.”
  • The paper relies on this without support.

    Dataset-size imbalance among different participating institutions frequently occurs in the real world.

    “Similar dataset-size imbalance among different participating institutions frequently occurs in the real-world and is also reflected in the imbalanced distribution of patient cases among the original tissue source sites in the TCGA.”
  • The paper relies on this without support.

    The randomized mechanism (Gaussian noise) used in federated averaging preserves differential privacy.

    “while also allowing participating institutions to preserve differential privacy via a randomized mechanism.”
10 of 10

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Research works printed but not linked: 1

  1. Carlini, N., Liu, C., Erlingsson, Ú., Kos, J. and Song, D., 2019. The secret sharer: Evaluating and testing unintended memorization in neural networks. In 28th {USENIX} Security Symposium ({USENIX} Security 19) (pp. 267-284).

Cited by

Sorted by citations

See all 267

Metadata

Authors
Lu, M. Y., Chen, R. J., Kong, D., Lipkova, J., Singh, R., Williamson, D. F. K., Chen, T. Y., & Mahmood, F.
Affiliations
  • Broad Institute (US)
  • Brigham and Women's Hospital (US)
  • Northwell Health (US)
Corresponding author
Mahmood, Faisal
Venue
Medical Image Analysis
Publisher
Elsevier BV
Published
2022-02
Volume
76
Pages
102298
ISSN
1361-8415 (ISSN-L)
OpenAlex
W3089056520
Cited by
267
References
65
Citations by year
2026: 52 · 2025: 62 · 2024: 60 · 2023: 49 · 2022: 36
FWCI
13.47
Type
journal article
Language
en
Open access
Locations
  • Medical Image Analysis (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion, cc-by-nc-nd)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • Brigham and Women's Hospital
  • National Institutes of Health
  • Google
  • National Human Genome Research Institute
  • National Science Foundation
  • National Institute of General Medical Sciences (R35GM138216)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
Keywords
federated learning, computational pathology, whole slide images, differential privacy, weakly supervised learning, survival prediction, patient stratification, privacy-preserving data sharing, histology, multicenter collaboration
MeSH terms
Algorithms, Histological Techniques, Humans, Privacy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review