CoChat
Sign Up
Cited by 278COVID-19 diagnosis using AIPrivacy-Preserving Technologies in DataArtificial Intelligence in Healthcare and Education

Federated deep learning for detecting COVID-19 lung abnormalities in CT: a privacy-preserving multinational validation study

Open access
Dou, Qi, So, Tiffany Y., Jiang, Meirui, Liu, Quande, Vardhanabhuti, Varut, Kaissis, Georgios, Li, Zeju, Si, Weixin +13 more·2021·npj Digital Medicine
DOI 10.1038/s41746-021-00431-6PMC PMC8007806
Open this paper in CoChat — free accountOpen PDF

Abstract

Data privacy mechanisms are essential for rapidly scaling medical training databases to capture the heterogeneity of patient data distributions toward robust and generalizable machine learning systems. In the current COVID-19 pandemic, a major focus of artificial intelligence (AI) is interpreting chest CT, which can be readily used in the assessment and management of the disease. This paper demonstrates the feasibility of a federated learning method for detecting COVID-19 related CT abnormalities with external validation on patients from a multinational study. We recruited 132 patients from seven multinational different centers, with three internal hospitals from Hong Kong for training and testing, and four external, independent datasets from Mainland China and Germany, for validating model generalizability. We also conducted case studies on longitudinal scans for automated estimation of lesion burden for hospitalized COVID-19 patients. We explore the federated learning algorithms to develop a privacy-preserving AI model for COVID-19 medical image diagnosis with good generalization capability on unseen multinational datasets. Federated learning could provide an effective mechanism during pandemics to rapidly develop clinically useful AI across institutions and countries overcoming the burden of central aggregation of large amounts of sensitive data.

AI snapshot

Written from the paper’s full text.

TL;DR

This proof-of-concept study shows that a federated deep learning model can detect COVID-19 lung abnormalities on chest CT across multinational centers without sharing patient data. The federated model generalized better than single-site, ensemble, and centralized joint-training baselines on external cohorts from China and Germany.

Key findings

  • On internal Hong Kong testing, the federated model achieved AUC 95.27% (95% CI 93.98–96.51), mAP 91.28% (84.91–97.65), sensitivity 93.54% (87.33–99.75), and precision 84.21% (78.61–89.81), outperforming the model ensemble consistently across metrics.
  • On external validation, the federated model achieved the best performance among six methods: External-Set-1 AUC 95.66% (94.17–97.14), mAP 87.83% (84.18–91.48); External-Set-2 Germany AUC 88.15% (86.38–89.91), mAP 71.48% (68.69–74.27), sensitivity 73.31% (70.44–76.18), precision 91.93% (89.48–94.38); External-Set-3 AUC 91.99% (89.97–93.88), mAP 83.04% (72.5–93.58), sensitivity 85.06% (74.82–95.3), precision 87.94% (83.24–92.64). Statistical comparison with the other five models showed p < 0.05 across all external sets.
  • Transfer learning from a DeepLesion-pretrained model increased mAP by 2.56% on internal testing and by 3.31%, 5.51%, and 2.00% on the three external sets, respectively.
  • In two longitudinal case studies, automated lesion burden estimates correlated with clinical symptoms; the model took around 40 ms to test one CT volume.

Methods

  • Multicenter study with 132 patients from seven centers: three internal Hong Kong hospitals for training/testing (75 patients) and four external datasets from Mainland China and Germany used solely for testing. Federated learning used three local nodes with Federated Averaging (FedAvg), exchanging model parameters rather than data.
  • The CNN detector was an improved RetinaNet pretrained on the public DeepLesion CT lesion dataset, then fine-tuned on internal COVID-19 CT images; input consisted of three adjacent axial CT slices. Comparisons included three single-site models, their ensemble, a centralized joint model, and the federated model.
  • Evaluation used ROC/AUC with 95% CI, mAP, and sensitivity/precision at 0.1 false positives per slice; p-values were computed with two-sided Student’s t-test and significance defined as p < 0.05. Preprocessing included HU clipping, instance-level normalization, non-maximum suppression, and lung segmentation post-processing.

Limitations

  • The paper states that patient numbers per center were relatively small and imbalanced across sites, reflecting the scarcity of COVID-19 cases at most single sites; future work will include more patients and centers.
  • The paper reports that the model generalized less well on the German cohort, possibly due to demographic differences and concept shift from differing annotation procedures; after excluding 15 very mild, 5 severe diffuse, and 1 no-CT-finding case, 35 cases remained with AUC 88.15% (86.38–89.91), whereas testing on all 56 German cases gave AUC 77.15% (72.84–81.47).
  • The text makes evident that this was an opportunistic proof-of-concept with independently collected cohorts and no standard operating procedure across sites; external data were used only for testing, and the lesion segmentation branch was frozen during fine-tuning because dense lesion annotations for the COVID-19 dataset were unavailable.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41746-021-00431-6

Cites

Statistics

Nothing to recompute.

Claims the paper makes

0 of 19 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Transfer learning from a large-scale dataset was helpful to reduce the false-positive predictions for lesion detection.

    “In addition, we visually observed that transfer learning from a large-scale dataset was helpful to reduce the false-positive predictions for lesion detection.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Statistical analysis between our federated learning model with other five models revealed a p-value <0.05 across all external sets.

    “Statistical analysis between our federated learning model with other five models revealed a p-value <0.05 across all external sets.”
    Quote in the paperSubstantial claim
  • Findingown data

    For External-Set-2 from Germany with expectable differences in population demographics compared to the training cohorts, the model generalization generally decreased for all models compared with their performance on External-Set-1.

    “For External-Set-2 from Germany with expectable differences in population demographics compared to the training cohorts, the model generalization generally decreased for all models compared with their performance on External-Set-1.”
    Quote in the paperNot a substantial claim
  • Findingown data

    For Case-1, the patient's condition worsened upon admission to hospital, with lesion burden increasing from day-4.

    “For Case-1 (57 years old, male), the patient’s condition worsened upon admission to hospital (accordingly lesion burden increased from on day-4), and afterward gradually relieved in remaining time (accordingly lesion burden decreased after the peak, and to a very low rate at the last CT before discharge).”
    Quote in the paperNot a substantial claim
  • Findingown data

    Transfer learning from a pretrained model is helpful to reduce false-positive predictions.

    “Transfer learning from pretrained model is helpful to reduce false-positive predictions.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 22
  • Classified as probably not a substantial claim. 21

  • May combine two claims.

    Findingown data

    External-Set-3 showed similar results, with the federated learning model attaining the highest AUC of 91.99% (89.97-93.88) among all models.

    “External-Set-3 showed similar results, with the federated learning model attaining the highest AUC of 91.99% (89.97–93.88) among all models.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For Case-1, the patient's condition afterward gradually relieved, with lesion burden decreasing after the peak to a very low rate at the last CT before discharge.

    “For Case-1 (57 years old, male), the patient’s condition worsened upon admission to hospital (accordingly lesion burden increased from on day-4), and afterward gradually relieved in remaining time (accordingly lesion burden decreased after the peak, and to a very low rate at the last CT before discharge).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    For Case-2, the estimated lesion burden decreased from the peak at the first scan, slightly relapsed in the middle period, then recovered on day-57.

    “For Case-2 (63 years old, male), the estimated lesion burden decreased from the peak at the first scan, and slightly relapsed in the middle period, then recovered on day-57.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Individual-model-2 trained with the largest single-site database (with 9398 lesions) obtained a higher AUC of 95.40% (95% CI 93.80-96.98) than the other two individual models, with mAP of 89.98% (82.27-97.69), sensitivity of 92.03% (84.23-99.83), and precision of 82.1% (76.16-88.04).

    “Among three individual models, the Individual-model-2 trained with the largest single-site database (with 9398 lesions) obtained a higher AUC of 95.40% (95% CI 93.80–96.98) than the other two individual models, with the mAP of 89.98% (82.27–97.69), sensitivity of 92.03% (84.23–99.83), and precision of 82.1% (76.16–88.04).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The highest performance on External-Set-2 was achieved by the federated learning model, with AUC of 88.15% (86.38-89.91), mAP of 71.48% (68.69-74.27), sensitivity of 73.31% (70.44-76.18), and precision of 91.93% (89.48-94.38).

    “The highest performance on this set was also achieved by the federated learning model, with AUC of 88.15% (86.38–89.91), mAP of 71.48% (68.69–74.27), sensitivity of 73.31% (70.44–76.18), and precision of 91.93% (89.48–94.38).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Using multicenter data for model training, the federated learning model obtained comparable AUC of 95.27% (93.98-96.51) as Individual-model-2, with slightly higher mAP of 91.28% (84.91-97.65), sensitivity of 93.54% (87.33-99.75), and precision of 84.21% (78.61-89.81).

    “Using multicenter data for model training, the federated learning model obtained comparable AUC of 95.27% (93.98–96.51) as Individual-model-2, with slightly higher mAP of 91.28% (84.91–97.65), sensitivity of 93.54% (87.33–99.75), and precision of 84.21% (78.61–89.81).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The joint model trained with all internal datasets performs slightly lower than model ensemble, with the AUC of 92.97% (91.71-94.21).

    “The joint model trained with all internal datasets performs slightly lower than model ensemble, with the AUC of 92.97% (91.71–94.21).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Compared with training from scratch, the transfer learning model increased mAP by 2.56% on internal testing set and 3.31%, 5.51%, 2.00% on three external sets, respectively.

    “Compared with training from scratch, the transfer learning model increased mAP by 2.56% on internal testing set and 3.31%, 5.51%, 2.00% on three external sets, respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    After excluding cases, 35 cases remained, and the model obtained an AUC of 88.15% (95% CI 86.38-89.91).

    “By doing this, 35 cases remained with the model obtaining AUC of 88.15% (95% CI 86.38–89.91).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The AI model trained on Hong Kong cohorts showed high performance not only on internal testing cases, but also on external, unseen, independent datasets collected from hospitals in Asia and Europe.

    “Importantly, the AI model trained on Hong Kong cohorts showed high performance not only on internal testing cases, but also on external, unseen, independent datasets collected from hospitals in Asia and Europe.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The severity of COVID-19 pneumonia differed across the patients involved in the study.

    “The scanner brands, imaging protocols, and patient populations varied across these multi-national centers, and the severity of COVID-19 pneumonia differed across the patients involved in the study”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingnone given

    The German cohort contained a few pleural effusion lesions, which are atypically seen in COVID-19, with relatively low contrast against normal lung tissue.

    “For example, the training data annotated ground-glass opacification and consolidation, while the German cohort contained a few pleural effusion lesions, which are atypically seen in COVID-19, with relatively low contrast against normal lung tissue.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The scanner brands, imaging protocols, and patient populations varied across the multi-national centers.

    “The scanner brands, imaging protocols, and patient populations varied across these multi-national centers, and the severity of COVID-19 pneumonia differed across the patients involved in the study”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Besides the joint model, all other five methods were free from data exchange or centralization in the multicenter study setting, thus protecting the privacy of the patient health data.

    “Besides the joint model, all other five methods were free from data exchange or centralization in the multicenter study setting, thus protecting the privacy of the patient health data.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Recent study has revealed the potential of federated learning models for generalizability outside federation on brain tumor application.

    “Recent study10 has revealed the potential of federated learning models for generalizability outside federation on brain tumor application.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [10] Sheller, M. J. et al. Federated learning in medicine: facilitating multi-institutional collaborations without sharing patient data. Sci. Rep. 10, 1–2 (2020). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The main findings of COVID-19 infection on CT scans are bilateral and peripheral ground-glass and consolidative pulmonary opacities.

    “The main findings of COVID-19 infection on CT scans are bilateral and peripheral ground-glass and consolidative pulmonary opacities14.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [14] Bernheim, A. et al. Chest CT findings in coronavirus disease-19 (COVID-19): relationship to duration of infection. Radiology 295, 685–691 (2020). source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    The data from all four external centers were used solely for testing purposes.

    “Note that the data from all four external centers were used solely for testing purposes.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Existing AI models to date are mostly designed for lung lesion segmentation using convolutional neural networks (CNNs), requiring dense pixel-wise labels through time-consuming, labor-intensive manual annotation from experts who are scarce during the crisis.

    “Existing AI models to date are mostly designed for lung lesion segmentation using convolutional neural networks (CNNs)15,16, requiring dense pixel-wise labels through time-consuming, labor-intensive manual annotation from experts who are scarce during the crisis.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [15] Zhang, K. et al. Clinically applicable AI system for accurate diagnosis, quantitative measurements, and prognosis of COVID-19 pneumonia using computed tomography. Cell 181, 1423–1433 (2020). source

      Not checked in the free run.

    • cites [16] Shi, F. et al. Review of artificial intelligence techniques in imaging data acquisition, segmentation and diagnosis for COVID-19. IEEE Rev. Biomed. Eng. 14, 4–15 (2020). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Multicenter collaborative research efforts have been expected to coordinate data sources for maximizing the potential of data-driven AI technologies.

    “Multicenter collaborative research efforts have been expected to coordinate data sources for maximizing the potential of data-driven AI technologies3,4.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [3] Peiffer-Smadja, N. et al. Machine learning for COVID-19 needs global collaboration and data-sharing. Nat. Mach. Intell. 2, 293–294 (2020). source

      Not checked in the free run.

    • cites [4] Luengo-Oroz, M. et al. Artificial intelligence cooperation to support the global response to COVID-19. Nat. Mach. Intell. 2, 295–297 (2020). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Premisecitation

    Enabling privacy protected data sharing across clinical centers is advocated as an essential pathway to promote collaborations internationally yet underexplored so far.

    “Enabling privacy protected data sharing across clinical centers is advocated as an essential pathway to promote collaborations internationally yet underexplored so far6,7.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [6] Kaissis, G. A., Makowski, M. R., Rückert, D. & Braren, R. F. Secure, privacy-preserving and federated machine learning in medical imaging. Nat. Mach. Intell. 2, 305–311 (2020). source

      Not checked in the free run.

    • cites [7] Rieke, N. et al. The future of digital health with federated learning. npj Digital Med. 3, 1–7 (2020). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation

    Validation of AI models on multiple, unseen, independent external cohorts has to be a crucial criteria for assessing scalable usability toward wide model sharing.

    “For the testing phase, validation of AI models on multiple, unseen, independent external cohorts has to be a crucial criteria for assessing scalable usability toward wide model sharing8,9.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [8] Shah, P. et al. Artificial intelligence and machine learning in clinical development: a translational perspective. npj Digital Med. 2, 1–5 (2019). source

      Not checked in the free run.

    • cites [9] Esteva, A. et al. A guide to deep learning in healthcare. Nat. Med. 25, 24–29 (2019). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    A major focus of AI fighting against COVID-19 is interpreting radiological images, mainly chest CT, which has been widely applied for detecting lung changes to inform patient management, assessment of severity, and monitoring of the disease.

    “A major focus of AI fighting against COVID-19 is interpreting radiological images, mainly chest CT which has been widely applied for detecting lung changes to inform patient management, assessment of severity, and monitoring of the disease11–13.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [11] Yan, L. et al. An interpretable mortality prediction model for COVID-19 patients. Nat. Mach. Intell. 2, 283–288 (2020). source

      Not checked in the free run.

    • cites [12] Hu, Y. et al. The challenges of deploying artificial intelligence models in a rapidly evolving pandemic. Nat. Mach. Intell. 2, 298–300 (2020). source

      Not checked in the free run.

    • cites [13] Burian, E. et al. Intensive care risk estimation in COVID-19 pneumonia based on clinical and imaging parameters: experiences from the munich cohort. J. Clin. Med. 9, 1514 (2020). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data

    The real-world heterogeneous environments of CT medical imaging in clinical practice could be reflected to a certain extent in this proof-of-concept study.

    “The real-world heterogeneous environments of CT medical imaging in clinical practice could be reflected to a certain extent in this proof-of-concept study.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    A reason for the model's poorer generalization on the German cohort may be the patient populations coming from different demographics.

    “A reason may be the patient populations coming from different demographics.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    The DeepLesion dataset is currently the most comprehensive open-source data for CT lesions, including 32,120 axial CT slices from 10,594 studies of 4427 unique patients.

    “The dataset is currently the most comprehensive open-source data for CT lesions, that includes 32,120 axial CT slices (from body parts of liver, lung, mediastinum, kidney, pelvis, bone, abdomen, and soft tissue) from 10,594 studies of 4427 unique patients.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [19] Yan, K., Wang, X., Lu, L. & Summers, R. M. DeepLesion: automated mining of large-scale lesion annotations and universal lesion detection with deep learning. J. Med. Imaging 5, 036501 (2018). source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductown data

    In the current proof-of-concept study, the relatively small numbers of patients did not impede the development of a deep learning model, because the numbers of individual lesions identified from all CT volumes were fairly large (14,095 overall in our datasets).

    “Nevertheless, it is worth noting that, in the current proof-of-concept study, the relatively small numbers of patients did not impede the development of a deep learning model, because the numbers of individual lesions identified from all CT volumes were fairly large (14,095 overall in our datasets).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The lesion annotation procedures differed between different clinical sites, causing concept shift where the manual annotations in the German cohort were not directly compatible with training data.

    “In addition, the lesion annotation procedures differed between different clinical sites causing what is known in machine learning as concept shift where the manual annotations in this cohort were not directly compatible with training data.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    This decentralized training strategy may be a key enabler of scalable AI-based technology during a pandemic when there is no time to set up complicated data-sharing agreements across institutions or even countries.

    “This decentralized training strategy may be a key enabler of scalable AI-based technology during a pandemic when there is no time to set up complicated data-sharing agreements across institutions or even countries.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    To our knowledge, this is a prior work to demonstrate the feasibility and effectiveness of federated learning for COVID-19 image analysis.

    “To our knowledge, this is a prior work to demonstrate the feasibility and effectiveness of federated learning for COVID-19 image analysis, where collaborative effort is especially valuable at the time of global crisis.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Multicenter training could enhance the generalizability of AI models, as previously demonstrated in other medical imaging scenarios.

    “This study hypothesized that multicenter training could enhance the generalizability of AI models, which has previously been demonstrated in other medical imaging scenarios25–27.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [25] Liu, Q., Dou, Q., Yu, L. & Heng, P. A. MS-Net: multi-site network for improving prostate segmentation with heterogeneous MRI data. IEEE Trans. Med. Imaging 39, 2713–2724 (2020). source

      Not checked in the free run.

    • cites [26] Zech, J. R. et al. Variable generalization performance of a deep learning model to detect pneumonia in chest radiographs: a cross-sectional study. PLoS Med. 15, e1002683 (2018). source

      Not checked in the free run.

    • cites [27] Chartrand, G. et al. Deep learning: a primer for radiologists. Radiographics 37, 2113–2131 (2017). source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    In addition to increased data scale, richer data diversity associated with imaging scanners and protocols may be equally important for reducing model bias thus improving generalizability.

    “To some extent, this could reflect that, in addition to increased data scale, richer data diversity associated with imaging scanners and protocols may be equally important for reducing model bias thus improving generalizability.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Ethical approvals were obtained in accordance with all relevant laws and regulations for each recruiting hospital.

    “The ethical approvals were obtained in accordance with all relevant laws and regulations for each recruiting hospital, i.e., PWH approved by the Joint Chinese University of Hong Kong - New Territories East Cluster Clinical Research Ethics Committee, PMH approved by the Kowloon West Cluster Research Ethics Committee, TMH approved by the New Territory West Cluster Research Ethics Committee.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The model pretrained on various other types of lesions in CT could capture general patterns of abnormalities, which is also applied for the novel disease of COVID-19 to some extent.

    “This supported that the model pretrained on various other types of lesions in CT could capture general patterns of abnormalities, which is also applied for the novel disease of COVID-19 to some extent.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given

    The expert in the loop acts as a safeguard against erroneous predictions such as false positives in non-suspicious scans.

    “At the same time, the expert in the loop acts as a safeguard against erroneous predictions such as false positives in non-suspicious scans.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Collaboration across multiple clinical centers is an essential component to pave the way for developing AI systems for wide-spread deployment, especially when faced with COVID-19 pandemic where multinational efforts are crucial.

    “In this sense, collaboration across multiple clinical centers is an essential component to pave the way for developing AI systems for wide-spread deployment, especially when faced with COVID-19 pandemic where multinational efforts are crucial.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: COVID-19 patients were generally too scarce for most single sites to train individual in-house AI models.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    The multinational validation confirms the potential for the approach while highlighting the real-world challenges in such studies.

    “Despite these limitations, we believe that our multinational validation confirms the potential for our approach while highlighting the real-world challenges in such studies.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    Concept shift as present in the German cohort can be avoided in future multicenter studies that may use federated learning when all sites follow a standard operating procedure.

    “It is also worth noting that concept shift as present in the German cohort can be avoided in future multicenter studies that may use federated learning when all sites follow a standard operating procedure.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    The wide generalizability to regional and international external cohorts, benefited from including diverse datasets, shows the promise of AI providing low-cost and scalable tools for lesion burden estimation to support clinical disease management.

    “The wide generalizability to regional and international external cohorts, benefited from including diverse datasets, shows the promise of AI providing low-cost and scalable tools for lesion burden estimation to support clinical disease management.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given

    The AI tool supports the expert by providing quantitative measurements during clinical decision-making.

    “In that way, the AI tool supports the expert by providing quantitative measurements during clinical decision-making.”
    Quote in the paperNot a substantial claim
39 of 39

  • The paper relies on this without support.

    There is little evidence reported to date on the generalization performance of decentrally developed AI models for widely collected COVID-19 cohorts, especially in multinational evaluation of heterogeneous patient cohorts.

    “However, there is still little evidence been reported to date on the generalization performance of decentrally developed AI models for widely collected COVID-19 cohorts, especially in the setting of multinational evaluation of heterogeneous patient cohorts.”

  • The paper relies on this without support.

    Detecting lesion bounding boxes requires annotations that are easier and quicker to obtain than dense pixel-wise labels while maintaining clinical utility of quantifying the burden of infection.

    “We instead consider detecting lesion bounding boxes, for which annotations are easier and quicker to obtain while maintaining clinical utility of quantifying the burden of infection.”

  • The paper relies on this without support.

    Artificial intelligence has the potential to provide access to accurate, low-cost, and scalable solutions in combating COVID-19 through automated analysis of patient data.

    “Artificial intelligence (AI) has the potential to provide access to accurate, low-cost, and scalable solutions in combating COVID-19 through automated analysis of patient data.”

  • The paper relies on this without support.

    Manual labeling of the lesions would be too laborious to extract an image-derived parameter like lesion burden.

    “especially given that manual labeling of the lesions would be too laborious to extract such an image-derived parameter like lesion burden.”

  • The paper relies on this without support.

    COVID-19 patients were generally too scarce for most single sites to train individual in-house AI models.

    “To a certain extent, this reflected the practical situation that COVID-19 patients were generally too scarce for most single sites to train individual in-house AI models.”
5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • A federated deep learning method for detecting COVID-19 lung abnormalities in CT is validated on external cohorts from Mainland China and Germany.

    “The internal datasets are collected from a total of 75 patients confirmed COVID-19 infection at three local hospitals in Hong Kong for FL simulations, and then the generalizability is validated on external cohorts from Mainland China and Germany.”

    From Federated learning for smart healthcare: A survey (2023) · cites this paper as [117]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Training-slice counts include held-out testing slices

    Minor

    The discussion describes 958, 4146, and 660 lesion-containing slices as used to train the three individual models. Table 2 identifies those as the full cohort totals; the respective training splits contain 676, 3333, and 521 slices.

1 of 1

The citation lists haven’t been fetched yet

References 33 · Cited by 278

Authors
Dou, Q., So, T. Y., Jiang, M., Liu, Q., Vardhanabhuti, V., Kaissis, G., Li, Z., Si, W., Lee, H. H. C., Yu, K., Feng, Z., Dong, L., Burian, E., Jungmann, F., Braren, R., Makowski, M., Kainz, B., Rueckert, D., Glocker, B., Yu, S. C. H., & Heng, P. A.
Affiliations
  • Chinese University of Hong Kong (HK)
  • University of Hong Kong (HK)
  • Imperial College London (GB)
  • Chinese Academy of Sciences (CN)
  • Princess Margaret Hospital (HK)
  • Tuen Mun Hospital (CN)
  • Peking University Shenzhen Hospital (CN)
  • Technical University of Munich (DE)
  • German Cancer Research Center (DE)
Corresponding author
  • Dou, Qi
  • Glocker, Ben
  • Yu, Simon C. H.
  • Heng, Pheng Ann
ORCID
Venue
npj Digital Medicine
Publisher
Springer Science and Business Media LLC
Published
2021-03-29
Volume
4
Issue
1
Pages
60-60
ISSN
2398-6352 (ISSN-L)
DOI
10.1038/s41746-021-00431-6
PMC
PMC8007806
OpenAlex
W3150684546
Semantic Scholar
fb37f2259f63bb0bffd6a001cdf0168b928dc488
Cited by
278
References
33
Citations by year
2026: 30 · 2025: 54 · 2024: 46 · 2023: 79 · 2022: 50 · 2021: 19
FWCI
25.60
Type
journal article
Language
en
Update notices
correction (2022-04-24)
Open access
Licence
Locations
  • npj Digital Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • mediaTUM – the media and publications repository of the Technical University Munich (Technical University Munich) (submittedVersion)
  • Spiral (Imperial College London) (submittedVersion, other-oa)
Funders
  • Deutsche Forschungsgemeinschaft
  • National Natural Science Foundation of China (61802385)
  • Chinese University of Hong Kong (MMT-p5-20)
  • Technische Universität München
  • Innovation and Technology Fund (ITS/426/17FP)
  • Shun Hing Institute of Advanced Engineering (MMT-p5-20)
  • H2020 European Research Council (757173)
Topics
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › COVID-19 diagnosis using AI
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
federated learning, COVID-19 pandemic, chest CT, lung abnormalities, data privacy, model generalization, external validation, privacy-preserving machine learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review