CoChat
Sign Up
Cited by 45Privacy-Preserving Technologies in DataBlockchain Technology Applications and SecurityMachine Learning in Healthcare

Benchmarking PySyft federated learning framework on MIMIC-III dataset

Open access
Budrionis, Andrius, Miara, Magda, Miara, Piotr, Wilk, Szymon, Bellika, Johan Gustav·2021·IEEE Access
DOI 10.1109/access.2021.3105929
Open this paper in CoChat — free accountOpen PDF

Abstract

The adoption of the advanced data analytics methods has been limited in industries governed by strict data reuse regulations, such as healthcare. Barriers to data access and sharing have affected numerous research and development initiatives in healthcare resulting in major delays, extensive use of resources for data access and findings originating from datasets that are too small to be generalizable. Federated machine learning presents a solution to the problems health data analytics projects are facing by providing a way of complying with strict regulatory requirements without sacrificing privacy. Computing frameworks supporting federated machine learning are still in their infancy and their performance in realistic settings has been studied only to a limited extent. To expand the existing knowledge on federated learning in realistic deployment settings three groups of experiments comparing the performance of a neural network-based model trained in federated manner to that of an equivalent baseline model trained on centralized data storage were designed. Experiments were conducted on the MIMIC-III dataset and modelled a binary classification problem predicting in-hospital mortality. The effect that varying amounts of data, number of computational nodes, and data distribution in the federated network had on model performance and on training and inference durations were studied. Experiments demonstrated predictive performance comparable to that of the baseline for models trained in federated settings in terms of area under the ROC and F1 scores. Data distribution across computing nodes showed minimal to no effect on model performance or on training and inference durations. However, federated model training and inference took approximately 9 and 40 times longer, respectively, than the equivalent tasks executed in centralized settings. These results indicate that federated learning is a viable solution for enabling advanced data analytics in environments regulated by strict privacy requirements.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/access.2021.3105929

Cites

19 of 19 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Budrionis et al.'s empirical study using PySyft on a medical dataset demonstrated that the performance of machine learning models trained with federated learning is comparable to those trained on centralized data.

    “Their experimental results demonstrate that the performance of machine learning models trained with federated learning is comparable to those trained on centralized data.”

    From Federated learning for medical image analysis: A survey (2024) · cites this paper as [96]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. 2011). Institute of Medicine (US) and National Academy of Engineer- ing (US) Roundtable on Value & Science-Driven Health Care, Engi- neering a Learning Healthcare System: A Look at the Future: Work- shop Summary. Washington (DC): National Academies Press (US). Accessed: Nov. 18, 2015. [Online]. Available: http://www.ncbi.nlm.nih. gov/books/NBK61965/

Cited by

Sorted by citations

Related

Authors
Budrionis, A., Miara, M., Miara, P., Wilk, S., & Bellika, J. G.
Affiliations
  • University Hospital of North Norway (NO)
  • Poznań University of Technology (PL)
ORCID
Venue
IEEE Access
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2021
Volume
9
Pages
116869-116878
ISSN
2169-3536 (ISSN-L)
DOI
10.1109/access.2021.3105929
OpenAlex
W3194908699
Cited by
45
References
19
Citations by year
2026: 7 · 2025: 15 · 2024: 10 · 2023: 9 · 2022: 3 · 2021: 1
FWCI
2.02
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Access (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Research Council of Norway to Norwegian Centre for E-health Research, University Hospital of North Norway (248150/O70)
  • Institute of Computing Science at the Poznan University of Technology
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Information Systems › Blockchain Technology Applications and Security
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
Keywords
federated learning, MIMIC-III, in-hospital mortality prediction, healthcare data privacy, data distribution skew, neural network performance

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review