Cited by 240Privacy-Preserving Technologies in DataIoT and Edge/Fog ComputingInternet of Things and AI

End-to-End Evaluation of Federated Learning and Split Learning for Internet of Things

Open access
Gao, Yansong, Kim, Minki, Abuadbba, Sharif, Kim, Yeonjae, Thapa, Chandra, Kim, Kyuyeon, Camtep, Seyit A., Kim, Hyoungshick +1 more·2020·2020 International Symposium on Reliable Distributed Systems (SRDS)

Abstract

Federated learning (FL) and split neural networks (SplitNN) are state-of-art distributed machine learning techniques to enable machine learning without directly accessing raw data on clients or end devices. In theory, such distributed machine learning techniques have great potential in distributed applications, in which data are typically generated and collected at the client-side while the collected data should be processed by the application deployed at the server-side. However, there is still a significant gap in evaluating the performance of those techniques concerning their practicality in the Internet of Things (IoT)-enabled distributed systems constituted by resource-constrained devices. This work is the first attempt to provide empirical comparisons of FL and SplitNN in real-world IoT settings in terms of learning performance and device implementation overhead. We consider a variety of datasets, different model architectures, multiple clients, and various performance metrics. For the learning performance (i.e., model accuracy and convergence time), we empirically evaluate both FL and SplitNN under different types of data distributions such as imbalanced and non-independent and identically distributed (non-IID) data. We show that the learning performance of SplitNN is better than FL under an imbalanced data distribution but worse than FL under an extreme non-IID data distribution. For implementation overhead, we mount both FL and SplitNN on Raspberry Pi devices and comprehensively evaluate their overhead, including training time, communication overhead, power consumption, and memory usage. Our key observations are that under the IoT scenario where the communication traffic is the primary concern, FL appears to perform better over SplitNN because FL has a significantly lower communication overhead compared with SplitNN. However, our experimental results also demonstrate that neither FL or SplitNN can be applied to a heavy model, e.g., with several million parameters, on resource-constrained IoT devices because its training cost would be too expensive for such devices. Source code is released and available: https://github.com/Minki-Kim95/Federated-Learning-and-Split-Learning-with-raspberry-pi.

Integrity signals

Registry

Crossref · 10.1109/srds51746.2020.00017

Cites

28 of 28 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Pruning can be an effective defense against backdoor attacks.

    “Plus, pruning can be an effective defense against backdoor attacks [66].”

    From Vulnerabilities in federated learning (2021) · cites this paper as [66]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Gao, Y., Kim, M., Abuadbba, S., Kim, Y., Thapa, C., Kim, K., Camtep, S. A., Kim, H., & Nepal, S.
Affiliations
  • Commonwealth Scientific and Industrial Research Organisation (AU)
  • Cyber Security Cooperative Research Centre,Australia
  • Data61, CSIRO,Syndey,Australia
Venue
2020 International Symposium on Reliable Distributed Systems (SRDS)
Publisher
IEEE
Published
2020-09
Pages
91-100
ISSN
1060-9857 (ISSN-L)
OpenAlex
W3105915864
Cited by
240
References
28
Citations by year
2026: 16 · 2025: 45 · 2024: 59 · 2023: 57 · 2022: 31 · 2021: 26 · 2020: 6
FWCI
11.46
Type
conference paper
Language
en
Open access
Locations
  • IEEE Symposium on Reliable Distributed Systems (SRDS) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
Australian Government
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
  • Physical Sciences › Computer Science › Information Systems › Internet of Things and AI
Keywords
federated learning, Internet of Things, resource-constrained devices, non-IID data, class imbalance, communication overhead, power consumption, memory usage, training duration, Raspberry Pi, distributed learning, end-to-end evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review