PreprintCited by 588Privacy-Preserving Technologies in DataStochastic Gradient Optimization TechniquesDistributed Sensor Networks and Detection Algorithms

FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization

Open access
Reisizadeh, Amirhossein, Mokhtari, Aryan, Hassani, Hamed, Jadbabaie, Ali, Pedarsani, Ramtin·2019·arXiv

Abstract

Federated learning is a distributed framework according to which a model is trained over a set of devices, while keeping data localized. This framework faces several systems-oriented challenges which include (i) communication bottleneck since a large number of devices upload their local updates to a parameter server, and (ii) scalability as the federated network consists of millions of devices. Due to these systems challenges as well as issues related to statistical heterogeneity of data and privacy concerns, designing a provably efficient federated learning method is of significant importance yet it remains challenging. In this paper, we present FedPAQ, a communication-efficient Federated Learning method with Periodic Averaging and Quantization. FedPAQ relies on three key features: (1) periodic averaging where models are updated locally at devices and only periodically averaged at the server; (2) partial device participation where only a fraction of devices participate in each round of the training; and (3) quantized message-passing where the edge nodes quantize their updates before uploading to the parameter server. These features address the communications and scalability challenges in federated learning. We also show that FedPAQ achieves near-optimal theoretical guarantees for strongly convex and non-convex loss functions and empirically demonstrate the communication-computation tradeoff provided by our method.

Integrity signals

Registry

DataCite · 10.48550/arxiv.1909.13014

Cites

37 of 37 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • MASM can reduce energy cost and delay while guaranteeing DL inference accuracy by optimizing workload assignment weights and computing capacities of VMs.

    “by optimizing the workload assignment weights and computing capacities of VMs, MASM [173] can reduce the energy cost and delay while guaranteeing the DL inference accuracy.”

Cites

Flagged first, then by citations

3 of the 40 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Reisizadeh, A., Mokhtari, A., Hassani, H., Jadbabaie, A., & Pedarsani, R.
Venue
arXiv
Publisher
arXiv
Published
2019
Pages
2021-2031
ISSN
2331-8422 (ISSN-L)
OpenAlex
W2975043678
Cited by
587
References
40
Citations by year
2025: 41 · 2024: 146 · 2023: 156 · 2022: 96 · 2021: 100 · 2020: 46 · 2019: 2
Type
posted content
Language
en
Open access
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
  • International Conference on Artificial Intelligence and Statistics
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
  • Physical Sciences › Computer Science › Computer Networks and Communications › Distributed Sensor Networks and Detection Algorithms
Keywords
federated learning, communication efficiency, scalability, computation-communication trade-off, strongly convex loss, non-convex loss

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review