Cited by 292Privacy-Preserving Technologies in DataAdversarial Robustness in Machine LearningSecurity and Verification in Computing

Back to the Drawing Board: A Critical Evaluation of Poisoning Attacks on Production Federated Learning

Open access
Shejwalkar, Virat, Houmansadr, Amir, Kairouz, Peter, Ramage, Daniel·2022·2022 IEEE Symposium on Security and Privacy (SP)

Abstract

While recent works have indicated that federated learning (FL) may be vulnerable to poisoning attacks by compromised clients, their real impact on production FL systems is not fully understood. In this work, we aim to develop a comprehensive systemization for poisoning attacks on FL by enumerating all possible threat models, variations of poisoning, and adversary capabilities. We specifically put our focus on un-targeted poisoning attacks, as we argue that they are significantly relevant to production FL deployments. We present a critical analysis of untargeted poisoning attacks under practical, production FL environments by carefully characterizing the set of realistic threat models and adversarial capabilities. Our findings are rather surprising: contrary to the established belief, we show that FL is highly robust in practice even when using simple, low-cost defenses. We go even further and propose novel, state-of-the-art data and model poisoning attacks, and show via an extensive set of experiments across three benchmark datasets how (in)effective poisoning attacks are in the presence of simple defense mechanisms. We aim to correct previous misconceptions and offer concrete guidelines to conduct more accurate (and more realistic) research on this topic.

Integrity signals

Registry

Crossref · 10.1109/sp46214.2022.9833647

Cites

63 of 63 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Virat et al. argued that there are two kinds of label-flipping strategies, namely static label flipping (SLF) and dynamic label flipping (DLF).

    “Virat et al. [23] argued that there are two kinds of label-flipping strategies, namely static label flipping (SLF) and dynamic label flipping (DLF).”

    From Poisoning attacks in federated learning: A survey (2023) · cites this paper as [23]

  • To generate a malicious model, the attacker can use a specific dataset or train the model on a benign dataset by projected gradient ascent.

    “To generate a malicious model w k+1 0 , the attacker can use a specific dataset (e.g., different kinds of green cars labeled as birds [29]) or train the model in a benign dataset by projected gradient ascent (while the benign clients use stochastic gradient descent) algorithm [23], [67], [77].”

    From Poisoning attacks in federated learning: A survey (2023) · cites this paper as [23]

  • Model updates should not deviate too far from benign updates to avoid detection.

    “To avoid detection, Virat et al. [23] also state that model updates should not deviate too far from benign updates to avoid detection.”

    From Poisoning attacks in federated learning: A survey (2023) · cites this paper as [23]

  • Model poisoning attacks tend to be more effective in practice than data poisoning attacks because they can directly modify local model parameters.

    “However, model poisoning attacks tend to work better in practice because they can directly modify local model parameters [11], [23], [33], [40].”

    From Poisoning attacks in federated learning: A survey (2023) · cites this paper as [23]

  • The gradient ascent attack can be applied on a batch of target data samples all at the same time.

    “This attack can be applied on a batch of target data samples all at the same time [73].”

Cites

Flagged first, then by citations

1 of the 64 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Shejwalkar, V., Houmansadr, A., Kairouz, P., & Ramage, D.
Affiliations
  • University of Massachusetts Amherst (US)
  • Google (United States) (US)
Venue
2022 IEEE Symposium on Security and Privacy (SP)
Publisher
IEEE
Published
2022-05
Pages
1354-1371
ISSN
1081-6011 (ISSN-L)
OpenAlex
W4288057793
Cited by
292
References
64
Citations by year
2026: 53 · 2025: 78 · 2024: 97 · 2023: 52 · 2022: 12
FWCI
32.95
Type
conference paper
Language
en
Open access
Locations
  • IEEE Symposium on Security and Privacy (S&P) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Science Foundation (1553301, 1953786, 1739462)
  • U.S. Department of Defense
  • Defense Advanced Research Projects Agency
  • Naval Information Warfare Center Pacific
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
Keywords
federated learning, poisoning attacks, untargeted attacks, threat model, adversary capabilities, data poisoning, model poisoning, defense mechanisms, production systems

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review