CoChat
Sign Up
Cited by 170Adversarial Robustness in Machine LearningPrivacy-Preserving Technologies in DataDigital Media Forensic Detection

The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index; none of the 23 sources its claims cite is checked for support yet.

Deep Model Poisoning Attack on Federated Learning

Open access
Zhou, Xingchen, Xu, Ming, Wu, Yiming, Zheng, Ning·2021·Future Internet
DOI 10.3390/fi13030073
Open this paper in CoChat — free accountOpen PDF

Abstract

Federated learning is a novel distributed learning framework, which enables thousands of participants to collaboratively construct a deep learning model. In order to protect confidentiality of the training data, the shared information between server and participants are only limited to model parameters. However, this setting is vulnerable to model poisoning attack, since the participants have permission to modify the model parameters. In this paper, we perform systematic investigation for such threats in federated learning and propose a novel optimization-based model poisoning attack. Different from existing methods, we primarily focus on the effectiveness, persistence and stealth of attacks. Numerical experiments demonstrate that the proposed method can not only achieve high attack success rate, but it is also stealthy enough to bypass two existing defense methods.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

The paper proposes an optimization-based model poisoning attack for federated learning that injects poisoning neurons into redundant network space and uses alternating minimization with regularization to improve effectiveness, persistence, and stealth. Experiments on MNIST and CIFAR-10 show high attack success rates and bypass of Krum and norm-bounded defenses.

Key findings

  • The proposed attack achieves over 90% attack success rate on CIFAR-10 in the multiple-shot scenario, higher than the backdoor attack in [4]; it also achieves over 90% attack success rate on MNIST against the norm-bounded defense.
  • In the single-shot scenario, the attack success rate of [4] drops dramatically, while the proposed attack falls slowly and retains higher main-task accuracy, indicating mitigation of catastrophic forgetting.
  • The attack bypasses two defenses: Krum [16] and the norm-bounded defense [22]; Krum rejects the model update in some rounds, but the adversary still maintains high adversarial-task accuracy when not selected.
  • Weight update distributions show malicious clients’ layer0, layer1, and layer3 differ from benign clients’, while layer2 is similar; the all-layer distribution is similar because layer2 has the most neurons.

Methods

  • Threat model: a single non-colluding malicious client in federated learning can directly modify local model updates but cannot control aggregation or know benign participants’ local training; the goal is targeted misclassification while preserving test accuracy.
  • Attack design: alternating minimization of a main task (clean data plus regularization to make malicious updates similar to benign updates) and an adversarial task (poisoning data plus Hessian-based structural regularization to avoid neurons important to the main task), then boosting the malicious update by factor n/η.
  • Experiments: MNIST and CIFAR-10 with non-I.I.D. distributions, 10,000 test images, LetNet and ResNet, |S_t|/N of 10/20 and 10/50, internal epochs 5 and 10, |D_p| = 100; comparisons against backdoor attack [4] and model poisoning [8], and defenses [16,22].

Limitations

  • Stated: time constraints greatly limit exploration; the malicious client must compute the Hessian matrix, which is non-trivial and time consuming, and a possible defense is to drop clients that cannot reply on time.
  • Stated: future work is needed to explore additional attacks and efficient mechanisms (e.g., meta-learning) for finding neurons to inject adversarial tasks.
  • Evident: evaluation is limited to MNIST and CIFAR-10 with specific architectures and parameters; effectiveness and persistence experiments assume the malicious client is selected each round; no real-world deployment or broader dataset evaluation is reported.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/fi13030073

Cites

51 of 51 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 of 23 cited sources checked · 0 supported · 0 flagged

Show all 35 claims

  • It may state two separate claims as one. 14
  • Probably not a substantial claim. 9
22 of 22

  • The authors state this as an assumption.

    The attacker cannot control the aggregation algorithm nor has knowledge of benign participants' local training.

    “The attacker cannot control the aggregation algorithm, which is used to combine participants' model updates into the shared global model, nor does it have any knowledge of the benign participants' local training.”

  • The authors state this as an assumption.

    The malicious client is non-colluding and has no partner to exchange information.

    “We assume a non-colluding malicious client, where the adversary has no partner to exchange any information.”

  • The paper relies on this without support.

    Catastrophic forgetting is an inevitable feature of neural network models when trained sequentially on multiple tasks.

    “catastrophic forgetting is an inevitable feature of neural network models, specifically when the network is trained sequentially on multiple tasks.”

  • The authors state this as an assumption.

    The attacker can directly influence the weights of the joint model and train in any way that benefits the attack.

    “as defined in standard federated learning framework, the attacker can directly influence the weights of the joint model and train in any way that benefits the attack (e.g., adopting alternating minimization training strategy, introducing adversarial task).”

  • The authors state this as an assumption.

    The malicious client is selected by the central server each round.

    “In this experiment, the malicious client is assumed to be selected each round”
    “We make the assumption that the malicious client is selected by central server each round in order to evaluate the effectiveness and persistence of attack.”

  • The paper relies on this without support.

    Only a small fraction of the clients' updates will be selected each round.

    “Because only a small fraction of the clients' updates will be selected each round, the effectiveness, persistence, and stealth is becoming more challenging for model poisoning attack in federated learning.”

  • The paper relies on this without support.

    The main idea behind Byzantine-robust defense methods is to detect abnormal weight updates and reject those models.

    “The main idea behind the Byzantine-robust defense method is to detect abnormal weight updates and reject those models.”

  • The paper relies on this without support.

    Adversarial techniques (e.g., backdoor, adversarial example) try to infer the model by creating a misleading minima.

    “The adversarial techniques (e.g., backdoor, adversarial example) try to inference the model by creating a misleading minima.”
8 of 8

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Byzantine-robust methods citation points to a ResNet paper

    Minor

    The discussion cites [44] as a Byzantine-robust federated learning method, but reference [44] is "Deep residual learning for image recognition."

  • ResNet citation points to a stereo-matching paper

    Minor

    The experiment setup cites [43] for ResNet, but [43] is a paper on stereo matching. The reference list instead gives the deep residual learning paper as [44].

  • LetNet citation points to a continual-learning paper

    Minor

    The experiment setup cites reference [42] for LetNet, but [42] is titled "Continual Learning Through Synaptic Intelligence."

  • Continual-learning citation points to a Byzantine-learning paper

    Minor

    Reference [35] is included among examples of continual-learning studies, but its listed title concerns a vulnerability of distributed learning in Byzantium.

  • Equation (2) is assigned to the wrong task

    Minor

    The model-update procedure calls Equation (2) the adversarial-task objective, but the paper identifies its loss as belonging to the main task; Equation (3) is presented as the adversarial-task objective.

  • Funding statement conflicts with acknowledged grants

    Minor

    The acknowledgments say the work was supported by two grants and a provincial project, while the funding section says the research received no external funding.

6 of 6

Citations

Cites

Flagged first, then by citations

Show all 43 references

Research works printed but not linked: 1

  1. Bhagoji, A.N., Chakraborty, S., Mittal, P., and Calo, S. (2018, January 3–8). Model poisoning attacks in federated learning. Proceedings of the Workshop on Security in Machine Learning (SecML), Collocated with the 32nd Conference on Neural Information Processing Systems (NeurIPS’18), Montréal, QC, Canada.

Cited by

Sorted by citations

Items 101–170 of 170

Related

See all

Metadata

Authors
Zhou, X., Xu, M., Wu, Y., & Zheng, N.
Affiliations
Hangzhou Dianzi University (CN)
Corresponding author
Xu, Ming
ORCID
Venue
Future Internet
Publisher
MDPI AG
Published
2021-03-14
Volume
13
Issue
3
Pages
73
ISSN
1999-5903 (ISSN-L)
DOI
10.3390/fi13030073
OpenAlex
W3136620885
Semantic Scholar
1bdb7429302500aa559c4e1653543802365abc47
Cited by
170
References
43
Citations by year
2026: 18 · 2025: 38 · 2024: 51 · 2023: 40 · 2022: 15 · 2021: 8
FWCI
10.08
Type
journal article
Language
en
Open access
Licence
Locations
  • Future Internet (publishedVersion, cc-by)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
Natural Science Foundation of China; Key Research and Development Plan Project of Zhejiang Province (61803135, 61702150; 2017C01065)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
Keywords
federated learning, model poisoning, deep learning, attack stealth, attack success rate, defense evasion, data confidentiality

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review