CoChat
Sign Up
Cited by 548Privacy-Preserving Technologies in DataAdversarial Robustness in Machine LearningCryptography and Data Security

Privacy and Robustness in Federated Learning: Attacks and Defenses

Open access
Lyu, Lingjuan, Yu, Han, Ma, Xingjun, Chen, Chen, Sun, Lichao, Zhao, Jun, Yang, Qiang, Yu, Philip S.·2024·IEEE Transactions on Neural Networks and Learning Systems
DOI 10.1109/tnnls.2022.3216981
Open this paper in CoChat — free accountOpen PDF

Abstract

As data are increasingly being stored in different silos and societies becoming more aware of data privacy issues, the traditional centralized training of artificial intelligence (AI) models is facing efficiency and privacy challenges. Recently, federated learning (FL) has emerged as an alternative solution and continues to thrive in this new reality. Existing FL protocol designs have been shown to be vulnerable to adversaries within or outside of the system, compromising data privacy and system robustness. Besides training powerful global models, it is of paramount importance to design FL systems that have privacy guarantees and are resistant to different types of adversaries. In this article, we conduct a comprehensive survey on privacy and robustness in FL over the past five years. Through a concise introduction to the concept of FL and a unique taxonomy covering: 1) threat models; 2) privacy attacks and defenses; and 3) poisoning attacks and defenses, we provide an accessible review of this important topic. We highlight the intuitions, key techniques, and fundamental assumptions adopted by various attacks and defenses. Finally, we discuss promising future research directions toward robust and privacy-preserving FL, and their interplays with the multidisciplinary goals of FL.

AI snapshot

Written from the paper’s full text.

TL;DR

This survey reviews privacy and robustness threats in federated learning over the past five years, organizing threat models, privacy attacks and defenses, and poisoning attacks and defenses. It highlights gradient leakage and malicious-participant/server vulnerabilities, compares defenses, and identifies open challenges including privacy–robustness conflicts and gaps in H2C/H2B and VFL/FTL.

Key findings

  • FL gradients can leak class representatives, membership, properties, and even training inputs and labels; DLG recovers raw images/texts, and iDLG extracts labels for differentiable models trained with cross-entropy loss and one-hot labels.
  • Byzantine attacks can let a single malicious participant control aggregation without defense; Krum theoretically withstands up to 33% adversaries and requires \(n \ge 2f+3\), while AUROR drops accuracy by only 3% even when 30% of users are adversarial.
  • Privacy defenses involve trade-offs: HE/SMC incur substantial communication and computation overhead; CDP has expected additive error \(\Theta(1/\epsilon)\), LDP requires at least \(\Omega(\sqrt{n}/\epsilon)\), and Bhowmick et al. needed more than 200 communication rounds with \(\epsilon=500\) on MNIST and \(\epsilon=5000\) on CIFAR10.
  • Many Byzantine-robust aggregators have breaking points—Krum \((n-2)/2n\), coordinate-wise median \(1/2\), Bulyan \((n-3)/4n\)—and are brittle against targeted/model poisoning; production H2C/H2B poisoning attacks may have little impact per Shejwalkar et al.

Methods

  • Design: comprehensive survey of privacy and robustness in FL over the past 5 years, using a taxonomy covering threat models, privacy attacks and defenses, and poisoning attacks and defenses.
  • Data/sample: literature review with no primary dataset; summarizes representative attacks, defenses, and comparative tables across HFL/H2B/H2C, homogeneous/heterogeneous architectures, and briefly VFL/FTL.
  • Analysis: conceptual synthesis and comparative tables rather than empirical meta-analysis, highlighting assumptions, key techniques, breaking points, and future research directions.

Limitations

  • The paper states that the survey mainly focuses on HFL and homogeneous architectures, and mainly on training-phase attacks and defenses; VFL/FTL and test-phase privacy/robustness remain less explored.
  • The paper states that many attacks assume sophisticated technical capabilities, unlimited computational resources, and repeated participant selection, which are impractical in H2C; defenses are brittle, and no existing work satisfies all six goals simultaneously, with an inherent conflict between privacy and robustness.
  • Evident from the text: as a survey, it provides no new experiments or quantitative validation; numerical claims are drawn from cited works rather than this paper’s own empirical results.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/tnnls.2022.3216981

Cites

Statistics

Nothing to recompute.

Claims the paper makes

0 of 197 cited sources checked · 0 supported · 0 flagged

  • Interpretationnone given
    § B. Categorization of Federated Learning based on Architectures · p. 2

    The sharing methods discussed did not provide defense against privacy and poisoning attacks, the two main sources of threats to federated learning.

    “In summary, all the above sharing methods did not provide defense against privacy and poisoning attacks -two main source of threats to FL.”
    Quote in the paperSubstantial claim
  • Interpretationreasoning
    § D. Secure FL · p. 3

    The noise addition required by differential privacy is hard to execute in federated learning.

    “The noise addition required by DP is also hard to execute in FL.”
    Quote in the paperNot a substantial claim
  • Premisecitation
    § C. Threats to FL · p. 2

    By simply observing the gradients, a malicious attacker can successfully steal the training data.

    “Recent works further show that, by simply observing the gradients, a malicious attacker can successfully steal the training data [29], [32].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [29], [32]

      Not checked in the free run.

  • Interpretationreasoning
    § D. Secure FL · p. 3

    Homomorphic encryption and secure multiparty computation may not be applicable to large-scale federated learning because they incur substantial communication and computation overhead.

    “However, HE and SMC may not be applicable to large-scale FL, as they incur substantial communication and computation overhead.”
    Quote in the paperNot a substantial claim
  • Interpretationreasoning
    § C. Threats to FL · p. 2

    Federated learning may not always provide sufficient privacy and robustness guarantees.

    “Nevertheless, recent works have demonstrated that FL may not always provide sufficient privacy and robustness guarantees.”
    Quote in the paperSubstantial claim
  • May combine two claims. 76
  • Classified as probably not a substantial claim. 42
  • The quote only partly states this claim. 1

  • May combine two claims.

    Premisecitation
    § B. Categorization of Federated Learning based on Architectures · p. 2

    Sharing model prediction instead of model parameters or updates removes the obstacle of heterogeneous model architectures and eliminates the risk of white-box inference attacks in conventional federated learning.

    “Sharing model prediction instead of model parameters or updates removes this obstacle and eliminates the risk of white-box inference attacks in conventional federated learning [26], [25].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [26], [25]

      Not checked in the free run.

  • May combine two claims.

    Premisereasoning
    § C. Threats to FL · p. 2

    Malicious participants can attack the convergence of the global model or implant backdoor triggers into the global model by deliberately altering their local data or their gradient uploads.

    “Malicious participants can attack the convergence of the global model or implant backdoor triggers into the global model by deliberately altering their local data (data poisoning) or their gradients uploads (model poisoning).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § B. Categorization of Federated Learning based on Architectures · p. 2

    Sharing logits reduces communication costs without significantly affecting utility.

    “Another obvious benefit of sharing logits is the reduced communication costs, without significantly affecting utility [25].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [25] Li, 2019, FedMD: Heterogenous federated learning via model distillation, arXiv:1910.03581

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation
    § I. INTRODUCTION · p. 1

    Federated learning enables a multitude of participants to construct a joint machine learning model without exposing their private training data.

    “FL enables a multitude of participants to construct a joint ML model without exposing their private training data [4], [5].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [4], [5]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § C. Threats to FL · p. 3

    In centralized learning, the server is responsible for all the participants' privacy and model robustness.

    “In centralized learning, the server is responsible for all the participants' privacy and model robustness.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § C. Threats to FL · p. 3

    In federated learning, any participant can attack the server and spy on other participants, sometimes even without involving the server.

    “However, in FL, any participant can attack the server and spy on other participants, which sometimes even without involving the server.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § C. Threats to FL · p. 2

    Federated learning systems are vulnerable to both data poisoning and model poisoning attacks.

    “In terms of robustness, FL systems are vulnerable to both data poisoning [33], [34] and model poisoning attacks [35], [36], [37], [38].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [33], [34]

      Not checked in the free run.

    • cites [35], [36], [37], [38]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § C. Threats to FL · p. 2

    Communicating gradients throughout the training process can reveal sensitive information, even cause deep leakage, either to a third party or the central server.

    “In terms of privacy leakage, communicating gradients throughout the training process can reveal sensitive information [27], [28], even cause deep leakage [29], either to a third party or the central server [7], [30].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [27], [28]

      Not checked in the free run.

    • cites [29] Zhu, Deep leakage from gradients

      Not checked in the free run.

    • cites [7], [30]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § I. INTRODUCTION · p. 1

    Federated learning can handle unbalanced and non-independent and identically distributed data that naturally arise in the real world.

    “It can also handle unbalanced and non-independent and identically distributed (non-I.I.D.) data, which naturally arise in the real world [6].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [6] McMahan, 2017, Communication-efficient learning of deep networks from decentralized data

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation
    § Privacy and Robustness in Federated Learning: Attacks and Defenses · p. 1

    Existing federated learning protocol designs are vulnerable to adversaries within or outside of the system, compromising data privacy and system robustness.

    “Existing FL protocol designs have been shown to be vulnerable to adversaries within or outside of the system, compromising data privacy and system robustness.”
    • Also stated in: § C. Threats to FL · p. 2 · plain
      “Existing FL protocol designs are vulnerable to: (1) a malicious server who aims to infer sensitive information from individual updates over time, tamper with the training process or control the view of the participants on the global parameters; (2) any adversarial participant who can infer other participants' sensitive information, tamper the global parameter aggregation or poison the global model”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § III. PRIVACY ATTACKS · p. 5

    An attacker can infer labels from shared gradients and recover the original training samples without any prior knowledge about the training data.

    “Even worse, an attacker can infer labels from the shared gradients and recover the original training samples without any prior knowledge about the training data [29].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [29] Zhu, Deep leakage from gradients

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § B. Inferring Membership · p. 6

    The gradient ascent attack can be applied on a batch of target data samples all at the same time.

    “This attack can be applied on a batch of target data samples all at the same time [73].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [73] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § B. Inferring Membership · p. 6

    In federated learning, the adversary can infer if a particular sample belongs to the private training data of a particular participant or any participant.

    “In FL, the adversary can infer if a particular sample belongs to the private training data of a particular participant (if the target update is from a single participant) or any participant (if the target update is the aggregate).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationcitation
    § D. Inferring Training Inputs and Labels · p. 6

    Inference attacks highlight the need for gradient protection in FL, possibly through various privacy-preserving mechanisms.

    “These inference attacks highlight the need for gradient protection in FL, possibly through various privacy-preserving mechanisms [3] detailed in Section IV.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [3] Yang, 2019, Federated learning, Synth. Lect. Artif. Intell. Mach. Learn.

      Not checked in the free run.

  • May combine two claims.

    Premisereasoning
    § A. Inferring Class Representatives · p. 5

    The GAN attack assumes the entire training corpus for a given class comes from a single participant, so the GAN-constructed representatives are similar to the training data only when all class members are similar.

    “It should be noted that GAN attack assumes the entire training corpus for a given class comes from a single participant, which means the GAN-constructed representatives are similar to the training data only when all class members are similar.”
    Quote in the paperNot a substantial claim
    • The authors state this as an assumption: The GAN attack assumes the entire training corpus for a given class comes from a single participant.

  • May combine two claims.

    Premisecitation
    § D. Robustness: Untargeted v.s. Targeted · p. 5

    The targeted poisoning attack induces the model to output the target label specified by the adversary for particular testing examples, while the testing error for other testing examples is unaffected.

    “The targeted poisoning attack induces the model to output the target label specified by the adversary for particular testing examples, while the testing error for other testing examples is unaffected [35], [36], [37], [38].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [35], [36], [37], [38]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § IV. DEFENSES AGAINST PRIVACY ATTACKS · p. 6

    Privacy preservation in federated learning can be more challenging than in machine learning due to sporadic access to power and network connectivity, statistical heterogeneity in the data, etc.

    “While privacy preservation has been extensively studied in the machine learning community, privacy preservation in federated learning can be more challenging due to the sporadic access to power and network connectivity, statistical heterogeneity in the data, etc.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Privacy preservation in federated learning is more challenging than in centralized machine learning due to sporadic access to power and network connectivity and statistical heterogeneity in the data.

  • Classified as probably not a substantial claim.

    Premisenone given
    § III. PRIVACY ATTACKS · p. 5

    In deep learning models, gradients of a given layer are computed based on the layer's features and the error from the layer after.

    “In deep learning models, gradients of a given layer are computed based on the layer's features and the error from the layer after ( i.e. , backpropagation).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given
    § D. Robustness: Untargeted v.s. Targeted · p. 5

    The untargeted poisoning attack aims to arbitrarily compromise the integrity of the target model.

    “The untargeted poisoning attack aims to arbitrarily compromise the integrity of the target model.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § D. Robustness: Untargeted v.s. Targeted · p. 5

    Byzantine attack is one type of untargeted poisoning attack that uploads arbitrarily malicious gradients to the server so as to cause the failure of the global model.

    “Byzantine attack is one type of the untargeted poisoning attacks that uploads arbitrarily malicious gradients to the server so as to cause the failure of the global model [69], [22], [56], [39], [70].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [69], [22], [56], [39], [70]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § B. Training Phase v.s. Inference Phase · p. 4

    During the training phase, the attacker can run data poisoning attacks to compromise the integrity of the training dataset or model poisoning attacks to compromise the integrity of the learning process.

    “During the training phase, the attacker can run data poisoning attacks to compromise the integrity of the training dataset [37], [35], [63], [64], [65], [66], or model poisoning attacks to compromise the integrity of the learning process [36], [67].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [37], [35], [63], [64], [65], [66]

      Not checked in the free run.

    • cites [36], [67]

      Not checked in the free run.

  • May combine two claims.

    Premisenone given
    § B. Inferring Membership · p. 6

    In the active case, the attacker can tamper with the federated learning model training protocol and perform a more powerful attack against other participants.

    “In the active case, the attacker can tamper with the FL model training protocol and perform a more powerful attack against other participants.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § III. PRIVACY ATTACKS · p. 5

    Although federated learning prevents participants from directly sharing their private data, exchanging gradients in federated learning can leak sensitive information about participants' private data to either passive or active attackers.

    “Although FL prevents the participants from directly sharing their private data, a series of works have demonstrated that exchanging gradients in FL can also leak sensitive information about the participants' private data to either passive or active attackers [71], [72], [28], [29], [73].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [71], [72], [28], [29], [73]

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § E. Motivation of this Survey and Our Contribution · p. 3

    Very few efforts have categorized privacy and robustness threats to federated learning in a systemic manner, and these threats have not been systematically explored.

    “Although these works attempt to discover the vulnerabilities of FL and aim to enhance the privacy and system robustness of FL, there are very few efforts for categorizing them in a systemic manner, and privacy and robustness threats to FL have not been systematically explored.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § C. Inferring Properties · p. 6

    Property inference attacks assume that the adversary has auxiliary training data that are correctly labelled with the target property.

    “Property inference attacks assume that the adversary has auxiliary training data that are correctly labelled with the target property.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [28] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationnone given
    § D. Inferring Training Inputs and Labels · p. 6

    Most inference attacks assume that the adversarial participants can be selected to update the global model in many rounds of the FL training process.

    “Moreover, most attacks assume that the adversarial participants can be selected (to update the global model) in many rounds of the FL training process.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § III. PRIVACY ATTACKS · p. 5

    Gradients can cause privacy leakage because they are derived from participants' private training data, and a learning model can be considered as a representation of the high-level statistics of the dataset it was trained on.

    “The reason why gradients can cause privacy leakage is that the gradients are derived from the participants' private training data, and a learning model can be considered as a representation of the high-level statistics of the dataset it was trained on [75].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [75] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisereasoning
    § D. Secure FL · p. 3

    A defense method for federated learning has to be robust to both data poisoning and model poisoning attacks.

    “Second, the defense method has to be robust to both data poisoning and model poisoning attacks.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § D. Inferring Training Inputs and Labels · p. 6

    Inference attacks generally assume that the adversaries possess sophisticated technical capabilities and unlimited computational resources.

    “In summary, inference attacks generally assume that the adversaries possess sophisticated technical capabilities and unlimited computational resources.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § B. Inferring Membership · p. 6

    Attackers in a federated learning system can conduct both active and passive membership inference attacks.

    “Attackers in a FL system can conduct both active and passive membership inference attacks [73], [28].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [73], [28]

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning
    § C. Inferring Properties · p. 6

    The assumption of auxiliary training data in property inference attacks may limit its applicability in H2C.

    “The assumption of auxiliary training data in property inference attacks may limit its applicability in H2C.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § A. Insider v.s. Outsider · p. 4

    Insider attacks are generally more dangerous than outsider attacks because insider status strictly enhances the capability of the adversary.

    “Insider attacks are generally more dangerous than outsider attacks, as it strictly enhances the capability of the adversary.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § D. Inferring Training Inputs and Labels · p. 6

    In FL, the assumptions of inference attacks are generally not practical in H2C scenarios, but more likely happen in H2B scenarios.

    “In FL, these assumptions are generally not practical in H2C scenarios, but more likely happen in H2B scenarios.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: In H2C scenarios, participants generally lack the sophisticated technical capabilities and unlimited computational resources assumed by inference attacks.

  • May combine two claims.

    Premisecitation
    § C. Inferring Properties · p. 6

    An adversary can launch both passive and active property inference attacks to infer certain properties of other participants' training data.

    “An adversary can launch both passive and active property inference attacks to infer certain properties of other participants' training data [28].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [28] source

      Not checked in the free run.

  • May combine two claims.

    Premisereasoning
    § C. Privacy-Preservation through Differential Privacy · p. 9

    In DDP, only the sum of the individually released statistics is (ε, δ)-differentially private but not the individually released statistic.

    “However, in DDP, only the sum of the individually released statistics is ( ϵ, δ ) -differentially private but not the individually released statistic, i.e. , ∑ r i is sufficient for this level of differential privacy, but individual noise r i alone is not sufficient, thus x i + r i cannot be released directly.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning
    § C. Privacy-Preservation through Differential Privacy · p. 9

    Theoretically, DDP offers the same utility as CDP, as the total amount of noise is the same.

    “Theoretically, DDP offers the same utility as CDP, as the total amount of noise is the same.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The total amount of noise in DDP is the same as in CDP.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given
    § A. Untargeted Attacks · p. 9

    Untargeted poisoning attacks aim to arbitrarily compromise the integrity of the target model.

    “Untargeted poisoning attacks aim to arbitrarily compromise the integrity of the target model.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § A. Privacy Preservation through Homomorphic Encryption · p. 7

    Polynomial approximations need to be made to evaluate non-linear functions in machine learning algorithms, resulting in a trade-off between utility and privacy.

    “Moreover, polynomial approximations need to be made to evaluate non-linear functions in machine learning algorithms, resulting in a trade-off between utility and privacy [79], [80].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [79] source

      Not checked in the free run.

    • cites [80] Fang, Local model poisoning attacks to Byzantine-robust federated learning

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § A. Privacy Preservation through Homomorphic Encryption · p. 6

    Fully homomorphic encryption can support arbitrary computation on ciphertexts, but is less efficient.

    “Fully homomorphic encryption can support arbitrary computation on ciphertexts, but is less efficient [92].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [92] Shah, 2021, Adversarial training in communication constrained federated learning, arXiv:2103.01319

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § B. Privacy-Preservation through SMC · p. 7

    SecureML comes at a cost of high computation and communication overhead, which may hamper participants' interest to collaborate.

    “However, this comes at a cost of high computation and communication overhead, which may hamper participants' interest to collaborate.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [89] Szegedy, 2013, Intriguing properties of neural networks, arXiv:1312.6199

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Untargeted Attacks · p. 9

    Byzantine attack is one type of untargeted poisoning attacks that uploads arbitrarily malicious gradients to the server so as to cause the failure of the global model.

    “Byzantine attack is one type of untargeted poisoning attacks that uploads arbitrarily malicious gradients to the server so as to cause the failure of the global model [69], [22], [56], [39], [70].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [69] source

      Not checked in the free run.

    • cites [22] source

      Not checked in the free run.

    • cites [56] Chan, 2019, Poison as a cure: Detecting & neutralizing variable-sized backdoor attacks in deep neural networks, arXiv:1911.08040

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § C. Privacy-Preservation through Differential Privacy · p. 7

    CDP is geared to tackle thousands of users for training to converge and achieve an acceptable trade-off between privacy and accuracy, resulting in a convergence problem with a small number of participants.

    “However, CDP is geared to tackle thousands of users for training to converge and achieve an acceptable trade-off between privacy and accuracy [7], resulting in a convergence problem with a small number of participants [110].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [7] McMahan, Learning differentially private recurrent language models

      Not checked in the free run.

    • cites [110] Nguyen, Input-aware dynamic backdoor attack

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning
    § C. Privacy-Preservation through Differential Privacy · p. 8

    The assumption of a trusted server in CDP is ill-suited in many applications as it constitutes a single point of failure for data breaches, and saddles the trusted curator with legal and ethical obligations to keep the user data secure.

    “Meanwhile, the assumption of a trusted server in CDP is ill-suited in many applications as it constitutes a single point of failure for data breaches, and saddles the trusted curator with legal and ethical obligations to keep the user data secure.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given
    § C. Privacy-Preservation through Differential Privacy · p. 7

    Differential privacy trades off privacy and accuracy by perturbing the data in a way that is computationally efficient, does not allow an attacker to recover the original data, and does not severely affect the utility.

    “In comparison with encryption based approaches, differential privacy trades off privacy and accuracy by perturbing the data in a way that (i) is computationally efficient, (ii) does not allow an attacker to recover the original data, and (iii) does not severely affect the utility.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § C. Privacy-Preservation through Differential Privacy · p. 9

    DDP avoids placing trust in any server, and offers better utility than LDP.

    “Therefore, DDP avoids placing trust in any server, and offers better utility than LDP.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § C. Privacy-Preservation through Differential Privacy · p. 9

    PATE requires a lot of participants to achieve reasonable accuracy, and each participant needs to have enough data to train an accurate model, which might not hold in FL system, where the data distribution of participants might be highly unbalanced, making this approach unsuitable to FL system.

    “PATE requires a lot of participants to achieve reasonable accuracy, and each participant needs to have enough data to train an accurate model, which might not hold in FL system, where the data distribution of participants might be highly unbalanced, making this approach unsuitable to FL system.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The data distribution of participants in FL might be highly unbalanced.

  • May combine two claims.

    Premisecitation
    § A. Privacy Preservation through Homomorphic Encryption · p. 6

    Somewhat homomorphic encryption and partially homomorphic encryption are more efficient but are specified by a limited number of operations.

    “On the other hand, somewhat homomorphic encryption and partially homomorphic encryption are more efficient but are specified by a limited number of operations [97], [98], [91], [90].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [97] Guo, 2022, Threats to pre-trained language models: Survey and taxonomy, arXiv:2202.06862

      Not checked in the free run.

    • cites [98] source

      Not checked in the free run.

    • cites [91] Hong, 2021, Federated robustness propagation: Sharing robustness in heterogeneous federated learning, arXiv:2106.10196

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § V. POISONING ATTACKS · p. 9

    Poisoning attacks can be broadly classified into untargeted poisoning attacks and targeted poisoning attacks.

    “Depending on the attacker's objective, poisoning attacks can be broadly classified into two categories: 1) untargeted poisoning attacks [123], [124], [125], [70], [67]; and 2) targeted poisoning attacks [126], [127], [35], [36], [128], [129], [130], [131].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [123] source

      Not checked in the free run.

    • cites [126] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § C. Privacy-Preservation through Differential Privacy · p. 9

    Distributed Differential Privacy (DDP) bridges the gap between LDP and CDP, while ensuring the privacy of each individual by combining with cryptographic protocols.

    “Distributed Differential Privacy (DDP) bridges the gap between LDP and CDP, while ensuring the privacy of each individual by combining with cryptographic protocols [101], [102], [103], [104], [30].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [101] Nelson, Exploiting machine learning to subvert your spam filter

      Not checked in the free run.

    • cites [102] source

      Not checked in the free run.

    • cites [103] Chen, 2017, Targeted backdoor attacks on deep learning systems using data poisoning, arXiv:1712.05526

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given
    § V. POISONING ATTACKS · p. 9

    Poisoning attacks aim to compromise the system robustness.

    “Different from privacy attacks that are targeting at data privacy, poisoning attacks aim to compromise the system robustness.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § C. Privacy-Preservation through Differential Privacy · p. 8

    The gap between CDP and LDP error is essential for eliminating the trust in the centralized server, and cannot be removed by algorithmic improvement.

    “This gap is essential for eliminating the trust in the centralized server, and cannot be removed by algorithmic improvement [118].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [118] Geyer, 2017, Differentially private federated learning: A client level perspective, arXiv:1712.07557

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given
    § B. Privacy-Preservation through SMC · p. 7

    All cryptography based protocols prevent anyone from auditing participants' updates to the joint model, which leaves spaces for malicious participants to attack.

    “Lastly, all the cryptography based protocols prevent anyone from auditing participants' updates to the joint model, which leaves spaces for the malicious participants to attack.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § V. POISONING ATTACKS · p. 9

    Model poisoning subsumes data poisoning in FL settings, as data poisoning attacks eventually change a subset of updates sent to the model at any given iteration.

    “In fact, model poisoning subsumes data poisoning in FL settings, as data poisoning attacks eventually change a subset of updates sent to the model at any given iteration.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § C. Privacy-Preservation through Differential Privacy · p. 9

    DDP necessitates the help of SMC to maintain utility and ensure aggregator obliviousness.

    “Therefore, DDP necessitates the help of SMC to maintain utility and ensure aggregator obliviousness, as evidenced in [101], [102], [103], [104], [30], [40], [88].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [101] Nelson, Exploiting machine learning to subvert your spam filter

      Not checked in the free run.

    • cites [102] source

      Not checked in the free run.

    • cites [103] Chen, 2017, Targeted backdoor attacks on deep learning systems using data poisoning, arXiv:1712.05526

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § C. Privacy-Preservation through Differential Privacy · p. 7

    CDP can achieve acceptable accuracy only with a large number of participants, thus not applicable to H2B with relatively a small number of participants.

    “Moreover, CDP can achieve acceptable accuracy only with a large number of participants, thus not applicable to H2B with relatively a small number of participants.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § B. Privacy-Preservation through SMC · p. 7

    SMC cannot fully guarantee protection from information leakage, which requires additional differential privacy techniques to be incorporated into the multi-party protocol.

    “That said, SMC cannot fully guarantee protection from information leakage, which requires additional differential privacy techniques to be incorporated into the multi-party protocol to address such concerns [101], [102], [103], [104].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [101] Nelson, Exploiting machine learning to subvert your spam filter

      Not checked in the free run.

    • cites [102] source

      Not checked in the free run.

    • cites [103] Chen, 2017, Targeted backdoor attacks on deep learning systems using data poisoning, arXiv:1712.05526

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § B. Privacy-Preservation through SMC · p. 7

    SMC techniques ensure a high level of privacy and accuracy, at the expense of high computation and communication overhead, thereby doing a disservice to attracting participation.

    “In general, SMC techniques ensure a high level of privacy and accuracy, at the expense of high computation and communication overhead, thereby doing a disservice to attracting participation.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given
    § A. Defenses against Untargeted Attacks · p. 12

    RFA can only handle a few types of poisoning attackers and is not applicable to Byzantine attacks.

    “Unfortunately, RFA can only handle a few types of poisoning attackers, but not applicable to Byzantine attacks.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation
    § A. Defenses against Untargeted Attacks · p. 12

    MultiKrum is not robust against untargeted poisoning because it is based on the distance between each gradient vector and the mean vector, while the mean vector is not robust against untargeted poisoning.

    “This is because Multi-Krum is based on the distance between each gradient vector and the mean vector, while the mean vector is not robust against untargeted poisoning.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Defenses against Untargeted Attacks · p. 11

    Krum uses the Euclidean distance to determine which gradient contributions should be removed.

    “Krum uses the Euclidean distance to determine which gradient contributions should be removed”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [22] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § B. Targeted Attacks · p. 11

    Clean-label attacks are arguably stealthier as they do not change the labels.

    “Clean-label attacks are arguably stealthier as they do not change the labels.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § B. Defenses against Targeted Attacks · p. 12

    Detection methods exploit activation statistics or model properties to determine whether a model is backdoored or whether a training/test example is a backdoor example.

    “Detection methods exploit activation statistics or model properties to determine whether a model is backdoored [151], [152], or whether a training/test example is a backdoor example [47].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [151] source

      Not checked in the free run.

    • cites [152] source

      Not checked in the free run.

    • cites [47] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § A. Defenses against Untargeted Attacks · p. 11

    MultiKrum interpolates between Krum and averaging, combining Krum's resilience properties with averaging's convergence speed.

    “MultiKrum is a variant of Krum, which intuitively interpolates between Krum and averaging, thereby combining the resilience properties of Krum with the convergence speed of averaging.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [22] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § A. Defenses against Untargeted Attacks · p. 11

    Krum filters outliers based on the entire update vector but does not filter coordinate-wise outliers.

    “Essentially, Krum filters outliers based on the entire update vector, but does not filter coordinate-wise outliers.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationnone given
    § B. Targeted Attacks · p. 11

    The impact on the FL model depends on the extent to which the backdoor participants engage in the attacks, and the amount of training data being poisoned.

    “The impact on the FL model depends on the extent to which the backdoor participants engage in the attacks, and the amount of training data being poisoned.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation
    § A. Defenses against Untargeted Attacks · p. 11

    AUROR is based on the observation that indicative features from the majority of honest users will exhibit a similar distribution while those from malicious users will exhibit an anomalous distribution.

    “AUROR is based on the observation that indicative features (most important model features) from the majority of honest users will exhibit a similar distribution while those from malicious users will exhibit an anomalous distribution.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [139] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § B. Targeted Attacks · p. 10

    Targeted attacks are more difficult to conduct than untargeted attacks as the attacker has a specific goal to achieve.

    “Generally, targeted attacks is more difficult to conduct than untargeted attacks as the attacker has a specific goal to achieve.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § A. Defenses against Untargeted Attacks · p. 12

    Byzantine-robust aggregation rules may ensure that the influence of Byzantine workers in any single round is limited, but attackers can couple attacks across rounds, moving weights significantly away from the desired direction and lowering model quality.

    “[133], [70] showed that while the Byzantine-robust aggregation rules may ensure that the influence of the Byzantine workers in any single round is limited, the attackers can couple their attacks across the rounds, moving weights significantly away from the desired direction and thus achieve the goal of lowering the model quality.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [133] source

      Not checked in the free run.

    • cites [70] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation
    § A. Defenses against Untargeted Attacks · p. 12

    DRACO assumes that each participant can access other participants' data, limiting its practicability in FL.

    “However, DRACO assumes that each participant can access other participants' data, limiting its practicability in FL.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [145] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § A. Defenses against Untargeted Attacks · p. 12

    Yin et al. proposed two robust distributed gradient descent algorithms, one based on coordinate-wise median and the other based on coordinate-wise trimmed mean.

    “Yin et al. [23] proposed two robust distributed gradient descent algorithms, one based on coordinate-wise median, and the other based on the coordinatewise trimmed mean.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [23] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § A. Defenses against Untargeted Attacks · p. 11

    Krum can theoretically withstand poisoning attacks of up to 33% adversaries in the participant pool, requiring n ≥ 2f + 3.

    “can theoretically withstand poisoning attacks of up to 33% adversaries in the participant pool, i.e. , given n agents of which f are Byzantine, Krum requires that n ≥ 2 f + 3 .”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [22] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § A. Defenses against Untargeted Attacks · p. 11

    Krum removes the top f contributions to the model that are furthest from the mean participant contribution from the aggregation.

    “Blanchard et al. proposed Krum [22], in which, the top f contributions to the model that are furthest from the mean participant contribution are removed from the aggregation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [22] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § B. Targeted Attacks · p. 11

    A system that allows participants to join and leave is susceptible to Sybil attacks, in which an attacker gains influence by joining a system to inject c fake participants into the FL system or compromise c benign participants.

    “A system that allows participants to join and leave is susceptible to Sybil attacks [138], in which an attacker gains influence by joining a system to inject c fake participants into the FL system or compromise c benign participants [37].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [138] Shi, Privacy-preserving aggregation of time-series data

      Not checked in the free run.

    • cites [37] Fung, The limitations of federated learning in Sybil settings

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § A. Defenses against Untargeted Attacks · p. 12

    DRACO is robust to arbitrarily malicious computing nodes while being orders of magnitude faster than state-of-the-art robust distributed systems.

    “DRACO is robust to arbitrarily malicious computing nodes, while being orders of magnitude faster than state-of-the-art robust distributed systems.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [145] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § B. Defenses against Targeted Attacks · p. 13

    For production cross-silo FL (H2B), data poisoning attacks are completely ineffective, and model poisoning attacks are unlikely to play a major risk when clients are bound by contract and their software stacks professionally maintained.

    “For production crosssilo FL (H2B), which contains up to hundred clients, data poisoning attacks are completely ineffective; model poisoning attacks are unlikely to play a major risk when the clients involved are bound by contract and their software stacks professionally maintained (e.g., in banks, hospitals).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [141] Chen, 2019, Secure computation for machine learning with SPDZ, arXiv:1901.00329

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 13

    Sharing model parameters may not be a strong design choice in FL because it opens all internal state of the model to inference attacks and maximizes the model's malleability by poisoning attacks.

    “Thus, sharing model parameters may not be a strong design choice in FL, it opens all the internal state of the model to inference attacks, and maximizes the model's malleability by poisoning attacks.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Most FL algorithms require overwriting the local model parameters with the global model.

  • May combine two claims.

    Premisecitation
    § B. Defenses against Targeted Attacks · p. 13

    Fung et al. exploited the characteristic behavior that sybils are more similar to each other than the similarity observed amongst honest clients, and proposed FoolsGold, a defense scheme against FL Sybil attacks by adapting the learning rate of participants based on contribution similarity.

    “To defend against the targeted poisoning attack by sybil clones, Fung et al. [37] exploited the characteristic behavior that sybils are more similar to each other than the similarity observed amongst the honest clients, and proposed FoolsGold: a new defense scheme against FL Sybil attacks by adapting the learning rate of participants based on contribution similarity.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [37] Fung, The limitations of federated learning in Sybil settings

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § B. Defenses against Targeted Attacks · p. 12

    The current state-of-the-art erasing methods are Mode Connectivity Repair (MCR) and Neural Attention Distillation (NAD).

    “The current state-of-the-art erasing methods are Mode Connectivity Repair (MCR) [155] and Neural Attention Distillation (NAD) [54].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [155] Hamm, Learning privately from multiparty data

      Not checked in the free run.

    • cites [54] Tang, Demon in the variant: Statistical analysis of DNNs for robust backdoor contamination detection

      Not checked in the free run.

  • The quote only partly states this claim.

    Interpretationreasoning
    § B. Defenses against Targeted Attacks · p. 13

    None of the existing backdoor defenses in FL provides certified robustness guarantees.

    “However, None of these provides certified robustness guarantees.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationnone given
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 14

    FL is vulnerable to both privacy and robustness attacks during the test/inference phase by users of the final FL model when deployed as a service.

    “In fact, FL is also vulnerable to both privacy and robustness attacks during test/inference phase by the users of the final FL model when it is deployed as a service.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § B. Defenses against Targeted Attacks · p. 13

    FoolsGold requires no auxiliary information beyond the learning process and makes fewer assumptions about participants and their data.

    “Additionally, FoolsGold requires no auxiliary information beyond the learning process, and makes fewer assumptions about participants and their data.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [37] Fung, The limitations of federated learning in Sybil settings

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 14

    Record-level DP bounds the success of membership inference but does not prevent property inference applied to a group of training records.

    “In terms of differential privacy (DP) based methods [7], [166], [84], [167], [168], [169], record-level DP bounds the success of membership inference, but does not prevent property inference applied to a group of training records [28].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [28] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § B. Defenses against Targeted Attacks · p. 12

    Extensive experiments on multiple benchmark datasets against 10 state-of-the-art attacks empirically show that ABL can automatically prevent backdoor attacks during training without degrading main performance.

    “Extensive experiments on multiple benchmark datasets against 10 state-of-the-art attacks empirically show that ABL can automatically prevent backdoor attacks during training, without degrading the main performance.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [55] Soremekun, 2020, Exposing backdoors in robust machine learning models, arXiv:2003.00865

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § B. Defenses against Targeted Attacks · p. 12

    MCR mitigates backdoors by selecting a robust model in the path of loss landscape, while NAD leverages knowledge distillation to erase triggers.

    “MCR mitigates the backdoors by selecting a robust model in the path of loss landscape, while NAD leverages knowledge distillation to erase triggers.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [155] Hamm, Learning privately from multiparty data

      Not checked in the free run.

    • cites [54] Tang, Demon in the variant: Statistical analysis of DNNs for robust backdoor contamination detection

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 14

    Participant-level DP is unsuitable for H2B scenarios because the FL model fails to converge with a small number of participants.

    “The FL model fails to converge with a small number of participants, making it unsuitable for H2B scenarios.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The FL model fails to converge with a small number of participants.

  • May combine two claims.

    Premisecitation
    § B. Defenses against Targeted Attacks · p. 13

    The robustness of FoolsGold holds for different distributions of participant data, varying poisoning targets, and various Sybil strategies, and can be applied successfully on both FedSGD and FedAvg.

    “The robustness of FoolsGold holds for different distributions of participant data, varying poisoning targets, and various Sybil strategies, and can be applied successfully on both FedSGD and FedAvg.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [37] Fung, The limitations of federated learning in Sybil settings

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation
    § B. Defenses against Targeted Attacks · p. 12

    ABL identifies two inherent characteristics of backdoor attacks as weaknesses: models learn backdoored data much faster than clean data, and the backdoor task is tied to a specific class.

    “From this view, they identify two inherent characteristics of backdoor attacks as their weaknesses: 1) the models learn backdoored data much faster than learning with clean data, and the stronger the attack the faster the model converges on backdoored data; 2) the backdoor task is tied to a specific class (the backdoor target class).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [55] Soremekun, 2020, Exposing backdoors in robust machine learning models, arXiv:2003.00865

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § B. Defenses against Targeted Attacks · p. 12

    Anti-Backdoor Learning (ABL) aims to train clean models given backdoor-poisoned data.

    “Another more recent work called Anti-Backdoor Learning (ABL) [55] aims to train clean models given backdoor-poisoned data.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [55] Soremekun, 2020, Exposing backdoors in robust machine learning models, arXiv:2003.00865

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 15

    Decentralized FL is an emerging research area where there is no single central server in the system.

    “Decentralized FL is an emerging research area, where there is no single central server in the system [7], [3], [167], [168].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [7] McMahan, Learning differentially private recurrent language models

      Not checked in the free run.

    • cites [3] Yang, 2019, Federated learning, Synth. Lect. Artif. Intell. Mach. Learn.

      Not checked in the free run.

    • cites [167] Steinhardt, 2017, Resilience: A criterion for learning in the presence of arbitrary outliers, arXiv:1703.04940

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 15

    Secureboost considered user privacy and data confidentiality in VFL and presented an approach to train a high-quality tree boosting model collaboratively.

    “For VFL, Secureboost [180] considered user privacy and data confidentiality in VFL, and presented an approach to train a high-quality tree boosting model collaboratively.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [180] Wang, Improving adversarial robustness requires revisiting misclassified examples

      Not checked in the free run.

  • May combine two claims.

    Interpretationnone given
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 16

    Although using encryption or DP-based techniques can provide provably privacy preservation, they are not robust to poisoning attacks and may produce models with undesirably poor privacy-utility trade-offs.

    “Although using encryption or DP-based techniques can provide provably privacy preservation, they are not robust to poisoning attacks and may produce models with undesirably poor privacy-utility trade-offs.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 14

    Adversarial training typically requires significant computation and longer time to converge.

    “Moreover, adversarial training typically requires significant computation and longer time to converge, and it is unclear how it performs in non-IID settings.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [165] Alistarh, Byzantine stochastic gradient descent

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 15

    CAFE uncovered the risk of catastrophic data leakage in VFL through a novel algorithm that can perform large-batch data leakage with high data recovery quality and theoretical guarantees.

    “Another recent work [182] uncovered the risk of catastrophic data leakage in vertical federated learning (CAFE) through a novel algorithm that can perform largebatch data leakage with high data recovery quality and theoretical guarantees.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [182] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 15

    FedSyn is the first method that can be practically applied to various real-world applications because it requires no additional information except model parameters to be transferred, does not require an auxiliary dataset, and is the first to consider both model and statistical heterogeneities in FL.

    “FedSyn is the first method that can be practically applied to various real-world applications due to the following benefits: (1) FedSyn requires no additional information (except the model parameters) to be transferred between participants and the server; (2) FedSyn does not require any auxiliary dataset for training; (3) FedSyn is the first to consider both model and statistical heterogeneities”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [191] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 15

    Decentralized FL is potentially more useful for H2B scenarios where business participants do not trust any third party.

    “Decentralized FL is potentially more useful for H2B scenarios where the business participants do not trust any third party.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 15

    The secret sharing approach offers no accuracy loss and computation much faster than HE.

    “By contrast, the secret sharing approach offers the following advantages: (i) there is no accuracy loss, (ii) computation is much faster than HE approach.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 15

    A recent work proposed a more practical data-free approach named FedSyn for one-shot FL framework with heterogeneity.

    “A recent work proposed a more practical data-free approach named FedSyn for one-shot FL framework with heterogeneity [191].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [191] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 15

    The HE approach is simple but computationally expensive.

    “The HE approach is simple but computationally expensive.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 15

    The major drawback of the secret sharing approach is that one has to offline generate and store many triplets before online computation.

    “The major drawback of the secret sharing approach is that one has to offline generate and store many triplets before online computation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 16

    There is an inherent conflict between privacy and robustness: defending against robustness attacks usually requires complete control of the training process or access to the training data, which goes against the privacy requirements of FL.

    “However, it is important to highlight that there is an inherent conflict between privacy and robustness: defending against robustness attacks usually require complete control of the training process or access to the training data [193], [139], [22], [23], [37], [194], which goes against the privacy requirements of FL.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [193] Jin, Catastrophic data leakage in vertical federated learning

      Not checked in the free run.

    • cites [139] source

      Not checked in the free run.

    • cites [22] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 15

    Swarm learning can be deemed as a decentralized FL framework, which unites edge computing, blockchain-based peer-to-peer networking and coordination while maintaining confidentiality without the need for a central coordinator.

    “The recent emerging swarm learning [187] can be deemed as a decentralized FL framework, which unites edge computing, blockchain-based peer-to-peer networking and coordination while maintaining confidentiality without the need for a central coordinator.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [187] Dong, Privacy for free: How does dataset condensation help privacy?

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 14

    Conducting adversarial training on only a proportion of participants leads to better convergence.

    “To solve this problem, they conducted AT on only a proportion of participants for better convergence.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [175] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 14

    The training of all federated adversarial training works is unstable, which potentially hurts convergence and performance.

    “The training of all the above FAT works are unstable, which potentially hurts the convergence and performance.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [175] source

      Not checked in the free run.

    • cites [176] Chang, 2019, Cronus: Robust and heterogeneous collaborative learning with black-box knowledge transfer, arXiv:1912.11279

      Not checked in the free run.

    • cites [177] source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 14

    Secure aggregation in FL amplifies the lack of transparency and fairness in FL systems, thus fails to fully comply with GDPR requirements of fairness and transparency; malicious participants in FL may conduct data or model poisoning attacks for an unauthorised purpose, making local ML model parameters no longer minimal for the original purpose.

    “For example, secure aggregation mechanism in FL amplifies the lack of transparency and fairness in FL systems, thus fails to fully comply with the GDPR requirements of fairness and transparency; malicious participants in FL may conduct either data or model poisoning attack for an unauthorised purpose, local ML model parameters obtained from participants is no longer minimal for the original”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [179] Wang, On the convergence and robustness of adversarial training

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 15

    CAFE can recover large-scale private data from shared aggregated gradients in VFL settings, overcoming the batch limitation problem in current data leakage attacks.

    “They empirically demonstrated that CAFE can recover large-scale private data from the shared aggregated gradients in VFL settings, overcoming the batch limitation problem in current data leakage attacks.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [182] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 15

    FederBoost pointed out that Secureboost is expensive since it requires cryptographic computation and communication for each possible split, and proposed vertical FederBoost which does not require any cryptographic operation.

    “A recent work called FederBoost [181] pointed out that Secureboost is expensive since it requires cryptoraphic computation and communication for each possible split, thus they proposed a vertical FederBoost which does not require any cryptographic operation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [181] Tsipras, Robustness may be at odds with accuracy

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation
    § VII. DISCUSSIONS AND PROMISING DIRECTIONS · p. 14

    Conducting adversarial training on all participants leads to divergence of the model.

    “[175] noticed that conducting AT on all participants leads to divergence of the model.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [175] source

      Not checked in the free run.

107 of 107

  • The paper relies on this without support.

    In many real-world federated learning scenarios, participants do not trust the central server or each other.

    “However, in many real-world scenarios, the participants may not trust the central server nor each other.”

  • The authors state this as an assumption.

    The GAN attack assumes the entire training corpus for a given class comes from a single participant.

    “It should be noted that GAN attack assumes the entire training corpus for a given class comes from a single participant, which means the GAN-constructed representatives are similar to the training data only when all class members are similar.”

  • The paper relies on this without support.

    Privacy preservation in federated learning is more challenging than in centralized machine learning due to sporadic access to power and network connectivity and statistical heterogeneity in the data.

    “While privacy preservation has been extensively studied in the machine learning community, privacy preservation in federated learning can be more challenging due to the sporadic access to power and network connectivity, statistical heterogeneity in the data, etc.”

  • The paper relies on this without support.

    In H2C scenarios, participants generally lack the sophisticated technical capabilities and unlimited computational resources assumed by inference attacks.

    “In FL, these assumptions are generally not practical in H2C scenarios, but more likely happen in H2B scenarios.”

  • The paper relies on this without support.

    The total amount of noise in DDP is the same as in CDP.

    “Theoretically, DDP offers the same utility as CDP, as the total amount of noise is the same.”

  • The paper relies on this without support.

    The data distribution of participants in FL might be highly unbalanced.

    “where the data distribution of participants might be highly unbalanced”

  • The paper relies on this without support.

    The server in FL only observes model parameters/updates sent as part of the iterative ML algorithm.

    “the server only observes model parameters/updates sent as part of the iterative ML algorithm [37].”

  • The paper relies on this without support.

    The estimation error scales up with the size of the model in a square-root manner.

    “Specifically, the estimation error scales up with the size of the model in a square-root manner.”

  • The paper relies on this without support.

    The FL model fails to converge with a small number of participants.

    “The FL model fails to converge with a small number of participants, making it unsuitable for H2B scenarios.”

  • The paper relies on this without support.

    Communication cost is a major concern that may hinder the practical deployment of FL.

    “In addition to privacy and robustness, communication cost is another major concern that may hinder the practical deployment of FL.”

  • The paper relies on this without support.

    Most FL algorithms require overwriting the local model parameters with the global model.

    “Most FL algorithms require overwriting the local model parameters with the global model.”

  • The paper relies on this without support.

    When deploying defense mechanisms, the FL server will need additional computational cost.

    “When deploying defense mechanisms to check if any adversary is attacking the FL system, the FL server will need additional computational cost.”

  • The paper relies on this without support.

    Different types of defense mechanisms may have different effectiveness against different attacks, and incur different costs.

    “In addition, different types of defense mechanisms may have different effectiveness against different attacks, and incur different costs.”
13 of 13

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Two incompatible descriptions of Krum's selection rule

    Minor

    One description says Krum removes the f updates furthest from the mean. The subsequent algorithm instead scores updates by distances to their nearest neighbors and selects the single update with the lowest score.

  • Targeted attacks were tested against defenses said to be untested

    Minor

    The paper says the listed defenses, including Krum and coordinate-wise median, have never been tested on targeted backdoor attacks. Later it reports targeted model poisoning against both defenses.

  • Krum's stated 33% limit conflicts with its participant bound

    Minor

    The stated condition n ≥ 2f + 3 permits more than 33% Byzantine participants: for example, n = 15 and f = 6 satisfy it, giving 40%. Thus 33% is not the maximum implied by the paper's own condition.

3 of 3

The citation lists haven’t been fetched yet

References 230 · Cited by 548

Research works printed but not linked: 21

  1. McMahan, 2017, Communication-efficient learning of deep networks from decentralized data, 1273
  2. Cui, Exploiting data sparsity in secure cross-platform social recommendation, 10524
  3. Fung, The limitations of federated learning in Sybil settings, 301
  4. Li, Anti-backdoor learning: Training clean models on poisoned data, 14900
  5. Kairouz, 2019, Advances and open problems in federated learning, arXiv:1912.04977 · arXiv
  6. Huang, Evaluating gradient inversion attacks and defenses in federated learning, 7232
  7. Fang, Local model poisoning attacks to Byzantine-robust federated learning, 1605
  8. 10.5555/3241094.3241142 · DOI
  9. Xu, Beyond model extraction: Imitation attack for black-box NLP APIs, 1
  10. He, CATER: Intellectual property protection on text generation APIs via conditional watermarks, 1
  11. Shafahi, Poison frogs! Targeted clean-label poisoning attacks on neural networks, 6103
  12. Nguyen, Input-aware dynamic backdoor attack, 3454
  13. Shi, Privacy-preserving aggregation of time-series data, 1
  14. Hamm, Learning privately from multiparty data, 555
  15. Jin, Accelerated federated learning with decoupled adaptive optimization, 10298
  16. Tsipras, Robustness may be at odds with accuracy, 1
  17. Dong, Privacy for free: How does dataset condensation help privacy?, 1
  18. Chen, CalFAT: Calibrated federated adversarial training with label skewness, 1
  19. Jin, Catastrophic data leakage in vertical federated learning, 2021, 994
  20. Dennis, Heterogeneity for the win: One-shot federated clustering, 2611
  21. Steinhardt, Certified defenses for data poisoning attacks, 3517

Authors
Lyu, L., Yu, H., Ma, X., Chen, C., Sun, L., Zhao, J., Yang, Q., & Yu, P. S.
Affiliations
  • Sony (Japan) (JP)
  • Nanyang Technological University (SG)
  • Fudan University (CN)
  • Lehigh University (US)
  • Hong Kong University of Science and Technology (HK)
  • University of Illinois Chicago (US)
ORCID
Venue
IEEE Transactions on Neural Networks and Learning Systems
Publisher
Institute of Electrical and Electronics Engineers (IEEE)
Published
2024-07
Volume
35
Issue
7
Pages
8726-8746
ISSN
  • 2162-237X (ISSN-L)
  • 2162-2388
DOI
10.1109/tnnls.2022.3216981
OpenAlex
W3111919937
Semantic Scholar
8448010d9adad18bf36070c012770a10ecb21c76
Cited by
548
References
230
Citations by year
2026: 96 · 2025: 169 · 2024: 141 · 2023: 80 · 2022: 38 · 2021: 24
FWCI
35.21
Type
journal article
Language
en
Open access
Licence
Locations
  • IEEE Transactions on Neural Networks and Learning Systems (publishedVersion)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • DR-NTU (Nanyang Technological University) (submittedVersion)
  • Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
  • Sony AI
  • Joint NTU-WeBank Research Centre on Fintech (NWJ-2020-008)
  • Nanyang Technological University
  • Joint SDU-NTU Centre for Artificial Intelligence Research (NSC-2019-011)
  • National Research Foundation, Singapore, under its AI Singapore Programme (AISG2-RP-2020-019)
  • RIE 2020 Advanced Manufacturing and Engineering (AME) Programmatic Fund, Singapore (A20G8b0102)
  • Nanyang Technological University through the Nanyang Assistant Professorship
  • Future Communications Research & Development Programme (FCPNTU-RG-2021-014)
  • Hong Kong RGC Theme-Based Research Scheme (T41-603/20-R)
  • NSF (III-1763325, III-1909323, III-2106758, SaTC-1930941)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Cryptography and Data Security
Keywords
federated learning, privacy preservation, robustness, privacy attacks, poisoning attacks, defense mechanisms, threat model, data privacy, adversarial robustness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review