Poisoning attacks in federated learning: A survey

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

Poisoning attacks are becoming more effective, stealthy, and powerful.

“Our observations show that poisoning attacks are becoming more effective, stealthy, and powerful.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

In a data poisoning attack, an attacker cannot directly modify the local model parameters and can only perform the poisoning attack by falsifying or tampering with the training data.

“In the data poisoning attack, an attacker cannot directly modify the local model parameters and can only perform the poisoning attack by falsifying or tampering with the training data.”

Classified as probably not a substantial claim.

Federated learning is considered effective in protecting user privacy during training.

“Therefore, federated learning is considered effective in protecting user privacy during training.”

May combine two claims.

The clean label attack methods that use public data or approximate user feature vectors all assume that the data distribution is IID; once the training data is non-IID, the attacker cannot execute the attack in the above way.

“However, these methods all assume that the data distribution is IID.Once the training data is non-IID, the attacker cannot execute the attack in the above way.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Virat et al. argued that there are two kinds of label-flipping strategies, namely static label flipping (SLF) and dynamic label flipping (DLF).

“Virat et al. [23] argued that there are two kinds of label-flipping strategies, namely static label flipping (SLF) and dynamic label flipping (DLF).”

• cites [23] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Static label flipping is inefficient, and dynamic label flipping schemes have been proposed to improve the attack's efficiency.

“However, SLF is inefficient, and some researchers have proposed DLF schemes to improve the attack's efficiency.”

Probably not a substantial claim: may not pass S2 (checkable).

Systematic reviews of poisoning attacks and their corresponding defense strategies from a privacy-preserving perspective still need more effort.

“However, the systematic reviews of poisoning attacks and their corresponding defense strategies from a privacy-preserving perspective still need more effort.”

• Also stated in: § I. INTRODUCTION · plain

  “While several researchers have discussed poisoning attacks in their surveys, few have analyzed the privacy issues that may arise from poisoning attacks and poisoning attack defense strategies.”

Classified as probably not a substantial claim.

Federated learning allows each client to jointly train a model without sharing data.

“To address this problem, Google developed a distributed machine learning framework called federated learning (FL) in 2016, which allows each client to jointly train a model without sharing data [6].”

• cites [6] McMahan, Communication-efficient learning of deep networks from decentralized dataNot checked yet

  Not checked in the free run.

May combine two claims.

Poisoning attacks can significantly impact global models, and malicious attackers can prevent global models from converging or manipulate the prediction results of global models.

“Among them, poisoning attacks can significantly impact global models, and malicious attackers can prevent global models from converging or even manipulating the prediction results of global models.”

• cites [11] sourceNot checked yet

  Not checked in the free run.

• cites [12] sourceNot checked yet

  Not checked in the free run.

• cites [13] Mammen, 2021, Federated learning: Opportunities and challenges, arXiv:2101.05428Not checked yet

  Not checked in the free run.

May combine two claims.

Federated learning faces many security and privacy risks, including poisoning attacks and privacy leakage.

“However, further research shows that federated learning also faces many security and privacy risks [7], [8], [9], [10], such as poisoning attacks [11], [12], [13], [14], [15] and privacy leakage [16], [17], [18], [19], [20], [21].”

• cites [7] sourceNot checked yet

  Not checked in the free run.

• cites [11] sourceNot checked yet

  Not checked in the free run.

• cites [16] Fu, Label inference attacks against vertical federated learningNot checked yet

  Not checked in the free run.

May combine two claims.

Jiale et al. use a generative adversarial network to generate samples similar to those of other clients and then use these fake samples to execute an attack, allowing an attacker to execute a poisoning attack without prior knowledge.

“To solve this problem, Jiale et al. [26] use a generative adversarial network to generate samples similar to those of other clients and then use these fake samples to execute an attack.”

• Also stated in: § a: CLEAN LABEL ATTACK · plain

  “With this approach, an attacker can execute a poisoning attack without prior knowledge.”

• cites [26] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

Only a tiny fraction of neurons change during the training phase, while the vast majority of neurons remain close to zero; those unchanged neurons are called the redundant space of neural networks.

“The principle of this approach is that only a tiny fraction of neurons changed during the training phase, while the vast majority of neurons were close to zero.Those unchanged neurons are called the redundant space of neural networks.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The attacker's goals of implementing a poisoning attack can be classified into two categories: reducing the accuracy of the global model on all tasks and affecting the performance of the global model on specific tasks.

“The attacker's goals of implementing a poisoning attack can be classified into two categories: 1) reducing the accuracy of the global model on all tasks and 2) affecting the performance of the global model on specific tasks.”

May combine two claims.

Model poisoning attacks tend to be more effective in practice than data poisoning attacks because they can directly modify local model parameters.

“However, model poisoning attacks tend to work better in practice because they can directly modify local model parameters [11], [23], [33], [40].”

• cites [11] sourceNot checked yet

  Not checked in the free run.

• cites [23] sourceNot checked yet

  Not checked in the free run.

• cites [33] sourceNot checked yet

  Not checked in the free run.

Classified as probably not a substantial claim. May combine two claims.

If the attack distance between the source class and the target class is small, the scale of the malicious updates required in the adversary's local model is also small, so the attack has a greater chance of surviving aggregation and produces a more significant effect.

“The idea of the attack in this setup is that if the AD between the source class c S and the target class c T is small, the scale of the malicious updates required in the adversary's local model is also small.Then the attack has a greater chance of surviving the aggregation of overwhelming legitimate updates, resulting in a more significant effect.”

May combine two claims.

To generate a malicious model, the attacker can use a specific dataset or train the model on a benign dataset by projected gradient ascent.

“To generate a malicious model w k+1 0 , the attacker can use a specific dataset (e.g., different kinds of green cars labeled as birds [29]) or train the model in a benign dataset by projected gradient ascent (while the benign clients use stochastic gradient descent) algorithm [23], [67], [77].”

• cites [23] sourceNot checked yet

  Not checked in the free run.

• cites [67] Guerraoui, The hidden vulnerability of distributed learning in ByzantiumNot checked yet

  Not checked in the free run.

• cites [77] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Model updates should not deviate too far from benign updates to avoid detection.

“To avoid detection, Virat et al. [23] also state that model updates should not deviate too far from benign updates to avoid detection.”

• cites [23] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The optimization-based model poisoning attack method can resist catastrophic forgetting in neural networks.

“Moreover, this method can resist catastrophic forgetting in neural networks.”

• cites [33] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In a model poisoning attack, the attacker uses a more direct way to manipulate the client's local model parameters.

“In a model poisoning attack, the attacker uses a more direct way to manipulate the client's local model parameters.”

May combine two claims.

In the non-IID setting, both the attacker and the server face significant challenges, but the server is affected more.

“Second, in the non-IID setting, both the attacker and the server faced significant challenges but affected the server even more.”

May combine two claims.

In a poisoning attack, the data distribution affects both the effectiveness of the attack and its success or failure.

“In a poisoning attack, the data distribution can not only affect the effectiveness of the attack but also determine the success or failure of the attack.”

May combine two claims.

Multiplying the local gradient by a factor increases detection risk, but choosing the right amplification factor can effectively improve attack efficiency.

“This approach increases the risk of being detected by the server, but the right choice of amplification factor can effectively improve the attack's efficiency.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Semi-targeted attacks can improve attack performance by fixing the source class and choosing the optimal target class.

“The semi-targeted attack can improve the performance of the attack by fixing the source class and choosing the optimal target class.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Data poisoning attacks allow attackers to implement label flipping according to their intentions.

“For example, data poisoning attacks are more intuitive, and attackers can implement label flipping according to their intentions.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Untargeted poisoning attacks affect all tasks, so they are more easily detected by the server.

“Since untargeted poisoning attacks affect all tasks, they are more easily detected by the server.”

May combine two claims.

Attackers can implement a more stealthy and persistent poisoning attack using redundant neurons, which requires the ability to modify model parameters directly.

“They can implement a more stealthy and persistent poisoning attack using these redundant neurons, which requires the ability to modify the model parameters directly.”

• cites [33] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Targeted poisoning attacks are highly stealthy and more challenging to detect, especially when the data is non-IID.

“On the other hand, targeted poisoning attacks are highly stealthy and more challenging to detect, especially when the data is non-IID.”

May combine two claims.

Only a tiny fraction of neurons are changed during the training phase, while most neurons are close to zero.

“First, Xingchen et al. [33] found that only a tiny fraction of neurons were changed during the training phase, while most neurons were close to zero.”

• cites [33] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

Many attackers avoid detection by bounding the norm of their malicious gradients, and the range of this norm is strongly influenced by the data distribution.

“In addition, many attackers avoid detection by bounding the norm of their malicious gradients, the range of which is strongly influenced by the data distribution.”

May combine two claims.

Mixing a small number of malicious updates with numerous benign updates can avoid server detection, but the server can still identify the attack by comparing historical updates from the client.

“Although Thien et al. [80] propose that it can mix a small number of malicious updates with numerous benign updates to avoid server detection, the server can still identify the attack by comparing historical updates from the client.”

• cites [80] sourceNot checked yet

  Not checked in the free run.

May combine two claims.

If the server can access the client's local model directly, there is a great risk of privacy leakage because the server may recover the original training data from the client's gradients.

“It is worth noting that if the server can access the client's local model directly, there is a great risk of privacy leakage because the server may recover the original training data from the client's gradients [16], [17], [18], [19].”

• cites [16] Fu, Label inference attacks against vertical federated learningNot checked yet

  Not checked in the free run.

• cites [17] Geiping, Inverting gradients—How easy is it to break privacy in federated learning?Not checked yet

  Not checked in the free run.

• cites [18] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Benign models and malicious models are inherently different from each other.

“This assumption is always valid because benign models and malicious models are inherently different from each other.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The gradients uploaded by honest clients are more diverse than those uploaded by malicious clients.

“For example, Clement et al. [53] found that the gradients uploaded by honest clients were more diverse than those uploaded by malicious clients.”

• cites [53] Fung, 2018, Mitigating sybils in federated learning poisoning, arXiv:1808.04866Not checked yet

  Not checked in the free run.

May combine two claims.

Both LDP and CDP are effective against poisoning attacks, and CDP is more effective than LDP against poisoning attacks.

“The experimental results show that both LDP and CDP are effective against poisoning attacks, and CDP is more effective than LDP against poisoning attacks.”

• cites [66] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

In a typical gradient clipping approach, a threshold is set for model parameters and all parameters that exceed the threshold are discarded.

“In a typical gradient clipping approach, we can set a threshold for the model parameters and discard all parameters that exceed the threshold [29], [65].”

• cites [29] Sun, 2019, Can you really backdoor federated learning?, arXiv:1911.07963Not checked yet

  Not checked in the free run.

• cites [65] Panda, SparseFed: Mitigating model poisoning attacks in federated learning with sparsificationNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Byzantine robust aggregation (BRA) is a passive defense strategy that mitigates the damage of poisoning attacks by changing the aggregation method of the global model.

“BRA is a passive defense strategy that mitigates the damage of poisoning attacks by changing the aggregation method of the global model.”

May combine two claims.

Many FL frameworks use secure aggregation algorithms, which invalidate most of the poisoning attack defense methods mentioned above.

“Therefore, many FL frameworks use secure aggregation [81], [82], [83], [84] algorithms in the aggregation process, at which point most of the poisoning attack defense methods mentioned above will be invalid.”

• cites [81] sourceNot checked yet

  Not checked in the free run.

• cites [82] Zhang, BatchCrypt: Efficient homomorphic encryption for cross-silo federated learningNot checked yet

  Not checked in the free run.

• cites [83] Bhowmick, 2018, Protection against reconstruction and its applications in private federated learning, arXiv:1812.00984Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Methods for testing global/local models require the server to collect a test set, and the quality of this test set dramatically affects defense effectiveness.

“The methods mentioned above for testing global/local models require the server to collect a test set, and the quality of this test set dramatically affects the effectiveness of the defense.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Because poisoning attacks are executed through malicious models, an intuitive defense strategy is to analyze the model.

“Poisoning attacks are executed through malicious models, so an intuitive defense strategy is to analyze the model.”

May combine two claims.

Among model analysis methods, direct analysis of the global model supports secure aggregation, and analysis of the encrypted local model can defend against poisoning attacks while protecting clients' data privacy.

“Among the model analysis methods, we found that in addition to the direct analysis of the global model that supports secure aggregation, the analysis of the encrypted local model can defend against poisoning attacks and protect clients' data privacy.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Directly verifying the client's local data is not always allowed, but introducing a trusted third party can solve this problem.

“In addition, directly verifying the client's local data is not always allowed, but introducing a trusted third party can solve this problem.”

May combine two claims.

Differential privacy is a secure aggregation strategy that can also resist poisoning attacks, but its biggest problem is that it reduces the accuracy of the global model.

“Finally, differential privacy is a secure aggregation strategy that can also resist poisoning attacks, but its biggest problem is that it reduces the accuracy of the global model.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

Trusted third parties are difficult to guarantee in real-world settings, but this approach provides lessons for subsequent research.

“Although trusted third parties are difficult to guarantee in real-world settings, this approach also provides lessons for subsequent research.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

One of the most remarkable features of federated learning is that it protects the data privacy of the clients involved in training.

“One of the most remarkable features of federated learning is that it protects the data privacy of the clients involved in training.”

May combine two claims.

Although defense efforts follow the principle that the client's original data is not released, intermediate data in federated learning may cause privacy leaks.

“While these efforts follow the fundamental principle that the client's original data is not released to the public, the intermediate data in federated learning may cause privacy leaks.”

Probably not a substantial claim: may not pass S3 (load-bearing).

An attacker with sufficiently long continuous learning cycles can achieve high accuracy on backdoor samples.

“Florian et al. [88] pointed out that an attacker with sufficiently long continuous learning cycles can achieve high accuracy on backdoor samples.”

• cites [88] sourceNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Many efforts to defend against poisoning attacks will likely lead to client data privacy leakage.

“However, many efforts to defend against poisoning attacks will likely lead to client data privacy leakage.”

The quote only partly states this claim.

ADFL cannot cope with data poisoning attacks.

“This approach also cannot cope with data poisoning attacks.”

May combine two claims.

A major problem with analyzing the global model is that current research uses predictive accuracy on a validation dataset to determine whether an attack has occurred, and targeted poisoning attacks may evade this defense.

“The biggest problem with the latter is that current research is based on the predictive accuracy of the global model on the validation dataset to determine whether an attack has been poisoned, and targeted poisoning attacks may evade this defense.”

May combine two claims.

Some researchers have proposed an iterative median approximation method that can approximate the median of all models without accessing the local model and thus can support secure aggregation.

“However, some researchers [68] have proposed an iterative median approximation method that can approximate the median of all models without accessing the local model and thus can also support the secure aggregation strategy.”

• cites [68] Pillutla, 2019, Robust aggregation for federated learning, arXiv:1912.13445Not checked yet

  Not checked in the free run.

May combine two claims.

A major problem with the SVM-based approach is that it requires thorough training on poisoned and non-poisoned datasets similar to the client's data source, so it may not apply to non-IID scenarios.

“The biggest problem with this approach is that it requires thorough training of an SVM on poisoned and non-poisoned datasets that are relatively similar to the client's data source and thus may not apply to scenarios where the data is Non-IID.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

Research on implementing poisoning attacks and defending against poisoning attacks has made significant progress in recent years, but some issues still need further research.

“Research on implementing poisoning attacks and defending against poisoning attacks has made significant progress in recent years, but some issues still need further research.”

Classified as probably not a substantial claim.

In each training round of federated learning, some clients are selected randomly.

“First, each training round of FL selects some clients randomly.”

May combine two claims.

In Henger et al.'s approach, the malicious client uses a gradient-inversion-based inference attack to learn approximations of other clients' data distributions from model updates received by the server, then uses its local data and the learned distributions to build a simulator of the FL environment, and finally uses reinforcement learning to learn an adaptive attack strategy.

“Furthermore, in the approach proposed by Henger et al. [32], the malicious client first uses a gradient inversionbased inference attack to learn approximations of other client data distributions from model updates received by the server.Then, the attacker uses his local data and the learned distributions to construct a simulator of the FL environment.Finally, the attacker uses reinforcement learn”

• cites [32] Li, Learning to attack federated learning: A model-based reinforcement learning attack frameworkNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Frequent attacks increase the risk of exposure.

“Second, frequent attacks increase the risk of exposure.”

Probably not a substantial claim: may not pass S4 (based).

Existing attacks are constantly enriched with increasingly sophisticated attack methods being proposed.

“However, the existing attacks are constantly enriched with increasingly sophisticated attack methods being proposed.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Attackers have already solved the feasibility problem.

“Attackers have already solved the feasibility problem.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

In poisoning attacks, multiple attacks may be combined, for example with privacy inference attacks.

“In poisoning attacks, multiple attacks may be combined (e.g., with privacy inference attacks).”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

Once an attacker knows these defense strategies exist, it is easy to bypass them.

“Once an attacker knows these defense strategies exist, it is easy to bypass them.”

May combine two claims.

Many researchers assume that the adversary in their experiments undisguisedly performs the attack, making them vulnerable to more sophisticated poisoning attacks despite positive experimental results.

“At the same time, many researchers assume that the adversary in their experiments undisguisedly performs the attack.Thus, they are vulnerable to more sophisticated poisoning attacks, despite the positive results they have achieved in their experiments.”

May combine two claims.

Existing defense strategies are primarily targeted at specific attacks and can only defend against known attack methods.

“Another problem with existing defense strategies is that they are primarily targeted at specific attacks.They can only defend against known attack methods.”

Probably not a substantial claim: may not pass S2 (checkable).

The defense against poisoning attacks, although well studied, still has many issues to be addressed.

“On the other hand, the defense against poisoning attacks, although well studied, still has many issues to be addressed.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In model replacement, the attacker attempts to replace the global model with the malicious model.

“In model replacement, the attacker attempts to replace the global model w (k+1) 0 with the malicious model w k+1 0 .”

• cites [29] Sun, 2019, Can you really backdoor federated learning?, arXiv:1911.07963Not checked yet

  Not checked in the free run.

• cites [30] Bagdasaryan, How to backdoor federated learningNot checked yet

  Not checked in the free run.

• cites [31] Xie, DBA: Distributed backdoor attacks against federated learningNot checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

One existing trend is to design defense strategies against poisoning attacks in the cryptographic mode with existing privacy-preserving cryptographic algorithms.

“One existing trend is to design defense strategies against poisoning attacks in the cryptographic mode with existing privacy-preserving cryptographic algorithms.”

Classified as probably not a substantial claim.

Effectively preventing or detecting poisoning attacks remains an open problem.

“At the same time, the challenge for defenders is much tougher, as effectively preventing or detecting poisoning attacks remains an open problem.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Attackers may be more concerned with the trade-off between efficiency and stealth.

“As a result, they may be more concerned with the trade-off between efficiency and stealth.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Attackers have used methods such as reinforcement learning to improve the efficiency of their attacks.

“Attackers have used methods such as reinforcement learning to improve the efficiency of their attacks.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

There are relatively few studies on cryptographic-mode poisoning defenses, mainly focusing on homomorphic encryption and differential privacy.

“However, there are relatively few studies in this direction, mainly focusing on homomorphic encryption and differential privacy.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

Many existing studies assume the server has access to the client's local model, which poses a significant risk of privacy leakage.

“Many existing studies assume the server has access to the client's local model, which poses a significant risk of privacy leakage.”

Probably not a substantial claim: may not pass S4 (based).

These defense strategies cannot be directly integrated with existing privacy protection strategies.

“Worse, these defense strategies cannot be directly integrated with existing privacy protection strategies.”

May combine two claims.

During the federated learning training process, an attacker can execute a poisoning attack by modifying the client's local data or directly modifying the local model parameters.

“During the federated learning training process, an attacker can execute a poisoning attack by modifying the client's local data or directly modifying the local model parameters.”