Data and Model Poisoning Backdoor Attacks on Wireless Federated Learning, and the Defense Mechanisms: A Comprehensive Survey

In the paper

“Moreover, with the broadcast nature of wireless interfaces, adversaries can potentially overhear local model updates from multiple or even all clients and accordingly create malicious local models to compromise WFL, e.g., using generative neural networks and machine learning technologies.”

What the sources say

• Supported

  The Threat Model section says that wireless radio’s broadcast nature lets an attacker eavesdrop on local models from some or all benign devices and generate malicious local models. The GAE for Malicious Model Generation section describes a graph-autoencoder method for doing so. Page numbers are not available in our copy. (LLM, conf=0.94)

  cites [35] source
• Not checked yetcites [114] source

In the paper

“With the additional knowledge of the local models along with the global model, the model-dependent triggers can be potentially further enhanced, e.g., by employing adversarial generative networks and training [174].”

What the sources say

• Partly supported

  The Pipeline (p. 4) describes a generative network that uses a target model’s gradients to produce adversarial examples, and the Conclusion (p. 12) says the networks are trained jointly. This supports the proposed use of model-informed generation and training, but the paper does not address model-dependent triggers or the use of both local and global models to enhance them. (LLM, conf=0.84)

  cites [174] sourceExperimental — accuracy not yet measured

In the paper

“In return, the attack strength is weak, and the backdoor effect will be quickly diluted.”

In the paper

“The backdoor effect can last longer if the attack success rate is higher.”

In the paper

“Consequently, the attackers need to know more about the entire network, and the attack cost is greatly increased.”

In the paper

“The trigger-based backdoor attack can be improved using a coordinated trigger pattern, achieving a higher attack success rate.”

In the paper

“In general, the model poisoning-based backdoor attacks outperform data poisoning-based backdoor attacks while yielding a higher complexity.”

In the paper

“It can be seen that the global model replacement backdoor attack [37] launched during the aggregation process achieves the highest BAR while also maintaining a reasonable MAR.”

What the sources say

• Not checked yetcites [37] source

In the paper

“It is noted that all defense methods in different phases can reduce backdoor effects while maintaining a steady MAR.”

In the paper

“It is worth mentioning that, in most of the existing attack methods, a typical assumption is that, malicious participants are more likely to be selected in each training round, and the proportion of clients controlled by the attacker is typically high.”

In the paper

“It is worth mentioning that, in most of the existing attack methods, a typical assumption is that, malicious participants are more likely to be selected in each training round, and the proportion of clients controlled by the attacker is typically high.”

In the paper

“In fact, backdoor detection relies on threshold design in different defense mechanisms.”

In the paper

“Meanwhile, several defense strategies are not compatible with the security aggregation protocols.”

In the paper

“A critical challenge faced by detecting and defending against backdoor attacks is that backdoor attacks often do not negatively impact the accuracy of the model on primary tasks and do not disrupt the normal system operation before they are activated [32].”

What the sources say

• Not checked yetcites [32] Bagdasaryan, How to backdoor federated learning

In the paper

“However, WFL will be more vulnerable to backdoor attacks than its wired counterpart due to a combination of factors inherent to wireless networks.”

In the paper

“By taking advantage of the overheard local models, the misbehaved participants can construct malicious local models to escape malicious model detection at the central server, poison the global model and subsequently the benign local models, and manipulate WFL [35].”

What the sources say

• Not checked yetcites [35] source

In the paper

“The WFL vulnerabilities potentially open the door for backdoor attacks.”

In the paper

“The detection of backdoors typically relies on threshold design, and the training of existing defense mechanisms often requires access to historical information about the attacks.”

In the paper

“In [52] and [53], the theoretical working mechanisms of backdoor attacks and defenses for WFL were reviewed. However, the limitations of the existing methodologies were not systematically analyzed.”

What the sources say

• Not checked yetcites [52] source
• Not checked yetcites [53] Nguyen, 2023, Backdoor attacks and defenses in federated learning: Survey, challenges and future research directions, arXiv:2303.02213

In the paper

“On the other hand, in [48], [49], [50], and [51], WFL was considered as one of the deep learning applications when discussing the impact of backdoor attacks, but no detailed analysis of vulnerabilities of backdoor attacks on WFL was provided.”

What the sources say

• Not checked yetcites [48] Gao, 2020, Backdoor attacks and countermeasures on deep learning: A comprehensive review, arXiv:2007.10760
• Not checked yetcites [49] source
• Not checked yetcites [50] source

In the paper

“This makes them more threatening and more difficult to defend against, as the defender needs to consider all steps in the process.”

In the paper

“Despite the rapid growth of WFL as a research area, there has not yet been a comprehensive and up-to-date review of it from the perspective of backdoor attacks.”

In the paper

“This property allows the backdoor to remain effective for a longer duration, making it difficult to eliminate fully.”

In the paper

“The detection of backdoors typically relies on threshold design, and the training of existing defense mechanisms often requires access to historical information about the attacks.”

In the paper

“In the survey [50], the backdoor attack strategies were discussed in depth, but they were not discussed in the context of WFL.”

What the sources say

• Not checked yetcites [50] source

In the paper

“This can make it more challenging to maintain constant security measures, making the system more prone to backdoor attacks.”

In the paper

“This inconsistency can lead to data corruption or loss, and an attacker might exploit the inconsistency to inject malicious backdoor attacks.”

In the paper

“For example, in [35], [114], a new adversarial attack on WFL is studied by exploiting the broadcast nature of wireless channels, where a rogue client collects the local model updates that benign clients upload. The rogue client can take these overheard local model updates as input to a graph adversarial encoder (GAE) to produce a malicious local model update. The malicious local model update can”

What the sources say

• Not checked yetcites [35] source
• Not checked yetcites [114] source

In the paper

“In OTA-WFL with uncoded transmission, gradients are transmitted directly over wireless channels without encoding or encryption. The lack of coding or encryption in transmission means that any vulnerability in the wireless channel, such as weak signal encryption or interception by unauthorized receivers, can lead to significant privacy breaches. Moreover, since gradients carry information about the”

What the sources say

• Not checked yetcites [97] source

In the paper

“This makes managing and securing each device more difficult and increases the overall attack surface.”

In the paper

“Data transmissions over wireless links are through airwaves, making it easier for attackers to eavesdrop [75]-[78], jam [79], and manipulate data, e.g., replay attack and Sybil attack, or intrude the networks [80], [81].”

What the sources say

• Not checked yetcites [75]-[78]
• Not checked yetcites [79] source
• Not checked yetcites [80], [81]

In the paper

“The primary objective of the backdoor attack is to generate a converged global model with high accuracy of the main task and the backdoor-injected sub-task [107].”

What the sources say

• Not checked yetcites [107] source

In the paper

“Under the setting of OTA-WFL, the authors of [97] control the relative intensity of the receiver noise power at the central server by controlling the transmit powers of the clients. The resulting relative intensity of the receiver noises can offer privacy-preserving capability, like DP noises.”

What the sources say

• Not checked yetcites [97] source

In the paper

“The effectiveness of an aggregation phase attack is mainly determined by the knowledge of the model controlled by attackers.”

In the paper

“This is due to the fact that the random errors resulting from wireless fading and receiver noises can potentially serve to perturb the model updates, hence achieving the effect of DP and alleviating the impact of the attacks.”

In the paper

“The objective of a backdoor attack on WFL is to mislead the global model to misclassify malicious inputs with injected backdoor patterns into the target output, with application to image classification [127]-[136], word prediction [137][142], etc.”

What the sources say

• Not checked yetcites [127]-[136]
• Not checked yetcites [137][142]

In the paper

“Backdoor attacks conducted during the aggregation process generally do not alter the targeted model. Instead, the attacker is keen on tricking it into producing wrong predictions and causing security problems [32].”

What the sources say

• Not checked yetcites [32] Bagdasaryan, How to backdoor federated learning

In the paper

“The model will behave normally and predict a target class when the trigger appears, e.g., in the same room with a facial recognition lock, all men with glasses will be authorized to enter.”

In the paper

“Generally, a high BAR is equivalent to a high ASR and BTA.”

What the sources say

• Not checked yetcites [143] source
• Not checked yetcites [144] source
• Not checked yetcites [32] Bagdasaryan, How to backdoor federated learning

In the paper

“Compared with training process attacks, the aggregation phase attacks can have stronger attack effects while there is a trade-off between the stealthiness and the attack success rate (ASR).”

In the paper

“In a local data collection phase backdoor attack, an individual feature or a small region of the local training data is modified by malicious users deliberately [117].”

What the sources say

• Not checked yetcites [117] Schwarzschild, Just how toxic is data poisoning? A unified benchmark for backdoor and data poisoning attacks

In the paper

“The primary objective of backdoor attacks is to manipulate the converged global model to misclassify specifically crafted inputs with hidden backdoors into a desired target label while ensuring that normal inputs are classified accurately.”

In the paper

“The inevitable problem of the simple label-flipping method still exists in that the flipped data set under the targeted label results in a remarkable outlier in the model training. Thus, it can be detected by norm clipping [164] or differential privacy [38], [38], [164], [165].”

What the sources say

• Not checked yetcites [164] Xie, CRFL: Certifiably robust federated learning against backdoor attacks
• Not checked yetcites [38] Nguyen, FLAME: Taming backdoors in federated learning
• Not checked yetcites [165] Zhang, 2023, Bounded and unbiased composite differential privacy, arXiv:2311.02324

In the paper

“In general, label-flipping depends on the data sets and is agnostic to the models to be trained and the communication interfaces.”

In the paper

“Due to the nature of FL, the backdoor effect in the global model tends to be diluted with the increment of the training round.”

In the paper

“In label-flipping-based backdoor attacks, the training data set under the targeted label is replaced by the backdoored data set [162].”

What the sources say

• Not checked yetcites [162] source

In the paper

“Each client's data distribution is not representative of the global data distribution, implying that the categories in each client are incomplete.”

In the paper

“To improve the stealthiness of the label flipping-based backdoor attack, the target label is selected on the tail in [33]. In this case, the data distribution is required, and the edge data set is selected as the target.”

What the sources say

• Not checked yetcites [33] Wang, Attack of the tails: Yes, you really can backdoor federated learning

In the paper

“Different from Byzantine attacks, which aim to jeopardize the convergence of the global model [145]-[147], the backdoor attacks focus on inserting backdoor patterns into the global model while maintaining a reasonable MAR.”

What the sources say

• Not checked yetcites [145]-[147]

In the paper

“Different from backdoor attack designs, which pursue high BAR, MAR, and long lifespan, the defenders aim to minimize the BAR and lifespan of the backdoor effect while preventing the MAR from degradation.”

In the paper

“As shown in Table IV, label flipping attacks, coordinated trigger attacks, local model replacement attacks, and aggregation process attacks are likely to be exacerbated in WFL.”

In the paper

“Replacing the edge target data set with a backdoored data set, the local model behaves correctly with normal inputs while misclassifying the adversarial inputs on the edge label.”

What the sources say

• Not checked yetcites [33] Wang, Attack of the tails: Yes, you really can backdoor federated learning

In the paper

“This is due to the broadcast nature of wireless communications, which facilitates various forms of cyber threats, such as spoofing attacks, Sybil attacks, and eavesdropping attacks.”

In the paper

“The primary objective of triggerbased backdoor attacks is to mislead the model to classify the trigger-mounted inputs into the target label while behaving normally with non-trigger inputs [38], [116], [126].”

What the sources say

• Not checked yetcites [38] Nguyen, FLAME: Taming backdoors in federated learning
• Not checked yetcites [116] source
• Not checked yetcites [126] Zhang, Neurotoxin: Durable backdoors in federated learning

In the paper

“In a random trigger-based backdoor attack, the trigger is generated independently of the model [122], [140], [169], [170].”

What the sources say

• Not checked yetcites [122] Xie, DBA: Distributed backdoor attacks against federated learning
• Not checked yetcites [140] Pan, Hidden trigger backdoor attack on NLP models via linguistic style manipulation
• Not checked yetcites [169] source

In the paper

“According to the number of adversarial participants involved in an attack, trigger-based backdoor attacks can be classified into centralized trigger-based backdoor attacks and coordinated triggerbased backdoor attacks.”

In the paper

“The advantage is that the attack cost is greatly reduced.”

In the paper

“Depending on different patterns of the trigger, the triggerbased backdoor attacks can be further divided into random trigger backdoor attacks and model-dependent trigger backdoor attacks, as follows.”

In the paper

“The model-dependent trigger can help to excite the target label and has a higher probability of being successfully injected into the trained local model before aggregation [172].”

What the sources say

• Not checked yetcites [172] source

In the paper

“Both label-flipping-based and triggerbased data poisoning backdoor attacks can be treated as blackbox attacks [33].”

What the sources say

• Not checked yetcites [33] Wang, Attack of the tails: Yes, you really can backdoor federated learning

In the paper

“In the local model training phase, the backdoor attacks can be conducted by manipulating the model parameters during the training process or modifying the training algorithm [126], [177].”

What the sources say

• Not checked yetcites [126] Zhang, Neurotoxin: Durable backdoors in federated learning
• Not checked yetcites [177] source

In the paper

“The local training time that remains in a communication round would be short, leading to insufficiently trained local models and, hence, vulnerabilities to backdoor attacks.”

In the paper

“For instance, the increasingly widely deployed wireless communication technologies, e.g., 5G, operate in high-frequency bands of up to 40 GHz, and the wavelength of the carrier frequency can be as short as 7.5 millimeters [175].”

What the sources say

• Not checked yetcites [175] source

In the paper

“With a newly designed training algorithm, the norm difference between the backdoored and benign models decreases, the backdoor becomes harder to detect, and the attack success rate improves.”

In the paper

“The common problem of centralized triggerbased backdoor attacks is that the single attacker's effect is limited, and it cannot be ensured that the attacker is selected in every round.”

In the paper

“It has been discussed in the previous section that the simple label-flipping attack exhibits a low attack success rate.”

In the paper

“The attacks at this stage are also known as white box attacks.”

In the paper

“Furthermore, the knowledge of the defense mechanism adopted in the WFL network is also required to ensure the injection of the backdoor.”

In the paper

“The attacker needs to master the training data set, as well as the local training process.”

In the paper

“On the other hand, when frequency-division multiple-access (FDMA) protocols, including orthogonal frequency-division multiple-access (OFDMA) protocols, are considered, each selected client would enjoy much narrower bandwidths, which would prolong their model uploading time at the cost of their model training time [180].”

What the sources say

• Not checked yetcites [180] source

In the paper

“The proposed attack ensures that the backdoor is bound to be injected in one round and will not be diluted.”

In the paper

“The injected backdoor pattern is also easy to detect, and the life span of the backdoor effect is short.”

In the paper

“Such black-box attacks featured low attack costs but also relatively low backdoor effects [155].”

What the sources say

• Not checked yetcites [155] source

In the paper

“Once the attacker obtains the knowledge of the training algorithm, it can fully control the compromised client and initiate the model poisoning backdoor attack during the local model training phase.”

In the paper

“Moreover, the attack shows better performance when the global model is close to complete convergence, so the attacker also needs to know the information on the convergence progress of the global model.”

In the paper

“The label-flippingbased data poisoning backdoor attack is the easiest to implement.”

In the paper

“The malicious attacker can then conduct the attack by hijacking one or more compromised clients that are close to the essential node. Thus, the backdoor effect is more likely to be injected into the network and spread more broadly.”

In the paper

“However, in decentralized WFL, each client can behave as the server, and only receives model information from nodes within its neighborhood.”

In the paper

“A backdoor attack at this phase can only pollute the training data, and the attackers do not necessarily know the training algorithm and aggregation protocol.”

In the paper

“The global model replacement method aims to manipulate the local training algorithm, and the adversarial updates are guaranteed to be uploaded for aggregation.”

In the paper

“However, training the backdoor filter on the server side requires prior information about the backdoor patterns and backdoored data sets.”

In the paper

“The backdoor attack success rate under the defense mechanism reduces to zero, while the stability-based defense can ensure a high success rate for the main task.”

What the sources say

• Not checked yetcites [191] source

In the paper

“The attacks at this stage have a better attack success rate, but the requirement of the model training information increases the attack cost.”

In the paper

“The defense schemes can be classified into four types: defense at the local training phase, defense before the aggregation phase, defense during the aggregation phase, and defense after the aggregation phase.”

In the paper

“In return, the attackers need to master the local training data, local training algorithm, and the global aggregation protocol, making it hard to implement in practical scenarios.”

In the paper

“Moreover, the blur operation can only deal with the trigger in the pixel pattern.”

In the paper

“In the experimental setup, the number of participants is set to 10, and the training data are IID.”

What the sources say

• Not checked yetcites [191] source

In the paper

“Compared with benign clients, the backdoor attackers are likely to upload updates with relatively larger norms.”

In the paper

“Thus, the trade-off between the primary task accuracy and attack defense rate must be considered when adding the noise.”

In the paper

“The defense mechanisms at this stage are also compatible with the SecAgg protocol.”

In the paper

“The defense performance might degrade in the actual WFL, where the number of clients is larger, and the training data sets are typically non-IID.”

In the paper

“For this reason, the UDP is more commonly adopted in WFL [201].”

What the sources say

• Not checked yetcites [201] source

In the paper

“For instance, the TCP can ensure the reliable delivery of data across a network but requires one or multiple retransmissions in each communication cycle [200].”

What the sources say

• Not checked yetcites [200] source

In the paper

“Another commonly adopted defense strategy against backdoor attacks is to detect the anomaly updates before sending them to the aggregator.”

In the paper

“Wireless devices can be resource-constrained, e.g., in terms of energy supply and communication resources.”

In the paper

“The remaining challenges are that the defender requires the proper knowledge of the backdoor patterns, and the trade-off between the defense performance and significant task accuracy must also be considered [192].”

What the sources say

• Not checked yetcites [192] source

In the paper

“Many efforts have been carried out to deal with both data poisoning backdoor attack [193]-[195] and model poisoning backdoor attack [196]-[198].”

What the sources say

• Not checked yetcites [193]-[195]
• Not checked yetcites [196]-[198]

In the paper

“This can penalize the local models by reducing the local training time in a communication cycle.”

In the paper

“Client selection and scheduling are indispensable in the context of WFL.”

In the paper

“There is a significant opportunity to couple the designs of both in a holistic way to improve defense effectiveness and efficiency.”

In the paper

“To date, the client selection/scheduling has been designed in parallel to the backdoor defense studies.”

In the paper

“The encoder-decoder architecture requires extra unbiased updates to train the detector, and it is pretty hard to implement in real WFL applications.”

What the sources say

• Not checked yetcites [202] Li, 2020, Learning to detect malicious clients for robust federated learning, arXiv:2002.00211

In the paper

“After removing the noisy and redundant features of these data instances, the difference between benign and malicious updates becomes apparent.”

In the paper

“The basic idea of low-dimensional anomaly detection methods is to project the compact computed local model onto its low-dimension space, as given by:”

In the paper

“The malicious attacker cannot inject completed backdoor patterns in a short time, thus improving the robustness of the global model.”

What the sources say

• Not checked yetcites [204] source

In the paper

“The PCA-based method requires prior knowledge of the targeted backdoor label, which is also hard to know in practical defense implementation.”

What the sources say

• Not checked yetcites [203] source

In the paper

“The low-dimensional methods are also incompatible with the SecAgg protocol, and the privacy of the participants in the WFL network cannot be guaranteed.”

What the sources say

• Not checked yetcites [202] Li, 2020, Learning to detect malicious clients for robust federated learning, arXiv:2002.00211
• Not checked yetcites [203] source

In the paper

“The clipping should filter not only the updates from clients but different communication channels as well.”

In the paper

“Due to the feature of WCN, the communication channels with better signals can be better selected in WFL.”

In the paper

“The reconstruction error of the benign updates is much smaller than that of the malicious updates.”

What the sources say

• Not checked yetcites [202] Li, 2020, Learning to detect malicious clients for robust federated learning, arXiv:2002.00211