Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descent
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
We consider the distributed statistical learning problem over decentralized systems that are prone to adversarial attacks. This setup arises in many practical applications, including Google's Federated Learning. Formally, we focus on a decentralized system that consists of a parameter server and m working machines; each working machine keeps N/m data samples, where N is the total number of samples. In each iteration, up to q of the m working machines suffer Byzantine faults -- a faulty machine in the given iteration behaves arbitrarily badly against the system and has complete knowledge of the system. Additionally, the sets of faulty machines may be different across iterations. Our goal is to design robust algorithms such that the system can learn the underlying true parameter, which is of dimension d, despite the interruption of the Byzantine attacks. In this paper, based on the geometric median of means of the gradients, we propose a simple variant of the classical gradient descent method. We show that our method can tolerate q Byzantine failures up to 2(1+ε)q ≤ for an arbitrarily small but fixed constant ε > 0. The parameter estimate converges in O(log N) rounds with an estimation error on the order of max{√dq/N, √d/N, which is larger than the minimax-optimal error rate √d/N in the centralized and failure-free setting by at most a factor of √q. The total computational complexity of our algorithm is of O((Nd/m) log N) at each working machine and O(md + kd log3 N) at the cen…
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Apart from preserving privacy for individual users, edge AI also needs to be robust to errors and adversarial attackers, as the decentralized nature makes it easy to be unreliable in the learning process or even completely controlled by external attackers [74].”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [74]
Citations
Cites
MapReduce: simplified data processing on large clusters
Dean, J., & Ghemawat, S. (2008) · Communications of the ACM · doi:10.1145/1327452.1327492 · Cited by 18,476 · cited 2×
“Many efficient distributed machine learning algorithms [6, 13] and system implementations [24, 28, 10, 19] have been proposed and studied.”
BackgroundBoyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075
(1992) · Lecture Notes in Computer Science · doi:10.1007/bfb0022433 · Cited by 3,955
Privacy-preserving data mining
Agrawal, R., & Srikant, R. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335438 · Cited by 2,961
Distributed GraphLab: a framework for machine learning and data mining in the cloud
Low, Y., Bickson, D., Gonzalez, J., Guestrin, C., Kyrola, A., & Hellerstein, J. M. (2012) · Proceedings of the VLDB Endowment · doi:10.14778/2212351.2212354 · Cited by 1,653 · cited 2×
“Many efficient distributed machine learning algorithms [6, 13] and system implementations [24, 28, 10, 19] have been proposed and studied.”
BackgroundNetwork Security: Private Communication in a Public World
Kaufman, C., Perlman, R., & Speciner, M. (1995) · Cited by 730
Geometric median and robust estimation in Banach spaces
Minsker, S. (2015) · Bernoulli · doi:10.3150/14-bej645 · Cited by 261
Agnostic Estimation of Mean and Covariance
Lai, K. A., Rao, A. B., & Vempala, S. (2016) · 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS) · doi:10.1109/focs.2016.76 · Cited by 204 · cited 1×
“A recent line of work [8, 15] presents polynomial algorithms to consistently estimate the mean and covariance of a distribution from N i.”
BackgroundRobust Estimators in High Dimensions without the Computational Intractability
Diakonikolas, I., Kamath, G., Kane, D. M., Li, J., Moitra, A., & Stewart, A. (2016) · 2016 IEEE 57th Annual Symposium on Foundations of Computer Science (FOCS) · doi:10.1109/focs.2016.85 · Cited by 200 · cited 1×
“A recent line of work [8, 15] presents polynomial algorithms to consistently estimate the mean and covariance of a distribution from N i.”
BackgroundCommunication-efficient algorithms for statistical optimization
(2013) · Journal of Machine Learning Research · doi:10.5555/2567709.2567769 · Cited by 195 · cited 1×
“One-Shot Aggregation: On the other end of the spectrum, using a One-Short Aggregation method, each working machine computes an estimate of the model parameter using only its local data and reports it to the server, which then aggregates all the estimates reported to obtain a final estimate [33, 32].”
Method
Research works printed but not linked: 15
- Peva Blanchard , El Mahdi El Mhamdi , Rachid Guerraoui, and Julien Stainer. 2017 . Byzantine-Tolerant Machine Learning . arXiv preprint arXiv:1703.02757 (2017). Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. 2017. Byzantine-Tolerant Machine Learning. arXiv preprint arXiv:1703.02757 (2017). · arXiv
- Stephen Boyd and Lieven Vandenberghe . 2004. Convex optimization . Cambridge university press . Stephen Boyd and Lieven Vandenberghe. 2004. Convex optimization. Cambridge university press.
- John Duchi Martin J Wainwright and Michael I Jordan. 2013. Local privacy and minimax bounds: Sharp rates for probability estimation. Advances in Neural Information Processing Systems. 1529--1537. John Duchi Martin J Wainwright and Michael I Jordan. 2013. Local privacy and minimax bounds: Sharp rates for probability estimation. Advances in Neural Information Processing Systems. 1529--1537.
- Jiashi Feng , Huan Xu , and Shie Mannor . 2014. Distributed Robust Learning. arXiv preprint arXiv:1409.5937 ( 2014 ). Jiashi Feng, Huan Xu, and Shie Mannor. 2014. Distributed Robust Learning. arXiv preprint arXiv:1409.5937 (2014). · arXiv
- Michael I Jordan , Jason D Lee , and Yun Yang . 2016. Communication-Efficient Distributed Statistical Inference. arXiv preprint arXiv:1605.07689 ( 2016 ). Michael I Jordan, Jason D Lee, and Yun Yang. 2016. Communication-Efficient Distributed Statistical Inference. arXiv preprint arXiv:1605.07689 (2016). · arXiv
- JHB Kemperman . 1987. The median of a finite measure on a Banach space. Statistical data analysis based on the L1-norm and related methods (Neuchâtel , 1987 ) (1987), 217--230. JHB Kemperman. 1987. The median of a finite measure on a Banach space. Statistical data analysis based on the L1-norm and related methods (Neuchâtel, 1987) (1987), 217--230.
- Jakub Konevcnỳ , Brendan McMahan , and Daniel Ramage . 2015. Federated optimization: Distributed optimization beyond the datacenter. arXiv preprint arXiv:1511.03575 ( 2015 ). Jakub Konevcnỳ, Brendan McMahan, and Daniel Ramage. 2015. Federated optimization: Distributed optimization beyond the datacenter. arXiv preprint arXiv:1511.03575 (2015). · arXiv
- Hendrik P Lopuhaa and Peter J Rousseeuw . 1991. Breakdown points of affine equivariant estimators of multivariate location and covariance matrices. The Annals of Statistics ( 1991 ), 229--248. Hendrik P Lopuhaa and Peter J Rousseeuw. 1991. Breakdown points of affine equivariant estimators of multivariate location and covariance matrices. The Annals of Statistics (1991), 229--248.
- Nancy A. Lynch . 1996. Distributed Algorithms . Morgan Kaufmann Publishers Inc ., San Francisco, CA, USA. \showISBNx9780080504704 Nancy A. Lynch. 1996. Distributed Algorithms. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA. \showISBNx9780080504704
- Song Mei , Yu Bai , and Andrea Montanari . 2016. The landscape of empirical risk for non-convex losses. arXiv preprint arXiv:1607.06534 ( 2016 ). Song Mei, Yu Bai, and Andrea Montanari. 2016. The landscape of empirical risk for non-convex losses. arXiv preprint arXiv:1607.06534 (2016). · arXiv
- Philipp Moritz , Robert Nishihara , Ion Stoica , and Michael I Jordan . 2015 . Sparknet: Training deep networks in spark. arXiv preprint arXiv:1511.06051 (2015). Philipp Moritz, Robert Nishihara, Ion Stoica, and Michael I Jordan. 2015. Sparknet: Training deep networks in spark. arXiv preprint arXiv:1511.06051 (2015). · arXiv
- Charles P. Pfleeger and Shari Lawrence Pfleeger . 2002 . Security in Computing (3rd ed.). Prentice Hall Professional Technical Reference . Charles P. Pfleeger and Shari Lawrence Pfleeger. 2002. Security in Computing (3rd ed.). Prentice Hall Professional Technical Reference.
- Foster J Provost and Daniel N Hennessy . 1996 . Scaling up: Distributed machine learning with cooperation . AAAI/IAAI , Vol. 1 . Citeseer, 74--79. Foster J Provost and Daniel N Hennessy. 1996. Scaling up: Distributed machine learning with cooperation. AAAI/IAAI, Vol. 1. Citeseer, 74--79.
- R. Vershynin . 2010. Introduction to the non-asymptotic analysis of random matrices. Arxiv preprint arxiv:1011.3027 ( 2010 ). R. Vershynin. 2010. Introduction to the non-asymptotic analysis of random matrices. Arxiv preprint arxiv:1011.3027 (2010). · arXiv
- 10.5555/1833515.1833620 · doi:10.5555/1833515.1833620
Cited by
Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms
Zhang, K., Yang, Z., & Başar, T. (2021) · Studies in Systems, Decision and Control · doi:10.1007/978-3-030-60990-0_12 · Cited by 1,264
Federated Learning via Over-the-Air Computation
Yang, K., Jiang, T., Shi, Y., & Ding, Z. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2961673 · Cited by 1,084
“Byzantine failures [20]) bring critical security issues for large-scale distributed learning, which will incur a major degradation of the learning performance [21].”
BackgroundRobust Aggregation for Federated Learning
Pillutla, K., Kakade, S. M., & Harchaoui, Z. (2022) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2022.3153135 · Cited by 801
“Byzantine robustness, resilience to arbitrary behavior of some devices [51], was studied in distributed optimization with gradient aggregation [5, 16, 20, 21, 23, 88].”
BackgroundEdge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
Privacy and Robustness in Federated Learning: Attacks and Defenses
Lyu, L., Yu, H., Ma, X., Chen, C., Sun, L., Zhao, J., Yang, Q., & Yu, P. S. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3216981 · Cited by 548
Communication-efficient edge ai: algorithms and systems
Shi, Y., Yang, K., Jiang, T., Zhang, J., & Letaief, K. B. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3007787 · Cited by 488
Privacy-Enhanced Federated Learning Against Poisoning Adversaries
Liu, X., Li, H., Xu, G., Chen, Z., Huang, X., & Lu, R. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2021.3108434 · Cited by 351
Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses
Goldblum, M., Tsipras, D., Xie, C., Chen, X., Schwarzschild, A., Song, D., Madry, A., Li, B., & Goldstein, T. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3162397 · Cited by 362
“Another class of approaches employs the coordinate-wise median gradient [192], geometric median of means [193], and approximate geometric median [194] since median-based computations are more resistant to outliers than mean-based aggregation.”
BackgroundVeriFL: Communication-Efficient and Fast Verifiable Aggregation for Federated Learning
Guo, X., Liu, Z., Li, J., Gao, J., Hou, B., Dong, C., & Baker, T. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2020.3043139 · Cited by 314
Byzantine-Resilient Secure Federated Learning
So, J., Guler, B., & Avestimehr, A. S. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3041404 · Cited by 313
Blockchain-empowered Federated Learning: Challenges, Solutions, and Future Directions
Zhu, J., Cao, J., Saxena, D., Jiang, S., & Ferradi, H. (2023) · ACM Computing Surveys · doi:10.1145/3570953 · Cited by 309
A Survey of Incentive Mechanism Design for Federated Learning
Zhan, Y., Zhang, J., Hong, Z., Wu, L., Li, P., & Guo, S. (2021) · IEEE Transactions on Emerging Topics in Computing · doi:10.1109/tetc.2021.3063517 · Cited by 284
A survey of federated learning for edge computing: Research problems and solutions
Xia, Q., Ye, W., Tao, Z., Wu, J., & Li, Q. (2021) · High-Confidence Computing · doi:10.1016/j.hcc.2021.100008 · Cited by 279
“…al. proposed geometric median, marginal median and median-around-median [81] , Yin et al. proposed coordinate-wised median [90] , Su et al. proposed a batch normalized median [93] , Alistarh et al. proposed a more complicated modification of median-based methods that is called Byzanti-neSGD [94] .”
MethodEdge Intelligence: Empowering Intelligence to the Edge of Network
Xu, D., Li, T., Li, Y., Su, X., Tarkoma, S., Jiang, T., Crowcroft, J., & Hui, P. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3119950 · Cited by 272
Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient Descent
Chen, Y., Su, L., & Xu, J. (2018) · Abstracts of the 2018 ACM International Conference on Measurement and Modeling of Computer Systems · doi:10.1145/3219617.3219655 · Cited by 228
“∗The full version of this paper is available as [1] †This is the corresponding author”
MethodSelf-citationFederated Variance-Reduced Stochastic Gradient Descent With Robustness to Byzantine Attacks
Wu, Z., Ling, Q., Chen, T., & Giannakis, G. B. (2020) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2020.3012952 · Cited by 226
A Privacy-Aware and Incremental Defense Method Against GAN-Based Poisoning Attack
Qiao, F., Li, Z., & Kong, Y. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2023.3263241 · Cited by 221
Fed-IIoT: A Robust Federated Malware Detection Architecture in Industrial IoT
Taheri, R., Shojafar, M., Alazab, M., & Tafazolli, R. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.3043458 · Cited by 218
MPAF: Model Poisoning Attacks to Federated Learning based on Fake Clients
Cao, X., & Gong, N. Z. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw56347.2022.00383 · Cited by 205
“The first type of defense [10, 13, 19, 20, 29, 30] designs Byzantine-robust aggregation rules.”
BackgroundOn the Byzantine Robustness of Clustered Federated Learning
Sattler, F., Muller, K.-R., Wiegand, T., & Samek, W. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054676 · Cited by 166
“This setting has been extensively studied and a variety of robust aggregation rules have been proposed which rely on gradient similarity [7][8], geometric median aggregation [11], redundant communication [12] or adaptive model quality estimation [13].”
MethodFederated Learning: A signal processing perspective
Gafni, T., Shlezinger, N., Cohen, K., Eldar, Y. C., & Poor, H. V. (2022) · IEEE Signal Processing Magazine · doi:10.1109/msp.2021.3125282 · Cited by 176
SEAR: Secure and Efficient Aggregation for Byzantine-Robust Federated Learning
Zhao, L., Jiang, J., Feng, B. Y., Wang, Q., Shen, C. Q., & Li, Q. L. (2021) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2021.3093711 · Cited by 179
Zhan, Y., Li, P., & Guo, S. (2020) · 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps47924.2020.00033 · Cited by 124
ByRDiE: Byzantine-Resilient Distributed Coordinate Descent for Decentralized Learning
Yang, Z., & Bajwa, W. U. (2019) · IEEE Transactions on Signal and Information Processing over Networks · doi:10.1109/tsipn.2019.2928176 · Cited by 172
A Decentralized Federated Learning Framework via Committee Mechanism With Convergence Guarantee
Che, C., Li, X., Chen, C., He, X., & Zheng, Z. (2022) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2022.3202887 · Cited by 149
Shielding Collaborative Learning: Mitigating Poisoning Attacks through Client-Side Detection
Zhao, L., Hu, S., Wang, Q., Jiang, J., Chao, S., Luo, X., & Hu, P. (2020) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2020.2986205 · Cited by 127
Ng, J. S., Lim, W. Y. B., Luong, N. C., Xiong, Z., Asheralieva, A., Niyato, D. T., Leung, C., & Miao, C. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3091684 · Cited by 160
FedRecover: Recovering from Poisoning Attacks in Federated Learning using Historical Information
Cao, X., Jia, J., Zhang, Z., & Gong, N. Z. (2023) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46215.2023.10179336 · Cited by 203
“Different FL methods essentially use different aggregation rules [8], [11], [16], [24], [25], [36] in Step III.”
BackgroundMethodAn Experimental Study of Byzantine-Robust Aggregation Schemes in Federated Learning
Li, S., Ngai, E. C. ‐., & Voigt, T. (2023) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2023.3237397 · Cited by 176
Privacy-Preserving Federated Learning With Malicious Clients and Honest-but-Curious Servers
Le, J., Zhang, D., Lei, X., Jiao, L., Zeng, K., & Liao, X. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3295949 · Cited by 125
Zhang, Z., Zhang, Y., Guo, D., Yao, L. Y., & Zhao, L. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.04.010 · Cited by 121
Threats to Training: A Survey of Poisoning Attacks and Defenses on Machine Learning Systems
Wang, Z., Ma, J., Wang, X., Hu, J., Qin, Z., & Ren, K. (2022) · ACM Computing Surveys · doi:10.1145/3538707 · Cited by 179
“Inspired by geometric median, researches [11, 88] presented two Byzantine-robust gradient descent (GD) algorithms.”
MethodByzantine-robust Federated Learning through Collaborative Malicious Gradient Filtering
Xu, J., Huang, S., Song, L., & Lan, T. (2022) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs54860.2022.00120 · Cited by 145
“Recently, much research attention has focused on mitigating Byzantine attacks either by leveraging statistic-based outlier detection techniques [15], [16] or by utilizing auxiliary labeled data collected by PS to verify the correctness of received gradients [17], [18].”
BackgroundSelf-citationAccelerating Decentralized Federated Learning in Heterogeneous Edge Computing
Wang, L., Xu, Y., Xu, H., Chen, M., & Huang, L. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2022.3178378 · Cited by 125
Yang, Z., Gang, A., & Bajwa, W. U. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2973345 · Cited by 112
Byzantine-Resilient Multiagent Optimization
Su, L., & Vaidya, N. H. (2020) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2020.3008139 · Cited by 114
Self-citationA Secure Federated Learning Framework for Residential Short-Term Load Forecasting
Husnoo, M. A., Anwar, A., Hosseinzadeh, N., Islam, S. N., Mahmood, A. N., & Doss, R. (2023) · IEEE Transactions on Smart Grid · doi:10.1109/tsg.2023.3292382 · Cited by 112
A Survey of Trustworthy Federated Learning: Issues, Solutions, and Challenges
Zhang, Y., Zeng, D., Luo, J., Fu, X., Chen, G., Xu, Z., & King, I. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3678181 · Cited by 136
FLARE: Defending Federated Learning against Model Poisoning Attacks via Latent Space Representations
Wang, N., Xiao, Y., Chen, Y., Yang, H., Lou, W., & Hou, Y. T. (2022) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3488932.3517395 · Cited by 125
Self-citationFedFAIM: A Model Performance-Based Fair Incentive Mechanism for Federated Learning
Shi, Z., Zhang, L., Yao, Z., Lyu, L., Chen, C., Wang, L., Wang, J., & Li, X. (2022) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2022.3183614 · Cited by 86
Federated Anomaly Analytics for Local Model Poisoning Attack
Shi, S., Hu, C., Wang, D., Zhu, Y., & Han, Z. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118347 · Cited by 64
Siren: Byzantine-robust Federated Learning via Proactive Alarming
Guo, H., Wang, H., Song, T., Yang, H., Lv, Z., Jin, X., Xue, Z., Ma, R., & Guan, H. (2021) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/3472883.3486990 · Cited by 75
“In this case, most current Byzantine-robust aggregation rules [10, 27, 32] only do the weight analysis and only deploy defense on the server, which makes the system vulnerable.”
BackgroundDynamic defense against byzantine poisoning attacks in federated learning
Rodríguez-Barroso, N., Martínez-Cámara, E., Luzón, M. V., & Herrera, F. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.03.003 · Cited by 54
Communication Efficient Federated Learning With Energy Awareness Over Wireless Networks
Jin, R., He, X., & Dai, H. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2021.3138394 · Cited by 66
RoseAgg: Robust Defense Against Targeted Collusion Attacks in Federated Learning
Yang, H., Xi, W., Shen, Y., Wu, C., & Zhao, J. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3352415 · Cited by 54
Detecting Malicious Model Updates from Federated Learning on Conditional Variational Autoencoder
Gu, Z., & Yang, Y. (2021) · 2021 IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps49936.2021.00075 · Cited by 47
Privacy Preservation for Federated Learning With Robust Aggregation in Edge Computing
Liu, W., Xu, X., Li, D., Qi, L., Dai, F., Dou, W., & Ni, Q. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3229122 · Cited by 50
Yang, H., Zhang, X., Fang, M., & Liu, J. (2019) · 2019 IEEE 58th Conference on Decision and Control (CDC) · doi:10.1109/cdc40024.2019.9029245 · Cited by 40
Resilient Primal–Dual Optimization Algorithms for Distributed Resource Allocation
Turan, B., Uribe, C. A., Wai, H.-T., & Alizadeh, M. (2020) · IEEE Transactions on Control of Network Systems · doi:10.1109/tcns.2020.3024485 · Cited by 49
Fault-Tolerance in Distributed Optimization: The Case of Redundancy
Gupta, N., & Vaidya, N. H. (2020) · ACM Symposium on Principles of Distributed Computing (PODC) · doi:10.1145/3382734.3405748 · Cited by 63
Collaborative Machine Learning: Schemes, Robustness, and Privacy
Wang, J., Pal, A., Yang, Q., Kant, K., Zhu, K., & Guo, S. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3169347 · Cited by 54
Security of Federated Learning: Attacks, Defensive Mechanisms, and Challenges
Benmalek, M., Benrekia, M. A., & Challal, Y. (2022) · Revue d intelligence artificielle · doi:10.18280/ria.360106 · Cited by 57
L4L: Experience-Driven Computational Resource Control in Federated Learning
Zhan, Y., Li, P., Wu, L., & Guo, S. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2021.3068219 · Cited by 43
A Multi-Shuffler Framework to Establish Mutual Confidence for Secure Federated Learning
Zhou, Z., Xu, C., Wang, M., Kuang, X., Zhuang, Y., & Yu, S. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2022.3215574 · Cited by 43
“Dynamic malice scaling (DMS) strategy stems from mitigation-based security strategies like [15], [16], [17].”
BackgroundEfficiently Achieving Privacy Preservation and Poisoning Attack Resistance in Federated Learning
Li, X.-Y., Yang, X. S., Zhou, Z., & Lu, R. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3378006 · Cited by 54
Edge Learning: The Enabling Technology for Distributed Big Data Analytics in the Edge
Zhang, J., Qu, Z., Chen, C., Wang, H., Zhan, Y., Ye, B., & Guo, S. (2021) · ACM Computing Surveys · doi:10.1145/3464419 · Cited by 83
Fan, X., Wang, Y., Huo, Y., & Tian, Z. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3164339 · Cited by 43
Byzantine-Resilient Federated Learning at Edge
Tao, Y., Cui, S., Xu, W., Yin, H., Yu, D., Liang, W., & Cheng, X. (2023) · IEEE Transactions on Computers · doi:10.1109/tc.2023.3257510 · Cited by 49
Securing Distributed Gradient Descent in High Dimensional Statistical Learning
Su, L., & Xu, J. (2019) · Abstracts of the 2019 SIGMETRICS/Performance Joint International Conference on Measurement and Modeling of Computer Systems · doi:10.1145/3309697.3331499 · Cited by 35
Self-citationApproximate Byzantine Fault-Tolerance in Distributed Optimization
Liu, S., Gupta, N., & Vaidya, N. H. (2021) · ACM Symposium on Principles of Distributed Computing (PODC) · doi:10.1145/3465084.3467902 · Cited by 65
FLTracer: Accurate Poisoning Attack Provenance in Federated Learning
Zhang, X., Liu, Q., Ba, Z., Hong, Y., Zheng, T., Lin, F., Lü, L., & Ren, K. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3410014 · Cited by 46
“Byzantine-robust Aggregation Algorithms: give provable convergence guarantees for each round of FL aggregation against poisoning attacks caused by Byzantine client failures [43], [44], [45].”
BackgroundCRACAU: Byzantine Machine Learning Meets Industrial Edge Computing in Industry 5.0
Du, A., Shen, Y., Zhang, Q., Tseng, L., & Aloqaily, M. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2021.3097072 · Cited by 42
Auto-weighted Robust Federated Learning with Corrupted Data Sources
Li, S., Ngai, E. C. ‐., Ye, F., & Voigt, T. (2022) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3517821 · Cited by 43
“[9] proposed to apply GM as a gradient aggregation protocol in robust distributed learning and showed that it can tolerate up to half malicious clients while estimating the underlying true gradients.”
BackgroundAli, H., Chen, D., Harrington, M., Salazar, N., Ameedi, M. A., Khan, A. F., Butt, A. R., & Cho, J.-H. (2023) · IEEE Access · doi:10.1109/access.2023.3326410 · Cited by 47
“A simple solution from distributed ML is to replace traditional Federated Averaging step [84] with simply taking the median of all received gradients [109], [110].”
MethodOptimizing Federated Learning With Aggregation Strategies: A Comprehensive Survey
Khan, N., Nisar, S., Khan, M. A., Rehman, Y. A. U., Noor, F., & Barb, G. (2025) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2025.3590102 · Cited by 46
“Median-based rules were introduced by Chen et al. [39] and Yin et al. [94] to mitigate outlier influence.”
Towards Efficient Synchronous Federated Training: A Survey on System Optimization Strategies
Jiang, Z., Wang, W., Li, B., & Yang, Q. (2022) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2022.3177222 · Cited by 46
Unsupervised Federated Learning for Unbalanced Data
Servetnyk, M., Fung, C. C., & Han, Z. (2020) · Globecom · doi:10.1109/globecom42002.2020.9348203 · Cited by 40
WMDefense: Using Watermark to Defense Byzantine Attacks in Federated Learning
Zheng, X., Dong, Q., & Fu, A. (2022) · IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS) · doi:10.1109/infocomwkshps54753.2022.9798217 · Cited by 37
The Convergence of Stochastic Gradient Descent in Asynchronous Shared Memory
Alistarh, D., De, C., & Konstantinov, N. (2018) · ACM Symposium on Principles of Distributed Computing (PODC) · doi:10.1145/3212734.3212763 · Cited by 41
Byzantine-Resilient SGD in High Dimensions on Heterogeneous Data
Data, D., & Diggavi, S. N. (2021) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit45174.2021.9518248 · Cited by 46
Kapoor, A., & Kumar, D. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3434510 · Cited by 39
Byzantine-Resilient Decentralized Policy Evaluation With Linear Function Approximation
Wu, Z., Shen, H., Chen, T., & Ling, Q. H. (2021) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2021.3090952 · Cited by 40
Byzantine-robust federated learning via credibility assessment on non-IID data
Zhai, K., Ren, Q., Wang, J., & Yan, C. (2021) · Mathematical Biosciences & Engineering · doi:10.3934/mbe.2022078 · Cited by 36
RobustFed: A Truth Inference Approach for Robust Federated Learning
Tahmasebian, F., Lou, J., & Xiong, L. (2022) · Proceedings of the 31st ACM International Conference on Information & Knowledge Management · doi:10.1145/3511808.3557439 · Cited by 26
Resilient Primal-Dual Optimization Algorithms for Distributed Resource Allocation
Turan, B., Uribe, C. A., Wai, H.-T., & Alizadeh, M. (2020) · eScholarship (California Digital Library) · Cited by 26
Jatain, D., Sıngh, V., & Dahiya, N. (2021) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2021.05.016 · Cited by 45
Untargeted Poisoning Attack Detection in Federated Learning via Behavior AttestationAl
Mallah, R. A., Lopez, D. L., Badu-Marfo, G., & Farooq, B. (2023) · IEEE Access · doi:10.1109/access.2023.3330144 · Cited by 61
Byzantine-Resilient Federated Machine Learning via Over-the-Air Computation
Huang, S., Zhou, Y., Wang, T., & Shi, Y. (2021) · IEEE International Conference on Communications workshops · doi:10.1109/iccworkshops50388.2021.9473694 · Cited by 31
“tion rule linearly converges to a neighborhood of the optimal solution [29], [30].”
BackgroundDefending Against Byzantine Attacks in Quantum Federated Learning
Xia, Q., Tao, Z., & Li, Q. (2021) · International Conference on Mobility, Sensing and Networking (MSN) · doi:10.1109/msn53354.2021.00035 · Cited by 38
Byzantine-Robust Distributed Learning With Compression
Zhu, H., & Ling, Q. H. (2023) · IEEE Transactions on Signal and Information Processing over Networks · doi:10.1109/tsipn.2023.3265892 · Cited by 36
Practical Differentially Private and Byzantine-resilient Federated Learning
Xiang, Z., Wang, T., Lin, W., & Wang, D. (2023) · Proceedings of the ACM on Management of Data · doi:10.1145/3589264 · Cited by 49
“To mitigate this issue, recent work proposes Byzantine-resilient machine learning approaches, such as diagnosing and rejecting gradients with abnormal features [9, 11, 15, 50, 52].”
BackgroundMethodInfluentialDong, X., Wu, Z., Ling, Q. H., & Tian, Z. (2023) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2023.3340028 · Cited by 32
Distributed Statistical Machine Learning in Adversarial Settings
Chen, Y., Su, L., & Xu, J. (2018) · ACM SIGMETRICS Performance Evaluation Review · doi:10.1145/3292040.3219655 · Cited by 22
Self-citationCommunication-Efficient and Byzantine-Robust Distributed Learning With Error Feedback
Ghosh, A., Maity, R. K., Kadhe, S., Mazumdar, A., & Ramchandran, K. (2021) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2021.3105076 · Cited by 35
Byzantine Fault-Tolerance in Decentralized Optimization under 2f-Redundancy
Gupta, N., Doan, T. T., & Vaidya, N. H. (2021) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc50511.2021.9483067 · Cited by 33
Byzantine Resilient Distributed Learning in Multirobot Systems
Li, J., Abbas, W., Shabbir, M., & Koutsoukos, X. (2022) · IEEE Transactions on Robotics · doi:10.1109/tro.2022.3178296 · Cited by 30
“Well-known majority-based aggregation rules include the CM [12]–[14], coordinate-wise trimmed mean [12], [15], GM [16], [17], Krum and multi-Krum”
BackgroundMethodTowards federated learning: An overview of methods and applications
Silva, P. R., Vinagre, J., & da Gama, J. M. P. (2023) · Wiley Interdisciplinary Reviews Data Mining and Knowledge Discovery · doi:10.1002/widm.1486 · Cited by 33
zPROBE: Zero Peek Robustness Checks for Federated Learning
Ghodsi, Z., Javaheripi, M., Sheybani, N., Zhang, X., Huang, K., & Koushanfar, F. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00448 · Cited by 36
NSPFL: A Novel Secure and Privacy-Preserving Federated Learning With Data Integrity Auditing
Zhang, Z., & Li, Y. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3379852 · Cited by 30
Communication-Efficient and Byzantine-Robust Distributed Learning
Ghosh, A., Maity, R. K., Kadhe, S., Mazumdar, A., & Ramchandran, K. (2020) · Information Theory and Applications Workshop (ITA) · doi:10.1109/ita50056.2020.9245017 · Cited by 22
Collaborative Byzantine Resilient Federated Learning
Gouissem, A., Abualsaud, K., Yaacoub, E. E., Khattab, T. M., & Guizani, M. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3266347 · Cited by 26
“2) Geometric mean (GeoMean), where the aggregation is done using a geometric mean cross distance minimization scheme as detailed in [15].”
BackgroundMethodFair Detection of Poisoning Attacks in Federated Learning
Singh, A. K., Blanco-Justicia, A., Domingo‐Ferrer, J., Sánchez, D., & Rebollo‐Monedero, D. (2020) · IEEE International Conference on Tools with Artificial Intelligence (ICTAI) · doi:10.1109/ictai50040.2020.00044 · Cited by 25
Optimal Communication-Computation Trade-Off in Heterogeneous Gradient Coding
Jahani-Nezhad, T., & Maddah-Ali, M. A. (2021) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2021.3085676 · Cited by 25
Byzantine-Robust Federated Learning with Variance Reduction and Differential Privacy
Zhang, Z., & Hu, R. (2023) · IEEE Conference on Communications and Network Security · doi:10.1109/cns59707.2023.10288938 · Cited by 31
Efficient and Privacy-Preserving Federated Learning Against Poisoning Adversaries
Zhao, J., Zhu, H., Wang, F., Zheng, Y., Lu, R., & Li, H. (2024) · IEEE Transactions on Services Computing · doi:10.1109/tsc.2024.3377931 · Cited by 31
PEFL: Privacy-Preserved and Efficient Federated Learning With Blockchain
Tian, L., Lin, F., Gan, J., Jia, R., Zheng, Z., & Li, M. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3479328 · Cited by 22
Securing Distributed Gradient Descent in High Dimensional Statistical Learning
Su, L., & Xu, J. (2019) · Proceedings of the ACM on Measurement and Analysis of Computing Systems · doi:10.1145/3322205.3311083 · Cited by 50
Self-citationData Encoding Methods for Byzantine-Resilient Distributed Optimization
Data, D., Song, L., & Diggavi, S. N. (2019) · IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849857 · Cited by 20
Adversarial Robustness through Bias Variance Decomposition: A New Perspective for Federated Learning
Zhou, Y., Wu, J., Wang, H., & He, J. (2022) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3511808.3557232 · Cited by 42
Secure Distributed Estimation Over Multitask Networks Against Multiple Attacks
Liu, Q., Ye, M., & Chen, F. (2022) · IEEE Transactions on Aerospace and Electronic Systems · doi:10.1109/taes.2022.3215948 · Cited by 18
Metadata
- Authors
- Chen, Y., Su, L., & Xu, J.
- Affiliations
- Cornell University (US)
- University of Illinois Urbana-Champaign (US)
- Purdue University West Lafayette (US)
- Venue
- Proceedings of the ACM on Measurement and Analysis of Computing Systems
- Published
- 2017-12-19
- Volume
- 1
- Issue
- 2
- Pages
- 1-25
- ISSN
- 2476-1249 (ISSN-L)
- OpenAlex
- W4252654521
- Cited by
- 390
- References
- 15
- Citations by year
- 2026: 36 · 2025: 85 · 2024: 68 · 2023: 61 · 2022: 47 · 2021: 44 · 2020: 28 · 2019: 14 · 2018: 7
- FWCI
- 8.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Proceedings of the ACM on Measurement and Analysis of Computing Systems (publishedVersion)
- Funders
- NSF (CRII award 1657420; CCF-1704828; ECCS-1610543; CCF-0939370)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Engineering › Control and Systems Engineering › Distributed Control Multi-Agent Systems
- Keywords
- Byzantine faults, federated learning, decentralized systems, parameter server, linear regression, communication complexity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.