The DOI matches its Crossref record; 21 of the 21 references it cites were found in a registry or an index.
RSA: Byzantine-Robust Stochastic Aggregation Methods for Distributed Learning from Heterogeneous Datasets
Open accessAbstract
In this paper, we propose a class of robust stochastic subgradient methods for distributed learning from heterogeneous datasets at presence of an unknown number of Byzantine workers. The Byzantine workers, during the learning process, may send arbitrary incorrect messages to the master due to data corruptions, communication failures or malicious attacks, and consequently bias the learned model. The key to the proposed methods is a regularization term incorporated with the objective function so as to robustify the learning task and mitigate the negative effects of Byzantine attacks. The resultant subgradient-based algorithms are termed Byzantine-Robust Stochastic Aggregation methods, justifying our acronym RSA used henceforth. In contrast to most of the existing algorithms, RSA does not rely on the assumption that the data are independent and identically distributed (i.i.d.) on the workers, and hence fits for a wider class of applications. Theoretically, we show that: i) RSA converges to a near-optimal solution with the learning error dependent on the number of Byzantine workers; ii) the convergence rate of RSA under Byzantine attacks is the same as that of the stochastic gradient descent method, which is free of Byzantine attacks. Numerically, experiments on real dataset corroborate the competitive performance of RSA and a complexity reduction compared to the state-of-the-art alternatives.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795
Security, privacy and trust in Internet of Things: The road ahead
Sicari, S., Rizzardi, A., Grieco, L. A., & Coen-Porisini, A. (2015) · Computer Networks · doi:10.1016/j.comnet.2014.11.008 · Cited by 2,187
Robust Stochastic Approximation Approach to Stochastic Programming
Nemirovski, A., Juditsky, A., Lan, G., & Shapiro, A. (2009) · SIAM Journal on Optimization · doi:10.1137/070704277 · Cited by 1,945
(1996) · Lecture Notes in Computer Science · doi:10.1007/3-540-61769-8 · Cited by 1,865
(2017) · neural information processing systems · Cited by 1,413
Byzantine-Robust Distributed Learning: Towards Optimal Statistical Rates
Yin, D., Chen, Y., Ramchandran, K., & Bartlett, P. (2018) · arXiv · doi:10.48550/arxiv.1803.01498 · Cited by 1,130
On the Linear Convergence of the ADMM in Decentralized Consensus Optimization
Shi, W., Ling, Q., Yuan, K., Wu, G., & Yin, W. (2014) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2014.2304432 · Cited by 820
Self-citationCommunication Efficient Distributed Machine Learning with the Parameter Server
Li, M., Andersen, D. G., Smola, A. J., & Yu, K. (2014) · neural information processing systems · Cited by 715
Federated Learning: Collaborative Machine Learning without Centralized Training Data
A, A. V., Binny, S., R, J. T., Raj, N., & Thomas, V. (2022) · international journal of engineering technology and management sciences · doi:10.46647/ijetms.2022.v06i05.052 · Cited by 489
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2011) · Chapman & Hall/CRC Computer Science & Data Analysis · doi:10.1201/b11429-4 · Cited by 460
Lynch, N. (1996) · Cited by 354
On the point for which the sum of the distances to n given points is minimum
Weiszfeld, E., & Plastria, F. (2009) · Annals of Operations Research · doi:10.1007/s10479-008-0352-z · Cited by 248
DRACO: Byzantine-resilient Distributed Training via Redundant GradientsPreprint
Chen, L., Wang, H., Charles, Z., & Papailiopoulos, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.09877 · Cited by 191
Distributed Statistical Machine Learning in Adversarial Settings: Byzantine Gradient DescentPreprint
Chen, Y., Su, L., & Xu, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.05491 · Cited by 170
Generalized Byzantine-tolerant SGD
Xie, C., Koyejo, S., & Gupta, I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.10116 · Cited by 166
Byzantine Stochastic Gradient Descent
Alistarh, D., Allen-Zhu, Z., & Li, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1803.08917 · Cited by 163
Phocas: dimensional Byzantine-resilient stochastic gradient descentPreprint
Xie, C., Koyejo, S., & Gupta, I. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.09682 · Cited by 57
Robust Distributed Consensus Using Total Variation
Ben‐Ameur, W., Bianchi, P., & Jakubowicz, J. (2015) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2015.2471755 · Cited by 47
Zeno: Byzantine-suspicious stochastic gradient descent.Preprint
Xie, C., Koyejo, S., & Gupta, I. (2018) · arXiv (Cornell University) · Cited by 39
Securing Distributed Machine Learning in High DimensionsPreprint
Su, L., & Xu, J. (2018) · arXiv (Cornell University) · Cited by 25
Asynchronous periodic event-triggered coordination of multi-agent systems
Liu, Y., Nowzari, C., Tian, Z., & Ling, Q. H. (2017) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2017.8264668 · Cited by 15
Self-citation
Cited by
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
Li, Q., Wen, Z., Wu, Z., Hu, S., Wang, N., Li, Y., Liu, X., & He, B. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3124599 · Cited by 1,407
FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping
Cao, X., Fang, M., Liu, J., & Gong, N. Z. (2021) · Proceedings 2021 Network and Distributed System Security Symposium · doi:10.14722/ndss.2021.24434 · Cited by 999
Robust Aggregation for Federated Learning
Pillutla, K., Kakade, S. M., & Harchaoui, Z. (2022) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2022.3153135 · Cited by 801
An Efficient Framework for Clustered Federated Learning
Ghosh, A., Chung, J., Yin, D., & Ramchandran, K. (2022) · IEEE Transactions on Information Theory · doi:10.1109/tit.2022.3192506 · Cited by 623
Wahab, O. A., Mourad, A., Otrok, H., & Taleb, T. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3058573 · Cited by 487
Metadata
- Authors
- Li, L., Xu, W., Chen, T., Giannakis, G. B., & Ling, Q.
- Venue
- Proceedings of the AAAI Conference on Artificial Intelligence
- Published
- 2019-07-17
- Volume
- 33
- Issue
- 01
- Pages
- 1544-1551
- ISSN
- 2159-5399 (ISSN-L)
- 2374-3468
- OpenAlex
- W2963334472
- Cited by
- 373
- References
- 21
- FWCI
- 13.52
- Type
- conference paper
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.