Federated Learning: Strategies for Improving Communication Efficiency
Open accessAbstract
Federated Learning is a machine learning setting where the goal is to train a high-quality centralized model while training data remains distributed over a large number of clients each with unreliable and relatively slow network connections. We consider learning algorithms for this setting where on each round, each client independently computes an update to the current model based on its local data, and communicates this update to a central server, where the client-side updates are aggregated to compute a new global model. The typical clients in this setting are mobile phones, and communication efficiency is of the utmost importance. In this paper, we propose two ways to reduce the uplink communication costs: structured updates, where we directly learn an update from a restricted space parametrized using a smaller number of variables, e.g. either low-rank or a random mask; and sketched updates, where we learn a full model update and then compress it using a combination of quantization, random rotations, and subsampling before sending it to the server. Experiments on both convolutional and recurrent networks show that the proposed methods can reduce the communication cost by two orders of magnitude.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Federated learning proposes a distributed way of updating a centralised model by aggregating each patient's local updates into a central server.
“Federated learning proposes a distributed way of updating a centralised model by aggregating each patient’s local updates into a central server119.”
From The future of sleep health: a data-driven revolution in sleep science and medicine (2020) · cites this paper as [119]
The Introduction (pp. 1–2) describes clients updating a shared model using local data and sending updates to a central server for aggregation. It does not describe patients as the clients; its motivating example is mobile devices, and the experiments use CIFAR-10 and Reddit data (Experiments, p. 4). The patient-specific part of the claim is therefore not supported by this paper. (LLM, conf=0.88)
Experimental — accuracy not yet measuredFederated Learning guarantees data privacy because data do not leave the local device and only model updates are transmitted.
“Since data do not leave the local device, only the transmission of model updates is required across distributed devices. This lightens the training process for network-compromised settings and guarantees data privacy [416].”
From Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI (2020) · cites this paper as [416]
The Introduction (pp. 1–2) says training data remain on devices, clients send model updates to a server, and the paper studies ways to reduce communication costs on limited connections. It does not establish that this arrangement guarantees privacy; the Introduction (p. 2) instead discusses cryptographic protocols to prevent inspection of individual updates. (LLM, conf=0.90)
Experimental — accuracy not yet measuredFederated learning approaches have arguably become the most widely used next-generation privacy preservation technique, both in industry and medical AI applications.
“Thus, federated learning approaches have arguably become the most widely used next-generation privacy preservation technique, both in industry 49 and medical AI applications 50 .”
From Secure, privacy-preserving and federated machine learning in medical imaging (2020) · cites this paper as [49]
The Introduction (p. 1) describes federated learning as a way to train on mobile-app data while keeping it on users’ devices, supporting its relevance to privacy-conscious applications. It presents mobile apps as a motivating example, not evidence that federated learning is the most widely used privacy-preservation technique in industry. The medical-AI portion has a separate citation in the citing sentence. (LLM, conf=0.83)
Experimental — accuracy not yet measuredFL is a distributed collaborative AI approach that allows for data training by coordinating multiple devices with a central server without sharing actual datasets.
“Technically, FL is a distributed collaborative AI approach that allows for data training by coordinating multiple devices with a central server without sharing actual datasets [6].”
From Federated Learning for Internet of Things: A Comprehensive Survey (2021) · cites this paper as [6]
Federated Learning computes ML models on locally captured data, transmits encrypted model updates to a central server, and preserves privacy because no client data is delivered to the server.
“By contrast, in Federated Learning [414, 415, 416], the computation of ML models is made on data captured locally by remote client devices”
From Explainable Artificial Intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI (2020) · cites this paper as [416]
Local differential privacy has been deployed effectively to gather statistics on popular items across large userbases by Google, Apple and Microsoft.
“LDP has been deployed effectively to gather statistics on popular items across large userbases by Google, Apple and Microsoft [177, 154, 155].”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [177]
The Introduction (p. 1) discusses federated learning on user devices, and Experiments (p. 4) describes simulations using Reddit data. Searches for local differential privacy, popular-item statistics, and deployment by Google, Apple, or Microsoft found no passage in our copy addressing the attributed claim. (LLM, conf=0.76)
Experimental — accuracy not yet measuredLDP protocols are effective at enforcing privacy and have theoretical justifications.
“While LDP protocols are effective at enforcing privacy and have theoretical justifications [177, 154, 155]”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [177]
The Introduction (pp. 1–2) discusses keeping training data on devices and mentions cryptographic protocols that prevent inspection of individual updates. Searches for LDP, local differential privacy, privacy guarantees, and theoretical justification found no passage addressing whether LDP protocols are effective or theoretically justified. (LLM, conf=0.76)
Experimental — accuracy not yet measuredFederated learning was initially proposed to address the learning problem in which datasets are distributed across multiple devices and not allowed to be leaked to others [220].
“Federated learning was initially proposed to address the learning problem in which datasets are distributed across multiple devices and not allowed to be leaked to others [220].”
From Empowering things with intelligence: a survey of the progress, challenges, and opportunities in artificial intelligence of things (2021) · cites this paper as [220]
FL enables collaborative training of a shared AI model across multiple mobile devices without exposing raw data, enhancing privacy.
“FL allows devices and the BS to train a common model while raw datasets are kept locally at users where the data reside.”
From Federated learning meets blockchain in edge computing: Opportunities and challenges (2021) · cites this paper as [17]
FL has transformed many applications by providing distributed AI solutions at the network edge with cooperation of multiple mobile devices.
“Since the invention in 2016 [17], FL has transformed many applications by providing distributed AI solutions at the network edge with the cooperation of multiple mobile devices.”
From Federated learning meets blockchain in edge computing: Opportunities and challenges (2021) · cites this paper as [17]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 21
- Rami Al-Rfou, Marc Pickett, Javier Snaider, Yun-hsuan Sung, Brian Strope, and Ray Kurzweil. Conversational contextual cues: The case of personalization and history for response ranking. arXiv:1606.00372 , 2016. · arXiv
- Dan Alistarh, Jerry Li, Ryota Tomioka, and Milan Vojnovic. QSGD: Randomized quantization for communication-optimal stochastic gradient descent. arXiv:1610.02132 , 2016. · arXiv
- Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for privacy preserving machine learning. In ACM Conference on Computer and Communications Security (ACM CCS) , 2017.
- Trishul Chilimbi, Yutaka Suzue, Johnson Apacible, and Karthik Kalyanaraman. Project adam: Building an efficient and scalable deep learning training system. In 11th USENIX Symposium on Operating Systems Design and Implementation (OSDI 14) , pp. 571-582, 2014.
- Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Andrew Senior, Paul Tucker, Ke Yang, Quoc V Le, et al. Large scale distributed deep networks. In NIPS , pp. 1223-1231, 2012.
- Misha Denil, Babak Shakibi, Laurent Dinh, Nando de Freitas, et al. Predicting parameters in deep learning. In NIPS , pp. 2148-2156, 2013.
- Mostafa El Gamal and Lifeng Lai. On randomized distributed coordinate descent with quantized updates. arXiv:1609.05539 , 2016. · arXiv
- Daniel Golovin, D. Sculley, H. Brendan McMahan, and Michael Young. Large-scale learning with less ram via randomization. In ICML , 2013.
- Song Han, Huizi Mao, and William J Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149 , 2015. · arXiv
- Jakub Koneˇ cn´ y and Peter Richt´ arik. Randomized distributed mean estimation: Accuracy vs communication. arXiv:1611.07555 , 2016. · arXiv
- Jakub Koneˇ cn´ y, H. Brendan McMahan, Daniel Ramage, and Peter Richt´ arik. Federated optimization: Distributed machine learning for on-device intelligence. arXiv preprint arXiv:1610.02527 , 2016. · arXiv
- Chenxin Ma, Jakub Koneˇ cn´ y, Martin Jaggi, Virginia Smith, Michael I Jordan, Peter Richt´ arik, and Martin Tak´ aˇ c. Distributed optimization with arbitrary local solvers. Optimization Methods & Software , 32(4): 813-848, 2017.
- H. Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication-efficient learning of deep networks from decentralized data. In Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS) , 2017.
- M.G. Rabbat and R.D. Nowak. Quantized incremental algorithms for distributed optimization. IEEE Journal on Selected Areas in Communications , 23(4):798-808, 2005.
- Sashank J Reddi, Jakub Koneˇ cn´ y, Peter Richt´ arik, Barnab´ as P´ ocz´ os, and Alex Smola. AIDE: Fast and communication efficient distributed optimization. arXiv:1608.06879 , 2016. · arXiv
- Ohad Shamir, Nathan Srebro, and Tong Zhang. Communication-efficient distributed optimization using an approximate Newton-type method. In ICML , pp. 1000-1008, 2014.
- Reza Shokri and Vitaly Shmatikov. Privacy-preserving deep learning. In Proceedings of the 22Nd ACM SIGSAC Conference on Computer and Communications Security , CCS '15, 2015. speedtest.net. Speedtest market report. http://www.speedtest.net/reports/united-states/ , August 2016.
- Jost Tobias Springenberg, Alexey Dosovitskiy, Thomas Brox, and Martin Riedmiller. Striving for simplicity: The all convolutional net. arXiv:1412.6806 , 2014. · arXiv
- Ananda Theertha Suresh, Felix X. Yu, Sanjiv Kumar, and H. Brendan McMahan. Distributed mean estimation with limited communication. In Proceedings of the 34th International Conference on Machine Learning , pp. 3329-3337, 2017.
- David P. Woodruff. Sketching as a tool for numerical linear algebra. Foundations and Trends in Theoretical Computer Science , 10(12):1-157, 2014. ISSN 1551-305X. doi: 10.1561/0400000060. · DOI
- Yuchen Zhang and Xiao Lin. DiSCO: Distributed optimization for self-concordant empirical loss. In ICML , pp. 362-370, 2015.
Cited by
Sorted by citations
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · see citing works
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772 · see citing works
“Methods combining Federated Averaging with sparsification and/or quantization of model updates to a small number of bits have demonstrated significant reductions in communication cost with minimal impact on training accuracy [245].”
BackgroundMethodSelf-citationFederated learning: Challenges, methods, and future directions
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2975749 · Cited by 6,210 · see citing works
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215 · see citing works
“[23] presented algorithms for further decreasing communication costs.”
BackgroundSelf-citationFederated Learning in Mobile Edge Networks: A Comprehensive Survey
Lim, W. Y. B., Luong, N. C., Hoang, D. T., Jiao, Y., Liang, Y.-C., Yang, Q., Niyato, D., & Miao, C. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2986024 · Cited by 2,565 · see citing works
Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing
Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2918951 · Cited by 2,455 · see citing works
“[47] propose to reduce communication cost with two new update schemes, namely, structured update and sketched update.”
BackgroundFederated Learning With Differential Privacy: Algorithms and Performance Analysis
Wei, K., Li, J., Ding, M., Ma, C., Yang, H. H., Farokhi, F., Jin, S., Quek, T. Q. S., & Vincent Poor, H. (2020) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2020.2988575 · Cited by 2,400 · see citing works
Adaptive Federated Learning in Resource Constrained Edge Computing Systems
Wang, S., Tuor, T., Salonidis, T., Leung, K. K., Makaya, C., He, T., & Chan, K. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2904348 · Cited by 2,232 · see citing works
Robust and Communication-Efficient Federated Learning From Non-i.i.d. Data
Sattler, F., Wiedemann, S., Muller, K.-R., & Samek, W. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2944481 · Cited by 1,725 · see citing works
Nasr, M., Shokri, R., & Houmansadr, A. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00065 · Cited by 1,760 · see citing works
On the Convergence of FedAvg on Non-IID Data
Li, X., Huang, K., Yang, W., Wang, S., & Zhang, Z. (2019) · arXiv · doi:10.48550/arxiv.1907.02189 · Cited by 1,575 · see citing works
Federated Learning for Healthcare Informatics
Xu, J., Glicksberg, B. S., Su, C., Walker, P., Bian, J., & Wang, F. (2021) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-020-00082-4 · Cited by 1,542 · see citing works
“, sparse, low-rank [58], or more specifically, pruning the least useful connections in a network [37, 113], weight quantization [17, 89], and model distillation [43].”
BackgroundMethodZhu, L., & Han, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-63076-8_2 · Cited by 1,906 · see citing works
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498 · see citing works
“gation, combined operations of subsampling, probabilistic quantization, and structured random rotations are performed to compress the full updates [203].”
BackgroundMethodA review of applications in federated learning
Li, L., Fan, Y., Tse, M., & Lin, K.-Y. (2020) · Computers & Industrial Engineering · doi:10.1016/j.cie.2020.106854 · Cited by 1,559 · see citing works
Federated Learning for Internet of Things: A Comprehensive Survey
Nguyen, D. C., Ding, M., Pathirana, P. N., Seneviratne, A., Li, J., & Vincent Poor, H. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3075439 · Cited by 1,503 · see citing works
Secure, privacy-preserving and federated machine learning in medical imaging
Kaissis, G. A., Makowski, M. R., Rückert, D., & Braren, R. F. (2020) · Nature Machine Intelligence · doi:10.1038/s42256-020-0186-1 · Cited by 1,501 · see citing works
How to backdoor federated learning
Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., & Shmatikov, V. (2018) · arXiv · doi:10.48550/arxiv.1807.00459 · Cited by 1,495 · see citing works
A survey on security and privacy of federated learning
Mothukuri, V., Parizi, R. M., Pouriyeh, S., Huang, Y., Dehghantanha, A., & Srivastava, G. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2020.10.007 · Cited by 1,480 · see citing works
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
Li, Q., Wen, Z., Wu, Z., Hu, S., Wang, N., Li, Y., Liu, X., & He, B. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3124599 · Cited by 1,407 · see citing works
Sattler, F., Muller, K.-R., & Samek, W. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3015958 · Cited by 1,355 · see citing works
FLTrust: Byzantine-robust Federated Learning via Trust Bootstrapping
Cao, X., Fang, M., Liu, J., & Gong, N. Z. (2021) · Proceedings 2021 Network and Distributed System Security Symposium · doi:10.14722/ndss.2021.24434 · Cited by 999 · see citing works
Swarm Learning for decentralized and confidential clinical machine learning
Warnat-Herresthal, S., Schultze, H., Shastry, K. L., Manamohan, S., Mukherjee, S., Garg, V., Sarveswara, R., Händler, K., Pickkers, P., Aziz, N. A., et al. (2021) · Nature · doi:10.1038/s41586-021-03583-3 · Cited by 884 · see citing works
Abdulrahman, S., Tout, H., Ould-Slimane, H., Mourad, A., Talhi, C., & Guizani, M. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3030072 · Cited by 796 · see citing works
Zhang, J., & Tao, D. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3039359 · Cited by 774 · see citing works
A Survey on Federated Learning for Resource-Constrained IoT Devices
Imteaj, A., Thakker, U., Wang, S., Li, J., & Amini, M. H. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3095077 · Cited by 774 · see citing works
Blockchain Empowered Asynchronous Federated Learning for Secure Data Sharing in Internet of Vehicles
Lu, Y., Huang, X., Zhang, K., Maharjan, S., & Zhang, Y. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.2973651 · Cited by 733 · see citing works
Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications
Aledhari, M., Razzak, R., Parizi, R. M., & Saeed, F. (2020) · IEEE Access · doi:10.1109/access.2020.3013541 · Cited by 733 · see citing works
Algorithmic fairness in artificial intelligence for medicine and healthcare
Chen, R. J., Wang, J. J., Williamson, D. F. K., Chen, T. Y., Lipkova, J., Lu, M. Y., Sahai, S., & Mahmood, F. (2023) · Nature Biomedical Engineering · doi:10.1038/s41551-023-01056-8 · Cited by 750 · see citing works
Privacy-Preserving Traffic Flow Prediction: A Federated Learning Approach
Liu, Y., Yu, J. J. Q., Kang, J., Niyato, D., & Zhang, S. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2991401 · Cited by 712 · see citing works
Scheduling Policies for Federated Learning in Wireless Networks
Yang, H. H., Liu, Z., Quek, T. Q. S., & Poor, H. V. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2944169 · Cited by 640 · see citing works
Federated learning meets blockchain in edge computing: Opportunities and challenges
Nguyen, D. C., Ding, M., Pham, Q.-V., Pathirana, P. N., Le, L. B., Seneviratne, A., Li, J., Niyato, D., & Poor, H. V. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3072611 · Cited by 661 · see citing works
Federated Learning Over Wireless Fading Channels
Amiri, M. M., & Gunduz, D. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2974748 · Cited by 625 · see citing works
A Blockchain-Based Decentralized Federated Learning Framework with Committee Consensus
Li, Y., Chen, C., Liu, N., Huang, H., Zheng, Z., & Yan, Q. (2021) · IEEE Network · doi:10.1109/mnet.011.2000263 · Cited by 645 · see citing works
Decentralized federated learning: Fundamentals, state of the art, frameworks, trends, and challenges
Martínez Beltrán, E. T., Pérez, M. Q., Sánchez, P. M. S., Bernal, S. L., Bovet, G., Pérez, M. G., Pérez, G. M., & Celdrán, A. H. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3315746 · Cited by 625 · see citing works
“Federation Models KPIs Evaluation of the federated models of each node of the network considering their performance and trustworthiness [16], [89]–[93]”
MethodDÏoT: A Federated Self-learning Anomaly Detection System for IoT
Nguyen, T. D., Marchal, S., Miettinen, M., Fereidooni, H., Asokan, N., & Sadeghi, A.-R. (2019) · 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2019.00080 · Cited by 663 · see citing works
Privacy-Preserving Blockchain-Based Federated Learning for IoT Devices
Zhao, Y., Zhao, J., Jiang, L., Tan, R., Niyato, D., Li, Z., Lyu, L., & Liu, Y. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3017377 · Cited by 613 · see citing works
FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantization
Reisizadeh, A., Mokhtari, A., Hassani, H., Jadbabaie, A., & Pedarsani, R. (2019) · arXiv · doi:10.48550/arxiv.1909.13014 · Cited by 587 · see citing works
Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
Mohammadi Amiri, M., & Gunduz, D. (2020) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2020.2981904 · Cited by 548 · see citing works
SecureBoost: A Lossless Federated Learning Framework
Cheng, K., Fan, T., Jin, Y., Liu, Y., Chen, T., Papadopoulos, D., & Yang, Q. (2021) · IEEE Intelligent Systems · doi:10.1109/mis.2021.3082561 · Cited by 669 · see citing works
Communication-efficient edge ai: algorithms and systems
Shi, Y., Yang, K., Jiang, T., Zhang, J., & Letaief, K. B. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.3007787 · Cited by 488 · see citing works
Distributed Federated Learning for Ultra-Reliable Low-Latency Vehicular Communications
Samarakoon, S., Bennis, M., Saad, W., & Debbah, M. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2956472 · Cited by 459 · see citing works
Federated learning for 6G communications: Challenges, methods, and future directions
Liu, Y., Yuan, X., Xiong, Z., Kang, J., Wang, X., & Niyato, D. (2020) · China Communications · doi:10.23919/jcc.2020.09.009 · Cited by 464 · see citing works
Federated Learning for intrusion detection system: Concepts, challenges and future directions
Agrawal, S., Sarkar, S., Aouedi, O., Yenduri, G., Piamrat, K., Alazab, M., Bhattacharya, S., Maddikunta, P. K. R., & Gadekallu, T. R. (2022) · Computer Communications · doi:10.1016/j.comcom.2022.09.012 · Cited by 457 · see citing works
PRADA: Protecting Against DNN Model Stealing Attacks
Juuti, M., Szyller, S., Marchal, S., & Asokan, N. (2019) · 2019 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2019.00044 · Cited by 440 · see citing works
“Increasing availability of lightweight hardware enhanced for DNNs [22] and the rise of federated learning will push machine learning computation to the edge [24], [46].”
BackgroundCommunication-Efficient Federated Learning for Wireless Edge Intelligence in IoT
Mills, J., Hu, J., & Min, G. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2956615 · Cited by 436 · see citing works
Federated Learning With Cooperating Devices: A Consensus Approach for Massive IoT Networks
Savazzi, S., Nicoli, M., & Rampa, V. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2964162 · Cited by 431 · see citing works
Privacy Preserving Machine Learning with Homomorphic Encryption and Federated Learning
Fang, H., & Qian, Q. (2021) · Future Internet · doi:10.3390/fi13040094 · Cited by 424 · see citing works
Efficient Acceleration of Deep Learning Inference on Resource-Constrained Edge Devices: A Review
Shuvo, M. M. H., Islam, S. K., Cheng, J., & Morshed, B. I. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2022.3226481 · Cited by 436 · see citing works
“There are research efforts to minimize the number of update rounds in federated learning to ensure communication efficiency [359], [363], [364].”
BackgroundWang, X., Wang, C., Li, X., Leung, V. C. M., & Taleb, T. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2986803 · Cited by 402 · see citing works
Attack of the tails: Yes, you really can backdoor federated learning
Wang, H., Sreenivasan, K., Rajput, S., Vishwakarma, H., Agarwal, S., Sohn, J.-Y., Lee, K., & Papailiopoulos, D. (2020) · arXiv · doi:10.48550/arxiv.2007.05084 · Cited by 406 · see citing works
Lu, Y., Huang, X., Dai, Y., Maharjan, S., & Zhang, Y. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2942179 · Cited by 387 · see citing works
Rahman, A., Hossain, M. S., Muhammad, G., Kundu, D., Debnath, T., Rahman, M., Khan, M. S. I., Tiwari, P., & Band, S. S. (2023) · Cluster Computing · doi:10.1007/s10586-022-03658-4 · Cited by 399 · see citing works
FedHome: Cloud-Edge Based Personalized Federated Learning for In-Home Health Monitoring
Wu, Q., Chen, X., Zhou, Z., & Zhang, J. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.3045266 · Cited by 392 · see citing works
Privacy preservation in federated learning: An insightful survey from the GDPR perspective
Truong, N., Sun, K., Wang, S., Guitton, F., & Guo, Y. (2021) · Computers & Security · doi:10.1016/j.cose.2021.102402 · Cited by 399 · see citing works
Improving Federated Learning Personalization via Model Agnostic Meta Learning
Jiang, Y., Konečný, J., Rush, K., & Kannan, S. (2019) · arXiv · doi:10.48550/arxiv.1909.12488 · Cited by 381 · see citing works
“Successful algorithmic extensions to the central idea include training a differential private model (McMahan et al., 2018), compression (Koneˇcn´y et al., 2016b; Caldas et al., 2018a), secure aggregation (Bonawitz et al., 2017), and a smaller number of always-participating nodes (Yang et al., 2019).”
MethodSelf-citationAccelerating Federated Learning via Momentum Gradient Descent
Liu, W., Chen, L., Chen, Y., & Zhang, W. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.2975189 · Cited by 374 · see citing works
Biscotti: A blockchain system for private and secure federated learning
Shayan, M., Fung, C., Yoon, C. J. M., & Beschastnikh, I. (2021) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.3044223 · Cited by 354 · see citing works
Privacy-preserving blockchain-based federated learning for traffic flow prediction
Qi, Y., Hossain, M. S., Nie, J., & Li, X. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2020.12.003 · Cited by 339 · see citing works
A Crowdsourcing Framework for On-Device Federated Learning
Pandey, S. R., Tran, N. H., Bennis, M., Tun, Y. K., Manzoor, A., & Hong, C. S. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.2971981 · Cited by 318 · see citing works
Chained Anomaly Detection Models for Federated Learning: An Intrusion Detection Case Study
Preuveneers, D., Rimmer, V., Tsingenopoulos, I., Spooren, J., Joosen, W., & Ilie-Zudor, E. (2018) · Applied Sciences · doi:10.3390/app8122663 · Cited by 323 · see citing works
The future of sleep health: a data-driven revolution in sleep science and medicine
Perez-Pozuelo, I., Zhai, B., Palotti, J., Mall, R., Aupetit, M., Garcia-Gomez, J. M., Taheri, S., Guan, Y., & Fernandez-Luque, L. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0244-4 · Cited by 311 · see citing works
Turbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning
So, J., Guler, B., & Avestimehr, A. S. (2021) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2021.3054610 · Cited by 318 · see citing works
Byzantine-Resilient Secure Federated Learning
So, J., Guler, B., & Avestimehr, A. S. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3041404 · Cited by 313 · see citing works
Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
Caldas, S., Konečny, J., McMahan, H. B., & Talwalkar, A. (2018) · arXiv · doi:10.48550/arxiv.1812.07210 · Cited by 312 · see citing works
Self-citationRodríguez-Barroso, N., Jiménez-López, D., Luzón, M. V., Herrera, F., & Martínez-Cámara, E. (2023) · Information Fusion · doi:10.1016/j.inffus.2022.09.011 · Cited by 296 · see citing works
Federated Learning for Keyword Spotting
Leroy, D., Coucke, A., Lavril, T., Gisselbrecht, T., & Dureau, J. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683546 · Cited by 291 · see citing works
Zhang, X., Peng, M., Yan, S., & Sun, Y. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2962715 · Cited by 282 · see citing works
UVeQFed: Universal Vector Quantization for Federated Learning
Shlezinger, N., Chen, M., Eldar, Y. C., Poor, H. V., & Cui, S. (2021) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2020.3046971 · Cited by 275 · see citing works
Federated learning for computational pathology on gigapixel whole slide images
Lu, M. Y., Chen, R. J., Kong, D., Lipkova, J., Singh, R., Williamson, D. F. K., Chen, T. Y., & Mahmood, F. (2022) · Medical Image Analysis · doi:10.1016/j.media.2021.102298 · Cited by 267 · see citing works
Deep learning for edge computing applications: A State-of-the-Art survey
Wang, F., Zhang, M., Wang, X., Ma, X., & Liu, J. (2020) · IEEE Access · doi:10.1109/access.2020.2982411 · Cited by 259 · see citing works
Shejwalkar, V., Houmansadr, A., Kairouz, P., & Ramage, D. (2022) · 2022 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp46214.2022.9833647 · Cited by 292 · see citing works
Vulnerabilities in federated learning
Bouacida, N., & Mohapatra, P. (2021) · IEEE Access · doi:10.1109/access.2021.3075203 · Cited by 248 · see citing works
Sparse Binary Compression: Towards Distributed Deep Learning with minimal Communication
Sattler, F., Wiedemann, S., Muller, K.-R., & Samek, W. (2019) · 2019 International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn.2019.8852172 · Cited by 240 · see citing works
Swarm learning for decentralized artificial intelligence in cancer histopathology
Saldanha, O. L., Quirke, P., West, N. P., James, J. A., Loughrey, M. B., Grabsch, H. I., Salto-Tellez, M., Alwers, E., Cifci, D., Ghaffari Laleh, N., et al. (2022) · Nature Medicine · doi:10.1038/s41591-022-01768-5 · Cited by 227 · see citing works
When Edge Meets Learning: Adaptive Control for Resource-Constrained Distributed Machine Learning
Wang, S., Tuor, T., Salonidis, T., Leung, K. K., Makaya, C., He, T., & Chan, K. (2018) · IEEE INFOCOM 2018 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2018.8486403 · Cited by 209 · see citing works
FedEraser: Enabling Efficient Client-Level Data Removal from Federated Learning Models
Liu, G., Ma, X., Yang, Y., Wang, C., & Liu, J. (2021) · 2021 IEEE/ACM 29th International Symposium on Quality of Service (IWQOS) · doi:10.1109/iwqos52092.2021.9521274 · Cited by 248 · see citing works
Federated learning for secure IoMT-applications in smart healthcare systems: A comprehensive review
Rani, S., Kataria, A., Kumar, S., & Tiwari, P. (2023) · Knowledge-Based Systems · doi:10.1016/j.knosys.2023.110658 · Cited by 228 · see citing works
Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Sampling
Luo, B., Xiao, W., Wang, S., Huang, J., & Tassiulas, L. (2022) · IEEE INFOCOM 2022 - IEEE Conference on Computer Communications · doi:10.1109/infocom48880.2022.9796935 · Cited by 229 · see citing works
Fed-IIoT: A Robust Federated Malware Detection Architecture in Industrial IoT
Taheri, R., Shojafar, M., Alazab, M., & Tafazolli, R. (2021) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2020.3043458 · Cited by 218 · see citing works
Duan, M., Liu, D., Chen, X., Tan, Y., Ren, J., Qiao, L., & Liang, L. (2019) · 2019 IEEE 37th International Conference on Computer Design (ICCD) · doi:10.1109/iccd46524.2019.00038 · Cited by 250 · see citing works
Design and Analysis of Uplink and Downlink Communications for Federated Learning
Zheng, S., Shen, C., & Chen, X. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3041388 · Cited by 191 · see citing works
More Than Privacy: Applying Differential Privacy in Key Areas of Artificial Intelligence
Zhu, T., Ye, D., Wang, W., Zhou, W., & Yu, P. S. (2022) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.3014246 · Cited by 219 · see citing works
Decentralized Federated Learning With Unreliable Communications
Ye, H., Liang, L., & Li, G. Y. (2022) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2022.3152445 · Cited by 184 · see citing works
Ali, M., Karimipour, H., & Tariq, M. (2021) · Computers & Security · doi:10.1016/j.cose.2021.102355 · Cited by 177 · see citing works
Myrzashova, R., Alsamhi, S. H., Shvetsov, A. V., Hawbani, A., & Wei, X. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3263598 · Cited by 178 · see citing works
Deep Model Poisoning Attack on Federated Learning
Zhou, X., Xu, M., Wu, Y., & Zheng, N. (2021) · Future Internet · doi:10.3390/fi13030073 · Cited by 170 · see citing works
Handling privacy-sensitive medical data with federated learning: Challenges and future directions
Aouedi, O., Sacco, A., Piamrat, K., & Marchetto, G. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3185673 · Cited by 165 · see citing works
POSEIDON: Privacy-Preserving Federated Neural Network Learning
Sav, S., Pyrgelis, A., Troncoso-Pastoriza, J. R., Froelicher, D., Bossuat, J.-P., Sousa, J. S., & Hubaux, J.-P. (2021) · Proceedings 2021 Network and Distributed System Security Symposium · doi:10.14722/ndss.2021.24119 · Cited by 162 · see citing works
Human Activity Recognition Using Federated Learning
Sozinov, K., Vlassov, V., & Girdzijauskas, S. (2018) · 2018 IEEE Intl Conf on Parallel & Distributed Processing with Applications, Ubiquitous Computing & Communications, Big Data & Cloud Computing, Social Computing & Networking, Sustainable Computing & Communications (ISPA/IUCC/BDCloud/SocialCom/SustainCom) · doi:10.1109/bdcloud.2018.00164 · Cited by 167 · see citing works
Applications of Federated Learning; Taxonomy, Challenges, and Research Trends
Shaheen, M., Farooq, M. S., Umer, T., & Kim, B.-S. (2022) · Electronics · doi:10.3390/electronics11040670 · Cited by 157 · see citing works
Local and Central Differential Privacy for Robustness and Privacy in Federated Learning
Naseri, M., Hayes, J., & De Cristofaro, E. (2022) · Proceedings 2022 Network and Distributed System Security Symposium · doi:10.14722/ndss.2022.23054 · Cited by 188 · see citing works
Federated Unlearning via Class-Discriminative Pruning
Wang, J., Guo, S., Xie, X., & Qi, H. (2022) · Proceedings of the ACM Web Conference 2022 · doi:10.1145/3485447.3512222 · Cited by 157 · see citing works
Rodríguez-Barroso, N., Stipcich, G., Jiménez-López, D., Ruiz-Millán, J. A., Martínez-Cámara, E., González-Seco, G., Luzón, M. V., Veganzones, M. A., & Herrera, F. (2020) · Information Fusion · doi:10.1016/j.inffus.2020.07.009 · Cited by 148 · see citing works
Federated learning-based cognitive detection of jamming attack in flying Ad-Hoc network
Mowla, N. I., Tran, N. H., Doh, I., & Chae, K. (2020) · IEEE Access · doi:10.1109/access.2019.2962873 · Cited by 147 · see citing works
Detailed comparison of communication efficiency of split learning and federated learning
Singh, A., Vepakomma, P., Gupta, O., & Raskar, R. (2019) · arXiv · doi:10.48550/arxiv.1909.09145 · Cited by 148 · see citing works
Challenges, Applications and Design Aspects of Federated Learning: A Survey
Rahman, K. M. J., Ahmed, F., Akhter, N., Hasan, M., Amin, R., Aziz, K. E., Islam, A. K. M. M., Mukta, M. S. H., & Islam, A. K. M. N. (2021) · IEEE Access · doi:10.1109/access.2021.3111118 · Cited by 147 · see citing works
Wan, Y., Qu, Y., Ni, W., Xiang, Y., Gao, L., & Hossain, E. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3361451 · Cited by 152 · see citing works
“First proposed by Google [18], FL is a machine learning paradigm in which multiple devices form a distributed network while maintaining the same machine learning model.”
BackgroundShort-Term Traffic Flow Prediction Based on Graph Convolutional Networks and Federated Learning
Xia, M., Jin, D., & Chen, J. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2022.3179391 · Cited by 142 · see citing works
- Authors
- Konečný, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D.
- Affiliations
- University of Edinburgh (GB)
- King Abdullah University of Science and Technology (SA)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2016
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1610.05492
- OpenAlex
- W2535838896
- Cited by
- 3,766
- References
- 0
- Citations by year
- 2026: 19 · 2025: 237 · 2024: 653 · 2023: 677 · 2022: 613 · 2021: 733 · 2020: 562 · 2019: 223 · 2018: 40 · 2017: 7 · 2016: 2
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1351108, CAREER)
- Office of Naval Research (N00014-17-1, N00014, N00014-17)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- federated learning, communication efficiency, low-rank updates, random mask, sketching, quantization, random rotation, subsampling, mobile devices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
