A Survey on Federated Learning for Resource-Constrained IoT Devices
The DOI matches its Crossref record; 224 of the 224 references it cites were found in a registry or an index.
Abstract
Federated learning (FL) is a distributed machine learning strategy that generates a global model by learning from multiple decentralized edge clients. FL enables on-device training, keeping the client’s local data private, and further, updating the global model based on the local model updates. While FL methods offer several advantages, including scalability and data privacy, they assume there are available computational resources at each edge-device/client. However, the Internet-of-Things (IoT)-enabled devices, e.g., robots, drone swarms, and low-cost computing devices (e.g., Raspberry Pi), may have limited processing ability, low bandwidth and power, or limited storage capacity. In this survey article, we propose to answer this question: how to train distributed machine learning models for resource-constrained IoT devices? To this end, we first explore the existing studies on FL, relative assumptions for distributed implementation using IoT devices, and explore their drawbacks. We then discuss the implementation challenges and issues when applying FL to an IoT environment. We highlight an overview of FL and provide a comprehensive survey of the problem statements and emerging challenges, particularly during applying FL within heterogeneous IoT environments. Finally, we point out the future research directions for scientists and researchers who are interested in working at the intersection of FL and resource-constrained IoT environments.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Pan, S. J., & Yang, Q. (2010) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2009.191 · Cited by 23,387 · cited 1×
“SecureBoost [25] and FedBCD [26] are some of the examples of vertical FL. c) Federated transfer learning: Federated transfer learning (FTL) [27] can be considered as the combination of both horizontal and vertical partitioning of data (see Fig.”
Advances in neural information processing systems 7
(1996) · Computers & Mathematics with Applications · doi:10.1016/s0898-1221(96)90463-0 · Cited by 14,281
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215 · cited 10×
“For instance, the key idea of [20] was to allow for more computation on the mobile-edge side, e.g., by conducting more local updates before interacting with the server.”
InfluentialAdvances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,887
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772
Caruana, R. (1997) · Machine Learning · doi:10.1023/a:1007379606734 · Cited by 6,636 · cited 1×
“Several ML works focused on the designing of statistical heterogeneity via meta-learning [166], [167], multitask learning [168], [169], which are further extended to FL settings [24], [48], [170]–[173].”
Federated Machine Learning: Concept and Applications
Yang, Q., Liu, Y., Chen, T., & Tong, Y. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3298981 · Cited by 6,322
Federated learning: Challenges, methods, and future directions
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2975749 · Cited by 6,210 · cited 5×
“FL survey papers [7], [7]–[12] are mostly focused on FL settings, system design, and components, implementation challenges, or on recent advancements.”
InfluentialA Survey on Mobile Edge Computing: The Communication Perspective
Mao, Y., You, C., Zhang, J., Huang, K., & Letaief, K. B. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2745201 · Cited by 5,607
(2013) · Cited by 5,372
Federated Optimization in Heterogeneous Networks
Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., & Smith, V. (2018) · arXiv · doi:10.48550/arxiv.1812.06127 · Cited by 4,023 · cited 4×
“Li et al. [21] proposed an algorithm named FedProx that is particularly useful for the resource-constrained FL-based IoT environment.”
InfluentialPractical Secure Aggregation for Privacy-Preserving Machine Learning
Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., Ramage, D., Segal, A., & Seth, K. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3133982 · Cited by 3,956 · cited 1×
“Alternatively, such decisions can be made by a leader client, through a collaborative consensus scheme [56], [59], [83].”
Federated Learning: Strategies for Improving Communication Efficiency
Konečný, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D. (2016) · arXiv · doi:10.48550/arxiv.1610.05492 · Cited by 3,766 · cited 4×
“McMahan et al. [20], Koneˇcn`y et al. [33], [42], and Bonawitz et al. [73] focused on analyzing federated optimization that considers synchronous communication during training between the FL server and clients.”
InfluentialProtocols for secure computations
Yao, A. C. (1982) · 23rd Annual Symposium on Foundations of Computer Science (sfcs 1982) · doi:10.1109/sfcs.1982.38 · Cited by 3,306
Mobile Edge Computing: A Survey on Architecture and Computation Offloading
Mach, P., & Becvar, Z. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2682318 · Cited by 3,258
Eyeriss: An Energy-Efficient Reconfigurable Accelerator for Deep Convolutional Neural Networks
Chen, Y.-H., Krishna, T., Emer, J. S., & Sze, V. (2017) · IEEE Journal of Solid-State Circuits · doi:10.1109/jssc.2016.2616357 · Cited by 3,247
Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures
Fredrikson, M., Jha, S., & Ristenpart, T. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813677 · Cited by 2,961 · cited 1×
“However, it is possible to leak sensitive information [58], [127]–[129] through sharing model update during the training process.”
Advances in Neural Information Processing Systems 20
Bonilla, E. V., Chai, K. M. A., & Williams, C. K. I. (2008) · neural information processing systems · Cited by 2,846
Decentralizing Privacy: Using Blockchain to Protect Personal Data
Zyskind, G., Nathan, O., & Pentland, A. '. (2015) · 2015 IEEE Security and Privacy Workshops · doi:10.1109/spw.2015.27 · Cited by 2,617 · cited 1×
“Some incentives or regulations schemes are implemented in blockchain [217]–[219], and some incentive mechanisms are proposed for high-quality federated data [220], [221].”
Federated Learning in Mobile Edge Networks: A Comprehensive Survey
Lim, W. Y. B., Luong, N. C., Hoang, D. T., Jiao, Y., Liang, Y.-C., Yang, Q., Niyato, D., & Miao, C. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2986024 · Cited by 2,565 · cited 1×
“Besides, a federated optimization-based framework is proposed in [10], which is constructed by addressing challenges related to both system and statistical heterogeneity.”
Mobile Edge Computing: A Survey
Abbas, N., Zhang, Y., Taherkordi, A., & Skeie, T. (2018) · IEEE Internet of Things Journal · doi:10.1109/jiot.2017.2750180 · Cited by 2,474
Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing
Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2918951 · Cited by 2,455 · cited 1×
“On the other hand, edge computing surveys [13]– [19] are mainly conducted on edge computing infrastructure, applications including ML and AI, resource management, wireless communication, and security and privacy issues.”
Federated Learning With Differential Privacy: Algorithms and Performance Analysis
Wei, K., Li, J., Ding, M., Ma, C., Yang, H. H., Farokhi, F., Jin, S., Quek, T. Q. S., & Vincent Poor, H. (2020) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2020.2988575 · Cited by 2,400
Self-citationTowards Federated Learning at Scale: System Design
Bonawitz, K., Eichner, H., Grieskamp, W., Huba, D., Ingerman, A., Ivanov, V., Kiddon, C., Konečný, J., Mazzocchi, S., McMahan, H. B., et al. (2019) · arXiv · doi:10.48550/arxiv.1902.01046 · Cited by 2,334 · cited 5×
“It is possible that some participants can generate local models utilizing their old data repeatedly while skipping new data [73].”
InfluentialAdaptive Federated Learning in Resource Constrained Edge Computing Systems
Wang, S., Tuor, T., Salonidis, T., Leung, K. K., Makaya, C., He, T., & Chan, K. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2904348 · Cited by 2,232 · cited 3×
“Wang et al. [67], Das and Brunschwiler [92], Jiang et al. [93], and Xu et al. [94] analyzed hardware limitation challenges in the implementation of FL by considering Raspberry Pi and other types of resource-limited clients.”
Self-citationFederated Learning with Non-IID Data
Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., & Chandra, V. (2018) · arXiv · doi:10.48550/arxiv.1806.00582 · Cited by 2,077 · cited 2×
“Besides, Eichner et al. [172] aimed to identify cyclic patterns within data samples, while a detailed analysis of transfer learning strategy for personalization during FL can be seen in [170].”
Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge
Nishio, T., & Yonetani, R. (2019) · ICC 2019 - 2019 IEEE International Conference on Communications (ICC) · doi:10.1109/icc.2019.8761315 · Cited by 1,875 · cited 1×
“Nishio and Yonetani [66] design a client selection protocol, using FL edge server.”
Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge
Nishio, T., & Yonetani, R. (2018) · arXiv · doi:10.48550/arxiv.1804.08333 · Cited by 1,875 · cited 1×
“Nishio and Yonetani [66] design a client selection protocol, using FL edge server.”
Nasr, M., Shokri, R., & Houmansadr, A. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00065 · Cited by 1,760
Exploiting Unintended Feature Leakage in Collaborative Learning
Melis, L., Song, C., De Cristofaro, E., & Shmatikov, V. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00029 · Cited by 1,734 · cited 1×
“However, it is possible to leak sensitive information [58], [127]–[129] through sharing model update during the training process.”
Robust and Communication-Efficient Federated Learning From Non-i.i.d. Data
Sattler, F., Wiedemann, S., Muller, K.-R., & Samek, W. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2944481 · Cited by 1,725
Learning IoT in Edge: Deep Learning for the Internet of Things with Edge Computing
Li, H., Ota, K., & Dong, M. (2018) · IEEE Network · doi:10.1109/mnet.2018.1700202 · Cited by 1,566
A Joint Learning and Communications Framework for Federated Learning Over Wireless Networks
Chen, M., Yang, Z., Saad, W., Yin, C., Poor, H. V., & Cui, S. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3024629 · Cited by 1,547 · cited 2×
“Chen et al. [123] also designed a framework of joint-learning by considering the effect of wireless factors on participants in the FL scenario.”
A Joint Learning and Communications Framework for Federated Learning over Wireless Networks
Chen, M., Yang, Z., Saad, W., Yin, C., Poor, H. V., & Cui, S. (2019) · arXiv · doi:10.48550/arxiv.1909.07972 · Cited by 1,547 · cited 2×
“Chen et al. [123] also designed a framework of joint-learning by considering the effect of wireless factors on participants in the FL scenario.”
How to backdoor federated learning
Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., & Shmatikov, V. (2018) · arXiv · doi:10.48550/arxiv.1807.00459 · Cited by 1,495 · cited 1×
“Bagdasaryan et al. [60] proposed a backdoor FL to identify malicious attacks during federated aggregation.”
Regularized multi--task learning
Evgeniou, T., & Pontil, M. (2004) · Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1014052.1014067 · Cited by 1,433 · cited 1×
“Several ML works focused on the designing of statistical heterogeneity via meta-learning [166], [167], multitask learning [168], [169], which are further extended to FL settings [24], [48], [170]–[173].”
The dining cryptographers problem: Unconditional sender and recipient untraceability
Chaum, D. (1988) · Journal of Cryptology · doi:10.1007/bf00206326 · Cited by 1,408
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
Li, Q., Wen, Z., Wu, Z., Hu, S., Wang, N., Li, Y., Liu, X., & He, B. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3124599 · Cited by 1,407
DianNao: a small-footprint high-throughput accelerator for ubiquitous machine-learning
Chen, T., Du, Z., Sun, N., Wang, J., Wu, C., Chen, Y., & Temam, O. (2014) · Proceedings of the 19th international conference on Architectural support for programming languages and operating systems · doi:10.1145/2541940.2541967 · Cited by 1,297
Blockchain and Federated Learning for Privacy-Preserved Data Sharing in Industrial IoT
Lu, Y., Huang, X., Dai, Y., Maharjan, S., & Zhang, Y. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2942190 · Cited by 1,245
Energy Efficient Federated Learning Over Wireless Communication Networks
Yang, Z., Chen, M., Saad, W., Hong, C. S., & Shikh-Bahaei, M. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3037554 · Cited by 1,153
Federated Learning for Mobile Keyboard Prediction
Hard, A., Rao, K., Mathews, R., Ramaswamy, S., Beaufays, F., Augenstein, S., Eichner, H., Kiddon, C., & Ramage, D. (2018) · arXiv · doi:10.48550/arxiv.1811.03604 · Cited by 1,148 · cited 4×
“The most popular ML model that is used in FL is federated stochastic gradient descent (Fed-SGD) coupled with the neural network (NN), e.g., image classification [20] and word prediction [22], [29].”
InfluentialData Privacy through Optimal k-Anonymization
Bayardo, R. J., & Agrawal, R. (2005) · 21st International Conference on Data Engineering (ICDE'05) · doi:10.1109/icde.2005.42 · Cited by 1,131
A perspective view and survey of meta-learning
Vilalta, R., & Drissi, Y. (2002) · Artificial Intelligence Review · doi:10.1023/a:1019956318069 · Cited by 1,108 · cited 1×
“Several ML works focused on the designing of statistical heterogeneity via meta-learning [166], [167], multitask learning [168], [169], which are further extended to FL settings [24], [48], [170]–[173].”
Federated Learning via Over-the-Air Computation
Yang, K., Jiang, T., Shi, Y., & Ding, Z. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2961673 · Cited by 1,084
Differentially Private Federated Learning: A Client Level Perspective
Geyer, R. C., Klein, T., & Nabi, M. (2017) · arXiv · doi:10.48550/arxiv.1712.07557 · Cited by 1,052 · cited 3×
“Though several privacy definitions for FL are stated in [40], [56], [58], [128], and [133]–[138], we can classify them as global and local privacy.”
In-Edge AI: Intelligentizing Mobile Edge Computing, Caching and Communication by Federated Learning
Wang, X., Han, Y., Wang, C., Zhao, Q., Chen, X., & Chen, M. (2019) · IEEE Network · doi:10.1109/mnet.2019.1800286 · Cited by 1,030 · cited 2×
“Besides, Wang et al. [11] designed a framework to exchange learning parameters of the clients through the collaboration for generating better local training models.”
A Hybrid Approach to Privacy-Preserving Federated Learning
Truex, S., Baracaldo, N., Anwar, A., Steinke, T., Ludwig, H., Zhang, R., & Zhou, Y. (2019) · Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security · doi:10.1145/3338501.3357370 · Cited by 1,029
Federated Learning over Wireless Networks: Optimization Model Design and Analysis
Tran, N. H., Bao, W., Zomaya, A., Nguyen, M. N. H., & Hong, C. S. (2019) · IEEE INFOCOM 2019 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2019.8737464 · Cited by 1,013 · cited 1×
“A detailed discussion on the tradeoffs between FL training period and energy requirement cost can be seen in [114] and [156]–[158].”
DBA: Distributed Backdoor Attacks against Federated Learning
Xie, C., Huang, K., Chen, P., & Li, B. (2020) · International Conference on Learning Representations · Cited by 1,009
Kang, J., Xiong, Z., Niyato, D., Xie, S., & Zhang, J. (2019) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2940820 · Cited by 953
A Survey on Mobile Edge Networks: Convergence of Computing, Caching and Communications
Wang, S., Zhang, X., Zhang, Y., Wang, L., YANG, J., & Wang, W. (2017) · IEEE Access · doi:10.1109/access.2017.2685434 · Cited by 952
Self-citationOutrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., & Dean, J. (2017) · arXiv · doi:10.48550/arxiv.1701.06538 · Cited by 933 · cited 1×
“By gating experts that are deemed unimportant for the input, one can achieve faster computation [187].”
LEAF: A Benchmark for Federated Settings
Caldas, S., Duddu, S. M. K., Wu, P., Li, T., Konečný, J., McMahan, H. B., Smith, V., & Talwalkar, A. (2018) · arXiv · doi:10.48550/arxiv.1812.01097 · Cited by 894
Blockchained On-Device Federated Learning
Kim, H., Park, J., Bennis, M., & Kim, S.-L. (2020) · IEEE Communications Letters · doi:10.1109/lcomm.2019.2921755 · Cited by 887 · cited 2×
“Some blockchain paradigm for on-device training is discussed in [36] and [222], but they designed that the framework without considering the challenges of the weak-processing unit and limited memory of IoT devices.”
Blockchained On-Device Federated Learning
Kim, H., Park, J., Bennis, M., & Kim, S.-L. (2018) · arXiv · doi:10.48550/arxiv.1808.03949 · Cited by 882 · cited 2×
“Some blockchain paradigm for on-device training is discussed in [36] and [222], but they designed that the framework without considering the challenges of the weak-processing unit and limited memory of IoT devices.”
Beyond Inferring Class Representatives: User-Level Privacy Leakage From Federated Learning
Wang, Z., Song, M., Zhang, Z., Song, Y., Wang, Q., & Qi, H. (2019) · IEEE INFOCOM 2019 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2019.8737416 · Cited by 881
Learning Differentially Private Recurrent Language Models
McMahan, H. B., Ramage, D., Talwar, K., & Zhang, L. (2017) · arXiv · doi:10.48550/arxiv.1710.06963 · Cited by 849
Once-for-All: Train One Network and Specialize it for Efficient Deployment
Cai, H., Gan, C., Wang, T., Zhang, Z., & Han, S. (2019) · arXiv · doi:10.48550/arxiv.1908.09791 · Cited by 845 · cited 1×
“A lot of edge applications are now using DNN-based algorithms [102], [103] and there is an increasing focus on making DNN inference efficient on edge devices [104], [105].”
Federated Learning for Wireless Communications: Motivation, Opportunities, and Challenges
Niknam, S., Dhillon, H. S., & Reed, J. H. (2020) · IEEE Communications Magazine · doi:10.1109/mcom.001.1900461 · Cited by 820 · cited 1×
“Yang et al. [8] focused on the categorization of FL settings while Niknam et al. [9] presented the issues of FL in a wireless environment.”
Blockchain Empowered Asynchronous Federated Learning for Secure Data Sharing in Internet of Vehicles
Lu, Y., Huang, X., Zhang, K., Maharjan, S., & Zhang, Y. (2020) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2020.2973651 · Cited by 733 · cited 1×
“Xie et al. [96] proposed asynchronous federated optimization, [159] discussed on the asynchronous FL for geospatial applications, [160]–[162] proposed a DP-based asynchronous FL strategy for MEC, and [163] presented a blockchain-based secure data sharing strategy for asynchronous FL.”
Communication Efficient Distributed Machine Learning with the Parameter Server
Li, M., Andersen, D. G., Smola, A. J., & Yu, K. (2014) · neural information processing systems · Cited by 715
Analyzing Federated Learning through an Adversarial Lens
Bhagoji, A. N., Chakraborty, S., Mittal, P., & Calo, S. (2018) · arXiv · doi:10.48550/arxiv.1811.12470 · Cited by 703
Applied Federated Learning: Improving Google Keyboard Query Suggestions
Yang, T., Andrew, G., Eichner, H., Sun, H., Li, W., Kong, N., Ramage, D., & Beaufays, F. (2018) · arXiv · doi:10.48550/arxiv.1812.02903 · Cited by 684 · cited 4×
“Recently, Google has invented an FL-based keyboard suggestion [29] by training the model on-device of the user and aggregate the model information in the server.”
InfluentialSecureBoost: A Lossless Federated Learning Framework
Cheng, K., Fan, T., Jin, Y., Liu, Y., Chen, T., Papadopoulos, D., & Yang, Q. (2021) · IEEE Intelligent Systems · doi:10.1109/mis.2021.3082561 · Cited by 669 · cited 3×
“In tree-based FL, a model is generated for training single or multiple DTs. Cheng et al. [25] and Li et al. [30] designed gradient boosting DTs (GBDTs) by considering both horizontal and vertical partitioned data schemes.”
SecureBoost: A Lossless Federated Learning Framework
Cheng, K., Fan, T., Jin, Y., Liu, Y., Chen, T., Papadopoulos, D., & Yang, Q. (2019) · arXiv · doi:10.48550/arxiv.1901.08755 · Cited by 668 · cited 3×
“In tree-based FL, a model is generated for training single or multiple DTs. Cheng et al. [25] and Li et al. [30] designed gradient boosting DTs (GBDTs) by considering both horizontal and vertical partitioned data schemes.”
DÏoT: A Federated Self-learning Anomaly Detection System for IoT
Nguyen, T. D., Marchal, S., Miettinen, M., Fereidooni, H., Asokan, N., & Sadeghi, A.-R. (2019) · 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2019.00080 · Cited by 663 · cited 1×
“Anomaly Detection: An autonomous self-learning scheme is proposed in [215] to identify compromised devices within IoT networks.”
A secure federated transfer learning framework
Liu, Y., Kang, Y., Xing, C., Chen, T., & Yang, Q. (2020) · IEEE Intelligent Systems · doi:10.1109/mis.2020.2988525 · Cited by 645
Secure Federated Transfer Learning
Liu, Y., Kang, Y., Xing, C., Chen, T., & Yang, Q. (2018) · arXiv · doi:10.48550/arxiv.1812.03337 · Cited by 645 · cited 1×
“Liu et al. [28] designed a framework that can learn a feature representation of multiple parties based on common instances.”
SkipNet: Learning Dynamic Routing in Convolutional Networks
Wang, X., Yu, F., Dou, Z.-Y., Darrell, T., & Gonzalez, J. E. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01261-8_25 · Cited by 625
Privacy-Preserving Blockchain-Based Federated Learning for IoT Devices
Zhao, Y., Zhao, J., Jiang, L., Tan, R., Niyato, D., Li, Z., Lyu, L., & Liu, Y. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3017377 · Cited by 613
A Learning-Based Incentive Mechanism for Federated Learning
Zhan, Y., Li, P., Qu, Z., Zeng, D., & Guo, S. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2967772 · Cited by 602
Federated Machine Learning: Concept and Applications
Yang, Q., Liu, Y., Chen, T., & Tong, Y. (2019) · arXiv · doi:10.48550/arxiv.1902.04885 · Cited by 600
Model Pruning Enables Efficient Federated Learning on Edge Devices
Jiang, Y., Wang, S., Valls, V., Ko, B. J., Lee, W.-H., Leung, K. K., & Tassiulas, L. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3166101 · Cited by 589 · cited 1×
“Wang et al. [67], Das and Brunschwiler [92], Jiang et al. [93], and Xu et al. [94] analyzed hardware limitation challenges in the implementation of FL by considering Raspberry Pi and other types of resource-limited clients.”
Self-citationPrivacy-Preserving Federated Brain Tumour Segmentation
Li, W., Milletarì, F., Xu, D., Rieke, N., Hancox, J., Zhu, W., Baust, M., Cheng, Y., Ourselin, S., Cardoso, M. J., et al. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-32692-0_16 · Cited by 578
Chen, Y., Sun, X., & Jin, Y. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2953131 · Cited by 573 · cited 1×
“Another strategy is to perform asynchronous training, i.e., updating the global model whenever it receives a model update from any of the participated clients [67], [165].”
Efficient and Privacy-Enhanced Federated Learning for Industrial Artificial Intelligence
Hao, M., Li, H., Luo, X., Xu, G., Yang, H., & Liu, S. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2945367 · Cited by 569
Training Deep Nets with Sublinear Memory Cost
Chen, T., Xu, B., Zhang, C., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1604.06174 · Cited by 561 · cited 1×
“Training on edge devices can bene-fit by adapting techniques that have served well in managing this problem in the server context, such as efficient gradient checkpointing [108], tensor rematerialization [109], and recompute [110].”
Federated Optimization:Distributed Optimization Beyond the Datacenter
Konečný, J., McMahan, B., & Ramage, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.03575 · Cited by 544 · cited 1×
“McMahan et al. [20], Koneˇcn`y et al. [33], [42], and Bonawitz et al. [73] focused on analyzing federated optimization that considers synchronous communication during training between the FL server and clients.”
Liu, Y., Garg, S., Nie, J., Zhang, Y., Xiong, Z., Kang, J., & Hossain, M. S. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3011726 · Cited by 541
Federated Learning for Edge Networks: Resource Optimization and Incentive Mechanism
Khan, L. U., Pandey, S. R., Tran, N. H., Saad, W., Han, Z., Nguyen, M. N. H., & Hong, C. S. (2020) · IEEE Communications Magazine · doi:10.1109/mcom.001.1900649 · Cited by 506 · cited 1×
“Khan et al. [117] designed an incentive-based FL model via a Stackelberg game for motivating client participation in the learning process.”
Fair Resource Allocation in Federated Learning
Li, T., Sanjabi, M., Beirami, A., & Smith, V. (2019) · arXiv · doi:10.48550/arxiv.1905.10497 · Cited by 475 · cited 2×
“Li et al. [63] used a α -fairness metric and proposed a q -Fair FL to ensure fair accuracy distribution.”
Jeong, E., Oh, S., Kim, H., Park, J., Bennis, M., & Kim, S.-L. (2018) · arXiv · doi:10.48550/arxiv.1811.11479 · Cited by 465
Local Differential Privacy-Based Federated Learning for Internet of Things
Zhao, Y., Zhao, J., Yang, M., Wang, T., Wang, N., Lyu, L., Niyato, D., & Lam, K.-Y. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3037194 · Cited by 456 · cited 1×
“Geyer et al. [40], McMahan et al. [56], and Zhao et al. [200] applied DP to FL to achieve a global DP, but the hyperparameters of these approaches affect the communication and model accuracy.”
Decentralized Privacy Using Blockchain-Enabled Federated Learning in Fog Computing
Qu, Y., Gao, L., Luan, T. H., Xiang, Y., Yu, S., Li, B., & Zheng, G. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2977383 · Cited by 451 · cited 1×
“Furthermore, some prior works [202]–[205] proposed mechanisms to preserve privacy in a blockchain-enabled FL-based IoT environment.”
FedMD: Heterogenous Federated Learning via Model Distillation
Li, D., & Wang, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.03581 · Cited by 450 · cited 1×
“Li and Wang [175] proposed a centralized approach of handling heterogeneity of FL model training but did not consider specific support and analysis for statistical heterogeneity.”
Secure Federated Matrix Factorization
Chai, D., Wang, L., Chen, K., & Yang, Q. (2021) · IEEE Intelligent Systems · doi:10.1109/mis.2020.3014880 · Cited by 449
Privacy-Preserving Ridge Regression on Hundreds of Millions of Records
Nikolaenko, V., Weinsberg, U., Ioannidis, S., Joye, M., Boneh, D., & Taft, N. (2013) · 2013 IEEE Symposium on Security and Privacy · doi:10.1109/sp.2013.30 · Cited by 439 · cited 1×
“Different linear models (e.g., linear regression and classification, logistic regression, and support vector machine (SVM) [31], [32]) are convenient to handle.”
Asynchronous Decentralized Parallel Stochastic Gradient DescentPreprint
Lian, X., Zhang, W., Zhang, C., & Liu, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.06952 · Cited by 436 · cited 1×
“Xie et al. [96], Zheng et al. [97], and Lian et al. [98] analyzed asynchronous FL training with provable convergence by combining it with federated optimization.”
PDA: Privacy-Preserving Data Aggregation in Wireless Sensor Networks
He, W., Liu, X., Nguyen, H., Nahrstedt, K., & Abdelzaher, T. (2007) · IEEE INFOCOM 2007 - 26th IEEE International Conference on Computer Communications · doi:10.1109/infcom.2007.237 · Cited by 434 · cited 2×
“6) During the FL process, we may observe statistical heterogeneity among the client data.”
Hardy, S., Henecka, W., Ivey-Law, H., Nock, R., Patrini, G., Smith, G., & Thorne, B. (2017) · arXiv · doi:10.48550/arxiv.1711.10677 · Cited by 431 · cited 1×
“Different linear models (e.g., linear regression and classification, logistic regression, and support vector machine (SVM) [31], [32]) are convenient to handle.”
Communication-Efficient Distributed Optimization using an Approximate Newton-type Method
Shamir, O., Srebro, N., & Zhang, T. (2013) · arXiv · doi:10.48550/arxiv.1312.7853 · Cited by 424 · cited 2×
“Such a technique is useful in practice and distributed approximate Newton (DANE) [84], CoCoA [85], and DiSCO [86] are some of the examples of such distributed optimization techniques.”
Asynchronous Federated Optimization
Xie, C., Koyejo, S., & Gupta, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1903.03934 · Cited by 410 · cited 2×
“Xie et al. [96] proposed asynchronous federated optimization, [159] discussed on the asynchronous FL for geospatial applications, [160]–[162] proposed a DP-based asynchronous FL strategy for MEC, and [163] presented a blockchain-based secure data sharing strategy for asynchronous FL.”
Chameleon: A Hybrid Secure Computation Framework for Machine Learning Applications
Riazi, M. S., Weinert, C., Tkachenko, O., Songhori, E. M., Schneider, T., & Koushanfar, F. (2018) · Proceedings of the 2018 on Asia Conference on Computer and Communications Security · doi:10.1145/3196494.3196522 · Cited by 410 · cited 1×
“The protocol outcome enables multiple data owners to collaboratively agree to generate a function without leaking any information [130]–[132].”
TinyTL: Reduce Memory, Not Parameters for Efficient On-Device LearningPreprint
Cai, H., Gan, C., Zhu, L., & Han, S. (2020) · arXiv (Cornell University) · Cited by 404
Adaptive Gradient-Based Meta-Learning MethodsPreprint
Khodak, M., Balcan, M.-F., & Talwalkar, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.02717 · Cited by 402 · cited 1×
“Several ML works focused on the designing of statistical heterogeneity via meta-learning [166], [167], multitask learning [168], [169], which are further extended to FL settings [24], [48], [170]–[173].”
Lu, Y., Huang, X., Dai, Y., Maharjan, S., & Zhang, Y. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2942179 · Cited by 387 · cited 1×
“Xie et al. [96] proposed asynchronous federated optimization, [159] discussed on the asynchronous FL for geospatial applications, [160]–[162] proposed a DP-based asynchronous FL strategy for MEC, and [163] presented a blockchain-based secure data sharing strategy for asynchronous FL.”
Mitigating Sybils in Federated Learning Poisoning
Fung, C., Yoon, C. J. M., & Beschastnikh, I. (2018) · arXiv · doi:10.48550/arxiv.1808.04866 · Cited by 385 · cited 1×
“Fung et al. [176] explained the vulnerability of sybil attacks in the FL process.”
Improving Federated Learning Personalization via Model Agnostic Meta Learning
Jiang, Y., Konečný, J., Rush, K., & Kannan, S. (2019) · arXiv · doi:10.48550/arxiv.1909.12488 · Cited by 381
Federated Learning in Vehicular Edge Computing: A Selective Model Aggregation Approach
Ye, D., Yu, R., Pan, M., & Han, Z. (2020) · IEEE Access · doi:10.1109/access.2020.2968399 · Cited by 378 · cited 1×
“Ye et al. [125] proposed a selective client model aggregation-based FL framework for vehicular edge computing.”
Mohri, M., Sivek, G., & Suresh, A. T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.00146 · Cited by 373 · cited 2×
“We realized that a major part of the existing approaches are based on server-centric design using trusted servers (e.g., [38]).”
SAFA: A Semi-Asynchronous Protocol for Fast Federated Learning With Low Overhead
Wu, W., He, L., Lin, W., Mao, R., Maple, C., & Jarvis, S. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2020.2994391 · Cited by 362
A Survey of Neuromorphic Computing and Neural Networks in Hardware
Schuman, C. D., Potok, T. E., Patton, R. M., Birdwell, J. D., Dean, M. E., Rose, G. S., & Plank, J. S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.06963 · Cited by 352 · cited 2×
“We refer the readers to [197] to get a better overview of this field.”
Personalized Federated Learning: A Meta-Learning Approach
Fallah, A., Mokhtari, A., & Ozdaglar, A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.07948 · Cited by 345 · cited 1×
“In this regard, the authors in [26], [90], [151]–[155] studied the relation between communication cost and effective resource utilization, though they did not discuss the complications of the local problem’s solution.”
MLbase: A Distributed Machine-learning System
Kraska, T., Talwalkar, A., & Duchi, J. C. (2013) · Cited by 324
Expanding the Reach of Federated Learning by Reducing Client Resource Requirements
Caldas, S., Konečny, J., McMahan, H. B., & Talwalkar, A. (2018) · arXiv · doi:10.48550/arxiv.1812.07210 · Cited by 312 · cited 1×
“With the same motivation, Caldas et al. [150] applied lossy compression and federated dropout to train a smaller sub-set of local clients and reduce client-to-server interaction and local computation (see Fig.”
Training and Inference with Integers in Deep Neural Networks
Wu, S., Li, G., Chen, F., & Shi, L. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.04680 · Cited by 306 · cited 1×
“Wu et al. [106] proposed an approach to reduce the cost of training and inference by using lower bitwidth integers for both stages of the application.”
Multi-Objective Evolutionary Federated Learning
Zhu, H., & Jin, Y. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2919699 · Cited by 301
The ARM Scalable Vector Extension
Stephens, N., Biles, S., Boettcher, M., Eapen, J., Eyole, M., Gabrielli, G., Horsnell, M., Magklis, G., Martinez, A., Premillieu, N., et al. (2017) · IEEE Micro · doi:10.1109/mm.2017.35 · Cited by 295 · cited 1×
“The works in this area can be categorized in the domain of novel instructions for executing ML in the CPU [188], [189], design of specialized accelerators [190]–[192], optimized software library [193], [194], development of new memory technologies [195], and near data processing to enable large…”
A survey on application of machine learning for Internet of Things
Cui, L., Yang, S., Chen, F., Ming, Z., Lu, N., & Qin, J. (2018) · International Journal of Machine Learning and Cybernetics · doi:10.1007/s13042-018-0834-5 · Cited by 293
Federated Learning for Keyword Spotting
Leroy, D., Coucke, A., Lavril, T., Gisselbrecht, T., & Dureau, J. (2019) · ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp.2019.8683546 · Cited by 291
CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs
Oliveira, A., Costa, D., Moreira, G., Martins, J., & Pinto, S. (2018) · arXiv (Cornell University) · doi:10.4230/oasics.ng-res.2025.4 · Cited by 288 · cited 1×
“…in the domain of novel instructions for executing ML in the CPU [188], [189], design of specialized accelerators [190]–[192], optimized software library [193], [194], development of new memory technologies [195], and near data processing to enable large storage using smaller energy budget [196].”
Lu, Y., Huang, X., Zhang, K., Maharjan, S., & Zhang, Y. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3015772 · Cited by 283
A Survey of On-Device Machine Learning: An Algorithms and Learning Theory Perspective
Dhar, S., Guo, J., Liu, J., Tripathi, S., Kurup, U., & Shah, M. (2021) · ACM Transactions on Internet of Things · doi:10.1145/3450494 · Cited by 276
Privacy-Preserving Federated Learning Framework Based on Chained Secure Multiparty Computing
Li, Y., Zhou, Y., Jolfaei, A., Yu, D., Xu, G., & Zheng, X. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3022911 · Cited by 265 · cited 1×
“Furthermore, some prior works [202]–[205] proposed mechanisms to preserve privacy in a blockchain-enabled FL-based IoT environment.”
Privacy-Preserving Federated Learning in Fog Computing
Zhou, C., Fu, A., Yu, S., Yang, W., Wang, H., & Zhang, Y. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2987958 · Cited by 252
Cambricon: an instruction set architecture for neural networks
Liu, S., Du, Z., Tao, J., Han, D., Luo, T., Xie, Y., Chen, Y., & Chen, T. (2016) · ACM SIGARCH Computer Architecture News · doi:10.1145/3007787.3001179 · Cited by 243 · cited 1×
“The works in this area can be categorized in the domain of novel instructions for executing ML in the CPU [188], [189], design of specialized accelerators [190]–[192], optimized software library [193], [194], development of new memory technologies [195], and near data processing to enable large…”
DiSCO: Distributed Optimization for Self-Concordant Empirical Loss
Zhang, Y., & Xiao, L. (2015) · International Conference on Machine Learning · Cited by 243
Liu, B., Wang, L., & Liu, M. (2019) · IEEE Robotics and Automation Letters · doi:10.1109/lra.2019.2931179 · Cited by 242 · cited 1×
“Smart Robotics: A lifelong reinforcement FL framework for mobile robots is proposed in [50].”
Protection Against Reconstruction and Its Applications in Private Federated Learning
Bhowmick, A., Duchi, J. C., Freudiger, J., Kapoor, G., & Rogers, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.00984 · Cited by 240 · cited 3×
“Though several privacy definitions for FL are stated in [40], [56], [58], [128], and [133]–[138], we can classify them as global and local privacy.”
Federated Learning for Emoji Prediction in a Mobile Keyboard
Ramaswamy, S., Mathews, R., Rao, K., & Beaufays, F. (2019) · arXiv · doi:10.48550/arxiv.1906.04329 · Cited by 231
Federated Learning in the Sky: Aerial-Ground Air Quality Sensing Framework With UAV Swarms
Liu, Y., Nie, J., Li, X., Ahmed, S. H., Lim, W. Y. B., & Miao, C. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3021006 · Cited by 227 · cited 1×
“Resource-Efficient Training of UAV-Enabled IoT Devices: A particle swarm-based air quality monitoring framework is proposed in [57].”
Federated Learning in the Sky: Aerial-Ground Air Quality Sensing Framework with UAV Swarms
Liu, Y., Nie, J., Li, X., Ahmed, S. H., Lim, W. Y. B., & Miao, C. (2020) · arXiv · doi:10.48550/arxiv.2007.12004 · Cited by 227 · cited 1×
“Resource-Efficient Training of UAV-Enabled IoT Devices: A particle swarm-based air quality monitoring framework is proposed in [57].”
A Fairness-aware Incentive Scheme for Federated Learning
Yu, H., Liu, Z., Liu, Y., Chen, T., Cong, M., Weng, X., Niyato, D., & Yang, Q. (2020) · Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3375627.3375840 · Cited by 226 · cited 3×
“With the motivation of addressing issues related to costs and mismatch between client’s contributions and receiving incentives, Yu et al. [116], [122] proposed a payoff-sharing scheme named FL Incentivizer (FLI).”
FLChain: A Blockchain for Auditable Federated Learning with Trust and Incentive
Bao, X., Su, C., Xiong, Y., Huang, W., & Hu, Y. (2019) · 2019 5th International Conference on Big Data Computing and Communications (BIGCOM) · doi:10.1109/bigcom.2019.00030 · Cited by 225 · cited 1×
“A trust and incentive-based FL model is designed in [119], where they proposed to add local computation results of the clients using the concept of blockchain consensus to establish a public auditable and decentralized FL ecosystem.”
Liu, Y., Liu, Y., Liu, Z., Liang, Y., Meng, C., Zhang, J., & Zheng, Y. (2020) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2020.2992755 · Cited by 216
Benchmarking TinyML Systems: Challenges and Direction
Banbury, C. R., Reddi, V. J., Lam, M., Fu, W., Fazel, A., Holleman, J., Huang, X., Hurtado, R., Kanter, D., Lokhmotov, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2003.04821 · Cited by 216 · cited 1×
“A lot of edge applications are now using DNN-based algorithms [102], [103] and there is an increasing focus on making DNN inference efficient on edge devices [104], [105].”
Self-citationProtecting Data Privacy in Private Information Retrieval Schemes
Gertner, Y., Ishai, Y., Kushilevitz, E., & Malkin, T. (2000) · Journal of Computer and System Sciences · doi:10.1006/jcss.1999.1689 · Cited by 214
Motivating Workers in Federated Learning: A Stackelberg Game Perspective
Sarikaya, Y., & Ercetin, O. (2020) · IEEE Networking Letters · doi:10.1109/lnet.2019.2947144 · Cited by 207 · cited 1×
“Some incentives or regulations schemes are implemented in blockchain [217]–[219], and some incentive mechanisms are proposed for high-quality federated data [220], [221].”
Federated Collaborative Filtering for Privacy-Preserving Personalized Recommendation System
Ammad-ud-din, M., Ivannikova, E., Khan, S. A., Oyomno, W., Fu, Q., Tan, K. E., & Flanagan, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1901.09888 · Cited by 203
Personalized Federated Learning with Moreau Envelopes
Dinh, C. T., Tran, N. H. K., & Nguyen, T. D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.08848 · Cited by 203
Practical Federated Gradient Boosting Decision Trees
Li, Q., Wen, Z., & He, B. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5895 · Cited by 201
Wireless Federated Learning with Local Differential Privacy
Seif, M., Tandon, R., & Li, M. (2020) · 2020 IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit44484.2020.9174426 · Cited by 200
Dynamic Channel Pruning: Feature Boosting and Suppression
Gao, X., Zhao, Y., Dudziak, Ł., Mullins, R., & Xu, C. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.05331 · Cited by 195 · cited 1×
“DCP techniques identify channels in the OFM that are deemed unimportant and skip computations for those channels ([183], [184]).”
Distributed optimization with arbitrary local solvers
Ma, C., Konečný, J., Jaggi, M., Smith, V., Jordan, M. I., Richtárik, P., & Takáč, M. (2017) · Optimization Methods and Software · doi:10.1080/10556788.2016.1278445 · Cited by 191 · cited 2×
“Ma et al. [90] analyzed the tradeoffs of communication and resource expense; however, they did not address the complexity of the clients’ local model solutions.”
A training-integrity privacy-preserving federated learning scheme with trusted execution environment
Chen, Y., Luo, F., Li, T., Xiang, T., Liu, Z., & Li, J. (2020) · Information Sciences · doi:10.1016/j.ins.2020.02.037 · Cited by 188
Self-citationCollaborative Fairness in Federated Learning
Lyu, L., Xu, X., Wang, Q., & Yu, H. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-63076-8_14 · Cited by 184
FedSel: Federated SGD Under Local Differential Privacy with Top-k Dimension Selection
Liu, R., Cao, Y., Yoshikawa, M., & Chen, H. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59410-7_33 · Cited by 182
Adaptive Gradient Sparsification for Efficient Federated Learning: An Online Learning Approach
Han, P., Wang, S., & Leung, K. K. (2020) · 2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs47774.2020.00026 · Cited by 181 · cited 1×
“10, we see that a particular data collection point According to [33] and [149], a novel approach is proposed to share compressed sizes of message and carry out the reduced number of communication rounds to attain the target model.”
Self-citationcpSGD: Communication-efficient and differentially-private distributed SGD
Agarwal, N., Suresh, A. T., Yu, F., Kumar, S., & McMahan, H. B. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.10559 · Cited by 180 · cited 1×
“Besides, DP can be coupled with model compression strategies to reduce communication overhead and attain an improved version of privacy simultaneously [7], [135].”
Privacy-preserving SVM classification
Vaidya, J., Yu, H., & Jiang, X. (2008) · Knowledge and Information Systems · doi:10.1007/s10115-007-0073-7 · Cited by 178
Langford, J. C., Smola, A. J., & Zinkevich, M. (2009) · arXiv (Cornell University) · doi:10.48550/arxiv.0911.0491 · Cited by 178 · cited 1×
“Asynchronous training [95] is applied in some recent works because of its faster convergence when communication latency is comparatively higher and heterogeneous across the clients.”
FedCoin: A Peer-to-Peer Payment System for Federated Learning
Liu, Y., Ai, Z., Sun, S., Zhang, S., Liu, Z., & Yu, H. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-63076-8_9 · Cited by 177
Communication-Efficient Distributed Dual Coordinate Ascent
Jaggi, M., Smith, V., Takáč, M., Terhorst, J., Krishnan, S., Hofmann, T., & Jordan, M. I. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1409.1458 · Cited by 177 · cited 1×
“Such a technique is useful in practice and distributed approximate Newton (DANE) [84], CoCoA [85], and DiSCO [86] are some of the examples of such distributed optimization techniques.”
Asynchronous Federated Learning for Geospatial Applications
Sprague, M., Jalalirad, A., Scavuzzo, M., Capota, C., Neun, M., Do, L., & Kopp, M. (2019) · Communications in computer and information science · doi:10.1007/978-3-030-14880-5_2 · Cited by 175 · cited 2×
“Sprague et al. [159] analyzed the issues of ensuring convergence of asynchronous FL but did not present a solution to overcome such issues.”
Neurocube: A Programmable Digital Neuromorphic Architecture with High-Density 3D Memory
Kim, D., Kung, J., Chai, S., Yalamanchili, S., & Mukhopadhyay, S. (2016) · 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA) · doi:10.1109/isca.2016.41 · Cited by 171
Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication
Koloskova, A., Stich, S. U., & Jaggi, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.00340 · Cited by 163
Asynchronous Stochastic Gradient Descent with Delay Compensation
Zheng, S., Meng, Q., Wang, T., Chen, W., Yu, N., Ma, Z.-M., & Liu, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.08326 · Cited by 154 · cited 1×
“Xie et al. [96], Zheng et al. [97], and Lian et al. [98] analyzed asynchronous FL training with provable convergence by combining it with federated optimization.”
Privacy preserving regression modelling via distributed computation
Sanil, A. P., Karr, A. F., Lin, X., & Reiter, J. P. (2004) · Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1014052.1014139 · Cited by 146
Peer-to-peer Federated Learning on Graphs
Lalitha, A., Kilinc, O. C., Javidi, T., & Koushanfar, F. (2019) · arXiv · doi:10.48550/arxiv.1901.11173 · Cited by 146
Battery-constrained federated edge learning in UAV-enabled IoT for B5G/6G networks
Tang, S., Zhou, W., Chen, L., Lai, L., Xia, J., & Fan, L. (2021) · Physical Communication · doi:10.1016/j.phycom.2021.101381 · Cited by 140 · cited 1×
“Besides, Tang et al. [216] addressed the issue of reducing latency and improving the energy efficiency of UAV-enabled IoT devices by optimizing battery resources and wireless bandwidth.”
Bayesian Nonparametric Federated Learning of Neural Networks
Yurochkin, M., Agarwal, M., Ghosh, S., Greenewald, K. H., Hoang, T. N., & Khazaeni, Y. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.12022 · Cited by 133
Semi-Stochastic Gradient Descent Methods
Konečný, J., & Richtárik, P. (2017) · Frontiers in Applied Mathematics and Statistics · doi:10.3389/fams.2017.00009 · Cited by 132 · cited 1×
“There have been various methods to deal with distributed online optimization and distributed learning, including stochastic variance reduced gradient (SVRG) [87], [88], DANE [84] that is particularly for distributed optimization, naive Federated SVRG, and Federated SVRG ( FSVRG ) .”
Overcoming Noisy and Irrelevant Data in Federated Learning
Tuor, T., Wang, S., Ko, B. J., Liu, C., & Leung, K. K. (2021) · 2020 25th International Conference on Pattern Recognition (ICPR) · doi:10.1109/icpr48806.2021.9412599 · Cited by 131 · cited 1×
“Tuor et al. [113] proposed a method of finding and ignoring irrelevant data (possibly due to mislabeling) from FL.”
Self-citationGuha, N., Talwalkar, A., & Smith, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.11175 · Cited by 126 · cited 1×
“The protocol outcome enables multiple data owners to collaboratively agree to generate a function without leaking any information [130]–[132].”
Channel Gating Neural Networks
Hua, W., Zhou, Y. J., De, C., Zhang, Z., & Suh, G. E. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.12549 · Cited by 124 · cited 1×
“DSG techniques ([181], [182]) identify spatial regions in the OFM that are deemed important and focus their attention only on those parts of the OFM.”
A sustainable incentive scheme for federated learning
Yu, H., Liu, Z., Liu, Y., Chen, T., Cong, M., Weng, X., Niyato, D., & Yang, Q. (2020) · IEEE Intelligent Systems · doi:10.1109/mis.2020.2987774 · Cited by 123 · cited 1×
“With the motivation of addressing issues related to costs and mismatch between client’s contributions and receiving incentives, Yu et al. [116], [122] proposed a payoff-sharing scheme named FL Incentivizer (FLI).”
Stochastic Gradient Push for Distributed Deep Learning
Assran, M., Loizou, N., Ballas, N., & Rabbat, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.10792 · Cited by 120 · cited 1×
“Fully decentralized SGD and other optimization algorithms are recently getting popular for scalability in large-scale systems [74] and decentralization of networks devices [71], [75]–[82].”
Low-Memory Neural Network Training: A Technical ReportPreprint
Sohoni, N. S., Aberger, C. R., Leszczynski, M., Zhang, J., & Ré, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.10631 · Cited by 118 · cited 1×
“Training on edge devices can bene-fit by adapting techniques that have served well in managing this problem in the server context, such as efficient gradient checkpointing [108], tensor rematerialization [109], and recompute [110].”
Energy-Aware Analog Aggregation for Federated Learning with Redundant Data
Sun, Y., Zhou, S., & Gunduz, D. (2020) · ICC 2020 - 2020 IEEE International Conference on Communications (ICC) · doi:10.1109/icc40277.2020.9148853 · Cited by 111 · cited 1×
“To reduce communication overhead, Imteaj and Amini [147] and Sun et al. [148] discussed infusing redundancy among the client data set to reach convergence with fewer communication rounds.”
Bitcoin-NG: A Scalable Blockchain Protocol
Eyal, I., Gencer, A. E., Sirer, E. G., & van Renesse, R. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.02037 · Cited by 111 · cited 1×
“Some incentives or regulations schemes are implemented in blockchain [217]–[219], and some incentive mechanisms are proposed for high-quality federated data [220], [221].”
InPrivate Digging: Enabling Tree-based Distributed Data Mining with Differential Privacy
Zhao, L., Ni, L., Hu, S., Chen, Y., Zhou, P., Xiao, F., & Wu, L. (2018) · IEEE INFOCOM 2018 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2018.8486352 · Cited by 108 · cited 1×
“In Table III, we explore such studies (e.g., [39]) as “Client-centric” in the “Decision-making” column.”
FedNER: Privacy-preserving Medical Named Entity Recognition with Federated LearningPreprint
Ge, S., Wu, F., Wu, C., Qi, T., Huang, Y., & Xie, X. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.09288 · Cited by 107
Variational Federated Multi-Task Learning
Corinzia, L., Beuret, A., & Buhmann, J. M. (2019) · arXiv · doi:10.48550/arxiv.1906.06268 · Cited by 105 · cited 1×
“Corinzia and Buhmann [171] modeled a Bayesian network by performing variational inference during learning.”
Koppula, S., Orosa, L., Yağlıkçı, A. G., Azizi, R., Shahroodi, T., Kanellopoulos, K., & Mutlu, O. (2019) · Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture · doi:10.1145/3352460.3358280 · Cited by 104 · cited 1×
“…in the domain of novel instructions for executing ML in the CPU [188], [189], design of specialized accelerators [190]–[192], optimized software library [193], [194], development of new memory technologies [195], and near data processing to enable large storage using smaller energy budget [196].”
Gossip Dual Averaging for Decentralized Optimization of Pairwise FunctionsPreprint
Colin, I., Bellet, A., Salmon, J., & Clémençon, S. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.02421 · Cited by 103 · cited 1×
“Fully decentralized SGD and other optimization algorithms are recently getting popular for scalability in large-scale systems [74] and decentralization of networks devices [71], [75]–[82].”
First Analysis of Local GD on Heterogeneous Data
Khaled, A., Mishchenko, K., & Richtárik, P. (2019) · arXiv · doi:10.48550/arxiv.1909.04715 · Cited by 103 · cited 1×
“Each client device implements a fixed number of GD steps (declares from the average model) using their local function independently in parallel [70].”
Smith, V., Chiang, C.-K., Sanjabi, M., & Talwalkar, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.10467 · Cited by 101 · cited 3×
“Through multitask learning [48], such a statistical challenge can be handled.”
Federated Learning Of Out-Of-Vocabulary Words
Chen, M., Mathews, R., Ouyang, T., & Beaufays, F. (2019) · arXiv · doi:10.48550/arxiv.1903.10635 · Cited by 97
Toward Communication-Efficient Federated Learning in the Internet of Things With Edge Computing
Sun, H., Li, S., Yu, F. R., Qi, Q., Wang, J., & Liao, J. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2994596 · Cited by 95 · cited 1×
“In this regard, the authors in [26], [90], [151]–[155] studied the relation between communication cost and effective resource utilization, though they did not discuss the complications of the local problem’s solution.”
Boosting Privately: Federated Extreme Gradient Boosting for Mobile Crowdsensing
Liu, Y., Ma, Z., Liu, X., Ma, S., Nepal, S., Deng, R. H., & Ren, K. (2020) · 2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs47774.2020.00017 · Cited by 93
Privacy-preserving ridge regression on distributed data
Chen, Y.-R., Rezapour, A., & Tzeng, W.-G. (2018) · Information Sciences · doi:10.1016/j.ins.2018.03.061 · Cited by 89
Decentralized Collaborative Learning of Personalized Models over Networks
Vanhaesebrouck, P., Bellet, A., & Tommasi, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.05202 · Cited by 89 · cited 2×
“Though large organizations and companies may afford to place a powerful, robust, and secure central server to carry out the training process, all types of sectors cannot adapt that [71].”
Federated Meta-Learning for Recommendation
Chen, F., Dong, Z., Li, Z., & He, X. (2018) · Cited by 87
Federated Learning for Resource-Constrained IoT Devices: Panoramas and State of the Art
Imteaj, A., Mamun Ahmed, K., Thakker, U., Wang, S., Li, J., & Amini, M. H. (2023) · Adaptation, Learning, and Optimization · doi:10.1007/978-3-031-11748-0_2 · Cited by 86
Self-citationFederated Learning with Matched Averaging
Wang, H., Yurochkin, M., Sun, Y., Papailiopoulos, D., & Khazaeni, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.06440 · Cited by 86 · cited 1×
“In a nut-shell, many FL applications and frameworks are proposed on FedSGD [20], [21], [33], [34].”
Measuring statistical heterogeneity: The Pietra index
Eliazar, I. I., & Sokolov, I. M. (2010) · Physica A: Statistical Mechanics and its Applications · doi:10.1016/j.physa.2009.08.006 · Cited by 85 · cited 1×
“A local dissimilarity approach is proposed [174] to quantify statistical sample variation, where the resource-quantification starts after the training starts.”
Differentially Private Meta-Learning
Li, J., Khodak, M., Caldas, S., & Talwalkar, A. (2019) · arXiv · doi:10.48550/arxiv.1909.05830 · Cited by 82 · cited 1×
“Another interesting approach to a DP mechanism based on metalearning is proposed in [136] that can be used in FL through personalization.”
Self-citationDifferentially Private Learning with Adaptive Clipping
Andrew, G., Thakkar, O. D., McMahan, H. B., & Ramaswamy, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.03871 · Cited by 80 · cited 1×
“An adaptive gradient clipping technique is presented in [201] to handle this issue.”
PEFL: A Privacy-Enhanced Federated Learning Scheme for Big Data Analytics
Zhang, J., Chen, B., Yu, S., & Deng, H. (2019) · 2019 IEEE Global Communications Conference (GLOBECOM) · doi:10.1109/globecom38437.2019.9014272 · Cited by 79
Trading Redundancy for Communication: Speeding up Distributed SGD for Non-convex Optimization
Haddadpour, F., Kamani, M. M., Mahdavi, M., & Cadambe, V. R. (2019) · Cited by 79
ELFISH: Resource-Aware Federated Learning on Heterogeneous Edge Devices.Preprint
Xu, Z., Zhao, Y., Xiong, J., Yang, J., & Chen, X. (2019) · arXiv (Cornell University) · Cited by 76
Differentially Private Learning with Adaptive ClippingPreprint
Thakkar, O. D., Andrew, G., & McMahan, H. B. (2021) · arXiv (Cornell University) · Cited by 72
Federated Learning for Ranking Browser History SuggestionsPreprint
Hartmann, F., Suh, S., Komarzewski, A., Smith, T. D., & Segall, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.11807 · Cited by 67
FedAR: Activity and Resource-Aware Federated Learning Model for Distributed Mobile Robots
Imteaj, A., & Hadi Amini, M. (2020) · 2020 19th IEEE International Conference on Machine Learning and Applications (ICMLA) · doi:10.1109/icmla51294.2020.00185 · Cited by 66 · cited 1×
“Besides, an activity and resource-aware FL model is presented in [124].”
Self-citationFederated Learning: Privacy and Incentive
Yang, Q., Fan, L., & Yu, H. (2020) · Federated Learning · doi:10.1007/978-3-030-63076-8 · Cited by 66
Semi-Cyclic Stochastic Gradient Descent
Eichner, H., Koren, T., McMahan, H. B., Srebro, N., & Talwar, K. (2019) · arXiv · doi:10.48550/arxiv.1904.10120 · Cited by 66 · cited 1×
“Besides, Eichner et al. [172] aimed to identify cyclic patterns within data samples, while a detailed analysis of transfer learning strategy for personalization during FL can be seen in [170].”
Distributed Sensing Using Smart End-User Devices: Pathway to Federated Learning for Autonomous IoT
Imteaj, A., & Amini, M. H. (2019) · 2019 International Conference on Computational Science and Computational Intelligence (CSCI) · doi:10.1109/csci49370.2019.00218 · Cited by 65 · cited 1×
“To reduce communication overhead, Imteaj and Amini [147] and Sun et al. [148] discussed infusing redundancy among the client data set to reach convergence with fewer communication rounds.”
Self-citation(2020) · arXiv (Cornell University) · Cited by 64
Scalable and Differentially Private Distributed Aggregation in the Shuffled Model
Ghazi, B., Pagh, R., & Velingker, A. (2019) · arXiv · doi:10.48550/arxiv.1906.08320 · Cited by 62 · cited 1×
“Though several privacy definitions for FL are stated in [40], [56], [58], [128], and [133]–[138], we can classify them as global and local privacy.”
A federated learning framework for healthcare IoT devices
Yuan, B., Ge, S., & Xing, W. (2020) · arXiv · doi:10.48550/arxiv.2005.05083 · Cited by 59 · cited 1×
“Yuan et al. [214] developed an FL framework for smart healthcare by applying the FL mechanism and reduced computation load of IoT devices during training.”
A Communication Efficient Vertical Federated Learning Framework.Preprint
Liu, Y., Kang, Y., Zhang, X., Li, L., Cheng, Y., Chen, T., Hong, M., & Yang, Q. (2019) · arXiv (Cornell University) · Cited by 58
Asynchronous Federated Learning with Differential Privacy for Edge IntelligencePreprint
Li, Y., Yang, S., Ren, X., & Zhao, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.07902 · Cited by 53 · cited 1×
“Xie et al. [96] proposed asynchronous federated optimization, [159] discussed on the asynchronous FL for geospatial applications, [160]–[162] proposed a DP-based asynchronous FL strategy for MEC, and [163] presented a blockchain-based secure data sharing strategy for asynchronous FL.”
Edge-Assisted Hierarchical Federated Learning with Non-IID DataPreprint
Liu, L., Zhang, J., Song, S., & Letaief, K. B. (2019) · arXiv (Cornell University) · Cited by 52
Boosting Privately: Privacy-Preserving Federated Extreme Boosting for Mobile Crowdsensing
Liu, Y., Ma, Z., Liu, X., Ma, S., Nepal, S., & Deng, R. (2019) · arXiv · doi:10.48550/arxiv.1907.10218 · Cited by 52
Personalized and Private Peer-to-Peer Machine Learning
Bellet, A., Guerraoui, R., Taziki, M., & Tommasi, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08435 · Cited by 51
Asynchronous Online Federated Learning for Edge Devices.Preprint
Chen, Y., Ning, Y., & Rangwala, H. (2019) · arXiv (Cornell University) · Cited by 49
Coded Federated Computing in Wireless Networks with Straggling Devices and Imperfect CSI
Ha, S., Zhang, J., Simeone, O., & Kang, J. (2019) · 2019 IEEE International Symposium on Information Theory (ISIT) · doi:10.1109/isit.2019.8849507 · Cited by 47 · cited 1×
“If the number of such straggler clients is high, then the overall model convergence would be at stake [99], [100].”
On-device machine learning An algorithms and learning theory perspective
Dhar, S., Guo, J., Liu, J., Tripathi, S., Kurup, U., & Shah, M. (2019) · arXiv · doi:10.48550/arxiv.1911.00623 · Cited by 47 · cited 1×
“We can define an edge device as a resource-constrained entity with limited computational power, storage capacity, transmission range, and battery [89].”
An incentive compatible reputation mechanism
Jurca, R., & Faltings, B. V. (2003) · IEEE Conference on Commerce and Enterprise Computing (CEC) · doi:10.1109/coec.2003.1210263 · Cited by 47 · cited 1×
“Some incentives or regulations schemes are implemented in blockchain [217]–[219], and some incentive mechanisms are proposed for high-quality federated data [220], [221].”
Research works printed but not linked: 10
- 10.5555/3295222.3295285 · doi:10.5555/3295222.3295285
- Tang, $D^{2}$ : Decentralized training over decentralized data, 4848
- 10.5555/2999611.2999647 · doi:10.5555/2999611.2999647
- Chai, Towards taming the resource and data heterogeneity in federated learning, 19
- 10.1109/comst.2020.2986024 · DOI · Paper page
- Thrun, 2012, Learning to Learn
- Kumar, Resource-efficient machine learning in 2 KB RAM for the Internet of Things, 1935
- Gupta, ProtoNN: Compressed and accurate kNN for resource-scarce devices, 1331
- Li, Federated optimization in heterogeneous networks, 2, 429
- Xu, MicroFL: A lightweight, secure-by-design edge network fabric for decentralized IoT systems
Cited by
Federated learning on non-IID data: A survey
Zhu, H., Xu, J., Liu, S., & Jin, Y. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.098 · Cited by 1,062
Xu, W., Yang, Z., Ng, D. W. K., Levorato, M., Eldar, Y. C., & Debbah, M. (2023) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2023.3239189 · Cited by 660
“Compared with these works [10], [19], [20], [21], [22], [23], the main focus of this paper is to provide a comprehensive overview of state-of-the-art signal processing techniques for EL over B5G networks.”
Federated learning for the Internet of Things: Applications, challenges, and opportunities
Zhang, T., Gao, L., He, C., Zhang, M., Krishnamachari, B., & Avestimehr, A. S. (2022) · IEEE Internet of Things Magazine · doi:10.1109/iotm.004.2100182 · Cited by 412
A state-of-the-art survey on solving non-IID data in Federated Learning
Ma, X., Zhu, J., Lin, Z., Chen, S., & Qin, Y. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.05.003 · Cited by 399
Decentralized Federated Averaging
Sun, T., Li, D., & Wang, B. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3196503 · Cited by 388
“The federated learningmethod has been applied to Internet of Things (IoT) researches [13].”
Method
Metadata
- Authors
- Imteaj, A., Thakker, U., Wang, S., Li, J., & Amini, M. H.
- Affiliations
- Florida International University (US)
- IBM (United States) (US)
- Binghamton University (US)
- Venue
- IEEE Internet of Things Journal
- Published
- 2022-01-01
- Volume
- 9
- Issue
- 1
- Pages
- 1-24
- ISSN
- 2327-4662 (ISSN-L)
- 2372-2541
- OpenAlex
- W3182125009
- Cited by
- 774
- References
- 223
- Citations by year
- 2026: 113 · 2025: 201 · 2024: 209 · 2023: 155 · 2022: 85
- FWCI
- 40.49
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Internet of Things Journal (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Information Systems › Internet of Things and AI
- Keywords
- federated learning, resource-constrained devices, Internet of Things, edge computing, on-device learning, data privacy, distributed learning, low-power devices, implementation barriers
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.