CEFL: Online Admission Control, Data Scheduling, and Accuracy Tuning for Cost-Efficient Federated Learning Across Edge Nodes
The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index.
Abstract
With the proliferation of Internet of Things (IoT), zillions of bytes of data are generated at the network edge, incurring an urgent need to push the frontiers of artificial intelligence (AI) to network edge so as to fully unleash the potential of the IoT big data. To materialize such a vision which is known as edge intelligence, federated learning is emerging as a promising solution to enable edge nodes to collaboratively learn a shared model in a privacy-preserving and communication-efficient manner, by keeping the data at the edge nodes. While pilot efforts on federated learning have mostly focused on reducing the communication overhead, the computation efficiency of those resource-constrained edge nodes has been largely overlooked. To bridge this gap, in this article, we investigate how to coordinate the edge and the cloud to optimize the system-wide cost efficiency of federated learning. Leveraging the Lyapunov optimization theory, we design and analyze a cost-efficient optimization framework CEFL to make online yet near-optimal control decisions on admission control, load balancing, data scheduling, and accuracy tuning for the dynamically arrived training data samples, reducing both computation and communication cost. In particular, our control framework CEFL can be flexibly extended to incorporate various design choices and practical requirements of federated learning, such as exploiting the cheaper cloud resource for model training with better cost efficiency yet still facilitating on-demand privacy preservation. Via both rigorous theoretical analysis and extensive trace-driven evaluations, we verify the cost efficiency of our proposed CEFL framework.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215 · cited 2×
“To avoid from transmitted the huge raw data from widespread edge devices to the cloud, Google proposed a distributed learning technique named federated learning [14].”
BackgroundEdge computing: Vision and challenges
Shi, W., Cao, J., Zhang, Q., Li, Y., & Xu, L. (2016) · IEEE Internet of Things Journal · doi:10.1109/jiot.2016.2579198 · Cited by 8,733 · cited 1×
“To address the above challenges, edge computing [3] has”
BackgroundFederated Learning: Strategies for Improving Communication Efficiency
Konečný, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D. (2016) · arXiv · doi:10.48550/arxiv.1610.05492 · Cited by 3,766 · cited 2×
“An emerging decentralized learning framework which is know as federated learning [12], [14], which coordinates the distributed edge nodes and the centralized cloud to solves the global optimization problem (1) in an iterative manner.”
BackgroundEdge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing
Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2918951 · Cited by 2,455 · cited 1×
““edge intelligence” or “edge AI” [4], [5] that refers to perform-”
Self-citationTowards Federated Learning at Scale: System Design
Bonawitz, K., Eichner, H., Grieskamp, W., Huba, D., Ingerman, A., Ivanov, V., Kiddon, C., Konečný, J., Mazzocchi, S., McMahan, H. B., et al. (2019) · arXiv · doi:10.48550/arxiv.1902.01046 · Cited by 2,334 · cited 2×
“Specifically, in Google’s implementation [13], not all the data samples would be used to train the AI model, instead, only data samples with certain freshness (e.g., the latest 100 data samples) would be selected to train the AI model.”
BackgroundMethodAdaptive Federated Learning in Resource Constrained Edge Computing Systems
Wang, S., Tuor, T., Salonidis, T., Leung, K. K., Makaya, C., He, T., & Chan, K. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2904348 · Cited by 2,232 · cited 1×
“To navigate such an accuracy–cost tradeoff, Wang et al. [15] proposed a control algorithm that determines the best tradeoff between local update and global parameter aggregation under a given resource budget.”
BackgroundStochastic Network Optimization with Application to Communication and Queueing Systems
Neely, M. J. (2010) · Synthesis lectures on communication networks · doi:10.2200/s00271ed1v01y201006cnt007 · Cited by 2,218
Federated Learning over Wireless Networks: Optimization Model Design and Analysis
Tran, N. H., Bao, W., Zomaya, A., Nguyen, M. N. H., & Hong, C. S. (2019) · IEEE INFOCOM 2019 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2019.8737464 · Cited by 1,013 · cited 1×
“When considering both energy cost and training latency of federated learning, Tran et al. [19] jointly applied CPU speed scaling, shared-network allocation and local model accuracy tuning to achieved well-balanced tradeoff between: 1) energy cost and training latency and 2) communication overhead…”
BackgroundEdge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computing
Li, E., Zeng, L., Zhou, Z., & Chen, X. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2946140 · Cited by 929 · cited 1×
“…efforts focus on optimizing the inference phase of edge intelligence, via approaches, such as model compression, model partition, model right sizing, and model selection to reduce the latency and/or resource consumption of model inference on resource-and energy-constrained edge nodes [4]–[11].”
Self-citationBenchmark analysis of representative deep neural network architectures
Bianco, S., Cadene, R., Celona, L., & Napoletano, P. (2018) · IEEE Access · doi:10.1109/access.2018.2877890 · Cited by 838 · cited 1×
“For example, for the commonly adopted vision-oriented DNN, such as Alexnet and VGG-16, the CPU cycles required for training one image data sample are 727 MFLOPS and 16 GFLOPS, respectively [25].”
MethodWrite off-loading: Practical power management for enterprise storage
Narayanan, D., Donnelly, A., & Rowstron, A. (2008) · ACM Transactions on Storage · doi:10.1145/1416944.1416949 · Cited by 762 · cited 1×
“Specifically, we associate with the one-week quarterly traces of the data centers of Google [33], Facebook [34], HP [35], and Microsoft [36] to each of the I = 4 edge nodes, respectively.”
MethodNeurosurgeon: Collaborative Intelligence Between the Cloud and Mobile Edge
Kang, Y., Hauswald, J., Gao, C., Rovinski, A., Mudge, T., Mars, J., & Tang, L. (2017) · Proceedings of the Twenty-Second International Conference on Architectural Support for Programming Languages and Operating Systems · doi:10.1145/3037697.3037698 · Cited by 673 · cited 1×
“In particular, kang et al. [6] proposed the method of DNN partitioning to coordinate the edge and the cloud, the intuition is that the DNN model can be carefully partitioned into two parts, and the more resource demanding part can be offloaded to the cloud at a low communication overhead.”
MethodCMFL: Mitigating Communication Overhead for Federated Learning
WANG, L., WANG, W., & LI, B. (2019) · 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2019.00099 · Cited by 475 · cited 1×
“…that some local model updates trained over client-specific and biased data are irrelevant to the global convergence, Wang et al. developed communication-mitigated federated learning (CMFL) [17] to identify irrelevant updates and preclude them from being uploaded for communication cost reduction.”
BackgroundThe Case for Evaluating MapReduce Performance Using Workload Suites
Chen, Y., Ganapathi, A., Griffith, R., & Katz, R. (2011) · 2011 IEEE 19th Annual International Symposium on Modelling, Analysis, and Simulation of Computer and Telecommunication Systems · doi:10.1109/mascots.2011.12 · Cited by 428 · cited 1×
“Specifically, we associate with the one-week quarterly traces of the data centers of Google [33], Facebook [34], HP [35], and Microsoft [36] to each of the I = 4 edge nodes, respectively.”
MethodCloud Programming Simplified: A Berkeley View on Serverless Computing
Jonas, E., Schleier-Smith, J., Sreekanti, V., Tsai, C.-C., Khandelwal, A., Pu, Q., Shankar, V., Carreira, J., Krauth, K., Yadwadkar, N. J., et al. (2019) · arXiv (Cornell University) · doi:10.4230/oasics.microservices.2020-2022.5 · Cited by 409 · cited 1×
“Note that such a fine-grained linear and the continuous cost model well captures the recent trend that the emerging paradigm of serverless computing [30] is increasingly employed to host AIenabled IoT (AIoT) applications.”
BackgroundEdge Intelligence: On-Demand Deep Learning Model Co-Inference with Device-Edge Synergy
Li, E., Zhou, Z., & Chen, X. (2018) · Proceedings of the 2018 Workshop on Mobile Edge Communications · doi:10.1145/3229556.3229562 · Cited by 385 · cited 3×
“…efforts focus on optimizing the inference phase of edge intelligence, via approaches, such as model compression, model partition, model right sizing, and model selection to reduce the latency and/or resource consumption of model inference on resource-and energy-constrained edge nodes [4]–[11].”
BackgroundSelf-citationLow Latency Geo-distributed Data Analytics
Pu, Q., Ananthanarayanan, G., Bodik, P., Kandula, S., Akella, A., Bahl, P., & Stoica, I. (2015) · Proceedings of the 2015 ACM Conference on Special Interest Group on Data Communication · doi:10.1145/2785956.2787505 · Cited by 246 · cited 1×
“leakage can also be a major concern [2].”
BackgroundAccelerating DNN Training in Wireless Federated Edge Learning Systems
Ren, J., Yu, G., & Ding, G. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3036971 · Cited by 226 · cited 1×
“For example, to accelerate the DNN training in the wireless federated edge learning system, Ren et al. [18] studied how to optimize the batch size of shared-network allocation for multiple edge nodes.”
MethodGeographical load balancing with renewables
Liu, Z., Lin, M., Wierman, A., Low, S. H., & Andrew, L. L. H. (2011) · ACM SIGMETRICS Performance Evaluation Review · doi:10.1145/2160803.2160862 · Cited by 221 · cited 1×
“Specifically, we associate with the one-week quarterly traces of the data centers of Google [33], Facebook [34], HP [35], and Microsoft [36] to each of the I = 4 edge nodes, respectively.”
MethodOn-Demand Deep Model Compression for Mobile Devices: A Usage-Driven Model Selection Framework
Liu, S., Lin, Y., Zhou, Z., Nan, K., Liu, H., & Du, J. (2018) · Proceedings of the 16th Annual International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3210240.3210337 · Cited by 217 · cited 1×
“Finally, the recent work AdaDeep [7] jointly considers the goals of accuracy, latency, storage, and energy, and leverages deep reinforcement learning (DRL) to efficiently find a good configuration of the applied model compression technique.”
BackgroundSelf-citationDistributed optimization with arbitrary local solvers
Ma, C., Konečný, J., Jaggi, M., Smith, V., Jordan, M. I., Richtárik, P., & Takáč, M. (2017) · Optimization Methods and Software · doi:10.1080/10556788.2016.1278445 · Cited by 191 · cited 1×
“…global model accuracy target (cid:4) which is typically predetermined and keeps constant over the lifecycle of the AI model, theoretical studies [23] have shown that the general upper bound on the number of on global iterations is where θ( t ) is the aforementioned local model accuracy at time…”
BackgroundSemi-Stochastic Gradient Descent Methods
Konečný, J., & Richtárik, P. (2017) · Frontiers in Applied Mathematics and Statistics · doi:10.3389/fams.2017.00009 · Cited by 132 · cited 1×
“For the local iterations in each global iteration, to achieve a local model accuracy of θ(t), the number of local iterations is upper bounded by O(log(1/θ(t))) for a wide range of iterative algorithms to solve problem (2), such as gradient descent, coordinate descent, or stochastic dual coordinate descent [24].”
BackgroundAn Efficient Application Partitioning Algorithm in Mobile Environments
Wu, H., Knottenbelt, W. J., & Wolter, K. (2019) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2019.2891695 · Cited by 125
Online Orchestration of Cross-Edge Service Function Chaining for Cost-Efficient Edge Computing
Zhou, Z., Wu, Q., & Chen, X. (2019) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2019.2927070 · Cited by 119 · cited 1×
“In line with the recent modeling work on edge computing [20]–[22], we adopt a discrete time-slotted model to fully characterize and leverage the system dynamics (e.”
MethodSelf-citationStochastic Analysis of Delayed Mobile Offloading in Heterogeneous Networks
Wu, H., & Wolter, K. (2018) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2017.2711014 · Cited by 114
Join the Shortest Queue with Many Servers. The Heavy-Traffic Asymptotics
Eschenfeldt, P. C., & Gamarnik, D. (2018) · Mathematics of Operations Research · doi:10.1287/moor.2017.0887 · Cited by 108 · cited 1×
“Such a routing policy is an intuitive “join-the-shortest-queue” policy for the purpose of load balancing among the I edge nodes, which is consistent with recent work on job scheduling for cloud servers [28].”
ResultFoggyCache: Cross-Device Approximate Computation Reuse
Guo, P., Hu, B., Li, R., & Hu, W. (2018) · Proceedings of the 24th Annual International Conference on Mobile Computing and Networking · doi:10.1145/3241539.3241557 · Cited by 91 · cited 1×
“Moreover, since the DNN inference tasks submitted by different devices generally show strong similarity, the inference results can be cached at the edge node, and then reused by nearby devices [8].”
BackgroundLi, Y., Park, J., Alian, M., Yuan, Y., Qu, Z., Pan, P., Wang, R., Schwing, A., Esmaeilzadeh, H., & Kim, N. S. (2018) · 2018 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1109/micro.2018.00023 · Cited by 85 · cited 1×
“Along a different route, Li et al. [16] demonstrated how to quantizing the model updates to low-precision float-value representations to reduce the communication overhead.”
BackgroundAn Online Market Mechanism for Edge Emergency Demand Response via Cloudlet Control
Chen, S., Jiao, L., Wang, L., & Liu, F. (2019) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2019.8737574 · Cited by 70 · cited 2×
“Following the recent literature on edge computing, we set the length of one time slot to be 15 min [20], [21].”
BackgroundMethodCloud-Based or On-Device: An Empirical Study of Mobile Deep Inference
Guo, T. (2018) · 2018 IEEE International Conference on Cloud Engineering (IC2E) · doi:10.1109/ic2e.2018.00042 · Cited by 66
Cost-Effective Cloud Server Provisioning for Predictable Performance of Big Data Analytics
Xu, F., Zheng, H., Jiang, H., Shao, W., Liu, H., & Zhou, Z. (2018) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2018.2873397 · Cited by 53 · cited 1×
“For the price of the cloud resource, to envisage its temporal variability, we resort to the dynamical price of a spot instance on the Amazon EC2 spot market, with the same period of October 10, 2018–October 16, 2018 [37].”
BackgroundSelf-citationMultiple Granularity Online Control of Cloudlet Networks for Edge Computing
Jiao, L., Pu, L., Wang, L., Lin, X., & Li, J. (2018) · IEEE International Conference on Sensing, Communication, and Networking (SECON) · doi:10.1109/sahcn.2018.8397141 · Cited by 42 · cited 3×
“Following the recent literature on edge computing, we set the length of one time slot to be 15 min [20], [21].”
BackgroundMethodSelf-citationOn the Power-of-d-choices with Least Loaded Server Selection
Hellemans, T., & Houdt, B. V. (2018) · doi:10.1145/3219617.3219664 · Cited by 30 · cited 1×
“To mitigate this complexity, we can resort to a recently developed alternative load balancing strategy of “power-of-m-choices” [29], which randomly selects a subset of m edge nodes and routes the admitted data samples to the one with the least queue backlog.”
Method
Research works printed but not linked: 4
- 2019, Cisco Global Cloud Index: Forecast and Methodology, 2016–2021 White Paper
- 2019, Online Technical Report
- Krizhevsky, 2019, Convolutional Deep Belief Networks on CIFAR-10
- 2019, Google Cluster Data
Cited by
Federated Learning for Internet of Things: A Comprehensive Survey
Nguyen, D. C., Ding, M., Pathirana, P. N., Seneviratne, A., Li, J., & Vincent Poor, H. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3075439 · Cited by 1,503
“For this problem, the authors in [56] suggest a cost-efficient optimization framework that can coordinate edge devices and cloud by minimizing the communication latency.”
BackgroundFederated Learning for Internet of Things: A Comprehensive Survey
Nguyen, D., Ding, M., Pathirana, P. N., Seneviratne, A., Li, J., & Poor, H. (2021) · Deakin Research Online (Deakin University) · Cited by 889
Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications
Aledhari, M., Razzak, R., Parizi, R. M., & Saeed, F. (2020) · IEEE Access · doi:10.1109/access.2020.3013541 · Cited by 733
Federated learning review: Fundamentals, enabling technologies, and future applications
Banabilah, S., Aloqaily, M., Alsayed, E., Malik, N., & Jararweh, Y. (2022) · Information Processing & Management · doi:10.1016/j.ipm.2022.103061 · Cited by 556
Wahab, O. A., Mourad, A., Otrok, H., & Taleb, T. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3058573 · Cited by 487
“In [120], the authors aim to reduce the computation cost on resource-constrained edge nodes through proposing a coordination scheme between the edge and cloud layers to optimize the system-wide cost-efficiency.”
Background
Metadata
- Authors
- Zhou, Z., Yang, S., Pu, L., & Yu, S.
- Affiliations
- Sun Yat-sen University (CN)
- Beijing Institute of Technology (CN)
- Nankai University (CN)
- Venue
- IEEE Internet of Things Journal
- Published
- 2020-10
- Volume
- 7
- Issue
- 10
- Pages
- 9341-9356
- ISSN
- 2327-4662 (ISSN-L)
- 2372-2541
- OpenAlex
- W3014208382
- Cited by
- 77
- References
- 33
- Citations by year
- 2026: 3 · 2025: 7 · 2024: 13 · 2023: 19 · 2022: 13 · 2021: 13 · 2020: 9
- FWCI
- 6.34
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Internet of Things Journal (publishedVersion)
- Funders
- National Key Research and Development Program of China (2017YFB1001703)
- National Science Foundation of China (61802449, 61702287, 61802018)
- Guangdong Natural Science Funds (2018A030313032)
- Program for Basic and Applied Basic Research Fund of Guangdong (2019A1515010030)
- Natural Science Foundation of Tianjin (18JCQNJC00200)
- Young Elite Scientists Sponsorship Program by Tianjin (TJSQNTJ-2018-19)
- Beijing Institute of Technology Research Fund Program
- Topics
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Hardware and Architecture › Energy Efficiency in Computing
- Keywords
- federated learning, edge computing, cost efficiency, admission control, data scheduling, load balancing, Lyapunov optimization, computational cost, communication overhead, Internet of Things, edge AI, privacy preservation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.