Adaptive Federated Learning in Resource Constrained Edge Computing Systems
Open accessThe DOI matches its Crossref record; 41 of the 41 references it cites were found in a registry or an index.
Abstract
Emerging technologies and applications including Internet of Things, social networking, and crowd-sourcing generate large amounts of data at the network edge. Machine learning models are often built from the collected data, to enable the detection, classification, and prediction of future events. Due to bandwidth, storage, and privacy concerns, it is often impractical to send all the data to a centralized location. In this paper, we consider the problem of learning model parameters from data distributed across multiple edge nodes, without sending raw data to a centralized place. Our focus is on a generic class of machine learning models that are trained using gradient-descent-based approaches. We analyze the convergence bound of distributed gradient descent from a theoretical point of view, based on which we propose a control algorithm that determines the best tradeoff between local update and global parameter aggregation to minimize the loss function under a given resource budget. The performance of the proposed algorithm is evaluated via extensive experiments with real datasets, both on a networked prototype system and in a larger-scale simulated environment. The experimentation results show that our proposed approach performs near to the optimum with various machine learning models and different data distributions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The problem studied in [15] captures two tradeoffs: (a) UE energy consumption and FL time via variations in device CPU-cycles/sec and (b) computational and communication latencies for FL accuracy.”
From Federated Learning for Edge Networks: Resource Optimization and Incentive Mechanism (2020) · cites this paper as [15]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Federated learning is a special kind of distributed learning that is characterized by various unstable users distributed in different places, each of whom has control over their devices and data [115, 116].”
From Machine Unlearning: A Survey (2024) · cites this paper as [115]
Experimental — accuracy not yet measuredIn the citing paper
“FL-based mechanism aggregation at the edge may be beneficial because the volumes of information gained from nodes are quite high in rate using such approaches [174].”
From Federated learning-based AI approaches in smart healthcare: Concepts, taxonomies, challenges and open issues (2023) · cites this paper as [174]
In the citing paper
“In our previous work [15], we jointly optimized the computing time and energy consumption of FL over wireless networks.”
From Federated Learning for Edge Networks: Resource Optimization and Incentive Mechanism (2020) · cites this paper as [15]
In the citing paper
“However, while several recent studies have investigated convergence guarantees of variants of FL methods [95]- [97], little work has evaluated the impact of failures over the FL algorithm or studied directly the effect of dropped devices.”
From Handling privacy-sensitive medical data with federated learning: Challenges and future directions (2023) · cites this paper as [96]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · cited 3×
“CNN is trained using SGD on three different datasets, including MNIST-O as described above, the fashion MNIST dataset (referred to as MNIST-F ) which has the same format as MNIST-O but includes images of fashion items instead of digits [41], and the CIFAR-10 dataset which includes 60 , 000 color images ( 50 , 000 for…”
MethodInfluentialLearning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms
Xiao, H., Rasul, K., & Vollgraf, R. (2017) · arXiv · doi:10.48550/arxiv.1708.07747 · Cited by 6,108
Large-Scale Machine Learning with Stochastic Gradient Descent
Bottou, L. (2010) · Proceedings of COMPSTAT'2010 · doi:10.1007/978-3-7908-2604-3_16 · Cited by 5,795
A Survey on Mobile Edge Computing: The Communication Perspective
Mao, Y., You, C., Zhang, J., Huang, K., & Letaief, K. B. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2745201 · Cited by 5,607
Federated Learning: Strategies for Improving Communication Efficiency
Konečný, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D. (2016) · arXiv · doi:10.48550/arxiv.1610.05492 · Cited by 3,766
Mobile Edge Computing: A Survey on Architecture and Computation Offloading
Mach, P., & Becvar, Z. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2682318 · Cited by 3,258
Understanding Machine Learning: From Theory to Algorithms
Shalev-Shwartz, S., & Ben-David, S. (2014) · doi:10.1017/cbo9781107298019 · Cited by 3,115
Fog and IoT: An Overview of Research Opportunities
Chiang, M., & Zhang, T. (2016) · IEEE Internet of Things Journal · doi:10.1109/jiot.2016.2584538 · Cited by 2,362
Federated Learning with Non-IID Data
Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., & Chandra, V. (2018) · arXiv · doi:10.48550/arxiv.1806.00582 · Cited by 2,077
Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge
Nishio, T., & Yonetani, R. (2019) · ICC 2019 - 2019 IEEE International Conference on Communications (ICC) · doi:10.1109/icc.2019.8761315 · Cited by 1,875
Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge
Nishio, T., & Yonetani, R. (2018) · arXiv · doi:10.48550/arxiv.1804.08333 · Cited by 1,875
Federated Optimization: Distributed Machine Learning for On-Device Intelligence
Konečný, J., McMahan, H. B., Ramage, D., & Richtárik, P. (2016) · arXiv · doi:10.48550/arxiv.1610.02527 · Cited by 1,563
Convex Optimization: Algorithms and Complexity
Bubeck, S. (2015) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000050 · Cited by 1,001
Scaling Distributed Machine Learning with the Parameter Server
Li, M. (2014) · Proceedings of the 2014 International Conference on Big Data Science and Computing · doi:10.1145/2640087.2644155 · Cited by 979
Data driven prediction models of energy use of appliances in a low-energy house
Candanedo, L. M., Feldheim, V., & Deramaix, D. (2017) · Energy and Buildings · doi:10.1016/j.enbuild.2017.01.083 · Cited by 554
Revisiting Distributed Synchronous SGD
Pan, X., Chen, J., Monga, R., Bengio, S., & Józefowicz, R. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1702.05800 · Cited by 548
A hierarchical edge cloud architecture for mobile computing
Tong, L., Li, Y., & Gao, W. (2016) · IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications · doi:10.1109/infocom.2016.7524340 · Cited by 518
Yu, H., Yang, S., & Zhu, S. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33015693 · Cited by 497
Federated Learning: Collaborative Machine Learning without Centralized Training Data
A, A. V., Binny, S., R, J. T., Raj, N., & Thomas, V. (2022) · international journal of engineering technology and management sciences · doi:10.46647/ijetms.2022.v06i05.052 · Cited by 489
Lian, X., Zhang, C., Zhang, H., Hsieh, C., Zhang, W., & Liu, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.09056 · Cited by 395
Asynchronous Decentralized Parallel Stochastic Gradient DescentPreprint
Lian, X., Zhang, W., Zhang, C., & Liu, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1710.06952 · Cited by 392
Wang, J., & Joshi, G. (2018) · arXiv.org · doi:10.48550/arxiv.1808.07576 · Cited by 365
Asynchronous Parallel Stochastic Gradient for Nonconvex Optimization
Lian, X., Huang, Y., Li, Y., & Liu, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.08272 · Cited by 324
Communication-efficient algorithms for statistical optimization
Zhang, Y., Duchi, J. C., & Wainwright, M. J. (2012) · 2012 IEEE 51st IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2012.6426691 · Cited by 283
Online job dispatching and scheduling in edge-clouds
Tan, H., Han, Z., Li, X.-Y., & Lau, F. C. M. (2017) · IEEE INFOCOM 2017 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2017.8057116 · Cited by 280
Asynchronous Stochastic Gradient Descent with Delay CompensationPreprint
Zheng, S., Meng, Q., Wang, T., Chen, W., Yu, N., Ma, Z.-M., & Liu, T. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.08326 · Cited by 278
Distributed delayed stochastic optimization
Agarwal, A., & Duchi, J. C. (2012) · 2012 IEEE 51st IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc.2012.6426626 · Cited by 243
QoE and power efficiency tradeoff for fog computing networks with fog node cooperation
Xiao, Y., & Krunz, M. (2017) · IEEE INFOCOM 2017 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2017.8057196 · Cited by 240
Dynamic Service Placement for Mobile Micro-Clouds with Predicted Future Costs
Wang, S., Urgaonkar, R., He, T., Chan, K., Zafer, M., & Leung, K. K. (2017) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2016.2604814 · Cited by 229
Self-citationWhen Edge Meets Learning: Adaptive Control for Resource-Constrained Distributed Machine Learning
Wang, S., Tuor, T., Salonidis, T., Leung, K. K., Makaya, C., He, T., & Chan, K. (2018) · IEEE INFOCOM 2018 - IEEE Conference on Computer Communications · doi:10.1109/infocom.2018.8486403 · Cited by 209
Self-citationDistributed optimization with arbitrary local solvers
Ma, C., Konečný, J., Jaggi, M., Smith, V., Jordan, M. I., Richtárik, P., & Takáč, M. (2017) · Optimization Methods and Software · doi:10.1080/10556788.2016.1278445 · Cited by 191
Adaptive Communication Strategies to Achieve the Best Error-Runtime Trade-off in Local-Update SGDPreprint
Wang, J., & Joshi, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1810.08313 · Cited by 175
Communication Complexity of Distributed Convex Learning and Optimization
Arjevani, Y., & Shamir, O. (2015) · arXiv · doi:10.48550/arxiv.1506.01900 · Cited by 133
The development of intuitive knowledge classifier and the modeling of domain dependent data
Kahraman, H. T., Sağıroğlu, Ş., & Çolak, İ. (2012) · Knowledge-Based Systems · doi:10.1016/j.knosys.2012.08.009 · Cited by 120
Online Resource Allocation for Arbitrary User Mobility in Distributed Edge Clouds
Wang, L., Jiao, L., Li, J., & Mühlhäuser, M. (2017) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2017.30 · Cited by 113
Wireless Network Intelligence at the Edge
Park, J., Samarakoon, S., Bennis, M., & Debbah, M. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2941458 · Cited by 92
Some Bounds for the Logarithmic Function
Topsøe, F. (2004) · Victoria University Research Repository (Victoria University) · Cited by 85
Distributed deep learning on edge-devices: Feasibility via adaptive compression
Hardy, C., Merrer, E. L., & Séricola, B. (2017) · IEEE International Symposium on Network Computing and Applications (NCA) · doi:10.1109/nca.2017.8171350 · Cited by 85
Application-aware traffic scheduling for workload offloading in mobile clouds
Liang, T., & Gao, W. (2016) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom.2016.7524520 · Cited by 39
Distributed Machine Learning in Coalition Environments: Overview of Techniques
Tuor, T., Wang, S., Leung, K. K., & Chan, K. (2018) · International Conference on Information Fusion (FUSION) · doi:10.23919/icif.2018.8455532 · Cited by 12
Self-citation
Research works printed but not linked: 6
- bonawitz, 2016, Practical secure aggregation for federated learning on user-held data, Proc NIPS Workshop Private Multi-Party Mach Learn
- kelly, 2015, Internet of Things data to top 1 6 zettabytes by 2020
- goodfellow, 2016, Deep Learning
- mcmahan, 2017, Communication-efficient learning of deep networks from decentralized data, Proc Int Conf Artif Intell Statist (AISTATS), 54, 1273
- 2019, Convolutional neural network
- hsieh, 2017, Gaia: Geo-distributed machine learning approaching LAN speeds, Proc USENIX Symp Netw Syst Design Implem (NSDI), 629
Cited by
Federated Optimization in Heterogeneous Networks
Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., & Smith, V. (2018) · arXiv · doi:10.48550/arxiv.1812.06127 · Cited by 4,023
Federated Learning in Mobile Edge Networks: A Comprehensive Survey
Lim, W. Y. B., Luong, N. C., Hoang, D. T., Jiao, Y., Liang, Y.-C., Yang, Q., Niyato, D., & Miao, C. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2986024 · Cited by 2,565
Edge Intelligence: Paving the Last Mile of Artificial Intelligence With Edge Computing
Zhou, Z., Chen, X., Li, E., Zeng, L., Luo, K., & Zhang, J. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2918951 · Cited by 2,455
“[51] propose a control algorithm that determines the best tradeoff between the local update and global parameter aggregation under a given resource budget.”
BackgroundFederated Learning With Differential Privacy: Algorithms and Performance Analysis
Wei, K., Li, J., Ding, M., Ma, C., Yang, H. H., Farokhi, F., Jin, S., Quek, T. Q. S., & Vincent Poor, H. (2020) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2020.2988575 · Cited by 2,400
“The work in [20] obtained a convergence bound for SGD based FL that incorporates non-independent-and-identically-distributed (non-i.”
BackgroundMethodA survey on federated learning
Zhang, C., Xie, Y., Bai, H., Yu, B., Li, W., & Gao, Y. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.106775 · Cited by 1,899
Metadata
- Authors
- Wang, S., Tuor, T., Salonidis, T., Leung, K. K., Makaya, C., He, T., & Chan, K.
- Affiliations
- IBM Research - Thomas J. Watson Research Center (US)
- Imperial College London (GB)
- Pennsylvania State University (US)
- DEVCOM Army Research Laboratory (US)
- Venue
- IEEE Journal on Selected Areas in Communications
- Published
- 2019-06
- Volume
- 37
- Issue
- 6
- Pages
- 1205-1221
- ISSN
- 0733-8716 (ISSN-L)
- 1558-0008
- OpenAlex
- W2963318081
- Cited by
- 2,232
- References
- 41
- Citations by year
- 2026: 189 · 2025: 317 · 2024: 341 · 2023: 369 · 2022: 366 · 2021: 374 · 2020: 234 · 2019: 39
- FWCI
- 115.53
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Licence
- Locations
- IEEE Journal on Selected Areas in Communications (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Spiral (Imperial College London) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Army Research Laboratory (W911NF-16-3-0001)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Computer Networks and Communications › IoT and Edge/Fog Computing
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Keywords
- adaptive federated learning, edge computing, resource-constrained systems, distributed gradient descent, convergence bounds, parameter aggregation, local updates, resource budget, privacy-preserving machine learning, Internet of Things
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.