Joint Parameter-and-Bandwidth Allocation for Improving the Efficiency of Partitioned Edge Learning
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
To leverage data and computation capabilities of mobile devices, machine learning algorithms are deployed at the network edge for training artificial intelligence (AI) models, resulting in the new paradigm of edge learning. In this paper, we consider the framework of partitioned edge learning for iteratively training a large-scale model using many resource-constrained devices (called workers). To this end, in each iteration, the model is dynamically partitioned into parametric blocks, which are downloaded to worker groups for updating using data subsets. Then, the local updates are uploaded to and cascaded by the server for updating a global model. To reduce resource usage by minimizing the total learning-and-communication latency, this work focuses on the novel joint design of parameter (computation load) allocation and bandwidth allocation (for downloading and uploading). Two design approaches are adopted. First, a practical sequential approach, called partially integrated parameter-and-bandwidth allocation (PABA), yields two schemes, namely bandwidth aware parameter allocation and parameter aware bandwidth allocation. The former minimizes the load for the slowest (in computing) of worker groups, each training a same parametric block. The latter allocates the largest bandwidth to the worker being the latency bottleneck. Second, PABA are jointly optimized. Despite it being a nonconvex problem, an efficient and optimal solution algorithm is derived by intelligently nesting a bisection search and solving a convex problem. Experimental results using real data demonstrate that integrating PABA can substantially improve the performance of partitioned edge learning in terms of latency (by e.g., 46%) and accuracy (by e.g., 4% given the latency of 100 seconds).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Parikh, N., & Boyd, S. (2014) · Foundations and Trends® in Optimization · doi:10.1561/2400000003 · Cited by 4,102
Federated Learning in Mobile Edge Networks: A Comprehensive Survey
Lim, W. Y. B., Luong, N. C., Hoang, D. T., Jiao, Y., Liang, Y.-C., Yang, Q., Niyato, D., & Miao, C. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2986024 · Cited by 2,565
A Joint Learning and Communications Framework for Federated Learning over Wireless Networks
Chen, M., Yang, Z., Saad, W., Yin, C., Poor, H. V., & Cui, S. (2019) · arXiv · doi:10.48550/arxiv.1909.07972 · Cited by 1,547
A Joint Learning and Communications Framework for Federated Learning Over Wireless Networks
Chen, M., Yang, Z., Saad, W., Yin, C., Poor, H. V., & Cui, S. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3024629 · Cited by 1,547
Wright, S. J. (2015) · Mathematical Programming · doi:10.1007/s10107-015-0892-3 · Cited by 1,438
Energy Efficient Federated Learning Over Wireless Communication Networks
Yang, Z., Chen, M., Saad, W., Hong, C. S., & Shikh-Bahaei, M. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3037554 · Cited by 1,153
Federated Learning via Over-the-Air Computation
Yang, K., Jiang, T., Shi, Y., & Ding, Z. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2019.2961673 · Cited by 1,084 · cited 1×
“ion, many techniques are designed for efficient transportation of high-dimensional data over wireless channels. New multi-access schemes for FEEL, called “over-the-air computing”, are proposed in [16]–[18] to support fast 4 “over-the-air” model/gradient aggregation using the waveform superposition property of a multiac…”
BackgroundSparse Communication for Distributed Gradient Descent
Aji, A. F., & Heafield, K. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1045 · Cited by 585
Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air
Mohammadi Amiri, M., & Gunduz, D. (2020) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2020.2981904 · Cited by 548
Hierarchical Federated Learning ACROSS Heterogeneous Cellular Networks
Abad, M. S. H., Ozfatura, E., GUndUz, D., & Ercetin, O. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054634 · Cited by 424
Research works printed but not linked: 16
- 10.1109/CVPR.2016.165 · DOI
- chen, 2016, Revisiting distributed synchronous SGD, arXiv 1604 00981
- kamp, 2018, Efficient decentralized deep learning by dynamic model averaging, Proc European Conf Machine Learning and Knowledge Discovery in Databases (ECML PKDD), 394
- chen, 2018, Lag: Lazily aggregated gradient for communication-efficient distributed learning, Proc Conf Neural Inf Process Syst (NIPS), 5050
- 10.1109/TWC.2019.2946245 · DOI · Paper page
- 10.1016/B978-1-55860-377-6.50048-7 · DOI · Paper page
- li, 2013, Parameter server for distributed machine learning, NIPS Big Learning Workshop, 2
- 10.1145/2640087.2644155 · DOI · Paper page
- ho, 2013, More effective distributed ml via a stale synchronous parallel parameter server, Proc Conf Neural Inf Process Syst (NIPS), 1223
- 10.1145/3343180.3343192 · doi:10.1145/3343180.3343192
- 10.1109/MCOM.001.1900103 · DOI · Paper page
- carreira-perpinan, 2014, Distributed optimization of deeply nested systems, Proc Int Workshop Artif Intell Statist (AISTATS), 10
- 10.1109/JSAC.2019.2934006 · DOI
- 10.1109/TCOMM.2019.2944169 · DOI · Paper page
- ren, 2019, Accelerating DNN training in wireless federated edge learning system, arXiv 1905 09712
- ruder, 2016, An overview of gradient descent optimization algorithms, arXiv 1609 04747
Cited by
Edge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
“Specifically, for edge training systems, we need to jointly optimize the subcarrier and bandwidth allocation [241], [88], [90], transmit power”
MethodLi, L., Yang, L., Guo, X., Shi, Y., Wang, H., Chen, W., & Letaief, K. B. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118431 · Cited by 37
“, the training bottleneck caused by the slowest participating devices) [22]–[28], resource allocation for improving the transmission efficiency [27]–[30], adaptive aggregation for dynamically controlling the aggregation frequency [31] or calibrating aggregation weights [32], batchsize selection for improving learning…”
BackgroundFast Convergence Algorithm for Analog Federated Learning
Xia, S., Zhu, J., Yang, Y., Zhou, Y., Shi, Y., & Chen, W. (2021) · ICC 2021 - IEEE International Conference on Communications · doi:10.1109/icc42927.2021.9500875 · Cited by 33
Metadata
- Authors
- Wen, D., Bennis, M., & Huang, K.
- Venue
- IEEE Transactions on Wireless Communications
- Published
- 2020-12
- Volume
- 19
- Issue
- 12
- Pages
- 8272-8286
- ISSN
- 1536-1276 (ISSN-L)
- 1558-2248
- OpenAlex
- W3086073975
- Cited by
- 64
- References
- 13
- FWCI
- 4.71
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- Funders
- Hong Kong Research Grants Council (17208319, 17209917)
- Innovation and Technology Fund (GHP/016/18GD, 17209917)
- Guangdong Basic and Applied Basic Research Foundation (2019B1515130003, 17209917)
- European Union CHIST-ERA (EU-CHISTERA) Program projects CONNECT and LeadingEdge
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Joint Parameter-and-Bandwidth Allocation for Improving the Efficiency of Partitioned Edge Learning. https://app.cochat.ai/papers/10.1109/twc.2020.3021177
- The paper
- Wen, D., Bennis, M., & Huang, K. (2020). Joint Parameter-and-Bandwidth Allocation for Improving the Efficiency of Partitioned Edge Learning. IEEE Transactions on Wireless Communications. https://doi.org/10.1109/twc.2020.3021177
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.