The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Accelerating DNN Training in Wireless Federated Edge Learning Systems
Open accessAbstract
Training task in classical machine learning models, such as deep neural networks, is generally implemented at a remote cloud center for centralized learning, which is typically time-consuming and resource-hungry. It also incurs serious privacy issue and long communication latency since a large amount of data are transmitted to the centralized node. To overcome these shortcomings, we consider a newly-emerged framework, namely federated edge learning, to aggregate local learning updates at the network edge in lieu of users' raw data. Aiming at accelerating the training process, we first define a novel performance evaluation criterion, called learning efficiency. We then formulate a training acceleration optimization problem in the CPU scenario, where each user device is equipped with CPU. The closed-form expressions for joint batchsize selection and communication resource allocation are developed and some insightful results are highlighted. Further, we extend our learning framework to the GPU scenario. The optimal solution in this scenario is manifested to have the similar structure as that of the CPU scenario, recommending that our proposed algorithm is applicable in more general systems. Finally, extensive experiments validate the theoretical analysis and demonstrate that the proposed algorithm can reduce the training time and improve the learning accuracy simultaneously.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215
A Survey on Mobile Edge Computing: The Communication Perspective
Mao, Y., You, C., Zhang, J., Huang, K., & Letaief, K. B. (2017) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2017.2745201 · Cited by 5,607 · cited 1×
“To address the first issue, an innovative architecture called mobile edge computing (MEC) has been developed by implementing cloud computation capability at the network edge and migrating learning tasks from the cloud center to the edge server [8].”
Federated Learning in Mobile Edge Networks: A Comprehensive Survey
Lim, W. Y. B., Luong, N. C., Hoang, D. T., Jiao, Y., Liang, Y.-C., Yang, Q., Niyato, D., & Miao, C. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2986024 · Cited by 2,565
Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour
Goyal, P., Dollár, P., Girshick, R., Noordhuis, P., Wesolowski, L., Kyrola, A., Tulloch, A., Jia, Y., & He, K. (2017) · arXiv · doi:10.48550/arxiv.1706.02677 · Cited by 2,294 · cited 1×
“tation keeps fixed to guarantee convergence and learning accuracy [31], [32].”
BackgroundA Joint Learning and Communications Framework for Federated Learning over Wireless Networks
Chen, M., Yang, Z., Saad, W., Yin, C., Poor, H. V., & Cui, S. (2019) · arXiv · doi:10.48550/arxiv.1909.07972 · Cited by 1,547
A Joint Learning and Communications Framework for Federated Learning Over Wireless Networks
Chen, M., Yang, Z., Saad, W., Yin, C., Poor, H. V., & Cui, S. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3024629 · Cited by 1,547
Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Training
Lin, Y., Han, S., Mao, H., Wang, Y., & Dally, W. J. (2017) · arXiv · doi:10.48550/arxiv.1712.01887 · Cited by 1,134
Blockchained On-Device Federated Learning
Kim, H., Park, J., Bennis, M., & Kim, S.-L. (2020) · IEEE Communications Letters · doi:10.1109/lcomm.2019.2921755 · Cited by 887 · cited 1×
“Besides, a block-chained FEEL architecture was introduced in [23], where the optimal block generation rate was derived to minimize the end-to-end latency.”
Blockchained On-Device Federated Learning
Kim, H., Park, J., Bennis, M., & Kim, S.-L. (2018) · arXiv · doi:10.48550/arxiv.1808.03949 · Cited by 882 · cited 1×
“Particularly, a block-chained FL architecture has been introduced in [28], where the optimal block generation rate has been derived to minimize the end-to-end latency.”
Background
Research works printed but not linked: 22
- 2020, Technical Specification Group Radio Access Network Evolved Universal Terrestrial Radio Access (E-UTRA) Physical channels and modulation (Release 16)
- krizhevsky, 2014, One weird trick for parallelizing convolutional neural networks, arXiv 1404 5997
- 10.1145/2623330.2623612 · DOI · Paper page
- dean, 2012, Large scale distributed deep networks, Proc Adv Neural Inf Process Syst (NeurIPS), 1223
- geisser, 2006, Modes of Parametric Statistical Inference
- 10.1109/TWC.2016.2633522 · DOI · Paper page
- 10.1109/TWC.2017.2717986 · DOI · Paper page
- 10.1109/TWC.2019.2942929 · DOI
- 10.1109/MCOM.001.1900103 · DOI · Paper page
- kone?n?y, 2016, Federated learning: Strategies for improving communication efficiency, Proc NeurIPS Workshop Private Multi-Party Mach Learn, 1
- alistarh, 2017, QSGD: Communication-efficient sgd via gradient quantization and encoding, Proc Adv Neural Inf Process Syst (NeurIPS), 1707
- 10.1109/ICC.2016.7510807 · DOI
- 10.1109/COMST.2019.2904897 · DOI · Paper page
- 10.1109/INFOCOM.2019.8737464 · DOI · Paper page
- 10.1109/LWC.2017.2757490 · DOI · Paper page
- 10.1109/TVT.2019.2904244 · DOI · Paper page
- 10.1109/COMST.2019.2926625 · DOI · Paper page
- 10.1109/TWC.2018.2845360 · DOI · Paper page
- smith, 2017, Federated multi-task learning, Proc Adv Neural Inf Process Syst (NeurIPS), 4427
- zhao, 2018, Federated learning with non-IID data, arXiv 1806 00582
- 10.1109/TWC.2019.2946245 · DOI · Paper page
- 10.1109/JSAC.2019.2904348 · DOI · Paper page
Cited by
Federated Learning for Healthcare Informatics
Xu, J., Glicksberg, B. S., Su, C., Walker, P., Bian, J., & Wang, F. (2021) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-020-00082-4 · Cited by 1,542
Federated Learning for Healthcare Informatics
Xu, J., Glicksberg, B. S., Su, C., Walker, P., Bian, J., & Wang, F. (2019) · arXiv · doi:10.48550/arxiv.1911.06270 · Cited by 1,538
Edge artificial intelligence for 6g: vision, enabling technologies, and applications
Letaief, K. B., Shi, Y., Lu, J., & Lu, J. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3126076 · Cited by 829
Abdulrahman, S., Tout, H., Ould-Slimane, H., Mourad, A., Talhi, C., & Guizani, M. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3030072 · Cited by 796
Luo, S., Chen, X., Wu, Q., Zhou, Z., & Yu, S. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3003744 · Cited by 455
Hierarchical Federated Learning ACROSS Heterogeneous Cellular Networks
Abad, M. S. H., Ozfatura, E., GUndUz, D., & Ercetin, O. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054634 · Cited by 424
Convergence Time Optimization for Federated Learning Over Wireless Networks
Chen, M., Poor, H. V., Saad, W., & Cui, S. (2021) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3042530 · Cited by 403
“In [15], the authors define a metric to evaluate the FL performance and optimized the batchsize selection and communication resource allocation for the acceleration of the FL training process.”
BackgroundMulti-Armed Bandit-Based Client Scheduling for Federated Learning
Xia, W., Quek, T. Q. S., Guo, K., Wen, W., Yang, H. H., & Zhu, H. (2020) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2020.3008091 · Cited by 313
Fu, Y., Li, C., Yu, F. R., Luan, T. H., & Zhao, P. (2023) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2023.3297996 · Cited by 96
Zhou, Z., Yang, S., Pu, L., & Yu, S. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2984332 · Cited by 77
Li, L., Yang, L., Guo, X., Shi, Y., Wang, H., Chen, W., & Letaief, K. B. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118431 · Cited by 37
“, the training bottleneck caused by the slowest participating devices) [22]–[28], resource allocation for improving the transmission efficiency [27]–[30], adaptive aggregation for dynamically controlling the aggregation frequency [31] or calibrating aggregation weights [32], batchsize selection for improving learning…”
Background
Metadata
- Authors
- Ren, J., Yu, G., & Ding, G.
- Venue
- IEEE Journal on Selected Areas in Communications
- Published
- 2021-01
- Volume
- 39
- Issue
- 1
- Pages
- 219-232
- ISSN
- 0733-8716 (ISSN-L)
- 1558-0008
- OpenAlex
- W3110962653
- Cited by
- 226
- References
- 16
- FWCI
- 14.49
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.