The DOI matches its Crossref record; 25 of the 25 references it cites were found in a registry or an index.
Communication-Efficient Federated Deep Learning With Layerwise Asynchronous Model Update and Temporally Weighted Aggregation
Open accessAbstract
Federated learning obtains a central model on the server by aggregating models trained locally on clients. As a result, federated learning does not require clients to upload their data to the server, thereby preserving the data privacy of the clients. One challenge in federated learning is to reduce the client-server communication since the end devices typically have very limited communication bandwidth. This article presents an enhanced federated learning technique by proposing an asynchronous learning strategy on the clients and a temporally weighted aggregation of the local models on the server. In the asynchronous learning strategy, different layers of the deep neural networks (DNNs) are categorized into shallow and deep layers, and the parameters of the deep layers are updated less frequently than those of the shallow layers. Furthermore, a temporally weighted aggregation strategy is introduced on the server to make use of the previously trained local models, thereby enhancing the accuracy and convergence of the central model. The proposed algorithm is empirically on two data sets with different DNNs. Our results demonstrate that the proposed asynchronous federated deep learning outperforms the baseline algorithm both in terms of communication cost and model accuracy.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Boyd, S., Parikh, N., Chu, E., Peleato, B., & Eckstein, J. (2011) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000016 · Cited by 17,075
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215
Boyd, S. (2010) · now publishers, Inc. eBooks · doi:10.1561/9781601984616 · Cited by 12,677
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955 · cited 1×
“In recent years, DNNs have been successfully applied to many complex problem-solvings, including text classification, image classification, and speech recognition [4], [5], [6].”
MethodHow transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061
Shin, H.-C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., & Summers, R. M. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2528162 · Cited by 5,759
Learning to forget: Continual prediction with LSTM
Gers, F. A., Schmidhuber, J., & Cummins, F. (2000) · Neural Computation · doi:10.1162/089976600300015015 · Cited by 4,823
Large Scale Distributed Deep Networks
Dean, J. B., Corrado, G. S., Monga, R., Chen, K., Devin, M., Mao, M., Ranzato, M., Senior, A., Tucker, P. A., Yang, K., et al. (2012) · neural information processing systems · Cited by 4,489
Practical Secure Aggregation for Privacy-Preserving Machine Learning
Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., Ramage, D., Segal, A., & Seth, K. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3133982 · Cited by 3,956
Federated Learning: Strategies for Improving Communication Efficiency
Konečný, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D. (2016) · arXiv · doi:10.48550/arxiv.1610.05492 · Cited by 3,766
Learning to forget: continual prediction with LSTM
Gers, F. A. (1999) · 9th International Conference on Artificial Neural Networks: ICANN '99 · doi:10.1049/cp:19991218 · Cited by 2,337
(2013) · RECERCAT (Consorci de Serveis Universitaris de Catalunya) · Cited by 1,959
Federated Optimization: Distributed Machine Learning for On-Device Intelligence
Konečný, J., McMahan, H. B., Ramage, D., & Richtárik, P. (2016) · arXiv · doi:10.48550/arxiv.1610.02527 · Cited by 1,563
Differentially Private Federated Learning: A Client Level Perspective
Geyer, R. C., Klein, T., & Nabi, M. (2017) · arXiv · doi:10.48550/arxiv.1712.07557 · Cited by 1,052
One weird trick for parallelizing convolutional neural networks
Krizhevsky, A. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1404.5997 · Cited by 919
Project Adam: building an efficient and scalable deep learning training system
Chilimbi, T. M., Suzue, Y., Apacible, J., & Kalyanaraman, K. (2014) · Operating Systems Design and Implementation · doi:10.5555/2685048.2685094 · Cited by 550
Federated Optimization:Distributed Optimization Beyond the Datacenter
Konečný, J., McMahan, B., & Ramage, D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1511.03575 · Cited by 544
Communication-Efficient Distributed Optimization using an Approximate Newton-type Method
Shamir, O., Srebro, N., & Zhang, T. (2013) · arXiv · doi:10.48550/arxiv.1312.7853 · Cited by 424
Multi-Objective Evolutionary Federated Learning
Zhu, H., & Jin, Y. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2919699 · Cited by 301
Self-citationDiSCO: Distributed Optimization for Self-Concordant Empirical Loss
Zhang, Y., & Xiao, L. (2015) · International Conference on Machine Learning · Cited by 243
Distributed optimization with arbitrary local solvers
Ma, C., Konečný, J., Jaggi, M., Smith, V., Jordan, M. I., Richtárik, P., & Takáč, M. (2017) · Optimization Methods and Software · doi:10.1080/10556788.2016.1278445 · Cited by 191
AIDE: Fast and Communication Efficient Distributed OptimizationPreprint
Reddi, S. J., Konečný, J., Richtárik, P., Póczos, B., & Smola, A. J. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1608.06879 · Cited by 171
Cited by
A review of applications in federated learning
Li, L., Fan, Y., Tse, M., & Lin, K.-Y. (2020) · Computers & Industrial Engineering · doi:10.1016/j.cie.2020.106854 · Cited by 1,559
Federated Learning for Healthcare Informatics
Xu, J., Glicksberg, B. S., Su, C., Walker, P., Bian, J., & Wang, F. (2019) · arXiv · doi:10.48550/arxiv.1911.06270 · Cited by 1,538
Federated Learning for Healthcare Informatics
Xu, J., Glicksberg, B. S., Su, C., Walker, P., Bian, J., & Wang, F. (2021) · Journal of Healthcare Informatics Research · doi:10.1007/s41666-020-00082-4 · Cited by 1,542
A survey on security and privacy of federated learning
Mothukuri, V., Parizi, R. M., Pouriyeh, S., Huang, Y., Dehghantanha, A., & Srivastava, G. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2020.10.007 · Cited by 1,480
Towards Personalized Federated Learning
Tan, A. Z., Yu, H., Cui, L., & Yang, Q. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3160699 · Cited by 1,182
Metadata
- Authors
- Chen, Y., Sun, X., & Jin, Y.
- Venue
- IEEE Transactions on Neural Networks and Learning Systems
- Published
- 2020-10
- Volume
- 31
- Issue
- 10
- Pages
- 4229-4238
- ISSN
- 2162-237X (ISSN-L)
- 2162-2388
- OpenAlex
- W2998045710
- Cited by
- 573
- References
- 25
- FWCI
- 28.66
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.