Sparse Communication for Distributed Gradient Descent
Open accessThe DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
We make distributed stochastic gradient descent faster by exchanging sparse updates instead of dense updates.Gradient updates are positively skewed as most updates are near zero, so we map the 99% smallest updates (by absolute value) to zero then exchange sparse matrices.This method can be combined with quantization to further improve the compression.We explore different configurations and apply them to neural machine translation and MNIST image classification tasks.Most configurations work on MNIST, whereas different configurations reduce convergence rate on the more complex translation task.Our experiments show that we can achieve up to 49% speed up on MNIST and 22% on NMT without damaging the final accuracy or BLEU.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures








Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Sparsification, on the other hand, proposes to only communicate the informative elements of the gradient or model vectors among nodes [128], [129].”
From Edge artificial intelligence for 6g: vision, enabling technologies, and applications (2022) · cites this paper as [129]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 16
- Tolu Alabi, Jeffrey D Blanchard, Bradley Gordon, and Russel Steinbach. 2012. Fast k-selection algorithms for graphics processing units. Journal of Experi- mental Algorithmics (JEA) 17:4-2.
- Dan Alistarh, Jerry Li, Ryota Tomioka, and Milan Vojnovic. 2016. QSGD: randomized quan- tization for communication-optimal stochas- tic gradient descent. CoRR abs/1610.02132. http://arxiv.org/abs/1610.02132. · arXiv
- Trishul M Chilimbi, Yutaka Suzue, Johnson Apacible, and Karthik Kalyanaraman. 2014. Project adam: Building an efficient and scalable deep learning training system. In OSDI. volume 14, pages 571- 582.
- Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Andrew Senior, Paul Tucker, Ke Yang, Quoc V Le, et al. 2012. Large scale distributed deep networks. In Advances in neural information processing systems. pages 1223- 1231.
- Nikoli Dryden, Sam Ade Jacobs, Tim Moon, and Brian Van Essen. 2016. Communication quantization for data-parallel training of deep neural networks. In Proceedings of the Workshop on Machine Learn- ing in High Performance Computing Environments. IEEE Press, pages 1-8.
- Marcin Junczys-Dowmunt, Tomasz Dwojak, and Hieu Hoang. 2016. Is neural machine translation ready for deployment? A case study on 30 translation directions. In Program of the 13th International Workshop on Spoken Language Translation (IWSLT 2016).
- Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. 1998. Gradient-based learning applied to document recognition. Proceedings of the IEEE 86(11):2278-2324.
- J. Lei Ba, J. R. Kiros, and G. E. Hinton. 2016. Layer Normalization. ArXiv e-prints .
- Ramón P Ñeco and Mikel L Forcada. 1996. Beyond mealy machines: Learning translators with recurrent neural networks. In Proceedings of the 1996 Inter- national Neural Network Society Annual Meeting. San Diego, California, USA.
- Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. BLEU: A method for automatic evalution of machine translation. In Proceedings 40th Annual Meeting of the Association for Com- putational Linguistics. Philadelphia, PA, pages 311- 318.
- Rajat Raina, Anand Madhavan, and Andrew Y Ng. 2009. Large-scale deep unsupervised learning using graphics processors. In Proceedings of the 26th an- nual international conference on machine learning. ACM, pages 873-880.
- Benjamin Recht, Christopher Re, Stephen Wright, and Feng Niu. 2011. Hogwild: A lock-free approach to parallelizing stochastic gradient de- scent. In J. Shawe-Taylor, R. S. Zemel, P. L. Bartlett, F. Pereira, and K. Q. Weinberger, editors, Advances in Neural Information Processing Sys- tems 24, Curran Associates, Inc., pages 693-701. http://papers.nips.cc/paper/4390-hogwild-a-lock- free-approach-to-parallelizing-stochastic-gradient- descent.pdf.
- Frank Seide, Hao Fu, Jasha Droppo, Gang Li, and Dong Yu. 2014. 1-bit stochastic gradi- ent descent and application to data-parallel distributed training of speech DNNs. In Interspeech. https://www.microsoft.com/en- us/research/publication/1-bit-stochastic-gradient- descent-and-application-to-data-parallel- distributed-training-of-speech-dnns/.
- Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Edinburgh neural machine translation sys- tems for WMT 16. In Proceedings of the ACL 2016 First Conference on Machine Translation (WMT16).
- Martin Zinkevich, Markus Weimer, Lihong Li, and Alex J Smola. 2010. Parallelized stochastic gradient descent. In Advances in neural information process- ing systems. pages 2595-2603.
- Daniel Zwillinger and Stephen Kokoska. 1999. CRC standard probability and statistics tables and formu- lae. CRC Press.
Cited by
Items 501–600 of 783
Sparse Training for Federated Learning With Regularized Error Correction
Greidi, R., & Cohen, K. (2024) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2024.3463505 · Cited by 6
FedMQ: Multi-grained Quantization for Heterogeneous Federated Learning
Mei, C., Zhang, T., Zhang, B., Lǚ, J., Shen, Z., & Zhao, M. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-71467-2_1 · Cited by 3
Sparsified Random Partial Model Update for Personalized Federated Learning
Hu, X., Chen, Z., Feng, C., Min, G., Quek, T. Q. S., & Yang, H. H. (2024) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2024.3507286 · Cited by 3
Zhao, H. V., Yue, P., & Fang, C. (2026) · IEEE Transactions on Automatic Control · doi:10.1109/tac.2026.3677732 · Cited by 3
Adaptive Scheduling for Machine Learning Tasks over Networks
Gatsis, K. (2021) · Proceedings of the ... American Control Conference/Proceedings of the American Control Conference · doi:10.23919/acc50511.2021.9482692 · Cited by 4
Communication-Memory-Efficient Decentralized Learning For Audio Representation
Li, L., Wang, J., Qu, X., & Xiao, J. Z. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534356 · Cited by 3
AI-Powered IoT System at the Edge
Chen, Y., Li, A., Yang, H., Zhang, T., Yang, Y., Li, H. H., Banerjee, S., & Pajić, M. (2021) · IEEE International Conference on Cognitive Machine Intelligence (CogMI) · doi:10.1109/cogmi52975.2021.00039 · Cited by 3
Double Perturbation-Based Privacy-Preserving Federated Learning against Inference Attack
Jiang, Y., Shi, Y., & Chen, S. (2022) · Globecom · doi:10.1109/globecom48099.2022.10001075 · Cited by 4
SuperNeurons: FFT-based Gradient Sparsification in the Distributed Training of Deep Neural NetworksPreprint
Wang, L., Wu, W., Zhang, J., Liu, H., Bosilca, G., Herlihy, M. P., & Fonseca, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.08596 · Cited by 3
Accelerated Sparsified SGD with Error FeedbackPreprint
Murata, T., & Suzuki, T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.12224 · Cited by 3
LASER: Linear Compression in Wireless Distributed OptimizationPreprint
Makkuva, A. V., Bondaschi, M., Vogels, T., Jaggi, M., Kim, H., & Gastpar, M. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2310.13033 · Cited by 6
Multimodal Federated Learning: A Survey through the Lens of Different FL ParadigmsPreprint
Peng, Y., Bian, J., Wang, L., Huang, Y., & Xu, J. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2505.21792 · Cited by 6
Du, R., He, D., Ding, Z., Wang, M., Chan, S., & Li, X. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3403178 · Cited by 4
Near-Lossless Gradient Compression for Data-Parallel Distributed DNN Training
Li, X., Guo, C., Qian, K., Zhang, M., Yang, M., & Xu, M. (2024) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/3698038.3698541 · Cited by 2
Gao, J., Li, Y., Wang, J., Lepech, M. D., Zhao, Y., & Campbell, B. (2026) · npj Artificial Intelligence · doi:10.1038/s44387-026-00155-6 · Cited by 6
Machine learning for sensory data analytics
Guo, Z., Ye, M., & Tan, J. (2025) · Elsevier eBooks · doi:10.1016/b978-0-443-21640-4.00013-2 · Cited by 2
Alleviating straggler impacts for data parallel deep learning with hybrid parameter update
Li, H., Tian, Q., Xu, D. H., Zhao, H., & Xu, Z. (2025) · Future Generation Computer Systems · doi:10.1016/j.future.2025.107775 · Cited by 3
Federated learning for connected autonomous vehicles: enhancing mobility, safety, and sustainability
Ali, A., Huang, J., Jabbar, A., Ali, M., & Ali, S. (2025) · Physica Scripta · doi:10.1088/1402-4896/addca2 · Cited by 4
NetSenseML: Network-Adaptive Compression for Efficient Distributed Machine Learning
Wang, Y., Li, X., Wu, R., Chen, H., & Kutscher, D. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-99872-0_20 · Cited by 2
FedBand: Adaptive Federated Learning Under Strict Bandwidth Constraints
Alanazi, T., Fahim, A., Ibnath, M., Güler, B., Chowdhury, A. R., Swami, A., Papalexakis, E. E., & Krishnamurthy, S. V. (2025) · International Conference on Computer Communications and Networks (ICCCN) · doi:10.1109/icccn65249.2025.11133779 · Cited by 2
Failure-Aware Tasking for Teams of Drones
Diller, J., Teoh, Y. S., Byers, R., Han, Q., Rogers, J. G., & Dantam, N. T. (2025) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs63083.2025.00043 · Cited by 2
Hou, Z., Yan, K., Qiu, Y., Wu, Q., & Liu, Z. (2025) · Electronics · doi:10.3390/electronics14224405 · Cited by 2
Federated Unlearning with Oriented Saliency Compression
Xiao, B., Liu, S., & Ling, Q. H. (2025) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn64981.2025.11228643 · Cited by 2
Yan, G., Xie, L., Shen, Q., Fang, Y., & Wu, Z. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792144 · Cited by 6
GUIDE: Enhancing Gradient Inversion Attacks in Federated Learning With Denoising Models
Carletti, V., Foggia, P., Mazzocca, C., Parrella, G., & Vento, M. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3726586 · Cited by 2
Information Leakage and Privacy Protection in Industrial Federated Learning
Lin, Z., Jiang, X., Zhang, X., Yao, L., & Song, Z. (2025) · IEEE Transactions on Systems Man and Cybernetics Systems · doi:10.1109/tsmc.2025.3633589 · Cited by 2
Computation and Communication Resource Optimization for Efficient Hierarchical Federated Learning
Liu, C., Zhao, J., & Wang, Y. L. (2025) · IEEE Transactions on Networking · doi:10.1109/ton.2025.3600163 · Cited by 2
Amber: Towards Fast and Space-Efficient Incremental Checkpointing in Large Language Model Training
Wang, Z., Zhu, W., Zhang, Z., Yan, C., Guo, F., Li, Y., & Xu, Y. (2025) · Proceedings of the International Conference on Parallel Processing · doi:10.1145/3754598.3754606 · Cited by 2
Abbass, W., Abbas, N., Majeed, U., & Hassan, M. E. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-08162-1 · Cited by 2
Liu, Y., Rini, S., & Chen, J. (2026) · IEEE Internet of Things Journal · doi:10.1109/jiot.2026.3687127 · Cited by 2
Unfederated: Open Challenges, Deployment Gaps, and Emerging Directions in Federated Learning
Waref, D., Alayary, Y., Ghany, M. A. A. E., & Salem, M. A. ‐. (2026) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-026-10696-3 · Cited by 3
Wu, L., Yue, H., Chen, P., Wu, D., & Qiu, J. (2019) · IEEE Access · doi:10.1109/access.2019.2926993 · Cited by 3
Grouping Synchronous to Eliminate Stragglers with Edge Computing in Distributed Deep Learning
Gui, Z., Yang, X., Yang, H., Li, W., Zhang, L., Qi, Q., Wang, J., Sun, H., & Liao, J. (2021) · IEEE International Symposium on Parallel and Distributed Processing with Applications (ISPA) · doi:10.1109/ispa-bdcloud-socialcom-sustaincom52081.2021.00066 · Cited by 2
Adaptive sparse ternary gradient compression for distributed DNN training in edge computing
Mao, Y., Wu, J., Xu, X., & Wang, L. (2022) · CCF Transactions on High Performance Computing · doi:10.1007/s42514-022-00091-2 · Cited by 2
BS-pFL: Enabling Low-Cost Personalized Federated Learning by Exploring Weight Gradient Sparsity
Wang, L., Sistla, M., Chen, M., & Fu, X. (2022) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn55064.2022.9892137 · Cited by 2
Compressing Model before Federated Learning by Transferrable Surrogate Lottery Ticket
Tanimura, T., & Takase, M. (2023) · IEEE Consumer Communications and Networking Conference · doi:10.1109/ccnc51644.2023.10060578 · Cited by 2
An Overview of Energy-Efficient DNN Training Processors
Han, D., & Yoo, H. (2023) · Springer eBooks · doi:10.1007/978-3-031-34237-0_8 · Cited by 3
Online Distributed Optimization with Efficient Communication via Temporal Similarity
Wang, J., Liang, B., Dong, M., Boudreau, G., & Afana, A. (2023) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom53939.2023.10229086 · Cited by 5
Wang, J., & Guo, S. (2023) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2023.3328931 · Cited by 3
Fast On-Device Learning Framework for Single-Image Super-Resolution
Lee, S. H., Park, K., Cho, S., Lee, H.-S., Choi, K., & Cho, N. I. (2024) · IEEE Access · doi:10.1109/access.2024.3375120 · Cited by 3
Near-Linear Scaling Data Parallel Training with Overlapping-Aware Gradient Compression
Meng, L., Sun, Y., & Li, W. (2023) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads60453.2023.00011 · Cited by 4
Communication-Efficient Distributed Learning via Sparse and Adaptive Stochastic Gradient
Deng, X., Li, D., Sun, T., & Lu, X. (2024) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2024.3407510 · Cited by 2
Approximate Wireless Communication for Lossy Gradient Updates in IoT Federated Learning
Ma, X., Sun, H., Hu, R. Q., & Qian, Y. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3488377 · Cited by 3
Communication-Efficient Federated Learning via Sparse Training with Regularized Error Correction
Greidi, R., & Cohen, K. (2024) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton63246.2024.10735294 · Cited by 2
Energy-Efficient Deep Learning Training
Guan, L., Zhang, S., & Chen, Y. (2024) · IntechOpen eBooks · doi:10.5772/intechopen.1007491 · Cited by 4
Sinaei, S., Mohammadi, M., Shrestha, R., Alibeigi, M., & Eklund, D. (2024) · 2024 27th Euromicro Conference on Digital System Design (DSD) · doi:10.1109/dsd64264.2024.00063 · Cited by 2
A Method for Parameter Updating in Distributed Deep Learning
He, M., Guo, X., Wang, D., & Qin, S. (2024) · International Conference on Next Generation Data-driven Networks (NGDN) · doi:10.1109/ngdn61651.2024.10744125 · Cited by 2
Veeraragavan, N. R., Auensen, S., Knoors, D., & Nygård, J. F. (2024) · International Conference on Federated Learning Technologies and Applications (FLTA) · doi:10.1109/flta63145.2024.10840145 · Cited by 2
Gao, W., Zheng, P., Hu, Y., Xu, L., & Schmeink, A. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3620877 · Cited by 3
Liu, B., Jiang, W., Zhao, S., Jin, H., & He, B. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-64243-3_6 · Cited by 2
Federated Learning Communication-Efficiency Framework via Corset Construction
Li, K., & Wang, H. (2022) · The Computer Journal · doi:10.1093/comjnl/bxac062 · Cited by 2
Approximate Computing for Scientific Applications
Anzt, H., Casas, M., Malossi, A. C. I., Quintana–Ort́ı, E. S., Scheidegger, F., & Zhuang, S. (2022) · Springer eBooks · doi:10.1007/978-3-030-94705-7_14 · Cited by 2
FNCS: Federated Learning Strategy Based on Cosine Similarity under Resource Constraints
Li, R., Qin, Y., & Zang, L. (2021) · IEEE International Conference on High Performance Computing and Communications; IEEE International Conference on Smart City; IEEE International Conference on Data Science and Systems (HPCC/SmartCity/DSS) · doi:10.1109/hpcc-dss-smartcity-dependsys53884.2021.00114 · Cited by 2
ASTC: An Adaptive Gradient Compression Scheme for Communication-Efficient Edge Computing
Wu, J., Mao, Y., & Xu, X. (2021) · IEEE International Conference on High Performance Computing and Communications; IEEE International Conference on Smart City; IEEE International Conference on Data Science and Systems (HPCC/SmartCity/DSS) · doi:10.1109/hpcc-dss-smartcity-dependsys53884.2021.00142 · Cited by 2
Communication-efficient Federated Learning with an Event-triggering Strategy
Li, Y., Bai, J., Li, D., & Li, W. (2022) · IEEE Data Driven Control and Learning Systems Conference (DDCLS) · doi:10.1109/ddcls55054.2022.9858572 · Cited by 3
CoBAAF: Controlled Bayesian Air Aggregation Federated Learning from Heterogeneous Data
Gafni, T., Cohen, K., & Eldar, Y. C. (2022) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton49937.2022.9929426 · Cited by 2
Sparsification on Different Federated Learning Schemes: Comparative Analysis
Lee, J., Jang, Y., Lee, J., & Kang, J. (2022) · International Conference on Information and Communication Technology Convergence (ICTC) · doi:10.1109/ictc55196.2022.9952431 · Cited by 2
Secure Sparse Gradient Aggregation in Distributed Architectures
van Rooij, M., van Rooij, S. B., Bouma, H., & Pimentel, A. D. (2022) · International Conference on Internet of Things: Systems, Management and Security (IOTSMS) · doi:10.1109/iotsms58070.2022.10062180 · Cited by 2
Cai, Y., Li, S., Xia, Z., & Tan, Y. (2022) · IEEE International Conference on Control Science and Systems Engineering (ICCSSE) · doi:10.1109/iccsse55346.2022.10079164 · Cited by 2
Adaptive Ratio-Based-Threshold Gradient Sparsification Scheme for Federated Learning
Kong, J. M., & Sousa, E. S. (2023) · International Symposium on Networks, Computers and Communications (ISNCC) · doi:10.1109/isncc58260.2023.10323644 · Cited by 2
Towards Heterogeneous Clients with Elastic Federated LearningPreprint
Ma, Z., Lu, Y. L., Lu, Z., Li, W., Yi, J., & Cui, S. R. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09433 · Cited by 3
Distributed and Stochastic Optimization Methods with Gradient Compression and Local StepsPreprint
Gorbunov, E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2112.10645 · Cited by 2
Peering Beyond the Gradient Veil with Distributed Auto DifferentiationPreprint
Baker, B. T., Khanal, A., Calhoun, V. D., Pearlmutter, B. A., & Plis, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.09631 · Cited by 2
Federated Learning of Artificial Neural Networks
Kiss, P. T. (2022) · Eötvös Loránd Tudományegyetem · doi:10.15476/elte.2021.124 · Cited by 2
An Efficient Gradient-Aware Error-Bounded Lossy Compressor for Federated LearningPreprint
Ye, Z., Di, S., Wang, J., Zhong, Z., Zhang, Z., & Yu, X. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2511.05770 · Cited by 3
SiFAR: Synchronization-Free All-Reduce for Low-Latency LLM InferencePreprint
Taneja, H., Saxena, A., Revinipati, A., Ju, J. H., Crago, N. C., & Qureshi, M. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2607.08973 · Cited by 2
Bridging the Gap Between Memory and Communication Efficiency on Distributed Deep Learning Systems
Zhao, S., Liu, B., Wang, F., & Feng, D. (2021) · IEEE Access · doi:10.1109/access.2021.3071579 · Cited by 1
SAGE: toward on-the-fly gradient compression ratio scaling
Yoon, D., Jeong, M., & Oh, S. (2023) · The Journal of Supercomputing · doi:10.1007/s11227-023-05120-7 · Cited by 1
AOPT-FL: A Communication-Efficient Federated Learning Method with Clusterd and Sparsification
Zhang, D., Xia, G., & Liu, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-0862-8_20 · Cited by 1
Lossy Lempel-Ziv Coding for Federated Learning
Zhong, H., Liang, K., & Wu, Y. (2023) · IEEE Globecom Workshops (GC Wkshps) · doi:10.1109/gcwkshps58843.2023.10464660 · Cited by 1
More communication-efficient distributed sparse learning
Zhou, X., & Yang, G. (2024) · Information Sciences · doi:10.1016/j.ins.2024.120523 · Cited by 1
Advances in Energy Recovery and Efficiency Technologies
Ismail, B. I. A. (2024) · IntechOpen eBooks · doi:10.5772/intechopen.1003402 · Cited by 2
Dual Adaptive Compression for Efficient Communication in Heterogeneous Federated Learning
Mao, Y., Wang, Z., Li, C., Zhang, J., Xu, S., & Wu, J. (2024) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid59990.2024.00035 · Cited by 1
Amplified Gradient Inversion Attacks on Federated Learning Frameworks
Tu, T. A., Thanh, D. C., & Tran, D. S. (2024) · Journal of Science and Technology on Information security · doi:10.54654/isj.v3i23.1066 · Cited by 1
Lossy Compression for Sparse Aggregation
Fan, Y., Ye, F., & Yeung, R. W. (2024) · IEEE Information Theory Workshop (ITW) · doi:10.1109/itw61385.2024.10806992 · Cited by 1
FedSSA: Reducing Overhead of Additive Cryptographic Methods in Federated Learning With Sketch
Ou, Z., Han, S., Zeng, Q., & Huang, Q. (2024) · IEEE International Conference on Network Protocols (ICNP) · doi:10.1109/icnp61940.2024.10858511 · Cited by 1
Federated Average Clustering Learning Based on Time-Asynchronous Similarity
Zheng, X., Li, H., Wu, D., Bu, T., Hu, P., & Zhang, J. (2025) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2025.3566605 · Cited by 2
SwitchTop- k : Scaling Top- k Compression on Programmable Switches
Li, Y., Huang, J., Liu, J., Li, Z., Jiang, W., & Wang, J. (2025) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3711896.3737142 · Cited by 3
GIDM: Gradient Inversion of Federated Diffusion Models
Huang, J., Hong, C., Roos, S., & Chen, L. Y. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-00624-0_19 · Cited by 1
Federated Learning Framework with Personalized Model Compression and Privacy Protection
Ding, C., Xiao, M., Xu, Y., & Wu, J. (2025) · International Conference on Computer Communications and Networks (ICCCN) · doi:10.1109/icccn65249.2025.11133897 · Cited by 1
UAV-Assisted Heterogeneous Federated Learning over the Air Against Byzantine Attacks
Fan, X., Li, G., Luo, C., Hong, Y., Chen, Z., Chen, T., & Huo, Y. (2025) · Tsinghua Science & Technology · doi:10.26599/tst.2024.9010167 · Cited by 1
Adaptive Sparsification for Communication-Efficient Distributed Learning
Kalarde, F. M., Liang, B., Dong, M., Ahmed, Y., & Cheng, H. T. (2025) · ACM International Symposium on Theory, Algorithmic Foundations, and Protocol Design for Mobile Networks and Mobile Computing (MobiHoc) · doi:10.1145/3704413.3764446 · Cited by 1
Regularized Top-$ k $: A Bayesian Framework for Gradient Sparsification
Bereyhi, A., Liang, B., Boudreau, G., & Afana, A. (2025) · IEEE Transactions on Signal Processing · doi:10.1109/tsp.2025.3624791 · Cited by 1
Enhancing Federated Learning in Edge Computing With Secure Aggregation and Dynamic Verification
Li, M., Tian, Y., Zhang, J., Xiong, J., Cui, X., & Ma, J. (2025) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2025.3625654 · Cited by 1
Hidayaturrahman, H. (2025) · International Journal of Computer Science and Humanitarian AI · doi:10.21512/ijcshai.v2i2.14533 · Cited by 1
Communication Efficient Over-the-Air Federated Learning With Random FLARE Algorithm
Seo, Y., Lim, H., & Yu, N. Y. (2025) · IEEE Signal Processing Letters · doi:10.1109/lsp.2025.3633592 · Cited by 1
A multi-threaded federated learning framework for reducing communication epochs
Luo, C.-Y., Tian, S., Li, F., Chen, X., Dai, Q., & Liu, J.-Q. (2025) · Applied Soft Computing · doi:10.1016/j.asoc.2025.114247 · Cited by 2
Federated Drift-Aware Graph Neural Forecasting for Real-Time Passenger Flow
Сахипов, А., Uzdenbayev, Z., Begisbayev, D., Mektepbayeva, A., Seiitbek, R., & Yedilkhan, D. (2025) · Procedia Computer Science · doi:10.1016/j.procs.2025.10.185 · Cited by 1
Trustworthy Efficient Communication for Distributed Learning Using LQ-SGD Algorithm
Li, H., Bai, L., Wu, C., Chadli, M., Mammar, S., & Bouvry, P. (2025) · European Signal Processing Conference (EUSIPCO) · doi:10.23919/eusipco63237.2025.11226134 · Cited by 1
Sketched Gaussian Mechanism for Private Federated Learning
Li, Q., Chen, Z., & Banerjee, A. (2025) · neural information processing systems · doi:10.52202/085713-4394 · Cited by 2
Jin, Z., Xu, R., Li, C., Liu, Y., Li, J., & Joshi, J. (2026) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2026/73 · Cited by 1
Helmert-based A-law and $$\mu $$-law vector gradient quantization in deep neural networks
Panic, S. R., Milenković, V., & Ivković, R. (2026) · Signal Image and Video Processing · doi:10.1007/s11760-025-05050-2 · Cited by 1
Memory-Efficient Distributed Unlearning
Lang, N., Helvits, A., & Shlezinger, N. (2026) · IEEE Access · doi:10.1109/access.2026.3663428 · Cited by 1
Tan, Z., Pan, Z., Liang, Y., & Yang, S. (2026) · Electronics · doi:10.3390/electronics15050928 · Cited by 1
Ajdani, M. (2026) · International Journal of Information Security and Privacy · doi:10.4018/ijisp.402007 · Cited by 1
Anomaly-Based Intrusion Detection System Using Federated Learning for IoT Devices
Hasan, M. Z., & Ahmed, I. (2025) · International Conference on Computer and Information Technology (ICCIT) · doi:10.1109/iccit68739.2025.11491092 · Cited by 1
Programmable In-Network Aggregation for Communication-Aware Federated Learning in 5G RANs
Paolini, E., Pinto, A., Valcarenghi, L., & Esposito, F. (2026) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2026.3697723 · Cited by 3
Chittaiah, N. M., & Manjula, S. H. (2026) · Engineering Technology & Applied Science Research · doi:10.48084/etasr.18548 · Cited by 1
Event-Triggered Gossip for Distributed Learning
Zhai, Z., Yuan, X., Wang, X., Zhang, R., & Li, G. Y. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3718498 · Cited by 1
FedAdOb: Privacy-Preserving Fed erated Deep Learning With Ad aptive Ob fuscation
Gu, H., Luo, J., Kang, Y., Yao, Y., Zhu, G., Li, B., Fan, L., & Yang, Q. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3715766 · Cited by 1
Metadata
- Authors
- Aji, A. F., & Heafield, K.
- Affiliations
- University of Edinburgh (GB)
- ORCID
- Heafield, Kenneth
- Venue
- Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
- Published
- 2017
- Pages
- 440-445
- OpenAlex
- W2606891064
- Cited by
- 585
- References
- 0
- Citations by year
- 2026: 57 · 2025: 89 · 2024: 104 · 2023: 88 · 2022: 79 · 2021: 83 · 2020: 54 · 2019: 24 · 2018: 7
- FWCI
- 10.60
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
- arXiv (Cornell University)
- Funders
- Nvidia
- Alan Turing Institute
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Physical Sciences › Computer Science › Hardware and Architecture › Parallel Computing and Optimization Techniques
- Keywords
- sparse communication, distributed gradient descent, stochastic gradient descent, gradient compression, quantization, neural machine translation, convergence rate, communication efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.