Communication-Efficient On-Device Machine Learning: Federated Distillation and Augmentation under Non-IID Private Data
Open accessAbstract
On-device machine learning (ML) enables the training process to exploit a massive amount of user-generated private data samples. To enjoy this benefit, inter-device communication overhead should be minimized. With this end, we propose federated distillation (FD), a distributed model training algorithm whose communication payload size is much smaller than a benchmark scheme, federated learning (FL), particularly when the model size is large. Moreover, user-generated data samples are likely to become non-IID across devices, which commonly degrades the performance compared to the case with an IID dataset. To cope with this, we propose federated augmentation (FAug), where each device collectively trains a generative model, and thereby augments its local data towards yielding an IID dataset. Empirical studies demonstrate that FD with FAug yields around 26x less communication overhead while achieving 95-98% test accuracy compared to FL.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Using an online version of KD can reduce communication cost when training a large DL model by exchanging model outputs rather than updated model parameters.
“Using an online version of KD can reduce such kind of communication cost [194].In other words, the model outputs rather the updated model parameters on each device are exchanged, making the training of large-sized local models possible.”
From Convergence of Edge Computing and Deep Learning: A Comprehensive Survey (2020) · cites this paper as [194]
Cites
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215 · see citing works
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works
Differentially Private Federated Learning: A Client Level Perspective
Geyer, R. C., Klein, T., & Nabi, M. (2017) · arXiv · doi:10.48550/arxiv.1712.07557 · Cited by 1,052 · see citing works
Flagged first, then by citations
1 of the 4 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772 · see citing works
Self-citationFederated learning: Challenges, methods, and future directions
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2975749 · Cited by 6,210 · see citing works
Federated Optimization in Heterogeneous Networks
Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., & Smith, V. (2018) · arXiv · doi:10.48550/arxiv.1812.06127 · Cited by 4,023 · see citing works
Convergence of Edge Computing and Deep Learning: A Comprehensive Survey
Wang, X., Han, Y., Leung, V. C. M., Niyato, D., Yan, X., & Chen, X. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2970550 · Cited by 1,498 · see citing works
“Using an online version of KD can reduce such kind of communication cost [194].”
MethodA survey on security and privacy of federated learning
Mothukuri, V., Parizi, R. M., Pouriyeh, S., Huang, Y., Dehghantanha, A., & Srivastava, G. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2020.10.007 · Cited by 1,480 · see citing works
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
Li, Q., Wen, Z., Wu, Z., Hu, S., Wang, N., Li, Y., Liu, X., & He, B. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3124599 · Cited by 1,407 · see citing works
Towards Personalized Federated Learning
Tan, A. Z., Yu, H., Cui, L., & Yang, Q. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3160699 · Cited by 1,366 · see citing works
FedProto: Federated Prototype Learning across Heterogeneous Clients
Tan, Y., Long, G., Liu, L., Zhou, T., Lu, Q., Jiang, J. B., & Zhang, C. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20819 · Cited by 809 · see citing works
A Survey on Federated Learning for Resource-Constrained IoT Devices
Imteaj, A., Thakker, U., Wang, S., Li, J., & Amini, M. H. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3095077 · Cited by 774 · see citing works
Ensemble Distillation for Robust Model Fusion in Federated LearningPreprint
Lin, T., Kong, L., Stich, S. U., & Jaggi, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07242 · Cited by 733 · see citing works
Distributed Learning in Wireless Networks: Recent Progress and Future Challenges
Chen, M., Gündüz, D., Huang, K., Saad, W., Bennis, M., Feljan, A. V., & Poor, H. V. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118346 · Cited by 558 · see citing works
Self-citationPrivacy and Robustness in Federated Learning: Attacks and Defenses
Lyu, L., Yu, H., Ma, X., Chen, C., Sun, L., Zhao, J., Yang, Q., & Yu, P. S. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3216981 · Cited by 548 · see citing works
Towards a standard for identifying and managing bias in artificial intelligence
Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., & Hall, P. (2022) · doi:10.6028/nist.sp.1270 · Cited by 548 · see citing works
“data generation bias Arises from the addition of synthetic or redundant data samples to a dataset [277].”
BackgroundWahab, O. A., Mourad, A., Otrok, H., & Taleb, T. (2021) · IEEE Communications Surveys and Tutorials · doi:10.1109/comst.2021.3058573 · Cited by 444 · see citing works
Federated learning for 6G communications: Challenges, methods, and future directions
Liu, Y., Yuan, X., Xiong, Z., Kang, J., Wang, X., & Niyato, D. (2020) · China Communications · doi:10.23919/jcc.2020.09.009 · Cited by 464 · see citing works
Huang, L., Shea, A. L., Qian, H., Masurkar, A., Deng, H., & Liu, D. (2019) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2019.103291 · Cited by 437 · see citing works
Federated Learning With Non-IID Data: A Survey
Lu, Z., Pan, H., Dai, Y., Si, X., & Zhang, Y. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3376548 · Cited by 450 · see citing works
Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge Based Framework
Wu, Q., He, K., & Chen, X. (2020) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2020.2993259 · Cited by 424 · see citing works
Federated Learning With Cooperating Devices: A Consensus Approach for Massive IoT Networks
Savazzi, S., Nicoli, M., & Rampa, V. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2964162 · Cited by 431 · see citing works
Hierarchical Federated Learning ACROSS Heterogeneous Cellular Networks
Abad, M. S. H., Ozfatura, E., GUndUz, D., & Ercetin, O. (2020) · ICASSP 2020 - 2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) · doi:10.1109/icassp40776.2020.9054634 · Cited by 424 · see citing works
A state-of-the-art survey on solving non-IID data in Federated Learning
Ma, X., Zhu, J., Lin, Z., Chen, S., & Qin, Y. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.05.003 · Cited by 404 · see citing works
Federated Learning for Vehicular Internet of Things: Recent Advances and Open Issues
Du, Z., Wu, C., Yoshinaga, T., Yau, K.-L. A., Ji, Y., & Li, J. (2020) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2020.2992630 · Cited by 393 · see citing works
FedHome: Cloud-Edge Based Personalized Federated Learning for In-Home Health Monitoring
Wu, Q., Chen, X., Zhou, Z., & Zhang, J. (2022) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2020.3045266 · Cited by 392 · see citing works
FedML: A Research Library and Benchmark for Federated Machine LearningPreprint
He, C., Li, S., So, J., Zeng, X., Zhang, M., Wang, H., Wang, X., Vepakomma, P., Singh, A., Qiu, H., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.13518 · Cited by 504 · see citing works
Itahara, S., Nishio, T., Koda, Y., Morikura, M., & Yamamoto, K. (2023) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2021.3070013 · Cited by 338 · see citing works
Blockchain-empowered Federated Learning: Challenges, Solutions, and Future Directions
Zhu, J., Cao, J., Saxena, D., Jiang, S., & Ferradi, H. (2023) · ACM Computing Surveys · doi:10.1145/3570953 · Cited by 309 · see citing works
An Efficiency-Boosting Client Selection Scheme for Federated Learning With Fairness Guarantee
Huang, T., Lin, W., Wu, W., He, L., Li, K., & Zomaya, A. Y. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.3040887 · Cited by 294 · see citing works
A Review of Federated Learning Methods in Heterogeneous Scenarios
Pei, J., Liu, W., Li, J., Wang, L., & Liu, C. (2024) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2024.3385440 · Cited by 289 · see citing works
Federated Learning for Open Banking
Long, G., Tan, Y., Jiang, J. B., & Zhang, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-63076-8_17 · Cited by 286 · see citing works
Group Knowledge Transfer: Federated Learning of Large CNNs at the EdgePreprint
He, C., Annavaram, M., & Avestimehr, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.14513 · Cited by 309 · see citing works
Layer-wised Model Aggregation for Personalized Federated Learning
Ma, X., Zhang, J., Guo, S., & Xu, W. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00985 · Cited by 253 · see citing works
End-Edge-Cloud Collaborative Computing for Deep Learning: A Comprehensive Survey
Wang, Y., Yang, C., Lan, S.-L., Zhu, L.-H., & Zhang, Y. (2024) · IEEE Communications Surveys and Tutorials · doi:10.1109/comst.2024.3393230 · Cited by 255 · see citing works
Chellapandi, V. P., Yuan, L., Brinton, C. G., Żak, S. H., & Wang, Z. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3332675 · Cited by 249 · see citing works
Federated Learning for Generalization, Robustness, Fairness: A Survey and Benchmark
Huang, W., Ye, M., Shi, Z., Wan, G., He, L., Du, B., & Yang, Q. (2024) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2024.3418862 · Cited by 251 · see citing works
No Fear of Heterogeneity: Classifier Calibration for Federated Learning with Non-IID DataPreprint
Luo, M., Chen, F., Hu, D., Zhang, Y., Liang, J., & Feng, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.05001 · Cited by 211 · see citing works
Li, D., Han, D., Weng, T., Zheng, Z., Li, H., Liu, H., Castiglione, A., & Li, K. (2021) · Soft Computing · doi:10.1007/s00500-021-06496-5 · Cited by 207 · see citing works
Handling privacy-sensitive medical data with federated learning: Challenges and future directions
Aouedi, O., Sacco, A., Piamrat, K., & Marchetto, G. (2023) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2022.3185673 · Cited by 165 · see citing works
Mhaisen, N., Abdellatif, A. A., Mohamed, A. M. S., Erbad, A., & Guizani, M. (2021) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2021.3053588 · Cited by 159 · see citing works
Data Heterogeneity-Robust Federated Learning via Group Client Selection in Industrial IoT
Li, Z., He, Y., Yu, H., Kang, J., Li, X., Xu, Z., & Niyato, D. T. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3161943 · Cited by 157 · see citing works
Zhou, X., Yang, Q., Zheng, X., Liang, W., Wang, K. I., Ma, J., Pan, Y. F., & Jin, Q. (2024) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2023.3345431 · Cited by 152 · see citing works
Semisupervised Federated-Learning-Based Intrusion Detection Method for Internet of Things
Zhao, R.-J., Wang, Y.-J., Xue, Z., Ohtsuki, T., Adebisi, B., & Gui, G. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3175918 · Cited by 131 · see citing works
Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated LearningPreprint
Yang, H., Fang, M., & Liu, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.11203 · Cited by 150 · see citing works
Challenges, Applications and Design Aspects of Federated Learning: A Survey
Rahman, K. M. J., Ahmed, F., Akhter, N., Hasan, M., Amin, R., Aziz, K. E., Islam, A. K. M. M., Mukta, M. S. H., & Islam, A. K. M. N. (2021) · IEEE Access · doi:10.1109/access.2021.3111118 · Cited by 147 · see citing works
Adaptive Federated Learning on Non-IID Data With Resource Constraint
Zhang, J., Guo, S., Qu, Z., Zeng, D., Zhan, Y., Liu, Q., & Akerkar, R. A. (2021) · IEEE Transactions on Computers · doi:10.1109/tc.2021.3099723 · Cited by 144 · see citing works
Ensemble Attention Distillation for Privacy-Preserving Federated Learning
Gong, X., Sharma, A., Karanam, S., Wu, Z., Chen, T., Doermann, D., & Innanje, A. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01480 · Cited by 138 · see citing works
Stochastic Client Selection for Federated Learning With Volatile Clients
Huang, T., Lin, W., Shen, L., Li, K., & Zomaya, A. Y. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3172113 · Cited by 131 · see citing works
Zhang, J., Liu, Y., Yang, H., & Cao, J. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i15.29617 · Cited by 130 · see citing works
Parameterized Knowledge Transfer for Personalized Federated LearningPreprint
Zhang, J., Guo, S., Ma, X., Wang, H., Xu, W., & Wu, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.02862 · Cited by 122 · see citing works
Federated unsupervised representation learning
Zhang, F., Kuang, K., Chen, L., You, Z., Shen, T., Xiao, J., Zhang⋆, Y., Wu, C., Wu, F. Q., Zhuang, Y., et al. (2023) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2200268 · Cited by 121 · see citing works
FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learning
Sattler, F., Korjakow, T., Rischke, R., & Samek, W. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3129371 · Cited by 122 · see citing works
Edge Intelligence for Data Handling and Predictive Maintenance in IIOT
Hafeez, T., Xu, L., & McArdle, G. (2021) · IEEE Access · doi:10.1109/access.2021.3069137 · Cited by 124 · see citing works
Personalized Edge Intelligence via Federated Self-Knowledge Distillation
Jin, H., Bai, D., Yao, D., Dai, Y., Gu, L., Chen, Y., & Sun, L. (2022) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2022.3225185 · Cited by 119 · see citing works
Wireless Federated Distillation for Distributed Edge Learning with Heterogeneous Data
Ahn, J.-H., Simeone, O., & Kang, J. (2019) · IEEE International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC) · doi:10.1109/pimrc.2019.8904164 · Cited by 114 · see citing works
Variational Federated Multi-Task LearningPreprint
Corinzia, L., Beuret, A., & Buhmann, J. M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.06268 · Cited by 118 · see citing works
Al-Quraan, M. M., Mohjazi, L., Bariah, L., Centeno, A., Zoha, A., Arshad, K., Assaleh, K. T., Muhaidat, S., Debbah, M. A., & Imran, M. A. (2023) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2023.3251404 · Cited by 145 · see citing works
A Survey on Approximate Edge AI for Energy Efficient Autonomous Driving Services
Katare, D., Perino, D., Nurmi, J., Warnier, M., Janssen, M., & Ding, A. Y. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3302474 · Cited by 119 · see citing works
Model optimization techniques in personalized federated learning: A survey
Sabah, F., Chen, Y., Yang, Z., Azam, M., Ahmad, N., & Sarwar, R. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.122874 · Cited by 113 · see citing works
Federated Learning-Based Misbehavior Detection for the 5G-Enabled Internet of Vehicles
Rani, P., Sharma, C., Ramesh, J. V. N., Verma, S., Sharma, R., Alkhayyat, A. H., & Kumar, S. (2023) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2023.3328020 · Cited by 118 · see citing works
Federated Learning and Meta Learning: Approaches, Applications, and Directions
Liu, X., Deng, Y., Nallanathan, A., & Bennis, M. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3330910 · Cited by 115 · see citing works
Self-citationWang, C., Yang, Y., & Zhou, P. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.3023905 · Cited by 106 · see citing works
Towards Fair Federated Learning with Zero-Shot Data Augmentation
Hao, W., El‐Khamy, M., Lee, J., Zhang, J., Liang, K. J., Chen, C., & Carin, L. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00369 · Cited by 107 · see citing works
Cui, Y., Cao, K., Cao, G., Meikang Qiu, A., & Wei, T. (2021) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2021.3110743 · Cited by 105 · see citing works
Federated Graph Classification over Non-IID GraphsPreprint
Xie, H., Ma, J., Xiong, L., & Yang, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.13423 · Cited by 105 · see citing works
Architectural patterns for the design of federated learning systems
Lo, S. K., Lu, Q., Zhu, L., Paik, H.-Y., Xu, X., & Wang, C. (2022) · Journal of Systems and Software · doi:10.1016/j.jss.2022.111357 · Cited by 109 · see citing works
Continual Federated Learning Based on Knowledge Distillation
Ma, Y., Xie, Z., Wang, J., Chen, K., & Shou, L. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/303 · Cited by 102 · see citing works
The Disruptions of 5G on Data-Driven Technologies and Applications
Loghin, D., Cai, S., Chen, G., Dinh, T. T. A., Fan, F., Lin, Q., Ng, J., Ooi, B. C., Sun, X., Ta, Q.-T., et al. (2020) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2020.2967670 · Cited by 101 · see citing works
Lee, J.-H., Park, J., Bennis, M., & Ko, Y. (2022) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2022.3220696 · Cited by 99 · see citing works
Self-citationSemisupervised Distributed Learning With Non-IID Data for AIoT Service Platform
Chiu, T.-C., Shih, Y.-Y., Pang, A.-C., Wang, C.-S., Weng, W., & Chou, C.-T. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2995162 · Cited by 98 · see citing works
Multimodal federated learning: Concept, methods, applications and future directions
Huang, W. X., Wang, D., Ouyang, X., Wan, J., Liu, J., & Li, T. (2024) · Information Fusion · doi:10.1016/j.inffus.2024.102576 · Cited by 99 · see citing works
Towards Faster and Better Federated Learning: A Feature Fusion Approach
Yao, X., Huang, T., Wu, C., Zhang, R.-X., & Sun, L. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803001 · Cited by 97 · see citing works
Trusted AI in Multiagent Systems: An Overview of Privacy and Security for Distributed Learning
Ma, C., Li, J., Wei, K., Liu, B., Ding, M., Yuan, L., Han, Z., & Vincent Poor, H. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3306773 · Cited by 96 · see citing works
Improving Semi-supervised Federated Learning by Reducing the Gradient Diversity of Models
Zhang, Z. J., Yang, Y. L., Yao, Z., Yan, Y., Gonzalez, J. E., Ramchandran, K., & Mahoney, M. W. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9671693 · Cited by 104 · see citing works
Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge TransferPreprint
Chang, H., Shejwalkar, V., Shokri, R., & Houmansadr, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.11279 · Cited by 131 · see citing works
Zhang, L., Luo, Y., Bai, Y., Du, B., & Duan, L. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00438 · Cited by 92 · see citing works
Preserving Privacy in Federated Learning with Ensemble Cross-Domain Knowledge Distillation
Gong, X., Sharma, A., Karanam, S., Wu, Z., Chen, T., Doermann, D., & Innanje, A. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i11.21446 · Cited by 90 · see citing works
Continual Local Training For Better Initialization Of Federated Models
Yao, X., & Sun, L. (2020) · Proceedings - International Conference on Image Processing · doi:10.1109/icip40778.2020.9190968 · Cited by 88 · see citing works
Federated Learning for Privacy-Preserving Open Innovation Future on Digital Health
Long, G., Shen, T., Tan, Y., Gerrard, L., Clarke, A., & Jiang, J. (2022) · Humanity Driven AI · doi:10.1007/978-3-030-72188-6_6 · Cited by 85 · see citing works
DaFKD: Domain-aware Federated Knowledge Distillation
Wang, H., Li, Y., Xu, W., Li, R., Zhan, Y., & Zeng, Z. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01955 · Cited by 82 · see citing works
Matsubara, Y., Callegaro, D., Baidya, S., Levorato, M., & Singh, S. (2020) · IEEE Access · doi:10.1109/access.2020.3039714 · Cited by 81 · see citing works
CFD: Communication-Efficient Federated Distillation via Soft-Label Quantization and Delta Coding
Sattler, F., Marbán, A., Rischke, R., & Samek, W. (2021) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2021.3081748 · Cited by 132 · see citing works
A survey: Distributed Machine Learning for 5G and beyond
Nassef, O., Sun, W., Purmehdi, H., Tatipamula, M., & Mahmoodi, T. (2022) · Computer Networks · doi:10.1016/j.comnet.2022.108820 · Cited by 76 · see citing works
FedSAE: A Novel Self-Adaptive Federated Learning Framework in Heterogeneous Systems
Li, L., Duan, M., Liu, D., Zhang, Y., Ren, A., Chen, X., Tan, Y., & Wang, C. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533876 · Cited by 73 · see citing works
Wireless Network Intelligence at the Edge
Park, J., Samarakoon, S., Bennis, M., & Debbah, M. (2019) · Proceedings of the IEEE · doi:10.1109/jproc.2019.2941458 · Cited by 92 · see citing works
Self-citationHelios: Heterogeneity-Aware Federated Learning with Dynamically Balanced Collaboration
Xu, Z., Yu, F., Xiong, J., & Chen, X. (2021) · ACM/IEEE Design Automation Conference (DAC) · doi:10.1109/dac18074.2021.9586241 · Cited by 72 · see citing works
Integration of large language models and federated learning
Chen, C., Feng, X., Li, Y., Lyu, L., Zhou, J., Zheng, X., & Yin, J. (2024) · Patterns · doi:10.1016/j.patter.2024.101098 · Cited by 103 · see citing works
Vahidian, S., Morafah, M., Chen, C., Shah, M., & Lin, B. (2023) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2023.3293068 · Cited by 71 · see citing works
Sun, Y., Ochiai, H., & Esaki, H. (2021) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3133819 · Cited by 73 · see citing works
A survey for federated learning evaluations: Goals and measures
Chai, D., Wang, L., Yang, L., Zhang, J., Chen, K., & Yang, Q. (2024) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2024.3382002 · Cited by 64 · see citing works
Communication-efficient distributed AI strategies for the IoT edge
Mwase, C., Jin, Y., Westerlund, T., Tenhunen, H., & Zou, Z. (2022) · Future Generation Computer Systems · doi:10.1016/j.future.2022.01.013 · Cited by 64 · see citing works
Clustered Federated Learning in Heterogeneous Environment
Yan, Y.-H., Tong, X.-J., & Wang, S. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3264740 · Cited by 66 · see citing works
Towards Data-Independent Knowledge Transfer in Model-Heterogeneous Federated Learning
Zhang, J., Guo, S., Guo, J., Zeng, D., Zhou, J., & Zomaya, A. Y. (2023) · IEEE Transactions on Computers · doi:10.1109/tc.2023.3272801 · Cited by 62 · see citing works
Gad, G., Gad, E., Fadlullah, Z. M., Fouda, M. M., & Kato, N. (2024) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2024.3423718 · Cited by 64 · see citing works
Performance Analysis and Characterization of Training Deep Learning Models on Mobile Device
Liu, J., Liu, J., Du, W., & Li, D. (2019) · IEEE International Conference on Parallel and Distributed Systems (ICPADS) · doi:10.1109/icpads47876.2019.00077 · Cited by 62 · see citing works
Classifying UAVs With Proprietary Waveforms via Preamble Feature Extraction and Federated Learning
Reus-Muns, G., & Chowdhury, K. R. (2021) · IEEE Transactions on Vehicular Technology · doi:10.1109/tvt.2021.3081049 · Cited by 61 · see citing works
XOR Mixup: Privacy-Preserving Data Augmentation for One-Shot Federated LearningPreprint
Shin, M., Hwang, C., Kim, J., Park, J., Bennis, M., & Kim, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.05148 · Cited by 61 · see citing works
Self-citationHFedMS: Heterogeneous Federated Learning With Memorable Data Semantics in Industrial Metaverse
Zeng, S., Li, Z., Yu, H., Zhang, Z., Luo, L., Li, B., & Niyato, D. T. (2023) · IEEE Transactions on Cloud Computing · doi:10.1109/tcc.2023.3254587 · Cited by 59 · see citing works
FedMes: Speeding Up Federated Learning With Multiple Edge Servers
Han, D.-J., Choi, M., Park, J., & Moon, J. J. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118422 · Cited by 58 · see citing works
Self-citationA Survey of Federated Learning for Connected and Automated Vehicles
Chellapandi, V. P., Yuan, L., Żak, S. H., & Wang, Z. (2023) · IEEE International Conference on Intelligent Transportation Systems (ITSC) · doi:10.1109/itsc57777.2023.10421974 · Cited by 59 · see citing works
Asynchronous Federated Learning for Sensor Data with Concept Drift
Chen, Y., Chai, Z., Cheng, Y., & Rangwala, H. (2021) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata52589.2021.9671924 · Cited by 57 · see citing works
Personalized Federated Few-Shot Learning
Zhao, Y., Yu, G., Wang, J., Domeniconi, C., Guo, M., Zhang, X., & Cui, L. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3190359 · Cited by 56 · see citing works
- Authors
- Jeong, E., Oh, S., Kim, H., Park, J., Bennis, M., & Kim, S.-L.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1811.11479
- OpenAlex
- W2903471046
- Cited by
- 465
- References
- 4
- Citations by year
- 2026: 1 · 2025: 41 · 2024: 102 · 2023: 106 · 2022: 74 · 2021: 67 · 2020: 58 · 2019: 15 · 2018: 1
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- on-device learning, federated distillation, non-IID data, communication efficiency, generative models, data augmentation, communication cost reduction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
