The DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
Heterogeneous Federated Learning: State-of-the-art and Research Challenges
Open accessAbstract
Federated learning (FL) has drawn increasing attention owing to its potential use in large-scale industrial applications. Existing FL works mainly focus on model homogeneous settings. However, practical FL typically faces the heterogeneity of data distributions, model architectures, network environments, and hardware devices among participant clients. Heterogeneous Federated Learning (HFL) is much more challenging, and corresponding solutions are diverse and complex. Therefore, a systematic survey on this topic about the research challenges and state-of-the-art is essential. In this survey, we firstly summarize the various research challenges in HFL from five aspects: statistical heterogeneity, model heterogeneity, communication heterogeneity, device heterogeneity, and additional challenges. In addition, recent advances in HFL are reviewed and a new taxonomy of existing HFL methods is proposed with an in-depth analysis of their pros and cons. We classify existing methods from three different levels according to the HFL procedure: data-level, model-level, and server-level. Finally, several critical and promising future research directions in HFL are discussed, which may facilitate further developments in this field. A periodically updated collection on HFL is available at https://github.com/marswhu/HFL_Survey.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Overcoming catastrophic forgetting in neural networks
Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1611835114 · Cited by 7,899
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772
Federated Machine Learning: Concept and Applications
Yang, Q., Liu, Y., Chen, T., & Tong, Y. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3298981 · Cited by 6,322
Federated learning: Challenges, methods, and future directions
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2975749 · Cited by 6,210
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem
McCloskey, M., & Cohen, N. J. (1989) · Psychology of Learning and Motivation · doi:10.1016/s0079-7421(08)60536-8 · Cited by 3,770
Robust and Communication-Efficient Federated Learning From Non-i.i.d. Data
Sattler, F., Wiedemann, S., Muller, K.-R., & Samek, W. (2020) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2019.2944481 · Cited by 1,725
Privacy-Preserving Deep Learning via Additively Homomorphic Encryption
Phong, L. T., Aono, Y., Hayashi, T., Wang, L., & Moriai, S. (2018) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2017.2787987 · Cited by 1,662
Federated Learning for Internet of Things: A Comprehensive Survey
Nguyen, D. C., Ding, M., Pathirana, P. N., Seneviratne, A., Li, J., & Vincent Poor, H. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3075439 · Cited by 1,503 · cited 3×
“[162] provide a survey and analysis of federated learning in IoT services (e.”
BackgroundA survey on security and privacy of federated learning
Mothukuri, V., Parizi, R. M., Pouriyeh, S., Huang, Y., Dehghantanha, A., & Srivastava, G. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2020.10.007 · Cited by 1,480
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
Li, Q., Wen, Z., Wu, Z., Hu, S., Wang, N., Li, Y., Liu, X., & He, B. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3124599 · Cited by 1,407
Sattler, F., Muller, K.-R., & Samek, W. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.3015958 · Cited by 1,355
Towards Personalized Federated Learning
Tan, A. Z., Yu, H., Cui, L., & Yang, Q. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3160699 · Cited by 1,182
Federated learning on non-IID data: A survey
Zhu, H., Xu, J., Liu, S., & Jin, Y. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.098 · Cited by 1,062
FedHealth: A Federated Transfer Learning Framework for Wearable Healthcare
Chen, Y., Qin, X., Wang, J., Yu, C., & Gao, W. (2020) · IEEE Intelligent Systems · doi:10.1109/mis.2020.2988604 · Cited by 1,005
Federated Learning for Internet of Things: Recent Advances, Taxonomy, and Open Challenges
Khan, L. U., Saad, W., Han, Z., Hossain, E., & Hong, C. S. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3090430 · Cited by 892
Federated Learning for Internet of Things: A Comprehensive Survey
Nguyen, D., Ding, M., Pathirana, P. N., Seneviratne, A., Li, J., & Poor, H. (2021) · Deakin Research Online (Deakin University) · Cited by 889
Federated learning for predicting clinical outcomes in patients with COVID-19
Dayan, I., Roth, H. R., Zhong, A., Harouni, A., Gentili, A., Abidin, A. Z., Liu, A., Costa, A. B., Wood, B. J., Tsai, C.-S., et al. (2021) · Nature Medicine · doi:10.1038/s41591-021-01506-3 · Cited by 842 · cited 1×
“Federated learning has drawn increasing attention in both academia and industry owing to its potential utility in large-scale applications [48, 85, 185], which has been widely explored in the fields of healthcare [39, 178, 214], medical analysis [54, 97], and data security [205], etc.”
BackgroundFederated Learning for Wireless Communications: Motivation, Opportunities, and Challenges
Niknam, S., Dhillon, H. S., & Reed, J. H. (2020) · IEEE Communications Magazine · doi:10.1109/mcom.001.1900461 · Cited by 820
Federated-Learning-Based Anomaly Detection for IoT Security Attacks
Mothukuri, V., Khare, P., Parizi, R. M., Pouriyeh, S., Dehghantanha, A., & Srivastava, G. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3077803 · Cited by 722
A Blockchain-Based Decentralized Federated Learning Framework with Committee Consensus
Li, Y., Chen, C., Liu, N., Huang, H., Zheng, Z., & Yan, Q. (2021) · IEEE Network · doi:10.1109/mnet.011.2000263 · Cited by 645
Yin, X., Zhu, Y., & Hu, J. (2022) · ACM Computing Surveys · doi:10.1145/3460427 · Cited by 638
Communication-efficient federated learning via knowledge distillation
Wu, C., Wu, F., Lyu, L., Huang, Y., & Xie, X. (2022) · Nature Communications · doi:10.1038/s41467-022-29763-x · Cited by 618
Privacy and Robustness in Federated Learning: Attacks and Defenses
Lyu, L., Yu, H., Ma, X., Chen, C., Sun, L., Zhao, J., Yang, Q., & Yu, P. S. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3216981 · Cited by 548
End-to-end privacy preserving deep learning on multi-institutional medical imaging
Kaissis, G., Ziller, A., Passerat-Palmbach, J., Ryffel, T., Usynin, D., Trask, A., Lima, I., Mancuso, J., Jungmann, F., Steinborn, M.-M., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00337-8 · Cited by 513
Wahab, O. A., Mourad, A., Otrok, H., & Taleb, T. (2021) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2021.3058573 · Cited by 487
On-Line Expectation–Maximization Algorithm for latent Data Models
Cappé, O., & Moulines, E. (2009) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.1467-9868.2009.00698.x · Cited by 477
Fog Computing for Sustainable Smart Cities: A Survey
Perera, C., Qin, Y., Estrella, J. C., Reiff-Marganiec, S., & Vasilakos, A. V. (2018) · ACM Computing Surveys · doi:10.1145/3057266 · Cited by 459
Decentralized Privacy Using Blockchain-Enabled Federated Learning in Fog Computing
Qu, Y., Gao, L., Luan, T. H., Xiang, Y., Yu, S., Li, B., & Zheng, G. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2977383 · Cited by 451
Personalized Federated Learning for Intelligent IoT Applications: A Cloud-Edge Based Framework
Wu, Q., He, K., & Chen, X. (2020) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2020.2993259 · Cited by 424
Communication-efficient federated learning
Chen, M., Shlezinger, N., Poor, H. V., Eldar, Y. C., & Cui, S. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2024789118 · Cited by 397
Deep Learning: The Frontier for Distributed Attack Detection in Fog-to-Things Computing
Abeshu, A., & Chilamkurti, N. (2018) · IEEE Communications Magazine · doi:10.1109/mcom.2018.1700332 · Cited by 390
Personalized Federated Learning With Differential Privacy
Hu, R., Guo, Y., Li, H., Pei, Q., & Gong, Y. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.2991416 · Cited by 350
FedMCCS: Multicriteria Client Selection Model for Optimal IoT Federated Learning
Abdulrahman, S., Tout, H., Mourad, A., & Talhi, C. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3028742 · Cited by 320
Multi-center federated learning: clients clustering for better personalization
Long, G., Xie, M., Shen, T., Zhou, T., Wang, X., & Jiang, J. (2023) · World Wide Web · doi:10.1007/s11280-022-01046-x · Cited by 320
Lim, W. Y. B., Ng, J. S., Xiong, Z., Jin, J., Zhang, Y., Niyato, D., Leung, C., & Miao, C. (2022) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2021.3096076 · Cited by 312
PoisonGAN: Generative Poisoning Attacks Against Federated Learning in Edge Computing Systems
Zhang, J., Chen, B., Cheng, X., Binh, H. T. T., & Yu, S. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3023126 · Cited by 291
FedProc: Prototypical contrastive federated learning on non-IID data
Mu, X., Shen, Y., Cheng, K., Geng, X., Fu, J., Zhang, T., & Zhang, Z. (2023) · Future Generation Computer Systems · doi:10.1016/j.future.2023.01.019 · Cited by 274
Towards Fair and Privacy-Preserving Federated Deep Models
Lyu, L., Yu, J., Nandakumar, K., Li, Y., Ma, X., Jin, J., Yu, H., & Ng, K. S. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.2996273 · Cited by 250 · cited 1×
“Besides, the potential relationship between collaborative fairness and privacy protection [91, 145] needs to be further investigated.”
BackgroundClient Selection for Federated Learning With Non-IID Data in Mobile Edge Computing
Zhang, W., Wang, X., Zhou, P., Wu, W., & Zhang, X. (2021) · IEEE Access · doi:10.1109/access.2021.3056919 · Cited by 230
Privacy-first health research with federated learning
Sadilek, A., Liu, L., Nguyen, D., Kamruzzaman, M., Serghiou, S., Rader, B., Ingerman, A., Mellem, S., Kairouz, P., Nsoesie, E. O., et al. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00489-2 · Cited by 226
Threats to federated learning A survey
Lyu, L., Yu, H., & Yang, Q. (2020) · arXiv · doi:10.48550/arxiv.2003.02133 · Cited by 221
On the Reconstruction of Face Images from Deep Face Templates
Mai, G., Cao, K., Yuen, P. C., & Jain, A. K. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2827389 · Cited by 220
Self-citationFederated unsupervised representation learning
Zhang, F., Kuang, K., Chen, L., You, Z., Shen, T., Xiao, J., Zhang⋆, Y., Wu, C., Wu, F. Q., Zhuang, Y., et al. (2023) · Frontiers of Information Technology & Electronic Engineering · doi:10.1631/fitee.2200268 · Cited by 195
Froelicher, D., Troncoso-Pastoriza, J. R., Raisaro, J. L., Cuendet, M. A., Sousa, J. S., Cho, H., Berger, B., Fellay, J., & Hubaux, J.-P. (2021) · Nature Communications · doi:10.1038/s41467-021-25972-y · Cited by 195
FedOpt: Towards Communication Efficiency and Privacy Preservation in Federated Learning
Asad, M., Moustafa, A., & Ito, T. (2020) · Applied Sciences · doi:10.3390/app10082864 · Cited by 174
GossipFL: A Decentralized Federated Learning Framework With Sparsified and Adaptive Communication
Tang, Z., Shi, S., Li, B., & Chu, X. (2023) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2022.3230938 · Cited by 168
Decentralized Federated Learning via Mutual Knowledge Transfer
Li, C., Li, G., & Varshney, P. K. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3078543 · Cited by 167
From Federated to Fog Learning: Distributed Machine Learning over Heterogeneous Wireless Networks
Hosseinalipour, S., Brinton, C. G., Aggarwal, V., Dai, H., & Chiang, M. (2020) · IEEE Communications Magazine · doi:10.1109/mcom.001.2000410 · Cited by 164
Lim, W. Y. B., Ng, J. S., Xiong, Z., Niyato, D., Miao, C., & Kim, D. I. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2021.3118401 · Cited by 160
Decentralized federated learning through proxy model sharing
Kalra, S., Wen, J., Cresswell, J. C., Volkovs, M., & Tizhoosh, H. R. (2023) · Nature Communications · doi:10.1038/s41467-023-38569-4 · Cited by 160
PMF: A Privacy-preserving Human Mobility Prediction Framework via Federated Learning
Feng, J., Rong, C., Sun, F., Guo, D., & Li, Y. (2020) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3381006 · Cited by 158
Stochastic Client Selection for Federated Learning With Volatile Clients
Huang, T., Lin, W., Shen, L., Li, K., & Zomaya, A. Y. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3172113 · Cited by 141
Augmentation Invariant and Instance Spreading Feature for Softmax Embedding
Ye, M., Shen, J., Zhang, X., Yuen, P. C., & Chang, S. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2020.3013379 · Cited by 137
Self-citationRobust Federated Learning With Noisy Labels
Yang, S., Park, H., Byun, J., & Kim, C. (2022) · IEEE Intelligent Systems · doi:10.1109/mis.2022.3151466 · Cited by 129
Highly efficient federated learning with strong privacy preservation in cloud computing
Chen, F., Guo, Y., Wang, N., & Ju, A. (2020) · Computers & Security · doi:10.1016/j.cose.2020.101889 · Cited by 127
SecureFace: Face Template Protection
Mai, G., Cao, K., Lan, X., & Yuen, P. C. (2020) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2020.3009590 · Cited by 96
Self-citationSafe: Synergic Data Filtering for Federated Learning in Cloud-Edge Computing
Xu, X., Li, H., Li, Z., & Zhou, X. (2022) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2022.3195896 · Cited by 76
The OARF Benchmark Suite: Characterization and Implications for Federated Learning Systems
Hu, S., Li, Y., Liu, X., Li, Q., Wu, Z., & He, B. (2022) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3510540 · Cited by 73
Coordinated Backdoor Attacks against Federated Learning with Model-Dependent Triggers
Gong, X., Chen, Y., Huang, H., Liao, Y., Wang, S., & Wang, Q. (2022) · IEEE Network · doi:10.1109/mnet.011.2000783 · Cited by 72
Adversarial interference and its mitigations in privacy-preserving collaborative machine learning
Usynin, D., Ziller, A., Makowski, M. R., Braren, R., Rueckert, D., Glocker, B., Kaissis, G. A., & Passerat‐Palmbach, J. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00390-3 · Cited by 68
Functional genomics data: privacy risk assessment and technological mitigation
Gürsoy, G., Li, T., Liu, S. X., Ni, E., Brannon, C. M., & Gerstein, M. (2021) · Nature Reviews Genetics · doi:10.1038/s41576-021-00428-7 · Cited by 53
Hybrid differential privacy based federated learning for Internet of Things
Liu, W., Cheng, J., Wang, X., Lu, X., & Yin, J. (2022) · Journal of Systems Architecture · doi:10.1016/j.sysarc.2022.102418 · Cited by 45
PFL-Bench: A Comprehensive Benchmark for Personalized Federated Learning
Chen, D., Gao, D., Kuang, W., Li, Y., & Ding, B. (2022) · neural information processing systems · doi:10.52202/068431-0679 · Cited by 45
Federated Generalized Face Presentation Attack Detection
Shao, R., Perera, P., Yuen, P. C., & Patel, V. M. (2022) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2022.3172316 · Cited by 44
Self-citationFedFog: Network-Aware Optimization of Federated Learning Over Wireless Fog-Cloud Systems
Nguyen, V., Chatzinotas, S., Ottersten, B., & Duong, T. Q. (2022) · IEEE Transactions on Wireless Communications · doi:10.1109/twc.2022.3167263 · Cited by 38
Learnable Privacy-Preserving Anonymization for Pedestrian Images
Zhang, J., Ye, M., & Yang, Y. (2022) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3503161.3548766 · Cited by 20
Self-citationFederated learning with superquantile aggregation for heterogeneous data
Pillutla, K., Laguel, Y., Malick, J., & Harchaoui, Z. (2023) · Machine Learning · doi:10.1007/s10994-023-06332-x · Cited by 18
Device Heterogeneity in Federated Learning: A Superquantile Approach
Laguel, Y., Pillutla, K., Malick, J., & Harchaoui, Z. (2020) · arXiv · doi:10.48550/arxiv.2002.11223 · Cited by 15
Federated Learning for Personalized Humor Recognition
Guo, X., Yu, H., Li, B., Wang, H., Xing, P., Feng, S., Nie, Z., & Miao, C. (2022) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3511710 · Cited by 13
Cited by
Decentralized Federated Learning: A Survey and Perspective
Yuan, L., Wang, Z., Sun, L., Yu, P. S., & Brinton, C. G. (2024) · IEEE Internet of Things Journal · doi:10.1109/jiot.2024.3407584 · Cited by 290
Horizontal Federated Recommender System: A Survey
Wang, L., Zhou, H., Bao, Y., Yan, X., Shen, G., & Kong, X. (2024) · ACM Computing Surveys · doi:10.1145/3656165 · Cited by 37
AI4Research: a survey of artificial intelligence for scientific research
Chen, Q., Yang, M., Qin, L., Liu, J., Yan, Z., Guan, J., Peng, D., Ji, Y., Li, H., Hu, M., et al. (2026) · Science China Information Sciences · doi:10.1007/s11432-025-5000-y · Cited by 4
Metadata
- Authors
- Ye, M., Fang, X., Du, B., Yuen, P. C., & Tao, D.
- Venue
- ACM Computing Surveys
- Published
- 2023-10-21
- Volume
- 56
- Issue
- 3
- Pages
- 1-44
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W4387105517
- Cited by
- 608
- References
- 69
- FWCI
- 53.41
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.