The DOI matches its Crossref record; 34 of the 34 references it cites were found in a registry or an index.
Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learning
Open accessAbstract
Federated Learning (FL) is an emerging distributed learning paradigm under privacy constraint. Data heterogeneity is one of the main challenges in FL, which results in slow convergence and degraded performance. Most existing approaches only tackle the heterogeneity challenge by restricting the local model update in client, ignoring the performance drop caused by direct global model aggregation. Instead, we propose a data-free knowledge distillation method to fine-tune the global model in the server (FedFTG), which relieves the issue of direct model aggregation. Concretely, FedFTG explores the input space of local models through a generator, and uses it to transfer the knowledge from local models to the global model. Besides, we propose a hard sample mining scheme to achieve effective knowledge distillation throughout the training. In addition, we develop customized label sampling and class-level ensemble to derive maximum utilization of knowledge, which implicitly mitigates the distribution discrepancy across clients. Extensive experiments show that our FedFTG significantly outperforms the state-of-the-art (SOTA) FL algorithms and can serve as a strong plugin for enhancing FedAvg, FedProx, FedDyn, and SCAFFOLD.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Federated Optimization in Heterogeneous Networks
Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., & Smith, V. (2018) · arXiv · doi:10.48550/arxiv.1812.06127 · Cited by 4,023
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
Karimireddy, S. P., Kale, S., Mohri, M., Reddi, S. J., Stich, S. U., & Suresh, A. T. (2019) · arXiv · doi:10.48550/arxiv.1910.06378 · Cited by 1,734
On the Convergence of FedAvg on Non-IID Data
Li, X., Huang, K., Yang, W., Wang, S., & Zhang, Z. (2019) · arXiv · doi:10.48550/arxiv.1907.02189 · Cited by 1,575
Model-Contrastive Federated Learning
Li, Q., He, B., & Song, D. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.01057 · Cited by 1,534
Federated Learning for Mobile Keyboard Prediction
Hard, A., Rao, K., Mathews, R., Ramaswamy, S., Beaufays, F., Augenstein, S., Eichner, H., Kiddon, C., & Ramage, D. (2018) · arXiv · doi:10.48550/arxiv.1811.03604 · Cited by 1,148
A large-scale car dataset for fine-grained categorization and verification
Yang, L., Luo, P., Loy, C. C., & Tang, X. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299023 · Cited by 825
Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
Hsu, T.-M. H., Qi, H., & Brown, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06335 · Cited by 610
Liu, Q., Chen, C., Qin, J., Dou, Q., & Heng, P.-A. (2021) · 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr46437.2021.00107 · Cited by 541
Dreaming to Distill: Data-Free Knowledge Transfer via DeepInversion
Yin, H., Molchanov, P., Alvarez, J. M., Li, Z., Mallya, A., Hoiem, D., Jha, N. K., & Kautz, J. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00874 · Cited by 540
A Field Guide to Federated OptimizationPreprint
Wang, J., Charles, Z., Xu, Z., Joshi, G., McMahan, H. B., y Arcas, B. A., Al-Shedivat, M., Andrew, G., Avestimehr, S., Daly, K., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.06917 · Cited by 493
Ensemble Distillation for Robust Model Fusion in Federated Learning
Lin, T., Kong, L., Stich, S. U., & Jaggi, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.07242 · Cited by 476
Communication-Efficient Federated Learning for Wireless Edge Intelligence in IoT
Mills, J., Hu, J., & Min, G. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2019.2956615 · Cited by 436
Data-Free Learning of Student Networks
Chen, H., Wang, Y., Xu, C., Yang, Z., Liu, C., Shi, B., Xu, C., Xu, C., & Tian, Q. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00361 · Cited by 346
FedML: A Research Library and Benchmark for Federated Machine Learning
He, C., Li, S., So, J., Zeng, X., Zhang, M., Wang, H., Wang, X., Vepakomma, P., Singh, A., Qiu, H., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.13518 · Cited by 314
Self-citationFederated Learning in Smart City Sensing: Challenges and Opportunities
Jiang, J. C., Kantarci, B., Oktug, S., & Soyata, T. (2020) · Sensors · doi:10.3390/s20216230 · Cited by 310
Applications of federated learning in smart cities: recent advances, taxonomy, and open challenges
Zheng, Z., Zhou, Y., Sun, Y., Wang, Z., Liu, B., & Li, K. (2022) · Connection Science · doi:10.1080/09540091.2021.1936455 · Cited by 206
Optimal Client Sampling for Federated LearningPreprint
Chen, W., Horváth, S., & Richtárik, P. (2020) · arXiv (Cornell University) · doi:10.17863/cam.89518 · Cited by 187
Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge
He, C., Annavaram, M., & Avestimehr, S. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.14513 · Cited by 177
Data-Free Adversarial DistillationPreprint
Fang, G., Song, J., Shen, C., Wang, X., Chen, D., & Song, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.11006 · Cited by 172
MIO-TCD: A New Benchmark Dataset for Vehicle Classification and Localization
Luo, Z., Branchaud-Charron, F., Lemaire, C., Konrad, J., Li, S., Mishra, A. K., Achkar, A., Eichel, J. A., & Jodoin, P. (2018) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2848705 · Cited by 162
Stochastic Client Selection for Federated Learning With Volatile Clients
Huang, T., Lin, W., Shen, L., Li, K., & Zomaya, A. Y. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3172113 · Cited by 141
Self-citationFraboni, Y., Vidal, R., Kameni, L., & Lorenzi, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.05883 · Cited by 133
FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learning
Sattler, F., Korjakow, T., Rischke, R., & Samek, W. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3129371 · Cited by 131
Performance Optimization of Federated Person Re-identification via Benchmark Analysis
Zhuang, W., Wen, Y., Zhang, X., Gan, X., Yin, D., Zhou, D., Zhang, S., & Yi, S. (2020) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3394171.3413814 · Cited by 115
HHHFL: Hierarchical Heterogeneous Horizontal Federated Learning for ElectroencephalographyPreprint
Gao, D., Ju, C., Wei, X., Liu, Y., Chen, T., & Yang, Q. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.05784 · Cited by 103
Particle Swarm Optimized Federated Learning For Industrial IoT and Smart City Services
Qolomany, B., Ahmad, K., Al‐Fuqaha, A., & Qadir, J. (2020) · Globecom · doi:10.1109/globecom42002.2020.9322464 · Cited by 91
Private Federated Learning with Domain AdaptationPreprint
Peterson, D. W., Kanani, P. H., & Marathe, V. J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.06733 · Cited by 83
Federated Learning for Ranking Browser History SuggestionsPreprint
Hartmann, F., Suh, S., Komarzewski, A., Smith, T. D., & Segall, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.11807 · Cited by 65
From Local SGD to Local Fixed Point Methods for Federated Learning
Malinovsky, G., Kovalev, D. A., Gasanov, E., Condat, L., & Richtárik, P. (2020) · International Conference on Machine Learning · Cited by 59
Decentralised Learning from Independent Multi-Domain Labels for Person Re-Identification
Wu, G., & Gong, S. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i4.16396 · Cited by 37
Learn electronic health records by fully decentralized federated learning
Lu, S., Zhang, Y., Wang, Y., & Mack, C. (2019) · arXiv · doi:10.48550/arxiv.1912.01792 · Cited by 23
Layer-Peeled Model: Toward Understanding Well-Trained Deep Neural Networks.Preprint
Fang, C., He, H., Qi, L., & Su, W. (2021) · arXiv (Cornell University) · Cited by 8
Cited by
A Review of Federated Learning Methods in Heterogeneous Scenarios
Pei, J., Liu, W., Li, J., Wang, L., & Liu, C. (2024) · IEEE Transactions on Consumer Electronics · doi:10.1109/tce.2024.3385440 · Cited by 281
Trusted AI in Multiagent Systems: An Overview of Privacy and Security for Distributed Learning
Ma, C., Li, J., Wei, K., Liu, B., Ding, M., Yuan, L., Han, Z., & Vincent Poor, H. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3306773 · Cited by 96
A Survey of Security Strategies in Federated Learning: Defending Models, Data, and Privacy
Manzoor, H. U., Shabbir, A., Chen, A., Flynn, D., & Zoha, A. (2024) · Future Internet · doi:10.3390/fi16100374 · Cited by 58
Threats and Defenses in the Federated Learning Life Cycle: A Comprehensive Survey and Challenges
Li, Y., Guo, Z., Yang, N., Chen, H., Yuan, D., & Ding, W. (2025) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2025.3563537 · Cited by 38
Metadata
- Authors
- Zhang, L., Shen, L., Ding, L., Tao, D., & Duan, L.-Y.
- Venue
- 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- Publisher
- IEEE
- Published
- 2022-06
- Pages
- 10164-10173
- ISSN
- 1063-6919 (ISSN-L)
- OpenAlex
- W4226101686
- Cited by
- 371
- References
- 34
- FWCI
- 48.08
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.