The DOI matches its OpenAlex record; 76 of the 76 references it cites were found in a registry or an index.
Ensemble Distillation for Robust Model Fusion in Federated Learning
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Densely Connected Convolutional Networks
Huang, G., Liu, Z., Van Der Maaten, L., & Weinberger, K. Q. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.243 · Cited by 48,566
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
GAN(Generative Adversarial Nets)
(2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 19,270
EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks
Tan, M., & Le, Q. V. (2019) · arXiv · doi:10.48550/arxiv.1905.11946 · Cited by 17,072
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772
ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design
Ma, N., Zhang, X., Zheng, H.-T., & Sun, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_8 · Cited by 7,356 · cited 6×
“Batch Normalization (BN) (Ioffe & Szegedy, 2015) is the current workhorse in convolutional deep learning tasks and has been employed by default in most SOTA CNNs (He et al., 2016; Huang et al., 2017; Sandler et al., 2018; Ma et al., 2018; Tan & Le, 2019), however often fails on heterogeneous training data.”
BackgroundInfluentialRecursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878
Federated learning: Challenges, methods, and future directions
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2975749 · Cited by 6,210
DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019) · arXiv · doi:10.48550/arxiv.1910.01108 · Cited by 4,653
XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks
Rastegari, M., Ordonez, V., Redmon, J., & Farhadi, A. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_32 · Cited by 4,408
Federated Optimization in Heterogeneous Networks
Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., & Smith, V. (2018) · arXiv · doi:10.48550/arxiv.1812.06127 · Cited by 4,023
Large Scale GAN Training for High Fidelity Natural Image Synthesis
Brock, A., Donahue, J., & Simonyan, K. (2018) · arXiv · doi:10.48550/arxiv.1809.11096 · Cited by 3,717
Wu, Y., & He, K. (2020) · International Journal of Computer Vision · doi:10.1007/s11263-019-01198-w · Cited by 3,601
A theory of learning from different domains
Ben-David, S., Blitzer, J., Crammer, K., Kulesza, A., Pereira, F., & Vaughan, J. W. (2010) · Machine Learning · doi:10.1007/s10994-009-5152-4 · Cited by 3,515
Understanding Machine Learning: From Theory to Algorithms
Shalev-Shwartz, S., & Ben-David, S. (2014) · doi:10.1017/cbo9781107298019 · Cited by 3,115
FitNets: Hints for Thin Deep Nets
Romero, A., Ballas, N., Kahou, S. E., Chassang, A., Gatta, C., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1412.6550 · Cited by 2,859
(2015) · Cited by 2,857
Privacy-preserving deep learning
Shokri, R., & Shmatikov, V. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813687 · Cited by 2,356
Measures of Diversity in Classifier Ensembles and Their Relationship with the Ensemble Accuracy
Kuncheva, L. I., & Whitaker, C. J. (2003) · Machine Learning · doi:10.1023/a:1022859003006 · Cited by 2,335
Towards Federated Learning at Scale: System Design
Bonawitz, K., Eichner, H., Grieskamp, W., Huba, D., Ingerman, A., Ivanov, V., Kiddon, C., Konečný, J., Mazzocchi, S., McMahan, H. B., et al. (2019) · arXiv · doi:10.48550/arxiv.1902.01046 · Cited by 2,334
Buciluǎ, C., Caruana, R., & Niculescu-Mizil, A. (2006) · Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining · doi:10.1145/1150402.1150464 · Cited by 2,068
Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation
Bengio, Y., Léonard, N., & Courville, A. (2013) · arXiv · doi:10.48550/arxiv.1308.3432 · Cited by 1,910
SCAFFOLD: Stochastic Controlled Averaging for Federated Learning
Karimireddy, S. P., Kale, S., Mohri, M., Reddi, S. J., Stich, S. U., & Suresh, A. T. (2019) · arXiv · doi:10.48550/arxiv.1910.06378 · Cited by 1,734
Self-citationHubara, I., Soudry, D., & Yaniv, R. E. (2016) · arXiv · doi:10.48550/arxiv.1602.02505 · Cited by 1,667
(2018) · International Conference on Learning Representations · Cited by 1,621
Contrastive Representation DistillationPreprint
Tian, Y., Krishnan, D., & Isola, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.10699 · Cited by 1,324
Federated Learning of Deep Networks using Model AveragingPreprint
McMahan, H. B., Moore, E., Ramage, D., & y Arcas, B. A. (2016) · arXiv (Cornell University) · Cited by 1,288
Federated Learning with Matched AveragingPreprint
Wang, H., Yurochkin, M., Sun, Y., Papailiopoulos, D., & Khazaeni, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.06440 · Cited by 1,088
Similarity-Preserving Knowledge Distillation
Tung, F., & Mori, G. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00145 · Cited by 1,079
Differentially Private Federated Learning: A Client Level Perspective
Geyer, R. C., Klein, T., & Nabi, M. (2017) · arXiv · doi:10.48550/arxiv.1712.07557 · Cited by 1,052
SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems
Wang, A., Pruksachatkun, Y., Nangia, N., Singh, A., Michael, J., Hill, F., Levy, O., & Bowman, S. R. (2019) · arXiv · doi:10.48550/arxiv.1905.00537 · Cited by 1,051
LEAF: A Benchmark for Federated Settings
Caldas, S., Duddu, S. M. K., Wu, P., Li, T., Konečný, J., McMahan, H. B., Smith, V., & Talwalkar, A. (2018) · arXiv · doi:10.48550/arxiv.1812.01097 · Cited by 894
Adaptive Federated Optimization
Reddi, S., Charles, Z., Zaheer, M., Garrett, Z., Rush, K., Konečný, J., Kumar, S., & McMahan, H. B. (2020) · arXiv · doi:10.48550/arxiv.2003.00295 · Cited by 840
Variational Information Distillation for Knowledge Transfer
Ahn, S., Hu, S. X., Damianou, A., Lawrence, N. D., & Dai, Z. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00938 · Cited by 671
Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification
Hsu, T.-M. H., Qi, H., & Brown, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.06335 · Cited by 610
Fair Resource Allocation in Federated Learning
Li, T., Sanjabi, M., Beirami, A., & Smith, V. (2020) · International Conference on Learning Representations · Cited by 512
Jeong, E., Oh, S., Kim, H., Park, J., Bennis, M., & Kim, S.-L. (2018) · arXiv · doi:10.48550/arxiv.1811.11479 · Cited by 465
FedMD: Heterogenous Federated Learning via Model Distillation
Li, D., & Wang, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.03581 · Cited by 450
Large scale distributed neural network training through online distillationPreprint
Anil, R., Pereyra, G., Passos, A., Ormándi, R., Dahl, G. E., & Hinton, G. E. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.03235 · Cited by 440
(2008) · Cited by 436
A Simple Baseline for Bayesian Uncertainty in Deep Learning
Maddox, W., Garipov, T., Izmailov, P., Vetrov, D., & Wilson, A. G. (2019) · arXiv · doi:10.48550/arxiv.1902.02476 · Cited by 429
The Non-IID Data Quagmire of Decentralized Machine LearningPreprint
Hsieh, K., Phanishayee, A., Mutlu, O., & Gibbons, P. B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.00189 · Cited by 414
Furlanello, T., Lipton, Z. C., Tschannen, M., Itti, L., & Anandkumar, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.04770 · Cited by 407
Learning from Multiple Teacher Networks
You, S., Xu, C., Xu, C., & Tao, D. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098135 · Cited by 395
Mohri, M., Sivek, G., & Suresh, A. T. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.00146 · Cited by 373
Learning with Ensembles: How over-fitting can be useful
Peter Sollich, A., & Krogh, A. (1995) · Cited by 353
A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets
Chrabąszcz, P., Loshchilov, I., & Hutter, F. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.08819 · Cited by 336
Neural Networks for Machine Learning
(2017) · Human and Machine Hearing · doi:10.1017/9781139051699.031 · Cited by 325
FedBE: Making Bayesian Model Ensemble Applicable to Federated LearningPreprint
Chen, H.-Y., & Chao, W. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.01974 · Cited by 319
Adaptive Personalized Federated Learning
Deng, Y., Kamani, M. M., & Mahdavi, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.13461 · Cited by 310
Like What You Like: Knowledge Distill via Neuron Selectivity Transfer
Huang, Z., & Wang, N. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.01219 · Cited by 300
Overcoming Forgetting in Federated Learning on Non-IID DataPreprint
Shoham, N., Avidor, T., Keren, A., Israel, N., Benditkis, D., Mor-Yosef, L., & Zeitak, I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.07796 · Cited by 299
One-Shot Federated LearningPreprint
Guha, N., Talwalkar, A., & Smith, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.11175 · Cited by 285
SCAFFOLD: Stochastic Controlled Averaging for On-Device Federated Learning.
Karimireddy, S. P., Kale, S., Mohri, M., Reddi, S. J., Stich, S. U., & Suresh, A. T. (2019) · Cited by 227
Self-citationCronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge TransferPreprint
Chang, H., Shejwalkar, V., Shokri, R., & Houmansadr, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.11279 · Cited by 225
Nedic, A. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2975210 · Cited by 211
Don't Use Large Mini-Batches, Use Local SGD
Lin, T., Stich, S. U., Patel, K. K., & Jaggi, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1808.07217 · Cited by 193
Self-citationDiversity With Cooperation: Ensemble Methods for Few-Shot Classification
Dvornik, N., Mairal, J., & Schmid, C. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00382 · Cited by 189
Distilled One-Shot Federated LearningPreprint
Zhou, Y., Pu, G., Ma, X., Li, X., & Wu, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.07999 · Cited by 181
Federated Visual Classification with Real-World Data Distribution
Hsu, T.-M. H., Qi, H., & Brown, M. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58607-2_5 · Cited by 178
Federated Model Distillation with Noise-Free Differential Privacy
Sun, L., & Lyu, L. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/216 · Cited by 130
Dynamic Model Pruning with Feedback
Lin, T., Stich, S. U., Barba, L., Dmitriev, D., & Jaggi, M. (2020) · arXiv · doi:10.48550/arxiv.2006.07253 · Cited by 126
Self-citationDistilled Person Re-Identification: Towards a More Scalable System
Wu, A., Zheng, W., Guo, X., & Lai, J. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00128 · Cited by 110
Algorithms and Theory for Multiple-Source AdaptationPreprint
Hoffman, J., Mohri, M., & Zhang, N. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.08727 · Cited by 108
FEED: Feature-level Ensemble for Knowledge DistillationPreprint
Park, S., & Kwak, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.10754 · Cited by 46
LIT: Learned Intermediate Representation Training for Model Compression
Koratana, A., Kang, D. D., Bailis, P. D., & Zaharia, M. (2019) · International Conference on Machine Learning · Cited by 42
Group Knowledge Transfer: Collaborative Training of Large CNNs on the Edge.Preprint
He, C., Annavaram, M., & Avestimehr, S. (2020) · arXiv (Cornell University) · Cited by 18
Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations
Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., & Bengio, Y. (2016) · arXiv · doi:10.48550/arxiv.1609.07061
Cited by
Knowledge Distillation: A Survey
Gou, J., Yu, B., Maybank, S. J., & Tao, D. (2021) · International Journal of Computer Vision · doi:10.1007/s11263-021-01453-z · Cited by 3,931
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
Li, Q., Wen, Z., Wu, Z., Hu, S., Wang, N., Li, Y., Liu, X., & He, B. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3124599 · Cited by 1,407
Federated Learning on Non-IID Data Silos: An Experimental Study
Li, Q., Diao, Y., Chen, Q., & He, B. (2022) · 2022 IEEE 38th International Conference on Data Engineering (ICDE) · doi:10.1109/icde53745.2022.00077 · Cited by 1,132
Federated learning on non-IID data: A survey
Zhu, H., Xu, J., Liu, S., & Jin, Y. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.07.098 · Cited by 1,062
FedProto: Federated Prototype Learning across Heterogeneous Clients
Tan, Y., Long, G., LIU, L., Zhou, T., Lu, Q., Jiang, J., & Zhang, C. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20819 · Cited by 761
Metadata
- Authors
- Lin, T., Kong, L., Stich, S. U., & Jaggi, M.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2020-06-12
- Volume
- 33
- Pages
- 2351-2363
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2006.07242
- OpenAlex
- W3035453001
- Cited by
- 476
- References
- 76
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.