The DOI matches its DataCite record; 2 of the 4 references it cites were found in a registry or an index, and 2 are not checked yet.
Learning Differentially Private Recurrent Language Models
Open accessAbstract
We demonstrate that it is possible to train large recurrent language models with user-level differential privacy guarantees with only a negligible cost in predictive accuracy. Our work builds on recent advances in the training of deep networks on user-partitioned data and privacy accounting for stochastic gradient descent. In particular, we add user-level privacy protection to the federated averaging algorithm, which makes "large step" updates from user-level data. Our work demonstrates that given a dataset with a sufficiently large number of users (a requirement easily met by even small internet-scale datasets), achieving differential privacy comes at the cost of increased computation, rather than in decreased utility as in most prior work. We find that our private LSTM language models are quantitatively and qualitatively similar to un-noised models when trained on a large dataset.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This proved that, with sufficient devices participating in federated training, privacy protection could be achieved with a small additional overhead.”
From Survey: federated learning data security and privacy-preserving in edge-Internet of Things (2024) · cites this paper as McMahan et al. 2017
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Simultaneously, both approaches guaranteed high model accuracy.”
From Survey: federated learning data security and privacy-preserving in edge-Internet of Things (2024) · cites this paper as McMahan et al. 2017
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Therefore, federated learning has a promising application in NLP like virtual keyboard prediction [7, 39, 70].”
From Federated Learning for Healthcare Informatics (2021) · cites this paper as [70]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Research works printed but not linked: 14
- Martin Abadi, Andy Chu, Ian Goodfellow, Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In 23rd ACM Conference on Computer and Communications Security (ACM CCS) , 2016a.
- Rami Al-Rfou, Marc Pickett, Javier Snaider, Yun-Hsuan Sung, Brian Strope, and Ray Kurzweil. Conversational contextual cues: The case of personalization and history for response ranking. CoRR , abs/1606.00372, 2016. URL http://arxiv.org/abs/1606.00372 . · arXiv
- R. Bassily, K. Nissim, U. Stemmer, and A. Thakurta. Practical locally private heavy hitters. ArXiv e-prints , July 2017. URL https://arxiv.org/abs/1707.04982 . · arXiv
- N. Carlini, C. Liu, J. Kos, ´ U. Erlingsson, and D. Song. The Secret Sharer: Measuring Unintended Neural Network Memorization & Extracting Secrets. ArXiv e-prints , February 2018. URL https://arxiv. org/abs/1802.08232 .
- T-H Hubert Chan, Mingfei Li, Elaine Shi, and Wenchang Xu. Differentially private continual monitoring of heavy hitters from distributed streams. In International Symposium on Privacy Enhancing Technologies Symposium , pages 140-159. Springer, 2012.
- Yarin Gal and Zoubin Ghahramani. A theoretically grounded application of dropout in recurrent neural networks. In Advances in Neural Information Processing Systems , pages 1019-1027, 2016.
- Edouard Grave, Armand Joulin, and Nicolas Usunier. Improving neural language models with a continuous cache. arXiv preprint arXiv:1612.04426 , 2016. · arXiv
- Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv preprint arXiv:1609.07843 , 2016. · arXiv
- Nicolas Papernot, Mart´ ın Abadi, ´ Ulfar Erlingsson, Ian Goodfellow, and Kunal Talwar. Semi-supervised knowledge transfer for deep learning from private training data. In Proceedings of the International Conference on Learning Representations , 2017. URL https://arxiv.org/abs/1610.05755 . · arXiv
- Ofir Press and Lior Wolf. Using the output embedding to improve language models. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers , pages 157-163. Association for Computational Linguistics, 2017.
- Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. In IEEE Symposium on Security and Privacy, Oakland , 2017. To Appear.
- Martin Sundermeyer, Ralf Schl¨ uter, and Hermann Ney. LSTM neural networks for language modeling. In Interspeech , pages 194-197, 2012.
- Xi Wu, Fengan Li, Arun Kumar, Kamalika Chaudhuri, Somesh Jha, and Jeffrey F. Naughton. Bolt-on differential privacy for scalable stochastic gradient descent-based analytics. In Proceedings of SIGMOD , 2017.
- Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. 2017. URL https://arxiv.org/abs/1611.03530 . · arXiv
Cited by
Sorted by citations
Items 101–200 of 852
Federated Model Distillation with Noise-Free Differential Privacy
Sun, L., & Lyu, L. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/216 · Cited by 117
Al-Quraan, M. M., Mohjazi, L., Bariah, L., Centeno, A., Zoha, A., Arshad, K., Assaleh, K. T., Muhaidat, S., Debbah, M. A., & Imran, M. A. (2023) · IEEE Transactions on Emerging Topics in Computational Intelligence · doi:10.1109/tetci.2023.3251404 · Cited by 159
Fairness and privacy preserving in federated learning: A survey
Rafi, T. H., Noor, F. A., Hussain, T., & Chae, D. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102198 · Cited by 138
The distributed discrete gaussian mechanism for federated learning with secure aggregation
Kairouz, P., Liu, Z., & Steinke, T. (2021) · arXiv · doi:10.48550/arxiv.2102.06387 · Cited by 117
Source Inference Attacks in Federated Learning
Hu, H., Salcic, Z. A., Sun, L., Dobbie, G., & Zhang, X. (2021) · Proceedings · doi:10.1109/icdm51629.2021.00129 · Cited by 124
Deep Learning towards Mobile Applications
Wang, J., Cao, B., Yu, P. S., Sun, L., Bao, W., & Zhu, X. (2018) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2018.00139 · Cited by 118
Gradient-Leakage Resilient Federated Learning
Wei, W., Liu, L., Wu, Y., Su, G., & Iyengar, A. (2021) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs51616.2021.00081 · Cited by 123
PFLF: Privacy-Preserving Federated Learning Framework for Edge Computing
Zhou, H., Yang, G., Dai, H., & Liu, G. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3174394 · Cited by 116
Billion-scale federated learning on mobile clients
Niu, C., Wu, F., Tang, S., Hua, L., Jia, R., Lv, C., Wu, Z., & Chen, G. (2020) · ACM Annual International Conference on Mobile Computing and Networking (MobiCom) · doi:10.1145/3372224.3419188 · Cited by 111
FLAME: Differentially Private Federated Learning in the Shuffle Model
Liu, R., Cao, Y., Chen, H., Guo, R., & Yoshikawa, M. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i10.17053 · Cited by 111
FedDM: Iterative Distribution Matching for Communication-Efficient Federated Learning
Xiong, Y., Wang, R., Cheng, M., Yu, F., & Hsieh, C. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.01566 · Cited by 119
FedGKD: Toward Heterogeneous Federated Learning via Global Knowledge Distillation
Yao, D., Pan, W., Dai, Y., Wan, Y., Ding, X., Chen, Y., Jin, H., Xu, Z., & Sun, L. (2023) · IEEE Transactions on Computers · doi:10.1109/tc.2023.3315066 · Cited by 107
FedMix: Approximation of Mixup under Mean Augmented Federated LearningPreprint
Yoon, T., Shin, S., Hwang, S. J., & Yang, E. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.00233 · Cited by 152
Privacy in Deep Learning: A Survey
Mireshghallah, F., Taram, M., Vepakomma, P., Singh, A., Raskar, R., & Esmaeilzadeh, H. (2020) · arXiv · doi:10.48550/arxiv.2004.12254 · Cited by 106
Privacy-Preserving Federated Learning With Malicious Clients and Honest-but-Curious Servers
Le, J., Zhang, D., Lei, X., Jiao, L., Zeng, K., & Liao, X. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3295949 · Cited by 110
Make Landscape Flatter in Differentially Private Federated Learning
Shi, Y., Liu, Y., Wei, K., Shen, L., Wang, X., & Tao, D. (2023) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52729.2023.02352 · Cited by 113
Survey: federated learning data security and privacy-preserving in edge-Internet of Things
Li, H., Ge, L., & Tian, L. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10774-7 · Cited by 112
Generative Models for Effective ML on Private, Decentralized Datasets
Augenstein, S., McMahan, H. B., Ramage, D., Ramaswamy, S., Kairouz, P., Chen, M., Mathews, R., & Arcas, B. A. Y. (2019) · arXiv · doi:10.48550/arxiv.1911.06679 · Cited by 103
Self-citationPractical One-Shot Federated Learning for Cross-Silo Setting
Li, Q., He, B., & Song, D. X. (2021) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2021/205 · Cited by 147
Tempered Sigmoid Activations for Deep Learning with Differential Privacy
Papernot, N., Thakurta, A., Song, S., Chien, S., & Erlingsson, Ú. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i10.17123 · Cited by 110
A Comprehensive Survey on Local Differential Privacy
Xiong, X., Liu, S., Li, D., Cai, Z., & Niu, X. (2020) · Security and Communication Networks · doi:10.1155/2020/8829523 · Cited by 110
A Review of Privacy-Preserving Federated Learning for the Internet-of-Things
Briggs, C., Fan, Z., & Andras, P. (2021) · Studies in Computational Intelligence · doi:10.1007/978-3-030-70604-3_2 · Cited by 101
RoFL: Robustness of Secure Federated Learning
Lycklama, H., Burkhalter, L., Viand, A., Küchler, N., & Hithnawi, A. (2023) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46215.2023.10179400 · Cited by 145
Differentially Private Fine-tuning of Language Models
Yu, D., Naik, S., Bačkurs, A., Gopi, S., Inan, H. A., Kamath, G., Kulkarni, J., Lee, Y. T., Manoel, A., Wutschitz, L., et al. (2024) · Journal of Privacy and Confidentiality · doi:10.29012/jpc.880 · Cited by 307
Differentially Private Fine-tuning of Language Models
Yu, D., Naik, S., Backurs, A., Gopi, S., Inan, H. A., Kamath, G., Kulkarni, J., Lee, Y. T., Manoel, A., Wutschitz, L., et al. (2021) · arXiv · doi:10.48550/arxiv.2110.06500 · Cited by 101
Large Language Models Can Be Strong Differentially Private LearnersPreprint
Li, X., Tramèr, F., Liang, P., & Hashimoto, T. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.05679 · Cited by 291
Generalised Differential Privacy for Text Document Processing
Fernandes, N., Dras, M., & McIver, A. K. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-17138-4_6 · Cited by 104
Understanding Gradient Clipping in Private SGD: A Geometric PerspectivePreprint
Chen, X., Wu, Z. S., & Hong, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.15429 · Cited by 98
Security and Privacy Threats to Federated Learning: Issues, Methods, and Challenges
Zhang, J., Zhu, H., Wang, F., Zhao, J., Xu, Q., & Li, H. (2022) · Security and Communication Networks · doi:10.1155/2022/2886795 · Cited by 105
Cronus: Robust and Heterogeneous Collaborative Learning with Black-Box Knowledge TransferPreprint
Chang, H., Shejwalkar, V., Shokri, R., & Houmansadr, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.11279 · Cited by 176
Gradient Inversion with Generative Image PriorPreprint
Jeon, J., Kim, J., Lee, K., Oh, S., & Ok, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.14962 · Cited by 97
Enabling Execution Assurance of Federated Learning at Untrusted Participants
Zhang, X., Li, F., Zhang, Z., Li, Q. L., Wang, C., & Wu, J. (2020) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom41043.2020.9155414 · Cited by 95
Differentially Private Learning Needs Better Features (or Much More Data)Preprint
Tramèr, F., & Boneh, D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.11660 · Cited by 171
No One Left Behind: Inclusive Federated Learning over Heterogeneous Devices
Liu, R., Wu, F., Wu, C., Wang, Y., Lyu, L., Chen, H., & Xie, X. (2022) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3534678.3539086 · Cited by 99
Privacy-Preserving Machine Learning: Methods, Challenges and DirectionsPreprint
Xu, R., Baracaldo, N., & Joshi, J. B. D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.04417 · Cited by 233
Private Retrieval, Computing, and Learning: Recent Progress and Future Challenges
Ulukuş, Ş., Avestimehr, S., Gastpar, M., Jafar, S. A., Tandon, R., & Tian, C. (2022) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2022.3142358 · Cited by 92
Protect Privacy from Gradient Leakage Attack in Federated Learning
Wang, J., Guo, S., Xie, X. N., & Qi, H. (2022) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom48880.2022.9796841 · Cited by 94
Split Federated Learning for 6G Enabled-Networks: Requirements, Challenges, and Future Directions
Hafi, H., Brik, B., Frangoudis, P. A., Ksentini, A., & Bagaa, M. (2024) · IEEE Access · doi:10.1109/access.2024.3351600 · Cited by 90
Differentially Private Federated Learning With an Adaptive Noise Mechanism
Xue, R., Xue, K., Zhu, B., Luo, X., Zhang, T., Sun, Q., & Lü, J. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3318944 · Cited by 93
Boosting Privately: Federated Extreme Gradient Boosting for Mobile Crowdsensing
Liu, Y., Ma, Z., Liu, X., Ma, S., Nepal, S., Deng, R. H., & Ren, K. (2020) · 2020 IEEE 40th International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs47774.2020.00017 · Cited by 93
Efficient Federated Learning for Cloud-Based AIoT Applications
Zhang, X., Hu, M., Xia, J., Wei, T., Chen, M., & Hu, S. (2020) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2020.3046665 · Cited by 89
A Study of Face Obfuscation in ImageNetPreprint
Yang, K., Yau, J., Fei-Fei, L., Deng, J., & Russakovsky, O. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.06191 · Cited by 116
Analyzing Information Leakage of Updates to Natural Language Models
Zanella-Béguelin, S., Wutschitz, L., Tople, S., Rühle, V., Paverd, A., Ohrimenko, O., Köpf, B., & Brockschmidt, M. (2020) · Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3372297.3417880 · Cited by 86
MaskCrypt: Federated Learning With Selective Homomorphic Encryption
Hu, C., & Li, B. (2024) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2024.3392424 · Cited by 98
A Comprehensive Empirical Study of Heterogeneity in Federated Learning
Abdelmoniem, A. M., Ho, C.-Y., Papageorgiou, P., & Canini, M. (2023) · IEEE Internet of Things Journal · doi:10.1109/jiot.2023.3250275 · Cited by 85
Label Leakage and Protection in Two-party Split LearningPreprint
Li, O., Sun, J., Xin, Y., Gao, W., Zhang, H., Xie, J., Smith, V., & Wang, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.08504 · Cited by 83
Gradient Leakage Attack Resilient Deep Learning
Wei, W., & Liu, L. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2021.3139777 · Cited by 87
Anonymisation Models for Text Data: State of the art, Challenges and Future Directions
Lison, P., Pilán, I., Sanchez, D., Batet, M., & Øvrelid, L. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.323 · Cited by 80
Petersen, E., Potdevin, Y., Mohammadi, E., Zidowitz, S., Breyer, S., Nowotka, D., Henn, S., Pechmann, L., Leucker, M., Rostalski, P., et al. (2022) · IEEE Access · doi:10.1109/access.2022.3178382 · Cited by 92
Pilán, I., Lison, P., Øvrelid, L., Papadopoulou, A. I., Sánchez, D., & Batet, M. (2022) · Computational Linguistics · doi:10.1162/coli_a_00458 · Cited by 107
Federated Reconstruction: Partially Local Federated LearningPreprint
Singhal, K., Sidahmed, H., Garrett, Z., Wu, S., Rush, K., & Prakash, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.03448 · Cited by 78
Membership Privacy for Machine Learning Models Through Knowledge Transfer
Shejwalkar, V., & Houmansadr, A. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i11.17150 · Cited by 90
A survey: Distributed Machine Learning for 5G and beyond
Nassef, O., Sun, W., Purmehdi, H., Tatipamula, M., & Mahmoodi, T. (2022) · Computer Networks · doi:10.1016/j.comnet.2022.108820 · Cited by 79
Arunan, A., Qin, Y., Li, X., & Yuen, C. (2023) · IEEE Transactions on Automation Science and Engineering · doi:10.1109/tase.2023.3274648 · Cited by 80
Xu, M., Qian, F., Mei, Q., Huang, K. R., & Liu, X. (2018) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3287075 · Cited by 76
FedSAE: A Novel Self-Adaptive Federated Learning Framework in Heterogeneous Systems
Li, L., Duan, M., Liu, D., Zhang, Y., Ren, A., Chen, X., Tan, Y., & Wang, C. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9533876 · Cited by 78
Federated Unlearning: A Survey on Methods, Design Guidelines, and Evaluation Metrics
Romandini, N., Mora, A., Mazzocca, C., Montanari, R., & Bellavista, P. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3478334 · Cited by 76
Scalable Privacy-Preserving Distributed Learning
Froelicher, D., Troncoso-Pastoriza, J. R., Pyrgelis, A., Sav, S., Sousa, J. S., Bossuat, J.-P., & Hubaux, J. (2021) · Proceedings on Privacy Enhancing Technologies · doi:10.2478/popets-2021-0030 · Cited by 82
Performance-Enhanced Federated Learning With Differential Privacy for Internet of Things
Shen, X., Liu, Y., & Zhang, Z. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2022.3189361 · Cited by 75
Real-World Image Datasets for Federated LearningPreprint
Luo, J., Wu, X., Luo, Y., Huang, A., Huang, Y., Liu, Y., & Yang, Q. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1910.11089 · Cited by 110
A robust analysis of adversarial attacks on federated learning environments
Nair, A. K., Raj, E. D., & Sahoo, J. (2023) · Computer Standards & Interfaces · doi:10.1016/j.csi.2023.103723 · Cited by 70
Ahmadi, M., Taghavirashidizadeh, A., Javaheri, D., Masoumian, A., Ghoushchi, S. J., & Pourasad, Y. (2021) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2021.08.019 · Cited by 70
Federated Learning and Privacy
Bonawitz, K., Kairouz, P., McMahan, H. B., & Ramage, D. (2021) · Queue · doi:10.1145/3494834.3500240 · Cited by 76
Self-citationFederated Learning with Privacy-preserving and Model IP-right-protection
Yang, Q., Huang, A., Fan, L., Chan, C. S., Lim, J. H., Ng, K. W., Ong, D. S., & Li, B. (2023) · Machine Intelligence Research · doi:10.1007/s11633-022-1343-2 · Cited by 88
Survey: Leakage and Privacy at Inference Time
Jegorova, M., Kaul, C., Mayor, C., O'Neil, A. Q., Weir, A., Murray-Smith, R., & Tsaftaris, S. A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3229593 · Cited by 77
A Survey of Privacy Vulnerabilities of Mobile Device Sensors
Delgado-Santos, P., Stragapede, G., Tolosana, R., Guest, R., Deravi, F., & Vera-Rodríguez, R. (2022) · ACM Computing Surveys · doi:10.1145/3510579 · Cited by 79
Partially Encrypted Multi-Party Computation for Federated Learning
Sotthiwat, E., Zhen, L., Li, Z., & Zhang, C. (2021) · IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGrid) · doi:10.1109/ccgrid51090.2021.00101 · Cited by 69
Sun, Y., Ochiai, H., & Esaki, H. (2021) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2021.3133819 · Cited by 77
Triastcyn, A., & Faltings, B. V. (2020) · IEEE Intelligent Systems · doi:10.1109/mis.2020.2993966 · Cited by 71
Toward Secure and Privacy-Preserving Distributed Deep Learning in Fog-Cloud Computing
Li, Y., Li, H., Xu, G., Xiang, T., Huang, X., & Lu, R. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3012480 · Cited by 66
Federated learning meets blockchain at 6G edge
Pokhrel, S. R. (2020) · ACM MobiCom Workshop on Drone Assisted Wireless Communications for 5G and Beyond (DroneCom) · doi:10.1145/3414045.3415949 · Cited by 65
Toward Multiple Federated Learning Services Resource Sharing in Mobile Edge Networks
Nguyen, M. N. H., Tran, N. H. K., Tun, Y. K., Han, Z., & Hong, C. S. (2021) · IEEE Transactions on Mobile Computing · doi:10.1109/tmc.2021.3085979 · Cited by 64
Fog-Embedded Deep Learning for the Internet of Things
Lyu, L., Bezdek, J. C., He, X., & Jin, J. (2019) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2912465 · Cited by 61
Federated User Modeling from Hierarchical Information
Liu, Q., Wu, J., Huang, Z., Wang, H., Ning, Y., Chen, M., Chen, E., Yi, J., & Zhou, B. (2023) · ACM Transactions on Information Systems · doi:10.1145/3560485 · Cited by 65
Privacy-Preserving Incentive Mechanism Design for Federated Cloud-Edge Learning
Liu, T., Di, B., An, P., & Song, L. (2021) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2021.3100096 · Cited by 62
Adap DP-FL: Differentially Private Federated Learning with Adaptive Noise
Fu, J., Chen, Z., & Han, X. (2022) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom56396.2022.00094 · Cited by 70
Federated Heavy Hitters Discovery with Differential PrivacyPreprint
Zhu, W., Kairouz, P., McMahan, H. B., Sun, H., & Li, W. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.08534 · Cited by 66
Self-citationDifferentially Private Federated Learning on Heterogeneous DataPreprint
Noble, M., Bellet, A., & Dieuleveut, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.09278 · Cited by 60
Federated Learning of N-Gram Language Models
Chen, M., Suresh, A. T., Mathews, R., Wong, A., Allauzen, C., Beaufays, F., & Riley, M. (2019) · Conference on Computational Natural Language Learning · doi:10.18653/v1/k19-1012 · Cited by 81
Enhancing Privacy via Hierarchical Federated Learning
Wainakh, A., Guinea, A. S., Grube, T., & Mühlhäuser, M. (2020) · IEEE European Symposium on Security and Privacy Workshops (EuroS&PW) · doi:10.1109/eurospw51379.2020.00053 · Cited by 60
Federated Learning of Gboard Language Models with Differential Privacy
Xu, Z., Zhang, Y., Andrew, G., Choquette-Choo, C. A., Kairouz, P., McMahan, H. B., Rosenstock, J., & Zhang, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-industry.60 · Cited by 114
Self-citationDP-Forward: Fine-tuning and Inference on Language Models with Differential Privacy in Forward Pass
Du, M., Yue, X., Chow, S. S. M., Wang, T., Huang, C., & Sun, H. (2023) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3576915.3616592 · Cited by 82
Advances in Robust Federated Learning: A Survey With Heterogeneity Considerations
Chen, C., Liao, T., Deng, X., Wu, Z., Huang, S., & Zheng, Z. (2025) · IEEE Transactions on Big Data · doi:10.1109/tbdata.2025.3527202 · Cited by 68
On lightweight privacy-preserving collaborative learning for internet-of-things objects
Jiang, L., Tan, R., Lou, X., & Lin, G. (2019) · IEEE/ACM International Conference on Internet-of-Things Design and Implementation (IoTDI) · doi:10.1145/3302505.3310070 · Cited by 56
Returning the N to NLP: Towards Contextually Personalized Classification Models
Flek, L. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.700 · Cited by 59
Empirical Studies of Institutional Federated Learning For Natural Language Processing
Zhu, X., Wang, J., Hong, Z., & Xiao, J. Z. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.55 · Cited by 55
Differential Privacy for Text Analytics via Natural Text Sanitization
Yue, X., Du, M., Wang, T., Li, Y., Sun, H., & Chow, S. S. M. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.337 · Cited by 96
Federated learning attack surface: taxonomy, cyber defences, challenges, and future directions
Qammar, A., Ding, J., & Ning, H. (2021) · Artificial Intelligence Review · doi:10.1007/s10462-021-10098-w · Cited by 58
Security and Privacy Issues in Deep Learning
Bae, H., Jang, J., Jung, D., Jang, H., Ha, H., Lee, H., & Yoon, S. (2018) · arXiv · doi:10.48550/arxiv.1807.11655 · Cited by 56
Decentralised federated learning with adaptive partial gradient aggregation
Jiang, J., & Hu, L. (2020) · CAAI Transactions on Intelligence Technology · doi:10.1049/trit.2020.0082 · Cited by 56
Empowering federated learning techniques for privacy-preserving PV forecasting
Michalakopoulos, V., Sarantinopoulos, E., Sarmas, E., & Marinakis, V. (2024) · Energy Reports · doi:10.1016/j.egyr.2024.08.033 · Cited by 59
Averaging Is Probably Not the Optimum Way of Aggregating Parameters in Federated Learning
Xiao, P., Cheng, S., Stankovic, V., & Vukobratovic, D. (2020) · Entropy · doi:10.3390/e22030314 · Cited by 54
Differentially Private AirComp Federated Learning with Power Adaptation Harnessing Receiver Noise
Koda, Y., Yamamoto, K., Nishio, T., & Morikura, M. (2020) · Global Communications Conference · doi:10.1109/globecom42002.2020.9322199 · Cited by 57
Advancing Differential Privacy: Where We Are Now and Future Directions for Real-World Deployment
Cummings, R., Desfontaines, D., Evans, D., Geambasu, R., Huang, Y., Jagielski, M., Kairouz, P., Kamath, G., Oh, S., Ohrimenko, O., et al. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.d3197524 · Cited by 114
Naseri, M., Hayes, J., & Cristofaro, E. D. (2020) · arXiv (Cornell University) · Cited by 54
Boosting Privately: Privacy-Preserving Federated Extreme Boosting for Mobile Crowdsensing
Liu, Y., Ma, Z., Liu, X., Ma, S., Nepal, S., & Deng, R. (2019) · arXiv · doi:10.48550/arxiv.1907.10218 · Cited by 52
An Accuracy-Lossless Perturbation Method for Defending Privacy Attacks in Federated Learning
Yang, X. S., Feng, Y., Fang, W., Shao, J., Tang, X., Xia, S., & Lu, R. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512233 · Cited by 60
Practical and private (deep) learning without sampling or shuffling
Kairouz, P., McMahan, B., Song, S., Thakkar, O., Thakurta, A., & Xu, Z. (2021) · arXiv · doi:10.48550/arxiv.2103.00039 · Cited by 52
Self-citationPPFL: Privacy-preserving Federated Learning with Trusted Execution EnvironmentsPreprint
Mo, F., Haddadi, H., Katevas, K., Marin, E., Perino, D., & Kourtellis, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.14380 · Cited by 52
Lyu, L., He, X., & Li, Y. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.213 · Cited by 51
Metadata
- Authors
- McMahan, H. B., Ramage, D., Talwar, K., & Zhang, L.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2017
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1710.06963
- OpenAlex
- W2784621220
- Cited by
- 849
- References
- 0
- Citations by year
- 2026: 2 · 2025: 66 · 2024: 162 · 2023: 130 · 2022: 116 · 2021: 183 · 2020: 134 · 2019: 44 · 2018: 12
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
- Social Sciences › Sociology and Political Science › Privacy, Security, and Data Protection
- Keywords
- differential privacy, FedAvg, LSTM, privacy-utility trade-off, stochastic gradient descent, privacy accounting, large-scale datasets, privacy-preserving deep learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
