Applied Federated Learning: Improving Google Keyboard Query Suggestions
Abstract
Federated learning is a distributed form of machine learning where both the training data and model training are decentralized. In this paper, we use federated learning in a commercial, global-scale setting to train, evaluate and deploy a model to improve virtual keyboard search suggestion quality without direct access to the underlying user data. We describe our observations in federated training, compare metrics to live deployments, and present resulting quality increases. In whole, we demonstrate how federated learning can be applied end-to-end to both improve user experiences and enhance user privacy.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Google makes extensive use of federated learning in the Gboard mobile keyboard, as well as in features on Pixel phones and in Android Messages.
“Google makes extensive use of federated learning in the Gboard mobile keyboard [376, 222, 491, 112, 383], as well as in features on Pixel phones [14] and in Android Messages [439].”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [14]
Cross-device federated learning and federated data analysis are now being applied in consumer digital products.
“Cross-device federated learning and federated data analysis are now being applied in consumer digital products.”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
No references could be resolved yet.
Research works printed but not linked: 14
- Brendan McMahan, Eider Moore, Daniel Ram- age, Seth Hampson, and Blaise Agüera y Arcas, "Communication-efficient learning of deep networks from decentralized data," in Proceedings of the 20th International Conference on Artificial Intelligence and Statistics, AISTATS 2017, 20-22 April 2017, Fort Laud- erdale, FL, USA, 2017, pp. 1273-1282.
- Jakub Konen, H. Brendan McMahan, Felix X. Yu, Pe- ter Richtarik, Ananda Theertha Suresh, and Dave Ba- con, "Federated learning: Strategies for improving com- munication efficiency," in NIPS Workshop on Private Multi-Party Machine Learning, 2016.
- Jakub Konecný, H. Brendan McMahan, Daniel Ram- age, and Peter Richtárik, "Federated optimization: Dis- tributed machine learning for on-device intelligence," CoRR, vol. abs/1610.02527, 2016. · arXiv
- Virginia Smith, Chao-Kai Chiang, Maziar Sanjabi, and Ameet S Talwalkar, "Federated multi-task learning," in Advances in Neural Information Processing Systems 30, I.
- Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fer- gus, S. Vishwanathan, and R. Garnett, Eds., pp. 4424- 4434.
- Blake E Woodworth, Jialei Wang, Adam Smith, Bren- dan McMahan, and Nati Srebro, "Graph oracle mod- els, lower bounds, and gaps for parallel stochastic opti- mization," in Advances in Neural Information Process- ing Systems 31, S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds., pp. 8504-8514.
- Cynthia Dwork, "Differential privacy," in 33rd Inter- national Colloquium on Automata, Languages and Pro- gramming, part II (ICALP 2006), Venice, Italy, July 2006, vol. 4052, pp. 1-12, Springer Verlag.
- H. Brendan McMahan, Daniel Ramage, Kunal Talwar, and Li Zhang, "Learning differentially private lan- guage models without losing accuracy," CoRR, vol. abs/1710.06963, 2017. · arXiv
- Keith Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth, "Practical secure aggregation for privacy-preserving machine learning," in Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, New York, NY, USA, 2017, CCS '17, pp. 1175-1191, ACM.
- Naman Agarwal, Ananda Theertha Suresh, Felix Xin- nan X Yu, Sanjiv Kumar, and Brendan McMahan, "cpsgd: Communication-efficient and differentially- private distributed sgd," in Advances in Neural In- formation Processing Systems 31, S. Bengio, H. Wal- lach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, Eds., pp. 7574-7585.
- Martin Abadi, Andy Chu, Ian Goodfellow, Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang, "Deep learning with differential privacy," in 23rd ACM Conference on Computer and Communications Security (ACM CCS), 2016, pp. 308-318.
- Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, San- jay Ghemawat, Geoffrey Irving, Michael Isard, Man- junath Kudlur, Josh Levenberg, Rajat Monga, Sherry Moore, Derek Gordon Murray, Benoit Steiner, Paul A. Tucker, Vijay Vasudevan, Pete Warden, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng, "Tensorflow: A system for large-scale machine learning," in 12th USENIX Sym- posium on Operating Systems Design and Implementa- tion, OSDI 2016, Savannah, GA, USA, November 2-4, 2016., 2016, pp. 265-283.
- S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural Computation, vol. 9, no. 8, pp. 1735- 1780, Nov 1997.
- G. Legros, World Health Organization, and United Na- tions Development Programme, The Energy Access Sit- uation in Developing Countries: A Review Focusing on the Least Developed Countries and Sub-Saharan Africa, World Health Organization, 2009.
Cited by
Sorted by citations
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772 · see citing works
Self-citationA survey on federated learning
Zhang, C., Xie, Y., Bai, H., Yu, B., Li, W., & Gao, Y. (2021) · Knowledge-Based Systems · doi:10.1016/j.knosys.2021.106775 · Cited by 1,899 · see citing works
Self-citationA review of applications in federated learning
Li, L., Fan, Y., Tse, M., & Lin, K.-Y. (2020) · Computers & Industrial Engineering · doi:10.1016/j.cie.2020.106854 · Cited by 1,559 · see citing works
A survey on security and privacy of federated learning
Mothukuri, V., Parizi, R. M., Pouriyeh, S., Huang, Y., Dehghantanha, A., & Srivastava, G. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2020.10.007 · Cited by 1,480 · see citing works
A Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
Li, Q., Wen, Z., Wu, Z., Hu, S., Wang, N., Li, Y., Liu, X., & He, B. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3124599 · Cited by 1,407 · see citing works
Abdulrahman, S., Tout, H., Ould-Slimane, H., Mourad, A., Talhi, C., & Guizani, M. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3030072 · Cited by 796 · see citing works
A Survey on Federated Learning for Resource-Constrained IoT Devices
Imteaj, A., Thakker, U., Wang, S., Li, J., & Amini, M. H. (2022) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3095077 · Cited by 774 · see citing works
Federated Learning: A Survey on Enabling Technologies, Protocols, and Applications
Aledhari, M., Razzak, R., Parizi, R. M., & Saeed, F. (2020) · IEEE Access · doi:10.1109/access.2020.3013541 · Cited by 733 · see citing works
Privacy Preserving Machine Learning with Homomorphic Encryption and Federated Learning
Fang, H., & Qian, Q. (2021) · Future Internet · doi:10.3390/fi13040094 · Cited by 424 · see citing works
Attack of the tails: Yes, you really can backdoor federated learning
Wang, H., Sreenivasan, K., Rajput, S., Vishwakarma, H., Agarwal, S., Sohn, J.-Y., Lee, K., & Papailiopoulos, D. (2020) · arXiv · doi:10.48550/arxiv.2007.05084 · Cited by 406 · see citing works
Privacy preservation in federated learning: An insightful survey from the GDPR perspective
Truong, N., Sun, K., Wang, S., Guitton, F., & Guo, Y. (2021) · Computers & Security · doi:10.1016/j.cose.2021.102402 · Cited by 399 · see citing works
Dynamic-Fusion-Based Federated Learning for COVID-19 Detection
Zhang, W., Zhou, T., Lu, Q., Wang, X., Zhu, C., Sun, H., Wang, Z., Lo, S. K., & Wang, F.-Y. (2021) · IEEE Internet of Things Journal · doi:10.1109/jiot.2021.3056185 · Cited by 329 · see citing works
Self-citationTurbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning
So, J., Guler, B., & Avestimehr, A. S. (2021) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2021.3054610 · Cited by 318 · see citing works
Byzantine-Resilient Secure Federated Learning
So, J., Guler, B., & Avestimehr, A. S. (2021) · IEEE Journal on Selected Areas in Communications · doi:10.1109/jsac.2020.3041404 · Cited by 313 · see citing works
Duan, M., Liu, D., Chen, X., Tan, Y., Ren, J., Qiao, L., & Liang, L. (2019) · 2019 IEEE 37th International Conference on Computer Design (ICCD) · doi:10.1109/iccd46524.2019.00038 · Cited by 250 · see citing works
Applications of Federated Learning; Taxonomy, Challenges, and Research Trends
Shaheen, M., Farooq, M. S., Umer, T., & Kim, B.-S. (2022) · Electronics · doi:10.3390/electronics11040670 · Cited by 157 · see citing works
Local and Central Differential Privacy for Robustness and Privacy in Federated Learning
Naseri, M., Hayes, J., & De Cristofaro, E. (2022) · Proceedings 2022 Network and Distributed System Security Symposium · doi:10.14722/ndss.2022.23054 · Cited by 188 · see citing works
Challenges, Applications and Design Aspects of Federated Learning: A Survey
Rahman, K. M. J., Ahmed, F., Akhter, N., Hasan, M., Amin, R., Aziz, K. E., Islam, A. K. M. M., Mukta, M. S. H., & Islam, A. K. M. N. (2021) · IEEE Access · doi:10.1109/access.2021.3111118 · Cited by 147 · see citing works
Challenges and future directions of secure federated learning: a survey
Zhang, K., Song, X., Zhang, C., & Yu, S. (2022) · Frontiers of Computer Science · doi:10.1007/s11704-021-0598-z · Cited by 127 · see citing works
A comprehensive survey on federated learning techniques for healthcare informatics
Dasaradharami Reddy, K., & Gadekallu, T. R. (2023) · Computational Intelligence and Neuroscience · doi:10.1155/2023/8393990 · Cited by 126 · see citing works
Pandl, K. D., Thiebes, S., Schmidt-Kraepelin, M., & Sunyaev, A. (2020) · IEEE Access · doi:10.1109/access.2020.2981447 · Cited by 115 · see citing works
Generative Models for Effective ML on Private, Decentralized Datasets
Augenstein, S., McMahan, H. B., Ramage, D., Ramaswamy, S., Kairouz, P., Chen, M., Mathews, R., & Arcas, B. A. Y. (2019) · arXiv · doi:10.48550/arxiv.1911.06679 · Cited by 103 · see citing works
Self-citationAdam, M., & Baroudi, U. (2024) · IEEE Open Journal of the Communications Society · doi:10.1109/ojcoms.2024.3506214 · Cited by 46 · see citing works
BayBFed: Bayesian Backdoor Defense for Federated Learning
Kumari, K., Rieger, P., Fereidooni, H., Jadliwala, M., & Sadeghi, A.-R. (2023) · 2023 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp46215.2023.10179362 · Cited by 37 · see citing works
On the Outsized Importance of Learning Rates in Local Update Methods
Charles, Z., & Konečný, J. (2020) · arXiv · doi:10.48550/arxiv.2007.00878 · Cited by 32 · see citing works
Communication-Efficient and Distributed Learning Over Wireless Networks: Principles and Applications
Park, J., Samarakoon, S., Elgabli, A., Kim, J., Bennis, M., Kim, S.-L., & Debbah, M. (2021) · Proceedings of the IEEE · doi:10.1109/jproc.2021.3055679 · Cited by 25 · see citing works
A comprehensive survey and taxonomy on privacy-preserving deep learning
Tran, A.-T., Luong, T.-D., & Huynh, V.-N. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127345 · Cited by 21 · see citing works
Robust Federated Learning for Ubiquitous Computing through Mitigation of Edge-Case Backdoor Attacks
Elhattab, F., Bouchenak, S., Talbi, R., & Nitu, V. (2022) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3569492 · Cited by 11 · see citing works
- Authors
- Yang, T., Andrew, G., Eichner, H., Sun, H., Li, W., Kong, N., Ramage, D., & Beaufays, F.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2018
- arXiv
- 1812.02903
- References
- 0
- Type
- Preprint
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
