CoChat
Sign Up
PreprintCited by 7,857Privacy-Preserving Technologies in DataStochastic Gradient Optimization TechniquesDomain Adaptation and Few-Shot Learning

Advances and Open Problems in Federated Learning

Open access
Kairouz, Peter, McMahan, H. Brendan, Avent, Brendan, Bellet, Aurélien, Bennis, Mehdi, Bhagoji, Arjun Nitin, Bonawitz, Kallista, Charles, Zachary +51 more·2019·arXiv
DOI 10.48550/arxiv.1912.04977arXiv 1912.04977
Open this paper in CoChat — free accountOpen PDF

Abstract

Federated learning (FL) is a machine learning setting where many clients (e.g. mobile devices or whole organizations) collaboratively train a model under the orchestration of a central server (e.g. service provider), while keeping the training data decentralized. FL embodies the principles of focused data collection and minimization, and can mitigate many of the systemic privacy risks and costs resulting from traditional, centralized machine learning and data science approaches. Motivated by the explosive growth in FL research, this paper discusses recent advances and presents an extensive collection of open problems and challenges.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.1912.04977

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Researchers and AI developers stand to benefit from access to a potentially vast collection of real-world data, particularly impacting smaller research labs and start-ups, but it will be necessary to conduct research on algorithmic strategies for federated training, and FL-based development implies that the researcher cannot investigate or visualise all of the data on which the model is trained.

    “Researchers and AI developers stand to benefit from access to a potentially vast collection of real-world data, which will particularly impact smaller research labs and start-ups.”

    From The future of digital health with federated learning (2020) · cites this paper as [12]

    Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • Federated learning does not solve all potential privacy issues and will always carry some risks; privacy-preserving techniques for FL offer levels of protection exceeding current commercially available ML models, but there is a trade-off in performance and these techniques may affect accuracy, and future techniques or ancillary data could compromise a model previously considered low-risk.

    “It is important to note that FL does not solve all potential privacy issues and—similar to ML algorithms in general—will always carry some risks.”

    From The future of digital health with federated learning (2020) · cites this paper as [12]

    Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • Already today, federated learning has an impact on nearly all stakeholders and the entire treatment cycle, ranging from improved medical image analysis to precision medicine and collaborative drug discovery; not all technical questions have been answered yet and FL will certainly be an active research area throughout the next decade.

    “However, already today, FL has an impact on nearly all stakeholders and the entire treatment cycle, ranging from improved medical image analysis providing clinicians with better diagnostic tools, over true precision medicine by helping to find similar patients, to collaborative and accelerated drug discovery decreasing cost and time-to-market for pharma companies.”

    From The future of digital health with federated learning (2020) · cites this paper as [12]

    Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • The more rounds of local training are performed, the less it is guaranteed that the overall procedure is minimising the global loss function.

    “The more rounds of local training are performed, the less it is guaranteed that the overall procedure is minimising (Eq. 1)9,12.”

    From The future of digital health with federated learning (2020) · cites this paper as [12]

  • Federated learning is perhaps best-known from the work of Konečnỳ et al., but various other definitions have been proposed in the literature.

    “FL is perhaps best-known from the work of Konečnỳ et al.55, but various other definitions have been proposed in the literature9,11,12,20.”

    From The future of digital health with federated learning (2020) · cites this paper as [12]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

No references could be resolved yet.

Research works printed but not linked: 439

  1. Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep learning with differential privacy. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pages 308-318. ACM, 2016.
  2. Omid Abari, Hariharan Rahul, and Dina Katabi. Over-the-air function computation in sensor networks. CoRR, abs/1612.02307, 2016. URL http://arxiv.org/abs/1612.02307. · arXiv
  3. Nazmiye Ceren Abay, Yan Zhou, Murat Kantarcioglu, Bhavani Thuraisingham, and Latanya Sweeney. Privacy preserving synthetic data release using deep learning. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 510-526. Springer, 2018.
  4. John M Abowd and Ian M Schmutte. An economic analysis of privacy protection and statistical accuracy as social choices. American Economic Review, 109(1):171-202, 2019.
  5. Jayadev Acharya, Clément L Canonne, and Himanshu Tyagi. Inference under information constraints i: Lower bounds from chi-square contraction. IEEE Transactions on Information Theory, 66(12):7835-7855, 2020.
  6. Gergely Ács and Claude Castelluccia. I have a DREAM!: DIfferentially PrivatE smart Metering. In Proceed- ings of the 13th International Conference on Information Hiding, IH'11, pages 118-132, Berlin, Heidelberg, 2011. Springer-Verlag. ISBN 978-3-642-24177-2. URL http://dl.acm.org/citation.cfm?id= 2042445.2042457.
  7. Naman Agarwal, Ananda Theertha Suresh, Felix X. Yu, Sanjiv Kumar, and Brendan McMahan. cpSGD: Communication-efficient and differentially-private distributed SGD. In Advances in Neural Information Pro- cessing Systems, pages 7564-7575, 2018.
  8. Nitin Agrawal, Ali Shahin Shamsabadi, Matt J. Kusner, and Adrià Gascón. QUOTIENT: two-party secure neural network training and prediction. In In Proceedings of the ACM Conference on Computer and Communi- cation Security (CCS), 2019.
  9. Rakesh Agrawal and Ramakrishnan Srikant. Privacy-preserving data mining. In ACM SIGMOD International Conference on Management of Data, 2000.
  10. Carlos Aguilar-Melchor, Joris Barrier, Laurent Fousse, and Marc-Olivier Killijian. XPIR: Private information retrieval for everyone. Proceedings on Privacy Enhancing Technologies, 2016(2):155-174, 2016.
  11. Dan Alistarh, Demjan Grubic, Jerry Li, Ryota Tomioka, and Milan Vojnovic. QSGD: Communication-efficient SGD via gradient quantization and encoding. In NIPS -Advances in Neural Information Processing Systems, pages 1709-1720, 2017.
  12. Dan Alistarh, Zeyuan Allen-Zhu, and Jerry Li. Byzantine stochastic gradient descent. In NIPS, 2018.
  13. Inês Almeida and João Xavier. DJAM: Distributed Jacobi Asynchronous Method for Learning Personal Models. IEEE Signal Processing Letters, 25(9):1389-1392, 2018.
  14. Scott Ames, Carmit Hazay, Yuval Ishai, and Muthuramakrishnan Venkitasubramaniam. Ligero: Lightweight sublinear arguments without a trusted setup. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, CCS '17, 2017.
  15. Kareem Amin, Alex Kulesza, Andres Munoz, and Sergei Vassilvtiskii. Bounding user contributions: A bias- variance trade-off in differential privacy. In International Conference on Machine Learning, pages 263-271, 2019.
  16. Sebastian Angel, Hao Chen, Kim Laine, and Srinath T. V. Setty. PIR with compressed queries and amortized query processing. In IEEE Symposium on Security and Privacy, pages 962-979. IEEE Computer Society, 2018.
  17. George J Annas. HIPAA regulations-a new era of medical-record privacy? New England Journal of Medicine, 348(15):1486-1490, 2003.
  18. Apple. Private Federated Learning (NeurIPS 2019 Expo Talk Abstract). https://nips.cc/ ExpoConferences/2019/schedule?talk_id=40, 2019.
  19. Toshinori Araki, Jun Furukawa, Yehuda Lindell, Ariel Nof, and Kazuma Ohara. High-throughput semi-honest secure three-party computation with an honest majority. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pages 805-817. ACM, 2016.
  20. Mahmoud Assran, Nicolas Loizou, Nicolas Ballas, and Michael Rabbat. Stochastic gradient push for distributed deep learning. In ICML, 2019.
  21. Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give a false sense of security: Circumventing defenses to adversarial examples. ICML, 2018.
  22. Sean Augenstein, H. Brendan McMahan, Daniel Ramage, Swaroop Ramaswamy, Peter Kairouz, Mingqing Chen, Rajiv Mathews, and Blaise Aguera y Arcas. Generative models for effective ML on private, decentralized datasets, 2019. URL https://arxiv.org/abs/1911.06679. · arXiv
  23. Brendan Avent, Yatharth Dubey, and Aleksandra Korolova. The power of the hybrid model for mean estimation. Proceedings on Privacy Enhancing Technologies (PETS), 2020(4):48 -68, 01 Oct. 2020. doi: https://doi.org/ 10.2478/popets-2020-0062. URL https://content.sciendo.com/view/journals/popets/ 2020/4/article-p48.xml. · DOI · Paper page
  24. Brendan Avent, Aleksandra Korolova, David Zeber, Torgeir Hovden, and Benjamin Livshits. BLENDER: Enabling local search with a hybrid differential privacy model. In 26th USENIX Security Symposium (USENIX Security 17), pages 747-764, Vancouver, BC, August 2017. USENIX Association. ISBN 978-1-931971-40-9. URL https://www.usenix.org/conference/usenixsecurity17/technical-sessions/ presentation/avent.
  25. Pranjal Awasthi, Corinna Cortes, Yishay Mansour, and Mehryar Mohri. Beyond individual and group fairness. CoRR, abs/2008.09490, 2020. · arXiv
  26. László Babai, Lance Fortnow, Leonid A. Levin, and Mario Szegedy. Checking computations in polylogarithmic time. In STOC, pages 21-31. ACM, 1991.
  27. Eugene Bagdasaryan and Vitaly Shmatikov. Differential privacy has disparate impact on model accuracy. CoRR, abs/1905.12101, 2019. URL http://arxiv.org/abs/1905.12101. · arXiv
  28. Eugene Bagdasaryan, Andreas Veit, Yiqing Hua, Deborah Estrin, and Vitaly Shmatikov. How to backdoor federated learning. arXiv preprint arXiv:1807.00459, 2018. · arXiv
  29. Borja Balle, James Bell, Adrià Gascón, and Kobbi Nissim. The privacy blanket of the shuffle model. In Advances in Cryptology -CRYPTO 2019 -39th Annual International Cryptology Conference, Santa Barbara, CA, USA, August 18-22, 2019, Proceedings, Part II, pages 638-667, 2019. doi: 10.1007/978-3-030-26951-7\ 22. URL https://doi.org/10.1007/978-3-030-26951-7_22. · DOI · Paper page
  30. Borja Balle, James Bell, Adrià Gascón, and Kobbi Nissim. Private summation in the multi-message shuffle model. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, page 657-676. ACM, 2020.
  31. Leighton Pate Barnes, Yanjun Han, and Ayfer Ozgur. Lower bounds for learning distributions under commu- nication constraints via fisher information. Journal of Machine Learning Research, 21(236):1-30, 2020. URL http://jmlr.org/papers/v21/19-737.html.
  32. Leighton Pate Barnes, Huseyin A. Inan, Berivan Isik, and Ayfer Ozgur. rtop-k: A statistical estimation approach to distributed sgd. arXiv preprint arXiv:2005.10761, 2020. · arXiv
  33. Moran Baruch, Gilad Baruch, and Yoav Goldberg. A little is enough: Circumventing defenses for distributed learning. arXiv preprint arXiv:1902.06156, 2019. · arXiv
  34. Raef Bassily and Adam Smith. Local, private, efficient protocols for succinct histograms. In STOC, pages 127-135, 2015.
  35. Raef Bassily, Uri Stemmer, Abhradeep Guha Thakurta, et al. Practical locally private heavy hitters. In Advances in Neural Information Processing Systems, pages 2288-2296, 2017.
  36. Debraj Basu, Deepesh Data, Can Karakus, and Suhas N Diggavi. Qsparse-local-sgd: Distributed sgd with quantization, sparsification, and local computations. IEEE Journal on Selected Areas in Information Theory, 1 (1):217-226, 2020.
  37. Jonathan Baxter. A model of inductive bias learning. Journal of Artificial Intelligence Research, 12:149-198, 2000.
  38. Amos Beimel, Aleksandra Korolova, Kobbi Nissim, Or Sheffet, and Uri Stemmer. The power of synergy in differential privacy: Combining a small curator with local randomizers. In Conference on Information-Theoretic Cryptography (ITC), 2020. URL https://arxiv.org/abs/1912.08951. · arXiv
  39. James Henry Bell, Kallista A. Bonawitz, Adrià Gascón, Tancrède Lepoint, and Mariana Raykova. Secure single- server aggregation with (poly)logarithmic overhead. In Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, page 1253-1269. ACM, 2020.
  40. Aurélien Bellet, Rachid Guerraoui, Mahsa Taziki, and Marc Tommasi. Personalized and Private Peer-to-Peer Machine Learning. In AISTATS, 2018.
  41. Irwan Bello, Barret Zoph, Vijay Vasudevan, and Quoc V Le. Neural optimizer search with reinforcement learning. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 459- 468. JMLR. org, 2017.
  42. Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. A theory of learning from different domains. Machine learning, 79(1-2):151-175, 2010.
  43. Eli Ben-Sasson, Alessandro Chiesa, Christina Garman, Matthew Green, Ian Miers, Eran Tromer, and Madars Virza. Zerocash: Decentralized anonymous payments from bitcoin. In IEEE Symposium on Security and Privacy, pages 459-474. IEEE Computer Society, 2014.
  44. Eli Ben-Sasson, Iddo Bentov, Yinon Horesh, and Michael Riabzev. Scalable zero knowledge with no trusted setup. In CRYPTO (3), volume 11694 of Lecture Notes in Computer Science, pages 701-732. Springer, 2019.
  45. James S Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. Algorithms for hyper-parameter optimiza- tion. In Advances in Neural Information Processing Systems, pages 2546-2554, 2011.
  46. Martín Bertrán, Natalia Martínez, Afroditi Papadaki, Qiang Qiu, Miguel R. D. Rodrigues, Galen Reeves, and Guillermo Sapiro. Learning adversarially fair and transferable representations. In ICML, 2019.
  47. Arjun Nitin Bhagoji, Supriyo Chakraborty, Prateek Mittal, and Seraphin Calo. Analyzing federated learning through an adversarial lens. In Proceedings of the 36th International Conference on Machine Learning, pages 634-643, 2019.
  48. Abhishek Bhowmick, John Duchi, Julien Freudiger, Gaurav Kapoor, and Ryan Rogers. Protection against reconstruction and its applications in private federated learning. arXiv preprint arXiv:1812.00984, 2018. · arXiv
  49. Battista Biggio, Blaine Nelson, and Pavel Laskov. Poisoning attacks against support vector machines. In Pro- ceedings of the 29th International Coference on International Conference on Machine Learning, ICML'12, pages 1467-1474, USA, 2012. Omnipress. ISBN 978-1-4503-1285-1. URL http://dl.acm.org/ citation.cfm?id=3042573.3042761.
  50. Battista Biggio, Igino Corona, Davide Maiorca, Blaine Nelson, Nedim Šrndić, Pavel Laskov, Giorgio Giacinto, and Fabio Roli. Evasion attacks against machine learning at test time. In ECML-PKDD, pages 387-402. Springer, 2013.
  51. Nir Bitansky, Ran Canetti, Alessandro Chiesa, and Eran Tromer. From extractable collision resistance to suc- cinct non-interactive arguments of knowledge, and back again. In Proceedings of the 3rd Innovations in Theo- retical Computer Science Conference, ITCS '12, 2012.
  52. R. Bitar and S. E. Rouayheb. Staircase-PIR: Universally robust private information retrieval. In 2018 IEEE Information Theory Workshop (ITW), pages 1-5, Nov 2018. doi: 10.1109/ITW.2018.8613532. · DOI · Paper page
  53. Andrea Bittau, Úlfar Erlingsson, Petros Maniatis, Ilya Mironov, Ananth Raghunathan, David Lie, Mitch Rudominer, Ushasree Kode, Julien Tinnes, and Bernhard Seefeld. Prochlo: Strong privacy for analytics in the crowd. In Proceedings of the 26th Symposium on Operating Systems Principles, SOSP '17, pages 441- 459, New York, NY, USA, 2017. ACM. ISBN 978-1-4503-5085-3. doi: 10.1145/3132747.3132769. URL http://doi.acm.org/10.1145/3132747.3132769. · DOI · Paper page
  54. Davis Blalock, Jose Javier Gonzalez Ortiz, Jonathan Frankle, and John Guttag. What is the state of neural network pruning? arXiv preprint arXiv:2003.03033, 2020. · arXiv
  55. Peva Blanchard, El Mahdi El Mhamdi, Rachid Guerraoui, and Julien Stainer. Machine learning with adver- saries: Byzantine tolerant gradient descent. In Advances in Neural Information Processing Systems, 2017.
  56. Peva Blanchard, Rachid Guerraoui, Julien Stainer, et al. Machine Learning with Adversaries: Byzantine Toler- ant Gradient Descent. In Advances in Neural Information Processing Systems, pages 118-128, 2017.
  57. Dan Bogdanov, Riivo Talviste, and Jan Willemson. Deploying secure multi-party computation for financial data analysis -(short paper). In Financial Cryptography, volume 7397 of Lecture Notes in Computer Science, pages 57-64. Springer, 2012.
  58. Peter Bogetoft, Dan Lund Christensen, Ivan Damgård, Martin Geisler, Thomas P. Jakobsen, Mikkel Krøigaard, Janus Dam Nielsen, Jesper Buus Nielsen, Kurt Nielsen, Jakob Pagter, Michael I. Schwartzbach, and Tomas Toft. Secure multiparty computation goes live. In Financial Cryptography, volume 5628 of Lecture Notes in Computer Science, pages 325-343. Springer, 2009.
  59. K. A. Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H. Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for federated learning on user-held data. arXiv preprint arXiv:1611.04482, 2016. · arXiv
  60. K. A. Bonawitz, Vladimir Ivanov, Ben Kreuter, Antonio Marcedone, H Brendan McMahan, Sarvar Patel, Daniel Ramage, Aaron Segal, and Karn Seth. Practical secure aggregation for privacy-preserving machine learning. In Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security, pages 1175- 1191. ACM, 2017.
  61. K. A. Bonawitz, Hubert Eichner, Wolfgang Grieskamp, Dzmitry Huba, Alex Ingerman, Vladimir Ivanov, Chloé M Kiddon, Jakub Konečný, Stefano Mazzocchi, Brendan McMahan, Timon Van Overveldt, David Petrou, Daniel Ramage, and Jason Roselander. Towards federated learning at scale: System design. In SysML 2019, 2019. URL https://arxiv.org/abs/1902.01046. · arXiv
  62. K. A. Bonawitz, Fariborz Salehi, Jakub Konečný, Brendan McMahan, and Marco Gruteser. Federated learning with autotuned communication-efficient secure aggregation. In 2019 53nd Asilomar Conference on Signals, Systems, and Computers. IEEE, 2019.
  63. Dan Boneh, Elette Boyle, Henry Corrigan-Gibbs, Niv Gilboa, and Yuval Ishai. Zero-knowledge proofs on secret-shared data via fully linear PCPs. In CRYPTO (3), volume 11694 of Lecture Notes in Computer Science, pages 67-97. Springer, 2019.
  64. Florian Bourse, Michele Minelli, Matthias Minihold, and Pascal Paillier. Fast homomorphic evaluation of deep discretized neural networks. In CRYPTO (3), volume 10993 of Lecture Notes in Computer Science, pages 483-512. Springer, 2018.
  65. Stephen Boyd, Arpita Ghosh, Balaji Prabhakar, and Devavrat Shah. Randomized gossip algorithms. IEEE Transactions on Information Theory, 52(6):2508-2530, 2006.
  66. Zvika Brakerski. Fully homomorphic encryption without modulus switching from classical gapsvp. In CRYPTO, volume 7417 of Lecture Notes in Computer Science, pages 868-886. Springer, 2012.
  67. Zvika Brakerski, Craig Gentry, and Vinod Vaikuntanathan. (leveled) fully homomorphic encryption without bootstrapping. In ITCS, pages 309-325. ACM, 2012.
  68. Mark Braverman, Ankit Garg, Tengyu Ma, Huy L. Nguyen, and David P. Woodruff. Communication lower bounds for statistical estimation problems via a distributed data processing inequality. In Proceedings of the forty-eighth annual ACM symposium on Theory of Computing, page 1011-1020. ACM, 2016.
  69. Wieland Brendel, Jonas Rauber, and Matthias Bethge. Decision-based adversarial attacks: Reliable attacks against black-box machine learning models. arXiv preprint arXiv:1712.04248, 2017. · arXiv
  70. Theodora S Brisimi, Ruidi Chen, Theofanie Mela, Alex Olshevsky, Ioannis Ch Paschalidis, and Wei Shi. Fed- erated learning of predictive models from federated electronic health records. International journal of medical informatics, 112:59-67, 2018.
  71. Benedikt Bünz, Jonathan Bootle, Dan Boneh, Andrew Poelstra, Pieter Wuille, and Gregory Maxwell. Bullet- proofs: Short proofs for confidential transactions and more. In 2018 IEEE Symposium on Security and Privacy, SP 2018, Proceedings, 21-23 May 2018, San Francisco, California, USA, 2018.
  72. Joy Buolamwini and Timnit Gebru. Gender shades: Intersectional accuracy disparities in commercial gender classification. In Conference on fairness, accountability and transparency, pages 77-91, 2018.
  73. Sebastian Caldas, Jakub Konečný, H Brendan McMahan, and Ameet Talwalkar. Expanding the reach of feder- ated learning by reducing client resource requirements. arXiv preprint arXiv:1812.07210, 2018. · arXiv
  74. Sebastian Caldas, Peter Wu, Tian Li, Jakub Konečný, H Brendan McMahan, Virginia Smith, and Ameet Tal- walkar. LEAF: A benchmark for federated settings. arXiv preprint arXiv:1812.01097, 2018. · arXiv
  75. Clément L Canonne, Gautam Kamath, Audra McMillan, Adam Smith, and Jonathan Ullman. The structure of optimal private tests for simple hypotheses. AarXiv preprint arXiv:1811.11148, 2019. · arXiv
  76. Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In 2017 IEEE Symposium on Security and Privacy (SP), pages 39-57. IEEE, 2017.
  77. Nicholas Carlini, Chang Liu, Jernej Kos, Úlfar Erlingsson, and Dawn Song. The secret sharer: Measuring unintended neural network memorization & extracting secrets. arXiv preprint arXiv:1802.08232, 2018. · arXiv
  78. Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, et al. Extracting training data from large language models. arXiv preprint arXiv:2012.07805, 2020. · arXiv
  79. Iker Ceballos, Vivek Sharma, Eduardo Mugica, Abhishek Singh, Albert Roman, Praneeth Vepakomma, and Ramesh Raskar. SplitNN-driven vertical partitioning. arXiv preprint arXiv:2008.04137, 2018. · arXiv
  80. Khe Chai Sim, Franc ¸oise Beaufays, Arnaud Benard, Dhruv Guliani, Andreas Kabel, Nikhil Khare, Tamar Lucassen, Petr Zadrazil, Harry Zhang, Leif Johnson, et al. Personalization of end-to-end speech recognition on mobile devices for named entities. arXiv, pages arXiv-1912, 2019.
  81. T-H Hubert Chan, Elaine Shi, and Dawn Song. Privacy-preserving stream aggregation with fault tolerance. In International Conference on Financial Cryptography and Data Security, pages 200-214. Springer, 2012.
  82. Ken Chang, Niranjan Balachandar, Carson Lam, Darvin Yi, James Brown, Andrew Beers, Bruce Rosen, Daniel L Rubin, and Jayashree Kalpathy-Cramer. Distributed deep learning networks among institutions for medical imaging. Journal of the American Medical Informatics Association, 25(8):945-954, 2018.
  83. Wei-Ting Chang and Ravi Tandon. On the upload versus download cost for secure and private matrix multipli- cation. ArXiv, abs/1906.10684, 2019. · arXiv
  84. Zachary Charles and Jakub Konečnỳ. On the outsized importance of learning rates in local update methods. arXiv preprint arXiv:2007.00878, 2020. · arXiv
  85. David Chaum. Untraceable electronic mail, return addresses, and digital pseudonyms. Communications of the ACM, 24(2), 1981.
  86. Bryant Chen, Wilka Carvalho, Nathalie Baracaldo, Heiko Ludwig, Benjamin Edwards, Taesung Lee, Ian Mol- loy, and Biplav Srivastava. Detecting backdoor attacks on deep neural networks by activation clustering. arXiv preprint arXiv:1811.03728, 2018. · arXiv
  87. Chien-Lun Chen, Leana Golubchik, and Marco Paolieri. Backdoor attacks on federated meta-learning. arXiv preprint arXiv:2006.07026, 2020. · arXiv
  88. Lingjiao Chen, Hongyi Wang, Zachary B. Charles, and Dimitris S. Papailiopoulos. DRACO: Byzantine- resilient distributed training via redundant gradients. In Proceedings of the 35th International Conference on Machine Learning, ICML, 2018.
  89. Mingqing Chen, Rajiv Mathews, Tom Ouyang, and Franc ¸oise Beaufays. Federated learning of out-of- vocabulary words. arXiv preprint 1903.10635, 2019. URL http://arxiv.org/abs/1903.10635. · arXiv
  90. Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho-Jui Hsieh. ZOO: Zeroth order optimization based black-box attacks to deep neural networks without training substitute models. In Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security, pages 15-26. ACM, 2017.
  91. Wei-Ning Chen, Peter Kairouz, and Ayfer Ozgur. Breaking the communication-privacy-accuracy trilemma. Advances in Neural Information Processing Systems, 33, 2020.
  92. Xinyun Chen, Chang Liu, Bo Li, Kimberly Lu, and Dawn Song. Targeted backdoor attacks on deep learning systems using data poisoning. arXiv preprint arXiv:1712.05526, 2017. · arXiv
  93. Massimo Chenal and Qiang Tang. On key recovery attacks against existing somewhat homomorphic encryption schemes. In LATINCRYPT, volume 8895 of Lecture Notes in Computer Science, pages 239-258. Springer, 2014.
  94. Kewei Cheng, Tao Fan, Yilun Jin, Yang Liu, Tianjian Chen, and Qiang Yang. SecureBoost: A lossless federated learning framework. CoRR, abs/1901.08755, 2019. URL http://arxiv.org/abs/1901.08755. · arXiv
  95. Raymond Cheng, Fan Zhang, Jernej Kos, Warren He, Nicholas Hynes, Noah Johnson, Ari Juels, Andrew Miller, and Dawn Song. Ekiden: A platform for confidentiality-preserving, trustworthy, and performant smart contracts. In 2019 IEEE European Symposium on Security and Privacy (EuroS&P), pages 185-200. IEEE, 2019.
  96. Albert Cheu, Adam Smith, Jonathan Ullman, David Zeber, and Maxim Zhilyaev. Distributed differential privacy via shuffling. In Annual International Conference on the Theory and Applications of Cryptographic Techniques, pages 375-403. Springer, 2019.
  97. Benny Chor, Eyal Kushilevitz, Oded Goldreich, and Madhu Sudan. Private information retrieval. J. ACM, 45 (6):965-981, November 1998. ISSN 0004-5411. doi: 10.1145/293347.293350. URL http://doi.acm. org/10.1145/293347.293350. · DOI · Paper page
  98. Edward Chou, Florian Tramèr, and Giancarlo Pellegrino. SentiNet: Detecting physical attacks against deep learning systems. arXiv preprint arXiv:1812.00292, 2018. · arXiv
  99. Sélim Chraibi, Ahmed Khaled, Dmitry Kovalev, Peter Richtárik, Adil Salim, and Martin Takáč. Distributed fixed point methods with compressed iterates. arXiv preprint arXiv:1912.09925, 2019. · arXiv
  100. P. Christen. Data matching: concepts and techniques for record linkage, entity resolution, and duplicate detec- tion. Springer Science & Business Media, 2012.
  101. Gregory Cohen, Saeed Afshar, Jonathan Tapson, and André van Schaik. EMNIST: an extension of MNIST to handwritten letters. arXiv preprint arXiv:1702.05373, 2017. · arXiv
  102. Igor Colin, Aurélien Bellet, Joseph Salmon, and Stéphan Clémenc ¸on. Gossip dual averaging for decentralized optimization of pairwise functions. In ICML, 2016.
  103. Graham Cormode, Tejas Kulkarni, and Divesh Srivastava. Marginal release under local differential privacy. In Proceedings of the 2018 International Conference on Management of Data, pages 131-146. ACM, 2018.
  104. Jean-Sébastien Coron, Tancrède Lepoint, and Mehdi Tibouchi. Scale-invariant fully homomorphic encryption over the integers. In Public Key Cryptography, volume 8383 of Lecture Notes in Computer Science, pages 311-328. Springer, 2014.
  105. Henry Corrigan-Gibbs and Dan Boneh. Prio: Private, robust, and scalable computation of aggregate statistics. In 14th {USENIX} Symposium on Networked Systems Design and Implementation ({NSDI} 17), pages 259- 282, 2017.
  106. Corinna Cortes, Mehryar Mohri, Ananda Theertha Suresh, and Ningshan Zhang. Multiple-source adaptation with domain classifiers. arXiv preprint arXiv:2008.11036, 2020. · arXiv
  107. Victor Costan and Srinivas Devadas. Intel SGX explained. IACR Cryptology ePrint Archive, 2016(086):1-118, 2016.
  108. Victor Costan, Ilia Lebedev, and Srinivas Devadas. Sanctum: Minimal hardware extensions for strong software isolation. In 25th {USENIX} Security Symposium ({USENIX} Security 16), pages 857-874, 2016.
  109. Craig Costello, Cédric Fournet, Jon Howell, Markulf Kohlweiss, Benjamin Kreuter, Michael Naehrig, Bryan Parno, and Samee Zahur. Geppetto: Versatile verifiable computation. In IEEE Symposium on Security and Privacy, pages 253-270. IEEE Computer Society, 2015.
  110. Andrew Cotter, Ohad Shamir, Nati Srebro, and Karthik Sridharan. Better mini-batch algorithms via ac- celerated gradient methods. In J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K. Q. Wein- berger, editors, Advances in Neural Information Processing Systems, volume 24, pages 1647-1655.
  111. Matthieu Courbariaux, Yoshua Bengio, and Jean-Pierre David. BinaryConnect: Training deep neural networks with binary weights during propagations. In Advances in neural information processing systems, pages 3123- 3131, 2015.
  112. Pierre Courtiol, Charles Maussion, Matahi Moarii, Elodie Pronier, Samuel Pilcer, Meriem Sefta, Pierre Manceron, Sylvain Toldo, Mikhail Zaslavskiy, Nolwenn Le Stang, et al. Deep learning-based classification of mesothelioma improves prediction of patient outcome. Nature medicine, pages 1-7, 2019.
  113. Thomas M Cover and Joy A Thomas. Elements of information theory. John Wiley & Sons, 2012.
  114. Gabriela F Cretu, Angelos Stavrou, Michael E Locasto, Salvatore J Stolfo, and Angelos D Keromytis. Casting out demons: Sanitizing training data for anomaly sensors. In 2008 IEEE Symposium on Security and Privacy (sp 2008), pages 81-95. IEEE, 2008.
  115. Rachel Cummings, Sara Krehbiel, Kevin Lai, and Uthaipon Tantitongpipat. Differential privacy for growing databases. In Advances in Neural Information Processing Systems 31, NeurIPS '18, pages 8864-8873, 2018.
  116. Rachel Cummings, Sara Krehbiel, Yajun Mei, Rui Tuo, and Wanrong Zhang. Differentially private change- point detection. In Advances in Neural Information Processing Systems 31, NeurIPS '18, pages 10825-10834, 2018.
  117. Rachel Cummings, Varun Gupta, Dhamma Kimpara, and Jamie Morgenstern. On the compatibility of privacy and fairness. In Proceedings of Fairness in User Modeling, Adaptation and Personalization, FairUMAP, 2019.
  118. Edwige Cyffers and Aurélien Bellet. Privacy amplification by decentralization. arXiv preprint arXiv:2012.05326, 2020. · arXiv
  119. Deepesh Data, Linqi Song, and Suhas Diggavi. Data encoding for byzantine-resilient distributed optimization. IEEE Transactions on Information Theory, 2020.
  120. Jeffrey Dean, Greg S. Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Quoc V. Le, Mark Z. Mao, Marc'Aurelio Ranzato, Andrew Senior, Paul Tucker, Ke Yang, and Andrew Y. Ng. Large scale distributed deep networks. In Proceedings of the International Conference on Neural Information Processing Systems, pages 1223-1231, 2012.
  121. Ilias Diakonikolas, Gautam Kamath, Daniel Kane, Jerry Li, Jacob Steinhardt, and Alistair Stewart. Sever: A robust meta-algorithm for stochastic optimization. In Kamalika Chaudhuri and Ruslan Salakhutdinov, editors, Proceedings of the 36th International Conference on Machine Learning, volume 97 of Proceedings of Machine Learning Research, pages 1596-1606, Long Beach, California, USA, 09-15 Jun 2019. PMLR. URL http: //proceedings.mlr.press/v97/diakonikolas19a.html.
  122. Mario Diaz, Peter Kairouz, Jiachun Liao, and Lalitha Sankar. Theoretical guarantees for model auditing with finite adversaries. arXiv preprint arXiv:1911.03405, 2019. · arXiv
  123. Differential Privacy Team. Learning with privacy at scale. Apple Machine Learning Journal, 1(8), 2017. URL https://machinelearning.apple.com/2017/12/06/ learning-with-privacy-at-scale.html.
  124. Bolin Ding, Janardhan Kulkarni, and Sergey Yekhanin. Collecting telemetry data privately. In Advances in Neural Information Processing Systems 30, December 2017. URL https://www.microsoft.com/ en-us/research/publication/collecting-telemetry-data-privately/.
  125. Zeyu Ding, Yuxin Wang, Guanhong Wang, Danfeng Zhang, and Daniel Kifer. Detecting violations of dif- ferential privacy. In Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security, CCS '18, pages 475-489, New York, NY, USA, 2018. ACM. ISBN 978-1-4503-5693-0. doi: 10.1145/3243734.3243818. URL http://doi.acm.org/10.1145/3243734.3243818. · DOI
  126. Canh T. Dinh, Nguyen H. Tran, and Tuan Dung Nguyen. Personalized Federated Learning with Moreau En- velopes. In NeurIPS, 2020.
  127. Rafael G. L. D'Oliveira and S. E. Rouayheb. Lifting private information retrieval from two to any number of messages. In 2018 IEEE International Symposium on Information Theory (ISIT), pages 1744-1748, June 2018. doi: 10.1109/ISIT.2018.8437805. · DOI
  128. John R. Douceur. The sybil attack. In Revised Papers from the First International Workshop on Peer-to-Peer Systems, IPTPS '01, pages 251-260, London, UK, UK, 2002. Springer-Verlag. ISBN 3-540-44179-4. URL http://dl.acm.org/citation.cfm?id=646334.687813.
  129. John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of machine learning research, 12(7), 2011.
  130. John C Duchi, Michael I Jordan, and Martin J Wainwright. Local privacy and statistical minimax rates. In Foundations of Computer Science (FOCS), 2013 IEEE 54th Annual Symposium on, pages 429-438. IEEE, 2013.
  131. Sanghamitra Dutta, Gauri Joshi, Soumyadip Ghosh, Parijat Dube, and Priya Nagpurkar. Slow and Stale Gradi- ents Can Win the Race: Error-Runtime Trade-offs in Distributed SGD. International Conference on Artificial Intelligence and Statistics (AISTATS), April 2018. URL https://arxiv.org/abs/1803.01113. · arXiv
  132. Cynthia Dwork. Differential privacy: A survey of results. In International Conference on Theory and Applica- tions of Models of Computation, pages 1-19. Springer, 2008.
  133. Cynthia Dwork and Aaron Roth. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science, 9(3-4):211-407, 2014.
  134. Cynthia Dwork, Krishnaram Kenthapadi, Frank McSherry, Ilya Mironov, and Moni Naor. Our data, ourselves: Privacy via distributed noise generation. In Annual International Conference on the Theory and Applications of Cryptographic Techniques, pages 486-503. Springer, 2006.
  135. Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam D. Smith. Calibrating noise to sensitivity in private data analysis. In IACR Theory of Cryptography Conference (TCC), New York, New York, volume 3876 of Lecture Notes in Computer Science, pages 265-284. Springer-Verlag, 2006. doi: 10.1007/11681878 14. · DOI
  136. Cynthia Dwork, Guy N. Rothblum, and Salil Vadhan. Boosting and differential privacy. In Proceedings of the IEEE 51st Annual Symposium on Foundations of Computer Science, FOCS '10, pages 51-60, 2010.
  137. Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. Fairness through aware- ness. In Proceedings of the 3rd innovations in theoretical computer science conference, pages 214-226. ACM, 2012.
  138. Laurel Eckhouse, Kristian Lum, Cynthia Conti-Cook, and Julie Ciccolini. Layers of bias: A unified approach for understanding problems with risk assessment. Criminal Justice and Behavior, 46(2):185-209, 2019.
  139. Hubert Eichner, Tomer Koren, H. Brendan McMahan, Nathan Srebro, and Kunal Talwar. Semi-cyclic stochastic gradient descent. In Accepted to ICML 2019., 2019. URL https://arxiv.org/abs/1904.10120. · arXiv
  140. Karim Eldefrawy, Gene Tsudik, Aurélien Francillon, and Daniele Perito. SMART: secure and minimal archi- tecture for (establishing dynamic) root of trust. In NDSS. The Internet Society, 2012.
  141. Anis Elgabli, Jihong Park, Amrit S Bedi, Mehdi Bennis, and Vaneet Aggarwal. GADMM: Fast and communi- cation efficient framework for distributed machine learning. arXiv preprint arXiv:1909.00047, 2019. · arXiv
  142. Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. Efficient multi-objective neural architecture search via Lamarckian evolution. arXiv preprint arXiv:1804.09081, 2018. · arXiv
  143. Logan Engstrom, Brandon Tran, Dimitris Tsipras, Ludwig Schmidt, and Aleksander Madry. A rotation and a translation suffice: Fooling CNNs with simple transformations. arXiv preprint arXiv:1712.02779, 2017. · arXiv
  144. Úlfar Erlingsson, Vasyl Pihur, and Aleksandra Korolova. RAPPOR: Randomized aggregatable privacy- preserving ordinal response. In ACM CCS, 2014. ISBN 978-1-4503-2957-6. doi: 10.1145/2660267.2660348. URL http://doi.acm.org/10.1145/2660267.2660348. · DOI · Paper page
  145. Úlfar Erlingsson, Vitaly Feldman, Ilya Mironov, Ananth Raghunathan, Kunal Talwar, and Abhradeep Thakurta. Amplification by shuffling: From local to central differential privacy via anonymity. In SODA, pages 2468- 2479, 2019.
  146. Stefan Falkner, Aaron Klein, and Frank Hutter. BOHB: Robust and efficient hyperparameter optimization at scale. arXiv preprint arXiv:1807.01774, 2018. · arXiv
  147. Alireza Fallah, Aryan Mokhtari, and Asuman Ozdaglar. Personalized federated learning: A meta-learning approach. arXiv preprint arXiv:2002.07948, 2020. · arXiv
  148. Minghong Fang, Xiaoyu Cao, Jinyuan Jia, and Neil Zhenqiang Gong. Local model poisoning attacks to Byzantine-robust federated learning. arXiv preprint arXiv:1911.11815, 2019. · arXiv
  149. Vitaly Feldman, Ilya Mironov, Kunal Talwar, and Abhradeep Thakurta. Privacy amplification by iteration. In 2018 IEEE 59th Annual Symposium on Foundations of Computer Science (FOCS), pages 521-532. IEEE, 2018.
  150. Clément Feutry, Pablo Piantanida, Yoshua Bengio, and Pierre Duhamel. Learning anonymized representations with adversarial neural networks. CoRR, abs/1802.09386, 2018. URL http://arxiv.org/abs/1802. 09386. · arXiv
  151. Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks. In Proceedings of the 34th International Conference on Machine Learning, 2017.
  152. Aurélien Francillon, Quan Nguyen, Kasper Bonne Rasmussen, and Gene Tsudik. A minimalist approach to remote attestation. In DATE, pages 1-6. European Design and Automation Association, 2014.
  153. Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. Model inversion attacks that exploit confidence infor- mation and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security, pages 1322-1333. ACM, 2015.
  154. Clement Fung, Chris JM Yoon, and Ivan Beschastnikh. Mitigating sybils in federated learning poisoning. arXiv preprint arXiv:1808.04866, 2018. · arXiv
  155. Jun Furukawa, Yehuda Lindell, Ariel Nof, and Or Weinstein. High-throughput secure three-party computation for malicious adversaries and an honest majority. In EUROCRYPT (2), volume 10211 of Lecture Notes in Computer Science, pages 225-255, 2017.
  156. Adam Gaier and David Ha. Weight agnostic neural networks. arXiv preprint arXiv:1906.04358, 2019. · arXiv
  157. Venkata Gandikota, Raj Kumar Maity, and Arya Mazumdar. vqSGD: Vector quantized stochastic gradient descent. arXiv preprint arXiv:1911.07971, 2019. · arXiv
  158. Adrià Gascón, Phillipp Schoppmann, Borja Balle, Mariana Raykova, Jack Doerner, Samee Zahur, and David Evans. Privacy-preserving distributed linear regression on high-dimensional data. PoPETs, 2017(4):345-364, 2017.
  159. Rosario Gennaro, Craig Gentry, and Bryan Parno. Non-interactive verifiable computing: Outsourcing compu- tation to untrusted workers. In CRYPTO, volume 6223 of Lecture Notes in Computer Science, pages 465-482. Springer, 2010.
  160. Rosario Gennaro, Craig Gentry, Bryan Parno, and Mariana Raykova. Quadratic span programs and succinct NIZKs without PCPs. In EUROCRYPT, volume 7881 of Lecture Notes in Computer Science, pages 626-645. Springer, 2013.
  161. Craig Gentry. Fully homomorphic encryption using ideal lattices. In Proceedings of the forty-first annual ACM symposium on Theory of computing, pages 169-178, 2009.
  162. Craig Gentry and Shai Halevi. Compressible FHE with applications to PIR. In TCC (2), volume 11892 of Lecture Notes in Computer Science, pages 438-464. Springer, 2019.
  163. Robin C. Geyer, Tassilo Klein, and Moin Nabi. Differentially private federated learning: A client level perspec- tive. CoRR, abs/1712.07557, 2017. URL http://arxiv.org/abs/1712.07557. · arXiv
  164. Badih Ghazi, Noah Golowich, Ravi Kumar, Rasmus Pagh, and Ameya Velingker. On the power of multiple anonymous messages. arXiv:1908.11358, 2019. · arXiv
  165. Badih Ghazi, Rasmus Pagh, and Ameya Velingker. Scalable and differentially private distributed aggregation in the shuffled model. arXiv preprint arXiv:1906.08320, 2019. · arXiv
  166. Badih Ghazi, Noah Golowich, Ravi Kumar, Pasin Manurangsi, Rasmus Pagh, and Ameya Velingker. Pure differentially private summation from anonymous messages. In ITC, pages 15:1-15:23, 2020.
  167. Badih Ghazi, Ravi Kumar, Pasin Manurangsi, and Rasmus Pagh. Private counting from anonymous messages: Near-optimal accuracy with vanishing communication overhead. In ICML, 2020.
  168. Badih Ghazi, Pasin Manurangsi, Rasmus Pagh, and Ameya Velingker. Private aggregation from fewer anony- mous messages. In EUROCRYPT, pages 798-827, 2020.
  169. Arpita Ghosh, Tim Roughgarden, and Mukund Sundararajan. Universally utility-maximizing privacy mecha- nisms. In Proceedings of the Forty-first Annual ACM Symposium on Theory of Computing, STOC '09, pages 351-360, New York, NY, USA, 2009. ACM. ISBN 978-1-60558-506-2. doi: 10.1145/1536414.1536464. URL http://doi.acm.org/10.1145/1536414.1536464. · DOI
  170. Ran Gilad-Bachrach, Nathan Dowlin, Kim Laine, Kristin E. Lauter, Michael Naehrig, and John Wernsing. CryptoNets: Applying neural networks to encrypted data with high throughput and accuracy. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, pages 201-210, 2016. URL http://proceedings.mlr.press/v48/gilad-bachrach16. html.
  171. Antonious M Girgis, Deepesh Data, Suhas Diggavi, Peter Kairouz, and Ananda Theertha Suresh. Shuf- fled model of federated learning: Privacy, communication and accuracy trade-offs. arXiv preprint arXiv:2008.07180, 2020. · arXiv
  172. O. Goldreich, S. Micali, and A. Wigderson. How to play any mental game. In Proceedings of the Nine- teenth Annual ACM Symposium on Theory of Computing, STOC '87, pages 218-229, New York, NY, USA, 1987. ACM. ISBN 0-89791-221-7. doi: 10.1145/28395.28420. URL http://doi.acm.org/10.1145/ 28395.28420. · DOI · Paper page
  173. Shafi Goldwasser, Silvio Micali, and Charles Rackoff. The knowledge complexity of interactive proof systems. SIAM J. Comput., 18(1):186-208, 1989.
  174. Shafi Goldwasser, Yael Tauman Kalai, and Guy N. Rothblum. Delegating computation: interactive proofs for muggles. In STOC, pages 113-122. ACM, 2008.
  175. Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http://arxiv.org/abs/1412.6572. · arXiv
  176. Ian J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. ICLR, 2015.
  177. Slawomir Goryczka and Li Xiong. A comprehensive comparison of multiparty secure additions with differential privacy. IEEE Trans. Dependable Sec. Comput., 14(5):463-477, 2017. doi: 10.1109/TDSC.2015.2484326. URL https://doi.org/10.1109/TDSC.2015.2484326. · DOI
  178. Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. BadNets: Identifying vulnerabilities in the machine learning model supply chain. arXiv preprint arXiv:1708.06733, 2017. · arXiv
  179. Otkrist Gupta and Ramesh Raskar. Distributed learning of deep neural network over multiple agents. Journal of Network and Computer Applications, 116:1-8, 2018.
  180. Farzin Haddadpour, Mohammad Mahdi Kamani, Mehrdad Mahdavi, and Viveck R Cadambe. Local SGD with periodic averaging: Tighter analysis and adaptive synchronization. arXiv preprint arXiv:1910.13598, 2019. · arXiv
  181. Andreas Haeberlen, Benjamin C Pierce, and Arjun Narayan. Differential privacy under fire. In USENIX Security Symposium, 2011.
  182. Shai Halevi, Yehuda Lindell, and Benny Pinkas. Secure computation on the web: Computing without simulta- neous interaction. In Annual Cryptology Conference, pages 132-150. Springer, 2011.
  183. Jenny Hamer, Mehryar Mohri, and Ananda Theertha Suresh. Fedboost: A communication-efficient algorithm for federated learning. In International Conference on Machine Learning, pages 3973-3983. PMLR, 2020.
  184. Song Han, Huizi Mao, and William J Dally. Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding. arXiv preprint arXiv:1510.00149, 2015. · arXiv
  185. Yanjun Han, Ayfer Özgür, and Tsachy Weissman. Geometric lower bounds for distributed parameter estimation under communication constraints. In Proceedings of Machine Learning Research, pages 1-26, 75, 2018.
  186. Andrew Hard, Kanishka Rao, Rajiv Mathews, Franc ¸oise Beaufays, Sean Augenstein, Hubert Eichner, Chloé Kiddon, and Daniel Ramage. Federated learning for mobile keyboard prediction. arXiv preprint 1811.03604, 2018. · arXiv
  187. Moritz Hardt, Eric Price, and Nathan Srebro. Equality of opportunity in supervised learning. In Advances in Neural Information Processing Systems, 2016.
  188. Stephen Hardy, Wilko Henecka, Hamish Ivey-Law, Richard Nock, Giorgio Patrini, Guillaume Smith, and Brian Thorne. Private federated learning on vertically partitioned data via entity resolution and additively homomor- phic encryption. arXiv preprint arXiv:1711.10677, 2017. · arXiv
  189. Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demograph- ics in repeated loss minimization. In International Conference on Machine Learning, pages 1934-1943, 2018.
  190. Chaoyang He, Conghui Tan, Hanlin Tang, Shuang Qiu, and Ji Liu. Central server free federated learning over single-sided trust social networks. arXiv preprint arXiv:1910.04956, 2019. · arXiv
  191. Chaoyang He, Murali Annavaram, and Salman Avestimehr. Group knowledge transfer: Federated learning of large cnns at the edge. In Advances in Neural Information Processing Systems 34, 2020.
  192. Chaoyang He, Haishan Ye, Li Shen, and Tong Zhang. Milenas: Efficient neural architecture search via mixed- level reformulation. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2020.
  193. Lie He, An Bian, and Martin Jaggi. COLA: Decentralized linear learning. In NeurIPS 2018 -Advances in Neural Information Processing Systems 31, 2018.
  194. Úrsula Hébert-Johnson, Michael Kim, Omer Reingold, and Guy Rothblum. Multicalibration: Calibration for the (computationally-identifiable) masses. In International Conference on Machine Learning, pages 1944- 1953, 2018.
  195. Judy Hoffman, Mehryar Mohri, and Ningshan Zhang. Algorithms and theory for multiple-source adaptation. In Advances in Neural Information Processing Systems, pages 8246-8256, 2018.
  196. Samuel Horvath, Chen-Yu Ho, Ludovit Horvath, Atal Narayan Sahu, Marco Canini, and Peter Richtarik. Nat- ural compression for distributed deep learning. arXiv preprint arXiv:1905.10988, 2019. · arXiv
  197. Kevin Hsieh, Amar Phanishayee, Onur Mutlu, and Phillip B. Gibbons. The non-IID data quagmire of decen- tralized machine learning, 2019. URL https://arxiv.org/abs/1910.00189. · arXiv
  198. Tzu-Ming Harry Hsu, Hang Qi, and Matthew Brown. Measuring the effects of non-identical data distribution for federated visual classification. arXiv preprint arXiv:1909.06335, 2019. · arXiv
  199. Zhouyuan Huo, Bin Gu, and Heng Huang. Training neural networks using features replay. In Advances in Neural Information Processing Systems, pages 6659-6668, 2018.
  200. Yuval Ishai, Joe Kilian, Kobbi Nissim, and Erez Petrank. Extending oblivious transfers efficiently. In CRYPTO, volume 2729 of Lecture Notes in Computer Science, pages 145-161. Springer, 2003.

200 of 439 shown

Cited by

Sorted by citations

Authors
Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., Bonawitz, K., Charles, Z., Cormode, G., Cummings, R., D'Oliveira, R. G. L., Eichner, H., Rouayheb, S. E., Evans, D., Gardner, J., Garrett, Z., Gascón, A., Ghazi, B., Gibbons, P. B., Gruteser, M., Harchaoui, Z., He, C., He, L., Huo, Z., Hutchinson, B., Hsu, J., Jaggi, M., Javidi, T., Joshi, G., Khodak, M., Konečný, J., Korolova, A., Koushanfar, F., Koyejo, S., Lepoint, T., Liu, Y., Mittal, P., Mohri, M., Nock, R., Özgür, A., Pagh, R., Raykova, M., Qi, H., Ramage, D., Raskar, R., Song, D., Song, W., Stich, S. U., Sun, Z., Suresh, A. T., Tramèr, F., Vepakomma, P., Wang, J., Xiong, L., Xu, Z., Yang, Q., Yu, F. X., Yu, H., & Zhao, S.
Affiliations
Google (United States) (US)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2019
Volume
14
Issue
1-2
Pages
1-210
ISSN
1935-8237 (ISSN-L)
DOI
10.48550/arxiv.1912.04977
arXiv
1912.04977
OpenAlex
W2995022099
Cited by
7,857
References
0
Citations by year
2026: 1798 · 2025: 1737 · 2024: 1367 · 2023: 1156 · 2022: 786 · 2021: 700 · 2020: 299 · 2019: 13 · 2018: 1
FWCI
286.43
Type
journal article
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • Foundations and Trends® in Machine Learning (publishedVersion)
  • Warwick Research Archive Portal (University of Warwick) (acceptedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • HAL (submittedVersion, other-oa)
  • HAL (submittedVersion, cc-by)
  • Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
  • LillOA (Université de Lille (University Of Lille)) (submittedVersion, other-oa)
  • Oskar-Bordeaux (Universite de Bordeaux) (submittedVersion)
  • Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
  • arXiv (Cornell University)
Funders
Agence Nationale de la Recherche (ANR-16-CE23-0016, ANR-20-CE23-0015)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Artificial Intelligence › Stochastic Gradient Optimization Techniques
  • Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
Keywords
federated learning, decentralized data, privacy-preserving machine learning, data minimization, open problems, mobile devices, data collection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review