CoChat
Sign Up
Cited by 1,377Generative Adversarial Networks and Image SynthesisAdvanced Neural Network ApplicationsNeural Networks and Applications

Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning

Open access
Hitaj, Briland, Ateniese, Giuseppe, Perez-Cruz, Fernando·2017·Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security
DOI 10.1145/3133956.3134012
Open this paper in CoChat — free accountOpen PDF

Abstract

Deep Learning has recently become hugely popular in machine learning for its ability to solve end-to-end learning systems, in which the features and the classifiers are learned simultaneously, providing significant improvements in classification accuracy in the presence of highly-structured and large databases.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3133956.3134012

Cites

34 of 34 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • By definition, federated learning systems avoid sharing healthcare data among participating institutions, but the shared information may still indirectly expose private data used for local training, e.g., by model inversion, gradients or adversarial attacks; developing counter-measures such as limiting update granularity, adding noise and ensuring adequate differential privacy may be needed and is

    “By definition, FL systems avoid sharing healthcare data among participating institutions.”

    From The future of digital health with federated learning (2020) · cites this paper as [63]

    Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • It is possible to reconstruct parts of the training data from the algorithm weights themselves on a decentralized node.

    “It is therefore possible to reconstruct parts of the training data from the algorithm weights themselves on a decentralized node [52][53][54] .”

    From Secure, privacy-preserving and federated machine learning in medical imaging (2020) · cites this paper as [54]

  • Attackers can recover data from gradients uploaded by customers.

    “Attackers can recover data from gradients uploaded by customers [4].”

    From Privacy-Preserving Blockchain-Based Federated Learning for IoT Devices (2021) · cites this paper as [4]

  • Their experiments confirmed that GAN-based approaches might not work well when the selected privacy parameter was smaller than 10, which is the upper bound of privacy parameter in our experiment.

    “But their experiments confirmed that GAN-based approaches might not work well when the selected privacy parameter was smaller than 10 , which is the upper bound of privacy parameter in our experiment, and we can achieve the accuracy of 97% .”

    From Privacy-Preserving Blockchain-Based Federated Learning for IoT Devices (2021) · cites this paper as [4]

  • There are studies discussing potential risks of information leakage in FL in which attackers may infer customers' private data from gradients.

    “There are studies discussing potential risks of information leakage in FL [4,66] in which attackers may infer customers' private data from gradients.”

    From Privacy-Preserving Blockchain-Based Federated Learning for IoT Devices (2021) · cites this paper as [4]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 48

  1. [7]7. Jeremiah Blocki, Anupam Datta, and Joseph Bonneau. 2016. Differentially Private Password Frequency Lists. In NDSS'16.
  2. [9]9. Diane Bouchacourt, Pawan K Mudigonda, and Sebastian Nowozin. 2016. DISCO Nets: DISsimilarity COefficients Networks. In Advances in Neural Information Processing Systems. 352-360.
  3. [10]10. Leo Breiman. 2001. Random Forests. Machine Learning 45, 1 (2001), 5-32.
  4. [11]11. Mark Bun and Thomas Steinke. 2016. Concentrated differential privacy: Simpli- fications, extensions, and lower bounds. In Theory of Cryptography Conference. Springer, 635-658.
  5. [12]12. Jan Camenisch, Mark Manulis, Gene Tsudik, and Rebecca N. Wright. 2012. Privacy-Oriented Cryptography (Dagstuhl Seminar 12381). Dagstuhl Reports 2 (2012), 165-183. http://drops.dagstuhl.de/opus/volltexte/2013/3755/pdf/dagrep_ v002_i009_p165_s12381.pdf
  6. [13]13. Kamalika Chaudhuri, Claire Monteleoni, and Anand D Sarwate. 2011. Differen- tially private empirical risk minimization. Journal of machine learning research: JMLR 12 (2011), 1069.
  7. [15]15. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research 12, Aug (2011), 2493-2537.
  8. [16]16. Angel Alfonso Cruz-Roa, John Edison Arevalo Ovalle, Anant Madabhushi, and Fabio Augusto González Osorio. 2013. A deep learning architecture for image representation, visual interpretability and automated basal-cell carcinoma cancer detection. In International Conference on Medical Image Computing and Computer- Assisted Intervention. Springer Berlin Heidelberg, 403-410.
  9. [17]17. Jeffrey Dean, Greg Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Mark Mao, Andrew Senior, Paul Tucker, Ke Yang, Quoc V Le, et al. 2012. Large scale distributed deep networks. In Advances in neural information processing systems. 1223-1231.
  10. [19]19. Ilias Diakonikolas, Moritz Hardt, and Ludwig Schmidt. 2015. Differentially private learning of structured discrete distributions. In Advances in Neural Information Processing Systems. 2566-2574.
  11. [21]21. Cynthia Dwork. 2006. Differential privacy. In Automata, Languages and Program- ming, 33rd International Colloquium, ICALP 2006, Venice, Italy, July 10-14, 2006, Proceedings, Part II. Springer Berlin Heidelberg, 1-12.
  12. [22]22. Cynthia Dwork and Moni Naor. 2008. On the difficulties of disclosure prevention in statistical databases or the case for differential privacy. Journal of Privacy and Confidentiality 2, 1 (2008), 8.
  13. [23]23. Cynthia Dwork and Aaron Roth. 2014. The algorithmic foundations of differential privacy. Foundations and Trends in Theoretical Computer Science 9, 3-4 (2014), 211-407.
  14. [25]25. Fabienne Eigner, Aniket Kate, Matteo Maffei, Francesca Pampaloni, and Ivan Pryvalov. 2014. Differentially private data aggregation with optimal utility. In Proceedings of the 30th Annual Computer Security Applications Conference. ACM, 316-325.
  15. [26]26. Rasool Fakoor, Faisal Ladhak, Azade Nazi, and Manfred Huber. 2013. Using deep learning to enhance cancer diagnosis and classification. In The 30th International Conference on Machine Learning (ICML 2013),WHEALTH workshop.
  16. [27]27. Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. 2015. Model inversion attacks that exploit confidence information and basic countermeasures. In Pro- ceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security. ACM, 1322-1333.
  17. [28]28. Matthew Fredrikson, Eric Lantz, Somesh Jha, Simon Lin, David Page, and Thomas Ristenpart. 2014. Privacy in pharmacogenetics: An end-to-end case study of personalized warfarin dosing. In 23rd USENIX Security Symposium (USENIX Security 14). 17-32.
  18. [29]29. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep learning. MIT Press.
  19. [30]30. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672-2680.
  20. [34]34. Alex Graves, Abdel-rahman Mohamed, and Geoffrey Hinton. 2013. Speech recognition with deep recurrent neural networks. In 2013 IEEE international conference on acoustics, speech and signal processing. IEEE, 6645-6649.
  21. [37]37. Andreas Haeberlen, Benjamin C. Pierce, and Arjun Narayan. 2011. Differential Privacy Under Fire. In Proceedings of the 20th USENIX Conference on Security (SEC'11). USENIX Association, Berkeley, CA, USA, 33-33. http://dl.acm.org/ citation.cfm?id=2028067.2028100
  22. [39]39. Shiva Prasad Kasiviswanathan, Homin K Lee, Kobbi Nissim, Sofya Raskhodnikova, and Adam Smith. 2011. What can we learn privately? SIAM J. Comput. 40, 3 (2011), 793-826.
  23. [40]40. Daniel Kifer and Ashwin Machanavajjhala. 2011. No free lunch in data privacy. In Proceedings of the 2011 ACM SIGMOD International Conference on Management of data. ACM, 193-204.
  24. [43]43. Alex M Lamb, Anirudh Goyal ALIAS PARTH GOYAL, Ying Zhang, Saizheng Zhang, Aaron C Courville, and Yoshua Bengio. 2016. Professor forcing: A new algorithm for training recurrent networks. In Advances In Neural Information Processing Systems. 4601-4609.
  25. [44]44. Pavel Laskov et al. 2014. Practical evasion of a learning-based classifier: A case study. In Security and Privacy (SP), 2014 IEEE Symposium on. IEEE, 197-211.
  26. [45]45. Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. Nature 521, 7553 (2015), 436-444.
  27. [48]48. Changchang Liu, Supriyo Chakraborty, and Prateek Mittal. 2016. Dependence Makes You Vulnerable: Differential Privacy Under Dependent Tuples. In The Network and Distributed System Security Symposium 2016 (NDSS '16). 1322-1333. https://www.internetsociety.org/sites/default/files/blogs-media/ dependence-makes-you-vulnerable-differential-privacy-under-dependent-tuples. pdf
  28. [55]55. Frank McSherry and Ilya Mironov. 2009. Differentially private recommender systems: building privacy into the Netflix Prize contenders. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 627-636.
  29. [61]61. Payman Mohassel and Yupeng Zhang. 2017. SecureML: A System for Scalable Privacy-Preserving Machine Learning. In IEEE Symposium on Security and Pri- vacy.
  30. [62]62. Arjun Narayan, Ariel Feldman, Antonis Papadimitriou, and Andreas Haeberlen. 2015. Verifiable differential privacy. In Proceedings of the Tenth European Confer- ence on Computer Systems. ACM, 28.
  31. [63]63. Olga Ohrimenko, Felix Schuster, Cédric Fournet, Aastha Mehta, Sebastian Nowozin, Kapil Vaswani, and Manuel Costa. 2016. Oblivious Multi-Party Machine Learning on Trusted Processors. In USENIX Security.
  32. [66]66. Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. 2015. The Limitations of Deep Learning in Adversarial Settings. Proceedings of the 1st IEEE European Symposium on Security and Privacy (2015).
  33. [67]67. Manas Pathak, Shantanu Rane, and Bhiksha Raj. 2010. Multiparty differential privacy via aggregation of locally trained classifiers. In Advances in Neural Infor- mation Processing Systems. 1876-1884.
  34. [69]69. NhatHai Phan, Yue Wang, Xintao Wu, and Dejing Dou. 2016. Differential Pri- vacy Preservation for Deep Auto-Encoders: an Application of Human Behavior Prediction. In Proceedings of the 30th AAAI Conference on Artificial Intelligence, AAAI. 12-17.
  35. [70]70. Alec Radford, Luke Metz, and Soumith Chintala. 2016. Unsupervised Representa- tion Learning with Deep Convolutional Generative Adversarial Networks. In 4th International Conference on Learning Representations.
  36. [71]71. C. E. Rasmussen and C. K. I. Williams. 2006. Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA.
  37. [72]72. Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. 2016. Improved techniques for training gans. In Advances in Neural Information Processing Systems. 2226-2234.
  38. [73]73. Ferdinando S Samaria and Andy C Harter. 1994. Parameterisation of a stochastic model for human face identification. In Applications of Computer Vision, 1994., Proceedings of the Second IEEE Workshop on. IEEE, 138-142.
  39. [74]74. Anand D Sarwate and Kamalika Chaudhuri. 2013. Signal processing and machine learning with differential privacy: Algorithms and challenges for continuous data. IEEE signal processing magazine 30, 5 (2013), 86-94.
  40. [75]75. Jürgen Schmidhuber. 2015. Deep learning in neural networks: An overview. Neural networks 61 (2015), 85-117.
  41. [76]76. Bernhard Scholkopf and Alexander J Smola. 2001. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press.
  42. [78]78. Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. 2017. Mem- bership Inference Attacks against Machine Learning Models. In IEEE Symposium on Security and Privacy (S&P), Oakland.
  43. [79]79. Shuang Song, Kamalika Chaudhuri, and Anand D Sarwate. 2013. Stochastic gradient descent with differentially private updates. In Global Conference on Signal and Information Processing (GlobalSIP), 2013 IEEE. IEEE, 245-248.
  44. [81]81. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. In International Conference on Learning Representations. http://arxiv.org/abs/1312. 6199
  45. [83]83. Florian Tramèr, Fan Zhang, Ari Juels, Michael K Reiter, and Thomas Ristenpart. 2016. Stealing Machine Learning Models via Prediction APIs. In USENIX Security.
  46. [84]84. Vladimir Naumovich Vapnik and Vlamimir Vapnik. 1998. Statistical learning theory. Vol. 1. Wiley New York.
  47. [87]87. Weilin Xu, Yanjun Qi, and David Evans. 2016. Automatically evading classifiers. In NDSS'16.
  48. [90]90. Xiang Zhang and Yann André LeCun. 2016. Text Understanding from Scratch. arXiv preprint arXiv:1502.01710v5 (2016). · arXiv

Cited by

Sorted by citations

Related

Authors
Hitaj, B., Ateniese, G., & Perez-Cruz, F.
Affiliations
  • Stevens Institute of Technology (US)
  • Stevens Institute of Technology, Hoboken, NJ, USA
ORCID
Venue
Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security
Publisher
ACM
Published
2017-10-30
Pages
603-618
ISSN
1543-7221 (ISSN-L)
DOI
10.1145/3133956.3134012
OpenAlex
W2591882872
Semantic Scholar
44a97f4eaaefaf5338f8aed2913d5debb2459f7e
Cited by
1,377
References
16
Citations by year
2026: 80 · 2025: 192 · 2024: 214 · 2023: 229 · 2022: 176 · 2021: 204 · 2020: 147 · 2019: 93 · 2018: 39 · 2017: 3
FWCI
22.62
Type
conference paper
Language
en
Open access
Licence
www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations
  • ACM SIGSAC Conference on Computer and Communications Security (CCS) (publishedVersion)
  • IRIS Research product catalog (Sapienza University of Rome) (submittedVersion)
Topics
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Physical Sciences › Computer Science › Artificial Intelligence › Neural Networks and Applications
Keywords
generative adversarial networks, deep learning, end-to-end learning, representation learning, classifier training, structured data

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review