CoChat
Sign Up
Cited by 285Privacy-Preserving Technologies in DataAdversarial Robustness in Machine LearningSecurity and Verification in Computing

A Survey of Privacy Attacks in Machine Learning

Open access
Rigaki, Maria, Garcia, Sebastian·2024·ACM Computing Surveys
DOI 10.1145/3624010
Open this paper in CoChat — free accountOpen PDF

Abstract

As machine learning becomes more widely used, the need to study its implications in security and privacy becomes more urgent. Although the body of work in privacy has been steadily growing over the past few years, research on the privacy aspects of machine learning has received less focus than the security aspects. Our contribution in this research is an analysis of more than 45 papers related to privacy attacks against machine learning that have been published during the past seven years. We propose an attack taxonomy, together with a threat model that allows the categorization of different attacks based on the adversarial knowledge, and the assets under attack. An initial exploration of the causes of privacy leaks is presented, as well as a detailed analysis of the different attacks. Finally, we present an overview of the most commonly proposed defenses and a discussion of the open problems and future directions identified during our analysis.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3624010

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Various attacks can compromise both the training and inference stages of the ML pipeline, as well as data collection and storage processes in ML-based IoV applications.

    “In ML-based IoV applications, various attacks can compromise both the training and inference stages of the ML pipeline, as well as the data collection and storage processes [116, 161].”

    From Privacy-preserving Machine Learning in Internet of Vehicle Applications: Fundamentals, Recent Advances, and Future Direction (2025) · cites this paper as [116]

    Sections 2.3.3 and 3 describe privacy attacks during both inference and training. The survey does not establish these attacks specifically for ML-based IoV applications, and Section 3 explicitly excludes attacks against the infrastructure hosting data or models; it therefore does not support the claim about attacks on data collection and storage processes. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Vehicular cloud platforms struggle to balance utility with privacy; models trained on cloud-stored data may inadvertently leak attributes of legitimate users.

    “Vehicular cloud platforms, crucial for applications like collaborative event sharing and anomaly detection, struggle to balance utility with privacy -models trained on cloud-stored data may inadvertently leak attributes of legitimate users [116].”

    From Privacy-preserving Machine Learning in Internet of Vehicle Applications: Fundamentals, Recent Advances, and Future Direction (2025) · cites this paper as [116]

    The Introduction (page not available in our copy) says models trained on sensitive data can reveal information about that data, supporting the general risk of user-information leakage. The passages found do not address vehicular cloud platforms, collaborative event sharing or anomaly detection, or a utility–privacy trade-off in that setting. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Studies [116], [161], [97], [16] reviewed privacy issues in deep learning, including attacks, privacy-preserving techniques and their challenges.

    “For instance, studies [116], [161], [97], [16] reviewed privacy issues in deep learning (DL), including attacks, privacy-preserving techniques and their challenges.”

    From Privacy-preserving Machine Learning in Internet of Vehicle Applications: Fundamentals, Recent Advances, and Future Direction (2025) · cites this paper as [116]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 113

  1. Martin Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. 2016. Deep learning with differential privacy. In Proceedings of the ACM SIGSAC Conference on Computer and Communica- tions Security (CCS'16). ACM, New York, NY, 308-318. https://doi.org/10.1145/2976749.2978318 · DOI · Paper page
  2. Mohammad Al-Rubaie and Morris J. Chang. 2019. Privacy-preserving machine learning: Threats and solutions. IEEE Secur. Privacy 17, 2 (2019), 49-58. https://doi.org/10.1109/MSEC.2018.2888775 · DOI · Paper page
  3. Giuseppe Ateniese, Luigi V. Mancini, Angelo Spognardi, Antonio Villani, Domenico Vitali, and Giovanni Felici. 2015. Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers. Int. J. Secur. Netw. 10, 3 (Sept. 2015), 137-150. https://doi.org/10.1504/IJSN.2015.071829 · DOI
  4. Michael Backes, Mathias Humbert, Jun Pang, and Yang Zhang. 2017. Walk2friends: Inferring social links from mobil- ity profiles. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS'17). ACM, New York, NY, 1943-1957. https://doi.org/10.1145/3133956.3133972 · DOI
  5. Antonio Barbalau, Adrian Cosma, Radu Tudor Ionescu, and Marius Popescu. 2020. Black-box ripper: copying black- box models using generative evolutionary algorithms. In Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS'20), Vol. 33. 20120-20129.
  6. David Berthelot, Nicholas Carlini, Ian Goodfellow, Nicolas Papernot, Avital Oliver, and Colin A. Raffel. 2019. Mixmatch: A holistic approach to semi-supervised learning. In Advances in Neural Information Processing Systems. NeurIPS, 5050-5060.
  7. Battista Biggio and Fabio Roli. 2018. Wild patterns: Ten years after the rise of adversarial machine learning. Pattern Recogn. 84 (2018), 317-331.
  8. Christopher M. Bishop. 2006. Pattern Recognition and Machine Learning (Information Science and Statistics). Springer- Verlag, Berlin.
  9. Leo Breiman. 2001. Random forests. Mach. Learn. 45, 1 (2001), 5-32.
  10. Nicholas Carlini, Chang Liu, Úlfar Erlingsson, Jernej Kos, and Dawn Song. 2019. The secret sharer: Evaluating and testing unintended memorization in neural networks. In Proceedings of the 28th USENIX Security Symposium (USENIX Security'19). USENIX Association, 267-284.
  11. Nicholas Carlini, Florian Tramèr, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Úlfar Erlingsson, Alina Oprea, and Colin Raffel. 2021. Extracting training data from large language models. In Proceedings of the 30th USENIX Security Symposium (USENIX Security'21). USENIX Association, 2633-2650.
  12. Augustin Cauchy et al. 1847. Méthode générale pour la résolution des systemes d'équations simultanées. Comp. Rend. Sci. Paris 25, 1847 (1847), 536-538.
  13. Varun Chandrasekaran, Kamalika Chaudhuri, Irene Giacomelli, Somesh Jha, and Songbai Yan. 2020. Exploring con- nections between active learning and model extraction. In Proceedings of the 29th USENIX Security Symposium (USENIX Security'20). USENIX Association.
  14. Dingfan Chen, Ning Yu, Yang Zhang, and Mario Fritz. 2020. GAN-leaks: A taxonomy of membership inference attacks against generative models. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (Virtual Event) (CCS'20). ACM, New York, NY, 343-362. https://doi.org/10.1145/3372297.3417238 · DOI · Paper page
  15. Christopher A. Choquette-Choo, Florian Tramer, Nicholas Carlini, and Nicolas Papernot. 2021. Label-only mem- bership inference attacks. In Proceedings of the 38th International Conference on Machine Learning (Proceedings of Machine Learning Research, Vol. 139), Marina Meila and Tong Zhang (Eds.). PMLR, 1964-1974.
  16. International Warfarin Pharmacogenetics Consortium. 2009. Estimation of the warfarin dose with clinical and phar- macogenetic data. New Engl. J. Med. 360, 8 (2009), 753-764.
  17. Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. 2016. The cityscapes dataset for semantic urban scene understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3213-3223.
  18. Jackson Rodrigues Correia-Silva, Rodrigo F. Berriel, Claudine Badue, Alferto F. de Souza, and Thiago Oliveira-Santos. 2018. Copycat CNN: Stealing knowledge by persuading confession with random non-labeled data. In Proceedings of the International Joint Conference on Neural Networks (IJCNN). IEEE, 1-8. https://doi.org/10.1109/IJCNN.2018.8489592 · DOI
  19. Jeffrey Dean, Greg S. Corrado, Rajat Monga, Kai Chen, Matthieu Devin, Quoc V. Le, Mark Z. Mao, Marc'Aurelio Ranzato, Andrew Senior, Paul Tucker, Ke Yang, and Andrew Y. Ng. 2012. Large scale distributed deep networks. In Proceedings of the 25th International Conference on Neural Information Processing Systems-Volume 1 (NIPS'12).
  20. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 248-255.
  21. Cynthia Dwork and Aaron Roth. 2013. The algorithmic foundations of differential privacy. Found. Trends Theor. Comput. Sci. 9, 3-4 (2013), 211-487. https://doi.org/10.1561/0400000042 · DOI · Paper page
  22. Ronald A. Fisher. 1936. The use of multiple measurements in taxonomic problems. Annals of Eugenics 7, 2 (1936), 179-188.
  23. Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. 2015. Model inversion attacks that exploit confidence informa- tion and basic countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (CCS'15). ACM, New York, NY, 1322-1333. https://doi.org/10.1145/2810103.2813677 · DOI · Paper page
  24. Matthew Fredrikson, Eric Lantz, Somesh Jha, Simon Lin, David Page, and Thomas Ristenpart. 2014. Privacy in phar- macogenetics: An end-to-end case study of personalized warfarin dosing. In Proceedings of the 23rd USENIX Security Symposium (USENIX Security'14). USENIX Association, 17-32.
  25. Athinodoros S. Georghiades, Peter N. Belhumeur, and David J. Kriegman. 2001. From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Trans. Pattern Anal. Mach. Intell. 23, 6 (2001), 643-660.
  26. Neil Zhenqiang Gong and Bin Liu. 2016. You are who you know and how you behave: Attribute inference attacks via users' social friends and behaviors. In Proceedings of the 25th USENIX Security Symposium (USENIX Security'16). USENIX Association, 979-995.
  27. Xueluan Gong, Yanjiao Chen, Wenbin Yang, Guanghao Mei, and Qian Wang. 2021. InverseNet: Augmenting model extraction attacks with training data inversion. In Proceedings of the 30th International Joint Conference on Artificial Intelligence (IJCAI'21), Zhi-Hua Zhou (Ed.). International Joint Conferences on Artificial Intelligence Organization, 2439-2447. https://doi.org/10.24963/ijcai.2021/336 Main Track. · DOI
  28. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press, Cambridge, MA.
  29. Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Proceedings of the 27th International Conference on Neural Information Processing Systems-Volume 2 (NIPS'14). MIT Press, Cambridge, MA, 2672-2680.
  30. F. Maxwell Harper and Joseph A. Konstan. 2015. The movielens datasets: History and context. ACM Trans. Interact. Intell. Syst. 5, 4, Article 19 (Dec. 2015), 19 pages. https://doi.org/10.1145/2827872 · DOI
  31. Johann Hauswald, Thomas Manville, Qi Zheng, Ronald Dreslinski, Chaitali Chakrabarti, and Trevor Mudge. 2014. A hybrid approach to offloading mobile image classification. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP'14). IEEE, 8375-8379.
  32. Yang He, Shadi Rahimian, Bernt Schiele, and Mario Fritz. 2020. Segmentations-leak: Membership inference at- tacks and defenses in semantic image segmentation. In Proceedings of the European Conference on Computer Vision (ECCV'20). Springer International Publishing, Cham, 519-535.
  33. Seira Hidano, Takao Murakami, Shuichi Katsumata, Shinsaku Kiyomoto, and Goichiro Hanaoka. 2017. Model inver- sion attacks for prediction systems: Without knowledge of non-sensitive attributes. In Proceedings of the 15th Annual Conference on Privacy, Security and Trust (PST'17). IEEE, 115-124.
  34. Benjamin Hilprecht, Martin Härterich, and Daniel Bernau. 2019. Monte Carlo and reconstruction membership inference attacks against generative models. In Proceedings on Privacy Enhancing Technologies. 232-249. https: //doi.org/10.2478/popets-2019-0067 · DOI
  35. Geoffrey Hinton, Nitsh Srivastava, and Kevin Swersky. 2012. Neural networks for machine learning. Coursera Video Lect. 264, 1 (2012).
  36. Sorami Hisamoto, Matt Post, and Kevin Duh. 2020. Membership inference attacks on sequence-to-sequence models: Is my data in your machine translation system? Trans. Assoc. Comput. Ling. 8 (2020), 49-63. https://doi.org/10.1162/ tacl_a_00299 · DOI
  37. Gary B. Huang, Marwan Mattar, Tamara Berg, and Eric Learned-Miller. 2008. Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. In Proceedings of the Workshop on Faces in "Real-life" Images: Detection, Alignment, and Recognition. Erik Learned-Miller and Andras Ferencz and Frédéric Jurie, HAL. Retrieved from https://hal.inria.fr/inria-00321923
  38. International Conference on Spoken Language Translation 2015. IWSLT Evaluation 2015. Retrieved April 17, 2020 from https://sites.google.com/site/iwsltevaluation2015
  39. Matthew Jagielski, Nicholas Carlini, David Berthelot, Alex Kurakin, and Nicolas Papernot. 2020. High accuracy and high fidelity extraction of neural networks. In Proceedings of the 29th USENIX Security Symposium (USENIX Secu- rity'20). USENIX Association.
  40. Bargav Jayaraman and David Evans. 2019. Evaluating differentially private machine learning in practice. In Proceed- ings of the 28th USENIX Security Symposium (USENIX Security'19). USENIX Association, 1895-1912.
  41. Jinyuan Jia and Neil Zhenqiang Gong. 2018. AttriGuard: A practical defense against attribute inference attacks via adversarial machine learning. In Proceedings of the 27th USENIX Security Symposium (USENIX Security'18). USENIX Association, 513-529.
  42. Jinyuan Jia, Ahmed Salem, Michael Backes, Yang Zhang, and Neil Zhenqiang Gong. 2019. MemGuard: Defending against black-box membership inference attacks via adversarial examples. In Proceedings of the ACM SIGSAC Con- ference on Computer and Communications Security (CCS'19). ACM, New York, NY, 259-274. https://doi.org/10.1145/ 3319535.3363201 · DOI
  43. Alistair E. W. Johnson, Tom J. Pollard, Lu Shen, H. Lehman Li-wei, Mengling Feng, Mohammad Ghassemi, Benjamin Moody, Peter Szolovits, Leo Anthony Celi, and Roger G. Mark. 2016. MIMIC-III, a freely accessible critical care database. Sci. Data 3 (2016), 160035.
  44. Yiping Kang, Johann Hauswald, Cao Gao, Austin Rovinski, Trevor Mudge, Jason Mars, and Lingjia Tang. 2017. Neu- rosurgeon: Collaborative intelligence between the cloud and mobile edge. In Proceedings of the 22nd International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS'17). ACM, New York, NY, 615-629. https://doi.org/10.1145/3037697.3037698 · DOI · Paper page
  45. Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. Light- GBM: A highly efficient gradient boosting decision tree. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NIPS'17).
  46. Diederik P. Kingma and Max Welling. 2014. Auto-encoding variational bayes. In Proceedings of the 2nd International Conference on Learning Representations (ICLR'14), Vol. 1. ICLR.
  47. Alex Krizhevsky, Geoffrey Hinton et al. 2009. Learning multiple layers of features from tiny images. MSc thesis. http://www.cs.utoronto.ca/~kriz/learning-features-2009-TR.pdf
  48. Klas Leino and Matt Fredrikson. 2020. Stolen memories: Leveraging model memorization for calibrated white-box membership inference. In Proceedings of the 29th USENIX Security Symposium (USENIX Security'20). 1605-1622.
  49. Zheng Li and Yang Zhang. 2021. Membership leakage in label-only exposures. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS'21). ACM, New York, NY, 880-895. https://doi.org/10. 1145/3460120.3484575 · DOI
  50. Dong C. Liu and Jorge Nocedal. 1989. On the limited memory BFGS method for large scale optimization. Math. Program. 45, 1-3 (1989), 503-528.
  51. Jian Liu, Mika Juuti, Yao Lu, and N. Asokan. 2017. Oblivious neural network predictions via MiniONN transforma- tions. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS'17). ACM, New York, NY, 619-631. https://doi.org/10.1145/3133956.3134056 · DOI
  52. Shuying Liu and Weihong Deng. 2015. Very deep convolutional neural network based image classification using small training sample size. In Proceedings of the 3rd IAPR Asian Conference on Pattern Recognition (ACPR'15). IEEE, 730-734.
  53. Ximeng Liu, Lehui Xie, Yaopeng Wang, Jian Zou, Jinbo Xiong, Zuobin Ying, and Athanasios V. Vasilakos. 2021. Privacy and security issues in deep learning: A survey. IEEE Access 9 (2021), 4566-4593. https://doi.org/10.1109/ ACCESS.2020.3045078 · DOI
  54. Yugeng Liu, Rui Wen, Xinlei He, Ahmed Salem, Zhikun Zhang, Michael Backes, Emiliano De Cristofaro, Mario Fritz, and Yang Zhang. 2021. ML-doctor: Holistic risk assessment of inference attacks against machine learning models. Retrieved from https://arXiv:2102.02551
  55. Yunhui Long, Vincent Bindschaedler, Lei Wang, Diyue Bu, Xiaofeng Wang, Haixu Tang, Carl A. Gunter, and Kai Chen. 2018. Understanding membership inferences on well-generalized learning models. Retrieved from https://arXiv:1802. 04889
  56. Minh-Thang Luong and Christopher D. Manning. 2015. Stanford neural machine translation systems for spoken language domain. In Proceedings of the International Workshop on Spoken Language Translation.
  57. Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards deep learning models resistant to adversarial attacks. In Proceedings of the 6th International Conference on Learning Repre- sentations (ICLR'18). ICLR.
  58. S. Mahloujifar, E. Ghosh, and M. Chase. 2022. Property inference from poisoning. In Proceedings of the IEEE Sympo- sium on Security and Privacy (SP'22). IEEE Computer Society, 1569-1569. https://doi.org/10.1109/SP46214.2022.00140 · DOI
  59. Davide Maiorca, Battista Biggio, and Giorgio Giacinto. 2019. Towards adversarial malware detection: Lessons learned from PDF-based attacks. ACM Comput. Surv. 52, 4, Article 78 (Aug. 2019), 36 pages. https://doi.org/10.1145/3332184 · DOI
  60. Luca Melis, Congzheng Song, Emiliano De Cristofaro, and Vitaly Shmatikov. 2019. Exploiting unintended feature leakage in collaborative learning. In Proceedings of the IEEE Symposium on Security and Privacy (SP'19). IEEE, San Francisco, CA, 691-706.
  61. Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. Pointer sentinel mixture models. Re- trieved from https://arXiv:1609.07843
  62. Smitha Milli, Ludwig Schmidt, Anca D. Dragan, and Moritz Hardt. 2019. Model reconstruction from model explana- tions. In Proceedings of the Conference on Fairness, Accountability, and Transparency (FAT*'19). ACM, New York, NY, 1-9. https://doi.org/10.1145/3287560.3287562 · DOI
  63. Christopher Morris, Nils M. Kriege, Franka Bause, Kristian Kersting, Petra Mutzel, and Marion Neumann. 2020. TUDataset: A collection of benchmark datasets for learning with graphs. In Proceedings of the ICML Workshop on Graph Representation Learning and Beyond (GRL+'20). www.graphlearning.io
  64. Kevin P. Murphy. 2012. Machine Learning: A Probabilistic Perspective. MIT Press, Cambridge, MA.
  65. Milad Nasr, Reza Shokri, and Amir Houmansadr. 2018. Machine learning with membership privacy using adversarial regularization. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (Toronto, Canada) (CCS'18). ACM, New York, NY, 634-646. https://doi.org/10.1145/3243734.3243855 · DOI · Paper page
  66. Milad Nasr, Reza Shokri, and Amir Houmansadr. 2019. Comprehensive privacy analysis of deep learning: Passive and active white-box inference attacks against centralized and federated learning. In Proceedings of the IEEE Symposium on Security and Privacy (SP'19). IEEE, 739-753.
  67. Yuval Netzer, Tao Wang, Adam Coates, Alessandro Bissacco, Bo Wu, and Andrew Y. Ng. 2011. Reading digits in natural images with unsupervised feature learning. In Proceedings of the NIPS Workshop on Deep Learning and Unsu- pervised Feature Learning. NIPS, Granada, Spain.
  68. Hong-Wei Ng and Stefan Winkler. 2014. A data-driven approach to cleaning large face datasets. In Proceedings of the IEEE International Conference on Image Processing (ICIP'14). IEEE, 343-347.
  69. Anh Nguyen, Alexey Dosovitskiy, Jason Yosinski, Thomas Brox, and Jeff Clune. 2016. Synthesizing the preferred inputs for neurons in neural networks via deep generator networks. In Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS'16).
  70. Jorge Nocedal and Stephen J Wright. 2006. Numerical Optimization. Springer, Berlin.
  71. Seong Joon Oh, Max Augustin, Mario Fritz, and Bernt Schiele. 2018. Towards reverse-engineering black-box neural networks. In Proceedings of the 6th International Conference on Learning Representations (ICLR'18). ICLR.
  72. Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. 2019. Knockoff nets: Stealing functionality of black-box models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 4954-4963.
  73. Tribhuvanesh Orekondy, Bernt Schiele, and Mario Fritz. 2020. Prediction poisoning: Towards defenses against DNN model stealing attacks. In Proceedings of the 8th International Conference on Learning Representations (ICLR'20).
  74. Soham Pal, Yash Gupta, Aditya Shukla, Aditya Kanade, Shirish Shevade, and Vinod Ganapathy. 2020. ActiveThief: Model extraction using active learning and unannotated public data. In Proceedings of the AAAI Conference on Arti- ficial Intelligence, Vol. 34. AAAI, 865-872. https://doi.org/10.1609/aaai.v34i01.5432 · DOI
  75. Bo Pang and Lillian Lee. 2005. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics (ACL'05). Association for Computational Linguistics, 115-124. https://doi.org/10.3115/1219840.1219855 · DOI
  76. Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami. 2017. Practical black-box attacks against machine learning. In Proceedings of the ACM on Asia Conference on Computer and Communications Security (ASIA CCS'17). ACM, New York, NY, 506-519. https://doi.org/10.1145/3052973.3053009 · DOI · Paper page
  77. Nicholas Papernot, Patrick McDaniel, Arunesh Sinha, and Michael P. Wellman. 2018. SoK: Security and privacy in machine learning. In Proceedings of the IEEE European Symposium on Security and Privacy (EuroSP'18). IEEE, 399-414.
  78. Boris T. Polyak. 1964. Some methods of speeding up the convergence of iteration methods. Ussr Computational Mathematics and Mathematical Physics 4, 5 (1964), 1-17.
  79. Md Atiqur Rahman, Tanzila Rahman, Robert Laganière, Noman Mohammed, and Yang Wang. 2018. Membership inference attack against differentially private deep learning model. Trans. Data Priv. 11, 1 (2018), 61-79.
  80. Alexandre Sablayrolles, Matthijs Douze, Cordelia Schmid, Yann Ollivier, and Herve Jegou. 2019. White-box vs black- box: Bayes optimal strategies for membership inference. In Proceedings of the 36th International Conference on Ma- chine Learning (Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 5558-5567.
  81. Ahmed Salem, Apratim Bhattacharya, Michael Backes, Mario Fritz, and Yang Zhang. 2020. Updates-leak: Data set in- ference and reconstruction attacks in online learning. In Proceedings of the 29th USENIX Security Symposium (USENIX Security'20). USENIX Association, 1291-1308.
  82. Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. 2019. ML-leaks: Model and data independent membership inference attacks and defenses on machine learning models. In Proceedings of the 26th Annual Network and Distributed System Security Symposium (NDSS'19).
  83. Todd E. Scheetz, Kwang-Youn A. Kim, Ruth E. Swiderski, Alisdair R. Philp, Terry A. Braun, Kevin L. Knudtson, Anne M. Dorrance, Gerald F. DiBona, Jian Huang, Thomas L. Casavant et al. 2006. Regulation of gene expression in the mammalian eye and its relevance to eye disease. Proc. Natl. Acad. Sci. U.S.A. 103, 39 (2006), 14429-14434.
  84. Ozan Sener and Silvio Savarese. 2018. Active learning for convolutional neural networks: A core-set approach. In Proceedings of the 6th International Conference on Learning Representations (ICLR'18).
  85. Shai Shalev-Shwartz and Shai Ben-David. 2014. Understanding Machine Learning: From Theory to Algorithms. Cam- bridge University Press.
  86. Mahmood Sharif, Sruti Bhagavatula, Lujo Bauer, and Michael K. Reiter. 2016. Accessorize to a Crime: Real and stealthy attacks on state-of-the-art face recognition. In Proceedings of the ACM SIGSAC Conference on Computer and Commu- nications Security (CCS'16). ACM, New York, NY, 1528-1540. https://doi.org/10.1145/2976749.2978392 · DOI · Paper page
  87. Reza Shokri and Vitaly Shmatikov. 2015. Privacy-preserving deep learning. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (CCS'15). ACM, New York, NY, 1310-1321. https://doi.org/10. 1145/2810103.2813687 · DOI · Paper page
  88. Congzheng Song and Ananth Raghunathan. 2020. Information leakage in embedding models. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS'20). ACM, New York, NY, 377-390. https: //doi.org/10.1145/3372297.3417270 · DOI
  89. Congzheng Song and Vitaly Shmatikov. 2019. Auditing data provenance in text-generation models. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD'19). ACM, New York, NY, 196-206. https://doi.org/10.1145/3292500.3330885 · DOI
  90. Congzheng Song and Vitaly Shmatikov. 2020. Overlearning reveals sensitive attributes. In Proceedings of the 8th International Conference on Learning Representations (ICLR'20).
  91. Liwei Song, Reza Shokri, and Prateek Mittal. 2019. Privacy risks of securing machine learning models against adver- sarial examples. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS'19). ACM, New York, NY, 241-257. https://doi.org/10.1145/3319535.3354211 · DOI · Paper page
  92. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1 (2014), 1929-1958.
  93. Johannes Stallkamp, Marc Schlipsing, Jan Salmen, and Christian Igel. 2011. The German traffic sign recognition benchmark: A multi-class classification competition. In Proceedings of the International Joint Conference on Neural Networks. IEEE, 1453-1460.
  94. Richard S. Sutton and Andrew G. Barto. 2018. Reinforcement Learning: An Introduction. MIT Press, Cambridge, MA.
  95. Florian Tramèr, Fan Zhang, Ari Juels, Michael K. Reiter, and Thomas Ristenpart. 2016. Stealing machine learning models via prediction APIs. In Proceedings of the 25th USENIX Security Symposium (USENIX Security'16). USENIX Association, 601-618.
  96. Stacey Truex, Ling Liu, Mehmet Emre Gursoy, Lei Yu, and Wenqi Wei. 2021. Demystifying membership Infer- ence attacks in machine learning as a service. IEEE Transactions on Services Computing 14, 6 (2021), 2073-2089. DOI:10.1109/TSC.2019.2897554 · DOI
  97. V. Vapnik. 1991. Principles of risk minimization for learning theory. In Proceedings of the 4th International Conference on Neural Information Processing Systems (NIPS'91). Morgan Kaufmann Publishers, 831-838.
  98. Ben Verhoeven and Walter Daelemans. 2014. CLiPS stylometry investigation (CSI) corpus: A Dutch corpus for the de- tection of age, gender, personality, sentiment and deception in text. In Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC'14). European Language Resources Association (ELRA), 3081-3085. [129] VoxForge 2009. VoxForge Speech Corpus. Retrieved April 17, 2020 from http://www.voxforge.org/
  99. Binghui Wang and Neil Zhenqiang Gong. 2018. Stealing hyperparameters in machine learning. In Proceedings of the IEEE Symposium on Security and Privacy (SP'18). IEEE, 36-52.
  100. Xianmin Wang, Jing Li, Xiaohui Kuang, Yu-an Tan, and Jin Li. 2019. The security of machine learning in an adversarial setting: A survey. J. Parallel Distrib. Comput. 130 (2019), 12-23.
  101. Xiaosong Wang, Yifan Peng, Le Lu, Zhiyong Lu, Mohammadhadi Bagheri, and Ronald M. Summers. 2017. ChestX- ray8: Hospital-scale chest X-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'17). IEEE.
  102. Zhibo Wang, Mengkai Song, Zhifei Zhang, Yang Song, Qian Wang, and Hairong Qi. 2019. Beyond inferring class rep- resentatives: User-level privacy leakage from federated learning. In Proceedings of the IEEE Conference on Computer Communications (INFOCOM'19). IEEE, 2512-2520.
  103. Xi Wu, Matthew Fredrikson, Somesh Jha, and Jeffrey F. Naughton. 2016. A methodology for formalizing model- inversion attacks. In Proceedings of the 29th Computer Security Foundations Symposium (CSF'16). IEEE, 355-370.
  104. Han Xiao, Kashif Rasul, and Roland Vollgraf. 2017. Fashion-mnist: A novel image dataset for benchmarking machine learning algorithms. Retrieved from https://arXiv:1708.07747
  105. Dingqi Yang, Daqing Zhang, Longbiao Chen, and Bingqing Qu. 2015. Nationtelescope: Monitoring and visualizing large-scale collective behavior in LBSNs. J. Netw. Comput. Appl. 55 (2015), 170-180.
  106. Ziqi Yang, Jiyi Zhang, Ee-Chien Chang, and Zhenkai Liang. 2019. Neural network inversion in adversarial setting via background knowledge alignment. In Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS'19). ACM, New York, NY, 225-240. https://doi.org/10.1145/3319535.3354261 · DOI
  107. Jason Yosinski, Jeff Clune, Anh Nguyen, Thomas Fuchs, and Hod Lipson. 2015. Understanding neural networks through deep visualization. Retrieved from https://arXiv:1506.06579
  108. Honggang Yu, Kaichen Yang, Teng Zhang, Yun-Yun Tsai, Tsung-Yi Ho, and Yier Jin. 2020. CloudLeak: Large-scale deep learning models stealing through adversarial examples. In Proceedings of the Network and Distributed Systems Security Symposium (NDSS'20). The Internet Society. https://doi.org/10.14722/ndss.2020.24178 · DOI
  109. Xiaohua Zhai, Avital Oliver, Alexander Kolesnikov, and Lucas Beyer. 2019. S4l: Self-supervised semi-supervised learn- ing. In Proceedings of the IEEE International Conference on Computer Vision. IEEE, 1476-1485.
  110. Ning Zhang, Manohar Paluri, Yaniv Taigman, Rob Fergus, and Lubomir Bourdev. 2015. Beyond frontal faces: Improv- ing person recognition using multiple cues. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 4804-4813.
  111. Yuheng Zhang, Ruoxi Jia, Hengzhi Pei, Wenxiao Wang, Bo Li, and Dawn Song. 2020. The secret revealer: Generative model-inversion attacks against deep neural networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 253-261.
  112. Zhikun Zhang, Min Chen, Michael Backes, Yun Shen, and Yang Zhang. 2022. Inference attacks against graph neural networks. In Proceedings of the 31st USENIX Security Symposium (USENIX Security'22). USENIX Association.
  113. Huadi Zheng, Qingqing Ye, Haibo Hu, Chengfang Fang, and Jie Shi. 2019. BDPL: A boundary differentially private layer against machine learning model extraction attacks. In Proceedings of the European Symposium on Research in Computer Security (ESORICS'19). Springer International Publishing, Cham, 66-83.

Cited by

Sorted by citations

Authors
Rigaki, M., & Garcia, S.
Affiliations
Czech Technical University in Prague, Czech Republic
ORCID
Venue
ACM Computing Surveys
Publisher
Association for Computing Machinery (ACM)
Published
2023-11-10
Volume
56
Issue
4
Pages
1-34
ISSN
  • 0360-0300 (ISSN-L)
  • 1557-7341
DOI
10.1145/3624010
OpenAlex
W3043638540
Cited by
285
References
18
Citations by year
2026: 68 · 2025: 80 · 2024: 74 · 2023: 20 · 2022: 25 · 2021: 16 · 2020: 2
FWCI
21.42
Type
journal article
Language
en
Open access
Licence
Locations
  • ACM Computing Surveys (publishedVersion, cc-by-nc)
  • arXiv (Cornell University) (publishedVersion)
  • arXiv (Cornell University)
Funders
  • Avast Software and the OP RDE
  • Research Center for Informatics (CZ.02.1.01/0.0./0.0./16_019/0000765)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
Keywords
privacy attacks, machine learning privacy, attack taxonomy, threat model, privacy leakage, privacy defenses, privacy-preserving machine learning

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review