The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Demystifying Membership Inference Attacks in Machine Learning as a Service
Abstract
Membership inference attacks seek to infer membership of individual training instances of a model to which an adversary has black-box access through a machine learning-as-a-service API. In providing an in-depth characterization of membership privacy risks against machine learning models, this paper presents a comprehensive study towards demystifying membership inference attacks from two complimentary perspectives. First, we provide a generalized formulation of the development of a black-box membership inference attack model. Second, we characterize the importance of model choice on model vulnerability through a systematic evaluation of a variety of machine learning models and model combinations using multiple datasets. Through formal analysis and empirical evidence from extensive experimentation, we characterize under what conditions a model may be vulnerable to such black-box membership inference attacks. We show that membership inference vulnerability is data-driven and corresponding attack models are largely transferable. Though different model types display different vulnerabilities to membership inference, so do different datasets. Our empirical results additionally show that (1) using the type of target model under attack within the attack model may not increase attack effectiveness and (2) collaborative learning exposes vulnerabilities to membership inference risks when the adversary is a participant. We also discuss countermeasure and mitigation strategies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016
Efficient Estimation of Word Representations in Vector Space
Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013) · arXiv · doi:10.48550/arxiv.1301.3781 · Cited by 26,568
UCI Machine Learning Repository
Asuncion, A. (2007) · Medical Entomology and Zoology · Cited by 25,411
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,395
Hinton, G., Deng, L., Yu, D., Dahl, G., Mohamed, A.-R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., et al. (2012) · IEEE Signal Processing Magazine · doi:10.1109/msp.2012.2205597 · Cited by 10,260 · cited 1×
“nition [36], [37] to natural language processing [38].”
Towards Evaluating the Robustness of Neural Networks
Carlini, N., & Wagner, D. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.49 · Cited by 8,859
Ensemble methods in machine learning
Dietterich, T. G. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45014-9_1 · Cited by 8,408
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143
The Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2014) · Foundations and Trends® in Theoretical Computer Science · doi:10.1561/0400000042 · Cited by 5,557
Adversarial examples in the physical world
Kurakin, A., Goodfellow, I. J., & Bengio, S. (2018) · Artificial Intelligence Safety and Security · doi:10.1201/9781351251389-8 · Cited by 5,375
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754
Semi-Supervised Learning Literature Survey
Zhu, X. (2005) · Minds at UW (University of Wisconsin) · Cited by 4,506
The Limitations of Deep Learning in Adversarial Settings
Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z. B., & Swami, A. (2016) · 2016 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2016.36 · Cited by 3,993
The Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2013) · doi:10.1561/9781601988195 · Cited by 3,856
Differential Privacy: A Survey of Results
Dwork, C. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-79228-4_1 · Cited by 3,542 · cited 1×
“This divergence separates membership inference from existing areas of privacy research, such as differential privacy [11], [12], [13] or secure multiparty computation [14], [15], [16].”
A desicion-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1995) · Lecture Notes in Computer Science · doi:10.1007/3-540-59119-2_166 · Cited by 2,252
Adversarial Examples: Attacks and Defenses for Deep Learning
Yuan, X., He, P., Zhu, Q., & Li, X. (2019) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2018.2886017 · Cited by 1,733
Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
Papernot, N., McDaniel, P., & Goodfellow, I. (2016) · arXiv · doi:10.48550/arxiv.1605.07277 · Cited by 1,326
Deng, L., Hinton, G., & Kingsbury, B. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6639344 · Cited by 1,178
Practical privacy: the SuLQ framework
Blum, A., Dwork, C., McSherry, F., & Nissim, K. (2005) · Proceedings of the twenty-fourth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems · doi:10.1145/1065167.1065184 · Cited by 797
The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks
Carlini, N., Liu, C., Erlingsson, Ú., Kos, J., & Song, D. (2018) · arXiv · doi:10.48550/arxiv.1802.08232 · Cited by 629
Secure Multiparty Computation and Secret Sharing
Cramer, R., Damgård, I. B., & Nielsen, J. B. (2015) · doi:10.1017/cbo9781107337756 · Cited by 595
Medical big data: promise and challenges
Lee, C. H., & Yoon, H.-J. (2017) · Kidney Research and Clinical Practice · doi:10.23876/j.krcp.2017.36.1.3 · Cited by 594
Machine Learning with Membership Privacy using Adversarial Regularization
Nasr, M., Shokri, R., & Houmansadr, A. (2018) · Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3243734.3243855 · Cited by 458
Practical Black-Box Attacks against Deep Learning Systems using Adversarial Examples.Preprint
Papernot, N., McDaniel, P. D., Goodfellow, I., Jha, S., Celik, Z. B., & Swami, A. (2016) · arXiv (Cornell University) · Cited by 304
Membership Inference Attack against Differentially Private Deep Learning Model.
Rahman, M. A., Rahman, T., Laganière, R., & Mohammed, N. (2018) · Cited by 292
Adversarial Examples: Attacks and Defenses for Deep Learning
Yuan, X., He, P. L., Zhu, Q., & Li, X. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.07107 · Cited by 213
Privately Evaluating Decision Trees and Random Forests
Wu, D. J., Feng, T., Naehrig, M., & Lauter, K. (2016) · Proceedings on Privacy Enhancing Technologies · doi:10.1515/popets-2016-0043 · Cited by 183
Understanding Membership Inferences on Well-Generalized Learning Models
Long, Y., Bindschaedler, V., Wang, L., Bu, D., Wang, X., Tang, H., Gunter, C. A., & Chen, K. (2018) · arXiv · doi:10.48550/arxiv.1802.04889 · Cited by 166
Cock, M. D., Dowsley, R., Horst, C., Katti, R. S., Nascimento, A. C. A., Poon, W.-S., & Truex, S. (2017) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2017.2679189 · Cited by 149
Self-citationDifferential privacy under fire
Haeberlen, A., Pierce, B. C., & Narayan, A. (2011) · ScholarlyCommons (University of Pennsylvania) · Cited by 130
A Random Decision Tree Framework for Privacy-Preserving Data Mining
Vaidya, J. S., Shafiq, B., Fan, W., Mehmood, D., & Lorenzi, D. (2013) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2013.43 · Cited by 124
The Secret Sharer: Measuring Unintended Neural Network Memorization & Extracting SecretsPreprint
Carlini, N., Liu, C., Kos, J., Erlingsson, Ú., & Song, D. X. (2018) · arXiv (Cornell University) · Cited by 124
Online Active Learning Methods for Fast Label-Efficient Spam Filtering.
Sculley, D. (2007) · Cited by 96
Towards Measuring Membership PrivacyPreprint
Long, Y., Bindschaedler, V., & Gunter, C. A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.09136 · Cited by 95
Improving Committee Diagnosis with Resampling Techniques
Parmanto, B., Munro, P., & Doyle, H. R. (1995) · Cited by 92
Secure Multi-party Computation
Jajodia, S., Samarati, P., & Yung, M. (2025) · Springer eBooks · doi:10.1007/978-3-030-71522-9_300862 · Cited by 66
LOGAN: Evaluating Privacy Leakage of Generative Models Using Generative Adversarial Networks.Preprint
Hayes, J., Melis, L., Danezis, G., & Cristofaro, E. D. (2017) · arXiv (Cornell University) · Cited by 59
Are Accuracy and Robustness Correlated
Rozsa, A., Günther, M., & Boult, T. E. (2016) · International Conference on Machine Learning and Applications · doi:10.1109/icmla.2016.0045 · Cited by 54
The Unintended Consequences of Overfitting: Training Data Inference Attacks.Preprint
Yeom, S., Fredrikson, M., & Jha, S. (2017) · arXiv (Cornell University) · Cited by 22
Cited by
A survey on security and privacy of federated learning
Mothukuri, V., Parizi, R. M., Pouriyeh, S., Huang, Y., Dehghantanha, A., & Srivastava, G. (2021) · Future Generation Computer Systems · doi:10.1016/j.future.2020.10.007 · Cited by 1,480
Data Poisoning Attacks Against Federated Learning Systems
Tolpegin, V., Truex, S., Gursoy, M. E., & Liu, L. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58951-6_24 · Cited by 927
Self-citationYin, X., Zhu, Y., & Hu, J. (2022) · ACM Computing Surveys · doi:10.1145/3460427 · Cited by 638
Membership Inference Attacks on Machine Learning: A Survey
Hu, H., Salcic, Z., Sun, L., Dobbie, G., Yu, P. S., & Zhang, X. (2022) · ACM Computing Surveys · doi:10.1145/3523273 · Cited by 564
Security and Privacy Challenges of Large Language Models: A Survey
Das, B. C., Amini, M. H., & Wu, Y. (2025) · ACM Computing Surveys · doi:10.1145/3712001 · Cited by 445
Metadata
- Authors
- Truex, S., Liu, L., Gursoy, M. E., Yu, L., & Wei, W.
- Venue
- IEEE Transactions on Services Computing
- Published
- 2021-11-01
- Volume
- 14
- Issue
- 6
- Pages
- 2073-2089
- ISSN
- 1939-1374 (ISSN-L)
- 2372-0204
- OpenAlex
- W2912023992
- Cited by
- 282
- References
- 42
- FWCI
- 12.78
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.