Machine Learning–based Cyber Attacks Targeting on Controlled Information: A Survey
Open accessThe DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
Stealing attack against controlled information, along with the increasing number of information leakage incidents, has become an emerging cyber security threat in recent years. Due to the booming development and deployment of advanced analytics solutions, novel stealing attacks utilize machine learning (ML) algorithms to achieve high success rate and cause a lot of damage. Detecting and defending against such attacks is challenging and urgent so governments, organizations, and individuals should attach great importance to the ML-based stealing attacks. This survey presents the recent advances in this new type of attack and corresponding countermeasures. The ML-based stealing attack is reviewed in perspectives of three categories of targeted controlled information, including controlled user activities, controlled ML model-related information, and controlled authentication information. Recent publications are summarized to generalize an overarching attack methodology and to derive the limitations and future directions of ML-based stealing attacks. Furthermore, countermeasures are proposed towards developing effective protections from three aspects—detection, disruption, and isolation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Calibrating Noise to Sensitivity in Private Data Analysis
Dwork, C., McSherry, F., Nissim, K., & Smith, A. (2006) · Lecture Notes in Computer Science · doi:10.1007/11681878_14 · Cited by 7,272
The Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2014) · Foundations and Trends® in Theoretical Computer Science · doi:10.1561/0400000042 · Cited by 5,557
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754 · cited 10×
“The range of inputs should also be wide enough to include samples inside and outside a model’s training set [95, 101].”
BackgroundMethodInfluentialThe Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2013) · doi:10.1561/9781601988195 · Cited by 3,856
Differential Privacy: A Survey of Results
Dwork, C. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-79228-4_1 · Cited by 3,542 · cited 2×
“[125] Model extraction attack Rounding confidences [34]; Differential privacy (DP) [28, 55, 71, 130]; Ensemble methods [120] Logistic regression; Decision tree; SVM; Three-layer NN Test error; Uniform error; Extraction Faccuracy”
MethodPractical Black-Box Attacks against Machine Learning
Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z. B., & Swami, A. (2017) · Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security · doi:10.1145/3052973.3053009 · Cited by 3,495
Privacy-preserving deep learning
Shokri, R., & Shmatikov, V. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813687 · Cited by 2,356 · cited 1×
“A distributed, federated, or decentralized deep learning algorithm can process each users’ training set by sharing the subset of parameters obfuscated with differential privacy [1, 100].”
BackgroundA survey of mobile phone sensing
Lane, N., Miluzzo, E., Lu, H., Peebles, D., Choudhury, T., & Campbell, A. (2010) · IEEE Communications Magazine · doi:10.1109/mcom.2010.5560598 · Cited by 2,292
Exploiting Unintended Feature Leakage in Collaborative Learning
Melis, L., Song, C., De Cristofaro, E., & Shmatikov, V. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00029 · Cited by 1,734
Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
Hitaj, B., Ateniese, G., & Perez-Cruz, F. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134012 · Cited by 1,377 · cited 10×
“For example, CNN was used in Reference [42] to learn a representation of features based on the correlation among data for optimal classification.”
BackgroundMethodInfluentialHuang, L., Joseph, A. D., Nelson, B., Rubinstein, B. I. P., & Tygar, J. D. (2011) · Proceedings of the 4th ACM workshop on Security and artificial intelligence · doi:10.1145/2046684.2046692 · Cited by 1,194 · cited 2×
“An ML model is confidential due to the pay-per-query development in a cloud-based ML service [109] as well as the security mechanisms contained in detection applications [9, 44, 72, 105].”
BackgroundEvasion Attacks against Machine Learning at Test Time
Biggio, B., Corona, I., Maiorca, D., Nelson, B., Šrndić, N., Laskov, P., Giacinto, G., & Roli, F. (2013) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-40994-3_25 · Cited by 890 · cited 2×
“An ML model is confidential due to the pay-per-query development in a cloud-based ML service [109] as well as the security mechanisms contained in detection applications [9, 44, 72, 105].”
BackgroundLowd, D., & Meek, C. (2005) · Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining · doi:10.1145/1081870.1081950 · Cited by 755 · cited 2×
“An ML model is confidential due to the pay-per-query development in a cloud-based ML service [109] as well as the security mechanisms contained in detection applications [9, 44, 72, 105].”
BackgroundSoK: Security and Privacy in Machine Learning
Papernot, N., McDaniel, P., Sinha, A., & Wellman, M. P. (2018) · 2018 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2018.00035 · Cited by 502 · cited 1×
“In addition to this category, as shown in Table 6, the other two ways focus on attacks at training/inference phase and with black-/whitebox access [88].”
BackgroundSemi-supervised Knowledge Transfer for Deep Learning from Private Training Data
Papernot, N., Abadi, M., Erlingsson, Ú., Goodfellow, I., & Talwar, K. (2016) · arXiv · doi:10.48550/arxiv.1610.05755 · Cited by 492
Ganju, K., Wang, Q., Yang, W., Gunter, C. A., & Borisov, N. (2018) · Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3243734.3243834 · Cited by 475 · cited 6×
“Attack Type Attack Targets Attack Surfaces Attacker’s Capabilities Model Info Training Set Info Training Phase Inference Phase Black-box Access White-box Access Model extraction attack [109] YES no no YES YES no Model extraction attack [87] YES no no YES YES no Hyperparameter stealing attack [114] YES no no YES YES no…”
BackgroundMethodInfluentialMachine Learning with Membership Privacy using Adversarial Regularization
Nasr, M., Shokri, R., & Houmansadr, A. (2018) · Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3243734.3243855 · Cited by 458 · cited 2×
“A method was proposed in Reference [82] to prevent training set leakage against the membership inference attack [95],”
BackgroundMethodWhen Machine Learning Meets Privacy: A Survey and Outlook
Liu, B., Ding, M., Shaham, S., Rahayu, W., Farokhi, F., & Lin, Z. (2022) · ACM Computing Surveys · doi:10.1145/3436755 · Cited by 455
Robust Network Traffic Classification
Zhang, J., Chen, X., Xiang, Y., Zhou, W., & Wu, J. (2015) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2014.2320577 · Cited by 391
Self-citationTowards Reverse-Engineering Black-Box Neural Networks
Oh, S. J., Schiele, B., & Fritz, M. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-28954-6_7 · Cited by 291
Internet Traffic Classification by Aggregating Correlated Naive Bayes Predictions
Zhang, J., Chen, C., Xiang, Y., Zhou, W., & Xiang, Y. (2013) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2012.2223675 · Cited by 175
Self-citationLane, N. D., Xu, Y., Lu, H., Hu, S., Choudhury, T., Campbell, A. T., & Zhao, F. (2011) · Proceedings of the 13th international conference on Ubiquitous computing · doi:10.1145/2030112.2030160 · Cited by 171
Cited by
Qiu, S., Zhao, H., Jiang, N., Wang, Z., Liu, L., An, Y., Zhao, H., Miao, X., Liu, R., & Fortino, G. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.11.006 · Cited by 671
Deep Learning Based Attack Detection for Cyber-Physical System Cybersecurity: A Survey
Zhang, J., Pan, L., Han, Q.-L., Chen, C., Wen, S., & Xiang, Y. (2022) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2021.1004261 · Cited by 432
Self-citationIntegrating explanation and prediction in computational social science
Hofman, J. M., Watts, D. J., Athey, S., Garip, F., Griffiths, T. L., Kleinberg, J., Margetts, H., Mullainathan, S., Salganik, M. J., Vazire, S., et al. (2021) · Nature · doi:10.1038/s41586-021-03659-0 · Cited by 371
Detecting Vulnerability on IoT Device Firmware: A Survey
Feng, X., Zhu, X., Han, Q.-L., Zhou, W., Wen, S., & Xiang, Y. (2023) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2022.105860 · Cited by 185
Self-citationPrivacy Protection for Blockchain-Based Healthcare IoT Systems: A Survey
Qi, M., Wang, Z., Han, Q.-L., Zhang, J., Chen, S., & Xiang, Y. (2024) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2022.106058 · Cited by 44
Self-citation
Metadata
- Authors
- Miao, Y., Chen, C., Pan, L., Han, Q.-L., Zhang, J., & Xiang, Y.
- Venue
- ACM Computing Surveys
- Published
- 2021-07-18
- Volume
- 54
- Issue
- 7
- Pages
- 1-36
- ISSN
- 0360-0300 (ISSN-L)
- 1557-7341
- OpenAlex
- W3184914355
- Cited by
- 126
- References
- 22
- FWCI
- 8.82
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.