The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers
Abstract
Machine-learning ML enables computers to learn how to recognise patterns, make unintended decisions, or react to a dynamic environment. The effectiveness of trained machines varies because of more suitable ML algorithms or because superior training sets. Although ML algorithms are known and publicly released, training sets may not be reasonably ascertainable and, indeed, may be guarded as trade secrets. In this paper we focus our attention on ML classifiers and on the statistical information that can be unconsciously or maliciously revealed from them. We show that it is possible to infer unexpected but useful information from ML classifiers. In particular, we build a novel meta-classifier and train it to hack other classifiers, obtaining meaningful information about their training sets. Such information leakage can be exploited, for example, by a vendor to build more effective classifiers or to simply acquire trade secrets from a competitor's apparatus, potentially violating its intellectual property rights.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1967) · Cited by 23,719
(1992) · Cited by 23,431
Quinlan, J. R. (1986) · Machine Learning · doi:10.1007/bf00116251 · Cited by 21,439
The WEKA data mining software: an update
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1656274.1656278 · Cited by 17,750
A Tutorial on Support Vector Machines for Pattern Recognition
Burges, C. J. C. (1998) · Data Mining and Knowledge Discovery · doi:10.1023/a:1009715923555 · Cited by 16,429
A training algorithm for optimal margin classifiers
Boser, B. E., Guyon, I. M., & Vapnik, V. N. (1992) · Proceedings of the fifth annual workshop on Computational learning theory · doi:10.1145/130385.130401 · Cited by 11,992
Gene selection for cancer classification using support vector machines
Guyon, I., Weston, J., Barnhill, S., & Vapnik, V. (2002) · Machine Learning · doi:10.1023/a:1012487302797 · Cited by 9,954
Text categorization with Support Vector Machines: Learning with many relevant features
Joachims, T. (1998) · Lecture Notes in Computer Science · doi:10.1007/bfb0026683 · Cited by 7,999
Calibrating Noise to Sensitivity in Private Data Analysis
Dwork, C., McSherry, F., Nissim, K., & Smith, A. (2006) · Lecture Notes in Computer Science · doi:10.1007/11681878_14 · Cited by 7,272
Fast Training of Support Vector Machines Using Sequential Minimal Optimization
Platt, J. C. (1998) · Advances in Kernel Methods · doi:10.7551/mitpress/1130.003.0016 · Cited by 5,437
Dwork, C. (2006) · Lecture Notes in Computer Science · doi:10.1007/11787006_1 · Cited by 5,291
Divergence measures based on the Shannon entropy
Lin, J. (1991) · IEEE Transactions on Information Theory · doi:10.1109/18.61115 · Cited by 5,140
Artificial neural networks: a tutorial
Jain, A. K., Mao, J., & Mohiuddin, K. M. (1996) · Computer · doi:10.1109/2.485891 · Cited by 4,946
A TUTORIAL ON SUPPORT VECTOR MACHINES FOR PATTERN RECOGNITION, DATA MINING AND KNOWLEDGE DISCOVERY
Burger, C. J. C. (1998) · Cited by 3,313
Privacy-preserving data mining
Agrawal, R., & Srikant, R. (2000) · ACM SIGMOD Record · doi:10.1145/335191.335438 · Cited by 2,961
Statistical Inference for Probabilistic Functions of Finite State Markov Chains
Baum, L. E., & Petrie, T. (1966) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177699147 · Cited by 2,775
A survey of techniques for internet traffic classification using machine learning
Nguyen, T. T. T., & Armitage, G. (2008) · IEEE Communications Surveys & Tutorials · doi:10.1109/surv.2008.080406 · Cited by 1,664
Hidden Markov Models for Speech Recognition
Juang, B. H., & Rabiner, L. R. (1991) · Technometrics · doi:10.1080/00401706.1991.10484833 · Cited by 1,068
Quinlan (2003) · Cited by 1,000
State-of-the-art in privacy preserving data mining
Verykios, V. S., Bertino, E., Fovino, I. N., Provenza, L. P., Saygin, Y., & Theodoridis, Y. (2004) · ACM SIGMOD Record · doi:10.1145/974121.974131 · Cited by 816
Intrusion detection using neural networks and support vector machines
Mukkamala, S., Janoski, G., & Sung, A. (2003) · Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290) · doi:10.1109/ijcnn.2002.1007774 · Cited by 814
Practical privacy: the SuLQ framework
Blum, A., Dwork, C., McSherry, F., & Nissim, K. (2005) · Proceedings of the twenty-fourth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems · doi:10.1145/1065167.1065184 · Cited by 797
Lowd, D., & Meek, C. (2005) · Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining · doi:10.1145/1081870.1081950 · Cited by 755
Candidate Multilinear Maps from Ideal Lattices
Garg, S., Gentry, C., & Halevi, S. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-38348-9_1 · Cited by 744
Machine learning and its applications to biology
Tarca, A. L., Carey, V. J., Chen, X.-W., Romero, R., & Drăghici, S. (2007) · PLoS Computational Biology · doi:10.1371/journal.pcbi.0030116 · Cited by 656
Traffic classification on the fly
Bernaille, L., Teixeira, R., Akodkenou, I., Soule, A., & Salamatian, K. (2006) · ACM SIGCOMM Computer Communication Review · doi:10.1145/1129582.1129589 · Cited by 585
Julius --- an open source real-time large vocabulary recognition engine
Lee, A., Kawahara, T., & Shikano, K. (2001) · 7th European Conference on Speech Communication and Technology (Eurospeech 2001) · doi:10.21437/eurospeech.2001-396 · Cited by 535
Privacy-preserving logistic regression
Chaudhuri, K., & Monteleoni, C. (2008) · Cited by 525
Face recognition by support vector machines
Guo, G., Li, S. Z., & Chan, K. (2002) · Proceedings Fourth IEEE International Conference on Automatic Face and Gesture Recognition (Cat. No. PR00580) · doi:10.1109/afgr.2000.840634 · Cited by 520
Bayesian Neural Networks for Internet Traffic Classification
Auld, T., Moore, A. W., & Gull, S. F. (2007) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2006.883010 · Cited by 494
Security Evaluation of Pattern Classifiers under Attack
Biggio, B., Fumera, G., & Roli, F. (2014) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2013.57 · Cited by 411
(2013) · doi:10.4324/9780203763247 · Cited by 400
A new intrusion detection system using support vector machines and hierarchical clustering
Khan, L., Awad, M., & Thuraisingham, B. (2007) · The VLDB Journal · doi:10.1007/s00778-006-0002-5 · Cited by 398
Improve Computer-Aided Diagnosis With Machine Learning Techniques Using Undiagnosed Samples
Li, M., & Zhou, Z.-H. (2007) · IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans · doi:10.1109/tsmca.2007.904745 · Cited by 377
Machine Learning in Medical Imaging
Wernick, M., Yang, Y., Brankov, J., Yourganov, G., & Strother, S. (2010) · IEEE Signal Processing Magazine · doi:10.1109/msp.2010.936730 · Cited by 345
Good Word Attacks on Statistical Spam Filters.
Lowd, D., & Meek, C. (2005) · Conference on Email and Anti-Spam · Cited by 316
Offline/realtime traffic classification using semi-supervised learning
Erman, J., Mahanti, A., Arlitt, M., Cohen, I., & Williamson, C. (2007) · Performance Evaluation · doi:10.1016/j.peva.2007.06.014 · Cited by 287
Renovo: a hidden code extractor for packed executables
Kang, M. G., Poosankam, P., & Yin, H. (2007) · Proceedings of the 2007 ACM workshop on Recurring malcode · doi:10.1145/1314389.1314399 · Cited by 272
Support Vector Machines for TCP traffic classification
Este, A., Gringoli, F., & Salgarelli, L. (2009) · Computer Networks · doi:10.1016/j.comnet.2009.05.003 · Cited by 270
Decision Tree-Based Online Voltage Security Assessment Using PMU Measurements
Diao, R., Sun, K., Vittal, V., O'Keefe, R. J., Richardson, M. R., Bhatt, N., Stradford, D., & Sarawgi, S. K. (2009) · IEEE Transactions on Power Systems · doi:10.1109/tpwrs.2009.2016528 · Cited by 268
A review of large-vocabulary continuous-speech
Young, S. (1996) · IEEE Signal Processing Magazine · doi:10.1109/79.536824 · Cited by 216
On Attacking Statistical Spam Filters.
Wittel, G. L., & Wu, S. F. (2004) · Cited by 180
Secure Multiparty Computation for Privacy Preserving Data Mining
Lindell, Y. (2005) · Encyclopedia of Data Warehousing and Mining · doi:10.4018/978-1-59140-557-3.ch189 · Cited by 180
Query Strategies for Evading Convex-Inducing ClassifiersPreprint
Nelson, B. A., Rubinstein, B. I. P., Huang, L., Joseph, A. D., Lee, S. J., Rao, S. B., & Tygar, J. D. (2010) · arXiv (Cornell University) · doi:10.5555/2188385.2343688 · Cited by 109
Large-vocabulary speaker-independent continuous speech recognition using HMM
Lee, K.-F., & Hon, H.-W. (2003) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.1988.196527 · Cited by 95
No NAT'd User Left Behind: Fingerprinting Users behind NAT from NetFlow Records Alone
Verde, N. V., Ateniese, G., Gabrielli, E., Mancini, L. V., & Spognardi, A. (2014) · IEEE International Conference on Distributed Computing Systems (ICDCS) · doi:10.1109/icdcs.2014.30 · Cited by 45
Self-citationSubFlow: Towards practical flow-level traffic classification
Xie, G., Iliofotou, M., Keralapura, R., Faloutsos, M., & Nucci, A. (2012) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infcom.2012.6195649 · Cited by 36
Real Time Identification of SSH Encrypted Application Flows by Using Cluster Analysis Techniques
Maiolini, G., Baiocchi, A., Iacovazzi, A., & Rizzi, A. (2009) · Lecture notes in computer science · doi:10.1007/978-3-642-01399-7_15 · Cited by 33
Prediction in financial markets: The case for small disjuncts
Dhar, V. (2011) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/1961189.1961191 · Cited by 24
Morphological classification of medical images using nonlinear support vector machines
Davatzikos, C. A., Shen, D. G., Lao, Z., Xue, Z., & Karaçalı, B. (2005) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2004.1398606 · Cited by 3
Cited by
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754
Federated Learning in Mobile Edge Networks: A Comprehensive Survey
Lim, W. Y. B., Luong, N. C., Hoang, D. T., Jiao, Y., Liang, Y.-C., Yang, Q., Niyato, D., & Miao, C. (2020) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2020.2986024 · Cited by 2,565
Exploiting Unintended Feature Leakage in Collaborative Learning
Melis, L., Song, C., De Cristofaro, E., & Shmatikov, V. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00029 · Cited by 1,734
Nasr, M., Shokri, R., & Houmansadr, A. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00065 · Cited by 1,760
Deep Models Under the GAN: Information Leakage from Collaborative Deep Learning
Hitaj, B., Ateniese, G., & Perez-Cruz, F. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134012 · Cited by 1,377
Self-citation
Metadata
- Authors
- Ateniese, G., Mancini, L. V., Spognardi, A., Villani, A., Vitali, D., & Felici, G.
- Venue
- International Journal of Security and Networks
- Publisher
- Inderscience Publishers
- Published
- 2015
- Volume
- 10
- Issue
- 3
- Pages
- 137
- ISSN
- 1747-8405
- 1747-8413
- OpenAlex
- W2962835266
- Cited by
- 395
- References
- 50
- FWCI
- 12.51
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.