Property Inference Attacks on Fully Connected Neural Networks using Permutation Invariant Representations
Open accessAbstract
With the growing adoption of machine learning, sharing of learned models is becoming popular. However, in addition to the prediction properties the model producer aims to share, there is also a risk that the model consumer can infer other properties of the training data the model producer did not intend to share. In this paper, we focus on the inference of global properties of the training data, such as the environment in which the data was produced, or the fraction of the data that comes from a certain class, as applied to white-box Fully Connected Neural Networks (FCNNs). Because of their complexity and inscrutability, FCNNs have a particularly high risk of leaking unexpected information about their training sets; at the same time, this complexity makes extracting this information challenging. We develop techniques that reduce this complexity by noting that FCNNs are invariant under permutation of nodes in each layer. We develop our techniques using representations that capture this invariance and simplify the information extraction task. We evaluate our techniques on several synthetic and standard benchmark datasets and show that they are very effective at inferring various data properties. We also perform two case studies to demonstrate the impact of our attack. In the first case study we show that a classifier that recognizes smiling faces also leaks information about the relative attractiveness of the individuals in its training set. In the second case study we show that a classifier that recognizes Bitcoin mining from performance counters also leaks information about whether the classifier was trained on logs from machines that were patched for the Meltdown and Spectre attacks.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Sorting the posteriors before the difference or concatenation operations better captures the level of confidence of the model.
“In order to better capture the level of confidence of the model, one may also sort the posteriors before the difference or concatenation operations [20].”
From When Machine Unlearning Jeopardizes Privacy (2021) · cites this paper as [20]
Section 6.1, “Neuron Sorting” (page unavailable), describes sorting hidden-layer neurons by their weights to create a canonical model representation. Searches for sorting posteriors, confidence levels, and difference or concatenation operations found no passage addressing the cited claim. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Semi-supervised classification with graph convolutional networks
Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853 · see citing works
FaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267 · see citing works
The perceptron: A probabilistic model for information storage and organization in the brain.
Rosenblatt, F. (1958) · Psychological Review · doi:10.1037/h0042519 · Cited by 12,280 · see citing works
A Stochastic Approximation Method
Robbins, H., & Monro, S. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729586 · Cited by 9,643 · see citing works
Deep Learning with Differential Privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978318 · Cited by 6,578 · see citing works
DeepFace: Closing the Gap to Human-Level Performance in Face Verification
Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.220 · Cited by 6,542 · see citing works
DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks
Moosavi-Dezfooli, S.-M., Fawzi, A., & Frossard, P. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.282 · Cited by 5,317 · see citing works
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754 · see citing works
(2018) · Cited by 3,510 · see citing works
5th International Conference on Learning Representations (ICLR 17)
Havrylov, S., & Titov, I. (2017) · Cited by 3,503 · see citing works
Practical Black-Box Attacks against Machine Learning
Papernot, N., McDaniel, P., Goodfellow, I., Jha, S., Celik, Z. B., & Swami, A. (2017) · Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security · doi:10.1145/3052973.3053009 · Cited by 3,495 · see citing works
Rodinia: A benchmark suite for heterogeneous computing
Che, S., Boyer, M. J., Meng, J., et al. (2009) · IEEE International Symposium on Workload Characterization · doi:10.1109/iiswc.2009.5306797 · Cited by 3,050 · see citing works
Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures
Fredrikson, M., Jha, S., & Ristenpart, T. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813677 · Cited by 2,961 · see citing works
Understanding deep learning requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2016) · arXiv · doi:10.48550/arxiv.1611.03530 · Cited by 2,838 · see citing works
Universal Adversarial Perturbations
Moosavi-Dezfooli, S.-M., Fawzi, A., Fawzi, O., & Frossard, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.17 · Cited by 2,680 · see citing works
(2011) · IEEE Software · doi:10.1109/ms.2011.67 · Cited by 2,038 · see citing works
Spectre Attacks: Exploiting Speculative Execution
Kocher, P. C., Horn, J., Fogh, A., Genkin, D., Gruss, D., Haas, W., Hamburg, M., Lipp, M., Mangard, S., Prescher, T., et al. (2019) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2019.00002 · Cited by 1,847 · see citing works
Trojaning Attack on Neural Networks
Liu, Y., Ma, S., Aafer, Y., Lee, W.-C., Zhai, J., Wang, W., & Zhang, X. (2018) · Proceedings 2018 Network and Distributed System Security Symposium · doi:10.14722/ndss.2018.23291 · Cited by 1,270 · see citing works
Vanschoren, J., van Rijn, J. N., Bischl, B., & Torgo, L. (2014) · ACM SIGKDD Explorations Newsletter · doi:10.1145/2641190.2641198 · Cited by 1,233 · see citing works
Meltdown: reading kernel memory from user space
Lipp, M., Schwarz, M., Gruss, D., Prescher, T., Haas, W., Fogh, A., Horn, J., Mangard, S., Kocher, P. C., Genkin, D., et al. (2018) · USENIX Security Symposium · Cited by 821 · see citing works
Stealing Machine Learning Models via Prediction APIs
Tramèr, F., Zhang, F., Juels, A., Reiter, M. K., & Ristenpart, T. (2016) · arXiv · doi:10.48550/arxiv.1609.02943 · Cited by 796 · see citing works
Parboil: A Revised Benchmark Suite for Scientific and Commercial Throughput Computing
Stratton, J. A., Rodrigues, C. I., Sung, I.-J., Obeid, N., Chang, L., Anssari, N., Liu, G. D., & Hwu, W. (2012) · Cited by 707 · see citing works
Privacy in Pharmacogenetics: An End-to-End Case Study of Personalized Warfarin Dosing.
Fredrikson, M., Lantz, E., Jha, S., Lin, S. M., Page, D., & Ristenpart, T. (2014) · PubMed · Cited by 658 · see citing works
Machine Learning Models that Remember Too Much
Song, C., Ristenpart, T., & Shmatikov, V. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134077 · Cited by 526 · see citing works
Stealing Hyperparameters in Machine Learning
Wang, B., & Gong, N. Z. (2018) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2018.00038 · Cited by 491 · see citing works
Ateniese, G., Mancini, L. V., Spognardi, A., Villani, A., Vitali, D., & Felici, G. (2015) · International Journal of Security and Networks · doi:10.1504/ijsn.2015.071829 · Cited by 482 · see citing works
Robust Physical-World Attacks on Deep Learning ModelsPreprint
Eykholt, K., Evtimov, I., Fernandes, E., Li, B., Rahmati, A., Xiao, C., Prakash, A., Kohno, T., & Song, D. X. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.08945 · Cited by 443 · see citing works
On the feasibility of online malware detection with performance counters
Demme, J., Maycock, M., Schmitz, J., Tang, A., Waksman, A., Sethumadhavan, S., & Stolfo, S. J. (2013) · Proceedings - International Symposium on Computer Architecture/Proceedings · doi:10.1145/2485922.2485970 · Cited by 341 · see citing works
Zaheer, M., Kottur, S., Ravanbakhsh, S., Póczos, B., Salakhutdinov, R. R., & Smola, A. J. (2017) · neural information processing systems · Cited by 279 · see citing works
Real time detection of cache-based side-channel attacks using hardware performance counters
Chiappetta, M., Savaş, E., & Yılmaz, C. (2016) · Applied Soft Computing · doi:10.1016/j.asoc.2016.09.014 · Cited by 262 · see citing works
On the feasibility of online malware detection with performance counters
Demme, J., Maycock, M., Schmitz, J., Tang, A., Waksman, A., Sethumadhavan, S., & Stolfo, S. J. (2013) · ACM SIGARCH Computer Architecture News · doi:10.1145/2508148.2485970 · Cited by 232 · see citing works
Robust Physical-World Attacks on Machine Learning Models.Preprint
Evtimov, I., Eykholt, K., Fernandes, E., Kohno, T., Li, B., Prakash, A., Rahmati, A., & Song, D. X. (2017) · arXiv (Cornell University) · Cited by 187 · see citing works
Adversarial Examples that Fool DetectorsPreprint
Lu, J., Sibai, H., & Fabry, E. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.02494 · Cited by 102 · see citing works
Wang, X., & Karri, R. (2013) · Proceedings - ACM IEEE Design Automation Conference · doi:10.1145/2463209.2488831 · Cited by 100 · see citing works
Analysis of universal adversarial perturbationsPreprint
Moosavi-Dezfooli, S.-M., Fawzi, A., Fawzi, O., Pascal Frossard, B., & Soatto, S. (2017) · arXiv (Cornell University) · Cited by 86 · see citing works
Mining on Someone Else’s Dime: Mitigating Covert Mining Operations in Clouds and Enterprises
Tahir, R., Huzaifa, M., Das, A., Ahmad, M. A., Gunter, C. A., Zaffar, F., Caesar, M., & Borisov, N. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66332-6_13 · Cited by 55 · see citing works
Self-citationOpenML: networked science in machine learning
Vanschoren, J., van Rijn, J. N., Bischl, B., et al. (2014) · doi:10.48550/arxiv.1407.7722 · Cited by 53 · see citing works
Stealing Hyperparameters in Machine LearningPreprint
Wang, B., & Gong, N. Z. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1802.05351 · Cited by 52 · see citing works
Towards Reverse-Engineering Black-Box Neural NetworksPreprint
Oh, S. J., Augustin, M., Schiele, B., & Fritz, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.01768 · Cited by 42 · see citing works
Klein, S. (2017) · Apress eBooks · doi:10.1007/978-1-4842-2143-3_14 · Cited by 39 · see citing works
The Unintended Consequences of Overfitting: Training Data Inference Attacks.Preprint
Yeom, S., Fredrikson, M., & Jha, S. (2017) · arXiv (Cornell University) · Cited by 20 · see citing works
Flagged first, then by citations
Research works printed but not linked: 9
- Amazon. 2018. Machine Learning at AWS. https://aws.amazon.com/machine-learning/ Retrieved August 15, 2018 from Amazon. 2018. Machine Learning at AWS. https://aws.amazon.com/machine-learning/ Retrieved August 15, 2018 from
- BigML Inc. 2018. BigML. https://bigml.com/ Retrieved August 15, 2018 from BigML Inc. 2018. BigML. https://bigml.com/ Retrieved August 15, 2018 from
- Eric Florenzano . 2016. Gadientzoo. https://www.gradientzoo.com/ Retrieved August 15, 2018 from Eric Florenzano. 2016. Gadientzoo. https://www.gradientzoo.com/ Retrieved August 15, 2018 from
- Google Cloud . 2018. Cloud Machine Learning Engine. https://cloud.google.com/ml-engine/ Retrieved August 15, 2018 from Google Cloud. 2018. Cloud Machine Learning Engine. https://cloud.google.com/ml-engine/ Retrieved August 15, 2018 from
- Jamie Hayes , Luca Melis , George Danezis , and Emiliano De Cristofaro . 2017. LOGAN: evaluating privacy leakage of generative models using generative adversarial networks. arXiv preprint arXiv:1705.07663 ( 2017 ). Jamie Hayes, Luca Melis, George Danezis, and Emiliano De Cristofaro. 2017. LOGAN: evaluating privacy leakage of generative models using generative adversarial networks. arXiv preprint arXiv:1705.07663 (2017). · arXiv
- Zhiqiang Lin , Xiangyu Zhang , and Dongyan Xu . 2010 . Automatic reverse engineering of data structures from binary execution . In Proceedings of the 11th Annual Information Security Symposium. CERIAS-Purdue University, 5. Zhiqiang Lin, Xiangyu Zhang, and Dongyan Xu. 2010. Automatic reverse engineering of data structures from binary execution. In Proceedings of the 11th Annual Information Security Symposium. CERIAS-Purdue University, 5.
- PyTorch core team. 2018. Pytorch . http://pytorch.org/ Retrieved August 15, 2018 from PyTorch core team. 2018. Pytorch. http://pytorch.org/ Retrieved August 15, 2018 from
- Perf Wiki . 2018. Linux profiling with performance counters. https://perf.wiki.kernel.org/index.php/Main_Page Retrieved August 15, 2018 from Perf Wiki. 2018. Linux profiling with performance counters. https://perf.wiki.kernel.org/index.php/Main_Page Retrieved August 15, 2018 from
- Manzil Zaheer Satwik Kottur Siamak Ravanbakhsh Barnabas Poczos Ruslan R Salakhutdinov and Alexander J Smola. 2017. Deep sets. In Advances in Neural Information Processing Systems. 3394--3404. Manzil Zaheer Satwik Kottur Siamak Ravanbakhsh Barnabas Poczos Ruslan R Salakhutdinov and Alexander J Smola. 2017. Deep sets. In Advances in Neural Information Processing Systems. 3394--3404.
Cited by
Sorted by citations
Exploiting Unintended Feature Leakage in Collaborative Learning
Melis, L., Song, C., De Cristofaro, E., & Shmatikov, V. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00029 · Cited by 1,734 · see citing works
Nasr, M., Shokri, R., & Houmansadr, A. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00065 · Cited by 1,760 · see citing works
“class. This attack works on datasets where all of the members in the same classes are similar. No metric for leakage is used to measure privacy, except illustrating the generated images. Ganju et al. [26] designed a new property inference attack on fully connected networks. In this attack, the attacker goal is to extr…”
BackgroundInverting Gradients -- How easy is it to break privacy in federated learning?Preprint
Geiping, J., Bauermeister, H., Dröge, H., & Moeller, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.14053 · Cited by 749 · see citing works
“A related, but easier problem, compared to the full reconstruction of input images, is the retrieval of input attributes [23, 10] from local updates, e.”
BackgroundMembership Inference Attacks on Machine Learning: A Survey
Hu, H., Salcic, Z. A., Sun, L., Dobbie, G., Yu, P. S., & Zhang, X. (2022) · ACM Computing Surveys · doi:10.1145/3523273 · Cited by 586 · see citing works
“However, one must be aware that DP cannot only be used to mitigate MIAs, but also other forms of privacy attacks such as attribute inference attacks [47, 48] and property inference attacks [4, 49].”
BackgroundMembership Inference Attacks From First Principles
Carlini, N., Chien, S., Nasr, M., Song, S., Terzis, A., & Tramèr, F. (2022) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46214.2022.9833649 · Cited by 562 · see citing works
“For example, a clas-sifier trained on bitcoin logs can reveal whether or not the machines that generated the logs were patched for Meltdown and Spectre [15].”
BackgroundPrivacy preservation in federated learning: An insightful survey from the GDPR perspective
Truong, N., Sun, K., Wang, S., Guitton, F., & Guo, Y. (2021) · Computers & Security · doi:10.1016/j.cose.2021.102402 · Cited by 399 · see citing works
“cessary to fully reconstruct the trained data; instead, inferring attributes or membership of the original trained data from local model parameters can also induce serious privacy leakage [15], [95], [100]–[102] (e.g., an attacker can figure out whether a specific data sample (of a patient) is used to train a model of a…”
BackgroundJia, J., Salem, A., Michael Backes, A., Zhang, Y., & Gong, N. Z. (2019) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3319535.3363201 · Cited by 374 · see citing works
“Several works [2, 16, 36] studied property inference attacks, which aim to infer a certain property (e.”
BackgroundBlockchain-Based Federated Learning for Securing Internet of Things: A Comprehensive Survey
Issa, W., Moustafa, N., Turnbull, B., Sohrabi, N., & Tari, Z. (2022) · ACM Computing Surveys · doi:10.1145/3560816 · Cited by 338 · see citing works
“For example, adversaries can use face recognition models to generate data samples similar to the original data, or they can discover how many times each class appears in the training data and the fraction of data for each class [45].”
BackgroundModel inversion attacks against collaborative inference
He, Z., Zhang, T., & Lee, R. B. (2019) · Proceedings of the 35th Annual Computer Security Applications Conference · doi:10.1145/3359789.3359824 · Cited by 313 · see citing works
“, property inference attacks [3, 11], membership inference attacks [18, 29, 30, 39, 41, 49], model inversion attacks [9, 10].”
BackgroundDetecting AI Trojans Using Meta Neural Analysis
Xu, X., Wang, Q., Li, H., Borisov, N., Gunter, C. A., & Li, B. (2021) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp40001.2021.00034 · Cited by 278 · see citing works
“With white-box access to the target model, the features can be derived directly from the parameters of the model [19].”
BackgroundMethodInfluentialSelf-citationFeature Inference Attack on Model Predictions in Vertical Federated Learning
Luo, X., Wu, Y., Xiao, X., & Ooi, B. C. (2021) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde51399.2021.00023 · Cited by 253 · see citing works
“Property inference [16] attempts to extract some underlying properties or statistics of a party’s training dataset, which are uncorrelated to the training task [16, 39].”
BackgroundDeep Q-Learning Based Reinforcement Learning Approach for Network Intrusion Detection
Alavizadeh, H., Alavizadeh, H., & Jang‐Jaccard, J. (2022) · Computers · doi:10.3390/computers11030041 · Cited by 234 · see citing works
“The neural network output f ( x ) or y is represented in Equation (1) [31]. where x is the input, F i is a transformation function, and | f | denotes the total number of computational layers which can be either hidden layers and the output layer in the neural network.”
BackgroundDynamic Backdoor Attacks Against Machine Learning Models
Salem, A., Wen, R., Michael Backes, A., Ma, S., & Zhang, Y. (2022) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp53844.2022.00049 · Cited by 227 · see citing works
“Beside the above, there are multiple other types of attacks against machine learning models [26], such as membership inference [6, 15–17, 29, 31, 42, 46, 48, 60], model stealing [35, 45, 52, 54], model inversion [9, 10, 18], property inference [11, 30, 63], and dataset reconstruction [41].”
BackgroundA Survey of Privacy Attacks in Machine Learning
Rigaki, M., & Garcia, S. (2024) · ACM Computing Surveys · doi:10.1145/3624010 · Cited by 285 · see citing works
“A common design pattern for a lot of supervised learning attacks is the use of shadow models and meta-models or attack-models [3, 18, 29, 40, 46, 51, 68, 97, 106, 108, 110, 116, 125].”
BackgroundInfluentialBadNL: Backdoor Attacks against NLP Models with Semantic-preserving Improvements
Chen, X., Salem, A., Chen, D., Michael Backes, A., Ma, S., Shen, Q., Wu, Z., & Zhang, Y. (2021) · Annual Computer Security Applications Conference · doi:10.1145/3485832.3485837 · Cited by 221 · see citing works
“For instance, multiple works have explored the security and privacy threats of data used to train DNNmodels, such as membership inference attack [30, 35, 36], dataset reconstruction attack [33], and property inference attack [8, 12].”
BackgroundMembership Leakage in Label-Only Exposures
Li, Z., & Zhang, Y. (2021) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3460120.3484575 · Cited by 226 · see citing works
“[20] proposed a property inference attack aiming at inferring general properties of the training data (such as the proportion of each class in the training data).”
BackgroundA Privacy-Preserving Federated Learning for Multiparty Data Sharing in Social IoTs
Yin, L., Feng, J., Xun, H., Sun, Z., & Cheng, X. (2021) · IEEE Transactions on Network Science and Engineering · doi:10.1109/tnse.2021.3074185 · Cited by 191 · see citing works
Privacy Risks of General-Purpose Language Models
Pan, X., Zhang, M., Ji, S., & Yang, M. (2020) · 2020 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp40000.2020.00095 · Cited by 188 · see citing works
“Different from membership inference, another branch of works called property inference aims at infering whether the training set has certain global property, which was first studied by [10] on shallow models and later extended by [23] to deep models.”
BackgroundPrivacy Risks of Securing Machine Learning Models against Adversarial Examples
Song, L., Shokri, R., & Mittal, P. (2019) · Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3319535.3354211 · Cited by 229 · see citing works
“Attacks targeting data privacy include: the adversary inferring whether input examples were used to train the target model with membership inference attacks [38, 42, 59], learning global properties of training data with property inference attacks [12], reconstructing training data with model inversion attacks [10, 11]…”
MethodWhen Machine Unlearning Jeopardizes Privacy
Chen, M., Zhang, Z., Wang, T., Backes, M., Humbert, M., & Zhang, Y. (2021) · Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3460120.3484756 · Cited by 195 · see citing works
“[20] proposed a property inference attack aiming at inferring general properties of the training data (such as the proportion of each class in the training data).”
BackgroundUnleashing the Tiger: Inference Attacks on Split Learning
Pasquini, D., Ateniese, G., & Bernaschi, M. (2021) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3460120.3485259 · Cited by 172 · see citing works
“This form of inference was introduced in [7] and extended to neural networks in [18].”
BackgroundCONTRA: Defending Against Poisoning Attacks in Federated Learning
Awan, S., Luo, B., & Li, F. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-88418-5_22 · Cited by 182 · see citing works
“As a result, federated learning becomes a new target of various adversarial machine learning attacks, such as poisoning attacks [9,29] and exploratory attacks [3,15,32].”
BackgroundChundawat, V. S., Tarun, A. K., Mandal, M., & Kankanhalli, M. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3265506 · Cited by 162 · see citing works
“[35] introduced a subcategory of membership inference attack called property inference attack.”
BackgroundUpdates-Leak: Data Set Inference and Reconstruction Attacks in Online Learning
Salem, A., Bhattacharya, A., Backes, M., Fritz, M., & Zhang, Y. (2020) · Universität des Saarlandes · doi:10.22028/d291-47370 · Cited by 157 · see citing works
Adversarial machine learning :
Vassilev, A., Oprea, A., Fordyce, A. J., & Anderson, H. S. (2024) · doi:10.6028/nist.ai.100-2e2023 · Cited by 194 · see citing works
Local and Central Differential Privacy for Robustness and Privacy in Federated Learning
Naseri, M., Hayes, J., & De Cristofaro, E. (2022) · Proceedings 2022 Network and Distributed System Security Symposium · doi:10.14722/ndss.2022.23054 · Cited by 188 · see citing works
“More specifically, the adversary might infer: 1) information about the model, as with model [49, 79, 105, 109] or functionality extraction attacks [80, 82]; 2) class representatives, as in the case of model inversion [30, 31]; 3) training inputs [11, 44, 101]; 4) presence of target records in the training set [40, 67,…”
BackgroundEthical machines: The human-centric use of artificial intelligence
Lepri, B., Oliver, N., & Pentland, A. (2021) · iScience · doi:10.1016/j.isci.2021.102249 · Cited by 149 · see citing works
“, 2017, 2019), as well as the adoption of property inference attacks to learn global properties of training data (Ganju et al., 2018; Song et al., 2019).”
BackgroundWhen the Curious Abandon Honesty: Federated Learning Is Not Private
Boenisch, F., Dziedzic, A., Schuster, R., Shamsabadi, A. S., Shumailov, I., & Papernot, N. (2023) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp57164.2023.00020 · Cited by 173 · see citing works
“allow the attacker to tell if a given data point was used in training) but also properties of the training data [14], [34].”
BackgroundAttacking Graph-based Classification via Manipulating the Graph Structure
Wang, B., & Gong, N. Z. (2019) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3319535.3354206 · Cited by 140 · see citing works
“…examples [7,12,27,30,37,40,43,46,51,53,56, 68], poisoning attacks [8,18,31,38,67,70], privacy attacks for users (e.g., model inversion attacks [19,20], membership inference attacks [49,52], property inference attacks [3,21]), as well as model parameter and hyperparameter stealing attacks [60,61].”
BackgroundResultAuditing Privacy Defenses in Federated Learning via Generative Gradient Leakage
Li, Z., Zhang, J., Liu, L., & Liu, J. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.00989 · Cited by 138 · see citing works
“Moreover, researchers have discovered that the exchanged model updates can be utilized to further infer unintended private information, such as the retrieval of certain input attributes [11, 35] (e.”
BackgroundAziz, R., Banerjee, S., Bouzefrane, S., & Vinh, T. L. (2023) · Future Internet · doi:10.3390/fi15090310 · Cited by 141 · see citing works
Machine Learning–based Cyber Attacks Targeting on Controlled Information
Miao, Y., Chen, C., Pan, L., Han, Q., Zhang, J., & Xiang, Y. (2021) · ACM Computing Surveys · doi:10.1145/3465171 · Cited by 133 · see citing works
“Attack Type Attack Targets Attack Surfaces Attacker’s Capabilities Model Info Training Set Info Training Phase Inference Phase Black-box Access White-box Access Model extraction attack [109] YES no no YES YES no Model extraction attack [87] YES no no YES YES no Hyperparameter stealing attack [114] YES no no YES YES no…”
BackgroundMethodInfluentialAP2FL: Auditable Privacy-Preserving Federated Learning Framework for Electronics in Healthcare
Yazdinejad, A., Dehghantanha, A., & Srivastava, G. (2024) · IEEE transactions on consumer electronics · doi:10.1109/tce.2023.3318509 · Cited by 117 · see citing works
“Moreover, investigators have found that exchanged data can be exploited to extract unintended private information, such as the presence of eyeglasses [13].”
BackgroundAttacking and Protecting Data Privacy in Edge–Cloud Collaborative Inference Systems
He, Z., Zhang, T., & Lee, R. B.-L. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3022358 · Cited by 126 · see citing works
“Attacks were demonstrated in traditional machine learning classifiers [25] and fully connected neural networks [26].”
BackgroundRIGA: Covert and Robust White-Box Watermarking of Deep Neural Networks
Wang, T., & Kerschbaum, F. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3450000 · Cited by 131 · see citing works
“In our implementation, we sort the weights as the neural networks are invariant in terms of different permutations of neurons in a layer [9].”
MethodQuantifying Privacy Leakage in Graph Embedding
Duddu, V., Boutet, A., & Shejwalkar, V. (2020) · EAI International Conference on Mobile and Ubiquitous Systems: Computing, Networking and Services (MobiQuitous) · doi:10.1145/3448891.3448939 · Cited by 124 · see citing works
“Prior literature in privacy attacks focus on models trained on non-graph data including text, images and speech to study the vulnerability to membership inference [30, 31], attribute inference [2, 12], property inference [11], model inversion [10] attacks as well as model parameter and hyperparameter stealing attacks…”
BackgroundSource Inference Attacks in Federated Learning
Hu, H., Salcic, Z. A., Sun, L., Dobbie, G., & Zhang, X. (2021) · Proceedings · doi:10.1109/icdm51629.2021.00129 · Cited by 119 · see citing works
“Location, Purchase, CHMNIST, MNIST, and CIFAR-10 are realistic datasets which are widely used for evaluating privacy leakage on ML models [30, 14, 8, 37].”
BackgroundMembership Inference Attack on Graph Neural Networks
Olatunji, I. E., Nejdl, W., & Khosla, M. (2021) · IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA) · doi:10.1109/tpsisa52974.2021.00002 · Cited by 111 · see citing works
“Other attacks against machine learningmodels includes property inference [12] and poisoning attack [2, 8, 31].”
BackgroundPrivacy and Fairness in Federated Learning: On the Perspective of Tradeoff
Chen, H., Zhu, T., Zhang, T., Zhou, W., & Yu, P. S. (2023) · ACM Computing Surveys · doi:10.1145/3606017 · Cited by 109 · see citing works
“Ganju et al. [58] extended this attack to a fully connected neural network case.”
BackgroundPrivacy in Deep Learning: A Survey
Mireshghallah, F., Taram, M., Vepakomma, P., Singh, A., Raskar, R., & Esmaeilzadeh, H. (2020) · arXiv · doi:10.48550/arxiv.2004.12254 · Cited by 106 · see citing works
“Inference Parameter Inference Property Inference Access to Model Access to Model Output Membership Inference [13] # # # # Blackbox Logits Measuring Membership Privacy[35] # # # # Blackbox Logits ML-Leaks[36] # # # # Blackbox Logits The Natural Auditor [37] # # # # Blackbox Label LOGAN [38] # # # # Both Logits Privacy…”
BackgroundInfluentialA Framework for Evaluating Client Privacy Leakages in Federated Learning
Wei, W., Liu, L., Loper, M. L., Chow, K.-H., Gürsoy, M. E., Truex, S., & Wu, Y. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-58951-6_27 · Cited by 236 · see citing works
Towards Security Threats of Deep Learning Systems: A Survey
He, Y., Meng, G., Chen, K., Hu, X., & He, J. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2020.3034721 · Cited by 114 · see citing works
“In [66], they try to find if some classifiers have enough women or minorities in training dataset.”
BackgroundMethodSoK: Secure Aggregation Based on Cryptographic Schemes for Federated Learning
Mansouri, M., Önen, M., Jaballah, W. B., & Conti, M. (2023) · Proceedings on Privacy Enhancing Technologies · doi:10.56553/popets-2023-0009 · Cited by 105 · see citing works
Nasr, M., Shokri, R., & Houmansadr, A. (2018) · arXiv (Cornell University) · Cited by 104 · see citing works
Leaky DNN: Stealing Deep-Learning Model Secret with GPU Context-Switching Side-Channel
Wei, J., Zhang, Y., Zhou, Z. G., Li, Z., & Faruque, M. A. A. (2020) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn48063.2020.00031 · Cited by 99 · see citing works
“Research showed that the information about data providers [14], [15], properties of the training data [3], [17], membership of a sample [20], [34], [56], model parameters [48], [59], [61] can be inferred when the model developer publishes its model or allows public API access.”
BackgroundKnowledge-Enriched Distributional Model Inversion Attacks
Chen, S., Kahla, M., Jia, R., & Qi, G.-J. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01587 · Cited by 98 · see citing works
“Attacks can be categorized into four types according to the specific goals: model extraction [19, 14, 22, 3], membership inference [25], property inference [1, 6, 18], and model inversion [5, 4, 31, 28].”
BackgroundKoshiyama, A., Kazim, E., Treleaven, P., Rai, P., Szpruch, L., Pavey, G., Ahamat, G., Leutner, F., Goebel, R., Knight, A., et al. (2024) · Royal Society Open Science · doi:10.1098/rsos.230859 · Cited by 110 · see citing works
“Attacks such as statistical disclosure [62], model inversion [63], inferring class representatives [64], membership and property inference [65–67] are different criteria that can be applied to an algorithm to assess levels of vulnerability.”
BackgroundPractical Blind Membership Inference Attack via Differential Comparisons
Hui, B., Yang, Y., Yuan, H., Burlina, P., Gong, N. Z., & Cao, Y. (2021) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2021.24293 · Cited by 102 · see citing works
“Machine learning is vulnerable to different privacy attacks including model inversion [12], [13], membership inference [43], property inference [2], [14], as well as model and hyperparameter stealing [46], [47].”
BackgroundSecurity and Privacy Threats to Federated Learning: Issues, Methods, and Challenges
Zhang, J., Zhu, H., Wang, F., Zhao, J., Xu, Q., & Li, H. (2022) · Security and Communication Networks · doi:10.1155/2022/2886795 · Cited by 94 · see citing works
“'e passive attacker with auxiliary data similar to the training data distribution of participants trains the specific property classifier to infer information [64].”
BackgroundMembership Inference Attacks Against Adversarially Robust Deep Learning Models
Song, L., Shokri, R., & Mittal, P. (2019) · IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw.2019.00021 · Cited by 89 · see citing works
“Well-known privacy issues include membership inference to infer whether an input is part of the model’s training dataset [11]–[13]; property inference to infer global properties of training dataset, such as the fraction of a certain class [14]; model inversion to reconstruct the model’s input from model predictions [1…”
BackgroundA Framework for Evaluating Gradient Leakage Attacks in Federated LearningPreprint
Wei, W., Liu, L., Loper, M. L., Chow, K. H., Gürsoy, M. E., Truex, S., & Wu, Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.10397 · Cited by 92 · see citing works
“Ganju et al [9] infers the global properties of the training data from a trained white-box fully connected neural network.”
BackgroundPrivacy-Preserving Machine Learning: Methods, Challenges and DirectionsPreprint
Xu, R., Baracaldo, N., & Joshi, J. B. D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.04417 · Cited by 172 · see citing works
“An adversary can also infer private information by exploiting an ML model via various inference attacks such as membership inference attacks [11, 12, 13, 14, 15], model inversion attacks [16, 17, 18], property inference attacks [19, 20], and privacy leakage from gradients exchanged in distributed ML scenarios [21, 22].”
BackgroundOn the Robustness of Backdoor-based Watermarking in Deep Neural Networks
Shafieinejad, M., Lukas, N., Wang, J., Li, X., & Kerschbaum, F. (2021) · Proceedings of the 2021 ACM Workshop on Information Hiding and Multimedia Security · doi:10.1145/3437880.3460401 · Cited by 98 · see citing works
Reconstructing Training Data with Informed Adversaries
Balle, B., Cherubin, G., & Hayes, J. (2022) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46214.2022.9833677 · Cited by 93 · see citing works
“Related work: The idea of using “neural networks to attack neural networks” has been used in the literature to implement a number of attacks, including (black-box and white-box) membership inference [5, 19, 20], model inversion [18], and property inference [12, 13].”
BackgroundMethodInfluentialBadNL: Backdoor Attacks Against NLP ModelsPreprint
Chen, X., Salem, A., Michael Backes, A., Ma, S., & Zhang, Y. (2020) · arXiv (Cornell University) · Cited by 87 · see citing works
Inference Attacks Against Collaborative Learning.Preprint
Melis, L., Song, C., Cristofaro, E. D., & Shmatikov, V. (2018) · arXiv (Cornell University) · Cited by 93 · see citing works
Gradient Leakage Attack Resilient Deep Learning
Wei, W., & Liu, L. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2021.3139777 · Cited by 82 · see citing works
“There are also privacy threats in prediction phase, e.g., model inversion [11], [12], attribute inference [13], [14], membership inference [15]– [20] and GAN-based reconstruction attack [21], [22].”
Elhanashi, A., Dini, P., Saponara, S., & Zheng, Q. (2023) · Electronics · doi:10.3390/electronics12244925 · Cited by 89 · see citing works
Revisiting Membership Inference Under Realistic Assumptions
Jayaraman, B., Wang, L., Knipmeyer, K., Gu, Q., & Evans, D. (2021) · Proceedings on Privacy Enhancing Technologies · doi:10.2478/popets-2021-0031 · Cited by 98 · see citing works
“…et al., 2019, Yeom et al., 2018], attribute inference [Fredrikson et al., 2014, 2015, Yeom et al., 2018], property inference [Ateniese et al., 2015, Ganju et al., 2018], model stealing [Lowd and Meek, 2005, Tram`er et al., 2016] and hyperparameter stealing [Wang and Gong, 2018, Yan et al., 2020].”
BackgroundAdversarial Attack on Large Scale Graph
Li, J., Xie, T., Chen, L., Xie, F., He, X., & Zheng, Z. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3078755 · Cited by 81 · see citing works
“The robustness and security of deep learning models have received widespread attention, there has been a surge of interests in the adversarial attacks [3], [4], [21], [22], [23].”
BackgroundMembership Privacy for Machine Learning Models Through Knowledge Transfer
Shejwalkar, V., & Houmansadr, A. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i11.17150 · Cited by 83 · see citing works
A Survey of Machine and Deep Learning Methods for Privacy Protection in the Internet of Things
Rodríguez, E., Otero, B., & Canal, R. (2023) · Sensors · doi:10.3390/s23031252 · Cited by 82 · see citing works
“They have been conducted in Hidden Markov Models (HMM) [46], SVM [47], and Neural Networks (NN) [48].”
MethodPoisoning-Assisted Property Inference Attack Against Federated Learning
Wang, Z., Huang, Y., Song, M., Wu, L., Xue, F., & Ren, K. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2022.3196646 · Cited by 82 · see citing works
“There are mainly three types of inference attacks, i.e., membership inference attack [10], [11], [12], reconstruction attack [5], [7], [13], [14], [15], and property inference attack [8], [16], [17].”
BackgroundLFighter: Defending against the label-flipping attack in federated learning
Jebreel, N., Domingo‐Ferrer, J., Sánchez, D., & Blanco-Justicia, A. (2023) · Neural Networks · doi:10.1016/j.neunet.2023.11.019 · Cited by 116 · see citing works
Differential privacy in deep learning: A literature survey
Pan, K., Ong, Y.-S., Gong, M., Li, H., Qin, A. K., & Gao, Y. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127663 · Cited by 79 · see citing works
Shen, M., Wang, H., Zhang, B., Zhu, L., Xu, K., Li, Q. L., & Du, X. (2020) · IEEE Internet of Things Journal · doi:10.1109/jiot.2020.3028110 · Cited by 71 · see citing works
“The main task is to recognize the digit shown in the image, and we want to infer whether the target model was trained using noisy images [35].”
BackgroundMethodInfluentialA survey on edge computing for wearable technology
Jin, X., Li, L., Dang, F., Chen, X., & Liu, Y. (2021) · Digital Signal Processing · doi:10.1016/j.dsp.2021.103146 · Cited by 75 · see citing works
Interaction-level Membership Inference Attack Against Federated Recommender Systems
Yuan, W., Yang, C., Nguyen, Q. V. H., Cui, L., He, T., & Yin, H. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583359 · Cited by 72 · see citing works
“These attacks include threats such as model inversion [48], attribute inference [3], and membership inference.”
BackgroundPrivacy Inference Attack and Defense in Centralized and Federated Learning: A Comprehensive Survey
Rao, B., Zhang, J., Wu, D., Zhu, C., Sun, X., & Chen, B. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3363670 · Cited by 79 · see citing works
AI system engineering key challenges and lessons learned
Fischer, L., Ehrlinger, L., Geist, V., Ramler, R., Sobiezky, F., Zellinger, W., Brunner, D., Kumar, M., & Moser, B. (2020) · Machine Learning and Knowledge Extraction · doi:10.3390/make3010004 · Cited by 71 · see citing works
“In contrast, the goal of property inference attacks is not at the individual level (membership of a certain class), but rather at the level of aggregated properties of the training data such as amount of data [51].”
BackgroundSousa, S., & Kern, R. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10204-6 · Cited by 74 · see citing works
“These attacks pose privacy threats for NLP models since the predicted properties may not be shared by the model producer in a consenting manner (Ganju et al. 2018).”
BackgroundSurvey: Leakage and Privacy at Inference Time
Jegorova, M., Kaul, C., Mayor, C., O'Neil, A. Q., Weir, A., Murray-Smith, R., & Tsaftaris, S. A. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3229593 · Cited by 77 · see citing works
“and ImageNet [38], [51], [53], as well as more applied datasets/tasks, such as classification of potential customer value [20], [33], [34], classification of the income level based on the Census data [21], [50], [55], [59], diagnosing breast cancer [21], [50], [55] and classifying X-rays [81], [87].”
BackgroundInfluentialSecure Aggregation is Insecure: Category Inference Attack on Federated Learning
Gao, J., Hou, B., Guo, X., Liu, Z., Zhang, Y., Chen, K., & Li, J. (2021) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2021.3128679 · Cited by 70 · see citing works
“Property inference attacks [12], [13] focus on inferring the properties of training data, which can be regarded as property privacy .”
BackgroundMethodInfluentialShield Against Gradient Leakage Attacks: Adaptive Privacy-Preserving Federated Learning
Hu, J., Wang, Z., Yong-sheng, S., Lin, B., Sun, P., Pang, X., Liu, J., & Ren, K. (2023) · IEEE/ACM Transactions on Networking · doi:10.1109/tnet.2023.3317870 · Cited by 69 · see citing works
“In the case of attribute inference attacks [17], [18], [32], an adversarial participant can infer the properties of other participants’ input samples and some properties may be unintended for the training task, of which the latter is more serious.”
BackgroundLabel-Only Model Inversion Attacks via Boundary Repulsion
Kahla, M., Chen, S., Just, H. A., & Jia, R. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01462 · Cited by 69 · see citing works
“Inversion-Attacks-via-Boundary-Repulsion pates in training [10,14,16,21] or whether a training dataset satisfies certain properties [9,15], to more “fine-grained” information, such as reconstructing the raw data [2, 3, 7, 24].”
BackgroundMemGuard: Defending against Black-Box Membership Inference Attacks via Adversarial ExamplesPreprint
Jia, J., Salem, A., Michael Backes, A., Zhang, Y. V., & Gong, N. Z. (2019) · arXiv (Cornell University) · doi:10.22028/d291-47339 · Cited by 71 · see citing works
TAPFed: Threshold Secure Aggregation for Privacy-Preserving Federated Learning
Xu, R., Li, B. H., Li, C., Joshi, J. B. D., Ma, S., & Li, J. (2024) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2024.3350206 · Cited by 68 · see citing works
“…have shown that private information can still be inferred by exploiting the final ML model, such as through data extraction, membership inference, model inversion, and property inference attacks [3], [4], [5], [6], or by exploiting the exchanged model update during the learning phase [7], [8], [9].”
MethodZhao, J., Bagchi, S., Avestimehr, S., Chan, K., Chaterji, S., Dimitriadis, D., Li, J., Li, N., Nourian, A., & Roth, H. (2025) · ACM Computing Surveys · doi:10.1145/3724113 · Cited by 63 · see citing works
“[44,56,58,92] Attackerhasblack-boxaccess .”
Property Inference from Poisoning
Mahloujifar, S., Ghosh, E., & Chase, M. (2022) · 2022 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp46214.2022.9833623 · Cited by 67 · see citing works
“In this attack, as in previous work [3, 17] we assume that the adversary can sample from D−, D+.”
BackgroundMethodResultInfluentialAdversarial interference and its mitigations in privacy-preserving collaborative machine learning
Usynin, D., Ziller, A., Makowski, M. R., Braren, R., Rueckert, D., Glocker, B., Kaissis, G. A., & Passerat‐Palmbach, J. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-021-00390-3 · Cited by 64 · see citing works
“However, once the model has been successfully extracted, all above-mentioned attacks that require white-box access can be performed 13,52 .”
BackgroundMethodUsing Highly Compressed Gradients in Federated Learning for Data Reconstruction Attacks
Yang, H., Ge, M., Xiang, K., & Li, J. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3227761 · Cited by 62 · see citing works
“To better illustrate the advantage of our Init-Generation , we also consider another dummy data initialization method Init-Property inspired by Melis et al. [5] and Ganju et al. [6], in which the properties of the original trained data can be inferred from the captured gradients.”
MethodRecent Advances of Differential Privacy in Centralized Deep Learning: A Systematic Survey
Demelius, L., Kern, R., & Trügler, A. (2025) · ACM Computing Surveys · doi:10.1145/3712000 · Cited by 76 · see citing works
“Demonstrations in the literature include, for instance, the proportion of women in the training data of an income prediction model, or the proportion of older faces from an image classification model for gender classification [59].”
Federated Learning Attacks and Defenses: A Survey
Chen, Y., Gui, Y., Lin, H., Gan, W., & Wu, Y. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020431 · Cited by 63 · see citing works
“However, there are some research works [24], [25], [26], [27] proving that some private data can be inferred from the transmission of data.”
DP-Forward: Fine-tuning and Inference on Language Models with Differential Privacy in Forward Pass
Du, M., Yue, X., Chow, S. S. M., Wang, T., Huang, C., & Sun, H. (2023) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3576915.3616592 · Cited by 63 · see citing works
“We are not interested in any global property of an entire corpus [30].”
BackgroundStarFL: Hybrid Federated Learning Architecture for Smart Urban Computing
Huang, A., Liu, Y., Chen, T., Zhou, Y., Sun, Q., Chai, H., & Yang, Q. (2021) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3467956 · Cited by 58 · see citing works
“However, more and more studies have shown that private information can still be divulged by analyzing the uploaded parameters or gradients [66, 120], such as model inversion attack [28, 29, 115] and membership inference attack [30, 88].”
BackgroundLuzón, M. V., Rodríguez-Barroso, N., Argente-Garrido, A., Jiménez-López, D., Moyano, J. M., Ser, J. D., Ding, W., & Herrera, F. (2024) · IEEE/CAA Journal of Automatica Sinica · doi:10.1109/jas.2024.124215 · Cited by 59 · see citing works
Label-Only Model Inversion Attacks: Attack With the Least Information
Zhu, T., Ye, D., Zhou, S., Liu, B., & Zhou, W. (2022) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2022.3233190 · Cited by 57 · see citing works
“There is also a stream of research that focuses on inferring the properties of training datasets [2], [10], [18], [33], such as the fraction of data that comes from a certain class.”
BackgroundFLGUARD: Secure and Private Federated Learning
Nguyen, T. D., Rieger, P., Yalame, H., Möllering, H., Fereidooni, H., Marchal, S., Miettinen, M., Mirhoseini, A., Sadeghi, A., Schneider, T. H., et al. (2021) · TUbilio (Technical University of Darmstadt) · Cited by 55 · see citing works
A Comprehensive Defense Framework Against Model Extraction Attacks
Jiang, W., Li, H., Xu, G., Zhang, T., & Lu, R. (2023) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2023.3261327 · Cited by 55 · see citing works
“…can be facilitated significantly easily in this scenario; (c) launching privacy attacks such as the membership attack [19], model inversion attack [20] andpropertyinferenceattack[21]:theseprivacyattacksaremore effectivewiththeavailabilityofawhite-boxmodel(viamodel extraction)thanablack-boxone.”
BackgroundFederated Analysis for Privacy-Preserving Data Sharing: A Technical and Legal Primer
Casaletto, J. A., Bernier, A., McDougall, R., & Cline, M. (2023) · Annual Review of Genomics and Human Genetics · doi:10.1146/annurev-genom-110122-084756 · Cited by 71 · see citing works
“A property inference attack attempts to infer some aggregate information about the training set as a whole, such as the environment where the data were produced or the percentage of the data that comes from a particular class (i.e., exploiting skewness) (2, 32).”
BackgroundGenerative AI model privacy: a survey
Liu, Y., Huang, J., Li, Y., Wang, D., & Xiao, B. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-11024-6 · Cited by 65 · see citing works
Privacy-preserving Voice Analysis via Disentangled Representations
Aloufi, R., Haddadi, H., & Boyle, D. (2020) · Cloud Computing Security Workshop · doi:10.1145/3411495.3421355 · Cited by 54 · see citing works
“Attribute inference attacks have been shown to compromise user privacy in various application domains including recommender systems [34], side-channel attacks [83], location inference attacks [73], and property inference attacks [21].”
BackgroundNaseri, M., Hayes, J., & Cristofaro, E. D. (2020) · arXiv (Cornell University) · Cited by 54 · see citing works
Advancing Differential Privacy: Where We Are Now and Future Directions for Real-World Deployment
Cummings, R., Desfontaines, D., Evans, D., Geambasu, R., Huang, Y., Jagielski, M., Kairouz, P., Kamath, G., Oh, S., Ohrimenko, O., et al. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.d3197524 · Cited by 71 · see citing works
Athena: Probabilistic Verification of Machine Unlearning
Sommer, D. M., Song, L., Wagh, S., & Mittal, P. (2022) · Proceedings on Privacy Enhancing Technologies · doi:10.56553/popets-2022-0072 · Cited by 51 · see citing works
Visual privacy attacks and defenses in deep learning: a survey
Zhang, G., Liu, B., Zhu, T., Zhou, A., & Zhou, W. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-021-10123-y · Cited by 53 · see citing works
Industry-Scale Orchestrated Federated Learning for Drug Discovery
Oldenhof, M., Ács, G., Pejó, B., Schuffenhauer, A., Holway, N., Sturm, N., Dieckmann, A., Fortmeier, O., Boniface, E., Mayer, C., et al. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i13.26847 · Cited by 67 · see citing works
An Overview of Privacy in Machine LearningPreprint
Cristofaro, E. D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.08679 · Cited by 91 · see citing works
“Ganju et al. [19] also develop property inference attacks, mainly against fully connected, relatively shallow neural networks.”
BackgroundProperty Inference Attacks Against GANs
Zhou, J., Chen, Y., Shen, C. Q., & Zhang, Y. (2022) · Network and Distributed System Security Symposium · doi:10.14722/ndss.2022.23019 · Cited by 49 · see citing works
“[17] propose the first property inference attack against discriminative models, in particular fully connected neural networks.”
BackgroundMethodInfluentialLAFED: A lightweight authentication mechanism for blockchain-enabled federated learning system
Ji, S., Zhang, J., Zhang, Y., Han, Z., & Ma, C. (2023) · Future Generation Computer Systems · doi:10.1016/j.future.2023.03.014 · Cited by 51 · see citing works
- Authors
- Ganju, K., Wang, Q., Yang, W., Gunter, C. A., & Borisov, N.
- Affiliations
- University of Illinois Urbana-Champaign (US)
- The University of Texas at Dallas (US)
- Venue
- Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security
- Publisher
- ACM
- Published
- 2018-10-15
- Pages
- 619-633
- ISSN
- 1543-7221 (ISSN-L)
- OpenAlex
- W2897830718
- Semantic Scholar
- 490c30b1d6b680be3c5a13552073e4fc10a850dc
- Cited by
- 475
- References
- 37
- Citations by year
- 2026: 27 · 2025: 58 · 2024: 89 · 2023: 98 · 2022: 74 · 2021: 69 · 2020: 39 · 2019: 19 · 2018: 2
- FWCI
- 13.11
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM SIGSAC Conference on Computer and Communications Security (CCS) (publishedVersion)
- Funders
- U.S. Department of Energy (DE-OE0000780)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
- Keywords
- property inference attacks, fully connected neural network, permutation invariance, training data leakage, white-box attacks, model sharing, performance counters
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
