Property Inference Attacks on Fully Connected Neural Networks using Permutation Invariant Representations

“class. This attack works on datasets where all of the members in the same classes are similar. No metric for leakage is used to measure privacy, except illustrating the generated images. Ganju et al. [26] designed a new property inference attack on fully connected networks. In this attack, the attacker goal is to extr…”

“A related, but easier problem, compared to the full reconstruction of input images, is the retrieval of input attributes [23, 10] from local updates, e.”

“However, one must be aware that DP cannot only be used to mitigate MIAs, but also other forms of privacy attacks such as attribute inference attacks [47, 48] and property inference attacks [4, 49].”

“For example, a clas-sifier trained on bitcoin logs can reveal whether or not the machines that generated the logs were patched for Meltdown and Spectre [15].”

“cessary to fully reconstruct the trained data; instead, inferring attributes or membership of the original trained data from local model parameters can also induce serious privacy leakage [15], [95], [100]–[102] (e.g., an attacker can figure out whether a specific data sample (of a patient) is used to train a model of a…”

“Several works [2, 16, 36] studied property inference attacks, which aim to infer a certain property (e.”

“For example, adversaries can use face recognition models to generate data samples similar to the original data, or they can discover how many times each class appears in the training data and the fraction of data for each class [45].”

“, property inference attacks [3, 11], membership inference attacks [18, 29, 30, 39, 41, 49], model inversion attacks [9, 10].”

“With white-box access to the target model, the features can be derived directly from the parameters of the model [19].”

“Property inference [16] attempts to extract some underlying properties or statistics of a party’s training dataset, which are uncorrelated to the training task [16, 39].”

“The neural network output f ( x ) or y is represented in Equation (1) [31]. where x is the input, F i is a transformation function, and | f | denotes the total number of computational layers which can be either hidden layers and the output layer in the neural network.”

“Beside the above, there are multiple other types of attacks against machine learning models [26], such as membership inference [6, 15–17, 29, 31, 42, 46, 48, 60], model stealing [35, 45, 52, 54], model inversion [9, 10, 18], property inference [11, 30, 63], and dataset reconstruction [41].”

“A common design pattern for a lot of supervised learning attacks is the use of shadow models and meta-models or attack-models [3, 18, 29, 40, 46, 51, 68, 97, 106, 108, 110, 116, 125].”

“For instance, multiple works have explored the security and privacy threats of data used to train DNNmodels, such as membership inference attack [30, 35, 36], dataset reconstruction attack [33], and property inference attack [8, 12].”

“[20] proposed a property inference attack aiming at inferring general properties of the training data (such as the proportion of each class in the training data).”

“Different from membership inference, another branch of works called property inference aims at infering whether the training set has certain global property, which was first studied by [10] on shallow models and later extended by [23] to deep models.”

“Attacks targeting data privacy include: the adversary inferring whether input examples were used to train the target model with membership inference attacks [38, 42, 59], learning global properties of training data with property inference attacks [12], reconstructing training data with model inversion attacks [10, 11]…”

“[20] proposed a property inference attack aiming at inferring general properties of the training data (such as the proportion of each class in the training data).”

“This form of inference was introduced in [7] and extended to neural networks in [18].”

“As a result, federated learning becomes a new target of various adversarial machine learning attacks, such as poisoning attacks [9,29] and exploratory attacks [3,15,32].”

“[35] introduced a subcategory of membership inference attack called property inference attack.”

“More specifically, the adversary might infer: 1) information about the model, as with model [49, 79, 105, 109] or functionality extraction attacks [80, 82]; 2) class representatives, as in the case of model inversion [30, 31]; 3) training inputs [11, 44, 101]; 4) presence of target records in the training set [40, 67,…”

“, 2017, 2019), as well as the adoption of property inference attacks to learn global properties of training data (Ganju et al., 2018; Song et al., 2019).”

“allow the attacker to tell if a given data point was used in training) but also properties of the training data [14], [34].”

“…examples [7,12,27,30,37,40,43,46,51,53,56, 68], poisoning attacks [8,18,31,38,67,70], privacy attacks for users (e.g., model inversion attacks [19,20], membership inference attacks [49,52], property inference attacks [3,21]), as well as model parameter and hyperparameter stealing attacks [60,61].”

“Moreover, researchers have discovered that the exchanged model updates can be utilized to further infer unintended private information, such as the retrieval of certain input attributes [11, 35] (e.”

“Attack Type Attack Targets Attack Surfaces Attacker’s Capabilities Model Info Training Set Info Training Phase Inference Phase Black-box Access White-box Access Model extraction attack [109] YES no no YES YES no Model extraction attack [87] YES no no YES YES no Hyperparameter stealing attack [114] YES no no YES YES no…”

“Moreover, investigators have found that exchanged data can be exploited to extract unintended private information, such as the presence of eyeglasses [13].”

“Attacks were demonstrated in traditional machine learning classifiers [25] and fully connected neural networks [26].”

“In our implementation, we sort the weights as the neural networks are invariant in terms of different permutations of neurons in a layer [9].”

“Prior literature in privacy attacks focus on models trained on non-graph data including text, images and speech to study the vulnerability to membership inference [30, 31], attribute inference [2, 12], property inference [11], model inversion [10] attacks as well as model parameter and hyperparameter stealing attacks…”

“Location, Purchase, CHMNIST, MNIST, and CIFAR-10 are realistic datasets which are widely used for evaluating privacy leakage on ML models [30, 14, 8, 37].”

“Other attacks against machine learningmodels includes property inference [12] and poisoning attack [2, 8, 31].”

“Ganju et al. [58] extended this attack to a fully connected neural network case.”

“Inference Parameter Inference Property Inference Access to Model Access to Model Output Membership Inference [13] # # # # Blackbox Logits Measuring Membership Privacy[35] # # # # Blackbox Logits ML-Leaks[36] # # # # Blackbox Logits The Natural Auditor [37] # # # # Blackbox Label LOGAN [38] # # # # Both Logits Privacy…”

“In [66], they try to find if some classifiers have enough women or minorities in training dataset.”

“Research showed that the information about data providers [14], [15], properties of the training data [3], [17], membership of a sample [20], [34], [56], model parameters [48], [59], [61] can be inferred when the model developer publishes its model or allows public API access.”

“Attacks can be categorized into four types according to the specific goals: model extraction [19, 14, 22, 3], membership inference [25], property inference [1, 6, 18], and model inversion [5, 4, 31, 28].”

“Attacks such as statistical disclosure [62], model inversion [63], inferring class representatives [64], membership and property inference [65–67] are different criteria that can be applied to an algorithm to assess levels of vulnerability.”

“Machine learning is vulnerable to different privacy attacks including model inversion [12], [13], membership inference [43], property inference [2], [14], as well as model and hyperparameter stealing [46], [47].”

“'e passive attacker with auxiliary data similar to the training data distribution of participants trains the specific property classifier to infer information [64].”

“Well-known privacy issues include membership inference to infer whether an input is part of the model’s training dataset [11]–[13]; property inference to infer global properties of training dataset, such as the fraction of a certain class [14]; model inversion to reconstruct the model’s input from model predictions [1…”

“Ganju et al [9] infers the global properties of the training data from a trained white-box fully connected neural network.”

“An adversary can also infer private information by exploiting an ML model via various inference attacks such as membership inference attacks [11, 12, 13, 14, 15], model inversion attacks [16, 17, 18], property inference attacks [19, 20], and privacy leakage from gradients exchanged in distributed ML scenarios [21, 22].”

“Related work: The idea of using “neural networks to attack neural networks” has been used in the literature to implement a number of attacks, including (black-box and white-box) membership inference [5, 19, 20], model inversion [18], and property inference [12, 13].”

“There are also privacy threats in prediction phase, e.g., model inversion [11], [12], attribute inference [13], [14], membership inference [15]– [20] and GAN-based reconstruction attack [21], [22].”

“…et al., 2019, Yeom et al., 2018], attribute inference [Fredrikson et al., 2014, 2015, Yeom et al., 2018], property inference [Ateniese et al., 2015, Ganju et al., 2018], model stealing [Lowd and Meek, 2005, Tram`er et al., 2016] and hyperparameter stealing [Wang and Gong, 2018, Yan et al., 2020].”

“The robustness and security of deep learning models have received widespread attention, there has been a surge of interests in the adversarial attacks [3], [4], [21], [22], [23].”

“They have been conducted in Hidden Markov Models (HMM) [46], SVM [47], and Neural Networks (NN) [48].”

“There are mainly three types of inference attacks, i.e., membership inference attack [10], [11], [12], reconstruction attack [5], [7], [13], [14], [15], and property inference attack [8], [16], [17].”

“The main task is to recognize the digit shown in the image, and we want to infer whether the target model was trained using noisy images [35].”

“These attacks include threats such as model inversion [48], attribute inference [3], and membership inference.”

“In contrast, the goal of property inference attacks is not at the individual level (membership of a certain class), but rather at the level of aggregated properties of the training data such as amount of data [51].”

“These attacks pose privacy threats for NLP models since the predicted properties may not be shared by the model producer in a consenting manner (Ganju et al. 2018).”

“and ImageNet [38], [51], [53], as well as more applied datasets/tasks, such as classification of potential customer value [20], [33], [34], classification of the income level based on the Census data [21], [50], [55], [59], diagnosing breast cancer [21], [50], [55] and classifying X-rays [81], [87].”

“Property inference attacks [12], [13] focus on inferring the properties of training data, which can be regarded as property privacy .”

“In the case of attribute inference attacks [17], [18], [32], an adversarial participant can infer the properties of other participants’ input samples and some properties may be unintended for the training task, of which the latter is more serious.”

“Inversion-Attacks-via-Boundary-Repulsion pates in training [10,14,16,21] or whether a training dataset satisfies certain properties [9,15], to more “fine-grained” information, such as reconstructing the raw data [2, 3, 7, 24].”

“…have shown that private information can still be inferred by exploiting the final ML model, such as through data extraction, membership inference, model inversion, and property inference attacks [3], [4], [5], [6], or by exploiting the exchanged model update during the learning phase [7], [8], [9].”

“[44,56,58,92] Attackerhasblack-boxaccess .”

“In this attack, as in previous work [3, 17] we assume that the adversary can sample from D−, D+.”

“However, once the model has been successfully extracted, all above-mentioned attacks that require white-box access can be performed 13,52 .”

“To better illustrate the advantage of our Init-Generation , we also consider another dummy data initialization method Init-Property inspired by Melis et al. [5] and Ganju et al. [6], in which the properties of the original trained data can be inferred from the captured gradients.”

“Demonstrations in the literature include, for instance, the proportion of women in the training data of an income prediction model, or the proportion of older faces from an image classification model for gender classification [59].”

“However, there are some research works [24], [25], [26], [27] proving that some private data can be inferred from the transmission of data.”

“We are not interested in any global property of an entire corpus [30].”

“However, more and more studies have shown that private information can still be divulged by analyzing the uploaded parameters or gradients [66, 120], such as model inversion attack [28, 29, 115] and membership inference attack [30, 88].”

“There is also a stream of research that focuses on inferring the properties of training datasets [2], [10], [18], [33], such as the fraction of data that comes from a certain class.”

“…can be facilitated significantly easily in this scenario; (c) launching privacy attacks such as the membership attack [19], model inversion attack [20] andpropertyinferenceattack[21]:theseprivacyattacksaremore effectivewiththeavailabilityofawhite-boxmodel(viamodel extraction)thanablack-boxone.”

“A property inference attack attempts to infer some aggregate information about the training set as a whole, such as the environment where the data were produced or the percentage of the data that comes from a particular class (i.e., exploiting skewness) (2, 32).”

“Attribute inference attacks have been shown to compromise user privacy in various application domains including recommender systems [34], side-channel attacks [83], location inference attacks [73], and property inference attacks [21].”

“Ganju et al. [19] also develop property inference attacks, mainly against fully connected, relatively shallow neural networks.”

“[17] propose the first property inference attack against discriminative models, in particular fully connected neural networks.”