Differentially Private Mixture of Generative Neural Networks
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Generative models are used in an increasing number of applications that rely on large amounts of contextually rich information about individuals. Owing to possible privacy violations, however, publishing or sharing generative models is not always viable. In this paper, we introduce a novel solution for privately releasing generative models and entire high-dimensional datasets produced by these models. We model the generator distribution of the training data by a mixture of k generative neural networks. These are trained together and collectively learn the generator distribution of a dataset. Data is divided into k clusters, using a novel differentially private kernel k-means, then each cluster is given to separate generative neural networks, such as Restricted Boltzmann Machines or Variational Autoencoders, which are trained only on their own cluster using differentially private gradient descent. We evaluate our approach using the MNIST dataset and a large Call Detail Records dataset, showing that it produces realistic synthetic samples, which can also be used to accurately compute arbitrary number of counting queries.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696
k-ANONYMITY: A MODEL FOR PROTECTING PRIVACY
SWEENEY, L. (2002) · International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems · doi:10.1142/s0218488502001648 · Cited by 8,571 · cited 1×
“unting queries. 2Related work In this section, we review prior work on privacy-preserving mechanisms applied to data mining, machine learning, and deep learning. Private Data Release. The k-anonymity [45] paradigm aims to protect data by generalizing and suppressing certain identifying attributes, however, it does not…”
BackgroundImproved Techniques for Training GANs
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016) · arXiv · doi:10.48550/arxiv.1606.03498 · Cited by 6,965
Deep Learning with Differential Privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978318 · Cited by 6,578
The Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2014) · Foundations and Trends® in Theoretical Computer Science · doi:10.1561/0400000042 · Cited by 5,557
The Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2013) · doi:10.1561/9781601988195 · Cited by 3,856
Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures
Fredrikson, M., Jha, S., & Ristenpart, T. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813677 · Cited by 2,961
Unsupervised deep embedding for clustering analysis
Xie, J., Girshick, R., & Farhadi, A. (2015) · arXiv · doi:10.48550/arxiv.1511.06335 · Cited by 2,476
Privacy-preserving deep learning
Shokri, R., & Shmatikov, V. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813687 · Cited by 2,356 · cited 1×
“While previous work explored the use of differential privacy in different areas of machine learning, including deep learning [1,19,22], privacy of generative models has not received much attention so far.”
BackgroundA note on the evaluation of generative models
Theis, L., Oord, A. V. D., & Bethge, M. (2015) · arXiv · doi:10.48550/arxiv.1511.01844 · Cited by 1,026 · cited 1×
“These models are used in a plethora of applications, such as compression [42], denoising [5], inpainting [50], superresolution [26], semi-supervised learning [37], clustering [43], and deep neural networks pretraining [18] in cases where labeled data is expensive.”
BackgroundJoint Unsupervised Learning of Deep Representations and Image Clusters
Yang, J., Parikh, D., & Batra, D. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.556 · Cited by 879 · cited 1×
“atures for clustering. A different training method is used in [17,31,54], where autoencoders are initially pre-trained, and then fine-tuned using the cluster assignment loss. Finally, other techniques [24,57] combine clustering with standard convolutional neural networks (CNNs) for representation learning of images. 3P…”
MethodBoosting and Differential Privacy
Dwork, C., Rothblum, G. N., & Vadhan, S. (2010) · 2010 IEEE 51st Annual Symposium on Foundations of Computer Science · doi:10.1109/focs.2010.12 · Cited by 784 · cited 1×
“, noisy kmeans iterations followed by noisy SGD iterations) much more accurately than previous work [11].”
ResultSemi-supervised Knowledge Transfer for Deep Learning from Private Training Data
Papernot, N., Abadi, M., Erlingsson, Ú., Goodfellow, I., & Talwar, K. (2016) · arXiv · doi:10.48550/arxiv.1610.05755 · Cited by 492 · cited 1×
“While previous work explored the use of differential privacy in different areas of machine learning, including deep learning [1,19,22], privacy of generative models has not received much attention so far.”
BackgroundGeneralized Denoising Auto-Encoders as Generative Models
Bengio, Y., Yao, L., Alain, G., & Vincent, P. (2013) · arXiv · doi:10.48550/arxiv.1305.6663 · Cited by 324
Cited by
When Machine Learning Meets Privacy: A Survey and Outlook
Liu, B., Ding, M., Shaham, S., Rahayu, W., Farokhi, F., & Lin, Z. (2022) · ACM Computing Surveys · doi:10.1145/3436755 · Cited by 455
Synthetic Data for Deep Learning
Nikolenko, S. I. (2021) · Springer Optimization and Its Applications · doi:10.1007/978-3-030-75178-4 · Cited by 423
Generative Adversarial Networks: A Survey Toward Private and Secure Applications
Cai, Z., Xiong, Z., Xu, H., Wang, P., Li, W., & Pan, Y. (2022) · ACM Computing Surveys · doi:10.1145/3459992 · Cited by 376
“[2] presented a first-of-its-kind attempt to build private generative models based on GAN.”
BackgroundGAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models
Chen, D., Yu, N., Zhang, Y., & Fritz, M. (2020) · Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3372297.3417238 · Cited by 311
“The generative model, on the other hand, has also gained increasing attention and has delivered appealing applications including photorealistic image synthesis [20, 44, 52, 70], text and sound generation [4, 49, 64, 65], sanitized dataset generation [2, 7, 35, 66, 73], etc.”
BackgroundInfluentialPrivacy and Security Issues in Deep Learning: A Survey
Liu, X., Xie, L., Wang, Y., Zou, J., Xiong, J., Ying, Z., & Vasilakos, A. V. (2021) · IEEE Access · doi:10.1109/access.2020.3045078 · Cited by 282
Metadata
- Authors
- Acs, G., Melis, L., Castelluccia, C., & De Cristofaro, E.
- Venue
- 2017 IEEE International Conference on Data Mining (ICDM)
- Publisher
- IEEE
- Published
- 2017-11
- Volume
- 31
- Issue
- 6
- Pages
- 715-720
- ISSN
- 1550-4786 (ISSN-L)
- OpenAlex
- W2756198004
- Cited by
- 136
- References
- 15
- FWCI
- 5.21
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.