Differentially Private Mixture of Generative Neural Networks
Open accessThe DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
Generative models are used in an increasing number of applications that rely on large amounts of contextually rich information about individuals. Owing to possible privacy violations, however, publishing or sharing generative models is not always viable. In this paper, we introduce a novel solution for privately releasing generative models and entire high-dimensional datasets produced by these models. We model the generator distribution of the training data by a mixture of k generative neural networks. These are trained together and collectively learn the generator distribution of a dataset. Data is divided into k clusters, using a novel differentially private kernel k-means, then each cluster is given to separate generative neural networks, such as Restricted Boltzmann Machines or Variational Autoencoders, which are trained only on their own cluster using differentially private gradient descent. We evaluate our approach using the MNIST dataset and a large Call Detail Records dataset, showing that it produces realistic synthetic samples, which can also be used to accurately compute arbitrary number of counting queries.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
Ledig, C., Theis, L., Huszar, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., et al. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.19 · Cited by 12,696
k-ANONYMITY: A MODEL FOR PROTECTING PRIVACY
SWEENEY, L. (2002) · International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems · doi:10.1142/s0218488502001648 · Cited by 8,571 · cited 1×
“unting queries. 2Related work In this section, we review prior work on privacy-preserving mechanisms applied to data mining, machine learning, and deep learning. Private Data Release. The k-anonymity [45] paradigm aims to protect data by generalizing and suppressing certain identifying attributes, however, it does not…”
BackgroundImproved Techniques for Training GANs
Salimans, T., Goodfellow, I., Zaremba, W., Cheung, V., Radford, A., & Chen, X. (2016) · arXiv · doi:10.48550/arxiv.1606.03498 · Cited by 6,965
Deep Learning with Differential Privacy
Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016) · Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2976749.2978318 · Cited by 6,578
The Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2014) · Foundations and Trends® in Theoretical Computer Science · doi:10.1561/0400000042 · Cited by 5,557
The Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2013) · doi:10.1561/9781601988195 · Cited by 3,856
Model Inversion Attacks that Exploit Confidence Information and Basic Countermeasures
Fredrikson, M., Jha, S., & Ristenpart, T. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813677 · Cited by 2,961
Unsupervised deep embedding for clustering analysis
Xie, J., Girshick, R., & Farhadi, A. (2015) · arXiv · doi:10.48550/arxiv.1511.06335 · Cited by 2,476
Privacy-preserving deep learning
Shokri, R., & Shmatikov, V. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813687 · Cited by 2,356 · cited 1×
“While previous work explored the use of differential privacy in different areas of machine learning, including deep learning [1,19,22], privacy of generative models has not received much attention so far.”
Background
Cited by
When Machine Learning Meets Privacy: A Survey and Outlook
Liu, B., Ding, M., Shaham, S., Rahayu, W., Farokhi, F., & Lin, Z. (2022) · ACM Computing Surveys · doi:10.1145/3436755 · Cited by 455
Synthetic Data for Deep Learning
Nikolenko, S. I. (2021) · Springer Optimization and Its Applications · doi:10.1007/978-3-030-75178-4 · Cited by 423
Generative Adversarial Networks: A Survey Toward Private and Secure Applications
Cai, Z., Xiong, Z., Xu, H., Wang, P., Li, W., & Pan, Y. (2022) · ACM Computing Surveys · doi:10.1145/3459992 · Cited by 376
“[2] presented a first-of-its-kind attempt to build private generative models based on GAN.”
BackgroundGAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models
Chen, D., Yu, N., Zhang, Y., & Fritz, M. (2020) · Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3372297.3417238 · Cited by 311
“The generative model, on the other hand, has also gained increasing attention and has delivered appealing applications including photorealistic image synthesis [20, 44, 52, 70], text and sound generation [4, 49, 64, 65], sanitized dataset generation [2, 7, 35, 66, 73], etc.”
BackgroundInfluentialPrivacy and Security Issues in Deep Learning: A Survey
Liu, X., Xie, L., Wang, Y., Zou, J., Xiong, J., Ying, Z., & Vasilakos, A. V. (2021) · IEEE Access · doi:10.1109/access.2020.3045078 · Cited by 282
A Survey on Intelligent Internet of Things: Applications, Security, Privacy, and Future Directions
Aouedi, O., Vu, T.-H., Sacco, A., Nguyen, D. C., Piamrat, K., Marchetto, G., & Pham, Q.-V. (2025) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3430368 · Cited by 277
Generating synthetic data in finance: opportunities, challenges and pitfalls
Assefa, S. A., Dervovic, D., Mahfouz, M., Tillman, R. E., Reddy, P., & Veloso, M. (2020) · Proceedings of the First ACM International Conference on AI in Finance · doi:10.1145/3383455.3422554 · Cited by 185
“[41] and [42] extend differentially private synthetic data generation to the domain of unstructured data such as images and audio.”
BackgroundA Survey on Differentially Private Machine Learning [Review Article]
Gong, M., Xie, Y., Pan, K., Feng, K., & Qin, A. K. (2020) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2020.2976185 · Cited by 115
Privacy in Deep Learning: A Survey
Mireshghallah, F., Taram, M., Vepakomma, P., Singh, A., Raskar, R., & Esmaeilzadeh, H. (2020) · arXiv · doi:10.48550/arxiv.2004.12254 · Cited by 106
“DPSGD [22] # # MNIST, CIFAR-10 Image Classification w/ DNN DP LSTM [77] # # Reddit Posts Language Model w/ LSTMs DP LOR [78] # # Artificial Data Logistic Regression DP ERM [79] # # Adult, KDD-99 Classification w/ ERM DP GAN [80] # # MNIST, MIMIC-III Data Generation w/ GAN DP GM [81] # # MNIST, CDR, TRANSIT Data Genera…”
MethodSecurity and Privacy Issues in Deep Learning
Bae, H., Jang, J., Jung, D., Jang, H., Ha, H., Lee, H., & Yoon, S. (2018) · arXiv · doi:10.48550/arxiv.1807.11655 · Cited by 56
“Depending on where the noise is added, such approaches can be divided into three groups: gradient-level [Abadi et al. 2016b; McMahan et al. 2018; Xie et al. 2018; Acs et al. 2018; Yu et al. 2019], objective-level [Chaudhuri and Monteleoni 2009; Phan et al.”
BackgroundOn Utility and Privacy in Synthetic Genomic Data
Oprisanu, B., Ganev, G., & De Cristofaro, E. (2022) · Proceedings 2022 Network and Distributed System Security Symposium · doi:10.14722/ndss.2022.24092 · Cited by 23
A comprehensive survey and taxonomy on privacy-preserving deep learning
Tran, A.-T., Luong, T.-D., & Huynh, V.-N. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127345 · Cited by 21
Metadata
- Authors
- Acs, G., Melis, L., Castelluccia, C., & De Cristofaro, E.
- Venue
- 2017 IEEE International Conference on Data Mining (ICDM)
- Publisher
- IEEE
- Published
- 2017-11
- Volume
- 31
- Issue
- 6
- Pages
- 715-720
- ISSN
- 1550-4786 (ISSN-L)
- OpenAlex
- W2756198004
- Cited by
- 136
- References
- 15
- FWCI
- 5.21
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.