Robust Traceability from Trace Amounts
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
The privacy risks inherent in the release of a large number of summary statistics were illustrated by Homer et al. (PLoS Genetics, 2008), who considered the case of 1-way marginals of SNP allele frequencies obtained in a genome-wide association study: Given a large number of minor allele frequencies from a case group of individuals diagnosed with a particular disease, together with the genomic data of a single target individual and statistics from a sizable reference dataset independently drawn from the same population, an attacker can determine with high confidence whether or not the target is in the case group. In this work we describe and analyze a simple attack that succeeds even if the summary statistics are significantly distorted, whether due to measurement error or noise intentionally introduced to protect privacy. Our attack only requires that the vector of distorted summary statistics is close to the vector of true marginals in ℓ1norm. Moreover, the reference pool required by previous attacks can be replaced by a single sample drawn from the underlying population. The new attack, which is not specific to genomics and which handles Gaussian as well as Bernouilli data, significantly generalizes recent lower bounds on the noise needed to ensure differential privacy (Bun, Ullman, and Vadhan, STOC 2014, Steinke and Ullman, 2015), obviating the need for the attacker to control the exact distribution of the data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Calibrating Noise to Sensitivity in Private Data Analysis
Dwork, C., McSherry, F., Nissim, K., & Smith, A. (2006) · Lecture Notes in Computer Science · doi:10.1007/11681878_14 · Cited by 7,272
Self-citationOur Data, Ourselves: Privacy Via Distributed Noise Generation
Dwork, C., Kenthapadi, K., McSherry, F., Mironov, I., & Naor, M. (2006) · Lecture Notes in Computer Science · doi:10.1007/11761679_29 · Cited by 1,835
Self-citationHomer, N., Szelinger, S., Redman, M., Duggan, D., Tembe, W., Muehling, J., Pearson, J. V., Stephan, D. A., Nelson, S. F., & Craig, D. W. (2008) · PLoS Genetics · doi:10.1371/journal.pgen.1000167 · Cited by 1,221
Boosting and Differential Privacy
Dwork, C., Rothblum, G. N., & Vadhan, S. (2010) · 2010 IEEE 51st Annual Symposium on Foundations of Computer Science · doi:10.1109/focs.2010.12 · Cited by 784 · cited 3×
“Like the fingerprinting attacks of [14], [18], the lower bounds on noise implied by our attacks nearly match the upper bounds on noise sufficient to ensure the strong guarantees of differential privacy, for example, via the Gaussian or Laplace mechanisms [3], [21]–[24]).”
BackgroundSelf-citationIm, H. K., Gamazon, E. R., Nicolae, D. L., & Cox, N. J. (2012) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2012.02.008 · Cited by 103
Cited by
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754
“By contrast, the membership inference problem we study in this paper is essentially the same as the well-known problem of identifying the presence of an individual’s data in a mixed pool given some statistics about the pool [3], [15], [21], [29].”
MethodExploiting Unintended Feature Leakage in Collaborative Learning
Melis, L., Song, C., De Cristofaro, E., & Shmatikov, V. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00029 · Cited by 1,734
Nasr, M., Shokri, R., & Houmansadr, A. (2019) · 2019 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2019.00065 · Cited by 1,760
Machine Learning Models that Remember Too Much
Song, C., Ristenpart, T., & Shmatikov, V. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3134077 · Cited by 526
“Subsequent work extended this work to published noisy statistics [24] and MicroRNA-based studies [5].”
BackgroundMembership Inference Attacks From First Principles
Carlini, N., Chien, S., Nasr, M., Song, S., Terzis, A., & Tramèr, F. (2022) · 2022 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp46214.2022.9833649 · Cited by 502
Machine Learning with Membership Privacy using Adversarial Regularization
Nasr, M., Shokri, R., & Houmansadr, A. (2018) · Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3243734.3243855 · Cited by 458
“ata. The data required for training accurate models presents serious privacy issues. The leakage through complex machine learning models maybe less obvious, compared to, for example, linear statistics[18].However,machine learning models,similartoother types of computations, could significantly leak information about th…”
BackgroundMethodInfluentialDong, J., Roth, A., & Su, W. J. (2019) · arXiv · doi:10.48550/arxiv.1905.02383 · Cited by 389
Dong, J., Roth, A., & Su, W. J. (2022) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/rssb.12454 · Cited by 374
GAN-Leaks: A Taxonomy of Membership Inference Attacks against Generative Models
Chen, D., Yu, N., Zhang, Y., & Fritz, M. (2020) · Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3372297.3417238 · Cited by 311
“raining data. The resulting privacy breach would raise serious issues as training data contains sensitive attributes such as diagnosis and income. One such attack is membership inference attack (MIA) [5, 18, 24, 25, 58, 60] which aims to identify if a data record was used to train a machine learning model. Overfitting…”
BackgroundExposed! A Survey of Attacks on Private Data
Dwork, C., Smith, A., Steinke, T., & Ullman, J. (2017) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-060116-054123 · Cited by 259
“The specific technical conditions we need to impose are beyond the scope of this article but can be found in Dwork et al. (2015d).”
BackgroundMethodInfluentialSelf-citationAn economic analysis of privacy protection and statistical accuracy as social choices
Abowd, J. M., & Schmutte, I. M. (2019) · American Economic Review · doi:10.1257/aer.20170627 · Cited by 162
Chundawat, V. S., Tarun, A. K., Mandal, M., & Kankanhalli, M. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3265506 · Cited by 162
The distributed discrete gaussian mechanism for federated learning with secure aggregation
Kairouz, P., Liu, Z., & Steinke, T. (2021) · arXiv · doi:10.48550/arxiv.2102.06387 · Cited by 117
“…individual client updates, the server is still able to learn the sum of the updates, which itself may leak potentially sensitive information (Melis et al., 2019; Carlini et al., 2019; Song & Shmatikov, 2019a; Dwork et al., 2015; Song & Shmatikov, 2019b; Nasr et al., 2021; Shokri et al., 2017).”
BackgroundSelf-citationAuditing Data Provenance in Text-Generation Models
Song, C., & Shmatikov, V. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330885 · Cited by 103
Trusted AI in Multiagent Systems: An Overview of Privacy and Security for Distributed Learning
Ma, C., Li, J., Wei, K., Liu, B., Ding, M., Yuan, L., Han, Z., & Vincent Poor, H. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3306773 · Cited by 96
PUMA: Performance Unchanged Model Augmentation for Training Data Removal
Wu, G., Hashemi, M., & Srinivasa, C. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i8.20846 · Cited by 61
BayBFed: Bayesian Backdoor Defense for Federated Learning
Kumari, K., Rieger, P., Fereidooni, H., Jadliwala, M., & Sadeghi, A.-R. (2023) · 2023 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp46215.2023.10179362 · Cited by 37
“These defense mechanisms follow clipping of the weights and additive noising [10], to limit the impact of the adversarial updates.”
BackgroundCoinPress: Practical Private Mean and Covariance Estimation
Biswas, S., Dong, Y., Kamath, G., & Ullman, J. (2020) · arXiv · doi:10.48550/arxiv.2006.06618 · Cited by 29
Self-citationDifferential Privacy Defenses and Sampling Attacks for Membership Inference
Rahimian, S., Orekondy, T., & Fritz, M. (2021) · Proceedings of the 14th ACM Workshop on Artificial Intelligence and Security · doi:10.1145/3474369.3486876 · Cited by 24
Graves, L., Nagisetty, V., & Ganesh, V. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i13.17371 · Cited by 5
Understanding Unintended Memorization in Federated Learning
Thakkar, O., Ramaswamy, S., Mathews, R., & Beaufays, F. (2020) · arXiv · doi:10.48550/arxiv.2006.07490
Metadata
- Authors
- Dwork, C., Smith, A., Steinke, T., Ullman, J., & Vadhan, S.
- Venue
- 2015 IEEE 56th Annual Symposium on Foundations of Computer Science
- Publisher
- IEEE
- Published
- 2015-10
- Pages
- 650-669
- ISSN
- 0272-5428 (ISSN-L)
- OpenAlex
- W2198253679
- Cited by
- 137
- References
- 5
- FWCI
- 5.11
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.