Detecting Violations of Differential Privacy
Open accessThe DOI matches its Crossref record; 8 of the 8 references it cites were found in a registry or an index.
Abstract
The widespread acceptance of differential privacy has led to the publication of many sophisticated algorithms for protecting privacy. However, due to the subtle nature of this privacy definition, many such algorithms have bugs that make them violate their claimed privacy. In this paper, we consider the problem of producing counterexamples for such incorrect algorithms. The counterexamples are designed to be short and human-understandable so that the counterexample generator can be used in the development process -- a developer could quickly explore variations of an algorithm and investigate where they break down. Our approach is statistical in nature. It runs a candidate algorithm many times and uses statistical tests to try to detect violations of differential privacy. An evaluation on a variety of incorrect published algorithms validates the usefulness of our approach: it correctly rejects incorrect algorithms and provides counterexamples for them within a few seconds.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Calibrating Noise to Sensitivity in Private Data Analysis
Dwork, C., McSherry, F., Nissim, K., & Smith, A. (2006) · Lecture Notes in Computer Science · doi:10.1007/11681878_14 · Cited by 7,272
de Moura, L., & Bjørner, N. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-78800-3_24 · Cited by 6,595 · cited 1×
“The executor then uses an external MaxSMT solver such as Z3 [13] on all generated constraints to find arguments that maximizes the number of diverged branches.”
MethodThe Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2014) · Foundations and Trends® in Theoretical Computer Science · doi:10.1561/0400000042 · Cited by 5,557
Fisher, R. A. (1936) · BMJ · doi:10.1136/bmj.1.3923.554-a · Cited by 4,980
The Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2013) · doi:10.1561/9781601988195 · Cited by 3,856
Differential Privacy: A Survey of Results
Dwork, C. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-79228-4_1 · Cited by 3,542 · cited 2×
“49 Correct Exponential Noisy Max [15] 4.”
BackgroundOur Data, Ourselves: Privacy Via Distributed Noise Generation
Dwork, C., Kenthapadi, K., McSherry, F., Mironov, I., & Naor, M. (2006) · Lecture Notes in Computer Science · doi:10.1007/11761679_29 · Cited by 1,835 · cited 2×
“The term differential privacy covers a family of privacy definitions that include pure ε-differential privacy (the topic of this paper) [17] and its relaxations: approximate (ε,δ )differential privacy [16], concentrated differential privacy [8, 20], and Renyi differential privacy [31].”
BackgroundMironov, I. (2017) · 2017 IEEE 30th Computer Security Foundations Symposium (CSF) · doi:10.1109/csf.2017.11 · Cited by 1,255 · cited 1×
“The term differential privacy covers a family of privacy definitions that include pure ε-differential privacy (the topic of this paper) [17] and its relaxations: approximate (ε,δ )differential privacy [16], concentrated differential privacy [8, 20], and Renyi differential privacy [31].”
Background
Cited by
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772
Machine Learning Testing: Survey, Landscapes and Horizons
Zhang, J. M., Harman, M., Ma, L., & Liu, Y. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2962027 · Cited by 897
A Survey of Privacy Risks and Mitigation Strategies in the Artificial Intelligence Life Cycle
Shahriar, S., Allana, S., Hazratifard, S. M., & Dara, R. (2023) · IEEE Access · doi:10.1109/access.2023.3287195 · Cited by 114
“[172] demonstrated that there is no guarantee to confront the attacks by these methods comprehensively.”
BackgroundMaking Machine Learning Forget
Shintre, S., Roundy, K. A., & Dhaliwal, J. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-21752-5_6 · Cited by 13
“Practical implementations of solid differentially-private algorithms have been found to contain mistakes that result in significantly weaker privacy guarantees in practice than in theory [9] [23].”
Background
Metadata
- Authors
- Ding, Z., Wang, Y., Wang, G., Zhang, D., & Kifer, D.
- Venue
- Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security
- Publisher
- ACM
- Published
- 2018-10-15
- Pages
- 475-489
- OpenAlex
- W2886153332
- Cited by
- 103
- References
- 8
- FWCI
- 3.64
- Type
- conference paper
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Detecting Violations of Differential Privacy. https://app.cochat.ai/papers/10.1145/3243734.3243818
- The paper
- Ding, Z., Wang, Y., Wang, G., Zhang, D., & Kifer, D. (2018). Detecting Violations of Differential Privacy. Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. https://doi.org/10.1145/3243734.3243818
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.