On the Compatibility of Privacy and Fairness
The DOI matches its Crossref record; 15 of the 16 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
In this work, we investigate whether privacy and fairness can be simultaneously achieved by a single classifier in several different models. Some of the earliest work on fairness in algorithm design defined fairness as a guarantee of similar outputs for "similar'' input data, a notion with tight technical connections to differential privacy. We study whether tensions exist between differential privacy and statistical notions of fairness, namely Equality of False Positives and Equality of False Negatives (EFP/EFN). We show that even under full distributional access, there are cases where the constraint of differential privacy precludes exact EFP/EFN. We then turn to ask whether one can learn a differentially private classifier which approximately satisfies EFP/EFN, and show the existence of a PAC learner which is private and approximately fair with high probability. We conclude by giving an efficient algorithm for classification that maintains utility and satisfies both privacy and approximate fairness with high probability.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Cross-silo applications have also been proposed or described in myriad domains including finance risk prediction for reinsurance [476], pharmaceuticals discovery [179], electronic health records mining [184], medical data segmentation [15, 139], and smart manufacturing [354].”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [476]
In the citing paper
“For instance, different banks can collaboratively train classification or anomaly detection models for fraud detection [476], hospitals can build better diagnostic models [139], and so on.”
From Advances and Open Problems in Federated Learning (2021) · cites this paper as [476]
Citations
Cites
Calibrating Noise to Sensitivity in Private Data Analysis
Dwork, C., McSherry, F., Nissim, K., & Smith, A. (2006) · Lecture Notes in Computer Science · doi:10.1007/11681878_14 · Cited by 7,272
Proceedings of the 25th international conference on Machine learning - ICML '08
(2008) · doi:10.1145/1390156 · Cited by 6,680
Dwork, C., Hardt, M., Pitassi, T., Reingold, O., & Zemel, R. (2012) · Proceedings of the 3rd Innovations in Theoretical Computer Science Conference · doi:10.1145/2090236.2090255 · Cited by 3,501 · cited 1×
“The focus on fairness in machine learning and its relationship to differential privacy was explored in early work by the privacy community [7].”
BackgroundDifferentially Private Empirical Risk Minimization
Chaudhuri, K., Monteleoni, C., & Sarwate, A. (2009) · Journal of machine learning research · doi:10.5555/1953048.2021036 · Cited by 1,634 · cited 2×
“In our setting we consider only binary labels, and therefore consider randomized hypotheses h : X → [0, 1], where h(z) is the probability that h predicts 1 on example z. Definition 6 (Agnostic PAC Learning [3]).”
Towards Sharp Inapproximability For Any 2-CSP
Austrin, P. (2007) · 48th Annual IEEE Symposium on Foundations of Computer Science (FOCS'07) · doi:10.1109/focs.2007.41 · Cited by 1,294
(2011) · PubMed · Cited by 990 · cited 2×
“In our setting we consider only binary labels, and therefore consider randomized hypotheses h : X → [0, 1], where h(z) is the probability that h predicts 1 on example z. Definition 6 (Agnostic PAC Learning [3]).”
Smooth sensitivity and sampling in private data analysis
Nissim, K., Raskhodnikova, S., & Smith, A. (2007) · Proceedings of the thirty-ninth annual ACM symposium on Theory of computing · doi:10.1145/1250790.1250803 · Cited by 974 · cited 1×
“5Other methods for ensuring privacy under data-dependent sensitivity—such as Propose-Test-Release [5] or smoothed sensitivity [19]—could be used instead.”
MethodDifferential privacy and robust statistics
Dwork, C., & Lei, J. (2009) · Proceedings of the forty-first annual ACM symposium on Theory of computing · doi:10.1145/1536414.1536466 · Cited by 611 · cited 1×
“5Other methods for ensuring privacy under data-dependent sensitivity—such as Propose-Test-Release [5] or smoothed sensitivity [19]—could be used instead.”
MethodEfficient algorithms for online decision problems
Kalai, A., & Vempala, S. (2005) · Journal of Computer and System Sciences · doi:10.1016/j.jcss.2004.10.016 · Cited by 591
Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness
Kearns, M., Neel, S., Roth, A., & Wu, Z. S. (2017) · arXiv · doi:10.48550/arxiv.1711.05144 · Cited by 469 · cited 7×
“Definition 7 (False Positive (FP) Subgroup Fairness [15]).”
BackgroundMethodInfluential
Research works printed but not linked: 9
- Bird Sarah, 2016, Proceedings of the Workshop on Fairness, Accountability, and Transparency in Machine Learning
- Carlini Nicholas, 1802, The secret sharer: Measuring unintended neural network memorization and extracting secrets. arXiv pre-print
- Datta Anupam, 1808, Correspondences between privacy and nondiscrimination: Why they should be studied together. arXiv pre-print
- Ekstrand Michael D., 2018, Proceedings of the 1st Conference on Fairness, Accountability and Transparency, FAT* '18, 35
- Hardt Moritz, 2016, NIPS '16, 3315
- Jagielski Matthew, 2019, Differentially private fair learning. arXiv preprint arXiv:1812.02696 · arXiv
- Kearns Michael J., 2018, Proceedings of the 35th International Conference on Machine Learning, ICML '18. PMLR
- Narasimhan Harikrishna, 2018, Proceedings of the 21st International Conference on Artificial Intelligence and Statistics, AISTATS '18, 1646
- Woodworth Blake, 2017, Proceedings of the 2017 Conference on Learning Theory, COLT '17, 1920
Cited by
Advances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772
Self-citationA Review on Fairness in Machine Learning
Pessach, D., & Shmueli, E. (2023) · ACM Computing Surveys · doi:10.1145/3494672 · Cited by 598
“Most of these studies have investigated the case of a centralized setting, in which all the dataset is held by a single party [10, 52, 100, 106, 153, 210].”
BackgroundPrivacy-Preserving Machine Learning: Methods, Challenges and DirectionsPreprint
Xu, R., Baracaldo, N., & Joshi, J. B. D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.04417 · Cited by 300
Trustworthy ai: A computational perspective
Liu, H., Wang, Y., Fan, W., Liu, X., Li, Y., Jain, S., Liu, Y., Jain, A., & Tang, J. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3546872 · Cited by 261
“Second, if the tension is proven to be unsolvable, more research on this ield is needed for us to better understand how one dimension impacts one another, and how we can sacriice one for another based on our actual needs [105].”
BackgroundTowards Fair and Privacy-Preserving Federated Deep Models
Lyu, L., Yu, J., Nandakumar, K., Li, Y., Ma, X., Jin, J., Yu, H., & Ng, K. S. (2020) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2020.2996273 · Cited by 250
“Existing research on fairness mostly focuses on protecting sensitive attributes or reducing the variance of the prediction distribution across participants [6], [7].”
Background
Metadata
- Authors
- Cummings, R., Gupta, V., Kimpara, D., & Morgenstern, J.
- Affiliations
- Georgia Institute of Technology (US)
- Venue
- Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization
- Publisher
- ACM
- Published
- 2019-06-06
- Pages
- 309-315
- OpenAlex
- W2950103651
- Cited by
- 133
- References
- 15
- Citations by year
- 2026: 7 · 2025: 22 · 2024: 21 · 2023: 22 · 2022: 19 · 2021: 24 · 2020: 17 · 2019: 1
- FWCI
- 5.83
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Conference on User Modeling, Adaptation and Personalization (UMAP) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Explainable Artificial Intelligence (XAI)
- Keywords
- differential privacy, algorithmic fairness, PAC learning, machine learning classifiers
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: On the Compatibility of Privacy and Fairness. https://app.cochat.ai/papers/10.1145/3314183.3323847
- The paper
- Cummings, R., Gupta, V., Kimpara, D., & Morgenstern, J. (2019). On the Compatibility of Privacy and Fairness. Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization. https://doi.org/10.1145/3314183.3323847
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.