Privacy-Preserving Machine Learning: Methods, Challenges and Directions
Open accessThe DOI matches its OpenAlex record; 50 of the 50 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Communication-Efficient Learning of Deep Networks from Decentralized Data
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2016) · arXiv · doi:10.48550/arxiv.1602.05629 · Cited by 14,215
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
k-ANONYMITY: A MODEL FOR PROTECTING PRIVACY
SWEENEY, L. (2002) · International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems · doi:10.1142/s0218488502001648 · Cited by 8,571 · cited 6×
“Given a person-specific field-structured dataset, k-anonymity and its variations are dedicated to creating and concealing identifiers and quasiidentifiers in such a way that the individuals who are the subjects of the data cannot be re-identified while the data remain useful [39].”
BackgroundInfluentialAdvances and Open Problems in Federated Learning
Kairouz, P., & McMahan, H. B. (2021) · Foundations and Trends in Machine Learning · doi:10.1561/2200000083 · Cited by 7,772
Public-Key Cryptosystems Based on Composite Degree Residuosity Classes
Paillier, P. (2007) · Lecture Notes in Computer Science · doi:10.1007/3-540-48910-x_16 · Cited by 7,250
Fully homomorphic encryption using ideal lattices
Gentry, C. (2009) · Proceedings of the forty-first annual ACM symposium on Theory of computing · doi:10.1145/1536414.1536440 · Cited by 6,806 · cited 5×
“Existing cryptographic approaches for data publishing for ML training fall into following two directions: (i) utilizing traditional symmetric encryption schemes such as AES in conjunction with the garbled-circuits and oblivious transfer to achieve general secure multi-party computation protocols [103, 104, 105] and (a…”
BackgroundMethodInfluentialFederated Machine Learning: Concept and Applications
Yang, Q., Liu, Y., Chen, T., & Tong, Y. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3298981 · Cited by 6,322
Federated learning: Challenges, methods, and future directions
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020) · IEEE Signal Processing Magazine · doi:10.1109/msp.2020.2975749 · Cited by 6,210 · cited 1×
“We refer the reader to [25, 184, 26, 28] for details.”
BackgroundThe Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2014) · Foundations and Trends® in Theoretical Computer Science · doi:10.1561/0400000042 · Cited by 5,557
Membership Inference Attacks Against Machine Learning Models
Shokri, R., Stronati, M., Song, C., & Shmatikov, V. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.41 · Cited by 4,754 · cited 4×
“An adversary can also infer private information by exploiting an ML model via various inference attacks such as membership inference attacks [11, 12, 13, 14, 15], model inversion attacks [16, 17, 18], property inference attacks [19, 20], and privacy leakage from gradients exchanged in distributed ML scenarios [21, 22].”
BackgroundInfluentialPractical Secure Aggregation for Privacy-Preserving Machine Learning
Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., Ramage, D., Segal, A., & Seth, K. (2017) · Proceedings of the 2017 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3133956.3133982 · Cited by 3,956 · cited 5×
“Existing cryptographic approaches for data publishing for ML training fall into following two directions: (i) utilizing traditional symmetric encryption schemes such as AES in conjunction with the garbled-circuits and oblivious transfer to achieve general secure multi-party computation protocols [103, 104, 105] and (a…”
BackgroundMethodInfluentialThe Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2013) · doi:10.1561/9781601988195 · Cited by 3,856
L -diversity: Privacy beyond k -anonymity
Machanavajjhala, A., Kifer, D., Gehrke, J., & Venkitasubramaniam, M. (2007) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/1217299.1217302 · Cited by 3,617
Differential Privacy: A Survey of Results
Dwork, C. (2008) · Lecture Notes in Computer Science · doi:10.1007/978-3-540-79228-4_1 · Cited by 3,542 · cited 7×
“Differential Privacy: Typically, the perturbation-based privacy-preserving data publishing approaches primarily refer to (ε, δ)-differential privacy technique [45, 46, 47] and more recent (α, ε)- Rènyi differential privacy [93] that is based on Rényi divergence.”
BackgroundMethodInfluentialt-Closeness: Privacy Beyond k-Anonymity and l-Diversity
Li, N., Li, T., & Venkatasubramanian, S. (2007) · 2007 IEEE 23rd International Conference on Data Engineering · doi:10.1109/icde.2007.367856 · Cited by 3,451 · cited 4×
“Some common approaches include differential privacy mechanisms [43, 97] and k-anonymity mechanism [37] and its follow-up work such as l-diversity [38], t-disclosure [39].”
BackgroundMethodInfluentialHomomorphic Encryption for Arithmetic of Approximate Numbers
Cheon, J. H., Kim, A., Kim, M., & Song, Y. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-70694-8_15 · Cited by 2,522 · cited 4×
“Furthermore, with the exception of homomorphic encryption systems based on approximate number arithmetic, such as CKKS [77], the majority of secure computation-related cryptography schemes perform on the integer group.”
BackgroundMethodInfluentialPrivacy-preserving deep learning
Shokri, R., & Shmatikov, V. (2015) · Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2810103.2813687 · Cited by 2,356 · cited 2×
“Shokri and Shmatikov [177] propose a distributed selective SGD framework that enables multiple parties to jointly learn an accurate neural network model without sharing their input datasets.”
BackgroundSecureML: A System for Scalable Privacy-Preserving Machine Learning
Mohassel, P., & Zhang, Y. (2017) · 2017 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2017.12 · Cited by 1,936 · cited 7×
“The majority of PPML systems that emphasize privacy-preserving training via secure computation techniques also indicate theoretical support for privacy-preserving inference, such as in [87, 88, 89, 58, 64, 37]; these, therefore, may be regarded as full privacy-preserving pipeline approaches.”
BackgroundMethodInfluentialZhu, L., & Han, S. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-63076-8_2 · Cited by 1,906
Privacy-preserving data mining
Agrawal, R., & Srikant, R. (2000) · Proceedings of the 2000 ACM SIGMOD international conference on Management of data · doi:10.1145/342009.335438 · Cited by 1,719
Federated Learning for Healthcare Informatics
Xu, J., Glicksberg, B. S., Su, C., Walker, P., Bian, J., & Wang, F. (2019) · arXiv · doi:10.48550/arxiv.1911.06270 · Cited by 1,538
How to backdoor federated learning
Bagdasaryan, E., Veit, A., Hua, Y., Estrin, D., & Shmatikov, V. (2018) · arXiv · doi:10.48550/arxiv.1807.00459 · Cited by 1,495 · cited 1×
“In terms of poisoning attack, several typical attacks include, for instance, clean-label data poisoning attack [191] where the adversary is assumed not to change the label of any training data and hence the poisoning of data samples has to be imperceptible, dirty-label data poisoning attack [192] where the adversary i…”
BackgroundA Survey on Federated Learning Systems: Vision, Hype and Reality for Data Privacy and Protection
Li, Q., Wen, Z., Wu, Z., Hu, S., Wang, N., Li, Y., Liu, X., & He, B. (2023) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3124599 · Cited by 1,407 · cited 4×
“Hence, we refer the readers for the surveys or systemization of knowledge articles such as in [23, 24, 25, 26] in vanilla ML research areas instead of repeating those content here.”
BackgroundMethodInfluentialMultiparty Computation from Somewhat Homomorphic Encryption
Damgård, I., Pastro, V., Smart, N., & Zakarias, S. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-32009-5_38 · Cited by 1,331 · cited 3×
“Simultaneously, the core compute engine is built on secret sharing, provides efficient arithmetics, and supports SPDZ [142], and ABY3 [123].”
BackgroundMethodMironov, I. (2017) · 2017 IEEE 30th Computer Security Foundations Symposium (CSF) · doi:10.1109/csf.2017.11 · Cited by 1,255 · cited 1×
“Differential Privacy: Typically, the perturbation-based privacy-preserving data publishing approaches primarily refer to (ε, δ)-differential privacy technique [45, 46, 47] and more recent (α, ε)- Rènyi differential privacy [93] that is based on Rényi divergence.”
MethodPrivacy Preserving Data Mining
Lindell, Y., & Pinkas, B. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-44598-6_3 · Cited by 1,077
A Hybrid Approach to Privacy-Preserving Federated Learning
Truex, S., Baracaldo, N., Anwar, A., Steinke, T., Ludwig, H., Zhang, R., & Zhou, Y. (2019) · Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security · doi:10.1145/3338501.3357370 · Cited by 1,029
Self-citationLearning Differentially Private Recurrent Language Models
McMahan, H. B., Ramage, D., Talwar, K., & Zhang, L. (2017) · arXiv · doi:10.48550/arxiv.1710.06963 · Cited by 849
Boosting and Differential Privacy
Dwork, C., Rothblum, G. N., & Vadhan, S. (2010) · 2010 IEEE 51st Annual Symposium on Foundations of Computer Science · doi:10.1109/focs.2010.12 · Cited by 784 · cited 3×
“From the perspective of data, existing privacy-preserving approaches work on the following directions: (i) adopting the traditional anonymization mechanisms such as k-anonymity[37], l-diversity[38], and t-closeness [39] to remove the identifier information in the training data before sending out the data for training;…”
BackgroundMethodSecureBoost: A Lossless Federated Learning Framework
Cheng, K., Fan, T., Jin, Y., Liu, Y., Chen, T., Papadopoulos, D., & Yang, Q. (2021) · IEEE Intelligent Systems · doi:10.1109/mis.2021.3082561 · Cited by 669
SecureBoost: A Lossless Federated Learning Framework
Cheng, K., Fan, T., Jin, Y., Liu, Y., Chen, T., Papadopoulos, D., & Yang, Q. (2019) · arXiv · doi:10.48550/arxiv.1901.08755 · Cited by 668
Yin, X., Zhu, Y., & Hu, J. (2022) · ACM Computing Surveys · doi:10.1145/3460427 · Cited by 638 · cited 3×
“Recent ML-related or FL-oriented surveys such as in [23, 24, 25, 26, 27, 28] have partially illustrated or discussed the specific privacy and security issues in ML or FL systems.”
BackgroundEfficient and Privacy-Enhanced Federated Learning for Industrial Artificial Intelligence
Hao, M., Li, H., Luo, X., Xu, G., Yang, H., & Liu, S. (2020) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2019.2945367 · Cited by 569 · cited 1×
“FL has shown its promise in various application domains, including healthcare, vehicular networks, intelligent manufacturing, among others [6, 7, 8].”
BackgroundSoK: Security and Privacy in Machine Learning
Papernot, N., McDaniel, P., Sinha, A., & Wellman, M. P. (2018) · 2018 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2018.00035 · Cited by 502 · cited 1×
“ML security, such as issues of stealing the ML models, injecting Trojans and availability of ML services and corresponding countermeasures, have been discussed in various recent articles such as those related to the systematization of knowledge [31], or surveys/analyses [32, 33, 34].”
BackgroundSemi-supervised Knowledge Transfer for Deep Learning from Private Training Data
Papernot, N., Abadi, M., Erlingsson, Ú., Goodfellow, I., & Talwar, K. (2016) · arXiv · doi:10.48550/arxiv.1610.05755 · Cited by 492
Ganju, K., Wang, Q., Yang, W., Gunter, C. A., & Borisov, N. (2018) · Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3243734.3243834 · Cited by 475 · cited 1×
“An adversary can also infer private information by exploiting an ML model via various inference attacks such as membership inference attacks [11, 12, 13, 14, 15], model inversion attacks [16, 17, 18], property inference attacks [19, 20], and privacy leakage from gradients exchanged in distributed ML scenarios [21, 22].”
BackgroundDistributed Federated Learning for Ultra-Reliable Low-Latency Vehicular Communications
Samarakoon, S., Bennis, M., Saad, W., & Debbah, M. (2020) · IEEE Transactions on Communications · doi:10.1109/tcomm.2019.2956472 · Cited by 459
Machine Learning with Membership Privacy using Adversarial Regularization
Nasr, M., Shokri, R., & Houmansadr, A. (2018) · Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3243734.3243855 · Cited by 458 · cited 3×
“An adversary can also infer private information by exploiting an ML model via various inference attacks such as membership inference attacks [11, 12, 13, 14, 15], model inversion attacks [16, 17, 18], property inference attacks [19, 20], and privacy leakage from gradients exchanged in distributed ML scenarios [21, 22].”
BackgroundHardy, S., Henecka, W., Ivey-Law, H., Nock, R., Patrini, G., Smith, G., & Thorne, B. (2017) · arXiv · doi:10.48550/arxiv.1711.10677 · Cited by 431 · cited 3×
“For example, recent works, as those in [60, 61], employ secure multiparty computation and FL approaches to distribute training of machine learning models across many”
BackgroundMethodTurbo-Aggregate: Breaking the Quadratic Aggregation Barrier in Secure Federated Learning
So, J., Guler, B., & Avestimehr, A. S. (2021) · IEEE Journal on Selected Areas in Information Theory · doi:10.1109/jsait.2021.3054610 · Cited by 318 · cited 4×
“Recent proposals, such as in [36, 116, 117], usually rely on a set of cryptographic primitives.”
BackgroundMethodInfluentialModel inversion attacks against collaborative inference
He, Z., Zhang, T., & Lee, R. B. (2019) · Proceedings of the 35th Annual Computer Security Applications Conference · doi:10.1145/3359789.3359824 · Cited by 313 · cited 3×
“An adversary can also infer private information by exploiting an ML model via various inference attacks such as membership inference attacks [11, 12, 13, 14, 15], model inversion attacks [16, 17, 18], property inference attacks [19, 20], and privacy leakage from gradients exchanged in distributed ML scenarios [21, 22].”
BackgroundCryptoDL: Deep Neural Networks over Encrypted Data
Hesamifard, E., Takabi, H., & Ghasemi, M. (2017) · arXiv · doi:10.48550/arxiv.1711.05189 · Cited by 306
CrypTen: Secure Multi-Party Computation Meets Machine Learning
Knott, B., Venkataraman, S., Hannun, A., Sengupta, S., Ibrahim, M., & van der Maaten, L. (2021) · arXiv · doi:10.48550/arxiv.2109.00984 · Cited by 257 · cited 4×
“Representative mixed-protocol solutions include TASTY [161], ABY [124], ABY3 [123], Chameleon [106], CrypTen [162], Falcon [163], etc. Notably, while a portion of the solutions listed above were presented in Section 5.2.2 as the garbled-circuits approach, we revisit those frameworks in a mixed fashion here.”
BackgroundMethodInfluentialThreats to federated learning A survey
Lyu, L., Yu, H., & Yang, Q. (2020) · arXiv · doi:10.48550/arxiv.2003.02133 · Cited by 221 · cited 2×
“Recent surveys focused on ML, including Federated Learning such as in [23, 24, 25, 26, 27, 28] partially illustrate or discuss the specific privacy and security issues in ML or FL systems.”
BackgroundDifferential Privacy Has Disparate Impact on Model Accuracy
Bagdasaryan, E., & Shmatikov, V. (2019) · arXiv · doi:10.48550/arxiv.1905.12101 · Cited by 219
On the Compatibility of Privacy and Fairness
Cummings, R., Gupta, V., Kimpara, D., & Morgenstern, J. (2019) · Adjunct Publication of the 27th Conference on User Modeling, Adaptation and Personalization · doi:10.1145/3314183.3323847 · Cited by 133
Differentially Private Meta-Learning
Li, J., Khodak, M., Caldas, S., & Talwalkar, A. (2019) · arXiv · doi:10.48550/arxiv.1909.05830 · Cited by 82 · cited 2×
“The differential privacy mechanism, in particular, has been widely utilized in contemporary privacy-preserving deep learning and privacy-preserving federated learning systems, such as those proposed in [36, 51, 52, 53, 57, 35].”
BackgroundMethodDynamic Decentralized Functional Encryption
Chotard, J., Dufour-Sans, E., Gay, R., Phan, D. H., & Pointcheval, D. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-56784-2_25 · Cited by 54
Cited by
The Algorithmic Foundations of Differential Privacy
Dwork, C., & Roth, A. (2014) · Foundations and Trends® in Theoretical Computer Science · doi:10.1561/0400000042 · Cited by 5,557
Reproducibility in machine‐learning‐based research: Overview, barriers, and drivers
Semmelrock, H., Ross‐Hellauer, T., Kopeinik, S., Theiler, D., Haberl, A., Thalmann, S., & Kowald, D. (2025) · AI Magazine · doi:10.1002/aaai.70002 · Cited by 118
Trusted AI in Multiagent Systems: An Overview of Privacy and Security for Distributed Learning
Ma, C., Li, J., Wei, K., Liu, B., Ding, M., Yuan, L., Han, Z., & Vincent Poor, H. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3306773 · Cited by 96
Reproducibility in Machine Learning-based Research: Overview, Barriers and Drivers
Semmelrock, H., Ross-Hellauer, T., Kopeinik, S., Theiler, D., Haberl, A., Thalmann, S., & Kowald, D. (2024) · arXiv · doi:10.48550/arxiv.2406.14325 · Cited by 88
Sharma, T., Stangl, A., Zhang, L., Tseng, Y.-Y., Xu, I., Findlater, L., Gurari, D., & Wang, Y. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580922 · Cited by 23
“Privacy is a major challenge with the datasets that support AI model development [12, 60, 86].”
BackgroundInfluential
Metadata
- Authors
- Xu, R., Baracaldo, N., & Joshi, J. B. D.
- Venue
- arXiv (Cornell University)
- Publisher
- Cornell University
- Published
- 2021-08-10
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2108.04417
- OpenAlex
- W3192298118
- Cited by
- 78
- References
- 50
- Type
- posted content
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.