CoChat
Sign Up
PreprintCited by 146Privacy-Preserving Technologies in DataAdversarial Robustness in Machine LearningCloud Data Security Solutions

Privacy in Deep Learning: A Survey

Open access
Mireshghallah, Fatemehsadat, Taram, Mohammadkazem, Vepakomma, Praneeth, Singh, Abhishek, Raskar, Ramesh, Esmaeilzadeh, Hadi·2020·arXiv
DOI 10.48550/arxiv.2004.12254arXiv 2004.12254
Open this paper in CoChat — free accountOpen PDF

Abstract

The ever-growing advances of deep learning in many areas including vision, recommendation systems, natural language processing, etc., have led to the adoption of Deep Neural Networks (DNNs) in production systems. The availability of large datasets and high computational power are the main contributors to these advances. The datasets are usually crowdsourced and may contain sensitive information. This poses serious privacy concerns as this data can be misused or leaked through various vulnerabilities. Even if the cloud provider and the communication link is trusted, there are still threats of inference attacks where an attacker could speculate properties of the data used for training, or find the underlying model architecture and parameters. In this survey, we review the privacy concerns brought by deep learning, and the mitigating techniques introduced to tackle these issues. We also show that there is a gap in the literature regarding test-time inference privacy, and propose possible future research directions.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

DataCite · 10.48550/arxiv.2004.12254

Cites

147 of 147 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Advanced PPML techniques such as FL, HE, SMPC, and DP are being thoroughly studied to protect user data and model integrity in the ML pipeline.

    “To circumvent these privacy risks, advanced privacy-preserving ML (PPML) techniques such as federated learning (FL), homomorphic encryption (HE), secure multi-party computations (SMPC), and differential privacy (DP) are being thoroughly studied to protect user data and model integrity in the ML pipeline [16, 97, 161].”

    From Privacy-preserving Machine Learning in Internet of Vehicle Applications: Fundamentals, Recent Advances, and Future Direction (2025) · cites this paper as [97]

    The survey reviews privacy-preserving mechanisms across the ML pipeline (Section 3, p. 5) and discusses DP, HE, and SMC for private training (Section 3.2, p. 7), as well as federated learning combined with SMC and DP (Section 4.1, p. 14). These passages support the claim about studying techniques to protect user data, but they do not establish that the listed techniques protect model integrity. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • In the inference phase, membership inference, model inversion, or model extraction attacks can undermine the privacy and integrity of the model and its underlying data.

    “During the training phase, attacks such as data poisoning, property inference, or Byzantine attacks, and in the inference phase, membership inference, model inversion, or model extraction attacks can undermine the privacy and integrity of both the model and its underlying data [97, 161].”

    From Privacy-preserving Machine Learning in Internet of Vehicle Applications: Fundamentals, Recent Advances, and Future Direction (2025) · cites this paper as [97]

    The Introduction (p. 1) describes membership inference, model inversion, and attacks that steal information about a deployed model; §§2.2.1–2.2.2 (pp. 3–4) explain how attacks using a trained model can expose training-data membership or sensitive attributes. These passages support privacy risks to the model and its data, but do not establish that the attacks undermine their integrity. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • During the training phase, attacks such as data poisoning, property inference, or Byzantine attacks can undermine the privacy and integrity of the model and its underlying data.

    “During the training phase, attacks such as data poisoning, property inference, or Byzantine attacks, and in the inference phase, membership inference, model inversion, or model extraction attacks can undermine the privacy and integrity of both the model and its underlying data [97, 161].”

    From Privacy-preserving Machine Learning in Internet of Vehicle Applications: Fundamentals, Recent Advances, and Future Direction (2025) · cites this paper as [97]

    Section 2.2.4 (p. 5) describes property inference as revealing sensitive patterns in training data, supporting that privacy concern. But the Introduction (p. 2) identifies data poisoning as a security vulnerability outside this privacy survey’s scope, and the copy does not substantiate Byzantine attacks. It also describes property inference from a target model, not specifically as an attack during training, so the full training-phase claim is not supported. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Privacy risks can arise during the training and inference phases of the ML pipeline in an IoV application, compromising confidentiality and integrity.

    “In the ML pipeline of an IoV application, privacy risks can arise during the training and inference phases to compromise the confidentiality and integrity of the system [16, 97, 161].”

    From Privacy-preserving Machine Learning in Internet of Vehicle Applications: Fundamentals, Recent Advances, and Future Direction (2025) · cites this paper as [97]

    The Introduction (pp. 1–2) describes privacy threats involving training data and inference queries, and discusses protections for both training and test-time inference. It does not address IoV applications or say that these privacy risks compromise system integrity; it explicitly places other security vulnerabilities outside the survey's scope. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Data poisoning attacks are a significant risk in IoV.

    “data poisoning attacks involve adversaries manipulating training data to compromise model integrity or insert backdoors, a significant risk in IoV where data quality is crucial for safety”

    From Privacy-preserving Machine Learning in Internet of Vehicle Applications: Fundamentals, Recent Advances, and Future Direction (2025) · cites this paper as [97]

    The Introduction (p. 2) identifies data poisoning as a security vulnerability that can be exploited in deep learning models. It does not describe manipulation of training data or insertion of backdoors, nor does it discuss IoV or safety-related data quality. Its separate mention of a backdoor in §2.1 (p. 2) concerns access to datasets, not data poisoning. (LLM, conf=0.87)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

5 of the 152 references couldn’t be matched to a known paper.

Research works printed but not linked: 17

  1. [29][29] S. A. Ossia, A. Taheri, A. S. Shamsabadi, K. Katevas, H. Haddadi, and H. R. Rabiee, 'Deep private-feature extraction,' IEEE Transactions on Knowledge and Data Engineering , vol. 32, pp. 54-66, 2018. -
  2. [31][31] F. Mireshghallah, M. Taram, A. Jalali, D. Tullsen, and H. Esmaeilzadeh, 'Shredder: Learning noise distributions to protect inference privacy,' in Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems , ASPLOS '20, (New York, NY, USA), Association for Computing Machinery, March 2020. -
  3. [45][45] Z. Yang, E.-C. Chang, and Z. Liang, 'Adversarial neural network inversion via auxiliary knowledge alignment,' ArXiv , vol. abs/1902.08552, 2019. - · arXiv
  4. [56][56] X. Wu, M. Fredrikson, S. Jha, and J. F. Naughton, 'A methodology for formalizing modelinversion attacks,' in 2016 IEEE 29th Computer Security Foundations Symposium (CSF) , pp. 355-370, June 2016. -
  5. [70][70] P. Kairouz, S. Oh, and P. Viswanath, 'Extremal mechanisms for local differential privacy,' CoRR , vol. abs/1407.1338, 2014. - · arXiv
  6. [71][71] Ú. Erlingsson, A. Korolova, and V. Pihur, 'RAPPOR: randomized aggregatable privacypreserving ordinal response,' CoRR , vol. abs/1407.6981, 2014. - · arXiv
  7. [76][76] D. Kifer and A. Machanavajjhala, 'Pufferfish: A framework for mathematical privacy definitions,' ACM Trans. Database Syst. , vol. 39, Jan. 2014. -
  8. [78][78] M. E. Andrés, N. E. Bordenabe, K. Chatzikokolakis, and C. Palamidessi, 'Geoindistinguishability: Differential privacy for location-based systems,' in Proceedings of the 2013 ACM SIGSAC Conference on Computer and Communications Security , CCS '13, (New York, NY, USA), p. 901-914, Association for Computing Machinery, 2013. -
  9. [79][79] S. Goldwasser and S. Micali, 'Probabilistic encryption,' Journal of computer and system sciences , vol. 28, no. 2, pp. 270-299, 1984. -
  10. [110][110] P. Mohassel and P. Rindal, 'Aby3: A mixed protocol framework for machine learning,' in Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security , CCS '18, (New York, NY, USA), p. 35-52, Association for Computing Machinery, 2018. -
  11. [123][123] E. Bagdasaryan, O. Poursaeed, and V. Shmatikov, 'Differential privacy has disparate impact on model accuracy,' in Advances in Neural Information Processing Systems , pp. 15479-15488, 2019. -
  12. [125][125] B. Jayaraman and D. Evans, 'Evaluating differentially private machine learning in practice,' in 28th { USENIX } Security Symposium ( { USENIX } Security 19) , pp. 1895-1912, 2019. -
  13. [128][128] J. Wang, J. Zhang, W. Bao, X. Zhu, B. Cao, and P. S. Yu, 'Not just privacy,' Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining , Jul 2018. -
  14. [136][136] M. S. Riazi, C. Weinert, O. Tkachenko, E. M. Songhori, T. Schneider, and F. Koushanfar, 'Chameleon: A hybrid secure computation framework for machine learning applications,' CoRR , vol. abs/1801.03239, 2018. - · arXiv
  15. [140][140] C. Juvekar, V. Vaikuntanathan, and A. Chandrakasan, 'Gazelle: A low latency framework for secure neural network inference,' in Proceedings of the 27th USENIX Conference on Security Symposium , SEC'18, (USA), p. 1651-1668, USENIX Association, 2018. -
  16. [153][153] F. Mo, A. S. Shamsabadi, K. Katevas, S. Demetriou, I. Leontiadis, A. Cavallaro, and H. Haddadi, 'Darknetz: Towards model privacy at the edge using trusted execution environments,' ArXiv , vol. abs/2004.05703, 2020. - · arXiv
  17. [160][160] C. Canella, D. Genkin, L. Giner, D. Gruss, M. Lipp, M. Minkin, D. Moghimi, F. Piessens, M. Schwarz, B. Sunar, J. Van Bulck, and Y. Yarom, 'Fallout: Leaking data on meltdownresistant cpus,' in Proceedings of the ACM SIGSAC Conference on Computer and Communications Security (CCS) , ACM, 2019. -

Cited by

Sorted by citations

Related

Authors
Mireshghallah, F., Taram, M., Vepakomma, P., Singh, A., Raskar, R., & Esmaeilzadeh, H.
ORCID
Venue
arXiv
Publisher
arXiv
Published
2020
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2004.12254
arXiv
2004.12254
OpenAlex
W3018763859
Cited by
106
References
152
Citations by year
2026: 1 · 2025: 6 · 2024: 26 · 2023: 19 · 2022: 22 · 2021: 26 · 2020: 6
Type
posted content
Language
en
Open access
Licence
arxiv.org/licenses/nonexclusive-distrib/1.0/
Locations
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Information Systems › Cloud Data Security Solutions
Keywords
inference attacks, deep neural network, sensitive data, data leakage, privacy-preserving techniques, crowdsourced data

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review