CoChat
Sign Up
Cited by 1,760Privacy-Preserving Technologies in DataAdversarial Robustness in Machine LearningPrivacy, Security, and Data Protection

Comprehensive Privacy Analysis of Deep Learning: Passive and Active White-box Inference Attacks against Centralized and Federated Learning

Open access
Nasr, Milad, Shokri, Reza, Houmansadr, Amir·2019·2019 IEEE Symposium on Security and Privacy (SP)
DOI 10.1109/sp.2019.00065
Open this paper in CoChat — free accountOpen PDF

Abstract

Deep neural networks are susceptible to various inference attacks as they remember information about their training data. We design white-box inference attacks to perform a comprehensive privacy analysis of deep learning models. We measure the privacy leakage through parameters of fully trained models as well as the parameter updates of models during training. We design inference algorithms for both centralized and federated learning, with respect to passive and active inference attackers, and assuming different adversary prior knowledge. We evaluate our novel white-box membership inference attacks against deep learning algorithms to trace their training data records. We show that a straightforward extension of the known black-box attacks to the white-box setting (through analyzing the outputs of activation functions) is ineffective. We therefore design new algorithms tailored to the white-box setting by exploiting the privacy vulnerabilities of the stochastic gradient descent algorithm, which is the algorithm used to train deep neural networks. We investigate the reasons why deep learning models may leak information about their training data. We then show that even well-generalized models are significantly susceptible to white-box membership inference attacks, by analyzing state-of-the-art pre-trained and publicly available models for the CIFAR dataset. We also show how adversarial participants, in the federated learning setting, can successfully run active membership inference attacks against other participants, even when the global model achieves high prediction accuracies.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/sp.2019.00065

Cites

44 of 44 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • FL gradients, though designed to protect raw data, remain susceptible to inference attacks such as membership inference and model inversion.

    “Moreover, FL gradients, though designed to protect raw data, remain susceptible to inference attacks such as membership inference and model inversion [101].”

    From Privacy-preserving Machine Learning in Internet of Vehicle Applications: Fundamentals, Recent Advances, and Future Direction (2025) · cites this paper as [101]

    The Federated Learning Settings: Passive Inference Attacks section reports membership inference against participants using their parameter updates, and the Impact of gradients section reports that gradients leak membership information. The cited paper does not show model inversion attacks against federated-learning gradients; “model inversion” appears in a reference title, not as a finding of this study. The Federated Learning section describes parties sending parameter updates rather than raw training data. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

  • Membership inference is the most well-established privacy attack and arguably constitutes a bigger privacy threat than class-level attacks, such as the attacks in [7, 59] and model inversion [18, 19].

    “We emphasize that membership inference is the most wellestablished privacy attack and arguably constitutes a bigger privacy threat [13,28,30,37,46,48,60,66,67,75] than classlevel attacks, such as the attacks in [7, 59] and model inversion [18, 19].”

    From When Machine Unlearning Jeopardizes Privacy (2021) · cites this paper as [48]

    The Introduction (page not available) treats membership inference as a measure of privacy leakage, and the Membership Inference Attacks section surveys multiple studies, supporting its characterization as an established attack. The paper does not say that it poses a bigger privacy threat than class-level attacks or model inversion. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • Different security and privacy threats remain for federated learning, including attempted extraction of training data information from models, model theft, and submission of poison model updates.

    “These threats include attempted extraction of training data information from intermediate and final models, model theft, and submission of poison model updates with the goal of introducing unwanted model behavior (including incentivizing the model to memorize more information about the training data in support of subsequent extraction, i.e., leakage).”

    From Federated learning enables big data for rare cancer boundary detection (2022) · cites this paper as [56]

  • An adversary can deploy data attacks to communication channels established by clients and server to steal updated information, which can interrupt model computation at the global server due to insufficient local updates to build a global model.

    “Indeed, an adversary can deploy data attacks to the communication channels established by the clients and the server to steal the updated information which can interrupt the model computation at the global server due to the insufficient local updates to build a global model.”

    From Federated learning for smart healthcare: A survey (2023) · cites this paper as [35]

  • The exchange of local model updates to the server may be risky due to external threats.

    “The exchange of local model updates to the server may be risky due to external threats [35].”

    From Federated learning for smart healthcare: A survey (2023) · cites this paper as [35]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. I. Goodfellow, Y. Bengio, and A. Courville, Deep learning, 2016, vol. 1.

Cited by

Sorted by citations

Related

Authors
Nasr, M., Shokri, R., & Houmansadr, A.
Affiliations
  • University of Massachusetts Amherst (US)
  • National University of Singapore (SG)
ORCID
Venue
2019 IEEE Symposium on Security and Privacy (SP)
Publisher
IEEE
Published
2019-05
Pages
739-753
ISSN
1081-6011 (ISSN-L)
DOI
10.1109/sp.2019.00065
OpenAlex
W2930926105
Cited by
1,760
References
44
Citations by year
2026: 189 · 2025: 303 · 2024: 343 · 2023: 316 · 2022: 260 · 2021: 215 · 2020: 110 · 2019: 23 · 2018: 1
FWCI
56.90
Type
conference paper
Language
en
Open access
Licence
Locations
  • IEEE Symposium on Security and Privacy (S&P) (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • National University of Singapore (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Science Foundation (1525642)
  • Nvidia
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Social Sciences › Sociology and Political Science › Privacy, Security, and Data Protection
Keywords
membership inference attacks, deep learning privacy, federated learning, centralized learning, privacy leakage, stochastic gradient descent, training data memorization, CIFAR

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review