Cited by 362Adversarial Robustness in Machine LearningPrivacy-Preserving Technologies in DataSecurity and Verification in Computing

Dataset Security for Machine Learning: Data Poisoning, Backdoor Attacks, and Defenses

Open access
Goldblum, Micah, Tsipras, Dimitris, Xie, Chulin, Chen, Xinyun, Schwarzschild, Avi, Song, Dawn, Madry, Aleksander, Li, Bo +1 more·2023·IEEE Transactions on Pattern Analysis and Machine Intelligence

Abstract

As machine learning systems grow in scale, so do their training data requirements, forcing practitioners to automate and outsource the curation of training data in order to achieve state-of-the-art performance. The absence of trustworthy human supervision over the data collection process exposes organizations to security vulnerabilities; training data can be manipulated to control and degrade the downstream behaviors of learned models. The goal of this work is to systematically categorize and discuss a wide range of dataset vulnerabilities and exploits, approaches for defending against these threats, and an array of open problems in this space.

Integrity signals

Registry

Crossref · 10.1109/tpami.2022.3162397

Cites

223 of 223 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The DoD CDAO Responsible AI Toolkit focuses on ethical and operational considerations rather than cryptographic integrity.

    “The DoD CDAO Responsible AI Toolkit [14] focuses on ethical and operational considerations rather than cryptographic integrity.”
  • By integrating PQC-safe signatures, hybrid modes, provenance modeling, and continuous attestation, the framework addresses critical gaps in current AI assurance practices that existing governance frameworks-including the NIST AI RMF [5], NIST SSDF [6], and DoD CDAO Responsible AI Toolkit [14]-do not fill.

    “By integrating PQC-safe signatures, hybrid modes, provenance modeling, and continuous attestation, it addresses critical gaps in current AI assurance practices that existing governance frameworks-including the NIST AI RMF [5], NIST SSDF [6], and DoD CDAO Responsible AI Toolkit [14]-do not fill.”
  • To carry out model aggregation phase attacks, attackers need complete knowledge about the target WFL framework, including the number of participants, local training algorithm, aggregation protocol, and global model convergence process.

    “To carry out these attacks, attackers need to have complete knowledge about the target WFL framework, including the number of participants, local training algorithm, aggregation protocol, and global model convergence process [51], [243].”

Cites

Flagged first, then by citations

10 of the 233 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Goldblum, M., Tsipras, D., Xie, C., Chen, X., Schwarzschild, A., Song, D., Madry, A., Li, B., & Goldstein, T.
Affiliations
  • University of Maryland, College Park (US)
  • Massachusetts Institute of Technology (US)
  • University of Illinois Urbana-Champaign (US)
  • University of California, Berkeley (US)
  • Massachusetts Institute of Technology, Cambridge, MA, USA
Venue
IEEE Transactions on Pattern Analysis and Machine Intelligence
Published
2023-02-01
Volume
45
Issue
2
Pages
1563-1580
ISSN
  • 0162-8828 (ISSN-L)
  • 2160-9292
  • 1939-3539
OpenAlex
W3117572899
Cited by
362
References
233
Citations by year
2026: 76 · 2025: 91 · 2024: 78 · 2023: 66 · 2022: 29 · 2021: 21 · 2020: 1
FWCI
21.63
Type
journal article
Language
en
Open access
Locations
  • IEEE Transactions on Pattern Analysis and Machine Intelligence (publishedVersion)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University) (cc-by)
Funders
  • Defense Advanced Research Projects Agency
  • QED4RML
  • D3M programs
  • National Science Foundation (CCF-1910100, TWC-1409915, CCF-1553428, CNS-1815221, DMS-1912866)
  • Amazon Research Award Program
  • Berkeley DeepDrive
  • Facebook PhD Fellowship
  • ONR MURI Program
  • Alfred P. Sloan Foundation
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
  • Physical Sciences › Computer Science › Artificial Intelligence › Security and Verification in Computing
Keywords
data poisoning, backdoor attacks, machine learning security, attack defense

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review