The DOI matches its Crossref record; 103 of the 111 references it cites were found in a registry or an index.
Poisoning Web-Scale Training Datasets is Practical
Open accessAbstract
Deep learning models are often trained on distributed, web-scale datasets crawled from the internet. In this paper, we introduce two new dataset poisoning attacks that intentionally introduce malicious examples to a model’s performance. Our attacks are immediately practical and could, today, poison 10 popular datasets. Our first attack, split-view poisoning, exploits the mutable nature of internet content to ensure a dataset annotator’s initial view of the dataset differs from the view downloaded by subsequent clients. By exploiting specific invalid trust assumptions, we show how we could have poisoned 0.01% of the LAION-400M or COYO-700M datasets for just $60 USD. Our second attack, frontrunning poisoning, targets web-scale datasets that periodically snapshot crowd-sourced content—such as Wikipedia—where an attacker only needs a time-limited window to inject malicious examples. In light of both attacks, we notify the maintainers of each affected dataset and recommended several low-overhead defenses.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“NIST SP 800-204D [13] provides guidance for integrating software supply chain security controls into DevSecOps CI/CD pipelines, but it does not define AI-specific provenance structures or model assurance mechanisms.”
From AI Supply Chain Security: MBOM-PQC Provenance, PQC Attestation, and a Maturity Model for Quantum-Resistant Assurance (2026) · cites this paper as [13]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,889
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 1×
“Deep learning is most effective when applied to large datasets [10, 30].”
BackgroundBuilding a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015) · Procedings of the British Machine Vision Conference 2015 · doi:10.5244/c.29.41 · Cited by 5,479
Pointer Sentinel Mixture ModelsPreprint
Merity, S., Xiong, C., Bradbury, J., & Socher, R. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1609.07843 · Cited by 5,058
VGGFace2: A Dataset for Recognising Faces across Pose and Age
Cao, Q., Shen, L., Xie, W., Parkhi, O. M., & Zisserman, A. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00020 · Cited by 3,032
Research works printed but not linked: 3
- 10.18653/v1/n19–1423 · DOI · Paper page
- Guo, Wiki-40B: Multilingual language model dataset, 2440
- Radford, 2019, Language models are unsupervised multitask learners, OpenAI blog, 1, 9
Cited by
Sorted by citations
Items 101–143 of 143
Privatar: A Privacy-Preserving Framework for LLM Agents
Jian, R., Zhu, Z., Shang, Y., Shen, Y., & Liu, K. (2025) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw69685.2025.00011
Tiny Poison, Big Effect: Comprehensive Multi-Vector Poisoning Assessment of Small Language Models
Sharma, N., & Singh, U. (2026) · International Conference on Computing, Communication, Control and Cyber-Physical Systems (I5CPS) · doi:10.1109/i5cps67958.2026.11452345
PaperAsk: A Benchmark for Reliability Evaluation of LLMs in Paper Search and Reading
Wu, Y., Liu, X., Feng, Y., Ding, J., & Ma, X. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792597 · Cited by 2
CACTEE: Confidential Asset Certification using Trusted Execution Environments
Akkuş, İ. E., & Rimac, I. (2026) · Workshop on System Software for Trusted Execution (SysTEX) · doi:10.1145/3805690.3805734
Revisiting Backdoor Threat in Federated Instruction Tuning From a Signal Aggregation Perspective
Zhao, H., Hu, J., & Liu, G. H. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11462088
BadLLM-TG: A Backdoor Defender Powered by LLM Trigger Generator
Zhang, R., Gao, H., Jian, S., Tan, Y., & Zhou, H. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463345
Covert Knowledge Poisoning Attacks in Retrieval-Augmented Code Generation
Zhang, Y., Chen, M., He, X., Cong, T., Xu, K., & Li, Q. (2026) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2026.3686845
Explainability-Guided Untargeted Attacks on Knowledge Graph Embedding
Lin, Y., Yu, H., Liang, K., Wang, M., Yang, L., & Liu, X. Y. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3689651 · Cited by 1
Security as an Ethical Imperative in AI Systems
Andrade, R., Ayala, C., & Morillo, P. (2026) · Synthesis lectures on computer science · doi:10.1007/978-3-032-25789-5_5
Machine Learning and Hardware Security: The Role of AI for Hardware in the Security Era
Deng, S., Xu, Y., Li, M., Jin, Y., & Wang, J. (2026) · Springer eBooks · doi:10.1007/978-3-032-17948-7_12
Mitigating Security Risks in Large Language Models: A Full Lifecycle Perspective
Wang, Y. K., Bai, Z., Huo, J., Wang, B., Liu, J., Cao, H., Meng, F., Yang, X., & Gao, Y. (2026) · Machine Learning · doi:10.1007/s10994-025-06930-x
From AI-generated content to agentic action: Security and safety threats in generative AI
Zhang, Z., Li, Q., Cao, J., Liu, L., & Ni, J. (2026) · Journal of Information and Intelligence · doi:10.1016/j.jiixd.2026.05.002 · Cited by 1
Side-Channel Privacy Attacks on Energy-Efficient Mobile Agents: A Membership Inference Perspective
Ehsan, I., Niu, J., & Zhang, Y. (2026) · International Conference on Mobile Systems, Applications, and Services · doi:10.1145/3812836.3814743
Zheng, Z., Choi, Y., Huang, C., Cao, H., Tian, S., & Pei, T. (2026) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2026.3699202
Xu, Z., Gardiner, J., & Belguith, S. (2026) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3779208.3785271
A Formal Security Framework for Model Context Protocol-Based Tool Access inAgentic AI SystemsPreprint
Alam, M. Z. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.6891368
Infrastructure as Compromise: Abusing Residual Trust in Infrastructure as Code Tools
Yang, R., Chaiwut, N., & Nikiforakis, N. (2026) · ACM Conference on Data and Application Security and Privacy (CODASPY) · doi:10.1145/3800506.3803500
Berini, A. D. E., Jamil, N., Lakas, A., Doshmangir, L., Ferrag, M. A., & Lam, K. (2026) · ACM International Workshop on Security and Privacy Analytics (IWSPA) · doi:10.1145/3806007.3810962
Liu, S., & Yang, X. (2026) · Communications in computer and information science · doi:10.1007/978-981-95-9884-7_3
Evaluating Design Decisions and Bias Resistance for Passive DNS-Based Domain Rankings
Pochat, V. L., Fernandez, S., Tajalizadehkhoob, S., Desmet, L., Duda, A., Joosen, W., & Korczyński, M. (2026) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2026.3705306
Liu, C., Qian, L., Jadhav, C. D., & Zhao, J. Z. (2026) · IEEE Conference on Computer Communications (INFOCOM) · doi:10.1109/infocom59046.2026.11571647
Detecting Sentiment Steering Attacks on RAG-enabled Large Language Models
Mohan, A., Mahadik, S. S., Mukherjee, M., Pawar, P. M., Muthalagu, R., & Lloret, J. (2026) · International Conference on Communications · doi:10.1109/icc59461.2026.11588193
Fairness in vision-language text-to-image retrieval: Natural biases and adversarial threats
Lazzaro, D., Ciná, A. E., Roli, F., Anguita, D., Biggio, B., Laurita, G., & Oneto, L. (2026) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2026.111385
Affective Computing in the Age of GenAI: Balancing Risks and Rewards
Latif, S., Shoukat, M., Pervez, F., Shamshad, F., Usama, M., Sandhu, M., & Schuller, B. W. (2026) · IEEE Computational Intelligence Magazine · doi:10.1109/mci.2026.3678879
!Imperio, smolVLA: The Implications of Data Poisoning on Open Source Robotics
Bühler, S., & Schutera, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-32335-4_2
MC-PoisonVerif: Model Checking-Driven Poisoning Attack Formal Verification Algorithm
Lv, C., Song, Y., Li, Y., Zhou, W., Chenyu, G., & Zhang, X. (2026) · Mathematics · doi:10.3390/math14142617
Kakebayashi, R., & Mori, T. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31441-3_7
Reproducibility Is the New Copyleft: Defining AGI-Oriented Reproducible Builds
Hatta, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-33010-9_25
RefineRAG: Word-Level Poisoning Attacks via Retriever-Guided Text Refinement
Wang, Z., Wang, G., & Wang, K. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-31583-0_35
DidData: A Trust-Aware Protocol for Sovereign AI Data Supply Chains
Liu, Y., Cheng, T., Yin, Z., & Liu, Y. (2026) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac69091.2026.00105
Data security in large language models: risks, defense, and directions
Chen, K., Zhou, X., Yu, Y., Lin, Y., Chen, H., Cai, C., & Shen, L. (2026) · Journal of King Saud University - Computer and Information Sciences · doi:10.1007/s44443-026-01200-9 · Cited by 2
Gao, Q., Gao, Y., Kang, H., Jiang, L., Zhang, X., & Li, Z. (2026) · International Conference on Electronics Technology (ICET) · doi:10.1109/icet69987.2026.11659295
Counterfactual incremental defense against poisoning attacks
Wang, H., & Kandah, F. I. (2026) · Knowledge-Based Systems · doi:10.1016/j.knosys.2026.116930
Training on Clean Data but Getting Backdoored Models! A Poisoning Attack on Code Encoders
Xiao, Y., Liu, X., Yang, Z., Bo, L., & Sun, X. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3773236
SoK: On the Survivability of Backdoor Attacks on Unconstrained Face Recognition Systems
Roux, Q. L., Teglia, Y., Furon, T., Loubet-Moundi, P., & Bourbao, E. (2026) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml68715.2026.00068 · Cited by 1
Strategic Oversight for AI Safety: A Stackelberg Security Games Perspective
Kim, C. W., Choo, D., Neoh, T. Y., & Tambe, M. (2026) · IEEE Intelligent Systems · doi:10.1109/mis.2026.3725453
Ge, H., Shi, Z., & Nie, Y. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-5712-6_34
Perturb and Recover: Fine-Tuning for Effective Backdoor Removal from CLIP
Singh, N. D., Croce, F., & Hein, M. (2026) · doi:10.1109/cvprf71909.2026.00592
Ajibola, T., & Okpiyalele, K. (2026) · International Journal of Scientific Research in Artificial Intelligence and Machine Learning · doi:10.32628/ijsraiml2624121
RAGtrap: Source Revocation and Indexed Provenance Lookup for Poisoned RAG Corpora
Kapelinski, C., & Kreutz, D. (2026) · Simpósio Brasileiro de Cibersegurança (SBSeg) · doi:10.5753/sbseg_estendido.2026.29850
Aladraj, R. (2026) · Bangladesh Journal of Multidisciplinary Scientific Research · doi:10.46281/bjmsr.v12i1.2993
Sombatpoonsiri, J., Bradshaw, S., & Mishra, K. (2026) · Asian Politics & Policy · doi:10.1111/aspp.70116
Bhange, M., Alfaghih, M., Creager, E., Sansfaçon, A. P., & Aïvodji, U. (2026) · AI and Ethics · doi:10.1007/s43681-026-01445-0
Metadata
- Authors
- Carlini, N., Jagielski, M., Choquette-Choo, C. A., Paleka, D., Pearce, W., Anderson, H., Terzis, A., Thomas, K., & Tramèr, F.
- Affiliations
- Google DeepMind (United Kingdom) (GB)
- ETH Zurich (CH)
- Nvidia (United Kingdom) (GB)
- Robust Intelligence
- Google (United States) (US)
- Venue
- 2024 IEEE Symposium on Security and Privacy (SP)
- Publisher
- IEEE
- Published
- 2024-05-19
- Pages
- 407-425
- ISSN
- 1081-6011 (ISSN-L)
- OpenAlex
- W4402264526
- Cited by
- 138
- References
- 111
- Citations by year
- 2026: 53 · 2025: 44 · 2024: 29
- FWCI
- 37.18
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Symposium on Security and Privacy (S&P) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Keywords
- deep learning security, Wikipedia, crowdsourced data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
