Poisoning Web-Scale Training Datasets is Practical
Open accessAbstract
Deep learning models are often trained on distributed, web-scale datasets crawled from the internet. In this paper, we introduce two new dataset poisoning attacks that intentionally introduce malicious examples to a model’s performance. Our attacks are immediately practical and could, today, poison 10 popular datasets. Our first attack, split-view poisoning, exploits the mutable nature of internet content to ensure a dataset annotator’s initial view of the dataset differs from the view downloaded by subsequent clients. By exploiting specific invalid trust assumptions, we show how we could have poisoned 0.01% of the LAION-400M or COYO-700M datasets for just $60 USD. Our second attack, frontrunning poisoning, targets web-scale datasets that periodically snapshot crowd-sourced content—such as Wikipedia—where an attacker only needs a time-limited window to inject malicious examples. In light of both attacks, we notify the maintainers of each affected dataset and recommended several low-overhead defenses.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
NIST SP 800-204D provides guidance for integrating software supply chain security controls into DevSecOps CI/CD pipelines, but it does not define AI-specific provenance structures or model assurance mechanisms.
“NIST SP 800-204D [13] provides guidance for integrating software supply chain security controls into DevSecOps CI/CD pipelines, but it does not define AI-specific provenance structures or model assurance mechanisms.”
From AI Supply Chain Security: MBOM-PQC Provenance, PQC Attestation, and a Maturity Model for Quantum-Resistant Assurance (2026) · cites this paper as [13]
The cited paper’s Introduction and Section 6.5 discuss defenses and transparency for web-scale training datasets, not NIST SP 800-204D. Searches for the standard, DevSecOps CI/CD guidance, AI-specific provenance structures, and model assurance mechanisms found no passage addressing what the standard provides or omits. (LLM, conf=0.78)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · see citing works
Learning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,890 · see citing works
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
Learning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · see citing works
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · see citing works
Building a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405 · see citing works
Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015) · Procedings of the British Machine Vision Conference 2015 · doi:10.5244/c.29.41 · Cited by 5,479 · see citing works
Pointer Sentinel Mixture ModelsPreprint
Merity, S., Xiong, C., Bradbury, J., & Socher, R. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1609.07843 · Cited by 5,051 · see citing works
VGGFace2: A Dataset for Recognising Faces across Pose and Age
Cao, Q., Shen, L., Xie, W., Parkhi, O. M., & Zisserman, A. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00020 · Cited by 3,032 · see citing works
PaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · see citing works
Understanding deep learning (still) requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2021) · Communications of the ACM · doi:10.1145/3446776 · Cited by 2,400 · see citing works
Robust Speech Recognition via Large-Scale Weak Supervision
Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., & Sutskever, I. (2022) · arXiv · doi:10.48550/arxiv.2212.04356 · Cited by 1,953 · see citing works
80 Million Tiny Images: A Large Data Set for Nonparametric Object and Scene Recognition
Torralba, A., Fergus, R., & Freeman, W. T. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.128 · Cited by 1,921 · see citing works
Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning
Sharma, P., Ding, N., Goodman, S., & Soricut, R. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1238 · Cited by 1,879 · see citing works
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · Cited by 1,749 · cited 1× · see citing works
“Deep learning is most effective when applied to large datasets [10, 30].”
BackgroundThomée, B., Shamma, D. A., Friedland, G. H., Elizalde, B., Ni, K. S., Poland, D., Borth, D., & Li, L.-J. (2016) · Communications of the ACM · doi:10.1145/2812802 · Cited by 1,632 · see citing works
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · see citing works
Attribute and simile classifiers for face verification
Kumar, N., Berg, A. C., Belhumeur, P. N., & Nayar, S. K. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459250 · Cited by 1,383 · see citing works
Poisoning Attacks against Support Vector Machines
Biggio, B., Nelson, B., & Laskov, P. (2012) · arXiv · doi:10.48550/arxiv.1206.6389 · Cited by 1,331 · see citing works
BadNets: Evaluating Backdooring Attacks on Deep Neural Networks
Gu, T., Liu, K., Dolan-Gavitt, B., & Garg, S. (2019) · IEEE Access · doi:10.1109/access.2019.2909068 · Cited by 1,238 · see citing works
Bitcoin and Cryptocurrency Technologies : A Comprehensive Introduction
Bolt, W. (2017) · Data Archiving and Networked Services (DANS) · Cited by 1,064 · see citing works
BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
Gu, T., Dolan-Gavitt, B., & Garg, S. (2017) · arXiv · doi:10.48550/arxiv.1708.06733 · Cited by 1,061 · see citing works
Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
Chen, X., Liu, C., Li, B., Lu, K., & Song, D. (2017) · arXiv · doi:10.48550/arxiv.1712.05526 · Cited by 1,039 · see citing works
LAION-5B: An open large-scale dataset for training next generation image-text models
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.08402 · Cited by 1,022 · see citing works
Training Compute-Optimal Large Language Models
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. D. L., Hendricks, L. A., Welbl, J., Clark, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2176 · Cited by 932 · see citing works
The MegaFace Benchmark: 1 Million Faces for Recognition at Scale
Kemelmacher-Shlizerman, I., Seitz, S. M., Miller, D., & Brossard, E. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.527 · Cited by 912 · see citing works
Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning
Jagielski, M., Oprea, A., Biggio, B., Liu, C., Nita-Rotaru, C., & Li, B. (2018) · 2018 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2018.00057 · Cited by 861 · see citing works
Self-citationWikiQA: A Challenge Dataset for Open-Domain Question Answering
Yang, Y., Yih, W.-T., & Meek, C. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1237 · Cited by 829 · see citing works
Gao, Y., Xu, C., Wang, D., Chen, S., Ranasinghe, D. C., & Nepal, . (2019) · Annual Computer Security Applications Conference · doi:10.1145/3359789.3359790 · Cited by 719 · see citing works
A data-driven approach to cleaning large face datasets
Ng, H., & WINKLER, S. S. (2014) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2014.7025068 · Cited by 668 · see citing works
Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural NetworksPreprint
Shafahi, A., Huang, W., Najibi, M., Suciu, O., Studer, C., Dumitraş, T., & Goldstein, T. (2018) · arXiv (Cornell University) · doi:10.3929/ethz-b-000460348 · Cited by 626 · see citing works
Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts
Changpinyo, S., Sharma, P. K., Ding, N., & Soricut, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00356 · Cited by 626 · see citing works
FLAVA: A Foundational Language And Vision Alignment Model
Singh, A., Hu, R., Goswami, V., Couairon, G., Galuba, W., Rohrbach, M., & Kiela, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01519 · Cited by 597 · see citing works
Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimization
Muñoz-González, L., Biggio, B., Demontis, A., Paudice, A., Wongrassamee, V., Lupu, E. C., & Roli, F. (2017) · Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security · doi:10.1145/3128572.3140451 · Cited by 549 · see citing works
Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks
Liu, Y., Ma, X., Bailey, J., & Lu, F. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58607-2_11 · Cited by 531 · see citing works
Spectral Signatures in Backdoor AttacksPreprint
Tran, B., Li, J. Z., & Mądry, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.00636 · Cited by 506 · see citing works
The Pile: An 800GB Dataset of Diverse Text for Language Modeling
Gao, L., Biderman, S., Black, S., Golding, L., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., Nabeshima, N., et al. (2021) · arXiv · doi:10.48550/arxiv.2101.00027 · Cited by 492 · see citing works
Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
Chen, B., Carvalho, W., Baracaldo, N., Ludwig, H., Edwards, B., Lee, T., Molloy, I., & Srivastava, B. (2018) · arXiv · doi:10.48550/arxiv.1811.03728 · Cited by 485 · see citing works
Invisible Backdoor Attack with Sample-Specific Triggers
Li, Y., Li, Y., Wu, B., Li, L., He, R., & Lyu, S. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01615 · Cited by 472 · see citing works
Vggsound: A Large-Scale Audio-Visual Dataset
Chen, H., Xie, W., Vedaldi, A., & Zisserman, A. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053174 · Cited by 452 · see citing works
ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation
Liu, Y., Lee, W., Tao, G., Ma, S., Aafer, Y., & Zhang, X. (2019) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3319535.3363216 · Cited by 419 · see citing works
Attack of the tails: Yes, you really can backdoor federated learning
Wang, H., Sreenivasan, K., Rajput, S., Vishwakarma, H., Agarwal, S., Sohn, J.-Y., Lee, K., & Papailiopoulos, D. (2020) · arXiv · doi:10.48550/arxiv.2007.05084 · Cited by 406 · see citing works
Deep Learning is Robust to Massive Label Noise
Rolnick, D., Veit, A., Belongie, S., & Shavit, N. (2017) · arXiv · doi:10.48550/arxiv.1705.10694 · Cited by 402 · see citing works
Latent Backdoor Attacks on Deep Neural Networks
Yao, Y., Li, H., Zheng, H.-T., & Zhao, B. Y. (2019) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3319535.3354209 · Cited by 370 · see citing works
A New Backdoor Attack in CNNS by Training Set Corruption Without Label Poisoning
Barni, M., Kallas, K., & Tondi, B. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8802997 · Cited by 360 · see citing works
LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs
Schuhmann, C., Vencu, R., Beaumont, R., Kaczmarczyk, R., Mullis, C., Katta, A., Coombes, T., Jitsev, J., & Komatsuzaki, A. (2021) · arXiv · doi:10.48550/arxiv.2111.02114 · Cited by 358 · see citing works
Beyond vandalism: Wikipedia trolls
Shachaf, P., & Hara, N. (2010) · Journal of Information Science · doi:10.1177/0165551510365390 · Cited by 335 · see citing works
Support Vector Machines Under Adversarial Label Noise
Biggio, B., Nelson, B. A., & Laskov, P. (2011) · UNICA IRIS Institutional Research Information System (University of Cagliari) · Cited by 319 · see citing works
Is feature selection secure against training data poisoning?Preprint
Huang, X. X., Biggio, B., Gavin Brown, B., Fumera, G., Eckert, C. M., & Roli, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.07933 · Cited by 314 · see citing works
WaNet -- Imperceptible Warping-based Backdoor AttackPreprint
Nguyen, A., & Tran, A. N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.10369 · Cited by 296 · see citing works
Nikiforakis, N., Invernizzi, L., Kapravelos, A., Acker, S. V., Joosen, W., Kruegel, C., Piessens, F., & Vigna, G. (2012) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/2382196.2382274 · Cited by 279 · see citing works
Detecting AI Trojans Using Meta Neural Analysis
Xu, X., Wang, Q., Li, H., Borisov, N., Gunter, C. A., & Li, B. (2021) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp40001.2021.00034 · Cited by 278 · see citing works
Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems
Doan, B. G., Abbasnejad, E., & Ranasinghe, D. C. (2020) · Annual Computer Security Applications Conference · doi:10.1145/3427228.3427264 · Cited by 263 · see citing works
UMDFaces: An annotated face dataset for training deep networks
Bansal, A., Nanduri, A., Castillo, C. D., Ranjan, R., & Chellappa, R. (2017) · IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/btas.2017.8272731 · Cited by 246 · see citing works
Information quality work organization in wikipedia
Stvilia, B., Twidale, M. B., Smith, L. C., & Gasser, L. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20813 · Cited by 233 · see citing works
Dynamic Backdoor Attacks Against Machine Learning Models
Salem, A., Wen, R., Michael Backes, A., Ma, S., & Zhang, Y. (2022) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp53844.2022.00049 · Cited by 227 · see citing works
Input-Aware Dynamic Backdoor AttackPreprint
Nguyen, A., & Tran, A. T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.08138 · Cited by 221 · see citing works
DeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentation
Qiu, H., Zeng, Y., Guo, S., Zhang, T., Meikang Qiu, A., & Thuraisingham, B. M. (2021) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3433210.3453108 · Cited by 196 · see citing works
Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger
Qi, F., Li, M., Chen, Y., Zhang, Z., Liu, Z., Wang, Y., & Sun, M. (2021) · doi:10.18653/v1/2021.acl-long.37 · Cited by 190 · see citing works
Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation
Zhong, H., Liao, C., Squicciarini, A., Zhu, S., & Miller, D. J. (2020) · ACM Conference on Data and Application Security and Privacy (CODASPY) · doi:10.1145/3374664.3375751 · Cited by 186 · see citing works
Label-Consistent Backdoor AttacksPreprint
Turner, A., Tsipras, D., & Mądry, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02771 · Cited by 177 · see citing works
Laurie, B. (2014) · Communications of the ACM · doi:10.1145/2659897 · Cited by 174 · see citing works
Penedo, G., Malartic, Q., Hesslow, D., Cojocaru, R., Cappelli, A., Alobeidli, H., Pannier, B., Almazrouei, E., & Launay, J. (2023) · arXiv · doi:10.48550/arxiv.2306.01116 · Cited by 164 · see citing works
Multimodal datasets: misogyny, pornography, and malignant stereotypes
Birhane, A., Prabhu, V. U., & Kahembwe, E. (2021) · arXiv · doi:10.48550/arxiv.2110.01963 · Cited by 160 · see citing works
Anti-Backdoor Learning: Training Clean Models on Poisoned DataPreprint
Li, Y., Lyu, X., Koren, N., Lyu, L., Li, B., & Ma, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.11571 · Cited by 154 · see citing works
Learning to Break Deep Perceptual Hashing: The Use Case NeuralHash
Struppek, L., Hintersdorf, D., Neider, D., & Kersting, K. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533073 · Cited by 153 · see citing works
Adversarial Neuron Pruning Purifies Backdoored Deep ModelsPreprint
Wu, D., & Wang, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.14430 · Cited by 122 · see citing works
Concealed Data Poisoning Attacks on NLP Models
Wallace, E., Zhao, T., Feng, S., & Singh, S. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.13 · Cited by 115 · see citing works
Bagging Classifiers for Fighting Poisoning Attacks in Adversarial Classification Tasks
Biggio, B., Corona, I., Fumera, G., Giacinto, G., & Roli, F. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21557-5_37 · Cited by 111 · see citing works
Turner, A., Tsipras, D., & Mądry, A. (2018) · Cited by 104 · see citing works
MetaPoison: Practical General-purpose Clean-label Data PoisoningPreprint
Huang, W., Geiping, J., Fowl, L., Taylor, G., & Goldstein, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.00225 · Cited by 102 · see citing works
The Do’s and Don’ts for CNN-Based Face Verification
Bansal, A., Castillo, C. D., Ranjan, R., & Chellappa, R. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.299 · Cited by 90 · see citing works
Witches' Brew: Industrial Scale Data Poisoning via Gradient MatchingPreprint
Geiping, J., Fowl, L., Huang, W., Czaja, W., Taylor, G., Moeller, M., & Goldstein, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.02276 · Cited by 86 · see citing works
NeuronInspect: Detecting Backdoors in Neural Networks via Output ExplanationsPreprint
Huang, X., Alzantot, M., & Srivastava, M. B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.07399 · Cited by 85 · see citing works
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets
Wang, A., Liu, A., Zhang, R., Kleiman, A., Kim, L., Zhao, D., Shirai, I., Narayanan, A., & Russakovsky, O. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01625-5 · Cited by 85 · see citing works
You Autocomplete Me: Poisoning Vulnerabilities in Neural Code CompletionPreprint
Schuster, R., Song, C., Tromer, E., & Shmatikov, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.02220 · Cited by 83 · see citing works
Backdooring and Poisoning Neural Networks with Image-Scaling Attacks
Quiring, E., & Rieck, K. (2020) · IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw50608.2020.00024 · Cited by 74 · see citing works
Tramèr, F., Shokri, R., Joaquin, A. S., Le, H., Jagielski, M., Hong, S., & Carlini, N. (2022) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3548606.3560554 · Cited by 70 · see citing works
Self-citationBackdoor Scanning for Deep Neural Networks through K-Arm OptimizationPreprint
Shen, G., Liu, Y., Tao, G., An, S., Xu, Q., Cheng, S., Ma, S., & Zhang, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.05123 · Cited by 69 · see citing works
What’s in the Box? An Analysis of Undesirable Content in the Common Crawl Corpus
Luccioni, A., & Viviano, J. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers) · doi:10.18653/v1/2021.acl-short.24 · Cited by 69 · see citing works
Towards Inspecting and Eliminating Trojan Backdoors in Deep Neural Networks
Guo, W., Wang, L., Xu, Y. H., Xing, X., Du, M., & Song, D. X. (2020) · Proceedings · doi:10.1109/icdm50108.2020.00025 · Cited by 68 · see citing works
Poisoning and Backdooring Contrastive LearningPreprint
Carlini, N., & Terzis, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09667 · Cited by 64 · see citing works
Self-citationDomain-Z: 28 Registrations Later Measuring the Exploitation of Residual Trust in Domains
Lever, C., Walls, R. J., Nadji, Y., Dagon, D., McDaniel, P. D., & Antonakakis, M. (2016) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2016.47 · Cited by 64 · see citing works
Backdoor Attack in the Physical WorldPreprint
Li, Y., Zhai, T., Jiang, Y., Li, Z., & Xia, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.02361 · Cited by 58 · see citing works
Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch
Souri, H., Fowl, L. H., Chellappa, R., Goldblum, M., & Goldstein, T. (2022) · neural information processing systems · doi:10.52202/068431-1393 · Cited by 57 · see citing works
Poisoning the Unlabeled Dataset of Semi-Supervised LearningPreprint
Carlini, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.01622 · Cited by 37 · see citing works
Self-citationIt's Not What It Looks Like: Manipulating Perceptual Hashing based Applications
Hao, Q., Luo, L., Jan, S. T. K., & Wang, G. (2021) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3460120.3484559 · Cited by 37 · see citing works
Anti-Distillation Backdoor Attacks: Backdoors Can Really Survive in Knowledge Distillation
Ge, Y., Wang, Q., Zheng, B., Zhuang, X., Li, Q., Shen, C. Q., & Wang, C. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475254 · Cited by 34 · see citing works
Red-Teaming the Stable Diffusion Safety FilterPreprint
Rando, J., Paleka, D., Lindner, D., Heim, L., & Tramèr, F. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.04610 · Cited by 34 · see citing works
Self-citation(2016) · arXiv (Cornell University) · Cited by 29 · see citing works
Poison Forensics: Traceback of Data Poisoning Attacks in Neural NetworksPreprint
Shan, S., Bhagoji, A. N., Zheng, H.-T., & Zhao, B. Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.06904 · Cited by 27 · see citing works
Game of Registrars: An Empirical Analysis of Post-Expiration Domain Name Takeovers
Lauinger, T., Chaabane, A., Buyukkayhan, A. S., Onarlıoğlu, K., & Robertson, W. (2017) · Cited by 26 · see citing works
Contour: A Practical System for Binary Transparency
Al-Bassam, M., & Meiklejohn, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00305-0_8 · Cited by 23 · see citing works
Jain, S., Cretu, A.-M., & de Montjoye, Y.-A. (2021) · arXiv · doi:10.48550/arxiv.2106.09820 · Cited by 21 · see citing works
Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved with Text
Zhu, W., Hessel, J., Awadalla, A., Gadre, S. Y., Dodge, J., Fang, A., Yu, Y., Schmidt, L., Wang, W. Y., & Choi, Y. (2023) · neural information processing systems · doi:10.52202/075280-0393 · Cited by 21 · see citing works
The Ghosts of Banking Past: Empirical Analysis of Closed Bank Websites
Moore, T., & Clayton, R. (2014) · Lecture notes in computer science · doi:10.1007/978-3-662-45472-5_3 · Cited by 20 · see citing works
Poisoning Deep Reinforcement Learning Agents with In-Distribution TriggersPreprint
Ashcraft, C., & Karra, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.07798 · Cited by 19 · see citing works
The Abandoned Side of the Internet: Hijacking Internet Resources When Domain Names Expire
Schlamp, J., Gustafsson, J., Wählisch, M., Schmidt, T. C., Carle, G., München, T. U., & Berlin, F. U. (2015) · Cited by 16 · see citing works
Hidden Trigger Backdoor Attacks
Saha, A., Subramanya, A., & Pirsiavash, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6871 · Cited by 13 · see citing works
Venomave: Targeted Poisoning Against Speech Recognition
Aghakhani, H., Schönherr, L., Eisenhofer, T., Kolossa, D., Holz, T., Kruegel, C., & Vigna, G. (2023) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00035 · Cited by 12 · see citing works
Domains Do Change Their Spots: Quantifying Potential Abuse of Residual Trust
So, J. C. F., Miramirkhani, N., Ferdman, M., & Nikiforakis, N. (2022) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46214.2022.9833609 · Cited by 12 · see citing works
The Abandoned Side of the Internet: Hijacking Internet Resources When Domain Names Expire
Schlamp, J., Gustafsson, J., Wählisch, M., Schmidt, T. C., & Carle, G. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-17172-2_13 · Cited by 7 · see citing works
From Deletion to Re-Registration in Zero Seconds
Lauinger, T., Buyukkayhan, A. S., Chaabane, A., Robertson, W., & Kirda, E. (2018) · ACM Internet Measurement Conference (IMC) · doi:10.1145/3278532.3278560 · Cited by 5 · see citing works
Flagged first, then by citations
8 of the 111 references couldn’t be matched to a known paper.
Research works printed but not linked: 6
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
- Yansong Gao, Change Xu, Derui Wang, Shiping Chen, Damith C Ranasinghe, and Surya Nepal. Strip: A defence against trojan attacks on deep neural networks. In Proceedings of the 35th Annual Computer Security Applications Conference, pages 113-125, 2019.
- Mandy Guo, Zihang Dai, Denny Vrandečić, and Rami Al-Rfou. Wiki-40B: Multilingual language model dataset. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 2440-2452, 2020.
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
- Bart Thomee, David A Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. YFCC100M: The new data in multimedia research. Communications of the ACM, 59(2):64-73, 2016.
- Florian Tramèr, Reza Shokri, Ayrton San Joaquin, Hoang Le, Matthew Jagielski, Sanghyun Hong, and Nicholas Carlini. Truth serum: Poisoning machine learning models to reveal their secrets. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pages 2779-2792, 2022.
Cited by
Sorted by citations
Medical large language models are vulnerable to data-poisoning attacks
Alber, D. A., Yang, Z., Alyakin, A., Yang, E., Rai, S., Valliani, A. A., Zhang, J., Rosenbaum, G. R., Amend-Thomas, A. K., Kurland, D. B., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03445-1 · Cited by 195 · see citing works
Adversarial machine learning :
Vassilev, A., Oprea, A., Fordyce, A. J., & Anderson, H. S. (2024) · doi:10.6028/nist.ai.100-2e2023 · Cited by 193 · see citing works
Self-citationRethinking machine unlearning for large language models
Liu, S., Yao, Y., Jia, J., Casper, S., Baracaldo, N., Hase, P., Yao, Y., Liu, C. Y., Xu, X., Li, H., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-00985-0 · Cited by 173 · see citing works
Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models
Shan, S., Ding, W., Passananti, J., Wu, S. L., Zheng, H.-T., & Zhao, B. Y. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00207 · Cited by 117 · see citing works
Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
Casper, S., Davies, X., Shi, C., Gilbert, T. K., Scheurer, J., Rando, J., Freedman, R., Korbak, T., Lindner, D., Freire, P., et al. (2023) · arXiv · doi:10.48550/arxiv.2307.15217 · Cited by 101 · see citing works
“However, a similar attack is successful for instruction tuning with very few examples [Wan et al., 2023, Xu et al., 2023a], and poisoning web-scale datasets is possible under realistic assumptions [Carlini et al., 2023a].”
BackgroundOn the Risk of Misinformation Pollution with Large Language Models
Pan, Y., Pan, L., Chen, W., Nakov, P., Kan, M.-Y., & Wang, W. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.97 · Cited by 70 · see citing works
Medical large language models are susceptible to targeted misinformation attacks
Han, T., Nebelung, S., Khader, F., Wang, T., Müller-Franzes, G., Kuhl, C., Försch, S., Kleesiek, J., Haarburger, C., Bressem, K. K., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01282-7 · Cited by 62 · see citing works
Toward Federated Large Language Models: Motivations, Methods, and Future Directions
Cheng, Y., Zhang, W., Zhang, Z., Zhang, C., Wang, S., & Mao, S. (2024) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2024.3503680 · Cited by 65 · see citing works
Identifying and Mitigating the Security Risks of Generative AI
Barrett, C., Boyd, B., Bursztein, E., Carlini, N., Chen, B., Choi, J., Chowdhury, A. R., Christodorescu, M., Datta, A., Feizi, S., et al. (2023) · Foundations and Trends® in Privacy and Security · doi:10.1561/3300000041 · Cited by 57 · see citing works
Self-citationCleanCLIP: Mitigating Data Poisoning Attacks in Multimodal Contrastive Learning
Bansal, H., Yin, F., Singhi, N., Grover, A., Yu, Y., & Chang, K.-W. (2023) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv51070.2023.00017 · Cited by 36 · see citing works
Future Shock: Generative AI and the International AI Policy and Governance Crisis
Leslie, D., & Perini, A. M. (2024) · Harvard Data Science Review · doi:10.1162/99608f92.88b4cc98 · Cited by 33 · see citing works
Mengara, O., Avila, A. R., & Falk, T. H. (2024) · IEEE Access · doi:10.1109/access.2024.3355816 · Cited by 32 · see citing works
Three Challenges to Secure AI Systems in the Context of AI Regulations
Hamon, R., Junklewitz, H., Garrido, J. S., & Sánchez, I. (2024) · IEEE Access · doi:10.1109/access.2024.3391021 · Cited by 34 · see citing works
Patel, K., Shah, M., Qureshi, K. M., & Qureshi, M. R. N. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11435-z · Cited by 32 · see citing works
You, W., Hammoudeh, Z., & Lowd, D. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.833 · Cited by 31 · see citing works
Red teaming large language models: A comprehensive review and critical analysis
Jabbar, M. S., Al-Azani, S., Alotaibi, A., & Ahmed, M. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104239 · Cited by 25 · see citing works
Taylor, N., Kormilitzin, A., Lorge, I., Nevado‐Holgado, A. J., Cipriani, A., & Joyce, D. W. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102988 · Cited by 23 · see citing works
Training Data Extraction From Pre-trained Language Models: A Survey
Ishihara, S. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.23 · Cited by 26 · see citing works
Reclaiming the Digital Commons: A Public Data Trust for Training Data
Chan, A., Bradley, H., & Rajkumar, N. (2023) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3600211.3604658 · Cited by 23 · see citing works
A Survey of Reasoning with Foundation ModelsPreprint
Sun, J., Zheng, C., Xie, E., Liu, Z., Chu, R., Liu, J., Xu, J., Ding, M., Li, H., Geng, M., et al. (2023) · doi:10.31219/osf.io/ac4sp · Cited by 51 · see citing works
PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models
Zou, W., Geng, R., Wang, B., & Jia, J. (2024) · arXiv · doi:10.48550/arxiv.2402.07867 · Cited by 16 · see citing works
Threat Modeling AI/ML With the Attack Tree
Hoseini, S. V., Suutala, J., Partala, J., & Halunen, K. (2024) · IEEE Access · doi:10.1109/access.2024.3497011 · Cited by 16 · see citing works
Atlas: A Framework for ML Lifecycle Provenance & Transparency
Spoczynski, M., Melara, M. S., & Szyller, S. (2025) · IEEE European Symposium on Security and Privacy Workshops (EuroS&PW) · doi:10.1109/eurospw67616.2025.00058 · Cited by 15 · see citing works
Kumar, R., Prakash, R., Kumar, N., Chinara, S., & Mohammed, M. A. (2025) · Archives of Computational Methods in Engineering · doi:10.1007/s11831-025-10436-z · Cited by 14 · see citing works
Ecosystem Graphs: The Social Footprint of Foundation Models
Bommasani, R., Soylu, D., Liao, T., Creel, K., & Liang, P. (2023) · doi:10.21203/rs.3.rs-2961271/v1 · Cited by 14 · see citing works
Large Language Models in Health Care: Charting a Path Toward Accurate, Explainable, and Secure AI
Khullar, D., Wang, X., & Wang, F. (2024) · Journal of General Internal Medicine · doi:10.1007/s11606-024-08657-2 · Cited by 13 · see citing works
Machine Learning Security: Threat Model, Attacks, and Challenges
Koball, C., Wang, Y., Rimal, B. P., & Vaidyan, V. M. (2024) · Computer · doi:10.1109/mc.2024.3396357 · Cited by 11 · see citing works
AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases
Chen, Z., Xiang, Z., Xiao, C., Song, D., & Li, B. (2024) · arXiv · doi:10.48550/arxiv.2407.12784 · Cited by 10 · see citing works
“For example, an attacker can easily inject poisoned texts by maliciously editing Wikipedia pages [4]).”
Rozi, M. F., Ban, T., Ozawa, S., Yamada, A., Takahashi, T., & Inoue, D. (2024) · IEEE Access · doi:10.1109/access.2024.3467180 · Cited by 11 · see citing works
PwnPilot: Reflections on Trusting Trust in the Age of Large Language Models and AI Code Assistants
Horne, D. (2023) · Congress in Computer Science, Computer Engineering, and Applied Computing (CSCE) · doi:10.1109/csce60160.2023.00396 · Cited by 9 · see citing works
TrapNet: Model Inversion Defense via Trapdoor
Ma, W.-L., Wang, D.-R., Song, Y., Xue, M.-H., Wen, S., Li, Z.-D., & Xiang, Y. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3560557 · Cited by 11 · see citing works
Rose, C. J., Bidonde, J., Ringsten, M., Glanville, J., Berg, R. C., Cooper, C., Muller, A. E., Bergsund, H. B., Meneses-Echavez, J. F., & Potrebny, T. (2025) · BMC Medical Research Methodology · doi:10.1186/s12874-025-02631-0 · Cited by 5 · see citing works
Jiao, Y., Wang, X., & Yang, K. (2025) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3726302.3730058 · Cited by 8 · see citing works
Chen, Z., Gong, Y., Liu, J., Chen, M., Liu, H., Cheng, Q., Zhang, F., Lu, W. P., & Liu, X. (2025) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3719027.3765023 · Cited by 6 · see citing works
Common Canvas: Open Diffusion Models Trained on Creative-Commons Images
Gokaslan, A., Cooper, A. F., Collins, J., Seguin, L., Jacobson, A., Patel, M., Frankle, J., Stephenson, C., & Kuleshov, V. (2024) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52733.2024.00788 · Cited by 6 · see citing works
Heat Death of Generative Models in Closed-Loop Learning
Marchi, M., Soatto, S., Chaudhari, P., & Tabuada, P. (2024) · IEEE Conference on Decision and Control (CDC) · doi:10.1109/cdc56724.2024.10886816 · Cited by 6 · see citing works
LLM Safeguard is a Double-Edged Sword: Exploiting False Positives for Denial-of-Service Attacks
Zhang, Q.-Z., Xiong, Z., & Mao, M. (2024) · LAMPS@CCS · doi:10.1145/3733800.3763264 · Cited by 7 · see citing works
Architectural Neural Backdoors from First Principles
Langford, H., Shumailov, I., Zhao, Y., Mullins, R., & Papernot, N. (2025) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp61157.2025.00060 · Cited by 5 · see citing works
Adversarial Machine Learning for Social Good: Reframing the Adversary as an Ally
Al-Maliki, S., Qayyum, A., Ali, H. M., Abdallah, M. M., Qadir, J., Hoang, D. T., Niyato, D. T., & Al‐Fuqaha, A. (2024) · IEEE Transactions on Artificial Intelligence · doi:10.1109/tai.2024.3383407 · Cited by 5 · see citing works
Detecting AI-Generated Images via CLIPPreprint
Moskowitz, A., Gaona, T., & Peterson, J. J. (2024) · Research Square · doi:10.21203/rs.3.rs-4259468/v1 · Cited by 5 · see citing works
Sardar, T. H., Pandey, B., & Aldasheva, L. (2026) · IEEE International Conference on AI in Cybersecurity (ICAIC) · doi:10.1109/icaic67076.2026.11395730 · Cited by 4 · see citing works
Misactivation-Aware Stealthy Backdoor Attacks on Neural Code Understanding Models
Sun, X., Xiao, Y., Bo, L., Sun, W., Liu, X., Li, B., & Zhang, J. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3614128 · Cited by 4 · see citing works
Levy, M., Ravfogel, S., & Goldberg, Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.569 · Cited by 4 · see citing works
Mitigating Backdoor Threats to Large Language Models: Advancement and Challenges
Liu, Q., Mo, W., Tong, T., Xu, J., Wang, F., Xiao, C., & Chen, M. (2024) · Allerton Conference on Communication, Control, and Computing · doi:10.1109/allerton63246.2024.10735305 · Cited by 6 · see citing works
Mithridates: Auditing and Boosting Backdoor Resistance of Machine Learning Pipelines
Bagdasarian, E., & Shmatikov, V. (2024) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3658644.3690337 · Cited by 5 · see citing works
Carroll, J. (2025) · European Conference on Cyber Warfare and Security · doi:10.34190/eccws.24.1.3713 · Cited by 4 · see citing works
Mo, Y., Tang, M., Lin, R., Zhou, B., & Li, X. (2025) · IEEE Access · doi:10.1109/access.2025.3582519 · Cited by 3 · see citing works
Multi-Agent AI Framework for Threat Mitigation and Resilience in Machine Learning Systems
Foundjem, A. T., Tidjon, L. N., Silva, L. D., & Khomh, F. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3780095 · Cited by 3 · see citing works
Safety and security of large language models in healthcare
Clusmann, J., Freyer, O., Ostermann, M., Ferber, D., Laleh, N. G., Hilgers, L., Kolbinger, F. R., Schneider, C. V., Downing, A., Wekenborg, M., et al. (2026) · Nature · doi:10.1038/s41586-026-10687-1 · Cited by 4 · see citing works
Preference Poisoning Attacks on Reward Model Learning
Wu, J., Wang, J., Xiao, C., Wang, C., Zhang, N., & Vorobeychik, Y. (2025) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp61157.2025.00094 · Cited by 3 · see citing works
Rose, C. J., Ringsten, M., Bidonde, J., Glanville, J., Berg, R. C., Cooper, C., Muller, A. E., Bergsund, H. B., Meneses-Echavez, J. F., & Potrebny, T. (2023) · Research Square · doi:10.21203/rs.3.rs-3288515/v1 · Cited by 3 · see citing works
Phantom: Untargeted Poisoning Attacks on Semi-Supervised Learning
Knauer, J., Rieger, P., Fereidooni, H., & Sadeghi, A. (2024) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3658644.3690369 · Cited by 2 · see citing works
A Rusty Link in the AI Supply Chain: Detecting Evil Configurations in Model Repositories
Ding, Z., Fu, Q., Ding, J., Deng, G., Liu, Y., & Li, Y. (2025) · IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw67851.2025.00036 · Cited by 2 · see citing works
Attacking Attention of Foundation Models Disrupts Downstream Tasks
Silva, H. P., Becattini, F., & Seidenari, L. (2025) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw67362.2025.00338 · Cited by 2 · see citing works
Farooq, A., Anuragi, D., Li, Z., Ziat, M., Cooperstock, J. R., & Raisamo, R. (2025) · Human-Intelligent Systems Integration · doi:10.1007/s42454-025-00085-9 · Cited by 2 · see citing works
Kalyanapu, J., Dizani, F., Asher, D., Ghanbari, A., Cammarota, R., Aysu, A., & Ajorpaz, S. M. (2025) · IEEE/ACM International Symposium on Microarchitecture (MICRO) · doi:10.1145/3725843.3756097 · Cited by 2 · see citing works
Jędrasiak, K. (2025) · Bezpieczeństwo i Technika Pożarnicza · doi:10.12845/sft.66.2.2025.10 · Cited by 2 · see citing works
A Survey on Deep Learning for Face Anti-Spoofing in Automotive Applications
Ruiz, M., Hassani, H., & Nikan, S. (2026) · IEEE Transactions on Biometrics Behavior and Identity Science · doi:10.1109/tbiom.2026.3669032 · Cited by 2 · see citing works
Tan, M., Yamaguchi, K., Raney, A., Nockles, V., Leblanc, M., & Bendelac, S. (2024) · Assurance and Security for AI-enabled Systems · doi:10.1117/12.3021908 · Cited by 3 · see citing works
Adversarially Robust Anti-Backdoor Learning
Zhao, Q., & Wressnegger, C. (2024) · doi:10.1145/3689932.3694763 · Cited by 2 · see citing works
Integrating Generative AI in Cybersecurity Curricula
AlOmar, B., & Trabelsi, Z. (2025) · IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon62633.2025.11016426 · Cited by 2 · see citing works
Generalized Adversarial Code-Suggestions: Exploiting Contexts of LLM-based Code-Completion
Rubel, K., Noppel, M., & Wressnegger, C. (2025) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3708821.3733870 · Cited by 2 · see citing works
Model-agnostic clean-label backdoor mitigation in cybersecurity environments
Severi, G., Boboila, S., Holodnak, J. T., Kratkiewicz, K., Izmailov, R., Lucia, M. D., & Oprea, A. (2025) · IEEE Military Communications Conference (MILCOM) · doi:10.1109/milcom64451.2025.11310745 · Cited by 2 · see citing works
Towards the Effect of Examples on In‐Context Learning: A Theoretical Case Study
He, P., Cui, Y., Xu, H., Liu, H., Yamada, M., Tang, J., & Xing, Y. (2025) · Stat · doi:10.1002/sta4.70045 · Cited by 1 · see citing works
Anichkov, Y., Popov, V., & Bolovtsov, S. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-80853-1_31 · Cited by 1 · see citing works
Lee, C., Kao, C.-C., Yeh, C.-H., Lu, C.-S., Yu, C.-M., & Chen, C.-S. (2025) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv61041.2025.00630 · Cited by 1 · see citing works
Dual-Model Backdoor Defense with Contrast Learning
Chen, Y., & Yin, J.-L. (2025) · International Conference on Big Data and Privacy Computing (BDPC) · doi:10.1109/bdpc63545.2025.11135804 · Cited by 1 · see citing works
GASLITEing the Retrieval: Exploring Vulnerabilities in Dense Embedding-based Search
Ben-Tov, M., & Sharif, M. (2025) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3719027.3765095 · Cited by 1 · see citing works
LS 2 : Boosting Hidden Separation for Backdoor Defense with Learning Speed-driven Label Smoothing
Peng, J., Yang, H., He, H., Zhao, J., He, H., Dong, H., & Zhang, W. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3616621 · Cited by 1 · see citing works
Push and Pull : Defending against Retrieval Poisoning Attacks via Embedding Space Reshaping
He, L., Zhang, X., Liu, Q., Li, C., Liu, Z., Zhou, P., & Su, S. (2026) · ACM Transactions on Information Systems · doi:10.1145/3800948 · Cited by 1 · see citing works
Mitigating Cognitive Vulnerabilities in Code Generation via Multi-Agent Adversarial Debate
Liu, S., Guo, Q., Liu, X., & Liu, Y. (2026) · International World Wide Web Conference (WWW) · doi:10.1145/3774904.3792557 · Cited by 1 · see citing works
Exact Certification of Data-Poisoning Attacks Using Mixed-Integer Programming
Sosnin, P., Knapp, J., Kennedy, F., Collyer, J., & Tsay, C. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-27242-3_32 · Cited by 1 · see citing works
Bias in AI: a framework for understanding the sources of bias in artificial intelligence
Robinig, W., & Wallner, J. P. (2026) · AI and Ethics · doi:10.1007/s43681-026-01283-0 · Cited by 1 · see citing works
Guo, W., Ma, X., Ge, P., Chen, Y., Yue, Q., & Zhang, Y. (2024) · IEEE International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData) · doi:10.1109/ithings-greencom-cpscom-smartdata-cybermatics62450.2024.00097 · Cited by 1 · see citing works
Clean Label Attacks Against SLU Systems
Xinyuan, H. L., Joshi, S., Thebaud, T., Villalba, J., Dehak, N., & Khudanpur, S. P. (2024) · IEEE Spoken Language Technology Workshop (SLT) · doi:10.1109/slt61566.2024.10832144 · Cited by 1 · see citing works
PACOL: Poisoning Attacks Against Continual LearnersPreprint
Ditzler, G., & Li, H. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4517547 · Cited by 1 · see citing works
Campbell, R. (2026) · Systems · doi:10.3390/systems14050593 · Cited by 3 · see citing works
Ctrl-Z: Controlling AI Agents via Resampling
Bhatt, A., Rushing, C., Kaufman, A., Tracy, T., Georgiev, V., Matolcsi, D., Khan, A., & Shlegeris, B. (2025) · arXiv · doi:10.48550/arxiv.2504.10374
“Internally deployed LLM agents might not behave as intended for several reasons, including the possibility that they develop misaligned goals and “fake alignment” during testing (Greenblatt et al., 2024; Carl-smith, 2023), due to data poisoning (Carlini et al., 2024), or for other reasons.”
Bespoke Large Language Models for Digital Triage Assistance in Mental Health CarePreprint
Taylor, N., Kormilitzin, A., Lorge, I., Nevado‐Holgado, A. J., & Joyce, D. W. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4777351
SINTESIS TEKS KE GAMBAR: TINJAUAN ATAS DATASET
Arifin, S. R. (2024) · Jurnal EEICT (Electric Electronic Instrumentation Control Telecommunication) · doi:10.31602/eeict.v7i1.13066
Agent Deprogramming Finetuning Away Backdoor Triggers for Secure Machine Learning Models/LLMs
Edwards, D., & Rawat, D. B. (2024) · International Conference on Computer Communications and Networks (ICCCN) · doi:10.1109/icccn61486.2024.10637573
FCert: Certifiably Robust Few-Shot Classification in the Era of Foundation Models
Wang, Y., Zou, W., & Jia, J. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00240
Restoring Unintended Model Learning by Error Correcting Code
Ashizawa, N., Shibahara, T., Kiribuchi, N., Saisho, O., & Yanai, N. (2024) · Annual Computer Security Applications Conference Workshops (ACSAC Workshops) · doi:10.1109/acsacw65225.2024.00013
A Survey on Emoji and Unicode-Based Data Masking Attacks on AI Systems
Sakpal, A. (2025) · International Journal for Research in Applied Science and Engineering Technology · doi:10.22214/ijraset.2025.72523
SESP: Semantic Entropy-based Self-play Security Alignment Mechanism for Large Language ModelsPreprint
Sun, H., Cao, L., Zhao, Z., Du, X., Liu, A., Sang, W., & Li, Y. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5332865
Data Leakage and Data Poisoning
Kobayashi, M. (2025) · Springer eBooks · doi:10.1007/978-981-96-9223-1_5
Exploiting Contexts of LLM-based Code-Completion
Noppel, M., Rubel, K., & Wressnegger, C. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-02813-6_26
Neretin, O., & Kharchenko, V. S. (2025) · Lecture notes in networks and systems · doi:10.1007/978-3-032-05802-7_34
Large Language Models as Search Engines: Societal Challenges
Sadeddine, Z., Maxwell, W., Varoquaux, G. P., & Suchanek, F. M. (2025) · ACM SIGIR Forum · doi:10.1145/3769733.3769746
On the Resilience of Traditional AI Algorithms Toward Poisoning Attacks for Vulnerability Detection
González‐Manzano, L., & García-Alfaro, J. (2025) · IET Information Security · doi:10.1049/ise2/9997989
109PwnPilot: could an adversary be pair programming with our most trusted software engineers?
Horne, D. (2025) · De Gruyter eBooks · doi:10.1515/9783111436548-006
A Practical and Secure Byzantine Robust Aggregator
Lee, D. Z., Kolluri, A., Saxena, P., & Chang, E. (2025) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3719027.3744851
VillainNet: Targeted Poisoning Attacks Against SuperNets Along the Accuracy-Latency Pareto Frontier
Oygenblik, D., Vemulapalli, A. S., Agrawal, A., Sanyal, D., Tumanov, A., & Saltaformaggio, B. (2025) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3719027.3765185
CatPoison: Category-Oriented Knowledge Poisoning Attacks in Retrieval-Augmented Generation Systems
Mei, H., Du, Z., Cai, C., Cheng, C., & Chen, J. (2025) · IEEE International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom) · doi:10.1109/trustcom66490.2025.00254
CAN LARGE LANGUAGE MODELS ACT AS “CO-AUDITORS”?
Emekci, H. (2026) · Denetişim · doi:10.58348/denetisim.1782835
Secure Prompt Engineering Patterns for Cloud LLM Agents
Sudarsan, S., Patel, A., Upadhyay, C., Gudur, V., & Matam, P. (2026) · IEEE International Conference on AI in Cybersecurity (ICAIC) · doi:10.1109/icaic67076.2026.11395764
(2025) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-83549-238-320251010
Large Language Models as Search Engines: Societal Challenges
Sadeddine, Z., Maxwell, W., Varoquaux, G. P., & Suchanek, F. M. (2025) · HAL · doi:10.1145/3769733.376974
Privatar: A Privacy-Preserving Framework for LLM Agents
Jian, R., Zhu, Z., Shang, Y., Shen, Y., & Liu, K. (2025) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw69685.2025.00011
- Authors
- Carlini, N., Jagielski, M., Choquette-Choo, C. A., Paleka, D., Pearce, W., Anderson, H., Terzis, A., Thomas, K., & Tramèr, F.
- Affiliations
- Google DeepMind (United Kingdom) (GB)
- ETH Zurich (CH)
- Nvidia (United Kingdom) (GB)
- Robust Intelligence
- Google (United States) (US)
- Venue
- 2024 IEEE Symposium on Security and Privacy (SP)
- Publisher
- IEEE
- Published
- 2024-05-19
- Pages
- 407-425
- ISSN
- 1081-6011 (ISSN-L)
- OpenAlex
- W4402264526
- Cited by
- 138
- References
- 111
- Citations by year
- 2026: 53 · 2025: 44 · 2024: 29 · 2023: 12
- FWCI
- 37.18
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Symposium on Security and Privacy (S&P) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Keywords
- deep learning security, Wikipedia, crowdsourced data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
