The DOI matches its DataCite record; 104 of the 112 references it cites were found in a registry or an index.
Poisoning Web-Scale Training Datasets is Practical
Open accessAbstract
Deep learning models are often trained on distributed, web-scale datasets crawled from the internet. In this paper, we introduce two new dataset poisoning attacks that intentionally introduce malicious examples to a model's performance. Our attacks are immediately practical and could, today, poison 10 popular datasets. Our first attack, split-view poisoning, exploits the mutable nature of internet content to ensure a dataset annotator's initial view of the dataset differs from the view downloaded by subsequent clients. By exploiting specific invalid trust assumptions, we show how we could have poisoned 0.01% of the LAION-400M or COYO-700M datasets for just $60 USD. Our second attack, frontrunning poisoning, targets web-scale datasets that periodically snapshot crowd-sourced content -- such as Wikipedia -- where an attacker only needs a time-limited window to inject malicious examples. In light of both attacks, we notify the maintainers of each affected dataset and recommended several low-overhead defenses.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This oversight provides an exploitable attack surface, as malicious actors could intentionally seed misinformation into LLM training datasets through data-poisoning6 attacks that do not require direct access to model weights.”
From Medical large language models are vulnerable to data-poisoning attacks (2025) · cites this paper as [6]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“even for manually maintained ones like Wikipedia, it is still possible for misinformation to slip in (Carlini et al., 2023).”
From On the Risk of Misinformation Pollution with Large Language Models (2023) · cites this paper as Carlini et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“pora with data in URLs risk attackers hijacking expired domains and tampering with contents (Carlini et al., 2023).”
From On the Risk of Misinformation Pollution with Large Language Models (2023) · cites this paper as Carlini et al., 2023
The cited source says what the claim attributes to it.
In the citing paper
“Other studies have evaluated potential attack vectors against general knowledge6 and demonstrated that significant effects emerge with minimal poisoning of computer vision systems57.”
From Medical large language models are vulnerable to data-poisoning attacks (2025) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333 · cited 1×
“Datasets used to train deep learning models have grown from thousands of carefully-curated examples [20, 33, 41] to webscale datasets with billions of samples automatically crawled from the internet [10, 48, 53, 57].”
BackgroundLearning Multiple Layers of Features from Tiny Images
Krizhevsky, A. (2024) · doi:10.57702/zp44cu3g · Cited by 23,889
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 2×
“Datasets used to train deep learning models have grown from thousands of carefully-curated examples [20, 33, 41] to webscale datasets with billions of samples automatically crawled from the internet [10, 48, 53, 57].”
BackgroundLearning Transferable Visual Models From Natural Language Supervision
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. (2021) · arXiv · doi:10.48550/arxiv.2103.00020 · Cited by 15,293 · cited 2×
“Possibly most important among these is that uncurated data has serious implications for fairness, bias, and ethics [8, 50, 77].”
BackgroundExploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929 · cited 1×
“Datasets used to train deep learning models have grown from thousands of carefully-curated examples [20, 33, 41] to webscale datasets with billions of samples automatically crawled from the internet [10, 48, 53, 57].”
BackgroundScaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 1×
“Deep learning is most effective when applied to large datasets [10, 30].”
BackgroundBuilding a Large Annotated Corpus of English: The Penn Treebank
Marcus, M. (1993) · doi:10.21236/ada273556 · Cited by 7,405
Parkhi, O. M., Vedaldi, A., & Zisserman, A. (2015) · Procedings of the British Machine Vision Conference 2015 · doi:10.5244/c.29.41 · Cited by 5,479
Pointer Sentinel Mixture ModelsPreprint
Merity, S., Xiong, C., Bradbury, J., & Socher, R. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1609.07843 · Cited by 5,058 · cited 1×
“Indeed, many language modelling datasets heavily rely on the English Wikipedia, e.g., it formed over 75% of the words in the BERT training set [25], 1.5% of the Pile dataset [27], and the entirety of the WikiText dataset [60].”
InfluentialVGGFace2: A Dataset for Recognising Faces across Pose and Age
Cao, Q., Shen, L., Xie, W., Parkhi, O. M., & Zisserman, A. (2018) · 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018) · doi:10.1109/fg.2018.00020 · Cited by 3,032 · cited 1×
“Critically, the maintainer does not provide a snapshot of the data associated with urli, due either to untenable storage costs [3,4,17,31], privacy concerns [13, 72], or copyright limitations [16].”
BackgroundPaLM: Scaling Language Modeling with Pathways
Chowdhery, A., Narang, S., Devlin, J., Bosma, M., Mishra, G., Roberts, A., Barham, P., Chung, H. W., Sutton, C., Gehrmann, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2204.02311 · Cited by 2,590 · cited 1×
“For example, the scaling laws observed in the recent Chinchilla [27] language model indicate that training a compute-optimal 500 billion parameter model would require 11 trillion tokens of training data—over 10× more data than is currently used to train models of this size [19].”
MethodUnderstanding deep learning (still) requires rethinking generalization
Zhang, C., Bengio, S., Hardt, M., Recht, B., & Vinyals, O. (2021) · Communications of the ACM · doi:10.1145/3446776 · Cited by 2,400 · cited 1×
“This quantity-over-quality tradeoff has so far been deemed acceptable, both because modern neural networks are extremely resilient to large amounts of label noise [79], [114], and because training on noisy data can even improve model utility on out-of-distribution data [74], [75].”
BackgroundRobust Speech Recognition via Large-Scale Weak Supervision
Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., & Sutskever, I. (2022) · arXiv · doi:10.48550/arxiv.2212.04356 · Cited by 1,953 · cited 1×
“This quantity-over-quality tradeoff has so far been deemed acceptable, both because modern neural networks are extremely resilient to large amounts of label noise [79], [114], and because training on noisy data can even improve model utility on out-of-distribution data [74], [75].”
Background80 Million Tiny Images: A Large Data Set for Nonparametric Object and Scene Recognition
Torralba, A., Fergus, R., & Freeman, W. T. (2008) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2008.128 · Cited by 1,921
Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning
Sharma, P., Ding, N., Goodman, S., & Soricut, R. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1238 · Cited by 1,879
Thomée, B., Shamma, D. A., Friedland, G. H., Elizalde, B., Ni, K. S., Poland, D., Borth, D., & Li, L.-J. (2016) · Communications of the ACM · doi:10.1145/2812802 · Cited by 1,715
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings
Bolukbasi, T., Chang, K.-W., Zou, J., Saligrama, V., & Kalai, A. (2016) · arXiv · doi:10.48550/arxiv.1607.06520 · Cited by 1,615 · cited 1×
“Possibly most important among these is that uncurated data has serious implications for fairness, bias, and ethics [8, 50, 77].”
BackgroundAttribute and simile classifiers for face verification
Kumar, N., Berg, A. C., Belhumeur, P. N., & Nayar, S. K. (2009) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2009.5459250 · Cited by 1,403 · cited 1×
“[70] 276 2023 + ✗ 0.24% ≥ 0.02% - OBELISC [49] 353 2023 + ✗ 0.09% ≥ 0.01% - LAION-2B-en [83] 2323 2022 [46] 0 .”
InfluentialPoisoning Attacks against Support Vector Machines
Biggio, B., Nelson, B., & Laskov, P. (2012) · arXiv · doi:10.48550/arxiv.1206.6389 · Cited by 1,331 · cited 2×
“Uncurated training datasets are prime targets for poisoning attacks [6].”
BackgroundBadNets: Evaluating Backdooring Attacks on Deep Neural Networks
Gu, T., Liu, K., Dolan-Gavitt, B., & Garg, S. (2019) · IEEE Access · doi:10.1109/access.2019.2909068 · Cited by 1,238 · cited 2×
“001% of uncurated web-scale training datasets is sufficient to induce targeted model mistakes, or plant model “backdoors” [18, 24].”
BackgroundBitcoin and Cryptocurrency Technologies : A Comprehensive Introduction
Bolt, W. (2017) · Data Archiving and Networked Services (DANS) · Cited by 1,064
BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain
Gu, T., Dolan-Gavitt, B., & Garg, S. (2017) · arXiv · doi:10.48550/arxiv.1708.06733 · Cited by 1,061
Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning
Chen, X., Liu, C., Li, B., Lu, K., & Song, D. (2017) · arXiv · doi:10.48550/arxiv.1712.05526 · Cited by 1,039 · cited 2×
“001% of uncurated web-scale training datasets is sufficient to induce targeted model mistakes, or plant model “backdoors” [18, 24].”
BackgroundLAION-5B: An open large-scale dataset for training next generation image-text models
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al. (2022) · arXiv · doi:10.48550/arxiv.2210.08402 · Cited by 1,022 · cited 2×
“Datasets used to train deep learning models have grown from thousands of carefully-curated examples [20, 33, 41] to webscale datasets with billions of samples automatically crawled from the internet [10, 48, 53, 57].”
BackgroundTraining Compute-Optimal Large Language Models
Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., Casas, D. D. L., Hendricks, L. A., Welbl, J., Clark, A., et al. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-2176 · Cited by 932 · cited 2×
“For example, the scaling laws observed in the recent Chinchilla [36] language model indicate that training a compute-optimal 500 billion parameter model would require 11 trillion tokens of training data—over 10 × more data than is currently used to train models of this size [23].”
MethodThe MegaFace Benchmark: 1 Million Faces for Recognition at Scale
Kemelmacher-Shlizerman, I., Seitz, S. M., Miller, D., & Brossard, E. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.527 · Cited by 912 · cited 1×
“Critically, the maintainer does not provide a snapshot of the data associated with url i , due either to untenable storage costs [5], [6], [21], [43], privacy concerns [16], [98], or copyright limitations [19].”
BackgroundGao, Y., Xu, C., Wang, D., Chen, S., Ranasinghe, D. C., & Nepal, . (2019) · Annual Computer Security Applications Conference · doi:10.1145/3359789.3359790 · Cited by 886
Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learning
Jagielski, M., Oprea, A., Biggio, B., Liu, C., Nita-Rotaru, C., & Li, B. (2018) · 2018 IEEE Symposium on Security and Privacy (SP) · doi:10.1109/sp.2018.00057 · Cited by 861 · cited 1×
“…[54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66], [73], [81] that first presumes an adversary can modify a training dataset, and…”
Self-citationWikiQA: A Challenge Dataset for Open-Domain Question Answering
Yang, Y., Yih, W.-T., & Meek, C. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1237 · Cited by 829 · cited 1×
“Many task-specific datasets also rely on the English Wikipedia: e.g., the WikiQA [112] question answering dataset (30,000+ downloads), and the WikiBio [51] biography writing dataset (19,000+ downloads).”
BackgroundConceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Concepts
Changpinyo, S., Sharma, P. K., Ding, N., & Soricut, R. (2021) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr46437.2021.00356 · Cited by 701 · cited 2×
“06% ≥10 Conceptual 12M [16] 12 2021 7 1.”
BackgroundFLAVA: A Foundational Language And Vision Alignment Model
Singh, A., Hu, R., Goswami, V., Couairon, G., Galuba, W., Rohrbach, M., & Kiela, D. (2022) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr52688.2022.01519 · Cited by 678 · cited 2×
“For example, the Public Multimodal Dataset (PMD) [92] and LAION-Aesthetics [85] datasets consist almost entirely of images drawn from LAION-2B-en.”
BackgroundA data-driven approach to cleaning large face datasets
Ng, H., & WINKLER, S. S. (2014) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2014.7025068 · Cited by 673
Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural NetworksPreprint
Shafahi, A., Huang, W., Najibi, M., Suciu, O., Studer, C., Dumitraş, T., & Goldstein, T. (2018) · arXiv (Cornell University) · doi:10.3929/ethz-b-000460348 · Cited by 626
Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimization
Muñoz-González, L., Biggio, B., Demontis, A., Paudice, A., Wongrassamee, V., Lupu, E. C., & Roli, F. (2017) · Proceedings of the 10th ACM Workshop on Artificial Intelligence and Security · doi:10.1145/3128572.3140451 · Cited by 549
Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networks
Liu, Y., Ma, X., Bailey, J., & Lu, F. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-58607-2_11 · Cited by 531 · cited 1×
“…[37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66], [73], [81] that first presumes an adversary can modify a training dataset, and then asks (1) what impact this could have, (2) if…”
Spectral Signatures in Backdoor AttacksPreprint
Tran, B., Li, J. Z., & Mądry, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.00636 · Cited by 529 · cited 1×
“…[106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66], [73], [81] that first presumes an adversary can modify a training dataset, and then asks (1) what impact…”
Invisible Backdoor Attack with Sample-Specific Triggers
Li, Y., Li, Y., Wu, B., Li, L., He, R., & Lyu, S. (2021) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.01615 · Cited by 511 · cited 1×
“…is an exceptionally vast literature [1], [3], [7], [10], [11], [17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57],…”
Vggsound: A Large-Scale Audio-Visual Dataset
Chen, H., Xie, W., Vedaldi, A., & Zisserman, A. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9053174 · Cited by 507 · cited 1×
“Critically, the maintainer does not provide a snapshot of the data associated with urli, due either to untenable storage costs [3,4,17,31], privacy concerns [13, 72], or copyright limitations [16].”
BackgroundThe Pile: An 800GB Dataset of Diverse Text for Language Modeling
Gao, L., Biderman, S., Black, S., Golding, L., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., Nabeshima, N., et al. (2021) · arXiv · doi:10.48550/arxiv.2101.00027 · Cited by 492 · cited 1×
“5% of the Pile dataset [23], and the entirety of the WikiText dataset [43].”
BackgroundDetecting Backdoor Attacks on Deep Neural Networks by Activation Clustering
Chen, B., Carvalho, W., Baracaldo, N., Ludwig, H., Edwards, B., Lee, T., Molloy, I., & Srivastava, B. (2018) · arXiv · doi:10.48550/arxiv.1811.03728 · Cited by 485 · cited 1×
“…[17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66], [73], [81] that first presumes an adversary can modify a…”
Label-Consistent Backdoor AttacksPreprint
Turner, A., Tsipras, D., & Mądry, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02771 · Cited by 447 · cited 1×
“…[7], [10], [11], [17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66], [73], [81] that first presumes an…”
ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation
Liu, Y., Lee, W., Tao, G., Ma, S., Aafer, Y., & Zhang, X. (2019) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3319535.3363216 · Cited by 439 · cited 1×
“…an exceptionally vast literature [1], [3], [7], [10], [11], [17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65],…”
Attack of the tails: Yes, you really can backdoor federated learning
Wang, H., Sreenivasan, K., Rajput, S., Vishwakarma, H., Agarwal, S., Sohn, J.-Y., Lee, K., & Papailiopoulos, D. (2020) · arXiv · doi:10.48550/arxiv.2007.05084 · Cited by 406 · cited 1×
“Indeed there is an exceptionally vast literature [1], [3], [7], [10], [11], [17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29],…”
Deep Learning is Robust to Massive Label Noise
Rolnick, D., Veit, A., Belongie, S., & Shavit, N. (2017) · arXiv · doi:10.48550/arxiv.1705.10694 · Cited by 402 · cited 1×
“This quantity-over-quality tradeoff has so far been deemed acceptable, both because modern neural networks are extremely resilient to large amounts of label noise [55, 83], and because”
BackgroundWaNet -- Imperceptible Warping-based Backdoor AttackPreprint
Nguyen, A., & Tran, A. N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.10369 · Cited by 394
A New Backdoor Attack in CNNS by Training Set Corruption Without Label Poisoning
Barni, M., Kallas, K., & Tondi, B. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8802997 · Cited by 392 · cited 1×
“Indeed there is an exceptionally vast literature [1], [3], [7], [10], [11], [17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29],…”
Latent Backdoor Attacks on Deep Neural Networks
Yao, Y., Li, H., Zheng, H.-T., & Zhao, B. Y. (2019) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3319535.3354209 · Cited by 382
LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs
Schuhmann, C., Vencu, R., Beaumont, R., Kaczmarczyk, R., Mullis, C., Katta, A., Coombes, T., Jitsev, J., & Komatsuzaki, A. (2021) · arXiv · doi:10.48550/arxiv.2111.02114 · Cited by 358
Beyond vandalism: Wikipedia trolls
Shachaf, P., & Hara, N. (2010) · Journal of Information Science · doi:10.1177/0165551510365390 · Cited by 357 · cited 1×
“Indeed, it is easy to temporarily edit Wikipedia to vandalize its contents [63, 69].”
BackgroundBackdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation
Zhong, H., Liao, C., Squicciarini, A., Zhu, S., & Miller, D. J. (2020) · ACM Conference on Data and Application Security and Privacy (CODASPY) · doi:10.1145/3374664.3375751 · Cited by 332 · cited 1×
“…[10], [11], [17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66], [73], [81] that first presumes an adversary can…”
Is feature selection secure against training data poisoning?Preprint
Huang, X. X., Biggio, B., Gavin Brown, B., Fumera, G., Eckert, C. M., & Roli, F. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.07933 · Cited by 315
Support Vector Machines Under Adversarial Label Noise
Biggio, B., Nelson, B. A., & Laskov, P. (2011) · UNICA IRIS Institutional Research Information System (University of Cagliari) · Cited by 309
Detecting AI Trojans Using Meta Neural Analysis
Xu, X., Wang, Q., Li, H., Borisov, N., Gunter, C. A., & Li, B. (2021) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp40001.2021.00034 · Cited by 306 · cited 1×
“…[26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66], [73], [81] that first presumes an adversary can modify a training dataset, and then asks (1) what impact this could…”
Nikiforakis, N., Invernizzi, L., Kapravelos, A., Acker, S. V., Joosen, W., Kruegel, C., Piessens, F., & Vigna, G. (2012) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/2382196.2382274 · Cited by 279
Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems
Doan, B. G., Abbasnejad, E., & Ranasinghe, D. C. (2020) · Annual Computer Security Applications Conference · doi:10.1145/3427228.3427264 · Cited by 263 · cited 1×
“Indeed there is an exceptionally vast literature [1], [3], [7], [10], [11], [17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29],…”
Hidden Killer: Invisible Textual Backdoor Attacks with Syntactic Trigger
Qi, F., Li, M., Chen, Y., Zhang, Z., Liu, Z., Wang, Y., & Sun, M. (2021) · doi:10.18653/v1/2021.acl-long.37 · Cited by 255 · cited 1×
“…vast literature [1], [3], [7], [10], [11], [17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66],…”
Information quality work organization in wikipedia
Stvilia, B., Twidale, M. B., Smith, L. C., & Gasser, L. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20813 · Cited by 250 · cited 1×
“Indeed, it is easy to temporarily edit Wikipedia to vandalize its contents [63, 69].”
BackgroundUMDFaces: An annotated face dataset for training deep networks
Bansal, A., Nanduri, A., Castillo, C. D., Ranjan, R., & Chellappa, R. (2017) · IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/btas.2017.8272731 · Cited by 246 · cited 1×
“Critically, the maintainer does not provide a snapshot of the data associated with urli, due either to untenable storage costs [3,4,17,31], privacy concerns [13, 72], or copyright limitations [16].”
BackgroundInput-Aware Dynamic Backdoor AttackPreprint
Nguyen, T. A., & Tran, A. N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.08138 · Cited by 237 · cited 1×
“…[56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66], [73], [81] that first presumes an adversary can modify a training dataset, and then asks (1) what impact this could have, (2) if poisoning can…”
Dynamic Backdoor Attacks Against Machine Learning Models
Salem, A., Wen, R., Michael Backes, A., Ma, S., & Zhang, Y. (2022) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp53844.2022.00049 · Cited by 233 · cited 1×
“…[80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66], [73], [81] that first presumes an adversary can modify a training dataset, and then asks (1) what impact this could have, (2) if poisoning can be stealthy,…”
BackgroundDeepSweep: An Evaluation Framework for Mitigating DNN Backdoor Attacks using Data Augmentation
Qiu, H., Zeng, Y., Guo, S., Zhang, T., Qiu, M., & Thuraisingham, B. M. (2021) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3433210.3453108 · Cited by 205 · cited 1×
“…[86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66], [73], [81] that first presumes an adversary can modify a training dataset, and then asks (1)…”
Laurie, B. (2014) · Communications of the ACM · doi:10.1145/2659897 · Cited by 174 · cited 1×
“Data transparency around the set of { ( url i , c i , h i ) } distributed to clients—akin to certificate transparency [50]—can prevent transient failures or a compromised maintainer from distributing different datasets to different clients and to assist in the detection and removal of inaccurate c…”
BackgroundPenedo, G., Malartic, Q., Hesslow, D., Cojocaru, R., Cappelli, A., Alobeidli, H., Pannier, B., Almazrouei, E., & Launay, J. (2023) · arXiv · doi:10.48550/arxiv.2306.01116 · Cited by 164 · cited 1×
“[70] 276 2023 + ✗ 0.24% ≥ 0.02% - OBELISC [49] 353 2023 + ✗ 0.09% ≥ 0.01% - LAION-2B-en [83] 2323 2022 [46] 0 .”
InfluentialMultimodal datasets: misogyny, pornography, and malignant stereotypes
Birhane, A., Prabhu, V. U., & Kahembwe, E. (2021) · arXiv · doi:10.48550/arxiv.2110.01963 · Cited by 160 · cited 2×
“[7] note that LAION-400M has “troublesome and explicit images and text pairs of rape, pornography, malign stereotypes, racist and ethnic slurs, and other extremely problematic content”.”
BackgroundWitches' Brew: Industrial Scale Data Poisoning via Gradient MatchingPreprint
Geiping, J., Fowl, L., Huang, W., Czaja, W., Taylor, G., Moeller, M., & Goldstein, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.02276 · Cited by 156 · cited 1×
“Indeed there is an exceptionally vast literature [1], [3], [7], [10], [11], [17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29],…”
Anti-Backdoor Learning: Training Clean Models on Poisoned DataPreprint
Li, Y., Lyu, X., Koren, N., Lyu, L., Li, B., & Ma, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.11571 · Cited by 156 · cited 1×
“…[62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66], [73], [81] that first presumes an adversary can modify a training dataset, and then asks…”
Learning to Break Deep Perceptual Hashing: The Use Case NeuralHash
Struppek, L., Hintersdorf, D., Neider, D., & Kersting, K. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533073 · Cited by 153 · cited 1×
“Switching to a perceptual hash function (which aims for invariance to small image changes) would lead to higher utility, but would not meaningfully prevent our poisoning attacks because an attacker could upload poisoned images that were adversarially modified to fool the perceptual hash [26,29,68].”
BackgroundPoisoning and Backdooring Contrastive LearningPreprint
Carlini, N., & Terzis, A. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09667 · Cited by 132 · cited 6×
“5% of poisoning success is orders-of-magnitude higher than what is required in prior poisoning attacks [15].”
BackgroundInfluentialSelf-citationTurner, A., Tsipras, D., & Mądry, A. (2018) · Cited by 125
Adversarial Neuron Pruning Purifies Backdoored Deep ModelsPreprint
Wu, D., & Wang, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.14430 · Cited by 125 · cited 1×
“…[113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66], [73], [81] that first presumes an adversary can modify a training dataset, and then asks (1) what impact this…”
Concealed Data Poisoning Attacks on NLP Models
Wallace, E., Zhao, T., Feng, S., & Singh, S. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.13 · Cited by 115 · cited 1×
“, a poisoning attack [6]) suffices to introduce targeted mistakes in model behavior [14, 15, 60, 76].”
BackgroundBagging Classifiers for Fighting Poisoning Attacks in Adversarial Classification Tasks
Biggio, B., Corona, I., Fumera, G., Giacinto, G., & Roli, F. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21557-5_37 · Cited by 111 · cited 1×
“…[11], [17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66], [73], [81] that first presumes an adversary can…”
Backdoor Attack in the Physical WorldPreprint
Li, Y., Zhai, T., Jiang, Y., Li, Z., & Xia, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.02361 · Cited by 105 · cited 1×
“Indeed there is an exceptionally vast literature [1], [3], [7], [10], [11], [17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29],…”
MetaPoison: Practical General-purpose Clean-label Data PoisoningPreprint
Huang, W., Geiping, J., Fowl, L., Taylor, G., & Goldstein, T. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.00225 · Cited by 102 · cited 1×
“…there is an exceptionally vast literature [1], [3], [7], [10], [11], [17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29],…”
NeuronInspect: Detecting Backdoors in Neural Networks via Output ExplanationsPreprint
Huang, X., Alzantot, M., & Srivastava, M. B. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.07399 · Cited by 100
REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets
Wang, A., Liu, A., Zhang, R., Kleiman, A., Kim, L., Zhao, D., Shirai, I., Narayanan, A., & Russakovsky, O. (2022) · International Journal of Computer Vision · doi:10.1007/s11263-022-01625-5 · Cited by 95 · cited 1×
“Possibly most important among these is that uncurated data has serious implications for fairness, bias, and ethics [8, 50, 77].”
BackgroundThe Do’s and Don’ts for CNN-Based Face Verification
Bansal, A., Castillo, C. D., Ranjan, R., & Chellappa, R. (2017) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2017.299 · Cited by 90 · cited 1×
“Critically, the maintainer does not provide a snapshot of the data associated with url i , due either to untenable storage costs [5], [6], [21], [43], privacy concerns [16], [98], or copyright limitations [19].”
BackgroundYou Autocomplete Me: Poisoning Vulnerabilities in Neural Code CompletionPreprint
Schuster, R., Song, C., Tromer, E., & Shmatikov, V. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2007.02220 · Cited by 87 · cited 1×
“, a poisoning attack [6]) suffices to introduce targeted mistakes in model behavior [14, 15, 60, 76].”
BackgroundSleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch
Souri, H., Fowl, L. H., Chellappa, R., Goldblum, M., & Goldstein, T. (2022) · neural information processing systems · doi:10.52202/068431-1393 · Cited by 85 · cited 1×
“…[1], [3], [7], [10], [11], [17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66], [73], [81] that first…”
Tramèr, F., Shokri, R., Joaquin, A. S., Le, H., Jagielski, M., Hong, S., & Carlini, N. (2022) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3548606.3560554 · Cited by 79
Self-citationBackdoor Scanning for Deep Neural Networks through K-Arm OptimizationPreprint
Shen, G., Liu, Y., Tao, G., An, S., Xu, Q., Cheng, S., Ma, S., & Zhang, X. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.05123 · Cited by 77
Backdooring and Poisoning Neural Networks with Image-Scaling Attacks
Quiring, E., & Rieck, K. (2020) · IEEE Security and Privacy Workshops (SPW) · doi:10.1109/spw50608.2020.00024 · Cited by 74 · cited 1×
“…[71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66], [73], [81] that first presumes an adversary can modify a training dataset, and then asks (1) what impact this could have, (2) if poisoning can be…”
What’s in the Box? An Analysis of Undesirable Content in the Common Crawl Corpus
Luccioni, A., & Viviano, J. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers) · doi:10.18653/v1/2021.acl-short.24 · Cited by 69 · cited 1×
“Many language datasets also contain similarly harmful “hate speech and sexually explicit content, even after filtering” [40].”
BackgroundTowards Inspecting and Eliminating Trojan Backdoors in Deep Neural Networks
Guo, W., Wang, L., Xu, Y. H., Xing, X., Du, M., & Song, D. X. (2020) · Proceedings · doi:10.1109/icdm50108.2020.00025 · Cited by 68 · cited 1×
“…[22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66], [73], [81] that first presumes an adversary can modify a training…”
Domain-Z: 28 Registrations Later Measuring the Exploitation of Residual Trust in Domains
Lever, C., Walls, R. J., Nadji, Y., Dagon, D., McDaniel, P. D., & Antonakakis, M. (2016) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2016.47 · Cited by 64
It's Not What It Looks Like: Manipulating Perceptual Hashing based Applications
Hao, Q., Luo, L., Jan, S. T. K., & Wang, G. (2021) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3460120.3484559 · Cited by 43 · cited 1×
“Switching to a perceptual hash function (which aims for invariance to small image changes) would lead to higher utility, but would not meaningfully prevent our poisoning attacks because an attacker could upload poisoned images that were adversarially modified to fool the perceptual hash [26,29,68].”
BackgroundPoisoning the Unlabeled Dataset of Semi-Supervised LearningPreprint
Carlini, N. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.01622 · Cited by 40 · cited 3×
“Recent work [14, 15] shows that arbitrarily poisoning only 0.”
BackgroundSelf-citationAnti-Distillation Backdoor Attacks: Backdoors Can Really Survive in Knowledge Distillation
Ge, Y., Wang, Q., Zheng, B., Zhuang, X., Li, Q., Shen, C. Q., & Wang, C. (2021) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/3474085.3475254 · Cited by 34
Red-Teaming the Stable Diffusion Safety FilterPreprint
Rando, J., Paleka, D., Lindner, D., Heim, L., & Tramèr, F. (2022) · arXiv (Cornell University) · doi:10.48550/arxiv.2210.04610 · Cited by 34 · cited 2×
“We consider two poisoning attack objectives: (1) cause a particular image to be misclassified as some target label from ImageNet, and (2) cause a particular image to be classified as NSFW by the Stable Diffusion Safety Filter [78].”
Self-citationPoisoning Deep Reinforcement Learning Agents with In-Distribution TriggersPreprint
Ashcraft, C., & Karra, K. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.07798 · Cited by 31 · cited 1×
“Indeed there is an exceptionally vast literature [1], [3], [7], [10], [11], [17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29],…”
(2016) · arXiv (Cornell University) · Cited by 29
Poison Forensics: Traceback of Data Poisoning Attacks in Neural NetworksPreprint
Shan, S., Bhagoji, A. N., Zheng, H.-T., & Zhao, B. Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.06904 · Cited by 27 · cited 1×
“…[99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29], [57], [65], [66], [73], [81] that first presumes an adversary can modify a training dataset, and then asks (1) what…”
Game of Registrars: An Empirical Analysis of Post-Expiration Domain Name Takeovers
Lauinger, T., Chaabane, A., Buyukkayhan, A. S., Onarlıoğlu, K., & Robertson, W. (2017) · Cited by 26
Contour: A Practical System for Binary Transparency
Al-Bassam, M., & Meiklejohn, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00305-0_8 · Cited by 23 · cited 1×
“While many downloaders are already open source, binary transparency would bolster protection to prevent selective inclusion of malicious modules [1].”
BackgroundMultimodal C4: An Open, Billion-scale Corpus of Images Interleaved with Text
Zhu, W., Hessel, J., Awadalla, A., Gadre, S. Y., Dodge, J., Fang, A., Yu, Y., Schmidt, L., Wang, W. Y., & Choi, Y. (2023) · neural information processing systems · doi:10.52202/075280-0393 · Cited by 21
Jain, S., Cretu, A.-M., & de Montjoye, Y.-A. (2021) · arXiv · doi:10.48550/arxiv.2106.09820 · Cited by 21 · cited 2×
“Switching to a perceptual hash function (which aims for invariance to small image changes) would lead to higher utility, but would not meaningfully prevent our poisoning attacks because an attacker could upload poisoned images that were adversarially modified to fool the perceptual hash [26,29,68].”
BackgroundThe Ghosts of Banking Past: Empirical Analysis of Closed Bank Websites
Moore, T., & Clayton, R. (2014) · Lecture notes in computer science · doi:10.1007/978-3-662-45472-5_3 · Cited by 20 · cited 1×
“Attackers have in the past targeted residual trust to defunct banking domains [61] and imported JavaScript libraries [67] to serve malware or steal user data, to take over email addresses associated with the domain [82], to control authoritative nameservers [52], or simply to serve ads [48].”
BackgroundThe Abandoned Side of the Internet: Hijacking Internet Resources When Domain Names Expire
Schlamp, J., Gustafsson, J., Wählisch, M., Schmidt, T. C., Carle, G., München, T. U., & Berlin, F. U. (2015) · Cited by 16
Venomave: Targeted Poisoning Against Speech Recognition
Aghakhani, H., Schönherr, L., Eisenhofer, T., Kolossa, D., Holz, T., Kruegel, C., & Vigna, G. (2023) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml54575.2023.00035 · Cited by 16 · cited 1×
“Indeed there is an exceptionally vast literature [1], [3], [7], [10], [11], [17], [18], [22], [31], [54], [62], [86], [99], [106], [113], [26], [30], [37], [55], [56], [71], [80], [88], [91], [94], [101], [102], [115] [9], [20], [28], [34], [38], [40], [53], [72], [89], [100], [109]–[111], [29],…”
Hidden Trigger Backdoor Attacks
Saha, A., Subramanya, A., & Pirsiavash, H. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i07.6871 · Cited by 13
Domains Do Change Their Spots: Quantifying Potential Abuse of Residual Trust
So, J. C. F., Miramirkhani, N., Ferdman, M., & Nikiforakis, N. (2022) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46214.2022.9833609 · Cited by 12 · cited 1×
“We note that attackers already routinely buy expired domains to hijack the residual trust attached with these domains [47], [52], [93].”
BackgroundJi, Y., & Gao, S. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 10
The Abandoned Side of the Internet: Hijacking Internet Resources When Domain Names Expire
Schlamp, J., Gustafsson, J., Wählisch, M., Schmidt, T. C., & Carle, G. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-17172-2_13 · Cited by 7
From Deletion to Re-Registration in Zero Seconds
Lauinger, T., Buyukkayhan, A. S., Chaabane, A., Robertson, W., & Kirda, E. (2018) · ACM Internet Measurement Conference (IMC) · doi:10.1145/3278532.3278560 · Cited by 5
Research works printed but not linked: 6
- Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota, June 2019. Association for Computational Linguistics.
- Yansong Gao, Change Xu, Derui Wang, Shiping Chen, Damith C Ranasinghe, and Surya Nepal. Strip: A defence against trojan attacks on deep neural networks. In Proceedings of the 35th Annual Computer Security Applications Conference, pages 113-125, 2019.
- Mandy Guo, Zihang Dai, Denny Vrandečić, and Rami Al-Rfou. Wiki-40B: Multilingual language model dataset. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 2440-2452, 2020.
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9, 2019.
- Bart Thomee, David A Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. YFCC100M: The new data in multimedia research. Communications of the ACM, 59(2):64-73, 2016.
- Florian Tramèr, Reza Shokri, Ayrton San Joaquin, Hoang Le, Matthew Jagielski, Sanghyun Hong, and Nicholas Carlini. Truth serum: Poisoning machine learning models to reveal their secrets. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, pages 2779-2792, 2022.
Cited by
Sorted by citations
Medical large language models are vulnerable to data-poisoning attacks
Alber, D. A., Yang, Z., Alyakin, A., Yang, E., Rai, S., Valliani, A. A., Zhang, J., Rosenbaum, G. R., Amend-Thomas, A. K., Kurland, D. B., et al. (2025) · Nature Medicine · doi:10.1038/s41591-024-03445-1 · Cited by 242
“https://doi.org/10.1038/s41591-024-03445-1 previous studies exploring data poisoning 6,22,23 to the high-risk medical domain by examining the harm potential of practical data-poisoning attacks not requiring direct access to model weights, instead relying on misinformation uploaded to the Internet…”
InfluentialAdversarial machine learning :
Vassilev, A., Oprea, A., Fordyce, A. J., & Anderson, H. S. (2024) · doi:10.6028/nist.ai.100-2e2023 · Cited by 230
Self-citationThe Curse of Recursion: Training on Generated Data Makes Models Forget
Shumailov, I., Shumaylov, Z., Zhao, Y., Gal, Y., Papernot, N., & Anderson, R. (2023) · arXiv · doi:10.48550/arxiv.2305.17493 · Cited by 151
“Recent studies have demonstrated that web-scale datasets can be poisoned by introducing malicious data into a small percentage of samples [Carlini and Terzis, 2021, Carlini et al., 2023].”
BackgroundRethinking machine unlearning for large language models
Liu, S., Yao, Y., Jia, J., Casper, S., Baracaldo, N., Hase, P., Yao, Y., Liu, C. Y., Xu, X., Li, H., et al. (2025) · Nature Machine Intelligence · doi:10.1038/s42256-025-00985-0 · Cited by 374
Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models
Shan, S., Ding, W., Passananti, J., Wu, S. L., Zheng, H.-T., & Zhao, B. Y. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00207 · Cited by 166
“This creates the possibility of data poisoning attacks [30].”
Background
Metadata
- Authors
- Carlini, N., Jagielski, M., Choquette-Choo, C. A., Paleka, D., Pearce, W., Anderson, H., Terzis, A., Thomas, K., & Tramèr, F.
- Affiliations
- Google DeepMind (United Kingdom) (GB)
- ETH Zurich (CH)
- Nvidia (United Kingdom) (GB)
- Google (United States) (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2023
- Pages
- 407-425
- ISSN
- 1081-6011 (ISSN-L)
- arXiv
- 2302.10149
- OpenAlex
- W4402264526
- Semantic Scholar
- 2cf43a61d0937ad25f23eaef7c90253ab799b3c7
- Cited by
- 138
- References
- 112
- Citations by year
- 2026: 53 · 2025: 44 · 2024: 29 · 2023: 12
- FWCI
- 37.13
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE Symposium on Security and Privacy (S&P) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Privacy-Preserving Technologies in Data
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Digital Media Forensic Detection
- Keywords
- deep learning security, Wikipedia, crowdsourced data
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
