Imitation Attacks and Defenses for Black-box Machine Translation Systems
Open accessAbstract
Adversaries may look to steal or attack blackbox NLP systems, either for financial gain or to exploit model errors.One setting of particular interest is machine translation (MT), where models have high commercial value and errors can be costly.We investigate possible exploitations of black-box MT systems and explore a preliminary defense against such threats.We first show that MT systems can be stolen by querying them with monolingual sentences and training models to imitate their outputs.Using simulated experiments, we demonstrate that MT model stealing is possible even when imitation models have different input data or architectures than their target models.Applying these ideas, we train imitation models that reach within 0.6 BLEU of three production MT systems on both high-resource and low-resource language pairs.We then leverage the similarity of our imitation models to transfer adversarial examples to the production systems.We use gradient-based attacks that expose inputs which lead to semanticallyincorrect translations, dropped content, and vulgar model outputs.To mitigate these vulnerabilities, we propose a defense that modifies translation outputs in order to misdirect the optimization of imitation models.This defense degrades the adversary's BLEU score and attack success rate at some cost in the defender's BLEU and inference speed.Transfer Solve Eq. (2) Save me it's over 100°F Phase One: Model Imitation German
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Extracting reasoning yields a substantially more effective form of capability stealing than output-only distillation because a reasoning trace reveals the intermediate solution trajectory and supplies a far denser supervision signal.
“Extracting reasoning matters not because output-only distillation is ineffective, but because reasoning yields a substantially more effective form of capability stealing.”
From Stealing Reasoning Traces from Proprietary LLM APIs (2026) · cites this paper as Wallace et al., 2020
The 'Past Work on Distillation and Stealing' section describes training an imitation model on a victim's translations, and 'Data Efficiency' reports that such output-based stealing can succeed. That supports the sentence's premise that output-only imitation is not ineffective. The paper does not examine reasoning traces or compare reasoning extraction with output-only imitation, so it does not support the claimed advantage. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344 · see citing works
Distilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204 · see citing works
WordNet: a lexical database for English
Miller, G. A. (1995) · Communications of the ACM · doi:10.1145/219717.219748 · Cited by 14,015 · see citing works
Explaining and Harnessing Adversarial Examples
Goodfellow, I. J., Shlens, J., & Szegedy, C. (2014) · arXiv · doi:10.48550/arxiv.1412.6572 · Cited by 10,381 · see citing works
Intriguing properties of neural networks
Szegedy, C., Zaremba, W., Sutskever, I., Bruna, J., Erhan, D., Goodfellow, I., & Fergus, R. (2013) · arXiv · doi:10.48550/arxiv.1312.6199 · Cited by 8,143 · see citing works
Kudo, T., & Richardson, J. (2018) · Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d18-2012 · Cited by 4,454 · see citing works
fairseq: A Fast, Extensible Toolkit for Sequence Modeling
Ott, M., Edunov, S., Baevski, A., Fan, A., Gross, S., Ng, N., Grangier, D., & Auli, M. (2019) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n19-4009 · Cited by 3,538 · see citing works
Universal Adversarial Perturbations
Moosavi-Dezfooli, S.-M., Fawzi, A., Fawzi, O., & Frossard, P. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.17 · Cited by 2,680 · see citing works
A Call for Clarity in Reporting BLEU Scores
Post, M. (2018) · Proceedings of the Third Conference on Machine Translation: Research Papers · doi:10.18653/v1/w18-6319 · Cited by 2,112 · see citing works
5.7 The WORDNET lexical database of English
Ooi, V. B. Y. (2019) · Edinburgh University Press eBooks · doi:10.1515/9781474471459-050 · Cited by 2,034 · see citing works
Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples
Papernot, N., McDaniel, P., & Goodfellow, I. (2016) · arXiv · doi:10.48550/arxiv.1605.07277 · Cited by 1,326 · see citing works
Delving into Transferable Adversarial Examples and Black-box Attacks
Liu, Y., Chen, X., Liu, C., & Song, D. (2016) · arXiv · doi:10.48550/arxiv.1611.02770 · Cited by 1,310 · see citing works
Self-citationThe Curious Case of Neural Text Degeneration
Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09751 · Cited by 1,304 · see citing works
Hierarchical Neural Story Generation
Fan, A., Lewis, M., & Dauphin, Y. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1082 · Cited by 1,266 · see citing works
Hierarchical Neural Story Generation
Fan, A., Lewis, M., & Dauphin, Y. (2018) · arXiv · doi:10.48550/arxiv.1805.04833 · Cited by 1,242 · see citing works
Sequence-Level Knowledge Distillation
Kim, Y., & Rush, A. M. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1139 · Cited by 836 · see citing works
Born Again Neural NetworksPreprint
Furlanello, T., Lipton, Z. C., Tschannen, M., Itti, L., & Anandkumar, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.04770 · Cited by 798 · see citing works
Lowd, D., & Meek, C. (2005) · Proceedings of the eleventh ACM SIGKDD international conference on Knowledge discovery in data mining · doi:10.1145/1081870.1081950 · Cited by 755 · see citing works
Universal Adversarial Triggers for Attacking and Analyzing NLP
Wallace, E., Feng, S., Kandpal, N., Gardner, M., & Singh, S. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1221 · Cited by 642 · see citing works
Self-citationProtecting Intellectual Property of Deep Neural Networks with Watermarking
Zhang, J., Gu, Z., Jang, J., Wu, H., Stoecklin, M. P., Huang, H., & Molloy, I. (2018) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3196494.3196550 · Cited by 556 · see citing works
Knockoff Nets: Stealing Functionality of Black-Box Models
Orekondy, T., Schiele, B., & Fritz, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00509 · Cited by 549 · see citing works
Evaluating Gender Bias in Machine Translation
Stanovsky, G., Smith, N. A., & Zettlemoyer, L. (2019) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1164 · Cited by 492 · see citing works
(2020) · arXiv (Cornell University) · Cited by 461 · see citing works
PRADA: Protecting Against DNN Model Stealing Attacks
Juuti, M., Szyller, S., Marchal, S., & Asokan, N. (2019) · 2019 IEEE European Symposium on Security and Privacy (EuroS&P) · doi:10.1109/eurosp.2019.00044 · Cited by 440 · cited 1× · see citing works
“For example, Juuti et al. (2019) assume adversaries will make consecutive out-of-distribution queries and can thus be detected.”
BackgroundDiverse Beam Search: Decoding Diverse Solutions from Neural Sequence ModelsPreprint
Vijayakumar, A. K., Cogswell, M., Selvaraju, R. R., Sun, Q., Lee, S., Crandall, D., & Batra, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.02424 · Cited by 350 · see citing works
Insertion Transformer: Flexible Sequence Generation via Insertion OperationsPreprint
Stern, M., Chan, W., Kiros, J., & Uszkoreit, J. (2019) · International Conference on Machine Learning · doi:10.48550/arxiv.1902.03249 · Cited by 300 · see citing works
Self-citationMultilingual Neural Machine Translation with Knowledge DistillationPreprint
Tan, X., Ren, Y., He, D., Qin, T., Zhao, Z., & Liu, T.-Y. (2019) · International Conference on Learning Representations · doi:10.48550/arxiv.1902.10461 · Cited by 270 · see citing works
Guzmán, F., Chen, P., Ott, M., Pino, J., Lample, G., Koehn, P., Chaudhary, V., & Ranzato, M. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1632 · Cited by 231 · see citing works
Certified Robustness to Adversarial Word Substitutions
Jia, R., Raghunathan, A., Göksel, K., & Liang, P. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1423 · Cited by 225 · see citing works
Seq2Sick: Evaluating the Robustness of Sequence-to-Sequence Models with Adversarial Examples
Cheng, M., Yi, J., Chen, P., Zhang, H., & Hsieh, C. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i04.5767 · Cited by 218 · see citing works
A causal framework for explaining the predictions of black-box sequence-to-sequence models
Alvarez-Melis, D., & Jaakkola, T. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1042 · Cited by 185 · see citing works
On Adversarial Examples for Character-Level Neural Machine TranslationPreprint
Ebrahimi, J., Lowd, D., & Dou, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.09030 · Cited by 172 · see citing works
Understanding Knowledge Distillation in Non-autoregressive Machine TranslationPreprint
Zhou, C., Neubig, G., & Gu, J. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.02727 · Cited by 144 · see citing works
Self-Distillation Amplifies Regularization in Hilbert SpacePreprint
Mobahi, H., Farajtabar, M., & Bartlett, P. L. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.05715 · Cited by 128 · see citing works
Achieving Verified Robustness to Symbol Substitutions via Interval Bound Propagation
Huang, P.-S., Stanforth, R., Welbl, J., Dyer, C., Yogatama, D., Gowal, S., Dvijotham, K., & Kohli, P. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1419 · Cited by 116 · see citing works
Prediction Poisoning: Towards Defenses Against DNN Model Stealing AttacksPreprint
Orekondy, T., Schiele, B., & Fritz, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.10908 · Cited by 116 · see citing works
Thieves on Sesame Street! Model Extraction of BERT-based APIs
Krishna, K., Tomar, G. S., Parikh, A. P., Papernot, N., & Iyyer, M. (2019) · arXiv · doi:10.48550/arxiv.1910.12366 · Cited by 111 · see citing works
On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models
Michel, P., Li, X., Neubig, G., & Pino, J. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1314 · Cited by 108 · see citing works
REFIT: A Unified Watermark Removal Framework For Deep Learning Systems With Limited Data
Chen, X., Wang, W., Bender, C., Ding, Y., Jia, R., Li, B., & Song, D. (2021) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3433210.3453079 · Cited by 100 · see citing works
Self-citationExploring Connections Between Active Learning and Model ExtractionPreprint
Chandrasekaran, V., Chaudhuri, K., Giacomelli, I., Jha, S., & Yan, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.02054 · Cited by 99 · see citing works
Hisamoto, S., Post, M., & Duh, K. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00299 · Cited by 86 · see citing works
DAWN: Dynamic Adversarial Watermarking of Neural NetworksPreprint
Szyller, S., Atli, B. G., Marchal, S., & Asokan, N. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1906.00830 · Cited by 58 · see citing works
From Research to Production and Back: Ludicrously Fast Neural Machine Translation
Kim, Y. J., Junczys-Dowmunt, M., Hassan, H., Aji, A. F., Heafield, K., Grundkiewicz, R., & Bogoychev, N. (2019) · Workshop on Neural Generation and Translation (WNGT) · doi:10.18653/v1/d19-5632 · Cited by 56 · see citing works
Adding Robustness to Support Vector Machines Against Adversarial Reverse Engineering
Alabdulmohsin, I., Gao, X. D., & Zhang, X. (2014) · doi:10.1145/2661829.2662047 · Cited by 37 · see citing works
A framework for the extraction of Deep Neural Networks by leveraging public dataPreprint
Pal, S., Gupta, Y., Shukla, A., Kanade, A., Shevade, S., & Ganapathy, V. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1905.09165 · Cited by 36 · see citing works
Defending Against Machine Learning Model Stealing Attacks Using Deceptive PerturbationsPreprint
Lee, T., Edwards, B., Molloy, I., & Su, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.00054 · Cited by 30 · see citing works
Constant-Time Machine Translation with Conditional Masked Language Models.Preprint
Ghazvininejad, M., Levy, O., Liu, Y., & Zettlemoyer, L. (2019) · arXiv (Cornell University) · Cited by 28 · see citing works
Exploring the Robustness of NMT Systems to Nonsensical InputsPreprint
Chaturvedi, A., Kp, A., & Garain, U. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.01165 · Cited by 8 · see citing works
Knowing When to Stop: Evaluation and Verification of Conformity to Output-Size Specifications
Wang, C., Bunel, R., Dvijotham, K., Huang, P.-S., Grefenstette, E., & Kohli, P. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.01254 · Cited by 7 · see citing works
Flagged first, then by citations
5 of the 55 references couldn’t be matched to a known paper.
Research works printed but not linked: 4
- Varun Chandrasekaran, K Chaudhari, Irene Giacomelli, Somesh Jha, and Songbai Yan. 2020. Exploring con- nections between active learning and model extrac- tion. In USENIX Security Symposium.
- Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hi- erarchical neural story generation. In ACL.
- Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Improving neural machine translation models with monolingual data. In ACL.
- Florian Tramèr, Fan Zhang, Ari Juels, Michael K Re- iter, and Thomas Ristenpart. 2016. Stealing ma- chine learning models via prediction APIs. In USENIX Security Symposium.
Cited by
Sorted by citations
Adversarial examples for models of code
Yefet, N., Alon, U., & Yahav, E. (2020) · Proceedings of the ACM on Programming Languages · doi:10.1145/3428230 · Cited by 171 · see citing works
Wang, Y., Sun, T. S., Li, S., Yuan, X., Ni, W., Hossain, E., & Poor, H. V. (2023) · IEEE Communications Surveys & Tutorials · doi:10.1109/comst.2023.3319492 · Cited by 207 · see citing works
A Survey of Adversarial Defenses and Robustness in NLP
Goyal, S., Doddapaneni, S., Khapra, M. M., & Ravindran, B. (2023) · ACM Computing Surveys · doi:10.1145/3593042 · Cited by 147 · see citing works
Hidden Backdoors in Human-Centric Language Models
Li, S., Liu, H., Dong, T., Zhao, B. Z. H., Xue, M., Zhu, H., & Lu, J. (2021) · Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/3460120.3484576 · Cited by 119 · see citing works
Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey
Saunders, D. (2022) · Journal of Artificial Intelligence Research · doi:10.1613/jair.1.13566 · Cited by 96 · see citing works
Model Extraction and Adversarial Transferability, Your BERT is Vulnerable!
He, X., Lyu, L., Sun, L., & Xu, Q. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.161 · Cited by 71 · see citing works
Protecting Intellectual Property of Language Generation APIs with Lexical Watermark
He, X., Xu, Q., Lyu, L., Wu, F., & Wang, C. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21321 · Cited by 56 · see citing works
The False Promise of Imitating Proprietary LLMs
Gudibande, A., Wallace, E., Snell, C., Geng, X., Liu, H., Abbeel, P., Levine, S., & Song, D. (2023) · arXiv · doi:10.48550/arxiv.2305.15717 · Cited by 48 · see citing works
“In this work, we study one possible resolution to this question: model imitation (Wallace et al., 2020; Orekondy et al., 2019).”
BackgroundMethodInfluentialSelf-citationDemystifying the Transferability of Adversarial Attacks in Computer Networks
Nowroozi, E., Mekdad, Y., Berenjestanaki, M. H., Conti, M., & Fergougui, A. E. (2022) · IEEE Transactions on Network and Service Management · doi:10.1109/tnsm.2022.3164354 · Cited by 46 · see citing works
Mix and Match: Learning-free Controllable Text Generationusing Energy Language Models
Mireshghallah, F., Goyal, K., & Berg-Kirkpatrick, T. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.31 · Cited by 47 · see citing works
Winsorization for Robust Bayesian Neural Networks
Sharma, S., & Chatterjee, S. (2021) · Entropy · doi:10.3390/e23111546 · Cited by 36 · see citing works
Crafting Adversarial Examples for Neural Machine Translation
Zhang, X., Zhang, J., Chen, Z., & He, K. (2021) · doi:10.18653/v1/2021.acl-long.153 · Cited by 38 · see citing works
Backdoor Attack and Defense on Deep Learning: A Survey
Bai, Y., Xing, G., Wu, H., Rao, Z., Ma, C., Wang, S., Liu, X., Zhou, Y., Tang, J., Huang, K., et al. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3482723 · Cited by 36 · see citing works
Adversarial Machine Learning: Attacks From Laboratories to the Real World
Lin, H.-Y., & Biggio, B. (2021) · Computer · doi:10.1109/mc.2021.3057686 · Cited by 32 · see citing works
Red Teaming Language Model Detectors with Language Models
Shi, Z., Wang, Y., Yin, F., Chen, X., Chang, K.-W., & Hsieh, C. (2024) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00639 · Cited by 39 · see citing works
Adversarial Semantic Collisions
Song, C., Rush, A. M., & Shmatikov, V. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.344 · Cited by 34 · see citing works
ModelShield: Adaptive and Robust Watermark Against Model Extraction Attack
Pang, K., Qi, T., Wu, C., Bai, M., Jiang, M., & Huang, Y. (2025) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2025.3530691 · Cited by 29 · see citing works
Han, X., Kumar, S., & Tsvetkov, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.647 · Cited by 41 · see citing works
Controlled Text Generation as Continuous Optimization with Multiple ConstraintsPreprint
Kumar, S., Malmi, E., Severyn, A., & Tsvetkov, Y. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.01850 · Cited by 29 · see citing works
Gradient-based Constrained Sampling from Language Models
Kumar, S., Paria, B., & Tsvetkov, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.144 · Cited by 28 · see citing works
A Targeted Attack on Black-Box Neural Machine Translation with Parallel Data Poisoning
Xu, C., Wang, J., Tang, Y., Guzmán, F., Rubinstein, B. I. P., & Cohn, T. (2021) · International World Wide Web Conference (WWW) · doi:10.1145/3442381.3450034 · Cited by 28 · see citing works
Liu, J., Kang, Y., Tang, D., Song, K., Sun, C., Wang, X., Lu, W. P., & Liu, X. (2022) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3548606.3560683 · Cited by 28 · see citing works
Designing Learning Media Using Augmented Reality for Engineering Mechanics Course
Fortuna, A., Waskito, Purwantono, Kurniawan, A., Andriani, W., & Alimin, M. (2023) · Journal of Engineering Researcher and Lecturer · doi:10.58712/jerel.v2i1.20 · Cited by 23 · see citing works
On Extracting Specialized Code Abilities from Large Language Models: A Feasibility Study
Li, Z.-J., Wang, C.-Z., Ma, P., Liu, C., Wang, S., Wu, D.-Y., Gao, C., & Liu, Y. (2023) · International Conference on Software Engineering · doi:10.1145/3597503.3639091 · Cited by 49 · see citing works
Wang, J., Xu, C., Guzmán, F., El-Kishky, A., Tang, Y., Rubinstein, B. I. P., & Cohn, T. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.127 · Cited by 21 · see citing works
Model-Based Robust Deep Learning: Generalizing to Natural, Out-of-Distribution DataPreprint
Robey, A., Hassani, H., & Pappas, G. J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2005.10247 · Cited by 37 · see citing works
Stealing the Decoding Algorithms of Language Models
Naseh, A., Krishna, K., Iyyer, M., & Houmansadr, A. (2023) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3576915.3616652 · Cited by 17 · see citing works
White-Box Multi-Objective Adversarial Attack on Dialogue Generation
Li, Y., Li, Z., Gao, Y., & Liu, C. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.100 · Cited by 18 · see citing works
Single-node attacks for fooling graph neural networks
Finkelshtein, B., Baskin, C., Zheltonozhskii, E., & Alon, U. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.09.115 · Cited by 15 · see citing works
GrOVe: Ownership Verification of Graph Neural Networks using Embeddings
Waheed, A., Duddu, V., & Asokan, N. (2024) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp54263.2024.00050 · Cited by 15 · see citing works
Token-modification adversarial attacks for natural language processing: A survey
Roth, T., Gao, Y., Abuadbba, A., Nepal, ., & Liu, W. (2024) · AI Communications · doi:10.3233/aic-230279 · Cited by 17 · see citing works
Solano, J., López, C., Rivera, E., Castelblanco, A., Tengana, L., & Ochoa, M. (2020) · ACM Workshop on Artificial Intelligence and Security (AISec) · doi:10.1145/3411508.3421378 · Cited by 13 · see citing works
Transferable adversarial examples can efficiently fool topic models
Wang, Z., Zheng, Y., Zhu, H., Yang, C., & Chen, T. (2022) · Computers & Security · doi:10.1016/j.cose.2022.102749 · Cited by 12 · see citing works
CROSS: A framework for cyber risk optimisation in smart homes
Zhang, Y., Malacaria, P., Loukas, G., & Panaousis, E. (2023) · Computers & Security · doi:10.1016/j.cose.2023.103250 · Cited by 12 · see citing works
Iterative Adversarial Attack on Image-Guided Story Ending Generation
Wang, Y., Hu, W., & Hong, R. (2023) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2023.3345167 · Cited by 11 · see citing works
Are GPT Embeddings Useful for Ads and Recommendation?
Peng, W., Xu, D., Xu, T., Zhang, J., & Chen, E. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-40292-0_13 · Cited by 10 · see citing works
Attack and defense techniques in large language models: A survey and new perspectives
Liao, Z., Chen, K., Lin, Y., Li, K., Liu, Y., Chen, H., Huang, X., & Yu, Y. (2025) · Neural Networks · doi:10.1016/j.neunet.2025.108388 · Cited by 19 · see citing works
Practical and Efficient Model Extraction of Sentiment Analysis APIs
Wu, W., Zhang, J., Wei, V. J., Chen, X., Zheng, Z., King, I., & Lyu, M. R.-T. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00054 · Cited by 10 · see citing works
A Survey on Out-of-Distribution Evaluation of Neural NLP Models
Li, X., Liu, M., Gao, S. B., & Buntine, W. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/749 · Cited by 12 · see citing works
Wang, J., Bao, R., Zhang, Z., & Zhao, H. Y. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.73 · Cited by 8 · see citing works
Targeted Adversarial Attacks Against Neural Machine Translation
Sadrizadeh, S., Aghdam, A. D., Dolamic, L., & Pascal Frossard, B. (2023) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49357.2023.10095342 · Cited by 7 · see citing works
Distillation-Resistant Watermarking for Model Protection in NLP
Zhao, X., Li, L., & Wang, Y. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.370 · Cited by 7 · see citing works
López, C., Solano, J., Rivera, E., Tengana, L., Florez-Lozano, J., Castelblanco, A., & Ochoa, M. (2023) · International Journal of Information Security · doi:10.1007/s10207-023-00711-0 · Cited by 7 · see citing works
Extracted BERT Model Leaks More Information than You Think!
He, X., Lyu, L., Chen, C., & Xu, Q. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.99 · Cited by 10 · see citing works
Zhang, Z., Setty, V., & Anand, A. (2022) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3477495.3531677 · Cited by 6 · see citing works
Security and privacy in LLMs: A comprehensive survey of threats and mitigation strategies
Berini, A. D. E., Jamil, N., Benrazek, A., Lakas, A., Ismail, L., Ferrag, M. A., & Lam, K. (2026) · Information Fusion · doi:10.1016/j.inffus.2026.104241 · Cited by 9 · see citing works
CycleGAN-Gradient Penalty for Enhancing Android Adversarial Malware Detection in Gray Box Setting
Atedjio, F. S., Lienou, J.-P., Nelson, F. F., Shetty, S. S., & Kamhoua, C. (2024) · IEEE Access · doi:10.1109/access.2024.3486734 · Cited by 5 · see citing works
A novel watermarking framework for intellectual property protection of NLG APIs
Li, M., Wu, H., & Zhang, X. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126700 · Cited by 7 · see citing works
Szyller, S., Duddu, V., Gröndahl, T., & Asokan, N. (2021) · arXiv (Cornell University) · Cited by 5 · see citing works
Anchor Points: Benchmarking Models with Much Fewer Examples
Vivek, R., Ethayarajh, K., Yang, D., & Kiela, D. (2024) · Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.eacl-long.95 · Cited by 4 · see citing works
Anchor Points: Benchmarking Models with Much Fewer Examples
Vivek, R., Ethayarajh, K., Yang, D., & Kiela, D. (2023) · arXiv · doi:10.48550/arxiv.2309.08638 · Cited by 4
An Effective Framework for Intellectual Property Protection of NLG Models
Li, M., Wang, Z., & Zhang, X. (2023) · Symmetry · doi:10.3390/sym15061287 · Cited by 4 · see citing works
Security Challenges in Natural Language Processing Models
Xu, Q., & He, X. (2023) · doi:10.18653/v1/2023.emnlp-tutorial.2 · Cited by 4 · see citing works
Query-efficient model extraction for text classification model in a hard label setting
Peng, H., Guo, S., Zhao, D., Wu, Y., Han, J., Wang, Z., Ji, S., & Zhong, M. J. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2023.02.019 · Cited by 3 · see citing works
Make Text Unlearnable: Exploiting Effective Patterns to Protect Personal Data
Li, X., & Liu, M. (2023) · Workshop on Trustworthy Natural Language Processing (TrustNLP) · doi:10.18653/v1/2023.trustnlp-1.22 · Cited by 4 · see citing works
What Models Know About Their Attackers: Deriving Attacker Information From Latent Representations
Xie, Z., Brophy, J., Noack, A., You, W., Asthana, K., Perkins, C., Reis, S., Hammoudeh, Z., Lowd, D., & Singh, S. (2021) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2021.blackboxnlp-1.6 · Cited by 3 · see citing works
Killing Two Birds with One Stone: Stealing Model and Inferring Attribute from BERT-based APIs.Preprint
Lyu, L., He, X., Wu, F., & Sun, L. (2021) · arXiv (Cornell University) · Cited by 4 · see citing works
Data protection in the era of large language models: challenges and opportunities
Revesai, Z., & Chipfumbu, C. T. (2026) · AI and Ethics · doi:10.1007/s43681-026-01223-y · Cited by 5 · see citing works
Rethinking Targeted Adversarial Attacks for Neural Machine Translation
Wu, J., Liu, L., Bi, W., & Yeung, D. (2024) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp48485.2024.10447368 · Cited by 2 · see citing works
Towards Variable-Length Textual Adversarial AttacksPreprint
Guo, J., Zhang, Z., Zhang, L., Xu, L., Chen, B., Chen, E., & Luo, W. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08139 · Cited by 3 · see citing works
Fully Synthetic Data Improves Neural Machine Translation with Knowledge DistillationPreprint
Aji, A. F., & Heafield, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.15455 · Cited by 2 · see citing works
Sampling and Filtering of Neural Machine Translation Distillation Data
Zouhar, V. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop (NAACL SRW) · doi:10.18653/v1/2021.naacl-srw.1 · Cited by 1 · see citing works
Liu, Z., Huang, P., Xu, Z., Li, X., Liu, S., Peng, C., Xin, H., Yan, Y., Wang, S., Han, X., et al. (2026) · AI Open · doi:10.1016/j.aiopen.2026.02.002 · Cited by 1 · see citing works
Li, J., Wang, J., Zhao, Z., & Wei, Z. (2026) · Computer Standards & Interfaces · doi:10.1016/j.csi.2026.104181 · Cited by 1 · see citing works
Imitation Attacks Can Steal More Than You Think from Machine Translation Systems
Hu, T., Zhang, P., Yang, B., Xie, J., & Wang, R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44693-1_32 · Cited by 1 · see citing works
How to Steal Reasoning Without Reasoning Traces
Zhang, T., Morris, J. X., & Shmatikov, V. (2026) · arXiv · doi:10.48550/arxiv.2603.07267
“In this paper, we focus on the setting where the teacher’s reasoning traces are not available .”
Stealing Reasoning Traces from Proprietary LLM APIs
Panfilov, A., Schmotz, D., Shumailov, I., Beurer-Kellner, L., Schaeffer, J., Prabhu, A., Geiping, J., & Andriushchenko, M. (2026) · arXiv · doi:10.48550/arxiv.2608.09867
Leveraging Extracted Model Adversaries for Improved Black Box Attacks
Nizar, N. J., & Kobren, A. (2020) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2020.blackboxnlp-1.6
CRank: Reusable Word Importance Ranking for Text Adversarial Attack
Chen, X., & Liu, B. (2021) · Applied Sciences · doi:10.3390/app11209570
Probabilistic Inverse Modeling: An Application in Hydrology
Sharma, S., Ghosh, R., Renganathan, A., Li, X., Chatterjee, S., Nieber, J. L., Duffy, C. J., & Kumar, V. (2023) · Society for Industrial and Applied Mathematics eBooks · doi:10.1137/1.9781611977653.ch95
Pulling Out All The Full Stops: Punctuation Sensitivity in Neural Machine Translation and Evaluation
Jwalapuram, P. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.381
StyleRec: A Benchmark Dataset for Prompt Recovery in Writing Style Transformation
Liu, S., Gao, Y., Zhai, S., & Wang, L. (2024) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata62323.2024.10825143
Securing NLP Systems: A Comprehensive AI-Based Approach
Huang, J.-J., Yang, C. T., & Wang, W.-H. (2025) · Smart innovation, systems and technologies · doi:10.1007/978-981-97-7786-0_12
Roy, S. K., & Shanbhag, N. R. (2024) · doi:10.1145/3676536.3676685
An imperceptible attack on machine translation system and the countermeasure
Jiang, J., Yao, X., & Ouyang, B. (2025) · Neural Computing and Applications · doi:10.1007/s00521-025-11308-0
GAMeta-Prompt: A Generative Adversarial Prompt-Based Meta-language Reasoning Enhancement Method
Qin, X., Hu, Y. J., Ma, C., & Tian, M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04630-7_11
An Ensemble Defense Method Against False Data in Structured Preference Learning
Zhang, M., Feng, S., Meng, Z., Liu, B., & Liu, Z. (2026) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp55912.2026.11463790
An LLM-Enabled End-to-End Attack Pipeline Against Speech-Based Machine Translation
Zhao, J., Liu, Z., Yao, Z., Li, F., & Luo, B. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-38702-8_20
- Authors
- Wallace, E., Stern, M., & Song, D.
- Affiliations
- University of California, Berkeley (US)
- ORCID
- Song, Dawn
- Venue
- Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- Published
- 2020
- Pages
- 5531-5546
- OpenAlex
- W3099729825
- Cited by
- 75
- References
- 55
- Citations by year
- 2026: 6 · 2025: 5 · 2024: 9 · 2023: 21 · 2022: 12 · 2021: 15 · 2020: 7
- FWCI
- 5.25
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Keywords
- model theft, imitation learning, adversarial attacks, gradient-based attacks, adversarial examples, attack defense, low-resource language pairs
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
