Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attack

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173

GAN（Generative Adversarial Nets）

淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 20,507

Bag of Tricks for Efficient Text Classification

Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545

Large Scale GAN Training for High Fidelity Natural Image Synthesis

Brock, A., Donahue, J., & Simonyan, K. (2018) · arXiv · doi:10.48550/arxiv.1809.11096 · Cited by 3,717

SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)

Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., & Kumar, R. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2010 · Cited by 816

Anyone Can Become a Troll

Cheng, J., Bernstein, M. S., Danescu-Niculescu-Mizil, C., & Leskovec, J. (2017) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2998181.2998213 · Cited by 499

The Second Conversational Intelligence Challenge (ConvAI2)

Dinan, E., Logacheva, V., Malykh, V., Miller, A., Shuster, K., Urbanek, J., Kiela, D., Szlam, A. D., Serban, I. V., Lowe, R., et al. (2019) · The Springer series on challenges in machine learning · doi:10.1007/978-3-030-29135-8_7 · Cited by 395

Deceiving Google's Perspective API Built for Detecting Toxic CommentsPreprint

Hosseini, H., Kannan, S., Zhang, B.-S., & Poovendran, R. (2017) · arXiv.org · doi:10.48550/arxiv.1702.08138 · Cited by 384

Beyond vandalism: Wikipedia trolls

Shachaf, P., & Hara, N. (2010) · Journal of Information Science · doi:10.1177/0165551510365390 · Cited by 357

Automated Hate Speech Detection and the Problem of Offensive LanguagePreprint

Davidson, T., Warmsley, D., Macy, M. W., & Weber, I. G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.04009 · Cited by 284

A First Look at Toxicity Injection Attacks on Open-domain Chatbots

Weeks, C., Cheruvu, A., Abdullah, S. M., Kanchi, S., Yao, D. D., & Viswanath, B. (2023) · Annual Computer Security Applications Conference · doi:10.1145/3627106.3627122 · Cited by 10

Zero-Shot Fine-Grained Style Transfer: Leveraging Distributed Continuous Style Representations to Transfer To Unseen StylesPreprint

Smith, E. M., Gonzalez-Rico, D., Dinan, E., & Boureau, Y.-L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.03914 · Cited by 11

Defense against adversarial attacks via textual embeddings based on semantic associative field

Huang, J., & Chen, L. (2023) · Neural Computing and Applications · doi:10.1007/s00521-023-08946-7 · Cited by 10

Revisiting Contextual Toxicity Detection in Conversations

Anuchitanukul, A., Ive, J., & Specia, L. (2022) · Journal of Data and Information Quality · doi:10.1145/3561390 · Cited by 10

ASSERT: Automated Safety Scenario Red Teaming for Evaluating the Robustness of Large Language Models

Mei, A., Levy, S., & Wang, W. Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.388 · Cited by 24

Improving Multi-label Malevolence Detection in Dialogues through Multi-faceted Label Correlation Enhancement

Zhang, Y., Ren, P., Deng, W., Chen, Z., & de Rijke, M. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.248 · Cited by 9

Robustness of models addressing Information Disorder: A comprehensive review and benchmarking study

Fenza, G., Loia, V., Stanzione, C., & Gisi, M. D. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.127951 · Cited by 11

The R-U-A-Robot Dataset: Helping Avoid Chatbot Deception by Detecting User Questions About Human or Non-Human Identity

Gros, D., Li, Y., & Yu, Z. (2021) · doi:10.18653/v1/2021.acl-long.544 · Cited by 15

The CRINGE Loss: Learning what language not to model

Adolphs, L., Gao, T., Xu, J., Shuster, K., Sukhbaatar, S., & Weston, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.493 · Cited by 30

Beyond plain toxic: building datasets for detection of flammable topics and inappropriate statements

Babakov, N., Logacheva, V., & Panchenko, A. (2023) · Language Resources and Evaluation · doi:10.1007/s10579-023-09682-z · Cited by 16

CoSafe: Evaluating Large Language Model Safety in Multi-Turn Dialogue Coreference

Yu, E., Li, J., Liao, M., Wang, S., Gao, Z., Mi, F., & Hong, L. (2024) · arXiv · doi:10.48550/arxiv.2406.17626 · Cited by 7

Learning Multi-Level Dependencies for Robust Word Recognition

Wang, Z., Liu, H., Tang, J., Yang, S., Huang, G. Y., & Liu, Z. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i05.6463 · Cited by 7

Improving Robustness of Intent Detection Under Adversarial Attacks: A Geometric Constraint Perspective

Qi, B., Zhou, B., Zhang, W., Liu, J., & Wu, L. (2023) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2023.3267460 · Cited by 7

Adapting to Change

Basharat, M., & Omar, M. (2024) · Advances in medical technologies and clinical practice book series · doi:10.4018/979-8-3693-1906-2.ch009 · Cited by 6

Thai-language chatbot security: Detecting instruction attacks with XLM-RoBERTa and Bi-GRU

Vajrobol, V., Gupta, B. B., & Gaurav, A. (2024) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2024.109186 · Cited by 8

Enhancing Offensive Language Detection with Data Augmentation and Knowledge Distillation

Deng, J., Chen, Z., Sun, H., Zhang, Z., Wu, J., Nakagawa, S., Ren, F., & Huang, M. (2023) · Research · doi:10.34133/research.0189 · Cited by 6

SafeConv: Explaining and Correcting Conversational Unsafe Behavior

Zhang, M., Jin, L., Song, L., Mi, H., Chen, W., & Yu, D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.2 · Cited by 8

Automatically Exposing Problems with Neural Dialog Models

Yu, D., & Sagae, K. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.37 · Cited by 8

AfriWOZ: Corpus for Exploiting Cross-Lingual Transfer for Dialogue Generation in Low-Resource, African Languages

Adewumi, T., Adeyemi, M., Aremu, A., Peters, B., Buzaaba, H., Oyerinde, S., Rufai, A. M., Ajibade, B., Gwadabe, T., Traore, M. M. K., et al. (2023) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn54540.2023.10191208 · Cited by 12

Query-Efficient Black-Box Red Teaming via Bayesian Optimization

Lee, D., Lee, J., Ha, J.-W., Kim, J.-H., Lee, S.-W., Lee, H., & Song, H. O. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.646 · Cited by 28

Traditional and Deep Learning Methods for Trustworthy Intent Classification in Conversational Agent Voting Advice Applications (CAVAAs)

Dieing, T. I. (2025) · Mensch und Computer (MuC) · doi:10.1145/3743049.3748572 · Cited by 4

Two in One: A multi-task framework for politeness turn identification and phrase extraction in goal-oriented conversations

Priya, P., Firdaus, M., & Ekbal, A. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101661 · Cited by 4

Jailbreaking large language models: models, origins, and evolution of attacks and defenses

Li, X. H., Wu, J., Zheng, Q., Wang, H., Fan, M. Y., Hu, S., Guo, J., & Ting, L. I. U. (2025) · Scientia Sinica Informationis · doi:10.1360/ssi-2024-0196 · Cited by 4

BiasX: “Thinking Slow” in Toxic Content Moderation with Explanations of Implied Social Biases

Zhang, Y., Nanduri, S., Jiang, L., Wu, T., & Sap, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.300 · Cited by 16

Distilling the Knowledge of Large-scale Generative Models into Retrieval Models for Efficient Open-domain Conversation

Kim, B., Seo, S., Han, S., Erdenee, E., & Chang, B. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.286 · Cited by 6

Athena 2.0: Contextualized Dialogue Management for an Alexa Prize SocialBot

Walker, M. A., Harrison, V., Juraska, J., Reed, L., Bowden, K. K., Cui, W. Q., Patil, O. R., & Ratnaparkhi, A. (2021) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/2021.emnlp-demo.15 · Cited by 5

Evolving Diverse Red-Team Language Models in Multi-Round Multi-Agent Games

Ma, C., Yang, Z., Ci, H., Gao, J., Gao, M., Pan, X., & Yang, Y. (2026) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3694367 · Cited by 34

Dialogue in the Wild: Learning from a Deployed Role-Playing Game with Humans and Bots

Shuster, K., Urbanek, J., Dinan, E., Szlam, A. D., & Weston, J. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.54 · Cited by 6

"Nice Try, Kiddo": Ad Hominems in Dialogue Systems.Preprint

Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2020) · arXiv (Cornell University) · Cited by 3

Bridging the Gap Between Clean Data Training and Real-World Inference for Spoken Language UnderstandingPreprint

Wu, D., Chen, Y., Ding, L., & Tao, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.06393 · Cited by 4

Adversarial Attacks and Defenses on Large Language Models: A Systematic Review

Li, R., Liu, J., Zhen, J., Bao, Y., Qiang, Y., Xu, Y., Gu, Y., Jin, Y., Yu, D., He, Y., et al. (2025) · IEEE International Conference on Big Data and Artificial Intelligence (BDAI) · doi:10.1109/bdai66031.2025.11325628 · Cited by 3

RogueGPT: transforming ChatGPT-4 into a rogue AI with dis-ethical tuning

Buscemi, A., & Proverbio, D. (2025) · AI and Ethics · doi:10.1007/s43681-025-00750-4 · Cited by 6

Towards Agile Text Classifiers for Everyone

Mozes, M., Hoffmann, J., Tomanek, K., Kouate, M., Thain, N., Yuan, A., Bolukbasi, T., & Dixon, L. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.30 · Cited by 13

Context-Aware Offensive Language Detection in Human-Chatbot Conversations

Shin, M., Chin, H., Song, H., Choi, Y., Choi, J., & Cha, M. (2024) · IEEE International Conference on Big Data and Smart Computing (BigComp) · doi:10.1109/bigcomp60711.2024.00049 · Cited by 5

Constructing Highly Inductive Contexts for Dialogue Safety through Controllable Reverse Generation

Zhang, Z., Cheng, J., Sun, H., Deng, J., Mi, F., Wang, Y., Shang, L., & Huang, M. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.270 · Cited by 9

QaDialMoE: Question-answering Dialogue based Fact Verification with Mixture of Experts

Wang, L., Zhang, P., Lu, X., Zhang, L., Yan, C., & Zhang, C. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.229 · Cited by 4

Robustness Testing of Language Understanding in Task-Oriented DialogPreprint

Liu, J., Takanobu, R., Wen, J., Wan, D., Li, H., Nie, W., Li, C., Peng, W., & Huang, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.15262 · Cited by 3

Benchmark for Detecting Child Pornography in Open Domain Dialogues Using Large Language Models

Yu, J., Huang, Z., Zhang, L., Lu, Y., Yan, Y., Xiao, Z., & Lan, Z. (2024) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn60899.2024.10650483 · Cited by 2

Sources of Noise in Dialogue and How to Deal with Them

Chen, D., & Zhou, Y. (2023) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2023.sigdial-1.1 · Cited by 2

InstructSafety: A Unified Framework for Building Multidimensional and Explainable Safety Detector through Instruction Tuning

Zhang, Z., Cheng, J., Sun, H., Deng, J., & Huang, M. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.700 · Cited by 2

An Adversarially-Learned Turing Test for Dialog Generation ModelsPreprint

Gao, X., Zhang, Y., Galley, M., & Dolan, B. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2104.08231 · Cited by 4

The R-U-A-Robot Dataset: Helping Avoid Chatbot Deception by Detecting User Questions About Human or Non-Human IdentityPreprint

Gros, D., Li, Y., & Yu, Z. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.02692 · Cited by 2

Semantically Improved Adversarial Attack Based on Masked Language Model via Context Preservation

Tian, H., Wu, H., Cheung, Y., He, J., & Tian, Z. (2025) · International Conference on Dependable Systems & Networks/Proceedings · doi:10.1109/dsn64029.2025.00029 · Cited by 1

Query-efficient and dataset-independent red teaming for LLMs content safety evaluation

Liu, S., Cheng, X., & Su, S. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.114404 · Cited by 2

Balancing AI responsibility with privacy, safety, and utility: Unlearning in large language models for mathematics education

Li, C., Gülfidan, G., & Zhang-Kopf, Y. (2026) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2026.100642 · Cited by 1

Measuring Adversarial Datasets

Bai, Y., Huang, R., Viswanathan, V., Kuo, T.-S., & Wu, T. (2023) · doi:10.18653/v1/2023.artofsafety-1.4 · Cited by 2

A Robust Representation with Pre-trained Start and End Characters Vectors for Noisy Word Recognition

Liu, C., Ma, X., Yu, M., Wu, X., Liu, M., Jiang, J., & Huang, W. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-55130-8_15

Adversarial Examples for Evaluating Math Word Problem SolversPreprint

Kumar, V., Maheshwary, R., & Pudi, V. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.05925 · Cited by 5

How Does Counterfactually Augmented Data Impact Models for Social Computing Constructs?Preprint

Sen, I., Samory, M., Floeck, F., Wagner, C., & Augenstein, I. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.07022 · Cited by 3

NaturalAdversaries: Can Naturalistic Adversaries Be as Effective as Artificial Adversaries?

Gabriel, S., Palangi, H., & Choi, Y. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.413

CESAR: Automatic Induction of Compositional Instructions for Multi-turn Dialogs

Aksu, T., Hazarika, D., Mehri, S., Kim, S., Hakkani-Tür, D. Z., Liu, Y., & Namazifar, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.717 · Cited by 1

Learning to love diligent trolls: Accounting for rater effects in the dialogue safety task

Ilagan, M. J. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.928

Investigating the Limitations of Adversarial Training for Language Models in Realistic Spam Filter Deployment Scenarios

Albtosh, L. B. (2024) · Advances in information security, privacy, and ethics book series · doi:10.4018/979-8-3693-6517-5.ch008

Multi-task learning on mental disorder detection, sentiment analysis, and emotion detection using social media posts.*

Armah, C., & Dehkharghani, R. (2024) · International Symposium on Innovative Approaches in Smart Technologies (ISAS) · doi:10.1109/isas64331.2024.10845733

A Context-Aware Framework for Reducing Offensive Language into Polite Equivalents in English-to-Tamil Neural Machine Translation

Rama, S., & Mythili, R. (2025) · International Conference on Data Science and Business Systems (ICDSBS) · doi:10.1109/icdsbs63635.2025.11031972

CapTune: Adapting Non-Speech Captions With Anchored Generative Models

Huang, J. Z., Pataca, C. D. L., Wu, L.-Y., & Jain, D. (2025) · International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS) · doi:10.1145/3663547.3746346 · Cited by 1

Avoiding toxicity and prejudice in chatbots with knowledge distillation

Wang, H., Putra, C. D., & Weng, H.-T. (2025) · Knowledge and Information Systems · doi:10.1007/s10115-025-02612-1

Defamiliarization Attack: Literary Theory Enabled Discussion of LLM Safety

Babu, B., Agafonova, I., Biedermann, S., & Yamshchikov, I. P. (2026) · Electronics · doi:10.3390/electronics15051047

Semantic Aggregated Adversarial Training Framework for Hate Speech Detection

Zhang, X., Tian, H., Zheng, X. B., Peng, J., & Zeng, D. D. (2026) · INFORMS journal on computing · doi:10.1287/ijoc.2023.0508

Empirical Evaluation of a Multi-Model Conversational Image Recognition System with Dynamic Routing and Real-Time Deployment

U, A. H. M., Prasad, S., Sortur, V. H., Ghanathe, H. S., & Manoj., M. (2026) · International Conference on Integrated Intelligence and Communication Systems (ICIICS) · doi:10.1109/iciics67880.2026.11483580

Bridging the Scale Gap: Augmenting Human Red-Teaming to Uncover Latent Risks in T2I Models

Quaye, J. A., Parrish, A., Rastogi, C., Kahng, M., Inel, O., Aroyo, L. M., & Reddi, V. J. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812210

LLM-as-a-Judge for Security Assessment: Limitations, Adversarial Vulnerabilities, and Mitigation Directions

Lin, J. (2026) · International Conference on Sustainable Development and Digital Transformation · doi:10.1145/3838938.3838949

Safety-Usefulness Relative Shift in Multi-turn Dialogues of Large Language Models

Li, W., Li, R., & Li, M. (2026) · Communications in computer and information science · doi:10.1007/978-981-92-6621-0_8

Bot-Adversarial Dialogue for Safe Conversational Agents

Xu, J., Ju, D., Li, M., Boureau, Y.-L., Weston, J., & Dinan, E. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.235 · Cited by 64

Attacks, Defenses and Evaluations for LLM Conversation Safety: A Survey

Dong, Z., Zhou, Z., Yang, C., Shao, J., & Qiao, Y. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.375 · Cited by 32

Adversarial attacks and defenses in natural language processing

Dong, X. (2022) · DR-NTU (Nanyang Technological University) · doi:10.32657/10356/159029 · Cited by 4

Semantics-preserving adversarial attacks in NLPPreprint

Singh, R., Joshi, T., Nair, V. N., & Sudjianto, A. (2020) · arXiv (Cornell University)

Contextualized Perturbation for Textual Adversarial Attack

Li, D., Zhang, Y., Peng, H., Chen, L., Brockett, C., Sun, M., & Dolan, B. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.400 · Cited by 228