Bot-Adversarial Dialogue for Safe Conversational Agents
Open accessAbstract
Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, Emily Dinan. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Gendered and racial disparities have been documented in a number of language technologies.
“Unsurprisingly, gendered and racial disparities have been documented in a number of language technologies [37, 79, 141]”
From Data Governance in the Age of Large-Scale Data-Driven Language Technology (2022) · cites this paper as [141]
The Introduction says dialogue models can learn biased language, and §7 Ethical Considerations reports racial bias in hate-speech datasets. The paper does not document gendered disparities or establish disparities across a number of language technologies; its mention of gender in Appendix A.2 defines a hate-speech category, not a measured disparity. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Automated Hate Speech Detection and the Problem of Offensive Language
Davidson, T., Warmsley, D., Macy, M., & Weber, I. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14955 · Cited by 2,610 · see citing works
A Survey on Hate Speech Detection using Natural Language Processing
Schmidt, A. G., & Wiegand, M. (2017) · International Workshop on Natural Language Processing for Social Media (SocialNLP) · doi:10.18653/v1/w17-1101 · Cited by 1,447 · see citing works
A Deep Reinforced Model for Abstractive Summarization
Paulus, R., Xiong, C., & Socher, R. (2017) · arXiv · doi:10.48550/arxiv.1705.04304 · Cited by 1,182 · see citing works
DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation
Zhang, Y., Sun, S., Galley, M., Chen, Y.-C., Brockett, C., Gao, X., Gao, J., Liu, J., & Dolan, B. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.18653/v1/2020.acl-demos.30 · Cited by 1,076 · see citing works
Reliability in content analysis: some common misconceptions and recommendations.
Krippendorff, K. (2004) · Human Communication Research · doi:10.1093/hcr/30.3.411 · Cited by 1,027 · see citing works
Baumgartner, J., Zannettou, S., Keegan, B., Squire, M., & Blackburn, J. (2020) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v14i1.7347 · Cited by 881 · see citing works
Hill, J., Randolph Ford, W., & Farreras, I. G. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.02.026 · Cited by 856 · see citing works
The Risk of Racial Bias in Hate Speech Detection
Sap, M., Card, D., Gabriel, S., Choi, Y., & Smith, N. A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1163 · Cited by 817 · see citing works
Recipes for Building an Open-Domain Chatbot
Roller, S., Dinan, E., Goyal, N., Ju, D., Williamson, M., Liu, Y., Xu, J., Ott, M., Smith, E. M., Boureau, Y.-L., et al. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.24 · Cited by 784 · see citing works
Self-citationPredicting the Type and Target of Offensive Posts in Social Media
Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., & Kumar, R. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1144 · Cited by 781 · see citing works
RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693 · see citing works
Measuring and Mitigating Unintended Bias in Text Classification
Dixon, L., Li, J., Sorensen, J., Thain, N., & Vasserman, L. (2018) · Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society · doi:10.1145/3278721.3278729 · Cited by 674 · see citing works
Adversarial NLI: A New Benchmark for Natural Language Understanding
Nie, Y., Williams, A., Dinan, E., Bansal, M., Weston, J., & Kiela, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.441 · Cited by 656 · see citing works
Self-citationDetecting Offensive Language in Social Media to Protect Adolescent Online Safety
Chen, Y., Zhou, Y., Zhu, S., & Xu, H. (2012) · International Conference on Social Computing · doi:10.1109/socialcom-passat.2012.55 · Cited by 653 · see citing works
Ex Machina: Personal Attacks Seen at Scale
Wulczyn, E., Thain, N., & Dixon, L. (2017) · Proceedings of the 26th International Conference on World Wide Web · doi:10.1145/3038912.3052591 · Cited by 636 · see citing works
Towards a Human-like Open-Domain Chatbot
Adiwardana, D., Luong, M.-T., So, D. R., Hall, J., Fiedel, N., Thoppilan, R., Yang, Z., Kulshreshtha, A., Nemade, G., Lu, Y., et al. (2020) · arXiv · doi:10.48550/arxiv.2001.09977 · Cited by 614 · see citing works
Wizard of Wikipedia: Knowledge-Powered Conversational agentsPreprint
Dinan, E., Roller, S., Shuster, K., Fan, A., Auli, M., & Weston, J. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.01241 · Cited by 610 · see citing works
Self-citationParlAI: A Dialog Research Software Platform
Miller, A., Feng, W., Batra, D., Bordes, A., Fisch, A., Lu, J., Parikh, D., & Weston, J. (2017) · Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP) · doi:10.18653/v1/d17-2014 · Cited by 326 · see citing works
Self-citationNeural Text Generation with Unlikelihood Training
Welleck, S., Kulikov, I., Roller, S., Dinan, E., Cho, K., & Weston, J. (2019) · arXiv · doi:10.48550/arxiv.1908.04319 · Cited by 254 · see citing works
Self-citationBuild it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attack
Dinan, E., Humeau, S., Chintagunta, B., & Weston, J. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1461 · Cited by 161 · see citing works
Self-citationBaumgartner, J., Zannettou, S., Keegan, B., Squire, M., & Blackburn, J. (2020) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.3608134 · Cited by 145 · see citing works
I hate you! Disinhibition with virtual partners
Angeli, A. D., & Brahnam, S. (2008) · Interacting with Computers · doi:10.1016/j.intcom.2008.02.004 · Cited by 140 · see citing works
ACUTE-EVAL: Improved Dialogue Evaluation with Optimized Questions and Multi-turn ComparisonsPreprint
Li, M., Weston, J., & Roller, S. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.03087 · Cited by 111 · see citing works
Self-citationNeural Text Generation With Unlikelihood Training
Welleck, S., Kulikov, I., Roller, S., Dinan, E., Cho, K., & Weston, J. (2020) · International Conference on Learning Representations · Cited by 106 · see citing works
Self-citationWhy We Should Have Seen That Coming
Wolf, M. J., Miller, K., & Grodzinsky, F. S. (2017) · ORBIT Journal · doi:10.29297/orbit.v1i2.49 · Cited by 96 · see citing works
Empathy Is All You Need: How a Conversational Agent Should Respond to Verbal Abuse
Chin, H., Molefi, L. W., & Yi, M. Y. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376461 · Cited by 96 · see citing works
Fortuna, P. (2017) · Open Repository of the University of Porto (University of Porto) · doi:10.34626/hy8t-f260 · Cited by 47 · see citing works
Judgment of the Humanness of an Interlocutor Is in the Eye of the Beholder
Lortie, C. L., & Guitton, M. J. (2011) · PLoS ONE · doi:10.1371/journal.pone.0025085 · Cited by 46 · see citing works
Neural Generation Meets Real People: Towards Emotionally Engaging Mixed-Initiative ConversationsPreprint
Paranjape, A., See, A., Kenealy, K., Li, H., Hardy, A., Qi, P., Sadagopan, K. R., Phu, N. M., Soylu, D., & Manning, C. D. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.12348 · Cited by 32 · see citing works
Should an Agent Be Ignoring It?
Chin, H., & Yi, M. Y. (2019) · Extended Abstracts of the CHI Conference on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3290607.3312826 · Cited by 28 · see citing works
Stupid computer! Abuse and social identities
Angeli, A. D. (2005) · Institutional Research Information System (Università degli Studi di Trento) · doi:10.1007/11555261 · Cited by 26 · see citing works
A Crowd-based Evaluation of Abuse Response Strategies in Conversational Agents
Curry, A. C., & Rieser, V. (2019) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/w19-5942 · Cited by 22 · see citing works
Flagged first, then by citations
1 of the 34 references couldn’t be matched to a known paper.
Research works printed but not linked: 2
- Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2019b. Wiz- ard of Wikipedia: Knowledge-powered conversa- tional agents. In Proceedings of the International Conference on Learning Representations.
- ACL.
Cited by
Sorted by citations
Red Teaming Language Models with Language Models
Perez, E., Huang, S., Song, F., Cai, T., Ring, R., Aslanides, J., Glaese, A., McAleese, N., & Irving, G. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.225 · Cited by 363 · see citing works
Predictability and Surprise in Large Generative Models
Ganguli, D., Hernandez, D., Lovitt, L., Askell, A., Bai, Y., Chen, A., Conerly, T., Dassarma, N., Drain, D., Elhage, N., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3533229 · Cited by 207 · see citing works
“…adversarial datasets to probe for weaknesses in computer vision systems [36]), as well as continuous evaluation by humans carrying out multi-step interactions (e.g, conversations [3, 88]) with these models, as well as plans for how to update the models in response to what these evaluations find.”
MethodTrustworthy ai: A computational perspective
Liu, H., Wang, Y., Fan, W., Liu, X., Li, Y., Jain, S., Liu, Y., Jain, A., & Tang, J. (2023) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3546872 · Cited by 261 · see citing works
COLD: A Benchmark for Chinese Offensive Language Detection
Deng, J., Zhou, J., Sun, H., Zheng, C., Mi, F., Meng, H. M. L., & Huang, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.796 · Cited by 103 · see citing works
“I’m sorry to hear that”: Finding new biases in language models with a holistic descriptor dataset
Smith, E. M., Hall, M., Kambadur, M., Presani, E., & Williams, A. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.625 · Cited by 97 · see citing works
ProsocialDialog: A prosocial backbone for conversational agents
Kim, H., Yu, Y., Jiang, L., Lu, X., Khashabi, D., Kim, G., Choi, Y., & Sap, M. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.267 · Cited by 80 · see citing works
The Moral Integrity Corpus: A Benchmark for Ethical Dialogue Systems
Ziems, C., Yu, J., Wang, Y., Halevy, A. Y., & Yang, D. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.261 · Cited by 73 · see citing works
Discovering Language Model Behaviors with Model-Written Evaluations
Perez, E., Ringer, S., Lukošiūtė, K., Nguyen, K., Chen, E., Heiner, S., Pettit, C., Olsson, C., Kundu, S., Kadavath, S., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.09251 · Cited by 53 · see citing works
On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark
Sun, H., Xu, G., Deng, J., Cheng, J., Zheng, C., Zhou, H., Peng, N., Zhu, X., & Huang, M. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.308 · Cited by 49 · see citing works
A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospects
Deng, Y., Lei, W., Lam, W. P., & Chua, T. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/738 · Cited by 47 · see citing works
Data Governance in the Age of Large-Scale Data-Driven Language Technology
Jernite, Y., Nguyen, H., Biderman, S., Rogers, A., Masoud, M., Danchev, V., Tan, S., Luccioni, A. S., Subramani, N., Johnson, I., et al. (2022) · 2022 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3531146.3534637 · Cited by 45 · see citing works
“Unsurprisingly, gendered and racial disparities have been documented in a number of language technologies [37, 79, 141], and processes of creating resources and technologies may further entrench such disparities [25, 38, 144].”
BackgroundProactive Conversational AI: A Comprehensive Survey of Advancements and Opportunities
Deng, Y., Liao, L., Lei, W., Yang, G. H., Lam, W. P., & Chua, T. (2025) · ACM Transactions on Information Systems · doi:10.1145/3715097 · Cited by 43 · see citing works
Si, W. M., Michael Backes, A., Blackburn, J., Cristofaro, E. D., Stringhini, G., Zannettou, S., & Zhang, Y. (2022) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3548606.3560599 · Cited by 47 · see citing works
InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning
Gupta, P., Jiao, C., Yeh, Y.-T., Mehri, S., Eskénazi, M., & Bigham, J. P. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.33 · Cited by 46 · see citing works
SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems
Dinan, E., Abercrombie, G., Bergman, A., Spruit, S., Hovy, D., Boureau, Y.-L., & Rieser, V. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.284 · Cited by 32 · see citing works
Self-citationKeep Me Updated! Memory Management in Long-term Conversations
Bae, S., Kwak, D., Kang, S., Lee, M. Y., Kim, S., Jeong, Y., Kim, H., Lee, S.-W., Park, W., & Sung, N. (2022) · Findings of the Association for Computational Linguistics: EMNLP 2022 · doi:10.18653/v1/2022.findings-emnlp.276 · Cited by 29 · see citing works
Building a Role Specified Open-Domain Dialogue System Leveraging Large-Scale Language Models
Bae, S., Kwak, D., Kim, S., Ham, D., Kang, S., Lee, S., & Park, W. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.155 · Cited by 28 · see citing works
Zhou, Y., Ni, T., Lee, W., & Zhao, Q. (2025) · Transactions on Artificial Intelligence · doi:10.53941/tai.2025.100003 · Cited by 34 · see citing works
SaFeRDialogues: Taking Feedback Gracefully after Conversational Safety Failures
Ung, M., Xu, J., & Boureau, Y.-L. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.447 · Cited by 24 · see citing works
Self-citationA prompt-based approach to adversarial example generation and robustness enhancement
Yang, Y., Huang, P., Cao, J., Li, J., Lin, Y., & Ma, F. (2023) · Frontiers of Computer Science · doi:10.1007/s11704-023-2639-2 · Cited by 32 · see citing works
Jalali, N. A., & Hongsong, C. (2024) · Cluster Computing · doi:10.1007/s10586-024-04515-2 · Cited by 21 · see citing works
AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-powered Applications
Radharapu, B., Robinson, K., Aroyo, L. M., & Lahoti, P. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-industry.37 · Cited by 34 · see citing works
Understanding Multi-Turn Toxic Behaviors in Open-Domain Chatbots
Chen, B., Wang, G., Guo, H., Wang, Y., & Yan, Q. (2023) · International Symposium on Research in Attacks, Intrusions and Defenses (RAID) · doi:10.1145/3607199.3607237 · Cited by 22 · see citing works
Guiding the Release of Safer E2E Conversational AI through Value Sensitive Design
Bergman, A. S., Abercrombie, G., Spruit, S. L., Hovy, D., Dinan, E., Boureau, Y.-L., & Rieser, V. (2022) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2022.sigdial-1.4 · Cited by 16 · see citing works
Self-citationKoSBI: A Dataset for Mitigating Social Bias Risks Towards Safer Large Language Model Applications
Lee, H., Hong, S., Park, J., Kim, T., Kim, G., & Ha, J.-W. (2023) · Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 5: Industry Track) · doi:10.18653/v1/2023.acl-industry.21 · Cited by 15 · see citing works
Kirk, H. R., Bean, A., Vidgen, B., Röttger, P., & Hale, S. A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.148 · Cited by 26 · see citing works
Using In-Context Learning to Improve Dialogue Safety
Meade, N., Gella, S., Hazarika, D., Gupta, P., Jin, D., Reddy, S., Liu, Y., & Hakkani-Tur, D. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.796 · Cited by 13 · see citing works
AdversaFlow: Visual Red Teaming for Large Language Models with Multi-Level Adversarial Flow
Deng, D., Zhang, C., Zheng, H., Pu, Y., Ji, S., & Wu, Y. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3456150 · Cited by 13 · see citing works
Collecting Qualitative Data at Scale with Large Language Models: A Case Study
Cuevas, A., Scurrell, J. V., Brown, E. M., Entenmann, J., & Daepp, M. I. G. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710947 · Cited by 23 · see citing works
Analyzing Dynamic Adversarial Training Data in the Limit
Wallace, E., Williams, A., Jia, R., & Kiela, D. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.18 · Cited by 15 · see citing works
Deceiving LLM through Compositional Instruction with Hidden Attacks
Jiang, S., Chen, X., & Tang, R. (2025) · ACM Transactions on Autonomous and Adaptive Systems · doi:10.1145/3717067 · Cited by 10 · see citing works
AURA: Amplifying Understanding, Resilience, and Awareness for Responsible AI Content Work
Zhang, A. Q., Amores, J., Shen, H., Czerwinski, M. P., Gray, M. L., & Suh, J. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3710931 · Cited by 13 · see citing works
Don’t Forget Your ABC’s: Evaluating the State-of-the-Art in Chat-Oriented Dialogue Systems
Finch, S. E., Finch, J. D., & Choi, J. D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.839 · Cited by 10 · see citing works
A First Look at Toxicity Injection Attacks on Open-domain Chatbots
Weeks, C., Cheruvu, A., Abdullah, S. M., Kanchi, S., Yao, D. D., & Viswanath, B. (2023) · Annual Computer Security Applications Conference · doi:10.1145/3627106.3627122 · Cited by 10 · see citing works
Revisiting Contextual Toxicity Detection in Conversations
Anuchitanukul, A., Ive, J., & Specia, L. (2022) · Journal of Data and Information Quality · doi:10.1145/3561390 · Cited by 10 · see citing works
ASSERT: Automated Safety Scenario Red Teaming for Evaluating the Robustness of Large Language Models
Mei, A., Levy, S., & Wang, W. Y. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.388 · Cited by 9 · see citing works
The CRINGE Loss: Learning what language not to model
Adolphs, L., Gao, T., Xu, J., Shuster, K., Sukhbaatar, S., & Weston, J. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.493 · Cited by 10 · see citing works
Self-citationThe Bias that Lies Beneath: Qualitative Uncovering of Stereotypes in Large Language Models
Babonnaud, W., Delouche, E., & Lahlouh, M. (2024) · Linköping electronic conference proceedings · doi:10.3384/ecp208022 · Cited by 6 · see citing works
On safe and usable chatbots for promoting voter participation
Muppasani, B. C., Pallagani, V., Lakkaraju, K., Lei, S., Srivastava, B., Robertson, B. W., Hickerson, A., & Narayanan, V. (2023) · AI Magazine · doi:10.1002/aaai.12109 · Cited by 6 · see citing works
Those Aren’t Your Memories, They’re Somebody Else’s: Seeding Misinformation in Chat Bot Memories
Atkins, C., Zhao, B. Z. H., Asghar, H. J., Wood, I., & Kâafar, M. A. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-33488-7_11 · Cited by 5 · see citing works
Lee, H., Hong, S., Park, J., Kim, T., Cha, M., Choi, Y., Kim, B., Kim, G., Lee, E., Lim, Y., et al. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.370 · Cited by 5 · see citing works
Click: Controllable Text Generation with Sequence Likelihood Contrastive Learning
Zheng, C., Ke, P., Zhang, Z., & Huang, M. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.65 · Cited by 5 · see citing works
Query-Efficient Black-Box Red Teaming via Bayesian Optimization
Lee, D., Lee, J., Ha, J.-W., Kim, J.-H., Lee, S.-W., Lee, H., & Song, H. O. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.646 · Cited by 5 · see citing works
Jailbreaking large language models: models, origins, and evolution of attacks and defenses
Li, X. H., Wu, J., Zheng, Q., Wang, H., Fan, M. Y., Hu, S., Guo, J., & Ting, L. I. U. (2025) · Scientia Sinica Informationis · doi:10.1360/ssi-2024-0196 · Cited by 4 · see citing works
The State of Profanity Obfuscation in Natural Language Processing Scientific Publications
Nozza, D., & Hovy, D. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.240 · Cited by 4 · see citing works
Improving Bot Response Contradiction Detection via Utterance Rewriting
Jin, D., Liu, S., Liu, Y., & Hakkani-Tür, D. Z. (2022) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2022.sigdial-1.56 · Cited by 4 · see citing works
Evolving Diverse Red-Team Language Models in Multi-Round Multi-Agent Games
Ma, C.-D., Yang, Z., Ci, H., Gao, J., Gao, M.-Q., Pan, X.-H., & Yang, Y. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2026.3694367 · Cited by 20 · see citing works
Ren, B., Cheon, E., & Li, J. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757641 · Cited by 6 · see citing works
Towards Agile Text Classifiers for Everyone
Mozes, M., Hoffmann, J., Tomanek, K., Kouate, M., Thain, N., Yuan, A., Bolukbasi, T., & Dixon, L. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.30 · Cited by 5 · see citing works
IberBench: LLM evaluation on Iberian languages
González, J. Á., Borrego-Obrador, I., Herrero, Á. R., Sarvazyan, A. M., Chinea-Ríos, M., Basile, A., & Franco-Salvador, M. (2025) · Computer Speech & Language · doi:10.1016/j.csl.2025.101899 · Cited by 3 · see citing works
Sources of Noise in Dialogue and How to Deal with Them
Chen, D., & Zhou, Y. (2023) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2023.sigdial-1.1 · Cited by 2 · see citing works
Adversarial Attacks on IoT Systems Leveraging Large Language Models
Shan, W., Long, T., & Zhou, Z. (2024) · International Conference on Identification, Information and Knowledge in the Internet of Things (IIKI) · doi:10.1109/iiki65561.2024.00035 · Cited by 1 · see citing works
Can Personal Health Information Be Secured in LLM? Privacy Attack and Defense in the Medical Domain
Kang, Y., Kim, E., & Cho, Y.-S. (2025) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3719027.3765105 · Cited by 1 · see citing works
Hu, J., Yao, X., Huang, Z., Liang, D., Yang, D., Liu, D., Li, J., Zhang, Y., & Ma, X. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3792787 · Cited by 1 · see citing works
Cheap Learning: Maximizing Performance of Language Models for Social Data Science Using Minimal Data
Castro-González, L., Chung, Y.-L., Kirk, H. R., Francis, J., Williams, A. R., Johansson, P., & Bright, J. (2025) · Sociological Methods & Research · doi:10.1177/00491241251340608 · Cited by 1 · see citing works
DialGuide: Aligning Dialogue Model Behavior with Developer Guidelines
Gupta, P., Liu, Y., Jin, D., Hedayatnia, B., Gella, S., Liu, S., Lange, P. L., Hirschberg, J., & Hakkani-Tür, D. Z. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.935 · Cited by 1 · see citing works
CESAR: Automatic Induction of Compositional Instructions for Multi-turn Dialogs
Aksu, T., Hazarika, D., Mehri, S., Kim, S., Hakkani-Tür, D. Z., Liu, Y., & Namazifar, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.717
Student-Teacher Prompting for Red Teaming to Improve Guardrails
Llaca, R. R., Leskoschek, V., Paiva, V. C., Lupău, C., Lippmann, P., & Yang, J. (2023) · doi:10.18653/v1/2023.artofsafety-1.2
LLM vs. LLM: uso de red team e AML para avaliar a segurança de LLMs
de Oliveira, D. E. G. C., & Miers, C. C. (2024) · Escola Regional de Redes de Computadores (ERRC) · doi:10.5753/errc.2024.4670
Fathima Zanha, K., Fathima, N., Salva Fathima, S., Rose, S., & Baby, D. E. (2025) · 2025 2nd International Conference on Trends in Engineering Systems and Technologies (ICTEST) · doi:10.1109/ictest64710.2025.11042287
Mitigating Security Risks in Large Language Models: A Full Lifecycle Perspective
Wang, Y. K., Bai, Z., Huo, J., Wang, B., Liu, J., Cao, H., Meng, F., Yang, X., & Gao, Y. (2026) · Machine Learning · doi:10.1007/s10994-025-06930-x
Going PLACES: Participatory Localized Red Teaming for Text-to-Image Safety in the Global South
Rastogi, C., Bhutani, M., Kahng, M., Muhammad, S. H., Razumovskaia, E., Suresh, P., Ahmad, I. S., Kalia, C., Mahomed, Y., Maji, M., et al. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812296
FairBotBench: A Bilingual Benchmark for Evaluating Chatbot Behavior in E-commerce
Janusko, T., Bochnia, R., Hempel, G., Tomschke, S., Anke, J., & Thiele, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-26717-7_8
Auditable LLM Safety-Judge Selection via Confidence-Interval Filtering on Unsafe Recall
Yang, J., Yi, J., Li, J., & Ma, S. (2026) · IEEE Access · doi:10.1109/access.2026.3726251
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling
Dinan, E., Abercrombie, G., Bergman, A. S., Spruit, S., Hovy, D., Boureau, Y.-L., & Rieser, V. (2021) · arXiv · doi:10.48550/arxiv.2107.03451 · Cited by 44 · see citing works
Self-citation
- Authors
- Xu, J., Ju, D., Li, M., Boureau, Y.-L., Weston, J., & Dinan, E.
- Affiliations
- Meta (United States) (US)
- Venue
- Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
- Published
- 2021
- Pages
- 2950-2968
- OpenAlex
- W3171850892
- Cited by
- 64
- References
- 34
- Citations by year
- 2026: 6 · 2025: 11 · 2024: 5 · 2023: 22 · 2022: 19 · 2021: 1
- FWCI
- 5.43
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Keywords
- chatbots, safety, dialogue evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review