The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attack
Open accessAbstract
Emily Dinan, Samuel Humeau, Bharath Chintagunta, Jason Weston. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recent methodologies demonstrate a significant reduction in explicit bias and toxicity within model outputs (Dinan et al. 2019; Bai et al. 2022; Ganguli et al. 2023), primarily by leveraging the models' ability to identify such undesirable behaviors upon inquiry.”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Dinan et al. 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 20,507
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545
Large Scale GAN Training for High Fidelity Natural Image Synthesis
Brock, A., Donahue, J., & Simonyan, K. (2018) · arXiv · doi:10.48550/arxiv.1809.11096 · Cited by 3,717
SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)
Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., & Kumar, R. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2010 · Cited by 816
Cheng, J., Bernstein, M. S., Danescu-Niculescu-Mizil, C., & Leskovec, J. (2017) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2998181.2998213 · Cited by 499
The Second Conversational Intelligence Challenge (ConvAI2)
Dinan, E., Logacheva, V., Malykh, V., Miller, A., Shuster, K., Urbanek, J., Kiela, D., Szlam, A. D., Serban, I. V., Lowe, R., et al. (2019) · The Springer series on challenges in machine learning · doi:10.1007/978-3-030-29135-8_7 · Cited by 395
Self-citationDeceiving Google's Perspective API Built for Detecting Toxic CommentsPreprint
Hosseini, H., Kannan, S., Zhang, B.-S., & Poovendran, R. (2017) · arXiv.org · doi:10.48550/arxiv.1702.08138 · Cited by 384
Beyond vandalism: Wikipedia trolls
Shachaf, P., & Hara, N. (2010) · Journal of Information Science · doi:10.1177/0165551510365390 · Cited by 357
Automated Hate Speech Detection and the Problem of Offensive LanguagePreprint
Davidson, T., Warmsley, D., Macy, M. W., & Weber, I. G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.04009 · Cited by 284
Research works printed but not linked: 16
- 2018. 6th International Conference on Learning Rep- resentations, ICLR 2018, Vancouver, BC, Canada, April 30 -May 3, 2018, Conference Track Proceed- ings. OpenReview.net.
- Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2018. Hotflip: White-box adversarial exam- ples for text classification. In Proceedings of the 56th Annual Meeting of the Association for Com- putational Linguistics, ACL 2018, Melbourne, Aus- tralia, July 15-20, 2018, Volume 2: Short Papers, pages 31-36. Association for Computational Lin- guistics. 3 https://github.com/twitter/twemoji
- Patxi Galán-García, José Gaviria de la Puerta, Car- los Laorden Gómez, Igor Santos, and Pablo García Bringas. 2016. Supervised machine learning for the detection of troll profiles in twitter social network: Application to a real case of cyberbullying. Logic Journal of the IGPL, 24(1):42-53.
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative ad- versarial nets. In Advances in neural information processing systems, pages 2672-2680.
- Tommi Gröndahl, Luca Pajola, Mika Juuti, Mauro Conti, and N Asokan. 2018. All you need is" love": Evading hate-speech detection. arXiv preprint arXiv:1808.09115. · arXiv
- Susan Herring, Kirk Job-Sluder, Rebecca Scheckler, and Sasha Barab. 2002. Searching for safety online: Managing" trolling" in a feminist forum. The infor- mation society, 18(5):371-384.
- Robin Jia and Percy Liang. 2017. Adversarial exam- ples for evaluating reading comprehension systems. In (Palmer et al., 2017), pages 2021-2031.
- Chandra Khatri, Behnam Hedayatnia, Rahul Goel, Anushree Venkatesh, Raefer Gabriel, and Arindam Mandal. 2018. Detecting offensive content in open-domain conversations using two stage semi- supervision. CoRR, abs/1811.12900. · arXiv
- Ritesh Kumar, Atul Kr. Ojha, Shervin Malmasi, and Marcos Zampieri. 2018. Benchmarking aggression identification in social media. In Proceedings of the First Workshop on Trolling, Aggression and Cyber- bullying (TRAC-2018), pages 1-11, Santa Fe, New Mexico, USA. Association for Computational Lin- guistics.
- Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter, and Dan Jurafsky. 2017. Adversarial learning for neural dialogue generation. In (Palmer et al., 2017), pages 2157-2169.
- Martha Palmer, Rebecca Hwa, and Sebastian Riedel, editors. 2017. Proceedings of the 2017 Confer- ence on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017. Association for Computa- tional Linguistics.
- Jeffrey Pennington, Richard Socher, and Christo- pher D. Manning. 2014. Glove: Global vectors for word representation. In Empirical Methods in Nat- ural Language Processing (EMNLP), pages 1532- 1543.
- Andrew Ruef, Michael Hicks, James Parker, Dave Levin, Michelle L Mazurek, and Piotr Mardziel. 2016. Build it, break it, fix it: Contesting secure development. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communica- tions Security, pages 690-703. ACM.
- Pnina Shachaf and Noriko Hara. 2010. Beyond vandal- ism: Wikipedia trolls. Journal of Information Sci- ence, 36(3):357-370.
- Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017. Ex machina: Personal attacks seen at scale. In Proceedings of the 26th International Conference on World Wide Web, WWW 2017, Perth, Australia, April 3-7, 2017, pages 1391-1399. ACM. · DOI · Paper page
- Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Per- sonalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Lin- guistics, ACL 2018, Melbourne, Australia, July 15- 20, 2018, Volume 1: Long Papers, pages 2204-2213. Association for Computational Linguistics.
Cited by
Sorted by citations
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Recipes for Building an Open-Domain Chatbot
Roller, S., Dinan, E., Goyal, N., Ju, D., Williamson, M., Liu, Y., Xu, J., Ott, M., Smith, E. M., Boureau, Y.-L., et al. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.24 · Cited by 784
Self-citationRealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693
Adversarial NLI: A New Benchmark for Natural Language Understanding
Nie, Y., Williams, A., Dinan, E., Bansal, M., Weston, J., & Kiela, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.441 · Cited by 656
Self-citationLarge language models (LLMs): survey, technical frameworks, and future challenges
Kumar, P. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10888-y · Cited by 395
Red Teaming Language Models with Language Models
Perez, E., Huang, S., Song, F., Cai, T., Ring, R., Aslanides, J., Glaese, A., McAleese, N., & Irving, G. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.225 · Cited by 363
Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpus
Dodge, J., Sap, M., Marasović, A., Agnew, W., Ilharco, G., Groeneveld, D., Mitchell, M., & Gardner, M. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.98 · Cited by 306
Bisk, Y., Holtzman, A., Thomason, J., Andreas, J., Bengio, Y., Chai, J. Y., Lapata, M., Lazaridou, A., May, J., Nisnevich, A., et al. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.703 · Cited by 421
ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection
Hartvigsen, T., Gabriel, S., Palangi, H., Sap, M., Ray, D., & Kamar, E. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.234 · Cited by 279
Dynabench: Rethinking Benchmarking in NLP
Kiela, D., Bartolo, M., Nie, Y., Kaushik, D., Geiger, A., Wu, Z., Vidgen, B., Prasad, G., Singh, A., Ringshia, P., et al. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.324 · Cited by 254
Jailbreaking Black Box Large Language Models in Twenty Queries
Chao, P., Robey, A., Dobriban, E., Hassani, H., Pappas, G. J., & Wong, E. (2025) · IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) · doi:10.1109/satml64287.2025.00010 · Cited by 1,298
HateCheck: Functional Tests for Hate Speech Detection Models
Röttger, P., Vidgen, B., Nguyen, D., Waseem, Z., Margetts, H. Z., & Pierrehumbert, J. B. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.4 · Cited by 287
Can You Put it All Together: Evaluating Conversational Agents’ Ability to Blend Skills
Smith, E. M., Williamson, M., Shuster, K., Weston, J., & Boureau, Y.-L. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.183 · Cited by 207
Self-citationLearning from the Worst: Dynamically Generated Datasets to Improve Online Hate Detection
Vidgen, B., Thrush, T., Waseem, Z., & Kiela, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.132 · Cited by 174
Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection
Sap, M., Swayamdipta, S., Vianna, L., Zhou, X., Choi, Y., & Smith, N. A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.431 · Cited by 224
Queens are Powerful too: Mitigating Gender Bias in Dialogue Generation
Dinan, E., Fan, A., Williams, A., Urbanek, J., Kiela, D., & Weston, J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.656 · Cited by 190
Self-citationA Survey of Adversarial Defenses and Robustness in NLP
Goyal, S., Doddapaneni, S., Khapra, M. M., & Ravindran, B. (2023) · ACM Computing Surveys · doi:10.1145/3593042 · Cited by 200
Eight Things to Know about Large Language Models
Bowman, S. R. (2023) · arXiv · doi:10.48550/arxiv.2304.00612 · Cited by 137
Eight Things to Know about Large Language Models
Bowman, S. R. (2024) · Critical AI · doi:10.1215/2834703x-11556011 · Cited by 305
A Holistic Approach to Undesired Content Detection in the Real World
Markov, T., Zhang, C., Agarwal, S., Nekoul, F. E., Lee, T. T. H., Adler, S., Jiang, A., & Weng, L. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i12.26752 · Cited by 264
WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation
Liu, A., Swayamdipta, S., Smith, N. A., & Choi, Y. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.508 · Cited by 178
Safeguarding large language models: a survey
Dong, Y., Mu, R., Zhang, Y., Sun, S., Zhang, T., Wu, C., Jin, G., Qi, Y., Hu, J., Meng, J., et al. (2025) · Artificial Intelligence Review · doi:10.1007/s10462-025-11389-2 · Cited by 242
Challenges in Automated Debiasing for Toxic Language Detection
Zhou, X., Sap, M., Swayamdipta, S., Choi, Y., & Smith, N. A. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.274 · Cited by 129
COLD: A Benchmark for Chinese Offensive Language Detection
Deng, J., Zhou, J., Sun, H., Zheng, C., Mi, F., Meng, H. M. L., & Huang, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.796 · Cited by 143
Beat the ai: Investigating adversarial human annotation for reading comprehension
Bartolo, M., Roberts, A., Welbl, J., Riedel, S., & Stenetorp, P. (2020) · UCL Discovery (University College London) · Cited by 102
Recipes for Safety in Open-domain ChatbotsPreprint
Xu, J., Ju, D., Li, M., Boureau, Y.-L., Weston, J., & Dinan, E. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.07079 · Cited by 236
Self-citationMulti-Dimensional Gender Bias Classification
Dinan, E., Fan, A., Wu, L., Weston, J., Kiela, D., & Williams, A. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.23 · Cited by 117
Self-citationDoes Gender Matter? Towards Fairness in Dialogue Systems
Liu, H., Dacon, J., Fan, W., Liu, H., Liu, Z., & Tang, J. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.390 · Cited by 81
Proactive Conversational Agents in the Post-ChatGPT World
Liao, L., Yang, G. H., & Shah, C. S. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3594250 · Cited by 95
Bot-Adversarial Dialogue for Safe Conversational Agents
Xu, J., Ju, D., Li, M., Boureau, Y.-L., Weston, J., & Dinan, E. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.235 · Cited by 64
Self-citationPerturbation augmentation for fairer NLP
Qian, R., Ross, C., Fernandes, J., Smith, E. M., Kiela, D., & Williams, A. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.646 · Cited by 69
Towards a Robust Deep Neural Network against Adversarial Texts: A Survey
Wang, W., Wang, R., Wang, L., Wang, Z., & Ye, A. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3117608 · Cited by 171
ProsocialDialog: A prosocial backbone for conversational agents
Kim, H., Yu, Y., Jiang, L., Lu, X., Khashabi, D., Kim, G., Choi, Y., & Sap, M. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.267 · Cited by 80
More Bang for Your Buck: Natural Perturbation for Robust Question Answering
Khashabi, D., Khot, T., & Sabharwal, A. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.12 · Cited by 62
Fanton, M., Bonaldi, H., Tekiroğlu, S. S., & Guerini, M. (2021) · doi:10.18653/v1/2021.acl-long.250 · Cited by 58
Towards a Robust Deep Neural Network in Texts: A SurveyPreprint
Wang, W., Wang, R., Wang, L., Wang, Z., & Ye, A. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.07285 · Cited by 58
Just Say No: Analyzing the Stance of Neural Dialogue Generation in Offensive Contexts
Baheti, A., Sap, M., Ritter, A., & Riedl, M. O. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.397 · Cited by 60
On the Safety of Conversational Models: Taxonomy, Dataset, and Benchmark
Sun, H., Xu, G., Deng, J., Cheng, J., Zheng, C., Zhou, H., Peng, N., Zhu, X., & Huang, M. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.308 · Cited by 71
A Survey on Proactive Dialogue Systems: Problems, Methods, and Prospects
Deng, Y., Lei, W., Lam, W. P., & Chua, T. (2023) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2023/738 · Cited by 64
Few-Shot Bot: Prompt-Based Learning for Dialogue SystemsPreprint
Madotto, A., Lin, Z., Winata, G. I., & Fung, P. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.08118 · Cited by 96
Know Your Limits: A Survey of Abstention in Large Language Models
Wen, B., Yao, J., Feng, S., Xu, C., Tsvetkov, Y., Howe, B., & Wang, L. L. (2025) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00754 · Cited by 56
Kirk, H. R., Vidgen, B., Röttger, P., Thrush, T., & Hale, S. A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.97 · Cited by 47
Anticipating Safety Issues in E2E Conversational AI: Framework and Tooling
Dinan, E., Abercrombie, G., Bergman, A. S., Spruit, S., Hovy, D., Boureau, Y.-L., & Rieser, V. (2021) · arXiv · doi:10.48550/arxiv.2107.03451 · Cited by 44
Self-citationChatbot Security and Privacy in the Age of Personal Assistants
Ye, W., & Li, Q. (2020) · IEEE/ACM Symposium on Edge Computing (SEC) · doi:10.1109/sec50012.2020.00057 · Cited by 44
Proactive Conversational AI: A Comprehensive Survey of Advancements and Opportunities
Deng, Y., Liao, L., Lei, W., Yang, G. H., Lam, W. P., & Chua, T. (2025) · ACM Transactions on Information Systems · doi:10.1145/3715097 · Cited by 67
Smith, E., Hsu, O., Qian, R., Roller, S., Boureau, Y.-L., & Weston, J. (2022) · Proceedings of the 4th Workshop on NLP for Conversational AI · doi:10.18653/v1/2022.nlp4convai-1.8 · Cited by 40
Self-citationConvAbuse: Data, Analysis, and Benchmarks for Nuanced Detection in Conversational AI
Curry, A. C., Abercrombie, G., & Rieser, V. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.587 · Cited by 64
Open-Domain Conversational Agents: Current Progress, Open Problems, and Future Directions
Roller, S., Boureau, Y.-L., Weston, J., Bordes, A., Dinan, E., Fan, A., Gunning, D., Ju, D., Li, M., Poff, S., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.12442 · Cited by 38
Self-citationSi, W. M., Michael Backes, A., Blackburn, J., Cristofaro, E. D., Stringhini, G., Zannettou, S., & Zhang, Y. (2022) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3548606.3560599 · Cited by 52
“Nice Try, Kiddo”: Investigating Ad Hominems in Dialogue Responses
Sheng, E., Chang, K.-W., Natarajan, P., & Peng, N. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.60 · Cited by 48
Lu, J., Xu, B., Zhang, X., Min, C., Yang, L., & Lin, H. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.898 · Cited by 44
Knowledge-Grounded Dialogue Flow Management for Social Robots and Conversational Agents
Grassi, L., Recchiuto, C. T., & Sgorbissa, A. (2022) · International Journal of Social Robotics · doi:10.1007/s12369-022-00868-z · Cited by 42
Multi-Modal Open-Domain Dialogue
Shuster, K., Smith, E. M., Ju, D., & Weston, J. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.398 · Cited by 44
Self-citationBalayn, A., Yang, J., Szlávik, Z., & Bozzon, A. (2021) · ACM Transactions on Social Computing · doi:10.1145/3479158 · Cited by 38
InstructDial: Improving Zero and Few-shot Generalization in Dialogue through Instruction Tuning
Gupta, P., Jiao, C., Yeh, Y.-T., Mehri, S., Eskénazi, M., & Bigham, J. P. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.33 · Cited by 75
SafetyKit: First Aid for Measuring Safety in Open-domain Conversational Systems
Dinan, E., Abercrombie, G., Bergman, A., Spruit, S., Hovy, D., Boureau, Y.-L., & Rieser, V. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.284 · Cited by 32
Self-citationPLACES: Prompting Language Models for Social Conversation Synthesis
Chen, M., Papangelis, A., Tao, C., Kim, S., Rosenbaum, A., Liu, Y., Zhou, Y., & Hakkani-Tür, D. Z. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.63 · Cited by 53
Towards Identifying Social Bias in Dialog Systems: Framework, Dataset, and Benchmark
Zhou, J., Deng, J., Mi, F., Li, Y., Wang, Y., Huang, M., Jiang, X., Liu, Q., & Meng, H. M. L. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.262 · Cited by 34
Robustness Testing of Language Understanding in Task-Oriented Dialog
Liu, J., Takanobu, R., Wen, J., Wan, D., Li, H., Nie, W., Li, C., Peng, W., & Huang, M. (2021) · doi:10.18653/v1/2021.acl-long.192 · Cited by 36
Shades of BLEU, Flavours of Success: The Case of MultiWOZ
Nekvinda, T., & Dušek, O. (2021) · Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2021.gem-1.4 · Cited by 29
Unveiling the Implicit Toxicity in Large Language Models
Wen, J., Ke, P., Sun, H., Zhang, Z., Li, C., Bai, J., & Huang, M. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.84 · Cited by 76
Linguistically-Informed Transformations (LIT): A Method for Automatically Generating Contrast Sets
Li, C., Shengshuo, L., Liu, Z., Wu, X., Zhou, X., & Steinert‐Threlkeld, S. (2020) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2020.blackboxnlp-1.12 · Cited by 30
ScatterShot: Interactive In-context Example Curation for Text Transformation
Wu, T., Shen, H., Weld, D. S., Heer, J., & Ribeiro, M. T. (2023) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3581641.3584059 · Cited by 34
Kaushik, D., Kiela, D., Lipton, Z. C., & Yih, W.-T. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.517 · Cited by 30
SaFeRDialogues: Taking Feedback Gracefully after Conversational Safety Failures
Ung, M., Xu, J., & Boureau, Y.-L. (2022) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2022.acl-long.447 · Cited by 31
How Robust are Fact Checking Systems on Colloquial Claims?
Kim, B., Kim, H., Hong, S., & Kim, G. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.121 · Cited by 23
How Does Counterfactually Augmented Data Impact Models for Social Computing Constructs?
Sen, I., Samory, M., Flöck, F., Wagner, C., & Augenstein, I. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.28 · Cited by 24
Counterfactually Augmented Data and Unintended Bias: The Case of Sexism and Hate Speech Detection
Sen, I., Samory, M., Wagner, C., & Augenstein, I. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.347 · Cited by 21
AART: AI-Assisted Red-Teaming with Diverse Data Generation for New LLM-powered Applications
Radharapu, B., Robinson, K., Aroyo, L. M., & Lahoti, P. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-industry.37 · Cited by 61
(2021) · arXiv (Cornell University) · Cited by 21
Am I Me or You? State-of-the-Art Dialogue Models Cannot Maintain an Identity
Shuster, K., Urbanek, J., Szlam, A., & Weston, J. (2022) · Findings of the Association for Computational Linguistics: NAACL 2022 · doi:10.18653/v1/2022.findings-naacl.182 · Cited by 20
Self-citationProactive Conversational Agents
Liao, L., Yang, G. H., & Shah, C. (2023) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3539597.3572724 · Cited by 20
Phadke, S., Samory, M., & Mitra, T. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479855 · Cited by 27
Human-Machine Collaboration Approaches to Build a Dialogue Dataset for Hate Speech Countering
Bonaldi, H., Dellantonio, S., Tekiroğlu, S. S., & Guerini, M. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.549 · Cited by 19
Towards an open-domain chatbot for language practice
Tyen, G., Brenchley, M., Caines, A., & Buttery, P. J. (2022) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/2022.bea-1.28 · Cited by 23
Understanding Multi-Turn Toxic Behaviors in Open-Domain Chatbots
Chen, B., Wang, G., Guo, H., Wang, Y., & Yan, Q. (2023) · International Symposium on Research in Attacks, Intrusions and Defenses (RAID) · doi:10.1145/3607199.3607237 · Cited by 26
Investigating Robustness of Dialog Models to Popular Figurative Language Constructs
Jhamtani, H., Gangal, V., Hovy, E. H., & Berg-Kirkpatrick, T. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.592 · Cited by 19
SGD-X: A Benchmark for Robust Generalization in Schema-Guided Dialogue Systems
Lee, H., Gupta, R., Rastogi, A., Cao, Y. J., Zhang, B., & Wu, Y. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21341 · Cited by 17
Models in the Loop: Aiding Crowdworkers with Generative Annotation Assistants
Bartolo, M., Thrush, T., Riedel, S., Stenetorp, P., Jia, R., & Kiela, D. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.275 · Cited by 28
Omar, M., & Mohaisen, A. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3487553.3524667 · Cited by 17
On-Device Smishing Classifier Resistant to Text Evasion Attack
Seo, J. W., Lee, J., Kim, H., Lee, J., Han, S., Cho, J., & Lee, C.-H. (2024) · IEEE Access · doi:10.1109/access.2024.3349577 · Cited by 18
Tekiroğlu, S. S., Bonaldi, H., Fanton, M., & Guerini, M. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.245 · Cited by 25
Guiding the Release of Safer E2E Conversational AI through Value Sensitive Design
Bergman, A. S., Abercrombie, G., Spruit, S. L., Hovy, D., Dinan, E., Boureau, Y.-L., & Rieser, V. (2022) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2022.sigdial-1.4 · Cited by 16
Self-citationConversational AI—A State‐of‐the‐Art Review
Bhardwaj, V., Kumar, M., Joshi, D., Chourasia, A., Bawaskar, B., & Sharma, S. (2024) · Wiley eBooks · doi:10.1002/9781394200801.ch31 · Cited by 16
Nguyen, A., Rai, A., & Maruping, L. (2024) · Journal of Management Information Systems · doi:10.1080/07421222.2024.2340831 · Cited by 9
Ocampo, N. B., Cabrio, E., & Villata, S. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.173 · Cited by 21
Deploying Lifelong Open-Domain Dialogue LearningPreprint
Shuster, K., Urbanek, J., Dinan, E., Szlam, A. D., & Weston, J. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.08076 · Cited by 22
Self-citationOn the Unexpected Abilities of Large Language Models
Nolfi, S. (2024) · Adaptive Behavior · doi:10.1177/10597123241256754 · Cited by 19
Li, C., Xing, W., Song, Y., & Lyu, B. (2024) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2024.100338 · Cited by 22
Adversarial Examples for Evaluating Math Word Problem Solvers
Kumar, V., Maheshwary, R., & Pudi, V. (2021) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2021.findings-emnlp.230 · Cited by 16
Know your enemy: Conversational agents for security, education, training, and awareness at scale
Giboney, J. S., Schuetzler, R. M., & Grimes, G. M. (2023) · Computers & Security · doi:10.1016/j.cose.2023.103207 · Cited by 13
MoralDial: A Framework to Train and Evaluate Moral Dialogue Systems via Moral Discussions
Sun, H., Zhang, Z., Mi, F., Wang, Y., Liu, W., Cui, J., Wang, B., Liu, Q., & Huang, M. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.123 · Cited by 22
Kirk, H. R., Bean, A., Vidgen, B., Röttger, P., & Hale, S. A. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.148 · Cited by 48
Using In-Context Learning to Improve Dialogue Safety
Meade, N., Gella, S., Hazarika, D., Gupta, P., Jin, D., Reddy, S., Liu, Y., & Hakkani-Tur, D. (2023) · Findings of the Association for Computational Linguistics: EMNLP 2023 · doi:10.18653/v1/2023.findings-emnlp.796 · Cited by 13
AdversaFlow: Visual Red Teaming for Large Language Models with Multi-Level Adversarial Flow
Deng, D., Zhang, C., Zheng, H., Pu, Y., Ji, S., & Wu, Y. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3456150 · Cited by 13
Analyzing Dynamic Adversarial Training Data in the Limit
Wallace, E., Williams, A., Jia, R., & Kiela, D. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.18 · Cited by 25
A Benchmark for Understanding Dialogue Safety in Mental Health Support
Qiu, H., Zhao, T., Li, A., Zhang, S., He, H., & Lan, Z. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-44696-2_1 · Cited by 11
Robust Conversational Agents against Imperceptible Toxicity Triggers
Mehrabi, N., Beirami, A., Morstatter, F., & Galstyan, A. (2022) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2022.naacl-main.204 · Cited by 21
Robots-Dont-Cry: Understanding Falsely Anthropomorphic Utterances in Dialog Systems
Gros, D., Li, Y., & Zhou, Y. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.215 · Cited by 16
Legilimens: Practical and Unified Content Moderation for Large Language Model Services
Wu, J., Deng, J., Pang, S., Chen, Y., Xu, J., Li, X., & Xu, W. (2024) · ACM SIGSAC Conference on Computer and Communications Security (CCS) · doi:10.1145/3658644.3690322 · Cited by 14
Metadata
- Authors
- Dinan, E., Humeau, S., Chintagunta, B., & Weston, J.
- Affiliations
- Meta (Israel) (IL)
- Virginia Tech (US)
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 4536-4545
- OpenAlex
- W2969117066
- Cited by
- 161
- References
- 15
- Citations by year
- 2026: 8 · 2025: 13 · 2024: 17 · 2023: 34 · 2022: 32 · 2021: 36 · 2020: 19 · 2019: 2
- FWCI
- 8.51
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- adversarial attacks, robustness, chatbots, adversarial training, dialogue systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
