The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Build it Break it Fix it for Dialogue Safety: Robustness from Adversarial Human Attack
Open accessAbstract
Emily Dinan, Samuel Humeau, Bharath Chintagunta, Jason Weston. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). 2019.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Recent methodologies demonstrate a significant reduction in explicit bias and toxicity within model outputs (Dinan et al. 2019; Bai et al. 2022; Ganguli et al. 2023), primarily by leveraging the models' ability to identify such undesirable behaviors upon inquiry.”
From Large language models (LLMs): survey, technical frameworks, and future challenges (2024) · cites this paper as Dinan et al. 2019
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
GAN（Generative Adversarial Nets）
淳司, 柴. (2017) · Journal of Japan Society for Fuzzy Theory and Intelligent Informatics · doi:10.3156/jsoft.29.5_177_2 · Cited by 20,507
Bag of Tricks for Efficient Text Classification
Joulin, A., Grave, E., Bojanowski, P., & Mikolov, T. (2017) · Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers · doi:10.18653/v1/e17-2068 · Cited by 4,545
Large Scale GAN Training for High Fidelity Natural Image Synthesis
Brock, A., Donahue, J., & Simonyan, K. (2018) · arXiv · doi:10.48550/arxiv.1809.11096 · Cited by 3,717
SemEval-2019 Task 6: Identifying and Categorizing Offensive Language in Social Media (OffensEval)
Zampieri, M., Malmasi, S., Nakov, P., Rosenthal, S., Farra, N., & Kumar, R. (2019) · International Workshop on Semantic Evaluation (SemEval) · doi:10.18653/v1/s19-2010 · Cited by 816
Cheng, J., Bernstein, M. S., Danescu-Niculescu-Mizil, C., & Leskovec, J. (2017) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2998181.2998213 · Cited by 499
The Second Conversational Intelligence Challenge (ConvAI2)
Dinan, E., Logacheva, V., Malykh, V., Miller, A., Shuster, K., Urbanek, J., Kiela, D., Szlam, A. D., Serban, I. V., Lowe, R., et al. (2019) · The Springer series on challenges in machine learning · doi:10.1007/978-3-030-29135-8_7 · Cited by 395
Self-citationDeceiving Google's Perspective API Built for Detecting Toxic CommentsPreprint
Hosseini, H., Kannan, S., Zhang, B.-S., & Poovendran, R. (2017) · arXiv.org · doi:10.48550/arxiv.1702.08138 · Cited by 384
Beyond vandalism: Wikipedia trolls
Shachaf, P., & Hara, N. (2010) · Journal of Information Science · doi:10.1177/0165551510365390 · Cited by 357
Automated Hate Speech Detection and the Problem of Offensive LanguagePreprint
Davidson, T., Warmsley, D., Macy, M. W., & Weber, I. G. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.04009 · Cited by 284
Bishop, J. (2013) · International Journal of Web Based Communities · doi:10.1504/ijwbc.2014.058384 · Cited by 230
Analyzing the Targets of Hate in Online Social MediaPreprint
da Silva, L. H. G., Mondal, M., Correa, D., Benevenuto, F., & Weber, I. G. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.07709 · Cited by 204
Why we should have seen that coming
Wolf, M. J., Miller, K., & Grodzinsky, F. S. (2017) · ACM SIGCAS Computers and Society · doi:10.1145/3144592.3144598 · Cited by 194
Ethical Challenges in Data-Driven Dialogue Systems
Henderson, P., Sinha, K., Gontier, N., Ke, N. R., Fried, G., Lowe, R., & Pineau, J. (2018) · Proceedings of the AAAI/ACM Conference on AI Ethics and Society · doi:10.1145/3278721.3278777 · Cited by 166
Towards Linguistically Generalizable NLP Systems: A Workshop and Shared Task
Ettinger, A., Rao, S., Daumé, H., & Bender, E. M. (2017) · doi:10.18653/v1/w17-5401 · Cited by 78
Machine Learning Suites for Online Toxicity DetectionPreprint
Noever, D. (2018) · arXiv (Cornell University) · doi:10.13140/rg.2.2.32630.45120 · Cited by 28
Research works printed but not linked: 16
- 2018. 6th International Conference on Learning Rep- resentations, ICLR 2018, Vancouver, BC, Canada, April 30 -May 3, 2018, Conference Track Proceed- ings. OpenReview.net.
- Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. 2018. Hotflip: White-box adversarial exam- ples for text classification. In Proceedings of the 56th Annual Meeting of the Association for Com- putational Linguistics, ACL 2018, Melbourne, Aus- tralia, July 15-20, 2018, Volume 2: Short Papers, pages 31-36. Association for Computational Lin- guistics. 3 https://github.com/twitter/twemoji
- Patxi Galán-García, José Gaviria de la Puerta, Car- los Laorden Gómez, Igor Santos, and Pablo García Bringas. 2016. Supervised machine learning for the detection of troll profiles in twitter social network: Application to a real case of cyberbullying. Logic Journal of the IGPL, 24(1):42-53.
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative ad- versarial nets. In Advances in neural information processing systems, pages 2672-2680.
- Tommi Gröndahl, Luca Pajola, Mika Juuti, Mauro Conti, and N Asokan. 2018. All you need is" love": Evading hate-speech detection. arXiv preprint arXiv:1808.09115. · arXiv
- Susan Herring, Kirk Job-Sluder, Rebecca Scheckler, and Sasha Barab. 2002. Searching for safety online: Managing" trolling" in a feminist forum. The infor- mation society, 18(5):371-384.
- Robin Jia and Percy Liang. 2017. Adversarial exam- ples for evaluating reading comprehension systems. In (Palmer et al., 2017), pages 2021-2031.
- Chandra Khatri, Behnam Hedayatnia, Rahul Goel, Anushree Venkatesh, Raefer Gabriel, and Arindam Mandal. 2018. Detecting offensive content in open-domain conversations using two stage semi- supervision. CoRR, abs/1811.12900. · arXiv
- Ritesh Kumar, Atul Kr. Ojha, Shervin Malmasi, and Marcos Zampieri. 2018. Benchmarking aggression identification in social media. In Proceedings of the First Workshop on Trolling, Aggression and Cyber- bullying (TRAC-2018), pages 1-11, Santa Fe, New Mexico, USA. Association for Computational Lin- guistics.
- Jiwei Li, Will Monroe, Tianlin Shi, Sébastien Jean, Alan Ritter, and Dan Jurafsky. 2017. Adversarial learning for neural dialogue generation. In (Palmer et al., 2017), pages 2157-2169.
- Martha Palmer, Rebecca Hwa, and Sebastian Riedel, editors. 2017. Proceedings of the 2017 Confer- ence on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017. Association for Computa- tional Linguistics.
- Jeffrey Pennington, Richard Socher, and Christo- pher D. Manning. 2014. Glove: Global vectors for word representation. In Empirical Methods in Nat- ural Language Processing (EMNLP), pages 1532- 1543.
- Andrew Ruef, Michael Hicks, James Parker, Dave Levin, Michelle L Mazurek, and Piotr Mardziel. 2016. Build it, break it, fix it: Contesting secure development. In Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communica- tions Security, pages 690-703. ACM.
- Pnina Shachaf and Noriko Hara. 2010. Beyond vandal- ism: Wikipedia trolls. Journal of Information Sci- ence, 36(3):357-370.
- Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017. Ex machina: Personal attacks seen at scale. In Proceedings of the 26th International Conference on World Wide Web, WWW 2017, Perth, Australia, April 3-7, 2017, pages 1391-1399. ACM. · DOI · Paper page
- Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Per- sonalizing dialogue agents: I have a dog, do you have pets too? In Proceedings of the 56th Annual Meeting of the Association for Computational Lin- guistics, ACL 2018, Melbourne, Australia, July 15- 20, 2018, Volume 1: Long Papers, pages 2204-2213. Association for Computational Linguistics.
Cited by
Sorted by citations
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Recipes for Building an Open-Domain Chatbot
Roller, S., Dinan, E., Goyal, N., Ju, D., Williamson, M., Liu, Y., Xu, J., Ott, M., Smith, E. M., Boureau, Y.-L., et al. (2021) · Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume · doi:10.18653/v1/2021.eacl-main.24 · Cited by 784
Self-citationRealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Models
Gehman, S., Gururangan, S., Sap, M., Choi, Y., & Smith, N. A. (2020) · Findings of the Association for Computational Linguistics: EMNLP 2020 · doi:10.18653/v1/2020.findings-emnlp.301 · Cited by 693
Adversarial NLI: A New Benchmark for Natural Language Understanding
Nie, Y., Williams, A., Dinan, E., Bansal, M., Weston, J., & Kiela, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.441 · Cited by 656
Self-citationLarge language models (LLMs): survey, technical frameworks, and future challenges
Kumar, P. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10888-y · Cited by 395
Metadata
- Authors
- Dinan, E., Humeau, S., Chintagunta, B., & Weston, J.
- Affiliations
- Meta (Israel) (IL)
- Virginia Tech (US)
- Venue
- Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)
- Published
- 2019
- Pages
- 4536-4545
- OpenAlex
- W2969117066
- Cited by
- 161
- References
- 15
- Citations by year
- 2026: 8 · 2025: 13 · 2024: 17 · 2023: 34 · 2022: 32 · 2021: 36 · 2020: 19 · 2019: 2
- FWCI
- 8.51
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- adversarial attacks, robustness, chatbots, adversarial training, dialogue systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
