Cited by 220Topic ModelingNatural Language Processing TechniquesSpeech and dialogue systems
GeDi: Generative Discriminator Guided Sequence Generation
Open accessKrause, Ben, Gotmare, Akhilesh Deepak, McCann, Bryan, Keskar, Nitish Shirish, Joty, Shafiq, Socher, Richard, Rajani, Nazneen Fatema·2021·Findings of the Association for Computational Linguistics: EMNLP 2021
Abstract
Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, Nazneen Fatema Rajani. Findings of the Association for Computational Linguistics: EMNLP 2021. 2021.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Crossref · 10.18653/v1/2021.findings-emnlp.424
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The idea of activation perturbation can be traced back to plug-and-play controllable text generation methods, which require repeated forward and backward propagation.
“The idea of activation perturbation can be traced back to plug-and-play controllable text generation methods (Dathathri et al., 2019; Krause et al., 2020; Li et al., 2022), which require repeated forward and backward propagation.”
From Inference-Time Intervention: Eliciting Truthful Answers from a Language Model (2023) · cites this paper as Krause et al., 2020
The Related Work section says that PPLM updates hidden states by backpropagating gradients and requires multiple forward and backward passes per generation step. It contrasts this with GeDi, which adds constant overhead; the repeated-pass requirement therefore applies to the earlier PPLM method, not to GeDi itself. (LLM, conf=0.87)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 56 · Cited by 220
Research works printed but not linked: 29
- Tolga Bolukbasi, Kai-Wei Chang, James Zou, Venkatesh Saligrama, and Adam Kalai. 2016. Man is to computer programmer as woman is to home- maker? debiasing word embeddings. In Advances in neural information processing systems.
- Shikha Bordia and Samuel R. Bowman. 2019. Identify- ing and reducing gender bias in word-level language models. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Work- shop, pages 7-15, Minneapolis, Minnesota. Associ- ation for Computational Linguistics. · DOI · Paper page
- Daniel Borkan, Lucas Dixon, Jeffrey Sorensen, Nithum Thain, and Lucy Vasserman. 2019. Nuanced met- rics for measuring unintended bias with real data for text classification. In Companion proceedings of the 2019 world wide web conference, pages 491-500. · DOI
- Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. In Advances in neural information process- ing systems.
- Alvin Chan, Yew-Soon Ong, Bill Pung, Aston Zhang, and Jie Fu. 2021. Cocon: A self-supervised ap- proach for controlled text generation. ICLR.
- Reuben Cohn-Gordon, Noah Goodman, and Christo- pher Potts. 2018. Pragmatically informative image captioning with character-level inference. In Pro- ceedings of the 2018 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, Vol- ume 2 (Short Papers), pages 439-443, New Orleans, Louisiana. Association for Computational Linguis- tics. · DOI
- Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. Plug and play language mod- els: a simple approach to controlled text generation. ICLR.
- Emily Dinan, Angela Fan, Adina Williams, Jack Ur- banek, Douwe Kiela, and Jason Weston. 2020. Queens are powerful too: Mitigating gender bias in dialogue generation. In Proceedings of the 2020 Conference on Empirical Methods in Natural Lan- guage Processing (EMNLP), pages 8173-8188, On- line. Association for Computational Linguistics. · DOI
- Alexander Fabbri, Irene Li, Tianwei She, Suyi Li, and Dragomir Radev. 2019. Multi-news: A large-scale multi-document summarization dataset and abstrac- tive hierarchical model. In Proceedings of the 57th Annual Meeting of the Association for Computa- tional Linguistics, pages 1074-1084, Florence, Italy. Association for Computational Linguistics. · DOI · Paper page
- Jessica Ficler and Yoav Goldberg. 2017. Controlling linguistic style aspects in neural language genera- tion. In Proceedings of the Workshop on Stylis- tic Variation, pages 94-104, Copenhagen, Denmark. Association for Computational Linguistics. · DOI
- Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020. RealToxi- cityPrompts: Evaluating neural toxic degeneration in language models. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3356-3369, Online. Association for Computational Linguistics. · DOI · Paper page
- Marjan Ghazvininejad, Xing Shi, Jay Priyadarshi, and Kevin Knight. 2017. Hafez: an interactive poetry generation system. In Proceedings of ACL 2017, System Demonstrations, pages 43-48, Vancouver, Canada. Association for Computational Linguistics.
- Noah D Goodman and Andreas Stuhlmüller. 2013. Knowledge and implicature: Modeling language un- derstanding as social cognition. Topics in cognitive science, 5(1):173-184.
- Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735-1780.
- Ari Holtzman, Jan Buys, Maxwell Forbes, Antoine Bosselut, David Golub, and Yejin Choi. 2018. Learning to write with cooperative discriminators. In Proceedings of the 56th Annual Meeting of the As- sociation for Computational Linguistics (Volume 1: Long Papers), pages 1638-1649, Melbourne, Aus- tralia. Association for Computational Linguistics. · DOI · Paper page
- Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2017. Toward con- trolled generation of text. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Re- search, pages 1587-1596, International Convention Centre, Sydney, Australia. PMLR.
- Nitish Shirish Keskar, Bryan McCann, Lav R Varshney, Caiming Xiong, and Richard Socher. 2019. Ctrl: A conditional transformer language model for control- lable generation. arXiv preprint arXiv:1909.05858. · arXiv
- Andrea Madotto, Etsuko Ishii, Zhaojiang Lin, Sumanth Dathathri, and Pascale Fung. 2020. Plug-and-play conversational models. In Findings of the Associa- tion for Computational Linguistics: EMNLP 2020, pages 2422-2433, Online. Association for Compu- tational Linguistics. · DOI
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog, 1(8):9.
- Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. Explain yourself! leveraging language models for commonsense rea- soning. In Proceedings of the 57th Annual Meet- ing of the Association for Computational Linguis- tics, pages 4932-4942, Florence, Italy. Association for Computational Linguistics. · DOI
- Björn Ross, Michael Rist, Guillermo Carbonell, Ben- jamin Cabrera, Nils Kurowsky, and Michael Wo- jatzki. 2017. Measuring the reliability of hate speech annotations: The case of the european refugee crisis. Proceedings of NLP4CMC III: 3rd Workshop on Natural Language Processing for Computer-Mediated Communication (Bochum), Bochumer Linguistische Arbeitsberichte, vol. 17, sep 2016, pp. 6-9.
- Sheng Shen, Daniel Fried, Jacob Andreas, and Dan Klein. 2019. Pragmatically informative text gen- eration. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4060-4067, Minneapolis, Minnesota. Associ- ation for Computational Linguistics. · DOI
- Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment tree- bank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1631-1642, Seattle, Washington, USA. Asso- ciation for Computational Linguistics.
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information pro- cessing systems, pages 5998-6008.
- Zeerak Waseem. 2016. Are you a racist or am I seeing things? annotator influence on hate speech detection on Twitter. In Proceedings of the First Workshop on NLP and Computational Social Science, pages 138- 142, Austin, Texas. Association for Computational Linguistics. · DOI
- Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Ja- son Weston, and Emily Dinan. 2020. Recipes for safety in open-domain chatbots. arXiv preprint arXiv:2010.07079. · arXiv
- Dani Yogatama, Chris Dyer, Wang Ling, and Phil Blun- som. 2017. Generative and discriminative text clas- sification with recurrent neural networks. arXiv preprint arXiv:1703.01898. · arXiv
- Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. 2017. Seqgan: Sequence generative adversarial nets with policy gradient. In Thirty-First AAAI Confer- ence on Artificial Intelligence.
- Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text clas- sification. In Advances in neural information pro- cessing systems, pages 649-657.
- Authors
- Krause, B., Gotmare, A. D., McCann, B., Keskar, N. S., Joty, S., Socher, R., & Rajani, N. F.
- Affiliations
- Salesforce (United States) (US)
- ORCID
- Venue
- Findings of the Association for Computational Linguistics: EMNLP 2021
- Published
- 2021
- Pages
- 4929-4952
- OpenAlex
- W3085190015
- Cited by
- 220
- References
- 56
- Citations by year
- 2026: 20 · 2025: 20 · 2024: 25 · 2023: 74 · 2022: 43 · 2021: 33 · 2020: 5
- FWCI
- 20.21
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Speech and dialogue systems
- Keywords
- sequence generation, generative adversarial networks, reinforcement learning from human feedback, language model fine-tuning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
