Learning to Write with Cooperative Discriminators
Open accessAbstract
Ari Holtzman, Jan Buys, Maxwell Forbes, Antoine Bosselut, David Golub, Yejin Choi. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2018.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Recent works have alluded to the disadvantages of generation by maximization, which tend to generate output with high grammaticality but low diversity.
“A number of recent works have alluded to the disadvantages of generation by maximization, which tend to generate output with high grammaticality but low diversity (Kulikov et al., 2019;Holtzman et al., 2018;Fan et al., 2018).”
From The Curious Case of Neural Text Degeneration (2019) · cites this paper as Holtzman et al., 2018
The Background (§2; page unavailable) says RNN language models often assign higher probability to repetitive, overly generic sentences, supporting the low-diversity disadvantage of probability-based generation. The Qualitative Analysis (§5.2; page unavailable) likewise describes generic output from a language-model baseline. Our copy does not establish that such output has high grammaticality. (LLM, conf=0.84)
Experimental — accuracy not yet measuredSeveral recent studies on open-ended generation have reported that maximization-based decoding does not lead to high quality text.
“However, several recent studies on open-ended generation have reported that maximization-based decoding does not lead to high quality text (Fan et al., 2018;Holtzman et al., 2018).”
From The Curious Case of Neural Text Degeneration (2019) · cites this paper as Holtzman et al., 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 37 · Cited by 192
Research works printed but not linked: 22
- Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large an- notated corpus for learning natural language infer- ence. In Proceedings of the Conference on Empiri- cal Methods in Natural Language Processing, pages 632-642. Association for Computational Linguis- tics. · DOI · Paper page
- Ido Dagan, Oren Glickman, and Bernardo Magnini. 2006. The pascal recognising textual entailment challenge. In Proceedings of the First Inter- national Conference on Machine Learning Chal- lenges: Evaluating Predictive Uncertainty Visual Object Classification, and Recognizing Textual En- tailment, MLCW'05, pages 177-190, Berlin, Hei- delberg. Springer-Verlag. · DOI
- Michael Denkowski and Alon Lavie. 2010. Extend- ing the METEOR Machine Translation Evaluation Metric to the Phrase Level. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 250-253.
- Michael C. Frank and Noah D. Goodman. 2012. Pre- dicting pragmatic reasoning in language games. Sci- ence, 336(6084):998-998. · DOI · Paper page
- Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative ad- versarial nets. In Advances in Neural Information Processing Systems, pages 2672-2680.
- Edouard Grave, Armand Joulin, and Nicolas Usunier. 2017. Improving neural language models with a continuous cache. In International Conference on Learning Representations.
- Guillaume Klein, Yoon Kim, Yuntian Deng, Jean Senellart, and Alexander M. Rush. 2017. Opennmt: Open-source toolkit for neural machine translation. In Proceedings of the Association of Computational Linguistics. · DOI
- Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Pro- ceedings of the 2003 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics on Human Language Technology -Vol- ume 1, pages 48-54. Association for Computational Linguistics. · DOI
- Jiwei Li, Michel Galley, Chris Brockett, Jianfeng Gao, and Bill Dolan. 2016a. A diversity-promoting ob- jective function for neural conversation models. In Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 110-119, San Diego, California. Association for Computational Linguis- tics.
- Jiwei Li, Will Monroe, Alan Ritter, Dan Jurafsky, Michel Galley, and Jianfeng Gao. 2016b. Deep rein- forcement learning for dialogue generation. In Pro- ceedings of the Conference on Empirical Methods in Natural Language Processing, pages 1192-1202, Austin, Texas. Association for Computational Lin- guistics.
- Stephen Merity, Nitish Shirish Keskar, and Richard Socher. 2018. Regularizing and optimizing lstm lan- guage models. ICLR.
- Franz Josef Och. 2003. Minimum error rate train- ing in statistical machine translation. In Proceed- ings of the 41st Annual Meeting of the Association for Computational Linguistics, pages 160-167, Sap- poro, Japan. Association for Computational Linguis- tics. · DOI
- Ankur Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit. 2016. A decomposable attention model for natural language inference. In Proceed- ings of the Conference on Empirical Methods in Nat- ural Language Processing, pages 2249-2255. Asso- ciation for Computational Linguistics. · DOI
- Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013. On the difficulty of training recurrent neural networks. In International Conference on Machine Learning (ICML), pages 1310-1318.
- Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the Conference on Empirical Methods in Natural Language Process- ing, pages 1532-1543, Doha, Qatar. Association for Computational Linguistics.
- Allen Schmaltz, Alexander M. Rush, and Stuart Shieber. 2016. Word ordering without syntax. In Proceedings of the Conference on Empirical Meth- ods in Natural Language Processing, pages 2319- 2324, Austin, Texas. Association for Computational Linguistics.
- Yuanlong Shao, Stephan Gouws, Denny Britz, Anna Goldie, Brian Strope, and Ray Kurzweil. 2017. Generating high-quality and informative conversa- tion responses with sequence-to-sequence models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 2210-2219. Association for Computational Linguis- tics.
- Ramakrishna Vedantam, C Lawrence Zitnick, and Devi Parikh. 2015. Cider: Consensus-based image de- scription evaluation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recog- nition, pages 4566-4575.
- Taro Watanabe, Jun Suzuki, Hajime Tsukada, and Hideki Isozaki. 2007. Online large-margin train- ing for statistical machine translation. In Proceed- ings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Com- putational Natural Language Learning (EMNLP- CoNLL), pages 764-773, Prague, Czech Republic. Association for Computational Linguistics.
- Lantao Yu, Weinan Zhang, Jun Wang, and Yong Yu. 2017a. Seqgan: Sequence generative adversarial nets with policy gradient. In Proceedings of the As- sociation for the Advancement of Artificial Intelli- gence, pages 2852-2858.
- Lei Yu, Phil Blunsom, Chris Dyer, Edward Grefen- stette, and Tomas Kocisky. 2017b. The neural noisy channel. In International Conference on Learning Representations.
- Licheng Yu, Hao Tan, Mohit Bansal, and Tamara L Berg. 2017c. A joint speaker-listener-reinforcer model for referring expressions. In Proceedings of the Conference on Computer Vision and Pattern Recognition, volume 2.
- Authors
- Holtzman, A., Buys, J., Forbes, M., Bosselut, A., Golub, D., & Choi, Y.
- Affiliations
- University of Washington (US)
- Allen Institute for Artificial Intelligence (US)
- Venue
- Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
- Published
- 2018
- Pages
- 1638-1649
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2798363023
- Cited by
- 192
- References
- 37
- Citations by year
- 2025: 9 · 2024: 10 · 2023: 25 · 2022: 20 · 2021: 60 · 2020: 35 · 2019: 32 · 2018: 1
- FWCI
- 13.06
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1524371)
- Tencent
- Defense Advanced Research Projects Agency
- Samsung
- Army Research Office (W911NF-15-1-, W911NF, W911NF-15)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Social Sciences › Education › Writing and Handwriting Education
- Keywords
- writing instruction, text generation, teacher model, human-in-the-loop learning
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
