Do Massively Pretrained Language Models Make Better Storytellers?
Open accessAbstract
Large neural language models trained on massive amounts of text have emerged as a formidable strategy for Natural Language Understanding tasks.However, the strength of these models as Natural Language Generators is less clear.Though anecdotal evidence suggests that these models generate better quality text, there has been no detailed study characterizing their generation abilities.In this work, we compare the performance of an extensively pretrained model, OpenAI GPT2-117 (Radford et al., 2019), to a state-of-the-art neural story generation model (Fan et al., 2018).By evaluating the generated text across a wide variety of automatic metrics, we characterize the ways in which pretrained models do, and do not, make better storytellers.We find that although GPT2-117 conditions more strongly on context, is more sensitive to ordering of events, and uses more unusual words, it is just as likely to produce repetitive and under-diverse text when using likelihood-maximizing decoding algorithms.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 48,293 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
Self-citationDeep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983 · see citing works
fairseq: A Fast, Extensible Toolkit for Sequence Modeling
Ott, M., Edunov, S., Baevski, A., Fan, A., Gross, S., Ng, N., Grangier, D., & Auli, M. (2019) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n19-4009 · Cited by 3,538 · see citing works
Kincaid, J. P., Fishburne, Jr., Robert P., R., Richard L., C., & Brad, S. (1975) · doi:10.21236/ada006655 · Cited by 3,497 · see citing works
Convolutional Sequence to Sequence Learning
Gehring, J., Auli, M., Grangier, D., Yarats, D., & Dauphin, Y. N. (2017) · arXiv · doi:10.48550/arxiv.1705.03122 · Cited by 2,850 · cited 2× · see citing works
“It is based on the Convolutional Seq2seq model of Gehring et al. (2017) and aims to improve two aspects of story generation: modeling long-range context and increasing relevance of the story to the prompt.”
MethodA Diversity-Promoting Objective Function for Neural Conversation Models
Li, J., Galley, M., Brockett, C., Gao, J., & Dolan, B. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1014 · Cited by 1,981 · see citing works
Concreteness ratings for 40 thousand generally known English word lemmas
Brysbaert, M., Warriner, A. B., & Kuperman, V. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0403-5 · Cited by 1,821 · see citing works
Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models
Serban, I. V., Sordoni, A., Bengio, Y., Courville, A. C., & Pineau, J. (2016) · AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) · doi:10.1609/aaai.v30i1.9883 · Cited by 1,639 · see citing works
Hierarchical Neural Story Generation
Fan, A., Lewis, M., & Dauphin, Y. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1082 · Cited by 1,266 · see citing works
Liu, C., Lowe, R., Serban, I. V., Noseworthy, M., Charlin, L., & Pineau, J. (2016) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d16-1230 · Cited by 1,262 · see citing works
Hierarchical Neural Story Generation
Fan, A., Lewis, M., & Dauphin, Y. (2018) · arXiv · doi:10.48550/arxiv.1805.04833 · Cited by 1,242 · see citing works
(2017) · International Conference on Learning Representations · Cited by 1,075 · see citing works
Neural Responding Machine for Short-Text Conversation
Shang, L., Lu, Z., & Li, H. (2015) · doi:10.3115/v1/p15-1152 · Cited by 1,009 · see citing works
Modeling Local Coherence: An Entity-Based Approach
Barzilay, R., & Lapata, M. (2008) · Computational Linguistics · doi:10.1162/coli.2008.34.1.1 · Cited by 673 · see citing works
Linguistic Features of Writing Quality
McNamara, D. S., Crossley, S. A., & McCarthy, P. M. (2010) · Written Communication · doi:10.1177/0741088309351547 · Cited by 570 · see citing works
GLTR: Statistical Detection and Visualization of Generated Text
Gehrmann, S., Strobelt, H., & Rush, A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.18653/v1/p19-3019 · Cited by 470 · see citing works
(2020) · arXiv (Cornell University) · Cited by 461 · see citing works
Why We Need New Evaluation Metrics for NLG
Novikova, J., Dušek, O., Cercas Curry, A., & Rieser, V. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1238 · Cited by 422 · see citing works
Language style matching in writing: synchrony in essays, correspondence, and poetry.
Ireland, M. E., & Pennebaker, J. W. (2010) · Journal of Personality and Social Psychology · doi:10.1037/a0020386 · Cited by 380 · see citing works
Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence ModelsPreprint
Vijayakumar, A. K., Cogswell, M., Selvaraju, R. R., Sun, Q., Lee, S., Crandall, D., & Batra, D. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1610.02424 · Cited by 350 · see citing works
Cold Fusion: Training Seq2Seq Models Together with Language Models
Sriram, A., Jun, H., Satheesh, S., & Coates, A. (2018) · Interspeech (USB) · doi:10.21437/interspeech.2018-1392 · Cited by 278 · see citing works
Syntactic complexity as a predictor of adolescent writing quality: Which measures? Which genre?
Beers, S. F., & Nagy, W. E. (2007) · Reading and Writing · doi:10.1007/s11145-007-9107-5 · Cited by 251 · see citing works
What makes a good conversation? How controllable attributes affect human judgments
See, A., Roller, S., Kiela, D., & Weston, J. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1170 · Cited by 247 · see citing works
Self-citationUnifying Human and Statistical Evaluation for Natural Language Generation
Hashimoto, T. B., Zhang, H., & Liang, P. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1169 · Cited by 161 · see citing works
Mutual Information and Diverse Decoding Improve Neural Machine TranslationPreprint
Li, J., & Jurafsky, D. (2016) · arXiv.org · doi:10.48550/arxiv.1601.00372 · Cited by 131 · see citing works
Caccia, M., Caccia, L., Fedus, W., Larochelle, H., Pineau, J., & Charlin, L. (2020) · International Conference on Learning Representations · Cited by 103 · see citing works
Generating More Interesting Responses in Neural Conversation Models with Distributional Constraints
Baheti, A., Ritter, A., Li, J., & Dolan, B. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1431 · Cited by 87 · see citing works
Learning to Control the Specificity in Neural Response Generation
Zhang, R., Guo, J., Fan, Y., Lan, Y., Xu, J., & Cheng, X. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1102 · Cited by 74 · see citing works
Improving Neural Story Generation by Targeted Common Sense Grounding
Mao, H. H., Majumder, B. P., McAuley, J., & Cottrell, G. W. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1615 · Cited by 50 · see citing works
Why are Sequence-to-Sequence Models So Dull? Understanding the Low-Diversity Problem of Chatbots
Jiang, S., & de Rijke, M. (2018) · doi:10.18653/v1/w18-5712 · Cited by 39 · see citing works
Importance of a Search Strategy in Neural Dialogue ModellingPreprint
Kulikov, I., Miller, A., Cho, K., & Weston, J. (2018) · arXiv (Cornell University) · Cited by 37 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- Shlomo Argamon, Moshe Koppel, and Galit Avneri. 1998. Routing documents according to style. In First International workshop on innovative informa- tion systems.
- Angela Fan, Mike Lewis, and Yann Dauphin. 2018. Hi- erarchical neural story generation. In Association for Computational Linguistics.
- Matthew Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. Deep contextualized word repre- sentations. In North American Chapter of the Asso- ciation for Computational Linguistics: Human Lan- guage Technologies.
- Melissa Roemmele, Andrew S Gordon, and Reid Swanson. 2017. Evaluating story generation sys- tems using automated linguistic analyses. In KDD Workshop on Machine Learning for Creativity.
- Zachary M Ziegler, Luke Melas-Kyriazi, Sebastian Gehrmann, and Alexander M Rush. 2019. Encoder- agnostic adaptation for conditional language gener- ation. arXiv preprint arXiv:1908.06938. · arXiv
Cited by
Sorted by citations
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
Self-citationOn Faithfulness and Factuality in Abstractive Summarization
Maynez, J., Narayan, S., Bohnet, B., & McDonald, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.173 · Cited by 1,185 · see citing works
Automatic Detection of Generated Text is Easiest when Humans are Fooled
Ippolito, D., Duckworth, D., Callison-Burch, C., & Eck, D. (2019) · arXiv · doi:10.48550/arxiv.1911.00650 · Cited by 264
“In fact, See et al. (2019) note that it takes setting k to 1000 to achieve about the same amount of rare word usage and fraction of non-stopword text as as human writing.3 This makes it very easy for the model to pick out machine-generated text based on these distributional differences.”
BackgroundAutomatic Detection of Generated Text is Easiest when Humans are Fooled
Ippolito, D., Duckworth, D., Callison-Burch, C., & Eck, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.164 · Cited by 264 · see citing works
A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation
Guan, J., Huang, F., Zhao, Z., Zhu, X., & Huang, M. (2020) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00302 · Cited by 230 · see citing works
Machine-Generated Text: A Comprehensive Survey of Threat Models and Detection Methods
Crothers, E. N., Japkowicz, N., & Viktor, H. L. (2023) · IEEE Access · doi:10.1109/access.2023.3294090 · Cited by 196 · see citing works
Homogenization Effects of Large Language Models on Human Creative Ideation
Anderson, B. R., Shah, J. H., & Kreminski, M. (2024) · Creativity and Cognition · doi:10.1145/3635636.3656204 · Cited by 188 · see citing works
Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense
Krishna, K., Song, Y., Karpinska, M., Wieting, J., & Iyyer, M. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-1195 · Cited by 150 · see citing works
Enabling Language Models to Fill in the Blanks
Donahue, C., Lee, M., & Liang, P. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.225 · Cited by 139 · see citing works
PlotMachines: Outline-Conditioned Generation with Dynamic Plot State Tracking
Rashkin, H., Çelikyılmaz, A., Choi, Y., & Gao, J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.349 · Cited by 130 · see citing works
Feature-based detection of automated language models: tackling GPT-2, GPT-3 and Grover
Fröhling, L., & Zubiaga, A. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.443 · Cited by 106 · see citing works
Content Planning for Neural Story Generation with Aristotelian Rescoring
Goldfarb-Tarrant, S., Chakrabarty, T., Weischedel, R., & Peng, N. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.351 · Cited by 104 · see citing works
Moral Stories: Situated Reasoning about Norms, Intents, Actions, and their Consequences
Emelin, D., Bras, R. J. L., Hwang, J. D., Forbes, M., & Choi, Y. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.54 · Cited by 98 · see citing works
Alhussain, A., & Azmi, A. M. (2021) · ACM Computing Surveys · doi:10.1145/3453156 · Cited by 87 · see citing works
TURINGBENCH: A Benchmark Environment for Turing Test in the Age of Neural Text Generation
Uchendu, A., Ma, Z., Le, T., Zhang, R., & Lee, D. (2021) · Findings of the Association for Computational Linguistics: EMNLP 2021 · doi:10.18653/v1/2021.findings-emnlp.172 · Cited by 82 · see citing works
STORIUM: A Dataset and Evaluation Platform for Machine-in-the-Loop Story Generation
Akoury, N., Wang, S., Whiting, J., Hood, S., Peng, N., & Iyyer, M. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.525 · Cited by 85 · see citing works
GPT-too: A Language-Model-First Approach for AMR-to-Text Generation
Mager, M., Astudillo, R. F., Naseem, T., Sultan, A., Lee, Y., Florian, R., & Roukos, S. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.167 · Cited by 76 · see citing works
Meister, C., Pimentel, T., Wiher, G., & Cotterell, R. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00536 · Cited by 95 · see citing works
McCoy, R. T., Smolensky, P., Linzen, T., Gao, J., & Celikyilmaz, A. (2023) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00567 · Cited by 70 · see citing works
McCoy, R. T., Smolensky, P., Linzen, T., Gao, J., & Celikyilmaz, A. (2021) · arXiv · doi:10.48550/arxiv.2111.09509 · Cited by 70 · see citing works
Building and Evaluating Open-Domain Dialogue Corpora with Clarifying Questions
Aliannejadi, M., Kiseleva, J., Chuklin, A., Dalton, J., & Burtsev, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.367 · Cited by 77 · see citing works
Progressive Generation of Long Text with Pretrained Language Models
Tan, B., Yang, Z., Al-Shedivat, M., Xing, E., & Hu, Z. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.341 · Cited by 69 · see citing works
Semantic Scene Understanding with Large Language Models on Unmanned Aerial Vehicles
de Curtò, J., de Zarzà, I., & Calafate, C. T. (2023) · Drones · doi:10.3390/drones7020114 · Cited by 68 · see citing works
A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation
Liu, T., Zhang, Y., Brockett, C., Mao, Y., Sui, Z., Chen, W., & Dolan, B. (2022) · Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2022.acl-long.464 · Cited by 63 · see citing works
GenAug: Data Augmentation for Finetuning Text Generators
Feng, S. Y., Gangal, V., Kang, D., Mitamura, T., & Hovy, E. H. (2020) · Workshop on Knowledge Extraction and Integration for Deep Learning Architectures: Deep Learning Inside Out (DeeLIO) · doi:10.18653/v1/2020.deelio-1.4 · Cited by 69 · see citing works
The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generation
Karpinska, M., Akoury, N., & Iyyer, M. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.97 · Cited by 56 · see citing works
Marasović, A., Bhagavatula, C., Park, J. S., Bras, R. J. L., Smith, N. A., & Choi, Y. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.253 · Cited by 52 · see citing works
Collaborative Storytelling with Large-scale Neural Language Models
Nichols, E., Gao, L., & Gomez, R. (2020) · Motion, Interaction and Games · doi:10.1145/3424636.3426903 · Cited by 50 · see citing works
Rigid Formats Controlled Text Generation
Li, P., Zhang, H., Liu, X., & Shi, S. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.68 · Cited by 48 · see citing works
Woo, D. J., Wang, Y., Susanto, H., & Guo, K. (2023) · Journal of Educational Computing Research · doi:10.1177/07356331231175999 · Cited by 44 · see citing works
Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writing
Chakrabarty, T., Padmakumar, V., & He, H. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.460 · Cited by 56 · see citing works
How Chatbot Language Shapes Consumer Perceptions: The Role of Concreteness and Shared Competence
Jiménez‐Barreto, J., Rubio, N., & Molinillo, S. (2023) · Journal of Interactive Marketing · doi:10.1177/10949968231177618 · Cited by 42 · see citing works
Transformer-based Conditional Variational Autoencoder for Controllable Story GenerationPreprint
Fang, L., Zeng, T., Liu, C., Bo, L., Dong, W., & Chen, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.00828 · Cited by 53 · see citing works
Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challenges
Huang, B., Chen, C., & Shu, K. (2025) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3715073.3715076 · Cited by 42 · see citing works
“Similar to the problem of human authorship attribution, linguistic features such as phrasal verbs, coreference, part-of-speech (POS) tags, and named entity (NE) tags are also useful in distinguishing LLM-generated text (Fröhling and Zubiaga, 2021, Nguyen-Son et al., 2017, See et al., 2019).”
Hot or Cold? Adaptive Temperature Sampling for Code Generation with Large Language Models
Zhu, Y., Li, J., Li, G., Zhao, Y., Li, J., Jin, Z. G., & Mei, H. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i1.27798 · Cited by 48 · see citing works
Automatic story and item generation for reading comprehension assessments with transformers
Bulut, O., & Yildirim‐Erbasli, S. N. (2022) · International Journal of Assessment Tools in Education · doi:10.21449/ijate.1124382 · Cited by 39 · see citing works
Efficiency of Automatic Text Generators for Online Review Content Generation
Perez-Castro, A., Torres, M. D. R. M., & Toral, S. L. (2022) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2023.122380 · Cited by 34 · see citing works
Abedi, M., Alshybani, I., Shahadat, M., & Murillo, M. S. (2023) · Qeios · doi:10.32388/md04b0 · Cited by 41 · see citing works
Generate, Prune, Select: A Pipeline for Counterspeech Generation against Online Hate Speech
Zhu, W., & Bhat, S. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.12 · Cited by 37 · see citing works
RankGen: Improving Text Generation with Large Ranking Models
Krishna, K., Chang, Y., Wieting, J., & Iyyer, M. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.15 · Cited by 30 · see citing works
The Next Chapter: A Study of Large Language Models in Storytelling
Xie, Z., Cohn, T., & Lau, J. (2023) · International Conference on Natural Language Generation · doi:10.18653/v1/2023.inlg-main.23 · Cited by 98 · see citing works
Unmet Creativity Support Needs in Computationally Supported Creative Writing
Kreminski, M., & Martens, C. (2022) · Proceedings of the First Workshop on Intelligent and Interactive Writing Assistants (In2Writing 2022) · doi:10.18653/v1/2022.in2writing-1.11 · Cited by 26 · see citing works
Can Machines Tell Stories? A Comparative Study of Deep Neural Language Models and Metrics
Das, A., & Verma, R. M. (2020) · IEEE Access · doi:10.1109/access.2020.3023421 · Cited by 26 · see citing works
Dungeons and Dragons as a Dialog Challenge for Artificial Intelligence
Callison-Burch, C., Tomar, G. S., Martin, L. J., Ippolito, D., Bailis, S., & Reitter, D. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.637 · Cited by 30 · see citing works
Decoding Methods in Neural Language Generation: A Survey
Zarrieß, S., Voigt, H., & Schüz, S. (2021) · Information · doi:10.3390/info12090355 · Cited by 26 · see citing works
Narrative Interpolation for Generating and Understanding StoriesPreprint
Wang, S., Durrett, G., & Erk, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.07466 · Cited by 28 · see citing works
Clark, E., & Smith, N. A. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.279 · Cited by 23 · see citing works
Progressive Generation of Long Text.Preprint
Tan, B., Yang, Z., Al-Shedivat, M., Xing, E. P., & Hu, Z. (2020) · arXiv (Cornell University) · Cited by 23 · see citing works
Collaborative Storytelling with Social Robots
Nichols, E., Gao, L., Vasylkiv, Y., & Gómez, R. (2021) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros51168.2021.9636409 · Cited by 22 · see citing works
NAREOR: The Narrative Reordering Problem
Gangal, V., Feng, S. Y., Alikhani, M., Mitamura, T., & Hovy, E. H. (2022) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v36i10.21309 · Cited by 24 · see citing works
Coarse2Fine: Fine-grained Text Classification on Coarsely-grained Annotated Data
Mekala, D., Gangal, V., & Shang, J. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.46 · Cited by 22 · see citing works
Abedi, M., Alshybani, I., Shahadat, M., & Murillo, M. S. (2023) · Qeios · doi:10.32388/md04b0.2 · Cited by 20 · see citing works
Li, P. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.04195 · Cited by 20 · see citing works
How Far are We from Robust Long Abstractive Summarization?
Koh, H. Y., Ju, J., Zhang, H., Liu, M., & Pan, S. (2022) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2022.emnlp-main.172 · Cited by 18 · see citing works
Open-world story generation with structured knowledge enhancement: A comprehensive survey
Wang, Y., Lin, J., Yu, Z., Hu, W. P., & Karlsson, B. F. (2023) · Neurocomputing · doi:10.1016/j.neucom.2023.126792 · Cited by 18 · see citing works
Toward Better Storylines with Sentence-Level Language Models
Ippolito, D., Grangier, D., Eck, D., & Callison-Burch, C. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.666 · Cited by 20 · see citing works
Assessing Discourse Relations in Language Generation from GPT-2
Ko, W.-J., & Li, J. J. (2020) · International Conference on Natural Language Generation · doi:10.18653/v1/2020.inlg-1.8 · Cited by 18 · see citing works
DiscoDVT: Generating Long Text with Discourse-Aware Discrete Variational Transformer
Ji, H., & Huang, M. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.347 · Cited by 19 · see citing works
A Knowledge-Enhanced Pretraining Model for Commonsense Story GenerationPreprint
Guan, J., Huang, F., Zhao, Z., Zhu, X., & Huang, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.05139 · Cited by 18 · see citing works
Design and Analysis of a Collaborative Story Generation Game for Social Robots
Nichols, E., Gao, L., Vasylkiv, Y., & Gómez, R. (2021) · Frontiers in Computer Science · doi:10.3389/fcomp.2021.674333 · Cited by 17 · see citing works
Li, C., Xing, W., Song, Y., & Lyu, B. (2024) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2024.100338 · Cited by 20 · see citing works
Visualize Before You Write: Imagination-Guided Open-Ended Text Generation
Zhu, W., Yan, A., Lu, Y., Xu, W., Wang, X. E., Eckstein, M. P., & Wang, W. Y. (2023) · Findings of the Association for Computational Linguistics: EACL · doi:10.18653/v1/2023.findings-eacl.5 · Cited by 23 · see citing works
Automated Storytelling via Causal, Commonsense Plot Ordering
Ammanabrolu, P., Cheung, W., Broniec, W., & Riedl, M. O. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v35i7.16733 · Cited by 19 · see citing works
Decoding Methods for Neural Narrative Generation
DeLucia, A., Mueller, A., Li, X. L., & Sedoc, J. (2021) · Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2021.gem-1.16 · Cited by 17 · see citing works
What Makes Data-to-Text Generation Hard for Pretrained Language Models?
Keymanesh, M., Benton, A., & Dredze, M. H. (2022) · Workshop on Generation, Evaluation and Metrics (GEM) · doi:10.18653/v1/2022.gem-1.50 · Cited by 12 · see citing works
The use of ChatGPT in creative writing assistance
Фіялка, С. Б., Kornieva, Z., & Гончарук, Т. Г. (2024) · XLinguae · doi:10.18355/xl.2024.17.01.01 · Cited by 11 · see citing works
Li, Q., Zeng, Z., Li, T., & Sun, S. (2024) · Journal of Information Science · doi:10.1177/01655515241281491 · Cited by 10 · see citing works
Nichols, E., Szapiro, D., Vasylkiv, Y., & Gómez, R. (2022) · IEEE RO-MAN · doi:10.1109/ro-man53752.2022.9900606 · Cited by 11 · see citing works
Roemmele, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.04007 · Cited by 16 · see citing works
Neary, M., Fulton, E. A., Rogers, V., Wilson, J., Griffiths, Z., Chuttani, R., & Sacher, P. M. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1460236 · Cited by 9 · see citing works
RecipeGM: A Hierarchical Recipe Generation Model
Reusch, A., Weber, A., Thiele, M., & Lehner, W. (2021) · IEEE International Conference on Data Engineering Workshops (ICDEW) · doi:10.1109/icdew53142.2021.00012 · Cited by 9 · see citing works
Structural Priming Demonstrates Abstract Grammatical Representations in Multilingual Language Models
Michaelov, J. A., Arnett, C., Chang, T. A., & Bergen, B. K. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.227 · Cited by 13 · see citing works
Changing the Mind of Transformers for Topically-Controllable Language Generation
Chang, H.-S., Yuan, J., Iyyer, M., & McCallum, A. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.223 · Cited by 10 · see citing works
To Point or Not to Point: Understanding How Abstractive Summarizers Paraphrase Text
Wilber, M., Timkey, W., & van Schijndel, M. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.298 · Cited by 9 · see citing works
Event Transition Planning for Open-ended Text Generation
Li, Q., Li, P., Bi, W., Ren, Z., Lai, Y., & Kong, L. (2022) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2022.findings-acl.269 · Cited by 9 · see citing works
Is Incoherence Surprising? Targeted Evaluation of Coherence Prediction from Language Models
Beyer, A., Loáiciga, S., & Schlangen, D. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.328 · Cited by 8 · see citing works
Chang, H.-S., Yao, Z., Gon, A., Yu, H. Z., & McCallum, A. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.805 · Cited by 8 · see citing works
Examining the rhetorical capacities of neural language models
Zhu, Z., Pan, C., Abdalla, M., & Rudzicz, F. (2020) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2020.blackboxnlp-1.3 · Cited by 9 · see citing works
Modeling Event Salience in Narratives via Barthes’ Cardinal Functions
Otake, T., Yokoi, S., Inoue, N., Takahashi, R., Kuribayashi, T., & Inui, K. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.160 · Cited by 8 · see citing works
KuiLeiXi: a Chinese Open-Ended Text Adventure Game
Xi, Y., Mao, X., Li, L., Lin, L., Chen, Y., Yang, S., Chen, X., Tao, K., Li, Z., Li, G., et al. (2021) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/2021.acl-demo.21 · Cited by 7 · see citing works
Relating Neural Text Degeneration to Exposure Bias
Chiang, T.-R., & Chen, Y.-N. (2021) · BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP (BlackboxNLP) · doi:10.18653/v1/2021.blackboxnlp-1.16 · Cited by 9 · see citing works
Neural Methods for Data-to-text Generation
Sharma, M., Gogineni, A. K., & Ramakrishnan, N. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3660639 · Cited by 17 · see citing works
Exploring Artificial Jabbering for Automatic Text Comprehension Question Generation
Steuer, T., Filighera, A., & Rensing, C. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-57717-9_1 · Cited by 8 · see citing works
Transformers Can Outperform Humans in Short Creative Writing TasksPreprint
Marco, G., Gonzalo, J. A., & Rello, L. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4673692 · Cited by 7 · see citing works
Seq2Seq dynamic planning network for progressive text generation
Wu, D., Cheng, P., & Zheng, Y. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101687 · Cited by 6 · see citing works
Fiction-Writing Mode: An Effective Control for Human-Machine Collaborative Writing
Zhong, W., Naradowsky, J., Takamura, H., Kobayashi, I., & Miyao, Y. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.128 · Cited by 6 · see citing works
CreativeBot: a Creative Storyteller Agent Developed by Leveraging Pre-trained Language Models
Elgarf, M., & Peters, C. (2022) · IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) · doi:10.1109/iros47612.2022.9981033 · Cited by 7 · see citing works
Unsupervised Pre-training for Natural Language Generation: A Literature ReviewPreprint
Liu, Y., & Lin, Z. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1911.06171 · Cited by 6 · see citing works
Human and Automatic Detection of Generated Text.Preprint
Ippolito, D., Duckworth, D., Callison-Burch, C., & Eck, D. (2019) · arXiv (Cornell University) · Cited by 6 · see citing works
A Temporal Variational Model for Story GenerationPreprint
Wilmot, D., & Keller, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.06807 · Cited by 7 · see citing works
Modelling Suspense in Short Stories as Uncertainty Reduction over Neural Representation
Wilmot, D., & Keller, F. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.161 · Cited by 8 · see citing works
Recent Advances in Chatbot Algorithms, Techniques, and Technologies
Caldarini, G., & Jaf, S. (2023) · Advances in web technologies and engineering book series · doi:10.4018/978-1-6684-6234-8.ch011 · Cited by 6 · see citing works
Sequentially Controlled Text Generation
Spangher, A., Ming, Y., Hua, X., & Peng, N. (2022) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2022.findings-emnlp.509 · Cited by 7 · see citing works
Data Augmentation for Fake Reviews Detection
Liu, M., & Poesio, M. (2023) · International conference Recent advances in natural language processing · doi:10.26615/978-954-452-092-2_073 · Cited by 5 · see citing works
Consistency and Coherency Enhanced Story Generation
Wang, W., Li, P., & Zheng, H.-T. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-72113-8_46 · Cited by 8 · see citing works
Bai, H., Shi, P., Lin, J., Tan, L., Xiong, K., Gao, W., Liu, J., & Li, M. (2021) · doi:10.18653/v1/2021.acl-srw.16 · Cited by 5 · see citing works
Islam, M. A., & Magnani, E. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.239 · Cited by 5 · see citing works
Outline to Story: Fine-grained Controllable Story Generation from Cascaded EventsPreprint
Fang, L., Zeng, T., Liu, C., Bo, L., Dong, W., & Chen, C. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2101.00822 · Cited by 10 · see citing works
- Authors
- See, A., Pappu, A., Saxena, R., Yerukola, A., & Manning, C. D.
- Affiliations
- Stanford University (US)
- Venue
- Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL)
- Published
- 2019
- Pages
- 843-861
- OpenAlex
- W2975003218
- Cited by
- 139
- References
- 31
- Citations by year
- 2026: 1 · 2025: 7 · 2024: 11 · 2023: 27 · 2022: 18 · 2021: 43 · 2020: 30 · 2019: 2
- FWCI
- 8.89
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Computational Natural Language Learning (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Tencent
- Defense Advanced Research Projects Agency
- Army Research Office (W911NF-15-1-, W911NF, W911NF-15)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Artificial Intelligence in Games
- Keywords
- story generation, pre-trained language models, GPT-2, natural language generation, text generation quality, automatic evaluation metrics, event ordering, lexical diversity, decoding algorithms, likelihood maximization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
