CoChat

Where to Hide a Stolen Elephant: Leaps in Creative Writing with Multimodal Machine Intelligence

Open access
Singh, Nikhil, Bernal, Guillermo, Savchenko, Daria, Glassman, Elena L. · 2023 · ACM Transactions on Computer-Human Interaction

The DOI matches its Crossref record; 79 of the 79 references it cites were found in a registry or an index.

Abstract

While developing a story, novices and published writers alike have had to look outside themselves for inspiration. Language models have recently been able to generate text fluently, producing new stochastic narratives upon request. However, effectively integrating such capabilities with human cognitive faculties and creative processes remains challenging. We propose to investigate this integration with a multimodal writing support interface that offers writing suggestions textually, visually, and aurally. We conduct an extensive study that combines elicitation of prior expectations before writing, observation and semi-structured interviews during writing, and outcome evaluations after writing. Our results illustrate the individual and situational variation in machine-in-the-loop writing approaches, suggestion acceptance, and ways the system is helpful. Centrally, we report how participants perform integrative leaps , by which they do cognitive work to integrate suggestions of varying semantic relevance into their developing stories. We interpret these findings, offering modeling and design recommendations for future creative writing support technologies.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3511599

Cites

79 of 79 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 79 references

Research works printed but not linked: 21

  1. Kenneth Arnold Kai-Wei Chang and Adam Kalai. 2017. Counterfactual Language Model Adaptation for Suggesting Phrases. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 2: Short Papers) . 49–54.
  2. Eden Bensaid. 2021. Multimodal generative models for storytelling. Master’s Thesis. Massachusetts Institute of Technology.
  3. Eden Bensaid Mauro Martino Benjamin Hoover Jacob Andreas and Hendrik Strobelt. 2021. FairyTailor: A multimodal generative framework for storytelling. arXiv:2108.04324. Retrieved from https://arxiv.org/abs/2108.04324. · arXiv
  4. Lukas Bieringer Kathrin Grosse Michael Backes and Katharina Krombholz. 2021. Mental Models of Adversarial Machine Learning. In ICCV 2021 Workshop on Adversarial Robustness in the Real World .
  5. Margaret A. Boden. 2004. The creative mind: Myths and mechanisms. Routledge.
  6. Tom B. Brown Benjamin Mann Nick Ryder Melanie Subbiah Jared Kaplan Prafulla Dhariwal Arvind Neelakantan Pranav Shyam Girish Sastry Amanda Askell Sandhini Agarwal Ariel Herbert-Voss Gretchen Krueger T. J. Henighan Rewon Child Aditya Ramesh Daniel M. Ziegler Jeff Wu Clemens Winter Christopher Hesse Mark Chen Eric Sigler Mateusz Litwin Scott Gray Benjamin Chess Jack Clark Christopher Berner Sam McCandlish Alec Radford Ilya Sutskever and Dario Amodei. 2020. Language models are few-shot learners. Advances in Neural Information Processing Systems 33 (2020) 1877–1901.
  7. Alex Calderwood, Vivian Qiu, Katy Ilonka Gero, and Lydia B. Chilton. 2020. How novelists use generative language models: An exploratory user study. In Proceedings of the HAI-GEN+ user2agent@ IUI.
  8. Orson Scott Card. 1990. How to Write Science Fiction and Fantasy. Writer’s digest books Cincinnati, OH.
  9. Michelene T. H. Chi. 2000. Self-explaining expository texts: The dual processes of generating inferences and repairing mental models. Advances in Instructional Psychology 5 (2000), 161–238.
  10. Stephen R. Donaldson. 2008. The Gap Into Conflict: The Real Story. CNIB.
  11. Angela Fan Mike Lewis and Yann Dauphin. 2018. Hierarchical Neural Story Generation. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . 889–898.
  12. 10.2307/356600 · doi:10.2307/356600 · Paper page
  13. 10.1207/s15516709cog1404_3 · doi:10.1207/s15516709cog1404_3 · Paper page
  14. Roger Levy and Galen Andrew. 2006. Tregex and Tsurgeon: Tools for querying and manipulating tree data structures.. In Proceedings of the LREC. Citeseer, 2231–2234.
  15. Ellen W. Nold. 1981. Revising. Writing: The Nature, Development, and Teaching of Written Communication 2 (1981), 67–79.
  16. Ben Pietrzak Ben Swanson Kory Mathewson Monica Dinculescu and Sherol Chen. 2021. Story centaur: Large language model few shot learning as a creative writing tool. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations.
  17. Paul A. Prior. 2005. A Sociocultural Theory of Writing. In The Handbook of Writing Research . Guilford Press 54–66.
  18. Alec Radford Jong Wook Kim Chris Hallacy Aditya Ramesh Gabriel Goh Sandhini Agarwal Girish Sastry Amanda Askell Pamela Mishkin Jack Clark Gretchen Krueger and Ilya Sutskever. 2021. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning . PMLR 8748–8763.
  19. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9.
  20. David Ramsay, Ishwarya Ananthabhotla, and Joseph Paradiso. 2019. The intrinsic memorability of everyday sounds. In Proceedings of the Audio Engineering Society Conference: 2019 AES International Conference on Immersive and Interactive Audio. Audio Engineering Society.
  21. Thomas B. Ward and E. Thomas Lawson. 2009. Creative Cognition in Science Fiction and Fantasy Writing . Cambridge University Press.

Cited by

Items 101–124 of 124

Metadata

Authors
Singh, N., Bernal, G., Savchenko, D., & Glassman, E. L.
Affiliations
  • MIT Media Lab (US)
  • Harvard University (US)
Venue
ACM Transactions on Computer-Human Interaction
Published
2023-09-23
Volume
30
Issue
5
Pages
1-57
ISSN
  • 1073-0516 (ISSN-L)
  • 1557-7325
OpenAlex
W4213451626
Cited by
119
References
77
Citations by year
2026: 15 · 2025: 32 · 2024: 38 · 2023: 23
FWCI
12.82
Type
journal article
Language
en
Open access
Locations
  • ACM Transactions on Computer-Human Interaction (publishedVersion, cc-by)
  • DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc)
Funders
NSF (2107391)
Topics
  • Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
  • Physical Sciences › Computer Science › Computer Graphics and Computer-Aided Design › Digital Media and Visual Art
  • Social Sciences › Arts and Humanities › Visual Arts and Performing Arts › Artistic and Creative Research
Keywords
creative writing, language models, text generation, human-AI collaboration, story development

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.