The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Sparks: Inspiration for Science Writing using Language Models
Abstract
Large-scale language models are rapidly improving, performing well on a wide variety of tasks with little to no customization. In this work we investigate how language models can support science writing, a challenging writing task that is both open-ended and highly constrained. We present a system for generating “sparks”, sentences related to a scientific concept intended to inspire writers. We find that our sparks are more coherent and diverse than a competitive language model baseline, and approach a human-written gold standard. We run a user study with 13 STEM graduate students writing on topics of their own selection and find three main use cases of sparks—inspiration, translation, and perspective—each of which correlates with a unique interaction pattern. We also find that while participants were more likely to select higher quality sparks, the average quality of sparks seen by a given participant did not correlate with their satisfaction with the tool. We end with a discussion about what impacts human satisfaction with AI support tools, considering participant attitudes towards influence, their openness to technology, as well as issues of plagiarism, trustworthiness, and bias in AI.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Gero et al.'s research [39] revealed that the idea generation phase can particularly threaten ownership, with some writers considering the struggle with writer's block as integral to their writerly identity.”
From Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process (2025) · cites this paper as [39]
In the citing paper
“A focus of researchers in this area is conducting empirical studies with writers to discover their writing strategies and requirements for support [9,39,53,63,77,133].”
From Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process (2025) · cites this paper as [39]
In the citing paper
“Researchers in HCI have worked extensively to investigate these dimensions, producing studies characterizing user values, social dynamics, and professional contexts [9,39,53,63,77,105] that influence ownership preferences and how they shape users' attitudes toward AI assistance.”
From Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process (2025) · cites this paper as [39]
In the citing paper
“Gero et al. [39] found that while authenticity and ownership are related, they are not directly correlated-users may not perceive a system that mimics their style as inauthentic.”
From Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process (2025) · cites this paper as Gero et al. [39]
In the citing paper
“Advanced Large Language Models (LLMs) such as ChatGPT [68], GPT-4 [69], PaLM 2 [32], LLaMA 2 [90], and Gemini [73] can enable writers to generate multiple variations of text via natural language prompting [8,15,26,66], potentially reducing the workload of generating text variations.”
From ABScribe: Rapid Exploration & Organization of Multiple Writing Variations in Human-AI Co-Writing Tasks using Large Language Models (2024) · cites this paper as [26]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 5×
“GPT-3 [11] or Megatron-LM [45], we wanted to limit the size of the model we used as larger models are more expensive to run and take more time to generate text.”
BackgroundMethodInfluentialSentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778 · cited 1×
“We measure diversity with sentence embeddings [40], in particular we report the average distance between outputs within a given prompt.”
MethodBraun, V., & Clarke, V. (2012) · APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological. · doi:10.1037/13620-004 · Cited by 10,299
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 4×
“There have been several calls to be more careful and explicit with the data that these pre-trained models are trained on [7, 8, 14].”
BackgroundMethodInfluentialDistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019) · arXiv · doi:10.48550/arxiv.1910.01108 · Cited by 4,653 · cited 4×
“Anecdotally, we found that DistilGPT2, a ‘distilled’, smaller version of GPT-2 [48], was not able to produce coherent responses to our prompts.”
BackgroundInfluentialSpeech and language processing
(2010) · Day-to-Day Dyslexia in the Classroom · doi:10.4324/9780203461891_chapter_3 · Cited by 3,544 · cited 1×
“Language models are trained on a dataset of text that does not need to be annotated in any way, as the model can simply be trained on what the next word in a sentence will be [29].”
BackgroundConstructing inferences during narrative text comprehension.
Graesser, A. C., Singer, M., & Trabasso, T. (1994) · Psychological Review · doi:10.1037//0033-295x.101.3.371 · Cited by 2,560
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Li, X. L., & Liang, P. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.353 · Cited by 2,473 · cited 2×
“These results however are highly dependent upon many factors1 [35].”
BackgroundMaking Pre-trained Language Models Better Few-shot Learners
Gao, T., Fisch, A., & Chen, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.295 · Cited by 1,380 · cited 4×
“Designing prompts for language models has become an active area of research, with many automatic methods being proposed [19, 34].”
BackgroundMethodInfluential
Research works printed but not linked: 6
- Yao Dou , Maxwell Forbes , Rik Koncel-Kedziorski , Noah A. Smith , and Yejin Choi . 2021 . Scarecrow: A Framework for Scrutinizing Machine Text. arXiv:2107.01294 [cs] (July 2021). http://arxiv.org/abs/2107.01294 arXiv:2107.01294. Yao Dou, Maxwell Forbes, Rik Koncel-Kedziorski, Noah A. Smith, and Yejin Choi. 2021. Scarecrow: A Framework for Scrutinizing Machine Text. arXiv:2107.01294 [cs] (July 2021). http://arxiv.org/abs/2107.01294 arXiv:2107.01294. · arXiv
- 10.2307/356600 · doi:10.2307/356600 · Paper page
- Katy Ilonka Gero Chris Kedzie Savvas Petridis and Lydia Chilton. 2021. Lightweight Decoding Strategies for Increasing Specificity. arXiv preprint arXiv:2110.11850(2021). Katy Ilonka Gero Chris Kedzie Savvas Petridis and Lydia Chilton. 2021. Lightweight Decoding Strategies for Increasing Specificity. arXiv preprint arXiv:2110.11850(2021). · arXiv
- Mina Lee Percy Liang and Qian Yang. 2022. CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities. CoRR abs/2201.06796(2022). arXiv:2201.06796https://arxiv.org/abs/2201.06796 Mina Lee Percy Liang and Qian Yang. 2022. CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities. CoRR abs/2201.06796(2022). arXiv:2201.06796https://arxiv.org/abs/2201.06796 · arXiv
- Oliver Schmitt and Daniel Buschek. 2021. CharacterChat: Supporting the Creation of Fictional Characters through Conversation and Progressive Manifestation with a Chatbot. In Creativity and Cognition. ACM Virtual Event Italy 1-10. https://doi.org/10.1145/3450741.3465253 · DOI · Paper page
- Tongshuang Wu , Michael Terry , and Carrie J. Cai . 2021. AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts. arXiv:2110.01691 [cs] (Oct . 2021 ). http://arxiv.org/abs/2110.01691 arXiv:2110.01691. Tongshuang Wu, Michael Terry, and Carrie J. Cai. 2021. AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts. arXiv:2110.01691 [cs] (Oct. 2021). http://arxiv.org/abs/2110.01691 arXiv:2110.01691. · arXiv
Cited by
Prompt Engineering with ChatGPT: A Guide for Academic Writers
Giray, L. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03272-4 · Cited by 776
Sharma, A., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00593-2 · Cited by 500
“…between expressed empathy and positive For AI-assisted writing, for instance, we must build AI models that generate actionable writing suggestions and simultaneously design human-facing systems that help people see, understand and act on those suggestions in a just-in-time fashion 17–23 .”
BackgroundMirowski, P., Mathewson, K. W., Pittman, J., & Evans, R. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581225 · Cited by 240
“Some studies consulted with expert linguists [31], university students in the sciences [42] or humanities [123], and amateur writers [2, 21, 62, 100, 125].”
BackgroundLarge-scale Text-to-Image Generation Models for Visual Artists’ Creative Works
Ko, H.-K., Park, G., Jeon, H., Jo, J., Kim, J., & Seo, J. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584078 · Cited by 189
“The vast majority of the publications targeted general public or they simply unspecified the user (45.8%, 33 publications).”
BackgroundInfluentialSuh, S., Chen, M., Min, B., Li, T. J.-J., & Xia, H. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642400 · Cited by 148
Metadata
- Authors
- Gero, K. I., Liu, V., & Chilton, L.
- Affiliations
- Columbia University (US)
- Venue
- Designing Interactive Systems Conference
- Publisher
- ACM
- Published
- 2022-06-13
- Pages
- 1002-1019
- OpenAlex
- W3206079177
- Cited by
- 161
- References
- 62
- Citations by year
- 2026: 24 · 2025: 52 · 2024: 54 · 2023: 28 · 2022: 3
- FWCI
- 30.63
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Designing Interactive Systems Conference (publishedVersion)
- Funders
- Brown Institute for Media Innovation (Magic Grant)
- NSF (DGE - 1644869)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- scientific writing, large language models, AI-assisted writing, Spark, scientific concepts, writing inspiration, STEM graduate students, user study, plagiarism, trust in AI, algorithmic bias, writing support tools
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Sparks: Inspiration for Science Writing using Language Models. https://app.cochat.ai/papers/10.1145/3532106.3533533
- The paper
- Gero, K. I., Liu, V., & Chilton, L. (2022). Sparks: Inspiration for Science Writing using Language Models. Designing Interactive Systems Conference. https://doi.org/10.1145/3532106.3533533
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.