Sparks: Inspiration for Science Writing using Language Models
The DOI matches its Crossref record; 69 of the 69 references it cites were found in a registry or an index.
Abstract
Large-scale language models are rapidly improving, performing well on a wide variety of tasks with little to no customization. In this work we investigate how language models can support science writing, a challenging writing task that is both open-ended and highly constrained. We present a system for generating “sparks”, sentences related to a scientific concept intended to inspire writers. We find that our sparks are more coherent and diverse than a competitive language model baseline, and approach a human-written gold standard. We run a user study with 13 STEM graduate students writing on topics of their own selection and find three main use cases of sparks—inspiration, translation, and perspective—each of which correlates with a unique interaction pattern. We also find that while participants were more likely to select higher quality sparks, the average quality of sparks seen by a given participant did not correlate with their satisfaction with the tool. We end with a discussion about what impacts human satisfaction with AI support tools, considering participant attitudes towards influence, their openness to technology, as well as issues of plagiarism, trustworthiness, and bias in AI.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Gero et al.'s research [39] revealed that the idea generation phase can particularly threaten ownership, with some writers considering the struggle with writer's block as integral to their writerly identity.”
From Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process (2025) · cites this paper as [39]
In the citing paper
“A focus of researchers in this area is conducting empirical studies with writers to discover their writing strategies and requirements for support [9,39,53,63,77,133].”
From Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process (2025) · cites this paper as [39]
In the citing paper
“Researchers in HCI have worked extensively to investigate these dimensions, producing studies characterizing user values, social dynamics, and professional contexts [9,39,53,63,77,105] that influence ownership preferences and how they shape users' attitudes toward AI assistance.”
From Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process (2025) · cites this paper as [39]
In the citing paper
“Gero et al. [39] found that while authenticity and ownership are related, they are not directly correlated-users may not perceive a system that mimics their style as inauthentic.”
From Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process (2025) · cites this paper as Gero et al. [39]
In the citing paper
“Advanced Large Language Models (LLMs) such as ChatGPT [68], GPT-4 [69], PaLM 2 [32], LLaMA 2 [90], and Gemini [73] can enable writers to generate multiple variations of text via natural language prompting [8,15,26,66], potentially reducing the workload of generating text variations.”
From ABScribe: Rapid Exploration & Organization of Multiple Writing Variations in Human-AI Co-Writing Tasks using Large Language Models (2024) · cites this paper as [26]
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 5×
“GPT-3 [11] or Megatron-LM [45], we wanted to limit the size of the model we used as larger models are more expensive to run and take more time to generate text.”
BackgroundMethodInfluentialSentence-BERT: Sentence Embeddings using Siamese BERT-Networks
Reimers, N., & Gurevych, I. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1410 · Cited by 12,778 · cited 1×
“We measure diversity with sentence embeddings [40], in particular we report the average distance between outputs within a given prompt.”
MethodBraun, V., & Clarke, V. (2012) · APA handbook of research methods in psychology, Vol 2: Research designs: Quantitative, qualitative, neuropsychological, and biological. · doi:10.1037/13620-004 · Cited by 10,299
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? 🦜
Bender, E. M., Gebru, T., McMillan-Major, A., & Shmitchell, S. (2021) · Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency · doi:10.1145/3442188.3445922 · Cited by 6,988 · cited 4×
“There have been several calls to be more careful and explicit with the data that these pre-trained models are trained on [7, 8, 14].”
BackgroundMethodInfluentialDistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter
Sanh, V., Debut, L., Chaumond, J., & Wolf, T. (2019) · arXiv · doi:10.48550/arxiv.1910.01108 · Cited by 4,653 · cited 4×
“Anecdotally, we found that DistilGPT2, a ‘distilled’, smaller version of GPT-2 [48], was not able to produce coherent responses to our prompts.”
BackgroundInfluentialSpeech and language processing
(2010) · Day-to-Day Dyslexia in the Classroom · doi:10.4324/9780203461891_chapter_3 · Cited by 3,544 · cited 1×
“Language models are trained on a dataset of text that does not need to be annotated in any way, as the model can simply be trained on what the next word in a sentence will be [29].”
BackgroundConstructing inferences during narrative text comprehension.
Graesser, A. C., Singer, M., & Trabasso, T. (1994) · Psychological Review · doi:10.1037//0033-295x.101.3.371 · Cited by 2,560
Prefix-Tuning: Optimizing Continuous Prompts for Generation
Li, X. L., & Liang, P. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.353 · Cited by 2,473 · cited 2×
“These results however are highly dependent upon many factors1 [35].”
BackgroundMaking Pre-trained Language Models Better Few-shot Learners
Gao, T., Fisch, A., & Chen, D. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.295 · Cited by 1,380 · cited 4×
“Designing prompts for language models has become an active area of research, with many automatic methods being proposed [19, 34].”
BackgroundMethodInfluential
Research works printed but not linked: 6
- Yao Dou , Maxwell Forbes , Rik Koncel-Kedziorski , Noah A. Smith , and Yejin Choi . 2021 . Scarecrow: A Framework for Scrutinizing Machine Text. arXiv:2107.01294 [cs] (July 2021). http://arxiv.org/abs/2107.01294 arXiv:2107.01294. Yao Dou, Maxwell Forbes, Rik Koncel-Kedziorski, Noah A. Smith, and Yejin Choi. 2021. Scarecrow: A Framework for Scrutinizing Machine Text. arXiv:2107.01294 [cs] (July 2021). http://arxiv.org/abs/2107.01294 arXiv:2107.01294. · arXiv
- 10.2307/356600 · doi:10.2307/356600 · Paper page
- Katy Ilonka Gero Chris Kedzie Savvas Petridis and Lydia Chilton. 2021. Lightweight Decoding Strategies for Increasing Specificity. arXiv preprint arXiv:2110.11850(2021). Katy Ilonka Gero Chris Kedzie Savvas Petridis and Lydia Chilton. 2021. Lightweight Decoding Strategies for Increasing Specificity. arXiv preprint arXiv:2110.11850(2021). · arXiv
- Mina Lee Percy Liang and Qian Yang. 2022. CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities. CoRR abs/2201.06796(2022). arXiv:2201.06796https://arxiv.org/abs/2201.06796 Mina Lee Percy Liang and Qian Yang. 2022. CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities. CoRR abs/2201.06796(2022). arXiv:2201.06796https://arxiv.org/abs/2201.06796 · arXiv
- Oliver Schmitt and Daniel Buschek. 2021. CharacterChat: Supporting the Creation of Fictional Characters through Conversation and Progressive Manifestation with a Chatbot. In Creativity and Cognition. ACM Virtual Event Italy 1-10. https://doi.org/10.1145/3450741.3465253 · DOI · Paper page
- Tongshuang Wu , Michael Terry , and Carrie J. Cai . 2021. AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts. arXiv:2110.01691 [cs] (Oct . 2021 ). http://arxiv.org/abs/2110.01691 arXiv:2110.01691. Tongshuang Wu, Michael Terry, and Carrie J. Cai. 2021. AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts. arXiv:2110.01691 [cs] (Oct. 2021). http://arxiv.org/abs/2110.01691 arXiv:2110.01691. · arXiv
Cited by
Prompt Engineering with ChatGPT: A Guide for Academic Writers
Giray, L. (2023) · Annals of Biomedical Engineering · doi:10.1007/s10439-023-03272-4 · Cited by 776
Sharma, A., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00593-2 · Cited by 500
“…between expressed empathy and positive For AI-assisted writing, for instance, we must build AI models that generate actionable writing suggestions and simultaneously design human-facing systems that help people see, understand and act on those suggestions in a just-in-time fashion 17–23 .”
BackgroundMirowski, P., Mathewson, K. W., Pittman, J., & Evans, R. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581225 · Cited by 240
“Some studies consulted with expert linguists [31], university students in the sciences [42] or humanities [123], and amateur writers [2, 21, 62, 100, 125].”
BackgroundLarge-scale Text-to-Image Generation Models for Visual Artists’ Creative Works
Ko, H.-K., Park, G., Jeon, H., Jo, J., Kim, J., & Seo, J. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584078 · Cited by 189
“The vast majority of the publications targeted general public or they simply unspecified the user (45.8%, 33 publications).”
BackgroundInfluentialSuh, S., Chen, M., Min, B., Li, T. J.-J., & Xia, H. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642400 · Cited by 148
Artificial intelligence as a tool for creativity
Ivcevic, Z., & Grandinetti, M. (2024) · Journal of Creativity · doi:10.1016/j.yjoc.2024.100079 · Cited by 134
A Design Space for Intelligent and Interactive Writing Assistants
Lee, M., Gero, K. I., Chung, J. J. Y., Shum, S. B., Raheja, V., Shen, H., Venugopalan, S., Wambsganss, T., Zhou, D., Alghamdi, E. A., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642697 · Cited by 136
Self-citationGraphologue: Exploring Large Language Model Responses with Interactive Diagrams
Jiang, P., Rayan, J., Dow, S. P., & Xia, H. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606737 · Cited by 135
“…release of LLMs have sparked imaginations for applying this technology to various domains such as programming [21, 67, 82], writing support [25, 35, 92], learning [7, 50, 56], and many others [11, 18, 30, 74], and further promoting the notion of natural language interfaces the HCI community…”
BackgroundSocial Dynamics of AI Support in Creative Writing
Gero, K. I., Long, T., & Chilton, L. B. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580782 · Cited by 129
“[15, 16, 22]) where the responses to tools can vary widely, and is often unexplained by system quality.”
BackgroundMethodInfluentialSelf-citationMindfulDiary: Harnessing Large Language Model to Support Psychiatric Patients' Journaling
Kim, T., Bae, S., Kim, H. A., Lee, S.-W., Hong, H., Yang, C., & Kim, Y.-H. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642937 · Cited by 112
“We hope that this research provides a case study and insight into the development of an LLM-driven chatbot for mental health that is clinically relevant and reflects the needs and experiences of MHPs.”
MethodHan, A., Zhou, X., Cai, Z., Han, S., Ko, R., Corrigan, S., & Peppler, K. A. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642438 · Cited by 114
Zhang, Z., Gao, J., Dhaliwal, R. S., & Li, T. J.-J. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606800 · Cited by 108
“For example, at the higher level of ideation, Sparks [22] facilitates ideation for scientifc writers by using an LLM to suggest potential ideas related to given sci-entifc concepts.”
BackgroundInfluentialAngleKindling: Supporting Journalistic Angle Ideation with Large Language Models
Petridis, S., Diakopoulos, N., Crowston, K., Hansen, M., Henderson, K., Jastrzebski, S., Nickerson, J. V., & Chilton, L. B. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580907 · Cited by 98
Self-citationWan, Q., Hu, S., Zhang, Y., Wang, P., Wen, B., & Lu, Z. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637361 · Cited by 88
“Furthermore, the Human-AI Co-creativity process enhances previous understanding of human-AI co-creative experience [13, 29, 52], by incorporatingWallas’ creativity theories in the two prewriting tasks.”
BackgroundResultInfluentialDang, H., Goller, S., Lehmann, F., & Buschek, D. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580969 · Cited by 83
A systematic review of artificial intelligence technologies used for story writing
Fang, X., Ng, D. T. K., Leung, J. K. L., & Chu, S. K. W. (2023) · Education and Information Technologies · doi:10.1007/s10639-023-11741-5 · Cited by 82
“…related to news writing (e.g., Wang, 2021), poetry writing (e.g., Li & Zhang, 2020), fiction writing (e.g., Candello et al., 2019), science writing (Gero et al., 2022), and English writing (e.g., Chang et al., 2021) by using AI technologies were excluded since they do not address the research…”
BackgroundBuilding machines that learn and think with people
Collins, K. M., Sucholutsky, I., Bhatt, U., Chandra, K., Wong, L., Lee, M., Zhang, C. E., Zhi-Xuan, T., Ho, M., Mansinghka, V., et al. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01991-9 · Cited by 73
Kim, T., Shin, D., Kim, Y.-H., & Hong, H. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642693 · Cited by 75
Kim, J., Suh, S., Chilton, L. B., & Xia, H. (2023) · Proceedings of the 2023 ACM Designing Interactive Systems Conference · doi:10.1145/3563657.3595996 · Cited by 61
“Because metaphors are essential in a variety of contexts—such as rhetoric [54, 68], poetry [34, 35], science writing [36], and teaching [20, 65], metaphor generation, along with metaphor identifca-tion, has long been a topic of interest in the artifcial intelligence and natural language processing…”
BackgroundSelf-citationChakrabarty, T., Padmakumar, V., Brahman, F., & Muresan, S. (2024) · Creativity and Cognition · doi:10.1145/3635636.3656201 · Cited by 61
August, T., Wang, L. L., Bragg, J., Hearst, M. A., Head, A., & Lo, K. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3589955 · Cited by 135
“In the context of writing tools, Gero et al. [ 44 ] used generation models to help researchers author “Tweetorials,” a threaded tweet meant to inform a general audience about a scientific concept on Twitter [ 19 ].”
BackgroundReelFramer: Human-AI Co-Creation for News-to-Video Translation
Wang, S., Menon, S., Long, T., Henderson, K., Li, D., Crowston, K., Hansen, M., Nickerson, J. V., & Chilton, L. B. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642868 · Cited by 55
“Generative AI has already been hugely successful in supporting creative tasks, including music composition [24], visual design [22, 44], and writing [7, 11, 39].”
BackgroundMethodSelf-citationKim, T. S., Lee, Y., Chang, M., & Kim, J. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606833 · Cited by 56
“Other tools support auxiliary processes during writing such as ideation [31, 32, 75, 116], revision [25, 56], and reflection [20, 91].”
BackgroundReza, M., Laundry, N. M., Musabirov, I., Dushniku, P., Yu, Z. Y. “., Mittal, K., Grossman, T., Liut, M., Kuzminykh, A., & Williams, J. J. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641899 · Cited by 52
DATATALES: Investigating the use of Large Language Models for Authoring Data-Driven Articles
Sultanum, N., & Srinivasan, A. (2023) · 2023 IEEE Visualization and Visual Analytics (VIS) · doi:10.1109/vis54172.2023.00055 · Cited by 52
“These include creative writing support tools such as plot suggestions [30], journalistic angle ideation [24] and co-writing of theater scripts [21]; as well as technical writing support such as argumentative writing [41], scientifc writing [9], and reverse outlining for manuscript revision [4].”
BackgroundContextCam: Bridging Context Awareness with Creative Human-AI Image Co-Creation
Fan, X., Wu, Z., Yu, C., Rao, F., Shi, W., & Tu, T. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642129 · Cited by 51
Sarkar, A. (2023) · Proceedings of the 2nd Annual Meeting of the Symposium on Human-Computer Interaction for Work · doi:10.1145/3596671.3597650 · Cited by 50
“Besides the studies mentioned above, there have been many other investigations of how contemporary generative AI supports creative and knowledge work [30, 33, 39, 57, 69, 95, 97, 107, 132, 134, 146].”
ResultHoque, M. N., Mashiat, T., Ghai, B., Shelton, C. D., Chevalier, F., Kraus, K., & Elmqvist, N. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641895 · Cited by 48
Moya, B., Eaton, S., Pethrick, H., Hayden, A., Brennan, R., Wiens, J., & McDermott, B. (2024) · Canadian Perspectives on Academic Integrity · doi:10.55016/ojs/cpai.v7i3.78123 · Cited by 48
“Results show no straightforward answer to this question but give some insights (Anson, 2022; Fyfe, 2022; Gero et al., 2022).”
BackgroundMethodInfluentialThe Silicon Ceiling: Auditing GPT’s Race and Gender Biases in Hiring
Armstrong, L., Liu, A., MacNeil, S., & Metaxa, D. (2024) · ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO) · doi:10.1145/3689904.3694699 · Cited by 120
Feng, K. J. K., Li, T. W., & Zhang, A. X. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581273 · Cited by 78
Zhou, Z., Li, J., Zhang, Z., Yu, J., & Duh, H. B. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2370635 · Cited by 55
“LLMs can enable open conversations that offer insights into human perspectives (Gero et al., 2022), collecting demographic information through open-ended questions (Dengel et al., 2023), real-world field studies (Cascella et al., 2023), and informal interviews conducted with everyday language…”
Rambler: Supporting Writing With Speech via LLM-Assisted Gist Manipulation
Lin, S., Warner, J., Zamfirescu-Pereira, J. D., Lee, M. G., Jain, S., Cai, S., Lertvittayakumjorn, P., Huang, M. X., Zhai, S., Hartmann, B., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642217 · Cited by 39
From Pen to Prompt: How Creative Writers Integrate AI into their Writing Practice
Guo, A., Sathyanarayanan, S., Wang, L., Heer, J., & Zhang, A. X. (2025) · Creativity and Cognition · doi:10.1145/3698061.3726910 · Cited by 79
Mirowski, P., Love, J., Mathewson, K., & Mohamed, S. (2024) · The 2024 ACM Conference on Fairness Accountability and Transparency · doi:10.1145/3630106.3658993 · Cited by 37
'It was 80% me, 20% AI': Seeking Authenticity in Co-Writing with Large Language Models
Hwang, A. H.-C., Liao, Q. V., Blodgett, S. L., Olteanu, A., & Trischler, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711020 · Cited by 42
Ding, Z., Srinivasan, A., Macneil, S., & Chan, J. (2023) · Creativity and Cognition · doi:10.1145/3591196.3593516 · Cited by 35
“For example, researchers have explored the potential of LLMs to replicate or assist creative writing tasks such as metaphor generation [9, 31], science writing [32] and storytelling [3, 18, 25, 62, 66, 69, 75].”
BackgroundUnderstanding Human-AI Workflows for Generating Personas
Shin, J., Hedderich, M. A., Rey, B. J., Lucero, A., & Oulasvirta, A. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3660729 · Cited by 83
“Accordingly, the authors developed ‘human frst–compute last’ workfows, wherein human experts set the initial parameters while AI performed the function of fne-tuning the parameters.”
BackgroundPromptInfuser: How Tightly Coupling AI and UI Design Impacts Designers’ Workflows
Petridis, S., Terry, M. A., & Cai, C. J. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661613 · Cited by 66
“These models are being used to create prototypes that support a variety of tasks, including music composition [24], designing visual art [22, 23, 38], simulating social communities [27, 28] and writing [4, 7, 16, 29, 48].”
MethodAshkinaze, J., Mendelsohn, J., Qiwei, L., Budak, C., & Gilbert, E. (2025) · Proceedings of the ACM Collective Intelligence Conference · doi:10.1145/3715928.3737481 · Cited by 33
PodReels: Human-AI Co-Creation of Video Podcast Teasers
Wang, S., Ning, Z., Truong, A., Dontcheva, M., Li, D., & Chilton, L. B. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661591 · Cited by 55
“Although a short summary might seem similar to a teaser due to their similar lengths, they serve very different purposes.”
BackgroundSelf-citationGuo, A., Pataranutaporn, P., & Maes, P. (2024) · Extended Abstracts of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613905.3650892 · Cited by 32
Generative AI and science communication in the physical sciences
Biyela, S., Dihal, K., Gero, K. I., Ippolito, D., Menczer, F., Schäfer, M. S., & Yokoyama, H. (2024) · Nature Reviews Physics · doi:10.1038/s42254-024-00691-7 · Cited by 40
Self-citationNot Just Novelty: A Longitudinal Study on Utility and Customization of an AI Workflow
Long, T., Gero, K. I., & Chilton, L. B. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661587 · Cited by 60
Self-citationPetridis, S., Wedin, B. D., Wexler, J., Pushkarna, M., Donsbach, A., Goyal, N., Cai, C. J., & Terry, M. (2024) · Proceedings of the 29th International Conference on Intelligent User Interfaces · doi:10.1145/3640543.3645144 · Cited by 31
Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process
Reza, M., Thomas-Mitchell, J., Dushniku, P., Laundry, N., Williams, J. J., & Kuzminykh, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757566 · Cited by 37
Exploring the application of LLM-based AI in UX design: an empirical case study of ChatGPT
Zhou, Z., Li, Y., & Yu, J. (2024) · Human-Computer Interaction · doi:10.1080/07370024.2024.2420991 · Cited by 45
“For example, LLM applications like ChatGPT can perform complex tasks, such as decision-making (Ferguson et al., 2023), summarizing information (Brown et al., 2020; H. Li, 2022) and brainstorming new ideas (Ilonka Gero et al., 2022a).”
Friction: Deciphering Writing Feedback into Writing Revisions through LLM-Assisted Reflection
Zhang, C., Ju, K., Bidoshi, P., Yen, Y.-C. G., & Rzeszotarski, J. M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714316 · Cited by 48
Generative AI in User-Generated Content
Hua, Y., Niu, S., Cai, J., Chilton, L. B., Heuer, H., & Wohn, D. Y. (2024) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3613905.3636315 · Cited by 51
Self-citationM4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection
Wang, Y., Mansurov, J., Ivanov, P., Su, J., Shelmanov, A., Tsvigun, A., Mohammed Afzal, O., Mahmoud, T., Puccetti, G., Arnold, T., et al. (2024) · Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/2024.acl-long.218 · Cited by 25
IdeationWeb: Tracking the Evolution of Design Ideas in Human-AI Co-Creation
Shen, H., Shen, L., Wu, W., & Zhang, K. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713375 · Cited by 24
“Co-creation with AI is increasingly being applied in various creative felds, such as writing [34, 81], music composition [49, 70, 115], and visual art [45, 80, 113].”
AI-mediated social support: the prospect of human–AI collaboration
Meng, J., Zhang, R., Qin, J., Lee, Y.-J., & Lee, Y. (2025) · Journal of Computer-Mediated Communication · doi:10.1093/jcmc/zmaf013 · Cited by 50
CausalMapper: Challenging designers to think in systems with Causal Maps and Large Language Model
Huang, Z., Quan, K., Chan, J., & MacNeil, S. (2023) · Creativity and Cognition · doi:10.1145/3591196.3596818 · Cited by 24
“Recent advances in Large Language Models (LLMs) trained on a massive corpus of human-generated text have shown the capability of generating diverse and personalized content, such as creative writing inspiration [13, 42] and code snippets [10, 22].”
BackgroundA Map of Exploring Human Interaction Patterns with LLM: Insights into Collaboration and Creativity
Li, J., Li, J., & Su, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60615-1_5 · Cited by 44
"Ronaldo's a poser!": How the Use of Generative AI Shapes Debates in Online Forums
Zeng, Y., Shi, Y., Huang, X., Nah, F. F., & Lc, R. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713829 · Cited by 34
“…is widely adopted in specific use cases such as fiction writing [89, 95], poetry writing [36], the-ater script writing [54], science and scientific writing [18, 32, 67], etc. Prior research has also highlighted the promising future of human-AI co-writing across various application scenarios [50].”
Read, Revise, Repeat: A System Demonstration for Human-in-the-loop Iterative Text Revision
Du, W., Kim, Z. M., Raheja, V., Kumar, D., & Kang, D. (2022) · Workshop on Intelligent and Interactive Writing Assistants (In2Writing) · doi:10.18653/v1/2022.in2writing-1.14 · Cited by 83
“While some prior works on text revision (Co-enen et al., 2021; Padmakumar and He, 2021; Gero et al., 2021; Lee et al., 2022) have proposed human-machine collaborative writing interfaces, they are mostly focused on collecting human-machine interaction data for training better neural models, rather…”
BackgroundWang, J., Hu, H., Wang, Z., Yan, S., Sheng, Y., & He, D. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641917 · Cited by 21
Lettersmith: Scaffolding Written Professional Communication Among College Students
Hui, J. S., & Sprouse, M. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581029 · Cited by 27
“for machine-in-the loop collaborative writing, such as machinegenerated sentence and phrasing suggestions [12, 29, 30, 42] and text evaluation [59, 102].”
BackgroundMethodPortrayal: Leveraging NLP and Visualization for Analyzing Fictional Characters
Hoque, M. N., Ghai, B., Kraus, K., & Elmqvist, N. (2023) · Designing Interactive Systems Conference · doi:10.1145/3563657.3596000 · Cited by 37
“These tools can generate text based on a prompt, often helping writers explore alternate narrative worlds and creative angles [8, 11, 23, 45, 53, 69].”
BackgroundMethodAuthors' Values and Attitudes Towards AI-bridged Scalable Personalization of Creative Language Arts
Kim, T., Han, H., Adar, E., Kay, M., & Chung, J. J. Y. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642529 · Cited by 20
“That is, instead of simply supporting the author’s writing process with LLMs [36, 58], we explore the concept of AI-bridged CLA, where the author conveys their broad artistic vision to AI models and these models present personalized versions of the vision to the audience at scale, according to the…”
BackgroundMethodWhen to Give Feedback: Exploring Tradeoffs in the Timing of Design Feedback
E, J. L., Yen, Y.-C. G., Pan, I. Y., Lin, G. H. C., Li, M., Jin, H., Chen, M., Xia, H., & Dow, S. P. (2024) · Creativity and Cognition · doi:10.1145/3635636.3656183 · Cited by 33
“Creativity tools that envision interactions with AI often focus on generating design inspiration [19, 48], examples [62, 64], or variants on the existing design [84, 97, 98, 100].”
BackgroundBeyond Productivity: Rethinking the Impact of Creativity Support Tools
Rhys Cox, S., Bøjer Djernæs, H., & van Berkel, N. (2025) · Proceedings of the 2025 Conference on Creativity and Cognition · doi:10.1145/3698061.3726924 · Cited by 28
Intelligent Support Engages Writers Through Relevant Cognitive Processes
Göldi, A., Wambsganß, T., Neshaei, S. P., & Rietsche, R. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642549 · Cited by 36
Nguyen, H., Nguyen, V., Ludovise, S., & Santagata, R. (2024) · Learning Media and Technology · doi:10.1080/17439884.2024.2435834 · Cited by 29
“Within science communication, scholars have examined the use of LLMs to provide writing inspiration (Gero, Liu, and Chilton 2022), generate content (Karinshak et al. 2023), and facilitate dialogs about scientific topics (Vaghefi et al. 2023).”
Prompting Generative AI with Interaction-Augmented Instructions
Shen, L., Li, H., Wang, Y., Xie, X., & Qu, H. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3720080 · Cited by 36
Reviewriter: AI-Generated Instructions For Peer Review Writing
Su, X., Wambsganss, T., Rietsche, R., Neshaei, S. P., & Kser, T. (2023) · Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023) · doi:10.18653/v1/2023.bea-1.5 · Cited by 17
“In addition, Gero et al. (2022) demonstrated that students find it faster and easier to draw on language from generated texts than to write a sentence from scratch, even when given well-known information.”
BackgroundInfluentialAkkan, H., & Seyyar, G. K. (2025) · Osteoporosis International · doi:10.1007/s00198-024-07358-0 · Cited by 21
Siddiqui, M. N., Pea, R. D., & Subramonyam, H. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714119 · Cited by 45
“Several LLM co-writing systems have emerged to bridge this gap [8, 12, 15, 16, 26–29, 44, 47, 50, 56, 69, 71].”
Collage is the New Writing: Exploring the Fragmentation of Text and User Interfaces in AI Tools
Buschek, D. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3660681 · Cited by 44
“These designs use a single writing area, plus “intel-ligent” support, such as corrective or reflective feedback [1, 79], examples [41], sentence continuations (e.g. one [11, 17, 44], multiple [17, 56, 58]), or inspiration in a separate view [33, 77].”
MethodCai, Z., Han, A., Zhou, X., Gazulla, E. D., & Peppler, K. (2025) · Proceedings of the 24th Interaction Design and Children · doi:10.1145/3713043.3731506 · Cited by 17
Chakrabarty, T., Laban, P., & Wu, C.-S. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713559 · Cited by 92
TutoAI: a cross-domain framework for AI-assisted mixed-media tutorial creation on physical tasks
Chen, Y., Morariu, V. I., Truong, A., & Liu, Z. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642443 · Cited by 23
Chen, A. Z., Park, C., Darkwa, A., Holliday, R. C., & Best, M. L. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3637429 · Cited by 23
“…especially with the rise of generative large language models, it is important to recognize the centrality of trusted messengers in doing the nuanced and often sensitive work of cultural interpretation and health communication, and for digital tools to support rather than replace them [40, 112].”
BackgroundGenerative AI and Perceptual Harms: Who's Suspected of using LLMs?
Kadoma, K., Metaxa, D., & Naaman, M. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713897 · Cited by 14
Nisi, V., James, S., Bala, P., Bue, A. D., & Nunes, N. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-47655-6_8 · Cited by 18
Do LLMs Meet the Needs of Software Tutorial Writers? Opportunities and Design Implications
Bhat, A., Shrivastava, D., & Guo, J. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3660692 · Cited by 19
“Human-LLM interactions for text generation have been studied across several dimensions, such as needs and values of users [7, 18, 27, 31, 53], writing domains [9, 43, 58], and writing stages [17].”
BackgroundThe Dearth of the Author in AI-Supported Writing
Kreminski, M. (2024) · Workshop on Intelligent and Interactive Writing Assistants (In2Writing) · doi:10.1145/3690712.3690725 · Cited by 35
SARD: A Human-AI Collaborative Story Generation
Radwan, A. Y., Alasmari, K. M., Abdulbagi, O. A., & Alghamdi, E. A. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-76806-4_7 · Cited by 28
Bonnet, S., & Teuteberg, F. (2025) · International Journal of Knowledge Management · doi:10.4018/ijkm.368223 · Cited by 16
Navigating the Unknown: A Chat-Based Collaborative Interface for Personalized Exploratory Tasks
Peng, Y., Qin, X., Zhang, Z., Zhang, J., Lin, Q., Yang, X., Zhang, D., Rajmohan, S., & Zhang, Q. (2025) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3708359.3712093 · Cited by 28
von Brackel-Schmidt, C., Kučević, E., Leible, S., Simic, D., Gücük, G.-L., & Schmidt, F. N. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-61315-9_19 · Cited by 21
Why So Serious? Exploring Timely Humorous Comments in AAC Through AI-Powered Interfaces
Weinberg, T. M., Kadoma, K., Penuela, R. E. G., Valencia, S., & Roumen, T. J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714102 · Cited by 21
Seeking Inspiration through Human-LLM Interaction
Lin, X., Huang, H., Huang, K., Shu, X., & Vines, J. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713259 · Cited by 24
Wambsganß, T., Su, X., Swamy, V., Neshaei, S. P., Rietsche, R., & Käser, T. (2023) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2023.findings-emnlp.689 · Cited by 38
Ding, Z. (2024) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3640544.3645241 · Cited by 18
“These include tasks such as crafting analogies, metaphors, analogous design concepts [2, 18, 26], slogans [9], and tweetorials [16].”
BackgroundLiu, Y., Wen, Z., Weng, L., Woodman, O., Yang, Y., & Chen, W. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3410523 · Cited by 18
“Other works focus on designing interfaces for specific writing tasks, such as email writing [52], story creation [14], argumentative writing [16], scientific writing [5] and journalistic angle ideation [53].”
Qiao, H., Vermeulen, J., Fitzmaurice, G., & Matejka, J. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714103 · Cited by 11
Prompt Engineering Guidelines for Using Large Language Models in Requirements Engineering
Ronanki, K., Arvidsson, S., & Axell, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04207-1_17 · Cited by 30
Large Language Models are Catalyzing Chemistry EducationPreprint
Du, Y., Duan, C., Bran, A. M., Sotnikova, A., Qu, Y., Kulik, H. J., Bosselut, A., Xu, J., & Schwaller, P. (2024) · ChemRxiv · doi:10.26434/chemrxiv-2024-h722v · Cited by 14
Mapping the Design Space of Interactions in Human-AI Text Co-creation Tasks
Ding, Z., & Chan, J. (2023) · arXiv · doi:10.48550/arxiv.2303.06430 · Cited by 9
“By creative, we mean outputs that are both novel and useful [22, 23], including generating analogies / metaphors / analogous design concepts [2, 16, 26, 29], slogans [8] and inspirations for tweetorials (short technical explanations on Twitter) [14].”
BackgroundFan, H., Zhou, C., Yu, H., Wu, X., Gu, J., & Peng, Z. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714111 · Cited by 24
Mixed-Initiative Interaction with Computational Generative Systems
Lehmann, F. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3577061 · Cited by 17
“[8], investigated how language models can support science writing.”
BackgroundSun, Y., Li, X., Peng, J., & Gao, Z. (2023) · ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp) · doi:10.1145/3594739.3610695 · Cited by 15
Empowering Children’s AI Literacy Through Co-Creating Stories with LLM
Han, A., & Han, S. (2025) · Proceedings of the 24th Interaction Design and Children · doi:10.1145/3713043.3731520 · Cited by 8
Imamura, S., Hiraki, H., & Rekimoto, J. (2024) · Augmented Humans International Conference (AHs) · doi:10.1145/3652920.3652931 · Cited by 13
Jääskeläinen, P., Holzapfel, A., & Eriksson, E. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3661619 · Cited by 11
“Although the VSD of Creative AI has been largely overlooked to date, user-centered AI design is well established [1, 6, 17, 40, 85].”
BackgroundBala, P., James, S., Bue, A. D., & Nisi, V. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-22298-6_9 · Cited by 10
An Exploratory Experiment Using ChatGPT in the Idea Generation Process for Product-Service System
Yi, Y., & Yeoun, M.-H. (2023) · Archives of Design Research · doi:10.15187/adr.2023.11.36.4.271 · Cited by 8
No Evidence for LLMs Being Useful in Problem Reframing
Shin, J., Polyanskaya, A., Lucero, A., & Oulasvirta, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713273 · Cited by 17
User-centric Reflective Writing Assistance: Leveraging RAG for Enhanced Personalized Support
Neshaei, S. P., Tashkovska, M., Mejia‐Domenzain, P., Wambsganß, T., & Käser, T. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719899 · Cited by 13
Metadata
- Authors
- Gero, K. I., Liu, V., & Chilton, L.
- Affiliations
- Columbia University (US)
- Venue
- Designing Interactive Systems Conference
- Publisher
- ACM
- Published
- 2022-06-13
- Pages
- 1002-1019
- OpenAlex
- W3206079177
- Cited by
- 161
- References
- 62
- Citations by year
- 2026: 24 · 2025: 52 · 2024: 54 · 2023: 28 · 2022: 3
- FWCI
- 30.63
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- Designing Interactive Systems Conference (publishedVersion)
- Funders
- Brown Institute for Media Innovation (Magic Grant)
- NSF (DGE - 1644869)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Keywords
- scientific writing, large language models, AI-assisted writing, Spark, scientific concepts, writing inspiration, STEM graduate students, user study, plagiarism, trust in AI, algorithmic bias, writing support tools
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.