The Impact of Multiple Parallel Phrase Suggestions on Email Input and Composition Behaviour of Native and Non-Native English Writers
Open accessThe DOI matches its Crossref record; 80 of the 81 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
We present an in-depth analysis of the impact of multi-word suggestion choices from a neural language model on user behaviour regarding input and text composition in email writing. Our study for the first time compares different numbers of parallel suggestions, and use by native and non-native English writers, to explore a trade-off of “efficiency vs ideation”, emerging from recent literature. We built a text editor prototype with a neural language model (GPT-2), refined in a prestudy with 30 people. In an online study (N=156), people composed emails in four conditions (0/1/3/6 parallel suggestions). Our results reveal (1) benefits for ideation, and costs for efficiency, when suggesting multiple phrases; (2) that non-native speakers benefit more from more suggestions; and (3) further insights into behaviour patterns. We discuss implications for research, the design of interactive suggestion systems, and the vision of supporting writers with AI instead of replacing them.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“Writing with suggestions correlates with shorter and more predictable texts being written [4], along with increased use of standard phrases when writing with a language model [17, 27].”
From Co-Writing with Opinionated Language Models Affects Users’ Views (2023) · cites this paper as [27]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Minimal AI assistance, such as simple word suggestions or basic grammatical corrections, may not offer substantial benefits to writers.This is particularly true for experienced writers who may already possess a rich vocabulary and a strong grasp of syntax.”
From Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models (2024) · cites this paper as [10]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Planning and translating support are intermingled due to the frequent use of longer AI outputs that directly ideate and write for the user, though these are generally presented as suggestions in order to preserve the user's engagement with the text [15,65].”
From Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process (2025) · cites this paper as [15]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“However, most previous work considered restricted interaction settings (e.g.strict turn-taking [15]) and adapted LMs and interfaces (e.g.fnd-tuned GPT-2 [58]) on specifc tasks (e.g.email writing [8]), thereby making it hard to generalize to other tasks and confgurations of the same model.”
From CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities (2022) · cites this paper as [8]
The cited source says what the claim attributes to it.
In the citing paper
“Although this approach may provide less rich insights than interviews, it can cover relatively diverse contexts across tasks and writers, while allowing for a fne-grained analysis of interactions.”
From CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities (2022) · cites this paper as [8]
10 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2025) · doi:10.32614/r.manuals · Cited by 352,181
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
Fitting Linear Mixed-Effects Models Using lme4
Bates, D., Mächler, M., Bolker, B., & Walker, S. (2015) · Journal of Statistical Software · doi:10.18637/jss.v067.i01 · Cited by 86,835
Exploiting Generative AI to Scale up Intelligent Tutoring Systems
Jakubův, J., Chvalovský, K., Goertzel, Z., Kaliszyk, C., Olšák, M., Piotrowski, B., Schulz, S., Suda, M., & Urban, J. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,658
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325 · cited 2×
“[12]) have improved many benchmarks, including language models for understanding and generating text [12, 21].”
BackgroundlmerTest Package: Tests in Linear Mixed Effects Models
Kuznetsova, A., Brockhoff, P. B., & Christensen, R. H. B. (2017) · Journal of Statistical Software · doi:10.18637/jss.v082.i13 · Cited by 24,346
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743 · cited 10×
“Columns list the involved HMM states (cf. Figure 3), describe the behaviour, and outline how this interpretation was derived from the state parameters. the rest for evaluation (11 states scored best, considering log-likelihood and BIC [15]).”
BackgroundInfluentialKaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,690
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
Research works printed but not linked: 4
- [14] N. Carr. 2015. The Glass Cage: Where Automation is Taking Us . Random House, London, UK. -
- 10.18653/v1 · doi:10.18653/v1
- [46] Eva Lindgren and Kirk Sullivan. 2019. Observing Writing . Brill, Leiden, Niederlande. https://doi.org/10.1163/9789004392526 - · DOI
- [57] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI Blog 1, 8 (2019), 9. -
Cited by
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676
Sharma, A., Lin, I. W., Miner, A. S., Atkins, D. C., & Althoff, T. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-022-00593-2 · Cited by 500
Lee, M., Liang, P., & Yang, Q. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502030 · Cited by 345
“email writing [8]), thereby making it hard to generalize to other tasks and confgurations of the same model.”
BackgroundInfluentialAI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances
Agarwal, D., Naaman, M., & Vashistha, A. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713564 · Cited by 307
Draxler, F., Werner, A., Lehmann, F., Hoppe, M., Schmidt, A., Buschek, D., & Welsch, R. (2023) · arXiv · doi:10.48550/arxiv.2303.03283 · Cited by 284
Self-citationCo-Writing with Opinionated Language Models Affects Users’ Views
Jakesch, M., Bhat, A., Buschek, D., Zalmanson, L., & Naaman, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581196 · Cited by 265
Self-citationTaleBrush: Sketching Stories with Generative Pretrained Language Models
Chung, J. J. Y., Kim, W., Yoo, K. M., Lee, H., Adar, E., & Chang, M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501819 · Cited by 206
Large-scale Text-to-Image Generation Models for Visual Artists’ Creative Works
Ko, H.-K., Park, G., Jeon, H., Jo, J., Kim, J., & Seo, J. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584078 · Cited by 189
Draxler, F., Werner, A., Lehmann, F., Hoppe, M., Schmidt, A., Buschek, D., & Welsch, R. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3637875 · Cited by 186
“Focusing on input behavior with such systems, Buschek et al. [15] quantitatively identified nine patterns, from ignoring suggestions to chaining several suggested phrases in a row.”
BackgroundSelf-citationYang, D., Zhou, Y., Zhang, Z., Li, T. J.-J., & Lc, R. (2022) · CityUHK Scholars · Cited by 185
Pang, R. Y., Schroeder, H., Smith, K. S., Barocas, S., Xiao, Z., Tseng, E., & Bragg, D. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713726 · Cited by 146
A Design Space for Intelligent and Interactive Writing Assistants
Lee, M., Gero, K. I., Chung, J. J. Y., Shum, S. B., Raheja, V., Shen, H., Venugopalan, S., Wambsganss, T., Zhou, D., Alghamdi, E. A., et al. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642697 · Cited by 136
“• Technology : Vipul Raheja (team lead), Subhashini Venu-gopalan (team lead), Seyed Parsa Neshaei (team member), Disha Shrivastava (team member), Antoine Bosselut (advisor), Sherol Chen (advisor), Joonsuk Park (advisor) • Interaction : Hua Shen (team lead), Thiemo Wambsganss (team lead), Sarah Sterman (team member), T…”
InfluentialSelf-citationWhere to Hide a Stolen Elephant: Leaps in Creative Writing with Multimodal Machine Intelligence
Singh, N., Bernal, G., Savchenko, D., & Glassman, E. L. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3511599 · Cited by 119
“, conducted in parallel with ours, has examined the efects of parallel phrase suggestions on writers in an email-writing task [17].”
BackgroundShaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models
Dhillon, P. S., Molaei, S., Li, J., Golub, M., Zheng, S., & Robert, L. P. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642134 · Cited by 111
Supporting Sensemaking of Large Language Model Outputs at Scale
Gero, K. I., Swoopes, C., Gu, Z., Kummerfeld, J. K., & Glassman, E. L. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642139 · Cited by 52
AI Suggestions Homogenize Writing Toward Western Styles and Diminish Cultural Nuances
Agarwal, D., Naaman, M., & Vashistha, A. (2024) · arXiv · doi:10.48550/arxiv.2409.11360 · Cited by 99
Will AI Console Me when I Lose my Pet? Understanding Perceptions of AI-Mediated Email Writing
Liu, Y., Mittal, A., Yang, D., & Bruckman, A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517731 · Cited by 92
“If an AI mediated communication increases the efficacy of humans engaging in such activities [8, 49], our work suggests that it might be a logical decision to use such AI mediation as receivers would not perceive such mediation negatively.”
BackgroundFairness in Language Models Beyond English: Gaps and Challenges
Ramesh, K., Sitaram, S., & Choudhury, M. (2023) · Findings of the Association for Computational Linguistics: EACL 2023 · doi:10.18653/v1/2023.findings-eacl.157 · Cited by 36
Chakrabarty, T., Laban, P., & Wu, C.-S. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713559 · Cited by 92
Park, H., & Ahn, D. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642785 · Cited by 84
“Because they produce language and dialogue similar to humans in terms of fluency and naturalness, people use and receive help from LLMs when doing various tasks, ranging from mundane (e.g., composing emails [10]) to creative (e.g., writing a novel [32, 49]).”
BackgroundBeyond Text Generation: Supporting Writers with Continuous Automatic Text Summaries
Dang, H., Benharrak, K., Lehmann, F., & Buschek, D. (2022) · Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3526113.3545672 · Cited by 84
Self-citationDang, H., Goller, S., Lehmann, F., & Buschek, D. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580969 · Cited by 83
“Single text suggestions can be shown inline [4, 8, 17, 48], whereas multiple suggestions are shown as popup lists of about three to six entries [6, 26].”
BackgroundInfluentialSelf-citationThe Value, Benefits, and Concerns of Generative AI-Powered Assistance in Writing
Li, Z., Liang, C., Peng, J., & Yin, M. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642625 · Cited by 81
“This assistance could also tailor to diverse writing tasks including stories [12, 13, 29, 41, 43, 45, 46], slogans [13], argumentative essays [29], emails [9, 19], and script writing [33].”
BackgroundBuilding machines that learn and think with people
Collins, K. M., Sucholutsky, I., Bhatt, U., Chandra, K., Wong, L., Lee, M., Zhang, C. E., Zhi-Xuan, T., Ho, M., Mansinghka, V., et al. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01991-9 · Cited by 73
LaMPost: Design and Evaluation of an AI-assisted Email Writing Prototype for Adults with Dyslexia
Goodman, S. M., Buehler, E., Clary, P., Coenen, A., Donsbach, A., Horne, T. N., Lahav, M., MacDonald, R., Michaels, R. B., Narayanan, A., et al. (2022) · Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility · doi:10.1145/3517428.3544819 · Cited by 71
Chakrabarty, T., Padmakumar, V., Brahman, F., & Muresan, S. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2309.12570 · Cited by 70
Chakrabarty, T., Padmakumar, V., Brahman, F., & Muresan, S. (2024) · Creativity and Cognition · doi:10.1145/3635636.3656201 · Cited by 61
Evaluating Human-Language Model Interaction
Lee, M., Srivastava, M., Hardy, A., Thickstun, J., Durmus, E., Paranjape, A., Gerard-Ursin, I., Li, X. L., Ladhak, F., Rong, F., et al. (2022) · arXiv · doi:10.48550/arxiv.2212.09746 · Cited by 61
“pletions (Buschek et al., 2021), and the language, ideation, and collaboration capabilities of the systems (Lee et al.”
BackgroundA Taxonomy for Human-LLM Interaction Modes: An Initial Exploration
Gao, J., Gebreegziabher, S. A., Choo, K. T. W., Li, T. J.-J., Perrault, S. T., & Malone, T. W. (2024) · Extended Abstracts of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613905.3650786 · Cited by 57
Kim, T. S., Lee, Y., Chang, M., & Kim, J. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606833 · Cited by 56
Yeo, S., Lim, G., Gao, J., Zhang, W., & Perrault, S. T. (2024) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642530 · Cited by 54
Towards Automatic Boundary Detection for Human-AI Collaborative Hybrid Essay in Education
Zeng, Z., Sha, L., Li, Y., Yang, K., Gašević, D., & Chen, G. (2024) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v38i20.30258 · Cited by 20
Reza, M., Laundry, N. M., Musabirov, I., Dushniku, P., Yu, Z. Y. “., Mittal, K., Grossman, T., Liut, M., Kuzminykh, A., & Williams, J. J. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3641899 · Cited by 52
Storyfier: Exploring Vocabulary Learning Support with Text Generation Models
Peng, Z., Wang, X., Han, Q., Zhu, J., Ma, X., & Qu, H. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606786 · Cited by 27
Friction: Deciphering Writing Feedback into Writing Revisions through LLM-Assisted Reflection
Zhang, C., Ju, K., Bidoshi, P., Yen, Y.-C. G., & Rzeszotarski, J. M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714316 · Cited by 48
Bhat, A., Agashe, S., Oberoi, P., Mohile, N., Jangir, R., & Joshi, A. (2023) · Proceedings of the 28th International Conference on Intelligent User Interfaces · doi:10.1145/3581641.3584060 · Cited by 47
Bhat, A., Agashe, S., Mohile, N., Oberoi, P., Jangir, R., & Joshi, A. (2022) · arXiv · doi:10.48550/arxiv.2208.00636 · Cited by 47
Siddiqui, M. N., Pea, R. D., & Subramonyam, H. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714119 · Cited by 45
Towards Explainable AI Writing Assistants for Non-native English SpeakersPreprint
Kim, Y., Lee, M., Kim, D., & Lee, S.-J. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2304.02625 · Cited by 46
A Map of Exploring Human Interaction Patterns with LLM: Insights into Collaboration and Creativity
Li, J., Li, J., & Su, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-60615-1_5 · Cited by 44
Collage is the New Writing: Exploring the Fragmentation of Text and User Interfaces in AI Tools
Buschek, D. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3660681 · Cited by 44
Self-citationZhou, Y., & Chen, C. (2025) · Design and Artificial Intelligence · doi:10.1016/j.daai.2025.100017 · Cited by 45
'It was 80% me, 20% AI': Seeking Authenticity in Co-Writing with Large Language Models
Hwang, A. H.-C., Liao, Q. V., Blodgett, S. L., Olteanu, A., & Trischler, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711020 · Cited by 42
Metacognition meets AI : Empowering reflective writing with large language models
Neshaei, S. P., Mejia‐Domenzain, P., Davis, R. L., & Käser, T. (2025) · British Journal of Educational Technology · doi:10.1111/bjet.13601 · Cited by 39
Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process
Reza, M., Thomas-Mitchell, J., Dushniku, P., Laundry, N., Williams, J. J., & Kuzminykh, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757566 · Cited by 37
“The General writing context contains systems that are presented for use in multiple contexts, or where the system design is not adapted to solving problems from a particular contextual domain.”
InfluentialVideoDiff: Human-AI Video Co-Creation with Alternatives
Huh, M., Li, D., Pimmel, K., Shin, H. V., Pavel, A., & Dontcheva, M. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713417 · Cited by 37
Ito, T., Yamashita, N., Kuribayashi, T., Hidaka, M., Suzuki, J., Gao, G., Jamieson, J., & Inui, K. (2023) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3586183.3606810 · Cited by 34
How Does the Disclosure of AI Assistance Affect the Perceptions of Writing?
Li, Z., Liang, C., Peng, J., & Yin, M. (2024) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2024.emnlp-main.279 · Cited by 33
Evaluating Human-AI Hybrid Conversational Systems with Chatbot Message Suggestions
Gao, Z., & Jiang, J. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482340 · Cited by 34
Navigating Complexity in Software Engineering: A Prototype for Comparing GPT-n Solutions
Treude, C. (2023) · IEEE/ACM International Workshop on Bots in Software Engineering (BotSE) · doi:10.1109/botse59190.2023.00008 · Cited by 32
Beyond the Chat: Executable and Verifiable Text-Editing with LLMs
Laban, P., Vig, J., Hearst, M., Xiong, C., & Wu, C.-S. (2024) · Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3654777.3676419 · Cited by 31
“For automated, LLM-based editing suggestions, much prior work [12, 13, 106] uses the LLM for text completion (or auto-complete), in which suggestions are made based on immediately preceding text, which is compatible with a language model’s training objective [103].”
Comparing Sentence-Level Suggestions to Message-Level Suggestions in AI-Mediated Communication
Fu, L., Newman, B., Jakesch, M., & Kreps, S. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581351 · Cited by 31
“More advanced systems combine behavioral data [18] with information at the sentence level [44] to predict users’ intentions to complete entire phrases or sentences [2, 11].”
BackgroundAkal Badi ya Bias: An Exploratory Study of Gender Bias in Hindi Language Technology
Hada, R., Husain, S., Gumma, V., Diddee, H., Yadavalli, A., Seth, A., Kulkarni, N., Gadiraju, U., Vashistha, A., Seshadri, V., et al. (2024) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3630106.3659017 · Cited by 33
ChartDetective: Easy and Accurate Interactive Data Extraction from Complex Vector Charts
Masson, D., Malacria, S., Vogel, D., Lank, E., & Casiez, G. (2023) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581113 · Cited by 31
AI Writing Assistants Influence Topic Choice in Self-Presentation
Poddar, R., Sinha, R., Naaman, M., & Jakesch, M. (2023) · Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544549.3585893 · Cited by 30
“While basic assistants predict single words based on frequency distributions [13, 17], more sophisticated assistants suggest sentence completions or entire paragraphs using the generative power of large language models [9].”
BackgroundSynthBio: A Case Study in Human-AI Collaborative Curation of Text DatasetsPreprint
Yuan, A., Ippolito, D., Nikolaev, V., Callison-Burch, C., Coenen, A., & Gehrmann, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2111.06467 · Cited by 29
Lucy, L., Blodgett, S. L., Shokouhi, M., Wallach, H., & Olteanu, A. (2024) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2024.naacl-long.61 · Cited by 26
Roemmele, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.04007 · Cited by 26
Lehmann, F., Markert, N., Dang, H., & Buschek, D. (2022) · Mensch und Computer 2022 · doi:10.1145/3543758.3543947 · Cited by 25
“Some are more general, not focused on mobile devices, such as a study about the number of suggestions provided during email writing [8].”
BackgroundMethodResultInfluentialSelf-citationTowards Automatic Boundary Detection for Human-AI Collaborative Hybrid Essay in Education
Zeng, Z., Sha, L., Li, Y., Yang, K., Gašević, D., & Chen, G. (2023) · arXiv · doi:10.48550/arxiv.2307.12267 · Cited by 25
“…entirely human-written or entirely AI-generated, Dugan et al. (2023); Cutler et al. (2021) noticed the trends in Human-AI collaborative writing (Buschek, Z¨urn, and Eiband 2021; Lee, Liang, and Yang 2022) and reflected on the binary classifi-cation setting, pointing out that a text could be…”
BackgroundThe AI Memory Gap: Users Misremember What They Created With AI or Without
Zindulka, T., Goller, S., Fernandes, D., Welsch, R., & Buschek, D. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791494 · Cited by 25
Self-citationVisual Story-Writing: Writing by Manipulating Visual Representations of Stories
Masson, D., Zhao, Z., & Chevalier, F. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747758 · Cited by 24
GlassMail: Towards Personalised Wearable Assistant for On-the-Go Email Creation on Smart Glasses
Zhou, C., Yan, Z., Ram, A., Gu, Y. S., Xiang, Y., Liu, C., Huang, Y., Ooi, W. T., & Zhao, S. (2024) · Designing Interactive Systems Conference · doi:10.1145/3643834.3660683 · Cited by 25
Amuse: Human-AI Collaborative Songwriting with Multimodal Inspirations
Kim, Y., Lee, S.-J., & Donahue, C. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713818 · Cited by 23
“We quantitatively assess this alignment by computing the Jensen-Shannon Divergence (JSD) [58] between the distributions of the generated chord progressions and real music data.”
InfluentialXiao, Y., Hancock, C., Agrawal, S., Mehandru, N., Salehi, N., Carpuat, M. J., & Gao, G. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713626 · Cited by 23
Who Wrote this? How Smart Replies Impact Language and Agency in the Workplace
Wenker, K. (2022) · Telematics and Informatics Reports · doi:10.1016/j.teler.2023.100062 · Cited by 24
Ink and algorithm: Exploring temporal dynamics in generative AI ‐assisted writing
Yang, K., Cheng, Y., Zhao, L., Raković, M., Swiecki, Z., Gašević, D., & Chen, G. (2026) · British Journal of Educational Technology · doi:10.1111/bjet.70031 · Cited by 22
LLM-mediated domain-specific voice agents: the case of TextileBot
Zhong, S., Gatti, E., Hardwick, J., Ribul, M., Cho, Y., & Obrist, M. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2456667 · Cited by 22
Does Writing with Language Models Reduce Content Diversity?
Padmakumar, V., & He, H. (2023) · arXiv · doi:10.48550/arxiv.2309.05196 · Cited by 21
Methods for the Design and Evaluation of HCI+NLP SystemsPreprint
Heuer, H., & Buschek, D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.13461 · Cited by 21
Self-citationMapping the Design Space of Human-AI Interaction in Text Summarization
Cheng, R., Smith-Renner, A., Zhang, K., Tetreault, J., & Jaimes-Larrarte, A. (2022) · Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2022.naacl-main.33 · Cited by 20
Authors' Values and Attitudes Towards AI-bridged Scalable Personalization of Creative Language Arts
Kim, T., Han, H., Adar, E., Kay, M., & Chung, J. J. Y. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642529 · Cited by 20
“Researchers also studied how specific designs of LLM, such as the number of suggestions [13] or allowing users to input instructional prompts or not [25] can impact writing.”
BackgroundEmails by LLMs: A Comparison of Language in AI-Generated and Human-Written Emails
Li, W., Lai, Y., Soni, S., & Saha, K. (2025) · ACM Web Science Conference (WebSci) · doi:10.1145/3717867.3717872 · Cited by 19
Reviewriter: AI-Generated Instructions For Peer Review Writing
Su, X., Wambsganss, T., Rietsche, R., Neshaei, S. P., & Kser, T. (2023) · Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023) · doi:10.18653/v1/2023.bea-1.5 · Cited by 17
Reactive Writers: How Co-Writing with AI Changes How We Engage with Ideas
Bhat, A., Quéré, M. A. L., Naaman, M., & Jakesch, M. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791529 · Cited by 17
Mixed-Initiative Interaction with Computational Generative Systems
Lehmann, F. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3577061 · Cited by 17
Understanding and Supporting Formal Email Exchange by Answering AI-Generated Questions
Miura, Y., Yang, C.-L., Kuribayashi, M., Matsumoto, K., Kuzuoka, H., & Morishima, S. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3714016 · Cited by 17
Examining Agency, Expertise, and Roles of AI Systems in AI-Mediated Communication
Mieczkowski, H., & Hancock, J. (2022) · doi:10.31219/osf.io/asnv4 · Cited by 16
A Decision Model for Designing NLP Applications
Chen, E., & Tseng, Y. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3487553.3524921 · Cited by 16
An HCI-Centric Survey and Taxonomy of Human-Generative-AI Interactions
Shi, J., Jain, R., Doh, H., Suzuki, R., & Ramani, K. (2023) · arXiv · doi:10.48550/arxiv.2310.07127 · Cited by 15
Xiao, Y., Chen, Y., Yamashita, N., Chen, Y., Liu, Z., & Gao, G. (2024) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3686942 · Cited by 15
Synthia: Visually Interpreting and Synthesizing Feedback for Writing Revision
Zhang, C., Ju, K. P., Han, Z., Yen, Y.-C. G., & Rzeszotarski, J. M. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747703 · Cited by 15
StorySage: Conversational Autobiography Writing Powered by a Multi-Agent Framework
Talaei, S., Li, M., Grover, K., Hippler, J. K., Yang, D., & Saberi, A. (2025) · ACM Symposium on User Interface Software and Technology (UIST) · doi:10.1145/3746059.3747681 · Cited by 15
Kim, Y., Le, T.-L. V., Kim, D., Lee, M., & Lee, S.-J. (2025) · Designing Interactive Systems Conference · doi:10.1145/3715336.3735740 · Cited by 15
Generative AI and Perceptual Harms: Who's Suspected of using LLMs?
Kadoma, K., Metaxa, D., & Naaman, M. (2025) · Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713897 · Cited by 14
“For the content evaluation and outcomes measurements, we hypothesize the non-dominant group will receive lower evaluations, and will be less likely to be recommended for hiring, controlling for whether they were suspected of using AI.”
Amin, R. M., Kühle, O. H., Buschek, D., & Butz, A. M. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3720243 · Cited by 14
Self-citationIntroduction to Intelligent User Interfaces
Schmidt, A., Mayer, S., & Buschek, D. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3445021 · Cited by 14
Self-citationInterpassivity instead of interactivity? The uses and gratifications of automated features
Chen, C., Lee, S., & Sundar, S. S. (2023) · Behaviour and Information Technology · doi:10.1080/0144929x.2023.2184174 · Cited by 14
Designing for Continuous Interaction with Artificial Intelligence Systems
Wintersberger, P., van Berkel, N., Fereydooni, N., Tag, B., Glassman, E. L., Buschek, D., BLANDFORD, A. E., & Michahelles, F. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3516409 · Cited by 13
Self-citationCollaborative Document Editing with Multiple Users and AI Agents
Lehmann, F., Shauchenka, K., & Buschek, D. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790648 · Cited by 11
Self-citationLehmann, F., & Buschek, D. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.10644 · Cited by 10
Self-citationAssessing Human Evaluations of Cover Letters Written or Edited by AI and Non-Native English Speakers
Tian, X., Hahn, H. M., Carangui, M. K. L., Si, Y., Chen, J., Li, X., & Fussell, S. R. (2025) · International ACM SIGGROUP Conference on Supporting Group Work (GROUP) · doi:10.1145/3688828.3699644 · Cited by 10
CoCo Matrix: Taxonomy of Cognitive Contributions in Co-writing with Intelligent Agents
Wan, R., Gebreegziabher, S. A., Li, T. J.-J., & Badillo-Urquiola, K. (2024) · Creativity and Cognition · doi:10.1145/3635636.3664260 · Cited by 9
Zeng, Z., Liu, S., Sha, L., Li, Z., Yang, K., Liu, S., Gasevic, D., & Chen, G. (2024) · Proceedings of the Thirty-ThirdInternational Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2024/835 · Cited by 8
Wan, Q., Li, J., Wang, H., & Lu, Z. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757497 · Cited by 9
Amin, R. M., Kühle, O. H., Buschek, D., & Butz, A. M. (2026) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3817049 · Cited by 8
Self-citationShah, H., Jaidka, K., Ungar, L., Fagan, J. M., & Grosser, T. J. (2023) · Information · doi:10.3390/info14120661 · Cited by 7
Fine-Grained Detection of AI-Generated Text Using Sentence-Level Segmentation
Teja, L. D. M. S. S., Yadagiri, A., Pakray, P., Chunka, C., & Vardhan, M. S. (2025) · International Joint Conference on Natural Language Processing and Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics (IJCNLP-AACL) · doi:10.18653/v1/2025.findings-ijcnlp.48 · Cited by 7
Baumler, C., & Daumé, H. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3790733 · Cited by 6
Taming Multi-Output Recommenders for Software Engineering
Treude, C. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3559557 · Cited by 6
Metadata
- Authors
- Buschek, D., Zürn, M., & Eiband, M.
- Affiliations
- University of Bayreuth (DE)
- Ludwig-Maximilians-Universität München (DE)
- Venue
- Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2021-05-06
- Pages
- 1-13
- OpenAlex
- W3123978464
- Cited by
- 108
- References
- 78
- Citations by year
- 2026: 13 · 2025: 28 · 2024: 26 · 2023: 21 · 2022: 14 · 2021: 6
- FWCI
- 12.52
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- CHI Conference on Human Factors in Computing Systems (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- ERef Bayreuth (University of Bayreuth) (submittedVersion)
- OSF Preprints (OSF Preprints) (submittedVersion)
- arXiv (Cornell University)
- Funders
- Bavarian State Ministry of Science and the Arts
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- email writing, neural language models, GPT-2, idea generation, writing efficiency, native vs non-native writers, non-native English speakers, AI-assisted writing, suggestion systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.