Beyond Text Generation: Supporting Writers with Continuous Automatic Text Summaries
Open accessThe DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
We propose a text editor to help users plan, structure and reflect on their writing process. It provides continuously updated paragraph-wise summaries as margin annotations, using automatic text summarization. Summary levels range from full text, to selected (central) sentences, down to a collection of keywords. To understand how users interact with this system during writing, we conducted two user studies (N=4 and N=8) in which people wrote analytic essays about a given topic and article. As a key finding, the summaries gave users an external perspective on their writing and helped them to revise the content and scope of their drafted paragraphs. People further used the tool to quickly gain an overview of the text and developed strategies to integrate insights from the automated summaries. More broadly, this work explores and highlights the value of designing AI tools for writers, with Natural Language Processing (NLP) capabilities that go beyond direct text generation and correction.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., & Liu, P. J. (2019) · arXiv · doi:10.48550/arxiv.1910.10683 · Cited by 10,929
Get To The Point: Summarization with Pointer-Generator Networks
See, A., Liu, P. J., & Manning, C. D. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1099 · Cited by 3,775
A cognitive process theory of writing
Flower, L., & Hayes, J. R. (1981) · College Composition & Communication · doi:10.58680/ccc198115885 · Cited by 1,025
Wordcraft: Story Writing With Large Language Models
Yuan, A., Coenen, A., Reif, E., & Ippolito, D. (2022) · 27th International Conference on Intelligent User Interfaces · doi:10.1145/3490099.3511105 · Cited by 374
Lee, M., Liang, P., & Yang, Q. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3502030 · Cited by 345
Rezwana, J., & Maher, M. L. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3519026 · Cited by 342
Digital support for academic writing: A review of technologies and pedagogies
Strobl, C., Ailhaud, E., Benetos, K., Devitt, A., Kruse, O., Proske, A., & Rapp, C. (2019) · Computers & Education · doi:10.1016/j.compedu.2018.12.005 · Cited by 298 · cited 1×
“[43] provide a recent survey of 44 academic writing tools.”
BackgroundCreative Writing with a Machine in the Loop: Case Studies on Slogans and Stories
Clark, E., Ross, A. S., Tan, C., Ji, Y., & Smith, N. A. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172983 · Cited by 257
TaleBrush: Sketching Stories with Generative Pretrained Language Models
Chung, J. J. Y., Kim, W., Yoo, K. M., Lee, H., Adar, E., & Chang, M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501819 · Cited by 206 · cited 1×
“Further work explored controllable generation of plots with AI [7, 46], or addressed writing poetry [14], metaphors [13], slogans [8], fictional characters [40], and emails [4, 18].”
BackgroundGmail Smart Compose: Real-Time Assisted Writing
Chen, M. X., Lee, B. N., Bansal, G., Cao, Y., Zhang, S., Lu, J., Tsay, J., Wang, Y., Dai, A. M., Chen, Z., et al. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330723 · Cited by 169
Metaphoria: An Algorithmic Companion for Metaphor Creation
Gero, K. I., & Chilton, L. B. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300526 · Cited by 146
Where to Hide a Stolen Elephant: Leaps in Creative Writing with Multimodal Machine Intelligence
Singh, N., Bernal, G., Savchenko, D., & Glassman, E. L. (2023) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3511599 · Cited by 119 · cited 2×
“for the next sentence [38] or paragraph [54] or in a sidebar [42]).”
BackgroundSketching NLP: A Case Study of Exploring the Right Things To Design with Language Intelligence
Yang, Q., Cranshaw, J., Amershi, S., Iqbal, S. T., & Teevan, J. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300415 · Cited by 90
Creative Help: A Story Writing Assistant
Roemmele, M., & Gordon, A. S. (2015) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-27036-4_8 · Cited by 83 · cited 1×
“for the next sentence [38] or paragraph [54] or in a sidebar [42]).”
BackgroundSchmitt, O., & Buschek, D. (2021) · Creativity and Cognition · doi:10.1145/3450741.3465253 · Cited by 41
Self-citationJustCorrect: Intelligent Post Hoc Text Correction Techniques on Smartphones
Cui, W., Zhu, S., Zhang, M. R., Schwartz, H. A., Wobbrock, J. O., & Bi, X. (2020) · Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3379337.3415857 · Cited by 24
Research works printed but not linked: 36
- 10.1145/2858036.2858407 · DOI
- Kenneth C Arnold , April M Volzer , and Noah G Madrid . 2021 . Generative Models can Help Writers without Writing for Them . 2nd Workshop on Human-AI Co-Creation with Generative Models - HAI-GEN 2021 (2021), 8. Kenneth C Arnold, April M Volzer, and Noah G Madrid. 2021. Generative Models can Help Writers without Writing for Them. 2nd Workshop on Human-AI Co-Creation with Generative Models - HAI-GEN 2021 (2021), 8.
- 10.1145/3468505 · DOI · Paper page
- 10.1145/3411764.3445372 · DOI · Paper page
- Hakan Ceylan and Rada Mihalcea . 2009. The Decomposition of Human-Written Book Summaries . In Computational Linguistics and Intelligent Text Processing, Alexander Gelbukh (Ed.). Vol. 5449. Springer Berlin Heidelberg , Berlin, Heidelberg, 582–593. https://doi.org/10.1007/978-3-642-00382-0_47 Series Title : Lecture Notes in Computer Science. 10.1007/978-3-642-00382-0_47 Hakan Ceylan and Rada Mihalcea. 2009. The Decomposition of Human-Written Book Summaries. In Computational Linguistics and Intelligent Text Processing, Alexander Gelbukh (Ed.). Vol. 5449. Springer Berlin Heidelberg, Berlin, Heidelberg, 582–593. https://doi.org/10.1007/978-3-642-00382-0_47 Series Title: Lecture Notes in Computer Science. · doi:10.1007/978-3-642-00382-0_47
- Juliet M Corbin . 1990. Basics of qualitative research: Grounded theory procedures and techniques . Sage . Juliet M Corbin. 1990. Basics of qualitative research: Grounded theory procedures and techniques. Sage.
- 10.1093/elt/54.3.235 · DOI · Paper page
- 10.2307/356600 · doi:10.2307/356600 · Paper page
- 10.18653/v1/P17-4008 · DOI
- Han L. Han , Miguel A. Renom , Wendy E. Mackay , and Michel Beaudouin-Lafon . 2020 . Textlets: Supporting Constraints and Consistency in Text Documents . Association for Computing Machinery , New York, NY, USA , 1–13. https://doi.org/10.1145/3313831.3376804 10.1145/3313831.3376804 Han L. Han, Miguel A. Renom, Wendy E. Mackay, and Michel Beaudouin-Lafon. 2020. Textlets: Supporting Constraints and Consistency in Text Documents. Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376804 · DOI
- 10.1145/3377325.3377485 · DOI
- 10.1145/2939672.2939801 · DOI · Paper page
- 10.1109/TPC.2012.2207838 · DOI
- Wojciech Kryscinski , Nitish Shirish Keskar , Bryan McCann , Caiming Xiong , and Richard Socher . 2019. Neural Text Summarization: A Critical Evaluation . In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) . Association for Computational Linguistics , Hong Kong , China, 540–551. https://doi.org/10. 1865 3/v1/D19-1051 10.18653/v1 Wojciech Kryscinski, Nitish Shirish Keskar, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. Neural Text Summarization: A Critical Evaluation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, Hong Kong, China, 540–551. https://doi.org/10.18653/v1/D19-1051 · doi:10.18653/v1
- 10.1016/j.ipl.2017.10.007 · doi:10.1016/j.ipl.2017.10.007
- Karen Sheriff LeVan and Marissa E King . 2017 . Self-Annotation as a Course Practice . Teaching English in the Two Year College 44 , 3 (2017), 289 . Karen Sheriff LeVan and Marissa E King. 2017. Self-Annotation as a Course Practice. Teaching English in the Two Year College 44, 3 (2017), 289. · doi:10.58680/tetyc201729004
- Daniel Li , Thomas Chen , Albert Tung , and Lydia B Chilton . 2021 . Hierarchical Summarization for Longform Spoken Dialog. In The 34th Annual ACM Symposium on User Interface Software and Technology ( Virtual Event, USA) (UIST ’21). Association for Computing Machinery, New York, NY, USA, 582–597. https://doi.org/10.1145/3472749.3474771 10.1145/3472749.3474771 Daniel Li, Thomas Chen, Albert Tung, and Lydia B Chilton. 2021. Hierarchical Summarization for Longform Spoken Dialog. In The 34th Annual ACM Symposium on User Interface Software and Technology (Virtual Event, USA) (UIST ’21). Association for Computing Machinery, New York, NY, USA, 582–597. https://doi.org/10.1145/3472749.3474771 · DOI
- Yang Li , Sayan Sarcar , Sunjun Kim , and Xiangshi Ren . 2020 . Swap: A Replacement-Based Text Revision Technique for Mobile Devices . Association for Computing Machinery , New York, NY, USA , 1–12. https://doi.org/10.1145/3313831.3376217 10.1145/3313831.3376217 Yang Li, Sayan Sarcar, Sunjun Kim, and Xiangshi Ren. 2020. Swap: A Replacement-Based Text Revision Technique for Mobile Devices. Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376217 · DOI
- 10.1109/TCOM.1982.1095380 · DOI
- Kristin Messuri . 2016 . Revision Strategies . The Southwest Respiratory and Critical Care Chronicles 4 , 14 (April 2016), 46–48. https://pulmonarychronicles.com/index.php/pulmonarychronicles/article/view/263 Number: 14. Kristin Messuri. 2016. Revision Strategies. The Southwest Respiratory and Critical Care Chronicles 4, 14 (April 2016), 46–48. https://pulmonarychronicles.com/index.php/pulmonarychronicles/article/view/263 Number: 14.
- 10.12746/swrccc.v4i15.290 · doi:10.12746/swrccc.v4i15.290
- Rada Mihalcea and Paul Tarau . 2004 . Textrank: Bringing order into text . In Proceedings of the 2004 conference on empirical methods in natural language processing. 404–411 . Rada Mihalcea and Paul Tarau. 2004. Textrank: Bringing order into text. In Proceedings of the 2004 conference on empirical methods in natural language processing. 404–411.
- Michael J. Muller and Sandra Kogan . 2012 . Grounded Theory Method in Human-Computer Interaction and Computer-Supported Cooperative Work. In The Human–Computer Interaction Handbook (3 ed.). CRC Press . Num Pages: 21. Michael J. Muller and Sandra Kogan. 2012. Grounded Theory Method in Human-Computer Interaction and Computer-Supported Cooperative Work. In The Human–Computer Interaction Handbook (3 ed.). CRC Press. Num Pages: 21.
- 10.1162/089120102762671927 · DOI · Paper page
- Hariharan Subramonyam , Colleen Seifert , Priti Shah , and Eytan Adar . 2020. TexSketch: Active Diagramming through Pen-and-Ink Annotations . Association for Computing Machinery , New York, NY, USA , 1–13. https://doi.org/10.1145/3313831.3376155 10.1145/3313831.3376155 Hariharan Subramonyam, Colleen Seifert, Priti Shah, and Eytan Adar. 2020. TexSketch: Active Diagramming through Pen-and-Ink Annotations. Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376155 · DOI · Paper page
- 10.1007/978-3-540-89454-4_5 · DOI
- Pradyumna Tambwekar Murtaza Dhuliawala Lara J. Martin Animesh Mehta Brent Harrison and Mark O. Riedl. 2018. Controllable Neural Story Plot Generation via Reinforcement Learning. arxiv:1809.10736 [cs.CL] Pradyumna Tambwekar Murtaza Dhuliawala Lara J. Martin Animesh Mehta Brent Harrison and Mark O. Riedl. 2018. Controllable Neural Story Plot Generation via Reinforcement Learning. arxiv:1809.10736 [cs.CL] · DOI
- 10.1145/3343413.3377971 · DOI · Paper page
- L. Danielle Tully . 2019 . Reverse Outlines: Fueling Revision & Preparing for Writing Conferences . The Second Draft 32 , 2 (2019), 6 . https://ssrn.com/abstract=3465807 L. Danielle Tully. 2019. Reverse Outlines: Fueling Revision & Preparing for Writing Conferences. The Second Draft 32, 2 (2019), 6. https://ssrn.com/abstract=3465807
- 10.1145/2851581.2856472 · DOI
- 10.1145/3027063.3027066 · DOI
- 10.1145/3472749.3474765 · DOI
- Harriet Salatas Waters and Wolfgang Schneider. 2009. Metacognition Strategy Use and Instruction. Guilford Press. Harriet Salatas Waters and Wolfgang Schneider. 2009. Metacognition Strategy Use and Instruction. Guilford Press.
- Daijin Yang , Yanpeng Zhou , Zhiyuan Zhang , and Toby Jia-Jun Li . 2022 . AI as an Active Writer: Interaction strategies with generated text in human-AI collaborative fiction writing . In IUI 2022 Workshop on Human-AI Co-Creation with Generative Models (HAI-GEN 2022). 10. Daijin Yang, Yanpeng Zhou, Zhiyuan Zhang, and Toby Jia-Jun Li. 2022. AI as an Active Writer: Interaction strategies with generated text in human-AI collaborative fiction writing. In IUI 2022 Workshop on Human-AI Co-Creation with Generative Models (HAI-GEN 2022). 10.
- Demet Yaylı . 2012 . Tracing the benefits of self annotation in genre-based writing . The Journal of Language Learning and Teaching 2 , 1 (2012), 45 – 58 . Demet Yaylı. 2012. Tracing the benefits of self annotation in genre-based writing. The Journal of Language Learning and Teaching 2, 1 (2012), 45–58.
- 10.1145/3332165.3347924 · DOI · Paper page
Cited by
AI literacy and its implications for prompt engineering strategies
Knoth, N., Tolzin, A., Janson, A., & Leimeister, J. M. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100225 · Cited by 368
Draxler, F., Werner, A., Lehmann, F., Hoppe, M., Schmidt, A., Buschek, D., & Welsch, R. (2023) · arXiv · doi:10.48550/arxiv.2303.03283 · Cited by 284
Self-citationCo-Writing with Opinionated Language Models Affects Users’ Views
Jakesch, M., Bhat, A., Buschek, D., Zalmanson, L., & Naaman, M. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581196 · Cited by 265
Self-citationDraxler, F., Werner, A., Lehmann, F., Hoppe, M., Schmidt, A., Buschek, D., & Welsch, R. (2024) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3637875 · Cited by 186
Self-citationSuh, S., Chen, M., Min, B., Li, T. J.-J., & Xia, H. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642400 · Cited by 148
Metadata
- Authors
- Dang, H., Benharrak, K., Lehmann, F., & Buschek, D.
- Venue
- Proceedings of the 35th Annual ACM Symposium on User Interface Software and Technology
- Publisher
- ACM
- Published
- 2022-10-28
- Pages
- 1-13
- OpenAlex
- W4292760860
- Cited by
- 84
- References
- 16
- FWCI
- 15.44
- Type
- conference paper
- Language
- en
- Open access
- green
- Funders
- Bavarian State Ministry of Science and the Arts
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.