Enhancing Peer Review with AI-Powered Suggestion Generation Assistance: Investigating the Design Dynamics
Open accessAbstract
While writing peer reviews resembles an important task in science, education, and large organizations, providing fruitful suggestions to peers is not a straightforward task, as different user interaction designs of text suggestion interfaces can have diverse effects on user behaviors when writing the review text. Generative language models might be able to support humans in formulating reviews with textual suggestions. Previous systems use two designs for providing text suggestions, but do not empirically evaluate them: inline and list of suggestions. To investigate the effects of embedding NLP text generation models in the two designs, we collected user requirements to implement Hamta as an example of assistants providing reviewers with text suggestions. Our experiment on comparing the two designs on 31 participants indicates that people using the inline interface provided longer reviews on average, while participants using the list of suggestions experienced more ease of use in using our tool. The results shed light on important design findings for embedding text generation models in user-centered assistants.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the Formal context, reviewing support was limited and focused on evaluation using quantitative feedback like readability metrics and visual feedback such as progress bars.
“Finally, reviewing support was limited, and focused on evaluation using quantitative feedback like readability metrics [88], and visual feedback such as progress bars [90].”
From Co-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process (2025) · cites this paper as [88]
The citation [90] is attached to the claim about visual feedback such as progress bars. Section 3.2 describes Hamta’s interface as providing inline or list-based text suggestions, a word count, and a help button—not progress-bar feedback. Section 6.2 says the study did not measure review quality, contrary to the characterization of its support as focused on evaluation. (LLM, conf=0.83)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
User Acceptance of Information Technology: Toward A Unified View1
Venkatesh, V., Morris, M. G., Davis, G. B., & Davis, F. D. (2003) · MIS Quarterly · doi:10.2307/30036540 · Cited by 45,035 · see citing works
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931 · see citing works
Hattie, J., & Timperley, H. (2007) · Review of Educational Research · doi:10.3102/003465430298487 · Cited by 12,905 · see citing works
Technology Acceptance Model 3 and a Research Agenda on Interventions
Venkatesh, V., & Bala, H. (2008) · Decision Sciences · doi:10.1111/j.1540-5915.2008.00192.x · Cited by 8,159 · see citing works
Agarwal, R., & Karahanna, E. (2000) · MIS Quarterly · doi:10.2307/3250951 · Cited by 4,726 · see citing works
Generative Adversarial Networks: An Overview
Creswell, A., White, T., Dumoulin, V., Arulkumaran, K., Sengupta, B., & Bharath, A. A. (2018) · IEEE Signal Processing Magazine · doi:10.1109/msp.2017.2765202 · Cited by 4,540 · see citing works
Generative Adversarial Networks
Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.2661 · Cited by 4,477 · see citing works
Beyond the Turk: Alternative platforms for crowdsourcing behavioral research
Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.01.006 · Cited by 2,957 · see citing works
Conditional Random Fields as Recurrent Neural Networks
Zheng, S., Jayasumana, S., Romera-Paredes, B., Vineet, V., Su, Z., Du, D., Huang, C., & Torr, P. H. S. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.179 · Cited by 2,386 · see citing works
Peer Assessment between Students in Colleges and Universities
Topping, K. (1998) · Review of Educational Research · doi:10.2307/1170598 · Cited by 2,173 · see citing works
Advances in Neural Information Processing Systems 19 (NIPS 2006)
Simonotto, E., Whalley, H. C., Lawrie, S. M., Murray, L. M., McGonigle, D. J., & Storkey, A. (2006) · Cited by 1,651 · see citing works
About Face 2.0: The Essentials of Interaction Design
Cooper, A., Reimann, R., Cronin, D., & Noessel, C. (1995) · Cited by 1,339 · see citing works
A cognitive process theory of writing
Flower, L., & Hayes, J. R. (1981) · College Composition & Communication · doi:10.58680/ccc198115885 · Cited by 1,025 · see citing works
Who are the crowdworkers?: shifting demographics in mechanical turk
Ross, J., Irani, L., Silberman, M. S., Zaldivar, A., & Tomlinson, B. (2010) · CHI '10 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/1753846.1753873 · Cited by 1,004 · see citing works
FEEDBACK-SEEKING IN INDIVIDUAL ADAPTATION: A RESOURCES PERSPECTIVE.
Ashford, S. J. (1986) · Academy of Management Journal · doi:10.2307/256219 · Cited by 996 · see citing works
To give is better than to receive: The benefits of peer review to the reviewer's own writing
Lundstrom, K., & Baker, W. (2009) · Journal of Second Language Writing · doi:10.1016/j.jslw.2008.06.002 · Cited by 843 · see citing works
Proceedings of the 18th European conference on information systems (ECIS2010)
Tervonen, T., Brock, D. B., Graeff, D. P., & Hillege, H. L. (2010) · Cited by 836 · see citing works
Writing With Power Techniques for Mastering the Writing Process
Elbow, P. H. (1998) · doi:10.1093/oso/9780195120172.001.0001 · Cited by 542 · see citing works
Quasi-Recurrent Neural NetworksPreprint
Bradbury, J., Merity, S., Xiong, C., & Socher, R. (2016) · International Conference on Learning Representations · doi:10.48550/arxiv.1611.01576 · Cited by 481 · see citing works
IntelliCode compose: code generation using transformer
Svyatkovskiy, A., Deng, S. K., Fu, S., & Sundaresan, N. (2020) · Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3368089.3417058 · Cited by 474 · cited 2× · see citing works
“As a result, GPT and Transformer models can generate texts in various domains, such as medical texts [6], text summaries [58], stories [34], patent claims [62], codes [27, 91], and research papers [94].”
BackgroundMethodThe Xerox Star: a retrospective
Johnson, J. A., Roberts, T. L., Verplank, W. L., Smith, D. C., Irby, C. H., Beard, M., & Mackey, K. (1989) · Computer · doi:10.1109/2.35211 · Cited by 433 · see citing works
Effects of Editorial Peer Review: A Systematic Review
Jefferson, T., Alderson, P., Wager, E., & Davidoff, F. (2002) · JAMA · doi:10.1001/jama.287.21.2784 · Cited by 347 · see citing works
Kocielnik, R., Amershi, S., & Bennett, P. N. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300641 · Cited by 277 · see citing works
Creative Writing with a Machine in the Loop: Case Studies on Slogans and Stories
Clark, E., Ross, A. S., Tan, C., Ji, Y., & Smith, N. A. (2018) · 23rd International Conference on Intelligent User Interfaces · doi:10.1145/3172944.3172983 · Cited by 257 · see citing works
The survey: Text generation models in deep learning
Iqbal, T., & Qureshi, S. (2020) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2020.04.001 · Cited by 241 · see citing works
Grammarly as AI-powered English Writing Assistant: Students' Alternative for Writing English
Fitria, T. N. (2021) · Metathesis: Journal of English Language, Literature, and Teaching · doi:10.31002/metathesis.v5i1.3519 · Cited by 221 · see citing works
Zha, Z.-J., Yang, L., Mei, T., Wang, M., & Wang, Z. (2009) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/1631272.1631278 · Cited by 180 · see citing works
Cheng, K.-H., Liang, J.-C., & Tsai, C.-C. (2015) · The Internet and Higher Education · doi:10.1016/j.iheduc.2015.02.001 · Cited by 174 · see citing works
Automated Essay Writing: An AIED Opinion
Sharples, M. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-022-00300-7 · Cited by 171 · see citing works
Gmail Smart Compose: Real-Time Assisted Writing
Chen, M. X., Lee, B. N., Bansal, G., Cao, Y., Zhang, S., Lu, J., Tsay, J., Wang, Y., Dai, A. M., Chen, Z., et al. (2019) · Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining · doi:10.1145/3292500.3330723 · Cited by 169 · see citing works
Peer review in higher education: Student perceptions before and after participation
Mulder, R. A., Pearce, J. M., & Baik, C. (2014) · Active Learning in Higher Education · doi:10.1177/1469787414527391 · Cited by 163 · see citing works
360 Degree Feedback and Leadership Development
Alimo‐Metcalfe, B. (1998) · International Journal of Selection and Assessment · doi:10.1111/1468-2389.00070 · Cited by 160 · see citing works
Patent claim generation by fine-tuning OpenAI GPT-2
Lee, J.-S., & Hsiang, J. (2020) · World Patent Information · doi:10.1016/j.wpi.2020.101983 · Cited by 160 · see citing works
ArgueTutor: An Adaptive Dialog-Based Learning System for Argumentation Skills
Wambsganss, T., Kueng, T., Soellner, M., & Leimeister, J. M. (2021) · Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411764.3445781 · Cited by 138 · see citing works
A survey on text generation using generative adversarial networks
de Rosa, G. H., & Papa, J. P. (2021) · Pattern Recognition · doi:10.1016/j.patcog.2021.108098 · Cited by 138 · see citing works
Alberts, B., Hanson, B., & Kelner, K. L. (2008) · Science · doi:10.1126/science.1162115 · Cited by 119 · see citing works
Transformer-Based Neural Network for Answer Selection in Question Answering
Shao, T., Guo, Y., Chen, H., & Hao, Z. (2019) · IEEE Access · doi:10.1109/access.2019.2900753 · Cited by 113 · see citing works
Predictive text entry methods for mobile phones
Dunlop, M. D., & Crossan, A. (2000) · Personal and Ubiquitous Computing · doi:10.1007/bf01324120 · Cited by 109 · see citing works
AL: An Adaptive Learning Support System for Argumentation Skills
Wambsganss, T., Niklaus, C., Cetto, M., Söllner, M., Handschuh, S., & Leimeister, J. M. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376732 · Cited by 101 · cited 3× · see citing works
“The two pre-survey constructs we chose have been employed in prior studies focusing on user-centric systems [101, 102] to check for valid randomization.”
BackgroundMethodSelf-citationOn the Ethics of Crowdsourced Research
Williamson, V. S. (2016) · PS Political Science & Politics · doi:10.1017/s104909651500116x · Cited by 91 · see citing works
On user interactions with query auto-completion
Mitra, B., Shokouhi, M., Radlinski, F., & Hofmann, K. (2014) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/2600428.2609508 · Cited by 85 · see citing works
A reciprocal peer review system to support college students' writing
Yang, Y. (2010) · British Journal of Educational Technology · doi:10.1111/j.1467-8535.2010.01059.x · Cited by 82 · see citing works
A Review of Peer Code Review in Higher Education
Indriasari, T. D., Luxton-Reilly, A., & Denny, P. C. (2020) · ACM Transactions on Computing Education · doi:10.1145/3403935 · Cited by 80 · see citing works
IntroAssist: A Tool to Support Writing Introductory Help Requests
Hui, J. S., Gergle, D., & Gerber, E. M. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173596 · Cited by 76 · see citing works
Osone, H., Lu, J.-L., & Ochiai, Y. (2021) · Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3411763.3450391 · Cited by 74 · see citing works
On Suggesting Phrases vs. Predicting Words for Mobile Text Composition
Arnold, K. C., Gajos, K. Z., & Kalai, A. T. (2016) · Proceedings of the 29th Annual Symposium on User Interface Software and Technology · doi:10.1145/2984511.2984584 · Cited by 69 · see citing works
Developing a writing assistant to help EAP writers with collocations in real time
Frankenberg‐Garcia, A., Lew, R., Roberts, J. C., Rees, G. P., & Sharma, N. (2018) · ReCALL · doi:10.1017/s0958344018000150 · Cited by 69 · see citing works
Word Prediction for Students with Severe Spelling Problems
MacArthur, C. A. (1999) · Learning Disability Quarterly · doi:10.2307/1511283 · Cited by 66 · see citing works
Using Peer Review to Improve Student Writing in Business Courses
Rieber, L. J. (2006) · Journal of Education for Business · doi:10.3200/joeb.81.6.322-326 · Cited by 64 · see citing works
Bi, X., Ouyang, T. Y., & Zhai, S. (2014) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2556288.2557414 · Cited by 59 · see citing works
Rivera, M., Qiu, L., Kumar, S., & Petrucci, T. (2021) · Information Systems Research · doi:10.1287/isre.2020.0979 · Cited by 59 · see citing works
A critical assessment of consumer reviews: A hybrid NLP-based methodology
Biswas, B. N., Sengupta, P., Kumar, A., Delen, D., & Gupta, S. (2022) · Decision Support Systems · doi:10.1016/j.dss.2022.113799 · Cited by 53 · see citing works
Can GPT-3 write an academic paper on itself, with minimal human input?Preprint
Transformer, G. G. P., Thunström, A. O., & Steingrímsson, S. (2022) · HAL (Le Centre pour la Communication Scientifique Directe) · Cited by 51 · see citing works
Fedor, D. B., Bettenhausen, K. L., & Davis, W. (1999) · Group & Organization Management · doi:10.1177/1059601199241006 · Cited by 50 · see citing works
Reflection in Agile Retrospectives
Andriyani, Y., Hoda, R., & Amor, R. (2017) · Lecture notes in business information processing · doi:10.1007/978-3-319-57633-6_1 · Cited by 48 · see citing works
Sushko, V. (2024) · bonndoc (University of Bonn) · Cited by 47 · see citing works
Benefits of Peer Review on Students’ Writing
Yalch, M. M., Vitale, E. M., & Ford, J. K. (2019) · Psychology Learning & Teaching · doi:10.1177/1475725719835070 · Cited by 40 · see citing works
Autocomplete as Research Tool: A Study on Providing Search Suggestions
Ward, D., Hahn, J., & Feist, K. (2012) · Information Technology and Libraries · doi:10.6017/ital.v31i4.1930 · Cited by 39 · see citing works
Yeh, S.-F., Wu, M.-H., Chen, T.-Y., Lin, Y.-C., Chang, X.-J., Chiang, Y.-H., & Chang, Y.-J. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3501941 · Cited by 38 · see citing works
Fictitious GAN: Training GANs with Historical Models
Ge, H., Yin, X., Chen, X., Berry, R., & Wu, Y. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01246-5_8 · Cited by 32 · see citing works
Supporting Cognitive and Emotional Empathic Writing of Students
Wambsganss, T., Niklaus, C., Söllner, M., Handschuh, S., & Leimeister, J. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.314 · Cited by 31 · see citing works
Self-citationRecurrent Neural Networks with External Memory for Language UnderstandingPreprint
Peng, B., & Yao, K. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.00195 · Cited by 29 · see citing works
Davis, R. L., Wambsganß, T., Jiang, W., Kim, K. G., Käser, T., & Dillenbourg, P. (2023) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3544549.3585644 · Cited by 28 · see citing works
Reviewriter: AI-Generated Instructions For Peer Review Writing
Su, X., Wambsganss, T., Rietsche, R., Neshaei, S. P., & Kser, T. (2025) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/2023.bea-1.5 · Cited by 27 · see citing works
Self-citationHow Network Embeddedness Affects Real-Time Performance Feedback: An Empirical Investigation
Petryk, M., Rivera, M., Bhattacharya, S., Qiu, L., & Kumar, S. (2022) · Information Systems Research · doi:10.1287/isre.2022.1110 · Cited by 25 · see citing works
A Design Space for Writing Support Tools Using a Cognitive Process Model of Writing
Gero, K., Calderwood, A., Li, C., & Chilton, L. (2022) · Proceedings of the First Workshop on Intelligent and Interactive Writing Assistants (In2Writing 2022) · doi:10.18653/v1/2022.in2writing-1.2 · Cited by 25 · see citing works
Adaptive Empathy Learning Support in Peer Review Scenarios
Wambsganss, T., Soellner, M., Koedinger, K. R., & Leimeister, J. M. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517740 · Cited by 24 · see citing works
A Taxonomy for Deep Learning in Natural Language Processing
Landolt, S., Wambsganss, T., & Söllner, M. (2021) · Hawaii International Conference on System Sciences · doi:10.24251/hicss.2021.129 · Cited by 18 · see citing works
A Corpus for Argumentative Writing Support in German
Wambsganß, T., Niklaus, C., Söllner, M., Handschuh, S., & Leimeister, J. M. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.74 · Cited by 16 · see citing works
Does emotion influence the use of auto-suggest during smartphone typing?
Ghosh, S., Hiware, K., Ganguly, N., Mitra, B., & De, P. (2019) · ACM International Conference on Intelligent User Interfaces (IUI) · doi:10.1145/3301275.3302329 · Cited by 15 · see citing works
Weber, F., Wambsganss, T., Neshaei, S. P., & Söllner, M. (2023) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2023.findings-acl.145 · Cited by 15 · see citing works
Self-citationA Blockchain Approach to Academic Assessment
Alipour, S., Elahimanesh, S., Jahanzad, S., Morassafar, P., & Neshaei, S. P. (2022) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3491101.3519682 · Cited by 8 · see citing works
Self-citationAn empirical study of search box and autocomplete design patterns in online bookstore
Thimthong, T., Chintakovid, T., & Krootjohn, S. (2012) · IEEE Symposium on Humanities, Science and Engineering Research · doi:10.1109/shuser.2012.6268796 · Cited by 8 · see citing works
Acosta, E., Otero, J. J. E., & Toletti, G. C. (2014) · The New Educational Review · doi:10.15804/tner.14.37.3.05 · Cited by 7 · see citing works
Jensen, E. T., Hansen, M., Eika, E., & Sandnes, F. E. (2020) · International Conference on Ubiquitous Information Management and Communication (IMCOM) · doi:10.1109/imcom48794.2020.9001795 · Cited by 5 · see citing works
E-mail autocomplete function using RNN Encoder-decoder sequence-to-sequence model
Modi, R., Naik, K., Vyas, T., Desai, S., & Degadwala, S. (2021) · International Conference on Electronics, Communication and Aerospace Technology (ICECA) · doi:10.1109/iceca52323.2021.9675961 · Cited by 3 · see citing works
Hybrid Question Generation Approach for Critical Review Writing Support
Liu, M., Calvo, R. A., & Rus, V. (2012) · International Conference on Computers in Education · doi:10.58459/icce.2012.558 · Cited by 1 · see citing works
Schäffer, P. (1966) · Nature · doi:10.1038/212223c0 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 35
- Ali Amin-Nejad, Julia Ive, and Sumithra Velupillai. 2020. Exploring transformer text generation for medical dataset augmentation. In Proceedings of the Twelfth Language Resources and Evaluation Conference. 4699-4708.
- Kathleen M Arnold, Sharda Umanath, Kara Thio, Walter B Reilly, Mark A Mc- Daniel, and Elizabeth J Marsh. 2017. Understanding the cognitive processes involved in writing to learn. Journal of Experimental Psychology: Applied 23, 2 (2017), 115.
- Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Ka- plan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33.
- Curran Associates, Inc., 1877-1901. https://proceedings.neurips.cc/paper/2020/ file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf
- Peng Cheng, Xiang Lian, Zhao Chen, Rui Fu, Lei Chen, Jinsong Han, and Jizhong Zhao. 2014. Reliable diversity-based spatial crowdsourcing by moving workers. arXiv preprint arXiv:1412.0223 (2014). · arXiv
- Junyoung Chung, Caglar Gulcehre, Kyunghyun Cho, and Yoshua Bengio. 2015. Gated feedback recurrent neural networks. In International conference on ma- chine learning. PMLR, 2067-2075.
- Alan Cooper, Robert Reimann, and David Cronin. 2007. About face 3: the essentials of interaction design. John Wiley & Sons.
- Arghavan Moradi Dakhel, Vahid Majdinasab, Amin Nikanjam, Foutse Khomh, Michel C Desmarais, Zhen Ming, et al. 2022. GitHub Copilot AI pair programmer: Asset or Liability? arXiv preprint arXiv:2206.15331 (2022). · arXiv
- Robert Dale. 2021. GPT-3: What's it good for? Natural Language Engineering 27, 1 (2021), 113-118.
- Katherine Elkins and Jon Chun. 2020. Can GPT-3 pass a Writer's turing test? Journal of Cultural Analytics 5, 2 (2020), 17212.
- Le Fang, Tao Zeng, Chaochun Liu, Liefeng Bo, Wen Dong, and Changyou Chen. 2021. Transformer-based conditional variational autoencoder for controllable story generation. arXiv preprint arXiv:2101.00828 (2021). · arXiv
- Donald B Fedor, Kenneth L Bettenhausen, and Walter Davis. 1999. Peer re- views: Employees' dual roles as raters and recipients. Group & Organization Management 24, 1 (1999), 92-120.
- David Galbraith. 2009. Cognitive models of writing. German as a foreign language 2-3 (2009), 7-22.
- John R Hayes and Linda S Flower. 1986. Writing research and the writer. American psychologist 41, 10 (1986), 1106.
- Yijue How and Min-Yen Kan. 2005. Optimizing predictive text entry for short message service on mobile phones. In Proceedings of HCII, Vol. 5. 2005.
- Ayush Jaiswal, Wael AbdAlmageed, Yue Wu, and Premkumar Natarajan. 2018. Capsulegan: Generative adversarial capsule network. In Proceedings of the Euro- pean conference on computer vision (ECCV) workshops. 0-0.
- Urvashi Khandelwal, Kevin Clark, Dan Jurafsky, and Lukasz Kaiser. 2019. Sample efficient text summarization using a single pre-trained transformer. arXiv preprint arXiv:1905.08836 (2019). · arXiv
- Surafel M Lakew, Mauro Cettolo, and Marcello Federico. 2018. A comparison of transformer and recurrent neural networks on multilingual neural machine translation. arXiv preprint arXiv:1806.06957 (2018). · arXiv
- Florian Lehmann and Daniel Buschek. 2022. Examining Autocompletion as a Basic Concept for Interaction with Generative AI. CoRR abs/2201.06892 (2022). arXiv:2201.06892 https://arxiv.org/abs/2201.06892 · arXiv
- Shilong Liu, Lei Zhang, Xiao Yang, Hang Su, and Jun Zhu. 2021. Query2label: A simple transformer way to multi-label classification. arXiv preprint arXiv:2107.10834 (2021). · arXiv
- Lawrence C Loh. 2016. Autocomplete: Dr Google's "helpful" assistant? Canadian Family Physician 62, 8 (2016), 622-623.
- David Nicol. 2014. Guiding principles for peer review: unlocking learners' evaluative skills. Advances and Innovations in University Assessment and Feedback (2014), 197-224.
- OECD. 2018. The Future of Education and Skills -Education 2030. https: //doi.org/2018-06-15
- Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.
- Roman Rietsche, Daniel Frei, Emanuel Stöckli, and Matthias Söllner. 2019. Not All Reviews are Equal -a Literature Review on Online Review Helpfulness. In Proceedings of the 27th European Conference on Information Systems (ECIS).
- Chi Sun, Xipeng Qiu, Yige Xu, and Xuanjing Huang. 2019. How to fine-tune BERT for text classification?. In China national conference on Chinese computa- tional linguistics. Springer, 194-206.
- Vadim Sushko, Jurgen Gall, and Anna Khoreva. 2021. One-shot GAN: Learning to generate samples from single images and videos. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2596-2600.
- Jonas Thiergart, Stefan Huber, and Thomas Übellacker. 2021. Understanding Emails and Drafting Responses-An Approach Using GPT-3. arXiv preprint arXiv:2102.03062 (2021). · arXiv
- Keith J Topping. 2010. Methodological quandaries in studying process and outcomes in peer assessment. Learning and instruction 20, 4 (2010), 339-343.
- Lewis Tunstall, Leandro von Werra, and Thomas Wolf. 2022. Natural Language Processing with Transformers. " O'Reilly Media, Inc. ".
- Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention Is All You Need. In Advances in neural information processing systems Nips (2017), 5998-6008.
- Thiemo Wambsganss, Xiaotian Su, Vinitra Swamy, Seyed Parsa Neshaei, Roman Rietsche, and Tanja Käser. 2023. Unraveling Downstream Gender Bias from Large Language Models: A Study on AI Educational Writing Assistance. arXiv preprint arXiv:2311.03311 (2023). · arXiv
- Thiemo Wambsganss, Vinitra Swamy, Roman Rietsche, and Tanja Käser. 2022. Bias at a Second Glance: A Deep Dive into Bias for German Educational Peer- Review Data Modeling. In Proceedings of the 29th International Conference on Computational Linguistics, COLING 2022, Gyeongju, Republic of Korea, October 12- 17, 2022,
- Nicoletta Calzolari, Chu-Ren Huang, Hansaem Kim, James Pustejovsky, Leo Wanner, Key-Sun Choi, Pum-Mo Ryu, Hsin-Hsi Chen, Lucia Donatelli, Heng Ji, Sadao Kurohashi, Patrizia Paggio, Nianwen Xue, Seokhwan Kim, Younggyun Hahm, Zhong He, Tony Kyungil Lee, Enrico Santus, Francis Bond, and Seung- Hoon Na (Eds.). International Committee on Computational Linguistics, 1344- 1356. https://aclanthology.org/2022.coling-1.115
- World Economic Forum WEF. 2018. The Future of Jobs Report 2018. Technical Report. https://doi.org/10.1177/0891242417690604 · DOI
Cited by
Sorted by citations
Metacognition meets <scp>AI</scp> : Empowering reflective writing with large language models
Neshaei, S. P., Mejia‐Domenzain, P., Davis, R. L., & Käser, T. (2025) · British Journal of Educational Technology · doi:10.1111/bjet.13601 · Cited by 39 · see citing works
Self-citationCo-Writing with AI, on Human Terms: Aligning Research with User Demands Across the Writing Process
Reza, M., Thomas-Mitchell, J., Dushniku, P., Laundry, N., Williams, J. J., & Kuzminykh, A. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3757566 · Cited by 37 · see citing works
“…processes and contexts–for example, strong representation of Generating and Translating activities in Creative and Academic settings, and limited focus on Monitoring across all contexts. process across them; we also report how many papers 1 were coded into each writing context, as shown in…”
Defining quality in peer review reports: a scoping review
Sizo, A., Lino, A. D. P., Rocha, Á., & Reis, L. P. (2025) · Knowledge and Information Systems · doi:10.1007/s10115-025-02435-0 · Cited by 24 · see citing works
Llerena-Izquierdo, J., & Ayala-Carabajo, R. (2025) · Informatics · doi:10.3390/informatics12040111 · Cited by 11 · see citing works
User-centric Reflective Writing Assistance: Leveraging RAG for Enhanced Personalized Support
Neshaei, S. P., Tashkovska, M., Mejia‐Domenzain, P., Wambsganß, T., & Käser, T. (2025) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3706599.3719899 · Cited by 7 · see citing works
Self-citationFeedQUAC: Quick Unobtrusive AI-Generated Commentary
Long, T., Wannamaker, K., Vermeulen, J., Fitzmaurice, G. W., & Matejka, J. (2025) · International Conference on Human-Agent Interaction (HAI) · doi:10.1145/3765766.3765861 · Cited by 6 · see citing works
Neshaei, S. P., Tolzin, A., Berkle, Y., Leuchter, M., Leimeister, J., Janson, A., & Wambsganss, T. (2025) · Proc. ACM Hum. Comput. Interact. · doi:10.1145/3711023 · Cited by 6 · see citing works
Self-citationAI-Powered Image Generation System using AI
Jamadade, V., Ghodake, M., Katakdhond, S., & Godase, V. V. (2025) · International Journal of Advanced Research in Science Communication and Technology · doi:10.48175/ijarsct-29529
AI-Powered Note Generation Application
Rajbhar, R., Singh, S., Singh, S., Saifi, S., & Mall, P. K. (2026) · Lecture notes in networks and systems · doi:10.1007/978-981-96-9935-3_6
- Authors
- Neshaei, S. P., Rietsche, R., Su, X., & Wambsganss, T.
- Affiliations
- École Polytechnique Fédérale de Lausanne (CH)
- Bern University of Applied Sciences (CH)
- Venue
- Proceedings of the 29th International Conference on Intelligent User Interfaces
- Publisher
- ACM
- Published
- 2024-03-18
- Pages
- 88-102
- OpenAlex
- W4393970617
- Cited by
- 9
- References
- 75
- Citations by year
- 2026: 1 · 2025: 8
- FWCI
- 2.90
- Type
- conference paper
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM International Conference on Intelligent User Interfaces (IUI) (publishedVersion)
- ARBOR - Bern University of Applied Sciences Repository (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Human-Computer Interaction › Usability and User Interface Design
- Physical Sciences › Computer Science › Information Systems › Recommender Systems and Techniques
- Physical Sciences › Computer Science › Human-Computer Interaction › Innovative Human-Technology Interaction
- Keywords
- peer review, generative language models, AI-assisted writing, user-centered design, ease of use, review length, user requirements
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
