How to provide automated feedback on the writing process? A participatory approach to design writing analytics tools
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Current writing support tools tend to focus on assessing final or intermediate products, rather than the writing process. However, sensing technologies, such as keystroke logging, can enable provision of automated feedback during, and on aspects of, the writing process. Despite this potential, little is known about the critical indicators that can be used to effectively provide this feedback. This article proposes a participatory approach to identify the indicators of students’ writing processes that are meaningful for educational stakeholders and that can be considered in the design of future systems to provide automated, personalized feedback. This approach is illustrated through a qualitative research design that included five participatory sessions with five distinct groups of stakeholders: bachelor and postgraduate students, teachers, writing specialists, and professional development staff. Results illustrate the value of the proposed approach, showing that students were especially interested in lower-level behavioral indicators, while the other stakeholders focused on higher-order cognitive and pedagogical constructs. These findings lay the groundwork for future work in extracting these higher-level indicators from in-depth analysis of writing processes. In addition, differences in terminology and the levels at which the indicators were discussed, highlight the need for human-centered, participatory approaches to design and develop writing analytics tools.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Using thematic analysis in psychology
Braun, V., & Clarke, V. (2006) · Qualitative Research in Psychology · doi:10.1191/1478088706qp063oa · Cited by 194,414
A meta-analysis of writing instruction for adolescent students
Graham, S., & Perin, D. (2007) · Journal of Educational Psychology · doi:10.1037/0022-0663.99.3.445 · Cited by 1,609
Intelligent tutoring systems and learning outcomes: A meta-analysis
Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q. (2014) · Journal of Educational Psychology · doi:10.1037/a0037123 · Cited by 840
Keystroke Logging in Writing Research: Using Inputlog to Analyze and Visualize Writing Processes
Leijten, M., & Van Waes, L. (2013) · Written Communication · doi:10.1177/0741088313491692 · Cited by 513
The Psychology of Written Composition
(2013) · doi:10.4324/9780203812310 · Cited by 505
Alamargot, D., & Chanquoy, L. (2001) · Studies in Writing · doi:10.1007/978-94-010-0804-4 · Cited by 378
Steenbergen-Hu, S., & Cooper, H. (2014) · Journal of Educational Psychology · doi:10.1037/a0034752 · Cited by 355
Linguistic features in writing quality and development: An overview
Crossley, S. (2020) · Journal of Writing Research · doi:10.17239/jowr-2020.11.03.01 · Cited by 310
Automated written corrective feedback: how well can students make use of it?
Ranalli, J. (2018) · Computer Assisted Language Learning · doi:10.1080/09588221.2018.1428994 · Cited by 228
Wang, Y.-J., Shang, H.-F., & Briody, P. (2013) · Computer Assisted Language Learning · doi:10.1080/09588221.2012.655300 · Cited by 185
Cited by
Human-centred learning analytics and AI in education: A systematic literature review
Alfredo, R., Echeverria, V., Jin, Y., Yan, L., Swiecki, Z., Gašević, D., & Martinez-Maldonado, R. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100215 · Cited by 281
“For instance, robust data privacy measures (e.g., Kivimäki et al., 2019; Ngoon et al., 2023) and transparent data collection procedures (e.g., Conijn et al., 2020a; Wang et al., 2022) can enhance the perceived safety of the system, thereby fostering trust among users.”
BackgroundBanihashem, S. K., Noroozi, O., van Ginkel, S., Macfadyen, L. P., & Biemans, H. J. A. (2022) · Educational Research Review · doi:10.1016/j.edurev.2022.100489 · Cited by 246
A systematic review of AI-based automated written feedback research
Shi, H., & Aryadoust, V. (2024) · ReCALL · doi:10.1017/s0958344023000265 · Cited by 222
Tang, L., Shen, K., Le, H., Shen, Y., Tan, S., Zhao, Y., Juelich, T., Li, X., Gašević, D., & Fan, Y. (2024) · Journal of Computer Assisted Learning · doi:10.1111/jcal.13036 · Cited by 25
Metadata
- Authors
- Conijn, R., Martinez-Maldonado, R., Knight, S., Buckingham Shum, S., Van Waes, L., & van Zaanen, M.
- Venue
- Computer Assisted Language Learning
- Publisher
- Informa UK Limited
- Published
- 2020-11-10
- Volume
- 35
- Issue
- 8
- Pages
- 1838-1868
- ISSN
- 0958-8221 (ISSN-L)
- 1744-3210
- OpenAlex
- W3100019532
- Cited by
- 44
- References
- 14
- FWCI
- 4.55
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How to provide automated feedback on the writing process? A participatory approach to design writing analytics tools. https://app.cochat.ai/papers/10.1080/09588221.2020.1839503
- The paper
- Conijn, R., Martinez-Maldonado, R., Knight, S., Buckingham Shum, S., Van Waes, L., & van Zaanen, M. (2022). How to provide automated feedback on the writing process? A participatory approach to design writing analytics tools. Computer Assisted Language Learning. https://doi.org/10.1080/09588221.2020.1839503
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.