Cited by 25Student Assessment and FeedbackOnline Learning and AnalyticsIntelligent Tutoring Systems and Adaptive Learning

Facilitating learners' self-assessment during formative writing tasks using writing analytics toolkit

Open access
Tang, Luzhen, Shen, Kejie, Le, Huixiao, Shen, Yuan, Tan, Shufang, Zhao, Yueying, Juelich, Torsten, Li, Xinyu +2 more·2024·Journal of Computer Assisted Learning

Abstract

Background: Learners' writing skills are critical to their academic and professional development. Previous studies have shown that learners' self‐assessment during writing is essential for assessing their writing products and monitoring their writing processes. However, conducting practical self‐assessments of writing remains challenging for learners without help, such as formative feedback. Objectives: To facilitate learners' self‐assessment in writing, we developed a writing analytics toolkit and used data visualisation and cutting‐edge machine learning technology that provides real‐time and formative feedback to learners. Methods: To investigate whether our newly‐developed tool affects the accuracy and process of learners' self‐assessment, we conducted a lab study. We assigned 59 learners to complete writing (2 h) and revising (1 h) tasks. During the revision stage, we randomly assigned the learners to two groups: one group used the writing analytics toolkit while the second group was not granted access to the toolkit. Learners' self‐assessment accuracy and process of self‐assessment were compared between the two groups. Results: In our study, we found the toolkit helped learners in the experimental group improve the self‐assessment accuracy of their writing products compared to the learners in the control group. In addition, we also found that the affordances of the toolkit affected the learners' self‐assessment process, and poor design affordances may have prevented the learners from reflecting by themselves. Conclusions: Together, our empirical study shed light on the design of future writing analytics tools which aim at improving learners' self‐assessment during formative writing processes.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1111/jcal.13036

Cites

107 of 107 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Multi-source assessment ecosystems combining self-assessment, peer evaluation, AI analytics, and instructor judgment provide triangulated evidence of competence.

    “Multi-source assessment ecosystems combining self-assessment, peer evaluation, AI analytics, and instructor judgment provide triangulated evidence of competence (Elshall and Badir, 2025; Francis et al., 2025; Tang et al., 2024; Zhang and Zhang, 2022).”

    From Reimagining writing assessment for the AI era: a systematic review on balancing AI support and authentic skill growth (2026) · cites this paper as Tang et al., 2024

    Section 2.1 recommends integrating feedback from learners, peers, teachers, and AI into writing analytics tools, and the Abstract identifies the lack of such integration as a design gap. The paper does not establish that an ecosystem combining all four provides triangulated evidence of competence; its discussion concerns formative writing feedback rather than competence assessment generally. No page numbers were available in our copy. (LLM, conf=0.84)

    Experimental — accuracy not yet measured
  • Two studies (n = 2) targeted AI-supported tools for writing assessment and feedback analytics.

    “two studies (n = 2) targeted AI-supported tools for writing assessment and feedback analytics (Tang et al., 2024; Li et al., 2024).”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

1 retracted

Flagged first, then by citations

Research works printed but not linked: 3

  1. Dang, H., Benharrak, K., Lehmann, F., and Buschek, D. (2022). Placeholder TextPlaceholder TextPlaceholder TextBeyond text generation: Sup- porting writers with continuous automatic text summaries. In Proceed- ings of the 35th annual ACM symposium on user interface software and technology (pp. 1-13).
  2. Spandel, V. (2005). Creating writers: Through 6-trait writing assessment and instruction (4th ed ed.). Pearson Allyn and Bacon.
  3. Winne, P. H., & Hadwin, A. F. (1998). Studying as self-regulated learning. In Metacognition in educational theory and practice. The educational psy- chology series (pp. 277-304).

Cited by

Sorted by citations

Authors
Tang, L., Shen, K., Le, H., Shen, Y., Tan, S., Zhao, Y., Juelich, T., Li, X., Gašević, D., & Fan, Y.
Affiliations
  • Peking University (CN)
  • Research Center for Data Hub and Security Zhejiang Lab Zhejiang China
  • University of Chinese Academy of Sciences (CN)
  • Monash University (AU)
Corresponding author
Fan, Yizhou
Venue
Journal of Computer Assisted Learning
Publisher
Wiley
Published
2024-07-11
Volume
40
Issue
6
Pages
2822-2839
ISSN
  • 0266-4909 (ISSN-L)
  • 1365-2729
OpenAlex
W4400598586
Cited by
25
References
106
Citations by year
2026: 12 · 2025: 10 · 2024: 3
FWCI
31.25
Type
journal article
Language
en
Open access
Locations
  • Journal of Computer Assisted Learning (publishedVersion, cc-by-nc)
  • Monash University Research Portal (Monash University) (submittedVersion, cc-by-nc)
  • HAL (submittedVersion)
Funders
Peking University (7310101005)
Topics
  • Social Sciences › Education › Student Assessment and Feedback
  • Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
Keywords
self-assessment, formative feedback, writing revision, self-assessment accuracy, data visualization, machine learning, reflective writing, writing skills

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review