Cited by 41Student Assessment and FeedbackEFL/ESL Teaching and LearningIntelligent Tutoring Systems and Adaptive Learning

Understanding the effectiveness of automated feedback: Using process data to uncover the role of behavioral engagement

Open access
Schiller, Ronja, Fleckenstein, Johanna, Mertens, Ute, Horbach, Andrea, Meyer, Jennifer·2024·Computers & Education

Abstract

In the last couple of years, feedback research has shifted towards a feedback-as-process approach, taking a learner-centered perspective and focusing on the proactive role of the learner in feedback effectiveness. Process measures can provide new insights into the role of the learner by making learners’ actual behavioral engagement visible. We conducted an experimental study, comparing two groups (feedback vs. no feedback) of English-as-a-foreign-language learners in lower secondary schools ( N = 189). The learners completed a writing task and revised it with or without feedback. A second writing task served as a transfer task. Performance was automatically assessed using a scoring algorithm. To determine the level of learners’ behavioral engagement during the text revision, we used the revision time and the edit distance (i.e., a similarity measure) as behavioral measures. Our analyses showed a positive effect of feedback on text revision. We found a full mediation of the effect of feedback on text revision through revision time with an estimated portion of mediation (POM) of .63∗∗∗ and a partial mediation of the feedback effect on text revision through the edit distance with a POM of .30∗∗. We did not find significant mediation effects of either engagement variable regarding performance in a transfer task. Our findings contribute to the understanding of feedback effectiveness, highlighting the central role of learner engagement in the feedback process.

Integrity signals

Registry

Crossref · 10.1016/j.compedu.2024.105163

Cites

77 of 77 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The impact of feedback on learning outcomes is mediated through complex internal mechanisms involving cognitive processing, behavioral patterns, and affective states.

    “The impact of feedback on learning outcomes is mediated through complex internal mechanisms. Three commonly-discussed mechanisms involve different dimensions of learners' engagement (Fredricks et al., 2004): cognitive processing, behavioral patterns, and affective states(Schiller et al., 2024).”

Cites

Flagged first, then by citations

2 of the 79 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Schiller, R., Fleckenstein, J., Mertens, U., Horbach, A., & Meyer, J.
Affiliations
  • Leibniz Institute for Science and Mathematics Education (DE)
  • University of Hildesheim (DE)
Corresponding author
Schiller, Ronja
Venue
Computers & Education
Publisher
Elsevier BV
Published
2024-12
Volume
223
Pages
105163
ISSN
0360-1315 (ISSN-L)
OpenAlex
W4402490467
Cited by
36
References
79
Citations by year
2026: 19 · 2025: 16 · 2024: 1
FWCI
45.51
Type
journal article
Language
en
Open access
Locations
Computers & Education (publishedVersion, cc-by)
Funders
  • Federal Ministry of Education and Research Bonn Office (01JG2104)
  • Jacobs Foundation (2024-2026)
Topics
  • Social Sciences › Education › Student Assessment and Feedback
  • Social Sciences › Arts and Humanities › Language and Linguistics › EFL/ESL Teaching and Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
Keywords
automated feedback, feedback effectiveness, student engagement, behavioral engagement, textual revision, English as a foreign language, writing instruction, edit distance, automatic scoring, transfer tasks, mediation analysis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review