Cited by 176Writing and Handwriting EducationStudent Assessment and FeedbackEducational Technology and Assessment

The DOI matches its Crossref record; 76 of the 76 references it cites were found in a registry or an index.

Automated Writing Evaluation and Feedback: Multiple Metrics of Efficacy

Wilson, Joshua, Roscoe, Rod D.·2020·Journal of Educational Computing Research

Abstract

The present study extended research on the effectiveness of automated writing evaluation (AWE) systems. Sixth graders were randomly assigned by classroom to an AWE condition that used Project Essay Grade Writing ( n = 56) or a word-processing condition that used Google Docs ( n = 58). Effectiveness was evaluated using multiple metrics: writing self-efficacy, holistic writing quality, performance on a state English language arts test, and teachers' perceptions of AWE's social validity. Path analyses showed that after controlling for pretest measures, composing condition had no effect on holistic writing quality, but students in the AWE condition had more positive writing self-efficacy and better performance on the state English language arts test. Posttest writing self-efficacy partially mediated the effect of composing condition on state test performance. Teachers reported positive perceptions of AWE's social validity. Results emphasize the importance of using multiple metrics and considering both contextual factors and AWE implementation methods when evaluating AWE effectiveness.

Overview

Integrity signals

Registry

Crossref · 10.1177/0735633119830764

Cites

76 of 76 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 75 references

Research works printed but not linked: 12

  1. Archer A. L., 2011, Explicit instruction: Effective and efficient teaching
  2. Balfour S. P., 2013, Research & Practice in Assessment, 8, 40
  3. Bennett R. E., 2004, Moving the field forward some thoughts on validity and automated scoring. Research Memorandum (RM-04-01)
  4. Graham S., 2016, Teaching secondary students to write effectively
  5. Grimes D., 2010, Journal of Technology, Learning, and Assessment, 8, 1
  6. Keith T. Z., 2003, Automated essay scoring: A cross-disciplinary perspective, 147
  7. Muthén L. K., Mplus user's guide
  8. National Center for Education Statistics, 2012, The Nation's Report Card: Writing 2011
  9. Page E. B., 2003, Automated essay scoring: A cross-disciplinary perspective, 43
  10. Shermis M. D., 2016, Handbook of writing research, 395
  11. Skalicky S., 2016, Advances in Language and Literacy Studies, 7, 248
  12. Wilson J., 2014, Learning Disabilities: A Contemporary Journal, 12, 93

Cited by

Sorted by citations

Metadata

Authors
Wilson, J., & Roscoe, R. D.
Affiliations
  • University of Delaware (US)
  • Arizona State University (US)
Corresponding author
Wilson, Joshua
Venue
Journal of Educational Computing Research
Published
2019-03-26
Volume
58
Issue
1
Pages
87-125
ISSN
  • 0735-6331 (ISSN-L)
  • 1541-4140
OpenAlex
W2923646821
Cited by
176
References
75
Citations by year
2026: 34 · 2025: 30 · 2024: 30 · 2023: 27 · 2022: 26 · 2021: 17 · 2020: 10 · 2019: 2
FWCI
37.10
Type
journal article
Language
en
Open access
closed
Locations
Journal of Educational Computing Research (publishedVersion)
Topics
  • Social Sciences › Education › Writing and Handwriting Education
  • Social Sciences › Education › Student Assessment and Feedback
  • Physical Sciences › Computer Science › Information Systems › Educational Technology and Assessment
Keywords
automated writing evaluation, writing self-efficacy, writing quality, Google Docs, writing instruction, teacher perceptions, social validity, junior high school students, writing feedback

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review