Automated writing evaluation (AWE) feedback: a systematic investigation of college students’ acceptance
The DOI matches its Crossref record; 16 of the 16 references it cites were found in a registry or an index.
Abstract
Automated writing evaluation (AWE) has been used increasingly to provide feedback on student writing. Previous research typically focused on its inter-rater reliability with human graders and validation frameworks. The limited body of research has only discussed students’ attitudes or perceptions in general. A systematic investigation of the driving factors contributing to students’ acceptance is still lacking. This study proposes an extended technology acceptance model (TAM) to identify the environmental, individual, educational, and systemic factors that influence college students’ acceptance of AWE feedback and examine how they affect college students’ usage intention. Structural equation modeling (SEM) was used to analyze the quantitative survey data from 448 Chinese college students who had used AWE feedback for at least one semester. Results revealed that students’ behavioral intention to use AWE feedback was affected by the subjective norm, facilitating conditions, perceived trust, AWE self-efficacy, cognitive feedback, and system characteristics. Among them, subjective norm, perceived trust, and cognitive feedback positively influenced perceived usefulness; facilitating conditions, AWE self-efficacy, and system characteristics were significant determinants of perceived ease of use; anxiety played no role for experienced users. Implications from these findings to AWE developers and practitioners are further elaborated.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Hu, L., & Bentler, P. M. (1999) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705519909540118 · Cited by 108,788
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483
Structural equation models with unobservable variables and measurement error: Algebra and statistics
Fornell, C., & Larcker, D. F. (1981) · Journal of Marketing Research · doi:10.1177/002224378101800313 · Cited by 13,287 · cited 1×
“…have sufficient correlations, which is considered adequate when a) the composite reliability value of each construct is above 0.70, b) the average variance extracted (AVE) of each construct is greater than 0.50, and c) the factor loading of each item exceeds 0.70 (Fornell & Larcker, 1981).”
BackgroundInfluentialHattie, J., & Timperley, H. (2007) · Review of Educational Research · doi:10.3102/003465430298487 · Cited by 12,905 · cited 1×
“Feedback is information provided by an agent, such as a teacher, peer, parent, or computer, specifically relating to one’s performance or understanding (Hattie & Timperley, 2007).”
BackgroundVenkatesh, V. (2000) · Information Systems Research · doi:10.1287/isre.11.4.342.11872 · Cited by 6,664
The Technology Acceptance Model: Past, Present, and Future
Lee, Y., Kozar, K. A., & Larsen, K. R. T. (2003) · Communications of the Association for Information Systems · doi:10.17705/1cais.01250 · Cited by 2,572 · cited 1×
“…R 2 value in behavioral intention to use is larger than that reported in the original TAM, which was between 45% and 49% (Davis et al., 1989), in TAM2, which was between 34% and 52% (Venkatesh & Davis, 2000), and in average TAM studies, which was between 30% and 40% (Lee, Kozar, & Larsen, 2003).”
ResultSung, Y.-T., Chang, K.-E., & Liu, T.-C. (2016) · Computers & Education · doi:10.1016/j.compedu.2015.11.008 · Cited by 1,574
Abdullah, F., & Ward, R. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2015.11.036 · Cited by 986 · cited 4×
“The result does not concur with previous studies (Abdullah & Ward, 2016; Aburub & Alnawas, 2019; Li et al., 2019), where computer anxiety was found to have a negative relationship with perceived ease of use.”
BackgroundResultInfluentialStudent engagement with teacher and automated feedback on L2 writing
Zhang, Z. (., & Hyland, K. (2018) · Assessing Writing · doi:10.1016/j.asw.2018.02.004 · Cited by 546
Mobile-based assessment: Investigating the factors that influence behavioral intention to use
Nikou, S. A., & Economides, A. A. (2017) · Computers & Education · doi:10.1016/j.compedu.2017.02.005 · Cited by 388 · cited 10×
“…perceived usefulness, perceived ease of use, and behavioral intention to use were taken from Coskuncay and Ozkan (2013), Venkatesh et al. (2003), Liu et al. (2015), Compeau and Higgins (1995), Nikou and Economides (2017), Davis (1989), and Hassanzadeh, Kanaani, and Elahi (2012), respectively.”
BackgroundMethodResultInfluential
Cited by
Comparing the quality of human and ChatGPT feedback of students’ writing
Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J., Moon, Y., Tseng, W., Warschauer, M., & Olson, C. B. (2024) · Learning and Instruction · doi:10.1016/j.learninstruc.2024.101894 · Cited by 509
A systematic review of AI-based automated written feedback research
Shi, H., & Aryadoust, V. (2024) · ReCALL · doi:10.1017/s0958344023000265 · Cited by 222
Automated feedback and writing: a multi-level meta-analysis of effects on students' performance
Fleckenstein, J., Liebenow, L. W., & Meyer, J. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1162454 · Cited by 145
“Two very recent meta-analyses have examined the effect of AWE systems on writing performance (Zhai and Ma, 2021; Ngo et al., 2022).”
BackgroundAljuaid, H. (2024) · Arab World English Journal · doi:10.24093/awej/chatgpt.2 · Cited by 129
Wei, P., Wang, X., & Dong, H. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1249991 · Cited by 89
Metadata
- Authors
- Zhai, N., & Ma, X.
- Venue
- Computer Assisted Language Learning
- Publisher
- Informa UK Limited
- Published
- 2021-04-07
- Volume
- 35
- Issue
- 9
- Pages
- 2817-2842
- ISSN
- 0958-8221 (ISSN-L)
- 1744-3210
- OpenAlex
- W3144434203
- Cited by
- 111
- References
- 16
- FWCI
- 12.58
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Automated writing evaluation (AWE) feedback: a systematic investigation of college students’ acceptance. https://app.cochat.ai/papers/10.1080/09588221.2021.1897019
- The paper
- Zhai, N., & Ma, X. (2022). Automated writing evaluation (AWE) feedback: a systematic investigation of college students’ acceptance. Computer Assisted Language Learning. https://doi.org/10.1080/09588221.2021.1897019
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.