Computer-assisted EFL writing and evaluations based on artificial intelligence: a case from a college reading and writing course
The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
Purpose: The aim of this study is to explore students' expectations and perceived effectiveness of computer-assisted review tools, and the differences in reliability and validity between human evaluation and automatic evaluation, to find a way to improve students' English writing ability. Design/methodology/approach: Based on the expectancy disconfirmation theory (EDT) and Intelligent Computer-Assisted Language Learning (ICALL) theory, an experiment is conducted through the observation method, semistructured interview method and questionnaire survey method. In the experiment, respondents were asked to write and submit four essays on three online automated essay evaluation (AEE) systems in total, one essay every two weeks. Also, two teacher raters were invited to score the first and last papers of each student. The respondents' feedbacks were investigated to confirm the effectiveness of the AEE system; the evaluation results of the AEE systems and teachers were compared; descriptive statistics was used to analyze the experimental data. Findings: The experiment revealed that the respondents held high expectations for the computer-assisted evaluation tools, and the effectiveness of computer scoring feedback on students was higher than that of teacher scoring feedback. Moreover, at the end of the writing project, the students' independent learning ability and English writing ability were significantly improved. Besides, there was a positive correlation between students' initial expectations of computer-assisted learning tools and the final evaluation of learning results. Originality/value: The innovation lies in the use of observation methods, questionnaire survey methods, data analysis, and other methods for the experiment, and the combination of deep learning theory, EDT and descriptive statistics, which has particular reference value for future works.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Wilson, J., & Czik, A. (2016) · Computers & Education · doi:10.1016/j.compedu.2016.05.004 · Cited by 297
Wang, Y.-J., Shang, H.-F., & Briody, P. (2013) · Computer Assisted Language Learning · doi:10.1080/09588221.2012.655300 · Cited by 185
The debate on automated essay grading
Hearst, M. A. (2000) · IEEE Intelligent Systems and their Applications · doi:10.1109/5254.889104 · Cited by 176
The Effect of Using Automated Essay Evaluation on ESL Undergraduate Students’ Writing Skill
Aluthman, E. S. (2016) · International Journal of English Linguistics · doi:10.5539/ijel.v6n5p54 · Cited by 45
Research works printed but not linked: 46
- 2011, Blogs in language learning: maximizing students' collaborative, Procedia Social and Behavioral Sciences, 18, 537
- 2007, Construct Validity of E-Rater in Scoring TOEFL Essays (Research Report No. RR‐07‐21)
- 2019, AAEE–automated evaluation of students' essays in Arabic language, Information Processing and Management, 56, 1736, 10.1016/j.ipm.2019.05.008 · DOI
- 2019, An overview of automatic essay evaluation in the past two decades, Foreign Language Research, 65
- 2014, Evaluation of computer based foreign language learning software by teachers and students, The Turkish Online Journal of Educational Technology, 13, 71
- 2019, Deep learning in automated essay scoring, 10.1007/978-3-319-91464-0_30 · doi:10.1007/978-3-319-91464-0_30
- 2017, Open innovation in supply networks: an expectation disconfirmation, Journal of Business and Industrial Marketing, 32, 432, 10.1108/JBIM-07-2016-0150 · DOI
- 2004, Automated essay evaluation: the criterion online writing service, AI Magazine, 25, 27
- 2008, Beyond the design of automated writing evaluation: pedagogical practices and perceived earning effectiveness in EFL writing classes, Language Learning and Technology, 12, 94
- 2015, Impacts of the test of English listening comprehension on students' English learning expectations in taiwan, Language and Curriculum, 28, 191, 10.1080/07908318.2015.1027216 · DOI
- 2017, Evaluating the use of a social media tool for collaborative group writing of secondary school students in Hong Kong, Computers and Education, 110, 170
- 2016, L2 collaborative E-writing, Procedia-Social and Behavioral Sciences, 228, 601
- 2020, Instructor feedback on free writing and automated corrective feedback in drills: intensity and efficacy, Language Teaching Research, 601, 10.1177/1362168820915337 · DOI
- 2018, The effect of automated writing evaluation feedback on students' vocabulary revision -taking Pigai.org for example, Modern Educational Technology, 2, 71
- 2019, The development of vocabulary and morphological awareness: a longitudinal study with college EFL students, Applied Psycholinguistics, 40, 877, 10.1017/S014271641900002X · DOI
- 2014, An evidenced-based rearch on the role of web 2.0 in building up college students' collaborative learning capabilities, China Educational Technology, 12, 139
- Khadka, S. (2020), “Meaning and teaching of writing in higher education in Nepal”, in Bista, K., Sharma, S. and Raby, R.L. (Eds), Higher Education in Nepal: Policies and Perspectives, Routledge, Oxford, pp. 201-213. · doi:10.4324/9781351139205-15
- 2016, Korean EFL freshman students' English learning experiences in high school and in college, The Journal of Studies in Language, 32, 1, 10.18627/jslg.32.1.201605.1 · doi:10.18627/jslg.32.1.201605.1
- 2017, Science-writing in the blogosphere as a tool to promote autonomous motivation in education, The Internet and Higher Education, 35, 48
- 2019, The impact of the integrated feedback on students' writing revision based on the AWE, Foreign Language Education, 40, 72
- 2011, Study on the effect of web-based collaborative writing, Web-Assisted Education, 219, 67
- 2015, Freshmen's new words consciousness and its influence on language output competence measured on a corpus basis, Higher Education Exploration, 2015, 91
- 2018, Group awareness increases student engagement in online collaborative writing, The Internet and Higher Education, 38, 1
- 2016, A study of the second writing process based on an automated essay evaluation tools, Foreign Language World, 2016, 88
- 1977, Effect of expectation and disconfirmation on postexposure product evaluations–an alternative interpretation, Journal of Applied Psychology, 62, 480, 10.1037/0021-9010.62.4.480 · DOI · Paper page
- 1980, A cognitive model of the antecedents and consequences of satisfaction decisions, Journal of Marketing Research, 17, 460, 10.1177/002224378001700405 · DOI · Paper page
- 1966, The imminence of grading essays by computer, Phi Delta Kappan, 47, 238
- 2019, Towards context-aware automated writing evaluation systems, 10.1145/3340435.3342722 · DOI
- 2018, The reliability and validity of automated tools for examining variation in syntactic complexity across genres, International Journal of Applied Linguistics, 128, 165
- 2000, Comparing the Validity of Automated and Human Scoring of Essays
- 2017, Automated short essay scoring system using normalized simpson methods, 10.1063/1.5028081 · doi:10.1063/1.5028081
- 2009, Evaluating the Construct-Coverage of the E-Rater® Scoring Engine
- 2006, An evaluation of the IntelliMetricSM essay scoring system, The Journal of Technology, Learning, and Assessment, 4, 1
- 2019, Comparison of automatic and human evaluation of L2 texts in Czech, Issledovanija po slavjanskim jazykam, 24, 93, 10.30530/JSL.2019.04.24.1.93 · doi:10.30530/jsl.2019.04.24.1.93
- 2019, Automatic scoring method of English composition based on language depth perception, 10.1088/1742-6596/1486/4/042045 · doi:10.1088/1742-6596/1486/4/042045
- 2011, Using automated writing evaluation in classroom assessment: a critical review, Foreign Language Teaching and Research, 2011, 273
- 2020, Learner engagement with automated feedback, peer feedback and teacher feedback in an online EFL writing context, System, 91, 10.1016/j.system.2020.102247 · DOI · Paper page
- 2013, An experimental test of the expectancy-disconfirmation theory of citizen satisfaction, Journal of Policy Analysis and Management, 32, 597, 10.1002/pam.21702 · DOI · Paper page
- 2013, Can automated writing evaluation programs help students improve their English writing?, International Journal of Applied Linguistics and English Literature, 2, 6, 10.7575/ijalel.v.2n.1p.6 · DOI
- 2014, Does automated feedback improve writing quality?, Learning Disabilities: A Contemporary Journal, 12, 93
- Winke, P.M. and Isbell, D.R. (2017), “Computer-assisted language assessment”, in Thorne, S. and May, S. (Eds), Language, Education and Technology. Encyclopedia of Language and Education, 3rd ed., Springer, Cham, pp. 1-13. · DOI
- 2015, An empirical study on college English autonomous writing teaching model based on www. pigai.Org, Computer-assisted Foreign Language Education, 2015, 17
- 2013, Blog-assisted process writing for English majors and pedagogic implications, Computer-Assisted Foreign Language Education, 153, 46
- 2015, Incorporation of automated writing evaluation software in language education: a case of evening university, International Journal of Information and Education Technology, 5, 808, 10.7763/IJIET.2015.V5.616 · DOI
- 2014, A discussion on the connection of English teaching between high school and college, Education exploration, 2014, 35
- 2015, An empirical study of blog-assisted EFL process writing: evidence from Chinese non-English majors, Journal of Language Teaching and Research, 6, 189, 10.17507/jltr.0601.23 · doi:10.17507/jltr.0601.23
Cited by
Song, C., & Song, Y. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1260843 · Cited by 717
Marzuki, Widiati, U., Rusdin, D., Darwin, & Indrawati, I. (2023) · Cogent Education · doi:10.1080/2331186x.2023.2236469 · Cited by 524
“Some research suggests that adopting AI-powered writing tools can help students improve their writing (e.g., Kurniati & Fithriani, 2022; Wang, 2022; Zhao, 2022), while others have voiced concerns about the side effect of these tools (e.g., Y. Liu et al., 2022; Lund & Wang, 2023; Qadir, 2022).”
BackgroundLiu, C., Hou, J., Tu, Y.-F., Wang, Y., & Hwang, G.-J. (2023) · Interactive Learning Environments · doi:10.1080/10494820.2021.2012812 · Cited by 231
“More popular in the fi eld of EFL, an automated evaluation system with the support of NLP was developed by researchers to support students ’ EFL reading and writing (Wang, 2020).”
MethodSystematic review for AI-based language learning tools
Woo, J. H., & Choi, H. (2021) · arXiv · doi:10.48550/arxiv.2111.04455 · Cited by 107
Hwang, W.-Y., Nurtantyana, R., Purba, S. W. D., Hariyanti, U., Indrihapsari, Y., & Surjono, H. D. (2023) · Journal of Educational Computing Research · doi:10.1177/07356331221137253 · Cited by 104
Metadata
- Authors
- Wang, Z.
- Venue
- Library Hi Tech
- Publisher
- Emerald
- Published
- 2020-10-14
- Volume
- 40
- Issue
- 1
- Pages
- 80-97
- ISSN
- 0737-8831 (ISSN-L)
- OpenAlex
- W3093253033
- Cited by
- 116
- References
- 4
- FWCI
- 31.93
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.emerald.com/insight/site-policies (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Computer-assisted EFL writing and evaluations based on artificial intelligence: a case from a college reading and writing course. https://app.cochat.ai/papers/10.1108/lht-05-2020-0113
- The paper
- Wang, Z. (2022). Computer-assisted EFL writing and evaluations based on artificial intelligence: a case from a college reading and writing course. Library Hi Tech. https://doi.org/10.1108/lht-05-2020-0113
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.