CoChat

Computer-assisted EFL writing and evaluations based on artificial intelligence: a case from a college reading and writing course

Wang, Zhijie · 2022 · Library Hi Tech
Cited by 116

The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.

Abstract

Purpose: The aim of this study is to explore students' expectations and perceived effectiveness of computer-assisted review tools, and the differences in reliability and validity between human evaluation and automatic evaluation, to find a way to improve students' English writing ability. Design/methodology/approach: Based on the expectancy disconfirmation theory (EDT) and Intelligent Computer-Assisted Language Learning (ICALL) theory, an experiment is conducted through the observation method, semistructured interview method and questionnaire survey method. In the experiment, respondents were asked to write and submit four essays on three online automated essay evaluation (AEE) systems in total, one essay every two weeks. Also, two teacher raters were invited to score the first and last papers of each student. The respondents' feedbacks were investigated to confirm the effectiveness of the AEE system; the evaluation results of the AEE systems and teachers were compared; descriptive statistics was used to analyze the experimental data. Findings: The experiment revealed that the respondents held high expectations for the computer-assisted evaluation tools, and the effectiveness of computer scoring feedback on students was higher than that of teacher scoring feedback. Moreover, at the end of the writing project, the students' independent learning ability and English writing ability were significantly improved. Besides, there was a positive correlation between students' initial expectations of computer-assisted learning tools and the final evaluation of learning results. Originality/value: The innovation lies in the use of observation methods, questionnaire survey methods, data analysis, and other methods for the experiment, and the combination of deep learning theory, EDT and descriptive statistics, which has particular reference value for future works.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1108/lht-05-2020-0113

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 4 references

Research works printed but not linked: 46

  1. 2011, Blogs in language learning: maximizing students' collaborative, Procedia Social and Behavioral Sciences, 18, 537
  2. 2007, Construct Validity of E-Rater in Scoring TOEFL Essays (Research Report No. RR‐07‐21)
  3. 2019, AAEE–automated evaluation of students' essays in Arabic language, Information Processing and Management, 56, 1736, 10.1016/j.ipm.2019.05.008 · DOI
  4. 2019, An overview of automatic essay evaluation in the past two decades, Foreign Language Research, 65
  5. 2014, Evaluation of computer based foreign language learning software by teachers and students, The Turkish Online Journal of Educational Technology, 13, 71
  6. 2019, Deep learning in automated essay scoring, 10.1007/978-3-319-91464-0_30 · doi:10.1007/978-3-319-91464-0_30
  7. 2017, Open innovation in supply networks: an expectation disconfirmation, Journal of Business and Industrial Marketing, 32, 432, 10.1108/JBIM-07-2016-0150 · DOI
  8. 2004, Automated essay evaluation: the criterion online writing service, AI Magazine, 25, 27
  9. 2008, Beyond the design of automated writing evaluation: pedagogical practices and perceived earning effectiveness in EFL writing classes, Language Learning and Technology, 12, 94
  10. 2015, Impacts of the test of English listening comprehension on students' English learning expectations in taiwan, Language and Curriculum, 28, 191, 10.1080/07908318.2015.1027216 · DOI
  11. 2017, Evaluating the use of a social media tool for collaborative group writing of secondary school students in Hong Kong, Computers and Education, 110, 170
  12. 2016, L2 collaborative E-writing, Procedia-Social and Behavioral Sciences, 228, 601
  13. 2020, Instructor feedback on free writing and automated corrective feedback in drills: intensity and efficacy, Language Teaching Research, 601, 10.1177/1362168820915337 · DOI
  14. 2018, The effect of automated writing evaluation feedback on students' vocabulary revision -taking Pigai.org for example, Modern Educational Technology, 2, 71
  15. 2019, The development of vocabulary and morphological awareness: a longitudinal study with college EFL students, Applied Psycholinguistics, 40, 877, 10.1017/S014271641900002X · DOI
  16. 2014, An evidenced-based rearch on the role of web 2.0 in building up college students' collaborative learning capabilities, China Educational Technology, 12, 139
  17. Khadka, S. (2020), “Meaning and teaching of writing in higher education in Nepal”, in Bista, K., Sharma, S. and Raby, R.L. (Eds), Higher Education in Nepal: Policies and Perspectives, Routledge, Oxford, pp. 201-213. · doi:10.4324/9781351139205-15
  18. 2016, Korean EFL freshman students' English learning experiences in high school and in college, The Journal of Studies in Language, 32, 1, 10.18627/jslg.32.1.201605.1 · doi:10.18627/jslg.32.1.201605.1
  19. 2017, Science-writing in the blogosphere as a tool to promote autonomous motivation in education, The Internet and Higher Education, 35, 48
  20. 2019, The impact of the integrated feedback on students' writing revision based on the AWE, Foreign Language Education, 40, 72
  21. 2011, Study on the effect of web-based collaborative writing, Web-Assisted Education, 219, 67
  22. 2015, Freshmen's new words consciousness and its influence on language output competence measured on a corpus basis, Higher Education Exploration, 2015, 91
  23. 2018, Group awareness increases student engagement in online collaborative writing, The Internet and Higher Education, 38, 1
  24. 2016, A study of the second writing process based on an automated essay evaluation tools, Foreign Language World, 2016, 88
  25. 1977, Effect of expectation and disconfirmation on postexposure product evaluations–an alternative interpretation, Journal of Applied Psychology, 62, 480, 10.1037/0021-9010.62.4.480 · DOI · Paper page
  26. 1980, A cognitive model of the antecedents and consequences of satisfaction decisions, Journal of Marketing Research, 17, 460, 10.1177/002224378001700405 · DOI · Paper page
  27. 1966, The imminence of grading essays by computer, Phi Delta Kappan, 47, 238
  28. 2019, Towards context-aware automated writing evaluation systems, 10.1145/3340435.3342722 · DOI
  29. 2018, The reliability and validity of automated tools for examining variation in syntactic complexity across genres, International Journal of Applied Linguistics, 128, 165
  30. 2000, Comparing the Validity of Automated and Human Scoring of Essays
  31. 2017, Automated short essay scoring system using normalized simpson methods, 10.1063/1.5028081 · doi:10.1063/1.5028081
  32. 2009, Evaluating the Construct-Coverage of the E-Rater® Scoring Engine
  33. 2006, An evaluation of the IntelliMetricSM essay scoring system, The Journal of Technology, Learning, and Assessment, 4, 1
  34. 2019, Comparison of automatic and human evaluation of L2 texts in Czech, Issledovanija po slavjanskim jazykam, 24, 93, 10.30530/JSL.2019.04.24.1.93 · doi:10.30530/jsl.2019.04.24.1.93
  35. 2019, Automatic scoring method of English composition based on language depth perception, 10.1088/1742-6596/1486/4/042045 · doi:10.1088/1742-6596/1486/4/042045
  36. 2011, Using automated writing evaluation in classroom assessment: a critical review, Foreign Language Teaching and Research, 2011, 273
  37. 2020, Learner engagement with automated feedback, peer feedback and teacher feedback in an online EFL writing context, System, 91, 10.1016/j.system.2020.102247 · DOI · Paper page
  38. 2013, An experimental test of the expectancy-disconfirmation theory of citizen satisfaction, Journal of Policy Analysis and Management, 32, 597, 10.1002/pam.21702 · DOI · Paper page
  39. 2013, Can automated writing evaluation programs help students improve their English writing?, International Journal of Applied Linguistics and English Literature, 2, 6, 10.7575/ijalel.v.2n.1p.6 · DOI
  40. 2014, Does automated feedback improve writing quality?, Learning Disabilities: A Contemporary Journal, 12, 93
  41. Winke, P.M. and Isbell, D.R. (2017), “Computer-assisted language assessment”, in Thorne, S. and May, S. (Eds), Language, Education and Technology. Encyclopedia of Language and Education, 3rd ed., Springer, Cham, pp. 1-13. · DOI
  42. 2015, An empirical study on college English autonomous writing teaching model based on www. pigai.Org, Computer-assisted Foreign Language Education, 2015, 17
  43. 2013, Blog-assisted process writing for English majors and pedagogic implications, Computer-Assisted Foreign Language Education, 153, 46
  44. 2015, Incorporation of automated writing evaluation software in language education: a case of evening university, International Journal of Information and Education Technology, 5, 808, 10.7763/IJIET.2015.V5.616 · DOI
  45. 2014, A discussion on the connection of English teaching between high school and college, Education exploration, 2014, 35
  46. 2015, An empirical study of blog-assisted EFL process writing: evidence from Chinese non-English majors, Journal of Language Teaching and Research, 6, 189, 10.17507/jltr.0601.23 · doi:10.17507/jltr.0601.23

Cited by

See all 116

Metadata

Authors
Wang, Z.
Venue
Library Hi Tech
Publisher
Emerald
Published
2020-10-14
Volume
40
Issue
1
Pages
80-97
ISSN
0737-8831 (ISSN-L)
OpenAlex
W3093253033
Cited by
116
References
4
FWCI
31.93
Type
journal article
Language
en
Open access
closed
Licence
www.emerald.com/insight/site-policies (tdm)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Computer-assisted EFL writing and evaluations based on artificial intelligence: a case from a college reading and writing course. https://app.cochat.ai/papers/10.1108/lht-05-2020-0113
The paper
Wang, Z. (2022). Computer-assisted EFL writing and evaluations based on artificial intelligence: a case from a college reading and writing course. Library Hi Tech. https://doi.org/10.1108/lht-05-2020-0113

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.