“A neural approach to automated essay scoring,”
Open accessAbstract
Traditional automated essay scoring systems rely on carefully designed features to evaluate and score essays.The performance of such systems is tightly bound to the quality of the underlying features.However, it is laborious to manually design the most informative features for such a system.In this paper, we develop an approach based on recurrent neural networks to learn the relation between an essay and its assigned score, without any feature engineering.We explore several neural network models for the task of automated essay scoring and perform some analysis to get some insights of the models.The results show that our best system, which is based on long short-term memory networks, outperforms a strong baseline by 5.6% in terms of quadratic weighted Kappa, without requiring any feature engineering.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Over the last decade, AI-based essay grading tools evolved from using handcrafted features to automatically extracted features using deep neural network variants.
“Over the last decade, AI-based essay grading tools evolved from using handcrafted features such as word/sentence count, mean word/sentence length, n-grams, word error rates, POS tags, grammar, and punctuation (Adamson et al., 2014; Phandi et al., 2015; Cummins et al., 2016; Contreras et al., 2018) to automatically extracted features using deep neural network variants (Taghipour and Ng, 2016; Dasgu”
From Proactive and reactive engagement of artificial intelligence methods for education: a review (2023) · cites this paper as Taghipour and Ng, 2016
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
(2014) · Cited by 33,419 · see citing works
Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1179 · Cited by 25,545 · see citing works
Elman, J. L. (1990) · Cognitive Science · doi:10.1207/s15516709cog1402_1 · Cited by 11,212 · see citing works
Natural Language Processing (almost) from ScratchPreprint
Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., & Kuksa, P. P. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1103.0398 · Cited by 6,920 · see citing works
On the difficulty of training recurrent neural networks
Pascanu, R., Mikolov, T., & Bengio, Y. (2012) · arXiv · doi:10.48550/arxiv.1211.5063 · Cited by 4,310 · see citing works
End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF
Ma, X., & Hovy, E. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1101 · Cited by 2,567 · see citing works
Character-Aware Neural Language Models
Kim, Y., Jernite, Y., Sontag, D., & Rush, A. M. (2015) · arXiv · doi:10.48550/arxiv.1508.06615 · Cited by 1,530 · see citing works
AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0
Attali, Y., & Burstein, J. C. (2004) · ETS Research Report Series · doi:10.1002/j.2333-8504.2004.tb01972.x · Cited by 630 · see citing works
Bilingual Word Embeddings for Phrase-Based Machine Translation
Zou, W. Y., Socher, R., Cer, D. M., & Manning, C. D. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1141 · Cited by 529 · see citing works
From Feedforward to Recurrent LSTM Neural Networks for Language Modeling
Sundermeyer, M., Ney, H., & Schlüter, R. (2015) · IEEE/ACM Transactions on Audio Speech and Language Processing · doi:10.1109/taslp.2015.2400218 · Cited by 509 · see citing works
Opinion Mining with Deep Recurrent Neural Networks
Irsoy, O., & Cardie, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1080 · Cited by 384 · cited 1× · see citing works
“Apart from these examples, recurrent neural networks have also been used for opinion mining (Irsoy and Cardie, 2014), sequence labeling (Ma and Hovy, 2016), language modeling (Kim et al.”
Automatic Text Scoring Using Neural Networks
Alikaniotis, D., Yannakoudakis, H., & Rei, M. (2016) · Apollo (University of Cambridge) · doi:10.17863/cam.376 · Cited by 267 · see citing works
Flexible Domain Adaptation for Automated Essay Scoring Using Correlated Linear Regression
Phandi, P., Chai, K. M. A., & Ng, H. T. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1049 · Cited by 171 · see citing works
Self-citationModeling Argument Strength in Student Essays
Persing, I., & Ng, V. (2015) · doi:10.3115/v1/p15-1053 · Cited by 150 · see citing works
Automated Essay Scoring by Maximizing Human-Machine Agreement
Chen, H., & He, B. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1180 · Cited by 131 · see citing works
Improved Deep Learning Baselines for Ubuntu Corpus DialogsPreprint
Kadlec, R., Schmid, M., & Kleindienst, J. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.03753 · Cited by 91 · see citing works
Task-Independent Features for Automated Essay Grading
Zesch, T., Wojatzki, M., & Scholten-Akoun, D. (2015) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.3115/v1/w15-0626 · Cited by 79 · see citing works
Modeling Organization in Student Essays
Persing, I., Davis, A., & Ng, V. (2010) · Cited by 77 · see citing works
Evaluating the performance of Automated Text Scoring systems
Yannakoudakis, H., & Cummins, R. (2015) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.3115/v1/w15-0625 · Cited by 46 · see citing works
Flagged first, then by citations
Research works printed but not linked: 6
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Ben- gio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the 3rd International Conference on Learning Represen- tations.
- Yann N. Dauphin, Harm de Vries, and Yoshua Bengio. 2015. Equilibrated adaptive learning rates for non- convex optimization. In Advances in Neural Informa- tion Processing Systems 28.
- Peter W Foltz, Darrell Laham, and Thomas K Landauer. 1999. The Intelligent Essay Assessor: Applications to educational technology. Interactive Multimedia Elec- tronic Journal of Computer-Enhanced Learning.
- Mark D. Shermis and Jill Burstein, editors. 2013. Hand- book of Automated Essay Evaluation: Current Appli- cations and New Directions. Routledge.
- Oriol Vinyals, Lukasz Kaiser, Terry Koo, Slav Petrov, Ilya Sutskever, and Geoffrey Hinton. 2015. Grammar as a foreign language. In Advances in Neural Informa- tion Processing Systems 28.
- Helen Yannakoudakis and Ronan Cummins. 2015. Eval- uating the performance of automated text scoring sys- tems. In Proceedings of the Tenth Workshop on Inno- vative Use of NLP for Building Educational Applica- tions.
Cited by
Sorted by citations
Exploring the potential of using an AI language model for automated essay scoring
Mizumoto, A., & Eguchi, M. (2023) · Research Methods in Applied Linguistics · doi:10.1016/j.rmal.2023.100050 · Cited by 647 · see citing works
WILDS: A Benchmark of in-the-Wild Distribution Shifts
Koh, P. W., Sagawa, S., Marklund, H., Xie, S. M., Zhang, M., Balsubramani, A., Hu, W., Yasunaga, M., Phillips, R. L., Gao, I., et al. (2020) · arXiv · doi:10.48550/arxiv.2012.07421 · Cited by 518 · see citing works
“…models can help in educational settings in a variety of ways: e.g., assisting in grading (Piech et al., 2013; Shermis, 2014; Kulkarni et al., 2014; Taghipour and Ng, 2016), estimating student knowledge and ability (Desmarais and Baker, 2012; Wu et al., 2020), identifying students who need…”
A Systematic Review of Deep Learning Approaches to Educational Data Mining
Hernández-Blanco, A., Herrera-Flores, B., Tomás, D., & Navarro-Colorado, B. (2019) · Complexity · doi:10.1155/2019/1306039 · Cited by 230 · see citing works
Proactive and reactive engagement of artificial intelligence methods for education: a review
Mallik, S., & Gangopadhyay, A. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1151391 · Cited by 139 · see citing works
Guo, S., Zheng, Y., & Zhai, X. (2024) · Education and Information Technologies · doi:10.1007/s10639-024-12491-8 · Cited by 117 · see citing works
Hahn, M. G., Navarro, S. M. B., De La Fuente Valentin, L., & Burgos, D. (2021) · IEEE Access · doi:10.1109/access.2021.3100890 · Cited by 111 · see citing works
“Other inputs include a short free form (e.g., a short sentence [12]), long free form (e.g., an essay [13]).”
BackgroundOn Automated Essay Grading using Large Language Models
Liew, P. Y., & Tan, I. K. T. (2024) · Proceedings of the 2024 8th International Conference on Computer Science and Artificial Intelligence · doi:10.1145/3709026.3709030 · Cited by 41 · see citing works
Comparing GPT-based approaches in automated writing evaluation
Liu, Y., Lu, X., & Qi, H. (2025) · Assessing Writing · doi:10.1016/j.asw.2025.100961 · Cited by 19 · see citing works
Masikisiki, B., Marivate, V., & Hlophe, Y. (2023) · Proceedings of the 4th African Human Computer Interaction Conference · doi:10.1145/3628096.3628747 · Cited by 13 · see citing works
- Authors
- Taghipour, K., & Ng, H. T.
- Affiliations
- National University of Singapore (SG)
- ORCID
- Taghipour, Kaveh
- Venue
- Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing
- Published
- 2016
- Pages
- 1882-1891
- OpenAlex
- W2567547739
- Cited by
- 537
- References
- 20
- Citations by year
- 2026: 64 · 2025: 68 · 2024: 41 · 2023: 74 · 2022: 59 · 2021: 59 · 2020: 65 · 2019: 53 · 2018: 43 · 2017: 11
- FWCI
- 30.14
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Conference on Empirical Methods in Natural Language Processing (EMNLP) (publishedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Information Systems › Educational Technology and Assessment
- Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Keywords
- automated essay scoring, artificial neural networks, writing assessment, natural language processing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
