Automated essay scoring: Psychometric guidelines and practices
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. E., & Cohen, J. (1989) · Technometrics · doi:10.2307/1270020 · Cited by 99,588 · see citing works
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,406 · see citing works
Fleiss, J. L., & Cohen, J. (1973) · Educational and Psychological Measurement · doi:10.1177/001316447303300309 · Cited by 3,354 · see citing works
MacArthur, C. A., Graham, S. W., & Fitzgerald, J. (2005) · Cited by 2,327 · see citing works
Shanahan, C. R. (2011) · Cited by 866 · see citing works
AUTOMATED ESSAY SCORING WITH E‐RATER® V.2.0
Attali, Y., & Burstein, J. C. (2004) · ETS Research Report Series · doi:10.1002/j.2333-8504.2004.tb01972.x · Cited by 630 · see citing works
Rating the ratings: Assessing the psychometric quality of rating data.
Saal, F. E., Downey, R. G., & Lahey, M. A. (1980) · Psychological Bulletin · doi:10.1037/0033-2909.88.2.413 · Cited by 545 · see citing works
Shermis, M. D., & Burstein, J. C. (2003) · Routledge eBooks · doi:10.4324/9781410606860 · Cited by 504 · see citing works
Shanahan, T. (2015) · The Elementary School Journal · doi:10.1086/681130 · Cited by 469 · see citing works
(1966) · Phi Delta Kappan · Cited by 449 · see citing works
A framework for evaluation and use of automated scoring
Williamson, D. M., Xi, X., & Breyer, F. J. (2012) · Educational Measurement: Issues and Practice · doi:10.1111/j.1745-3992.2011.00223.x · Cited by 430 · see citing works
Self-citation(2006) · The Journal of Technology, Learning and Assessment (Boston College) · Cited by 424 · see citing works
C-rater: Automated Scoring of Short-Answer Questions
Leacock, C., & Chodorow, M. S. (2003) · Computers and the Humanities · doi:10.1023/a:1025779619903 · Cited by 384 · see citing works
Rearticulating Writing Assessment for Teaching and Learning
Huot, B. (2003) · Utah State University Press eBooks · doi:10.2307/j.ctt46nx5z · Cited by 355 · see citing works
Automated writing evaluation: Defining the classroom research agenda
Warschauer, M., & Ware, P. (2006) · Language Teaching Research · doi:10.1191/1362168806lr190oa · Cited by 337 · see citing works
An Overview of Current Research on Automated Essay Grading
Valenti, S., Neri, F., & Cucchiarelli, A. (2003) · Journal of Information Technology Education Research · doi:10.28945/331 · Cited by 313 · see citing works
On the relation between automated essay scoring and modern views of the writing construct
Deane, P. (2012) · Assessing Writing · doi:10.1016/j.asw.2012.10.002 · Cited by 274 · see citing works
Decision Making while Rating ESL/EFL Writing Tasks: A Descriptive Framework
Cumming, A. H., Kantor, R. N., & Powers, D. E. (2002) · Modern Language Journal · doi:10.1111/1540-4781.00137 · Cited by 243 · see citing works
Automated Scoring of Complex Tasks in Computer-Based Testing
(2006) · Routledge eBooks · doi:10.4324/9780415963572 · Cited by 229 · see citing works
Measuring the Impact of Judge Severity on Examination Scores
Lunz, M. E., Wright, B. D., & Linacre, J. M. (1990) · Applied Measurement in Education · doi:10.1207/s15324818ame0304_3 · Cited by 225 · see citing works
The E-rater® Scoring Engine: Automated Essay Scoring With Natural Language Processing
Shermis, M. D., & Burstein, J. C. (2003) · Routledge eBooks · doi:10.4324/9781410606860-16 · Cited by 159 · see citing works
Weigle, S. C. (1999) · Assessing Writing · doi:10.1016/s1075-2935(00)00010-6 · Cited by 157 · see citing works
Automated Scoring and Annotation of Essays with the Intelligent Essay Assessor™
Shermis, M. D., & Burstein, J. C. (2003) · Routledge eBooks · doi:10.4324/9781410606860-15 · Cited by 153 · see citing works
Towards robust computerised marking of free-text responses
Mitchell, T. M., Russell, T., Broomhead, P., & Aldridge, N. (2002) · Loughborough University Research Repository (Loughborough University) · Cited by 151 · see citing works
Assessing Second Language Writing in Academic Contexts
Leki, I., & Hamp‐Lyons, L. (1993) · College Composition and Communication · doi:10.2307/358997 · Cited by 147 · see citing works
Stumping e-rater:challenging the validity of automated essay scoring
Powers, D. E., Burstein, J. C., Chodorow, M. S., Fowles, M. E., & Kukich, K. (2002) · Computers in Human Behavior · doi:10.1016/s0747-5632(01)00052-8 · Cited by 146 · see citing works
Validity and Automad Scoring: It's Not Only the Scoring
Bennett, R. E., & Bejar, I. I. (1998) · Educational Measurement Issues and Practice · doi:10.1111/j.1745-3992.1998.tb00631.x · Cited by 145 · see citing works
Comparison of Human and Machine Scoring of Essays: Differences by Gender, Ethnicity, and Country
Bridgeman, B., Trapani, C. S., & Attali, Y. (2012) · Applied Measurement in Education · doi:10.1080/08957347.2012.635502 · Cited by 144 · see citing works
The use of the computer in analyzing student essays
Page, E. B. (1968) · International Review of Education · doi:10.1007/bf01419938 · Cited by 137 · see citing works
On a Scale: A Social History of Writing Assessment in America
Elliot, N. (2005) · Cited by 135 · see citing works
A Review of Strategies for Validating Computer-Automated Scoring
Yang, Y., Buckendahl, C. W., Juszkiewicz, P. J., & Bhola, D. S. (2002) · Applied Measurement in Education · doi:10.1207/s15324818ame1504_04 · Cited by 127 · see citing works
Leckie, G., & Baird, J. (2011) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.2011.00152.x · Cited by 111 · see citing works
An Evaluation of IntelliMetric™ Essay Scoring System
Rudner, L. M., García, V. A., & Welch, C. J. (2006) · Open Access Journals at BC (Boston College) · Cited by 101 · see citing works
Understanding Scoring Reliability: Experiments in Calibrating Essay Readers
Braun, H. I. (1988) · Journal of Educational Statistics · doi:10.2307/1164948 · Cited by 98 · see citing works
Cognitive Differences in Proficient and Nonproficient Essay Scorers
Wolfe, E. W., KAO, C.-W., & Ranney, M. A. (1998) · Written Communication · doi:10.1177/0741088398015004002 · Cited by 94 · see citing works
Davis, M. T. (1988) · The Journal of Higher Education · doi:10.1080/00221546.1988.11778315 · Cited by 94 · see citing works
Page, E. B. (2003) · Cited by 91 · see citing works
What Happens When Machines Read Our Students' Writing?
Herrington, A. J., & Moran, C. (2001) · College English · doi:10.2307/378891 · Cited by 81 · see citing works
Validity Issues for Performance-Based Tests Scored With Computer-Automated Scoring Systems
Clauser, B. E., Kane, M. T., & Swanson, D. B. (2002) · Applied Measurement in Education · doi:10.1207/s15324818ame1504_05 · Cited by 80 · see citing works
VALIDITY AND AUTOMATED SCORING: IT'S NOT ONLY THE SCORING
Bennett, R. E., & Bejar, I. I. (1997) · ETS Research Report Series · doi:10.1002/j.2333-8504.1997.tb01734.x · Cited by 69 · see citing works
The SRI EduSpeak System: Recognition and Pronunciation Scoring for Language Learning
Franco, H., Abrash, V., Precoda, K., Bratt, H., Rao, R., Butzberger, J., Rossier, R., & Cesari, F. (2007) · Cited by 67 · see citing works
Automated Scoring Technologies and the Rising Influence of Error
Cheville, J. (2004) · The English Journal · doi:10.2307/4128980 · Cited by 66 · see citing works
Rater effects: Ego engagement in rater decision-making
Wiseman, C. S. (2012) · Assessing Writing · doi:10.1016/j.asw.2011.12.001 · Cited by 66 · see citing works
Callear, D., Jerrams-Smith, J., & Soh, V. C.-M. (2001) · Loughborough University Institutional Repository (Loughborough University) · Cited by 64 · see citing works
Performance of a Generic Approach in Automated Essay Scoring
Attali, Y., Bridgeman, B., & Trapani, C. S. (2010) · The Journal of Technology, Learning and Assessment (Boston College) · Cited by 58 · see citing works
‘Mental Model’ Comparison of Automated and Human Scoring
Williamson, D. M., Bejar, I. I., & Hone, A. S. (1999) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1999.tb00552.x · Cited by 58 · see citing works
Self-citationCOMPARING THE VALIDITY OF AUTOMATED AND HUMAN SCORING OF ESSAYS
Powers, D. E., Burstein, J. C., Chodorow, M. S., Fowles, M. E., & Kukich, K. (2002) · Journal of Educational Computing Research · doi:10.2190/cx92-7wkv-n7wc-jl0a · Cited by 49 · see citing works
CCCC Position Statement on Teaching, Learning, and Assessing Writing in Digital Environments
(2004) · College Composition and Communication · doi:10.58680/ccc20042788 · Cited by 46 · see citing works
Towards automatic scoring of non-native spontaneous speech
Zechner, K., & Bejar, I. I. (2006) · doi:10.3115/1220835.1220863 · Cited by 44 · see citing works
Two Experiments on Automatic Scoring of Spoken Language Proficiency
Bernstein, J., de Jong, J. H. A. L., Pisoni, D. B., & Townshend, B. (2000) · Cited by 43 · see citing works
Reducing Errors Due to the Use of Judges
Rudner, L. M. (2020) · Scholarworks (University of Massachusetts Amherst) · doi:10.7275/w4a1-cb66 · Cited by 41 · see citing works
Hamp‐Lyons, L. (2007) · Assessing Writing · doi:10.1016/j.asw.2007.05.002 · Cited by 41 · see citing works
Hales, L. W., & Tokar, E. (1975) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1975.tb01014.x · Cited by 40 · see citing works
Sukkarieh, J. Z., & Pulman, S. (2005) · Cited by 38 · see citing works
Ramineni, C., Trapani, C. S., Williamson, D. M., Davey, T., & Bridgeman, B. (2012) · ETS Research Report Series · Cited by 34 · see citing works
Self-citationValidating automated essay scoring for online writing placement
Ramineni, C. (2012) · Assessing Writing · doi:10.1016/j.asw.2012.10.005 · Cited by 31 · see citing works
Self-citationMoving the Field Forward: Some Thoughts on Validity and Automated Scoring
(2006) · Routledge eBooks · doi:10.4324/9780415963572-15 · Cited by 28 · see citing works
A human-computer collaborative approach to the marking of free text answers
Sargeant, J., Wood, M. M., & Anderson, S. M. (2004) · Loughborough University Research Repository (Loughborough University) · Cited by 26 · see citing works
THE USE OF MODEL ESSAYS TO REDUCE CONTEXT EFFECTS IN ESSAY SCORING
HUGHES, D. C., & Keeling, B. (1984) · Journal of Educational Measurement · doi:10.1111/j.1745-3984.1984.tb01034.x · Cited by 26 · see citing works
Automated Scoring of Spontaneous Speech Using SpeechRater? v1.0. Research Report. ETS RR-08-62.
Xi, X., Higgins, D., Zechner, K., & Williamson, D. M. (2008) · ETS Research Report Series · Cited by 24 · see citing works
Self-citationEvaluating the Construct-Coverage of the e-rater[R] Scoring Engine. Research Report. ETS RR-09-01.
Quinlan, T., Higgins, D., & Wolff, S. (2009) · TOEFL iBT research reports · Cited by 22 · see citing works
Automated Scoring of Constructed-Response Literacy and Mathematics Items
Elliot, R. (2011) · Cited by 19 · see citing works
A Regression-Based Procedure for Automated Scoring of a Complex Medical Performance Assessment
(2006) · Routledge eBooks · doi:10.4324/9780415963572-7 · Cited by 18 · see citing works
Ramineni, C., Trapani, C. S., Williamson, D. M., Davey, T., & Bridgeman, B. (2012) · ETS Research Report Series · Cited by 15 · see citing works
Self-citationNEW SAT® WRITING PROMPT STUDY: ANALYSES OF GROUP IMPACT AND RELIABILITY
Breland, H. M., Kubota, M. Y., Nickerson, K., Trapani, C. S., & Walker, M. E. (2004) · ETS Research Report Series · doi:10.1002/j.2333-8504.2004.tb01930.x · Cited by 14 · see citing works
How important is content in the ratings of essay assessments?
Shermis, M. D., Shneyderman, A., & Attali, Y. (2008) · Assessment in Education Principles Policy and Practice · doi:10.1080/09695940701876219 · Cited by 11 · see citing works
Testing and Assessing Mathematical Skills by a Script Based System
Risse, T. (2007) · Cited by 9 · see citing works
Flagged first, then by citations
3 of the 70 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Baker, R. S., & Hawn, A. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00285-9 · Cited by 993 · see citing works
A hierarchical classification approach to automated essay scoring
McNamara, D. S., Crossley, S. A., Roscoe, R. D., Allen, L. K., & Dai, J. (2014) · Assessing Writing · doi:10.1016/j.asw.2014.09.002 · Cited by 243 · see citing works
Shermis, M. D. (2014) · Assessing Writing · doi:10.1016/j.asw.2013.04.001 · Cited by 227 · see citing works
Stop! Grammar time: University students’ perceptions of the automated feedback program Grammarly
ONeill, R., & Russell, A. (2019) · Australasian Journal of Educational Technology · doi:10.14742/ajet.3795 · Cited by 180 · see citing works
Graduate Theses and Dissertations
Davis, M., Davis, K. J., & Dunagan, M. M. (2012) · Elsevier eBooks · doi:10.1016/b978-0-12-384727-0.00006-9 · Cited by 165 · see citing works
Initial investigation into computer scoring of candidate essays for personnel selection.
Campion, M. C., Campion, M. A., Campion, E. D., & Reider, M. H. (2016) · Journal of Applied Psychology · doi:10.1037/apl0000108 · Cited by 130 · see citing works
Bui, N. M., & Barrot, J. S. (2025) · Education and Information Technologies · doi:10.1007/s10639-024-12891-w · Cited by 113 · see citing works
Condon, W. (2012) · Assessing Writing · doi:10.1016/j.asw.2012.11.001 · Cited by 113 · see citing works
The role of automated writing evaluation holistic scores in the ESL classroom
Li, Z., Link, S., Ma, H., Yang, H., & Hegelheimer, V. H. (2014) · System · doi:10.1016/j.system.2014.02.007 · Cited by 113 · see citing works
Beyond single-word measures: L2 writing assessment, lexical richness and formulaic competence
Bestgen, Y. (2017) · System · doi:10.1016/j.system.2017.08.004 · Cited by 107 · see citing works
A systematic review of automated writing evaluation systems
Shi, H., & Aryadoust, V. (2022) · Education and Information Technologies · doi:10.1007/s10639-022-11200-7 · Cited by 111 · see citing works
Appropriating automated feedback in L2 writing: experiences of Chinese EFL student writers
Jiang, L. G., & Yu, S. (2020) · Computer Assisted Language Learning · doi:10.1080/09588221.2020.1799824 · Cited by 88 · see citing works
Automated Scoring of Nonnative Speech Using the SpeechRaterSMv. 5.0 Engine
Chen, L., Zechner, K., Yoon, S., Evanini, K., Wang, X., Loukina, A., Tao, J., Davis, L., Lee, C., Ma, M., et al. (2018) · ETS Research Report Series · doi:10.1002/ets2.12198 · Cited by 69 · see citing works
The many dimensions of algorithmic fairness in educational applications
Loukina, A., Madnani, N., & Zechner, K. (2019) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/w19-4401 · Cited by 64 · see citing works
What has been assessed in writing and how? Empirical evidence from Assessing Writing (2000–2018)
Zheng, Y., & Yu, S. (2019) · Assessing Writing · doi:10.1016/j.asw.2019.100421 · Cited by 60 · see citing works
Automated Evaluation of Writing – 50 Years and Counting
Klebanov, B. B., & Madnani, N. (2020) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2020.acl-main.697 · Cited by 60 · see citing works
Feature selection for automated speech scoring
Loukina, A., Zechner, K., Chen, L., & Heilman, M. (2015) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.3115/v1/w15-0602 · Cited by 52 · see citing works
Advances in the Field of Automated Essay Evaluation
Zupanc, K., & Bosnić, Z. (2016) · Informatica · Cited by 46 · see citing works
Building Better Open-Source Tools to Support Fairness in Automated Scoring
Madnani, N., Loukina, A., von Davier, A. A., Burstein, J. C., & Cahill, A. (2017) · doi:10.18653/v1/w17-1605 · Cited by 45 · see citing works
Thompson, I. B., Koenig, N., Mracek, D. L., & Tonidandel, S. (2023) · Journal of Business and Psychology · doi:10.1007/s10869-023-09874-y · Cited by 44 · see citing works
AI-assisted automated scoring of picture-cued writing tasks for language assessment
Zhao, R., Zhuang, Y., Zou, D., Xie, Q., & Yu, P. L. H. (2023) · Education and Information Technologies · doi:10.1007/s10639-022-11473-y · Cited by 40 · see citing works
Building e‐rater® Scoring Models Using Machine Learning Methods
Chen, J., Fife, J. H., Bejar, I. I., & Rupp, A. A. (2016) · ETS Research Report Series · doi:10.1002/ets2.12094 · Cited by 35 · see citing works
A Fairness Evaluation of Automated Methods for Scoring Text Evidence Usage in Writing
Litman, D. J., Zhang, H., Correnti, R. J., Matsumura, L. C., & Wang, E. L. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-78292-4_21 · Cited by 33 · see citing works
Validating automated essay scoring for online writing placement
Ramineni, C. (2012) · Assessing Writing · doi:10.1016/j.asw.2012.10.005 · Cited by 31 · see citing works
Self-citationAutomated Writing Analysis for Writing Pedagogy
Cotos, E. (2015) · Writing & Pedagogy · doi:10.1558/wap.v7i2-3.26381 · Cited by 37 · see citing works
Evaluating China’s Automated Essay Scoring System iWrite
Qian, L., Zhao, Y., & Cheng, Y. (2019) · Journal of Educational Computing Research · doi:10.1177/0735633119881472 · Cited by 31 · see citing works
Ramineni, C., & Williamson, D. M. (2018) · ETS Research Report Series · doi:10.1002/ets2.12192 · Cited by 29 · see citing works
Self-citationAn NCME Instructional Module on Data Mining Methods for Classification and Regression
Sinharay, S. (2016) · Educational Measurement Issues and Practice · doi:10.1111/emip.12115 · Cited by 30 · see citing works
Automated Essay Scoring in Innovative Assessments of Writing from Sources
Deane, P., Williams, F., Weng, V. Z., & Trapani, C. S. (2013) · eScholarship (California Digital Library) · Cited by 28 · see citing works
Automated scoring in context: Rapid assessment for placed students
Klobucar, A., Elliot, N., Deess, P., Rudniy, O., & Joshi, K. (2012) · Assessing Writing · doi:10.1016/j.asw.2012.10.001 · Cited by 28 · see citing works
Validity of automated essay scores for elementary-age English language learners: Evidence of bias?
Wilson, J., & Huang, Y. (2024) · Assessing Writing · doi:10.1016/j.asw.2024.100815 · Cited by 27 · see citing works
Automated Essay Scoring Systems
Ifenthaler, D. (2022) · Handbook of Open, Distance and Digital Education · doi:10.1007/978-981-19-0351-9_59-1 · Cited by 26 · see citing works
Automated Scoring with Validity in Mind
Bejar, I. I., Mislevy, R. J., & Zhang, M. (2016) · John Wiley & Sons, Inc. eBooks · doi:10.1002/9781118956588.ch10 · Cited by 25 · see citing works
Automated Writing Instruction and Feedback: Instructional Mode, Attitudes, and Revising
Roscoe, R. D., Allen, L. K., Johnson, A. C., & McNamara, D. S. (2018) · Proceedings of the Human Factors and Ergonomics Society Annual Meeting · doi:10.1177/1541931218621471 · Cited by 26 · see citing works
Combining machine translation and automated scoring in international large-scale assessments
Jung, J. Y., Tyack, L., & von Davier, M. (2024) · Large-scale Assessments in Education · doi:10.1186/s40536-024-00199-7 · Cited by 21 · see citing works
An exploration of automated narrative analysis via machine learning
Jones, S., Fox, C., Gillam, S., & Gillam, R. B. (2019) · PLOS ONE · doi:10.1371/journal.pone.0224634 · Cited by 20 · see citing works
Xie, Q. (2015) · Assessing Writing · doi:10.1016/j.asw.2015.05.001 · Cited by 20 · see citing works
Vo, Y., Rickels, H., Welch, C., & Dunbar, S. (2023) · Assessing Writing: An International Journal · doi:10.1016/j.asw.2023.100719 · Cited by 21 · see citing works
Impact of AWE Rubrics and Automated Assessment on EFL Writing Instruction
Tang, J., & Wu, Y. (2017) · International Journal of Computer-Assisted Language Learning and Teaching · doi:10.4018/ijcallt.2017040104 · Cited by 15 · see citing works
Raczynski, K. R., & Cohen, A. S. (2018) · Applied Measurement in Education · doi:10.1080/08957347.2018.1464449 · Cited by 14 · see citing works
Lillo-Fuentes, F., René Venegas, V., & Lobos, I. (2023) · Perspectiva educacional · doi:10.4151/07189729-vol.62-iss.2-art.1420 · Cited by 11 · see citing works
Suhan, M., & Wolf, M. K. (2025) · Language Testing · doi:10.1177/02655322251346860 · Cited by 10 · see citing works
Text Mining and Automated Scoring
Flor, M., & Hao, J. (2021) · Methodology of educational measurement and assessment · doi:10.1007/978-3-030-74394-9_14 · Cited by 9 · see citing works
Myers, M. C., & Wilson, J. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-022-00301-6 · Cited by 9 · see citing works
Evaluation of e‐rater® for the Praxis I® Writing Test
Ramineni, C., Trapani, C. S., & Williamson, D. M. (2015) · ETS Research Report Series · doi:10.1002/ets2.12047 · Cited by 9 · see citing works
Self-citationWang, J., & Bai, L. (2021) · International Journal of English Linguistics · doi:10.5539/ijel.v11n2p68 · Cited by 9 · see citing works
Automated Essay Scoring Systems
Ifenthaler, D. (2023) · Handbook of Open, Distance and Digital Education · doi:10.1007/978-981-19-2080-6_59 · Cited by 9 · see citing works
Linguistic predictors of L2 writing performance: Variations across genres
Yang, W., Cushing, S. T., & Yu, G. (2025) · Assessing Writing · doi:10.1016/j.asw.2025.100985 · Cited by 8 · see citing works
Automated evaluation of the quality of ideas in compositions based on concept maps
Yang, L., Xin, T., Luo, F., Zhang, S., & Tian, X. (2021) · Natural Language Engineering · doi:10.1017/s1351324921000103 · Cited by 8 · see citing works
Human Versus Automated Essay Scoring: A Critical Review
Journal, A. W. E., & Sevcikova, B. L. (2018) · Arab World English Journal · doi:10.2139/ssrn.3201916 · Cited by 8 · see citing works
Makassar, U. M., Syamsuddin, A., Jannah, M., & Kristiawati, K. (2019) · MaPan · doi:10.24252/mapan.2019v7n1a11 · Cited by 8 · see citing works
Human Systems Engineering and Educational Technology
Roscoe, R. D., Branaghan, R. J., Cooke, N. J., & Craig, S. D. (2017) · Advances in educational technologies and instructional design book series · doi:10.4018/978-1-5225-2639-1.ch001 · Cited by 8 · see citing works
Oğuz, E. (2025) · Assessing Writing · doi:10.1016/j.asw.2025.100981 · Cited by 11 · see citing works
Intelligent Tutoring Systems for Literacy: Existing Technologies and Continuing Challenges.
Jacovina, M. E., & McNamara, D. S. (2016) · Grantee Submission · Cited by 7 · see citing works
Klebanov, B. B., & Madnani, N. (2021) · Synthesis lectures on human language technologies · doi:10.2200/s01121ed1v01y202108hlt052 · Cited by 7 · see citing works
Automatic Medical Case Study Essay Scoring by Support Vector Machine and Genetic Algorithms
Yenaeng, S., Saelee, S., & Samai, W. (2013) · International Journal of Information and Education Technology · doi:10.7763/ijiet.2014.v4.384 · Cited by 7 · see citing works
Reproducing Monolingual, Multilingual and Cross-Lingual CEFR Predictions
Bestgen, Y. (2020) · Digital Access to Libraries (Université catholique de Louvain (UCL), l'Université de Namur (UNamur) and the Université Saint-Louis (USL-B)) · Cited by 7 · see citing works
How Reliable Is It to Automatically Score Open-Ended Items? An Application in the Turkish Language
UYSAL, İ., & Doğan, N. (2021) · Eğitimde ve Psikolojide Ölçme ve Değerlendirme Dergisi · doi:10.21031/epod.817396 · Cited by 6 · see citing works
Transformer-LSTM Models for Automatic Scoring and Feedback in English Writing Assessment
Xuan, Y. (2025) · IEEE Access · doi:10.1109/access.2025.3562493 · Cited by 5 · see citing works
Student Engagement With AI Feedback: Evidence From Ethnic Minority Students in China
Yang, F., Li, K., & Qian, L. (2025) · International Journal of Applied Linguistics · doi:10.1111/ijal.70066 · Cited by 5 · see citing works
Bestgen, Y. (2019) · Revue française de linguistique appliquée · doi:10.3917/rfla.241.0081 · Cited by 5 · see citing works
Computer Scoring of Emotional Awareness in a Nonclinical Population of Young Adults
Barchard, K. A., & Picker, C. J. (2017) · Journal of Personality Assessment · doi:10.1080/00223891.2017.1282866 · Cited by 5 · see citing works
A Study for the Development of Automated Essay Scoring (AES) in Malaysian English Test Environment
Wong, W. S., & Bong, C. H. (2019) · International Journal of Innovative Computing · doi:10.11113/ijic.v9n1.220 · Cited by 5 · see citing works
Feenstra, H. (2014) · doi:10.3990/1.9789036537254 · Cited by 5 · see citing works
Campion, M. A. (2025) · Human Resource Management · doi:10.1002/hrm.70025 · Cited by 4 · see citing works
Jiang, Y., Klebanov, B. B., Hao, J., Deane, P., & Livne, O. E. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.70014 · Cited by 5 · see citing works
Predicting Evaluations of Essay by Computational Graph-Based Features
Yang, L., Xin, T., & Cao, C. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.531262 · Cited by 4 · see citing works
Adapting for Scalability: Automating the Video Assessment of Instructional Learning
Roberts, A., LoCasale‐Crouch, J., Hamre, B. K., & Buckrop, J. (2017) · Online Learning · doi:10.24059/olj.v21i1.961 · Cited by 4 · see citing works
Zhai, Y. (2025) · BMC Psychology · doi:10.1186/s40359-025-03765-y · Cited by 3 · see citing works
Human Systems Engineering and Educational Technology
Roscoe, R. D., Branaghan, R. J., Cooke, N. J., & Craig, S. D. (2018) · IGI Global eBooks · doi:10.4018/978-1-5225-5472-1.ch104 · Cited by 3 · see citing works
Predicting Second Language Learner Successes and Mistakes by Means of Conjunctive Features
Bestgen, Y. (2018) · Workshop on Innovative Use of NLP for Building Educational Applications · doi:10.18653/v1/w18-0542 · Cited by 3 · see citing works
Kara, A., Kara, Z. A., & Yıldırım, S. (2025) · International Journal of Assessment Tools in Education · doi:10.21449/ijate.1687429 · Cited by 2 · see citing works
Source integration in source-based writing: From theoretical modeling to predictive analytics
Potter, A., Tian, Y., Balyan, R., Goldshtein, M., Allen, L. K., & McNamara, D. S. (2026) · Assessing Writing · doi:10.1016/j.asw.2026.101081 · Cited by 3 · see citing works
Nazari, S., Leite, W. L., & Huggins‐Manley, A. C. (2024) · Educational and Psychological Measurement · doi:10.1177/00131644241255109 · Cited by 2 · see citing works
Modeling Directional Testlet Effects on Multiple Open‐Ended Questions
Jin, K., & Siu, W. (2024) · Journal of Educational Measurement · doi:10.1111/jedm.12422 · Cited by 2 · see citing works
Context-aware Argument Mining and Its Applications in Education
Nguyen, H. V. (2018) · D-Scholarship@Pitt (University of Pittsburgh) · Cited by 2 · see citing works
User-centered & Robust NLP OSS: Lessons Learned from Developing & Maintaining RSMTool
Madnani, N., & Loukina, A. (2020) · Workshop for NLP Open Source Software (NLP-OSS) · doi:10.18653/v1/2020.nlposs-1.20 · Cited by 2 · see citing works
Enhancing Rubric Development in Science Education through Topic Modeling Techniques
Hong, M., & You, H. S. (2024) · The Journal of Experimental Education · doi:10.1080/00220973.2024.2329094 · Cited by 2 · see citing works
An Evaluation of China’s Automated Scoring System Bingo English
Gao, J., Li, X., Gu, P., & Liu, Z. (2020) · International Journal of English Linguistics · doi:10.5539/ijel.v10n6p30 · Cited by 2 · see citing works
Improvement of GAN-LCS Performance with Synonym Recognition
Saskia, L., Hidayah, I., & Kusumawardani, S. S. (2022) · International Conference on Information Technology, Information Systems and Electrical Engineering (ICITISEE) · doi:10.1109/icitisee57756.2022.10057648 · Cited by 1 · see citing works
Kekurangan Aplikasi Grammarly dalam Mendeteksi Kesalahan pada Karya Tulis Mahasiswa
Tambunsaribu, G., & Galingging, Y. (2023) · DIALEKTIKA JURNAL BAHASA SASTRA DAN BUDAYA · doi:10.33541/dia.v10i1.4838 · Cited by 1 · see citing works
Effects of Prompts and Time on the Automated Scoring of English Argumentative Essays by ChatGPT 4
Lin, D., & Pu, X. (2024) · International Symposium on Artificial Intelligence for Education (ISAIE) · doi:10.1145/3700297.3700350 · Cited by 3 · see citing works
Owen, N., & Senel, A. (2025) · Review of Education · doi:10.1002/rev3.70096 · Cited by 4 · see citing works
Johari, M. R., & Mukherjee, S. (2026) · Education Sciences · doi:10.3390/educsci16020211 · Cited by 1 · see citing works
Uto, M. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29744-0_37 · Cited by 1 · see citing works
Semiautomatic Grading of Short Texts for Open Answers in Higher Education
de‐la‐Fuente‐Valentín, L., Verdú, E., Padilla-Zea, N., Villalonga, C., Valencia, X. P. B., & Baldiris, S. (2022) · Communications in computer and information science · doi:10.1007/978-3-030-96060-5_4 · Cited by 1 · see citing works
Keyword-Based Processing for Assessing Short Answers in the Educational Field
Sanz-Fayos, J., de‐la‐Fuente‐Valentín, L., & Verdú, E. (2022) · Communications in computer and information science · doi:10.1007/978-3-031-19647-8_10 · Cited by 1 · see citing works
Retrospective. From Isolation to Integration: Technical Issues in the Assessment of Writing
Slomp, D. (2024) · The WAC Clearinghouse; University Press of Colorado eBooks · doi:10.37514/per-b.2024.2166.2.21 · Cited by 1 · see citing works
Williamson, M. T. (2024) · The WAC Clearinghouse; University Press of Colorado eBooks · doi:10.37514/per-b.2024.2166.2.08 · Cited by 1 · see citing works
Chapter 10. Globalizing Plagiarism and Writing Assessment: A Case Study of Turnitin
Canzonetta, J., & Kannan, V. (2024) · The WAC Clearinghouse; University Press of Colorado eBooks · doi:10.37514/per-b.2024.2166.2.10 · Cited by 1 · see citing works
Responsible Implementation of AI in Higher Education: Ethical Factors Guiding Dutch IT Teachers
van Meerten, J., Smit, K., van Steenbergen, M., & Berkhout, M. (2024) · Resilience Through Digital Innovation: Enabling the Twin Transition · doi:10.18690/um.fov.4.2024.19
Exploring Learner Engagement with the AWE-Peer-Teacher Feedback Mode in L2 Writing
Zhou, S., Liu, Q., & Zhang, Y. (2024) · Lecture notes in computer science · doi:10.1007/978-981-97-4246-2_3 · Cited by 2 · see citing works
Navigating the Ethical Frontier
Öztürk, H. İ., Doğuş, M., Şahin, V., Çakmak, M., & Gürses, İ. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-0122-8.ch011 · Cited by 1 · see citing works
Yu, W., & Barker, T. (2020) · Education Language and Sociology Research · doi:10.22158/elsr.v1n1p20 · Cited by 1 · see citing works
L1 feedback in automated writing evaluation: From learners' perspectives
Wilken, J. L. (2013) · doi:10.31274/etd-180810-3388 · Cited by 1 · see citing works
No Right Answer: A Feasibility Study of Essay Assessment with LDA
Roberts, T. (2013)
Yenaeng, S., Saelee, S., & Samai, W. (2018) · AIP conference proceedings · doi:10.1063/1.5019548
채점 자질 설계를 통한 지도 학습 기반 작문 자동 채점의 타당도 확보 방안 탐색
Ju, K. S. (2019) · Journal of CheongRam Korean Language Education · doi:10.26589/jockle..69.201903.265
- Authors
- Ramineni, C., & Williamson, D. M.
- Affiliations
- Educational Testing Service (US)
- Corresponding author
- Ramineni, Chaitanya
- Venue
- Assessing Writing
- Publisher
- Elsevier BV
- Published
- 2013-01
- Volume
- 18
- Issue
- 1
- Pages
- 25-39
- ISSN
- 1075-2935 (ISSN-L)
- OpenAlex
- W2088070094
- Cited by
- 127
- References
- 70
- Citations by year
- 2026: 11 · 2025: 15 · 2024: 26 · 2023: 6 · 2022: 7 · 2021: 7 · 2020: 7 · 2019: 9 · 2018: 10 · 2017: 7 · 2016: 6 · 2015: 4 · 2014: 4 · 2013: 4 · 2012: 4
- FWCI
- 13.89
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Assessing Writing (publishedVersion)
- Funders
- Educational Testing Service
- Topics
- Physical Sciences › Computer Science › Information Systems › Educational Technology and Assessment
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Education › Student Assessment and Feedback
- Keywords
- automated essay scoring, e-rater, psychometric assessment, writing assessment, large-scale assessment, performance expectations, Educational Testing Service
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
