CoChat
Sign Up
Cited by 41Educational Technology and AssessmentArtificial Intelligence in EducationTopic Modeling

Automatic Short Answer Grading in the LLM Era: Does GPT-4 with Prompt Engineering beat Traditional Models?

Open access
Ferreira Mello, Rafael, Pereira Junior, Cleon, Rodrigues, Luiz, Pereira, Filipe Dwan, Cabral, Luciano, Costa, Newarney, Ramalho, Geber, Gasevic, Dragan·2025·Proceedings of the 15th International Learning Analytics and Knowledge Conference
DOI 10.1145/3706468.3706481
Open this paper in CoChat — free account

Abstract

Assessing short answers in educational settings is challenging due to the need for scalability and accuracy, which led to the field of Automatic Short Answer Grading (ASAG). Traditional machine learning models, such as ensemble and embeddings, have been widely researched in ASAG, but they often suffer from generalizability issues. Recently, Large Language Models (LLMs) emerged as an alternative to optimize ASAG systems. However, previous research has failed to present a comprehensive analysis of LLMs' performance powered by prompt engineering strategies and compare its capabilities to traditional models. This study presents a comparative analysis between traditional machine learning models and GPT-4 in the context of ASAG. We investigated the effectiveness of different models and text representation techniques and explored prompt engineering strategies for LLMs. The results indicate that traditional machine learning models outperform LLMs. However, GPT-4 showed promising capabilities, especially when configured with optimized prompt components, such as few-shot examples and clear instructions. This study contributes to the literature by providing a detailed evaluation of LLM performance compared to traditional machine learning models in a multilingual ASAG context, offering insights for developing more efficient automatic grading systems.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3706468.3706481

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Universities increasingly rely on AI systems to assess, classify, and enforce academic norms.

    “Because universities increasingly rely on AI systems to assess, classify, and enforce academic norms [16, 27, 37, 63]”

    From The Due Process Deficit: Auditing AI Governance in U.S. Higher Education (2026) · cites this paper as [27]

    The Introduction describes automatic short-answer grading as a promising way to assess student responses, supporting the assessment part of the claim. It does not establish that universities increasingly rely on these systems, or that they use them to classify and enforce academic norms. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • LLMs have demonstrated substantial capabilities in educational practice, e.g., automatic grading, answer generation, and student-chatbot interaction.

    “Large Language Models (LLMs) have demonstrated substantial capabilities in educational practice, e.g., automatic grading (Ferreira Mello et al., 2025), answer generation (Rodrigues et al., 2025), and student-chatbot interaction (Woollaston et al., 2025; Hao et al., 2026; Zhang et al., 2025).”

    From Assessing the Impact and Underlying Pathways of Sequenced AI feedback on Student Learning (2026) · cites this paper as Ferreira Mello et al., 2025

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 53

  1. [1][1] Sridevi Bonthu, S Rama Sree, and MHM Krishna Prasad. 2021. Automated short answer grading using deep learning: A survey. In Machine Learning and Knowledge Extraction: 5th IFIP TC 5, TC 12, WG 8.4, WG 8.9, WG 12.9 International Cross-Domain Conference, CD-MAKE 2021, Virtual Event, August 17–20, 2021, Proceedings 5. Springer, 61–78. Digital Library Google Scholar a [...] process without compromising quality b [...] standard answers or specific criteria
  2. [2][2] S Brindha, K Prabha, and Sandeep Sukumaran. 2016. A survey on classification techniques for text mining. In 2016 3rd International Conference on Advanced Computing and Communication Systems (ICACCS), Vol. 1. IEEE, 1–5. Crossref Google Scholar a [...] problems in the field of Text Mining b [...] are from the actual values on average
  3. [3][3] Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. arxiv:https://arXiv.org/abs/2005.14165 [cs.CL] Google Scholar a [...] In another study, Brown et al. b [...] responses (referred to as few-shot prompts) · arXiv
  4. [4][4] Steven Burrows, Iryna Gurevych, and Benno Stein. 2015. The eras and trends of automatic short answer grading. International journal of artificial intelligence in education 25 (2015), 60–117. Google Scholar a [...] approaches to maximize learning outcomes b [...] to open-ended questions automatically
  5. [5][5] Leon Camus and Anna Filighera. 2020. Investigating transformers for automatic short answer grading. In Artificial Intelligence in Education: 21st International Conference, AIED 2020, Ifrane, Morocco, July 6–10, 2020, Proceedings, Part II 21. Springer, 43–48. Go to Citation Digital Library Google Scholar
  6. [6][6] Leon Camus and Anna Filighera. 2020. Investigating Transformers for Automatic Short Answer Grading. In Artificial Intelligence in Education, Ig Ibert Bittencourt, Mutlu Cukurova, Kasia Muldner, Rose Luckin, and Eva Millán (Eds.). Springer International Publishing, Cham, 43–48. Go to Citation Digital Library Google Scholar
  7. [7][7] Dan Carpenter, Wookhee Min, Seung Lee, Gamze Ozogul, Xiaoying Zheng, and James Lester. 2024. Assessing Student Explanations with Large Language Models Using Fine-Tuning and Few-Shot Learning. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), Ekaterina Kochmar, Marie Bexte, Jill Burstein, Andrea Horbach, Ronja Laarmann-Quante, Anaïs Tack, Victoria Yaneva, and Zheng Yuan (Eds.). Association for Computational Linguistics, Mexico City, Mexico, 403–413. https://aclanthology.org/2024.bea-1.33 Go to Citation Google Scholar
  8. [8][8] Dan Carpenter, Wookhee Min, Seung Lee, Gamze Ozogul, Xiaoying Zheng, and James Lester. 2024. Assessing student explanations with large language models using fine-tuning and few-shot learning. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024). 403–413. Google Scholar a [...] examined LLMs for ASAG, Carpenter et al. b [...] also evaluated different prompts for ASAG c [...] did not improve the final result of GPT
  9. [9][9] Imran Chamieh, Torsten Zesch, and Klaus Giebermann. 2024. LLMs in Short Answer Scoring: Limitations and Promise of Zero-Shot and Few-Shot Approaches. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024). 309–315. Google Scholar a [...] factors when evaluating their effectiveness b [...] of these models in educational contexts c [...] The research by Chamieh et al. d [...] also evaluated different prompts for ASAG e [...] to traditional machine learning models
  10. [10][10] Li-Hsin Chang and Filip Ginter. 2024. Automatic Short Answer Grading for Finnish with ChatGPT. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 23173–23181. Go to Citation Digital Library Google Scholar
  11. [12][12] Benjamin Clavié and Kobi Gal. 2019. Edubert: Pretrained deep language models for learning analytics. arXiv preprint arXiv:https://arXiv.org/abs/1912.00690 (2019). Go to Citation Google Scholar · arXiv
  12. [13][13] Aubrey Condor, Max Litster, and Zachary A. Pardos. 2021. Automatic short answer grading with SBERT on out-of-sample questions. In Educational Data Mining. https://api.semanticscholar.org/CorpusID:236880619 Google Scholar a [...] standard answers or specific criteria b [...] of models for ASAG. Condor et al.
  13. [14][14] Emiliano del Gobbo, Alfonso Guarino, Barbara Cafarelli, and Luca Grilli. 2023. GradeAid: a framework for automatic short answers grading in educational contexts—design, implementation and evaluation. Knowledge and Information Systems 65, 10 (2023), 4295––4334. Go to Citation Digital Library Google Scholar
  14. [15][15] Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:https://arXiv.org/abs/1810.04805 (2018). Google Scholar a [...] ] and advanced transformers such as BERT b [...] for a deep understanding of language c [...] classifiers in several NLP tasks d [...] enabling the capture of meaning variations · arXiv
  15. [17][17] Rafael Ferreira, Elyda Freitas, Luciano Cabral, Filipe Dawn, Luiz Rodrigues, Mladen Rakovic, Jackson Raniel, and Dragan Gasevic. 2024. Words of Wisdom: A Journey through the Realm of NLP for Learning Analytics-A Systematic Literature Review. Journal of Learning Analytics (2024), 1–24. Google Scholar a [...] similar meanings Ferreira et al. b [...] the most suitable model for ASAG c [...] to open-ended questions automatically
  16. [19][19] Lucas Busatta Galhardi and Jacques Duílio Brancher. 2018. Machine learning approach for automatic short answer grading: A systematic review. In Advances in Artificial Intelligence-IBERAMIA 2018: 16th Ibero-American Conference on AI, Trujillo, Peru, November 13-16, 2018, Proceedings 16. Springer, 380–391. Go to Citation Digital Library Google Scholar
  17. [20][20] Luyu Gao, Aman Madaan, Shuyan Zhou, Uri Alon, Pengfei Liu, Yiming Yang, Jamie Callan, and Graham Neubig. 2023. Pal: Program-aided language models. In International Conference on Machine Learning. PMLR, 10764–10799. Go to Citation Google Scholar
  18. [21][21] Louie Giray. 2023. Prompt Engineering with ChatGPT: A Guide for Academic Writers. Annals of Biomedical Engineering (2023), 1–5. Go to Citation Google Scholar
  19. [22][22] Luis Gutiérrez and Brian Keith. 2019. A systematic literature review on word embeddings. In Trends and Applications in Software Engineering: Proceedings of the 7th International Conference on Software Process Improvement (CIMPS 2018) 7. Springer, 132–141. Go to Citation Crossref Google Scholar
  20. [23][23] Jochen Hartmann, Juliana Huppertz, Christina Schamp, and Mark Heitmann. 2019. Comparing automated text classification methods. International Journal of Research in Marketing 36, 1 (2019), 20–38. Go to Citation Google Scholar
  21. [25][25] Abdullah İhsanoğlu, Mounes Zaval, and Olcay Taner Yıldız. 2024. Comparison of Machine Learning Algorithms and Large Language Models for Product Categorization. In 2024 32nd Signal Processing and Communications Applications Conference (SIU). IEEE, 1–4. Go to Citation Crossref Google Scholar
  22. [26][26] Shubhra Kanti Karmaker Santu and Dongji Feng. 2023. TELeR: A General Taxonomy of LLM Prompts for Benchmarking Complex Tasks. In Findings of the Association for Computational Linguistics: EMNLP 2023, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 14197–14203. Crossref Google Scholar a [...] of LLMs in specific educational tasks b [...] consistency, and overall effectiveness c [...] responses for accuracy and relevance d [...] Karmaker Santu and Feng e [...] performance through prompt engineering f [...] ], (iii) defining the model’s role g [...] the model to explain its reasoning
  23. [27][27] Faiza Khan Khattak, Serena Jeblee, Chloé Pou-Prom, Mohamed Abdalla, Christopher Meaney, and Frank Rudzicz. 2019. A survey of word embeddings for clinical text. Journal of Biomedical Informatics 100 (2019), 100057. Go to Citation Google Scholar
  24. [28][28] Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. Advances in neural information processing systems 35 (2022), 22199–22213. Google Scholar a [...] the model time to process the information b [...] previous research in prompt engineering
  25. [29][29] Junyoung Lee and Elizabeth Koh. 2023. Teamwork dimensions classification using bert. In International conference on artificial intelligence in education. Springer, 254–259. Go to Citation Crossref Google Scholar
  26. [30][30] Jiazheng Li, Lin Gui, Yuxiang Zhou, David West, Cesare Aloisi, and Yulan He. 2023. Distilling ChatGPT for Explainable Automated Student Answer Assessment. arXiv preprint arXiv:https://arXiv.org/abs/2305.12962 (2023). Go to Citation Google Scholar · arXiv
  27. [31][31] Zongmin Li, Qi Zhang, Yuhong Wang, and Shihang Wang. 2020. Social media rumor refuter feature analysis and crowd identification based on XGBoost and NLP. Applied Sciences 10, 14 (2020), 4711. Go to Citation Google Scholar
  28. [32][32] Joseph P Magliano and Arthur C Graesser. 2012. Computer-based assessment of student-constructed responses. Behavior Research Methods 44 (2012), 608–621. Go to Citation Google Scholar
  29. [33][33] Christopher Manning. 1999. Foundations of statistical natural language processing. The MIT Press. Digital Library Google Scholar a [...] consisting of the weights of the terms b [...] several pre-processing methods were applied
  30. [34][34] Elijah Mayfield and Alan W Black. 2020. Should you fine-tune BERT for automated essay scoring?. In Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications. 151–162. Go to Citation Crossref Google Scholar
  31. [36][36] Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, and Armand Joulin. 2017. Advances in pre-training distributed word representations. arXiv preprint arXiv:https://arXiv.org/abs/1712.09405 (2017). Go to Citation Google Scholar · arXiv
  32. [37][37] Michael Mohler, Razvan Bunescu, and Rada Mihalcea. 2011. Learning to grade short answer questions using semantic similarity measures and dependency graph alignments. In HLT ’11: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies - Volume 1. Association for Computational Linguistics, 752––762. Digital Library Google Scholar a [...] We used two datasets b [...] , created by Mohler et al.
  33. [38][38] Michael Mohler and Rada Mihalcea. 2009. Text-to-text semantic similarity for automatic short answer grading. In Proceedings of the 12th Conference of the European Chapter of the ACL (EACL 2009). 567–575. Crossref Google Scholar a [...] process without compromising quality b [...] standard answers or specific criteria
  34. [39][39] Ben Naismith, Phoebe Mulcaire, and Jill Burstein. 2023. Automated evaluation of written discourse coherence using GPT-4. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023). 394–403. Crossref Google Scholar a [...] In a recent study, Naismith et al. b [...] to traditional machine learning models
  35. [40][40] Huy A Nguyen, Hayden Stec, Xinying Hou, Sarah Di, and Bruce M McLaren. 2023. Evaluating chatgpt’s decimal skills and feedback generation in a digital learning game. In European Conference on Technology Enhanced Learning. Springer, 278–293. Google Scholar a [...] answer reading comprehension questions b [...] contexts. Similarly, Nguyen et al. c [...] to traditional machine learning models
  36. [42][42] Shweta Patil and Krishnakant P Adhiya. 2022. Automated evaluation of short answers: A systematic review. Intelligent Data Communication Technologies and Internet of Things: Proceedings of ICICI 2021 (2022), 953–963. Go to Citation Google Scholar
  37. [43][43] Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 1532–1543. Crossref Google Scholar a [...] ] to the word embedding models like GloVe b [...] the semantic relationships between words
  38. [44][44] Marko Putnikovic and Jelena Jovanovic. 2023. Embeddings for Automatic Short Answer Grading: A Scoping Review. IEEE Transactions on Learning Technologies (2023). Google Scholar a [...] in generating multiple answer options b [...] standard answers or specific criteria c [...] expressed in its presentation and meaning
  39. [45][45] Dadi Ramesh and Suresh Kumar Sanampudi. 2022. An automated essay scoring systems: a systematic literature review. Artificial Intelligence Review 55, 3 (2022), 2495–2527. Google Scholar a [...] ] and specifically for ASAG b [...] are from the actual values on average
  40. [46][46] Chaudhary Jashubhai Rameshbhai and Joy Paulose. 2019. Opinion mining on newspaper headlines using SVM and NLP. International journal of electrical and computer engineering (IJECE) 9, 3 (2019), 2152–2163. Go to Citation Google Scholar
  41. [47][47] Luiz Rodrigues, Filipe Dwan Pereira, Luciano Cabral, Dragan Gašević, Geber Ramalho, and Rafael Ferreira Mello. 2024. Assessing the quality of automatic-generated short answers using GPT-4. Computers and Education: Artificial Intelligence 7 (2024), 100248. Go to Citation Google Scholar
  42. [48][48] Archana Sahu and Plaban Kumar Bhowmick. 2020. Feature Engineering and Ensemble-Based Approach for Improving Automatic Short-Answer Grading Performance. IEEE Transactions on Learning Technologies 13, 1 (2020), 77–90. Crossref Google Scholar a [...] ]. For instance, Sahu and Bhowmick b [...] on the University of North Texas dataset
  43. [49][49] Gerard Salton and Christopher Buckley. 1988. Term-weighting approaches in automatic text retrieval. Information processing & management 24, 5 (1988), 513–523. Go to Citation Google Scholar
  44. [50][50] S Selva Birunda and R Kanniga Devi. 2021. A review on word embedding techniques for text classification. Innovative Data Communication Technologies and Application: Proceedings of ICIDCA 2020 (2021), 267–281. Go to Citation Google Scholar
  45. [51][51] Chamuditha Senanayake and Dinesh Asanka. 2024. Rubric Based Automated Short Answer Scoring using Large Language Models (LLMs). In 2024 International Research Conference on Smart Computing and Systems Engineering (SCSE), Vol. 7. IEEE, 1–6. Crossref Google Scholar a [...] learning models and human evaluations b [...] related to spelling and text structure c [...] to traditional machine learning models
  46. [52][52] Sima Siami-Namini, Neda Tavakoli, and Akbar Siami Namin. 2019. The performance of LSTM and BiLSTM in forecasting time series. In 2019 IEEE International conference on big data (Big Data). IEEE, 3285–3292. Crossref Google Scholar a [...] information for use with current data b [...] enhancing its ability to capture context
  47. [53][53] Chul Sung, Tejas Indulal Dhamecha, and Nirmal Mukhi. 2019. Improving Short Answer Grading Using Transformer-Based Pre-training. In Artificial Intelligence in Education, Seiji Isotani, Eva Millán, Amy Ogan, Peter Hastings, Bruce McLaren, and Rose Luckin (Eds.). Springer International Publishing, Cham, 469–481. Go to Citation Digital Library Google Scholar
  48. [54][54] Neslihan Süzen, Alexander N Gorban, Jeremy Levesley, and Evgeny M Mirkes. 2020. Automatic short answer grading and feedback using text mining methods. Procedia computer science 169 (2020), 726–743. Go to Citation Google Scholar
  49. [55][55] Felipe Urrutia and Roberto Araya. 2024. Who’s the Best Detective? Large Language Models vs. Traditional Machine Learning in Detecting Incoherent Fourth Grade Math Answers. Journal of Educational Computing Research 61, 8 (2024), 187–218. Google Scholar a [...] ]. In this regard, Urrutia and Araya b [...] to traditional machine learning models
  50. [56][56] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems 35 (2022), 24824–24837. Google Scholar a [...] consistency, and overall effectiveness b [...] the performance LLMs across various tasks c [...] For instance, Wei et al. d [...] performance through prompt engineering e [...] (known as the Chain-of-Thought approach)
  51. [57][57] Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, and Douglas C Schmidt. 2023. A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:https://arXiv.org/abs/2302.11382 (2023). Go to Citation Google Scholar · arXiv
  52. [59][59] Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, et al. 2023. A survey of large language models. arXiv preprint arXiv:https://arXiv.org/abs/2303.18223 (2023). Go to Citation Google Scholar · arXiv
  53. [60][60] Araz Zirar. 2023. Exploring the impact of language models, such as ChatGPT, on student learning and assessment. Review of Education 11, 3 (2023). Go to Citation Google Scholar Cited By View all Alam A(2026)Prompt Engineering as Distributed Cognition and Epistemic Infrastructure in Higher EducationPrompt Literacy in Higher Education10.4018/979-8-3373-8332-3.ch004(109-150)Online publication date: 3-Nov-2026 https://doi.org/10.4018/979-8-3373-8332-3.ch004 Prof. Dr. Aniruddha Rumale, Dr. Sushma Zade, Ganesh More Sushant Gholap, Atmanand Mashal, Kalynai Patil (2026)Generative AI for Question Generation, Answer Scoring and Faculty-Supervised AssessmentInternational Journal of Advanced Research in Science Communication and Technology10.48175/IJARSCT-40148(346)Online publication date: 25-Oct-2026 https://doi.org/10.48175/IJARSCT-40148 Domonkos ÁEkler P(2026)Grading Real Coursework with a Local Language ModelComputers10.3390/computers1510071415:10(714)Online publication date: 14-Oct-2026 https://doi.org/10.3390/computers15100714 Show More Cited By Index Terms Index Terms (Premium feature) Automatic Short Answer Grading in the LLM Era: Does GPT-4 with Prompt Engineering beat Traditional Models? Applied computing Education Computing methodologies Artificial intelligence Natural language processing Natural language generation Recommendations Recommendations (Premium feature) Analysis of Grading Times of Short Answer Questions L@S '20: Proceedings of the Seventh ACM Conference on Learning @ Scale We present an analysis of factors correlated to grading speed in short answer questions from college level STEM courses using a novel dataset collected by an online education company. By analyzing timestamp data, we were able to estimate how long ... Read More “I Understand Why I Got This Grade”: Automatic Short Answer Grading (ASAG) with Feedback Artificial Intelligence in Education · DOI

Cited by

Sorted by citations

Authors
Ferreira Mello, R., Pereira Junior, C., Rodrigues, L., Pereira, F. D., Cabral, L., Costa, N., Ramalho, G., & Gasevic, D.
Affiliations
  • Monash University, Melbourne, Australia
  • Instituto Federal Goiano, Iporá, Brazil
  • Universidade Federal de Alagoas (BR)
  • Universidade Federal de Roraima (BR)
  • CJBG-IFPE, NEES-UFAL, CESAR School, Jaboatão dos Guararapes, Brazil
  • Instituto Federal Goiano (BR)
  • Universidade Federal de Pernambuco (BR)
ORCID
Venue
Proceedings of the 15th International Learning Analytics and Knowledge Conference
Publisher
ACM
Published
2025-03-03
Pages
93-103
DOI
10.1145/3706468.3706481
OpenAlex
W4407841980
Cited by
41
References
4
Citations by year
2026: 29 · 2025: 12
FWCI
49.20
Type
conference paper
Language
en
Open access
Licence
Locations
  • International Conference on Learning Analytics and Knowledge (LAK) (publishedVersion, cc-by)
  • Monash University Research Portal (Monash University) (submittedVersion, cc-by)
  • HAL (submittedVersion)
Funders
  • Conselho Nacional de Desenvolvimento Científico e Tecnológico (310888/2021- 2)
  • OpenAI research Program (1)
Topics
  • Physical Sciences › Computer Science › Information Systems › Educational Technology and Assessment
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
Keywords
automatic short answer grading, GPT-4, prompt engineering, large language models, few-shot learning, text representation, automated assessment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review