The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
On the Application of Sentence Transformers to Automatic Short Answer Grading in Blended Assessment
Open accessAbstract
In Natural Language Processing, automatic short answer grading remains a necessary launch-pad for the analysis of human responses in a blended learning setting. This study presents pre-trained neural language models that use context dependent Sentence-Transformers to automatically grade student responses with two different input settings. It is found that the use of these models achieves promising results when compared to conventional Bidirectional Encoder Representation Transformer, (BERT), approaches when applying various text similarity-based tasks. This work presents experiments using the benchmark Mohler dataset to test these new models. In summary, an excellent Pearson Correlation score of 0.82 and a Root Mean Square Error of 0.69 is exhibited across a representative experiment sample size.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Likewise, LLMs-based innovations' performances on automatic short-answer grading were also highly correlated with human ratings (Pearson's correlation between 0.75 to 0.82) (Ahmed et al., 2022;Sawatzki et al., 2022).”
From Practical and ethical challenges of large language models in education: A systematic scoping review (2024) · cites this paper as Ahmed et al., 2022
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Research works printed but not linked: 33
- 10.1007/s40593-014-0026-8 · DOI
- 10.3115/1609067.1609130 · DOI
- Mohler, Learning to grade short answer questions using semantic similarity measures and dependency graph alignments, 752
- Mikolov, 2013, Efficient estimation of word representations in vector space, arXiv preprint
- 10.3115/v1/D14-1162 · DOI · Paper page
- 10.18653/v1/N16-1123 · DOI
- Roy, Wisdom of students: A consistent automatic short answer grading technique, 178
- Magooda, Vector based techniques for short answer grading
- 10.1109/ACCESS.2021.3054346 · DOI
- 10.1007/978-3-319-93843-1_37 · DOI
- Surya, 2019, Deep learning for short answer scoring, Int. J. Recent. Technol. Eng.(IJRTE), 7
- Vaswani, 2017, Attention is all you need, arXiv preprint
- 10.18653/v1/D19-1410 · DOI · Paper page
- 10.29007/132n · DOI
- 10.48550/ARXIV.1907.11692 · DOI · Paper page
- Radford, 2018, Improving language understanding by generative pre-training
- 10.14569/ijacsa.2012.031119 · DOI
- 10.3115/v1/W15-0612 · DOI
- 10.18653/v1/W17-5017 · DOI
- 10.24963/ijcai.2017/284 · DOI
- 10.5121/cseij.2012.2107 · DOI
- 10.1109/tale48000.2019.9226026 · DOI
- 10.14569/ijacsa.2018.091048 · DOI
- 10.18653/v1/D19-1628 · DOI
- 10.1007/978-3-030-23204-7_39 · DOI
- 10.1007/978-3-030-52240-7_8 · DOI
- Sasi, 2020, Comparative Evaluation of Pretrained Transfer Learning Models on Automatic Short Answer Grading, arXiv pre-print server, 2020-09-02
- 10.5220/0009422400580067 · DOI
- 10.1007/978-3-030-52240-7_14 · DOI
- 10.1007/978-3-030-14118-9_59 · DOI
- 10.1007/978-3-030-52240-7_41 · DOI
- Condor, 2021, Automatic Short Answer Grading with SBERT on Out-of-Sample Questions
- 10.1109/ACCESS.2021.3054346 · DOI
Cited by
Sorted by citations
Practical and ethical challenges of large language models in education: A systematic scoping review
Yan, L., Sha, L., Zhao, L., Li, Y., Martinez‐Maldonado, R., Chen, G., Li, X., Jin, Y., & Gašević, D. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13370 · Cited by 1,435
“Likewise, LLMs-based innovations’ performances on automatic short-answer grading were also highly correlated with human ratings (Pearson’s correlation between 0.75 to 0.82) (Ahmed et al., 2022; Sawatzki et al., 2022).”
BackgroundInfluential
Metadata
- Authors
- Ahmed, A., Joorabchi, A., & Hayes, M. J.
- Affiliations
- University of Limerick (IE)
- Venue
- 2022 33rd Irish Signals and Systems Conference (ISSC)
- Publisher
- IEEE
- Published
- 2022-06-09
- Pages
- 1-6
- OpenAlex
- W4286282840
- Cited by
- 13
- References
- 3
- Citations by year
- 2026: 2 · 2025: 5 · 2024: 4 · 2023: 2
- FWCI
- 1.52
- Type
- conference paper
- Language
- en
- Open access
- green
- hdl.handle.net
- Locations
- Irish Signals and Systems Conference (publishedVersion)
- University of Limerick Institutional Repository (University of Limerick) (submittedVersion, cc-by-nc-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Physical Sciences › Computer Science › Information Systems › Educational Technology and Assessment
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Keywords
- automatic short answer grading, Sentence Transformers, student response evaluation, text similarity, BERT comparison
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
