Open Source Language Models Can Provide Feedback: Evaluating LLMs' Ability to Help Students Using GPT-4-As-A-Judge
Open accessThe DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.
Abstract
Large language models (LLMs) have shown great potential for the automatic generation of feedback in a wide range of computing contexts. However, concerns have been voiced around the privacy and ethical implications of sending student work to proprietary models. This has sparked considerable interest in the use of open source LLMs in education, but the quality of the feedback that such open models can produce remains understudied. This is a concern as providing flawed or misleading generated feedback could be detrimental to student learning. Inspired by recent work that has utilised very powerful LLMs, such as GPT-4, to evaluate the outputs produced by less powerful models, we conduct an automated analysis of the quality of the feedback produced by several open source models using a dataset from an introductory programming course. First, we investigate the viability of employing GPT-4 as an automated evaluator by comparing its evaluations with those of a human expert. We observe that GPT-4 demonstrates a bias toward positively rating feedback while exhibiting moderate agreement with human raters, showcasing its potential as a feedback evaluator. Second, we explore the quality of feedback generated by several leading open-source LLMs by using GPT-4 to evaluate the feedback. We find that some models offer competitive performance with popular proprietary LLMs, such as ChatGPT, indicating opportunities for their responsible use in educational settings.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741
LLaMA: Open and Efficient Foundation Language Models
Touvron, H., Lavril, T., Izacard, G., Martinet, X., Lachaux, M.-A., Lacroix, T., Rozière, B., Goyal, N., Hambro, E., Azhar, F., et al. (2023) · arXiv · doi:10.48550/arxiv.2302.13971 · Cited by 4,602
HuggingFace's Transformers: State-of-the-art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.03771 · Cited by 3,347
Direct Preference Optimization: Your Language Model is Secretly a Reward Model
Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., & Finn, C. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2338 · Cited by 686
The Robots Are Here: Navigating the Generative AI Revolution in Computing Education
Prather, J., Denny, P., Leinonen, J., Becker, B. A., Albluwi, I., Craig, M., Keuning, H., Kiesler, N., Kohn, T., Luxton-Reilly, A., et al. (2023) · Proceedings of the 2023 Working Group Reports on Innovation and Technology in Computer Science Education · doi:10.1145/3623762.3633499 · Cited by 396 · cited 1×
“Indeed, there has been great interest in generating automated feedback using language models in the past year [1, 6, 14, 19, 20, 26, 28, 29, 31].”
BackgroundSelf-citationA systematic literature review of automated feedback generation for programming exercises
Keuning, H., Jeuring, J., & Heeren, B. (2019) · ACM Transactions on Computing Education · doi:10.1145/3231711 · Cited by 370 · cited 4×
“Generating automated feedback is one of the long-standing challenges in computing education and has intrigued researchers for decades [13].”
BackgroundInfluentialStudying the effect of AI Code Generators on Supporting Novice Learners in Introductory Programming
Kazemitabaar, M., Chow, J., Ma, C. K. T., Ericson, B. J., Weintrop, D., & Grossman, T. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580919 · Cited by 366
Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., Casas, D. D. L., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., et al. (2023) · arXiv · doi:10.48550/arxiv.2310.06825 · Cited by 340
Computing Education in the Era of Generative AI
Denny, P., Prather, J., Becker, B. A., Finnie-Ansley, J., Hellas, A., Leinonen, J., Luxton-Reilly, A., Reeves, B. N., Santos, E. A., & Sarsa, S. (2024) · Communications of the ACM · doi:10.1145/3624720 · Cited by 316
Self-citationDenny, P., Kumar, V., & Giacaman, N. (2023) · Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3545945.3569823 · Cited by 287
Self-citationLIMA: Less Is More for Alignment
Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L., et al. (2023) · Advances in Neural Information Processing Systems 36 · doi:10.52202/075280-2400 · Cited by 275 · cited 1×
“While recent work [32, 45] has already used GPT-4 as part of their evaluation strategies, also in educational settings [8, 21], the question of how effective LLMs (or in particular GPT-4) can be in assessing the quality of the feedback generated by other LLMs has not been studied explicitly.”
BackgroundLIMA: Less Is More for Alignment
Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.11206 · Cited by 275
Using Large Language Models to Enhance Programming Error Messages
Leinonen, J., Hellas, A., Sarsa, S., Reeves, B., Denny, P., Prather, J., & Becker, B. A. (2023) · Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3545945.3569770 · Cited by 235
Self-citationHuang, F., Kwak, H., & An, J. (2023) · arXiv · doi:10.48550/arxiv.2302.07736 · Cited by 231
Exploring the Responses of Large Language Models to Beginner Programmers' Help Requests
Hellas, A., Leinonen, J., Sarsa, S., Koutcheme, C., Kujanpää, L., & Sorva, J. (2023) · arXiv · doi:10.48550/arxiv.2306.05715 · Cited by 211
Self-citationCodeHelp: Using Large Language Models with Guardrails for Scalable Support in Programming Classes
Liffiton, M., Sheese, B. E., Savelka, J., & Denny, P. (2023) · Proceedings of the 23rd Koli Calling International Conference on Computing Education Research · doi:10.1145/3631802.3631830 · Cited by 192
Self-citationPrompt Problems: A New Programming Exercise for the Generative AI Era
Denny, P., Leinonen, J., Prather, J., Luxton-Reilly, A., Amarouche, T., Becker, B. A., & Reeves, B. N. (2024) · Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3626252.3630909 · Cited by 187 · cited 1×
“Although there will always be situations where students choose to use closed source models to seek feedback or answer questions, a wide range of new educational tools powered by LLMs are being developed and integrated into computing classrooms [4, 11, 20].”
BackgroundSelf-citationExploring the Responses of Large Language Models to Beginner Programmers’ Help Requests
Hellas, A., Leinonen, J., Sarsa, S., Koutcheme, C., Kujanpää, L., & Sorva, J. (2023) · Proceedings of the 2023 ACM Conference on International Computing Education Research V.1 · doi:10.1145/3568813.3600139 · Cited by 145
Self-citationMeasuring Coding Challenge Competence With APPS
Hendrycks, D., Basart, S., Kadavath, S., Mazeika, M., Arora, A., Guo, E., Burns, C., Puranik, S., He, H., Song, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2105.09938 · Cited by 142 · cited 1×
“We chose these models because of their extensive documentation, community adoption, respective performance on code and language reasoning benchmarks (HumanEval [2] and MMLU [7]) and ability to follow instructions.”
MethodSavelka, J., Agarwal, A., An, M., Bogart, C., & Sakr, M. (2023) · Proceedings of the 2023 ACM Conference on International Computing Education Research V.1 · doi:10.1145/3568813.3600142 · Cited by 141 · cited 1×
“Following prior work [6, 36], we generate a single feedback using greedy decoding (i.e., selecting the token with the highest probability as the next element in the sequence).”
Method(2023) · doi:10.1109/sp46215.2023.10179324 · Cited by 131
Investigating the Potential of GPT-3 in Providing Feedback for Programming Assessments
Balse, R., Valaboju, B., Singhal, S., Warriem, J. M., & Prasad, P. (2023) · Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 · doi:10.1145/3587102.3588852 · Cited by 63
PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization
Wang, Y., Yu, Z., Zeng, Z., Yang, L., Wang, C., Chen, H., Jiang, C., Xie, R., Wang, J., Xie, X., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05087 · Cited by 34 · cited 2×
“However, the proprietary nature of the state-of-the-art LLM has led to the rise of judges based on open-source LLMs. PandaLM [40] is an open-source LLM that has been fine-tuned for selecting the best response to an (instruction, input) pair and providing an explanation together with its decision.”
BackgroundGenerating High-Precision Feedback for Programming Syntax Errors using Large Language Models
Phung, T., Cambronero, J., Gulwani, S., Kohn, T., Majumdar, R., Singla, A., & Soares, G. (2023) · arXiv · doi:10.48550/arxiv.2302.04662 · Cited by 16 · cited 2×
“This is in line with the findings of Leinonen et al. [19] and Phung et al. [28] who both found that LLMs could be used to give feedback on syntax errors.”
BackgroundResult
Cited by
Qian, K., Liu, S., Li, T., Raković, M., Li, X., Guan, R., Molenaar, I., Nawaz, S., Swiecki, Z., Yan, L., et al. (2026) · Computers & Education · doi:10.1016/j.compedu.2025.105448 · Cited by 39
“…over 80% agreement—comparable to inter-human agreement levels, and it has also been applied in evaluating educational feedback (Zheng et al., 2024; Koutcheme et al., 2024; Huang et al., 2024; Weyssow et al., 2024; Zhang et al., 2023b; Liu et al., 2024a), highlighting its potential of offering…”
Prather, J., Reeves, B. N., Denny, P., Leinonen, J., MacNeil, S., Luxton-Reilly, A., Orvalho, J., Alipour, A., Alfageeh, A., Amarouche, T., et al. (2024) · arXiv · doi:10.48550/arxiv.2412.12800 · Cited by 20
Self-citationNavigating the Ethical and Societal Impacts of Generative AI in Higher Computing Education
Mak, J., Nakatumba-Nabende, J., Clear, T., Clear, A., Abluwi, I., Andrei, O., Angeli, L., MacNeil, S., Oyelere, S. S., Hale Rattigan, M., et al. (2025) · Proceedings of the 2025 Working Group Reports on Innovation and Technology in Computer Science Education · doi:10.1145/3760545.3783975 · Cited by 12
Ali, M., Ma, M., Muneeb, M., & Wong, G. K. W. (2025) · Metrics · doi:10.3390/metrics2040023 · Cited by 6
Mapeamento Sistemático sobre o Uso de IAs Generativas no Ensino de Programação
Ferreira de Sousa, J., & Silveira, F. R. D. V. (2026) · Revista Eletrônica de Iniciação Científica em Computação · doi:10.5753/reic.2026.7169
Metadata
- Authors
- Koutcheme, C., Dainese, N., Sarsa, S., Hellas, A., Leinonen, J., & Denny, P.
- Venue
- Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1
- Publisher
- ACM
- Published
- 2024-07-03
- Pages
- 52-58
- OpenAlex
- W4400267611
- Cited by
- 47
- References
- 24
- FWCI
- 18.47
- Type
- conference paper
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.