Embracing LLM Feedback: the role of feedback providers and provider information for feedback effectiveness
Open accessThe DOI matches its Crossref record; 49 of the 49 references it cites were found in a registry or an index.
Abstract
Feedback is an integral part of learning in higher education and is increasingly being provided to students via modern technologies like Large Language Models (LLMs). But students’ perception of feedback from LLMs vs. feedback from educators remains unclear even though it is an important facet of feedback effectiveness. Further, feedback effectiveness can be negatively influenced by various factors; For example, (not) knowing certain characteristics about the feedback provider may bias a student’s reaction to the feedback process. To assess perceptions of LLM feedback and mitigate the negative effects of possible biases, this study investigated the potential of providing provider-information about feedback providers. In a 2×2 between-subjects design with the factors feedback provider (LLM vs. educator) and provider-information (yes vs. no), 169 German students evaluated feedback message and provider perceptions. Path analyses showed that the LLM was perceived as more trustworthy than an educator and that the provision of provider-information led to improved perceptions of the feedback. Furthermore, the effect of the provider and the feedback on perceived trustworthiness and fairness changed when provider-information was provided. Overall, our study highlights the importance of further research on feedback processes that include LLMs due to their influential nature and suggests practical recommendations for designing digital feedback processes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Hattie, J., & Timperley, H. (2007) · Review of Educational Research · doi:10.3102/003465430298487 · Cited by 12,905
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-019-0171-0 · Cited by 7,242
Humans and automation: Use, misuse, disuse, abuse
Parasuraman, R., & Riley, V. (1997) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/001872097778543886 · Cited by 4,554
What is AI Literacy? Competencies and Design Considerations
Long, D., & Magerko, B. (2020) · Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376727 · Cited by 3,241
Trust in Automation: Integrating Empirical Evidence on Factors That Influence Trust
Hoff, K. A., & Bashir, M. (2015) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720814547570 · Cited by 2,817
The development of student feedback literacy: Enabling uptake of feedback
Carless, D., & Boud, D. (2018) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2018.1463354 · Cited by 2,428
Conceptualizing AI literacy: An exploratory review
Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100041 · Cited by 2,205
Consequences of individual feedback on behavior in organizations.
Ilgen, D. R., Fisher, C. D., & Taylor, M. S. (1979) · Journal of Applied Psychology · doi:10.1037/0021-9010.64.4.349 · Cited by 1,822
Rethinking models of feedback for learning: the challenge of design
Boud, D., & Molloy, E. (2013) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2012.691462 · Cited by 1,636
Ethics of AI in Education: Towards a Community-Wide Framework
Holmes, W., Porayska-Pomsta, K., Holstein, K., Sutherland, E., Baker, T., Shum, S. B., Santos, O. C., Rodrigo, M. T., Cukurova, M., Bittencourt, I. I., et al. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00239-1 · Cited by 1,437
The Power of Feedback Revisited: A Meta-Analysis of Educational Feedback Research
Wisniewski, B., Zierer, K., & Hattie, J. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.03087 · Cited by 1,199
Chiu, T. K. F., Xia, Q., Zhou, X., Chai, C. S., & Cheng, M. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100118 · Cited by 1,181
Differing perceptions in the feedback process
Carless, D. (2006) · Studies in Higher Education · doi:10.1080/03075070600572132 · Cited by 1,102
Grassini, S. (2023) · Education Sciences · doi:10.3390/educsci13070692 · Cited by 1,048
Winstone, N. E., Nash, R. A., Parker, M., & Rowntree, J. (2017) · Educational Psychologist · doi:10.1080/00461520.2016.1207538 · Cited by 884
Metzger, M. J. (2007) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20672 · Cited by 871
Explainable Artificial Intelligence in education
Khosravi, H., Shum, S. B., Chen, G., Conati, C., Tsai, Y.-S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gašević, D. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100074 · Cited by 863
Strzelecki, A. (2024) · Interactive Learning Environments · doi:10.1080/10494820.2023.2209881 · Cited by 807
AI literacy in K-12: a systematic literature review
Casal-Otero, L., Catala, A., Fernández-Morante, C., Taboada, M., Cebreiro, B., & Barro, S. (2023) · International Journal of STEM Education · doi:10.1186/s40594-023-00418-7 · Cited by 724
Trustworthy Artificial Intelligence: A Review
Kaur, D., Uslu, S., Rittichier, K. J., & Durresi, A. (2023) · ACM Computing Surveys · doi:10.1145/3491209 · Cited by 709
Similarities and differences between human–human and human–automation trust: an integrative review
Madhavan, P., & Wiegmann, D. A. (2007) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220500337708 · Cited by 577
Alqahtani, T., Badreldin, H. A., Alrashed, M., Alshaya, A. I., Alghamdi, S. S., bin Saleh, K., Alowais, S. A., Alshaya, O. A., Rahman, I., Al Yami, M. S., et al. (2023) · Research in Social and Administrative Pharmacy · doi:10.1016/j.sapharm.2023.05.016 · Cited by 565
Memarian, B., & Doleck, T. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100152 · Cited by 416
Assessment in the age of artificial intelligence
Swiecki, Z., Khosravi, H., Chen, G., Martinez-Maldonado, R., Lodge, J. M., Milligan, S., Selwyn, N., & Gašević, D. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100075 · Cited by 409
Trust in Artificial Intelligence: Meta-Analytic Findings
Kaplan, A. D., Kessler, T. T., Brill, J. C., & Hancock, P. A. (2023) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1177/00187208211013988 · Cited by 397
Can Large Language Models Provide Feedback to Students? A Case Study on ChatGPT
Dai, W., Lin, J., Jin, H., Li, T., Tsai, Y.-S., Gašević, D., & Chen, G. (2023) · 2023 IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt58122.2023.00100 · Cited by 395
Can Large Language Models Provide Feedback to Students? A Case Study on ChatGPT
Dai, W., Lin, J., Jin, F., Li, T., Tsai, Y.-S., Gasevic, D., & Chen, G. (2023) · doi:10.35542/osf.io/hcgzj · Cited by 395
A review of feedback models and typologies: Towards an integrative model of feedback elements
Panadero, E., & Lipnevich, A. A. (2022) · Educational Research Review · doi:10.1016/j.edurev.2021.100416 · Cited by 389
Hendriks, F., Kienhues, D., & Bromme, R. (2015) · PLOS ONE · doi:10.1371/journal.pone.0139309 · Cited by 373
Chang, D. H., Lin, M. P.-C., Hajian, S., & Wang, Q. Q. (2023) · Sustainability · doi:10.3390/su151712921 · Cited by 370
A Review of Feedback Models and Theories: Descriptions, Definitions, and Conclusions
Lipnevich, A. A., & Panadero, E. (2021) · Frontiers in Education · doi:10.3389/feduc.2021.720195 · Cited by 368
The challenges of feedback in higher education
Henderson, M., Ryan, T., & Phillips, M. (2019) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2019.1599815 · Cited by 335
A Meta-Analysis of the Effects of Feedback in Computer-Based Instruction
Azevedo, R., & Bernard, R. M. (1995) · Journal of Educational Computing Research · doi:10.2190/9lmd-3u28-3a0g-ftqt · Cited by 308
Panadero, E., & Alqassab, M. (2019) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2019.1600186 · Cited by 232
Bozkurt, A. (2023) · Zenodo · doi:10.5281/zenodo.7716416 · Cited by 224
Understanding User Trust in Artificial Intelligence-Based Educational Systems: Evidence from China
Qin, F., Li, K., & Yan, J. (2020) · British Journal of Educational Technology · doi:10.1111/bjet.12994 · Cited by 204
Student Perceptions of Assessment Feedback: A Critical Scoping Review and Call for Research
Van der Kleij, F. M., & Lipnevich, A. A. (2021) · Educational Assessment, Evaluation and Accountability · doi:10.1007/s11092-020-09331-x · Cited by 189
Factors Influencing Learner Attitudes Towards ChatGPT-Assisted Language Learning in Higher Education
Cai, Q., Lin, Y., & Yu, Z. (2024) · International Journal of Human–Computer Interaction · doi:10.1080/10447318.2023.2261725 · Cited by 180
An Instrument for Measuring Teachers’ Trust in AI-Based Educational Technology
Nazaretsky, T., Cukurova, M., & Alexandron, G. (2022) · LAK22: 12th International Learning Analytics and Knowledge Conference · doi:10.1145/3506860.3506866 · Cited by 158
AcaWriter: A learning analytics tool for formative feedback on academic writing
Knight, S., Shibani, A., Abel, S., Gibson, A., & Ryan, P. (2020) · Journal of Writing Research · doi:10.17239/jowr-2020.12.01.06 · Cited by 151
Automated feedback and writing: a multi-level meta-analysis of effects on students' performance
Fleckenstein, J., Liebenow, L. W., & Meyer, J. (2023) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2023.1162454 · Cited by 145
Explainable Artificial Intelligence: An Introduction to Interpretable Machine Learning
Kamath, U., & Liu, J. (2021) · doi:10.1007/978-3-030-83356-5 · Cited by 126
Toward a Model of Student Response to Feedback
(2016) · Dialnet (Universidad de la Rioja) · doi:10.4324/9781315749136.ch10 · Cited by 122
Wilson, J., Ahrendt, C., Fudge, E. A., Raiche, A., Beard, G., & MacArthur, C. (2021) · Computers & Education · doi:10.1016/j.compedu.2021.104208 · Cited by 118
Flanagin, A. J., Winter, S., & Metzger, M. J. (2020) · Information, Communication & Society · doi:10.1080/1369118x.2018.1547411 · Cited by 116
Langer, M., Hunsicker, T., Feldkamp, T., König, C. J., & Grgić-Hlača, N. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517527 · Cited by 55
Cited by
Brügge, E., Ricchizzi, S., Arenbeck, M., Keller, M. N., Schur, L., Stummer, W., Holling, M., Lu, M. H., & Darici, D. (2024) · BMC Medical Education · doi:10.1186/s12909-024-06399-7 · Cited by 137
Teacher, peer, or AI? Comparing effects of feedback sources in higher education
Weidlich, J., Gotsch, F., Schudel, K., Marusic-Würscher, C., Mazzarella, J., Bolten, H., Bütler, D., Luger, S., Wohlfender, B., & Merki, K. M. (2025) · Computers and Education Open · doi:10.1016/j.caeo.2025.100300 · Cited by 36
Who Gives Feedback Matters: Student Biases Towards Human and AI ‐Generated Formative Feedback
Nazaretsky, T., Mejia‐Domenzain, P., Swamy, V., Frej, J., & Käser, T. (2026) · Journal of Computer Assisted Learning · doi:10.1111/jcal.70153 · Cited by 26
Jovic, M., Papakonstantinidis, S., & Kirkpatrick, R. (2025) · Language Testing in Asia · doi:10.1186/s40468-025-00389-2 · Cited by 11
Evaluating Trust in AI, Human, and Co-produced Feedback Among Undergraduate Students
Zhang, A., Gao, Y., Suraworachet, W., Nazaretsky, T., & Cukurova, M. (2025) · arXiv · doi:10.48550/arxiv.2504.10961 · Cited by 7
Metadata
- Authors
- Ruwe, T., & Mayweg-Paus, E.
- Venue
- Frontiers in Education
- Publisher
- Frontiers Media SA
- Published
- 2024-10-16
- Volume
- 9
- ISSN
- 2504-284X (ISSN-L)
- OpenAlex
- W4403476385
- Cited by
- 33
- References
- 49
- FWCI
- 42.33
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.