Student reactions to AI versus human feedback in teamwork skills assessment
Open accessAbstract
Abstract As AI technologies become increasingly integrated into education, this research investigates how students react to AI-generated versus human feedback in teamwork skills assessment. In Study 1, 108 students completed a virtual teamwork simulation and received assessment feedback framed as either AI- or human-generated. Students showed a clear preference for human feedback over AI feedback, revealing a bias against machine-generated evaluations. Study 2, a scenario-based experiment involving 322 students, confirmed these findings and tested whether enhancing AI feedback with credibility and empathy cues could improve perceptions. These enhancements significantly improved reactions to AI feedback, and when both credibility and empathy were emphasized, reactions approached those for unenhanced human feedback. However, even with enhancements, AI feedback did not fully match the positive perceptions of human feedback. These findings highlight the need for thoughtful design to mitigate biases against AI feedback and suggest that blending AI and human feedback offers an effective approach for improving acceptance and engagement in educational contexts.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Learners' perceptions of the feedback source directly influence its acceptance and learning outcomes.
“Kotlyar and Krasman [60] indicate that learners’ perceptions of the feedback source directly influence its acceptance and learning outcomes.”
From AI-Powered Simulation for Nursing Education: Mixed Methods Systematic Review (2026) · cites this paper as [60]
The H2b section says perceived credibility and empathy of the feedback source affect feedback acceptance and that learning effectiveness depends in part on those perceptions. The Discussion: study 2 reports that credibility and empathy shaped reactions to feedback, but the Limitations and future directions section says the study captured initial reactions and calls for future research on learning outcomes. The claim’s assertion of a direct influence on learning outcomes is therefore stronger than the paper’s findings. (LLM, conf=0.84)
Experimental — accuracy not yet measuredIn learning tasks intensive in emotion and interpersonal interaction, learners tend to trust human feedback sources perceived as possessing benevolence and integrity over AI systems perceived as merely competent.
“In learning tasks intensive in emotion and interpersonal interaction, learners tend to trust human feedback sources perceived as possessing “benevolence” and “integrity” over AI systems perceived as merely “competent.””
From AI-Powered Simulation for Nursing Education: Mixed Methods Systematic Review (2026) · cites this paper as [60]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. E., & Cohen, J. (1989) · Technometrics · doi:10.2307/1270020 · Cited by 99,588 · see citing works
Perceived usefulness, perceived ease of use, and user acceptance of information technology
Davis, F. D. (1989) · MIS Quarterly · doi:10.2307/249008 · Cited by 68,483 · see citing works
Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007) · Behavior Research Methods · doi:10.3758/bf03193146 · Cited by 65,971 · see citing works
Statistical power analyses using G∗Power 3.1: tests for correlation and regression analyses
Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009) · Behavior Research Methods · doi:10.3758/brm.41.4.1149 · Cited by 37,615 · see citing works
The "what" and "why" of goal pursuits: Human needs and the self-determination of behavior
Deci, E. L., & Ryan, R. M. (2000) · Psychological Inquiry · doi:10.1207/s15327965pli1104_01 · Cited by 34,210 · cited 1× · see citing works
“…likely to form a positive connection with the feedback provider (Amer-storfer & Freiin von Munster-Kistner, 2021; Winstone et al., 2017b; Zhang et al., 2024, p. 5), which in turn enhances motivation, engagement, and openness to learning (Deci & Ryan, 2000; Gettinger & Walter, 2013; Shute, 2008).”
Venkatesh, V., Thong, J. Y. L., & Xu, X. (2012) · MIS Quarterly · doi:10.2307/41410412 · Cited by 15,756 · see citing works
Hattie, J., & Timperley, H. (2007) · Review of Educational Research · doi:10.3102/003465430298487 · Cited by 12,905 · see citing works
The Social psychology of intergroup relations
Austin, W. G., & Worchel, S. (1979) · Medical Entomology and Zoology · Cited by 11,577 · see citing works
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736 · see citing works
Psychology of intergroup relations
Austin, W. G., & Worchel, S. (1986) · Medical Entomology and Zoology · Cited by 10,626 · see citing works
The role of deliberate practice in the acquisition of expert performance
Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993) · Psychological Review · doi:10.1037/0033-295x.100.3.363 · Cited by 9,564 · see citing works
Assessment and Classroom Learning
Black, P., & Wiliam, D. (1998) · Assessment in Education: Principles, Policy & Practice · doi:10.1080/0969595980050102 · Cited by 8,152 · see citing works
Social Identity Theory and the Organization
Ashforth, B. E., & Mael, F. A. (1989) · Academy of Management Review · doi:10.5465/amr.1989.4278999 · Cited by 7,978 · see citing works
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-019-0171-0 · Cited by 7,242 · see citing works
Trust in Automation: Designing for Appropriate Reliance
Lee, J. D., & See, K. A. (2004) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/hfes.46.1.50_30392 · Cited by 6,605 · cited 3× · see citing works
“The findings highlight the importance of perceived credibility and empathy in shaping feedback acceptance, building on existing models of trust calibration (Lee & See, 2004) and feedback recipience (Winstone et al., 2017a).”
Kluger, A. N., & DeNisi, A. (1996) · Psychological Bulletin · doi:10.1037/0033-2909.119.2.254 · Cited by 5,882 · see citing works
Shute, V. J. (2008) · Review of Educational Research · doi:10.3102/0034654307313795 · Cited by 4,631 · see citing works
The social psychology of telecommunications
Gray, G. (1977) · Telecommunications Policy · doi:10.1016/0308-5961(77)90016-7 · Cited by 3,486 · see citing works
Feedback and Self-Regulated Learning: A Theoretical Synthesis
Butler, D. L., & Winne, P. H. (1995) · Review of Educational Research · doi:10.3102/00346543065003245 · Cited by 3,334 · see citing works
Algorithm aversion: People erroneously avoid algorithms after seeing them err.
Dietvorst, B. J., Simmons, J. P., & Massey, C. (2015) · Journal of Experimental Psychology: General · doi:10.1037/xge0000033 · Cited by 3,232 · cited 3× · see citing works
“This is consistent with algorithm aversion (Dietvorst et al., 2015), which reflects users’ reluctance to trust algorithmic judgments even when they are accurate and may be exacerbated by perceived lack of social presence, emotional warmth, or perceived authenticity in AI communication (Dzindolet et…”
InfluentialHuman trust in artificial intelligence: Review of empirical research
Glikson, E., & Woolley, A. W. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0057 · Cited by 2,921 · see citing works
Consequences of individual feedback on behavior in organizations.
Ilgen, D. R., Fisher, C. D., & Taylor, M. S. (1979) · Journal of Applied Psychology · doi:10.1037/0021-9010.64.4.349 · Cited by 1,822 · see citing works
Rethinking models of feedback for learning: the challenge of design
Boud, D., & Molloy, E. (2013) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2012.691462 · Cited by 1,636 · cited 2× · see citing works
“…well-established in educational research as core components of effective teaching and have been linked to greater student engagement and feedback acceptance (Amerstorfer & Freiin von Munster-Kistner, 2021; Boud & Molloy, 2013; Davis et al., 2012; Pishghadam & Karami, 2017; Pishghadam et al., 2018).”
A Unified Framework of Five Principles for AI in Society
Floridi, L., & Cowls, J. (2019) · Harvard Data Science Review · doi:10.1162/99608f92.8cd550d1 · Cited by 1,631 · cited 1× · see citing works
“Learning that the feedback was AI-generated can lead to confusion, distrust, or even reactance (Floridi & Cowls, 2019).”
A SWOT analysis of ChatGPT: Implications for educational practice and research
Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2195846 · Cited by 1,384 · see citing works
Making Sense of Assessment Feedback in Higher Education
Evans, C. (2013) · Review of Educational Research · doi:10.3102/0034654312474350 · Cited by 1,143 · see citing works
Historical threads, missing links, and future directions in AI in education
Williamson, B., & Eynon, R. (2020) · Learning, Media and Technology · doi:10.1080/17439884.2020.1798995 · Cited by 981 · see citing works
Winstone, N. E., Nash, R. A., Parker, M., & Rowntree, J. (2017) · Educational Psychologist · doi:10.1080/00461520.2016.1207538 · Cited by 884 · cited 2× · see citing works
“Winstone et al. (2017a) noted that feedback effectiveness depends not only on its content but also on how recipients perceive and engage with it.”
Applicant Reactions to Selection Procedures: An Updated Model and Meta‐Analysis
Hausknecht, J. P., Day, D. V., & Thomas, S. C. (2004) · Personnel Psychology · doi:10.1111/j.1744-6570.2004.00003.x · Cited by 816 · see citing works
What is feedback in clinical education?
Van De Ridder, J. M. M., Stokking, K. M., McGaghie, W. C., & Ten Cate, O. T. J. (2008) · Medical Education · doi:10.1111/j.1365-2923.2007.02973.x · Cited by 686 · see citing works
The teacher‐student relationship as an interpersonal relationship
Frymier, A. B., & Houser, M. L. (2000) · Communication Education · doi:10.1080/03634520009379209 · Cited by 636 · see citing works
AI-generated feedback on writing: insights into efficacy and ENL student preference
Escalante, J., Pack, A., & Barrett, A. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00425-2 · Cited by 589 · see citing works
Similarities and differences between human–human and human–automation trust: an integrative review
Madhavan, P., & Wiegmann, D. A. (2007) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220500337708 · Cited by 577 · see citing works
Tong, S., Jia, N., Luo, X., & Fang, Z. (2021) · Strategic Management Journal · doi:10.1002/smj.3322 · Cited by 531 · cited 3× · see citing works
“Applicants and employees often view AI evaluations in selection and assessment as less fair or trustworthy than human evaluations, even when AI is more objective and accurate (Suen et al., 2019; Tong et al., 2021).”
Comparing the quality of human and ChatGPT feedback of students’ writing
Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J., Moon, Y., Tseng, W., Warschauer, M., & Olson, C. B. (2024) · Learning and Instruction · doi:10.1016/j.learninstruc.2024.101894 · Cited by 509 · see citing works
Winstone, N. E., Nash, R. A., Rowntree, J., & Parker, M. (2017) · Studies in Higher Education · doi:10.1080/03075079.2015.1130032 · Cited by 417 · cited 2× · see citing works
“…and support, students are more likely to form a positive connection with the feedback provider (Amer-storfer & Freiin von Munster-Kistner, 2021; Winstone et al., 2017b; Zhang et al., 2024, p. 5), which in turn enhances motivation, engagement, and openness to learning (Deci & Ryan, 2000;…”
The Perceived Utility of Human and Automated Aids in a Visual Detection Task
Dzindolet, M. T., Pierce, L. G., Beck, H. P., & Dawe, L. A. (2002) · Human Factors: The Journal of the Human Factors and Ergonomics Society · doi:10.1518/0018720024494856 · Cited by 386 · see citing works
Amerstorfer, C. M., & von Münster-Kistner, C. F. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.713057 · Cited by 332 · see citing works
Effects of Frequent Classroom Testing
Bangert‐Drowns, R. L., Kulik, J. A., & Kulik, C. C. (1991) · The Journal of Educational Research · doi:10.1080/00220671.1991.10702818 · Cited by 320 · see citing works
Understanding the influence of emotions and reflection upon multi-source feedback acceptance and use
Sargeant, J., Mann, K., Sinclair, D., Van der Vleuten, C., & Metsemakers, J. (2008) · Advances in Health Sciences Education · doi:10.1007/s10459-006-9039-x · Cited by 274 · see citing works
Justice perceptions of artificial intelligence in selection
Açıkgöz, Y., Davison, H. K., Compagnone, M., & Laske, M. M. (2020) · International Journal of Selection and Assessment · doi:10.1111/ijsa.12306 · Cited by 263 · see citing works
Johnson, D. G., & Verdicchio, M. (2017) · Minds and Machines · doi:10.1007/s11023-017-9417-6 · Cited by 241 · see citing works
Suen, H.-Y., Chen, M. Y.-C., & Lu, S. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2019.04.012 · Cited by 231 · see citing works
Sources of feedback: A preliminary investigation
Greller, M. M., & Herold, D. M. (1975) · Organizational Behavior and Human Performance · doi:10.1016/0030-5073(75)90048-3 · Cited by 225 · see citing works
The uncanny of mind in a machine: Humanoid robots as tools, agents, and experiencers
Appel, M., Izydorczyk, D., Weber, S., Mara, M., & Lischetzke, T. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2019.07.031 · Cited by 211 · see citing works
Effects of differential feedback on students’ examination performance.
Lipnevich, A. A., & Smith, J. K. (2009) · Journal of Experimental Psychology: Applied · doi:10.1037/a0017841 · Cited by 199 · see citing works
Edwards, A., Edwards, C., Spence, P. R., Harris, C., & Gambino, A. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.06.005 · Cited by 174 · see citing works
Lipnevich, A. A., & Smith, J. K. (2009) · Educational Assessment Evaluation and Accountability · doi:10.1007/s11092-009-9082-2 · Cited by 171 · see citing works
Allying with AI? Reactions toward human-based, AI/ML-based, and augmented hiring processes
Gonzalez, M. F., Liu, W., Shirase, L., Tomczak, D. L., Lobbe, C. E., Justenhoven, R. T., & Martin, N. R. (2022) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107179 · Cited by 157 · see citing works
Kim, J., Merrill, K., Xu, K., & Kelly, S. E. (2022) · Computers in Human Behavior · doi:10.1016/j.chb.2022.107383 · Cited by 147 · see citing works
Mirowska, A., & Mesnet, L. (2021) · Human Resource Management Journal · doi:10.1111/1748-8583.12393 · Cited by 124 · see citing works
Lechermeier, J., & Faßnacht, M. (2018) · Management Review Quarterly · doi:10.1007/s11301-018-0136-8 · Cited by 115 · see citing works
Zhai, N., & Ma, X. (2021) · Computer Assisted Language Learning · doi:10.1080/09588221.2021.1897019 · Cited by 113 · see citing works
Group Feedback for Continuous Learning
London, M., & Sessa, V. I. (2006) · Human Resource Development Review · doi:10.1177/1534484306290226 · Cited by 97 · see citing works
Dai, W., Tsai, Y.-S., Lin, J., Aldino, A., Jin, H., Li, T., Gašević, D., & Chen, G. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100299 · Cited by 93 · see citing works
Classroom Strategies to Enhance Academic Engaged Time
Gettinger, M., & Walter, M. J. (2012) · Handbook of Research on Student Engagement · doi:10.1007/978-1-4614-2018-7_31 · Cited by 89 · see citing works
McCarthy, J. M., Iddekinge, C. H. V., Lievens, F., Kung, M., Sinar, E. F., & Campion, M. A. (2013) · Journal of Applied Psychology · doi:10.1037/a0034089 · Cited by 77 · see citing works
An Interpersonal Approach to Classroom Management: Strategies for Improving Student Engagement
Davis, H. A., Summers, J. J., & Miller, L. M. (2012) · Corwin Press eBooks · doi:10.4135/9781483387383 · Cited by 54 · see citing works
Bardach, L., Rushby, J. V., Kim, L. E., & Klassen, R. M. (2020) · European Journal of Work and Organizational Psychology · doi:10.1080/1359432x.2020.1736619 · Cited by 49 · see citing works
Zhang, S., Zhao, X., Nan, D., & Kim, J. H. (2024) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-024-00465-2 · Cited by 41 · see citing works
Richmond, A. S., Berglund, M. B., Epelbaum, V. B., & Klein, E. M. (2015) · Teaching of Psychology · doi:10.1177/0098628315569924 · Cited by 38 · see citing works
Level and Source of Feedback as Determinants of Feedback Effectiveness.
DeNisi, A. S., Randolph, W. A., & Blencoe, A. G. (1982) · Academy of Management Proceedings · doi:10.5465/ambpp.1982.4976533 · Cited by 21 · see citing works
Ackerman, D. S., Dommeyer, C. J., & Gross, B. L. (2016) · Journal of Marketing Education · doi:10.1177/0273475316628293 · Cited by 17 · see citing works
Virtual simulation: New method for assessing teamwork skills
Kotlyar, I., & Krasman, J. (2022) · International Journal of Selection and Assessment · doi:10.1111/ijsa.12368 · Cited by 17 · see citing works
Self-citationProbing Language Teachers’ Stroking and Credibility in Relation to Their Success in Class
Pishghadam, R., & Karami, M. (2017) · Libraries and Cultural Resources (University of Calgary) · doi:10.11575/ajer.v63i4.56431 · Cited by 14 · see citing works
Understanding Human-Ai Trust in EducationPreprint
Pitts, G., & Motamedi, S. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5325944 · Cited by 14 · see citing works
Oostrom, J. K., Bos-Broekema, L., Serlie, A. W., Born, M. P., & van der Molen, H. T. (2012) · Human Performance · doi:10.1080/08959285.2012.658928 · Cited by 13 · see citing works
Assessing Teamwork Skills: Can a Computer Algorithm Match Human Experts?
Kotlyar, I., Sharifi, T., & Fıksenbaum, L. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-022-00318-x · Cited by 11 · see citing works
Self-citationVirtual high-fidelity simulation assessment of teamwork skills: How do students react?
Kotlyar, I., Krasman, J., & Fıksenbaum, L. (2020) · Journal of Research on Technology in Education · doi:10.1080/15391523.2020.1783401 · Cited by 7 · see citing works
Self-citationEmotionally Enriched Feedback via Generative AIPreprint
Alsaiari, O. A. S., Baghaei, N., Lahza, H. F., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2410.15077 · Cited by 5 · see citing works
Pishghadam, R., Makiabadi, H., & Mohtasham, M. (2018) · Riviste UNIMI (Università degli studi di Milano) · doi:10.13130/2035-7680/10121 · Cited by 3 · see citing works
Kotlyar, I., Pearse, N. J., & Krasman, J. (2024) · Discover Education · doi:10.1007/s44217-024-00125-7 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 8
- Boud, D., Cohen, R., & Sampson, J. (Eds.). (2001). Peer learning in higher education: learning from and with each other.
- Chinn, C. A., Rinehart, R. W., & Buckland, L. A. (2014). Epistemic cognition and evaluating information: Applying the AIR model of epistemic cognition. In D. K. M. Kendeou, J. L. G. Braasch, & P. van den Broek (Eds.), Cognitive processes in comprehension of science texts (pp. 425-452). Routledge.
- Fahmy, Y. (2024). Student perception on AI-driven assessment: Motivation, engagement and feedback capabilities [Master's thesis, University of Twente]. University of Twente Student Theses. https:// essay. utwen te. nl/ 91297/
- Nargesi, M. N. (2021). Comparing AI and human feedback: The role of source in feedback perceptions [Doctoral dissertation, San Francisco State University]. https:// schol arwor ks. calst ate. edu/ downl oads/ r494v r478
- Siau, K., & Wang, W. (2018). Building trust in artificial intelligence, machine learning, and robotics. Cutter Business Technol- ogy Journal, 31(2), 47-53.
- Tajfel, H., & Turner, J. C. (1986). The social identity theory of intergroup behavior. In S. Worchel & W. G. Austin (Eds.), Psychol- ogy of intergroup relations (pp. 7-24).
- Topping, K. (1998). Peer assessment between students in colleges and universities. Review of Educational Research, 68(3), 249-276. https:// doi. org/ 10. 3102/ 00346 54306 80032 · DOI
- Zhang, S., Zhao, X., Nan, D., & Kim, J. H. (2025). Evaluating trust in AI, human, and co-produced feedback among undergradu- ate students. arXiv. https:// arxiv. org/ abs/ 2504. 10961
Cited by
Sorted by citations
Morgan, D. (2026) · Disability & Society · doi:10.1080/09687599.2026.2667528 · Cited by 4 · see citing works
“Research on AI-mediated support suggests that students are significantly less receptive to feedback they believe is AI-generated, perceiving it as less credible and empathetic than human feedback (Kotlyar and Krasman 2025; Nazaretsky et al. 2026).”
AI-Powered Simulation for Nursing Education: Mixed Methods Systematic Review
Jiang, H., Wang, Z., Shen, W., Meng, M., Yang, D., Li, X., & Hao, Y. (2026) · Journal of Medical Internet Research · doi:10.2196/95167 · Cited by 4 · see citing works
“Kotlyar and Krasman [60] indicate that learners’ perceptions of the feedback source directly influence its acceptance and learning outcomes.”
- Authors
- Kotlyar, I., & Krasman, J.
- Affiliations
- Ontario Tech University (CA)
- Corresponding author
- Kotlyar, Igor
- ORCID
- Kotlyar, Igor
- Venue
- International Journal of Educational Technology in Higher Education
- Published
- 2025-10-10
- Volume
- 22
- Issue
- 1
- ISSN
- 2365-9440 (ISSN-L)
- OpenAlex
- W4414991394
- Cited by
- 21
- References
- 73
- Citations by year
- 2026: 20 · 2025: 1
- FWCI
- 22.02
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- International Journal of Educational Technology in Higher Education (publishedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Social Sciences › Education › Student Assessment and Feedback
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Keywords
- AI-generated feedback, human feedback, student perceptions, feedback credibility
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
