How does artificial intelligence compare to human feedback? A meta-analysis of performance, feedback perception, and learning dispositions
The DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“a comprehensive meta-analysis by Kaliisa et al. (2026) synthesizing 41 studies found no statistically significant difference in learning outcomes between the two sources, suggesting functional equivalence in immediate performance.”
From Personalized language learning with an LLM chatbot: effects of immediate vs. delayed corrective feedback (2026) · cites this paper as Kaliisa et al. (2026)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
Hattie, J., & Timperley, H. (2007) · Review of Educational Research · doi:10.3102/003465430298487 · Cited by 12,905
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 3×
“However, even though AWE tools have been found useful in providing real-time feedback and personalised guidance, they are criticised for being less interpretable and biased, as the reasoning behind their decisions is often unclear to the users (Kasneci et al., 2023).”
InfluentialOutlier and influence diagnostics for meta-analysis
Viechtbauer, W., & Cheung, M. W.-L. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.11 · Cited by 2,949 · cited 2×
“Identifying such influential studies enables the identification of potential sources of heterogeneity and assess the robustness of the meta-analytic findings (Viechtbauer & Cheung, 2010).”
Robust variance estimation in meta‐regression with dependent effect size estimates
Hedges, L. V., Tipton, E., & Johnson, M. C. (2010) · Research Synthesis Methods · doi:10.1002/jrsm.5 · Cited by 2,086 · cited 1×
“…same study, we used Robust Variance Estimation—with sandwich-type estimator—to account for effect size dependence and avoid underestimation of variance, inflation of confidence intervals or Type I error that may arise from using traditional models (Hedges et al., 2010; Pustejovsky & Tipton, 2022).”
Methods to estimate the between‐study variance and its uncertainty in meta‐analysis
Veroniki, A. A., Jackson, D., Viechtbauer, W., Bender, R., Bowden, J., Knapp, G., Kuss, O., Higgins, J. P., Langan, D., & Salanti, G. (2016) · Research Synthesis Methods · doi:10.1002/jrsm.1164 · Cited by 1,685 · cited 1×
“We estimated the between-study variance using the restricted maximum-likelihood estimator (RMLE) (Viechtbauer, 2010), as recommended for continuous outcomes by recent guidelines (Veroniki et al., 2016).”
The Power of Feedback Revisited: A Meta-Analysis of Educational Feedback Research
Wisniewski, B., Zierer, K., & Hattie, J. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.03087 · Cited by 1,199 · cited 3×
“This aligns with Wisniewski et al. (2019), who emphasised the variability in feedback effectiveness.”
InfluentialMeta-analysis with robust variance estimation: Expanding the range of working models
Pustejovsky, J. E., & Tipton, E. (2022) · Prevention Science · doi:10.1007/s11121-021-01246-3 · Cited by 737 · cited 2×
“…same study, we used Robust Variance Estimation—with sandwich-type estimator—to account for effect size dependence and avoid underestimation of variance, inflation of confidence intervals or Type I error that may arise from using traditional models (Hedges et al., 2010; Pustejovsky & Tipton, 2022).”
Research works printed but not linked: 55
- 10.3991/ijet.v18i04.36349 · DOI
- 10.1177/21582440221123021 · DOI
- Bearman, M., Ajjawi, R., Boud, D., Tai, J., & Dawson, P. (2023). CRADLE suggests… assessment and genAI. Deakin University, Centre for Research in Assessment and Digital Learning.
- 10.2307/2533446 · DOI · Paper page
- 10.3102/00346543065003245 · DOI · Paper page
- 10.1186/s40862-022-00171-4 · DOI
- 10.1080/10494820.2019.1655769 · DOI
- Cheng, G., Law, E., & Wong, T.-L. (2017). Investigating effects of automated feedback on EFL students’ critical thinking skills. In Proceedings of the IEEE International Conference on Teaching, Assessment, and Learning for Engineering (TALE) (pp. 270–276). IEEE. https://doi.org/10.1109/TALE.2016.7851798 · DOI
- 10.1080/10494820.2022.2100426 · DOI
- Darvishi, A., Khosravi, H., Abdi, S., Kitto, K., & Gašević, D. (2022). Incorporating training, self-monitoring and AI-based feedback to improve students’ academic writing. In Proceedings of the ACM Conference on Learning@Scale (pp. 215–225). ACM. https://doi.org/10.1145/3491140.3528265 · DOI · Paper page
- 10.1111/j.0006-341x.2000.00455.x · DOI · Paper page
- 10.1080/07380569.2022.2150067 · DOI · Paper page
- 10.1177/21582440231181296 · DOI
- 10.3389/frai.2023.116245 · doi:10.3389/frai.2023.116245
- 10.1080/02602938.2014.893418 · DOI · Paper page
- 10.1007/s11301-021-00247-4 · DOI · Paper page
- 10.1111/jcal.12587 · DOI
- 10.1002/sim.1186 · DOI · Paper page
- 10.1080/10494820.2022.2160467 · DOI
- Huang, L. (2020). From feedback to revision: How can collaborative peer feedback activities improve students’ writing? In Proceedings of the International Conference on Educational Innovation through Technology (EITT) (pp. 52–56). IEEE. https://doi.org/10.1109/EITT53287.2021.00052 · DOI
- 10.18608/jla.2021.7184 · DOI
- Jin, Y., Yang, K., Yan, L., Echeverria, V., Zhao, L., Alfredo, R., & Martinez-Maldonado, R. (2025, March). Chatting with a learning analytics dashboard: The role of generative AI literacy on learner interaction with conventional and scaffolding chatbots. In Proceedings of the 15th International Learning Analytics and Knowledge Conference (pp. 579–590). · DOI
- 10.1080/1941126X.2018.1444339 · DOI · Paper page
- 10.1037/0033-2909.119.2.254 · DOI · Paper page
- 10.1111/j.1467-8535.2009.00959.x · DOI · Paper page
- 10.1016/j.jslw.2014.10.004 · DOI · Paper page
- Lin, J., Song, J., & Sun, L. (2020, June). The application of artificial intelligence video feedback system in tennis teaching in colleges and universities. In 2020 International Conference on Artificial Intelligence and Education (ICAIE) (pp. 28–31). IEEE. https://doi.org/10.1109/ICAIE50891.2020.00014 · doi:10.1109/icaie50891.2020.00014
- Liu, L. (2022, December). Application of automated writing evaluation (AWE) system based on intelligent technology. In 2022 IEEE 2nd International Conference on Mobile Networks and Wireless Communications (ICMNWC) (pp. 1–5). IEEE. https://doi.org/10.1109/ICMNWC56175.2022.10031924 · DOI
- 10.1109/TLT.2016.2612659 · DOI
- Lu, M., Deng, Q., & Yang, M. (2020). EFL writing assessment: Peer assessment vs. automated writing evaluation. In Lecture Notes in Computer Science (Vol. 12063, pp. 25–35). https://doi.org/10.1007/978-3-030-38778-5 · DOI
- 10.22034/ijlt.2022.364072.1201 · DOI
- Mohsen, M. A., & Alshahrani, A. (2019). The effectiveness of using a hybrid mode of automated writing evaluation and teacher feedback to improve EFL students’ writing. Teaching English with Technology, 19(3), 122–139.
- Mørch, A., Engeness, I., Cheng, V. C. W., Cheung, W. K., & Wong, K. (2017). EssayCritic: Writing to learn with a knowledge-based design critiquing system. Journal of Educational Technology & Society, 20(2), 213–223. https://eric.ed.gov/?id=EJ1137510
- 10.1080/10494820.2022.2096642 · DOI
- 10.1111/bjet.12987 · DOI
- 10.1002/jrsm.1053 · DOI · Paper page
- 10.1186/s41239-022-00372-4 · DOI · Paper page
- Pirttinen, N., & Leinonen, J. (2022). Can students review their peers? Comparison of peer and instructor reviews. In Proceedings of the 27th ACM Conference on Innovation and Technology in Computer Science Education (Vol. 1, pp. 12–18).
- 10.1007/s40593-015-0042-3 · DOI
- 10.1177/0033688214562799 · DOI · Paper page
- 10.1016/j.caeai.2023.100189 · DOI
- 10.1080/10494820.2019.1629601 · DOI · Paper page
- Shibani, A., Knight, S., & Buckingham Shum, S. (2017). Design and implementation of a pedagogic intervention using automated feedback in academic writing. In Proceedings of the International Conference on Computers in Education (ICCE) (pp. 65–74).
- Silitonga, L. M., Hawanti, S., & Aziez, F. (2023). The impact of AI chatbot-based learning on students’ writing outcomes. In Lecture Notes in Computer Science (Vol. 13991, pp. 678–690). https://doi.org/10.1007/978-3-031-40113-8_53 · DOI
- 10.1016/j.stueduc.2021.101123 · DOI
- 10.18637/jss.v036.i03 · DOI · Paper page
- 10.1080/17501229.2021.1914062 · DOI · Paper page
- 10.17507/tpls.0912.12 · DOI
- 10.3389/fpsyg.2022.909802 · DOI · Paper page
- Weitekamp, D., Harpstead, E., & Koedinger, K. R. (2020, April). An interaction design for machine teaching to develop AI tutors. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (pp. 1–11). https://doi.org/10.1145/3313831.3376226 · DOI
- Wilson, J., & Martin, T. (2015). Using PEGWriting® to support the writing motivation and performance of middle school students. Workshop on Innovative Use of NLP for Building Educational Applications (BEA) (pp. 45–52). ACL.
- 10.1016/j.cedpsych.2019.101826 · DOI
- 10.1016/j.compedu.2020.104059 · DOI · Paper page
- 10.1016/j.asej.2014.09.007 · DOI
- 10.1080/09588221.2022.2069822 · DOI
Cited by
M. Kamelabad, A., Turano, B., Lundin, M., & Skantze, G. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1703664 · Cited by 13
AI assistance in peer feedback provision: Pedagogically sound, but minimally adopted
Pozdniakov, S., Brazil, J., Banihashem, S. K., Noroozi, O., Gašević, D., Sadiq, S., & Khosravi, H. (2026) · Computers & Education · doi:10.1016/j.compedu.2026.105591 · Cited by 11
Kolhatin, A. O. (2025) · CEUR Workshop Proceedings, Vol-4060: Proceedings of the 8th International Workshop on Augmented Reality in Education (AREdu 2025) · doi:10.55056/ceur-ws.org/vol-4060/paper05.pdf · Cited by 6
LLM feedback for academic writing: Effects on students’ performance and engagement
Glüsing, R., Fleckenstein, J., Schmidt, F. T. C., & Möller, J. (2026) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2026.102463 · Cited by 2
Feedback futures: Beyond the limits of human and GenAI capacities
Zhan, Y., Wood, J., Carless, D., & Yan, Z. (2026) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2026.2672830 · Cited by 2
Metadata
- Authors
- Kaliisa, R., Misiejuk, K., López-Pernas, S., & Saqr, M.
- Affiliations
- Department of Education, University of Oslo
- Centre for the Science of Learning & Technology (SLATE), University of Bergen
- School of Computing, University of Eastern Finland
- Venue
- Educational Psychology
- Publisher
- Informa UK Limited
- Published
- 2025-09-24
- Volume
- 46
- Issue
- 1
- Pages
- 80-111
- ISSN
- 0144-3410
- 1469-5820
- Cited by
- 33
- References
- 26
- Language
- en
- Update notices
- correction (2026-10-08)
- Licence
- creativecommons.org/licenses/by/4.0/ (vor)
- doi.org/10.1080/policies-text-data-mining (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How does artificial intelligence compare to human feedback? A meta-analysis of performance, feedback perception, and learning dispositions. https://app.cochat.ai/papers/10.1080/01443410.2025.2553639
- The paper
- Kaliisa, R., Misiejuk, K., López-Pernas, S., & Saqr, M. (2026). How does artificial intelligence compare to human feedback? A meta-analysis of performance, feedback perception, and learning dispositions. Educational Psychology. https://doi.org/10.1080/01443410.2025.2553639
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.