The DOI matches its Crossref record; 129 of the 131 references it cites were found in a registry or an index, and 2 are not checked yet; none of the 19 sources its claims cite is checked for support yet.
Effects of Generative Artificial Intelligence on K-12 and Higher Education Students’ Learning Outcomes: A Meta-Analysis
Open accessAbstract
Generative artificial intelligence (GenAI) has significant potential for educational innovation, although its impact on students’ learning outcomes remains controversial. This study aimed to examine the impact of GenAI on the learning outcomes of K-12 and higher education students, and explore the moderating factors influencing this impact. A meta-analysis of 49 articles showed that the mean effect sizes of GenAI on students’ learning achievement and learning motivation were 0.857 and 0.803, respectively, indicating a positive impact of GenAI on education. However, this effect varied according to moderators, including education level, subject classification, GenAI interface, GenAI development, interaction approaches, and experimentation time, which enhanced the impact of GenAI on education. Specifically, GenAI had a greater impact on the academic performance of higher education students, and students interacted more effectively with GenAI using text than with mixed media, such as images or audio. Although GenAI has a novel effect on students’ learning motivation, the effect size decreases over time. These findings provide empirical support for the beneficial effects of GenAI on education and offer insights for optimizing its use in teaching practices.
Checked 2026-11-12 · 2 references not checked yet
Overview
CoChat Summary
TL;DR
This meta-analysis of 49 studies finds that generative AI has a large positive overall effect on students’ learning achievement (g = 0.857) and learning motivation (g = 0.803), though effects vary by education level, subject, interface, development approach, interaction mode, and experiment duration. The authors conclude GenAI benefits learning but that design and implementation choices matter.
Key findings
- GenAI showed a large positive overall effect on learning achievement (g = 0.857, 95% CI [0.601, 1.113], p < .05) and learning motivation (g = 0.803, 95% CI [0.445, 1.161], p < .05).
- Effects differed by education level: for achievement, higher education students benefited more (g = 0.959) than K-12 students (g = 0.495); for motivation, K-12 students benefited more (g = 1.651) than higher education students (g = 0.642).
- Application-based GenAI access had stronger achievement effects (g = 1.107) than web-based access (g = 0.259); text-only interaction had stronger achievement effects (g = 1.033) than mixed interaction (g = 0.816), and much stronger motivation effects (g = 2.086 vs. g = 0.621).
- Duration moderated outcomes: achievement was strongest at 5–10 weeks (g = 1.892) and declined beyond 15 weeks (g = 0.413); motivation was strongest at 1–5 weeks (g = 0.821) and decreased beyond 5 weeks (g = 0.697).
Methods
- PRISMA-guided meta-analysis searching WOS and Scopus on August 18, 2024, plus snowball sampling; from 3,760 initial records and 122 snowball records, 49 articles met inclusion criteria.
- Included empirical studies with a GenAI experimental group and non-GenAI control group, measured learning outcomes, and provided effect-size or sufficient statistical information; studies were coded independently by two researchers.
- Analyses used Comprehensive Meta-Analysis 3.0, Hedges’ g, 95% CIs, random-effects models due to significant heterogeneity, and moderator analyses for education level, subject classification, GenAI interface, GenAI development, interaction approach, and experiment duration.
Limitations
- The paper states it included only English-language literature from WOS and Scopus; excluded empirical studies lacking sufficient effect-size data; did not examine different types of GenAI as moderators; and found publication-bias test results varied across methods, possibly because negative results are more likely to be rejected.
- Evident from the text: several outcomes had too few studies for meta-analysis (critical thinking n = 4, knowledge retention n = 2, learning anxiety n = 2), and some moderator cells were very small, such as motivation for experiments >15 weeks (n = 1).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 5 Not valid · 16 Not checked · 51 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“we conducted a statistical analysis of learning outcomes (as shown in Table 3), which identified learning achievement, learning motivation, and self-efficacy as recurring themes across the 49 studies.”
§ Effects of Generative Artificial Intelligence on K-12 and Higher Education Students' Learning Outcomes: A Meta-Analysis > GenAI Interaction Approaches.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, this effect varied according to moderators, including education level, subject classification, GenAI interface, GenAI development, interaction approaches, and experimentation time, which enhanced the impact of GenAI on education.”
§ Effects of Generative Artificial Intelligence on K-12 and Higher Education Students' Learning Outcomes: A Meta-Analysis > Abstract
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For learning motivation, significant differences emerged across the four primary subject areas examined (Q = 14.981, p < .05).”
§ Effects of Generative Artificial Intelligence on K-12 and Higher Education Students' Learning Outcomes: A Meta-Analysis > Classification of Subject.
plainn = 4 ·- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In contrast, Figure 3, displaying results on learning motivation, shows slight asymmetry in the funnel plot.This asymmetry indicates that there may be some degree of publication bias affecting studies on learning motivation.”
§ Effects of Generative Artificial Intelligence on K-12 and Higher Education Students' Learning Outcomes: A Meta-Analysis > Analysis of Publication Bias
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Specifically, GenAI had a greater impact on the academic performance of higher education students”
§ Effects of Generative Artificial Intelligence on K-12 and Higher Education Students' Learning Outcomes: A Meta-Analysis > Abstract
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
25 more claims
Create a free account to see all 72 claims and to verify the 3 references still open.
- It may state two separate claims as one. 31
- The quote only partly states this claim. 3
- Probably not a substantial claim. 12
Statistics recomputed
Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.
- Consistentz = 4.758reported p < .05 · recomputed p < .001
“(z = 4.758, p < .05”
- Consistentz = 3.229reported p < .05 · recomputed p = .001
“(z = 3.229, p < .05”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Effects of Generative Artificial Intelligence on K-12 and Higher Education Students' Learning Outcomes: A Meta-Analysis > GenAI Development.
“However, for learning motivation, Self-created GenAI tools (g = 1.029, p < .05)have a significantly greater positive impact.”
§ Effects of Generative Artificial Intelligence on K-12 and Higher Education Students' Learning Outcomes: A Meta-Analysis > Limitations and Future Studies
“GenAI development Existing system 20 0.678 0.128 [0.426, 0.929] 5.276 .000*Self-created with targeted functions 10 1.029 0.477 [0.093, 1.965] 2.155 .031*Total between 0.505 .477”
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Effects of Generative Artificial Intelligence on K-12 and Higher Education Students' Learning Outcomes: A Meta-Analysis > Sensitivity Analysis
“All estimates of learning achievement (95% CI [0.251, 0.383]) and motivation (95% CI [0.471, 0.648]) fall within the 95% confidence limits, which means that the results of the meta-analysis are robust.”
§ Effects of Generative Artificial Intelligence on K-12 and Higher Education Students' Learning Outcomes: A Meta-Analysis > Analysis of the Overall Effect of GenAI Application on Students' Learning Achievement and Motivation
“the impact on learning achievement was substantial (g = 0.857, 95% CI [0.601, 1.113], p < .05)”
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
§ Effects of Generative Artificial Intelligence on K-12 and Higher Education Students' Learning Outcomes: A Meta-Analysis > Analysis of Publication Bias
“the random effects model removed 12 studies, yielding an adjusted effect size of 1.242 for learning achievement, with a revised 95% confidence interval [0.787, 1.695].”
§ Effects of Generative Artificial Intelligence on K-12 and Higher Education Students' Learning Outcomes: A Meta-Analysis > Limitations and Future Studies
“| | Adjusted values | 12 | 1.242 | [0.900, 1.583] 2133.737 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Effects of Generative Artificial Intelligence on K-12 and Higher Education Students' Learning Outcomes: A Meta-Analysis > Classification of Subject.
“GenAI had a significant positive impact on learning achievement in Arts and Humanities (g = 1.271, p < .05),Information and Communication Technologies (g = 0.814, p < .05),Health and Welfare (g = 0.939, p > .05),and Education (g = 0.926, p < .05).”
§ Effects of Generative Artificial Intelligence on K-12 and Higher Education Students' Learning Outcomes: A Meta-Analysis > Limitations and Future Studies
“Health and Welfare 5 0.939 0.490 [À0.021, 1.899] 1.917 .055”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Effects of Generative Artificial Intelligence on K-12 and Higher Education Students' Learning Outcomes: A Meta-Analysis > Analysis of Publication Bias
“For learning achievement and learning motivation, the Failsafe N values are 3540 and 1616, respectively”
§ Effects of Generative Artificial Intelligence on K-12 and Higher Education Students' Learning Outcomes: A Meta-Analysis > Limitations and Future Studies
“| Learning achievement | 49 | 3504 | 6.620 | 0.837 | .000 | 4.758 | .000 |”
2 more findings
Create a free account to see every finding of this check.
Citations
Cites
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,400
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Self-efficacy: Toward a unifying theory of behavioral change
Bandura, A. (1977) · Psychological Review · doi:10.1037/0033-295x.84.2.191 · Cited by 43,925
Quantifying heterogeneity in a meta‐analysis
Higgins, J. P. T., & Thompson, S. G. (2002) · Statistics in Medicine · doi:10.1002/sim.1186 · Cited by 36,763
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099
The PRISMA 2020 Statement: An Updated Guideline for Reporting Systematic Reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · International Journal of Surgery · doi:10.1016/j.ijsu.2021.105906 · Cited by 11,748
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 1×
“Kasneci et al. (2023) argued that both K-12 students and higher education students can bene fi t from GenAI ’ s integration into learning, teaching interactions, and personalized learning experiences.”
(1956) · Medical Entomology and Zoology · Cited by 4,596
Research works printed but not linked: 1
- Moore B. N., Parker R. (2012). Critical thinking. Mcgraw-Hill.
Cited by
Allison, J., Hwang, G., Mayer, R. E., Πέλλας, Ν., Karnalim, O., de Freitas, S., Ng, O., Huang, Y., Hooshyar, D., Seidman, R. H., et al. (2025) · Journal of Educational Computing Research · doi:10.1177/07356331251359964 · Cited by 77
Institutional approaches to generative AI management in higher education: a systematic review
Bacalso, F. D., Villamor, F. E., Paclipan, C. P., Orquia, C. J. B., Rendon, I. P., Albiso, A. T., & Himang, C. M. (2026) · Frontiers in Education · doi:10.3389/feduc.2026.1814426 · Cited by 20
Wang, Y. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1761562 · Cited by 16
Exploring the effect of GenAI on learning outcomes in higher education: a three-level meta-analysis
Fan, C., Ke, L., Chen, Z., & Lv, P. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1758670 · Cited by 15
The calculator analogy: Epistemic virtues for using LLMs
Voinea, C., Mann, S. P., Savulescu, J., & Earp, B. D. (2025) · Technology in Society · doi:10.1016/j.techsoc.2025.103198 · Cited by 12
Metadata
- Authors
- Liu, X., Guo, B., He, W., & Hu, X.
- Affiliations
- South China Normal University (CN)
- Corresponding author
- Hu, Xiaoyong
- Venue
- Journal of Educational Computing Research
- Publisher
- SAGE Publications
- Published
- 2025-04-11
- Volume
- 63
- Issue
- 5
- Pages
- 1249-1291
- ISSN
- 0735-6331 (ISSN-L)
- 1541-4140
- OpenAlex
- W4409352105
- Cited by
- 89
- References
- 125
- Citations by year
- 2026: 77 · 2025: 12
- FWCI
- 9.27
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of Educational Computing Research (publishedVersion)
- Funders
- China Postdoctoral Science Foundation (2024M760980)
- National Social Science Foundation of China (BCA240050)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- generative AI, learning outcomes, K-12 education, higher education, academic performance, student motivation, moderators, text-based interaction, educational technology
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-11-12, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-11-12 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
