Does Generative Artificial Intelligence Improve the Academic Achievement of College Students? A Meta-Analysis
Abstract
The use of generative artificial intelligence (Gen-AI) to assist college students in their studies has become a trend. However, there is no academic consensus on whether Gen-AI can enhance the academic achievement of college students. Using a meta-analytic approach, this study aims to investigate the effectiveness of Gen-AI in improving the academic achievement of college students and to explore the effects of different moderating variables. A total of 28 articles (65 independent studies, 1909 participants) met the inclusion criteria for this study. The results showed that Gen-AI significantly improved college students’ academic achievement with a medium effect size (Hedges’s g = 0.533, 95% CI [0.408,0.659], p < .05). There were within-group differences in the three moderator variables, activity categories, sample size, and generated content, when the generated content was text ( g = 0.554, p < .05), and sample size of 21–40 ( g = 0.776, p < .05), the use of independent learning styles ( g = 0.600, p < .05) had the most significant improvement in college student’s academic achievement. The intervention duration, the discipline types, and the assessment tools also had a moderate positive impact on college students’ academic achievement, but there were no significant within-group differences in any of the moderating variables. This study provides a theoretical basis and empirical evidence for the scientific application of Gen-AI and the development of educational technology pol…
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Working with generative AI tools such as ChatGPT increases productivity in complex problem-solving tasks in educational and professional settings.
“Working with generative AI tools, such as ChatGPT, increases productivity in complex problem-solving tasks in both educational and professional settings (Noy & Zhang, 2023;Sun & Zhou, 2024;Urban et al., 2024).”
From “ <scp>ChatGPT</scp> can make mistakes. Check important info.” Epistemic beliefs and metacognitive accuracy in students' integration of <scp>ChatGPT</scp> content into academic writing (2025) · cites this paper as Sun & Zhou, 2024
Cites
Cohen, J. (1992) · Current Directions in Psychological Science · doi:10.1111/1467-8721.ep10768783 · Cited by 5,870 · see citing works
Hattie, J. (2008) · doi:10.4324/9780203887332 · Cited by 3,051 · see citing works
Baidoo-Anu, D., & Owusu Ansah, L. (2023) · doi:10.2139/ssrn.4337484 · Cited by 3,004 · cited 1× · see citing works
“This is consistent with existing research, where Baidoo-Anu and Owusu Ansah (2023) reported a strong impact of Gen-AI on teaching and learning, where Gen-AI signi fi cantly improved not only the cognitive level, but also the critical thinking, creative thinking, and problem-solving skills of college…”
On the Opportunities and Risks of Foundation Models
Bommasani, R., Hudson, D. A., Adeli, E., Altman, R., Arora, S., von Arx, S., Bernstein, M. S., Bohg, J., Bosselut, A., Brunskill, E., et al. (2021) · arXiv · doi:10.48550/arxiv.2108.07258 · Cited by 2,676 · see citing works
Students' voices on generative ai: perceptions, benefits, and challenges in higher education
Chan, C. K. Y., & Hu, W. (2023) · arXiv · doi:10.48550/arxiv.2305.00290 · Cited by 2,504 · cited 1× · see citing works
“…analysis, the intervention duration of self-directed learning activities should be appropriately prolonged to take full advantage of the personalized feedback and guidance provided by the AI to help students construct their knowledge in-depth and enhance their writing skills (Chan & Hu, 2023).”
Students’ voices on generative AI: perceptions, benefits, and challenges in higher education
Chan, C. K. Y., & Hu, W. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00411-8 · Cited by 2,301 · cited 1× · see citing works
“…analysis, the intervention duration of self-directed learning activities should be appropriately prolonged to take full advantage of the personalized feedback and guidance provided by the AI to help students construct their knowledge in-depth and enhance their writing skills (Chan & Hu, 2023).”
Doing Meta-Analysis with R: A Hands-On Guide
Harrer, M., Cuijpers, P., Furukawa, T. A., & Ebert, D. D. (2021) · doi:10.1201/9781003107347 · Cited by 1,850 · see citing works
ChatGPT: Bullshit spewer or the end of traditional assessments in higher education?
(2023) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2023.6.1.9 · Cited by 1,806 · cited 1× · see citing works
“This is explained by Rudolph et al. (2023), who argued that Gen-AI has the potential to enhance the teaching and learning experience through personalized learning materials, simulated conversation partners, and automated grading, but that at present, there may be technological limitations, cost…”
Writing meta-analytic reviews.
Rosenthal, R. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.118.2.183 · Cited by 1,108 · see citing works
Ouyang, F., Zheng, L., & Jiao, P. (2022) · Education and Information Technologies · doi:10.1007/s10639-022-10925-9 · Cited by 831 · see citing works
Bahroun, Z., Anane, C., Ahmed, V., & Zacca, A. (2023) · Sustainability · doi:10.3390/su151712983 · Cited by 784 · see citing works
AI-generated feedback on writing: insights into efficacy and ENL student preference
Escalante, J., Pack, A., & Barrett, A. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00425-2 · Cited by 589 · see citing works
Generative AI in Education and Research: Opportunities, Concerns, and Solutions
Alasadi, E. A., & Baiz, C. R. (2023) · Journal of Chemical Education · doi:10.1021/acs.jchemed.3c00323 · Cited by 463 · see citing works
Essel, H. B., Vlachopoulos, D., Tachie-Menson, A., Johnson, E. E., & Baah, P. K. (2022) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-022-00362-6 · Cited by 428 · see citing works
Impact of ChatGPT on ESL students’ academic writing skills: a mixed methods intervention study
Mahapatra, S. (2024) · Smart Learning Environments · doi:10.1186/s40561-024-00295-9 · Cited by 398 · see citing works
ChatGPT improves creative problem-solving performance in university students: An experimental study
Urban, M., Děchtěrenko, F., Lukavský, J., Hrabalová, V., Svacha, F., Brom, C., & Urban, K. (2024) · Computers & Education · doi:10.1016/j.compedu.2024.105031 · Cited by 396 · see citing works
A meta-analysis of the impact of technology on learning effectiveness of elementary students
Chauhan, S. (2017) · Computers & Education · doi:10.1016/j.compedu.2016.11.005 · Cited by 395 · see citing works
Bozkurt, A., Xiao, J., Lambert, S., Pazurek, A., Crompton, H., Koseoglu, S., Farrow, R., Bond, M., Nerantzi, C., Honeychurch, S., et al. (2023) · Zenodo · doi:10.5281/zenodo.7636568 · Cited by 350 · see citing works
Computing Education in the Era of Generative AI
Denny, P., Prather, J., Becker, B. A., Finnie-Ansley, J., Hellas, A., Leinonen, J., Luxton-Reilly, A., Reeves, B. N., Santos, E. A., & Sarsa, S. (2024) · Communications of the ACM · doi:10.1145/3624720 · Cited by 316 · cited 1× · see citing works
“As students become more pro fi cient with Gen-AI, they can learn better, thus improving their academic achievement (Denny et al., 2023).”
Holy or Unholy? Interview with Open AI’s ChatGPT
Iskender, A. (2023) · European Journal of Tourism Research · doi:10.54055/ejtr.v34i.3169 · Cited by 260 · see citing works
The influence of AI text generators on critical thinking skills in UK business schools
Essien, A., Bukoye, O. T., O’Dea, X., & Kremantzis, M. (2024) · Studies in Higher Education · doi:10.1080/03075079.2024.2316881 · Cited by 252 · see citing works
ChatGPT is not all you need. A state-of-the-art review of large generative AI models
Gozalo-Brizuela, R., & Garrido-Merchan, E. C. (2023) · arXiv · doi:10.48550/arxiv.2301.04655 · Cited by 245 · see citing works
Better by You, better than Me? ChatGPT-3 as writing assistance in students’ essays.
Bašić, Ž., Banovac, A., Kružić, I., & Jerković, I. (2023) · doi:10.35542/osf.io/n5m7s · Cited by 189 · see citing works
Effects of chatbot-assisted in-class debates on students’ argumentation skills and task motivation
Guo, K., Zhong, Y., Li, D., & Chu, S. K. W. (2023) · Computers & Education · doi:10.1016/j.compedu.2023.104862 · Cited by 188 · see citing works
Han, J.-W., Park, J., & Lee, H. (2022) · BMC Medical Education · doi:10.1186/s12909-022-03898-3 · Cited by 187 · cited 1× · see citing works
“…students (Koç, 2024; Kuhail et al., 2022; Qureshi, 2023), foster critical thinking (Essien et al., 2024), and stimulate interest in learning (Han et al., 2022), enhancing motivation (Yilmaz & Yilmaz, 2023), learning engagement (Ouyang et al., 2022), learning satisfaction (Li, 2023), and…”
Prompt Problems: A New Programming Exercise for the Generative AI Era
Denny, P., Leinonen, J., Prather, J., Luxton-Reilly, A., Amarouche, T., Becker, B. A., & Reeves, B. N. (2024) · Proceedings of the 55th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3626252.3630909 · Cited by 187 · see citing works
Pellas, N. (2023) · Education Sciences · doi:10.3390/educsci13111155 · Cited by 95 · cited 1× · see citing works
“…(Koç, 2024; Kuhail et al., 2022; Qureshi, 2023) by developing their writing skills (Banihashem et al., 2024), critical thinking (Essien et al., 2024), problem-solving skills (Liu et al., 2024), and can also be effective in improving college students ’ motivation and self-ef fi cacy (Pellas, 2023).”
Xiao, P., Chen, Y., & Bao, W. (2023) · doi:10.2139/ssrn.4458269 · Cited by 93 · see citing works
Al-Abdullatif, A. M., Al-Dokhny, A. A., & Drwish, A. M. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1129070 · Cited by 80 · cited 1× · see citing works
“Generative content of text type and code type may be more easily integrated into the learning and assessment process due to its directness, logic, and ease of assessment, thus having a greater impact on academic achievement (Al-Abdullatif et al., 2023; Yilmaz & Yilmaz, 2023).”
Bonsu, E., & Baffour-Koduah, D. (2023) · doi:10.2139/ssrn.4387107 · Cited by 26 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Iqbal, J., Hashmi, Z. F., Asghar, M. Z., & Abid, M. N. (2025) · Scientific Reports · doi:10.1038/s41598-025-01676-x · Cited by 82 · see citing works
Salih, S., Husain, O., Hamdan, M., Abdelsalam, S., Elshafie, H., & Motwakel, A. (2025) · Results in Engineering · doi:10.1016/j.rineng.2024.103837 · Cited by 74 · see citing works
Kassenkhan, A. M., Moldagulova, A. N., & Serbin, V. V. (2025) · IEEE Access · doi:10.1109/access.2025.3576147 · Cited by 44 · see citing works
The Impact of <scp>ChatGPT</scp> on Students' Academic Achievement: A Meta‐Analysis
Liu, Z., Zuo, H., & Lu, Y. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.70096 · Cited by 44 · see citing works
Urban, M., Brom, C., Lukavský, J., Děchtěrenko, F., Hein, V., Svacha, F., Kmoníčková, P., & Urban, K. (2025) · British Journal of Educational Technology · doi:10.1111/bjet.13591 · Cited by 47 · see citing works
Wu, M., & Ouyang, F. (2026) · Educational Psychology Review · doi:10.1007/s10648-026-10211-x
- Authors
- Sun, L., & Zhou, L.
- Affiliations
- Minzu University of China (CN)
- Corresponding author
- Sun, Lihui
- Venue
- Journal of Educational Computing Research
- Publisher
- SAGE Publications
- Published
- 2024-08-27
- Volume
- 62
- Issue
- 7
- Pages
- 1676-1713
- ISSN
- 0735-6331 (ISSN-L)
- 1541-4140
- OpenAlex
- W4401971689
- Cited by
- 129
- References
- 30
- Citations by year
- 2026: 70 · 2025: 57 · 2024: 2
- FWCI
- 7.67
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Educational Computing Research (publishedVersion)
- Funders
- The financial supports by the National Social Science Foundation Youth Project in Pedagogy of China (CCA190261)
- Topics
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Keywords
- generative AI, academic performance, university students, learning styles, moderators, educational technology, AI-assisted learning, sample size effects, intervention duration, disciplinary differences, assessment instruments
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
