The effectiveness of artificial intelligence on learning achievement and learning perception: A meta-analysis
The DOI matches its Crossref record; 29 of the 29 references it cites were found in a registry or an index.
Abstract
Recently, artificial intelligence (AI) technologies have been widely used in the field of education, and artificial intelligence in education (AIEd) has gained increasing attention. However, no quantitative meta-analysis has been conducted on the overall effectiveness of AI on learning achievement and learning perception. To close this research gap, this study conducted a comprehensive meta-analysis of the effects of AI on learning achievement and learning perception. The present meta-analysis synthesized 24 articles with a total of 2908 participants from 2001 to 2020. The findings reveal that AI had a high effect size on learning achievement and a small effect size on learning perception. The effect sizes of 13 moderator variables were analyzed, including sample levels, sample size, learning domains, learning methods, research design, research settings, intervention duration, types of organization for treatment, role of AI, areas of AI application, AI software, AI hardware, and AI technologies. It was found that sample size, sample level, learning domains, types of organization, roles of AI, and hardware significantly moderated the effectiveness of AI. The results and the implications for educators and practitioners are discussed in depth.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“To identify high-quality reviews and ensure reliable results, this study covered journal papers only (Tlili et al., 2024;Zawacki-Richter et al., 2019;Zheng et al., 2023).”
From A systematic review of literature reviews on artificial intelligence in education (AIED): a roadmap to a future research agenda (2024) · cites this paper as Zheng et al., 2023
Citations
Cites
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009) · doi:10.1002/9780470743386 · Cited by 14,225 · cited 2×
“Furthermore, Borenstein et al. (2009) indicated that it is very signi fi cant to obtain the overall e ff ect size among di ff erent studies through a meta-analysis approach.”
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · cited 1×
“Fifth, publication bias was examined through classic fail-safe N (Rosenthal, 1979) and Orwin ’ s fail-safe N (Orwin, 1983).”
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-019-0171-0 · Cited by 7,242 · cited 1×
“Zawacki-Richter et al. (2019) conducted a systematic review of AI in higher education and concluded that AI was employed in four areas, including prediction, assessment, personalization, and tutoring.”
What is technological pedagogical content knowledge (TPACK)?
Koehler, M. J., Mishra, P., & Cain, W. (2013) · Journal of Education · doi:10.1177/002205741319300303 · Cited by 2,790 · cited 1×
“Koehler and Mishra (2009) proposed that technology, pedagogy, and content are three crucial components for teaching and should exist in a state of dynamic equilibrium.”
A Fail-Safe N for Effect Size in Meta-Analysis
Orwin, R. G. (1983) · Journal of Educational Statistics · doi:10.2307/1164923 · Cited by 2,064 · cited 1×
“Fifth, publication bias was examined through classic fail-safe N (Rosenthal, 1979) and Orwin ’ s fail-safe N (Orwin, 1983).”
Vision, challenges, roles and research issues of Artificial Intelligence in Education
Hwang, G.-J., Xie, H., Wah, B. W., & Gašević, D. (2020) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2020.100001 · Cited by 1,435 · cited 2×
“AI technologies have demonstrated promising potential for the realization of personalized learning (Hwang et al., 2020), which has contributed to transforming the traditional “ one size fi t all ” pedagogical approach (Khakpour & Colomo-Palacios, 2020).”
Tamim, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C., & Schmid, R. F. (2011) · Review of Educational Research · doi:10.3102/0034654310393361 · Cited by 1,029 · cited 1×
“This fi nding was consistent with a previous study showing that the application of technologies bene fi ted K-12 levels more than the postsecondary level (Tamim et al., 2011).”
Application and theory gaps during the rise of artificial intelligence in education
Chen, X., Xie, H., Zou, D., & Hwang, G.-J. (2020) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2020.100002 · Cited by 1,003 · cited 6×
“Recently, arti fi cial intelligence in education (AIEd) has undergone rapid development, and interest in AIEd is higher than ever (Chen et al., 2020).”
InfluentialAI technologies for education: Recent research & future directions
Zhang, K., & Aslan, A. B. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100025 · Cited by 923 · cited 2×
“Zhang and Aslan (2021) found that chatbots can increase students ’ learning engagement and that expert systems can improve the quality of interaction.”
Chan, K. S., & Zary, N. (2019) · JMIR Medical Education · doi:10.2196/13930 · Cited by 877 · cited 2×
“Recently, AI has experienced rapid development with the emergence of deep learning technologies (Chan & Zary, 2019).”
Research works printed but not linked: 2
- Nguyen, T. (2015). The effectiveness of online learning: Beyond no significant difference and future horizons. MERLOT Journal of Online Learning and Teaching, 11(2), 309–319. https://jolt.merlot.org/Vol11no2/Nguyen_0615.pdf
- 10.1109/ACCESS.2018.286788 · doi:10.1109/access.2018.286788
Cited by
Revolutionizing education with AI: Exploring the transformative potential of ChatGPT
Adiguzel, T., Kaya, M. H., & Cansu, F. K. (2023) · Contemporary Educational Technology · doi:10.30935/cedtech/13152 · Cited by 975
“When designing learning activities supported by AI, teachers and practitioners should consider six crucial elements to achieve the expected learning objectives: learners, context, tasks, pedagogical approaches, interaction methods, and the application of AI technologies (Zheng et al., 2021).”
BackgroundWei, L. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1261955 · Cited by 536
“In other words, the AI tool served as a catalyst, engaging learners in collaborative language learning activities that ultimately fostered the internalization of these skills (Zheng et al., 2021; Xu et al., 2022).”
BackgroundMethodResultInfluentialArtificial Intelligence (AI) Literacy in Early Childhood Education: The Challenges and Opportunities
Su, J., Ng, D. T. K., & Chu, S. K. W. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100124 · Cited by 489
Wang, J., & Fan, W. (2025) · Humanities and Social Sciences Communications · doi:10.1057/s41599-025-04787-y · Cited by 322
“This scheme was meticulously developed by thoroughly examining the included literature and drawing on insights by Zheng et al. (2021).”
RetractedLai, C. Y., Cheung, K. Y., & Chan, C. S. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100178 · Cited by 230
Metadata
- Authors
- Zheng, L., Niu, J., Zhong, L., & Gyasi, J. F.
- Affiliations
- Beijing Normal University (CN)
- Corresponding author
- Zheng, Lanqin
- Venue
- Interactive Learning Environments
- Publisher
- Informa UK Limited
- Published
- 2021-12-19
- Volume
- 31
- Issue
- 9
- Pages
- 5650-5664
- ISSN
- 1049-4820 (ISSN-L)
- 1744-5191
- OpenAlex
- W4200348706
- Cited by
- 183
- References
- 29
- Citations by year
- 2026: 46 · 2025: 67 · 2024: 49 · 2023: 17 · 2022: 4
- FWCI
- 2.67
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Interactive Learning Environments (publishedVersion)
- Funders
- This study is funded by the International Joint Research Project of Huiyan International College, Faculty of Education, Beijing Normal University
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Education › Educational Technology and Pedagogy
- Keywords
- artificial intelligence in education, academic performance, perceptions of learning, effect size, sample size, learning domains, AI accelerators, AI software, educational technology effectiveness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The effectiveness of artificial intelligence on learning achievement and learning perception: A meta-analysis. https://app.cochat.ai/papers/10.1080/10494820.2021.2015693
- The paper
- Zheng, L., Niu, J., Zhong, L., & Gyasi, J. F. (2023). The effectiveness of artificial intelligence on learning achievement and learning perception: A meta-analysis. Interactive Learning Environments. https://doi.org/10.1080/10494820.2021.2015693
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.