Cited by 463Artificial Intelligence in EducationIntelligent Tutoring Systems and Adaptive LearningArtificial Intelligence in Healthcare and Education

The DOI matches its Crossref record; 136 of the 137 references it cites were found in a registry or an index; none of the 28 sources its claims cite is checked for support yet.

RETRACTED ARTICLE: The effect of ChatGPT on students’ learning performance, learning perception, and higher-order thinking: insights from a meta-analysis

Open access
Wang, Jin, Fan, Wenxiang·2025·Humanities and Social Sciences Communications

Abstract

As a new type of artificial intelligence, ChatGPT is becoming widely used in learning. However, academic consensus regarding its efficacy remains elusive. This study aimed to assess the effectiveness of ChatGPT in improving students’ learning performance, learning perception, and higher-order thinking through a meta-analysis of 51 research studies published between November 2022 and February 2025. The results indicate that ChatGPT has a large positive impact on improving learning performance ( g = 0.867) and a moderately positive impact on enhancing learning perception ( g = 0.456) and fostering higher-order thinking ( g = 0.457). The impact of ChatGPT on learning performance was moderated by type of course ( Q B = 64.249, P < 0.001), learning model ( Q B = 76.220, P < 0.001), and duration ( Q B = 55.998, P < 0.001); its effect on learning perception was moderated by duration ( Q B = 19.839, P < 0.001); and its influence on the development of higher-order thinking was moderated by type of course ( Q B = 7.811, P < 0.05) and the role played by ChatGPT ( Q B = 4.872, P < 0.05). This study suggests that: (1) appropriate learning scaffolds or educational frameworks (e.g., Bloom’s taxonomy) should be provided when using ChatGPT to develop students’ higher-order thinking; (2) the broad use of ChatGPT at various grade levels and in different types of courses should be encouraged to support diverse learning needs; (3) ChatGPT should be actively integrated into different learning modes to enhance student learning, especially in problem-based learning; (4) continuous use of ChatGPT should be ensured to support student learning, with a recommended duration of 4–8 weeks for more stable effects; (5) ChatGPT should be flexibly integrated into teaching as an intelligent tutor, learning partner, and educational tool. Finally, due to the limited sample size for learning perception and higher-order thinking, and the moderately positive effect, future studies with expanded scope should further explore how to use ChatGPT more effectively to cultivate students’ learning perception and higher-order thinking.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

This PRISMA-guided meta-analysis of 51 experimental and quasi-experimental studies (November 2022–February 2025) finds that ChatGPT has a large positive effect on students' learning performance (g = 0.867) and moderate positive effects on learning perception (g = 0.456) and higher-order thinking (g = 0.457). Effects are moderated by course type, learning model, duration, and the role ChatGPT plays, leading the authors to recommend scaffolded, sustained (4–8 week) integration of ChatGPT across grade levels and course types.

Key findings

  • Overall effects across 72 independent effect sizes: learning performance g = 0.867 (k = 44, 95% CI [0.683, 1.052], p < 0.001); learning perception g = 0.456 (k = 19, 95% CI [0.287, 0.625], p < 0.001); higher-order thinking g = 0.457 (k = 9, 95% CI [0.255, 0.650], p < 0.001).
  • Learning performance was moderated by type of course (Q_B = 64.249, p < 0.001; strongest for skills and competencies development, g = 0.874), learning model (Q_B = 76.220, p < 0.001; strongest for problem-based learning, g = 1.113, weakest for project-based learning, g = 0.239), and duration (Q_B = 55.998, p < 0.001; strongest at 4–8 weeks, g = 0.999). Grade level, role of ChatGPT, and application area were not significant.
  • Learning perception was moderated only by duration (Q_B = 19.839, p < 0.001), with >8 weeks showing the largest effect (g = 1.054) versus ≤1 week (g = 0.203).
  • Higher-order thinking was moderated by type of course (Q_B = 7.811, p < 0.05; STEM g = 0.737, language/writing g = 0.334, skills development g = 0.296) and role of ChatGPT (Q_B = 4.872, p < 0.05; intelligent tutor g = 0.945).

Methods

  • Design: meta-analysis following PRISMA guidelines, with literature search, inclusion/exclusion screening, coding of study characteristics, and data analysis; search covered Web of Science Core Collection, ScienceDirect, Taylor & Francis, Wiley Online Library, and Scopus, restricted to November 2022–February 2025, starting from 6,621 records and yielding 51 included articles.
  • Sample: 51 (quasi-)experimental studies with an experimental group using ChatGPT and a control group without it, both using the same learning method; 72 independent effect sizes were extracted. Interrater reliability for study selection was Cronbach's α = 0.96.
  • Analysis: Hedges's g computed in Comprehensive Meta-Analysis (3.0/3.3.0), interpreted via Sawilowsky's (2009) thresholds; random-effects models were used given heterogeneity (I² = 89.243% for learning performance, 72.580% for learning perception, 55.813% for higher-order thinking); publication bias assessed via funnel plots and fail-safe N (5490, 399, and 101 respectively), with one-study-removal sensitivity analysis.

Limitations

  • Stated by the authors: the sample of 51 documents included only one primary-school study and none at kindergarten level; only 19 and 9 studies examined learning perception and higher-order thinking respectively, versus a much larger sample for learning performance; only seven moderating variables were explored, omitting factors such as cultural background, parents' occupations, and national policies; and only experimental/quasi-experimental studies were included, excluding qualitative and mixed-methods research.
  • Evident from the text: heterogeneity was high for learning performance (I² = 89.243%), and the authors note some funnel-plot points fell outside the diagonal lines, indicating potential bias or heterogeneity; several moderator categories (e.g., mixed roles, intelligent partners for higher-order thinking, primary school) were excluded from analyses due to insufficient data.
  • Evident from the bibliographic record: the article is labeled "RETRACTED ARTICLE" in the title, though the retraction is not discussed in the full text provided.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1057/s41599-025-04787-yThis paper is marked retracted.

Cites

136 of 137 references verified1 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

68 Own result · 19 Own method · 17 Not checked yet · 6 No basis given · 5 Own interpretation

Verify all 28 references — free account

Show all 79 claims

  • It may state two separate claims as one. 30
  • The quote only partly states this claim. 1
  • Probably not a substantial claim. 14
36 of 36
  • The paper relies on this without support.

    ChatGPT is based on pretrained models that rely on existing data and patterns.

    “ChatGPT is based on pretrained models that rely on existing data and patterns.”
  • The paper relies on this without support.

    The meta-analysis by Chen et al. (2025) on other AI assessment tools is comparable to this meta-analysis on ChatGPT.

    “The results indicate that the effect size for the impact of ChatGPT on students' learning performance is significantly larger than that of other AI assessment tools (g = 0.390).”
2 of 2

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

5 of 5

Citations

Cites

Show all 134 references

Research works printed but not linked: 5

  1. Alsharif MHK, Elamin AY, Almasaad JM, Bakhit NM, Alarifi A, Taha KM, Hassan WA, Zumrawi E (2024) Using ChatGPT to create engaging problem-based learning scenarios in anatomy: a step-by-step guide. New Armen Med J 18(4). https://doi.org/10.56936/18290825-4.v18.2024-98 · DOI
  2. Lipsey MW, Wilson DB (2001) Practical meta-analysis. Sage Publications, Inc
  3. Luo H, Liao X, Ru Q, Wang Z (2024) Generative AI-supported teacher comments: an empirical study based on junior high school mathematics classrooms. e-Educ Res 45(5):58–66. https://doi.org/10.13811/j.cnki.eer.2024.05.008 · DOI
  4. Poola I, Božic V (2023) Guiding AI with human intuition for solving mathematical problems in ChatGPT. Int J Eng Sci Res 11(07):31–40. https://www.researchgate.net/publication/373447147
  5. Vygotskij LS, Hanfmann E (1966) Thought and language. MIT Press

Cited by

Sorted by citations

Items 401–463 of 463

Metadata

Authors
Wang, J., & Fan, W.
Affiliations
Hangzhou Normal University (CN)
Corresponding author
Fan, Wenxiang
Venue
Humanities and Social Sciences Communications
Published
2025-05-06
Volume
12
Issue
1
ISSN
  • 2055-1045 (ISSN-L)
  • 2662-9992
OpenAlex
W4410129808
Cited by
322
References
134
Citations by year
2026: 217 · 2025: 105
FWCI
48.98
Type
journal article
Language
en
Status
Retracted
Open access
Locations
  • Humanities and Social Sciences Communications (publishedVersion, cc-by-nc-nd)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Major Project of Humanities and Social Sciences Key Research Base of the Ministry of Educatio (22JJD880023)
  • University-Industry Collaborative Education Program (240904171111322)
  • Hangzhou Social Science Routine Planning Project (Z24JC014)
  • Hangzhou Normal University’s 2023 Teaching Construction and Reform Project “Exploration and Practice of Generative Artificial Intelligence to Promote University Teaching Reform”
Topics
  • Social Sciences › Education › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
ChatGPT, generative AI in education, learning performance, perceptions of learning, higher-order thinking skills, AI-assisted learning, problem-based learning, Bloom's taxonomy, instructional scaffolding, intelligent tutoring systems, moderating effect

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review