CoChat
Sign Up
Cited by 37Artificial Intelligence in EducationEducation and Critical Thinking DevelopmentArtificial Intelligence in Healthcare and Education

Does Generative Artificial Intelligence Improve Students’ Higher-Order Thinking? A Meta-Analysis Based on 29 Experiments and Quasi-Experiments

Open access
Zhao, Yan, Yue, Yuhe, Sun, Zhonghua, Jiang, Qiang, Li, Gangsheng·2025·Journal of Intelligence
DOI 10.3390/jintelligence13120160PMC PMC12734368
Open this paper in CoChat — free accountOpen PDF

Abstract

The widespread application of Generative Artificial Intelligence (Gen-AI) is transforming educational practices and driving pedagogical innovation. While cultivating higher-order thinking (HOT) represents a central educational goal, its achievement remains an ongoing challenge. Current evidence regarding the impact of Gen-AI on HOT is relatively fragmented, lacking systematic integration, particularly in the analysis of moderating variables. To address this gap, a meta-analysis approach was employed, integrating data from 29 experimental and quasi-experimental studies to quantitatively assess the overall impact of Gen-AI on learners’ HOT and to examine potential moderating factors. The analysis revealed that Gen-AI exerts a moderate positive effect on HOT, with the most significant improvement observed in problem-solving abilities, followed by critical thinking, while its effect on creativity is relatively limited. Moderation analyses further indicated that the impact of Gen-AI is significantly influenced by experimental duration and learners’ self-regulated learning (SRL) abilities: effects were strongest when interventions lasted 8–16 weeks, and learners with higher SRL capacities benefited more substantially. Based on the research findings, this study proposed that Gen-AI should be systematically integrated as a targeted instructional tool to foster HOT. Medium- to long-term interventions (8–16 weeks) are recommended to enhance learners’ problem-solving and critical thinking abilities. At the same time, effective approaches should also be explored to promote creative thinking through Gen-AI within existing pedagogical frameworks. Furthermore, individual learner differences should be accounted for by adopting dynamic and personalized scaffolding strategies to foster SRL, thereby maximizing the educational potential of Gen-AI in cultivating innovative talents.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3390/jintelligence13120160

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • GenAI exerts a moderate positive effect on higher-order thinking, with the strongest gains for problem-solving and critical thinking and weaker effects for creativity.

    “Metaanalytic evidence further suggests that GenAI exerts a moderate positive effect on higher-order thinking, with the strongest gains observed for problem-solving and critical thinking, and weaker effects for creativity (Zhao et al., 2025).”

    From <b>Effects of Partial vs. Full Cognitive Offloading with Generative AI on Learning Retention </b> (2026) · cites this paper as Zhao et al., 2025

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

1 retracted

Flagged first, then by citations

Research works printed but not linked: 55

  1. Adijaya Made Aryawan Widiana I Wayan Parwata I Gusti Lanang Agung Antara I Gede Wahyu Suwela Bloom’s taxonomy revision-oriented learning activities to improve procedural capabilities and learning outcomes International Journal of Educational Methodology 2023 9 261 70 10.12973/ijem.9.1.261 · DOI
  2. Alkhatib Omar J. An Effective Assessment Method of Higher-Order Thinking Skills (Problem-Solving, Critical Thinking, Creative Thinking, and Decision-Making) in Engineering and Humanities Paper presented at 2022 Advances in Science and Engineering Technology International Conferences (ASET) Dubai, United Arab Emirates February 21–24 2022 1 6 10.1109/ASET53988.2022.9734856 · DOI
  3. Alzubi Ali Abbas Falah Nazim Mohd Alyami Naji Do AI-generative tools kill or nurture creativity in EFL teaching and learning? Education and Information Technologies 2025 30 15147 84 10.1007/s10639-025-13409-8 · DOI · Paper page
  4. Anderson Lorin W. Krathwohl David R. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives: Complete Edition Addison Wesley Longman, Inc. Albany 2001 Available online: http://eduq.info/xmlui/handle/11515/18824 (accessed on 6 November 2015)
  5. Arista Artika Shuib Liyana Ismail Maizatul Akmar A glimpse of ChatGPT: An introduction to features, challenges, and threats in higher education Paper presented at Informatics, Multimedia, Cyber and Information Systems Conference Jakarta Selatan, Indonesia November 7–8 2023 694 98 10.1109/ICIMCIS60089.2023.10349057 · DOI
  6. Borge M. Smith B. K. Aldemir T. Using generative AI as a simulation to support higher-order thinking International Journal of Computer-Supported Collaborative Learning 2024 19 479 532 10.1007/s11412-024-09437-0 · DOI
  7. Bower Matt Vlachopoulos Panos A critical analysis of technology—Enhanced learning design frameworks British Journal of Educational Technology 2018 49 981 97 10.1111/bjet.12668 · DOI
  8. Cam Emre Kiyici Mübin The impact of robotics-assisted programming education on academic success, problem-solving skills, and motivation Journal of Educational Technology and Online Learning 2022 5 47 65 10.31681/jetol.1028825 · DOI
  9. Chiu Thomas K. F. Moorhouse Benjamin Luke Chai Ching Sing Ismailov Murod Teacher support and student motivation to learn with Artificial Intelligence (AI) based chatbot Interactive Learning Environments 2023 32 3240 56 10.1080/10494820.2023.2172044 · DOI · Paper page
  10. Costa Alexandra R. Lima Natércia Viegas Clara Caldeira Amélia Critical minds: Enhancing education with ChatGPT Cogent Education 2024 11 2415286 10.1080/2331186X.2024.2415286 · DOI
  11. Deng Ruiqi Jiang Maoli Yu Xinlu Lu Yuyan Liu Shasha Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies Computers & Education 2025 227 105224 10.1016/j.compedu.2024.105224 · DOI · Paper page
  12. Ganesh D. Kumar M. Sunil Reddy P. Venkateswarlu Sadam Kavitha Murthy D. Sudarsana Implementation of AI pop bots and its allied applications for designing efficient curriculum in early childhood education International Journal of Early Childhood Special Education 2022 14 2282 92 10.9756/INT-JECSE/V14I3.271 · DOI
  13. García-Martínez Inmaculada Fernández-Batanero José María Fernández-Cerero José León Samuel P. Analysing the impact of artificial intelligence and computational sciences on student performance: Systematic review and meta-analysis Journal of New Approaches in Educational Research 2023 12 171 97 10.7821/naer.2023.1.1240 · DOI
  14. Gonsalves Chahna Generative AI’s Impact on Critical Thinking: Revisiting Bloom’s Taxonomy Journal of Marketing Education 2024 1 1 16 10.1177/02734753241305980 · DOI · Paper page
  15. Guo Kai Zhong Yuchun Li Danling Chu Samuel Kai Wah Effects of chatbot-assisted in-class debates on students’ argumentation skills and task motivation Computers & Education 2023 10 1 19 10.1016/j.compedu.2023.104862 · DOI · Paper page
  16. Heung Yuk Mui Elly Chiu Thomas K.F. How ChatGPT impacts student engagement from a systematic review and meta-analysis study Computers and Education: Artificial Intelligence 2025 8 100361 10.1016/j.caeai.2025.100361 · DOI
  17. Higgins Julian P. T. Thompson Simon G. Deeks Jonathan J. Altman Douglas G. Measuring inconsistency in meta-analyses The BMJ 2003 327 557 60 10.1136/bmj.327.7414.557 12958120 PMC192859 · DOI · Paper page
  18. Huang Kuo-Liang Liu Yi-Chen Dong Ming-Qing Incorporating AIGC into design ideation: A study on self-efficacy and learning experience acceptance under higher-order thinking Thinking Skills and Creativity 2024a 52 101508 10.1016/j.tsc.2024.101508 · DOI
  19. Huang Yueh-Min Wang Wei-Sheng Lee Hsin-Yu Lin Chia-Ju Wu Ting-Ting Empowering virtual reality with feedback and reflection in hands-on learning: Effect of learning engagement and higher-order thinking Journal of Computer Assisted Learning 2024b 40 1413 27 10.1111/jcal.12959 · DOI
  20. Hwang Gwo-Jen Lai Chiu-Lin Liang Jyh-Chong Chu Hui-Chun Tsai Chin-Chung A long-term experiment to investigate the relationships between high school students’ perceptions of mobile learning and peer interaction and higher-order thinking tendencies Educational Technology Research and Development 2018 66 75 93 10.1007/s11423-017-9540-3 · DOI · Paper page
  21. Hwang Sunghwan Examining the effects of artificial intelligence on elementary students’ mathematics achievement: A meta-Analysis Sustainability 2022 14 13185 10.3390/su142013185 · DOI
  22. Ilgun Dibek Munevver Kursad Merve Sahin Erdogan Tolga Influence of artificial intelligence tools on higher order thinking skills: A meta-analysis Interactive Learning Environments 2024 33 2216 38 10.1080/10494820.2024.2402028 · DOI · Paper page
  23. Johinke Rebecca Cummings Robert Di Lauro Frances Reclaiming the technology of higher education for teaching digital writing in a post—Pandemic world Journal of University Teaching & Learning Practice 2023 20 01 10.53761/1.20.02.01 · DOI
  24. Khan Talha Ahmed Alam Muhammad Rizvi Safdar Ali Shahid Zeeshan Mazliham M. S. Introducing AI applications in engineering education (PBL): An implemen-tation of power generation at minimum wind velocity and turbine faults classification using AI Computer Applications in Engineering Education 2023 32 438 10.1002/cae.22691 · DOI
  25. Kimberly A. Riegel Project-based learning (PBL): Less project, more problem The Journal of the Acoustical Society of America 2023 152 272 10.1121/10.0018820 · DOI
  26. Kimmel Sara AI Facilitated Critical Thinking in an Undergraduate Project Based Service-Learning Course Journal of Behavioral and Applied Management 2024 24 123 30 10.21818/001c.122149 · DOI
  27. Lewis Arthur Smith David Defining higher order thinking Theory into Practice 1993 32 131 37 10.1080/00405849309543588 · DOI
  28. Li Haifeng Wang Wei Human-machine collaborative in-depth inquiry teaching model: The case of ChatGPT and QQ-based collaborative inquiry learning system Open Education Research 2023 29 69 81 10.13966/j.cnki.kfjyyj.2023.06.008 · DOI
  29. Li Haifeng Wang Wei Human-machine debate inquiry method: A study on using debate-style intelligent dialogue robots to foster students’ higher-order thinking skills E-education Research 2024 45 106 12+128 10.13811/j.cnki.eer.2024.03.015 · DOI
  30. Li Haifeng Wang Wei Li Guangxin Wang Yuan Smart midwifery teaching method: The practice of “smart Socratic dialogue robot” teaching Open Education Research 2024a 30 89 99 10.13966/j.cnki.kfjyyj.2024.02.010 · DOI
  31. Li Pin-Hui Lee Hsin-Yu Lin Chia-Ju Wang Wei-Sheng Huang Yueh-Min InquiryGPT: Augmenting ChatGPT for enhancing inquiry-based learning in STEM education Journal of Educational Computing Research 2024b 62 1 30 10.1177/07356331241289824 · DOI
  32. Li Yan Xu Jie Jia Chengyuan Zhai Xuesong The current status and reflections on the application of generative artificial intelligence among university students: A case study from Zhejiang University Open Education Research 2024c 30 89 98 10.13966/j.cnki.kfjyyj.2024.01.010 · DOI
  33. Liu Jun Liu Zile Wang Cong Xu Yanhua Chen Jiayu Cheng Yichun K-12 students’ higher-order thinking skills: Conceptualization, components, and evaluation indicators Thinking Skills and Creativity 2024a 52 101551 10.1016/j.tsc.2024.101551 · DOI
  34. Liu Ming Guo Shuo Wu Zhongming Liao Jian Generative artificial intelligence reshaping the landscape of higher education: Content, cases, and pathways E-educational Research 2024b 45 57 65 10.13811/j.cnki.eer.2024.06.007 · DOI
  35. Liu Yangxia Connotation and cultivation of mathematical advanced thinking ability Creative Education Studies 2022 10 1294 300 10.12677/CES.2022.106208 · DOI
  36. Lu Kaili Zhu Jianrong Pang Feng Shadiev Rustam Understanding the relationship between college students’ artificial intelligence literacy and higher-order thinking skills using the 3P model: The mediating roles of behavioral engagement and peer interaction Educational Technology Research and Development 2024 72 10434 10.1007/s11423-024-10434-1 · DOI
  37. Luo Gongbo Niu Changsong Lu Lu Wu Ling Huang Lin Empirical study on the double-edged effects of generative artificial intelligence on team creativity among university students Education and Information Technologies 2025 30 18853 78 10.1007/s10639-025-13527-3 · DOI
  38. Maniktala Mehak Chi Min Barnes Tiffany Enhancing a student productivity model for adaptive problemsolving assistance User Modeling and User-Adapted Interaction 2023 33 159 88 10.1007/s11257-022-09338-7 35966028 PMC9362072 · DOI
  39. OECD Fostering Students’ Creativity and Critical Thinking: What it Means in School 2019 Available online: https://read.oecd-ilibrary.org/education/fostering-students-creativity-and-critical-thinking_62212c37-en (accessed on 24 October 2019)
  40. Pujawan I. G. N. Rediani N. N. Antara I. G. W. S. Putri N. N. C. A. Bayu G. W. Revised Bloom taxonomy-oriented learning activities to develop scientific literacy and creative thinking skills Jurnal Pendidikan IPA Indonesia 2022 11 47 60 10.15294/jpii.v11i1.34628 · DOI
  41. Putra Febri W. Rangka Itsar B. Aminah Siti Aditama Mint. H. R. ChatGPT in the higher education environment: Perspectives from the theory of higher-order thinking skills Journal of Public Health 2023 45 840 41 10.1093/pubmed/fdad120 37455540 · DOI
  42. Qi Jia Xu Yanru Liu Ji’an Xue Kai The impact of generative AI tools on critical thinking and independent learning ability in university students E-Education Research 2024 45 67 74 10.13811/j.cnki.eer.2024.12.009 · DOI
  43. Shi Wanruo Han Xibin The impact of generative artificial intelligence on learning analytics research: Current status and future directions—A review of LAK24 E-Educational Research 2024 45 113 20 10.13811/j.cnki.eer.2024.12.015 · DOI
  44. Sun Dan Zhu Chengcong Xu Zuodong Xu Guangtao Analysis of college students’ programming learning behavior based on generative AI E-Education Research 2024 45 113 20 10.13811/j.cnki.eer.2024.03.016 · DOI
  45. Tan Shuxin Cho Young Woo Towards a higher order thinking skills—Oriented translation competence model Translation, Cognition & Behavior 2021 4 187 211 10.1075/tcb.00053.tan · DOI
  46. Van der Willik Esmee M. van Zwet Erik W. Hoekstra Tiny van Ittersum Frans J. Hemmelder Marc H. Zoccali Carmine Jager Kitty J. Dekker Friedo W. Meuleman Yvette Funnel plots of patient-reported outcomes (PROs) to evaluate healthcare quality: Basic principles, pitfalls and considerations Nephrology 2021 26 95 104 10.1111/nep.13761 32725679 PMC7891340 · DOI
  47. Wang Siyao Huang Yating Facilitation or Inhibition: The Impact of Generative Artificial Intelligence on College Students’ Creativity China Higher Education Research 2024 11 29 36 10.16298/j.cnki.1004-3667.2024.11.05 · DOI
  48. Wang Yanbei Liu Liping Learning elements for developing higher-order thinking in a blended learning environment: A comprehensive survey of Chinese vocational high school students Education and Information Technologies 2024 29 19443 70 10.1007/s10639-024-12572-8 · DOI
  49. Wang Yan Lavonen Jari Tirri Kirsi Twenty-first century competencies in the Chinese science curriculum Nordic-Chinese Intersections Within Education. Palgrave Studies on Chinese Education in a Global Perspective Liu Haiqin Dervin Fred Du Xiangyun Palgrave Macmillan London 2019 151 71
  50. Xu Yeqing Zhu Jingdong Wang Minkai Qian Fang Yang Yiling Zhang Jie The impact of a digital game-based AI chatbot on students’ academic performance, higher-order thinking, and behavioral patterns in an information technology curriculum Applied Sciences 2024 14 6418 10.3390/app14156418 · DOI · Paper page
  51. Yang Ya-Ting Carolyn Virtual CEOs: A blended approach to digital gaming for enhancing higher-order thinking and academic achievement among vocational high school students Computers & Education 2015 81 281 95 10.1016/j.compedu.2014.10.004 · DOI
  52. Zawacki-Richter Olaf Marín Victoria I. Bond Melissa Gouverneur Franziska Systematic review of research on artificial intelligence applications in higher education—Where are the educators? International Journal of Educational Technology in Higher Education 2019 16 1 27 10.1186/s41239-019-0171-0 · DOI · Paper page
  53. Zhang Wenlan Hu Jiao Has project-based learning taken effect? A meta-analysis based on 46 experimental and quasi-experimental studies Audio-Visual Education Research 2019 2 95 104 10.13811/j.cnki.eer.2019.02.012 · DOI
  54. Zhao Guoqing Sheng Haixi Wang Yaxuan Cai Xiaohui Long Taotao Generative artificial intelligence amplifies the role of critical thinking skills and reduces reliance on prior knowledge while promoting in-depth learning Education Sciences 2025 15 554 10.3390/educsci15050554 · DOI
  55. Zheng Lanqin Gao Lei Huang Zichen Can generative AI-powered dialogue robots promote online collaborative learning performance? E-Education Research 2024 45 70 84 70–76+84 10.13811/j.cnki.eer.2024.03.010 · DOI

Cited by

Sorted by citations

Authors
Zhao, Y., Yue, Y., Sun, Z., Jiang, Q., & Li, G.
Affiliations
  • Changchun Normal University (CN)
  • Northeast Normal University (CN)
  • Ocean University of China (CN)
Corresponding author
  • Sun, Zhonghua
  • Li, Gangsheng
ORCID
Jiang, Qiang
Venue
Journal of Intelligence
Publisher
MDPI AG
Published
2025-12-05
Volume
13
Issue
12
Pages
160
ISSN
2079-3200 (ISSN-L)
DOI
10.3390/jintelligence13120160
PMC
PMC12734368
OpenAlex
W4417045567
Cited by
37
References
18
Citations by year
2026: 37
FWCI
5.69
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Intelligence (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Planning Project for Humanities and Social Sciences of the Ministry of Education (23YJA880023)
  • Jilin Province Educational Science Project (GH25412)
  • Project of the Jilin Provincial Higher Education Society (JGJX2025C107)
  • Social Science Foundation Projects of CCNU (CSJJ2022019SK)
  • National Natural Science Foundation of China (62577018)
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Social Sciences › Education › Education and Critical Thinking Development
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
generative AI, higher-order thinking skills, problem solving, critical thinking, creativity, student self-regulation, moderators, intervention duration, artificial intelligence in education

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review