Evaluation of Problem Based Gamification Learning (PBGL) Model on Critical Thinking Ability with Artificial Intelligence Approach Integrated with ChatGPT API: An Experimental Study
Open accessAbstract
The purpose of this research is to evaluate the effectiveness of Problem Based Gamification Learning (PBGL) model integrated with Artificial Intelligence (AI) through ChatGPT API in improving participants' critical thinking skills. The study used an experimental design with two groups, namely control and experimental, each consisting of 120 students (240 in total) from two countries (Timor Leste and Indonesia) studying basic Python programming. The main issues were complex system integration with ChatGPT API, technical challenges in AI implementation for automated feedback, and maintaining test consistency and fairness. Results showed the experimental group had a significant improvement in critical thinking skills, with an average score improvement of 23.57 points compared to 9.04 points in the control group, showing a mean difference of 14.53 points. Content Validity Ratio (CVR) testing showed good content validity with CVR values > 0.75 for all items. Construct Validity (CV) testing confirmed the instrument measured the desired construct with significant correlation coefficients. Test-retest Reliability testing showed excellent reliability with a Pearson correlation coefficient of 0.960. The findings confirm that the integration of AI in gamified learning can provide automatic feedback and additional relevant information, improving critical thinking and motivation of participants. The contribution of the research is the development and testing of an AI-supported PBGL model, which can serve as a reference in digital learning curriculum design and improve critical thinking skills. Practical implications show that the integration of AI in learning enriches the material and provides flexibility to participants and teachers. Overcoming the technical challenges and adaptability of AI models will be key to increasing the effectiveness of PBGL in various educational contexts in the future. This research provides new insights in the development of more effective and adaptive learning methods in the digital era.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AI tools supplied exemplar feedback, evaluation criteria, alternative solutions, brainstorming prompts, or code suggestions that students were required to analyze, critique, or revise.
“In these interventions, AI tools supplied exemplar feedback, evaluation criteria, alternative solutions, brainstorming prompts, or code suggestions that students were required to analyze, critique, or revise [13,50,71,93].”
From A Systematic Literature Review on the Pedagogical Implications and Impact of GenAI on Students' Critical Thinking (2026) · cites this paper as [50]
The Research Design and Participant Criteria section says ChatGPT provided explanations, correction suggestions, additional insights, and programming code examples that students could use to improve their understanding or correct answers. It does not say students were required to analyze, critique, or revise the AI output; it describes their use of it as optional. The paper also does not establish the full list of output types in the claim. (LLM, conf=0.86)
Experimental — accuracy not yet measuredA few studies developed custom GPT-based systems.
“Other tools such as Gemini, Ernie Bot, and multi-tool workflows were less common [61,70], and a few studies developed custom GPT-based systems ( [50,59,84,89,90]).”
From A Systematic Literature Review on the Pedagogical Implications and Impact of GenAI on Students' Critical Thinking (2026) · cites this paper as [50]
In project-based contexts, GenAI was embedded across multiple stages of project development, including idea generation, planning, analysis, and drafting.
“In project-based contexts, GenAI was embedded across multiple stages of project development, including idea generation, planning, analysis, and drafting [50,59].”
From A Systematic Literature Review on the Pedagogical Implications and Impact of GenAI on Students' Critical Thinking (2026) · cites this paper as [50]
The Research Design and Participant Criteria section describes ChatGPT providing explanations, answer corrections, and programming-code examples in a problem-based learning system. The Gamification Model Architecture section describes students answering leveled questions and receiving feedback. Our copy does not describe project-based work or GenAI use across idea generation, planning, analysis, and drafting; searches for those stages and project development found no supporting passage. Page numbers were not available in the indexed passages. (LLM, conf=0.76)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-019-0171-0 · Cited by 7,242 · cited 1× · see citing works
“This is reinforced by research by Zawacki-Richter et al. (2019) which showed that critical thinking skills allow participants to evaluate information objectively, crucial in today's digital age where false information can easily spread [34].”
BackgroundArtificial Intelligence in Education: A Review
Chen, L., Chen, P., & Lin, Z. (2020) · IEEE Access · doi:10.1109/access.2020.2988510 · Cited by 4,133 · cited 1× · see citing works
“A.I. is an intelligent technology capable of interacting with the environment and simulating human intelligence [76] and has adaptation and decision-making capabilities [77] A.I. systems have been applied and developed in society, government, industry, and academia [78-80].”
BackgroundFunctional Classification and Experimental Dissection of Long Noncoding RNAs
Kopp, F. B., & Mendell, J. T. (2018) · Cell · doi:10.1016/j.cell.2018.01.011 · Cited by 3,522 · see citing works
Human trust in artificial intelligence: Review of empirical research
Glikson, E., & Woolley, A. W. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0057 · Cited by 2,921 · cited 1× · see citing works
“A.I. is an intelligent technology capable of interacting with the environment and simulating human intelligence [76] and has adaptation and decision-making capabilities [77] A.I. systems have been applied and developed in society, government, industry, and academia [78-80].”
BackgroundSailer, M., Hense, J. U., Mayr, S. K., & Mandl, H. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2016.12.033 · Cited by 2,064 · see citing works
Gamifying education: what is known, what is believed and what remains uncertain: a critical review
Dichev, C., & Dicheva, D. (2017) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-017-0042-5 · Cited by 1,451 · cited 2× · see citing works
“Dichev and Dicheva (2017) showed that game elements such as points, badges and leaderboards can make the learning process more interesting and challenging for students [11].”
BackgroundShi, F., Wang, J., Shi, J., Wu, Z., Wang, Q., Tang, Z., He, K., Shi, Y., & Shen, D. (2021) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2020.2987975 · Cited by 1,404 · see citing works
Study on artificial intelligence: The state of the art and future prospects
Zhang, C., & Lu, Y. (2021) · Journal of Industrial Information Integration · doi:10.1016/j.jii.2021.100224 · Cited by 1,240 · cited 1× · see citing works
“A.I. is an intelligent technology capable of interacting with the environment and simulating human intelligence [76] and has adaptation and decision-making capabilities [77] A.I. systems have been applied and developed in society, government, industry, and academia [78-80].”
BackgroundSelf-Directed Learning in Problem-Based Learning and its Relationships with Self-Regulated Learning
Loyens, S. M. M., Magda, J., & Rikers, R. M. J. P. (2008) · Educational Psychology Review · doi:10.1007/s10648-008-9082-7 · Cited by 882 · cited 1× · see citing works
“Sofie et al. (2017) added that PBL teaches students to learn independently and develop time management skills that are essential for success in higher education [40].”
BackgroundChan, K. S., & Zary, N. (2019) · JMIR Medical Education · doi:10.2196/13930 · Cited by 765 · cited 1× · see citing works
“Chan and Zary (2019) showed that A.I. can impact personalized feedback and more efficient learning support [52].”
BackgroundGamification of e-learning in higher education: a systematic literature review
Khaldi, A., Bouzidi, R., & Nader, F. (2023) · Smart Learning Environments · doi:10.1186/s40561-023-00227-z · Cited by 575 · cited 1× · see citing works
“Therefore, further research is needed to explore the potential of ChatGPT API in creating a more interactive, adaptive, and personalized learning environment, and to assess its effectiveness in improving students' engagement, motivation, and learning outcomes in the long term [63] .”
BackgroundComparing the quality of human and ChatGPT feedback of students’ writing
Steiss, J., Tate, T., Graham, S., Cruz, J., Hebert, M., Wang, J., Moon, Y., Tseng, W., Warschauer, M., & Olson, C. B. (2024) · Learning and Instruction · doi:10.1016/j.learninstruc.2024.101894 · Cited by 509 · cited 1× · see citing works
“[89, 90].”
Smiderle, R., Rigo, S. J., Marques, L. B., Peçanha de Miranda Coelho, J. A., & Jaques, P. A. (2020) · Smart Learning Environments · doi:10.1186/s40561-019-0098-x · Cited by 502 · cited 1× · see citing works
“[51].”
Artificial Intelligence in Drug Discovery and Development
Agrawal, P. (2018) · Journal of Pharmacovigilance · doi:10.4172/2329-6887.1000e173 · Cited by 493 · cited 1× · see citing works
“A.I. is an intelligent technology capable of interacting with the environment and simulating human intelligence [76] and has adaptation and decision-making capabilities [77] A.I. systems have been applied and developed in society, government, industry, and academia [78-80].”
BackgroundGamification applications in E-learning: a literature review
Saleem, A. N., Noori, N. M., & Ozdamli, F. (2022) · Technology, Knowledge and Learning · doi:10.1007/s10758-020-09487-x · Cited by 478 · see citing works
van den Berg, G., & du Plessis, E. (2023) · Education Sciences · doi:10.3390/educsci13100998 · Cited by 401 · cited 1× · see citing works
“…researchers Berg and Plessis (2023) recommended the need for further research to explore the practice of ChatGPT integration in lesson planning [60], as well as research by Lopez et.al which provides advice on the continuation of the utilization of ChatGPT in the critical thinking dimension…”
BackgroundCan Large Language Models Provide Feedback to Students? A Case Study on ChatGPT
Dai, W., Lin, J., Jin, H., Li, T., Tsai, Y.-S., Gašević, D., & Chen, G. (2023) · 2023 IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt58122.2023.00100 · Cited by 395 · cited 1× · see citing works
“[89, 90].”
Dahalan, F., Alias, N., & Shaharom, M. S. N. (2023) · Education and Information Technologies · doi:10.1007/s10639-022-11548-w · Cited by 344 · cited 1× · see citing works
“In Asia, since 2020, gamification has focused on vocational schools, technical colleges, and healthcare [44].”
BackgroundTailored gamification in education: A literature review and future agenda
Oliveira, W., Hamari, J., Shi, L., Toda, A. M., Rodrigues, L., Palomino, P. T., & Isotani, S. (2023) · Education and Information Technologies · doi:10.1007/s10639-022-11122-4 · Cited by 341 · cited 2× · see citing works
“Research on using A.I. to analyze learner behavior and customize gamification elements shows great potential [12].”
BackgroundCritical Thinking as Discourse
Kuhn, D. (2019) · Human Development · doi:10.1159/000500171 · Cited by 265 · cited 1× · see citing works
“According to Kuhn (2019), critical thinking is a dialogical activity carried out by individuals, first interactively, then in a form that is internalized with others only implicitly [30].”
BackgroundRuiz-Rojas, L. I., Salvador-Ullauri, L., & Acosta-Vargas, P. (2024) · Sustainability · doi:10.3390/su16135367 · Cited by 263 · see citing works
Leveraging ChatGPT for Enhancing Critical Thinking Skills
Guo, Y., & Lee, D. (2023) · Journal of Chemical Education · doi:10.1021/acs.jchemed.3c00505 · Cited by 250 · cited 1× · see citing works
“[88].”
Mughairi, H. A., & Bhaskar, P. (2024) · Journal of Research in Innovative Teaching & Learning · doi:10.1108/jrit-09-2023-0129 · Cited by 237 · cited 1× · see citing works
“However, its capabilities are limited to text-based questions only [86].”
BackgroundJia, X.-H., & Tu, J.-C. (2024) · Systems · doi:10.3390/systems12030074 · Cited by 218 · cited 1× · see citing works
“There are also studies that assess academic integrity and learner attitudes towards using A.I. technologies [28], as well as looking at how A.I. affects critical thinking, motivation, and self-awareness [29].”
BackgroundCastonguay, A., Farthing, P., Davies, S., Vogelsang, L., Kleib, M., Risling, T., & Green, N. (2023) · Nurse Education Today · doi:10.1016/j.nedt.2023.105916 · Cited by 203 · cited 1× · see citing works
“This personalization ensures that learners stay engaged and get the tailored support they need to overcome barriers to learning and improve critical thinking skills [17, 18].”
BackgroundAdaptive gamification in E‐learning: A literature review and future challenges
Bennani, S., Maalel, A., & Ghézala, H. B. (2021) · Computer Applications in Engineering Education · doi:10.1002/cae.22477 · Cited by 198 · cited 1× · see citing works
“Alongside the development of gamification, the arrival of artificial intelligence in education has also opened up new opportunities for more personalized, intelligent, and automated learning experiences [13, 14].”
BackgroundUsing gamification to support learning in K‐12 education: A systematic literature review
Dehghanzadeh, H., Farrokhnia, M., Dehghanzadeh, H., Taghipour, K., & Noroozi, O. (2023) · British Journal of Educational Technology · doi:10.1111/bjet.13335 · Cited by 193 · cited 1× · see citing works
“Gamification is more suitable for use in problem solving [45] whose impact can improve the learning experience that was previously lacking and needs to be addressed again [46].”
BackgroundExploring the impact of ChatGPT on education: A web mining and machine learning approach
Rejeb, A., Rejeb, K., Appolloni, A., Treiblmaier, H., & Iranmanesh, M. (2024) · The International Journal of Management Education · doi:10.1016/j.ijme.2024.100932 · Cited by 181 · cited 1× · see citing works
“I can customize lesson content to suit individual needs, respond to learner progress, and provide feedback that helps them think more critically [16].”
BackgroundGamification in Education: Why, Where, When, and How?—A Systematic Review
Zeybek, N., & Saygı, E. (2024) · Games and Culture · doi:10.1177/15554120231158625 · Cited by 172 · see citing works
Gkintoni, E., Vantaraki, F., Skoulidi, C., Anastassopoulos, P., & Vantarakis, A. (2024) · Behavioral Sciences · doi:10.3390/bs14020102 · Cited by 137 · cited 1× · see citing works
“…academic performance, engagement, and motivation of learners [69] as well as having a psychological and intrinsic effect on participants [70, 71] gamification can be implemented in a variety of ways such as in focus group testing, online, utilizing software to developing concepts into the…”
BackgroundLi, T., Ji, Y., & Zhan, Z. (2024) · Asia Pacific Journal of Education · doi:10.1080/02188791.2024.2305163 · Cited by 133 · cited 1× · see citing works
“[87] in various programming topics, including Python, Java, and Web Programming Languages.”
BackgroundCrespí, P., García-Ramos, J. M., & Queiruga-Dios, M. (2022) · Journal of New Approaches in Educational Research · doi:10.7821/naer.2022.7.993 · Cited by 125 · cited 1×
“PBL also allows students to work in groups, which strengthens their social and communication skills, as well as the ability to work together in teams to achieve a common goal [38, 39].”
BackgroundAfzaal, M., Nouri, J., Zia, A., Papapetrou, P., Fors, U., Wu, Y., Li, X., & Weegar, R. (2021) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2021.723447 · Cited by 125 · see citing works
Rusandi, M. A., Ahman, Saripah, I., Khairun, D. Y., & Mutmainnah (2023) · Journal of Public Health · doi:10.1093/pubmed/fdad049 · Cited by 115 · cited 1× · see citing works
“The difference between the current research and previous research is more focused on the use of gamification to improve students' critical thinking skills in dealing with misinformation and fake news [19, 20].”
BackgroundNaidu, K., & Sevnarayan, K. (2023) · Online Journal of Communication and Media Technologies · doi:10.30935/ojcmt/13291 · Cited by 112 · cited 1× · see citing works
“In addition, other studies highlighted that A.I., including ChatGPT, has great potential in providing personalized feedback in the context of distance education [54].”
BackgroundThe dark side of the metaverse: The role of gamification in event virtualization
Flavián, C., Ibáñez‐Sánchez, S., Orús, C., & Barta, S. (2023) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2023.102726 · Cited by 94 · cited 1× · see citing works
“…performance, engagement, and motivation of learners [69] as well as having a psychological and intrinsic effect on participants [70, 71] gamification can be implemented in a variety of ways such as in focus group testing, online, utilizing software to developing concepts into the metaverse [72-75].”
BackgroundAisha: A Custom AI Library Chatbot Using the ChatGPT API
Lappalainen, Y., & Narayanan, N. (2023) · Journal of Web Librarianship · doi:10.1080/19322909.2023.2221477 · Cited by 89 · see citing works
Sotos-Martínez, V. J., Férriz-Valero, A., Martínez, S. G., & Tortosa‐Martínez, J. (2022) · Physical Education and Sport Pedagogy · doi:10.1080/17408989.2022.2039611 · Cited by 86 · cited 1× · see citing works
“…improve academic performance, engagement, and motivation of learners [69] as well as having a psychological and intrinsic effect on participants [70, 71] gamification can be implemented in a variety of ways such as in focus group testing, online, utilizing software to developing concepts into…”
BackgroundStambuk‐Castellano, M., Contreras‐McKay, I., Neyem, A., Farfán, E., Inzunza, Ó., Ottone, N. E., del Sol, M., Alario‐Hoyos, C., Alvarado, M. S., & Tubbs, R. S. (2023) · Clinical Anatomy · doi:10.1002/ca.24074 · Cited by 73 · cited 1× · see citing works
“I, gamification approaches will be more adaptive [47].”
BackgroundDeveloping critical thinking skills through gamification
Angelelli, C. V., Ribeiro, G. M. D. C., Severino, M. R., Johnstone, E., Borzenkova, G., & da Silva, D. C. O. (2023) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2023.101354 · Cited by 67 · cited 1× · see citing works
“The difference between the current research and previous research is more focused on the use of gamification to improve students' critical thinking skills in dealing with misinformation and fake news [19, 20].”
BackgroundBachiri, Y.-A., Mouncif, H., & Bouikhalene, B. (2023) · International Journal of Engineering Pedagogy (iJEP) · doi:10.3991/ijep.v13i8.40853 · Cited by 59 · cited 1× · see citing works
“This approach can make assessment more fun and impactful to learning, providing valuable feedback for educators to refine their pedagogical strategies [48] to mental health which holds great promise for improving one's emotional and psychological well-being [49].”
BackgroundBayesian Critical Values for Lawshe's Content Validity Ratio
Baghestani, A., Ahmadi, F., Tanha, A., et al. (2019) · doi:10.1080/07481756.2017.1308227 · Cited by 52 · cited 1×
“As for some of the tests used to test the validity and reliability of the data, namely; content validity test using lawshe technique (CVR) which evaluates the extent to which the items in the instrument cover all aspects measured by involving 8 experts [67].”
MethodPredicting Everyday Critical Thinking: A Review of Critical Thinking Assessments
Butler, H. A. (2024) · Journal of Intelligence · doi:10.3390/jintelligence12020016 · Cited by 50 · see citing works
Heliawati, L., Lidiawati, L., & Pursitasari, I. D. (2022) · International Journal of Evaluation and Research in Education (IJERE) · doi:10.11591/ijere.v11i3.22168 · Cited by 50 · cited 1× · see citing works
“[21], as well as software with gamification elements such as points, levels and scores designed to motivate and engage learners [22, 23].”
BackgroundNotes to Factor Analysis Techniques for Construct Validity
Alavi, M., Biroš, E., & Cleary, M. L. (2023) · Canadian Journal of Nursing Research · doi:10.1177/08445621231204296 · Cited by 45 · cited 1× · see citing works
“While reliability testing uses test-retest reliability to evaluate the consistency of the instrument in producing stable scores over time [68].”
MethodAlahmari, M., Jdaitawi, M. T., Rasheed, A., Abduljawad, R. M., Hussein, E. T., Alzahrani, M. A., & Awad, N. (2023) · Contemporary Educational Technology · doi:10.30935/cedtech/13177 · Cited by 43 · cited 1× · see citing works
“This finding is in line with research by Alahmari (2020) who found that gamification in science education can significantly increase student engagement [50].”
ResultA gamification strategy in engineering education—A case study on motivation and engagement
Gamarra, M., Dominguez, A., Velazquez, J., et al. (2021) · Computer Applications in Engineering Education · doi:10.1002/cae.22466 · Cited by 40 · cited 1×
“Gamification in other studies has had an impact that can improve academic performance, engagement, and motivation of learners [69] as well as having a psychological and intrinsic effect on participants [70, 71] gamification can be implemented in a variety of ways such as in focus group testing,…”
BackgroundOrji, C. T., & Ogbuanya, T. C. (2020) · Innovations in Education and Teaching International · doi:10.1080/14703297.2020.1813188 · Cited by 40 · cited 1× · see citing works
“According to research by Chibueze and Theresa (2011), PBL increases students' intrinsic motivation and facilitates deeper mastery of concepts through direct engagement with the learning material [35].”
Abolnejadian, M., Alipour, S., & Taeb, K. (2024) · CHI Extended Abstracts · doi:10.1145/3613905.3637148 · Cited by 39 · cited 1× · see citing works
“[91], which can be widely applied to improve critical thinking skills in programming education and other fields.”
BackgroundJodoi, K., Takenaka, N., Uchida, S., Nakagawa, S., & Inoue, N. (2021) · Heliyon · doi:10.1016/j.heliyon.2021.e08256 · Cited by 37 · cited 1× · see citing works
“[21], as well as software with gamification elements such as points, levels and scores designed to motivate and engage learners [22, 23].”
BackgroundTwelve TipsBecoming a student in a PBL course: twelve tips for successful group discussion
Azer, S. (2004) · Medical Teacher · doi:10.1080/0142159032000156533 · Cited by 33 · cited 1×
“PBL also allows students to work in groups, which strengthens their social and communication skills, as well as the ability to work together in teams to achieve a common goal [38, 39].”
BackgroundČubela, D., Rössner, A., & Neis, P. (2023) · Education Sciences · doi:10.3390/educsci13121223 · Cited by 31 · cited 1× · see citing works
“Reputable research shows that PBGL is effective in increasing learner motivation and engagement, as well as deepening understanding of complex concepts [1-3].”
BackgroundMeaningful Gamification for Journalism Students to Enhance Their Critical Thinking Skills
Huang, L.-Y., & Yeh, Y.-C. (2017) · International Journal of Games Based Learning · doi:10.4018/ijgbl.2017040104 · Cited by 31 · cited 1×
“In addition, several studies have also highlighted the use of gamification in online discussions both individually and in communities [24, 25].”
BackgroundPoonsawad, A., Srisomphan, J., & Sanrach, C. (2022) · International Journal of Emerging Technologies in Learning (iJET) · doi:10.3991/ijet.v17i05.28181 · Cited by 29 · see citing works
Experimental discovery of Weyl semimetal TaAs
Chen, X. (2015) · Science China Materials · doi:10.1007/s40843-015-0083-9 · Cited by 28 · see citing works
Imran, H. (2022) · Journal of Educational Computing Research · doi:10.1177/07356331221144074 · Cited by 27 · see citing works
Enhancing Classroom Learning: ChatGPT's Integration and Educational Challenges
AlAli, R. M., & Wardat, Y. (2024) · International Journal of Religion · doi:10.61707/znwnxd43 · Cited by 23 · cited 1× · see citing works
“This personalization ensures that learners stay engaged and get the tailored support they need to overcome barriers to learning and improve critical thinking skills [17, 18].”
BackgroundEducational gamification and artificial intelligence for promoting digital literacy
Koravuna, S., & Surepally, U. K. (2020) · doi:10.1145/3415088.3415107 · Cited by 22 · cited 1×
“Alongside the development of gamification, the arrival of artificial intelligence in education has also opened up new opportunities for more personalized, intelligent, and automated learning experiences [13, 14].”
BackgroundKladchuen, R., & Srisomphan, J. (2021) · International Journal of Emerging Technologies in Learning (iJET) · doi:10.3991/ijet.v16i14.20439 · Cited by 21 · cited 1×
“In PBGL, the learner is placed in a real or simulated situation that is challenging and requires the learner to apply learned knowledge and skills to solve a given problem in a variety of situations [4, 5].”
BackgroundBachiri, Y.-A., & Mouncif, H. (2023) · International Journal of Emerging Technologies in Learning (iJET) · doi:10.3991/ijet.v18i05.36589 · Cited by 21 · cited 1× · see citing works
“Recent research by Bachiri and Mouncif (2020) showed that A.I systems can increase learner engagement in online courses through automated and adaptive feedback [53].”
Tsai, Y.-C. (2023) · International Conference on Innovative Technologies and Learning · doi:10.48550/arxiv.2305.00821 · Cited by 18 · cited 1× · see citing works
“Existing research mostly discusses the use of A.I ChatGPT for general feedback without deeply integrating it into gamified learning systems [55, 56].”
BackgroundGamification in Ecology-Oriented Mobile Applications—Typologies and Purposes
Torres-Toukoumidis, Á., Vintimilla-León, D., Santis-Piras, A. D., & López-López, P. C. (2022) · Societies · doi:10.3390/soc12020042 · Cited by 16 · see citing works
Abasi, A., Hoseinabadi, R., Raji, P., Friedman, J. H., & Hadian, M. R. (2022) · Parkinson s Disease · doi:10.1155/2022/6913691 · Cited by 16 · cited 1× · see citing works
“[84, 85].”
Palacios, V. A. F., Jaramillo-Galarza, K. A., Calle, J. V. Q., & Orozco-Cantos, L. (2023) · Salud Ciencia y Tecnología · doi:10.56294/saludcyt2023639 · Cited by 15 · see citing works
Pishtari, G., Sarmiento‐Márquez, E. M., Rodríguez‐Triana, M. J., Wagner, M., & Ley, T. (2024) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3636555.3636862 · Cited by 15 · cited 1× · see citing works
“This research was developed to understand how the integration of A.I and gamification can provide more personalized insights and influence long-term learning outcomes [62].”
BackgroundMahdi, H. S., & Sahari, Y. (2024) · Journal of Pedagogical Research · doi:10.33902/jpr.202423345 · Cited by 14 · cited 1× · see citing works
“I, as well as how A.I is used to measure and improve individual capabilities [26, 27].”
BackgroundNappi, C., & Cuocolo, A. (2018) · Journal of Nuclear Cardiology · doi:10.1007/s12350-018-1344-2 · Cited by 14 · cited 1×
“I, as well as how A.I is used to measure and improve individual capabilities [26, 27].”
BackgroundGamification in e-learning: A Systematic Review of Benefits, Challenges, and Future Possibilities
Gejandran, P., & Abdullah, N. (2024) · Journal of Logistics Informatics and Service Science · doi:10.33168/jliss.2024.0206 · Cited by 12 · cited 1× · see citing works
“Studies show that the use of game elements, such as points, levels, and challenges, can increase participants' active participation, encourage teamwork, and facilitate more meaningful and enjoyable learning [6-8].”
BackgroundKahoot! gamification improves learning outcomes in problem-based learning classroom
Taesotikul, T., Chinpaisal, C., & Nawanopparatsakul, S. (2021) · International Conference on Modern Educational Technology (ICMET) · doi:10.1145/3468978.3468999 · Cited by 12 · see citing works
Резаев, А. В., & Трегубова, Н. Д. (2023) · Monitoring obŝestvennogo mneniâ: èkonomičeskie i socialʹnye peremeny · doi:10.14515/monitoring.2023.3.2384 · Cited by 11 · see citing works
Gamification in online discussions: How do game elements affect critical thinking?
Tzelepi, M., Makri, K., Petroulis, I., Moundridou, M., & Papanikolaou, K. A. (2020) · IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt49669.2020.00035 · Cited by 11 · cited 1× · see citing works
“In addition, several studies have also highlighted the use of gamification in online discussions both individually and in communities [24, 25].”
BackgroundChatGPT in Medical Education: Promoting Learning or Killing Critical Thinking?
Wong, R. S.-Y. (2024) · Education in Medicine Journal · doi:10.21315/eimj2024.16.2.13 · Cited by 10 · cited 1× · see citing works
“The integration of ChatGPT with PBGL can enable more interactive and adaptive feedback, thus enriching students' learning experience [43].”
MethodProkop‐Dorner, A., Piłat, A., Ślusarczyk, M., Świątkiewicz‐Mośny, M., Ożegalska-Łukasik, N., Potysz-Rzyman, A., Zarychta, M., Juszczyk, A., Kondyjowska, D., Magiera, A., et al. (2024) · PLoS ONE · doi:10.1371/journal.pone.0307094 · Cited by 10 · cited 1× · see citing works
“In addition, the habit of critical thinking supports the development of analytical abilities necessary for solving complex problems and making informed decisions [31, 32].”
BackgroundImpact of Artificial Intelligence on Gamification: Current Applications
Modi, T., & Gochhait, S. (2023) · International Conference on Innovative Data Communication Technologies and Application (ICIDCA) · doi:10.1109/icidca56705.2023.10099771 · Cited by 9 · cited 1× · see citing works
“…learning support systems, evaluation mechanisms, education management, and educators [58], as well as the impact of utilizing A.I in gamification [59] These are some of the areas that have been researched, but do not delve deeply into how APIs such as ChatGPT can be integrated to provide more…”
BackgroundTopic Evolution in the Research on Educational Gamification
Swacha, J. (2022) · Education Sciences · doi:10.3390/educsci12100640 · Cited by 8 · see citing works
Assessing ChatGPT's Influence on Critical Thinking in Sustainability Oriented Activities
Alarcón-López, C., Krütli, P., & Gillet, D. (2024) · 2024 IEEE Global Engineering Education Conference (EDUCON) · doi:10.1109/educon60312.2024.10578790 · Cited by 8 · cited 1× · see citing works
“…researchers Berg and Plessis (2023) recommended the need for further research to explore the practice of ChatGPT integration in lesson planning [60], as well as research by Lopez et.al which provides advice on the continuation of the utilization of ChatGPT in the critical thinking dimension [61].”
BackgroundSupnoon, A., & Chonchaiya, R. (2024) · International Journal of Education and Practice · doi:10.18488/61.v12i2.3718 · Cited by 7 · cited 1× · see citing works
“[21], as well as software with gamification elements such as points, levels and scores designed to motivate and engage learners [22, 23].”
BackgroundMutma’innah, F., & Hamimi, E. (2024) · International Journal of Interactive Mobile Technologies (iJIM) · doi:10.3991/ijim.v18i09.48097 · Cited by 7 · cited 1× · see citing works
“In addition, PBL not only focuses on problem solving but also on developing critical and analytical thinking skills that are important in professional and academic life [36].”
BackgroundGamification and its challenges in e-learning: a case study of computer science learners in KKHSOU
Barua, A. M., & Bharali, S. S. (2023) · AAOU Journal/AAOU journal · doi:10.1108/aaouj-01-2023-0001 · Cited by 6 · see citing works
ChatGPT-enabled Network Automation using API-based Prompts
Okunaiya, O., Austin, R., & Zhu, S.-Y. (2024) · IEEE/IFIP Network Operations and Management Symposium · doi:10.1109/noms59830.2024.10574943 · Cited by 6
Chen, B., & Chen, C. (2023) · IEEE International Conference on Knowledge Innovation and Invention (ICKII) · doi:10.1109/ickii58656.2023.10332623 · Cited by 4 · cited 1× · see citing works
“[81-83].”
Problem Based Learning Using Gamification: A Systematic Literature Review
Daba, J. B. R., Rosmansyah, Y., & Dabarsyah, B. (2019) · International Conference on Informatics, Multimedia, Cyber and Information System (ICIMCIS) · doi:10.1109/icimcis48181.2019.8985216 · Cited by 4 · see citing works
Monroe, C. M., Zosel, K. L., Stansbury, M. L., Younginer, N., Davis, R. E., Dutton, G. R., Newton, R. L., Cai, B., & West, D. S. (2024) · mHealth · doi:10.21037/mhealth-23-44 · Cited by 3 · cited 1× · see citing works
“…performance, engagement, and motivation of learners [69] as well as having a psychological and intrinsic effect on participants [70, 71] gamification can be implemented in a variety of ways such as in focus group testing, online, utilizing software to developing concepts into the metaverse [72-75].”
BackgroundEvolution of Research on Adaptive Gamification: A Bibliometric Study
Vijay, M., & Jayan, J. (2023) · International Conference on Circuit, Power and Computing Technologies · doi:10.1109/iccpct58313.2023.10245165 · Cited by 2 · cited 1× · see citing works
“The use of game elements in education, known as gamification, has grown rapidly since 2012 [9, 10].”
BackgroundKuchyn, I. L., Lymar, L., Bielka, K., et al. (2024) · Wiadomosci lekarskie · doi:10.36740/wlek202404108 · Cited by 2 · cited 1×
“There are also studies that assess academic integrity and learner attitudes towards using A.I. technologies [28], as well as looking at how A.I. affects critical thinking, motivation, and self-awareness [29].”
Background
Flagged first, then by citations
4 of the 89 references couldn’t be matched to a known paper.
Research works printed but not linked: 9
- Elbanna, S., & Armstrong, L. (2024). Exploring the integration of ChatGPT in education: Adapting for the future. Management & Sustainability: An Arab Review, 3(1), 16-29.
- Shafieieh, M., Ozturen, A., Rezapouraghdam, H., & Karatepe, O. M. (2024). Does critical thinking mediate the relationship between sustainability knowledge and tourism students' ability to make sustainable decisions? Sustainability, 16(13), 5655-5671. 518 VOLUME 4, No 3, 2024
- Fajari, L. E. W., Sarwanto, & Chumdari. (2020). Improving elementary school's critical thinking skills through three different PBL-assisted learning media viewed from learning styles. Journal of E-Learning and Knowledge Society, 16(1), 55-64.
- Freire-Palacios, V., Jaramillo-Galarza, K., Quito-Calle, J., & Orozco-Cantos, L. (2023). La inteligencia artificial en la gamificación para promover la salud mental de los estudiantes universitarios: una revisión de alcance. Salud, Ciencia y Tecnología, 3(1), 639-656.
- Резаев, А. В., & Трегубова, Н. Д. (2023). От социологии алгоритмов к социальной аналитике искусственной социальности: анализ кейсов API и ChatGPT. The Monitoring of Public Opinion: Economic & Social Changes, 3(3), 3-22.
- Babu, S. S., & Moorthy, A. D. (2024). Application of artificial intelligence in adaptation of gamification in education: A literature review. Computer Applications in Engineering Education, 32(1), e22683-e22698.
- Yu, P., & Wang, S. (2024). An examination and analysis of the integration of artificial intelligence and gamification in the pedagogy of Chinese higher education. In Engaged Learning and Innovative Teaching in Higher Education (1st ed., pp. 29-46). Singapore: Springer. 519 VOLUME 4, No 3, 2024
- Craig, S., Stark, P., Wilson, C. B., Carter, G., Clarke, S., & Mitchell, G. (2023). Evaluation of a dementia awareness game for undergraduate nursing students in Northern Ireland: A pre-/post-test study. BMC Nursing, 22(1), 177-1. 520 VOLUME 4, No 3, 2024
- Rusandi, M. A., Ahman, Saripah, I., Khairun, D. Y., & Mutmainnah. (2023). No worries with ChatGPT: Building bridges between artificial intelligence and education with critical thinking soft skills. Journal of Public Health (Bangkok), 45(3), e602-e603.
Cited by
Sorted by citations
Sardi, J., Darmansyah, Candra, O., Yuliana, D. F., Habibullah, Yanto, D. T. P., & Eliza, F. (2025) · International Journal of Engineering Pedagogy (iJEP) · doi:10.3991/ijep.v15i1.53379 · Cited by 105 · see citing works
Kassenkhan, A. M., Moldagulova, A. N., & Serbin, V. V. (2025) · IEEE Access · doi:10.1109/access.2025.3576147 · Cited by 44 · see citing works
“In a study conducted by Naatonis et al. [5] among students in Indonesia and Timor-Leste, the use of PBGL led to a significant improvement in critical thinking, with an average gain of 23.57 points, compared to only 9.04 points in the control group.”
InfluentialHelal, M. Y. I., Elgendy, I. A., Albashrawi, M. A., Dwivedi, Y. K., Al-Ahmadi, S., & Jeon, I. (2025) · Information Discovery and Delivery · doi:10.1108/idd-05-2025-0125 · Cited by 47 · see citing works
Enhancing critical thinking skills in ChatGPT-human interaction: a scoping review
Raitskaya, L., & Tikhonova, E. (2025) · Journal of Language and Education · doi:10.17323/jle.2025.27387 · Cited by 30 · see citing works
Balart, T., Díaz, B., & Shryock, K. (2026) · Algorithms · doi:10.3390/a19030179 · Cited by 15 · see citing works
“Other tools such as Gemini, Ernie Bot, and multi-tool workflows were less common [61,70], and a few studies developed custom GPT-based systems ([50,59,84,89,90]).”
InfluentialKassenkhan, A. M., Serbin, V. V., Beisembekova, R., Abshukirova, A. M., & Mendekina, B. (2026) · Information · doi:10.3390/info17030282 · Cited by 15 · see citing works
Pérez-Jorge, D., González-Afonso, M. C., Santos-Álvarez, A. G., Plasencia-Carballo, Z., & Perdomo-López, C. D. L. Á. (2025) · Information · doi:10.3390/info16070540 · Cited by 9 · see citing works
ChatGPT in education: A systematic review of its impact on critical thinking skills and dispositions
Guo, Y., Huang, L., Zhang, C., Li, Q., & Chen, M. (2025) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2025.102106 · Cited by 18 · see citing works
Bećirović, S., Augustín, M., Mattoš, B., & Dančo, J. (2026) · Discover Computing · doi:10.1007/s10791-026-10042-y · Cited by 7 · see citing works
Çakır, B., Berk, A., Berşe, S., Göv, P., & Güngör, M. İ. (2025) · Global Nursing Insights · doi:10.31125/globnursinsights.1762516 · Cited by 4 · see citing works
Pum, M., Mey, D., Van, S., & Seung, S. (2025) · Interdisciplinary Educational Technology · doi:10.71176/interedtech/17569 · Cited by 5 · see citing works
EFFICIENCY, ENGAGEMENT, AND COGNITIVE OFFLOADING: A REVIEW OF GENERATIVE AI IN EDUCATION
Khalid, N., Kasbun, R., & Rahman, N. F. A. (2026) · International Journal of Education Psychology and Counseling · doi:10.35631/ijepc.1162072 · Cited by 4 · see citing works
Pillpe, G. R., & Inca, K. S. M. (2024) · Revista Punto Cero · doi:10.35319/puntocero.202449241
“Esta mejora se atribuye a la capacidad de la IA de proporcionar comentarios y recomendaciones personalizados, lo que ayuda a los estudiantes a involucrarse más profundamente con el material (Naatonis et al., 2024).”
Meyrambaykyzy, K. A., & Maratuly, A. A. (2025) · International Conference on Control, Automation and Systems (ICCAS) · doi:10.23919/iccas66577.2025.11301135
Transforming Teaching and Learning in Business Education
Berraies, S., Rejeb, W. B., & Abidine, S. Z. E. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-4377-8.ch008
CodePlay: A Gamification-Based Approach to Improving Programming Skills
Reyes, A., Chicaiza, D., Guerrero, G., Avilés, D., & Coloma, K. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-16761-3_4
Nuryadin, A., Karlimah, K., & Nugraha, M. R. (2026) · Participatory Educational Research · doi:10.17275/per.26.27.13.2
Chai, H., Guo, F., Wang, J., Song, X., & Hu, T. (2026) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2026.102227 · Cited by 1 · see citing works
Serbin, V. V., Mendes, M., Kassenkhan, A. M., Bekarystankyzy, A., Ibragim, G., & Tolegenov, A. (2026) · Machine Learning and Knowledge Extraction · doi:10.3390/make8050120 · Cited by 1 · see citing works
Gutierrez, R., Govea, J., & Valencia-Silva, E. (2026) · Education Sciences · doi:10.3390/educsci16060858
Syarifah, D. F., Basyirun, B., Wijaya, M. B. R., et al. (2026) · Psychological Science and Education · doi:10.17759/pse.2026310308
Cong‐Lem, N. (2026) · Journal for the Study of Education and Development Infancia y Aprendizaje · doi:10.1177/02103702261463324
Chen, Z., Yan, Z., Fu, Z., & Huang, H. (2026) · Acta Psychologica · doi:10.1016/j.actpsy.2026.107734 · Cited by 1 · see citing works
Galindo‐Domínguez, H., Campo, L., Delgado, N., & Urruzola, M. V. (2026) · Thinking Skills and Creativity · doi:10.1016/j.tsc.2026.102385
- Authors
- Naatonis, R. N., Rusijono, R., Jannah, M., & Malahina, E. A. U.
- Affiliations
- Universitas Negeri Surabaya (ID)
- Universitas Uyelindo (ID)
- Venue
- Qubahan Academic Journal
- Publisher
- Qubahan
- Published
- 2024-09-10
- Volume
- 4
- Issue
- 3
- Pages
- 485-520
- ISSN
- 2709-8206 (ISSN-L)
- OpenAlex
- W4402412763
- Cited by
- 24
- References
- 89
- Citations by year
- 2026: 14 · 2025: 9 · 2024: 1
- FWCI
- 18.87
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- Locations
- Qubahan Academic Journal (publishedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Funders
- Lembaga Pengelola Dana Pendidikan
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Educational Games and Gamification
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Social Sciences › Education › Problem and Project Based Learning
- Keywords
- critical thinking skills, ChatGPT API, artificial intelligence in education, automated feedback, Python programming education, AI-assisted learning, student motivation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
