CoChat
Sign Up
Cited by 191Artificial Intelligence in EducationArtificial Intelligence in Healthcare and EducationIntelligent Tutoring Systems and Adaptive Learning

A systematic literature review of empirical research on ChatGPT in education

Open access
Albadarin, Yazid, Saqr, Mohammed, Pope, Nicolas, Tukiainen, Markku·2024·Discover Education
DOI 10.1007/s44217-024-00138-2
Open this paper in CoChat — free account

Abstract

Over the last four decades, studies have investigated the incorporation of Artificial Intelligence (AI) into education. A recent prominent AI-powered technology that has impacted the education sector is ChatGPT. This article provides a systematic review of 14 empirical studies incorporating ChatGPT into various educational settings, published in 2022 and before the 10th of April 2023—the date of conducting the search process. It carefully followed the essential steps outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2020) guidelines, as well as Okoli’s (Okoli in Commun Assoc Inf Syst, 2015) steps for conducting a rigorous and transparent systematic review. In this review, we aimed to explore how students and teachers have utilized ChatGPT in various educational settings, as well as the primary findings of those studies. By employing Creswell’s (Creswell in Educational research: planning, conducting, and evaluating quantitative and qualitative research [Ebook], Pearson Education, London, 2015) coding techniques for data extraction and interpretation, we sought to gain insight into their initial attempts at ChatGPT incorporation into education. This approach also enabled us to extract insights and considerations that can facilitate its effective and responsible use in future educational contexts. The results of this review show that learners have utilized ChatGPT as a virtual intelligent assistant, where it offered instant feedback, on-demand answers, and explanations of complex topics. Additionally, learners have used it to enhance their writing and language skills by generating ideas, composing essays, summarizing, translating, paraphrasing texts, or checking grammar. Moreover, learners turned to it as an aiding tool to facilitate their directed and personalized learning by assisting in understanding concepts and homework, providing structured learning plans, and clarifying assignments and tasks. However, the results of specific studies (n = 3, 21.4%) show that overuse of ChatGPT may negatively impact innovative capacities and collaborative learning competencies among learners. Educators, on the other hand, have utilized ChatGPT to create lesson plans, generate quizzes, and provide additional resources, which helped them enhance their productivity and efficiency and promote different teaching methodologies. Despite these benefits, the majority of the reviewed studies recommend the importance of conducting structured training, support, and clear guidelines for both learners and educators to mitigate the drawbacks. This includes developing critical evaluation skills to assess the accuracy and relevance of information provided by ChatGPT, as well as strategies for integrating human interaction and collaboration into learning activities that involve AI tools. Furthermore, they also recommend ongoing research and proactive dialogue with policymakers, stakeholders, and educational practitioners to refine and enhance the use of AI in learning environments. This review could serve as an insightful resource for practitioners who seek to integrate ChatGPT into education and stimulate further research in the field.

AI snapshot

Written from the paper’s full text.

TL;DR

This systematic review synthesizes 14 empirical studies published in 2022 and up to 10 April 2023 on how students and teachers initially used ChatGPT in education, finding it served as a virtual assistant, writing aid, and teaching productivity tool, while raising concerns about accuracy, academic integrity, and overuse harming collaboration and innovation. The authors call for structured training, clear guidelines, and redesigned assessment.

Key findings

  • Learners used ChatGPT as a virtual intelligent assistant (9 studies) for on-demand answers, immediate feedback, and learning resources; as a writing and language proficiency assistant (6 studies) for generating ideas, composing essays, summarizing, translating, paraphrasing, and grammar checking; as a resource for self-directed and personalized learning (5 studies); and as a tool for enhancing competencies and academic success (6 and 7 studies respectively).
  • Teachers used ChatGPT to create lesson plans, generate quizzes, and provide additional resources, enhancing productivity and efficiency and enabling new teaching methodologies (5 studies).
  • Negative effects: results of specific studies (n = 3, 21.4%) show overuse of ChatGPT may negatively impact innovative capacities and collaborative learning competencies among learners; one study [S5] reported students suggesting it may harm collaborative learning.
  • Most reviewed studies recommend structured training, support, and clear guidelines for learners and educators, including critical evaluation skills and strategies for integrating human interaction into AI-mediated learning.

Methods

  • Systematic literature review following PRISMA 2020 and Okoli's (2015) steps, with quality appraisal using Fink's standards (two articles excluded for insufficient methodological detail).
  • Search of Dimensions.ai, Scopus, Web of Science, ERIC, and Lens.org using ("ChatGPT" AND "Education"), conducted 3–10 April 2023, yielding 276 articles; after duplicate removal 135 unique articles, then 26 after inclusion/exclusion screening, and 14 eligible for analysis after excluding studies solely testing ChatGPT capabilities.
  • Data extraction and interpretation used Creswell's (2015) open, axial, and selective coding techniques; extracted data covered authors, titles, years, educational levels, methodologies, sample sizes, participants, aims, data sources, and analysis methods.

Limitations

  • Stated: the limited number of reviewed studies, attributed to the novelty of the technology, lack of clear guidelines or best practices, and institutional or governmental policies affecting its educational use.
  • Stated: the review covers the original ChatGPT based on GPT-3 or GPT-3.5, so studies using GPT-4 may yield different findings; follow-up systematic reviews and long-term studies are needed.
  • Evident from the text: the search window closed on 10 April 2023, only months after ChatGPT's November 2022 launch; nine of the reviewed studies had samples of fewer than 50 participants, and most were conducted in higher education institutions (11 of 14), limiting generalizability.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1007/s44217-024-00138-2

Cites

86 of 86 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 84 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    The majority of reviewed studies (11) were conducted in higher education institutions.

    “The majority of the reviewed studies, 11 studies, were conducted in higher education institutions [S1]-[S10] and [S13].”
    Quote in the paperNot a substantial claim
  • Findingown data
    § Abstract

    Learners used ChatGPT to enhance their writing and language skills by generating ideas, composing essays, summarizing, translating, paraphrasing texts, or checking grammar.

    “Additionally, learners have used it to enhance their writing and language skills by generating ideas, composing essays, summarizing, translating, paraphrasing texts, or checking grammar.”
    Quote in the paperSubstantial claim
  • Findingown data

    The reviewed studies included four study populations: university students, university teachers, university teachers and students, and elementary school teachers.

    “Four study populations were included: university students, university teachers, university teachers and students, and elementary school teachers.”
    Quote in the paperNot a substantial claim
  • Findingcitation
    § Abstract

    Results of three specific studies (n = 3, 21.4%) show that overuse of ChatGPT may negatively impact innovative capacities and collaborative learning competencies among learners.

    “However, the results of specific studies (n = 3, 21.4%) show that overuse of ChatGPT may negatively impact innovative capacities and collaborative learning competencies among learners.”
    Quote in the paperSubstantial claim
  • Findingown data

    Ten reviewed studies primarily focused on participants from single countries.

    “The majority of the studies, 10 in total, [S1]-[S3], [S5]-[S9], [S11], and [S14], primarily focused on participants from single countries such as Pakistan, the United Arab Emirates, China, Indonesia, Poland, Saudi Arabia, South Korea, Spain, Tajikistan, and the United States.”
    Quote in the paperNot a substantial claim
  • May combine two claims. 66
  • Classified as probably not a substantial claim. 36

  • May combine two claims.

    Findingown data

    One reviewed study focused on Social Science Education, one on Early Education, one on Journalism Education, and three did not specify the field of education.

    “One study focused on Social Science Education [S13]. One study focused on Early Education [S4]. One study focused on Journalism Education [S9]. Finally, three studies did not specify the field of education [S1], [S6], and [S7].”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § Abstract

    Learners used ChatGPT as a virtual intelligent assistant that offered instant feedback, on-demand answers, and explanations of complex topics.

    “The results of this review show that learners have utilized ChatGPT as a virtual intelligent assistant, where it offered instant feedback, on-demand answers, and explanations of complex topics.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingnone given
    § Abstract

    The reviewed studies recommend ongoing research and proactive dialogue with policymakers, stakeholders, and educational practitioners to refine and enhance the use of AI in learning environments.

    “Furthermore, they also recommend ongoing research and proactive dialogue with policymakers, stakeholders, and educational practitioners to refine and enhance the use of AI in learning environments.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data
    § Abstract

    The majority of the reviewed studies recommend structured training, support, and clear guidelines for learners and educators to mitigate drawbacks.

    “Despite these benefits, the majority of the reviewed studies recommend the importance of conducting structured training, support, and clear guidelines for both learners and educators to mitigate the drawbacks.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § Abstract

    Learners used ChatGPT as an aiding tool to facilitate directed and personalized learning by assisting with understanding concepts and homework, providing structured learning plans, and clarifying assignments and tasks.

    “Moreover, learners turned to it as an aiding tool to facilitate their directed and personalized learning by assisting in understanding concepts and homework, providing structured learning plans, and clarifying assignments and tasks.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingreasoning
    § Abstract

    The reviewed studies recommend developing critical evaluation skills to assess the accuracy and relevance of ChatGPT information and strategies for integrating human interaction and collaboration into AI learning activities.

    “This includes developing critical evaluation skills to assess the accuracy and relevance of information provided by ChatGPT, as well as strategies for integrating human interaction and collaboration into learning activities that involve AI tools.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Two reviewed studies did not specify the educational level of the population, and one focused on elementary education.

    “Two studies did not specify the educational level of the population [S12] and [S14], while one study focused on elementary education [S11].”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    One reviewed study was published in 2022 and the remaining 13 were published in 2023.

    “One study was published in 2022 [S1], while the remaining were published in 2023 [S2]-[S14].”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    All 14 reviewed studies were empirical and published in journals focusing on Education and Technology.

    “All studies were empirical studies and published in different journals focusing on Education and Technology.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § Abstract

    The review included 14 empirical studies incorporating ChatGPT into educational settings, published in 2022 and before 10 April 2023.

    “This article provides a systematic review of 14 empirical studies incorporating ChatGPT into various educational settings, published in 2022 and before the 10th of April 2023—the date of conducting the search process.”
    • Also stated in: § 3.1 Part 1: descriptive analysis · plain
      “The total number of the reviewed studies was 14.”
    • Also stated in: § 2.4 Literature search · plain
      “This resulted in 14 articles eligible for analysis.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § Abstract

    Educators used ChatGPT to create lesson plans, generate quizzes, and provide additional resources, which helped them enhance productivity and efficiency and promote different teaching methodologies.

    “Educators, on the other hand, have utilized ChatGPT to create lesson plans, generate quizzes, and provide additional resources, which helped them enhance their productivity and efficiency and promote different teaching methodologies.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Three reviewed studies focused on Arts and Humanities Education, specifically English Education; two focused on Engineering Education; and two focused on Mathematics Education.

    “Three studies focused on Arts and Humanities Education [S8], [S11], and [S14], specifically English Education. Two studies focused on Engineering Education, with one in Computer Engineering [S2] and the other in Construction Education [S3]. Two studies focused on Mathematics Education [S5] and [S12].”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Study S4 used a mixed methods approach with 10 participants for qualitative analysis and 110 for quantitative analysis.

    “It is worth mentioning that study [S4] adopted a mixed methods approach, including 10 participants for qualitative analysis and 110 participants for quantitative analysis.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Six reviewed studies involved university students, three focused on university teachers, one targeted elementary school teachers, and four included both university teachers and students.

    “Six studies involved university students [S2], [S3], [S5] and [S6]-[S8]. Three studies focused on university teachers [S1], [S4], and [S6], while one study specifically targeted elementary school teachers [S11]. Additionally, four studies included both university teachers and students [S10] and [12,13,14], and among them, study [S13] specifically included postgraduate students.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Four reviewed studies involved participants from multiple countries.

    “In contrast, four studies, [S4], [S10], [S12], and [S13], involved participants from multiple countries, including China and the United States [S4], China, the United Kingdom, and the United States [S10], the United Arab Emirates, Oman, Saudi Arabia, and Jordan [S12], Turkey, Sweden, Canada, and Australia [13].”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Nine reviewed studies had a sample size of less than 50 participants, three had 50-100 participants, and one had more than 100 participants.

    “In terms of the sample size of the reviewed studies, nine studies included a small sample size of less than 50 participants [S1], [S3], [S6], [S8], and [S10]-[S13]. Three studies had 50–100 participants [S2], [S9], and [S14]. Only one study had more than 100 participants [S7].”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning

    The authors eliminated empirical studies solely focused on checking ChatGPT capabilities, resulting in 14 articles eligible for analysis.

    “The authors agreed to eliminate any empirical studies solely focused on checking ChatGPT capabilities, as these studies do not guide us in addressing the research questions and achieving the study's objectives. This resulted in 14 articles eligible for analysis.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    The literature search was conducted between 3 and 10 April 2023 and resulted in 276 articles from all selected databases.

    “The search was conducted between the 3rd and 10th of April 2023, which resulted in 276 articles from all selected databases (111 articles from Dimensions.ai, 65 from Scopus, 28 from Web of Science, 14 from ERIC, and 58 from Lens.org).”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    After duplicate removal, 135 unique articles remained.

    “Subsequently, the first author manually reviewed the duplicated articles ensured that they had the same content, and then removed them, leaving us with 135 unique articles.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Data extraction and interpretation yielded several themes, which were categorized and presented in Table 2.

    “After a comprehensive analysis and capturing of the key information from the included studies, the data extraction and interpretation process yielded several themes. These themes have been categorized and are presented in Table 2.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning
    § 7 Conclusions

    Learners used ChatGPT as a virtual intelligent assistant, benefiting from its ability to provide immediate feedback, on-demand answers, and easy access to educational resources.

    “Learners have used it as a virtual intelligent assistant, benefiting from its ability to provide immediate feedback, on-demand answers, and easy access to educational resources.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    Some students indicated the need for improvement, as they found that ChatGPT answers are not always accurate, may provide misleading information, or may not align with their expectations.

    “some students indicated the need for improvement, as they found that answers are not always accurate [S2], and that misleading information may have been provided or that it may not always align with their expectations [S6] and [S10]”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation
    § 6 Discussion

    The review found that the potential misuse of ChatGPT might foster cheating, facilitate plagiarism, and threaten academic integrity.

    “The review found that the potential misuse of ChatGPT might foster cheating, facilitate plagiarism, and threaten academic integrity.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    ChatGPT delivered age-appropriate conversations and tailored teaching based on a child's interests.

    “It delivered age-appropriate conversations and tailored teaching based on a child's interests [S4]”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    ChatGPT allowed participants to translate content into their home languages, making it more accessible and relevant to their context.

    “ChatGPT allowed participants to translate content into their home languages, making it more accessible and relevant to their context [S4]”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    ChatGPT helped students in writing by making them active investigators rather than passive knowledge recipients and facilitated the development of their writing skills.

    “ChatGPT helped them in writing by making students active investigators rather than passive knowledge recipients and facilitated the development of their writing skills [S11] and [S14]”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    ChatGPT increased students' successful completion of important tasks in their studies, particularly those involving average difficulty writing tasks.

    “it increased students' successful completion of important tasks in their studies [S7], particularly those involving average difficulty writing tasks [S8]”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation
    § 6 Discussion

    The review found that frequent use of ChatGPT may result in a decrease in innovative capacities, collaborative skills, cognitive capacities, and students' motivation to attend classes, as well as reduced higher-order thinking skills among students.

    “it found that frequent use of ChatGPT may result in a decrease in innovative capacities, collaborative skills and cognitive capacities, and students' motivation to attend classes, as well as could lead to reduced higher-order thinking skills among students [22, 29]”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [22] Farrokhnia M, Banihashem SK, Noroozi O, Wals AEJ. A SWOT analysis of ChatGPT: implications for educational practice and research. Innov Educ Teach Int. 2023. https://doi.org/10.1080/14703297.2023.2195846. source

      Not checked in the free run.

    • cites [29] García-Peñalvo FJ. La percepción de la inteligencia artificial en contextos educativos tras el lanzamiento de ChatGPT: disrupción o pánico. Educ Knowl Soc. 2023;24: e31279. https://doi.org/10.14201/eks.31279. source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    ChatGPT aided students in deepening their understanding of various subjects by finding information, generating ideas, translating texts, and providing alternative questions.

    “by finding information, generating ideas, translating texts, and providing alternative questions, ChatGPT aided students in deepening their understanding of various subjects [S6]”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation
    § 6 Discussion

    The review found that teachers without the requisite skills to utilize ChatGPT realized that it did not contribute positively to their work and could potentially have adverse effects.

    “the review also found that teachers without the requisite skills to utilize ChatGPT realized that it did not contribute positively to their work and could potentially have adverse effects [37]”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [37] Iqbal N, Ahmed H, Azhar KA. Exploring teachers’ attitudes towards using CHATGPT. Globa J Manag Adm Sci. 2022;3(4):97–111. https://doi.org/10.46568/gjmas.v3i4.163. source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    ChatGPT offered step-by-step guidance to grasp concepts at students' own pace and enhance their understanding.

    “it offered step-by-step guidance to grasp concepts at their own pace and enhance their understanding [S5]”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Teacher negativity toward ChatGPT stemmed from a lack of necessary skills to use it effectively, limited familiarity with it, and occasional inaccuracies in the content provided by it.

    “This negativity stemmed from a lack of necessary skills to use it effectively [S1], a limited familiarity with it [S4], and occasional inaccuracies in the content provided by it [S10]”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Educators emphasized the importance of adopting new educational philosophies and approaches that can evolve with the introduction of ChatGPT into the classroom.

    “They also emphasized the importance of adopting new educational philosophies and approaches that can evolve with the introduction of ChatGPT into the classroom [S12]”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Five reviewed studies highlighted that educators found the necessity of redefining their teaching profession with the assistance of ChatGPT, developing new effective learning strategies, and adapting teaching strategies and methodologies to ensure the development of essential skills for future engineers.

    “Five of the reviewed studies highlighted that educators found the necessity of redefining their teaching profession with the assistance of ChatGPT [S11], developing new effective learning strategies [S4], and adapting teaching strategies and methodologies to ensure the development of essential skills for future engineers [S5]”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    ChatGPT proved valuable in constructing new knowledge and providing timely answers to students' questions in classrooms.

    “It also proved valuable in constructing new knowledge [S6] and providing timely answers to students' questions in classrooms [S11]”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    ChatGPT acted as a personalized learning assistant, adapted to students' needs and pace, and assisted them in understanding mathematical concepts.

    “acted as a personalized learning assistant, adapted to their needs and pace, which assisted them in understanding mathematical concepts [S12]”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Seven reviewed studies highlighted that students found ChatGPT beneficial for learning as it enhanced learning efficiency and improved the learning experience.

    “Seven of the reviewed studies highlighted that students found ChatGPT to be beneficial for learning as it enhanced learning efficiency and improved the learning experience.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    Students observed that the accuracy of ChatGPT depends on the quality and specificity of the user's input, the complexity of the question or topic, and the scope and relevance of its training data.

    “the accuracy of ChatGPT is dependent on several factors, including the quality and specificity of the user's input, the complexity of the question or topic, and the scope and relevance of its training data [S12]”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Students generally were confident in the accuracy of ChatGPT's responses, considering them relevant, reliable, and detailed.

    “students generally were confident in the accuracy of ChatGPT's responses, considering them relevant, reliable, and detailed [S3], [S4], [S5], and [S8]”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    ChatGPT improved students' learning experiences in solving mathematics problems and developing abilities.

    “It also improved students' learning experiences in solving mathematics problems and developing abilities [S5] and [S12]”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Participants used ChatGPT to check grammar or as a dictionary.

    “participants used it to check grammar or as a dictionary [S11]”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 7 Conclusions

    Learners used ChatGPT to enhance their writing and language skills, engaging in practices such as generating ideas, composing essays, summarizing, translating, paraphrasing texts, or checking grammar.

    “learners have used it to enhance their writing and language skills, engaging in practices such as generating ideas, composing essays, and performing tasks like summarizing, translating, paraphrasing texts, or checking grammar”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    Many students felt that ChatGPT's answers were not always accurate, and most believed that it requires good background knowledge to work with.

    “Many students felt that ChatGPT's answers were not always accurate and most of them believed that it requires good background knowledge to work with.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    ChatGPT provided learning resources and guidance on diverse educational topics and created a supportive home learning environment.

    “It provided learning resources and guidance on diverse educational topics and created a supportive home learning environment [S2] and [S4]”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Updating curricula to focus on improving human-specific features, such as emotional intelligence, creativity, and philosophical perspectives, was found to be essential.

    “updating curricula to focus on improving human-specific features, such as emotional intelligence, creativity, and philosophical perspectives [S13], was found to be essential”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 7 Conclusions

    Other learners utilized ChatGPT in supporting and facilitating their directed and personalized learning on a broad range of educational topics, assisting in understanding concepts and homework, providing structured learning plans, and clarifying assignments and tasks.

    “other learners have utilized it in supporting and facilitating their directed and personalized learning on a broad range of educational topics, assisting in understanding concepts and homework, providing structured learning plans, and clarifying assignments and tasks”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    ChatGPT allowed students to generate unique ideas and perspectives, leading to deeper analysis and reflection on their journalism writing.

    “ChatGPT allowed students to generate unique ideas and perspectives, leading to deeper analysis and reflection on their journalism writing [S9]”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    ChatGPT enhanced teachers' efficiency by generating new ideas for activities and preplanning activities for their students, including interactive language game partners.

    “ChatGPT enhanced teachers' efficiency by generating new ideas for activities and preplanning activities for their students [S4] and [S6], including interactive language game partners [S11]”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    According to one reviewed study, students suggested that using ChatGPT may negatively affect collaborative learning competencies between students.

    “according to one study [S5], students suggested that using ChatGPT may negatively affect collaborative learning competencies between students”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    ChatGPT assisted in studying geometry operations, empowering students to explore geometry operations at their own pace.

    “assisted in studying geometry operations, thereby empowering them to explore geometry operations at their own pace [S12]”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Two reviewed studies in mathematical education showed the positive impact of ChatGPT on students' problem-solving abilities and on enhancing their understanding of the problem-solving process.

    “two studies focused on mathematical education have shown the positive impact of ChatGPT on students' problem-solving abilities in unraveling problem-solving questions [S12] and enhancing the students' understanding of the problem-solving process [S5]”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Reviewed studies showed that participants' productivity and work efficiency were significantly enhanced by using ChatGPT as it enabled them to allocate more time to other tasks and reduce their overall workloads.

    “participants' productivity and work efficiency have been significantly enhanced by using ChatGPT as it enabled them to allocate more time to other tasks and reduce their overall workloads [S6], [S10], [S11], [S13], and [S14]”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    ChatGPT enhanced students' efficiency in solving mathematics problems and capabilities.

    “it also enhanced their efficiency in solving mathematics problems and capabilities [S5] and [S12]”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingreasoning
    § 7 Conclusions

    Learners might not engage deeply with the material or consider multiple solutions to a problem when relying on ChatGPT.

    “Learners might not engage deeply with the material or consider multiple solutions to a problem.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning
    § 7 Conclusions

    Overuse of ChatGPT can decrease innovative capacities and collaborative learning among learners.

    “The results revealed that overuse of ChatGPT could decrease innovative capacities and collaborative learning among learners.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given
    § 7 Conclusions

    Integrating ChatGPT into education raises concern about inequity in access.

    “issues of inequity in access”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1 Introduction

    ChatGPT is widely recognized as one of today's most advanced LLMs-based chatbots.

    “This cutting-edge natural language processing (NLP) tool is widely recognized as one of today's most advanced LLMs-based chatbots [70]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [70] Schöbel S, Schmitt A, Benner D, Saqr M, Janson A, Leimeister JM. Charting the evolution and future of conversational agents: a research agenda along five waves and new frontiers. Inf Syst Front. 2023. https://doi.org/10.1007/s10796-023-10375-9. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1 Introduction

    ChatGPT exhibited an impressive ability to formulate interpretable responses that were easily understood when compared with Google's feature snippet.

    “Similarly, ChatGPT exhibited an impressive ability to formulate interpretable responses that were easily understood when compared with Google's feature snippet [34].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [34] Hopkins AM, Logan JM, Kichenadasse G, Sorich MJ. Artificial intelligence chatbots will revolutionize how cancer patients access information: ChatGPT represents a paradigm-shift. JNCI Cancer Spectr. 2023. https://doi.org/10.1093/jncics/pkad010. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1 Introduction

    In a multidisciplinary test requiring higher-order thinking, ChatGPT achieved the highest score, surpassing Bing Chat and Bard.

    “The study revealed that ChatGPT achieved the highest score, surpassing Bing Chat and Bard [64].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [64] Rudolph J, Tan S, Tan S. War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.1.23. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Inclusion and exclusion criteria were determined based on the authors' experience and informed by previous successful systematic reviews.

    “To ensure the rigorous execution of this step, our inclusion and exclusion criteria were determined based on the authors' experience and informed by previous successful systematic reviews [21].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [21] Elmoazen R, Saqr M, Tedre M, Hirsto L. A systematic literature review of empirical research on epistemic network analysis in education. IEEE Access. 2022;10:17330–48. https://doi.org/10.1109/access.2022.3149812. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A protocol outlining the study process mitigates researcher bias in study selection and data extraction.

    “This protocol serves to outline the study process in a rigorous and transparent manner, mitigating researcher bias in study selection and data extraction [88].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [88] Xiao Y, Watson ME. Guidance on conducting a systematic literature review. J Plan Educ Res. 2017;39(1):93–112. https://doi.org/10.1177/0739456x17723971. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The synthesis stage is considered a transition from an author-centric to a concept-centric focus, enabling mapping of all provided information to achieve the most effective evaluation of the data.

    “The synthesis stage is considered a transition from an author-centric to a concept-centric focus, enabling us to map all the provided information to achieve the most effective evaluation of the data [87].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [87] Webster J, Watson RT. Analyzing the past to prepare for the future: writing a literature review. Manag Inf Syst Quart. 2002;26(2):3.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1 Introduction

    In a comparison by O'Leary, ChatGPT outperformed LaMDA and BlenderBot.

    “For example, O’Leary [53] compared two chatbots, LaMDA and BlenderBot, with ChatGPT and revealed that ChatGPT outperformed both.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [53] O’Leary DE. An analysis of three chatbots: BlenderBot, ChatGPT and LaMDA. Int J Intell Syst Account, Financ Manag. 2023;30(1):41–54. https://doi.org/10.1002/isaf.1531. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Coding categories were constructed using Creswell's coding techniques for data extraction and interpretation.

    “To improve efficiency, reduce variation among authors, and minimize errors in data analysis, the coding categories were constructed using Creswell's [15] coding techniques for data extraction and interpretation.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [15] Creswell JW. Educational research: planning, conducting, and evaluating quantitative and qualitative research [Ebook]. 4th ed. London: Pearson Education; 2015.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 1 Introduction

    ChatGPT is considered the fastest-growing technology in history, amassing an estimated 120 million monthly active users and 13 million daily queries in the three months following its public launch.

    “ChatGPT is considered the fastest-growing technology in history: in just three months following its public launch, it amassed an estimated 120 million monthly active users [16] with an estimated 13 million daily queries [49], surpassing all other applications [64].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [16] Curry D. ChatGPT Revenue and Usage Statistics (2023)—Business of Apps. 2023. https://www.businessofapps.com/data/chatgpt-statistics/

      Not checked in the free run.

    • cites [49] Meyer JG, Urbanowicz RJ, Martin P, O’Connor K, Li R, Peng P, Moore JH. ChatGPT and large language models in academia: opportunities and challenges. Biodata Min. 2023. https://doi.org/10.1186/s13040-023-00339-9. source

      Not checked in the free run.

    • cites [64] Rudolph J, Tan S, Tan S. War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.1.23. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 1 Introduction

    ChatGPT has been highlighted for fostering students' performance, engagement, and motivation and enhancing teaching practices.

    “foster students’ performance, engagement and motivation [14, 44, 58], and enhance teaching practices [17, 18, 64, 74].”
    Quote in the paperNot a substantial claim0 of 7 sources checked
    • cites [14] Crawford J, Cowling M, Allen K. Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). J Univ Teach Learn Pract. 2023. https://doi.org/10.53761/1.20.3.02. source

      Not checked in the free run.

    • cites [44] Lee H. The rise of ChatGPT: exploring its potential in medical education. Anat Sci Educ. 2023. https://doi.org/10.1002/ase.2270. source

      Not checked in the free run.

    • cites [58] Rahman MM, Watanobe Y (2023) ChatGPT for education and research: opportunities, threats, and strategies. Appl Sci 13(9):5783. https://doi.org/10.3390/app13095783 source

      Not checked in the free run.

    • cites [17] Day T. A preliminary investigation of fake peer-reviewed citations and references generated by ChatGPT. Prof Geogr. 2023. https://doi.org/10.1080/00330124.2023.2190373. source

      Not checked in the free run.

    • cites [18] De Castro CA. A Discussion about the Impact of ChatGPT in education: benefits and concerns. J Bus Theor Pract. 2023;11(2):p28. https://doi.org/10.22158/jbtp.v11n2p28. source

      Not checked in the free run.

    • cites [64] Rudolph J, Tan S, Tan S. War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.1.23. source

      Not checked in the free run.

    • cites [74] Su J, Yang W. Unlocking the power of ChatGPT: a framework for applying generative AI in education. ECNU Rev Educ. 2023. https://doi.org/10.1177/20965311231168423. source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductcitation
    § 2 Methodology

    The review followed PRISMA 2020 guidelines and Okoli's steps for conducting a rigorous and transparent systematic review.

    “To conduct this study, the authors followed the essential steps of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA 2020) and Okoli’s [54] steps for conducting a systematic review.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [54] Okoli C. A guide to conducting a standalone systematic literature review. Commun Assoc Inf Syst. 2015. https://doi.org/10.17705/1cais.03743. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 1 Introduction

    Concerns have been raised that ChatGPT may disseminate false information and references, reinforce bias, compromise academic integrity, and cause potential decline in students' skills.

    “These include the dissemination of false information and references [12, 23, 61, 85], biased reinforcement [47, 50], compromised academic integrity [18, 40, 66, 74], and the potential decline in students' skills [43, 61, 64, 74].”
    Quote in the paperNot a substantial claim0 of 10 sources checked
    • cites [12] Choi EPH, Lee JJ, Ho M, Kwok JYY, Lok KYW. Chatting or cheating? The impacts of ChatGPT and other artificial intelligence language models on nurse education. Nurse Educ Today. 2023;125:105796. https://doi.org/10.1016/j.nedt.2023.105796. source

      Not checked in the free run.

    • cites [23] Fergus S, Botha M, Ostovar M. Evaluating academic answers generated using ChatGPT. J Chem Educ. 2023;100(4):1672–5. https://doi.org/10.1021/acs.jchemed.3c00087. source

      Not checked in the free run.

    • cites [61] Ratnam M, Sharm B, Tomer A. ChatGPT: educational artificial intelligence. Int J Adv Trends Comput Sci Eng. 2023;12(2):84–91. https://doi.org/10.30534/ijatcse/2023/091222023. source

      Not checked in the free run.

    • cites [85] Wagner MW, Ertl-Wagner BB. Accuracy of information and references using ChatGPT-3 for retrieval of clinical radiological information. Can Assoc Radiol J. 2023. https://doi.org/10.1177/08465371231171125. source

      Not checked in the free run.

    • cites [47] Lo CK. What is the impact of ChatGPT on education? A rapid review of the literature. Educ Sci. 2023;13(4):410. https://doi.org/10.3390/educsci13040410. source

      Not checked in the free run.

    • cites [50] Mhlanga D. Open AI in education, the responsible and ethical use of ChatGPT towards lifelong learning. Soc Sci Res Netw. 2023. https://doi.org/10.2139/ssrn.4354422. source

      Not checked in the free run.

    • cites [18] De Castro CA. A Discussion about the Impact of ChatGPT in education: benefits and concerns. J Bus Theor Pract. 2023;11(2):p28. https://doi.org/10.22158/jbtp.v11n2p28. source

      Not checked in the free run.

    • cites [40] Khan RA, Jawaid M, Khan AR, Sajjad M. ChatGPT—Reshaping medical education and clinical management. Pak J Med Sci. 2023. https://doi.org/10.12669/pjms.39.2.7653. source

      Not checked in the free run.

    • cites [66] Sallam M, Salim NA, Barakat M, Al-Tammemi AB. ChatGPT applications in medical, dental, pharmacy, and public health education: a descriptive study highlighting the advantages and limitations. Narra J. 2023;3(1): e103. https://doi.org/10.52225/narra.v3i1.103. source

      Not checked in the free run.

    • cites [74] Su J, Yang W. Unlocking the power of ChatGPT: a framework for applying generative AI in education. ECNU Rev Educ. 2023. https://doi.org/10.1177/20965311231168423. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 1 Introduction

    ChatGPT has been banned in multiple countries and various educational institutions.

    “As a result, ChatGPT has been banned in multiple countries, including Russia, China, Venezuela, Belarus, and Iran, as well as in various educational institutions in India, Italy, Western Australia, France, and the United States [52, 90].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [52] Nolan B. Here are the schools and colleges that have banned the use of ChatGPT over plagiarism and misinformation fears. Business Insider. 2023. https://www.businessinsider.com

      Not checked in the free run.

    • cites [90] Yu H. Reflection on whether Chat GPT should be banned by academia from the perspective of education and teaching. Front Psychol. 2023;14:1181712. https://doi.org/10.3389/fpsyg.2023.1181712. source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation
    § 1 Introduction

    ChatGPT has potential benefits, limitations, challenges, and threats to education.

    “This indicates the necessity for further exploration to gain a deeper understanding of this technology and carefully evaluate its potential benefits, limitations, challenges, and threats to education [79].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [79] Tedre M, Kahila J, Vartiainen H. (2023). Exploration on how co-designing with AI facilitates critical evaluation of ethics of AI in craft education. In: Langran E, Christensen P, Sanson J (Eds). Proceedings of Society for Information Technology and Teacher Education International Conference. 2023. pp. 2289–2296.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationcitation
    § 1 Introduction

    The advent of ChatGPT has provoked significant controversy due to its potential impact on learning and teaching.

    “Clearly, the advent of chatbots, especially ChatGPT, has provoked significant controversy due to their potential impact on learning and teaching.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 1 Introduction

    In answering 30 mathematical questions and logic problems, ChatGPT was superior to Bard on Set A (unavailable online), while Bard had an advantage on Set B (available online) due to internet access.

    “The results revealed ChatGPT's superiority in Set (A) over Bard. Nevertheless, Bard's advantage emerged in Set (B) due to its capacity to access the internet directly and retrieve answers, a capability that ChatGPT does not possess [57].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [57] Plevris V, Papazafeiropoulos G, Rios AJ. Chatbots put to the test in math and logic problems: A preliminary comparison and assessment of ChatGPT-3.5, ChatGPT-4, and Google Bard. arXiv (Cornell University). 2023. https://doi.org/10.48550/arxiv.2305.18618 source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 6 Discussion

    Emerging technological advancements have opened up novel opportunities and means to support teaching and learning practices and enhance teachers' productivity.

    “emerging technological advancements have opened up novel opportunities and means to support teaching and learning practices, and enhance teachers’ productivity”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites Day [17] Day T. A preliminary investigation of fake peer-reviewed citations and references generated by ChatGPT. Prof Geogr. 2023. https://doi.org/10.1080/00330124.2023.2190373. source

      Not checked in the free run.

    • cites De Castro [18] De Castro CA. A Discussion about the Impact of ChatGPT in education: benefits and concerns. J Bus Theor Pract. 2023;11(2):p28. https://doi.org/10.22158/jbtp.v11n2p28. source

      Not checked in the free run.

    • cites Su and Yang [74] Su J, Yang W. Unlocking the power of ChatGPT: a framework for applying generative AI in education. ECNU Rev Educ. 2023. https://doi.org/10.1177/20965311231168423. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § 6 Discussion

    Personalization of learning through ChatGPT enhances the learning environment, leading to increased academic success.

    “This personalization enhances the learning environment, leading to increased academic success.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § 6 Discussion

    ChatGPT has the potential to offer personalized learning experiences and support an effective learning process by providing students with customized feedback and explanations tailored to their needs and abilities.

    “ChatGPT has the potential to offer personalized learning experiences and support an effective learning process by providing students with customized feedback and explanations tailored to their needs and abilities”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [26] Firat, M. (2023). How chat GPT can transform autodidactic experiences and open education. Department of Distance Education, Open Education Faculty, Anadolu Unive.‏ https://orcid.org/0000-0001-8707-5918 source

      Not checked in the free run.

    • cites [27] Firat M. What ChatGPT means for universities: perceptions of scholars and students. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.1.22. source

      Not checked in the free run.

    • cites [28] Fuchs K. Exploring the opportunities and challenges of NLP models in higher education: is Chat GPT a blessing or a curse? Front Educ. 2023. https://doi.org/10.3389/feduc.2023.1166682. source

      Not checked in the free run.

    • cites [60] Rasul T, Nair S, Kalendra D, Robin M, de Oliveira Santini F, Ladeira WJ, Heathcote L. The role of ChatGPT in higher education: benefits, challenges, and future research directions. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.1.29. source

      Not checked in the free run.

    • cites [66] Sallam M, Salim NA, Barakat M, Al-Tammemi AB. ChatGPT applications in medical, dental, pharmacy, and public health education: a descriptive study highlighting the advantages and limitations. Narra J. 2023;3(1): e103. https://doi.org/10.52225/narra.v3i1.103. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § 6 Discussion

    The adoption of ChatGPT introduces new opportunities to support teaching and learning practices, enhancing teacher productivity.

    “This adoption of technology introduces new opportunities to support teaching and learning practices, enhancing teacher productivity.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § 6 Discussion

    Chatbots, including ChatGPT, play a crucial role in supporting student learning, enhancing their learning experiences, and facilitating diverse learning approaches.

    “Chatbots, including ChatGPT, play a crucial role in supporting student learning, enhancing their learning experiences, and facilitating diverse learning approaches [42, 43]”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [42] Kooli C. Chatbots in education and research: a critical examination of ethical implications and solutions. Sustainability. 2023;15(7):5614. https://doi.org/10.3390/su15075614. source

      Not checked in the free run.

    • cites [43] Kuhail MA, Alturki N, Alramlawi S, Alhejori K. Interacting with educational chatbots: a systematic review. Educ Inf Technol. 2022;28(1):973–1018. https://doi.org/10.1007/s10639-022-11177-3. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 6 Discussion

    Plagiarism detection tools such as Turnitin, ZeroGPT, GPTZero, and Copyleaks have evolved, but studies found they are still not fully ready to accurately and reliably identify AI-generated text.

    “Turnitin and other similar plagiarism detection tools, such as ZeroGPT, GPTZero, and Copyleaks, have since evolved, incorporating enhanced techniques to detect AI-generated text, despite the possibility of false positives, as noted in different studies that have found these tools still not yet fully ready to accurately and reliably identify AI-generated text [10, 51]”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [10] Chaka C. Detecting AI content in responses generated by ChatGPT, YouChat, and Chatsonic: the case of five AI content detection tools. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.2.12. source

      Not checked in the free run.

    • cites [51] Neumann, M., Rauschenberger, M., & Schön, E. M. (2023). “We Need To Talk About ChatGPT”: The Future of AI and Higher Education.‏ https://doi.org/10.1109/seeng59157.2023.00010 source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 6 Discussion

    Incorporating chatbots such as ChatGPT into education presents challenges related to accuracy and reliability due to their training on a large corpus of data, which may contain inaccuracies, and the way users formulate or ask ChatGPT.

    “incorporating chatbots such as ChatGPT, into education presents challenges related to its accuracy and reliability due to its training on a large corpus of data, which may contain inaccuracies and the way users formulate or ask ChatGPT”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [6] Baidoo-Anu D, Ansah L. Education in the Era of generative artificial intelligence (AI): understanding the potential benefits of ChatGPT in promoting teaching and learning. Soc Sci Res Netw. 2023. https://doi.org/10.2139/ssrn.4337484. source

      Not checked in the free run.

    • cites [30] Gilson A, Safranek CW, Huang T, Socrates V, Chi L, Taylor A, Chartash D. How does ChatGPT perform on the United States medical Licensing examination? The implications of large language models for medical education and knowledge assessment. JMIR Med Educ. 2023;9: e45312. https://doi.org/10.2196/45312. source

      Not checked in the free run.

    • cites [35] Househ M, AlSaad R, Alhuwail D, Ahmed A, Healy MG, Latifi S, Sheikh J. Large Language models in medical education: opportunities, challenges, and future directions. JMIR Med Educ. 2023;9: e48291. https://doi.org/10.2196/48291. source

      Not checked in the free run.

    • cites [40] Khan RA, Jawaid M, Khan AR, Sajjad M. ChatGPT—Reshaping medical education and clinical management. Pak J Med Sci. 2023. https://doi.org/10.12669/pjms.39.2.7653. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning
    § 7 Conclusions

    This systematic review has highlighted the transformative potential of ChatGPT in education.

    “this systematic review has highlighted the transformative potential of ChatGPT in education”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 6 Discussion

    Chatbots present a substantial challenge concerning maintaining academic integrity and copyright violations, which are significant concerns in education.

    “chatbots present a substantial challenge concerning maintaining academic integrity [20, 56] and copyright violations [83], which are significant concerns in education”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [20] Eke DO. ChatGPT and the rise of generative AI: threat to academic integrity? J Responsib Technol. 2023;13:100060. https://doi.org/10.1016/j.jrt.2023.100060. source

      Not checked in the free run.

    • cites [56] Perkins M. Academic integrity considerations of AI large language models in the post-pandemic era: ChatGPT and beyond. J Univ Teach Learn Pract. 2023. https://doi.org/10.53761/1.20.02.07. source

      Not checked in the free run.

    • cites [83] Vartiainen H, Tedre M. Using artificial intelligence in craft education: crafting with text-to-image generative models. Digit Creat. 2023;34(1):1–21. https://doi.org/10.1080/14626268.2023.2174557. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductown data
    § Appendix

    Inclusion/exclusion criteria were applied in the review.

    “Following this, inclusion/exclusion criteria were applied”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § 6 Discussion

    Chatbots, including ChatGPT, have demonstrated the potential to reshape and revolutionize education, creating new opportunities for enhancing the learning process and outcomes, facilitating different learning approaches, and offering a range of pedagogical benefits.

    “chatbots, including ChatGPT, have demonstrated the potential to reshape and revolutionize education, creating new, novel opportunities for enhancing the learning process and outcomes [83], facilitating different learning approaches, and offering a range of pedagogical benefits [19, 43, 72]”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [83] Vartiainen H, Tedre M. Using artificial intelligence in craft education: crafting with text-to-image generative models. Digit Creat. 2023;34(1):1–21. https://doi.org/10.1080/14626268.2023.2174557. source

      Not checked in the free run.

    • cites [19] Deng X, Yu Z. A meta-analysis and systematic review of the effect of Chatbot technology use in sustainable education. Sustainability. 2023;15(4):2940. https://doi.org/10.3390/su15042940. source

      Not checked in the free run.

    • cites [43] Kuhail MA, Alturki N, Alramlawi S, Alhejori K. Interacting with educational chatbots: a systematic review. Educ Inf Technol. 2022;28(1):973–1018. https://doi.org/10.1007/s10639-022-11177-3. source

      Not checked in the free run.

    • cites [72] Sonderegger S, Seufert S. Chatbot-mediated learning: conceptual framework for the design of Chatbot use cases in education. Gallen: Institute for Educational Management and Technologies, University of St; 2022. https://doi.org/10.5220/0010999200003182. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 6 Discussion

    A fundamental challenge with LLM-based chatbots, including ChatGPT, is the accuracy and quality of the provided information and responses, as they provide false information as truth, a phenomenon often referred to as hallucination.

    “a fundamental challenge with LLM-based chatbots, including ChatGPT, is the accuracy and quality of the provided information and responses, as they provide false information as truth—a phenomenon often referred to as “hallucination” [3, 49]”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [3] Alkaissi H, McFarlane SI. Artificial hallucinations in ChatGPT: implications in scientific writing. Cureus. 2023. https://doi.org/10.7759/cureus.35179. source

      Not checked in the free run.

    • cites [49] Meyer JG, Urbanowicz RJ, Martin P, O’Connor K, Li R, Peng P, Moore JH. ChatGPT and large language models in academia: opportunities and challenges. Biodata Min. 2023. https://doi.org/10.1186/s13040-023-00339-9. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation
    § Appendix

    Data extraction was performed using Creswell's coding techniques to capture key information and identify common themes across the included studies.

    “data extraction was performed using Creswell's [15] coding techniques to capture key information and identify common themes across the included studies”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Creswell's [15] Creswell JW. Educational research: planning, conducting, and evaluating quantitative and qualitative research [Ebook]. 4th ed. London: Pearson Education; 2015.

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation
    § 6 Discussion

    Previous studies indicate the importance of rethinking, improving, and redesigning innovative assessment methods in the era of chatbots.

    “The findings from previous studies indicate the importance of rethinking, improving, and redesigning innovative assessment methods in the era of chatbots [14, 20, 64, 75]”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [14] Crawford J, Cowling M, Allen K. Leadership is needed for ethical ChatGPT: Character, assessment, and learning using artificial intelligence (AI). J Univ Teach Learn Pract. 2023. https://doi.org/10.53761/1.20.3.02. source

      Not checked in the free run.

    • cites [20] Eke DO. ChatGPT and the rise of generative AI: threat to academic integrity? J Responsib Technol. 2023;13:100060. https://doi.org/10.1016/j.jrt.2023.100060. source

      Not checked in the free run.

    • cites [64] Rudolph J, Tan S, Tan S. War of the chatbots: Bard, Bing Chat, ChatGPT, Ernie and beyond. The new AI gold rush and its impact on higher education. J Appl Learn Teach. 2023. https://doi.org/10.37074/jalt.2023.6.1.23. source

      Not checked in the free run.

    • cites [75] Sullivan M, Kelly A, McLaughlan P. ChatGPT in higher education: Considerations for academic integrity and student learning. J ApplLearn Teach. 2023;6(1):1–10. https://doi.org/10.37074/jalt.2023.6.1.17. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationreasoning
    § 6 Discussion

    The concern about teachers lacking the necessary skills to use ChatGPT could lead to inequity of access to the benefits of chatbots, including ChatGPT, resulting in disparities in educational outcomes and opportunities.

    “This concern could lead to inequity of access to the benefits of chatbots, including ChatGPT, as individuals who lack the necessary expertise may not be able to harness their full potential, resulting in disparities in educational outcomes and opportunities.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § 6 Discussion

    Chatbots, including ChatGPT, have the potential to significantly impact students' thinking abilities, including retention, reasoning, and analysis skills, and to foster innovation and creativity capabilities.

    “chatbots, including ChatGPT, have the potential to significantly impact students' thinking abilities, including retention, reasoning, analysis skills [19, 45], and foster innovation and creativity capabilities [83]”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [19] Deng X, Yu Z. A meta-analysis and systematic review of the effect of Chatbot technology use in sustainable education. Sustainability. 2023;15(4):2940. https://doi.org/10.3390/su15042940. source

      Not checked in the free run.

    • cites [45] Li L, Subbareddy R, Raghavendra CG. AI intelligence Chatbot to improve students learning in the higher education platform. J Interconnect Netw. 2022. https://doi.org/10.1142/s0219265921430325. source

      Not checked in the free run.

    • cites [83] Vartiainen H, Tedre M. Using artificial intelligence in craft education: crafting with text-to-image generative models. Digit Creat. 2023;34(1):1–21. https://doi.org/10.1080/14626268.2023.2174557. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 7 Conclusions

    The utilization of the original version of ChatGPT (GPT-3 or GPT-3.5) implies that new studies utilizing GPT-4 may lead to different findings.

    “the utilization of the original version of ChatGPT, based on GPT-3 or GPT-3.5, implies that new studies utilizing the updated version, GPT-4 may lead to different findings”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The original version of ChatGPT is based on GPT-3 or GPT-3.5.
89 of 89

  • The paper relies on this without support.

    The existing literature lacks a systematic literature review of empirical studies.

    “However, the existing literature lacks a systematic literature review of empirical studies.”

  • The paper relies on this without support.

    ChatGPT is widely adopted by students and teachers for various educational purposes, often without a thorough understanding of responsible and effective use or a clear recognition of its potential impact on learning and teaching.

    “Given the widespread adoption of ChatGPT by students and teachers for various educational purposes, often without a thorough understanding of responsible and effective use or a clear recognition of its potential impact on learning and teaching, the authors recognized the need for further exploration of ChatGPT's impact on education in this early stage.”

  • The paper relies on this without support.

    The 14 selected studies are representative of empirical research on ChatGPT in education, so that the review's synthesized conclusions can be generalized to ChatGPT in education.

    “From the analysis of 14 selected studies, the review revealed the dual-edged impact of ChatGPT in educational settings.”

  • The paper relies on this without support.

    The findings reported by the reviewed studies are valid and reliable enough to be synthesized into review-level conclusions.

    “The findings of the reviewed studies showed several ways of ChatGPT utilization in different learning and teaching practices as well as it provided insights and considerations that can facilitate its effective and responsible use in future educational contexts.”

  • The paper relies on this without support.

    New technologies often face initial skepticism and cautious adoption.

    “as new technologies often face initial skepticism and cautious adoption”

  • The paper relies on this without support.

    There is a lack of clear guidelines or best practices for leveraging ChatGPT for educational purposes.

    “the lack of clear guidelines or best practices for leveraging this technology for educational purposes”

  • The paper relies on this without support.

    Institutional or governmental policies affect the utilization of ChatGPT in educational contexts.

    “institutional or governmental policies affecting the utilization of this technology in educational contexts”

  • The paper relies on this without support.

    The original version of ChatGPT is based on GPT-3 or GPT-3.5.

    “the utilization of the original version of ChatGPT, based on GPT-3 or GPT-3.5”
8 of 8

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The growing body of literature on ChatGPT in education reflects a wide range of classroom applications, methodological approaches, and disciplinary perspectives.

    “The growing body of literature on ChatGPT in education reflects a wide range of classroom applications, methodological approaches, and disciplinary perspectives [34].”

    From Fluency Illusion: A Review on Influence of ChatGPT in Classroom Settings (2026) · cites this paper as [34]

  • ChatGPT improved students' learning process, supported their learning approaches, enhanced their competencies, and contributed to their academic success.

    “Similarly, another review by Albadarin et al. (2024) on ChatGPT affirmed thatChatGPT, improved students’ learning process, supported their learning approaches, enhanced their competencies, and contributed to their academic success (Albadarin et al., 2024).”

    From Current trends and future implications in the utilization of ChatGPT in nursing: a rapid review (2024) · cites this paper as Albadarin et al., 2024

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Journalism and construction findings cite studies assigned to different fields

    Minor

    The competencies section labels S5 as journalism and S9 as construction education. The field breakdown instead assigns S5 to mathematics, S9 to journalism, and S3 to construction education.

  • A student-writing finding cites a teacher-only study

    Minor

    The writing section attributes a finding about students' mastery to S11, whereas the population section says S11 specifically targeted elementary school teachers.

  • S6 appears in two separate study-population categories

    Minor

    S6 is listed both among studies involving university students and among those focused on university teachers, rather than in the category for studies including both.

  • S4 is classified as qualitative despite a reported mixed-methods design

    Minor

    The methodology classification places S4 among qualitative studies, while the sample-size discussion explicitly calls it a mixed-methods study with qualitative and quantitative analyses.

  • S4 also had more than 100 participants in one analysis

    Minor

    The text says S7 was the only study with more than 100 participants, but it reports 110 participants for S4's quantitative analysis alone.

  • The list of small-sample studies contains eight, not nine

    Minor

    The four individually named studies plus S10 through S13 make eight studies, although the text says nine.

  • S12 and S14 have both unspecified and university-level populations

    Minor

    The educational-level section says S12 and S14 did not specify their populations' educational level, but the population section identifies both as including university teachers and students.

7 of 7

Cites

Flagged first, then by citations

Research works printed but not linked: 7

  1. Ausat AMA, Massang B, Efendi M, Nofirman N, Riady Y. Can chat GPT replace the role of the teacher in the classroom: a fundamental analysis. J Educ. 2023;5(4):16100–6.
  2. Creswell JW. Educational research: planning, conducting, and evaluating quantitative and qualitative research [Ebook]. 4th ed. London: Pearson Education; 2015.
  3. Hirsto L, Saqr M, López-Pernas S, Valtonen T. (2022). A systematic narrative review of learning analytics research in K-12 and schools. Proceedings. https://ceur-ws.org/Vol-3383/FLAIEC22_paper_9536.pdf
  4. Limna P. A Review of Artificial Intelligence (AI) in Education during the Digital Era. 2022. https://ssrn.com/abstract=4160798
  5. Saqr M, Matcha W, Uzir N, Jovanović J, Gašević D, López-Pernas S. Transferring effective learning strategies across learning contexts matters: a study in problem-based learning. Australas J Educ Technol. 2023;39(3):9.
  6. Szabo A. ChatGPT is a breakthrough in science and education but fails a test in sports and exercise psychology. Balt J Sport Health Sci. 2023;1(128):25–40. https://doi.org/10.33607/bjshs.v127i4.1233. · DOI
  7. Tedre M, Kahila J, Vartiainen H. (2023). Exploration on how co-designing with AI facilitates critical evaluation of ethics of AI in craft education. In: Langran E, Christensen P, Sanson J (Eds). Proceedings of Society for Information Technology and Teacher Education International Conference. 2023. pp. 2289–2296.

Cited by

Sorted by citations

Related

Authors
Albadarin, Y., Saqr, M., Pope, N., & Tukiainen, M.
Affiliations
University of Eastern Finland (FI)
Corresponding author
Albadarin, Yazid
ORCID
Venue
Discover Education
Publisher
Springer Science and Business Media LLC
Published
2024-05-26
Volume
3
Issue
1
ISSN
2731-5525 (ISSN-L)
DOI
10.1007/s44217-024-00138-2
OpenAlex
W4399028114
Semantic Scholar
cd9e8c91a1262a02ec40ed650106b0d84e58ec36
Cited by
191
References
81
Citations by year
2026: 98 · 2025: 73 · 2024: 19 · 2012: 1
FWCI
11.30
Type
journal article
Language
en
Open access
Licence
Locations
  • Discover Education (publishedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
Keywords
ChatGPT, generative AI in education, artificial intelligence in education, student use of AI, teacher use of AI, personalized learning, cooperative learning, critical evaluation skills, responsible AI use, overreliance on AI, lesson planning, human-AI collaboration, AI policy in education

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-09.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-09.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review