CoChat
Sign Up
Cited by 76Artificial Intelligence in EducationArtificial Intelligence in Healthcare and EducationAcademic integrity and plagiarism

Artificial intelligence-powered tools and academic writing: to use or not to use ChatGPT

Open access
Al-Sofi, Bakr·2024·Saudi Journal of Language Studies
DOI 10.1108/sjls-06-2024-0029
Open this paper in CoChat — free accountOpen PDF

Abstract

Purpose This study investigates the potential effectiveness of ChatGPT in enhancing the academic writing skills of Saudi EFL undergraduate students. It also examines the challenges associated with its use and suggests effective ways to address them in the education sector. Design/methodology/approach The study employed a sequential mixed-methods approach, which involved distributing questionnaires to gather data from students, followed by conducting semi-structured interviews with a purposeful selection of eight students and six teachers. Findings The findings revealed that students were generally satisfied with the effectiveness of ChatGPT in enhancing their academic writing skills. However, they also pinpointed some challenges associated with using ChatGPT, including plagiarism, overreliance, inadequate documentation, threats to academic integrity, and inaccurate information. To alleviate these challenges, effective strategies include deploying detection tools, equipping students and educators with training sessions, and revisiting academic policies and assessment methods. It is recommended that ChatGPT be used responsibly as an assistant tool, in conjunction with students' ideas and teachers' feedback. This approach can significantly enhance students' writing skills and facilitate completing their research projects and assignments. Practical implications ChatGPT can be a valuable tool in the educational landscape, but it is essential to use it judiciously. Therefore, teachers' effective integration of ChatGPT into their classrooms can significantly enhance students' writing abilities and streamline their research process. Originality/value This study contributes to recent AI-based research and provides practical insights on the responsible integration of ChatGPT into education while addressing potential challenges.

AI snapshot

Written from the paper’s full text.

TL;DR

This mixed-methods study found Saudi EFL undergraduates generally agreed ChatGPT enhanced their academic writing, with an overall effectiveness mean of 3.93, but also identified challenges such as plagiarism, overreliance, and inaccurate information. It recommends using ChatGPT responsibly as an assistant tool alongside students’ own ideas and teacher feedback.

Key findings

  • Students generally agreed ChatGPT enhanced academic writing: overall weighted mean 3.93 (“agree”); highest items were generating helpful texts and summarizing literature (both M = 4.06), enhancing writing style (M = 4.04), and lowest was staying up-to-date with research trends (M = 3.80).
  • Challenges were rated neutral overall (M = 3.21); the highest-ranked challenge was ChatGPT potentially substituting for a teacher in feedback (M = 3.54), while inaccurate/misleading content was lowest (M = 2.81). Interviews raised plagiarism, overreliance, missing references, and reduced creativity/critical thinking.
  • Strategies to mitigate challenges were agreed overall (M = 3.87); top strategies were training on effective ChatGPT use (M = 4.27), using it for proofreading/editing already-written texts (M = 4.16), and critically evaluating its information (M = 4.12). The lowest strategy item, using ChatGPT as an assistant rather than relying on it exclusively, had M = 3.94.
  • Six interviewed teachers unanimously agreed ChatGPT enhanced students’ academic writing; students highlighted time-saving, user-friendly features, brainstorming, summarizing, and proofreading, with some also using it for voice modification, image processing, website design, and translation.

Methods

  • Sequential mixed-methods design: an online questionnaire was distributed first, followed by semi-structured interviews with 8 purposefully selected students and 6 teachers.
  • Participants were 102 of 117 Saudi EFL undergraduate students in the Department of English at the University of Bisha; 60 (58.8%) were enrolled in Basics of Scientific Research in English and 42 (41.2%) in Research Project. Data were collected in the 2023/2024 academic year via WhatsApp groups.
  • Questionnaire data were analyzed with descriptive statistics (frequencies and percentages) in SPSS Version 26; interview data were analyzed using content analysis with codes E-X, C-X, and S-X, then integrated side-by-side. Questionnaire reliability was reported as Cronbach’s alpha = 0.965.

Limitations

  • Stated: the study is limited to 102 students enrolled in two courses in one department at the University of Bisha; future research should include larger, more diverse samples including postgraduates. The paper also notes ChatGPT is relatively new and not widely adopted, so participants with greater exposure may report different perspectives.
  • Evident from the text: the study relies on self-reported questionnaire and interview perceptions rather than direct measures of writing performance; the paper itself notes students’ cautious reporting may lead to under-reporting of challenges.
  • Evident from the text: the qualitative sample is small (8 students and 6 teachers), which limits generalizability of the interview findings.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1108/sjls-06-2024-0029

Cites

32 of 32 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 31 cited sources checked · 0 supported · 0 flagged

  • Findingreasoning

    ChatGPT scored the lowest mean in keeping students up to date with the latest research trends (M = 3.80).

    “ChatGPT scored the lowest mean in keeping students up to date with the latest research trends (M 5 3.80).”
    Quote in the paperSubstantial claim
  • Findingown data

    The interviews revealed unanimous agreement among the six teachers regarding the effective role of ChatGPT in enhancing their students' academic writing skills.

    “The interviews revealed unanimous agreement among the six teachers regarding the effective role of ChatGPT in enhancing their students' academic writing skills.”
    Quote in the paperSubstantial claim
  • Findingown data

    The weighted mean for each item on the scale fell within the agree category.

    “and the weighted mean for each item on the scale also falls within the”
    Quote in the paperSubstantial claim
  • Findingown data

    Three interviewed students (G, H, and I) confirmed using ChatGPT to brainstorm ideas and choose a topic for their project.

    “Regarding the brainstorming theme, three of the interviewed students (G, H, and I) confirmed using ChatGPT to brainstorm ideas and choose a topic for their project.”
    Quote in the paperNot a substantial claim
  • Findingown data

    Students identified plagiarism, overreliance, inadequate documentation, threats to academic integrity, and inaccurate information as challenges associated with using ChatGPT.

    “However, they also pinpointed some challenges associated with using ChatGPT, including plagiarism, overreliance, inadequate documentation, threats to academic integrity, and inaccurate information.”
    Quote in the paperSubstantial claim
  • Classified as probably not a substantial claim. 32
  • May combine two claims. 29
  • The quote only partly states this claim. 3

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    These students highlighted ChatGPT's utility as a time-saving, cost-effective, and effort-reducing assistant tool for their studies.

    “These students highlighted ChatGPT's utility as a time-saving, cost-effective, and effort-reducing assistant tool for their studies.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Students interviewed emphasized that ChatGPT's standout features are its time-saving and user-friendly nature.

    “The students interviewed emphasized that ChatGPT's standout features are its timesaving and user-friendly nature.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The overall weighted mean for the entire scale was 3.93, falling within the agree category.

    “It is worth mentioning that the overall weighted mean (M 5 3.93) for the entire scale falls within the”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingreasoning

    Effective strategies to alleviate the challenges of using ChatGPT include deploying detection tools, equipping students and educators with training sessions, and revisiting academic policies and assessment methods.

    “To alleviate these challenges, effective strategies include deploying detection tools, equipping students and educators with training sessions, and revisiting academic policies and assessment methods.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing). May combine two claims.

    Findingown data

    A teacher raised the concern that students may not fully understand the concepts they write about, as they may merely copy and paste information provided by ChatGPT without genuinely understanding the subject matter.

    “The teacher raised the concern that students may not fully understand the concepts they write about, as they may merely copy and paste information provided by ChatGPT without genuinely understanding the subject matter.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The overall weighted mean for the challenges scale was 3.21, falling in the neutral category, indicating respondents neither agreed nor disagreed regarding challenges associated with using ChatGPT to enhance academic writing skills.

    “Table 3 displays that the overall weighted mean (M 5 3.21) for the entire scale fell within the”
    Quote in the paperNot a substantial claim

  • The quote only partly states this claim.

    Findingown data

    Participants agreed (M=3.23) that the use of ChatGPT compromised the integrity of academic writing.

    “Using ChatGPT in academic research writing reduces the integrity of academic writing N 11 30 10 27 24 3.23 1.378 3”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Teacher 4 disclosed that students simply copy and paste and do not know how to paraphrase generated ideas.

    “Teacher 4 disclosed that the students”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Respondents expressed that training sessions focusing on effective use of ChatGPT (M=4.27) and employing ChatGPT for proofreading and editing already-written texts rather than generating texts from scratch (M=4.16) were among the top viable methods to mitigate challenges.

    “More precisely, the respondents expressed that training sessions focusing on the effective use of ChatGPT (M 5 4.27) and employing ChatGPT for proofreading and editing already-written texts rather than generating texts from scratch (M 5 4.16) were among the top viable methods that can be used to mitigate the challenges associated with ChatGPT.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Teachers expressed concerns about the lack of creativity and critical thinking skills resulting from using ChatGPT.

    “The teachers also expressed concerns about the lack of creativity and critical thinking skills, resulting from using ChatGPT.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Teacher 6 stated that ChatGPT seems ineffective in AI-generated content and students do not seem to understand mistakes pointed out in ChatGPT content because it has not been created by them.

    “According to the sixth teacher,”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Teachers' responses about challenges centered on ethical issues and a lack of students' skills.

    “Most of their responses centered on ethical issues and a lack of students' skills.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    25 participants (24.5%) strongly disagreed and 35 participants (34.4%) disagreed that ChatGPT can replace teachers in providing feedback.

    “Specifically, some participants strongly disagreed (N 5 25, 24.5%) and (N 5 35, 34.4%) that ChatGPT can replace teachers in providing feedback.”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Findingown data

    Interviewed students reported that ChatGPT-generated content lacked citations and that information could be unverified.

    “Similarly, students (G, H, and I) mentioned the issue of plagiarism, stating that”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    Students acknowledged that using ChatGPT in academic writing presented several challenges, including a potential decrease in integrity and critical thinking, overreliance leading to a decline in creative abilities, risk of plagiarism, and generation of inaccurate or misleading content.

    “However, they also acknowledge that using ChatGPT in academic writing presented several challenges.These included a potential decrease in the integrity of academic writing and critical thinking, an overreliance on the tool leading to a decline in creative abilities, the risk of plagiarism, and the generation of inaccurate or misleading content.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Participants were somewhat cautious in their questionnaire responses; while they mostly agreed ChatGPT was beneficial, their responses regarding its limitations leaned towards neutrality.

    “However, it was observed that they were somewhat cautious in their responses to the questionnaire.While they mostly agreed that ChatGPT was beneficial, their responses regarding its limitations leaned towards neutrality.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Teacher 5 stated that ChatGPT may create a sense of dependency or complacency among students who rely too heavily on its assistance or output.

    “He further referred to the students' dependence on this tool, stating that”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    ChatGPT proved helpful for generating relevant texts for research work, summarizing literature, enhancing assignments and research papers, and assisting with proofreading.

    “It proved helpful for generating relevant texts for research work, summarizing literature, enhancing assignments and research papers, and assisting with proofreading.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Teacher 5 cited misinformation as a challenge and emphasized that ChatGPT can be a source of distraction if not used productively.

    “Interestingly, teacher 5 cited”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    Teachers' effective integration of ChatGPT into their classrooms can significantly enhance students' writing abilities and streamline their research process.

    “Therefore, teachers' effective integration of ChatGPT into their classrooms can significantly enhance students' writing abilities and streamline their research process.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Using ChatGPT to generate essays and presenting them as one's original work violates academic integrity.

    “Eke (2023) highlighted that using ChatGPT to generate essays and presenting them as one's original work violated academic integrity.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Eke, 2023

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationreasoning

    Challenges pose a difficulty for educators and supervisors in differentiating between AI-generated and human-written texts, potentially impacting academic integrity.

    “These challenges pose a difficulty for educators and supervisors in differentiating between AI-generated and human-written texts, potentially impacting academic integrity.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationown data

    Using ChatGPT responsibly as an assistant tool, in conjunction with students' ideas and teachers' feedback, can significantly enhance students' writing skills and facilitate completing their research projects and assignments.

    “It is recommended that ChatGPT be used responsibly as an assistant tool, in conjunction with students' ideas and teachers' feedback.This approach can significantly enhance students' writing skills and facilitate completing their research projects and assignments.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    ChatGPT is effective in organizing content but should be used with caution, especially in unfamiliar topics, as careful verification may be necessary.

    “Lingard (2023) proposed that while ChatGPT proves to be effective in organizing content, it should be used with caution, especially in unfamiliar topics, as careful verification may be necessary.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Lingard, 2023

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Study design and conductcitation

    The qualitative data enhance the validity of the claims and strengthen the study by offering a nuanced understanding of the topic under investigation.

    “Furthermore, the qualitative data enhance the validity of the claims and strengthen the study by offering a nuanced understanding of the topic under investigation (Mahapatra, 2024).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Mahapatra, 2024

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Interpretationown data

    ChatGPT can be a valuable tool in the educational landscape.

    “ChatGPT can be a valuable tool in the educational landscape, but it is essential to use it judiciously.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    Implementing actionable strategies can alleviate the challenges associated with using ChatGPT and foster the development of students' academic writing skills.

    “Implementing these actionable strategies can alleviate the challenges associated with using ChatGPT and foster the development of students' academic writing skills.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    Irresponsible use of ChatGPT poses challenges and ethical considerations for educators and educational institutions, including overreliance, plagiarism, inaccurate or biased information, improper referencing, a decrease in creativity and critical thinking, difficulties in evaluating ChatGPT-generated content, and misleading assessment of students' assignments.

    “These issues include overreliance, plagiarism, inaccurate or biased information, improper referencing, a decrease in creativity and critical thinking, difficulties in evaluating ChatGPT-generated content, and misleading assessment of students' assignments”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites AlAfnan et al., 2023

      Not checked in the free run.

    • cites Cotton et al., 2023

      Not checked in the free run.

    • cites Eke, 2023

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Increasing students' awareness of the potential benefits and limitations of ChatGPT positively influences their academic characteristics and motivates them to engage more effectively in metacognitive learning.

    “For instance, Kumar (2023) found that increasing students' awareness of the potential benefits and limitations of ChatGPT positively influences their academic characteristics and motivates them to engage more effectively in metacognitive learning.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Kumar, 2023

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The questionnaire's design was revised with the expertise of three linguists in the department.

    “The questionnaire's design underwent revision, benefiting from the expertise of three linguists in the department.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given

    A critical gap persists in academic research concerning the possible effectiveness of ChatGPT in enhancing EFL students' academic writing skills, particularly within the context of performing research projects and assignments.

    “Despite the growing use of ChatGPT, a critical gap persists in academic research concerning the possible effectiveness of this tool in enhancing English as a Foreign Language (EFL) students' academic writing skills, particularly within the context of performing research projects and assignments.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: Some students may excessively depend on ChatGPT or use it irresponsibly to perform their academic writing tasks, which may not accurately showcase their true writing skills.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Premisecitation

    ChatGPT should be employed as a supplementary writing tool to improve structure, brainstorm ideas, and proofread human-generated texts, rather than relying solely on it to generate texts from scratch.

    “However, similar to previously developed AI-driven applications, scholars suggested that ChatGPT should be employed as a supplementary writing tool to improve structure, brainstorm ideas, and proofread human-generated texts, rather than relying solely on it to generate texts from scratch.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationnone given

    ChatGPT has the potential to revolutionize the way users approach academic writing.

    “It has the potential to revolutionize the way users approach academic writing.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning

    Students' irresponsible use of ChatGPT, particularly overreliance on it as the sole tool for writing projects and text-based assignments, can lead to academic dishonesty and hinder the development of essential writing skills.

    “Students' irresponsible use of ChatGPT, particularly overreliance on it as the sole tool for writing projects and text-based assignments, can lead to academic dishonesty and hinder the development of essential writing skills.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Chinese students reported a generally positive view of ChatGPT, valuing its ability to enhance the entire writing process, but raised concerns regarding accuracy, privacy, ethical issues, and the potential impact on personal development, career prospects, and societal values.

    “The respondents reported a generally positive view of ChatGPT, valuing its ability to enhance the entire writing process, from initial brainstorming to final proofreading and editing.However, concerns were raised regarding accuracy, privacy, ethical issues, and the potential impact on personal development, career prospects, and societal values.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Chan and Hu, 2023

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    ChatGPT played a positive role in stimulating students' interest in developing their reading and writing skills.

    “Based on the findings of Bin-Hady et al. (2023) regarding the potential improvement of learners' language abilities through ChatGPT, Ali et al. (2023) found that ChatGPT played a positive role in stimulating students' interest in developing their reading and writing skills.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Bin-Hady et al., 2023

      Not checked in the free run.

    • cites Ali et al., 2023

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    ChatGPT provided students with accurate and reliable information for addressing theoretical and practical questions.

    “The study found that ChatGPT provided students with accurate and reliable information for addressing theoretical and practical questions.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites AlAfnan et al., 2023

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    Participants' cautious reporting may stem from the tool's novelty and students' natural inclination towards objectivity, which could lead to under-reporting of challenges encountered while using ChatGPT.

    “Their cautious reporting may stem from the tool's novelty and students' natural inclination towards objectivity.This could lead to under-reporting of challenges encountered while using ChatGPT.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning

    Like other AI-driven tools, the increasing use of ChatGPT makes it impractical to exclude it.

    “Like other AI-driven editing and proofreading tools (such as Grammarly, Quillbot, Wordtune, and Paperpal), tools for analyzing data (such as SPSS, AMOS, MAXQDA, and NVivo), and tools for referencing (such as Mendeley, Endnote, and Zetero), the increasing use of ChatGPT makes it impractical to exclude it.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    AI can assist students and researchers throughout the writing process, from initial brainstorming to post-publication communication.

    “Their review of 24 studies, retrieved from various databases, revealed that AI can assist students and researchers throughout the writing process, from initial brainstorming to post-publication communication.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Khalifa and Albadawy, 2024

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    The result that students heavily relied on ChatGPT supported the teachers' interviews indicating that students excessively relied on ChatGPT in academic writing tasks.

    “This result supported the findings of the teachers' interviews, which indicated that students excessively relied on ChatGPT in their academic writing tasks.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Although ChatGPT does not fully meet the rigorous standards of scholarly journals, it excels in promptly providing precise information and utilizing flawless language.

    “Consistent with Morgan's (2023) viewpoint, the results showed that although ChatGPT does not fully meet the rigorous standards of scholarly journals, it does excel in promptly providing precise information and utilizing flawless language.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites Mijwil et al., 2023

      Not checked in the free run.

    • cites Morgan, 2023

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    ChatGPT can be valuable as a writing assistant, improving writing skills and research efficiency.

    “The studies confirmed that ChatGPT can be valuable as a writing assistant, improving writing skills and research efficiency.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites Altm€ ae et al., 2023

      Not checked in the free run.

    • cites Dergaa et al., 2023

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Utilizing ChatGPT as a writing assistant had benefits and drawbacks for students and educators.

    “The results indicated that utilizing ChatGPT as a writing assistant had benefits and drawbacks for students and educators.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Imran and Almusharraf, 2023

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Indian undergraduate ESL students had positive perceptions of the role of ChatGPT on their academic writing skills.

    “The results indicated that they had positive perceptions of the role of ChatGPT on their academic writing skills.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Mahapatra, 2024

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data

    AI-based tools should be seen as complementary rather than substitutes for teachers in editing and proofreading students' generated texts.

    “This result confirms the vital role of teachers, highlighting that AI-based tools should be seen as complementary rather than substitutes for teachers in editing and proofreading students' generated texts.”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Interpretationown data

    The challenges highlighted by interviewed teachers and students were evident and reflected prior research and questionnaire responses, specifically plagiarism, excessive reliance on ChatGPT, and lack of creativity and critical thinking skills.

    “Specifically, students and teachers raised primary concerns, such as the potential for plagiarism in ChatGPT-generated texts, excessive reliance on ChatGPT, and a lack of creativity and critical thinking skills.”
    Quote in the paperSubstantial claim
47 of 47

  • The paper relies on this without support.

    Some students may excessively depend on ChatGPT or use it irresponsibly to perform their academic writing tasks, which may not accurately showcase their true writing skills.

    “However, some students may excessively depend on ChatGPT or use it irresponsibly to perform their academic writing tasks, which may not accurately showcase their true writing skills.”

  • The paper relies on this without support.

    The prior studies were conducted shortly after the emergence of the tool, at a time when it was inaccessible in some countries, including Saudi Arabia, and may be limited by stakeholders' unfamiliarity with ChatGPT's capabilities.

    “Notably, the prior studies were conducted shortly after the emergence of the tool, at a time when it was inaccessible in some countries, including Saudi Arabia.Furthermore, these studies may be limited by stakeholders' unfamiliarity with ChatGPT's capabilities.”

  • The paper relies on this without support.

    The present study was conducted after ChatGPT became accessible in Saudi Arabia and people gained sufficient knowledge and experience in its usage.

    “The present study was conducted after ChatGPT became accessible in Saudi Arabia and people gained sufficient knowledge and experience in its usage.”

  • The paper relies on this without support.

    AI is becoming an increasingly integral part of the learning process, moving away from being perceived as a threat or a substitute for teachers.

    “AI is becoming an increasingly integral part of the learning process, moving away from being perceived as a threat or a substitute for teachers.”

  • The paper relies on this without support.

    ChatGPT is a relatively new tool and has not yet been widely adopted by educational institutions.

    “Additionally, it is important to note that ChatGPT is a relatively new tool and has not yet been widely adopted by educational institutions.”
5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Reported benefits vary considerably across studies, with inconsistencies attributable to differences in instructional design, assessment rigor, and intervention duration-limiting cross-study comparability and generalizability of positive findings.

    “Nevertheless, reported benefits vary considerably across studies, with inconsistencies attributable to differences in instructional design, assessment rigor, and intervention duration—limiting cross-study comparability and generalizability of positive findings (Al-Dossary, 2024; Al-Sofi, 2024).”

    From From tool to scaffold: structured human–AI collaboration and its effects on academic writing and digital critical thinking among Saudi EFL learners (2026) · cites this paper as Al-Sofi, 2024

    Split into 2 atoms — 0 confirmed, 0 partially supported, 1 disputed, 0 misattributed, 1 unverifiable. Expand the claim to see per-atom verdicts.

    Checked by a reasoning model — may be wrong.

    Experimental — accuracy not yet measured

  • From a process-writing and self-regulation perspective, the educational value of AI depends on whether it makes the writing process more visible and subject to deliberate revision, rather than outsourcing final text production.

    “From a process-writing and self-regulation perspective, the educational value of AI depends on whether it makes the writing process more visible and subject to deliberate revision, rather than outsourcing final text production (Aljuaid, 2024; Al-Sofi, 2024; Deep and Chen, 2025).”

    From From tool to scaffold: structured human–AI collaboration and its effects on academic writing and digital critical thinking among Saudi EFL learners (2026) · cites this paper as Al-Sofi, 2024

    The Conclusion (page unavailable) recommends using ChatGPT as a writing assistant rather than relying on it to generate ideas from scratch or replace teachers’ feedback. That supports the distinction between assistance and outsourcing, but the paper does not establish that educational value depends on making the writing process visible or subject to deliberate revision. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Writing-focused research in Saudi contexts indicates that EFL learners commonly use AI for idea generation, initial drafting, and surface polishing, reporting improved clarity and correctness alongside mixed feelings regarding originality and authorial ownership.

    “Writing-focused research in Saudi contexts indicates that EFL learners commonly use AI for idea generation, initial drafting, and surface polishing, reporting improved clarity and correctness alongside mixed feelings regarding originality and authorial ownership (Al-Harbi and Al-Ahdal, 2025; Al-Sofi, 2024; Alyami et al., 2025).”

    From From tool to scaffold: structured human–AI collaboration and its effects on academic writing and digital critical thinking among Saudi EFL learners (2026) · cites this paper as Al-Sofi, 2024

    Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.

    Experimental — accuracy not yet measured

  • Generative AI tools such as ChatGPT, Gemini, DeepSeek, and other platforms have become increasingly popular among university students worldwide for academic tasks including writing assistance, research support, coding, and creative ideation.

    “Tools such as ChatGPT, Gemini, DeepSeek, and other generative AI platforms have become increasingly popular among university students worldwide for a range of academic tasks, including writing assistance, research support, coding, and creative ideation (Al-Sofi, 2024).”

    From Six Institutional Intervention Areas to Support Ethical and Effective Student Use of Generative AI in Higher Education: A Narrative Review (2026) · cites this paper as Al-Sofi, 2024

    The Introduction and Results and discussion describe students using ChatGPT for writing, research, and brainstorming; Previous studies calls its use in education widespread. Our copy does not establish increasing worldwide popularity among university students or address Gemini, DeepSeek, or coding as an academic use. (LLM, conf=0.88)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Strategies scale mean does not match its item means

    Major

    The seven item means in Table 4 average 4.05, not the reported overall weighted mean of 3.87. This changes a headline result for the proposed strategies.

    Reported 3.87 · recomputed 4.054286

  • Challenges scale mean does not match its item means

    Minor

    The eight item means in Table 3 average 3.09 rather than the reported 3.21.

    Reported 3.21 · recomputed 3.09

  • Claim of agreement on reduced integrity conflicts with responses

    Minor

    The text says participants agreed that ChatGPT reduces academic-writing integrity, but the corresponding item has 41 agree or strongly agree responses versus 51 disagree or strongly disagree responses. Its reported mean also appears to run in the opposite direction from the stated response scale.

  • Nine student labels appear despite eight reported interviews

    Minor

    The paper reports interviewing eight students, but it identifies interviewed students using labels from A through I, implying nine distinct students.

  • Claimed lowest strategies mean is not the lowest

    Minor

    Table 4 reports a mean of 3.82 for another strategy, below the 3.94 described as the lowest mean.

  • Percentage for 45 respondents is incorrect

    Minor

    In the first item of Table 4, 45 of the 102 respondents represent 44.1%, not 41.1%.

    Reported 41.1 · recomputed 44.117647

  • Disagree responses are called strongly disagree

    Minor

    For the teacher-substitution item, Table 3 places 35 responses under Disagree and 25 under Strongly disagree; the text calls both groups strongly disagree.

  • A five-point scale is described as ending at 2

    Minor

    A scale ranging from 1 to 2 conflicts with its description as five-point and with the reported item means above 2.

8 of 8

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. 2018, Research Design: Qualitative, Quantitative, and Mixed Methods Approaches

Cited by

Sorted by citations

Related

Authors
Al-Sofi, B.
Affiliations
University of Bisha (SA)
Corresponding author
Al-Sofi, Bakr
ORCID
Al-Sofi, Bakr
Venue
Saudi Journal of Language Studies
Publisher
Manarah
Published
2024-08-30
Volume
4
ISSN
  • 2634-243X (ISSN-L)
  • 2634-2448
DOI
10.1108/sjls-06-2024-0029
OpenAlex
W4401980541
Cited by
45
References
30
Citations by year
2026: 16 · 2025: 26 · 2024: 3
FWCI
16.84
Type
journal article
Language
en
Open access
Licence
Locations
Saudi Journal of Language Studies (publishedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Social Sciences › Safety Research › Academic integrity and plagiarism
Keywords
ChatGPT, academic writing, EFL learners, artificial intelligence in education, plagiarism, academic integrity, overreliance on AI, AI detection tools, teacher professional development, responsible AI use, writing assessment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review