The DOI matches its Crossref record; 71 of the 71 references it cites were found in a registry or an index; none of the 9 sources its claims cite is checked for support yet.
ChatGPT for Education and Research: Opportunities, Threats, and Strategies
Open accessPublished version of a preprint
Abstract
In recent years, the rise of advanced artificial intelligence technologies has had a profound impact on many fields, including education and research. One such technology is ChatGPT, a powerful large language model developed by OpenAI. This technology offers exciting opportunities for students and educators, including personalized feedback, increased accessibility, interactive conversations, lesson preparation, evaluation, and new ways to teach complex concepts. However, ChatGPT poses different threats to the traditional education and research system, including the possibility of cheating on online exams, human-like text generation, diminished critical thinking skills, and difficulties in evaluating information generated by ChatGPT. This study explores the potential opportunities and threats that ChatGPT poses to overall education from the perspective of students and educators. Furthermore, for programming learning, we explore how ChatGPT helps students improve their programming skills. To demonstrate this, we conducted different coding-related experiments with ChatGPT, including code generation from problem descriptions, pseudocode generation of algorithms from texts, and code correction. The generated codes are validated with an online judge system to evaluate their accuracy. In addition, we conducted several surveys with students and teachers to find out how ChatGPT supports programming learning and teaching. Finally, we present the survey results and analysis.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This paper examines ChatGPT’s opportunities and threats for education and research, with a focus on programming learning and teaching. It combines coding experiments on an online judge system with surveys of students and teachers, finding generally useful support but also accuracy, integrity, and over-reliance concerns.
Key findings
- In code-generation experiments, ChatGPT’s generated code was correct about 95.83% of the time on a basic compiler and about 75% on the Aizu Online Judge (AOJ), with an average accuracy of 85.42%.
- Among surveyed students, 78.8% had used ChatGPT for programming problems, 86.7% said its suggestions helped solve programming problems, and 92.9% found it useful for learning programming; most student satisfaction ratings were 4/5 (36.6%) or 5/5 (20%).
- About 60% of surveyed teachers had used ChatGPT for teaching programming, but about 50% rated satisfaction as 3/5, suggesting limited trust; one teacher said ChatGPT “is still not perfect to answer some exams of my Java class.”
- The paper identifies threats including cheating on online exams and assignments, diminished critical thinking, difficulty evaluating AI-generated text/code, ethical implications and bias, and blind reliance on generative AI; proposed strategies include clear assignment instructions, plagiarism detection, proctoring, comparing responses, and follow-up questions.
Methods
- The authors conducted coding-related experiments with ChatGPT: code generation from problem descriptions, pseudocode generation, code correction/debugging, and code optimization.
- For code generation, they used eight random “Algorithms and Data Structures” problems from AOJ, generated code three times per problem, and validated results on both a basic compiler and the AOJ platform.
- They also conducted surveys with undergraduate, Master’s, and doctoral students and with teachers about ChatGPT’s support for programming learning and teaching; the paper reports percentages but not the total number of respondents.
Limitations
- The paper states that experiments were conducted on ChatGPT during active development, so results may vary with new ChatGPT versions, different questions, different problem descriptions, and different solution codes for optimization.
- The paper notes that although ChatGPT generated about 85.42% correct code on average, adjustments may still be necessary for error-free compilation and acceptance, and it failed an elementary counting task involving the digit “8.”
- Evident from the text: the survey sample size is not reported, and the coding experiments are limited to eight AOJ problems with three generations each, which restricts generalizability.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 9 cited sources checked · 0 supported · 0 flagged
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the correctness rate of the generated code based on the basic compiler is approximately 95.83%”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 4.2. Solution Code Generation
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ChatGPT generates mostly correct code considering long problem descriptions, including constraints, algorithms, and input and output formatting.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 4.2. Solution Code Generation
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the correctness rate based on AOJ compilation is approximately 75%”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 4.2. Solution Code Generation
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In addition, we experimented with the ChatGPT model to find the derivatives of mathematical equations, and it solved all equations correctly.In this case, the correctness is 100%.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 3.1. Opportunities for Learners
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We also asked ChatGPT about quizzes of other levels, such as medium and easy, and we found that there was a significant difference between quizzes of different levels.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Rapid Assessment and Evaluation:
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ChatGPT can help optimize codes by suggesting ways to reduce the memory usage and time complexity.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 4.4. Solution Code Optimization
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, we asked ChatGPT to “prepare a detailed outline for the Algorithms and Data Structures course”, as depicted in Figure 2. It creates a complete table of contents for the Algorithms and Data Structures course, showing topics and a breakdown of each topic, along with learning objectives.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Lesson Planning:
Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found that most students rated their programming experiences between 1/5 and 3/5.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 5. Survey on ChatGPT Language Model for Programming Learning and Teaching Support
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The average accuracy of the code generated by ChatGPT is 85.42%.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 7. Limitations
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found that about 86.7% of the students answered “Yes”.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 5. Survey on ChatGPT Language Model for Programming Learning and Teaching Support
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passes
- It may state two separate claims as one. 18
- Probably not a substantial claim. 31
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The suggestions proved to be interesting to the programmers and helpful in resolving the code errors.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 4.3. Error Checking and Debugging in Code
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: The authors' assessment that ChatGPT's debugging suggestions were helpful reflects actual programmer benefit.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Of the participating students, 61.3% (35.5% first year, 16.1% second year, 6.5% third year, and 3.2% fourth year) were undergraduate students, 32.2% (16.1% first year and 16.1% second year) were Master's students, and 6.5% were second-year doctoral students.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 5. Survey on ChatGPT Language Model for Programming Learning and Teaching Support
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ChatGPT can identify errors in code and provide potential suggestions and code snippets.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 4.3. Error Checking and Debugging in Code
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, when we asked ChatGPT to “provide a pseudocode for a selection sort and an explanation”, it generated the pseudocode and explanation, as depicted in Figure 7.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 4.1. Conceptual Understanding
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The generated optimized code was validated on the AOJ platform, and it passed all test cases (10/10) and was accepted for the Bubble Sort problem”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 4.4. Solution Code Optimization
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ChatGPT can also be used to generate topic-specific quiz questions tailored to the subject matter and difficulty level.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Lesson Planning:
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ChatGPT can generate nearly accurate answers to technical questions from a wide range of topics and correct or partially correct programming code based on problem descriptions, algorithm and problem names, etc.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Ethical implications and potential biases:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the experimental results show that the accuracy of the generated code is 95.83% on the basic compiler and 75% on AOJ.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 7. Limitations
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“ChatGPT can be used to create lesson plans for specific courses, such as math, chemistry, physics, computer science, civil engineering, language, and literature.ChatGPT provides topic-specific illustrations, activities, and exercises to help educators better teach their students.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Lesson Planning:
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our surveys and experimental results show that ChatGPT is useful not only for programming education but also for education and research.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 8. Conclusions
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“During the experiments, we obtained significant results in code generation, error checking and debugging, and optimization of the solution code.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 7. Limitations
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is a rare case (or an example of erroneous code), but ChatGPT has optimized and fixed it.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 4.4. Solution Code Optimization
Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This can lead to social unrest, reputation damage, and even physical harm.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Ethical implications and potential biases:
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ChatGPT is a revolutionary LLM that can maintain humanlike conversations and generate human-like text for any natural language query that is nearly indistinguishable.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 8. Conclusions
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Simply acquiring answers and code from ChatGPT can be a barrier to improving learners' critical thinking and problem-solving skills.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Ethical implications and potential biases:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, the quality and suitability of these applications for programming learning remain unclear.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 4. Programming Learning with ChatGPT
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the solution codes generated by AI models can be used for academic coding exams and competitions”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Ethical implications and potential biases:
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Educators can use ChatGPT to provide personalized learning support for their students.Depending on a student's needs and learning style, ChatGPT can suggest customized resources and learning activities.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Lesson Planning:
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is because the ease of obtaining answers, problem-solving strategies, and scientific text generation can limit critical thinking and problem-solving skills.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Blind reliance on generative AI tools:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The model can be used to answer questions, write essays, solve problems, explain complex topics, provide virtual tutoring, practice languages, learn programming, teach, and support research.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 8. Conclusions
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the ethical implications (e.g., bias and discrimination, privacy and security, misuse of technology, accountability, transparency, and social impact) of ChatGPT are complex and multifaceted”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 8. Conclusions
Strength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“it can be difficult to hold anyone accountable for the responses generated”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Ethical implications and potential biases:
Strength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
- Rests on an assumption: ChatGPT generates responses without human intervention.
In the paper
“Cotton et al. [57] presented several strategies for recognizing texts from LLMs such as ChatGPT, including language inconsistencies, a lack of proper citations, factual errors, ambiguity, and poor context awareness.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > -Sam Altman, CEO of OpenAI
What the sources say
- Not checked yetcites [57] Cotton, D.R., Cotton, P.A., and Shipway, J.R. (2023). Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT. Innov. Educ. Teach. Int., 1–12. source
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ChatGPT and other generative models involve the collection and processing of personal data, which raises concerns about privacy and data security.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Ethical implications and potential biases:
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If the answer is identical to answers generated by ChatGPT, it may be a sign that the answer was not generated by a human.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Compare responses:
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“The answers and texts generated by ChatGPT are becoming indistinguishable from human-generated answers and texts.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > -Sam Altman, CEO of OpenAI
What the sources say
- Not checked yetcites [57] Cotton, D.R., Cotton, P.A., and Shipway, J.R. (2023). Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT. Innov. Educ. Teach. Int., 1–12. source
- Not checked yetcites [60] Elkins, 2020, Can GPT-3 pass a Writer’s turing test?, J. Cult. Anal.
- Not checked yetcites [61] Gao, C.A., Howard, F.M., Markov, N.S., Dyer, E.C., Ramesh, S., Luo, Y., and Pearson, A.T. (2022). Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers. bioRxiv. source
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In short, ChatGPT threatens the fairness and validity of online exams and assignments.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 6. Threats and Strategies
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
- Rests on an assumption: ChatGPT can generate human-like text for academic topics.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, the ChatGPT model can be used to solve technical (e.g., engineering and computer programming) and non-technical (e.g., language and literature) problems.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 8. Conclusions
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“First, it can effectively support the writing process of research.At its most basic, it can improve writing by finding and correcting typographical errors, improving grammatical inconsistencies, providing advanced vocabulary, and recommending improvement strategies.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 3.3. Opportunities for Researchers
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ChatGPT and other AI language models can be used to generate fake news, hate speech, and other harmful content.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Ethical implications and potential biases:
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This can help detect cases of fraud.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Compare responses:
Strength:plain· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ChatGPT responses typically have certain characteristics, such as a lack of personalization or a rather generic tone.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Ethical implications and potential biases:
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Educators can receive the help of ChatGPT to answer learners' questions.Furthermore, if educators can ask ChatGPT for explanations and examples on a particular topic, it will surely increase the effectiveness of teaching.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Lesson Planning:
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If the answers appear disjointed or nonsensical, this may indicate that they were generated by ChatGPT or another AI model.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Ethical implications and potential biases:
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“ChatGPT responses may not have a consistent or logical flow, especially when it is generating answers to complex questions.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Ethical implications and potential biases:
Strength:hedged· not yet calibratedCausal:conditional· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The model can also summarize published work on a particular topic, which helps researchers understand the work.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 3.3. Opportunities for Researchers
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“To the best of our knowledge, there are no such tools that can recognize code generated by AI models”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Ethical implications and potential biases:
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“it still has certain limitations, such as a lack of common sense, potential bias, difficulty with complex reasoning, and inability to process visual information.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 8. Conclusions
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The model can be used to check submitted assignments for plagiarism.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > Rapid Assessment and Evaluation:
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“It is worth noting that our experiments were conducted on ChatGPT, which is currently in an active development phase.”
§ ChatGPT for Education and Research: Opportunities, Threats, and Strategies > 7. Limitations
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper relies on this without support.
The eight randomly selected AOJ problems and three code generations per problem are representative of ChatGPT's general code-generation performance for programming learning.
“We leveraged the problem descriptions of eight random problems and generated codes three times for each problem on the basis of the same description.”
The paper relies on this without support.
The survey respondents are representative of students and teachers who use ChatGPT for programming learning and teaching.
“We conducted a comprehensive survey on how ChatGPT supports programming learning and teaching among undergraduate, Master's, and doctoral students and teachers.”
The paper relies on this without support.
Students' self-reported helpfulness of ChatGPT suggestions reflects actual improvement in solving programming problems.
“By asking the question “Do ChatGPT's suggestions help you in solving programming problems?”,we tried to assess how ChatGPT suggestions help students.We found that about 86.7% of the students answered “Yes”.”
The paper relies on this without support.
The authors' judgment that ChatGPT-generated explanations and pseudocode are easy to understand and useful reflects learners' actual understanding.
“The results demonstrate that ChatGPT can generate easy-to-understand explanations and pseudocode that are useful for learners to understand algorithmic concepts.”
The paper relies on this without support.
The authors' assessment that ChatGPT's debugging suggestions were helpful reflects actual programmer benefit.
“The suggestions proved to be interesting to the programmers and helpful in resolving the code errors.”
The paper relies on this without support.
ChatGPT can generate human-like text for academic topics.
“As ChatGPT can generate human-like text for academic topics”
The paper relies on this without support.
LLMs rely heavily on training data, and when the data contain biases or anomalies, it could lead to unfair results.
“Typically, LLMs rely heavily on training data, and when the data contain biases or anomalies, it could lead to unfair results.”
The paper relies on this without support.
The internal mechanisms and processes of ChatGPT are not sufficiently open and transparent to users about how they work.
“Furthermore, the internal mechanisms and processes are not sufficiently open and transparent to users about how they work.”
The paper relies on this without support.
ChatGPT generates responses without human intervention.
“Because ChatGPT generates responses without human intervention”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“Particularly, students that are more experienced using ChatGPT tend to be less concerned about its impact on academic integrity and more optimistic about its role in self-study (Nisar and Aslam 2023;Rahman and Watanobe 2023).”
From Widely used but barely trusted: understanding student perceptions on the use of generative AI in higher education (2025) · cites this paper as Rahman and Watanobe 2023
Experimental — accuracy not yet measuredIn the citing paper
“foster students’ performance, engagement and motivation [14, 44, 58], and enhance teaching practices [17, 18, 64, 74].”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [58]
In the citing paper
“fostering students' performance, engagement, and motivation, leading to increase students' academic success [14, 44, 58]”
From A systematic literature review of empirical research on ChatGPT in education (2024) · cites this paper as [58]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Table 1 names a different lead author for reference 25
MinorTable 1 attributes reference 25 to Simon et al., while the reference list identifies Frieder as its lead author.
“Simon et al. [25]”
“25. Frieder, S.; Pinchetti, L.; Griffiths, R.R.; Salvatori, T.; Lukasiewicz, T.; Petersen, P.C.; Chevalier, A.; Berner, J. Mathematical capabilities of chatgpt.”
Citations
Cites
1 retracted
Retracted: Chat With ChatGPT on Intelligent Vehicles: An IEEE TIV Perspective
Du, H., Teng, S., Chen, H., Ma, J., Wang, X., Gou, C., Li, B., Ma, S., Miao, Q., Na, X., et al. (2023) · IEEE Transactions on Intelligent Vehicles · doi:10.1109/tiv.2023.3253281 · Cited by 164 · cited 1×
“Moreover, ChatGPT has recently been applied to tasks in different domains such as software engineering [36], medical education [37], data augmentation [38], fifth industrial revolution [39], code generation [40], intelligent vehicles [41], and solving AI tasks [42].”
MethodRetractedVaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · cited 1×
“Therefore, the Transformer architecture was introduced [4] to alleviate this problem.”
MethodLanguage Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 1×
“Generative pretrained transformer (GPT)-3 is a large language model (LLM) based on the Transformer architecture [5,6] that has achieved significant success in NLP tasks.”
BackgroundChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 3×
“Furthermore, several studies [9–12 Frieder et al. [25] tested the mathematical capabilities of ChatGPT on GHOSTS and handcrafted datasets.”
BackgroundMethodTraining language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · cited 2×
“Finally, the resulting model is called InstructGPT [8].”
BackgroundMethodKung, T. H., Cheatham, M., ChatGPT, Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., et al. (2022) · doi:10.1101/2022.12.19.22283643 · Cited by 4,115
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · cited 1×
“Kung et al. [26] evaluated the performance of ChatGPT on the US Medical Licensing Exam, which includes three exams: Step 1, Step 2CK, and Step 3.”
BackgroundChatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087 · cited 3×
“Students can be given clear instructions on assignments and online exams on how to structure their assignments and answer their questions online [57].”
BackgroundChatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079
Flagged first, then by citations
Research works printed but not linked: 3
- Rahman, M.M. (2022). Data Analysis and Code Assessment Using Machine Learning Techniques for Programming Activities. [Ph.D. Thesis, The University of Aizu].
- Avila-Chauvet, L., Mejía, D., and Acosta Quiroz, C.O. (2023). Chatgpt as a Support Tool for Online Behavioral Task Programming. SSRN, Available online: https://ssrn.com/abstract=4329020.
- Jiao, W., Wang, W., Huang, J.t., Wang, X., and Tu, Z. (2023). Is ChatGPT a good translator? A preliminary study. arXiv.
Cited by
Sorted by citations
Items 301–400 of 1,004
Mirea, C.-M., Bologa, R., Toma, A., Clim, A., Plăcintă, D.-D., & Bobocea, A. (2025) · Applied Sciences · doi:10.3390/app15105785 · Cited by 31
AI chatbots in higher education: Students’ beliefs and concerns
Fošner, A., & Aver, B. (2025) · Sustainable Futures · doi:10.1016/j.sftr.2025.100734 · Cited by 27
Tai, H., Lin, M., & Chen, Y. (2025) · International Journal of Applied Linguistics · doi:10.1111/ijal.12777 · Cited by 23
Li, H., Huang, Q.-R., Wen, L.-P., Chen, W., & Xu, Z.-Z. (2025) · SAGE Open · doi:10.1177/21582440251377986 · Cited by 17
“The rise of generative artificial intelligence has made personalized learning feasible (Baidoo-Anu & Owusu Ansah, 2023; Farrokhnia et al., 2024; Rahman & Watanobe, 2023) and is anticipated to act as an instructional support tool for educators (Grassini, 2023; Polverini & Gregorcic, 2024).”
Marbun, T. O. (2023) · Journal Didaskalia · doi:10.33856/didaskalia.v6i2.309 · Cited by 15
Enhancing Computer Programming Education using ChatGPT- A Mini Review
Deriba, F. G., Sanusi, I. T., & Sunday, A. O. (2023) · Koli Calling International Conference on Computing Education Research · doi:10.1145/3631802.3631848 · Cited by 15
Centralized Database Access: Transformer Framework and LLM/Chatbot Integration-Based Hybrid Model
Bratić, D., Šapina, M., Jurečić, D., & Gršić, J. Ž. (2024) · Applied System Innovation · doi:10.3390/asi7010017 · Cited by 15
The Evolution of Learning: Assessing the Transformative Impact of Generative AI on Higher Education
Krause, S., Panchal, B. H., & Ubhe, N. (2024) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-981-97-9255-9_24 · Cited by 35
Navigating cognitive dissonance: master’s students’ experiences with ChatGPT in dissertation writing
Ren, X., Zheng, W., & Zhang, M. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1542559 · Cited by 18
Ngu, P., Chien, C., Ho, Y., & Hou, H. (2025) · Computers & Education · doi:10.1016/j.compedu.2025.105421 · Cited by 14
AI chatbots for better or worse? Trust, enjoyment and ethics in AI adoption through a UTAUT lens
Al‐Mamary, Y. H., & Alshammari, M. M. (2025) · Journal of Science and Technology Policy Management · doi:10.1108/jstpm-06-2025-0252 · Cited by 19
Wuisang, M. C., Kurniawan, M., Santosa, K. A. W., Gunawan, A. A. S., & Saputra, K. E. (2023) · International Seminar on Application for Technology of Information and Communication (iSemantic) · doi:10.1109/isemantic59612.2023.10295323 · Cited by 14
“ChatGPT is an NLP model launched in November 2022 that is capable of generating text in a human-like conversational style [14],[15].”
Prajapati, J. B., Kumar, A., Singh, S. K., Prajapati, B. G., Thakar, Y., Tambe, P. R., & Ved, A. D. (2024) · SN Social Sciences · doi:10.1007/s43545-023-00818-0 · Cited by 20
“In addition, ChatGPT can be used to solve technical and non-technical problems (Rahman and Watanobe 2023).”
Exploring the Role of Artificial Intelligence in Digital Marketing Strategies
Tauheed, J., Shabbir, A., & Pervez, M. S. (2024) · Journal of Business Communication & Technology · doi:10.56632/bct.2024.3105 · Cited by 35
“The fact that 85% of marketers use AI for content authoring exemplifies this development (Rahman & Watanobe, 2023).”
BackgroundDoctoral Programme in Education Science Universitas Pendidikan Ganesha, S., & Turmuzi, M. (2024) · International Journal of Information and Education Technology · doi:10.18178/ijiet.2024.14.5.2097 · Cited by 18
“ChatGPT can assist teachers with a range of tasks, including creating study materials, offering advice to students, performing research, enhancing the caliber of written work, and boosting productivity and efficiency at work [6, 8, 48].”
BackgroundParticipant Use of Artificial Intelligence in Online Focus Groups: An Experiential Account
Stafford, L., Preston, C., & Pike, A. C. (2024) · International Journal of Qualitative Methods · doi:10.1177/16094069241286417 · Cited by 19
“Therefore, we compared participant responses to ChatGPT ’ s answers to the same questions (Rahman & Watanobe, 2023).”
He, S., & Lu, Y. (2024) · Interactive Learning Environments · doi:10.1080/10494820.2024.2415444 · Cited by 19
“Students in this category rely heavily on the information feedback function of ChatGPT, leading to a situation where “ machine thinking ” replaces “ student thinking ” (Hamdani et al., 2022; Rahman & Watanobe, 2023).”
A High-Quality Generation Approach for Educational Programming Projects Using LLM
Song, T., Zhang, H., & Xiao, Y. (2024) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2024.3499751 · Cited by 20
“5 and GPT-4, such as conceptual understanding, essay evaluation, solution code generation, and error checking and debugging [8], [9].”
Teachers’ perceptions of the risks and benefits of AI in higher education: A case study of ERNIE Bot
Wang, N., & Li, M. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2024.2432429 · Cited by 21
“Generative AI models often provide false or outdated information (Sallam, 2023) and can generate fabricated data, raising authenticity concerns (Rahman & Watanobe, 2023).”
Unleashing ChatGPT: Redefining Technology Acceptance and Digital Transformation in Higher Education
Mourtajji, L., & Chiss, N. (2024) · Administrative Sciences · doi:10.3390/admsci14120325 · Cited by 17
Rane, N., Choudhary, S., & Rane, J. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4681718 · Cited by 14
Perkins, M., & Roe, J. (2023) · doi:10.35542/osf.io/z4cru · Cited by 117
Generative AI in Accounting Education: Evaluating ChatGPT's Role in Assessment and Skill Development
Tharapos, M., Lau, K. H., Peszynski, K., Nguyen, L. A., Magdziarz, S., Morton, E., Borg, V., & Duan, S. X. (2025) · Accounting and Finance · doi:10.1111/acfi.70051 · Cited by 44
Zhang, Y., Hsieh, C., & Liu, F. (2025) · Public Performance & Management Review · doi:10.1080/15309576.2025.2537107 · Cited by 14
“Additionally, the provision of personalized and adaptive assistance by ChatGPT may alleviate the workload and time pressure they experience (Rahman & Watanobe, 2023; Rakap, 2024).”
Aruleba, K., Sanusi, I. T., Obaido, G., Ogbuokiri, B., & Mienye, I. D. (2025) · Technology Knowledge and Learning · doi:10.1007/s10758-025-09899-7 · Cited by 13
Decision-making criteria for AI tools in digital education
Madanchian, M., & Taherdoost, H. (2025) · Digital engineering. · doi:10.1016/j.dte.2025.100069 · Cited by 27
Opportunities and limitations of the ChatGPT Advanced Data Analysis plugin for hydrological analyses
Irvine, D. J., Halloran, L. J. S., & Brunner, P. (2023) · Hydrological Processes · doi:10.1002/hyp.15015 · Cited by 13
“…of computer scripts and data analyses, with or without the Advanced Data Analysis plugin, has been explored in a university context (Malinka et al., 2023; Rahman & Watanobe, 2023), and specifically within the disciplines of climate science (Biswas, 2023) and bioinformatics (Wang et al., 2023).”
BackgroundMethodInformation Seeking Behavior in ChatGPT: The Case of Programming Students from a Developing Economy
Hernandez, A. A., Padilla, J. R., & Montefalcon, M. D. (2023) · IEEE International Conference on System Engineering and Technology (ICSET) · doi:10.1109/icset59111.2023.10295122 · Cited by 15
“Particularly in the educational context, ChatGPT has been explored to assist students with writing essays, problem-solving, and programming tasks [6].”
BackgroundStudi Kasus Penggunaan ChatGPT pada Mahasiswa di PTKI Lampung
Sugiarto, S., & Suhono, S. (2023) · Jurnal Al-Qiyam · doi:10.33648/alqiyam.v4i2.318 · Cited by 23
“ChatGPT mampu menghasilkan model bahasa yang sangat akurat yang dapat digunakan untuk memprediksi kata berikut dalam frasa berdasarkan kata sebelumnya dengan mempelajari kumpulan data besar percakapan manusia(Rahman & Watanobe, 2023).”
BackgroundYuan, S., Li, F., Browning, M. H. E. M., Bardhan, M., Zhang, K., McAnirlin, O., Patwary, M. M., & Reuben, A. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2024.1295275 · Cited by 13
“…could help explain code, generate queries, identify relevant coding libraries and functions, explain errors, assisting debug, or write prototype coding for future studies in almost every coding language, such as R, Python, Matlab, etc. (Biswas, 2023; Feng et al., 2023; Rahman and Watanobe, 2023).”
BackgroundPoleac, D. (2024) · Proceedings of the ... International Conference on Business Excellence · doi:10.2478/picbe-2024-0240 · Cited by 13
“Chatbots such as ChatGPT have evolved significantly with the growth of AI and are now used in various industries, such as customer service, health, finance, education (Kooli, 2023), content generation, translation, chatbot responses, question answering, programming code (Rahman & Watanobe, 2023).”
InfluentialPersonalisation is not Guaranteed: The Challenges of Using Generative AI for Personalised Learning
Laak, K.-J., Abdelghani, R., & Aru, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-65881-5_5 · Cited by 14
GPT vs. Google: A comparative study of self-code learning in ODL students
Fırat, M., & Kuleli, S. (2024) · Journal of Educational Technology and Online Learning · doi:10.31681/jetol.1508675 · Cited by 13
“Although Generative Artificial Intelligence (GAI) is still developing, it can already compete with traditional methods in specific areas, such as code interpretation (Popovici, 2023; Yilmaz & Yilmaz, 2023; Rahman & Watanobe, 2023; Shoufan, 2023).”
Exploring the integration of AI in library services: Perspectives and considerations
Patra, N. K., Gaitanou, P., Oladokun, B. D., & Sahu, S. C. (2025) · Business Information Review · doi:10.1177/02663821251343742 · Cited by 13
Yapay Zeka Sohbet Robotları ve ChatGPT’nin Hemşirelik Eğitiminde Kullanılması
İlaslan, E. (2023) · Akdeniz Hemşirelik Dergisi · doi:10.59398/ahd.1330341 · Cited by 14
Zhang, L., Jiang, Q., Xiong, W., & Zhao, W. (2025) · Journal of Educational Computing Research · doi:10.1177/07356331251333874 · Cited by 15
“However, the accuracy of ChatGPT in programming tasks is affected by factors such as task complexity, programming languages, problem descriptions, questioning approaches, and program size (Bucaioni et al., 2024; Rahman & Watanobe, 2023; Sun, Boudouaia, Zhu et al., 2024).”
Usability of large language models for building construction safety risk assessment
Oral, M., Alboga, Ö., Aydınlı, S., & Erdiş, E. (2025) · Engineering Construction & Architectural Management · doi:10.1108/ecam-08-2024-1143 · Cited by 15
Enhancing Programming Learning with LLMs: Prompt Engineering and Flipped Interaction
Cowan, B., Watanobe, Y., & Shirafuji, A. (2023) · Asia Service Sciences and Software Engineering Conference (ASSE) · doi:10.1145/3634814.3634816 · Cited by 17
Self-citationAcceptance and Use of ChatGPT Among Accounting and Finance Higher Education Students
Ziemba, E. W., Maruszewska, E. W., Grabara, D., & Renik, K. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-66761-9_16 · Cited by 17
Unraveling the Impact of ChatGPT as a Knowledge Anchor in Business Education
George, A., Storey, V. C., & Hong, S. (2024) · ACM Transactions on Management Information Systems · doi:10.1145/3705734 · Cited by 17
Vera, F. S. (2024) · Education Sciences · doi:10.3390/educsci15010026 · Cited by 22
Generative AI as a Coach to Help Students Enhance Proficiency in Question Formulation
Kim, P., Wang, W., & Bonk, C. J. (2025) · Journal of Educational Computing Research · doi:10.1177/07356331251314222 · Cited by 14
“Generative AI presents educators with many new opportunities as well as challenges in transforming teaching and learning as we know it (Adeshola & Adepoju, 2023; Baidoo-Anu & Ansah, 2023; Chan & Hu, 2023; Lo, 2023; Meyer et al., 2023; Rahman & Watanobe, 2023; Yu & Guo, 2023).”
May, T. A., Fan, Y., Stone, G. E., Koskey, K. L. K., Sondergeld, C. J., Folger, T. D., Archer, J. N., Provinzano, K., & Johnson, C. C. (2025) · Education Sciences · doi:10.3390/educsci15020144 · Cited by 12
Kaya, D., & Yavuz, S. (2025) · Journal of Intelligence · doi:10.3390/jintelligence13040043 · Cited by 22
“Research suggests that ChatGPT can enhance student learning experiences by explaining abstract concepts, reducing cognitive load, and adapting to individual proficiency levels (Rahman and Watanobe 2023).”
InfluentialCarou‐Senra, P., Delgado-Taboada, I., Alvarez‐Lorenzo, C., Díaz‐Rodríguez, P., & Goyanes, Á. (2025) · American Journal of Pharmaceutical Education · doi:10.1016/j.ajpe.2025.101462 · Cited by 12
Yakin, A. M. A., Muthmainnah, M., Apriani, E., Obaid, A. J., & Elngar, A. A. (2023) · Idarah (Jurnal Pendidikan dan Kependidikan) · doi:10.47766/idarah.v7i2.1514 · Cited by 25
“Chatbot ChatGPT, an artificial intelligence chat tool, is ready to answer any questions, similar to Rahman and Watanobe's research (Rahman & Watanobe, 2023).”
MethodClark, Q. M. (2025) · Frontiers in Education · doi:10.3389/feduc.2024.1396104 · Cited by 13
Altamimi, I., Alhumimidi, A. S., Alshehri, S., AlRumayan, A. F., Al-khlaiwi, T., Meo, S. A., & Temsah, M.-H. A. (2024) · Annals of Medicine and Surgery · doi:10.1097/ms9.0000000000002120 · Cited by 11
“However, their integration into traditional educational and research frameworks also poses challenges, particularly concerning the potential erosion of critical thinking skills [7,8] .”
BackgroundAutomated Evaluation Techniques and AI-Enhanced Methods
Ahmed, S., Zaki, A., & Bentley, Y. N. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-2145-4.ch001 · Cited by 12
Towards Integrating Emerging AI Applications in SE Education
Vierhauser, M., Groher, I., Antensteiner, T., & Sauerwein, C. (2024) · IEEE International Conference on Software Engineering Education and Training (CSEE&T) · doi:10.1109/cseet62301.2024.10663045 · Cited by 11
“Solutions to programming exercises are automatically generated by students using, for example, ChatGPT and pose further challenges for educators when conducting online exams [3].”
BackgroundJeong, S., Kim, S., & Lee, S. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-75599-6_7 · Cited by 11
Hussain, F., & Anwar, M. A. (2024) · Journal of Asian Public Policy · doi:10.1080/17516234.2024.2422917 · Cited by 12
“Despite these challenges, studies have emphasized the potential benefits of ChatGPT in higher education, including personalized feedback, improved accessibility, interactive learning, assistance with lesson planning and assessments, and innovative teaching techniques (M. M. Rahman & Watanobe, 2023).”
InfluentialAndersen-Kiel, N., & Linos, P. K. (2024) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie61694.2024.10892934 · Cited by 12
“Md. Mostafizer Rahman and Yutaka Watanobe delve into ChatGPT's multifaceted role in education and research, emphasizing its capabilities in facilitating research and improving educational practices [21].”
Akpan, I., Razavi, R., & Akpan, A. A. (2025) · Big Data · doi:10.1089/big.2024.0128 · Cited by 11
AlOmar, E. A. (2025) · ACM Transactions on Computing Education · doi:10.1145/3722229 · Cited by 18
“This technology has sparked a debate about its potential benefits and drawbacks for student learning [38].”
A Bibliometric Exposition and Review on Leveraging LLMs for Programming Education
Amos, J. P., Amodu, O. A., Mahmood, R. A. R., AbdulQudus, A. B., Zakaria, A. F., Iyanda, A. R., Bukar, U. A., & Hanapi, Z. M. (2025) · IEEE Access · doi:10.1109/access.2025.3554627 · Cited by 17
“The authors [24] investigate the use of ChatGPT in helping students improve their programming skills.”
ChatGPT: A reliable assistant for the evaluation of students’ written texts?
Atasoy, A., & Arani, S. M. N. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13553-1 · Cited by 19
“AI has the potential to tailor education to each learner, as described by researchers like Ngo (2023) and Rahman and Watanobe (2023), but this study makes it clear that AI on its own might not be ready to fully take on the detailed, personal parts of language that humans can rate so well.”
ChatGPT and Cheat Detection in CS1 Using a Program Autograding System
Pang, A., & Vahid, F. (2024) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3649217.3653558 · Cited by 15
“These tools have raised concerns among college educators who worry students will let the tool do their homework for them, such as writing essays, or in computer science courses, writing programs [2, 3, 4].”
BackgroundChatGPT-4o for English language teaching and learning: Features, applications, and future prospectsPreprint
Pang, S., Nol, E., & Heng, K. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4837988 · Cited by 11
Jazim, F., Al‐Mamary, Y. H., & Abubakar, A. A. (2025) · Acta Psychologica · doi:10.1016/j.actpsy.2025.105445 · Cited by 16
Luo, Z., & Tahir, R. (2025) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-025-00513-6 · Cited by 10
“For instance, Rahman and Watanobe (2023) emphasized ChatGPT’s versatility in generating lesson plans across a range of disciplines such as mathematics, science, and engineering.”
Liang, Z., Yang, K., Sha, L., Gašević, D., Yan, L., & Chen, G. (2026) · British Journal of Educational Technology · doi:10.1111/bjet.70055 · Cited by 22
Yapay Zeka Uygulamalarının Kimya Eğitiminde Kullanımı Alternatif Araçlar
Buluş, B., & Elmas, R. (2024) · Turkiye Kimya Dernegi Dergisi Kisim C Kimya Egitimi · doi:10.37995/jotcsc.1366999 · Cited by 14
Tapia-Mandiola, S., & Araya, R. (2024) · AI · doi:10.3390/ai5040093 · Cited by 11
Tamrin, S. I., Omar, N. F., Kamaruzaman, K. N., Zaghlol, A. K., & Aziz, M. R. A. (2024) · Information Management and Business Review · doi:10.22610/imbr.v16i3s(i)a.4191 · Cited by 17
“ChatGPT allows students to obtain quick and direct answers by simply asking questions, which may enhance their academic performance in the short term (Rahman & Watanobe, 2023).”
BackgroundAI chatbots and political learning
Taylor, J. B., & Richey, S. (2024) · Journal of Information Technology & Politics · doi:10.1080/19331681.2024.2422929 · Cited by 10
“Almost simultaneously with ChatGPT’s release, scholars assessed this new technology’s potential to affect human education and learning (see Lo, 2023; Rahman & Watanobe, 2023).”
Prompting for pedagogy? Australian F-10 teachers’ generative AI prompting use cases
Crosthwaite, P. R., Smala, S., & Spinelli, F. (2024) · The Australian Educational Researcher · doi:10.1007/s13384-024-00787-0 · Cited by 20
“GAI applications have demonstrated the capacity to produce detailed and high-quality multimodal educational materials (e.g., Rahman & Watanobe, 2023), with educators able to utilise ChatGPT to generate exemplary written texts and oral dialogues across a variety of genres (Crosthwaite & Baisa, 2023)…”
Towards Generative AI for Course Content Production: Expert Reflections
Ullmann, T., Edwards, C. B., Bektik, D., Herodotou, C., & Whitelock, D. M. (2024) · European Journal of Open Distance and E-Learning · doi:10.2478/eurodl-2024-0013 · Cited by 17
“…course content; generating engaging video scripts based on existing written content, including suggesting on-screen graphics; and generating scenarios or case studies based on existing content (Herft, 2023; Liu et al., 2023; McInnes, 2023; Michel-Villarreal et al., 2023; Rahman & Watanobe, 2023).”
Wang, Q., & Li, H. (2025) · Big Data and Cognitive Computing · doi:10.3390/bdcc9030050 · Cited by 13
“In the context of education, the threats are particularly severe as students can use LLMs like ChatGPT to complete coursework, online exams, and even graduation theses, which could undermine academic integrity and diminish students’ critical thinking and problem-solving skills [9,10].”
Zhu, M. (2025) · Journal of Computer Assisted Learning · doi:10.1111/jcal.70013 · Cited by 12
Exploring the difference and quality of AI-generated versus human-written texts
Tengler, K., & Brandhofer, G. (2025) · Discover Education · doi:10.1007/s44217-025-00529-z · Cited by 21
“AI-supported applications are used to answer questions, solve problems, explain complex topics, provide virtual tutoring, practice languages and write essays [3].”
InfluentialFacilitating student engagement through ChatGPT
Meşe, E., Özge, K. U. R. U., & Atay, D. (2025) · The Journal of Educational Research · doi:10.1080/00220671.2025.2510394 · Cited by 10
“ChatGPT also poses threats such as a reduced use of cognitive skills such as critical thinking (Rahman & Watanobe, 2023).”
Jing, B., Yang, H., Pi, Z., Zhou, Y., & Ma, H. (2025) · Interactive Learning Environments · doi:10.1080/10494820.2025.2514090 · Cited by 12
“AIGC tools can serve as multidisciplinary teaching assistants in areas such as mathematics (Urhan et al., 2024), programming tutoring (Rahman & Watanobe, 2023), and second-language learning (Yeh, 2024).”
Martha, A. S. D., Widowati, S., & Rahayu, D. P. (2025) · Journal of Information Technology Education Research · doi:10.28945/5599 · Cited by 24
“A Friend or A Foe” Determining Factors Contributed to the Use of ChatGPT among University Students
Yifan, W., Mengmeng, Y., & Omar, M. K. (2023) · International Journal of Academic Research in Progressive Education and Development · doi:10.6007/ijarped/v12-i2/17400 · Cited by 13
Application of Chatgpt in Higher Education and Research – a Futuristic AnalysisPreprint
Aithal, P. S., & Aithal, S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4674364 · Cited by 10
Barakat, M. M., Salim, N. A., & Sallam, M. (2024) · Research Square · doi:10.21203/rs.3.rs-3919524/v1 · Cited by 10
Do large language models have a legal duty to tell the truth?Preprint
Wachter, S., Mittelstadt, B., & Russell, C. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4771884 · Cited by 10
Seung, Y., & Basham, J. D. (2026) · Learning Disability Quarterly · doi:10.1177/07319487261439132 · Cited by 12
A FOUR-STAGE MODEL OF CHATGPT ADOPTION: APPLICATION IN TOURISM, HEALTHCARE, AND EDUCATION INDUSTRY
(2023) · Issues in Information Systems · doi:10.48009/3_iis_2023_121 · Cited by 11
“Adaptability: ChatGPT has many limitations, such as a lack of common sense, potential bias, difficulty with complex reasoning, and inability to process visual information (Rahman and Watanobe, 2023).”
Digital Learning Management using OpenAI ChatGPT: A Systematic Literature Review
Faruq, M. S. S. A., Sunoko, A., Ibda, H., & Wahyudi, K. (2023) · International Journal of Learning Teaching and Educational Research · doi:10.26803/ijlter.22.12.2 · Cited by 16
Boundaries and Future Trends of ChatGPT Based on AI and Security Perspectives
Alsumayt, A., Alfawaer, Z. M., El-Haggar, N., Alshammari, M., Alghamedy, F. H., Aljameel, S. S., Alabbad, D. A., & Aldossary, M. I. (2024) · HighTech and Innovation Journal · doi:10.28991/hij-2024-05-01-010 · Cited by 10
“ Unauthorized code execution: Exploiting the natural language prompts, attackers can execute malicious code, actions, or commands on the system by leveraging the capabilities of LLMs [30].”
BackgroundUsing Artificial Intelligence for Essay Writing
Jen, S. L., & Salam, A. R. (2024) · Arab World English Journal · doi:10.24093/awej/chatgpt.5 · Cited by 11
“This is further supported by Chiu et al. (2023), AbdAlgane and Jabir Othman (2023), Aydın and Karaarslan (2022), Farrelly and Baker (2023), Jarrah et al. (2023) and Rahman and Watanobe (2023) who asserted that AI has significantly affected education and classroom practice recently.”
ResultKamali, J., Paknejad, A., & Poorghorban, A. (2024) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2024.7.2.8 · Cited by 12
Ohm, M., Bungartz, C., Boes, F., & Meier, M. (2024) · International Conference on Availability, Reliability and Security (ARES) · doi:10.1145/3664476.3670446 · Cited by 9
“RahmanandWatanobe[14]identifyvariousopportunitiesand threats associated with using ChatGPT in higher education.”
AI-Assisted Academia: Navigating the Nuances of Peer Review With ChatGPT 4
Biswas, S. S. (2024) · The Journal of Pediatric Pharmacology and Therapeutics · doi:10.5863/1551-6776-29.4.441 · Cited by 9
Vindigni, G. (2024) · European Journal of Contemporary Education and E-Learning · doi:10.59324/ejceel.2024.2(4).10 · Cited by 19
“LLMs optimize the programming process with advanced support functions for writing and debugging and expand the capacities for advanced data analyses through sentiment analysis and entity recognition (Topsal & Akinci, 2023; Rahman & Watanobe, 2023).”
MethodDigital Tools in Language Learning: Optimizing Memory and Attention for College Students
Gu, J. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2400384 · Cited by 17
“Going forward, educators will play a key role by providing clear recommendations and guidance on the appropriate and ethical use of AI tools in assessment (Rahman & Watanobe, 2023).”
Examining the Accuracy of AI Detection Software Tools in Education
Halaweh, M., & Refae, G. A. E. (2024) · International Conference on Intelligent Data Science Technologies and Applications (IDSTA) · doi:10.1109/idsta62194.2024.10747004 · Cited by 13
“It has the ability to consistently maintain a coherent dialog style, effectively engage users, and furnish authentic and pertinent responses, thereby avoiding irrelevant answers to individual queries [2].”
BackgroundUnlocking the potential of ChatGPT in academic libraries
Hussain, A. (2024) · IP Indian Journal of Library Science and Information Technology · doi:10.18231/j.ijlsit.2024.015 · Cited by 15
A First Look at Financial Data Analysis Using ChatGPT-4o
Chou, W.-H., Feng, Z., Li, B., & Feng, L. (2025) · Journal of risk and financial management · doi:10.3390/jrfm18020099 · Cited by 20
Integration of the SECI model and ChatGPT in higher education
Barreto, U., & Abarca, Y. (2025) · Heliyon · doi:10.1016/j.heliyon.2025.e42814 · Cited by 9
“The studies by Rahman and Watanobe [17] and Romero-Rodríguez et al. [9] support these findings, emphasizing that the innovative features of ChatGPT, such as personalized feedback and an accessible interface, motivate students to engage more actively in educational settings.”
PyChatAI: Enhancing Python Programming Skills—An Empirical Study of a Smart Learning System
Alanazi, M., Soh, B., Samra, H., & Li, A. (2025) · Computers · doi:10.3390/computers14050158 · Cited by 15
Qualitative evaluation of a ChatGPT-integrated transcultural nursing course for a diverse group
Gümüş, E. Ç., & Dolu, İ. (2025) · Nurse Education in Practice · doi:10.1016/j.nepr.2025.104491 · Cited by 11
Kashiha, H. (2025) · Journal of Second Language Writing · doi:10.1016/j.jslw.2025.101254 · Cited by 13
Stoyanova, D., Stoyanova-Petrova, S. V., Shotarova, S., Lyubomirov, S. Y., & Mileva, N. S. (2025) · Education Sciences · doi:10.3390/educsci16010019 · Cited by 13
The Rise of ChatGPT and the Demise of Bloom's Taxonomy of Learning Stages
Damaševičius, R. (2023) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-0205-7.ch006 · Cited by 9
The Usage of Artificial Intelligence in Education in Light of the Spread of ChatGPT
Hassan, A. (2023) · Contributions to management science · doi:10.1007/978-981-99-6101-6_50 · Cited by 10
The Role of ChatGPT in Higher Education: Some Reflections from Public Administration Students
Aristovnik, A., Umek, L., Brezovar, N., Keržič, D., & Ravšelj, D. (2023) · Communications in computer and information science · doi:10.1007/978-981-99-8255-4_22 · Cited by 13
Exploring ChatGPT's Potential and Concerns in Higher Education
Tick, A. (2024) · 2024 IEEE 22nd Jubilee International Symposium on Intelligent Systems and Informatics (SISY) · doi:10.1109/sisy62279.2024.10737595 · Cited by 8
Metadata
- Authors
- Rahman, M. M., & Watanobe, Y.
- Affiliations
- University of Aizu (JP)
- Corresponding author
- Rahman, Md. Mostafizer
- Venue
- Applied Sciences
- Publisher
- MDPI AG
- Published
- 2023-05-08
- Volume
- 13
- Issue
- 9
- Pages
- 5783
- ISSN
- 2076-3417 (ISSN-L)
- OpenAlex
- W4381982883
- Semantic Scholar
- 3b69ec0fc17c5134c9b002bebe283e0189bfdae5
- Cited by
- 1,080
- References
- 58
- Citations by year
- 2026: 193 · 2025: 338 · 2024: 425 · 2023: 124
- FWCI
- 253.56
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Applied Sciences (publishedVersion, cc-by)
- Preprints.org (acceptedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Japan Society for the Promotion of Science (JSPS) KAKENHI (23H03508)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Computer Science Applications › Teaching and Learning Programming
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- ChatGPT, large language models, artificial intelligence in education, academic dishonesty, critical thinking, programming education, code generation, code correction, student and teacher perceptions
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review