What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education
Open accessThe DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index.
Abstract
Artificial Intelligence (AI) technologies have been progressing constantly and being more visible in different aspects of our lives. One recent phenomenon is ChatGPT, a chatbot with a conversational artificial intelligence interface that was developed by OpenAI. As one of the most advanced artificial intelligence applications, ChatGPT has drawn much public attention across the globe. In this regard, this study examines ChatGPT in education, among early adopters, through a qualitative instrumental case study. Conducted in three stages, the first stage of the study reveals that the public discourse in social media is generally positive and there is enthusiasm regarding its use in educational settings. However, there are also voices who are approaching cautiously using ChatGPT in educational settings. The second stage of the study examines the case of ChatGPT through lenses of educational transformation, response quality, usefulness, personality and emotion, and ethics. In the third and final stage of the study, the investigation of user experiences through ten educational scenarios revealed various issues, including cheating, honesty and truthfulness of ChatGPT, privacy misleading, and manipulation. The findings of this study provide several research directions that should be considered to ensure a safe and responsible adoption of chatbots, specifically ChatGPT, in education.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Similarly, Tlili et al. [31] advocate for cautious integration of GenAI in education to mitigate ethical and practical challenges, underlining the importance of establishing clear usage guidelines.”
From Generative AI and Academic Integrity in Higher Education: A Systematic Review and Research Agenda (2025) · cites this paper as [31]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This is in accordance with the generation of teaching materials such as rubrics or activities to implement in the classroom and even questionnaires of a certain quality, as recognized by other authors [17,27].”
From Is the Education System Prepared for the Irruption of Artificial Intelligence? A Study on the Perceptions of Students of Primary Education Degree from a Dual Perspective: Current Pupils and Future Teachers (2023) · cites this paper as [27]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These AI technologies can simulate realistic conversations, provide instant feedback, and adapt to individual learning needs, thereby offering accessible, flexible, and personalised learning opportunities [2, 3].”
From Enhancing professional communication training in higher education through artificial intelligence(AI)-integrated exercises: study protocol for a randomised controlled trial (2025) · cites this paper as [2]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“these scholars are currently navigating the complexities of under-regulated AI in the learning and teaching landscape (Tlili et al., 2023).”
From A systematic literature review of attitudes, intentions and behaviours of teaching academics pertaining to AI and generative AI (GenAI) in higher education: An analysis of GenAI adoption using the UTAUT framework (2024) · cites this paper as Tlili et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Such attacks could facilitate cheating, promote biased views, or expose students to offensive language [92].”
From Promises and challenges of generative artificial intelligence for human learning (2024) · cites this paper as [92]
Experimental — accuracy not yet measured
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(1984) · Cited by 81,983
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · cited 2×
“…which is the third iteration of the GPT series by OpenAI that is more advanced in terms of scale (175 billion parameters, compared to 1.5 billion of GPT-2), larger dataset as the training data, more fine-tuning, enhanced capabilities, and more human-like text generations (Brown et al., 2020).”
BackgroundInfluentialArrieta, A. B., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., et al. (2019) · arXiv · doi:10.48550/arxiv.1910.10045 · Cited by 11,321
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., Garcia, S., Gil-Lopez, S., Molina, D., Benjamins, R., et al. (2020) · Information Fusion · doi:10.1016/j.inffus.2019.12.012 · Cited by 10,423 · cited 1×
“…implementation and use of ethical, transparent, and accountable AI technology in order to reduce biases, promote fairness and equality, and help facilitate the interpretability and explainability of outcomes, which are particularly pertinent in an educational context (Barredo Arrieta et al., 2020).”
BackgroundI.—COMPUTING MACHINERY AND INTELLIGENCE
TURING, A. M. (1950) · Mind · doi:10.1093/mind/lix.236.433 · Cited by 10,290
ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education
Kasneci, E., Seßler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · doi:10.35542/osf.io/5er8f · Cited by 7,552
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 1×
“Janssen et al. (2021) described reasons for chatbots’ failure in practice, including not having enough resources, wrong use case (i.e., the basic chatbot technology did not match the required task), poor law regulations, data security, and liability concerns, ignorance of user expectation and bad…”
BackgroundKung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115 · cited 1×
“For instance, Kung et al. (2023) found the accuracy of ChatGPT to be around 60%, demanding careful assessment of its output before use.”
BackgroundKung, T. H., Cheatham, M., ChatGPT, Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., et al. (2022) · doi:10.1101/2022.12.19.22283643 · Cited by 4,115
Cook, K. S. (2020) · The Blackwell Encyclopedia of Sociology · doi:10.1002/9781405165518.wbeoss149.pub2 · Cited by 3,637
Research works printed but not linked: 7
- Asimov, I. (1950). I, Robot. Gnome Press.
- Guadamuz, A. (2018). Can the monkey selfie case teach us anything about copyright law? WIPO Magazine, 1, 40–46.
- Haque, M. U., Dharmadasa, I., Sworna, Z. T., Rajapakse, R. N., & Ahmad, H. (2022). I think this is the most disruptive technology: Exploring sentiments of ChatGPT early adopters using Twitter data. arXiv preprint arXiv:2212.05856. · arXiv
- McCarthy, J., Minsky, M., Rochester, N., & Shannon, C. (1955). A proposal for Dartmouth summer research project on artificial intelligence. AI Magazine, 27, 12.
- Stake, R. E. (1995). The art of case study research: Perspective in practice. Sage.
- Susnjak, T. (2022). ChatGPT: The end of online exam integrity?. arXiv preprint arXiv:2212.09292. · arXiv
- van der Maaten, L., & Hinton, G. (2008). Visualizing data using t-SNE. Journal of Machine Learning Research, 9(2008), 2579–2605.
Cited by
Ray, P. P. (2023) · Internet of Things and Cyber-Physical Systems · doi:10.1016/j.iotcps.2023.04.003 · Cited by 2,463
What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature
Lo, C. K. (2023) · Education Sciences · doi:10.3390/educsci13040410 · Cited by 2,024
“For example, some researchers [8,52,62] suggested designing assessment tasks that focus on creativity and critical thinking.”
BackgroundInfluentialA SWOT analysis of ChatGPT: Implications for educational practice and research
Farrokhnia, M., Banihashem, S. K., Noroozi, O., & Wals, A. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2195846 · Cited by 1,384
“ChatGPT has raised various ethical issues such as encouraging plagiarism and cheating (Gašević et al., 2023) and being prone to errors such as the provision of fake information (Tlili et al., 2023).”
BackgroundGrassini, S. (2023) · Education Sciences · doi:10.3390/educsci13070692 · Cited by 1,048
“Inter-estingly, when the researchers posed these concerns directly to ChatGPT, it contradicted the information on the official page, stating that it does not retain any conversation data [87].”
BackgroundChallenges and Opportunities of Generative AI for Higher Education as Explained by ChatGPT
Michel-Villarreal, R., Vilalta-Perdomo, E., Salinas-Navarro, D. E., Thierry-Aguilera, R., & Gerardou, F. S. (2023) · Education Sciences · doi:10.3390/educsci13090856 · Cited by 877
“In line with previou studi s, th resul s revealed t t ChatGPT h s the potential to revolutionize education in different ways [3,12,14,34].”
Result
Metadata
- Authors
- Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B.
- Affiliations
- Beijing Normal University (CN)
- Anadolu University (TR)
- Indiana University Bloomington (US)
- University of Wollongong (AU)
- Corresponding author
- Adarkwah, Michael Agyemang
- ORCID
- Venue
- Smart Learning Environments
- Published
- 2023-02-22
- Volume
- 10
- Issue
- 1
- ISSN
- 2196-7091 (ISSN-L)
- OpenAlex
- W4321499901
- Cited by
- 2,003
- References
- 33
- Citations by year
- 2026: 608 · 2025: 449 · 2024: 614 · 2023: 331
- FWCI
- 470.02
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Smart Learning Environments (publishedVersion, cc-by)
- Research Online (University of Wollongong) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Research Online (University of Wollongong) (cc-by)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Social Sciences › Business, Management and Accounting › Marketing › AI in Service Interactions
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- ChatGPT, educational chatbots, conversational AI, early adopters, public discourse on AI, educational transformation, response quality, AI ethics, user experience, academic dishonesty, privacy, manipulation, responsible AI adoption
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. https://app.cochat.ai/papers/10.1186/s40561-023-00237-x
- The paper
- Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023). What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education. Smart Learning Environments. https://doi.org/10.1186/s40561-023-00237-x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.