Hello GPT! Goodbye home examination? An exploratory study of AI chatbots impact on university teachers’ assessment practices
Open accessAbstract
AI chatbots have recently fuelled debate regarding education practices in higher education institutions worldwide. Focusing on Generative AI and ChatGPT in particular, our study examines how AI chatbots impact university teachers’ assessment practices, exploring teachers’ perceptions about how ChatGPT performs in response to home examination prompts in undergraduate contexts. University teachers (n = 24) from four different departments in humanities and social sciences participated in Turing Test-inspired experiments, where they blindly assessed student and ChatGPT-written responses to home examination questions. Additionally, we conducted semi-structured interviews in focus groups with the same teachers examining their reflections about the quality of the texts they assessed. Regarding chatbot-generated texts, we found a passing rate range across the cohort (37.5 − 85.7%) and a chatbot-written suspicion range (14–23%). Regarding the student-written texts, we identified patterns of downgrading, suggesting that teachers were more critical when grading student-written texts. Drawing on post-phenomenology and mediation theory, we discuss AI chatbots as a potentially disruptive technology in higher education practices.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
ChatGPT and other GAI tools can lead to untraceable plagiarism and compromised originality, making cheating harder to detect.
“First, ChatGPT and other GAI tools can lead to untraceable plagiarism and compromised originality, making cheating harder to detect, and therefore prompting the need for a reevaluation of traditional assessment methods (Farazouli, Cerratto-Pargman, and Bolander-Laksov 2024).”
From Widely used but barely trusted: understanding student perceptions on the use of generative AI in higher education (2025) · cites this paper as Farazouli, Cerratto-Pargman, and Bolander-Laksov 2024
The Plagiarism check section (page unavailable) reports that the software flagged only 4 of 14 ChatGPT texts, and that the flagged material would not count as plagiarism in a realistic assessment. The ChatGPT mediating role in assessment practices section (page unavailable) describes teachers questioning authorship and changing how they assess work. These passages support difficulty detecting AI-written submissions and pressure on assessment practices, but they do not establish that the texts were plagiarised, that originality was compromised, or that the findings extend to other GAI tools. (LLM, conf=0.85)
Experimental — accuracy not yet measuredThis has invoked an immediate reaction from the higher education community as the question arose as to how it may affect various forms of student performance evaluation (such as essay and thesis writing).
“This has invoked an immediate reaction from the higher education community as the question arose as to how it may affect various forms of student performance evaluation (such as essay and thesis writing) (Chaudhry et al. 2023; Yu, 2023; Farazouli et al. 2024).”
From AI and ethics: Investigating the first policy responses of higher education institutions to the challenge of generative AI (2024) · cites this paper as Farazouli et al. 2024
The Introduction (page unavailable) says ChatGPT’s launch sparked debate about its impact on education and prompted institutional responses, including bans and guidelines. The paper discusses essays and home examinations, but our copy does not address thesis writing specifically, so that example is not supported by this source. (LLM, conf=0.82)
Experimental — accuracy not yet measuredEven the latest studies focus mostly on issues related to assessment practices and often limit their scope to ChatGPT.
“This is especially true as even the latest studies focus mostly on issues related to assessment practices (Chan, 2023; Farazouli et al. 2024) and often limit their scope to ChatGPT (Cotton et al. 2024)”
From AI and ethics: Investigating the first policy responses of higher education institutions to the challenge of generative AI (2024) · cites this paper as Farazouli et al. 2024
The presence of AI chatbots may reduce teachers' confidence in students' texts, requiring them to be more thorough and suspicious when assessing a text.
“It was also concluded that the presence of AI chatbots may reduce teachers confidence in students’ texts, requiring them to be more thorough and suspicious when assessing a text. 3”
Chatbot-generated responses in take-home exams received pass rates ranging from 38% to 86%, while student-written texts were frequently downgraded.
“A study found that chatbot-generated responses in take-home exams received pass rates ranging from 38% to 86%, while student-written texts were frequently downgraded.”
From a teacher assessment perspective, distinguishing AI-generated texts from independently written ones is challenging.
“However, from a teacher assessment perspective, distinguishing AI-generated texts from independently written ones is challenging [7], and no current tool can reliably make this distinction [1].”
From Generative artificial intelligence in physiotherapy education: great potential amidst challenges- a qualitative interview study (2025) · cites this paper as [7]
Research highlights the difficulty of distinguishing AI-generated work from student writing.
“Research highlights the difficulty of distinguishing AI-generated work from student writing (Farazouli et al., 2024)”
From OTE Reconstructing the Assessment System of the Undergraduate Course “Plant Physiology” in the AI Era: A Framework for Authentic and AI-Resilient Task Design (2026) · cites this paper as Farazouli et al., 2024
Teachers in humanities and social sciences tended to be critical about students' texts, underestimated students' performance, and doubted that some student texts had been finished by GPT.
“They found that teachers tended to be critical about students' texts, underestimated students' performance, and doubted that some student texts had been finished by GPT.”
From A systematic review of the early impact of artificial intelligence on higher education curriculum, instruction, and assessment (2025) · cites this paper as Farazouli et al. (2024)
Farazouli et al. (2023) revealed that teachers were more critical when grading student-written texts as they thought their students might finish their assessment tasks with ChatGPT.
“For example, Farazouli et al. (2023) revealed that the teachers were more critical when grading student-written texts as they thought their students might finish their assessment tasks with ChatGPT.”
From A Scoping Review on How Generative Artificial Intelligence Transforms Assessment in Higher Education (2024) · cites this paper as Farazouli et al., 2023
These concerns negatively influenced the trust relationship between teachers and students.
“These concerns negatively influenced the trust relationship between teachers and students.”
From A systematic review of the early impact of artificial intelligence on higher education curriculum, instruction, and assessment (2025) · cites this paper as Farazouli et al. (2024)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372 · see citing works
Graneheim, U. H., & Lundman, B. (2004) · Nurse Education Today · doi:10.1016/j.nedt.2003.10.001 · Cited by 20,772 · cited 1× · see citing works
“Drawing on Graneheim and Lundman (2004) work we divided the text data into meaning units including participants’ reflections on their assessments during the TT experiment.”
MethodAdvances in Neural Information Processing Systems 29
Teymur, O., Zygalakis, K., & Calderhead, B. (2016) · Cited by 13,881 · see citing works
Kaugeranna, R., Kaugeranna, E., & Aion, (. S. 4. 6. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.giscience.2023.43 · Cited by 13,695 · see citing works
ChatGPT for good? On opportunities and challenges of large language models for education
Kasneci, E., Sessler, K., Küchemann, S., Bannert, M., Dementieva, D., Fischer, F., Gasser, U., Groh, G., Günnemann, S., Hüllermeier, E., et al. (2023) · Learning and Individual Differences · doi:10.1016/j.lindif.2023.102274 · Cited by 7,326 · cited 1× · see citing works
“…and weaknesses report that availability and usefulness are among the most critical success factors, while poor content, lack of legal frameworks and poor conversation design are the main reasons for the failures of the systems in practice (Janssen, Grützner, and Breitner 2021; Kasneci et al. 2023).”
BackgroundWhat if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education
Tlili, A., Shehata, B., Adarkwah, M. A., Bozkurt, A., Hickey, D. T., Huang, R., & Agyemang, B. (2023) · Smart Learning Environments · doi:10.1186/s40561-023-00237-x · Cited by 2,003 · cited 1× · see citing works
“Further research has examined the potential of AI chatbots to enhance education and the strategies education stakeholders need to develop to tackle the risks of possible harm and disruptions (Tlili et al. 2023).”
BackgroundThe Hawthorne Effect: a randomised, controlled trial
McCarney, R., Warner, J. P., Iliffe, S., van Haselen, R. A., Griffin, M., & Fisher, P. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-30 · Cited by 1,730 · see citing works
Emergent Abilities of Large Language Models
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.07682 · Cited by 1,264 · see citing works
ChatGPT User Experience: Implications for Education
Zhai, X. (2023) · doi:10.2139/ssrn.4312418 · Cited by 935 · cited 1× · see citing works
“A number of studies have also inquired whether GPT-3 could author an academic paper with minimum human involvement (Zhai 2022).”
BackgroundMoralizing Technology: Understanding and Designing the Morality of Things
Verbeek, P.-P. (2011) · doi:10.7208/chicago/9780226852904.001.0001 · Cited by 923 · see citing works
Materializing Morality: Design Ethics and Technological Mediation
Verbeek, P.-P. (2006) · Science, Technology, & Human Values · doi:10.1177/0162243905285847 · Cited by 683 · see citing works
A review of opportunities and challenges of chatbots in education
Hwang, G.-J., & Chang, C.-Y. (2023) · Interactive Learning Environments · doi:10.1080/10494820.2021.1952615 · Cited by 530 · see citing works
ChatGPT Goes to Law SchoolPreprint
Choi, J. H., Hickman, K. E., Monahan, A., & Schwarcz, D. B. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4335905 · Cited by 522 · see citing works
Postphenomenology: Essays in the Postmodern Context
Ihde, D. (1993) · Medical Entomology and Zoology · Cited by 411 · see citing works
Nikolic, S., Daniel, S., Haque, R., Belkina, M., Hassan, G. M., Grundy, S., Lyden, S., Neal, P., & Sandison, C. (2023) · European Journal of Engineering Education · doi:10.1080/03043797.2023.2213169 · Cited by 336 · see citing works
Postphenomenological Investigations: Essays on Human–Technology Relations
Ihde, D., Langsdorf, L., Besmer, K. M., Hoel, A. S., Carusi, A., Nizzi, M., Secomandi, F. D. C., Kiran, A. H., Eede, Y. V. D., Bottenberg, F., et al. (2015) · Cited by 300 · see citing works
Drawn to distraction: A qualitative study of off-task use of educational technology
Aagaard, J. (2015) · Computers & Education · doi:10.1016/j.compedu.2015.03.010 · Cited by 273 · see citing works
Sørensen, E. (2001) · Cambridge University Press eBooks · doi:10.1017/cbo9780511576362 · Cited by 256 · see citing works
Can GPT-3 pass a writer’s turing test?
Elkins, K., & Chun, J. (2020) · Journal of Cultural Analytics · doi:10.22148/001c.17212 · Cited by 202 · see citing works
New Modes of Learning Enabled by AI Chatbots: Three Methods and Assignments
Mollick, E. R., & Mollick, L. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4300783 · Cited by 176 · cited 1× · see citing works
“However, recent studies indicate that there is a need for empirical research to examine these developments in context where we can gain insights about the impact they have on practices (Mollick and Mollick 2022; Nikolic et al. 2023).”
BackgroundDesigning assessment in a digital world: an organising framework
Bearman, M., Nieminen, J. H., & Ajjawi, R. (2022) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2022.2069674 · Cited by 116 · see citing works
Qualitative differences in academics’ conceptions of e-assessment
Mimirinis, M. (2018) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2018.1493087 · Cited by 70 · see citing works
Toward a Theory of Technological Mediation: A Program for Postphenomenological Research
Verbeek, P. (2016) · University of Twente Research Information · Cited by 65 · see citing works
Towards a framework for designing and evaluating online assessments in business education
Huber, E., Harris, L. M., Wright, S., White, A. W., Raduescu, C., Zeivots, S., Cram, A., & Brodzeli, A. (2023) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2023.2183487 · Cited by 48 · see citing works
Why do Chatbots fail? A Critical Success Factors Analysis
Janssen, A., Grützner, L., & Breitner, M. H. (2021) · Journal of the Association for Information Systems · Cited by 34 · see citing works
Beyond open book versus closed book: a taxonomy of restrictions in online examinations
Dawson, P., Nicola‐Richmond, K., & Partridge, H. (2023) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2023.2209298 · Cited by 30 · see citing works
Virtual Reality learning environments and technological mediation in construction practice
Voordijk, H., & Vahdatikhaki, F. (2020) · European Journal of Engineering Education · doi:10.1080/03043797.2020.1795085 · Cited by 20 · see citing works
Talk2Learn: A Framework for Chatbot Learning
Bahja, M., Hammad, R., & Hassouna, M. S. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-29736-7_44 · Cited by 14 · see citing works
Miranda: A Chatbot for Supporting Self-regulated Learning
Maldonado‐Mahauad, J., Pérez‐Sanagustín, M., Carvallo-Vega, J., Narvaez, E., & Calle, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16290-9_36 · Cited by 8 · see citing works
Assessment design decisions in practice: Profile identification in approaches to assessment design
Ruiz, J. F., Panadero, E., García-Pérez, D., & Pinedo, L. (2021) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2021.1937512 · Cited by 7 · see citing works
Assessment from a disciplinary approach: design and implementation in three undergraduate programmes
Ruiz, J. F., Panadero, E., & García-Pérez, D. (2021) · Assessment in Education Principles Policy and Practice · doi:10.1080/0969594x.2021.1999210 · Cited by 6 · see citing works
Pargman, T. C., & Jahnke, I. (2019) · Springer eBooks · doi:10.1007/978-3-030-10764-2_1 · Cited by 5 · see citing works
An Educational Conversational Agent for GDPR
Mirzababaei, B., & Pammer‐Schindler, V. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16290-9_38 · Cited by 5 · see citing works
OpenAI, Achiam, J., Adler, S., Agarwal, S., Ahmad, L., Akkaya, I., Aleman, F. L., Almeida, D., Altenschmidt, J., Altman, S., et al. (2023) · arXiv · doi:10.48550/arxiv.2303.08774
Flagged first, then by citations
Research works printed but not linked: 2
- Cerratto-Pargman, T., O. Knutsson, and P. Karlström. 2015. "Materiality of Online Students' Peer-Review Activities in Higher Education. " https://www.semanticscholar.org/paper/Materiality-of-Online-Students%27-Peer-Review-in- Pargman-Knutsson/443f3043a94be285acc84f021d18df1a05b5070d.
- Radford, A., J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever. 2019. "Language Models Are Unsupervised Multitask Learners. " https://www.semanticscholar.org/paper/Language-Models-are-Unsupervised-Multitask-Learners-Radford-Wu/ 9405cc0d6169988371b2755e573cc28650d14dfe.
Cited by
Sorted by citations
A Scoping Review on How Generative Artificial Intelligence Transforms Assessment in Higher Education
Xia, Q., Weng, X., Ouyang, F., Lin, T. J., & Chiu, T. K. F. (2024) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-024-00468-z · Cited by 493 · see citing works
ChatGPT: The End of Online Exam Integrity?
Susnjak, T. (2022) · arXiv · doi:10.48550/arxiv.2212.09292 · Cited by 370 · see citing works
“ChatGPT’s ability to answer exam questions has already been demonstrated in multiple studies [21–23].”
Stöhr, C., Ou, A. W., & Malmström, H. (2024) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2024.100259 · Cited by 319 · see citing works
Batista, J., Mesquita, A., & Carnaz, G. (2024) · Information · doi:10.3390/info15110676 · Cited by 256 · see citing works
Dabis, A., & Csáki, C. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-03526-z · Cited by 181 · see citing works
“This is especially true as even the latest studies focus mostly on issues related to assessment practices (Chan, 2023; Farazouli et al. 2024) and often limit their scope to ChatGPT (Cotton et al. 2024) (this speci fi c tool still dominates discourses of LLMs despite the availability of many other…”
InfluentialPedagogical Applications of Generative AI in Higher Education: A Systematic Review of the Field
Qian, Y. (2025) · TechTrends · doi:10.1007/s11528-025-01100-1 · Cited by 206 · see citing works
“Similarly, the prominence of Writing/Composition (Cum-mings et al., 2024; Farazouli et al., 2024; Nguyen et al., 2024; Tsao and Nogues, 2024) and ESL/EFL studies (Esca-lante et al., 2024; Guo & Li, 2024; Guo et al., 2024; Waluyo & Kusumastuti, 2024; Wiboolyasarin et al., 2024) stems from GenAI’s…”
Generative AI and higher education: a review of claims from the first months of ChatGPT
Jensen, L. X., Buhl, A., Sharma, A., & Bearman, M. (2025) · Higher Education · doi:10.1007/s10734-024-01265-3 · Cited by 144 · see citing works
ChatGPT in Teaching and Learning: A Systematic Review
Ali, D., Fatemi, Y., Boskabadi, E., Nikfar, M., Ugwuoke, J., & Ali, H. (2024) · Education Sciences · doi:10.3390/educsci14060643 · Cited by 146 · see citing works
Wijaya, T. T., Yu, Q., Cao, Y., He, Y., & Leung, F. K. S. (2024) · Behavioral Sciences · doi:10.3390/bs14111008 · Cited by 103 · see citing works
Nikolic, S., Sandison, C., Haque, R., Daniel, S., Grundy, S., Belkina, M., Lyden, S., Hassan, G. M., & Neal, P. (2024) · Australasian Journal of Engineering Education · doi:10.1080/22054952.2024.2372154 · Cited by 98 · see citing works
Wijaya, T. T., Su, M., Cao, Y., Weinhandl, R., & Houghton, T. (2025) · Education and Information Technologies · doi:10.1007/s10639-024-12837-2 · Cited by 80 · see citing works
Liang, J., Stephens, J. M., & Brown, G. T. L. (2025) · Frontiers in Education · doi:10.3389/feduc.2025.1522841 · Cited by 112 · see citing works
Zhang, D., Wijaya, T. T., Wang, Y., Su, M., Li, X., & Damayanti, N. W. (2025) · Scientific Reports · doi:10.1038/s41598-025-99127-0 · Cited by 63 · see citing works
Alshamy, A., Al-Harthi, A. S. A., & Abdullah, S. (2025) · Education Sciences · doi:10.3390/educsci15040501 · Cited by 46 · see citing works
Generative AI and educational assessments: a systematic review
Graduate School of Education, T. U. O. W. A., Zhao, J., Chapman, E., Sabet, P. G. P., & Global Curtin, C. U. (2024) · Education Research and Perspectives · doi:10.70953/erpv51.2412006 · Cited by 45 · see citing works
Alharbi, M. K., Syed, W., Innab, A., Basil A. Al-Rawi, M., Alsadoun, A., & Bashatah, A. (2024) · Scientific Reports · doi:10.1038/s41598-024-73359-y · Cited by 36 · see citing works
Lindbäck, Y., Schröder, K., Engström, T., Valeskog, K., & Sonesson, S. (2025) · BMC Medical Education · doi:10.1186/s12909-025-07106-w · Cited by 22 · see citing works
“This raises concerns about assessment reliability and validity, academic integrity, and whether assessments accurately reflect students’ knowledge and skills [1, 7].”
Lindbäck, Y., Valeskog, K., Schröder, K., & Sonesson, S. (2025) · Journal of Medical Education and Curricular Development · doi:10.1177/23821205251378794 · Cited by 4 · see citing works
Cabrera, C., & Neville, R. (2025) · Perspectives: Policy and Practice in Higher Education · doi:10.1080/13603108.2025.2595453 · Cited by 4 · see citing works
Peng, J. (2026) · World Journal of Educational Research · doi:10.22158/wjer.v13n2p127
- Authors
- Farazouli, A., Cerratto-Pargman, T., Bolander-Laksov, K., & McGrath, C.
- Affiliations
- Stockholm University (SE)
- Stockholm University, Stockholm, Sweden
- Corresponding author
- Farazouli, Alexandra
- Venue
- Assessment & Evaluation in Higher Education
- Publisher
- Informa UK Limited
- Published
- 2023-08
- Volume
- 49
- Issue
- 3
- Pages
- 363-375
- ISSN
- 0260-2938 (ISSN-L)
- 1469-297X
- OpenAlex
- W4385455730
- Cited by
- 219
- References
- 34
- Citations by year
- 2026: 60 · 2025: 81 · 2024: 71 · 2023: 7
- FWCI
- 51.31
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Assessment & Evaluation in Higher Education (publishedVersion, cc-by)
- Funders
- Marcus och Amalia Wallenbergs minnesfond (2020.0138)
- Topics
- Physical Sciences › Computer Science › Information Systems › Artificial Intelligence in Education
- Social Sciences › Safety Research › Academic integrity and plagiarism
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- AI chatbots, ChatGPT, generative AI, university faculty, assessment practices, Turing test, teacher perceptions, grading, postphenomenology, mediation theory, disruptive technologies, higher education, undergraduate assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
