Generative AI chatbots in higher education: a review of an emerging research area
Open accessThe DOI matches its Crossref record; 30 of the 30 references it cites were found in a registry or an index.
Abstract
Artificial intelligence (AI) chatbots trained on large language models are an example of generative AI which brings promises and threats to the higher education sector. In this study, we examine the emerging research area of AI chatbots in higher education (HE), focusing specifically on empirical studies conducted since the release of ChatGPT. Our review includes 23 research articles published between December 2022 and December 2023 exploring the use of AI chatbots in HE settings. We take a three-pronged approach to the empirical data. We first examine the state of the emerging field of AI chatbots in HE. Second, we identify the theories of learning used in the empirical studies on AI chatbots in HE. Third, we scrutinise the discourses of AI in HE framing the latest empirical work on AI chatbots. Our findings contribute to a better understanding of the eclectic state of the nascent research area of AI chatbots in HE, the lack of common conceptual groundings about human learning, and the presence of both dystopian and utopian discourses about the future role of AI chatbots in HE.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
A typology of reviews: an analysis of 14 review types and associated methodologies
Grant, M. J., & Booth, A. (2009) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2009.00848.x · Cited by 10,817
Dwivedi, Y. K., Kshetri, N., Hughes, L., Slade, E. L., Jeyaraj, A., Kar, A. K., Baabdullah, A. M., Koohang, A., Raghavan, V., Ahuja, M., et al. (2023) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2023.102642 · Cited by 4,584
ELIZA—a computer program for the study of natural language communication between man and machine
Weizenbaum, J. (1966) · Communications of the ACM · doi:10.1145/365153.365168 · Cited by 4,452 · cited 1×
“Chatbots date back to the 1960s, when MIT’s first chatbot, ELIZA, could simulate a conversation (Weizenbaum, 1966).”
Discourse Analysis as Theory and Method
Jørgensen, M., & Phillips, L. (2002) · doi:10.4135/9781849208871 · Cited by 4,118 · cited 1×
“…key signifiers (i.e., power) and their potential combination with other key signifiers (i.e., future) to understand “the chains of meaning that discourses bring together in this way, […] [to] identify discourses (and identities and social spaces)” (Winther Jørgensen and Phillips, 2002 p.27).”
Students' voices on generative ai: perceptions, benefits, and challenges in higher education
Chan, C. K. Y., & Hu, W. (2023) · arXiv · doi:10.48550/arxiv.2305.00290 · Cited by 2,504 · cited 2×
“…Li et al. (2023), Yan (2023), and Lai et al. (2023) make explicit reference to learning theories, a number of other studies (Bernabei et al., 2023; Chan & Hu, 2023; Habibi et al., 2023; Maheshwari, 2023) draw on behavioural research theories such as the technology acceptance model (TAM), the…”
Students’ voices on generative AI: perceptions, benefits, and challenges in higher education
Chan, C. K. Y., & Hu, W. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00411-8 · Cited by 2,301 · cited 2×
“…Li et al. (2023), Yan (2023), and Lai et al. (2023) make explicit reference to learning theories, a number of other studies (Bernabei et al., 2023; Chan & Hu, 2023; Habibi et al., 2023; Maheshwari, 2023) draw on behavioural research theories such as the technology acceptance model (TAM), the…”
Ethics of AI in Education: Towards a Community-Wide Framework
Holmes, W., Porayska-Pomsta, K., Holstein, K., Sutherland, E., Baker, T., Shum, S. B., Santos, O. C., Rodrigo, M. T., Cukurova, M., Bittencourt, I. I., et al. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00239-1 · Cited by 1,437 · cited 1×
“We argue that education journals need to stay abreast of developments in technology-mediated learning to offer nuance from a position of education values and ethics (Holmes et al., 2022; McGrath et al., 2023).”
Emergent Abilities of Large Language Models
Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. (2022) · arXiv · doi:10.48550/arxiv.2206.07682 · Cited by 1,264 · cited 1×
“LLMs are artificial neural networks that process and generate natural language text and mimic human-created text (Wei et al., 2022).”
Yilmaz, R., & Karaoglan Yilmaz, F. G. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100147 · Cited by 745
Impact of ChatGPT on learners in a L2 writing practicum: an exploratory investigation
Yan, D. (2023) · Education and Information Technologies · doi:10.1007/s10639-023-11742-4 · Cited by 713 · cited 4×
“A number of the selected studies focus on second language learning ( n = 5) and suggest that chatbots help students structure their thoughts in the second language (e.g. Yan, 2023; Zou & Huang, 2023).”
InfluentialAI-generated feedback on writing: insights into efficacy and ENL student preference
Escalante, J., Pack, A., & Barrett, A. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00425-2 · Cited by 589 · cited 2×
“Studies such as Escalante et al. (2023), Farazouli et al. (2023), and Li et al. (2023) raise concerns about GAI chatbots’ disruptive impact on education practice, particularly their potential threat to assessment and their negative consequences for students’ reflective writing and critical thinking.”
Do AI chatbots improve students learning outcomes? Evidence from a meta‐analysis
Wu, R., & Yu, Z. (2024) · British Journal of Educational Technology · doi:10.1111/bjet.13334 · Cited by 576 · cited 4×
“Wu and Yu (2023) and Alemdag (2023) conduct meta-analyses on empirical studies and collect data on effect sizes irrespective of domain, discipline, and contextual setting in primary, secondary, and tertiary education settings.”
InfluentialDiscourses of artificial intelligence in higher education: a critical literature review
Bearman, M., Ryan, J., & Ajjawi, R. (2023) · Higher Education · doi:10.1007/s10734-022-00937-2 · Cited by 545
In search of artificial intelligence (AI) literacy in teacher education: A scoping review
Sperling, K., Stenberg, C.-J., McGrath, C., Åkerfeldt, A., Heintz, F., & Stenliden, L. (2024) · Computers and Education Open · doi:10.1016/j.caeo.2024.100169 · Cited by 498 · cited 1×
“Similar findings have been shown elsewhere and, more recently, illustrating how matters relevant to AI in education settings are more frequent in computer and technology journals (Sperling et al., 2024).”
Self-citationGuo, K., & Wang, D. (2024) · Education and Information Technologies · doi:10.1007/s10639-023-12146-0 · Cited by 449
Barrett, A., & Pack, A. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00427-0 · Cited by 391
Mohamed, A. M. (2024) · Education and Information Technologies · doi:10.1007/s10639-023-11917-z · Cited by 387 · cited 1×
“Additionally, Rodway and Schepman (2023) and Mohamed et al. (2023) discuss AI in education as enabling teachers to offer students individualised and customised experiences.”
Kohnke, L., Moorhouse, B. L., & Zou, D. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100156 · Cited by 309 · cited 1×
“Kohnke et al. (2023) suggest that familiarity and confidence in using AI-driven teaching tools play an important role in adoption, but stress there are challenges that language instructors face that require tailored support and professional development.”
ChatGPT in education: global reactions to AI innovations
Fütterer, T., Fischer, C., Alekseeva, A., Chen, X., Tate, T., Warschauer, M., & Gerjets, P. (2023) · Scientific Reports · doi:10.1038/s41598-023-42227-6 · Cited by 242 · cited 2×
“Major technological advances, such as GAI based on LLMs, can cause significant public and educational concerns (Barman et al., 2019; Fütterer et al., 2023).”
Lai, C. Y., Cheung, K. Y., & Chan, C. S. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100178 · Cited by 230
Farazouli, A., Cerratto-Pargman, T., Bolander-Laksov, K., & McGrath, C. (2024) · Assessment & Evaluation in Higher Education · doi:10.1080/02602938.2023.2241676 · Cited by 219
Self-citationAnsari, A. N., Ahmad, S., & Bhutta, S. M. (2024) · Education and Information Technologies · doi:10.1007/s10639-023-12223-4 · Cited by 198 · cited 3×
“To date, we identify three existing review studies examining the impact of GAI chatbots (Alemdag, 2023; Ansari et al., 2023; Wu & Yu, 2023).”
McGrath, C., Cerratto Pargman, T., Juth, N., & Palmgren, P. J. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100139 · Cited by 195 · cited 1×
“We argue that education journals need to stay abreast of developments in technology-mediated learning to offer nuance from a position of education values and ethics (Holmes et al., 2022; McGrath et al., 2023).”
Self-citationChatGPT in higher education learning: Acceptance and use
Habibi, A., Muhaimin, M., Danibao, B. K., Wibowo, Y. G., Wahyuni, S., & Octavia, A. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100190 · Cited by 173
Rodway, P., & Schepman, A. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100150 · Cited by 144 · cited 3×
“Survey studies ( n = 11) use self-reported data to identify the impact (Rodway & Schepman, 2023), readiness (Al-Zahrani, 2023), or willingness (Lai et al., 2023) to engage with GAI in educational practice.”
The impact of ChatGPT on L2 writing and expected responses: Voice from doctoral students
Zou, M., & Huang, L. (2024) · Education and Information Technologies · doi:10.1007/s10639-023-12397-x · Cited by 139 · cited 2×
“A number of the selected studies focus on second language learning ( n = 5) and suggest that chatbots help students structure their thoughts in the second language (e.g. Yan, 2023; Zou & Huang, 2023).”
Al-Zahrani, A. M. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2271445 · Cited by 120 · cited 6×
“…of utopia-around-the-corner discourses also appear in the discussion of the same paper, when the authors note: “In summary, the findings indicate the readiness of the higher education community in Saudi Arabi to integrate AI technologies in research and development” (Al-Zahrani, 2023, p. 11).”
InfluentialPursnani, V., Sermet, Y., Kurt, M., & Demir, I. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100183 · Cited by 106
Can large language models write reflectively
Li, Y., Sha, L., Yan, L., Lin, J., Raković, M., Galbraith, K., Lyons, K., Gašević, D., & Chen, G. (2023) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2023.100140 · Cited by 99 · cited 5×
“Of the 23 selected studies, only three make use of learning theories explicitly, referring to experiential learning (Li et al., 2023; Yan, 2023), reflective learning (Li et al., 2023; Yan, 2023), active learning (Lai et al., 2023), and self-regulated learning (SRL) (Lai et al., 2023).”
Influential
Cited by
Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies
Deng, R., Jiang, M., Yu, X., Lu, Y., & Liu, S. (2025) · Computers & Education · doi:10.1016/j.compedu.2024.105224 · Cited by 531
Modeling AI-assisted writing: How self-regulated learning influences writing outcomes
Jin, F., Lin, C.-H., & Lai, C. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2024.108538 · Cited by 93
Time, emotions and moral judgements: how university students position GenAI within their study
Bearman, M., Fawns, T., Corbin, T., Henderson, M., Liang, Y., Oberg, G., Walton, J., & Matthews, K. E. (2026) · Higher Education Research & Development · doi:10.1080/07294360.2025.2580616 · Cited by 10
“Indeed, a 2024 review suggests limited empirical research as only fourteen studies engaged with students as participants and only half of these studies elicited what students themselves thought about or did with respect to GenAI (McGrath et al., 2024).”
Kumar, A., Baijal, A., & Kharat, M. (2025) · Journal of International Education in Business · doi:10.1108/jieb-07-2024-0086 · Cited by 6
Metadata
- Authors
- McGrath, C., Farazouli, A., & Cerratto-Pargman, T.
- Venue
- Higher Education
- Published
- 2024-08-24
- Volume
- 89
- Issue
- 6
- Pages
- 1533-1549
- ISSN
- 0018-1560 (ISSN-L)
- 1573-174X
- OpenAlex
- W4401850697
- Cited by
- 136
- References
- 30
- FWCI
- 52.04
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.