Systematic review of research on artificial intelligence applications in higher education – where are the educators?
Open accessThe DOI matches its Crossref record; 158 of the 162 references it cites were found in a registry or an index; 7 of the 43 sources its claims cite were checked for support.
Abstract
Abstract According to various international reports, Artificial Intelligence in Education (AIEd) is one of the currently emerging fields in educational technology. Whilst it has been around for about 30 years, it is still unclear for educators how to make pedagogical advantage of it on a broader scale, and how it can actually impact meaningfully on teaching and learning in higher education. This paper seeks to provide an overview of research on AI applications in higher education through a systematic review. Out of 2656 initially identified publications for the period between 2007 and 2018, 146 articles were included for final synthesis, according to explicit inclusion and exclusion criteria. The descriptive results show that most of the disciplines involved in AIEd papers come from Computer Science and STEM, and that quantitative methods were the most frequently used in empirical studies. The synthesis of results presents four areas of AIEd applications in academic support services, and institutional and administrative services: 1. profiling and prediction, 2. assessment and evaluation, 3. adaptive systems and personalisation, and 4. intelligent tutoring systems. The conclusions reflect on the almost lack of critical reflection of challenges and risks of AIEd, the weak connection to theoretical pedagogical perspectives, and the need for further exploration of ethical and educational approaches in the application of AIEd in higher education.
Overview
CoChat Summary
TL;DR
This systematic review synthesizes 146 articles (from 2,656 initial records, 2007–2018) on AI applications in higher education, mapping four application areas and finding that Computer Science/STEM authors dominate while educators are largely absent. It concludes that critical reflection on ethical/pedagogical risks and explicit definitions of AI are rare, and calls for more educational perspectives and theory.
Key findings
- The review identifies four areas of AIEd applications in higher education: profiling and prediction, assessment and evaluation, adaptive systems and personalisation, and intelligent tutoring systems, with 17 sub-categories.
- First authors came mainly from Computer Science (n = 61) and STEM departments (n = 29); only 13 papers (8.9%) had first authors with an Education background. Half of all articles came from four countries: USA, China, Taiwan, and Turkey.
- Only 5 of 146 articles (3.4%) explicitly defined “Artificial Intelligence,” and only 2 of 146 (1.4%) critically reflected on ethical implications, challenges, and risks of applying AI in education.
- Most empirical studies used quantitative methods (73.3%), while 20.5% were theoretical/descriptive, 0.7% qualitative, and 5.5% mixed-methods. By level, 92 studies (63.0%) related to academic support services, 48 (32.8%) to administrative/institutional services, and 6 (4.1%) to both.
Methods
- Systematic review with explicit search and inclusion/exclusion criteria; searched EBSCO Education Source, Web of Science, and Scopus in November 2018, yielding 2,656 initial records and 146 articles included for synthesis.
- Screening was done by three coders; inter-rater reliability using Cohen’s kappa was κ = .79 between raters A and B, κ = .89 between A and C, and κ = .69 between B and C (median = .79). After initial screening, 332 articles remained for full-text screening; 41 could not be retrieved, 291 were retrieved and coded, and 149 were excluded, leaving 146.
- Data were coded in EPPI Reviewer and descriptively analysed in R; codes covered article information, study design and execution, AI applications in the student life cycle, specific applications and methods, presence of challenges/benefits, and whether AI was defined.
Limitations
- Stated in the paper: the review is limited by its search strategy—only peer-reviewed articles in English or Spanish indexed in three databases were included, excluding other languages, conference proceedings, book chapters, grey literature, and non-indexed journals; no specific Spanish search string was used, narrowing the possibility of including Spanish papers not indexed with the chosen keywords.
- Evident from the text: geographical distribution and author disciplinary affiliation were analysed using only the first author’s country and affiliation, so the results may not represent all co-authors.
- Evident from the text: most included intelligent tutoring system studies were implemented and validated over short-term periods (a course or a semester), and no longitudinal studies were identified except Jackson and Cossitt (2015), limiting conclusions about long-term effectiveness.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
6 Valid · 3 Not valid · 59 Not checked · 80 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“One study presented negative results regarding the type of support that the ITS provided (Adamson et al., 2014)”
§ Systematic review of research on artificial intelligence applications in higher educationwhere are the educators? > Intelligent tutoring systems
What the sources say
- Supportedcites Adamson et al., 2014 source
The Results section of our copy reports exactly this: in the study with more advanced learners, 'there was a trend for a negative effect of the Revoicing manipulation F(1, 104)=2.22,p=.13 ... within the APT Feedback condition, there was a significant negative effect of Revoicing (p<.05)', with essentially no effect of APT Feedback (p=.87), and the process analysis gives 'confirming evidence that there was a negative impact of the Revoicing manipulation with this population' while the APT Feedback condition showed 'no effect of any variable'. The Discussion frames these support types as unneeded for advanced learners, who benefit instead from Agree-Disagree support. Two caveats on wording rather than substance: the support was delivered by dynamic conversational tutoring agents within a CSCL chat environment (characterized as tutorial-dialogue-agent technology) rather than a classic ITS, and the negative findings are specific to the Revoicing manipulation with this population — other studies in the same paper (e.g. the 9th-grade Diffusion lab) reported positive effects. (LLM, conf=0.80)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The systematic review by Bartolomé et al. (2018) also revealed this lack of explicit pedagogical perspectives in the studies analysed.”
§ Systematic review of research on artificial intelligence applications in higher educationwhere are the educators? > International Journal of Artificial Intelligence in Education, Computers & Education, and the International Journal of Emerging Technologies in Learning.
What the sources say
- Supportedcites Bartolomé et al. (2018) source
The Abstract (p. 1) describes a systematic exploration of the literature and explicitly reports a lack of pedagogical perspective in the experiences analysed. The Conclusions (p. 14) likewise say that relevant educational approaches were not addressed in most texts in the original sample. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The descriptive results show that most of the disciplines involved in AIEd papers come from Computer Science and STEM, and that quantitative methods were the most frequently used in empirical studies.”
- Also stated in: § Author affiliations · plain
“Researchers working in departments of Computer Science contributed by far the greatest number of papers (n = 61) followed by Science, Technology, Engineering and Mathematics (STEM) departments (n = 29).”
§ Systematic review of research on artificial intelligence applications in higher educationwhere are the educators? > Abstract
plainn = 61- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Also stated in: § Author affiliations · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The synthesis of results presents four areas of AIEd applications in academic support services, and institutional and administrative services: 1. profiling and prediction, 2. assessment and evaluation, 3. adaptive systems and personalisation, and 4. intelligent tutoring systems.”
- Also stated in: § AI applications in higher education · plain
“The iterative coding process led to the following four areas of AI applications with 17 sub-categories, covered in the publications: a) adaptive systems and personalisation, b) assessment and evaluation, c) profiling and prediction, and d) intelligent tutoring systems.”
§ Systematic review of research on artificial intelligence applications in higher educationwhere are the educators? > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Also stated in: § AI applications in higher education · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The conclusions reflect on the almost lack of critical reflection of challenges and risks of AIEd, the weak connection to theoretical pedagogical perspectives, and the need for further exploration of ethical and educational approaches in the application of AIEd in higher education.”
§ Systematic review of research on artificial intelligence applications in higher educationwhere are the educators? > Abstract
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
55 more claims
Create a free account to see all 148 claims and to verify the 11 references still open.
- It may state two separate claims as one. 51
- Probably not a substantial claim. 49
- Borderline: one of the tests for a substantial claim is unclear. 2
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“However, the current body of literature is overly reliant on educators' perspectives on the use of generative AI, especially in the interpretation of findings.”
From The rapid rise of generative AI and its implications for academic integrity: Students’ perceptions and use of chatbots for assistance with assessments (2024) · cites this paper as Zawacki-Richter et al., 2019
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This capability to tailor educational experiences to individual needs is particularly valuable in addressing diverse student populations and enhancing overall educational quality (Zawacki-Richter et al., 2019).”
From Attitudes of faculty members in Palestinian universities toward employing artificial intelligence applications in higher education: opportunities and challenges (2024) · cites this paper as Zawacki-Richter et al., 2019
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These AI-assisted writing tools offer automated feedback on various aspects of writing, including organization, coherence, grammar, and vocabulary, thereby facilitating more effective writing performance improvements.”
From Enhancing academic writing skills and motivation: assessing the efficacy of ChatGPT in AI-assisted language learning for EFL students (2023) · cites this paper as Zawacki-Richter et al., 2019
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In their systematic review, Zawacki-Richter and colleagues (2019) found that the fields of Computer Science and STEM contributed the greatest number of papers on AI technology integration in higher education and only 8.9% of papers were written by authors in the field of education.”
From Procedure to Evaluate Artificial Intelligence (AI) Output for Educational and Clinical Tasks (2026) · cites this paper as Zawacki-Richter et al., 2019
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These studies suggest current gaps in GenAI in education research include addressing ethical and privacy concerns, integrating GenAI with educational theories and practices, and exploring fairness, accountability, and transparency [72–74].”
From Generative AI in Higher Education: Balancing Innovation and Integrity (2025) · cites this paper as [72]
Experimental — accuracy not yet measured
57 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Systematic review of research on artificial intelligence applications in higher educationwhere are the educators? > Search strategy
“The initial search string (see Table 1) and criteria (see Table 2) for this systematic review included peer-reviewed articles in English, reporting on artificial intelligence within education at any level”
§ Systematic review of research on artificial intelligence applications in higher educationwhere are the educators? > International Journal of Artificial Intelligence in Education, Computers & Education, and the International Journal of Emerging Technologies in Learning.
“| English or Spanish language | Not in English or Spanish |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Systematic review of research on artificial intelligence applications in higher educationwhere are the educators? > Methods
“Thirty studies (20.5%) were coded as being theoretical or descriptive in nature.”
§ Systematic review of research on artificial intelligence applications in higher educationwhere are the educators? > International Journal of Artificial Intelligence in Education, Computers & Education, and the International Journal of Emerging Technologies in Learning.
“| Empirical, primary research | Not primary research (e.g., review) |”
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
A Coefficient of Agreement for Nominal Scales
Cohen, J. (1960) · Educational and Psychological Measurement · doi:10.1177/001316446002000104 · Cited by 42,398 · cited 1×
“…were randomly selected to evaluate the coding decisions of the three coders (A, B and C) to determine inter-rater reliability using Cohen’s kappa (κ) (Cohen, 1960), which is a coefficient for the degree of consistency among raters, based on the number of codes in the coding scheme (Neumann, 2007,…”
MethodArtificial intelligence: a modern approach
(1995) · Choice Reviews Online · doi:10.5860/choice.33-1577 · Cited by 21,974
Statistical Methods for Rates and Proportions
Fleiss, J. L., Levin, B., & Paik, M. C. (2003) · Wiley Series in Probability and Statistics · doi:10.1002/0471445428 · Cited by 18,646
Artificial intelligence: A modern approach
Nilsson, N. J. (1996) · Artificial Intelligence · doi:10.1016/0004-3702(96)00007-0 · Cited by 10,583
Statistical Methods for Rates and Proportions.
Everitt, B. S., & Fleiss, J. L. (1981) · Biometrics · doi:10.2307/2530193 · Cited by 8,990
Social Research Methods: Qualitative and Quantitative Approaches
Djamba, Y. K., & Neuman, W. L. (2002) · Teaching Sociology · doi:10.2307/3211488 · Cited by 8,851
Social Research Methods: Qualitative and Quantitative Approaches
Neuman, W. L. (1991) · Cited by 7,055
Systematic reviews in the social sciences: a practical guide
(2006) · Choice Reviews Online · doi:10.5860/choice.43-5664 · Cited by 4,089
Research works printed but not linked: 3
- Alfarsi, G. M. S., Omar, K. A. M., & Alsinani, M. J. (2017). A rule-based system for advising undergraduate students. Journal of Theoretical and Applied Information Technology, 95(11) Retrieved from http://www.jatit.org .
- Baker, T., & Smith, L. (2019). Educ-AI-tion rebooted? Exploring the future of artificial intelligence in schools and colleges. Retrieved from Nesta Foundation website: https://media.nesta.org.uk/documents/Future_of_AI_and_education_v5_WEB.pdf
- Contact North. (2018). Ten facts about artificial intelligence in teaching and learning. Retrieved from https://teachonline.ca/sites/default/files/tools-trends/downloads/ten_facts_about_artificial_intelligence.pdf
Cited by
Chatting and Cheating. Ensuring academic integrity in the era of ChatGPT
Cotton, D., Cotton, P., & Shipway, J. R. (2023) · doi:10.35542/osf.io/mrz8h · Cited by 3,079
Chatting and cheating: Ensuring academic integrity in the era of ChatGPT
Cotton, D. R. E., Cotton, P. A., & Shipway, J. R. (2024) · Innovations in Education and Teaching International · doi:10.1080/14703297.2023.2190148 · Cited by 3,087
“GPT-3 could be used to generate customised exams or quizzes for each student based on their individual needs and abilities (Barber et al., 2021; Zawacki-Richter et al., 2019).”
BackgroundConceptualizing AI literacy: An exploratory review
Ng, D. T. K., Leung, J. K. L., Chu, S. K. W., & Qiao, M. S. (2021) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2021.100041 · Cited by 2,205
What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature
Lo, C. K. (2023) · Education Sciences · doi:10.3390/educsci13040410 · Cited by 2,024
“• RQ3: What are the potential issues associated with ChatGPT, and how can they be addressed?”
BackgroundArtificial intelligence in higher education: the state of the field
Crompton, H., & Burke, D. (2023) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-023-00392-8 · Cited by 2,010
“First authors were chosen as that person is the primary researcher of the study and this follows past research practice (e.g., Zawacki-Richter et al., 2019).”
BackgroundMethodInfluential
Metadata
- Authors
- Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F.
- Affiliations
- Carl von Ossietzky Universität Oldenburg (DE)
- Venue
- International Journal of Educational Technology in Higher Education
- Published
- 2019-10-28
- Volume
- 16
- Issue
- 1
- ISSN
- 2365-9440 (ISSN-L)
- OpenAlex
- W2981863007
- Semantic Scholar
- 04de4d9eb0d53025c8ab6c99d1e743f4c1bc1eb6
- Cited by
- 7,242
- References
- 162
- Citations by year
- 2026: 2999 · 2025: 2145 · 2024: 1131 · 2023: 525 · 2022: 239 · 2021: 139 · 2020: 62 · 2019: 1
- FWCI
- 19.23
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- International Journal of Educational Technology in Higher Education (publishedVersion, cc-by)
- UniSA Research Outputs Repository (University of South Australia) (submittedVersion, other-oa)
- Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- Repositori ObertUDL (University of Lleida) (submittedVersion, cc-by)
- Topics
- Social Sciences › Education › Artificial Intelligence in Education
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
- Keywords
- artificial intelligence in education, higher education, academic support, assessment and evaluation, adaptive learning systems, personalization, intelligent tutoring systems, ethical challenges of AI, conceptions of teaching, research gaps
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Systematic review of research on artificial intelligence applications in higher education – where are the educators? https://app.cochat.ai/papers/10.1186/s41239-019-0171-0
- The paper
- Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education. https://doi.org/10.1186/s41239-019-0171-0
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.