CoChat

Systematic review of research on artificial intelligence applications in higher education – where are the educators?

Open access
Zawacki-Richter, Olaf, Marín, Victoria I., Bond, Melissa, Gouverneur, Franziska · 2019 · International Journal of Educational Technology in Higher Education

The DOI matches its Crossref record; 158 of the 162 references it cites were found in a registry or an index; 7 of the 43 sources its claims cite were checked for support.

Abstract

Abstract According to various international reports, Artificial Intelligence in Education (AIEd) is one of the currently emerging fields in educational technology. Whilst it has been around for about 30 years, it is still unclear for educators how to make pedagogical advantage of it on a broader scale, and how it can actually impact meaningfully on teaching and learning in higher education. This paper seeks to provide an overview of research on AI applications in higher education through a systematic review. Out of 2656 initially identified publications for the period between 2007 and 2018, 146 articles were included for final synthesis, according to explicit inclusion and exclusion criteria. The descriptive results show that most of the disciplines involved in AIEd papers come from Computer Science and STEM, and that quantitative methods were the most frequently used in empirical studies. The synthesis of results presents four areas of AIEd applications in academic support services, and institutional and administrative services: 1. profiling and prediction, 2. assessment and evaluation, 3. adaptive systems and personalisation, and 4. intelligent tutoring systems. The conclusions reflect on the almost lack of critical reflection of challenges and risks of AIEd, the weak connection to theoretical pedagogical perspectives, and the need for further exploration of ethical and educational approaches in the application of AIEd in higher education.

Overview

CoChat Summary

TL;DR

This systematic review synthesizes 146 articles (from 2,656 initial records, 2007–2018) on AI applications in higher education, mapping four application areas and finding that Computer Science/STEM authors dominate while educators are largely absent. It concludes that critical reflection on ethical/pedagogical risks and explicit definitions of AI are rare, and calls for more educational perspectives and theory.

Key findings

  • The review identifies four areas of AIEd applications in higher education: profiling and prediction, assessment and evaluation, adaptive systems and personalisation, and intelligent tutoring systems, with 17 sub-categories.
  • First authors came mainly from Computer Science (n = 61) and STEM departments (n = 29); only 13 papers (8.9%) had first authors with an Education background. Half of all articles came from four countries: USA, China, Taiwan, and Turkey.
  • Only 5 of 146 articles (3.4%) explicitly defined “Artificial Intelligence,” and only 2 of 146 (1.4%) critically reflected on ethical implications, challenges, and risks of applying AI in education.
  • Most empirical studies used quantitative methods (73.3%), while 20.5% were theoretical/descriptive, 0.7% qualitative, and 5.5% mixed-methods. By level, 92 studies (63.0%) related to academic support services, 48 (32.8%) to administrative/institutional services, and 6 (4.1%) to both.

Methods

  • Systematic review with explicit search and inclusion/exclusion criteria; searched EBSCO Education Source, Web of Science, and Scopus in November 2018, yielding 2,656 initial records and 146 articles included for synthesis.
  • Screening was done by three coders; inter-rater reliability using Cohen’s kappa was κ = .79 between raters A and B, κ = .89 between A and C, and κ = .69 between B and C (median = .79). After initial screening, 332 articles remained for full-text screening; 41 could not be retrieved, 291 were retrieved and coded, and 149 were excluded, leaving 146.
  • Data were coded in EPPI Reviewer and descriptively analysed in R; codes covered article information, study design and execution, AI applications in the student life cycle, specific applications and methods, presence of challenges/benefits, and whether AI was defined.

Limitations

  • Stated in the paper: the review is limited by its search strategy—only peer-reviewed articles in English or Spanish indexed in three databases were included, excluding other languages, conference proceedings, book chapters, grey literature, and non-indexed journals; no specific Spanish search string was used, narrowing the possibility of including Spanish papers not indexed with the chosen keywords.
  • Evident from the text: geographical distribution and author disciplinary affiliation were analysed using only the first author’s country and affiliation, so the results may not represent all co-authors.
  • Evident from the text: most included intelligent tutoring system studies were implemented and validated over short-term periods (a course or a semester), and no longitudinal studies were identified except Jackson and Cossitt (2015), limiting conclusions about long-term effectiveness.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s41239-019-0171-0

Cites

158 of 158 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

6 Valid · 3 Not valid · 59 Not checked · 80 Own results and interpretations

55 more claims

Create a free account to see all 148 claims and to verify the 11 references still open.

See all claims — free account
  • It may state two separate claims as one. 51
  • Probably not a substantial claim. 49
  • Borderline: one of the tests for a substantial claim is unclear. 2

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

57 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Show all 158 references

Research works printed but not linked: 3

  1. Alfarsi, G. M. S., Omar, K. A. M., & Alsinani, M. J. (2017). A rule-based system for advising undergraduate students. Journal of Theoretical and Applied Information Technology, 95(11) Retrieved from http://www.jatit.org .
  2. Baker, T., & Smith, L. (2019). Educ-AI-tion rebooted? Exploring the future of artificial intelligence in schools and colleges. Retrieved from Nesta Foundation website: https://media.nesta.org.uk/documents/Future_of_AI_and_education_v5_WEB.pdf
  3. Contact North. (2018). Ten facts about artificial intelligence in teaching and learning. Retrieved from https://teachonline.ca/sites/default/files/tools-trends/downloads/ten_facts_about_artificial_intelligence.pdf

Cited by

See all 7,242

Metadata

Authors
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F.
Affiliations
Carl von Ossietzky Universität Oldenburg (DE)
Venue
International Journal of Educational Technology in Higher Education
Published
2019-10-28
Volume
16
Issue
1
ISSN
2365-9440 (ISSN-L)
OpenAlex
W2981863007
Cited by
7,242
References
162
Citations by year
2026: 2999 · 2025: 2145 · 2024: 1131 · 2023: 525 · 2022: 239 · 2021: 139 · 2020: 62 · 2019: 1
FWCI
19.23
Type
journal article
Language
en
Open access
Licence
Locations
  • International Journal of Educational Technology in Higher Education (publishedVersion, cc-by)
  • UniSA Research Outputs Repository (University of South Australia) (submittedVersion, other-oa)
  • Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • UCL Discovery (University College London) (submittedVersion)
  • Repositori ObertUDL (University of Lleida) (submittedVersion, cc-by)
Topics
  • Social Sciences › Education › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
Keywords
artificial intelligence in education, higher education, academic support, assessment and evaluation, adaptive learning systems, personalization, intelligent tutoring systems, ethical challenges of AI, conceptions of teaching, research gaps

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Systematic review of research on artificial intelligence applications in higher education – where are the educators? https://app.cochat.ai/papers/10.1186/s41239-019-0171-0
The paper
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019). Systematic review of research on artificial intelligence applications in higher education – where are the educators? International Journal of Educational Technology in Higher Education. https://doi.org/10.1186/s41239-019-0171-0

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.