Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 147 of the 147 references it cites were found in a registry or an index; 46 of the 95 sources its claims cite were checked for support.

A meta systematic review of artificial intelligence in higher education: a call for increased ethics, collaboration, and rigour

Open access
Bond, Melissa, Khosravi, Hassan, De Laat, Maarten, Bergdahl, Nina, Negrea, Violeta, Oxley, Emily, Pham, Phuong, Chong, Sin Wang, Siemens, George · 2024 · International Journal of Educational Technology in Higher Education

Abstract

Although the field of Artificial Intelligence in Education (AIEd) has a substantial history as a research domain, never before has the rapid evolution of AI applications in education sparked such prominent public discourse. Given the already rapidly growing AIEd literature base in higher education, now is the time to ensure that the field has a solid research and conceptual grounding. This review of reviews is the first comprehensive meta review to explore the scope and nature of AIEd in higher education (AIHEd) research, by synthesising secondary research (e.g., systematic reviews), indexed in the Web of Science, Scopus, ERIC, EBSCOHost, IEEE Xplore, ScienceDirect and ACM Digital Library, or captured through snowballing in OpenAlex, ResearchGate and Google Scholar. Reviews were included if they synthesised applications of AI solely in formal higher or continuing education, were published in English between 2018 and July 2023, were journal articles or full conference papers, and if they had a method section 66 publications were included for data extraction and synthesis in EPPI Reviewer, which were predominantly systematic reviews (66.7%), published by authors from North America (27.3%), conducted in teams (89.4%) in mostly domestic-only collaborations (71.2%). Findings show that these reviews mostly focused on AIHEd generally (47.0%) or Profiling and Prediction (28.8%) as thematic foci, however key findings indicated a predominance of the use of Adaptive Systems and Personalisation in higher education. Research gaps identified suggest a need for greater ethical, methodological, and contextual considerations within future research, alongside interdisciplinary approaches to AIHEd application. Suggestions are provided to guide future primary and secondary research.

Overview

CoChat Summary

TL;DR

This tertiary review synthesises 66 secondary studies on artificial intelligence in higher education (AIHEd) published 2018–July 2023, mapping review types, authorship, quality, applications, benefits, challenges, and gaps. It finds adaptive systems and personalisation dominate key findings, while ethics, collaboration, and methodological rigour remain major gaps.

Key findings

  • Of 66 included evidence syntheses, 66.7% were systematic literature reviews; 89.4% were team-authored, but 71.2% were domestic-only collaborations, and North America accounted for 27.3% of authorship.
  • Review focus was mostly General AIEd (47.0%) or Profiling and Prediction (28.8%); key findings most often discussed Adaptive Systems and Personalisation (54.5%), then Profiling and Prediction (48.5%), Assessment and Evaluation (39.4%), and Intelligent Tutoring Systems (21.2%).
  • Quality averaged 6.57/10; 45.5% did not undertake any quality assessment, 31.8% searched only one or two databases, 24.2% provided their exact data extraction coding scheme, and 34.8% did not reflect on review limitations; 65% were critically low to medium quality.
  • Top research gaps were ethics (40.9%), methodological rigour (36.4%), wider stakeholders (21.2%), disciplines (16.7%), and countries (15.2%); the top challenge was lack of ethical consideration (29.0%).

Methods

  • Tertiary review/review of reviews guided by PRISMA; searched Web of Science, Scopus, ERIC, EBSCOHost, IEEE Xplore, ScienceDirect, ACM Digital Library, OpenAlex, and Google Scholar (first 500 records), plus snowballing/manual additions; initial search 13 October 2022, updated to 18 July 2023.
  • Included secondary research with a method section on AI applications in formal higher or continuing education, published in English January 2018–18 July 2023 as journal articles or full conference papers; from 5,609 records and 5,160 title/abstract screened, 66 reviews solely focused on higher/continuing education were synthesised.
  • Data extracted in EPPI Reviewer; quality assessed with 10 criteria adapted from DARE, AMSTAR 2, and Buntins et al.; narrative synthesis and interactive evidence/gap maps.

Limitations

  • Stated: the protocol was not pre-registered in an official repository such as PROSPERO; only the first 500 Google Scholar records were considered versus the 1000 recommended; additional journals and non-English literature could have been searched to reduce language bias; the quality assessment tool is imperfect and yes/no/partly distances cannot be assumed equal.
  • Stated/design: generative AI and ChatGPT reviews were excluded; 32 reviews that included higher education but combined it with other levels were excluded to keep results higher-education-specific.
  • Evident from the text: English-language and 2018–July 2023 restrictions, plus reliance on available databases and snowballing, may limit coverage; the authors note future work will explore the full 307 AIEd evidence syntheses across educational levels.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s41239-023-00436-z

Cites

147 of 147 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

36 Valid · 0 Not valid · 28 Not checked · 127 Own results and interpretations

100 more claims · 6 with their cited sources checked

Create a free account to see all 191 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 70
  • Probably not a substantial claim. 25

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

8 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

5 of 5

Citations

Cites

Show all 135 references

Research works printed but not linked: 6

  1. *Algabri, H. K., Kharade, K. G., & Kamat, R. K. (2021). Promise, threats, and personalization in higher education with artificial intelligence. Webology, 18(6), 2129–2139.
  2. Centre for Reviews and Dissemination (UK). (1995). Database of Abstracts of Reviews of Effects (DARE): Quality-assessed Reviews. https://www.ncbi.nlm.nih.gov/books/NBK285222/. Accessed 4 January 2023.
  3. Chen, X., Zou, D., Xie, H., Cheng, G., & Liu, C. (2022). Two decades of artificial intelligence in education: Contributors, collaborations, research topics, challenges, and future directions. Educational Technology and Society, 25(1), 28–47. https://doi.org/10.2307/48647028 · doi:10.2307/48647028
  4. Chu, H.-C., Hwang, G.-H., Tu, Y.-F., & Yang, K.-H. (2022). Roles and research trends of artificial intelligence in higher education: A systematic review of the top 50 most-cited articles. Australasian Journal of Educational Technology, 38(3), 22–42. https://doi.org/10.14742/ajet.7526 · doi:10.14742/ajet.7526
  5. Mahmood, A., Sarwar, Q., & Gordon, C. (2022). A Systematic Review on Artificial Intelligence in Education (AIE) with a focus on Ethics and Ethical Constraints. Pakistan Journal of Multidisciplinary Research, 3(1). https://pjmr.org/pjmr/article/view/245
  6. Zawacki-Richter, O. (2023). Umbrella Review in ODDE. Herbsttagung der Sektion Medienpädagogik (DGfE), 22 September 2023.

Cited by

See all 852

Metadata

Authors
Bond, M., Khosravi, H., De Laat, M., Bergdahl, N., Negrea, V., Oxley, E., Pham, P., Chong, S. W., & Siemens, G.
Affiliations
  • University College London (GB)
  • The University of Queensland (AU)
  • Stockholm University (SE)
  • University of South Australia (AU)
  • University of St Andrews (GB)
Corresponding author
Bond, Melissa
Venue
International Journal of Educational Technology in Higher Education
Published
2024-01-19
Volume
21
Issue
1
ISSN
2365-9440 (ISSN-L)
OpenAlex
W4391006361
Cited by
852
References
135
Citations by year
2026: 474 · 2025: 279 · 2024: 99
FWCI
48.93
Type
journal article
Language
en
Open access
Licence
Locations
  • International Journal of Educational Technology in Higher Education (publishedVersion, cc-by)
  • Hogskolan Ihalmstad (Halmstad University) (submittedVersion, other-oa)
  • ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (submittedVersion, other-oa)
  • UCL Discovery (University College London) (submittedVersion, cc-by)
  • St Andrews Research Repository (St Andrews Research Repository) (submittedVersion, cc-by)
  • UniSA Research Outputs Repository (University of South Australia) (submittedVersion, cc-by)
  • Adelaide Research & Scholarship (AR&S) (University of Adelaide) (submittedVersion, cc-by)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
University of South Australia
Topics
  • Social Sciences › Education › Artificial Intelligence in Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Physical Sciences › Computer Science › Computer Science Applications › Online Learning and Analytics
Keywords
artificial intelligence in higher education, AI in education research, research ethics, research collaboration, scientific rigor, adaptive learning systems, personalization, research gaps, interdisciplinary approach

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-16, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-16 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A meta systematic review of artificial intelligence in higher education: a call for increased ethics, collaboration, and rigour. https://app.cochat.ai/papers/10.1186/s41239-023-00436-z
The paper
Bond, M., Khosravi, H., De Laat, M., Bergdahl, N., Negrea, V., Oxley, E., Pham, P., Chong, S. W., & Siemens, G. (2024). A meta systematic review of artificial intelligence in higher education: a call for increased ethics, collaboration, and rigour. International Journal of Educational Technology in Higher Education. https://doi.org/10.1186/s41239-023-00436-z

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.