Recent Meta-reviews and Meta–analyses of AIED Systems
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
It is pleasing to see that the 25th Anniversary of IJAIED also brings a certain maturity to the field.In the last 5 years there have been a number useful and positive analyses of the effectiveness of AIED systems designed to tutor one-to-one.For example, VanLehn (2011) found that the effectiveness of the Bstep-based^intelligent tutoring systems he surveyed were nearly as effective as average human tutors, and that neither of these were as effective as expert human tutors, but certainly both better than providing the domain content without tutoring.BStep-based^means that the tutor evaluates and reacts to each step that the learner makes in a problem-solving session as opposed to evaluating only the final answer given by the learner.In a meta-analysis, Ma et al. (2014) found similar positive results for step-based ITSs both when compared to a no tutoring condition and when compared to large group human teacher led-instruction, but no differences when compared to small group human tutoring or to one-to-one tutoring.The same authors analysed systems for teaching programming and also found a Ba significant advantage of ITSs over teacherled classroom instruction and non-ITS computer-based instruction^ (Nesbit et al. 2014).Likewise Kulik andFletcher (2015) found similar sized improvements but distinguished between studies that used standardised tests and those where the tests were more specifically tuned to the system providing tuition.Smaller effect sizes were found by Steenbergen-Hu and Cooper (2013) in their meta-analysis of pupils using ITSs in a School setting.They also noted that lower-achievers seemed to do worse with ITSs than did the broad spectrum of school pupils.In a parallel study of university students, Steenbergen-Hu and Cooper (2014) found more positive effects for ITSs as compared to conventional instruction.Finally in a large-scale study in the USA of the Cognitive Tutors, (Pane et al. 2014) found only limited evidence of the relative effectiveness of these tutors over conventional teaching, though we note that how the tutors were actually used in the classrooms
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
VanLEHN, K. (2011) · Educational Psychologist · doi:10.1080/00461520.2011.611369 · Cited by 2,323
Effectiveness of Intelligent Tutoring Systems: A Meta-Analytic Review
Kulik, J. A., & Fletcher, J. D. (2016) · Review of Educational Research · doi:10.3102/0034654315581420 · Cited by 930
Intelligent tutoring systems and learning outcomes: A meta-analysis
Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q. (2014) · Journal of Educational Psychology · doi:10.1037/a0037123 · Cited by 840 · cited 2×
“For example, VanLehn (2011) found that the effectiveness of the Bstep-based^ intelligent tutoring systems he surveyed were nearly as effective as average human tutors, and that neither of these were as effective as expert human tutors, but certainly both better than providing the domain content without tutoring.”
Steenbergen-Hu, S., & Cooper, H. (2014) · Journal of Educational Psychology · doi:10.1037/a0034752 · Cited by 355 · cited 1×
“In a parallel study of university students, Steenbergen-Hu and Cooper (2014) found more positive effects for ITSs as compared to conventional instruction.”
Effectiveness of Cognitive Tutor Algebra I at Scale
Pane, J. F., Griffin, B. A., McCaffrey, D. F., & Karam, R. (2014) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373713507480 · Cited by 317 · cited 3×
“Finally in a large-scale study in the USA of the Cognitive Tutors, (Pane et al. 2014) found only limited evidence of the relative effectiveness of these tutors over conventional teaching, though we note that how the tutors were actually used in the classrooms Int J Artif Intell Educ (2016)…”
Influential
Cited by
Explainable Artificial Intelligence in education
Khosravi, H., Shum, S. B., Chen, G., Conati, C., Tsai, Y.-S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gašević, D. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100074 · Cited by 863
Empowering Educators to Be AI-Ready
Luckin, R., Cukurova, M., Kent, C., & du Boulay, B. (2022) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2022.100076 · Cited by 422
“These results for AIED are particularly significant, as more general studies examining the positive impact of educational interventions are notoriously hard to reach statistical significance (see also, Du Boulay, 2016).”
BackgroundInfluentialCukurova, M. (2025) · British Journal of Educational Technology · doi:10.1111/bjet.13514 · Cited by 302
Bulathwela, S., Pérez-Ortiz, M., Holloway, C., Cukurova, M., & Shawe-Taylor, J. (2024) · Sustainability · doi:10.3390/su16020781 · Cited by 269
Artificial Intelligence as an Effective Classroom Assistant
du Boulay, B. (2016) · IEEE Intelligent Systems · doi:10.1109/mis.2016.93 · Cited by 213
Metadata
- Authors
- du Boulay, B.
- Venue
- International Journal of Artificial Intelligence in Education
- Published
- 2015-08-06
- Volume
- 26
- Issue
- 1
- Pages
- 536-537
- ISSN
- 1560-4292 (ISSN-L)
- 1560-4306
- OpenAlex
- W2253091614
- Cited by
- 45
- References
- 5
- FWCI
- 1.25
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Recent Meta-reviews and Meta–analyses of AIED Systems. https://app.cochat.ai/papers/10.1007/s40593-015-0060-1
- The paper
- du Boulay, B. (2016). Recent Meta-reviews and Meta–analyses of AIED Systems. International Journal of Artificial Intelligence in Education. https://doi.org/10.1007/s40593-015-0060-1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.