Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 102 of the 102 references it cites were found in a registry or an index.

Effectiveness of Intelligent Tutoring Systems: A Meta-Analytic Review

Kulik, James A., Fletcher, J. D. · 2016 · Review of Educational Research

Abstract

This review describes a meta-analysis of findings from 50 controlled evaluations of intelligent computer tutoring systems. The median effect of intelligent tutoring in the 50 evaluations was to raise test scores 0.66 standard deviations over conventional levels, or from the 50th to the 75th percentile. However, the amount of improvement found in an evaluation depended to a great extent on whether improvement was measured on locally developed or standardized tests, suggesting that alignment of test and instructional objectives is a critical determinant of evaluation results. The review also describes findings from two groups of evaluations that did not meet all of the selection requirements for the meta-analysis: six evaluations with nonconventional control groups and four with flawed implementations of intelligent tutoring systems. Intelligent tutoring effects in these evaluations were small, suggesting that evaluation results are also affected by the nature of control treatments and the adequacy of program implementations.

Overview

Integrity signals

Registry

Crossref · 10.3102/0034654315581420

Cites

102 of 102 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Verification

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Citations

Cites

Show all 100 references

Research works printed but not linked: 19

  1. Anania J. (1981). The effects of quality of instruction on the cognitive and affective learning of students (Doctoral dissertation). Available from ProQuest Dissertations & Theses database. (Order No. T-28171)
  2. *Atkinson R. K., 2007, An experimental evaluation of three computer-based reading comprehension tutors
  3. Beal C. R., 2010, Journal of Interactive Online Learning, 9, 64
  4. Beal C. R., 2007, Journal of Interactive Online Learning, 6, 43
  5. Biesinger K., 2008, Journal of Computers in Mathematics and Science Teaching, 27, 5
  6. Burke A. J. (1980). Students’ potential for learning contrasted under tutorial and group approaches to instruction. (Doctoral dissertation). Available from ProQuest Dissertations & Theses database. (Order No. T-28810)
  7. *Chang K.-E., 1999, Journal of Computers in Mathematics and Science Teaching, 18, 303
  8. Cohen J., 1988, Statistical power analysis for the social sciences
  9. Craig S. D., 2004, Journal of Educational Multimedia and Hypermedia, 13, 163
  10. Fletcher J. D., 1982, Psychology and society: Information technology in the 1980s, 166
  11. *Fletcher J. D., 2012, DARPA digital Tutor: Assessment data
  12. Hartley S. S. (1977). Meta-analysis of the effects of individually paced instruction in mathematics (Doctoral dissertation). Available from ProQuest Dissertations & Theses database. (Order No. 7729926)
  13. *Hategekimana C., 2008, Proceedings of Society for Information Technology & Teacher Education International Conference 2008, 414
  14. *Jeremic Z., 2009, Educational Technology & Society, 12, 111
  15. Koedinger K. R., 2002, Proceedings of PMENA XXII (North American Chapter of the International Group for the Psychology of Mathematics Education, 1, 21
  16. *Le N. T., Menzel W., Pinkwart N. (2009). Evaluation of a constraint-based homework assistance system for logic programming. In Leung H., Li R., Lau R., Li Q. (Eds.), Proceedings of the 6th International Conference on Web-based Learning, Edinburgh, UK, 2007 (pp. 367–379). Berlin, Germany: Springer. Retrieved from http://www.icce2009.ied.edu.hk/pdf/C1/proceedings051-058.pdf
  17. *Ritter S., 2007, Supporting learning flow through integrative technologies, 13
  18. *Stankov S., 2004, Eighth IEEE International Conference on Intelligent Engineering Systems, 545
  19. *Steuck K., 1997, Paper presented at the annual meeting of the American Educational Research Association, Chicago, IL

Cited by

See all 930

Metadata

Authors
Kulik, J. A., & Fletcher, J. D.
Affiliations
  • University of Michigan (US)
  • Institute For Defense Analyses (US)
Venue
Review of Educational Research
Published
2016-03
Volume
86
Issue
1
Pages
42-78
ISSN
  • 0034-6543 (ISSN-L)
  • 1935-1046
OpenAlex
W1985824294
Cited by
930
References
100
Citations by year
2026: 259 · 2025: 170 · 2024: 119 · 2023: 45 · 2022: 62 · 2021: 74 · 2020: 56 · 2019: 46 · 2018: 59 · 2017: 24 · 2016: 13
FWCI
31.68
Type
journal article
Language
en
Open access
closed
Licence
journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations
Review of Educational Research (publishedVersion)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
  • Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
  • Social Sciences › Education › Artificial Intelligence in Education
Keywords
intelligent tutoring systems, computer-assisted instruction, educational effectiveness, test score improvement, control group design, program implementation quality, educational technology evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Effectiveness of Intelligent Tutoring Systems: A Meta-Analytic Review. https://app.cochat.ai/papers/10.3102/0034654315581420
The paper
Kulik, J. A., & Fletcher, J. D. (2016). Effectiveness of Intelligent Tutoring Systems: A Meta-Analytic Review. Review of Educational Research. https://doi.org/10.3102/0034654315581420

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.