The DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
A meta-analysis of the effectiveness of intelligent tutoring systems on college students’ academic learning.
Open accessAbstract
© 2013 American Psychological Association.This meta-analysis synthesizes research on the effectiveness of intelligent tutoring systems (ITS) for college students. Thirty-five reports were found containing 39 studies assessing the effectiveness of 22 types of ITS in higher education settings. Most frequently studied were AutoTutor, Assessment and Learning in Knowledge Spaces, eXtended Tutor-Expert System, and Web Interface for Statistics Education. Major findings include (a) Overall, ITS had a moderate positive effect on college students' academic learning (g = .32 to g = .37); (b) ITS were less effective than human tutoring, but they outperformed all other instruction methods and learning activities, including traditional classroom instruction, reading printed text or computerized materials, computer-assisted instruction, laboratory or homework assignments, and no-treatment control; (c) ITS's effectiveness did not significantly differ by different ITS, subject domain, or the manner or degree of their involvement in instruction and learning; and (d) effectiveness in earlier studies appeared to be significantly greater than that in more recent studies. In addition, there is some evidence suggesting the importance of teachers and pedagogy in ITS-assisted learning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Another meta-analysis calculated a point estimate for intelligent tutoring systems in college-level business education of ḡ = 0.16 (Steenbergen-Hu & Cooper, 2014).”
From Synthesizing Results From Empirical Research on Computer-Based Scaffolding in STEM Education (2017) · cites this paper as Steenbergen-Hu & Cooper, 2014
In the citing paper
“and that ITS were less effective that human tutoring”
From Systematic review of research on artificial intelligence applications in higher education – where are the educators? (2019) · cites this paper as Steenbergen-Hu and Cooper (2014)
In the citing paper
“Steenbergen-Hu and Cooper (2014) found that ITS had a moderate effect of students' learning”
From Systematic review of research on artificial intelligence applications in higher education – where are the educators? (2019) · cites this paper as Steenbergen-Hu and Cooper (2014)
In the citing paper
“but ITS outperformed all other instruction methods (such as traditional classroom instruction, reading printed or digital text, or homework assignments).”
From Systematic review of research on artificial intelligence applications in higher education – where are the educators? (2019) · cites this paper as Steenbergen-Hu and Cooper (2014)
In the citing paper
“most of the strategies and functions that our participants desired to improve the usability issues of ChatGPT, such as hints, feedback, prompt suggestions, a learning map and navigation system, and AI-Helper agents, resemble the traits that ITSs have long had and are already proven to be effective.”
From The Promise and Peril of ChatGPT in Higher Education: Opportunities, Challenges, and Design Implications (2024) · cites this paper as [80]
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099 · cited 1×
“We further conducted Duval and Tweedie’s (2000) trim and fill procedures and found that the adjusted average effect sizes remained the same as the observed ones, which were g .”
The Role of Goal Orientation in Self-Regulated Learning
Pintrich, P. R. (2000) · Handbook of Self-Regulation · doi:10.1016/b978-012109890-2/50043-3 · Cited by 5,173 · cited 1×
“The last two questions are particularly important because research has found that high and low selfregulating learners usually exhibit different learning characteristics (Pintrich, 2000, 2004).”
Meta-analysis in social research
(1982) · Evaluation and Program Planning · doi:10.1016/0149-7189(82)90011-8 · Cited by 3,853 · cited 1×
“Graesser et al.’s (2011) review of ITS posed a number of questions that need answering.”
A Conceptual Framework for Assessing Motivation and Self-Regulated Learning in College Students
Pintrich, P. R. (2004) · Educational Psychology Review · doi:10.1007/s10648-004-0006-x · Cited by 3,269 · cited 1×
“The last two questions are particularly important because research has found that high and low selfregulating learners usually exhibit different learning characteristics (Pintrich, 2000, 2004).”
The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring
BLOOM, B. S. (1984) · Educational Researcher · doi:10.3102/0013189x013006004 · Cited by 2,734 · cited 7×
“Bloom (1984) reported that expert human tutors can help students achieve learning gains as large as two sigmas. Although not as effective as what Bloom (1984) found, a recent meta-review by VanLehn (2011) found that human tutoring had a positive impact of d .79 on students’ learning. Many ITS are designed to follow th…”
InfluentialVanLEHN, K. (2011) · Educational Psychologist · doi:10.1080/00461520.2011.611369 · Cited by 2,323 · cited 4×
“VanLehn (2011) reviewed randomized experiments that compared the effectiveness of human tutoring, computer tutoring, and no tutoring.”
InfluentialCognitive Tutors: Lessons Learned
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995) · Journal of the Learning Sciences · doi:10.1207/s15327809jls0402_2 · Cited by 1,982 · cited 1×
“For example, Cognitive Tutors are built on a cognitive theory called adaptive control of thought (ACT, or ACT-R in its updated form; Anderson et al., 1995).”
Sample criteria for testing outlying observations
Grubbs, F. E. (1950) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729885 · Cited by 1,631 · cited 4×
“Grubbs (1950) tests detected no effect-size outliers.”
InfluentialBuilding Intelligent Interactive Tutors : student-centered strategies for revolutionizing e-learning
Woolf, B. P. (2009) · HAL · Cited by 1,376
Intelligent Tutoring Goes To School in the Big City
Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (2018) · doi:10.1184/r1/6470153 · Cited by 1,105
Cited by
Zawacki-Richter, O., Marín, V. I., Bond, M., & Gouverneur, F. (2019) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-019-0171-0 · Cited by 7,242
What Is the Impact of ChatGPT on Education? A Rapid Review of the Literature
Lo, C. K. (2023) · Education Sciences · doi:10.3390/educsci13040410 · Cited by 2,024
“• RQ3: What are the potential issues associated with ChatGPT, and how can they be addressed?”
BackgroundVision, challenges, roles and research issues of Artificial Intelligence in Education
Hwang, G.-J., Xie, H., Wah, B. W., & Gašević, D. (2020) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2020.100001 · Cited by 1,435
Variables associated with achievement in higher education: A systematic review of meta-analyses.
Schneider, M., & Preckel, F. (2017) · Psychological Bulletin · doi:10.1037/bul0000098 · Cited by 1,083
Artificial Intelligence trends in education: a narrative overview
Chassignol, M., Khoroshavin, A., Klimova, A., & Bilyatdinova, A. (2018) · Procedia Computer Science · doi:10.1016/j.procs.2018.08.233 · Cited by 956
Metadata
- Authors
- Steenbergen-Hu, S., & Cooper, H.
- Venue
- Journal of Educational Psychology
- Published
- 2014-05
- Volume
- 106
- Issue
- 2
- Pages
- 331-347
- ISSN
- 0022-0663 (ISSN-L)
- 1939-2176
- OpenAlex
- W2316441559
- Cited by
- 355
- References
- 68
- Citations by year
- 2026: 48 · 2025: 55 · 2024: 50 · 2023: 28 · 2022: 24 · 2021: 31 · 2020: 31 · 2019: 22 · 2018: 22 · 2017: 13 · 2016: 15 · 2015: 12 · 2014: 4
- FWCI
- 1.50
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- cc-by-nc-nd
- Locations
- Journal of Educational Psychology (publishedVersion)
- DukeSpace (Duke University) (submittedVersion, cc-by-nc-nd)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
- Social Sciences › Education › Artificial Intelligence in Education
- Keywords
- intelligent tutoring systems, university students, academic learning, higher education, computer-assisted instruction, AutoTutor, ALKIS, effect size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.