A meta-analysis of the effectiveness of intelligent tutoring systems on college students’ academic learning.
Open accessThe DOI matches its Crossref record; 68 of the 68 references it cites were found in a registry or an index.
Abstract
© 2013 American Psychological Association.This meta-analysis synthesizes research on the effectiveness of intelligent tutoring systems (ITS) for college students. Thirty-five reports were found containing 39 studies assessing the effectiveness of 22 types of ITS in higher education settings. Most frequently studied were AutoTutor, Assessment and Learning in Knowledge Spaces, eXtended Tutor-Expert System, and Web Interface for Statistics Education. Major findings include (a) Overall, ITS had a moderate positive effect on college students' academic learning (g = .32 to g = .37); (b) ITS were less effective than human tutoring, but they outperformed all other instruction methods and learning activities, including traditional classroom instruction, reading printed text or computerized materials, computer-assisted instruction, laboratory or homework assignments, and no-treatment control; (c) ITS's effectiveness did not significantly differ by different ITS, subject domain, or the manner or degree of their involvement in instruction and learning; and (d) effectiveness in earlier studies appeared to be significantly greater than that in more recent studies. In addition, there is some evidence suggesting the importance of teachers and pedagogy in ITS-assisted learning.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Another meta-analysis calculated a point estimate for intelligent tutoring systems in college-level business education of ḡ = 0.16 (Steenbergen-Hu & Cooper, 2014).”
From Synthesizing Results From Empirical Research on Computer-Based Scaffolding in STEM Education (2017) · cites this paper as Steenbergen-Hu & Cooper, 2014
In the citing paper
“and that ITS were less effective that human tutoring”
From Systematic review of research on artificial intelligence applications in higher education – where are the educators? (2019) · cites this paper as Steenbergen-Hu and Cooper (2014)
In the citing paper
“Steenbergen-Hu and Cooper (2014) found that ITS had a moderate effect of students' learning”
From Systematic review of research on artificial intelligence applications in higher education – where are the educators? (2019) · cites this paper as Steenbergen-Hu and Cooper (2014)
In the citing paper
“but ITS outperformed all other instruction methods (such as traditional classroom instruction, reading printed or digital text, or homework assignments).”
From Systematic review of research on artificial intelligence applications in higher education – where are the educators? (2019) · cites this paper as Steenbergen-Hu and Cooper (2014)
In the citing paper
“most of the strategies and functions that our participants desired to improve the usability issues of ChatGPT, such as hints, feedback, prompt suggestions, a learning map and navigation system, and AI-Helper agents, resemble the traits that ITSs have long had and are already proven to be effective.”
From The Promise and Peril of ChatGPT in Higher Education: Opportunities, Challenges, and Design Implications (2024) · cites this paper as [80]
3 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Duval, S., & Tweedie, R. (2000) · Biometrics · doi:10.1111/j.0006-341x.2000.00455.x · Cited by 14,099 · cited 1×
“We further conducted Duval and Tweedie’s (2000) trim and fill procedures and found that the adjusted average effect sizes remained the same as the observed ones, which were g .”
The Role of Goal Orientation in Self-Regulated Learning
Pintrich, P. R. (2000) · Handbook of Self-Regulation · doi:10.1016/b978-012109890-2/50043-3 · Cited by 5,173 · cited 1×
“The last two questions are particularly important because research has found that high and low selfregulating learners usually exhibit different learning characteristics (Pintrich, 2000, 2004).”
Meta-analysis in social research
(1982) · Evaluation and Program Planning · doi:10.1016/0149-7189(82)90011-8 · Cited by 3,853 · cited 1×
“Graesser et al.’s (2011) review of ITS posed a number of questions that need answering.”
A Conceptual Framework for Assessing Motivation and Self-Regulated Learning in College Students
Pintrich, P. R. (2004) · Educational Psychology Review · doi:10.1007/s10648-004-0006-x · Cited by 3,269 · cited 1×
“The last two questions are particularly important because research has found that high and low selfregulating learners usually exhibit different learning characteristics (Pintrich, 2000, 2004).”
The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring
BLOOM, B. S. (1984) · Educational Researcher · doi:10.3102/0013189x013006004 · Cited by 2,734 · cited 7×
“Bloom (1984) reported that expert human tutors can help students achieve learning gains as large as two sigmas. Although not as effective as what Bloom (1984) found, a recent meta-review by VanLehn (2011) found that human tutoring had a positive impact of d .79 on students’ learning. Many ITS are designed to follow th…”
InfluentialVanLEHN, K. (2011) · Educational Psychologist · doi:10.1080/00461520.2011.611369 · Cited by 2,323 · cited 4×
“VanLehn (2011) reviewed randomized experiments that compared the effectiveness of human tutoring, computer tutoring, and no tutoring.”
InfluentialCognitive Tutors: Lessons Learned
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995) · Journal of the Learning Sciences · doi:10.1207/s15327809jls0402_2 · Cited by 1,982 · cited 1×
“For example, Cognitive Tutors are built on a cognitive theory called adaptive control of thought (ACT, or ACT-R in its updated form; Anderson et al., 1995).”
Sample criteria for testing outlying observations
Grubbs, F. E. (1950) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729885 · Cited by 1,631 · cited 4×
“Grubbs (1950) tests detected no effect-size outliers.”
InfluentialBuilding Intelligent Interactive Tutors : student-centered strategies for revolutionizing e-learning
Woolf, B. P. (2009) · HAL · Cited by 1,376
Intelligent Tutoring Goes To School in the Big City
Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (2018) · doi:10.1184/r1/6470153 · Cited by 1,105
Cited by
Items 101–200 of 355
Henkel, O., Horne-Robinson, H., Kozhakhmetova, N., & Lee, A. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-64315-6_34 · Cited by 30
Feasibility of adaptive teaching with technology: Which implementation conditions matter?
Sibley, L., Lachner, A., Plicht, C., Fabian, A., Backfisch, I., Scheiter, K., & Bohl, T. (2024) · Computers & Education · doi:10.1016/j.compedu.2024.105108 · Cited by 45
Learning with virtual humans: Introduction to the special issue
Schroeder, N. L., & Craig, S. D. (2021) · Journal of Research on Technology in Education · doi:10.1080/15391523.2020.1863114 · Cited by 34
“…lines of work around interactive virtual humans may increase with time given the effectiveness of intelligent tutoring systems (see Kulik & Fletcher, 2016; Ma et al., 2014; Steenbergen-Hu & Cooper, 2014) and other conversational agent systems (Graesser & McNamara, 2010; Graesser et al., 2017).”
BackgroundImpact of ChatGPT on Academic Performance among Bangladeshi Undergraduate Students
Tanvir, K., Islam, M. S., Sezan, S. B. K., Sanad, Z. A., & Ataur, A.-J. I. (2023) · International Journal of Research In Science & Engineering · doi:10.55529/ijrise.35.18.28 · Cited by 26
Radkowitsch, A., Sailer, M., Schmidmaier, R., Fischer, M. R., & Fischer, F. (2021) · Learning and Instruction · doi:10.1016/j.learninstruc.2021.101487 · Cited by 41
“To date, agents have mainly been used to assess competences (e.g., OECD, 2017) or as pedagogical agents to support specific competences of individuals or groups of learners (e.g., intelligent tutoring systems, Steenbergen-Hu & Cooper, 2014).”
AI chatbots: A disguised enemy for academic integrity?
Niloy, A. C., Hafiz, R., Hossain, B. M., Gulmeher, F., Sultana, N., Islam, K. F., Bushra, F., Islam, S., Hoque, S. I., Rahman, M., et al. (2024) · International Journal of Educational Research Open · doi:10.1016/j.ijedro.2024.100396 · Cited by 26
Intelligent Learning Environments: Design, Usage and Analytics for Future Schools
Mavrikis, M., & Holmes, W. (2019) · Perspectives on rethinking and reforming education · doi:10.1007/978-981-13-9439-3_4 · Cited by 26
Recommender in AI-enhanced Learning: An Assessment from the Perspective of Instructional Design
Kerres, M., & Buntins, K. (2020) · Open Education Studies · doi:10.1515/edu-2020-0119 · Cited by 34
“…(Goodyear, 1991; Mandl & Lesgold, 1988; Sleeman & Brown, 1982), practical implementations of this strand of research have hardly become visible; meta-analyses show rather small effects of these solutions (Kulik & Fletcher, 2016; Ma, Adesope, Nesbit, & Liu, 2014; Steenbergen-Hu & Cooper, 2014).”
Use of ChatGPT for education by engineering students in developing countries: a mixed-methods study
Elkefi, S., Tounsi, A., & Kefi, M. A. (2024) · Behaviour and Information Technology · doi:10.1080/0144929x.2024.2354428 · Cited by 34
Using artificial intelligence in digital video production: A systematic review study
Orak, C., & Turan, Z. (2024) · Journal of Educational Technology and Online Learning · doi:10.31681/jetol.1459434 · Cited by 44
“It can be used in academic education to increase success in disciplines where abstract concepts are predominant and students have difficulty in understanding (Steenbergen-Hu & Cooper, 2014).”
Adaptive teaching with technology enhances lasting learning
Sibley, L., Fabian, A., Plicht, C., Pagano, L., Ehrhardt, N., Wellert, L., Bohl, T., & Lachner, A. (2025) · Learning and Instruction · doi:10.1016/j.learninstruc.2025.102141 · Cited by 47
Ninghardjanti, P., Umam, M. C., Subarno, A., Winarno, W., Langgi, N. R., & Widodo, J. (2025) · Frontiers in Education · doi:10.3389/feduc.2025.1719625 · Cited by 53
Tenório, K., Dermeval, D., Monteiro, M. D. S., Peixoto, A., & da Silva, A. P. (2021) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00274-y · Cited by 34
Gupta, S., Ojeh, N., Sa, B., Majumder, M. A. A., Singh, K., & Adams, O. P. (2020) · Advances in Medical Education and Practice · doi:10.2147/amep.s267834 · Cited by 23
The Effects of ALEKS on Mathematics Learning in K-12 and Higher Education: A Meta-Analysis
Sun, S., Else-Quest, N. M., Hodges, L. C., French, A. M., & Dowling, R. (2021) · Investigations in Mathematics Learning · doi:10.1080/19477503.2021.1926194 · Cited by 23
“Our meta-analysis aimed to build upon and extend the meta-analytic studies of Fang et al. (2019) and Steenbergen-Hu and Cooper (2014) in at least five important ways.”
BackgroundMethodResultInfluentialAnalyzing the Efficacy of ECoach in Supporting Gateway Course Success Through Tailored Support
Matz, R. L., Schulz, K. W., Hanley, E., Derry, H. A., Hayward, B., Koester, B. P., Hayward, C., & McKay, T. A. (2021) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3448139.3448160 · Cited by 25
“hierarchies of problem types, computer-scored assessments, and just-in-time remediation, and they can have efficacy close to that of human tutoring [54, 58].”
Backgrounddel Olmo‐Muñoz, J., González‐Calero, J. A., Diago, P. D., Arnau, D., & Arevalillo‐Herráez, M. (2022) · British Journal of Educational Technology · doi:10.1111/bjet.13228 · Cited by 32
Leite, W. L., Cetin‐Berber, D. D., Huggins‐Manley, A. C., Collier, Z. K., & Beal, C. R. (2019) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12360 · Cited by 23
Using an Artificial Intelligence Based Program to Enhance Primary Stage Pupils' EFL Listening Skills
Ghoneim, N. M. M., & Elghotmy, H. E. A. (2021) · المجلة التربوية لکلية التربية بسوهاج · doi:10.21608/edusohag.2021.140694 · Cited by 21
Bertoletti, A., Soncin, M., Cannistrà, M., & Agasisti, T. (2023) · PLoS ONE · doi:10.1371/journal.pone.0280494 · Cited by 32
“, [29]) found no differences in the effect of digital tools on student learning between different grade levels.”
BackgroundWang, T., Li, S., Huang, X., & Lajoie, S. P. (2023) · Educational Technology Research and Development · doi:10.1007/s11423-023-10222-3 · Cited by 22
The Challenge and Opportunity of Competency‐Based Counselor Education
Akos, P. T., WASIK, S. Z., McDonald, A., Solér, M. L., & Lys, D. B. (2019) · Counselor Education and Supervision · doi:10.1002/ceas.12134 · Cited by 24
Badges and XP: An observational study about learning
Paiva, R., Barbosa, A., Batista, E., Pimentel, D., & Bittencourt, I. I. (2015) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie.2015.7344074 · Cited by 23
“There are ITSs for algebra, basic mathematics, statistics, assessment and learning in knowledge spaces, physics and computer science [5].”
BackgroundVanLehn, K. A., Chung, G. K. W. K., Grover, S., Madni, A., & Wetzel, J. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-015-0093-5 · Cited by 20
“The grade point average for Dragoon students on their math class in the preceding year was 3.17 vs. 3.00 for the control students.”
BackgroundInfluentialDeveloping a Teacher Dashboard For Use with Intelligent Tutoring Systems.
Aleven, V. A. W. M. M., Xhakaj, F., Holstein, K., & McLaren, B. M. (2016) · Cited by 19
Goldberg, B., Amburn, C. R., Ragusa, C., & Chen, D.-W. (2017) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-017-0155-y · Cited by 20
“…(ITS) have seen success among multiple academic and military applications (Kulik and Fletcher 2016; VanLehn 2011), most triumph stories are seen in well-defined cognitive problem spaces (e.g., VanLehn 2011; Stottler et al. 2001; Steenbergen-Hu and Cooper 2014; Steenbergen-Hu and Cooper 2013).”
BackgroundAlqahtani, F., & Ramzan, N. (2019) · Sensors · doi:10.3390/s19030460 · Cited by 17
“There are confirmatory results that ITS does not have a negative impact on K-12 level [17] or college level learning [18], making it an effective tool for use in the classroom.”
MethodMathematics intelligent tutoring systems with handwritten input: a scoping review
Rodrigues, L., Pereira, F. D., Marinho, M., Macario, V., Bittencourt, I. I., Isotani, S., Dermeval, D., & Mello, R. F. (2023) · Education and Information Technologies · doi:10.1007/s10639-023-12245-y · Cited by 21
“From a practical perspective, empirical evidence reiterates the importance of teachers and pedagogy in exploring ITSs to assist math learning (Steenbergen-Hu & Cooper, 2014).”
InfluentialWhy technology-supported classrooms: An analysis of classroom behavior data from AIGC
Dai, Y. M., Yizhe, H., Zhang, Y., & Xu, X. (2023) · International Conference on Intelligent Education and Intelligent Research (IEIR) · doi:10.1109/ieir59294.2023.10391211 · Cited by 20
“E-learning platforms like Coursera and Grammarly leverage AIGC technology to assist students in identifying assignment errors, and teachers can also use AIGC to create unique assessments [29].”
Sequential detection of learning in cognitive diagnosis
Ye, S., Fellouris, G., Culpepper, S. A., & Douglas, J. (2016) · British Journal of Mathematical and Statistical Psychology · doi:10.1111/bmsp.12065 · Cited by 18
Nesbit, J. C., Liu, L., Liu, Q., & Adesope, O. (2015) · ASEE Annual Conference & Exposition · doi:10.18260/p.25090 · Cited by 18
What to do and what to avoid on the use of gamified intelligent tutor system for low-income students
Joaquim, S., Bittencourt, I. I., Silva, R. D. A., Espinheira, P. L., & Reis, M. (2021) · Education and Information Technologies · doi:10.1007/s10639-021-10728-4 · Cited by 17
Arroyo-Machado, W., Ma, J., Chen, T., Johnson, T. P., Islam, S., Michalegko, L., & Welch, E. (2025) · PLOS One · doi:10.1371/journal.pone.0330416 · Cited by 15
“AI is used to personalize learning through adaptive tutoring systems and identify and address student learning obstacles [21,22].”
The Role of Artificial Intelligence in Education: A Systematic Literature Review
Bakti, I. K., Zulkarnain, Yarun, A., Rusdi, Syaifudin, M., & Syafaq, H. (2023) · JURNAL IQRA · doi:10.25217/ji.v8i2.3194 · Cited by 48
“Conversely, the majority of other studies concentrate on various specific topics related to artificial intelligence and technology within the realm of education (Heffernan and Heffernan 2014; Ma et al. 2014; Steenbergen-Hu and Cooper 2014).”
BackgroundMultimedia Learning with Intelligent Tutoring Systems
Nye, B. D., Graesser, A. C., & Hu, X. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9781139547369.035 · Cited by 16
Chevalère, J., Cazenave, L., Berthon, M., Martinez, R., Mazenod, V., Borion, M., Pailler, D., Rocher, N., Cadet, R., Lenne, C., et al. (2021) · PLoS ONE · doi:10.1371/journal.pone.0259664 · Cited by 20
Borchers, C., Fleischer, H., Schanze, S., Scheiter, K., & Aleven, V. A. W. M. M. (2025) · Computers & Education · doi:10.1016/j.compedu.2025.105364 · Cited by 25
Betts, A. L., Thai, K.-P., & Gunderia, S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-77857-6_3 · Cited by 18
Personalisation is not Guaranteed: The Challenges of Using Generative AI for Personalised Learning
Laak, K.-J., Abdelghani, R., & Aru, J. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-65881-5_5 · Cited by 28
Akın, A. (2022) · Education and Information Technologies · doi:10.1007/s10639-022-10931-x · Cited by 33
“This result conforms with the results of Hillmayr et al. ( 2020 ) and the hypothesis of Steenbergen-Hu and Cooper ( 2014 ) that WBMI could be more useful for adults who have better self-control abilities, digital skills, previous mathematical knowledge, and motivational beliefs than younger…”
ResultZhang, F., Gou, J., Shen, K. N., Camarinha-Matos, L. M., & Wang, Z. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13717-z · Cited by 18
“The first is the substitutional perspective, where AI systems such as automated grading, intelligent tutoring, or virtual teachers are used to replace certain tasks traditionally performed by humans (Steenbergen-Hu & Cooper, 2014; Camarinha-Matos et al., 1997).”
Rodrigues, L., Guerino, G. C., Challco, G. C., da Silva, T. E. V., de Oliveira, L. C., da Penha, R. S., Melo, R. F., Vieira, T. M. D. A., Marinho, M., Macario, V., et al. (2023) · Anais do ... Simpósio Brasileiro de Informática na Educação/Anais do Simpósio Brasileiro de Informática na Educação · doi:10.5753/sbie.2023.235159 · Cited by 13
“The teachers-ITS relationship is pivotal in harnessing their full potential [Steenbergen-Hu and Cooper 2014].”
BackgroundThe Application of Artificial Intelligence in Education – The Current State and Trends
Milićević, V., Lazarova, L. K., & Jordović-Pavlović, M. I. (2024) · International Journal of Cognitive Research in Science Engineering and Education · doi:10.23947/2334-8496-2024-12-2-259-272 · Cited by 22
Xu, Q. (2024) · Discover Education · doi:10.1007/s44217-024-00330-4 · Cited by 22
“Steenbergen-Hu& Cooper defined ITS, as a key application of AI in education, are able to intelligently analyze students’ learning data to provide each student with personalized tutoring [4].”
Using large language models for grading in education: an applied test for physics
Mok, R., Akhtar, F., Clare, L., Li, C., Ida, J., Ross, L., & Campanelli, M. (2025) · Physics Education · doi:10.1088/1361-6552/adb92b · Cited by 37
“For instance, ‘intelligent tutoring systems’ that interact with students to learn and solve problems without a human instructor have shown promising performance [2, 3].”
Carpenter, D. M., Kaka, S. J., Tygret, J., & Cathcart, K. (2017) · Research in Higher Education · doi:10.1007/s11162-017-9456-0 · Cited by 12
Personalized Learning and Online Instruction
Svenningsen, L., Bottomley, S., & Pear, J. J. (2018) · Advances in educational technologies and instructional design book series · doi:10.4018/978-1-5225-3940-7.ch008 · Cited by 17
Adaptive Learning Goes to China
Feng, M., Cui, W., & Wang, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93846-2_17 · Cited by 14
The Feedback Principle in Multimedia Learning
Johnson, C. I., & Marraffino, M. D. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108894333.042 · Cited by 15
Rodrigues, L., Guerino, G. C., da Silva, T. E. V., Bianchini, L. D. P., Xavier, M., Vieira, T. M. D. A., Marinho, M., Macario, V., Dermeval, D., Bittencourt, I. I., et al. (2024) · Anais do ... Simpósio Brasileiro de Informática na Educação/Anais do Simpósio Brasileiro de Informática na Educação · doi:10.5753/sbie.2024.242017 · Cited by 11
“This aligns with previous research suggesting that ITS can significantly reduce teachers’ bureacratic burdens and allow for more focused instructional time [Steenbergen-Hu and Cooper 2014].”
Moving-Target Intelligent Tutoring System for Marksmanship Training
Zotov, V., & Kramkowski, E. W. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-022-00308-z · Cited by 16
“Nevertheless, the majority were designed to train knowledge-based subjects—such as mathematics, economics, or computer science (e.g., Steenbergen-Hu & Cooper 2014).”
Artificial Intelligence in Education: Origin, Development and Rise
Ye, R., Sun, F., & Li, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-89092-6_49 · Cited by 11
Timing Game-Based Practice in a Reading Comprehension Strategy Tutor
Jacovina, M. E., Jackson, G. T., Snow, E. L., & McNamara, D. S. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-39583-8_6 · Cited by 12
“Intelligent Tutoring Systems (ITSs) have produced positive outcomes for students across a number of domains [1].”
BackgroundAsriyadin, A., Yulianci, S., Kaniawati, I., & Liliawati, W. (2021) · AIP conference proceedings · doi:10.1063/5.0043350 · Cited by 15
Fink, M. C., Heitzmann, N., Siebeck, M., Fischer, F., & Fischer, M. R. (2021) · BMC Medical Education · doi:10.1186/s12909-021-02937-9 · Cited by 14
ICT competency and practicum of preservice teachers as digital natives: a mixed-method study
Wang, P., & Ko, J. (2022) · Asia Pacific Journal of Education · doi:10.1080/02188791.2022.2083581 · Cited by 18
“Although it is theoretically grounded, ITSs are controversial tools that can provoke discussion among PSTs as meta-analysis has shown contradictory findings towards the effectiveness of ITSs (e.g., Ma, Adesope, Nesbit, & Liu, 2014; Steenbergen-Hu & Cooper, 2014).”
Kerres, M., Buntins, K., Buchner, J., Drachsler, H., & Zawacki‐Richter, O. (2023) · Springer eBooks · doi:10.1007/978-3-658-40079-8_6 · Cited by 16
DESIGN AND NEED ANALYSIS OF COMPUTER DEVICES’ EXPERT SYSTEM USING FORWARD CHAINING METHOD
Jalinus, N. (2019) · International Journal of Geomate · doi:10.21660/2019.61.icee408 · Cited by 12
“Individuals/children at the formal operational stage are already able to think abstractly [18].”
BackgroundLiao, M., Zhu, K., & Wang, G. (2024) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1288503 · Cited by 21
Lampropoulos, G., López-Belmonte, J., Pozo-Sánchez, S., & Marín-Marín, J.-A. (2026) · Advances in Computational Intelligence and Robotics · doi:10.4018/979-8-2600-0101-1.ch001 · Cited by 9
The ChatGPT revolution: Reshaping education through artificial intelligence
Kaur, K., Shilpa, S., & Jindal, R. (2024) · AIP conference proceedings · doi:10.1063/5.0221430 · Cited by 12
Automated intelligent feedback in university statistics education
Tacoma, S. (2020) · Utrecht University Repository (Utrecht University) · doi:10.33540/395 · Cited by 18
Charting the uncertain future of AI and education: promises, challenges and opportunities
Looi, C. (2024) · Pedagogies An International Journal · doi:10.1080/1554480x.2024.2379776 · Cited by 15
“While ITS can yield better post-test learning outcomes compared to traditional classroom settings, they fall short of the effectiveness of skilled human teachers in one-to-one interactions (see for example, Steenbergen-Hu & Cooper, 2014).”
The Future of Intelligent Tutoring Systems for Writing
Banawan, M., Butterfuss, R., Taylor, K. S., Christhilf, K., Hsu, C., O’Loughlin, C., Allen, L. K., Roscoe, R. D., & McNamara, D. S. (2023) · Springer eBooks · doi:10.1007/978-3-031-36033-6_23 · Cited by 12
Use of Generative Artificial Intelligence tools in higher education environments
Seco, D., Groesser, S. N., & Pedrosa, A. M. (2025) · Multidisciplinary Journal for Education Social and Technological Sciences · doi:10.4995/muse.2025.23623 · Cited by 13
Yang, J. W., Choe, K., Lim, J., & Park, J. (2025) · Interactive Learning Environments · doi:10.1080/10494820.2025.2488984 · Cited by 10
“Steenbergen-Hu and Cooper (2014) also reported a medium effect size of 0.32 for 39 intelligent tutoring systems widely used in higher education.”
InfluentialWang, J., Tigelaar, D. E. H., Ye, T., & Admiraal, W. (2025) · Journal of Computer Assisted Learning · doi:10.1002/jcal.70168 · Cited by 20
Intelligent Feedback on Hypothesis Testing
Tacoma, S., Heeren, B. J., Jeuring, J. T., & Drijvers, P. H. M. (2020) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-020-00218-y · Cited by 11
A Qualitative Journey on Instructors' Perceptions of Artificial Intelligence in EFL Education
Dinçer, N., & Bal, S. (2024) · Advances in educational technologies and instructional design book series · doi:10.4018/979-8-3693-0872-1.ch005 · Cited by 12
Learning spaces: built, natural and digital considerations for learning and learners
Kraftl, P., McKenzie, M., Gulson, K. Ν., Accioly, I., Burke, C., Clarke, D. A. G., Bailón, R. O. F., Goodyear, P., Hartong, S., Howard, S. K., et al. (2022) · UNESCO MGIEP eBooks · doi:10.56383/kwaw2002 · Cited by 14
Salamon, R. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.01742 · Cited by 11
Influence of LLMs on Learning and Teaching in Higher EducationPreprint
Upadhyay, A., Farahmand, E., Muñoz, I., Khan, M. A., & Witte, N. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4716855 · Cited by 15
Enhancing academic writing in English language education through generative AI integration
Zamorano, C. (2025) · Research Studies in English Language Teaching and Learning · doi:10.62583/rseltl.v3i3.87 · Cited by 14
“Literature constantly emphasises the ability of AI to customise learning and optimise student performance using intelligent tutoring systems and recommendation algorithms (Nemorin et al., 2023; Steenbergen-Hu & Cooper, 2014).”
What Is the Impact of ChatGPT on Education?
NAGAR, L. (2024) · INTERANTIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT · doi:10.55041/ijsrem32175 · Cited by 8
Enhancing Academic Tutoring with AI – A Conceptual Framework
Łuczak, K., Greńczuk, A., Chomiak-Orsa, I., & Piwoni-Krzeszowska, E. (2024) · Procedia Computer Science · doi:10.1016/j.procs.2024.09.709 · Cited by 9
Educators in Transition: Unpacking Argentinean Teachers' Attitudes Towards AI in Higher Education
de Salta, U. N., Bottiglieri, L. I., Irrazabal, M. F., de Salta, U. N., Ramallo, C., & de Salta, U. N. (2025) · Íkala Revista de Lenguaje y Cultura · doi:10.17533/udea.ikala.355874 · Cited by 13
Bhattacharya, N., Langhout, R. D., Vaccarino‐Ruiz, S. S., Jackson, N., Woolfe, M., Matta, W., Zúñiga, B. R., Rowe, Z., & Gibo, L. (2021) · American Journal of Community Psychology · doi:10.1002/ajcp.12573 · Cited by 10
Future Prospects of ChatGPT in Higher Education
Phutela, N., Grover, P., Singh, P., & Mittal, N. (2024) · International Conference on Reliability, Infocom Technologies and Optimization (Trends and Future Directions) (ICRITO) · doi:10.1109/icrito61523.2024.10522294 · Cited by 11
“Research shows that ITS discreetly influences the academic accomplishment of students in higher education institutions (HEIs) [4].”
BackgroundPedagogical agents in K-12 education: a scoping review
Zhang, S., Jaldi, C. D., Schroeder, N. L., & Gladstone, J. R. (2024) · Journal of Research on Technology in Education · doi:10.1080/15391523.2024.2381229 · Cited by 9
“We observed fewer conversational agents in the literature (Ma et al. 2014; Steenbergen-Hu and Cooper 2013, 2014; Wang et al. 2023a) that often incorporate visually-present agents.”
Dubey, P., & Crevar, A. R. (2025) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3373-0502-8.ch008 · Cited by 7
OCDE (2022) · Digital education outlook · doi:10.1787/d5fe6bd0-fr · Cited by 17
Meaney, P. A., Hokororo, A., Masenge, T. J., Mwanga, J. R., Kalabamu, F. S., Berg, M., Rozenfeld, B., Smith, Z. H., Chami, N., Mkopi, N. P., et al. (2023) · Digital Health · doi:10.1177/20552076231180471 · Cited by 6
Tang, X., Wang, Y., Ma, H., Wang, A., Zhou, Y. J., Li, S., Pei, R., Cui, H., Peng, Y., & Piao, M. (2024) · Sensors · doi:10.3390/s24072154 · Cited by 6
“Multimodal Learning Analytics (MMLA) present innovative solutions to these limitations by providing automated, real-time feedback [11].”
BackgroundDiscouraging Gaming the System Through Interventions of an Animated Pedagogical Agent
Nunes, T. M., Bittencourt, I. I., Isotani, S., & Jaques, P. A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-45153-4_11 · Cited by 9
Modeling of the Educational Process Based on Smart Technologies
Yablochnikov, S. L., Kuptsov, M. I., Bukhensky, K. V., & Kuptsov, I. M. (2022) · Lecture notes in networks and systems · doi:10.1007/978-3-030-98832-6_48 · Cited by 6
Bringing Non-programmer Authoring of Intelligent Tutors to MOOCs
Aleven, V., Baker, R. S., Wang, Y., Sewall, J., & Popescu, O. (2016) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/2876034.2893442 · Cited by 6
The Role of AI in Transforming Education: A Systematic Review of Trends
Ratul, Q. E. A., Zaman, E., Das, S. K., & Memari, M. (2025) · Intermountain Engineering, Technology and Computing (IETC) · doi:10.1109/ietc64455.2025.11039329 · Cited by 14
“Kulik and Fletcher [33] conducted a comprehensive review that indicates ITSs can lead to significant improvements in student performance, although some studies [34]–[36] report mixed results regarding their overall effectiveness.”
Artificial intelligence innovations challenges and emerging trends in engineering education
Rahman, A., Khandakar, A., Ayari, M. A., Naji, K. K., Al-Ali, A. K., Sellami, A., & Alhazbi, S. (2026) · Discover Education · doi:10.1007/s44217-026-01137-1 · Cited by 19
Böhme, R., & Munser-Kiefer, M. (2020) · MedienPädagogik Zeitschrift für Theorie und Praxis der Medienbildung · doi:10.21240/mpaed/jb17/2020.05.17.x · Cited by 9
Kuzminov, I., Вохмин, А. А., Sharaeva, K., Likhanov, M. V., Markovnikova, A., Lyamin, A. V., & Kurushkin, M. (2023) · Journal of Chemical Education · doi:10.1021/acs.jchemed.3c00138 · Cited by 7
Oliveira, A. S. F., Isotani, S., & Bittencourt, I. I. (2023) · Tecnologias Sociedade e Conhecimento · doi:10.20396/tsc.v10i2.18367 · Cited by 6
Vier, J., Gluz, J. C., & Jaques, P. A. (2015) · Revista Brasileira de Informática na Educação · doi:10.5753/rbie.2015.23.02.45 · Cited by 5
“Algumas revisões sistemáticas recentes mostraram que STIs podem ser tão eficientes quanto professores particulares [21, 43, 44, 46].”
BackgroundPedagogical Agents for Learning
Schroeder, N. L. (2016) · Advances in educational technologies and instructional design book series · doi:10.4018/978-1-4666-9837-6.ch010 · Cited by 5
Treinamento da Habilidade Metacognitiva de Monitoramento do Conhecimento em Sistemas Tutores
Kautzmann, T. R., & Jaques, P. A. (2016) · Revista Brasileira de Informática na Educação · doi:10.5753/rbie.2016.24.02.22 · Cited by 6
“Por fim, os estudos de meta-análise descritos em [58, 59] também encontraram que os STIs podem ser tão efetivos”
BackgroundConversational AIS as the Cornerstone of Hybrid Tutors
Hampton, A. J., & Wang, L. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-22341-0_49 · Cited by 5
Elearning 4.0 for higher education: literature review, trends and perspectives
Babori, A., Ghoulam, K., Falih, N., & Ouchitachen, H. (2021) · International Conference on Digital Age & Technological Advances for Sustainable Development (ICDATA) · doi:10.1109/icdata52997.2021.00032 · Cited by 8
Wang, L. (2023) · Soft Computing · doi:10.1007/s00500-023-08480-7 · Cited by 11
RetractedTowards Better Affect Detectors: Detecting Changes Rather Than States
Mandalapu, V., & Gong, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93846-2_36 · Cited by 5
“The intelligent tutoring systems have come to force in recent years, especially with the rise of affective computing that deals with the possibility of making computers to recognize human affective states in diverse ways [1, 2].”
BackgroundFoundational Principles and Design of a Hybrid Tutor
Hampton, A. J., & Graesser, A. C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-22341-0_8 · Cited by 5
Assessing the Student's Knowledge in Informatics Discipline Using the METEOR Metric
Dobre, I. (2014) · Mediterranean Journal of Social Sciences · doi:10.5901/mjss.2014.v5n19p84 · Cited by 5
“86 Another meta-analysis, more recent (Steenbergen-Hu and Cooper, 2014), “synthesizes research on the effectiveness of intelligent tutoring systems (ITS) for college students” (p.”
BackgroundMethodInfluential
Metadata
- Authors
- Steenbergen-Hu, S., & Cooper, H.
- Venue
- Journal of Educational Psychology
- Published
- 2014-05
- Volume
- 106
- Issue
- 2
- Pages
- 331-347
- ISSN
- 0022-0663 (ISSN-L)
- 1939-2176
- OpenAlex
- W2316441559
- Cited by
- 355
- References
- 68
- Citations by year
- 2026: 48 · 2025: 55 · 2024: 50 · 2023: 28 · 2022: 24 · 2021: 31 · 2020: 31 · 2019: 22 · 2018: 22 · 2017: 13 · 2016: 15 · 2015: 12 · 2014: 4
- FWCI
- 1.50
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- cc-by-nc-nd
- Locations
- Journal of Educational Psychology (publishedVersion)
- DukeSpace (Duke University) (submittedVersion, cc-by-nc-nd)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
- Social Sciences › Education › Artificial Intelligence in Education
- Keywords
- intelligent tutoring systems, university students, academic learning, higher education, computer-assisted instruction, AutoTutor, ALKIS, effect size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.