The DOI matches its Crossref record; 127 of the 127 references it cites were found in a registry or an index.
Intelligent tutoring systems and learning outcomes: A meta-analysis
Abstract
Intelligent Tutoring Systems (ITS) are computer programs that model learners’ psychological states to provide individualized instruction. They have been developed for diverse subject areas (e.g., algebra, medicine, law, reading) to help learners acquire domain-specific, cognitive and metacognitive knowledge. A meta-analysis was conducted on research that compared the outcomes from students learning from ITS to those learning from non-ITS learning environments. The meta-analysis examined how effect sizes varied with type of ITS, type of comparison treatment received by learners, type of learning outcome, whether knowledge to be learned was procedural or declarative, and other factors. After a search of major bibliographic databases, 107 effect sizes involving 14,321 participants were extracted and analyzed. The use of ITS was associated with greater achievement in comparison with teacher-led, large-group instruction (g .42), non-ITS computer-based instruction (g .57), and textbooks or workbooks (g .35). There was no significant difference between learning from ITS and learning from individualized human tutoring (g –.11) or small-group instruction (g .05). Significant, positive mean effect sizes were found regardless of whether the ITS was used as the principal means of instruction, a supplement to teacher-led instruction, an integral component of teacher-led instruction, or an aid to homework. Significant, positive effect sizes were found at all levels of education, in almost all subject domains evaluated, and whether or not the ITS provided feedback or modeled student misconceptions. The claim that ITS are relatively effective tools for learning is consistent with our analysis of potential publication bias.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Every learner has unique learning contexts based on their prior knowledge about the topic, social background, economic well-being and emotional state [41].”
From Artificial Intelligence in Education (AIEd): a high-level academic and industry note 2021 (2022) · cites this paper as [41]
In the citing paper
“Despite being limited in terms of the domain that a particular intelligent tutoring system addresses, they have proven to be effective in providing relevant content to students, interacting with students [6] and improving students’ academic performance [18, 41].”
From Artificial Intelligence in Education (AIEd): a high-level academic and industry note 2021 (2022) · cites this paper as [41]
Citations
Cites
(1988) · Cited by 16,582
Hattie, J., & Timperley, H. (2007) · Review of Educational Research · doi:10.3102/003465430298487 · Cited by 12,905
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,377
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2021) · doi:10.1002/9781119558378 · Cited by 9,799
(2000) · Cited by 9,198
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
An introduction to latent semantic analysis
Landauer, T. K., Foltz, P. W., & Laham, D. (1998) · Discourse Processes · doi:10.1080/01638539809545028 · Cited by 4,880
Fixed-and random-effects models in meta-analysis.
Hedges, L. V., & Vevea, J. L. (1998) · Psychological Methods · doi:10.1037/1082-989x.3.4.486 · Cited by 2,977
The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring
BLOOM, B. S. (1984) · Educational Researcher · doi:10.3102/0013189x013006004 · Cited by 2,734 · cited 1×
“Another notable contribution of VanLehn's review was that he reported an effect size of d ϭ 0.79 for human tutoring compared with no-treatment controls, a value much smaller than the frequently cited, twosigma effect sizes reported by Bloom (1984).”
ResultInfluentialHays, W. (1983) · Contemporary Psychology · doi:10.1037/022267 · Cited by 2,399
Cited by
Sorted by citations
Items 301–400 of 840
Analyzing Students' Behavior in Blended Learning Environment for Programming Education
Luo, J., & Wang, T. (2020) · World Symposium on Software Engineering (WSSE) · doi:10.1145/3425329.3425346 · Cited by 11
“Past researchers have considered the learner's background [8], prior knowledge [9], cognitive abilities [10], working hours [11], and learning attitudes [12] may be related to learning outcomes and behavior.”
BackgroundArtificial Intelligence in Education: Origin, Development and Rise
Ye, R., Sun, F., & Li, J. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-89092-6_49 · Cited by 11
Treceño-Fernández, D., Calabia-del-Campo, J., Bote-Lorenzo, M. L., Gómez-Sánchez, E., Luis-García, R. D., & Alberola-López, C. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105634 · Cited by 12
Enhancing a student productivity model for adaptive problem-solving assistance
Maniktala, M., Chi, M., & Barnes, T. M. (2022) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-022-09338-7 · Cited by 12
“Personalized instruction is considered an effective educational strategy (Bloom 1984), with research suggesting that students who receive this individualized instruction perform significantly better than those who receive classroom instruction (Ma et al. 2014; VanLehn 2011).”
BackgroundOverview of Applications of Artificial Intelligence (AI) in Diverse Fields
Kuhar, N., Kumria, P., & Rani, S. (2024) · Springer eBooks · doi:10.1007/978-3-031-69433-2_2 · Cited by 11
Ghosh, S. S. (2024) · Computers in the Schools · doi:10.1080/07380569.2024.2435308 · Cited by 12
“Although the potential advantages of this have been extensively studied in the field of education as a whole (Ma et al., 2014), their specific effects on language teacher education have not been thoroughly investigated.”
Yu-jie, H., Hong, S., Li, Z., & Lim, C. (2025) · TechTrends · doi:10.1007/s11528-025-01058-0 · Cited by 10
“A variety of ITSs has been defined under different forms of instructional strategies, including model-tracing ITSs, example-tracing ITSs, agent-based ITSs, expectation and misconception tailoring and constraint-based modeling (Ma et al., 2014; Vandewae-tere & Clarebout, 2014).”
Adaptive Learning Technologies: Navigating the Road from Hype to Reality
Ganesh, S. V., Nayagam, M. G., Suresh, V., Rajakarunakaran, S., & Bensujin, B. (2025) · Springer eBooks · doi:10.1007/978-981-96-2355-6_4 · Cited by 10
Beyond Good AI: The Need for Sound Learning Theories in AIED
Sinha, T. (2025) · Technology Knowledge and Learning · doi:10.1007/s10758-025-09843-9 · Cited by 13
“Bulk of the work in the area of intelligent tutoring systems in STEM domains falls in this category, where students receive step-by-step hints and prompts to nudge them towards problem-solving success (Ma et al., 2014 ; Kulik & Fletcher, 2016 ).”
AI-Supported Formative Assessments: Enhancing Student-Centered Learning and Teacher Perceptions
Mousa, M. (2025) · Journal of Pedagogy and Education Science · doi:10.56741/jpes.v4i02.857 · Cited by 10
“AI tools like intelligent tutoring systems (ITS) have been shown to help teachers make informed decisions on adjusting instructional strategies in real time, fostering a more dynamic and responsive learning environment [6],[7].”
A Conversation-Based Intelligent Tutoring System Benefits Adult Readers with Low Literacy Skills
Fang, Y., Lippert, A. M., Cai, Z., Hu, X., & Graesser, A. C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-22341-0_47 · Cited by 10
ICT competency and practicum of preservice teachers as digital natives: a mixed-method study
Wang, P., & Ko, J. (2022) · Asia Pacific Journal of Education · doi:10.1080/02188791.2022.2083581 · Cited by 10
The Design Implementation Framework
Stone, M. L., Kent, K. M., Roscoe, R. D., Corley, K. M., Allen, L. K., & McNamara, D. S. (2017) · Advances in educational technologies and instructional design book series · doi:10.4018/978-1-5225-2639-1.ch004 · Cited by 11
Tuti, T., Winters, N., Muinga, N., Wanyama, C. W., English, M., & Paton, C. (2019) · JMIR Research Protocols · doi:10.2196/13034 · Cited by 10
“…increase the number of well-trained health providers, are yet to capitalize on digital learning approaches that model learners’ knowledge states to provide individualized instruction and have been shown to have considerable positive effects on learning outcomes in other subject areas [5,24,25].”
InfluentialVirtual trainer system: a tool to increase exercise participation and work productivity
Kwok, R. C.-W., Leung, A. C. M., Hui, S. S., & Wong, C. C.-K. (2021) · Internet Research · doi:10.1108/intr-04-2020-0236 · Cited by 10
A Comparison of Hints vs. Scaffolding in a MOOC with Adult Learners
Zhou, Y., Andres-Bray, J. M. L., Hutt, S. J., Ostrow, K., & Baker, R. S. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-78270-2_76 · Cited by 10
Evaluating Adaptive Pedagogical Agents’ Prompting Strategies Effect on Students’ Emotions
Bouchet, F., Harley, J. M., & Azevedo, R. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-91464-0_4 · Cited by 10
“Adaptation is key to successful learning with intelligent tutoring systems (ITSs), as learners integrate instructional material with elements ITSs are designed to enhance: learning and problem solving [1].”
BackgroundAdaptive remediation for novice programmers through personalized prescriptive quizzes
Soltanpoor, R., Thevathayan, C., & D’Souza, D. (2018) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3197091.3197097 · Cited by 10
“form traditional teaching for specific cohorts [20].”
MethodCan Crowds Customize Instructional Materials with Minimal Expert Guidance?
Yang, K. B., Nagashima, T., Yao, J., Williams, J. J., Holstein, K., & Aleven, V. A. W. M. M. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449193 · Cited by 14
“AI-based educational software is increasingly used in K-12 classrooms, enabling teachers to facilitate more personalized learning experiences [36, 41, 50, 71].”
BackgroundFrom Robots to Books-An Introduction to Smart Applications of AI in Education (AIEd)Preprint
Narendra, A., Ojha, S., Mohapatra, S., & Misra, I. (2023) · doi:10.36227/techrxiv.21836610 · Cited by 11
Examining the Effects of a Real-Time, Knowledge-Aware Tool for Academic Writing Assessment
Li, X., Ouyang, F., Liu, J., Wei, C., & Chen, W. (2023) · Journal of Educational Computing Research · doi:10.1177/07356331221136889 · Cited by 9
Lin, C.-H., Kuo, B., & Chang, F. T. Y. (2025) · Educational Psychology · doi:10.1080/01443410.2025.2561028 · Cited by 14
“Existing research consistently demonstrates the independent positive effects of digital tools on mathematics achievement (Feng et al., 2023; Ma et al., 2014; Pellegrini et al., 2021) and effectively promote SRL (Molenaar et al., 2023; van der graaf et al., 2023).”
InfluentialBach, K. M., Reinhold, F., & Hofer, S. I. (2025) · npj Science of Learning · doi:10.1038/s41539-025-00358-7 · Cited by 16
“Learning at a suitable dif fi culty level helps alleviate anxiety 151 and can improve attitudes towardchallengingsubjectssuchasmath 162 .”
Adi, P. N., Köhler, T., Triyono, M. B., Priyanto, Handayani, S. A., & Maruanaya, R. F. (2026) · Journal of Computers in Education · doi:10.1007/s40692-026-00387-0 · Cited by 17
“These AI-driven systems excel at personalizing instruction and supporting evidence-based decision-making (Ma et al., 2014; Yilmaz, 2020 ; Maglogiannis, 2020 ; Apture et al., 2024).”
Rosenbaum, S., Moberg, J., Chesire, F., Mugisha, M., Ssenyonga, R., Ochieng, M., Simbi, C. M. C., Nakyejwe, E., Ngatia, B., Gabriel Rada, G., et al. (2024) · F1000Research · doi:10.12688/f1000research.132580.3 · Cited by 8
Charting the Future of Assessments
Kyllonen, P. C., Sevak, A., Ober, T. M., Choi, I., Sparks, J. R., & Fishtein, D. (2024) · ETS Research Report Series · doi:10.1002/ets2.12388 · Cited by 18
Affective Response Categories—Toward Personalized Reactions in Affect-Adaptive Tutoring Systems
Schmitz-Hübsch, A., Stasch, S.-M., Becker, R., Fuchs, S., & Wirzberger, M. (2022) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2022.873056 · Cited by 10
“Another definition by Ma et al. (2014) notes that ITS are computer programs that provide individualized instruction by analyzing model learners’ psychological states.”
BackgroundDevelopment of Computer Assisted Instruction (CAI) Based Teaching Materials in Junior High School
niversitas Jember, E. J., & Wahyuni, S. (2016) · International Journal of Learning and Teaching · doi:10.18178/ijlt.2.2.117-120 · Cited by 9
Njeru, A. M., & Paracha, S. (2017) · International Conference on Applied System Innovation · doi:10.1109/icasi.2017.7988616 · Cited by 9
“thesis SCHOLAR implemented the earliest intelligent tutor system that invited students to explore geographical features of South America [9], [10].”
BackgroundMorrison, D. M., & Miller, K. B. (2017) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-017-0153-0 · Cited by 9
“Comparisons across black-box categories such as Bhuman tutoring,^ Bintelligent tutoring,^ and Btraditional instruction^ (e.g., Ma et al. 2014; VanLehn 2011) are of limited practical value if we are not told what’s going on inside those black boxes.”
BackgroundCoherently Organized Digital Exercises and Expositions
Sangwin, C., & Kinnear, G. (2021) · PRIMUS · doi:10.1080/10511970.2021.1999352 · Cited by 9
“Previous studies, such as [25], have been cautiously optimistic about the effectiveness of computer based instruction in general.”
ResultGenerative AI and Higher Education Assessments: A Competency-Based AnalysisPreprint
Tenakwah, E. S., Boadu, G., Tenakwah, E. J., Parzakonis, M., Brady, M., Kansiime, P., Said, S. J., Ayilu, R. K., Radavoi, C. N., & Berman, A. (2023) · Research Square · doi:10.21203/rs.3.rs-2968456/v2 · Cited by 15
Lampropoulos, G., López-Belmonte, J., Pozo-Sánchez, S., & Marín-Marín, J.-A. (2026) · Advances in Computational Intelligence and Robotics · doi:10.4018/979-8-2600-0101-1.ch001 · Cited by 9
The Future of Intelligent Tutoring Systems for Writing
Banawan, M., Butterfuss, R., Taylor, K. S., Christhilf, K., Hsu, C., O’Loughlin, C., Allen, L. K., Roscoe, R. D., & McNamara, D. S. (2023) · Springer eBooks · doi:10.1007/978-3-031-36033-6_23 · Cited by 8
Intelligent Tutoring Systems Examined in Social Experiments—Is the Magic Gone? A Meta-Analysis
Tlili, A., Salha, S. H., Wang, H., & Huang, R. (2023) · IEEE International Conference on Advanced Learning Technologies (ICALT) · doi:10.1109/icalt58122.2023.00020 · Cited by 8
“However, interestingly, while conducting this present meta-analysis, it is seen that most of the ITSs reviewed in this present study or in the literature in general [3, 4] are treated as a black box.”
BackgroundResultInfluentialEducation artificial intelligence systems and their use in teaching
Mengliyev, I., Meylikulov, S., Fayzullayeva, Z., & Kobulova, M. (2024) · AIP conference proceedings · doi:10.1063/5.0241443 · Cited by 8
Yang, J. W., Choe, K., Lim, J., & Park, J. (2025) · Interactive Learning Environments · doi:10.1080/10494820.2025.2488984 · Cited by 9
“While the average effect size of ITS in terms of general academic achievement, such as test scores, across 50 research papers was 0.66 (Kulik & Fletcher, 2016), there were findings where performance did not significantly differ compared to small group instruction (Ma et al., 2014).”
InfluentialWeinhandl, R., Helm, C., Anđić, B., & Große, C. S. (2025) · Education and Information Technologies · doi:10.1007/s10639-025-13740-0 · Cited by 11
“The meta-analysis supports earlier findings on the benefits of digital technologies in education (Ma et al., 2014; Steenbergen & Cooper, 2013; Vogel et al., 2017).”
Wang, J., Tigelaar, D. E. H., Ye, T., & Admiraal, W. (2025) · Journal of Computer Assisted Learning · doi:10.1002/jcal.70168 · Cited by 8
Eti, N., Mosia, M. S., & Egara, F. O. (2026) · Frontiers in Computer Science · doi:10.3389/fcomp.2026.1813431 · Cited by 24
Tian, J., & Zhang, R. (2026) · Humanities and Social Sciences Communications · doi:10.1057/s41599-026-07153-8 · Cited by 18
“…beyond CT to include learning motivation, metacognitive monitoring, and academic anxiety, and examine how different instructional designs and AI-use policies shape the balance between immersion and dependency (Ali and Pearl, 2024; Bianchi, 2024; Gerlich, 2025; Ma et al., 2014; Zhong et al., 2024).”
Belland, B. R., Lee, E., Zhang, A. Y., & Kim, C. (2022) · Computer Applications in Engineering Education · doi:10.1002/cae.22556 · Cited by 17
“Scaffolding is an intervention that leads to stronger learning than lectures across learner populations, and education and assessment levels [6, 8, 9, 37, 55, 61].”
BackgroundAvcı, G., Owusu-Boateng, O., Vampa, M., & Khalid, A.-M. (2025) · doi:10.70593/978-93-49307-53-7_4 · Cited by 8
eLEM: A novel e-Learner Experience Model
Hammad, R., Odeh, M., & Khan, Z. (2017) · UWE Research Repository (UWE Bristol) · Cited by 8
Modeling Student Learning Behaviors in ALEKS: A Two-Layer Hidden Markov Modeling Approach
Wang, G., Tang, Y., Li, J., & Hu, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-93846-2_70 · Cited by 8
Zhang, Y., & Wang, Q. (2020) · International Journal of Mathematical Education in Science and Technology · doi:10.1080/0020739x.2020.1717659 · Cited by 8
Olivares, D. M., Hundhausen, C. D., & Ray, N. (2021) · ACM Transactions on Computing Education · doi:10.1145/3453165 · Cited by 8
Towards a Design of an Intelligent Educational System
Terzieva, V. T., Ilchev, S., Todorova, K., & Andreev, R. (2021) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2021.10.474 · Cited by 10
Heterogeneity of Treatment Effects of a Video Recommendation System for Algebra
Leite, W. L., Kuang, H., Shen, Z., Chakraborty, N., Michailidis, G. C., D’Mello, S. K., & Xing, W. (2022) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/3491140.3528275 · Cited by 8
“Ma et al. [9] performed a meta-analysis of the effects of ITS that included moderation by continent where the sample was taken, grade level, and level of prior knowledge of the students.”
BackgroundResultLearning spaces: built, natural and digital considerations for learning and learners
Kraftl, P., McKenzie, M., Gulson, K. Ν., Accioly, I., Burke, C., Clarke, D. A. G., Bailón, R. O. F., Goodyear, P., Hartong, S., Howard, S. K., et al. (2022) · UNESCO MGIEP eBooks · doi:10.56383/kwaw2002 · Cited by 8
Rosenbaum, S., Moberg, J., Chesire, F., Mugisha, M., Ssenyonga, R., Ochieng, M., Simbi, C. M. C., Nakyejwe, E., Ngatia, B., Gabriel Rada, G., et al. (2024) · F1000Research · doi:10.12688/f1000research.132580.2 · Cited by 8
Hu, Y., Wong, R. M., Adesope, O., & Taylor, M. E. (2020) · Computers & Education · doi:10.1016/j.compedu.2020.103816 · Cited by 8
Self-citationSuraworachet, W., Villa-Torrano, C., Zhou, Q., Asensio-Pérez, J. I., Dimitriadis, Y. A., & Cukurova, M. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-86436-1_13 · Cited by 9
“There are many tools developed to support writing instruction and assessment including automated essay scoring systems to assess writing quality [15], automated writing evaluation systems to provide feedback and correction suggestions [16] and intelligent tutoring systems that can provide automated feedback and provok…”
Authoring Intelligent Tutoring Systems Using Human Computation: Designing for Intrinsic Motivation
Olney, A. M., & Cade, W. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-20816-9_60 · Cited by 9
“Recent meta-analyses have found that ITS learning gains are indistinguishable from human tutor controls [21, 40], suggesting that ITS research has su ffi ciently matured to make it broadly applicable to K-16 education.”
BackgroundInfluence of LLMs on Learning and Teaching in Higher EducationPreprint
Upadhyay, A., Farahmand, E., Muñoz, I., Khan, M. A., & Witte, N. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4716855 · Cited by 9
Automation in education: trending issues concerning Artificial Intelligence
Lima, G. D. M., Ferreira, G. M. D. S., & Carvalho, J. D. S. (2024) · Educação e Pesquisa · doi:10.1590/s1678-4634202450273857en · Cited by 14
Automação na educação: caminhos da discussão sobre a inteligência artificial
Lima, G. D. M., Ferreira, G. M. D. S., & Carvalho, J. D. S. (2024) · Educação e Pesquisa · doi:10.1590/s1678-4634202450273857por · Cited by 60
Ahn, D., & Lim, H. (2025) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3706598.3713646 · Cited by 26
“This comprehensive vision spanned from administrative support to personalized instruction and assessment, demonstrating a broader range of AI applications in PE than typically found in existing AIED research that focused primarily on student-oriented solutions [21, 28, 29, 66, 113].”
Using AI to Personalize Learning in Accounting Education
Alsharari, N. M., & Habashi, J. I. (2025) · Studies in computational intelligence · doi:10.1007/978-3-031-89175-5_30 · Cited by 7
Harrasi, N. A., Subhi, N. A., Harthy, S. A., & Habsi, J. A. (2025) · Social Sciences & Humanities Open · doi:10.1016/j.ssaho.2025.101950 · Cited by 16
Effects of Artificial Intelligence on Educational Functioning: A Review and Meta-Analysis
Yeo, G., & Lansford, J. E. (2025) · Educational Psychology Review · doi:10.1007/s10648-025-10085-5 · Cited by 16
“Furthermore, meta-analyses of experimental studies have not adequately explored whether the observed effects of AI in educa - tion extend beyond basic effectiveness (e.g., Ma et al., 2014 ).”
InfluentialIntelligent Tutoring Systems Need Teachers
Bardach, L., Moeller, K., Ruiz-García, M., Strittmatter, Y., Meyer, J., Musslick, S., & Spitzer, M. (2025) · Journal of Computer Assisted Learning · doi:10.1002/jcal.70159 · Cited by 15
du Boulay, B. (2020) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-020-00198-z · Cited by 7
“…example, over the last decade there have been at least nine meta-reviews and meta-analyses that have compared the educational effectiveness of adaptive systems for a topic versus human teachers of that topic, working either with whole classes or one-to-one (see, e.g. Ma et al. 2014; VanLehn 2011).”
BackgroundIntelligent Systems in Learning and Education
Patel, V. L., & Dev, P. (2022) · Cognitive informatics in biomedicine and healthcare · doi:10.1007/978-3-031-09108-7_16 · Cited by 8
Rethinking Teachers’ Formative Assessment Practices Within Technology-Enhanced Classrooms
Cusi, A., Aldon, G., Barzel, B., & Olsher, S. (2024) · Springer international handbooks of education · doi:10.1007/978-3-030-95060-6_29-1 · Cited by 7
Maia, D., dos Santos, S. C., Lima, L. G. D., Franca, V. L., Lima, A. V. A., & Andrade, D. (2024) · IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie61694.2024.10893361 · Cited by 7
“These three factors are essential elements within a project for the success of the main objective for which that system was built [2][12][13].”
Renkl, A. (2020) · Springer eBooks · doi:10.1007/978-3-662-61403-7_1 · Cited by 9
Establishing a Student-Centered Environment to Support All Learners
Basham, J. D., Stahl, S., Hall, T., & Carter, R. A. (2017) · Advances in educational technologies and instructional design book series · doi:10.4018/978-1-5225-2520-2.ch007 · Cited by 7
Alday, R. B. (2018) · International Conference on Algorithms, Computing and Systems (ICACS) · doi:10.1145/3242840.3242867 · Cited by 7
“Intelligent Tutoring Systems or ITS was defined in [9] as a computer program that model learner’s psychological states to provide individual interactions.”
BackgroundTeam Measurement: Unobtrusive Strategies for Intelligent Tutoring Systems
DeCostanza, A. H., Gamble, K. R., Estrada, A. X., & Orvis, K. L. (2018) · Research on managing groups and teams · doi:10.1108/s1534-085620180000019009 · Cited by 7
Barzel, B., Ball, L., & Klinger, M. (2019) · Mathematics education in the digital era · doi:10.1007/978-3-030-19741-4_4 · Cited by 7
Cody, C., Maniktala, M., Lytle, N., Chi, M., & Barnes, T. M. (2021) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-021-00237-3 · Cited by 7
“Intelligent tutoring systems (ITS) provide adaptive assistance to students and have significant positive effects on learning (Murray 1999; Ma et al. 2014).”
BackgroundDeveloping Instructional Design Agents to Support Novice and K-12 Design Education
Schimpf, C., Huang, X., Xie, C., Sha, Z., & Massicotte, J. (2020) · ASEE Annual Conference & Exposition · doi:10.18260/1-2--32640 · Cited by 7
“ITS are computer programs that provide individualized instructions to learners by computationally or statistically modeling learners’ knowledge, cognitive or other related states and contrast students’ states with an expert model of a given domain to provide appropriate instruction 10,24 .”
BackgroundContrastive Reinforcement Learning of Symbolic Reasoning DomainsPreprint
Poesia, G., Dong, W., & Goodman, N. D. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.09146 · Cited by 7
“Intelligent Tutoring Systems [20, 28, 22] provide a key application for step-by-step solvers.”
BackgroundArtificial Intelligence and Learning
Lee, J., Lamb, R. L., & Kim, S. (2021) · Oxford University Press eBooks · doi:10.1093/obo/9780199756810-0269 · Cited by 9
Innovative Strategies for Accelerated Human Resources Development in South Asia:
Bank, A. D. (2017) · doi:10.22617/tcs179080 · Cited by 9
Idiographic learning analytics: Mapping of the ethical issues
Misiejuk, K., Samuelsen, J., Kaliisa, R., & Prinsloo, P. (2024) · Learning and Individual Differences · doi:10.1016/j.lindif.2024.102599 · Cited by 8
Application of Social Robots in Higher Education: A Long-Term Study
Donnermann, M., Schäper, P., & Lugrin, B. (2025) · International Journal of Social Robotics · doi:10.1007/s12369-025-01286-7 · Cited by 9
“Technology-supported learning tools are already widespread and beneficial for learning [ 27, 44].”
Lim, H., Choi, D., Choi, D., Nam, S., & Hong, H. (2025) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2025.103687 · Cited by 6
Lee, Y. O., Bang, B., Lee, J., & Oh, S. (2026) · IEEE Access · doi:10.1109/access.2026.3657726 · Cited by 6
“Tutors, ASSISTments [1], [2]; for a meta-analysis, see [3]) The widespread adoption of online and blended learning environments has familiarized students with web-based mathematics platforms and technology-mediated instruction.”
Liu, X., Tang, Q., Wu, W., Chen, X. Y., & Wang, C. (2026) · Interactive Learning Environments · doi:10.1080/10494820.2026.2672562 · Cited by 8
Effectiveness of a numeracy intelligent tutoring system in kindergarten: A conceptual replication
Grimes, K. R., Park, S., McClelland, A. M., Park, J., Lee, Y. R., Nozari, M., Umer, Z., Zaparolli, B., & Bryant, D. P. (2021) · Journal of Numerical Cognition · doi:10.5964/jnc.6931 · Cited by 10
Martínez, A. F., & Alonso, N. (2024) · Tripodos · doi:10.51698/tripodos.2024.55.07 · Cited by 9
“…intelligence is also considered a scientific field in which intelligent tools are developed, capable of responding to different problems through their ability to anticipate situations in their environment, as well as their elasticity (Ma et al. , 2014; Wang et al. , 2015; Tuomi et al. , 2018).”
BackgroundPedagogical agents in K-12 education: a scoping review
Zhang, S., Jaldi, C. D., Schroeder, N. L., & Gladstone, J. R. (2024) · Journal of Research on Technology in Education · doi:10.1080/15391523.2024.2381229 · Cited by 8
“We observed fewer conversational agents in the literature (Ma et al. 2014; Steenbergen-Hu and Cooper 2013, 2014; Wang et al. 2023a) that often incorporate visually-present agents.”
Kim, N. J. (2021) · Digital Commons - USU (Utah State University) · doi:10.26076/5859-913d · Cited by 7
Apprentissages scolaires et technologies numériques : Une revue critique des méta-analyses
Leroux, G., Monteil, J., & Huguet, P. (2017) · L’Année psychologique · doi:10.4074/s0003503317004018 · Cited by 6
Altun, D., Krauss, C., Streicher, A., Mueller, C., Atorf, D., Rerhaye, L., & Kunde, D. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-05887-5_18 · Cited by 6
Perspectives de l'OCDE sur l'éducation numérique 2021
OCDE (2022) · Digital education outlook · doi:10.1787/d5fe6bd0-fr · Cited by 16
Discouraging Gaming the System Through Interventions of an Animated Pedagogical Agent
Nunes, T. M., Bittencourt, I. I., Isotani, S., & Jaques, P. A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-45153-4_11 · Cited by 6
Application of planning techniques in knowledge-managed tutoring systems
Marinov, M., & Valova, I. (2016) · International Conference on Information Technology Based Higher Education and Training (ITHET) · doi:10.1109/ithet.2016.7760745 · Cited by 6
“The goal of these educational technology systems is to provide intelligent, one-on-one, computer based support to students as they learn to solve problems in a type of instruction that is often not available because of scarce (human) resources [2,5,7].”
BackgroundUsing Web-based Intelligent Tutoring Systems in Teaching Physics Subjects at Undergraduate Level
Erdemir, M. (2019) · Universal Journal of Educational Research · doi:10.13189/ujer.2019.070705 · Cited by 6
Conijn, R., van Zaanen, M. M., & Waes, L. V. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-29736-7_43 · Cited by 6
Die Zukunft des MINT-Lernens – Band 1
Röth, J., Baum, M., Eilerts, K., Hornung, G., & Trefzger, T. (2022) · Springer eBooks · doi:10.1007/978-3-662-66131-4 · Cited by 9
Bringing Non-programmer Authoring of Intelligent Tutors to MOOCs
Aleven, V., Baker, R. S., Wang, Y., Sewall, J., & Popescu, O. (2016) · ACM Conference on Learning @ Scale (L@S) · doi:10.1145/2876034.2893442 · Cited by 6
“There is substantial evidence that ITS can enhance student learning, compared to other methods of instruction [6][8][11][13][15].”
BackgroundOECD (2018) · Trends shaping education spotlights · doi:10.1787/9b181d3c-en · Cited by 10
AI as a teaching augmentor: A review of the integration of AI in science education
Mnguni, L. E. (2025) · Multidisciplinary Reviews · doi:10.31893/multirev.2025390 · Cited by 7
Adayilo, D. M., Oyefolahan, I. O., Ndunagu, J. N., Otuya, C., Malcalm, E., & Twabu, K. Y. (2025) · Journal of Future Artificial Intelligence and Technologies · doi:10.62411/faith.3048-3719-290 · Cited by 11
Exploring the Scholarly Landscape: AI Teaching and Learning in Adult Education
Akbar, Y. A. A., Anuar, A., Zani, R. M., Abdullah, F. N., & Sulaiman, E. S. E. (2024) · International Journal of Academic Research in Progressive Education and Development · doi:10.6007/ijarped/v13-i1/19354 · Cited by 8
“The results indicated a consistent enhancement in learning outcomes across several topic areas, as reported in previous studies by (Ma et al., 2014; Grgurovic et al., 2013).”
ResultAdolphe, M., Pech, M., Sawayama, M., Maurel, D., Delmas, A., Oudeyer, P., & Sauzeon, H. (2025) · PLoS ONE · doi:10.1371/journal.pone.0316860 · Cited by 10
“Several signals or performance dimensions can be used to guide the generation of a curriculum: some ITS are interested in using an optimal emotional level [81] or learning progress [82,83] or both [84].”
Exploring Causality Within Collaborative Problem Solving Using Eye-Tracking
Sharma, K., Olsen, J. K., Aleven, V., & Rummel, N. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-98572-5_32 · Cited by 5
Metadata
- Authors
- Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q.
- Affiliations
- Simon Fraser University (CA)
- Washington State University (US)
- Venue
- Journal of Educational Psychology
- Published
- 2014-11
- Volume
- 106
- Issue
- 4
- Pages
- 901-918
- ISSN
- 0022-0663 (ISSN-L)
- 1939-2176
- OpenAlex
- W2117655204
- Cited by
- 840
- References
- 127
- Citations by year
- 2026: 179 · 2025: 132 · 2024: 99 · 2023: 65 · 2022: 67 · 2021: 69 · 2020: 70 · 2019: 41 · 2018: 43 · 2017: 36 · 2016: 22 · 2015: 17
- FWCI
- 24.90
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Educational Psychology (publishedVersion)
- Funders
- Social Sciences and Humanities Research Council of Canada
- Washington State University
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Psychology › Developmental and Educational Psychology › Innovative Teaching and Learning Methods
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Visual and Cognitive Learning Processes
- Keywords
- intelligent tutoring systems, learning outcomes, personalized learning, computer-assisted instruction, teacher-centered instruction, small group instruction, homework help, metacognitive knowledge, feedback, misconceptions, publication bias, effect size
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
