Effectiveness of Cognitive Tutor Algebra I at Scale
The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
This article examines the effectiveness of a technology-based algebra curriculum in a wide variety of middle schools and high schools in seven states. Participating schools were matched into similar pairs and randomly assigned to either continue with the current algebra curriculum for 2 years or to adopt Cognitive Tutor Algebra I (CTAI), which uses a personalized, mastery-learning, blended-learning approach. Schools assigned to implement CTAI did so under conditions similar to schools that independently adopt it. Analysis of posttest outcomes on an algebra proficiency exam finds no effects in the first year of implementation, but finds evidence in support of positive effects in the second year. The estimated effect is statistically significant for high schools but not for middle schools; in both cases, the magnitude is sufficient to improve the median student’s performance by approximately eight percentile points.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“AI tools can enhance learning outcomes by providing personalised instruction and immediate feedback, thus supporting skill acquisition and knowledge retention [2,3].”
From AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking (2025) · cites this paper as [2]
In the citing paper
“The literature suggests that AI can enhance learning outcomes when used appropriately [2,3].”
From AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking (2025) · cites this paper as [2]
Citations
Cites
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1993) · doi:10.1007/978-1-4899-4541-9 · Cited by 21,230
Multiple Imputation for Nonresponse in Surveys
Rubin, D. B. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316696 · Cited by 20,701
National Council of Teachers of Mathematics
(1979) · The Mathematics Teacher · doi:10.5951/mt.72.8.0637 · Cited by 8,499
(2009) · US Department of Education · Cited by 3,654
Gill, J. (2005) · Encyclopedia of Social Measurement · doi:10.1016/b0-12-369398-5/00166-3 · Cited by 3,114
TIMSS 2011 International Results in Mathematics.
Mullis, I. V. S., Martín, M. O., Foy, P., & Arora, A. (2012) · Cited by 2,342
Measurement Error in Nonlinear Models
Carroll, R. J., Ruppert, D., Stefanski, L. A., & Crainiceanu, C. M. (2006) · Chapman & Hall/CRC eBooks · doi:10.1201/9781420010138 · Cited by 2,116
Cognitive Tutors: Lessons Learned
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995) · Journal of the Learning Sciences · doi:10.1207/s15327809jls0402_2 · Cited by 1,982 · cited 3×
“In addition, an experiment testing the efficacy of the related Cognitive Tutor Geometry curriculum in Grades 9 to 12 found a significant negative effect on student achievement (Pane, McCaffrey, Slaughter, Steele, & Ikemoto, 2010).”
MethodMeasurement error in nonlinear models: a modern perspective
Carroll, R. J. (2006) · Cited by 1,374
Research works printed but not linked: 10
- American College Testing, 2012, The reality of college readiness: National
- Fennema E., 1976, JSAS Catalog of Selected Documents of Psychology, 6, 31
- Greene W. H., 2003, Econometric analysis
- Karam R., 2013, Evaluating Cognitive Tutor Algebra 1 Curricula at Scale: Focus on implementation
- Koedinger K. R., 2000, Carnegie Learning’s Cognitive Tutor: Summary research results
- Lockwood J. R., Value-added modeling and growth modeling with particular application to teacher and school effectiveness
- Ritter S., 2007, Supporting Learning Flow Through Integrative Technologies, 162, 13
- U.S. Department of Education, 2011, What Works Clearinghouse: Procedures and standards handbook
- What Works Clearinghouse, 2004, WWC Intervention Report: Cognitive Tutor®
- What Works Clearinghouse, 2013, WWC Intervention Report: Carnegie Learning Curricula and Cognitive Tutor®
Cited by
Conducting Educational Design Research
McKenney, S., & Reeves, T. C. (2018) · Routledge eBooks · doi:10.4324/9781315105642 · Cited by 1,354
AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking
Gerlich, M. (2025) · doi:10.2139/ssrn.5082524 · Cited by 1,238
AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking
Gerlich, M. (2025) · Societies · doi:10.3390/soc15010006 · Cited by 1,248
State of the art and practice in AI in education
Holmes, W., & Tuomi, I. (2022) · European Journal of Education · doi:10.1111/ejed.12533 · Cited by 1,093
Effectiveness of Intelligent Tutoring Systems: A Meta-Analytic Review
Kulik, J. A., & Fletcher, J. D. (2016) · Review of Educational Research · doi:10.3102/0034654315581420 · Cited by 930
Teacher support and student motivation to learn with Artificial Intelligence (AI) based chatbot
Chiu, T. K. F., Moorhouse, B. L., Chai, C. S., & Ismailov, M. (2024) · Interactive Learning Environments · doi:10.1080/10494820.2023.2172044 · Cited by 707
“Relevant studies have revealed that well-designed AI systems such as, OLI learning course, developed by the Carne fi e Mellon University (Lovett et al., 2008) and Cognitive Tutor (Pane et al., 2014) can have positive impacts on student learning.”
The promise and challenges of generative AI in education
Giannakos, M., Azevedo, R., Brusilovsky, P., Cukurova, M., Dimitriadis, Y., Hernandez-Leo, D., Järvelä, S., Mavrikis, M., & Rienties, B. (2025) · Behaviour & Information Technology · doi:10.1080/0144929x.2024.2394886 · Cited by 543
“There are particular domains and contexts where AI has shown particular promise to improve learning outcomes, such as AI-assisted tutoring in algebra (see Pane et al. 2014).”
Designing educational technologies in the age of AI: A learning sciences‐driven approach
Luckin, R., & Cukurova, M. (2019) · British Journal of Educational Technology · doi:10.1111/bjet.12861 · Cited by 481
“…have been shown to have statistically significant positive impacts on student learning including, for example, OLI learning course (Lovett et al., 2008), SQL-Tutor (Mitrovic, & Ohlsson 1999), ALEKS (Craig et al. 2013), Cognitive Tutor (Pane et al. 2014), and ASSISTments (Koedinger et al. 2010).”
BackgroundStupid tutoring systems, intelligent humans
Baker, R. S. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-016-0105-0 · Cited by 425
“However, extremely high engagement along the lines reported by Schofield has been more recently reported among students using Cognitive Tutors in the Philippines (Rodrigo, Baker, & Rossi, 2013).”
BackgroundInfluentialEmpowering Educators to Be AI-Ready
Luckin, R., Cukurova, M., Kent, C., & du Boulay, B. (2022) · Computers and Education Artificial Intelligence · doi:10.1016/j.caeai.2022.100076 · Cited by 422
“Note that the evaluation of the Cognitive Tutor was conducted as a multi-state study using matched pairs of schools across the USA.”
Education Technology: An Evidence-Based Review
Escueta, M., Quan, V., Nickow, A. J., & Oreopoulos, P. (2017) · doi:10.3386/w23744 · Cited by 389
Artificial Intelligence in Education
Holmes, W. (2019) · Encyclopedia of Education and Information Technologies · doi:10.1007/978-3-319-60013-0_107-1 · Cited by 336
Kim, N. J., & Kim, M. K. (2022) · Frontiers in Education · doi:10.3389/feduc.2022.755914 · Cited by 298
“…147 middle and high schools in the United States, which analyzed the change in grades in algebra, the students who used the AI education software “MATHia” with traditional textbook showed significantly higher average scores than those who studies the algebra only with textbooks (Pane et al., 2014).”
BackgroundWalkington, C., & Bernacki, M. L. (2020) · Journal of Research on Technology in Education · doi:10.1080/15391523.2020.1747757 · Cited by 291
“However, results are variable; for example, in K-12 mathematics, evaluations of blended learning platforms have shown some positive results for Khan Academy (Murphy et al., 2014), Cognitive Tutor Algebra (Pane et al., 2014), and ASSISTments (Roschelle et al., 2016), and more recent null results for Teach to One: Math…”
ResultCukurova, M. (2025) · British Journal of Educational Technology · doi:10.1111/bjet.13514 · Cited by 302
Wang, S., Christensen, C., Cui, W., Tong, R., Yarnall, L., Shear, L., & Feng, M. (2023) · Interactive Learning Environments · doi:10.1080/10494820.2020.1808794 · Cited by 277
“After 2 years of the intervention, the average student’s performance improved by approximately 8 percentile points (Pane et al., 2014).”
BackgroundLearning is Not a Spectator Sport: Doing is Better than Watching for Learning from a MOOC
Koedinger, K. R., Kim, J., Jia, J. Z., McLaughlin, E. A., & Bier, N. L. (2015) · Proceedings of the Second (2015) ACM Conference on Learning @ Scale · doi:10.1145/2724660.2724681 · Cited by 271
Artificial Intelligence as an Effective Classroom Assistant
du Boulay, B. (2016) · IEEE Intelligent Systems · doi:10.1109/mis.2016.93 · Cited by 213
Instructional Complexity and the Science to Constrain It
Koedinger, K. R., Booth, J. L., & Klahr, D. (2013) · Science · doi:10.1126/science.1238056 · Cited by 202
A systematic review of AI-driven intelligent tutoring systems (ITS) in K-12 education
Létourneau, A., Deslandes Martineau, M., Charland, P., Karran, J. A., Boasen, J., & Léger, P. M. (2025) · npj Science of Learning · doi:10.1038/s41539-025-00320-7 · Cited by 185
“Incontrast to the results of Pane et al. 21 , Kulik and Fletcher 17 , in their review of ITS, indicated that only three studies were conducted in school settings and found no real improvement in school (K-12) performance due to the deploymentofITS.”
InfluentialUpgrading Education with Technology: Insights from Experimental Research
Escueta, M., Nickow, A. J., Oreopoulos, P., & Quan, V. (2020) · Journal of Economic Literature · doi:10.1257/jel.20191507 · Cited by 205
Strengthening the research base that informs STEM instructional improvement efforts: A meta-analysis
Lynch, K., Hill, H. C., Gonzalez, K. E., & Pollard, C. (2019) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373719849044 · Cited by 160
Song, P., & Wang, X. (2020) · Asia Pacific Education Review · doi:10.1007/s12564-020-09640-2 · Cited by 205
Endogenous Stratification in Randomized Experiments
Abadie, A., Chingos, M. M., & West, M. R. (2018) · The Review of Economics and Statistics · doi:10.1162/rest_a_00732 · Cited by 185
Artificial Intelligence in Education
Holmes, W. (2020) · Springer eBooks · doi:10.1007/978-3-030-10576-1_107 · Cited by 240
Wang, H., Tlili, A., Huang, R., Cai, Z., Li, M., Cheng, Z., Yang, D., Li, M., Zhu, X., & Fei, C. (2023) · Education and Information Technologies · doi:10.1007/s10639-022-11555-x · Cited by 145
“For attitude and confidence in math learning, Pane et al. (2014) reported that ITS did not have any significant effect.”
BackgroundInfluentialThe Beginning of a Beautiful Friendship? Intelligent Tutoring Systems and MOOCs
Aleven, V., Sewall, J., Popescu, O., Xhakaj, F., Chand, D., Baker, R., Wang, Y., Siemens, G., Rosé, C. P., & Gašević, D. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-19773-9_53 · Cited by 142
“Their effectiveness in helping students learn has been well-documented [5][9].”
BackgroundExample-Tracing Tutors: Intelligent Tutor Development for Non-programmers
Aleven, V., McLaren, B. M., Sewall, J., van Velsen, M., Popescu, O., Demi, S., Ringenberg, M., & Koedinger, K. R. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-015-0088-2 · Cited by 139
“…in students’ learning (Anderson et al. 1995; Koedinger et al. 1997), a finding that, with a few exceptions, was reproduced in later classroom studies (Koedinger and Aleven 2007; Ritter et al. 2007), including one conducted by a third party with 17,000 students in 150 schools (Pane et al. 2013).”
BackgroundResultGuo, L., Wang, D., Gu, F., Li, Y., Wang, Y., & Zhou, R. (2021) · Asia Pacific Education Review · doi:10.1007/s12564-021-09697-7 · Cited by 124
“The first identified paper was the “Effectiveness of Cognitive Tutor Algebra I at Scale” (Pane et al., 2014) (burst = 4.15), which reported on a two year study of Cognitive Tutor Algebra I (CTAI) to compare the learning effects of personalized, mastery learning, blended learning, and CTAI alone.”
BackgroundInfluentialModelling Human Teaching Tactics and Strategies for Tutoring Systems: 14 Years On
du Boulay, B., & Luckin, R. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-015-0053-0 · Cited by 124
“There can be empirical observations of real students or there can be analyses of the reactions of simulated students.”
BackgroundThe Struggle to Pass Algebra: Online vs. Face-to-Face Credit Recovery for At-Risk Urban Students
Heppen, J. B., Sorensen, N., Allensworth, E., Walters, K., Rickles, J., Taylor, S. S., & Michelman, V. (2017) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2016.1168500 · Cited by 101
Designing Adaptive Instruction for Teams: a Meta-Analysis
Sottilare, R. A., Burke, C. S., Salas, E., Sinatra, A. M., Johnston, J. H., & Gilbert, S. B. (2017) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-017-0146-z · Cited by 119
“A recent review of artificial intelligence in education (AIED) meta-analyses by du Boulay (2016) noted investigations by VanLehn (2011), Ma et al. (2014), Kulik and Fletcher (2015), Steenbergen-Hu and Cooper (2013, 2014), and Pane et al. (2014).”
BackgroundHR Analytics and Artificial Intelligence-Transforming Human Resource Management
Arora, M., Prakash, A., Mittal, A., & Singh, S. (2021) · International Conference on Decision Aid Sciences and Applications (DASA) · doi:10.1109/dasa53625.2021.9682325 · Cited by 155
Performance Comparison of an AI-Based Adaptive Learning System in China
Cui, W., Xue, Z., & Thai, K.-P. (2018) · 2018 Chinese Automation Congress (CAC) · doi:10.1109/cac.2018.8623327 · Cited by 93
ChatGPT and Generative AI Technology: A Mixed Bag of Concerns and New Opportunities
Lambert, J., & Stevens, M. (2024) · Computers in the Schools · doi:10.1080/07380569.2023.2256710 · Cited by 93
“PL, including approaches that utilize adaptive technologies, have been shown to increase student outcomes, and the longer students experience PL, the greater their achievement growth tends to be (Pane et al. 2014, 2017).”
Successful learning with multiple graphical representations and self-explanation prompts.
Rau, M. A., Aleven, V., & Rummel, N. (2015) · Journal of Educational Psychology · doi:10.1037/a0037211 · Cited by 89
Improving Sensor-Free Affect Detection Using Deep Learning
Botelho, A. F., Baker, R. S., & Heffernan, N. T. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-61425-0_4 · Cited by 111
Selecting Optimal Subgroups for Treatment Using Many Covariates
VanderWeele, T. J., Luedtke, A. R., van der Laan, M. J., & Kessler, R. C. (2019) · Epidemiology · doi:10.1097/ede.0000000000000991 · Cited by 104
Adoption of Artificial Intelligence in Schools: Unveiling Factors Influencing Teachers’ Engagement
Cukurova, M., Miao, X., & Brooker, R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-36272-9_13 · Cited by 142
Van Schoors, R., Elen, J., Raes, A., & Depaepe, F. (2021) · British Journal of Educational Technology · doi:10.1111/bjet.13148 · Cited by 87
Bang, H. J., Li, L., & Flynn, K. (2023) · Early Childhood Education Journal · doi:10.1007/s10643-022-01332-3 · Cited by 91
“However, many studies report minimal or mixed effects for educational technology interventions in mathematics (Campuzano et al., 2009; Pane et al., 2014; Rutherford et al., 2014; Schenke et al., 2014; Slavin et al., 2017; Steenbergen-Hu & Cooper, 2013).”
BackgroundDoss, C., Fahle, E. M., Loeb, S., & York, B. N. (2019) · Journal of Human Resources · doi:10.3368/jhr.54.3.0317-8637r · Cited by 84
Pai, K.-C., Kuo, B.-C., Liao, C.-H., & Liu, Y.-M. (2021) · Educational Psychology · doi:10.1080/01443410.2020.1731427 · Cited by 83
Escape from the Skinner Box: The case for contemporary intelligent learning environments
du Boulay, B. (2019) · British Journal of Educational Technology · doi:10.1111/bjet.12860 · Cited by 79
How Mastery Learning Works at Scale
Ritter, S., Yudelson, M., Fancsali, S. E., & Berman, S. R. (2016) · Proceedings of the Third (2016) ACM Conference on Learning @ Scale · doi:10.1145/2876034.2876039 · Cited by 77
“Mastery learning programs have a long history of success (Guskey and Gates, 1986; Kulik, Kulik & Bangert-Drowns, 1990), predating computer-based instruction.”
BackgroundResultAI and the Future of Learning: Expert Panel Report
Roschelle, J., Lester, J., & Fusco, J. (2020) · doi:10.51388/20.500.12265/106 · Cited by 76
“One well-known example is the Cognitive Tutor for Algebra I, which has been successfully commercialized and tested at scale (Pane et al., 2014).”
MethodThe Effect of Metacognitive Scaffolding for Learning by Teaching a Teachable Agent
Matsuda, N., Weng, W., & Wall, N. G. (2020) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-019-00190-2 · Cited by 80
“This finding that cognitive scaffolding does not facilitate tutor learning is somewhat surprising given a well know effect of cognitive tutoring (Pane et al. 2014; Ritter et al. 2007).”
BackgroundGraesser, A. C., Sabatini, J., & Li, H. (2021) · Annual Review of Psychology · doi:10.1146/annurev-psych-020821-113042 · Cited by 114
Handbook on Personalized Learning for States, Districts, and Schools
(2016) · Emerald Group Publishing Limited eBooks · doi:10.1108/978-1-68123-589-9 · Cited by 71
An Interaction Design for Machine Teaching to Develop AI Tutors
Weitekamp, D. P., Harpstead, E., & Koedinger, K. R. (2020) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3313831.3376226 · Cited by 78
Nye, B. D., Pavlik, P. I., Windsor, A., Olney, A. M., Hajeer, M., & Hu, X. (2018) · International Journal of STEM Education · doi:10.1186/s40594-018-0109-4 · Cited by 55
“A second study reported learning gains comparable to the cognitive tutor (Sabo et al. 2013), which has demonstrated statistically significant gains on large-scale randomized controlled studies (Pane et al. 2014).”
ResultImplementing an adaptive intelligent tutoring system as an instructional supplement
Phillips, A., Pane, J. F., Reumann-Moore, R., & Shenbanjo, O. (2020) · Educational Technology Research and Development · doi:10.1007/s11423-020-09745-w · Cited by 68
Self-citationIs the doer effect a causal relationship? how can we tell and why it's important
Koedinger, K. R., McLaughlin, E. A., Jia, J. Z., & Bier, N. L. (2016) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2883851.2883957 · Cited by 93
Thomas, D. R., Lin, J., Gatz, E., Gurung, A., Gupta, S., Norberg, K., Fancsali, S. E., Aleven, V. A. W. M. M., Branstetter, L., Brunskill, E., et al. (2024) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/3636555.3636896 · Cited by 91
“High-dosage human tutoring has been shown to double student learning [25, 26].”
BackgroundPersonalization of Instruction: Design Dimensions and Implications for Cognition
Walkington, C., & Bernacki, M. L. (2018) · The Journal of Experimental Education · doi:10.1080/00220973.2017.1380590 · Cited by 60
Pardos, Z. A., Tang, M., Anastasopoulos, I., Sheel, S. K., & Zhang, E. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3581574 · Cited by 55
Artificial Intelligence in Education and Ethics
du Boulay, B. (2023) · Handbook of Open, Distance and Digital Education · doi:10.1007/978-981-19-2080-6_6 · Cited by 89
Lechuga, C. G., & Doroudi, S. (2022) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-022-00309-y · Cited by 65
“While some research studies have shown promise in this regard (Koedinger et al., 2010; Pane et al., 2014), historically, the effectiveness of ITSs has been mixed and remains in question (Fang et al., 2019; Kulik & Fletcher, 2016; Reich, 2020; Sun et al., 2021) as ITSs and blended learning…”
An Interview Reflection on “Intelligent Tutoring Goes to School in the Big City”
Koedinger, K. R., & Aleven, V. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-015-0082-8 · Cited by 51
“The rigorous randomized field trial evaluations that have since been done (e.g., Ritter et al. 2007a, b; Pane et al. 2014) were completely appropriate (except for the limitation to a single post assessment) and have, for the most part, validated the impact of the approach.”
BackgroundEducational Game and Intelligent Tutoring System: A Classroom Study and Comparative Design Analysis
Long, Y., & Aleven, V. A. W. M. M. (2017) · ACM Transactions on Computer-Human Interaction · doi:10.1145/3057889 · Cited by 63
“A substantial amount of scientific evidence shows that ITSs can enhance learning, compared to business-as-usual in schools and other forms of instruction [Kulik and Fletcher 2015; Ma et al. 2014; Pane et al. 2013; Steenbergen-Hu and Cooper 2013a; Steenbergen-Hu and Cooper 2013b; VanLehn 2011].”
BackgroundMethodInfluentialMutambik, I. (2024) · Sustainability · doi:10.3390/su16145970 · Cited by 91
Decision Tree Modeling of Wheel- Spinning and Productive Persistence in Skill Builders
Kai, S., Almeda, M. V., Baker, R. S., Heffernan, C., & Heffernan, N. T. (2018) · Journal of Educational Data Mining · doi:10.5281/zenodo.3344810 · Cited by 48
Recent Meta-reviews and Meta–analyses of AIED Systems
du Boulay, B. (2016) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-015-0060-1 · Cited by 45
“Finally in a large-scale study in the USA of the Cognitive Tutors, (Pane et al. 2014) found only limited evidence of the relative effectiveness of these tutors over conventional teaching, though we note that how the tutors were actually used in the classrooms Int J Artif Intell Educ (2016)…”
InfluentialWilliams, R. T., Citkowicz, M., Miller, D. I., Lindsay, J., & Walters, K. (2022) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2021.2009072 · Cited by 59
“For instance, Cognitive TutorVR is a widely used and studied algebra intervention (Pane et al., 2014).”
Khachatryan, G. A., Romashov, A. V., Khachatryan, A. R., Gaudino, S., Khachatryan, J. M., Guarian, K. R., & Yufa, N. V. (2014) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-014-0019-7 · Cited by 49
“…in algebra have shown at-scale effects of between 0 and 0.3 standard deviations on standardized tests (Ritter et al., 2007; Campuzano et al., 2009; Pane et al., 2013) and 0.7–1.0 standard deviations on more proximal instruments (VanLehn, 2011; Anderson et al., 1995; Koedinger et al., 1997), and…”
BackgroundInfluentialToward Asset-based Instruction and Assessment in Artificial Intelligence in Education
Ocumpaugh, J. L., Roscoe, R. D., Baker, R. S., Hutt, S. J., & Aguilar, S. J. (2024) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-023-00382-x · Cited by 57
“For example, several large-scale studies of intelligent tutoring systems (ITS) have documented their efficacy (Feng et al., 2023; Pane et al., 2014; Roschelle et al., 2020).”
Conceptualizing Essential Components of Effective High Schools
Preston, C., Goldring, E. B., Guthrie, J. E., Ramsey, R., & Huff, J. (2016) · Leadership and Policy in Schools · doi:10.1080/15700763.2016.1205198 · Cited by 66
Domain-General Tutor Authoring with Apprentice Learner Models
MacLellan, C. J., & Koedinger, K. R. (2020) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-020-00214-2 · Cited by 62
Educational Equity Through Combined Human-AI Personalization: A Propensity Matching Evaluation
Chine, D. R., Brentley, C., Thomas-Browne, C., Richey, J. E., Gül, A., Carvalho, P. F., Branstetter, L., & Koedinger, K. R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-11644-5_30 · Cited by 44
Can Technology Help Promote Equality of Educational Opportunities?
Jacob, B. A., Berger, D., Hart, C. M. D., & Loeb, S. (2016) · RSF The Russell Sage Foundation Journal of the Social Sciences · doi:10.7758/rsf.2016.2.5.12 · Cited by 52
Culturally Responsive Personalized Learning: Recommendations for a Working Definition and Framework
Ober, T. M., Lehman, B. A., Gooch, R. M., Oluwalana, O., Solyst, J., Phelps, G., & Hamilton, L. S. (2023) · ETS Research Report Series · doi:10.1002/ets2.12372 · Cited by 66
Doss, C., Fahle, E. M., Loeb, S., & York, B. N. (2018) · National Bureau of Economic Research · doi:10.3386/w24450 · Cited by 40
SPACLE: investigating learning across virtual and physical spaces using spatial replays
Holstein, K., McLaren, B. M., & Aleven, V. (2017) · Proceedings of the Seventh International Learning Analytics & Knowledge Conference · doi:10.1145/3027385.3027450 · Cited by 34
“For example, a large-scale, two-year evaluation study of Carnegie Learning’s Algebra I tutor suggested that variability in the out-of-software support teachers provided to students may have been at least partly responsible for inconsistent results across evaluation years [29].”
ResultLee, Y. (2024) · Education and Information Technologies · doi:10.1007/s10639-024-13129-5 · Cited by 58
“These applications use computational algorithms to assess the performance of learners and provide targeted interventions and resources that can improve learning outcomes (Pane et al., 2014).”
The Effect of AI ‐Based Systems on Mathematics Achievement in Rural Context: A Quantitative Study
Khazanchi, R., Mitri, D. D., & Drachsler, H. (2024) · Journal of Computer Assisted Learning · doi:10.1111/jcal.13098 · Cited by 63
Artificial Intelligence in Technology-Enhanced Assessment: A Survey of Machine Learning
Caspari-Sadeghi, S. (2022) · Journal of Educational Technology Systems · doi:10.1177/00472395221138791 · Cited by 60
“Cognitive Tutor (Pane et al., 2014) is an ITS which uses BKT for students ’ modeling.”
MethodExamining the implementation of technology-based blended algebra I curriculum at scale
Karam, R., Pane, J. F., Griffin, B. A., Robyn, A., Phillips, A., & Daugherty, L. (2016) · Educational Technology Research and Development · doi:10.1007/s11423-016-9498-6 · Cited by 35
“…which compared the performance of students in mathematics in both the CTAI (treatment) and non-CTAI (control) classrooms found that the curriculum did not have significant effects in the first year of implementation, but had a significant positive effect in the second year (Pane et al. 2014).”
BackgroundSelf-citationRealizing the full potential of individualizing learning
Arvidsson, T. S., & Kuhn, D. (2021) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2021.101960 · Cited by 36
Gaur, A. S., Sharan, H. O., & Kumar, R. (2024) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-2964-1.ch003 · Cited by 33
Matching and Weighting With Functions of Error-Prone Covariates for Causal Inference
Lockwood, J. R., & McCaffrey, D. F. (2015) · Journal of the American Statistical Association · doi:10.1080/01621459.2015.1122601 · Cited by 40
“Kuroki and Pearl (2014) develop a theoretical framework for “effect restoration” in graphical models where potential confounders are observed by proxy variables and models relating outcomes to latent covariates are assumed.”
BackgroundSelf-citationZheng, G., Fancsali, S. E., RITTER, S., & Berman, S. (2019) · Journal of Learning Analytics · doi:10.18608/jla.2019.62.11 · Cited by 33
Artificial Intelligence in Education and Ethics
du Boulay, B. (2022) · Handbook of Open, Distance and Digital Education · doi:10.1007/978-981-19-0351-9_6-2 · Cited by 33
A Dashboard to Support Teachers During Students’ Self-paced AI-Supported Problem-Solving Practice
Aleven, V., Blankestijn, J., Lawrence, L., Nagashima, T., & Taatgen, N. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-16290-9_2 · Cited by 28
“The current work targets contexts in which students engage in self-paced, personalized learning with AI-based tutoring software, by now a common occurrence in K-12 in the US and elsewhere [16, 23].”
BackgroundSvane, R. P., Willemsen, M. M., Bleses, D., Krøjgaard, P., Verner, M., & Nielsen, H. S. (2023) · Frontiers in Education · doi:10.3389/feduc.2023.1229849 · Cited by 27
Conversational Agents in Support for Collaborative Learning in MOOCs: An Analytical Review
Caballé, S., & Conesa, J. (2018) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-319-98557-2_35 · Cited by 32
Predicting Standardized Test Scores from Cognitive Tutor Interactions.
RITTER, S., Joshi, A., Fancsali, S. E., & Nixon, T. M. (2013) · Educational Data Mining · Cited by 35
PISA 2022 Mathematics Framework
OECD (2023) · Programme for international student assessment/Internationale Schulleistungsstudie · doi:10.1787/7ea9ee19-en · Cited by 111
Towards the Future of AI-Augmented Human Tutoring in Math Learning
Aleven, V. A. W. M. M., Baraniuk, R. G., Brunskill, E., Crossley, S. A., Demszky, D., Fancsali, S. E., Gupta, S., Koedinger, K. R., Piech, C., RITTER, S., et al. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-36336-8_3 · Cited by 30
Intelligent Learning Environments: Design, Usage and Analytics for Future Schools
Mavrikis, M., & Holmes, W. (2019) · Perspectives on rethinking and reforming education · doi:10.1007/978-981-13-9439-3_4 · Cited by 26
Ayedoun, E., Hayashi, Y., & Seta, K. (2020) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2020.2989776 · Cited by 24
“Despite the potential of such systems, they have often been limited in their ability to produce skills that are transferable to the real world [5], [6].”
BackgroundTransforming Education: Adaptive Learning, AI, and Online Platforms for Personalization
Rizvi, I., Bose, C., & Tripathi, N. (2025) · Springer eBooks · doi:10.1007/978-981-96-1721-0_4 · Cited by 49
Bush, J. B. (2021) · British Journal of Educational Technology · doi:10.1111/bjet.13139 · Cited by 53
THE ROLE AND CHALLENGES OF ARTIFICIAL INTELLIGENCE IN INFORMATION TECHNOLOGY EDUCATION
Bai, X. (2024) · Pacific International Journal · doi:10.55014/pij.v7i1.524 · Cited by 39
Fancsali, S. E., & RITTER, S. (2014) · International Conference on Learning Analytics and Knowledge (LAK) · doi:10.1145/2567574.2567615 · Cited by 24
“The educational effectiveness of adapting to cognitive factors in this manner is well established [6][9], and many recent efforts have focused on enhancing both the educational and motivational effects of the software by personalizing instruction based on non-cognitive factors [2][4][12].”
BackgroundCompetency-Based Education in Three Pilot Programs: Examining Implementation and Outcomes
Steele, J., Lewis, M. W., Santibáñez, L., Faxon-Mills, S., Rudnick, M., Stecher, B. M., & Hamilton, L. S. (2014) · RAND Corporation eBooks · doi:10.7249/rr732 · Cited by 34
Rutherford, T. A., Duck, K. D., Rosenberg, J. M., & Patt, R. (2021) · Journal of Research on Technology in Education · doi:10.1080/15391523.2021.1920520 · Cited by 28
“In particular, use of tutorial and practice programs, such as DreamBox, ASSISTments, and Cognitive Tutor, have been linked with improved mathematics test scores outside of the platform (Pane et al., 2014; Roschelle et al., 2016; Wang & Woodworth, 2011).”
MethodLeite, W. L., Cetin‐Berber, D. D., Huggins‐Manley, A. C., Collier, Z. K., & Beal, C. R. (2019) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12360 · Cited by 23
Wang, S., Christensen, C., Xu, Y., Cui, W., Tong, R., & Shear, L. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.01803 · Cited by 32
Metadata
- Authors
- Pane, J. F., Griffin, B. A., McCaffrey, D. F., & Karam, R.
- Affiliations
- RAND Corporation (US)
- Educational Testing Service (US)
- Venue
- Educational Evaluation and Policy Analysis
- Published
- 2014-06
- Volume
- 36
- Issue
- 2
- Pages
- 127-144
- ISSN
- 0162-3737 (ISSN-L)
- 1935-1062
- OpenAlex
- W2113386642
- Cited by
- 317
- References
- 32
- Citations by year
- 2026: 34 · 2025: 50 · 2024: 44 · 2023: 34 · 2022: 22 · 2021: 20 · 2020: 29 · 2019: 16 · 2018: 16 · 2017: 11 · 2016: 18 · 2015: 11 · 2014: 10
- FWCI
- 15.12
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Educational Evaluation and Policy Analysis (publishedVersion)
- Funders
- Educational Testing Service
- Institute of Education Sciences
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Education › School Choice and Performance
- Social Sciences › Education › Mathematics Education and Teaching Techniques
- Keywords
- algebra instruction, technology-based curriculum, personalized learning, learning mastery, blended learning, curriculum adoption, junior high school, high schools, academic performance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.