Effectiveness of Cognitive Tutor Algebra I at Scale
The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
Abstract
This article examines the effectiveness of a technology-based algebra curriculum in a wide variety of middle schools and high schools in seven states. Participating schools were matched into similar pairs and randomly assigned to either continue with the current algebra curriculum for 2 years or to adopt Cognitive Tutor Algebra I (CTAI), which uses a personalized, mastery-learning, blended-learning approach. Schools assigned to implement CTAI did so under conditions similar to schools that independently adopt it. Analysis of posttest outcomes on an algebra proficiency exam finds no effects in the first year of implementation, but finds evidence in support of positive effects in the second year. The estimated effect is statistically significant for high schools but not for middle schools; in both cases, the magnitude is sufficient to improve the median student’s performance by approximately eight percentile points.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“AI tools can enhance learning outcomes by providing personalised instruction and immediate feedback, thus supporting skill acquisition and knowledge retention [2,3].”
From AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking (2025) · cites this paper as [2]
In the citing paper
“The literature suggests that AI can enhance learning outcomes when used appropriately [2,3].”
From AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking (2025) · cites this paper as [2]
Citations
Cites
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1994) · doi:10.1201/9780429246593 · Cited by 40,500
An Introduction to the Bootstrap
Efron, B., & Tibshirani, R. J. (1993) · doi:10.1007/978-1-4899-4541-9 · Cited by 21,230
Multiple Imputation for Nonresponse in Surveys
Rubin, D. B. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316696 · Cited by 20,701
National Council of Teachers of Mathematics
(1979) · The Mathematics Teacher · doi:10.5951/mt.72.8.0637 · Cited by 8,499
(2009) · US Department of Education · Cited by 3,654
Gill, J. (2005) · Encyclopedia of Social Measurement · doi:10.1016/b0-12-369398-5/00166-3 · Cited by 3,114
TIMSS 2011 International Results in Mathematics.
Mullis, I. V. S., Martín, M. O., Foy, P., & Arora, A. (2012) · Cited by 2,342
Measurement Error in Nonlinear Models
Carroll, R. J., Ruppert, D., Stefanski, L. A., & Crainiceanu, C. M. (2006) · Chapman & Hall/CRC eBooks · doi:10.1201/9781420010138 · Cited by 2,116
Cognitive Tutors: Lessons Learned
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995) · Journal of the Learning Sciences · doi:10.1207/s15327809jls0402_2 · Cited by 1,982 · cited 3×
“In addition, an experiment testing the efficacy of the related Cognitive Tutor Geometry curriculum in Grades 9 to 12 found a significant negative effect on student achievement (Pane, McCaffrey, Slaughter, Steele, & Ikemoto, 2010).”
MethodMeasurement error in nonlinear models: a modern perspective
Carroll, R. J. (2006) · Cited by 1,374
Miscellanea. Small-sample degrees of freedom with multiple imputation
Barnard, J. (1999) · Biometrika · doi:10.1093/biomet/86.4.948 · Cited by 829
Fennema, E., & Sherman, J. A. (1976) · Journal for Research in Mathematics Education · doi:10.5951/jresematheduc.7.5.0324 · Cited by 772
Wenglinsky, H. H. (1998) · Cited by 665
(1995) · Bayesian Data Analysis · doi:10.1201/9780429258411-25 · Cited by 553
Using Curriculum‐Based Measurement to Improve Student Achievement: Review of Research
Stecker, P. M., Fuchs, L. S., & Fuchs, D. (2005) · Psychology in the Schools · doi:10.1002/pits.20113 · Cited by 506
Cognitive Tutor: Applied research in mathematics education
Ritter, S., Anderson, J. R., Koedinger, K. R., & Corbett, A. (2007) · Psychonomic Bulletin & Review · doi:10.3758/bf03194060 · Cited by 483
Schacter, J. (1999) · Cited by 417
The Nation's Report Card: Mathematics, 2000.
Braswell, J. S., Lutkus, A. D., Grigg, W. S., Santapau, S. L., Tay-Lim, B., & Johnson, M. (2001) · Cited by 408
(2012) · TigerPrints (Clemson University) · Cited by 402
The prognostic analogue of the propensity score
Hansen, B. B. (2008) · Biometrika · doi:10.1093/biomet/asn004 · Cited by 371
STRATIFICATION BY A MULTIVARIATE CONFOUNDER SCORE
MIETTINEN, O. S. (1976) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a112339 · Cited by 348
Understanding the Implications of Online Learning for Educational Productivity
Bakia, M., Shear, L., Toyama, Y., & Lasseter, A. (2012) · Cited by 328
Organizing Instruction and Study to Improve Student Learning
Pashler, H., Bain, P. M., Bottge, B. A., Graesser, A., Koedinger, K., McDaniel, M., & Metcalfe, J. (2007) · PsycEXTRA Dataset · doi:10.1037/e607972011-001 · Cited by 306
Hiebert, J., Stigler, J. W., Jacobs, J. K., Givvin, K. B., Garnier, H., Smith, M., Hollingsworth, H., Manaster, A., Wearne, D., & Gallimore, R. (2005) · Educational Evaluation and Policy Analysis · doi:10.3102/01623737027002111 · Cited by 282
Hall, D. B. (2008) · Quarterly Publications of the American Statistical Association · Cited by 196
Arbogast, P. G., & Ray, W. A. (2011) · American Journal of Epidemiology · doi:10.1093/aje/kwr143 · Cited by 138
Campuzano, L., Dynarski, M., Agodini, R., & Rall, K. (2009) · National Center for Education Evaluation and Regional Assistance · Cited by 125
An experimental study of the effects of Cognitive Tutor® Algebra I on student knowledge and attitude
Morgan, P., & RITTER, S. (2002) · Cited by 56
An Experiment to Evaluate the Efficacy of Cognitive Tutor Geometry
Pane, J. F., McCaffrey, D. F., Slaughter, M. E., Steele, J. L., & Ikemoto, G. S. (2010) · Journal of Research on Educational Effectiveness · doi:10.1080/19345741003681189 · Cited by 39
Self-citationWhat Evidence Matters? A randomized field trial of Cognitive Tutor Algebra I
RITTER, S., Kulikowich, J. M., Lei, P., McGuire, C. L., & Morgan, P. (2007) · International Conference on Computers in Education · Cited by 36
Cabalo, J. V., Ma, B., & Jaciw, A. P. (2007) · Cited by 14
Analysis of Costs in an Algebra I Curriculum Effectiveness Study
Daugherty, L., Phillips, A., Pane, J. F., & Karam, R. (2012) · Cited by 3
Self-citation
Research works printed but not linked: 10
- American College Testing, 2012, The reality of college readiness: National
- Fennema E., 1976, JSAS Catalog of Selected Documents of Psychology, 6, 31
- Greene W. H., 2003, Econometric analysis
- Karam R., 2013, Evaluating Cognitive Tutor Algebra 1 Curricula at Scale: Focus on implementation
- Koedinger K. R., 2000, Carnegie Learning’s Cognitive Tutor: Summary research results
- Lockwood J. R., Value-added modeling and growth modeling with particular application to teacher and school effectiveness
- Ritter S., 2007, Supporting Learning Flow Through Integrative Technologies, 162, 13
- U.S. Department of Education, 2011, What Works Clearinghouse: Procedures and standards handbook
- What Works Clearinghouse, 2004, WWC Intervention Report: Cognitive Tutor®
- What Works Clearinghouse, 2013, WWC Intervention Report: Carnegie Learning Curricula and Cognitive Tutor®
Cited by
Conducting Educational Design Research
McKenney, S., & Reeves, T. C. (2018) · Routledge eBooks · doi:10.4324/9781315105642 · Cited by 1,354
AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking
Gerlich, M. (2025) · doi:10.2139/ssrn.5082524 · Cited by 1,238
AI Tools in Society: Impacts on Cognitive Offloading and the Future of Critical Thinking
Gerlich, M. (2025) · Societies · doi:10.3390/soc15010006 · Cited by 1,248
State of the art and practice in AI in education
Holmes, W., & Tuomi, I. (2022) · European Journal of Education · doi:10.1111/ejed.12533 · Cited by 1,093
Effectiveness of Intelligent Tutoring Systems: A Meta-Analytic Review
Kulik, J. A., & Fletcher, J. D. (2016) · Review of Educational Research · doi:10.3102/0034654315581420 · Cited by 930
Metadata
- Authors
- Pane, J. F., Griffin, B. A., McCaffrey, D. F., & Karam, R.
- Affiliations
- RAND Corporation (US)
- Educational Testing Service (US)
- Venue
- Educational Evaluation and Policy Analysis
- Published
- 2014-06
- Volume
- 36
- Issue
- 2
- Pages
- 127-144
- ISSN
- 0162-3737 (ISSN-L)
- 1935-1062
- OpenAlex
- W2113386642
- Cited by
- 317
- References
- 32
- Citations by year
- 2026: 34 · 2025: 50 · 2024: 44 · 2023: 34 · 2022: 22 · 2021: 20 · 2020: 29 · 2019: 16 · 2018: 16 · 2017: 11 · 2016: 18 · 2015: 11 · 2014: 10
- FWCI
- 15.12
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Educational Evaluation and Policy Analysis (publishedVersion)
- Funders
- Educational Testing Service
- Institute of Education Sciences
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Intelligent Tutoring Systems and Adaptive Learning
- Social Sciences › Education › School Choice and Performance
- Social Sciences › Education › Mathematics Education and Teaching Techniques
- Keywords
- algebra instruction, technology-based curriculum, personalized learning, learning mastery, blended learning, curriculum adoption, junior high school, high schools, academic performance
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.