Example-Tracing Tutors: Intelligent Tutor Development for Non-programmers
Open accessThe DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
In 2009, we reported on a new Intelligent Tutoring Systems (ITS) technology, example-tracing tutors, that can be built without programming using the Cognitive Tutor Authoring Tools (CTAT). Creating example-tracing tutors was shown to be 4–8 times as cost-effective as estimates for ITS development from the literature. Since 2009, CTAT and its associated learning management system, the Tutorshop, have been extended and have been used for both research and real-world instruction. As evidence that example-tracing tutors are an effective and mature ITS paradigm, CTAT-built tutors have been used by approximately 44,000 students and account for 40 % of the data sets in DataShop , a large open repository for educational technology data sets. We review 18 example-tracing tutors built since 2009, which have been shown to be effective in helping students learn in real educational settings, often with large pre/post effect sizes. These tutors support a variety of pedagogical approaches, beyond step-based problem solving, including collaborative learning, educational games, and guided invention activities. CTAT and other ITS authoring tools illustrate that non-programmer approaches to building ITS are viable and useful and will likely play a key role in making ITS widespread.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Knowledge tracing: Modeling the acquisition of procedural knowledge
Corbett, A. T., & Anderson, J. R. (1995) · User Modelling and User-Adapted Interaction · doi:10.1007/bf01099821 · Cited by 2,324 · cited 5×
“They also meet the second factor: As mentioned, example-tracing tutors and the Tutorshop support individualized problem selection based on Bayesian Knowledge Tracing (Corbett and Anderson 1995; Anderson et al. 1989; Corbett et al. 2000; VanLehn 2011).”
BackgroundMethodInfluentialVanLEHN, K. (2011) · Educational Psychologist · doi:10.1080/00461520.2011.611369 · Cited by 2,323
Cognitive Tutors: Lessons Learned
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995) · Journal of the Learning Sciences · doi:10.1207/s15327809jls0402_2 · Cited by 1,982 · cited 7×
“Example-tracing tutors”
BackgroundMethodResultInfluentialSelf-citationKalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_4 · Cited by 1,927
Paas, F. G. W. C., & Van Merriënboer, J. J. G. (1994) · Journal of Educational Psychology · doi:10.1037/0022-0663.86.1.122 · Cited by 1,198
Expertise Reversal Effect and Its Implications for Learner-Tailored Instruction
Kalyuga, S. (2007) · Educational Psychology Review · doi:10.1007/s10648-007-9054-3 · Cited by 1,056 · cited 1×
“For example, it would be useful to test whether the typical expertise reversal effect (studying examples is more effective early on, problem solving is more effective later on) (Kalyuga 2007; Kalyuga et al. 2003) occurs in the context of ITS.”
BackgroundPaas, F., Renkl, A., & Sweller, J. (2004) · Instructional Science · doi:10.1023/b:truc.0000021806.17516.d0 · Cited by 1,054
Koedinger, K. R., Corbett, A. T., & Perfetti, C. (2012) · Cognitive Science · doi:10.1111/j.1551-6709.2012.01245.x · Cited by 772 · cited 9×
“The work is grounded in the Knowledge-Learning-Instruction (KLI) framework (Koedinger et al. 2012), which links cognitive theory and instructional design.”
BackgroundMethodInfluentialSelf-citationSchwartz, D. L., & Martin, T. (2004) · Cognition and Instruction · doi:10.1207/s1532690xci2202_1 · Cited by 763
Van der Kleij, F. M., Feskens, R. C. W., & Eggen, T. J. H. M. (2015) · Review of Educational Research · doi:10.3102/0034654314564881 · Cited by 714 · cited 1×
“Step-level feedback is strongly supported in the empirical ITS and education literature as enhancing student learning (Arroyo et al. 2000; Kleij et al. 2015; McKendree 1990; VanLehn 2011).”
Background
Cited by
Artificial Intelligence and Its Role in Education
Ahmad, S. F., Rahmat, M. K., Mubarik, M. S., Alam, M. M., & Hyder, S. I. (2021) · Sustainability · doi:10.3390/su132212902 · Cited by 336
A Conceptual Framework for Human–AI Hybrid Adaptivity in Education
Holstein, K., Aleven, V., & Rummel, N. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-52237-7_20 · Cited by 168
“Authoring tools have been developed to support non-programmer authoring, but further research is needed to support easy authoring in everyday educational settings (e.g., by teachers or students) [1, 29, 37, 39].”
MethodSelf-citationEnhancing learning outcomes through self-regulated learning support with an Open Learner Model
Long, Y., & Aleven, V. (2017) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-016-9186-6 · Cited by 95
Self-citationSmart Environment for Adaptive Learning of Cybersecurity Skills
Vykopal, J., Seda, P., Švábenský, V., & Čeleda, P. (2023) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2022.3216345 · Cited by 48
Supporting skill integration in an intelligent tutoring system for code tracing
Huang, Y., Brusilovsky, P., Guerra, J., Koedinger, K., & Schunn, C. (2023) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12757 · Cited by 32
Self-citationA Dashboard to Support Teachers During Students’ Self-paced AI-Supported Problem-Solving Practice
Aleven, V., Blankestijn, J., Lawrence, L., Nagashima, T., & Taatgen, N. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-16290-9_2 · Cited by 28
Self-citationRodriguez-Gil, L., Garcia-Zubia, J., Orduna, P., Villar-Martinez, A., & Lopez-De-Ipina, D. (2019) · IEEE Access · doi:10.1109/access.2018.2883500 · Cited by 20
“CTAT’s example-tracing tutor creation requires no JESS knowledge and is simpler [37].”
Treceño-Fernández, D., Calabia-del-Campo, J., Bote-Lorenzo, M. L., Gómez-Sánchez, E., Luis-García, R. D., & Alberola-López, C. (2020) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2020.105634 · Cited by 12
Proceedings of the Eleventh ACM Conference on Learning @ Scale
(2024) · doi:10.1145/3657604 · Cited by 11
“Frame-works for intelligent tutoring include constraint-based tutoring, which uses constraint-based modeling (CBM) to specify domain principles that every correct solution must follow [18], and example-tracing tutoring, which uses generalized traces of problem-solving behavior [3].”
Background
Metadata
- Authors
- Aleven, V., McLaren, B. M., Sewall, J., van Velsen, M., Popescu, O., Demi, S., Ringenberg, M., & Koedinger, K. R.
- Venue
- International Journal of Artificial Intelligence in Education
- Published
- 2016-02-10
- Volume
- 26
- Issue
- 1
- Pages
- 224-269
- ISSN
- 1560-4292 (ISSN-L)
- 1560-4306
- OpenAlex
- W2261910606
- Cited by
- 139
- References
- 23
- FWCI
- 9.13
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.