Example-Tracing Tutors: Intelligent Tutor Development for Non-programmers
Open accessThe DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
In 2009, we reported on a new Intelligent Tutoring Systems (ITS) technology, example-tracing tutors, that can be built without programming using the Cognitive Tutor Authoring Tools (CTAT). Creating example-tracing tutors was shown to be 4–8 times as cost-effective as estimates for ITS development from the literature. Since 2009, CTAT and its associated learning management system, the Tutorshop, have been extended and have been used for both research and real-world instruction. As evidence that example-tracing tutors are an effective and mature ITS paradigm, CTAT-built tutors have been used by approximately 44,000 students and account for 40 % of the data sets in DataShop , a large open repository for educational technology data sets. We review 18 example-tracing tutors built since 2009, which have been shown to be effective in helping students learn in real educational settings, often with large pre/post effect sizes. These tutors support a variety of pedagogical approaches, beyond step-based problem solving, including collaborative learning, educational games, and guided invention activities. CTAT and other ITS authoring tools illustrate that non-programmer approaches to building ITS are viable and useful and will likely play a key role in making ITS widespread.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Knowledge tracing: Modeling the acquisition of procedural knowledge
Corbett, A. T., & Anderson, J. R. (1995) · User Modelling and User-Adapted Interaction · doi:10.1007/bf01099821 · Cited by 2,324 · cited 5×
“They also meet the second factor: As mentioned, example-tracing tutors and the Tutorshop support individualized problem selection based on Bayesian Knowledge Tracing (Corbett and Anderson 1995; Anderson et al. 1989; Corbett et al. 2000; VanLehn 2011).”
BackgroundMethodInfluentialVanLEHN, K. (2011) · Educational Psychologist · doi:10.1080/00461520.2011.611369 · Cited by 2,323
Cognitive Tutors: Lessons Learned
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995) · Journal of the Learning Sciences · doi:10.1207/s15327809jls0402_2 · Cited by 1,982 · cited 7×
“Example-tracing tutors”
BackgroundMethodResultInfluentialSelf-citationKalyuga, S., Ayres, P., Chandler, P., & Sweller, J. (2003) · Educational Psychologist · doi:10.1207/s15326985ep3801_4 · Cited by 1,927
Paas, F. G. W. C., & Van Merriënboer, J. J. G. (1994) · Journal of Educational Psychology · doi:10.1037/0022-0663.86.1.122 · Cited by 1,198
Expertise Reversal Effect and Its Implications for Learner-Tailored Instruction
Kalyuga, S. (2007) · Educational Psychology Review · doi:10.1007/s10648-007-9054-3 · Cited by 1,056 · cited 1×
“For example, it would be useful to test whether the typical expertise reversal effect (studying examples is more effective early on, problem solving is more effective later on) (Kalyuga 2007; Kalyuga et al. 2003) occurs in the context of ITS.”
BackgroundPaas, F., Renkl, A., & Sweller, J. (2004) · Instructional Science · doi:10.1023/b:truc.0000021806.17516.d0 · Cited by 1,054
Koedinger, K. R., Corbett, A. T., & Perfetti, C. (2012) · Cognitive Science · doi:10.1111/j.1551-6709.2012.01245.x · Cited by 772 · cited 9×
“The work is grounded in the Knowledge-Learning-Instruction (KLI) framework (Koedinger et al. 2012), which links cognitive theory and instructional design.”
BackgroundMethodInfluentialSelf-citationSchwartz, D. L., & Martin, T. (2004) · Cognition and Instruction · doi:10.1207/s1532690xci2202_1 · Cited by 763
Van der Kleij, F. M., Feskens, R. C. W., & Eggen, T. J. H. M. (2015) · Review of Educational Research · doi:10.3102/0034654314564881 · Cited by 714 · cited 1×
“Step-level feedback is strongly supported in the empirical ITS and education literature as enhancing student learning (Arroyo et al. 2000; Kleij et al. 2015; McKendree 1990; VanLehn 2011).”
BackgroundExploring the Assistance Dilemma in Experiments with Cognitive Tutors
Koedinger, K. R., & Aleven, V. (2007) · Educational Psychology Review · doi:10.1007/s10648-007-9049-0 · Cited by 632 · cited 2×
“…in students’ learning (Anderson et al. 1995; Koedinger et al. 1997), a finding that, with a few exceptions, was reproduced in later classroom studies (Koedinger and Aleven 2007; Ritter et al. 2007), including one conducted by a third party with 17,000 students in 150 schools (Pane et al. 2013).”
ResultSelf-citationHeffernan, N. T., & Heffernan, C. L. (2014) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-014-0024-x · Cited by 616
Toward an Instructionally Oriented Theory of Example‐Based Learning
Renkl, A. (2014) · Cognitive Science · doi:10.1111/cogs.12086 · Cited by 509
Effectiveness of Cognitive Tutor Algebra I at Scale
Pane, J. F., Griffin, B. A., McCaffrey, D. F., & Karam, R. (2014) · Educational Evaluation and Policy Analysis · doi:10.3102/0162373713507480 · Cited by 317 · cited 2×
“…in students’ learning (Anderson et al. 1995; Koedinger et al. 1997), a finding that, with a few exceptions, was reproduced in later classroom studies (Koedinger and Aleven 2007; Ritter et al. 2007), including one conducted by a third party with 17,000 students in 150 schools (Pane et al. 2013).”
BackgroundResultHow Fading Worked Solution Steps Works – A Cognitive Load Perspective
Renkl, A., Atkinson, R. K., & Große, C. S. (2004) · Instructional Science · doi:10.1023/b:truc.0000021815.74806.f6 · Cited by 272
Arroyo, I., Woolf, B. P., Burelson, W., Muldner, K., Rai, D., & Tai, M. (2014) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-014-0023-y · Cited by 263 · cited 3×
“Furthermore, a number of systems that have a legitimate claim to being adaptive and intelligent do not have a very elaborate inner loop, for example, ASSISTments (Heffernan and Heffernan 2014) and Wayang Outpost/Mathsprings (Arroyo et al. 2014).”
BackgroundTowards an Agile Approach to Adapting Dynamic Collaboration Support to Student Needs
Adamson, D., Dyke, G., Jang, H., & Rosé, C. P. (2014) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-013-0012-6 · Cited by 156 · cited 1×
“For example, it is not straightforward to provide feedback on students’ dialogue (e.g., Adamson et al. 2014) or on how students are collaborating (e.g., Rummel et al. under review; Walker et al. 2014).”
BackgroundPolite web-based intelligent tutors: Can they improve learning in classrooms?
McLaren, B. M., DeLeeuw, K. E., & Mayer, R. E. (2011) · Computers & Education · doi:10.1016/j.compedu.2010.09.019 · Cited by 98
Self-citationAccounting for Beneficial Effects of Worked Examples in Tutored Problem Solving
Salden, R. J. C. M., Koedinger, K. R., Renkl, A., Aleven, V., & McLaren, B. M. (2010) · Educational Psychology Review · doi:10.1007/s10648-010-9143-6 · Cited by 96 · cited 5×
“Regarding how examples and problems can be sequenced, as mentioned, CTAT now supports the gradual backward fading of worked examples, shown to be an effective method of transitioning fromworked examples to problems (Atkinson et al. 2003; Salden et al. 2010a).”
BackgroundMethodInfluentialSelf-citationGamification of Joint Student/System Control over Problem Selection in a Linear Equation Tutor
Long, Y., & Aleven, V. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-07221-0_47 · Cited by 71 · cited 6×
“…equation solving initially implemented as an example-tracing tutor (Long and Aleven 2013a, b; Waalkens et al. 2013) was re-implemented as a rule-based Cognitive Tutor (Long and Aleven 2014), in spite of the example-tracing tutor’s proven effectiveness in multiple classroom studies (see above).”
BackgroundMethodInfluentialSelf-citationInterleaved practice in multi-dimensional learning tasks: Which dimension should we interleave?
Rau, M. A., Aleven, V., & Rummel, N. (2013) · Learning and Instruction · doi:10.1016/j.learninstruc.2012.07.003 · Cited by 67 · cited 4×
“Stoichiometry Tutor AdaptErrEx (Adams et al., 2014; McLaren et al. 2015) is an example-tracing tutor for learning decimals (part of 6th-grade mathematics) that has students identify, correct, and explain incorrect steps in worked-out problem solutions AdaptErrEx – Erroneous Examples Decimal Point (Forlizzi et al., 201…”
BackgroundInfluentialSelf-citationExploring the Assistance Dilemma: Comparing Instructional Support in Examples and Problems
McLaren, B. M., van Gog, T., Ganoe, C., Yaron, D., & Karabinos, M. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-07221-0_44 · Cited by 34 · cited 9×
“The Stoichiometry Tutor (McLaren et al. 2014, 2015b, 2016) was extended so that students watch the step-by-step narrated playback of worked and erroneous examples in the tutor’s problem-solving interface.”
BackgroundInfluentialSelf-citationAdvancing Development of Intercultural Competence through Supporting Predictions in Narrative Video
Ogan, A., Aleven, V., & Jones, C. (2009) · International Journal of Artificial Intelligence in Education · doi:10.3233/irg-2009-19(3)02 · Cited by 34
Self-citation
Cited by
Artificial Intelligence and Its Role in Education
Ahmad, S. F., Rahmat, M. K., Mubarik, M. S., Alam, M. M., & Hyder, S. I. (2021) · Sustainability · doi:10.3390/su132212902 · Cited by 336
A Conceptual Framework for Human–AI Hybrid Adaptivity in Education
Holstein, K., Aleven, V., & Rummel, N. (2020) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-52237-7_20 · Cited by 168
“Authoring tools have been developed to support non-programmer authoring, but further research is needed to support easy authoring in everyday educational settings (e.g., by teachers or students) [1, 29, 37, 39].”
MethodSelf-citationEnhancing learning outcomes through self-regulated learning support with an Open Learner Model
Long, Y., & Aleven, V. (2017) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-016-9186-6 · Cited by 95
Self-citationSmart Environment for Adaptive Learning of Cybersecurity Skills
Vykopal, J., Seda, P., Švábenský, V., & Čeleda, P. (2023) · IEEE Transactions on Learning Technologies · doi:10.1109/tlt.2022.3216345 · Cited by 48
Supporting skill integration in an intelligent tutoring system for code tracing
Huang, Y., Brusilovsky, P., Guerra, J., Koedinger, K., & Schunn, C. (2023) · Journal of Computer Assisted Learning · doi:10.1111/jcal.12757 · Cited by 32
Self-citation
Metadata
- Authors
- Aleven, V., McLaren, B. M., Sewall, J., van Velsen, M., Popescu, O., Demi, S., Ringenberg, M., & Koedinger, K. R.
- Venue
- International Journal of Artificial Intelligence in Education
- Published
- 2016-02-10
- Volume
- 26
- Issue
- 1
- Pages
- 224-269
- ISSN
- 1560-4292 (ISSN-L)
- 1560-4306
- OpenAlex
- W2261910606
- Cited by
- 139
- References
- 23
- FWCI
- 9.13
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.