The DOI matches its Crossref record; 84 of the 84 references it cites were found in a registry or an index.
Help Helps, But Only So Much: Research on Help Seeking with Intelligent Tutoring Systems
Open accessAbstract
Help seeking is an important process in self-regulated learning (SRL). It may influence learning with intelligent tutoring systems (ITSs), because many ITSs provide help, often at the student’s request. The Help Tutor was a tutor agent that gave in-context, real-time feedback on students’ help-seeking behavior, as they were learning with an ITS. Key goals were to help students become better self-regulated learners and help them achieve better domain-level learning outcomes. In a classroom study, feedback on help seeking helped students to use on-demand help more deliberately, even after the feedback was no longer given, but not to achieve better learning outcomes. The work made a number of contributions, including the creation of a knowledge-engineered, rule-based, executable model of help seeking that can drive tutoring. We review these contributions from a contemporary perspective, with a theoretical analysis, a review of recent empirical literature on help seeking with ITSs, and methodological suggestions. Although we do not view on-demand, principle-based help during tutored problem solving as being as important as we once did, we still view it as helpful under certain circumstances, and recommend that it be included in ITSs. We view the goal of helping students become better self-regulated learners as one of the grand challenges in ITSs research today.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Zimmerman, B. J. (2000) · Handbook of Self-Regulation · doi:10.1016/b978-012109890-2/50031-7 · Cited by 6,069
The Cambridge Handbook of the Learning Sciences
(2005) · Cambridge University Press eBooks · doi:10.1017/cbo9780511816833 · Cited by 4,601
Zimmerman, B. J. (2008) · American Educational Research Journal · doi:10.3102/0002831207312909 · Cited by 3,735
A Conceptual Framework for Assessing Motivation and Self-Regulated Learning in College Students
Pintrich, P. R. (2004) · Educational Psychology Review · doi:10.1007/s10648-004-0006-x · Cited by 3,269
Knowledge tracing: Modeling the acquisition of procedural knowledge
Corbett, A. T., & Anderson, J. R. (1995) · User Modelling and User-Adapted Interaction · doi:10.1007/bf01099821 · Cited by 2,324
Metacognition in Educational Theory and Practice
(1998) · Routledge eBooks · doi:10.4324/9781410602350 · Cited by 2,119
Cognitive Tutors: Lessons Learned
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995) · Journal of the Learning Sciences · doi:10.1207/s15327809jls0402_2 · Cited by 1,982 · cited 8×
“…student’s request, including Andes (VanLehn et al. 2005), Assistments (Heffernan and Heffernan 2014; Razzaq and Heffernan 2010), Cognitive Tutors (Anderson et al. 1995; Koedinger and Corbett 2006), example-tracing tutors (Aleven et al. 2009), Sherlock (Katz et al. 1998; Lesgold et al. 1992),…”
BackgroundMethodInfluentialSelf-citation(1993) · Medical Entomology and Zoology · Cited by 1,882
Self‐Explanations: How Students Study and Use Examples in Learning to Solve Problems
Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989) · Cognitive Science · doi:10.1207/s15516709cog1302_1 · Cited by 1,714
Studying as Self-Regulated Learning
(1998) · Metacognition in Educational Theory and Practice · doi:10.4324/9781410602350-19 · Cited by 1,589
Self-regulation of learning and performance: Issues and educational applications.
Schunk, D. H., & Zimmerman, B. J. (1994) · Cited by 1,492
Inquiry, modeling, and metacognition: making science accessible to all students
White, B. Y., & Frederiksen, J. R. (1998) · Cognition and Instruction · doi:10.1207/s1532690xci1601_2 · Cited by 1,394 · cited 2×
“Second, a study by Stamper et al. (2011) compared versions of a tutoring system for logic proof with and without next-step hints.”
BackgroundAleven, V. A. W. M. M., & Koedinger, K. R. (2002) · Cognitive Science · doi:10.1207/s15516709cog2602_1 · Cited by 895
Self-citationChapter 3: Rethinking Transfer: A Simple Proposal With Multiple Implications
BRANSFORD, J. D., & SCHWARTZ, D. L. (1999) · Review of Research in Education · doi:10.3102/0091732x024001061 · Cited by 881
Koedinger, K. R., Corbett, A. T., & Perfetti, C. (2012) · Cognitive Science · doi:10.1111/j.1551-6709.2012.01245.x · Cited by 772 · cited 3×
“We frame the issue in terms of the Knowledge-Learning-Instruction (KLI) theoretical framework (Koedinger et al. 2012), a recent framework linking cognitive science and educational research to educational practice.”
BackgroundMethodSelf-citationThe Behavior of Tutoring Systems
VanLehn, K. (2006) · International Journal of Artificial Intelligence in Education · doi:10.3233/irg-2006-16(3)02 · Cited by 749
Heffernan, N. T., & Heffernan, C. L. (2014) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-014-0024-x · Cited by 616
Help Seeking and Help Design in Interactive Learning Environments
Aleven, V., Stahl, E., Schworm, S., Fischer, F., & Wallace, R. (2003) · Review of Educational Research · doi:10.3102/00346543073003277 · Cited by 609 · cited 2×
“Classroom evaluation”
BackgroundSelf-citationThe Andes Physics Tutoring System: Lessons Learned
VanLehn, K. A., Lynch, C. F., Schulze, K. G., Shapiro, J. A., Shelby, R., Taylor, L., Treacy, D. J., Weinstein, A., & Wintersgill, M. C. (2005) · doi:10.5555/1434930.1434932 · Cited by 518
Toward an Instructionally Oriented Theory of Example‐Based Learning
Renkl, A. (2014) · Cognitive Science · doi:10.1111/cogs.12086 · Cited by 509
Toward Meta-cognitive Tutoring: A Model of Help Seeking with a Cognitive Tutor
Aleven, V., McLaren, B., Roll, I., & Koedinger, K. (2006) · International Journal of Artificial Intelligence in Education · doi:10.3233/irg-2006-16(2)02 · Cited by 406
Self-citationSkill Acquisition and the LISP Tutor
Anderson, J. R., Conrad, F. G., & Corbett, A. T. (1989) · Cognitive Science · doi:10.1207/s15516709cog1304_1 · Cited by 386
Learning from Worked-Out Examples: The Effects of Example Variability and Elicited Self-Explanations
Renkl, A., Stark, R., Gruber, H., & Mandl, H. (1998) · Contemporary Educational Psychology · doi:10.1006/ceps.1997.0959 · Cited by 374
Designing Learning by Teaching Agents: The Betty's Brain System
Leelawong, K., & Biswas, G. (2008) · International Journal of Artificial Intelligence in Education · doi:10.3233/irg-2008-18(3)02 · Cited by 311
Toward Computer-Based Support of Meta-Cognitive Skills: a . . .
Conati, C., & VanLehn, K. A. (2000) · HAL · Cited by 293
Wittwer, J., & Renkl, A. (2008) · Educational Psychologist · doi:10.1080/00461520701756420 · Cited by 277
A New Paradigm for Intelligent Tutoring Systems: Example-Tracing Tutors
Aleven, V., McLaren, B. M., Sewall, J., & Koedinger, K. R. (2009) · doi:10.5555/1734243.1734245 · Cited by 277
Self-citationCognitive Tutors: Technology Bringing Learning Sciences to the Classroom
Koedinger, K. R., & Corbett, A. (2005) · The Cambridge Handbook of the Learning Sciences · doi:10.1017/cbo9780511816833.006 · Cited by 275
Self-citationHelp seeking, learning and contingent tutoring
Wood, H., & Wood, D. (1999) · Computers & Education · doi:10.1016/s0360-1315(99)00030-5 · Cited by 275
Arroyo, I., Woolf, B. P., Burelson, W., Muldner, K., Rai, D., & Tai, M. (2014) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-014-0023-y · Cited by 263
Why Students Engage in “Gaming the System” Behavior in Interactive Learning Environments
Baker, R. S., Walonoski, J. A., Heffernan, N. T., Roll, I., Corbett, A. T., & Koedinger, K. R. (2008) · The Journal of Interactive Learning Research · Cited by 256
Self-citationStudent Models that Invite the Learner In: The SMILI:() Open Learner Modelling Framework
Bull, S., & Kay, J. (2007) · International Journal of Artificial Intelligence in Education · doi:10.3233/irg-2007-17(2)02 · Cited by 250
SHERLOCK: A Coached Practice Environment for an Electronics Troubleshooting Job
Lesgold, A., Lajoie, S., Bunzo, M., & Eggan, G. (1988) · doi:10.21236/ada201748 · Cited by 239
Limitations of Student Control: Do Students Know when They Need Help?
Aleven, V., & Koedinger, K. R. (2000) · Lecture Notes in Computer Science · doi:10.1007/3-540-45108-0_33 · Cited by 236 · cited 7×
“The work started with a discovery in student log data from the Geometry Cognitive Tutor in an early educational data mining study (EDM) (Aleven and Koedinger 2000; 2001).”
BackgroundMethodInfluentialSelf-citationBull, S., & Kay, J. (2010) · Studies in Computational Intelligence · doi:10.1007/978-3-642-14363-2_15 · Cited by 209
Adaptive help seeking: A strategy of self-regulated learning.
Newman, R. S. (1994) · Cited by 205
Developing a generalizable detector of when students game the system
Baker, R. S. J. D., Corbett, A. T., Roll, I., & Koedinger, K. R. (2008) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-007-9045-6 · Cited by 203
Self-citationMotivation and performance in a game-based intelligent tutoring system.
Jackson, G. T., & McNamara, D. S. (2013) · Journal of Educational Psychology · doi:10.1037/a0032580 · Cited by 182
Computer-supported example-based learning: When instructional explanations reduce self-explanations
Schworm, S., & Renkl, A. (2006) · Computers & Education · doi:10.1016/j.compedu.2004.08.011 · Cited by 175
Sinatra, G. M., Kardash, C. M., Taasoobshirazi, G., & Lombardi, D. (2012) · Instructional Science · doi:10.1007/s11251-011-9166-5 · Cited by 173
STyLE-OLM: Interactive Open Learner Modelling
Dimitrova, V. (2003) · International Journal of Artificial Intelligence in Education · doi:10.3233/irg-2003-13(1)06 · Cited by 171
Evaluating the Effect of Open Student Models on Self-Assessment
Mitrovic, A., & Martin, B. (2007) · International Journal of Artificial Intelligence in Education · doi:10.3233/irg-2007-17(2)03 · Cited by 167
Adaptive Intelligent Support to Improve Peer Tutoring in Algebra
Walker, E., Rummel, N., & Koedinger, K. R. (2013) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-013-0001-9 · Cited by 163
Self-citationA Contextualized, Differential Sequence Mining Method to Derive Students' Learning Behavior Patterns
Kinnebrew, J. S., Loretz, K. M., & Biswas, G. (2013) · Journal of Educational Data Mining · doi:10.5281/zenodo.3554617 · Cited by 155
Effective Feedback Content for Tutoring Complex Skills
McKendree, J. (1990) · Human-Computer Interaction · doi:10.1207/s15327051hci0504_2 · Cited by 153
Evaluating metacognitive scaffolding in Guided Invention Activities
Roll, I., Holmes, N. G., Day, J., & Bonn, D. A. (2012) · Instructional Science · doi:10.1007/s11251-012-9208-7 · Cited by 152
Self-citationLarkin, J. H., & Chabay, R. W. (1992) · Cited by 143
Help seeking in academic settings: goals, groups, and contexts
(2006) · Choice Reviews Online · doi:10.5860/choice.44-1060 · Cited by 139
McLaren, B. M., van Gog, T., Ganoe, C. H., Karabinos, M., & Yaron, D. (2015) · Computers in Human Behavior · doi:10.1016/j.chb.2015.08.038 · Cited by 134
Self-citationA Response-Time Model for Bottom-Out Hints as Worked Examples
Shih, B., Koedinger, K. R., & Scheines, R. (2010) · Chapman & Hall/CRC data mining and knowledge discovery series · doi:10.1201/b10274-17 · Cited by 132
Self-citationBouchet, F., Harley, J. M., Trevors, G., & Azevedo, R. (2013) · Journal of Educational Data Mining · doi:10.5281/zenodo.3554613 · Cited by 131
Does Help Help? Introducing the Bayesian Evaluation and Assessment Methodology
Beck, J. E., Chang, K.-M. K., Mostow, J., & Corbett, A. T. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-69132-7_42 · Cited by 126
Analyzing the temporal evolution of students’ behaviors in open-ended learning environments
Kinnebrew, J. S., Segedy, J. R., & Biswas, G. (2014) · Metacognition and Learning · doi:10.1007/s11409-014-9112-4 · Cited by 123
On the Benefits of Seeking (and Avoiding) Help in Online Problem-Solving Environments
Roll, I., Baker, R. S. J. D., Aleven, V., & Koedinger, K. R. (2014) · Journal of the Learning Sciences · doi:10.1080/10508406.2014.883977 · Cited by 109
Self-citationSabourin, J. L., Shores, L. R., Mott, B., & Lester, J. C. (2013) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-013-0004-6 · Cited by 98
Accounting for Beneficial Effects of Worked Examples in Tutored Problem Solving
Salden, R. J. C. M., Koedinger, K. R., Renkl, A., Aleven, V., & McLaren, B. M. (2010) · Educational Psychology Review · doi:10.1007/s10648-010-9143-6 · Cited by 96
Self-citationRule-Based Cognitive Modeling for Intelligent Tutoring Systems
Aleven, V. A. W. M. M. (2010) · Studies in computational intelligence · doi:10.1007/978-3-642-14363-2_3 · Cited by 91
Self-citationEvaluating the Effectiveness of a Tutorial Dialogue System for Self-Explanation
Aleven, V., Ogan, A., Popescu, O., Torrey, C., & Koedinger, K. (2004) · International Conference on Intelligent Tutoring Systems · doi:10.1007/978-3-540-30139-4_42 · Cited by 88
Self-citationConati, C., & Kardan, S. (2013) · AI Magazine · doi:10.1609/aimag.v34i3.2483 · Cited by 88
Poitras, E., & Lajoie, S. P. (2013) · Metacognition and Learning · doi:10.1007/s11409-013-9104-9 · Cited by 87
Arroyo, I., Beck, J. E., Woolf, B. P., Beal, C. R., & Schultz, K. (2000) · Lecture notes in computer science · doi:10.1007/3-540-45108-0_61 · Cited by 79
Supporting Students’ Self-Regulated Learning with an Open Learner Model in a Linear Equation Tutor
Long, Y., & Aleven, V. A. W. M. M. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39112-5_23 · Cited by 74
Self-citationScaffolding vs. Hints in the Assistment System
Razzaq, L. M., & Heffernan, N. T. (2006) · Lecture notes in computer science · doi:10.1007/11774303_63 · Cited by 66
Azevedo, R., Landis, R. S., Feyzi-Behnagh, R., Duffy, M., Trevors, G., Harley, J. M., Bouchet, F., Burlison, J., Taub, M., Pacampara, N., et al. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-30950-2_27 · Cited by 58 · cited 2×
“Our hunch is that it will be useful to take a more comprehensive view of self-regulation, as a number of researchers have started to do, and study multiple SRL constructs in conjunction, consistent with SRL theories (Azevedo et al. 2012; Kinnebrew et al. 2013; Long and Aleven 2013b; Poitras and Lajoie 2013; Roll et al…”
ResultKoedinger, K. R., Aleven, V. A. W. M. M., Roll, I., & Baker, R. S. (2009) · doi:10.4324/9780203876428.ch20 · Cited by 52
Self-citationTowards Predicting Future Transfer of Learning
Baker, R. S., Gowda, S. M., & Corbett, A. T. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-21869-9_6 · Cited by 49
Effects of social presence and social role on help-seeking and learning
Howley, I. K., Kanda, T., Hayashi, K., & Rosé, C. P. (2014) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/2559636.2559667 · Cited by 43
Cognitive Task Analysis in Service of Intelligent Tutoring System Design: A Case Study in Statistics
Lovett, M. C. (1998) · Lecture notes in computer science · doi:10.1007/3-540-68716-5_29 · Cited by 41
Aleven, V. A. W. M. M. (2013) · Springer international handbooks of education · doi:10.1007/978-1-4419-5546-3_21 · Cited by 39
Self-citationHints: Is It Better to Give or Wait to Be Asked?
Razzaq, L. M., & Heffernan, N. T. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-13388-6_39 · Cited by 39
Modeling and Studying Gaming the System with Educational Data Mining
Baker, R. S., Corbett, A. T., Roll, I., Koedinger, K. R., Aleven, V. A. W. M. M., Cocea, M., Hershkovitz, A., de Caravalho, A. M. J. B., Mitrović, A., & Mathews, M. (2013) · Springer international handbooks of education · doi:10.1007/978-1-4419-5546-3_7 · Cited by 38
Self-citationTo Tutor or Not to Tutor: That is the Question
Leena, R., & Heffernan, N. T. (2009) · Frontiers in artificial intelligence and applications · doi:10.3233/978-1-60750-028-5-457 · Cited by 37
Butcher, K. R., & Aleven, V. A. W. M. M. (2013) · Journal of Educational Psychology · doi:10.1037/a0031756 · Cited by 32
Self-citationEnhancing the Automatic Generation of Hints with Expert Seeding
Stamper, J. C., Barnes, T. M., & Croy, M. J. (2011) · International Journal of Artificial Intelligence in Education · doi:10.3233/jai-2011-021 · Cited by 27
Learner Differences in Hint Processing.
Goldin, I. M., Koedinger, K. R., & Aleven, V. A. W. M. M. (2012) · Educational Data Mining · Cited by 26
Self-citationInvestigations into Help Seeking and Learning with a Cognitive Tutor
Aleven, V. A. W. M. M., & Koedinger, K. R. (2001) · Cited by 25
Self-citationVassileva, J., McCalla, G., & Greer, J. E. (2015) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-015-0073-9 · Cited by 24
Identifying students' inquiry planning using machine learning
Montalvo, O., Baker, R. S., Pedro, M. S., Nakama, A., & Gobert, J. D. (2010) · Educational Data Mining · Cited by 23
Long, Y., & Aleven, V. A. W. M. M. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39112-5_26 · Cited by 21
Self-citationInterpreting model discovery and testing generalization to a new dataset.
Liu, R., McLaughlin, E. A., & Koedinger, K. R. (2014) · Cited by 21
Self-citationAn Architecture to Combine Meta-Cognitive and Cognitive Tutoring: Pilot Testing the Help Tutor
Aleven, V. A. W. M. M., Roll, I., McLaren, B. M., Ryu, E. J., & Koedinger, K. R. (2018) · Figshare · doi:10.1184/r1/6469799 · Cited by 18
Self-citationAnalysing High-Level Help-Seeking Behaviour in ITSs
Mathews, M., Mitrović, T., & Thomson, D. (2008) · Lecture notes in computer science · doi:10.1007/978-3-540-70987-9_42 · Cited by 17
Teammate Relationships Improve Help-Seeking Behavior in an Intelligent Tutoring System
Tai, M., Arroyo, I., & Woolf, B. P. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-39112-5_25 · Cited by 10
Hints: You Can't Have Just One
Goldin, I. M., Koedinger, K. R., & Aleven, V. (2013) · Cited by 8
Self-citation
Cited by
Mining Big Data in Education: Affordances and Challenges
Fischer, C., Pardos, Z. A., Baker, R. S., Williams, J. J., Smyth, P., Yu, R., Slater, S., Baker, R., & Warschauer, M. (2020) · Review of Research in Education · doi:10.3102/0091732x20903304 · Cited by 412
Co-Designing a Real-Time Classroom Orchestration Tool to Support Teacher–AI Complementarity
Holstein, K., McLaren, B. M., & Aleven, V. (2019) · Journal of Learning Analytics · doi:10.18608/jla.2019.62.3 · Cited by 390
Self-citationAI in Education: A Systematic Literature Review
Tahiru, F. (2021) · Journal of Cases on Information Technology · doi:10.4018/jcit.2021010101 · Cited by 299
Developing Personalized Education: A Dynamic Framework
Tetzlaff, L., Schmiedek, F., & Brod, G. (2021) · Educational Psychology Review · doi:10.1007/s10648-020-09570-w · Cited by 266
Student Learning Benefits of a Mixed-Reality Teacher Awareness Tool in AI-Enhanced Classrooms
Holstein, K., McLaren, B. M., & Aleven, V. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-93843-1_12 · Cited by 235
Self-citation
Metadata
- Authors
- Aleven, V., Roll, I., McLaren, B. M., & Koedinger, K. R.
- Venue
- International Journal of Artificial Intelligence in Education
- Published
- 2016-01-13
- Volume
- 26
- Issue
- 1
- Pages
- 205-223
- ISSN
- 1560-4292 (ISSN-L)
- 1560-4306
- OpenAlex
- W2237600160
- Cited by
- 255
- References
- 84
- FWCI
- 17.15
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.