Teaching the tacit knowledge of programming to noviceswith natural language tutoring
The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
For beginning programmers, inadequate problem solving and planning skills are among the most salient of their weaknesses. In this paper, we test the efficacy of natural language tutoring to teach and scaffold acquisition of these skills. We describe ProPL (Pro-PELL), a dialogue-based intelligent tutoring system that elicits goal decompositions and program plans from students in natural language. The system uses a variety of tutoring tactics that leverage students' intuitive understandings of the problem, how it might be solved, and the underlying concepts of programming. We report the results of a small-scale evaluation comparing students who used ProPL with a control group who read the same content. Our primary findings are that students who received tutoring from ProPL seem to have developed an improved ability to solve the composition problem and displayed behaviors that suggest they were able to think at greater levels of abstraction than students in the read-only group.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring
BLOOM, B. S. (1984) · Educational Researcher · doi:10.3102/0013189x013006004 · Cited by 2,734 · cited 1×
“Students working one-on-one with expert human tutors often score 2.0 standard deviations higher than students working on the same topic in classrooms (Bloom, 1984).”
BackgroundInfluentialCognitive Tutors: Lessons Learned
Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995) · Journal of the Learning Sciences · doi:10.1207/s15327809jls0402_2 · Cited by 1,982 · cited 1×
“In contrast, the best ITS’s have effect sizes of only 1.0 standard deviations (Anderson, Corbett, Koedinger, & Pelletier, 1995).”
BackgroundInfluential(2001) · doi:10.1609/aimag.v22i4.1591 · Cited by 381 · cited 1×
“In efforts to narrow this gap, a number of dialogue-based tutoring systems have emerged that attempt to mimic the dialogue strategies and communicative patterns of human tutors (Graesser, VanLehn, Rose, Jordan, & Harter, 2001), something most traditional ITS’s do not do.”
BackgroundSelf-citationConditions of Learning in Novice Programmers
Perkins, D. N., Hancock, C., Hobbs, R., Martin, F., & Simmons, R. (1986) · Journal of Educational Computing Research · doi:10.2190/gujt-jcbj-q6qu-q9pl · Cited by 262 · cited 1×
“(Perkins, Hancock, Hobbs, Martin, & Simmons, 1989, p. 257).”
Cited by
VanLEHN, K. (2011) · Educational Psychologist · doi:10.1080/00461520.2011.611369 · Cited by 2,323
Self-citationEffectiveness of Intelligent Tutoring Systems: A Meta-Analytic Review
Kulik, J. A., & Fletcher, J. D. (2016) · Review of Educational Research · doi:10.3102/0034654315581420 · Cited by 930
Intelligent tutoring systems and learning outcomes: A meta-analysis
Ma, W., Adesope, O. O., Nesbit, J. C., & Liu, Q. (2014) · Journal of Educational Psychology · doi:10.1037/a0037123 · Cited by 840
When Are Tutorial Dialogues More Effective Than Reading?
VanLehn, K., Graesser, A. C., Jackson, G. T., Jordan, P., Olney, A., & Rosé, C. P. (2007) · Cognitive Science · doi:10.1080/03640210709336984 · Cited by 378
“…then interactive tutoring usually elicited larger learning gains than the control instruction as predicted by the interaction hypothesis (Evens & Michael , 2006; Graesser et al., 2003; Lane & VanLehn, 2005; Person, Graesser, Bautista, et al., 2001; Swanson, 1992; Wood et al., 1978).”
BackgroundMethodSelf-citationA systematic literature review of automated feedback generation for programming exercises
Keuning, H., Jeuring, J., & Heeren, B. (2019) · ACM Transactions on Computing Education · doi:10.1145/3231711 · Cited by 370
“For example, in PATTIE and ProPL the student engages in a conversation with an automated tutor to solve a programming problem.”
Steenbergen-Hu, S., & Cooper, H. (2014) · Journal of Educational Psychology · doi:10.1037/a0034752 · Cited by 355
“…Berger, Healy, & Romero, 2003; Assessment and Learning in Knowledge Spaces [ALEKS]: Doignon & Falmagne, 1999), physics (e.g., Andes, Atlas, and Why/Atlas: VanLehn et al., 2002, 2007), and computer science (e.g., dialogue-based ITS: Lane & VanLehn, 2005; ACT Programming Tutor: Corbett, 2001).”
Steenbergen-Hu, S., & Cooper, H. (2013) · Journal of Educational Psychology · doi:10.1037/a0032447 · Cited by 175
Influence of artificial intelligence tools on higher order thinking skills: a meta-analysis
Ilgun Dibek, M., Sahin Kursad, M., & Erdogan, T. (2025) · Interactive Learning Environments · doi:10.1080/10494820.2024.2402028 · Cited by 45
Zhu, M., & Zhang, K. (2025) · Technology, Knowledge and Learning · doi:10.1007/s10758-025-09859-1 · Cited by 13
Metadata
- Authors
- Lane, H. C., & VanLehn, K.
- Venue
- Computer Science Education
- Publisher
- Informa UK Limited
- Published
- 2005-09
- Volume
- 15
- Issue
- 3
- Pages
- 183-201
- ISSN
- 0899-3408 (ISSN-L)
- 1744-5175
- OpenAlex
- W1999645150
- Cited by
- 101
- References
- 4
- FWCI
- 9.94
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.