The Impact of Adding Textual Explanations to Next-step Hints in a Novice Programming Environment
The DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
Automated hints, a powerful feature of many programming environments, have been shown to improve students' performance and learning. New methods for generating these hints use historical data, allowing them to scale easily to new classrooms and contexts. These scalable methods often generate next-step, code hints that suggest a single edit for the student to make to their code. However, while these code hints tell the student what to do, they do not explain why, which can make these hints hard to interpret and decrease students' trust in their helpfulness. In this work, we augmented code hints by adding adaptive, textual explanations in a block-based, novice programming environment. We evaluated their impact in two controlled studies with novice learners to investigate how our results generalize to different populations. We measured the impact of textual explanations on novices' programming performance. We also used quantitative analysis of log data, self-explanation prompts, and frequent feedback surveys to evaluate novices' understanding and perception of the hints throughout the learning process. Our results showed that novices perceived hints with explanations as significantly more relevant and interpretable than those without explanations, and were also better able to connect these hints to their code and the assignment. However, we found little difference in novices' performance. Our results suggest that explanations have the potential to make code hints more useful, but it is unclear whether this translates into better overall performance and learning.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Failure rates in introductory programming revisited
Watson, C., & Li, F. W. B. (2014) · Proceedings of the 2014 conference on Innovation & technology in computer science education - ITiCSE '14 · doi:10.1145/2591708.2591749 · Cited by 639 · cited 1×
“Introductory programming courses often have high attrition and failure rates, with pass rates estimated as low as 68% [6, 38].”
BackgroundCorbett, A. T., & Anderson, J. R. (2001) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/365024.365111 · Cited by 280 · cited 2×
“[9] showed that students who received various forms of feedback while using the system completed subsequent programming”
BackgroundFactors Influencing Students' Help-Seeking Behavior while Programming with Human and Computer Tutors
Price, T. W., Liu, Z., Cateté, V., & Barnes, T. (2017) · Proceedings of the 2017 ACM Conference on International Computing Education Research · doi:10.1145/3105726.3106179 · Cited by 61 · cited 3×
“Previous work has shown this lack of explanations can make next-step hints difficult for novices to interpret and connect to their current code [29], leading to a decrease in how often students request and follow hints, and eroding their trust in the system [29, 32].”
BackgroundSelf-citation
Cited by
Comparing Code Explanations Created by Students and Large Language Models
Leinonen, J., Denny, P., MacNeil, S., Sarsa, S., Bernstein, S., Kim, J., Tran, A., & Hellas, A. (2023) · Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1 · doi:10.1145/3587102.3588785 · Cited by 210
Becker, B. A., Denny, P., Pettit, R., Bouchard, D., Bouvier, D. J., Harrington, B., Kamil, A., Karkare, A., McDonald, C., Osera, P.-M., et al. (2019) · Proceedings of the Working Group Reports on Innovation and Technology in Computer Science Education · doi:10.1145/3344429.3372508 · Cited by 199
MacNeil, S., Tran, A., Hellas, A., Kim, J., Sarsa, S., Denny, P., Bernstein, S., & Leinonen, J. (2023) · Proceedings of the 54th ACM Technical Symposium on Computer Science Education V. 1 · doi:10.1145/3545945.3569785 · Cited by 195
“Explanations help students to make connections and to develop their own understanding of how a code snippet executes [26].”
BackgroundMacNeil, S., Tran, A., Hellas, A., Kim, J., Sarsa, S., Denny, P., Bernstein, S., & Leinonen, J. (2022) · arXiv · doi:10.48550/arxiv.2211.02265 · Cited by 195
Metadata
- Authors
- Marwan, S., Lytle, N., Williams, J. J., & Price, T.
- Venue
- Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education
- Publisher
- ACM
- Published
- 2019-07-02
- Pages
- 520-526
- OpenAlex
- W2963562081
- Cited by
- 38
- References
- 3
- FWCI
- 6.18
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Impact of Adding Textual Explanations to Next-step Hints in a Novice Programming Environment. https://app.cochat.ai/papers/10.1145/3304221.3319759
- The paper
- Marwan, S., Lytle, N., Williams, J. J., & Price, T. (2019). The Impact of Adding Textual Explanations to Next-step Hints in a Novice Programming Environment. Proceedings of the 2019 ACM Conference on Innovation and Technology in Computer Science Education. https://doi.org/10.1145/3304221.3319759
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.