Re-Factoring Based Program Repair Applied to Programming Assignments
Open accessThe DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.
Abstract
Automated program repair has been used to provide feedback for incorrect student programming assignments, since program repair captures the code modification needed to make a given buggy program pass a given test-suite. Existing student feedback generation techniques are limited because they either require manual effort in the form of providing an error model, or require a large number of correct student submissions to learn from, or suffer from lack of scalability and accuracy. In this work, we propose a fully automated approach for generating student program repairs in real-time. This is achieved by first re-factoring all available correct solutions to semantically equivalent solutions. Given an incorrect program, we match the program with the closest matching refactored program based on its control flow structure. Subsequently, we infer the input-output specifications of the incorrect program's basic blocks from the executions of the correct program's aligned basic blocks. Finally, these specifications are used to modify the blocks of the incorrect program via search-based synthesis. Our dataset consists of almost 1,800 real-life incorrect Python program submissions from 361 students for an introductory programming course at a large public university. Our experimental results suggest that our method is more effective and efficient than recently proposed feedback generation approaches. About 30% of the patches produced by our tool Refactory are smaller than those produced by the state-of-art tool Clara, and can be produced given fewer correct solutions (often a single correct solution) and in a shorter time. We opine that our method is applicable not only to programming assignments, and could be seen as a general-purpose program repair method that can achieve good results with just a single correct reference solution.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Le Goues, C., Pradel, M., & Roychoudhury, A. (2019) · Communications of the ACM · doi:10.1145/3318162 · Cited by 410
Self-citationShaping program repair space with existing patches and similar code
Jiang, J., Xiong, Y., Zhang, H., Gao, Q., & Chen, X. (2018) · Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3213846.3213871 · Cited by 329 · cited 1×
“SimFix [19] mines repairs from similar code and past patches.”
Background
Cited by
GitHub Copilot AI pair programmer: Asset or Liability?
Moradi Dakhel, A., Majdinasab, V., Nikanjam, A., Khomh, F., Desmarais, M. C., & Jiang, Z. M. (. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111734 · Cited by 417
GitHub Copilot AI pair programmer: Asset or Liability?
Dakhel, A. M., Majdinasab, V., Nikanjam, A., Khomh, F., Desmarais, M. C., Ming, Z., & Jiang (2022) · arXiv · doi:10.48550/arxiv.2206.15331 · Cited by 417
“The second pipeline focuses on the assignments of a Python programming course [20].”
BackgroundMethodInfluentialEffective test generation using pre-trained Large Language Models and mutation testing
Dakhel, A. M., Nikanjam, A., Majdinasab, V., Khomh, F., & Desmarais, M. C. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107468 · Cited by 157
Enhanced automated code vulnerability repair using large language models
de-Fitero-Dominguez, D., Garcia-Lopez, E., Garcia-Cabot, A., & Martinez-Herraiz, J.-J. (2024) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2024.109291 · Cited by 43
Verifix: Verified Repair of Programming Assignments
Ahmed, U. Z., Fan, Z., Yi, J., Al-Bataineh, O. I., & Roychoudhury, A. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3510418 · Cited by 42
Self-citationCREF: An LLM-Based Conversational Software Repair Framework for Programming Tutors
Yang, B., Tian, H., Pian, W., Yu, H., Wang, H., Klein, J., Bissyandé, T. F., & Jin, S. (2024) · Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3650212.3680328 · Cited by 41
“In education scenarios, I ntelligent T utoring S ystem (ITS) for programming shows promising results in helping novice students learn programming [1, 13, 22, 23, 81].”
BackgroundMethodImproving Program Matching to Automatically Repair Introductory Programs
Contractor, M. R., & Rivero, C. R. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-09680-8_30 · Cited by 7
Counterexample Guided Program Repair Using Zero-Shot Learning and MaxSAT-based Fault Localization
Orvalho, P., Janota, M., & Manquinho, V. (2025) · arXiv · doi:10.48550/arxiv.2502.07786 · Cited by 4
Zhou, X., Weyssow, M., Widyasari, R., Zhang, T., He, J., Lyu, Y., Chang, J., Zhang, B., Huang, D., & Lo, D. (2025) · arXiv · doi:10.48550/arxiv.2502.06215 · Cited by 3
Metadata
- Authors
- Hu, Y., Ahmed, U. Z., Mechtaev, S., Leong, B., & Roychoudhury, A.
- Venue
- 2019 34th IEEE/ACM International Conference on Automated Software Engineering (ASE)
- Publisher
- IEEE
- Published
- 2019-11
- Pages
- 388-398
- OpenAlex
- W2998870469
- Cited by
- 87
- References
- 2
- FWCI
- 6.18
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.