The DOI matches its Crossref record; 66 of the 68 references it cites were found in a registry or an index.
Is the cure worse than the disease? overfitting in automated program repair
Abstract
Automated program repair has shown promise for reducing the significant manual effort debugging requires. This paper addresses a deficit of earlier evaluations of automated repair techniques caused by repairing programs and evaluating generated patches' correctness using the same set of tests. Since tests are an imperfect metric of program correctness, evaluations of this type do not discriminate between correct patches and patches that overfit the available tests and break untested but desired functionality. This paper evaluates two well-studied repair tools, GenProg and TrpAutoRepair, on a publicly available benchmark of bugs, each with a human-written patch. By evaluating patches using tests independent from those used during repair, we find that the tools are unlikely to improve the proportion of independent tests passed, and that the quality of the patches is proportional to the coverage of the test suite used during repair. For programs that pass most tests, the tools are as likely to break tests as to fix them. However, novice developers also overfit, and automated repair performs no worse than these developers. In addition to overfitting, we measure the effects of test suite coverage, test suite provenance, and starting program quality, as well as the difference in quality between novice-developer-written and tool-generated patches when quality is assessed with a test suite independent from the one used for patch generation.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“However, due to practical limitations, test suites are rarely exhaustive and often suffer from weaknesses [50].”
From Are “Solved Issues” in SWE-bench Really Solved Correctly? An Empirical Study (2026) · cites this paper as [50]
In the citing paper
“Smith el al. [50] found that using a test suite that is inaccessible to evaluated tools can help detect plausible but incorrect patches.”
From Are “Solved Issues” in SWE-bench Really Solved Correctly? An Empirical Study (2026) · cites this paper as [50]
In the citing paper
“The problem of weak test suites has been well studied in the area of test-based automatic program repair (APR) [37, 40, 46, 50].”
From Are “Solved Issues” in SWE-bench Really Solved Correctly? An Empirical Study (2026) · cites this paper as [50]
Citations
Cites
Genetic Programming: On the Programming of Computers by Means of Natural Selection
Koza, J. R. (1992) · Medical Entomology and Zoology · Cited by 13,082
Modeling by shortest data description
Rissanen, J. (1978) · Automatica · doi:10.1016/0005-1098(78)90005-5 · Cited by 6,011
KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs
Cadar, C., Dunbar, D., & Engler, D. (2008) · doi:10.5555/1855741.1855756 · Cited by 2,655
Defects4J: a database of existing faults to enable controlled testing studies for Java programs
Just, R., Jalali, D., & Ernst, M. D. (2014) · Proceedings of the 2014 International Symposium on Software Testing and Analysis · doi:10.1145/2610384.2628055 · Cited by 1,447
Mechanization in problem solving: The effect of Einstellung.
Luchins, A. S. (1942) · Psychological Monographs · doi:10.1037/h0093502 · Cited by 1,143
GenProg: A Generic Method for Automatic Software Repair
Goues, C. L., Nguyen, T. H., Forrest, S., & Weimer, W. R. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.104 · Cited by 1,084
Simplifying and isolating failure-inducing input
Zeller, A., & Hildebrandt, R. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/32.988498 · Cited by 1,084
Automating string processing in spreadsheets using input-output examples
Gulwani, S. (2011) · ACM-SIGACT Symposium on Principles of Programming Languages · doi:10.1145/1926385.1926423 · Cited by 1,039
Arcuri, A., & Briand, L. (2011) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/1985793.1985795 · Cited by 989
An experimental evaluation of the assumption of independence in multiversion programming
KNIGHT, J. C., & Leveson, N. G. (1986) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1986.6312924 · Cited by 768
Research works printed but not linked: 6
- L. Chen and A. Avižienis . N-version programming: A faulttolerance approach to reliability of software operation . In IEEE International Symposium on Fault-Tolerant Computing (FTCS) , pages 3– 9 , 1978 . L. Chen and A. Avižienis. N-version programming: A faulttolerance approach to reliability of software operation. In IEEE International Symposium on Fault-Tolerant Computing (FTCS), pages 3–9, 1978.
- 10.5555/2486788.2486893 · DOI
- 10.5555/2337223.2337225 · DOI
- 10.5555/2818754.2818811 · DOI
- 10.5555/541177 · DOI
- Research Triangle Institute . The economic impacts of inadequate infrastructure for software testing. NIST Planning Report 02-3, May 2002 . Research Triangle Institute. The economic impacts of inadequate infrastructure for software testing. NIST Planning Report 02-3, May 2002.
Cited by
Sorted by citations
Items 201–300 of 328
SampleFix: Learning to Generate Functionally Diverse Fixes
Hajipour, H., Bhattacharyya, A., Staicu, C.-A., & Fritz, M. (2021) · Communications in computer and information science · doi:10.1007/978-3-030-93733-1_8 · Cited by 6
Yang, D., Lei, Y., Mao, X., Qi, Y., & Yi, X. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3561382 · Cited by 7
START: A Framework for Trusted and Resilient Autonomous Vehicles (Practical Experience Report)
Leach, K., Timperley, C. S., Angstadt, K., Nguyen‐Tuong, A., Hiser, J. D., Paulos, A., Pal, P. P., Hurley, P., Thomas, C., Davidson, J. W., et al. (2022) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre55969.2022.00018 · Cited by 6
CirFix: Automated Hardware Repair and its Real-World Applications
Santiesteban, P., Huang, Y., Weimer, W. R., & Ahmad, H. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3269899 · Cited by 7
Test-based patch clustering for automatically-generated patches assessment
Martínez, M., Kechagia, M., Perera, A., Petke, J., Sarro, F., & Aleti, A. (2024) · Empirical Software Engineering · doi:10.1007/s10664-024-10503-2 · Cited by 6
A Survey on Automatic Bug Fixing
Cao, H., Meng, Y., Shi, J., Li, L., Liao, T., & Zhao, C. (2020) · International Symposium on System and Software Reliability (ISSSR) · doi:10.1109/isssr51244.2020.00029 · Cited by 6
High-Quality Automated Program Repair
Motwani, M. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion52605.2021.00134 · Cited by 6
On the evaluation of automatic program repair techniques and tools
Khalilian, A., Baraani-Dastjerdi, A., & Zamani, B. (2016) · Iranian Conference on Electrical Engineering · doi:10.1109/iraniancee.2016.7585390 · Cited by 6
ROSE: An IDE-Based Interactive Repair Framework for Debugging
Reiss, S. P., Wei, X., Yuan, J. P., & Xin, Q. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3705306 · Cited by 6
Speedup Automatic Program Repair Using Dynamic Software Updating
Guo, R., Gu, T., Yao, Y., Xu, F., & Ma, X. (2019) · Asia-Pacific Symposium on Internetware · doi:10.1145/3361242.3361245 · Cited by 5
Some automatically generated patches are more likely to be correct than others
Bennett, G. J., Hall, T., & Bowes, D. (2022) · IEEE/ACM International Workshop on Automated Program Repair (APR) · doi:10.1145/3524459.3527348 · Cited by 5
Strider: Signal Value Transition-Guided Defect Repair for HDL Programming Assignments
Yang, D., He, J., Mao, X., Li, T., Lei, Y., Yi, X., & Wu, J. (2023) · IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems · doi:10.1109/tcad.2023.3341750 · Cited by 5
Automatically Mitigating Vulnerabilities in Binary Programs via Partially Recompilable Decompilation
Reiter, P., Tay, H. J., Weimer, W. R., Doupé, A., Wang, R., & Forrest, S. (2024) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2024.3482413 · Cited by 5
Christi, A., Groce, A., & Wellman, A. (2019) · IEEE International Symposium on Software Reliability Engineering Workshops (ISSREW) · doi:10.1109/issrew.2019.00046 · Cited by 5
Nilizadeh, A., Leavens, G. T., Păsăreanu, C. S., & Noller, Y. (2023) · Formal Aspects of Computing · doi:10.1145/3607538 · Cited by 5
Nakajima, H., Higo, Y., Yokoyama, H., & Kusumoto, S. (2016) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec.2016.042 · Cited by 5
Khalilian, A., Baraani-Dastjerdi, A., & Zamani, B. (2019) · Journal of Computer Languages · doi:10.1016/j.cola.2019.100927 · Cited by 5
Improving Patch Quality by Enhancing Key Components of Automatic Program Repair
Soto, M. (2019) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2019.00147 · Cited by 5
A Revisit of the Integration of Metamorphic Testing and Test Suite Based Automated Program Repair
Jiang, M., Chen, T. Y., Kuo, F., Ding, Z., Choi, E.-H., & Mizuno, O. (2017) · IEEE/ACM International Workshop on Metamorphic Testing (MET) · doi:10.1109/met.2017.5 · Cited by 5
E-APR: Mapping the Effectiveness of Automated Program RepairPreprint
Aleti, A., & Martínez, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.03968 · Cited by 5
Integrating Various Software Artifacts for Better LLM-based Bug Localization and Program Repair
Feng, Q., Ma, X., Sheng, J., Feng, Z., Song, W., & Liang, P. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3770581 · Cited by 4
Automated program repair with canonical constraints
Hill, A., Păsăreanu, C. S., & Stolee, K. T. (2018) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3183440.3194999 · Cited by 4
Priority Measurement of Patches for Program Repair Based on Semantic Distance
Dong, Y., Wu, M., Zhang, L., Yin, W., Wu, M., & Li, H. (2020) · Symmetry · doi:10.3390/sym12122102 · Cited by 4
Petke, J., Martínez, M., Kechagia, M., Aleti, A., & Sarro, F. (2024) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3663529.3663776 · Cited by 4
Syntax-Guided Automated Program Repair for Hyperproperties
Beutner, R., Hsu, T.-H., Bonakdarpour, B., & Finkbeiner, B. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-65633-0_1 · Cited by 4
Automated program repair for variability bugs in software product line systems
Nguyen, T.-T., Zhang, X., Arcaini, P., Ishikawa, F., & Vo, H. D. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112152 · Cited by 5
Development of dynamic protection against timing channels
Kananizadeh, S., & Kononenko, K. (2016) · International Journal of Information Security · doi:10.1007/s10207-016-0356-7 · Cited by 4
The effectiveness of context-based change application on automatic program repair
Kim, J., Kim, J., Lee, E., & Kim, S. (2019) · Empirical Software Engineering · doi:10.1007/s10664-019-09770-1 · Cited by 4
Does Going Beyond Branch Coverage Make Program Repair Tools More Reliable?
Nilizadeh, A., Leavens, G. T., Păsăreanu, C. S., Le, X.-B. D., & Cok, D. R. (2024) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst60714.2024.00033 · Cited by 4
Woodward, J. R., Johnson, C. G., & Brownlee, A. E. I. (2016) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/2908961.2931728 · Cited by 4
Clarifications on the Construction and Use of the ManyBugs Benchmark
Goues, C. L., Brun, Y., Forrest, S., & Weimer, W. R. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2017.2755651 · Cited by 4
Self-citationTrident: Controlling Side Effects in Automated Program Repair
Parasaram, N., Barr, E. T., & Mechtaev, S. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3124323 · Cited by 4
Self-citationHow to Measure the Performance of Automated Program Repair
Qi, Y., Liu, W., Zhang, W., & Yang, D. (2018) · International Conference on Information Science and Control Engineering · doi:10.1109/icisce.2018.00059 · Cited by 4
Automatically Repairing Programs Using Both Tests and Bug Reports.Preprint
Motwani, M., & Brun, Y. (2020) · arXiv (Cornell University) · Cited by 4
Self-citationAre “Solved Issues” in SWE-bench Really Solved Correctly? An Empirical Study
Wang, Y., Pradel, M., & Liu, Z. (2026) · Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering · doi:10.1145/3744916.3764576 · Cited by 3
“However, due to practical limitations, test suites are rarely exhaustive and often suffer from weaknesses [50].”
EffFix: Efficient and Effective Repair of Pointer Manipulating Programs
Zhang, Y., Costea, A., Shariffdeen, R., McCall, D., & Roychoudhury, A. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3705310 · Cited by 3
RePurr: Automated Repair of Block-Based Learners’ Programs
Schweikl, S., & Fraser, G. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3715786 · Cited by 6
When debugging encounters artificial intelligence: state of the art and open challenges
Song, Y., Xie, X., & Xu, B. (2024) · Science China Information Sciences · doi:10.1007/s11432-022-3803-9 · Cited by 4
On the Time Performance of Automated Fixes
Yi, J. (2019) · Advances in intelligent systems and computing · doi:10.1007/978-3-030-14687-0_28 · Cited by 3
PRF: a framework for building automatic program repair prototypes for JVM-based languages
Ghanbari, A., & Marcus, A. (2020) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3368089.3417929 · Cited by 3
M3V: Multi-modal Multi-view Context Embedding for Repair Operator Prediction
Xu, X., Wang, X., & Xue, J. (2022) · Proceedings of the CGO/Proceedings of the ... CGO · doi:10.1109/cgo53902.2022.9741261 · Cited by 3
Evolutionary Testing for Program Repair
Ruan, H., Nguyen, H. L., Shariffdeen, R., Noller, Y., & Roychoudhury, A. (2024) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst60714.2024.00058 · Cited by 3
QEDCartographer: Automating Formal Verification Using Reward-Free Reinforcement Learning
Sanchez-Stern, A., Varghese, A., Kaufman, Z., Zhang, D., Ringer, T., & Brun, Y. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00033 · Cited by 4
Self-citationRevisiting Unnaturalness for Automated Program Repair in the Era of Large Language Models
Yang, A. Z. H., Kolak, S., Hellendoorn, V. J., Martins, R., & Goues, C. L. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00089 · Cited by 22
Multiplicative Weights Algorithms for Parallel Automated Software Repair
Renzullo, J., Weimer, W. R., & Forrest, S. (2021) · IEEE International Parallel and Distributed Processing Symposium (IPDPS) · doi:10.1109/ipdps49936.2021.00107 · Cited by 3
Progress on Approaches to Automatic Program Repair
Xuan, J., Zhi-Lei, R., Wang, Z., Xie, X., & Jiang, H. (2016) · Cited by 3
A revisit of the integration of metamorphic testing and test suite based automated program repair
Jiang, M., Chen, T. Y., Kuo, F., Ding, Z., Choi, E.-H., & Mizuno, O. (2017) · doi:10.5555/3103620.3103628 · Cited by 3
Wang, Y., Chen, Y., Shen, B., & Zhong, H. (2017) · Asia-Pacific Symposium on Internetware · doi:10.1145/3131704.3131720 · Cited by 3
An Evolutionary System for Better Automatic Software Repair
Yuan, Y., & Banzhaf, W. (2020) · Genetic and evolutionary computation · doi:10.1007/978-3-030-39958-0_19 · Cited by 3
Review4Repair: Code Review Aided Automatic Program Repairing
Huq, F., Hasan, M., Haque, M. A. P., Mahbub, S., Iqbal, A., & Ahmed, T. (2021) · Figshare · doi:10.5281/zenodo.4445746 · Cited by 3
When Automated Program Repair Meets Regression Testing—An Extensive Study on Two Million Patches
Lou, Y., Yang, J., Benton, S., Hao, D., Tan, L., Chen, Z., Zhang, L. H., & Zhang, L. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3672450 · Cited by 2
An Empirical Study on the Suitability of Test-based Patch Acceptance Criteria
Zemín, L., Godio, A., Cornejo, C., Degiovanni, R., Brida, S. G., Regis, G., Aguirre, N., & Frias, M. F. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3702971 · Cited by 2
Enhancing APR with PRISM: A Semantic-Based Approach to Overfitting Patch Detection
Song, D., & Oh, H. (2025) · Proceedings of the ACM on Programming Languages · doi:10.1145/3763170 · Cited by 2
Debugging the Undebuggable: Why Multi-Fault Programs Break Debugging and Repair Tools
Al-Bataineh, O. I. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00320 · Cited by 2
FURINA: A Multi-Agent Framework for Automated Patch Correctness Assessment
Li, Y., Ji, Y., Chen, X., Shen, Z., Ge, J., & Luo, B. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3811825 · Cited by 2
Heterogeneous Prompting and Execution Feedback for SWE Issue Test Generation and Selection
Ahmed, T., Ganhotra, J., Shinnar, A., & Hirzel, M. J. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3744916.3787837 · Cited by 2
Understanding Why and Predicting When Developers Adhere to Code-Quality Standards
Motwani, M., & Brun, Y. (2023) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip58684.2023.00045 · Cited by 2
Self-citationBenchmarking and Categorizing the Performance of Neural Program Repair Systems for Java
Zhong, W., Li, C., Liu, K., Ge, J., Luo, B., Bissyandé, T. F., & Ng, V. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3688834 · Cited by 3
The Sustainability Face of Automated Program Repair Tools
Martínez, M., Martínez‐Fernández, S., & Franch, X. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3744900 · Cited by 2
We’ve Got You Covered: Type-Guided Repair of Incomplete Input Generators
LaFontaine, P., Zhou, Z., Mishra, A., Jagannathan, S., & Delaware, B. (2025) · Proceedings of the ACM on Programming Languages · doi:10.1145/3763158 · Cited by 1
Refining Fitness Functions for Search-Based Automated Program Repair
Guizzo, G., Blot, A., Callan, J., Petke, J., & Sarro, F. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-88106-1_12 · Cited by 2
Guiding Unconstrained Genetic Improvement
White, D. R. (2016) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/2908961.2931688 · Cited by 2
Automated workarounds from Java program specifications based on SAT solving
Uva, M., Ponzio, P., Regis, G., Aguirre, N., & Frias, M. F. (2018) · International Journal on Software Tools for Technology Transfer · doi:10.1007/s10009-018-0503-8 · Cited by 2
[Journal First] Do Automated Program Repair Techniques Repair Hard and Important Bugs?
Motwani, M., Sankaranarayanan, S., Just, R., & Brun, Y. (2018) · Cited by 2
Self-citationDigging into Semantics: Where Do Search-Based Software Repair Methods Search?
Ahmad, H., Cashin, P., Forrest, S., & Weimer, W. R. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-14721-0_1 · Cited by 2
Interactive Patch Filtering via Test Generation
Zhang, Q., Zhai, X., Xu, S., Huang, W., Zhang, J., & Fan, Y. (2022) · International Conference on Dependable Systems and Their Applications (DSA) · doi:10.1109/dsa56465.2022.00015 · Cited by 2
Interactive Patch Filtering as Debugging AidPreprint
Liang, J., Ji, R., Jiang, J., Lou, Y., Xiong, Y., & Huang, G. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2004.08746 · Cited by 2
Adversarial Patch Generation for Automatic Program Repair.Preprint
Alhefdhi, A., Dam, H. K., Le, X.-B. D., & Ghose, A. (2020) · arXiv (Cornell University) · Cited by 2
Checking Patch Behaviour against Test SpecificationPreprint
Tian, H., Li, Y., Pian, W., Kaboré, A. K., Liu, K., Klein, J., & Bissyandé, T. F. (2021) · arXiv (Cornell University) · Cited by 2
Automatic unit test amplification for DevOps
Danglot, B. (2019) · doi:10.70675/5747c493z5931z462bz8c5az0f897179fd20 · Cited by 2
The Evolution of Automated Software Repair
Goues, C. L., Nguyen, T. H., Forrest, S., & Weimer, W. R. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3533309 · Cited by 1
MultiMend: multilingual program repair with context augmentation and multi-hunk patch generation
Gharibi, R., Sadreddini, M. H., & Fakhrahmad, S. M. (2026) · Automated Software Engineering · doi:10.1007/s10515-026-00611-2 · Cited by 7
Li, Y., Zhong, W., Shen, Z., Li, C., Chen, X., Ge, J., & Luo, B. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00502-y · Cited by 2
Yang, Y., Li, C., Han, Z., Li, R., Xu, K., Li, Q., Zhong, W., Shen, Z., Fei, Z., Ge, J., et al. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3764584 · Cited by 7
P³: Reasoning about Patches via Product Programs
Sharma, A., Schemmel, D., & Cadar, C. (2025) · Proceedings of the ACM on Programming Languages · doi:10.1145/3763145 · Cited by 1
Specification-Guided Repair of Arithmetic Errors in Dafny Programs Using LLMs
Wu, V., Mendes, A., & Abreu, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-10444-1_16 · Cited by 2
Iter-T: ITERative Test Suite Generation for Automated Program Repair
Godio, A., Brida, S. G., Regis, G., Bagheri, H., Nguyen, T. H., Aguirre, N., & Frias, M. F. (2026) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2026.3671416 · Cited by 1
Debugging Engine Enhanced by Prior Knowledge: Can We Teach LLM How to Debug?
Li, K., Wu, S., Tang, X., Yao, C., Bu, S., Xu, Q., & Chen, G. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797110 · Cited by 1
The Undecidability of Overfitting in Automated Program Repair
Al-Bataineh, O. I. (2026) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3786582.3786802 · Cited by 1
Automated Workarounds from Java Program Specifications Based on SAT Solving
Uva, M., Ponzio, P., Regis, G., Aguirre, N., & Frias, M. F. (2017) · Lecture notes in computer science · doi:10.1007/978-3-662-54494-5_20 · Cited by 1
Hey APR! Integrate Our Fault Localization Skill: Toward Better Automated Program Repair
Yamate, K., Kondo, M., Kashiwa, Y., Kamei, Y., & Ubayashi, N. (2022) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac54236.2022.00101 · Cited by 1
A method to identify overfitting program repair patches based on expression tree
Dong, Y., Cheng, X., Yang, Y., Zhang, L., Wang, S., & Kong, L. (2024) · Science of Computer Programming · doi:10.1016/j.scico.2024.103105 · Cited by 1
GenProgJS: A Baseline System for Test-Based Automated Repair of JavaScript Programs
Csuvik, V., Horváth, D., Lajkó, M., & Vidács, L. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3497798 · Cited by 1
Which Inputs Trigger My Patch?
Eberlein, M., Raselimo, M., & Grunske, L. (2025) · IEEE/ACM International Workshop on Automated Program Repair (APR) · doi:10.1109/apr66717.2025.00006 · Cited by 1
Empirical Analysis of 1-edit Degree Patches in Syntax-Based Automatic Program Repair
Dziurzański, P., Gerasimou, S., Kolovos, D. S., & Matragkas, N. (2020) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec48606.2020.9185913 · Cited by 1
Extending Context Awareness by Anticipating Uncertainty with Enki and Darjeeling
Jahan, S., Riley, I., Walter, C., & Gamble, R. (2020) · IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C) · doi:10.1109/acsos-c51401.2020.00051 · Cited by 1
EqFix: Fixing LaTeX Equation Errors by Examples
Zhu, F., & He, F. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-21213-0_7 · Cited by 1
Production-Driven Patch Generation and ValidationPreprint
Durieux, T., Hamadi, Y., & Monperrus, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1609.06848 · Cited by 1
Can defects be fixed with weak test suites? An analysis of 50 defects from Defects4JPreprint
Jiang, J., & Xiong, Y. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.04149 · Cited by 1
Obstacles in Fully Automatic Program Repair: A surveyPreprint
Mousavi, S. A., Babani, D. A., & Flammini, F. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2011.02714 · Cited by 1
When Automated Program Repair Meets Regression Testing -- An Extensive Study on 2 Million PatchesPreprint
Lou, Y., Yang, J., Benton, S., Hao, D., Tan, L., Chen, Z., Zhang, L., & Zhang, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2105.07311 · Cited by 1
Poster: Automated Program Repair with Canonical Constraints
Hill, A., Păsăreanu, C. S., & Stolee, K. T. (2018)
[Journal First] A Correlation Study Between Automated Program Repair and Test-Suite Metrics
Yi, J., Tan, S. H., Mechtaev, S., Böhme, M., & Roychoudhury, A. (2018)
Automated program repair: a perspective from metamorphic testing
Jiang, M. (2017) · Swinburne figshare (Swinburne University of Technology) · doi:10.25916/sut.26278918
Dissection on Java Organs in GitHub Repositories
Wang, S., Mao, X., & Yi, X. (2019) · IEEE Access · doi:10.1109/access.2019.2917330
Patch Quality and Diversity of Invariant-Guided Search-Based Program RepairPreprint
Ding, Z. Y. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.11667
Trusted and Resilient Mission Operations
Weimer, W. R., Davidson, J. W., Forrest, S., Goues, C. L., Pal, P. P., & Smith, E. (2020)
Self-citationCrossFix: Collaborative bug fixing by recommending similar bugsPreprint
Tan, S. H., Li, Z., & Yan, L. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.13453
Challenging the Stigma of Formal Program Repair
Rothenberg, B.-C. (2021) · IEEE/ACM International Workshop on Automated Program Repair (APR) · doi:10.1109/apr52552.2021.00015
Interactive Patch Filtering as Debugging Aid
Jingjing, L., Ji, R., Jiang, J., Zhou, S., Lou, Y., Xiong, Y., & Huang, G. (2021) · doi:10.26226/morressier.613b5418842293c031b5b610
Metadata
- Authors
- Smith, E. K., Barr, E. T., Le Goues, C., & Brun, Y.
- Affiliations
- University of Massachusetts Amherst (US)
- University College London (GB)
- Carnegie Mellon University, USA
- University of Massachusetts, USA
- Venue
- Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering
- Publisher
- ACM
- Published
- 2015-08-30
- Pages
- 532-543
- OpenAlex
- W2061575154
- Cited by
- 323
- References
- 68
- Citations by year
- 2026: 18 · 2025: 25 · 2024: 33 · 2023: 22 · 2022: 29 · 2021: 26 · 2020: 37 · 2019: 40 · 2018: 43 · 2017: 27 · 2016: 19 · 2015: 4
- FWCI
- 23.83
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) (publishedVersion)
- Funders
- National Science Foundation (CCF-1446683, CCF-1446966, CCF-1453508)
- Microsoft Research (Software Engineering Innovation Foundation Award)
- Topics
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
- Keywords
- automated program repair, overfitting, patch evaluation, test suite coverage, novice programmers, program correctness
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
