Cited by 328Software Testing and Debugging TechniquesSoftware Engineering ResearchEvolutionary Algorithms and Applications

The DOI matches its Crossref record; 66 of the 68 references it cites were found in a registry or an index.

Is the cure worse than the disease? overfitting in automated program repair

Smith, Edward K., Barr, Earl T., Le Goues, Claire, Brun, Yuriy·2015·Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering

DOI 10.1145/2786805.2786825

Open this paper in CoChat — free account

Abstract

Automated program repair has shown promise for reducing the significant manual effort debugging requires. This paper addresses a deficit of earlier evaluations of automated repair techniques caused by repairing programs and evaluating generated patches' correctness using the same set of tests. Since tests are an imperfect metric of program correctness, evaluations of this type do not discriminate between correct patches and patches that overfit the available tests and break untested but desired functionality. This paper evaluates two well-studied repair tools, GenProg and TrpAutoRepair, on a publicly available benchmark of bugs, each with a human-written patch. By evaluating patches using tests independent from those used during repair, we find that the tools are unlikely to improve the proportion of independent tests passed, and that the quality of the patches is proportional to the coverage of the test suite used during repair. For programs that pass most tests, the tools are as likely to break tests as to fix them. However, novice developers also overfit, and automated repair performs no worse than these developers. In addition to overfitting, we measure the effects of test suite coverage, test suite provenance, and starting program quality, as well as the difference in quality between novice-developer-written and tool-generated patches when quality is assessed with a test suite independent from the one used for patch generation.

Integrity signals

Registry

Crossref · 10.1145/2786805.2786825

Cites

66 of 66 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Genetic Programming: On the Programming of Computers by Means of Natural Selection
Koza, J. R. (1992) · Medical Entomology and Zoology · Cited by 13,082
Modeling by shortest data description
Rissanen, J. (1978) · Automatica · doi:10.1016/0005-1098(78)90005-5 · Cited by 6,011
KLEE: unassisted and automatic generation of high-coverage tests for complex systems programs
Cadar, C., Dunbar, D., & Engler, D. (2008) · doi:10.5555/1855741.1855756 · Cited by 2,655
Defects4J: a database of existing faults to enable controlled testing studies for Java programs
Just, R., Jalali, D., & Ernst, M. D. (2014) · Proceedings of the 2014 International Symposium on Software Testing and Analysis · doi:10.1145/2610384.2628055 · Cited by 1,447
Mechanization in problem solving: The effect of Einstellung.
Luchins, A. S. (1942) · Psychological Monographs · doi:10.1037/h0093502 · Cited by 1,143
GenProg: A Generic Method for Automatic Software Repair
Goues, C. L., Nguyen, T. H., Forrest, S., & Weimer, W. R. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.104 · Cited by 1,084
Simplifying and isolating failure-inducing input
Zeller, A., & Hildebrandt, R. (2002) · IEEE Transactions on Software Engineering · doi:10.1109/32.988498 · Cited by 1,084
Automating string processing in spreadsheets using input-output examples
Gulwani, S. (2011) · ACM-SIGACT Symposium on Principles of Programming Languages · doi:10.1145/1926385.1926423 · Cited by 1,039
A practical guide for using statistical tests to assess randomized algorithms in software engineering
Arcuri, A., & Briand, L. (2011) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/1985793.1985795 · Cited by 989
An experimental evaluation of the assumption of independence in multiversion programming
KNIGHT, J. C., & Leveson, N. G. (1986) · IEEE Transactions on Software Engineering · doi:10.1109/tse.1986.6312924 · Cited by 768

Show all 68 references

Research works printed but not linked: 6

L. Chen and A. Avižienis . N-version programming: A faulttolerance approach to reliability of software operation . In IEEE International Symposium on Fault-Tolerant Computing (FTCS) , pages 3– 9 , 1978 . L. Chen and A. Avižienis. N-version programming: A faulttolerance approach to reliability of software operation. In IEEE International Symposium on Fault-Tolerant Computing (FTCS), pages 3–9, 1978.
10.5555/2486788.2486893 · DOI
10.5555/2337223.2337225 · DOI
10.5555/2818754.2818811 · DOI
10.5555/541177 · DOI
Research Triangle Institute . The economic impacts of inadequate infrastructure for software testing. NIST Planning Report 02-3, May 2002 . Research Triangle Institute. The economic impacts of inadequate infrastructure for software testing. NIST Planning Report 02-3, May 2002.

Sorted by citations

Items 101–200 of 328

Empirical Study on Synthesis Engines for Semantics-Based Program Repair
Le, X.-B. D., Lo, D., & Goues, C. L. (2016) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2016.68 · Cited by 31
APPT: Boosting Automated Patch Correctness Prediction via Fine-Tuning Pre-Trained Models
Zhang, Q., Fang, C., Sun, W., Liu, Y., He, T., Hao, X., & Chen, Z. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3354969 · Cited by 44
Accelerating Search-Based Program Repair
Mehne, B., Yoshida, H., Prasad, M. R., Sen, K., Gopinath, D., & Sarfraz Khurshid, D. (2018) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst.2018.00031 · Cited by 30
Repairnator patches programs automatically
Monperrus, M., Urli, S., Durieux, T., Martínez, M., Baudry, B., & Seinturier, L. (2019) · Ubiquity · doi:10.1145/3349589 · Cited by 30
Using a probabilistic model to predict bug fixes
Soto, M., & Goues, C. L. (2018) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2018.8330211 · Cited by 29
SpecRover: Code Intent Extraction via LLMs
Ruan, H., Zhang, Y., & Roychoudhury, A. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00080 · Cited by 45
Improved representation and genetic operators for linear genetic programming for automated program repair
de Oliveira, V. Z. B., de Souza, E. F., Goues, C. L., & Camilo-Junior, C. G. (2018) · Empirical Software Engineering · doi:10.1007/s10664-017-9562-9 · Cited by 29
Utilizing Source Code Embeddings to Identify Correct Patches
Csuvik, V., Horvath, D., Horváth, F., & Vidács, L. (2020) · IEEE International Workshop on Intelligent Bug Fixing (IBF) · doi:10.1109/ibf50092.2020.9034714 · Cited by 30
Human-In-The-Loop Automatic Program Repair
Böhme, M., Kapugama, C. G., & Pham, V.-T. (2020) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst46399.2020.00036 · Cited by 28
Comparing developer-provided to user-provided tests for fault localization and automated program repair
Just, R., Parnin, C., Drosos, I., & Ernst, M. D. (2018) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3213846.3213870 · Cited by 29
SpecRover: Code Intent Extraction via LLMs
Ruan, H., Zhang, Y., & Roychoudhury, A. (2024) · arXiv · doi:10.48550/arxiv.2408.02232 · Cited by 28
I Want My App That Way: Reclaiming Sovereignty Over Personal Devices
Kollnig, K., Datta, S., & Kleek, M. G. V. (2021) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/3411763.3451632 · Cited by 27
CirFix: automatically repairing defects in hardware design code
Ahmad, H., Huang, Y., & Weimer, W. R. (2022) · International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS) · doi:10.1145/3503222.3507763 · Cited by 27
CCRep: Learning Code Change Representations via Pre-Trained Code Model and Query Back
Liu, Z., Tang, Z., Xia, X., & Yang, X. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00014 · Cited by 34
Better Automatic Program Repair by Using Bug Reports and Tests Together
Motwani, M., & Brun, Y. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00109 · Cited by 37
Self-citation
TransplantFix: Graph Differencing-based Code Transplantation for Automated Program Repair
Yang, D., Mao, X., Chen, L., Xu, X., Lei, Y., Lo, D., & He, J. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3556893 · Cited by 25
Invalidator: Automated Patch Correctness Assessment via Semantic and Syntactic Reasoning
Le-Cong, T., Luong, D.-M., Le, X.-B. D., Lo, D., Tran, N.-H., Quang-Huy, B., & Huynh, Q. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3255177 · Cited by 33
An Empirical Analysis of the Influence of Fault Space on Search-Based Automated Program RepairPreprint
Wen, M., Chen, J., Wu, R., Hao, D., & Cheung, S.-C. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1707.05172 · Cited by 23
Contract-Based Program Repair Without The Contracts: An Extended Study
Chen, L., Pei, Y., & Furia, C. A. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2020.2970009 · Cited by 23
SmartFix: Fixing Vulnerable Smart Contracts by Accelerating Generate-and-Verify Repair using Statistical Models
So, S., & Oh, H. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3616341 · Cited by 22
Automatic diagnosis and correction of logical errors for functional programming assignments
Lee, J., Song, D., So, S., & Oh, H. (2018) · Proceedings of the ACM on Programming Languages · doi:10.1145/3276528 · Cited by 22
Leveraging Program Invariants to Promote Population Diversity in Search-Based Automatic Program Repair
Ding, Z. Y., Lyu, Y., Timperley, C. S., & Goues, C. L. (2019) · IEEE/ACM International Workshop on Genetic Improvement (GI) · doi:10.1109/gi.2019.00011 · Cited by 22
Automatic Program Repair Using Formal Verification and Expression Templates
Nguyen, T.-T., Ta, Q.-T., & Chin, W.-N. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-11245-5_4 · Cited by 22
Understanding Automatically-Generated Patches Through Symbolic Invariant Differences
Cashin, P., Martinez, C., Weimer, W. R., & Forrest, S. (2019) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2019.00046 · Cited by 21
Improved Crossover Operators for Genetic Programming for Program Repair
de Oliveira, V. Z. B., de Souza, E. F., Goues, C. L., & Camilo-Junior, C. G. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-47106-8_8 · Cited by 20
The Fact Selection Problem in LLM-Based Program Repair
Parasaram, N., Yan, H., Yang, B., Flahy, Z., Qudsi, A., Ziaber, D., Barr, E. T., & Mechtaev, S. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00162 · Cited by 42
Self-citation
Watch out for this commit! A study of influential software changes
Li, D., Li, L., Kim, D., Bissyandé, T. F., Lo, D., & Traon, Y. L. (2019) · Journal of Software Evolution and Process · doi:10.1002/smr.2181 · Cited by 20
E-APR: Mapping the effectiveness of automated program repair techniques
Aleti, A., & Martínez, M. (2021) · Empirical Software Engineering · doi:10.1007/s10664-021-09989-x · Cited by 19
Interactive Patch Filtering as Debugging Aid
Liang, J., Ji, R., Jiang, J., Zhou, S., Lou, Y., Xiong, Y., & Huang, G. (2021) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme52107.2021.00028 · Cited by 19
ICEBAR: Feedback-Driven Iterative Repair of Alloy Specifications
Brida, S. G., Regis, G., Zheng, G., Bagheri, H., Nguyen, T. H., Aguirre, N., & Frias, M. F. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3556944 · Cited by 19
Better Code Search and Reuse for Better Program Repair
Qi, X., & Reiss, S. P. (2019) · IEEE/ACM International Workshop on Genetic Improvement (GI) · doi:10.1109/gi.2019.00012 · Cited by 22
Input Test Suites for Program Repair: A Novel Construction Method Based on Metamorphic Relations
Jiang, M., Chen, T. Y., Zhou, Z. Q., & Ding, Z. (2020) · IEEE Transactions on Reliability · doi:10.1109/tr.2020.3003313 · Cited by 19
Impact of tool support in patch construction
Koyuncu, A., Bissyandé, T. F., Kim, D., Klein, J., Monperrus, M., & Traon, Y. L. (2017) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3092703.3092713 · Cited by 18
Automated Program Repair, What Is It Good For? Not Absolutely Nothing!
Eladawy, H. M., Goues, C. L., & Brun, Y. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639095 · Cited by 18
Self-citation
Detection of Concurrency Errors in Multithreaded Applications Based on Static Source Code Analysis
Giebas, D., & Wojszczyk, R. (2021) · IEEE Access · doi:10.1109/access.2021.3073859 · Cited by 18
Program Repair Guided by Datalog-Defined Static Analysis
Liu, Y., Mechtaev, S., Subotić, P., & Roychoudhury, A. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3616363 · Cited by 17
CCBERT: Self-Supervised Code Change Representation Learning
Zhou, X., Xu, B., Han, D., Yang, Z., He, J., & Lo, D. (2023) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58846.2023.00028 · Cited by 21
Studying and Understanding the Tradeoffs Between Generality and Reduction in Software Debloating
Xin, Q., Zhang, Q., & Orso, A. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3556970 · Cited by 17
Aligning the Objective of LLM-Based Program Repair
Xu, J., Fu, Y., Tan, S. H., & He, P. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00169 · Cited by 47
Human-in-the-Loop Automatic Program Repair
Kapugama, C. G., Böhme, M., & Pham, V.-T. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3305052 · Cited by 16
Evolving Paradigms in Automated Program Repair: Taxonomy, Challenges, and Opportunities
Huang, K., Xu, Z., Yang, S., Sun, H., Li, X., Yan, Z., & Zhang, Y. (2024) · ACM Computing Surveys · doi:10.1145/3696450 · Cited by 16
Automatic and scalable detection of logical errors in functional programming assignments
Song, D., Lee, M., & Oh, H. (2019) · Proceedings of the ACM on Programming Languages · doi:10.1145/3360614 · Cited by 17
Restore: Retrospective Fault Localization Enhancing Automated Program Repair
Xu, T., Chen, L., Pei, Y., Zhang, T. Y., Pan, M., & Furia, C. A. (2020) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2020.2987862 · Cited by 19
Repairnator patches programs automatically
Monperrus, M., Urli, S., Durieux, T., Martínez, M., Baudry, B., & Seinturier, L. (2019) · LillOA (Université de Lille (University Of Lille)) · Cited by 16
More Reliable Test Suites for Dynamic APR by using Counterexamples
Nilizadeh, A., Calvo, M., Leavens, G. T., & Le, X.-B. D. (2021) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre52982.2021.00032 · Cited by 16
Search-Based Efficient Automated Program Repair Using Mutation and Fault Localization
Sun, S., Guo, J., Zhao, R., & Li, Z. (2018) · IEEE Annual International Computer Software and Applications Conference (COMPSAC) · doi:10.1109/compsac.2018.00030 · Cited by 16
NPEX
Lee, J., Hong, S., & Oh, H. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510186 · Cited by 15
Towards efficient and effective automatic program repair
Le, X.-B. D. (2016) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/2970276.2975934 · Cited by 15
Exploring Fitness and Edit Distance of Mutated Python Programs
Haraldsson, S. O., Woodward, J. R., Brownlee, A. E. I., & Cairns, D. R. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-55696-3_2 · Cited by 15
The impacts of techniques, programs and tests on automated program repair: An empirical study
Kong, X., Zhang, L., Wong, W. E., & Li, B. (2017) · Journal of Systems and Software · doi:10.1016/j.jss.2017.06.039 · Cited by 15
Generating counterexamples in the form of unit tests from hoare-style verification attempts
Nilizadeh, A., Calvo, M., Leavens, G. T., & Cok, D. R. (2022) · IEEE/ACM International Conference on Formal Methods in Software Engineering (FormaliSE) · doi:10.1145/3524482.3527656 · Cited by 15
Discovering common bug‐fix patterns: A large‐scale observational study
Campos, E. C., & Maia, M. D. A. (2019) · Journal of Software Evolution and Process · doi:10.1002/smr.2173 · Cited by 15
On the Efficiency of Test Suite based Program Repair: A Systematic Assessment of 16 Automated Repair Systems for Java Programs
Liu, K., Wang, S., Koyuncu, A., Kim, K., Bissyandé, T. F., Kim, D., Wu, P., Klein, J., Mao, X., & Traon, Y. L. (2020) · Open Repository and Bibliography (University of Luxembourg) · doi:10.48550/arxiv.2008.00914 · Cited by 14
ObjSim: lightweight automatic patch prioritization via object similarity
Ghanbari, A. (2020) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3395363.3404362 · Cited by 14
Evaluating the usage of fault localization in automated program repair: an empirical study
Yang, D., Qi, Y., Mao, X., & Lei, Y. (2020) · Frontiers of Computer Science · doi:10.1007/s11704-020-9263-1 · Cited by 14
Mining repair model for exception-related bug
Zhong, H., & Mei, H. (2018) · Journal of Systems and Software · doi:10.1016/j.jss.2018.03.046 · Cited by 14
A correlation study between automated program repair and test-suite metrics
Yi, J., Tan, S. H., Mechtaev, S., Böhme, M., & Roychoudhury, A. (2018) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3180155.3182517 · Cited by 14
Automated program repair: a step towards software automation
Roychoudhury, A., & Xiong, Y. (2019) · Science China Information Sciences · doi:10.1007/s11432-019-9947-6 · Cited by 13
Reliable Fix Patterns Inferred from Static Checkers for Automated Program Repair
Liu, K., Zhang, J., Li, L., Koyuncu, A., Kim, D., Ge, C., Liu, Z., Klein, J., & Bissyandé, T. F. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3579637 · Cited by 13
Do automated program repair techniques repair hard and important bugs?
Motwani, M., Sankaranarayanan, S., Just, R., & Brun, Y. (2018) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3180155.3182533 · Cited by 12
Self-citation
Review4Repair: Code review aided automatic program repairing
Huq, F., Hasan, M., Haque, M. M. A., Mahbub, S., Iqbal, A., & Ahmed, T. (2021) · Information and Software Technology · doi:10.1016/j.infsof.2021.106765 · Cited by 22
Basic block coverage for search-based unit testing and crash reproduction
Derakhshanfar, P., Devroey, X., & Zaidman, A. (2022) · Empirical Software Engineering · doi:10.1007/s10664-022-10155-0 · Cited by 12
Tackling Build Failures in Continuous Integration
Hassan, F. (2019) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2019.00150 · Cited by 12
Rete: Learning Namespace Representation for Program Repair
Parasaram, N., Barr, E. T., & Mechtaev, S. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00112 · Cited by 12
Self-citation
Deadlocks Detection in Multithreaded Applications Based on Source Code Analysis
Giebas, D., & Wojszczyk, R. (2020) · Applied Sciences · doi:10.3390/app10020532 · Cited by 12
Crex: Predicting patch correctness in automated repair of C programs through transfer learning of execution semantics
Yan, D., Liu, K., Niu, Y., Li, L., Liu, Z., Liu, Z., Klein, J., & Bissyandé, T. F. (2022) · Information and Software Technology · doi:10.1016/j.infsof.2022.107043 · Cited by 11
T5APR: Empowering automated program repair across languages through checkpoint ensemble
Gharibi, R., Sadreddini, M. H., & Fakhrahmad, S. M. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112083 · Cited by 15
Patch Correctness Assessment: A Survey
Fei, Z., Ge, J., Li, C., Wang, T., Li, Y., Zhang, H., Huang, L., & Luo, B. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3702972 · Cited by 11
A Large-Scale Empirical Review of Patch Correctness Checking Approaches
Yang, J., Wang, Y., Lou, Y., Wen, M., & Zhang, L. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3616331 · Cited by 24
Would You Fix This Code for Me? Effects of Repair Source and Commenting on Trust in Code Repair
Alarcon, G. M., Walter, C., Gibson, A. M., Gamble, R., Capiola, A., Jessup, S. A., & Ryan, T. J. (2020) · Systems · doi:10.3390/systems8010008 · Cited by 11
Reinforcement learning for mutation operator selection in automated program repair
Hanna, C., Blot, A., & Petke, J. (2025) · Automated Software Engineering · doi:10.1007/s10515-025-00501-z · Cited by 11
Rango: Adaptive Retrieval-Augmented Proving for Automated Software Verification
Thompson, K., Saavedra, N., Carrott, P., Fisher, K. D., Sanchez-Stern, A., Brun, Y., Ferreira, J. F., Lerner, S., & First, E. (2025) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse55347.2025.00161 · Cited by 14
Self-citation
Automated Program Repair and Test Overfitting: Measurements and Approaches using Formal Methods
Nilizadeh, A. (2022) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst53961.2022.00061 · Cited by 10
Practical Program Repair via Preference-based Ensemble Strategy
Zhong, W., Li, C., Liu, K., Xu, T., Ge, J., Bissyandé, T. F., Luo, B., & Ng, V. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3623310 · Cited by 14
A Survey of Test Based Automatic Program Repair
Liu, Y., Zhang, L., & Zhang, Z. (2018) · Journal of Software · doi:10.17706/jsw.13.8.437-452 · Cited by 10
Repairing Infrastructure-as-Code using Large Language Models
Low, E., Cheh, C., & Chen, B. (2024) · IEEE Secure Development Conference (SecDev) · doi:10.1109/secdev61143.2024.00008 · Cited by 9
AdverIntent-Agent: Adversarial Reasoning for Repair Based on Inferred Program Intent
Ye, H., Yang, A. Z. H., Hu, C., Wang, Y., Zhang, T., & Goues, C. L. (2025) · Proceedings of the ACM on software engineering. · doi:10.1145/3728939 · Cited by 11
PRoofster: Automated Formal Verification
Agrawal, A., First, E., Kaufman, Z., Reichel, T., Zhang, S., Zhou, T., Sanchez-Stern, A., Ringer, T., & Brun, Y. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-companion58688.2023.00018 · Cited by 9
Self-citation
User-Centric Deployment of Automated Program Repair at Bloomberg
Williams, D., Callan, J., Kirbas, S., Mechtaev, S., Petke, J., Prideaux-Ghee, T., & Sarro, F. (2024) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1145/3639477.3639756 · Cited by 9
Automatic Software Merging using Automated Program Repair
Xing, X., & Maruyama, K. (2019) · IEEE International Workshop on Intelligent Bug Fixing (IBF) · doi:10.1109/ibf.2019.8665493 · Cited by 9
Identifying Incorrect Patches in Program Repair Based on Meaning of Source Code
Phung, Q.-N., Kim, M., & Lee, E. (2022) · IEEE Access · doi:10.1109/access.2022.3145983 · Cited by 9
Automated Misconfiguration Repair of Configurable Cyber-Physical Systems with Search: an Industrial Case Study on Elevator Dispatching Algorithms
Valle, P., Arrieta, A., & Arratibel, M. (2023) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) · doi:10.1109/icse-seip58684.2023.00042 · Cited by 9
Improving Patch Correctness Analysis via Random Testing and Large Language Models
Molina, F., Copia, J. M., & Gorla, A. (2024) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst60714.2024.00036 · Cited by 8
A Sketching-Based Approach for Debugging Using Test Cases
Hua, J., & Sarfraz Khurshid, D. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46520-3_29 · Cited by 8
Generating Class-Level Integration Tests Using Call Site Information
Derakhshanfar, P., Devroey, X., Panichella, A., Zaidman, A., & Arie Van Deursen, A. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3209625 · Cited by 8
Rust-lancet: Automated Ownership-Rule-Violation Fixing with Behavior Preservation
Yang, W., Song, L., & Xue, Y. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3639103 · Cited by 8
A Comprehensive Review on Regression Test Case Prioritization Techniques for Web Services
Hasnain, M., Ghani, I., Pasha, M. F., Lim, C. H., & Jeong, S. R. (2020) · KSII Transactions on Internet and Information Systems · doi:10.3837/tiis.2020.05.001 · Cited by 8
Automatic Program Repair as Semantic Suggestions: An Empirical Study
Campos, D., Restivo, A., Ferreira, H. S., & Ramos, A. J. M. M. (2021) · IEEE International Conference on Software Testing, Verification and Validation (ICST) · doi:10.1109/icst49551.2021.00032 · Cited by 8
Language models can prioritize patches for practical program patching
Kang, S., & Yoo, S. (2022) · IEEE/ACM International Workshop on Automated Program Repair (APR) · doi:10.1145/3524459.3527343 · Cited by 15
An Analysis of the Suitability of Test-Based Patch Acceptance Criteria
Zemín, L., Brida, S. G., Godio, A., Cornejo, C., Degiovanni, R., Regis, G., Aguirre, N., & Frias, M. F. (2017) · IEEE/ACM International Workshop on Search-Based Software Testing (SBST) · doi:10.1109/sbst.2017.12 · Cited by 8
Vulnerability Repair via Concolic Execution and Code Mutations
Shariffdeen, R., Timperley, C. S., Noller, Y., Goues, C. L., & Roychoudhury, A. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3707454 · Cited by 11
Impact of Test Suite Coverage on Overfitting in Genetic Improvement of Software
Lim, M., Guizzo, G., & Petke, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59762-7_14 · Cited by 7
Exploring Plausible Patches Using Source Code Embeddings in JavaScript
Csuvik, V., Horváth, D., Lajkó, M., & Vidács, L. (2021) · IEEE/ACM International Workshop on Automated Program Repair (APR) · doi:10.1109/apr52552.2021.00010 · Cited by 7
Automated Program Repair from Fuzzing Perspective
Kim, Y., Han, S., Khamit, A. Y., & Yi, J. (2023) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3597926.3598101 · Cited by 7
The Future Can’t Help Fix The Past: Assessing Program Repair In The Wild
Kabadi, V., Kong, D., Xie, S., Bao, L., Prana, G. A. A., Le, T.-D. B., Le, X.-B. D., & Lo, D. (2023) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58846.2023.00017 · Cited by 7
RunBugRun: An executable dataset for automated program repair
Prenner, J. A., & Robbes, R. (2026) · Empirical Software Engineering · doi:10.1007/s10664-025-10790-3 · Cited by 33
Automated program repair using genetic programming and model checking
Zojaji, Z., Ladani, B. T., & Khalilian, A. (2016) · Applied Intelligence · doi:10.1007/s10489-016-0804-0 · Cited by 7
Towards Addressing the Patch Overfitting Problem
Qi, X. (2017) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse-c.2017.42 · Cited by 7
Toward introducing automated program repair techniques to industrial software development
Naitou, K., Tanikado, A., Matsumoto, S., Higo, Y., Kusumoto, S., Kirinuki, H., Kurabayashi, T., & Tanno, H. (2018) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1145/3196321.3196358 · Cited by 7
Uncovering Bugs in Code Coverage Profilers via Control Flow Constraint Solving
Wang, Y., Zhang, P., Sun, M., Lu, Z., Yang, Y., Tang, Y., Qian, J., Li, Z., & Zhou, Y. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3321381 · Cited by 6

Authors: Smith, E. K., Barr, E. T., Le Goues, C., & Brun, Y.
Affiliations: University of Massachusetts Amherst (US)
University College London (GB)
Carnegie Mellon University, USA
University of Massachusetts, USA
ORCID: Barr, Earl T.
Brun, Yuriy
Venue: Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering
Publisher: ACM
Published: 2015-08-30
Pages: 532-543
DOI: 10.1145/2786805.2786825
OpenAlex: W2061575154
Cited by: 323
References: 68
Citations by year: 2026: 18 · 2025: 25 · 2024: 33 · 2023: 22 · 2022: 29 · 2021: 26 · 2020: 37 · 2019: 40 · 2018: 43 · 2017: 27 · 2016: 19 · 2015: 4
FWCI: 23.83
Type: conference paper
Language: en
Open access: closed
Licence: www.acm.org/publications/policies/copyright_policy#Background (vor)
Locations: ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) (publishedVersion)
Funders: National Science Foundation (CCF-1446683, CCF-1446966, CCF-1453508)
Microsoft Research (Software Engineering Innovation Foundation Award)
Topics: Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
Physical Sciences › Computer Science › Information Systems › Software Engineering Research
Physical Sciences › Computer Science › Artificial Intelligence › Evolutionary Algorithms and Applications
Keywords: automated program repair, overfitting, patch evaluation, test suite coverage, novice programmers, program correctness

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Is the cure worse than the disease? overfitting in automated program repair

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks