The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each
Abstract
There are more bugs in real-world programs than human programmers can realistically address. This paper evaluates two research questions: “What fraction of bugs can be repaired automatically?” and “How much does it cost to repair a bug automatically?” In previous work, we presented GenProg, which uses genetic programming to repair defects in off-the-shelf C programs. To answer these questions, we: (1) propose novel algorithmic improvements to GenProg that allow it to scale to large programs and find repairs 68% more often, (2) exploit GenProg's inherent parallelism using cloud computing resources to provide grounded, human-competitive cost measurements, and (3) generate a large, indicative benchmark set to use for systematic evaluations. We evaluate GenProg on 105 defects from 8 open-source programs totaling 5.1 million lines of code and involving 10,193 test cases. GenProg automatically repairs 55 of those 105 defects. To our knowledge, this evaluation is the largest available of its kind, and is often two orders of magnitude larger than previous work in terms of code or test suite size or defect count. Public cloud computing prices allow our 105 runs to be reproduced for $403; a successful repair completes in 96 minutes and costs $7.32, on average.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Genetic Programming: On the Programming of Computers by Means of Natural Selection
Koza, J. R. (1992) · Medical Entomology and Zoology · Cited by 13,082
Uniform crossover in genetic algorithms
Syswerda, G. (1989) · international conference on Genetic algorithms · Cited by 1,965
Empirical evaluation of the tarantula automatic fault-localization technique
Jones, J. A., & Harrold, M. J. (2005) · Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering · doi:10.1145/1101908.1101949 · Cited by 1,238
Anvik, J., Hiew, L., & Murphy, G. C. (2006) · Proceedings of the 28th international conference on Software engineering · doi:10.1145/1134285.1134336 · Cited by 924
Automatically finding patches using genetic programming
Weimer, W., Nguyen, T., Le Goues, C., & Forrest, S. (2009) · 2009 IEEE 31st International Conference on Software Engineering · doi:10.1109/icse.2009.5070536 · Cited by 761
Self-citationThe Current State and Future of Search Based Software Engineering
Harman, M. (2007) · Future of Software Engineering (FOSE '07) · doi:10.1109/fose.2007.29 · Cited by 631
Scalable statistical bug isolation
Liblit, B., Naik, M., Zheng, A. X., Aiken, A., & Jordan, M. I. (2005) · Proceedings of the 2005 ACM SIGPLAN conference on Programming language design and implementation · doi:10.1145/1065010.1065014 · Cited by 472
Automatically patching errors in deployed software
Perkins, J. H., Kim, S., Larsen, S., Amarasinghe, S., Bachrach, J., Carbin, M., Pacheco, C., Sherwood, F., Sidiroglou, S., Sullivan, G., et al. (2009) · Proceedings of the ACM SIGOPS 22nd symposium on Operating systems principles · doi:10.1145/1629575.1629585 · Cited by 409
How Long Will It Take to Fix This Bug?
Weiss, C., Premraj, R., Zimmermann, T., & Zeller, A. (2007) · Fourth International Workshop on Mining Software Repositories (MSR'07:ICSE Workshops 2007) · doi:10.1109/msr.2007.13 · Cited by 375
Genetic Algorithms, Selection Schemes, and the Varying Effects of Noise
Miller, B. L., & Goldberg, D. E. (1996) · Evolutionary Computation · doi:10.1162/evco.1996.4.2.113 · Cited by 353
A novel co-evolutionary approach to automatic software bug fixing
Arcuri, A., & Yao, X. (2008) · 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence) · doi:10.1109/cec.2008.4630793 · Cited by 250
Yin, Z., Yuan, D., Zhou, Y., Pasupathy, S., & Bairavasundaram, L. (2011) · Proceedings of the 19th ACM SIGSOFT symposium and the 13th European conference on Foundations of software engineering · doi:10.1145/2025113.2025121 · Cited by 249
A genetic programming approach to automated software repair
Forrest, S., Nguyen, T., Weimer, W., & Le Goues, C. (2009) · Proceedings of the 11th Annual conference on Genetic and evolutionary computation · doi:10.1145/1569901.1570031 · Cited by 236
Self-citationAutomated fixing of programs with contracts
Wei, Y., Pei, Y., Furia, C. A., Silva, L. S., Buchholz, S., Meyer, B., & Zeller, A. (2010) · Proceedings of the 19th international symposium on Software testing and analysis · doi:10.1145/1831708.1831716 · Cited by 235
Using Mutation to Automatically Suggest Fixes for Faulty Programs
Debroy, V., & Wong, W. E. (2010) · 2010 Third International Conference on Software Testing, Verification and Validation · doi:10.1109/icst.2010.66 · Cited by 226
Modernizing Legacy Systems: Software Technologies, Engineering Process and Business Practices
Seacord, R. C., Plakosh, D., & Lewis, G. A. (2003) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 221
Dynamic test generation to find integer bugs in x86 binary linux programs
Molnár, D., Li, X. C., & Wagner, D. A. (2009) · Cited by 198
Sketching concurrent data structures
Solar-Lezama, A., Jones, C. G., & Bodik, R. (2008) · Proceedings of the 29th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/1375581.1375599 · Cited by 195
On Parameter Tuning in Search Based Software Engineering
Arcuri, A., & Fraser, G. (2011) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-23716-4_6 · Cited by 170
Yesterday, my Program Worked. Today, it Does Not. Why?
Zeller, A. (1999) · Lecture notes in computer science · doi:10.1007/3-540-48166-4_16 · Cited by 156
Automated atomicity-violation fixing
Jin, G., Song, L., Zhang, W., Lu, S., & Liblit, B. (2011) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/1993498.1993544 · Cited by 151
Countering Network Worms Through Automatic Patch Generation
Sidiroglou, S., & Keromytis, A. D. (2005) · IEEE Security & Privacy · doi:10.1109/msp.2005.144 · Cited by 136
Inference and enforcement of data structure consistency specifications
Demsky, B. C., Ernst, M. D., Guo, P. J., McCamant, S., Perkins, J. H., & Rinard, M. (2006) · International Symposium on Software Testing and Analysis · doi:10.1145/1146238.1146266 · Cited by 127
Weimer, W. R. (2006) · International Conference on Generative Programming and Component Engineering (GPCE) · doi:10.1145/1173706.1173734 · Cited by 123
Self-citationDIRA: Automatic Detection, Identification and Repair of Control-Hijacking Attacks.
Смирнов, А. В., & Chiueh, T. (2005) · Cited by 105
DebugAdvisor: a recommender system for debugging
Ashok, B., Joy, J. M., Liang, H., Rajamani, S. K., Srinivasa, G., & Vangala, V. (2009) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1595696.1595766 · Cited by 105
BugFix: A learning-based tool to assist developers in fixing bugs
Jeffrey, D., Feng, M., Gupta, N., & Gupta, R. (2009) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2009.5090029 · Cited by 81
Designing better fitness functions for automated program repair
Fast, E., Goues, C. L., Forrest, S., & Weimer, W. R. (2010) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/1830483.1830654 · Cited by 61
Self-citationAutomated Debugging Using Path-Based Weakest Preconditions
He, H., & Gupta, N. (2004) · Lecture notes in computer science · doi:10.1007/978-3-540-24721-0_20 · Cited by 51
Detecting and Escaping Infinite Loops with Jolt
Carbin, M., Misailovíc, S., Kling, M., & Rinard, M. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-22655-7_28 · Cited by 38
Fair and Balanced? Bias in bug-fix Datasets
Bird, C., Bachmann, A., Aune, E., Duffy, J., Bernstein, A., Filkov, V., & Dévanbu, P. (2009) · Cited by 30
Cited by
(2021) · doi:10.1109/tse.2019.2946563 · Cited by 574
Automatic patch generation learned from human-written patches
Kim, D., Nam, J., Song, J., & Kim, S. (2013) · 2013 35th International Conference on Software Engineering (ICSE) · doi:10.1109/icse.2013.6606626 · Cited by 449
Nopol: Automatic Repair of Conditional Statement Bugs in Java Programs
Xuan, J., Martinez, M., DeMarco, F., Clement, M., Marcote, S. L., Durieux, T., Le Berre, D., & Monperrus, M. (2017) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2560811 · Cited by 430
Mutation Testing Advances: An Analysis and Survey
(2019) · Advances in Computers · doi:10.1016/bs.adcom.2018.03.015 · Cited by 425
SEQUENCER: Sequence-to-Sequence Learning for End-to-End Program Repair
Chen, Z., Kommrusch, S. J., Tufano, M., Pouchet, L.-N., Poshyvanyk, D., & Monperrus, M. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2940179 · Cited by 396
Metadata
- Authors
- Le Goues, C., Dewey-Vogt, M., Forrest, S., & Weimer, W.
- Venue
- 2012 34th International Conference on Software Engineering (ICSE)
- Publisher
- IEEE
- Published
- 2012-06
- Pages
- 3-13
- ISSN
- 0270-5257 (ISSN-L)
- OpenAlex
- W4231241365
- Cited by
- 326
- References
- 31
- FWCI
- 11.37
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.