Cited by 372Software Engineering ResearchSoftware Testing and Debugging TechniquesSoftware Engineering Techniques and Practices

An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation

Open access
Tufano, Michele, Watson, Cody, Bavota, Gabriele, Penta, Massimiliano Di, White, Martin, Poshyvanyk, Denys·2019·ACM Transactions on Software Engineering and Methodology

Abstract

Millions of open source projects with numerous bug fixes are available in code repositories. This proliferation of software development histories can be leveraged to learn how to fix common programming bugs. To explore such a potential, we perform an empirical study to assess the feasibility of using Neural Machine Translation techniques for learning bug-fixing patches for real defects. First, we mine millions of bug-fixes from the change histories of projects hosted on GitHub in order to extract meaningful examples of such bug-fixes. Next, we abstract the buggy and corresponding fixed code, and use them to train an Encoder-Decoder model able to translate buggy code into its fixed version. In our empirical investigation, we found that such a model is able to fix thousands of unique buggy methods in the wild. Overall, this model is capable of predicting fixed patches generated by developers in 9--50% of the cases, depending on the number of candidate patches we allow it to generate. Also, the model is able to emulate a variety of different Abstract Syntax Tree operations and generate candidate patches in a split second.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3340544

Cites

107 of 107 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 2

  1. [51]51. S. Mechtaev, Y. Jooyong, and A. Roychoudhury. [n.d.]. Angelix: Scalable multiline program patch synthesis via sym- bolic analysis (ICSE'16).
  2. [54]54. Martin Monperrus and Matias Martinez. 2012. CVS-Vintage: A Dataset of 14 CVS Repositories of Java Software. (Dec. 2012). https://hal.archives-ouvertes.fr/hal-00769121 working paper or preprint.

Cited by

Sorted by citations

Authors
Tufano, M., Watson, C., Bavota, G., Penta, M. D., White, M., & Poshyvanyk, D.
Affiliations
  • William & Mary (US)
  • Università della Svizzera italiana (CH)
  • University of Sannio (IT)
Venue
ACM Transactions on Software Engineering and Methodology
Published
2019-09-02
Volume
28
Issue
4
Pages
1-29
ISSN
  • 1049-331X (ISSN-L)
  • 1557-7392
OpenAlex
W2905463374
Cited by
370
References
101
Citations by year
2026: 44 · 2025: 62 · 2024: 73 · 2023: 82 · 2022: 55 · 2021: 40 · 2020: 10 · 2019: 3 · 2018: 1
FWCI
34.22
Type
journal article
Language
en
Open access
Locations
  • ACM Transactions on Software Engineering and Methodology (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • NSF (CCF-1525902 and CCF-1815186)
  • SNF (175513)
  • Swiss National Science Foundation for the CCQR project
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
Keywords
neural machine translation, program translation, bug fixes, open source software, GitHub, encoder-decoder architecture, code mining, software defect repair

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review