An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation
Open accessAbstract
Millions of open source projects with numerous bug fixes are available in code repositories. This proliferation of software development histories can be leveraged to learn how to fix common programming bugs. To explore such a potential, we perform an empirical study to assess the feasibility of using Neural Machine Translation techniques for learning bug-fixing patches for real defects. First, we mine millions of bug-fixes from the change histories of projects hosted on GitHub in order to extract meaningful examples of such bug-fixes. Next, we abstract the buggy and corresponding fixed code, and use them to train an Encoder-Decoder model able to translate buggy code into its fixed version. In our empirical investigation, we found that such a model is able to fix thousands of unique buggy methods in the wild. Overall, this model is capable of predicting fixed patches generated by developers in 9--50% of the cases, depending on the number of candidate patches we allow it to generate. Also, the model is able to emulate a variety of different Abstract Syntax Tree operations and generate candidate patches in a split second.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Sequence to Sequence Learning with Neural Networks
Sutskever, I., Vinyals, O., & Le, Q. V. (2014) · arXiv · doi:10.48550/arxiv.1409.3215 · Cited by 17,766 · see citing works
Neural Machine Translation by Jointly Learning to Align and Translate
Bahdanau, D., Cho, K., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1409.0473 · Cited by 16,213 · see citing works
Effective Approaches to Attention-based Neural Machine Translation
Luong, T., Pham, H., & Manning, C. D. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1166 · Cited by 8,382 · see citing works
Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation
Cho, K., van Merrienboer, B., Gulcehre, C., Bahdanau, D., Bougares, F., Schwenk, H., & Bengio, Y. (2014) · arXiv · doi:10.48550/arxiv.1406.1078 · Cited by 3,317 · see citing works
Defects4J: a database of existing faults to enable controlled testing studies for Java programs
Just, R., Jalali, D., & Ernst, M. D. (2014) · Proceedings of the 2014 International Symposium on Software Testing and Analysis · doi:10.1145/2610384.2628055 · Cited by 1,447 · see citing works
Sequence Transduction with Recurrent Neural NetworksPreprint
Graves, A. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1211.3711 · Cited by 1,320 · see citing works
Statistical machine translation
Koehn, P. (2010) · Choice Reviews Online · doi:10.5860/choice.47-6293 · Cited by 1,309 · see citing works
Recurrent Continuous Translation Models
Kalchbrenner, N., & Blunsom, P. (2013) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d13-1176 · Cited by 1,277 · see citing works
GenProg: A Generic Method for Automatic Software Repair
Goues, C. L., Nguyen, T. H., Forrest, S., & Weimer, W. R. (2011) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2011.104 · Cited by 1,084 · see citing works
Statistical Machine Translation
Koehn, P. (2009) · Cambridge University Press eBooks · doi:10.1017/cbo9780511815829 · Cited by 874 · see citing works
Automatically finding patches using genetic programming
Weimer, W. R., Nguyen, T. H., Goues, C. L., & Forrest, S. (2009) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2009.5070536 · Cited by 768 · see citing works
A systematic review of software development cost estimation studies
Jørgensen, M., & Shepperd, M. (2007) · Brunel University Research Archive (BURA) (Brunel University London) · doi:10.1109/tse.2007.3 · Cited by 748 · see citing works
Effective Approaches to Attention-based Neural Machine TranslationPreprint
Luong, M.-T., Pham, H., & Manning, C. D. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1508.04025 · Cited by 742 · see citing works
Automatically learning semantic features for defect prediction
Wang, S., Liu, T., & Tan, L. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884804 · Cited by 720 · see citing works
Deep learning code fragments for code clone detection
White, M., Tufano, M., Vendome, C., & Poshyvanyk, D. (2016) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/2970276.2970326 · Cited by 613 · see citing works
Self-citationOn the naturalness of software
Hindle, A., Barr, E. T., Su, Z., Gabel, M., & Dévanbu, P. (2012) · doi:10.5555/2337223.2337322 · Cited by 574 · see citing works
Fine-grained and accurate source code differencing
Falleri, J., Morandat, F., Blanc, X., Martínez, M., & Monperrus, M. (2014) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/2642937.2642982 · Cited by 569 · see citing works
Code completion with statistical language models
Raychev, V., Vechev, M., & Yahav, E. (2014) · Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/2594291.2594321 · Cited by 555 · see citing works
Gu, X., Zhang, H., & Kim, S. (2018) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3180155.3180167 · Cited by 539 · see citing works
Roy, C. K., & Cordy, J. R. (2008) · 2008 16th IEEE International Conference on Program Comprehension · doi:10.1109/icpc.2008.41 · Cited by 533 · see citing works
Populating a Release History Database from version control and bug tracking systems
Fischer, M. D., Pinzger, M., & Gall, H. C. (2004) · International Conference on Software Maintenance · doi:10.1109/icsm.2003.1235403 · Cited by 531 · see citing works
Mechtaev, S., Yi, J., & Roychoudhury, A. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884807 · Cited by 515 · see citing works
Automatic patch generation by learning correct code
Long, F., & Rinard, M. (2016) · ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages (POPL) · doi:10.1145/2837614.2837617 · Cited by 493 · see citing works
Massive Exploration of Neural Machine Translation Architectures
Britz, D., Goldie, A., Luong, M.-T., & Le, Q. V. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1151 · Cited by 487 · see citing works
Massive Exploration of Neural Machine Translation Architectures
Britz, D., Goldie, A., Luong, M.-T., & Le, Q. (2017) · arXiv · doi:10.48550/arxiv.1703.03906 · Cited by 487 · see citing works
Gu, X., Zhang, H., Zhang, D., & Kim, S. (2016) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2950290.2950334 · Cited by 485 · see citing works
Automatic patch generation learned from human-written patches
Kim, D., Nam, J., Song, J., & Kim, S. (2013) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2013.6606626 · Cited by 471 · see citing works
Sajnani, H., Saini, V., Svajlenko, J., Roy, C. K., & Lopes, C. V. (2016) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/2884781.2884877 · Cited by 447 · see citing works
Automatic patch generation learned from human-written patches
Kim, D., Nam, J., Song, J., & Kim, S. (2013) · Cited by 438 · see citing works
Nopol: Automatic Repair of Conditional Statement Bugs in Java Programs
Xuan, J., Martínez, M., DeMarco, F., Clement, M., Marcote, S. R. L., Durieux, T., Berre, D. L., & Monperrus, M. (2016) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2016.2560811 · Cited by 430 · see citing works
SemFix: program repair via semantic analysis
Nguyen, H. D. T., Qi, D., Roychoudhury, A., & Chandra, S. (2013) · doi:10.5555/2486788.2486890 · Cited by 423 · see citing works
An analysis of patch plausibility and correctness for generate-and-validate patch generation systems
Qi, Z., Long, F., Achour, S., & Rinard, M. (2015) · Proceedings of the 2015 International Symposium on Software Testing and Analysis · doi:10.1145/2771783.2771791 · Cited by 420 · see citing works
Automatically patching errors in deployed software
Perkins, J. H., Kim, S., Larsen, S., Amarasinghe, S. P., Bachrach, J. R., Carbin, M., Pacheco, C., Sherwood, F., Sidiroglou, S., Sullivan, G., et al. (2009) · ACM SIGOPS Symposium on Operating Systems Principles (SOSP) · doi:10.1145/1629575.1629585 · Cited by 409 · see citing works
Monperrus, M. (2018) · ACM Computing Surveys · doi:10.1145/3105906 · Cited by 386 · see citing works
Zhou, J. H., Zhang, H., & Lo, D. (2012) · doi:10.5555/2337223.2337226 · Cited by 381 · see citing works
The Definitive ANTLR 4 Reference
Parr, T. (2013) · USF Scholarship Repository (University of San Francisco) · Cited by 381 · see citing works
Suggesting accurate method and class names
Allamanis, M., Barr, E. T., Bird, C., & Sutton, C. A. (2015) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2786805.2786849 · Cited by 379 · see citing works
How Long Will It Take to Fix This Bug?
Weiß, C., Premraj, R., Zimmermann, T., & Zeller, A. (2007) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr.2007.13 · Cited by 377 · see citing works
Is it a bug or an enhancement?
Antoniol, G., Ayari, K., Penta, M. D., Khomh, F., & Guéhéneuc, Y. (2008) · IEEE International Conference on Collaborative Advances in Software and Computing (CASCON) · doi:10.1145/1463788.1463819 · Cited by 367 · see citing works
Self-citationLe, X.-B. D., Lo, D., & Goues, C. L. (2016) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2016.76 · Cited by 341 · see citing works
A systematic study of automated program repair: Fixing 55 out of 105 bugs for $8 each
Goues, C. L., Dewey-Vogt, M., Forrest, S., & Weimer, W. R. (2012) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2012.6227211 · Cited by 327 · see citing works
The ManyBugs and IntroClass benchmarks for automated repair of C programs
Le Goues, C., Holtschulte, N., Smith, E. K., Brun, Y., Devanbu, P., Forrest, S., & Weimer, W. (2015) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2015.2454513 · Cited by 326 · cited 1× · see citing works
“o likely sources of imprecision in our analyses. To have enough training data, we mined bug-fixes in GitHub repositories rather than using curated bug-fix datasets such as Defects4j [34] or IntroClass[44], useful but very limited in size. To mitigate imprecisions in our datasets, we manually analyzed a sample of the e…”
MethodIs the cure worse than the disease? overfitting in automated program repair
Smith, E. K., Barr, E. T., Le Goues, C., & Brun, Y. (2015) · Proceedings of the 2015 10th Joint Meeting on Foundations of Software Engineering · doi:10.1145/2786805.2786825 · Cited by 323 · see citing works
SourcererCC: Scaling Code Clone Detection to Big CodePreprint
Sajnani, H., Saini, V., Svajlenko, J., Roy, C. K., & Lopes, C. V. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1512.06448 · Cited by 306 · see citing works
It's not a bug, it's a feature: How misclassification impacts bug prediction
Herzig, K., Just, S., & Zeller, A. (2013) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1109/icse.2013.6606585 · Cited by 292 · see citing works
The adverse effects of code duplication in machine learning models of code
Allamanis, M. (2019) · Proceedings of the 2019 ACM SIGPLAN International Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software · doi:10.1145/3359591.3359735 · Cited by 284 · see citing works
Leveraging program equivalence for adaptive program repair: Models and first results
Weimer, W. R., Fry, Z. P., & Forrest, S. (2013) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2013.6693094 · Cited by 271 · see citing works
A novel co-evolutionary approach to automatic software bug fixing
Arcuri, A., & Yao, X. (2008) · IEEE Congress on Evolutionary Computation (CEC) · doi:10.1109/cec.2008.4630793 · Cited by 251 · see citing works
Automatic repair of real bugs in java: a large-scale experiment on the defects4j dataset
Martínez, M., Durieux, T., Sommerard, R., Xuan, J., & Monperrus, M. (2016) · Empirical Software Engineering · doi:10.1007/s10664-016-9470-4 · Cited by 249 · see citing works
Machine Learning-Based Prototyping of Graphical User Interfaces for Mobile Apps
Moran, K., Bernal-Cárdenas, C., Curcio, M., Bonett, R., & Poshyvanyk, D. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2018.2844788 · Cited by 242 · see citing works
Self-citationA study of the uniqueness of source code
Gabel, M., & Su, Z. (2010) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/1882291.1882315 · Cited by 220 · see citing works
S3: syntax- and semantic-guided repair synthesis via programming by examples
Le, X.-B. D., Chu, D.-H., Lo, D., Goues, C. L., & Visser, W. (2017) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3106237.3106309 · Cited by 219 · see citing works
The plastic surgery hypothesis
Barr, E. T., Brun, Y., Dévanbu, P., Harman, M., & Sarro, F. (2014) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2635868.2635898 · Cited by 217 · see citing works
On Learning Meaningful Code Changes Via Neural Machine Translation
Tufano, M., Pantiuchina, J., Watson, C., Bavota, G., & Poshyvanyk, D. (2019) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse.2019.00021 · Cited by 212 · see citing works
Self-citationModernizing Legacy Systems: Software Technologies, Engineering Process and Business Practices
Seacord, R. C., Plakosh, D., & Lewis, G. A. (2003) · Addison-Wesley Longman Publishing Co., Inc. eBooks · Cited by 208 · see citing works
Bug Localization with Combination of Deep Learning and Information Retrieval
Lam, A. N., Nguyen, A. T., Nguyen, H. A., & Nguyen, T. N. (2017) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2017.24 · Cited by 201 · see citing works
Repairing Programs with Semantic Code Search (T)
Ke, Y., Stolee, K. T., Goues, C. L., & Brun, Y. (2015) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2015.60 · Cited by 196 · see citing works
DirectFix: Looking for Simple Program Repairs
Mechtaev, S., Yi, J., & Roychoudhury, A. (2015) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1109/icse.2015.63 · Cited by 181 · see citing works
Automatic inference of code transforms for patch generation
Long, F., Amidon, P., & Rinard, M. (2017) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3106237.3106253 · Cited by 179 · see citing works
Parr, T., & Fisher, K. (2011) · ACM SIGPLAN Notices · doi:10.1145/1993316.1993548 · Cited by 178 · see citing works
Toward deep learning software repositories
White, M., Vendome, C., & Poshyvanyk, D. (2015) · Cited by 177 · see citing works
Self-citationSorting and Transforming Program Repair Ingredients via Deep Learning Code Similarities
White, M., Tufano, M., Martínez, M., Monperrus, M., & Poshyvanyk, D. (2019) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2019.8668043 · Cited by 167 · see citing works
Self-citationDirectFix: looking for simple program repairs
Mechtaev, S., Yi, J., & Roychoudhury, A. (2015) · UCL Discovery (University College London) · doi:10.5555/2818754.2818811 · Cited by 165 · see citing works
What's a Typical Commit? A Characterization of Open Source Software Repositories
Alali, A., Kagdi, H., & Maletic, J. I. (2008) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2008.24 · Cited by 163 · see citing works
Deep learning similarities from different representations of source code
Tufano, M., Watson, C., Bavota, G., Di Penta, M., White, M., & Poshyvanyk, D. (2018) · Proceedings of the 15th International Conference on Mining Software Repositories · doi:10.1145/3196398.3196431 · Cited by 158 · see citing works
Self-citationAutomated atomicity-violation fixing
Jin, G., Song, L., Zhang, W., Lu, S., & Liblit, B. (2011) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/1993498.1993544 · Cited by 151 · see citing works
Bug prediction based on fine-grained module histories
Hata, H., Mizuno, O., & Kikuno, T. (2012) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1109/icse.2012.6227193 · Cited by 149 · see citing works
Mining API mapping for language migration
Zhong, H., Thummalapenta, S., Xie, T., Zhang, L., & Wang, Q. (2010) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/1806799.1806831 · Cited by 146 · see citing works
Audio Chord Recognition With Recurrent Neural Networks.
Boulanger-Lewandowski, N., Bengio, Y., & Vincent, P. (2013) · Zenodo (CERN European Organization for Nuclear Research) · doi:10.5281/zenodo.1418318 · Cited by 145 · see citing works
A study of repetitiveness of code changes in software evolution
Nguyen, H. A., Nguyen, A. T., Nguyen, T. T., Nguyen, T. N., & Rajan, H. (2013) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2013.6693078 · Cited by 138 · see citing works
Lexical statistical machine translation for language migration
Nguyen, A. T., Nguyen, T. T., & Nguyen, T. N. (2013) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/2491411.2494584 · Cited by 136 · see citing works
Automated atomicity-violation fixing
Jin, G., Song, L., Zhang, W., Lu, S., & Liblit, B. (2011) · ACM SIGPLAN Notices · doi:10.1145/1993316.1993544 · Cited by 135 · see citing works
Tufano, M., Watson, C., Bavota, G., Penta, M. D., White, M., & Poshyvanyk, D. (2018) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3238147.3240732 · Cited by 133 · see citing works
Self-citationBetter test cases for better automated program repair
Yang, J., Zhikhartsev, A., Liu, Y., & Tan, L. (2017) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3106237.3106274 · Cited by 124 · see citing works
Automatic error elimination by horizontal code transfer across multiple applications
Sidiroglou, S., Lahtinen, E., Long, F., & Rinard, M. (2015) · ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) · doi:10.1145/2737924.2737988 · Cited by 114 · see citing works
sk_p: a neural program corrector for MOOCs
Pu, Y., Narasimhan, K., Solar-Lezama, A., & Barzilay, R. (2016) · ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity (SPLASH) · doi:10.1145/2984043.2989222 · Cited by 112 · see citing works
Dissection of a Bug Dataset:Anatomy of 395 Patches from Defects4J
Sobreira, V. (., Durieux, T., Madeiral, F., Monperrus, M., & Maia, M. A. (. A. ). (2018) · LillOA (Université de Lille (University Of Lille)) · Cited by 107 · see citing works
Dissection of a bug dataset: Anatomy of 395 patches from Defects4J
Sobreira, V., Durieux, T., Madeiral, F., Monperrus, M., & Maia, M. D. A. (2018) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2018.8330203 · Cited by 104 · see citing works
An empirical study on real bug fixes
Zhong, H., & Su, Z. (2015) · doi:10.5555/2818754.2818864 · Cited by 103 · see citing works
Parr, T., & Fisher, K. (2011) · Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation · doi:10.1145/1993498.1993548 · Cited by 98 · see citing works
Statistical learning approach for mining API usage mappings for code migration
Nguyen, A. T., Nguyen, H. A., Nguyen, T. T., & Nguyen, T. N. (2014) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/2642937.2643010 · Cited by 97 · see citing works
Automatic recovery from runtime failures
Carzaniga, A., Gorla, A., Mattavelli, A., Perino, N., & Pezzè, M. (2013) · doi:10.5555/2486788.2486891 · Cited by 81 · see citing works
Scholtes, I., Mavrodiev, P., & Schweitzer, F. (2015) · Empirical Software Engineering · doi:10.1007/s10664-015-9406-4 · Cited by 81 · see citing works
Martínez, M., Weimer, W. R., & Monperrus, M. (2014) · Cited by 81 · see citing works
Representations and operators for improving evolutionary software repair
Goues, C. L., Weimer, W. R., & Forrest, S. (2012) · Genetic and Evolutionary Computation Conference (GECCO) · doi:10.1145/2330163.2330296 · Cited by 76 · see citing works
Martinez, M., Weimer, W., & Monperrus, M. (2014) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1145/2591062.2591114 · Cited by 76 · see citing works
Automated Correction for Syntax Errors in Programming Assignments using Recurrent Neural NetworksPreprint
Bhatia, S., & Singh, R. K. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1603.06129 · Cited by 69 · see citing works
Migrating code with statistical machine translation
Nguyen, A. T., Nguyen, T. T., & Nguyen, T. N. (2014) · Proceedings/Proceedings - International Conference on Software Engineering · doi:10.1145/2591062.2591072 · Cited by 62 · see citing works
DeepAM: Migrate APIs with Multi-modal Sequence to Sequence Learning
Gu, X., Zhang, H., Zhang, D., & Kim, S. (2017) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2017/514 · Cited by 62 · see citing works
Automatically diagnosing and repairing error handling bugs in C
Tian, Y., & Ray, B. (2017) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3106237.3106300 · Cited by 61 · see citing works
The care and feeding of wild-caught mutants
Brown, D. B., Vaughn, M. L., Liblit, B., & Reps, T. (2017) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3106237.3106280 · Cited by 49 · see citing works
Tortoise: Interactive system configuration repair
Weiss, A., Guha, A., & Brun, Y. (2017) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase.2017.8115673 · Cited by 44 · see citing works
An empirical study on developer‐related factors characterizing fix‐inducing commits
Tufano, M., Bavota, G., Poshyvanyk, D., Penta, M. D., Oliveto, R., & Lucia, A. D. (2016) · Journal of Software Evolution and Process · doi:10.1002/smr.1797 · Cited by 37 · see citing works
Self-citationUsing a probabilistic model to predict bug fixes
Soto, M., & Goues, C. L. (2018) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner.2018.8330211 · Cited by 29 · see citing works
Automatic error elimination by horizontal code transfer across multiple applications
Sidiroglou, S., Lahtinen, E., Long, F., & Rinard, M. (2015) · ACM SIGPLAN Notices · doi:10.1145/2813885.2737988 · Cited by 29 · see citing works
Watch out for this commit! A study of influential software changes
Li, D., Li, L., Kim, D., Bissyandé, T. F., Lo, D., & Traon, Y. L. (2019) · Journal of Software Evolution and Process · doi:10.1002/smr.2181 · Cited by 20 · see citing works
Deep Reinforcement Learning for Programming Language CorrectionPreprint
Gupta, R., Kanade, A., & Shevade, S. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1801.10467 · Cited by 19 · see citing works
A Model of the Commit Size Distribution of Open Source
Kolassa, C., Riehle, D., & Salim, M. A. (2013) · Cited by 19 · see citing works
DeepAM: Migrate APIs with Multi-modal Sequence to Sequence LearningPreprint
Gu, X., Zhang, H., Zhang, D., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.07734 · Cited by 16 · see citing works
An empirical exploration of regularities in open-source software lexicons
Pierret, D., & Poshyvanyk, D. (2009) · IEEE International Conference on Program Comprehension (ICPC) · doi:10.1109/icpc.2009.5090047 · Cited by 14 · see citing works
Self-citationA Systematic Review of Software Development Cost Estimation Studies
JorgensenMagne, & ShepperdMartin (2007) · IEEE Transactions on Software Engineering · doi:10.5555/1248721.1248736 · Cited by 12 · see citing works
Watch out for This Commit! A Study of Influential Software ChangesPreprint
Li, D., Li, L., Kim, D., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.03266 · Cited by 10 · see citing works
The CodRep Machine Learning on Source Code CompetitionPreprint
Chen, Z., & Monperrus, M. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.03200 · Cited by 8 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [51]51. S. Mechtaev, Y. Jooyong, and A. Roychoudhury. [n.d.]. Angelix: Scalable multiline program patch synthesis via sym- bolic analysis (ICSE'16).
- [54]54. Martin Monperrus and Matias Martinez. 2012. CVS-Vintage: A Dataset of 14 CVS Repositories of Java Software. (Dec. 2012). https://hal.archives-ouvertes.fr/hal-00769121 working paper or preprint.
Cited by
Sorted by citations
Wang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · arXiv · doi:10.48550/arxiv.2109.00859 · Cited by 1,495 · see citing works
Evaluating Large Language Models Trained on Code
Chen, M., Tworek, J., Jun, H., Yuan, Q., Pinto, H. P. D. O., Kaplan, J., Edwards, H., Burda, Y., Joseph, N., Brockman, G., et al. (2021) · arXiv · doi:10.48550/arxiv.2107.03374 · Cited by 1,436 · see citing works
“More recent works (Tufano et al., 2019; Drain et al., 2021) considered bug-fixing as neural machine translation from buggy to correct programs.”
BackgroundWang, Y., Wang, W., Joty, S., & Hoi, S. C. H. (2021) · Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2021.emnlp-main.685 · Cited by 1,495 · see citing works
Unified Pre-training for Program Understanding and Generation
Ahmad, W., Chakraborty, S., Ray, B., & Chang, K.-W. (2021) · Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · doi:10.18653/v1/2021.naacl-main.211 · Cited by 642 · see citing works
Vaithilingam, P., Zhang, T., & Glassman, E. L. (2022) · CHI Conference on Human Factors in Computing Systems Extended Abstracts · doi:10.1145/3491101.3519665 · Cited by 660 · see citing works
CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation
Lu, S., Guo, D., Ren, S., Huang, J., Svyatkovskiy, A., Blanco, A., Clement, C., Drain, D., Jiang, D., Tang, D., et al. (2021) · arXiv · doi:10.48550/arxiv.2102.04664 · Cited by 536 · see citing works
Self-citationLarge language models for software engineering: survey and open problems
Fan, A., Gokkaya, B., Harman, M., Lyubarskiy, M., Sengupta, S., Yoo, S., & Zhang, J. M. (2023) · 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00008 · Cited by 650 · see citing works
SequenceR: Sequence-to-Sequence Learning for End-to-End Program Repair
Chen, Z.-M., Kommrusch, S., Tufano, M., Pouchet, L., Poshyvanyk, D., & Martin, M. (2018) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2019.2940179 · Cited by 585 · see citing works
Self-citationCURE: Code-Aware Neural Machine Translation for Automatic Program Repair
Jiang, N., Lutellier, T., & Tan, L. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00107 · Cited by 336 · see citing works
Studying the Usage of Text-To-Text Transfer Transformer to Support Code-Related Tasks
Mastropaolo, A., Scalabrino, S., Cooper, N., Palacio, D. N., Poshyvanyk, D., Oliveto, R., & Bavota, G. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00041 · Cited by 271 · see citing works
Self-citationCodeBLEU: a Method for Automatic Evaluation of Code Synthesis
Ren, S., Guo, D., Lu, S., Zhou, L., Liu, S., Tang, D., Sundaresan, N., Zhou, M., Blanco, A., & Ma, S. (2020) · arXiv · doi:10.48550/arxiv.2009.10297 · Cited by 214 · see citing works
Examining Zero-Shot Vulnerability Repair with Large Language Models
Pearce, H. A., Tan, B., Ahmad, B., Karri, R., & Dolan-Gavitt, B. (2023) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp46215.2023.10179420 · Cited by 206 · see citing works
Automating code review activities by large-scale pre-training
Li, Z., Lu, S., Guo, D., Duan, N., Jannu, S., Jenks, G., Majumder, D., Green, J., Svyatkovskiy, A., Fu, S., et al. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549081 · Cited by 226 · see citing works
InferFix: End-to-End Program Repair with LLMs
Jin, M., Shahriar, S., Tufano, M., Shi, X., Lu, S., Sundaresan, N., & Svyatkovskiy, A. (2023) · Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3611643.3613892 · Cited by 191 · see citing works
Self-citationUsing pre-trained models to boost code review automation
Tufano, R., Masiero, S., Mastropaolo, A., Pascarella, L., Poshyvanyk, D., & Bavota, G. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510621 · Cited by 182 · see citing works
Self-citationRetrieval-Based Prompt Selection for Code-Related Few-Shot Learning
Nashid, N., Sintaha, M., & Mesbah, A. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00205 · Cited by 178 · see citing works
Neural Transfer Learning for Repairing Security Vulnerabilities in C Code
Chen, Z., Kommrusch, S., & Monperrus, M. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3147265 · Cited by 174 · see citing works
Neural program repair with execution-based backpropagation
Ye, H., Martínez, M., & Monperrus, M. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510222 · Cited by 161 · see citing works
An extensive study on pre-trained models for program understanding and generation
Zeng, Z., Tan, H., Zhang, H., Li, J., Zhang, Y., & Zhang, L. (2022) · Proceedings of the 31st ACM SIGSOFT International Symposium on Software Testing and Analysis · doi:10.1145/3533767.3534390 · Cited by 153 · see citing works
On the Robustness of Code Generation Techniques: An Empirical Study on GitHub Copilot
Mastropaolo, A., Pascarella, L., Guglielmi, E., Ciniselli, M., Scalabrino, S., Oliveto, R., & Bavota, G. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00181 · Cited by 149 · see citing works
Self-citationTowards Automating Code Review Activities
Tufano, R., Pascarella, L., Tufano, M., Poshyvanyk, D., & Bavota, G. (2021) · International Conference on Software Engineering · doi:10.1109/icse43902.2021.00027 · Cited by 230 · see citing works
Self-citationA Systematic Literature Review on the Use of Deep Learning in Software Engineering Research
Watson, C., Cooper, N., Palacio, D. N., Moran, K., & Poshyvanyk, D. (2022) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3485275 · Cited by 137 · see citing works
Self-citationA Survey of Learning-based Automated Program Repair
Zhang, Q., Fang, C., Ma, Y., Sun, W., & Chen, Z. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3631974 · Cited by 153 · see citing works
Natural Language Generation and Understanding of Big Code for AI-Assisted Programming: A Review
Wong, M.-F., Guo, S., Hang, C. N., Ho, S., & Tan, C. W. (2023) · Entropy · doi:10.3390/e25060888 · Cited by 137 · see citing works
(2023) · doi:10.1109/sp46215.2023.10179324 · Cited by 131 · see citing works
Lu, J., Yu, L., Xiao-jia, L., Li, Y., & Zuo, C. (2023) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre59848.2023.00026 · Cited by 133 · see citing works
Karampatsis, R.-M., Sutton, C., Informatics, S. O., Institute for Machine Learning, & Intelligence, D. S. A. A. (2020) · Edinburgh Research Explorer (University of Edinburgh) · doi:10.5281/zenodo.3653444 · Cited by 117 · see citing works
Wei, Y., Xia, C. S., & Zhang, L. (2023) · Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3611643.3616271 · Cited by 115 · see citing works
An Empirical Study on Fine-Tuning Large Language Models of Code for Automated Program Repair
Huang, K., Meng, X., Zhang, J., Liu, Y., Wang, W., Li, S., & Zhang, Y. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00181 · Cited by 113 · see citing works
A Survey on Software Defect Prediction Using Deep Learning
Akimova, E. N., Bersenev, A. Y., Deikov, A. A., Kobylkin, K. S., Konygin, A. V., Mezentsev, I. P., & Misilov, V. E. (2021) · Mathematics · doi:10.3390/math9111180 · Cited by 118 · see citing works
Niu, C., Li, C., Ng, V., Ge, J., Huang, L., & Luo, B. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3510096 · Cited by 125 · see citing works
NatGen: generative pre-training by “naturalizing” source code
Chakraborty, S., Ahmed, T., Ding, Y., Devanbu, P. T., & Ray, B. (2022) · Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3540250.3549162 · Cited by 105 · see citing works
On Hardware Security Bug Code Fixes by Prompting Large Language Models
Ahmad, B., Thakur, S., Tan, B., Karri, R., & Pearce, H. A. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3374558 · Cited by 104 · see citing works
Perfection Not Required? Human-AI Partnerships in Code Translation
Weisz, J. D., Muller, M., Houde, S., Richards, J., Ross, S. I., Martinez, F., Agarwal, M., & Talamadupula, K. (2021) · 26th International Conference on Intelligent User Interfaces · doi:10.1145/3397481.3450656 · Cited by 129 · see citing works
Deep learning-based software engineering: progress, challenges, and opportunities
Chen, X., Hu, X., Huang, Y., Jiang, H., Ji, W., Jiang, Y., Jiang, Y., Liu, B., Liu, H., Li, X., et al. (2024) · Science China Information Sciences · doi:10.1007/s11432-023-4127-5 · Cited by 114 · see citing works
What Makes Good In-Context Demonstrations for Code Intelligence Tasks with LLMs?
Gao, S., Wen, X.-C., Gao, C., Wang, W., Zhang, H., & Lyu, M. R. (2023) · 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00109 · Cited by 106 · see citing works
Keeping Deep Learning Models in Check: A History-Based Approach to Mitigate Overfitting
Li, H., Rajbahadur, G. K., Lin, D., Bezemer, C., & Jiang, Z. M. (2024) · IEEE Access · doi:10.1109/access.2024.3402543 · Cited by 99 · see citing works
Code Generation Using Machine Learning: A Systematic Review
Dehaerne, E., Dey, B., Halder, S., Gendt, S. D., & Meert, W. (2022) · IEEE Access · doi:10.1109/access.2022.3196347 · Cited by 95 · see citing works
An Empirical Study on the Usage of Transformer Models for Code Completion
Ciniselli, M., Cooper, N., Pascarella, L., Mastropaolo, A., Aghajani, E., Poshyvanyk, D., Penta, M. D., & Bavota, G. (2021) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2021.3128234 · Cited by 101 · see citing works
Self-citationRAP-Gen: Retrieval-Augmented Patch Generation with CodeT5 for Automatic Program Repair
Wang, W.-S., Wang, Y., Joty, S. R., & Hoi, S. (2023) · ESEC/SIGSOFT FSE · doi:10.1145/3611643.3616256 · Cited by 151 · see citing works
SelfAPR: Self-supervised Program Repair with Test Execution Diagnostics
Ye, H., Martínez, M., Luo, X., Zhang, T., & Monperrus, M. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3556926 · Cited by 88 · see citing works
An Empirical Study on the Usage of BERT Models for Code Completion
Ciniselli, M., Cooper, N., Pascarella, L., Poshyvanyk, D., Penta, M. D., & Bavota, G. (2021) · International Conference on Mining Software Repositories (MSR) · doi:10.1109/msr52588.2021.00024 · Cited by 93 · see citing works
Self-citationCoditT5: Pretraining for Source Code and Natural Language Editing
Zhang, J., Panthaplackel, S., Nie, P., Li, J. J., & Gligoric, M. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3556955 · Cited by 89 · see citing works
A Deep Dive into Large Language Models for Automated Bug Localization and Repair
Hossain, S. B., Jiang, N., Zhou, Q., Li, X., Chiang, W.-H., Lyu, Y., Nguyen, H. A., & Tripp, O. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3660773 · Cited by 84 · see citing works
An Empirical Comparison of Pre-Trained Models of Source Code
Niu, C., Li, C., Ng, V., Chen, D., Ge, J., & Luo, B. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00180 · Cited by 90 · see citing works
Unit Test Case Generation with Transformers and Focal ContextPreprint
Tufano, M., Drain, D., Svyatkovskiy, A., Deng, S. K., & Sundaresan, N. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2009.05617 · Cited by 158 · see citing works
Self-citationLearning How to Mutate Source Code from Bug-Fixes
Tufano, M., Watson, C., Bavota, G., Penta, M. D., White, M., & Poshyvanyk, D. (2019) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme.2019.00046 · Cited by 74 · see citing works
Self-citationUsing Transfer Learning for Code-Related Tasks
Mastropaolo, A., Cooper, N., Palacio, D. N., Scalabrino, S., Poshyvanyk, D., Oliveto, R., & Bavota, G. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3183297 · Cited by 74 · see citing works
Self-citationCCT5: A Code-Change-Oriented Pre-trained Model
Lin, B., Wang, S., Liu, Z., Liu, Y., Xia, X., & Mao, X. (2023) · Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3611643.3616339 · Cited by 69 · see citing works
Using deep learning to generate complete log statements
Mastropaolo, A., Pascarella, L., & Bavota, G. (2022) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3510003.3511561 · Cited by 68 · see citing works
Self-citationContraBERT: Enhancing Code Pre-trained Models via Contrastive Learning
Liu, S., Wu, B., Xie, X., Meng, G., & Liu, Y. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00207 · Cited by 77 · see citing works
SeqTrans: Automatic Vulnerability Fix Via Sequence to Sequence Learning
Chi, J., Qu, Y., Liu, T., Zheng, Q., & Yin, H. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3156637 · Cited by 70 · see citing works
Semantic bug seeding: a learning-based approach for creating realistic bugs
Patra, J., & Pradel, M. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3468623 · Cited by 66 · see citing works
DOBF: A Deobfuscation Pre-Training Objective for Programming LanguagesPreprint
Rozière, B., Lachaux, M.-A., Szafraniec, M., & Lample, G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2102.07492 · Cited by 96 · see citing works
CIRCLE: continual repair across programming languages
Yuan, W., Zhang, Q., He, T., Fang, C., Nguyen, Q. V. H., Hao, X., & Yin, H. (2022) · ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA) · doi:10.1145/3533767.3534219 · Cited by 65 · see citing works
CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Models
Jha, A., & Reddy, C. K. A. (2023) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v37i12.26739 · Cited by 69 · see citing works
On the Reproducibility and Replicability of Deep Learning in Software Engineering
Liu, C. C., Gao, C., Xia, X., Lo, D., Grundy, J., & Yang, X. (2021) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3477535 · Cited by 64 · see citing works
Pre-Trained Model-Based Automated Software Vulnerability Repair: How Far are We?
Zhang, Q., Fang, C., Yu, B., Sun, W., Zhang, T., & Chen, Z. (2023) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2023.3308897 · Cited by 60 · see citing works
KNOD: Domain Knowledge Distilled Tree Decoder for Automated Program Repair
Jiang, N., Lutellier, T., Lou, Y., Tan, L., Goldwasser, D., & Zhang, X. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00111 · Cited by 61 · see citing works
AI for DevSecOps: A Landscape and Future Opportunities
Fu, M. C., Pasuksmit, J., & Tantithamthavorn, C. K. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3712190 · Cited by 83 · see citing works
A Survey on Machine Learning Techniques for Source Code AnalysisPreprint
Sharma, T., Kechagia, M., Georgiou, S., Tiwari, R. K., Vats, I., Moazen, H., & Sarro, F. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2110.09610 · Cited by 56 · see citing works
Evaluating SZZ Implementations Through a Developer-Informed Oracle
Rosa, G., Pascarella, L., Scalabrino, S., Tufano, R., Bavota, G., Lanza, M., & Oliveto, R. (2021) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse43902.2021.00049 · Cited by 50 · see citing works
Self-citationAutomatically Learning Patterns for Self-Admitted Technical Debt Removal
Zampetti, F., Serebrenik, A., & Penta, M. D. (2020) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner48275.2020.9054868 · Cited by 51 · see citing works
Self-citationCode Review Automation: Strengths and Weaknesses of the State of the Art
Tufano, R., Dabić, O., Mastropaolo, A., Ciniselli, M., & Bavota, G. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3348172 · Cited by 48 · see citing works
Self-citationOn Multi-Modal Learning of Editing Source Code
Chakraborty, S., & Ray, B. (2021) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase51524.2021.9678559 · Cited by 50 · see citing works
ITER: Iterative Neural Repair for Multi-Location Patches
Ye, H., & Monperrus, M. (2024) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3597503.3623337 · Cited by 48 · see citing works
Recent Advances of Foundation Language Models-based Continual Learning: A Survey
Yang, Y., Zhou, J., Ding, X., Huai, T., Liu, S., Chen, Q., Xie, Y., & He, L. (2024) · ACM Computing Surveys · doi:10.1145/3705725 · Cited by 57 · see citing works
CORE: Resolving Code Quality Issues using LLMs
Wadhwa, N., Pradhan, J., Sonwane, A., Sahu, S. P., Natarajan, N., Kanade, A., Parthasarathy, S., & Rajamani, S. K. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3643762 · Cited by 63 · see citing works
Generating bug-fixes using pretrained transformers
Drain, D., Wu, C. H., Svyatkovskiy, A., & Sundaresan, N. (2021) · MAPS@PLDI · doi:10.1145/3460945.3464951 · Cited by 49 · see citing works
On the Effectiveness of Transfer Learning for Code Search
Salza, P., Schwizer, C., Gu, J. N., & Gall, H. C. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3192755 · Cited by 47 · see citing works
Tufano, M., Deng, S. K., Sundaresan, N., & Svyatkovskiy, A. (2022) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3524842.3528009 · Cited by 43 · see citing works
Self-citationA survey on machine learning techniques applied to source code
Sharma, T., Kechagia, M., Georgiou, S., Tiwari, R., Vats, I., Moazen, H., & Sarro, F. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111934 · Cited by 130 · see citing works
Huang, Q. Y., Yuan, Z., Xing, Z., Xu, X., Zhu, L., & Lu, Q. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3556912 · Cited by 42 · see citing works
Better Patching Using LLM Prompting, via Self-Consistency
Ahmed, T., & Dévanbu, P. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00065 · Cited by 44 · see citing works
A comprehensive empirical study on bug characteristics of deep learning frameworks
Yang, Y., He, T., Xia, Z., & Feng, Y. (2022) · Information and Software Technology · doi:10.1016/j.infsof.2022.107004 · Cited by 41 · see citing works
Deep Learning Meets Software Engineering: A Survey on Pre-Trained Models of Source Code
Niu, C., Li, C., Luo, B., & Ng, V. (2022) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2022/775 · Cited by 53 · see citing works
Leveraging automated unit tests for unsupervised code translation
Roziere, B., Zhang, J. M., Charton, F., Harman, M., Synnaeve, G., & Lample, G. (2021) · arXiv · doi:10.48550/arxiv.2110.06773 · Cited by 39 · see citing works
Towards AI-Assisted Synthesis of Verified Dafny Methods
Misu, M. R. H., Lopes, C. V., Ma, I., & Noble, J. (2024) · Proceedings of the ACM on Software Engineering · doi:10.1145/3643763 · Cited by 38 · see citing works
State of Practice: LLMs in Software Engineering and Software Architecture
Jahić, J., & Sami, A. (2024) · IEEE International Conference on Software Architecture Companion (ICSA-C) · doi:10.1109/icsa-c63560.2024.00059 · Cited by 38 · see citing works
(2021) · arXiv (Cornell University) · Cited by 37 · see citing works
Towards JavaScript program repair with generative pre-trained transformer (GPT-2)
Lajkó, M., Csuvik, V., & Vidács, L. (2022) · IEEE/ACM International Workshop on Automated Program Repair (APR) · doi:10.1145/3524459.3527350 · Cited by 38 · see citing works
A structural model for contextual code changes
Brody, S., Alon, U., & Yahav, E. (2020) · Proceedings of the ACM on Programming Languages · doi:10.1145/3428283 · Cited by 36 · see citing works
DeepMerge: Learning to Merge Programs
Dinella, E., Mytkowicz, T., Svyatkovskiy, A., Bird, C., Naik, M., & Lahiri, S. K. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3183955 · Cited by 36 · see citing works
Template-based Neural Program Repair
Meng, X., Wang, X., Zhang, H., Sun, H., Liu, X., & Hu, C. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00127 · Cited by 36 · see citing works
Learning to Generate Corrective Patches using Neural Machine TranslationPreprint
Hata, H., Shihab, E., & Neubig, G. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1812.07170 · Cited by 36 · see citing works
Unveiling ChatGPT’s usage in open source projects: A mining-based study
Tufano, R., Mastropaolo, A., Pepe, F., Dabic, O., Di Penta, M., & Bavota, G. (2024) · Proceedings of the 21st International Conference on Mining Software Repositories · doi:10.1145/3643991.3644918 · Cited by 38 · see citing works
Self-citationCerebro: Static Subsuming Mutant Selection
Garg, A., Ojdanić, M., Degiovanni, R., Chekam, T. T., Papadakis, M., & Traon, Y. L. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3140510 · Cited by 33 · see citing works
On the Reliability and Explainability of Language Models for Program Generation
Liu, Y., Tantithamthavorn, C. K., Liu, Y., & Li, L. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3641540 · Cited by 37 · see citing works
Multi-target Backdoor Attacks for Code Pre-trained Models
Li, Y., Liu, S., Chen, K., Xie, X., Zhang, T., & Liu, Y. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.399 · Cited by 37 · see citing works
APPT: Boosting Automated Patch Correctness Prediction via Fine-Tuning Pre-Trained Models
Zhang, Q., Fang, C., Sun, W., Liu, Y., He, T., Hao, X., & Chen, Z. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3354969 · Cited by 35 · see citing works
A systematic mapping study of source code representation for deep learning in software engineering
Samoaa, P., Bayram, F., Salza, P., & Leitner, P. (2022) · IET Software · doi:10.1049/sfw2.12064 · Cited by 29 · see citing works
Comprehensive Fine-Tuning Large Language Models of Code for Automated Program Repair
Huang, K., Zhang, J., Bao, X., Wang, X., & Liu, Y. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3532759 · Cited by 30 · see citing works
Machine Learning for Source Code Vulnerability Detection: What Works and What Isn’t There Yet
Marjanov, T., Николаевич, П. И., & Massacci, F. (2022) · IEEE Security & Privacy · doi:10.1109/msec.2022.3176058 · Cited by 28 · see citing works
Representation Learning for Stack Overflow Posts: How Far Are We?
He, J., Zhou, X., Xu, B., Zhang, T., Kim, K., Yang, Z., Thung, F., Irsan, I. C., & Lo, D. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3635711 · Cited by 32 · see citing works
Poison Attack and Poison Detection on Deep Source Code Processing Models
Li, J., Li, Z., Zhang, H., Li, G., Jin, Z. G., Hu, X., & Xia, X. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3630008 · Cited by 43 · see citing works
Design pattern recognition: a study of large language models
Pandey, S. K., Chand, S., Horkoff, J., Staron, M., Ochodek, M., & Durisic, D. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10625-1 · Cited by 27 · see citing works
Explaining Software Bugs Leveraging Code Structures in Neural Machine Translation
Mahbub, P., Shuvo, O., & Rahman, M. M. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00063 · Cited by 27 · see citing works
Learning from what we know: How to perform vulnerability prediction using noisy historical data
Garg, A., Degiovanni, R., Jimenez, M., Cordy, M., Papadakis, M., & Traon, Y. L. (2022) · Empirical Software Engineering · doi:10.1007/s10664-022-10197-4 · Cited by 26 · see citing works
Generating realistic vulnerabilities via neural code editing: an empirical study
Nong, Y., Ou, Y., Pradel, M., Chen, F., & Cai, H. (2022) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3540250.3549128 · Cited by 26 · see citing works
Ciniselli, M., Pascarella, L., & Bavota, G. (2022) · International Conference on Mining Software Repositories (MSR) · doi:10.1145/3524842.3528440 · Cited by 26 · see citing works
Self-citation
- Authors
- Tufano, M., Watson, C., Bavota, G., Penta, M. D., White, M., & Poshyvanyk, D.
- Affiliations
- William & Mary (US)
- Università della Svizzera italiana (CH)
- University of Sannio (IT)
- Venue
- ACM Transactions on Software Engineering and Methodology
- Published
- 2019-09-02
- Volume
- 28
- Issue
- 4
- Pages
- 1-29
- ISSN
- 1049-331X (ISSN-L)
- 1557-7392
- OpenAlex
- W2905463374
- Semantic Scholar
- 516497e7fc39ebb126bed28dcc5c6129e500acb8
- Cited by
- 370
- References
- 101
- Citations by year
- 2026: 44 · 2025: 62 · 2024: 73 · 2023: 82 · 2022: 55 · 2021: 40 · 2020: 10 · 2019: 3 · 2018: 1
- FWCI
- 34.22
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- ACM Transactions on Software Engineering and Methodology (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- NSF (CCF-1525902 and CCF-1815186)
- SNF (175513)
- Swiss National Science Foundation for the CCQR project
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Software › Software Testing and Debugging Techniques
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Keywords
- neural machine translation, program translation, bug fixes, open source software, GitHub, encoder-decoder architecture, code mining, software defect repair
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review