A Systematic Literature Review on the Use of Deep Learning in Software Engineering Research
The DOI matches its Crossref record; 207 of the 207 references it cites were found in a registry or an index.
Abstract
An increasingly popular set of techniques adopted by software engineering (SE) researchers to automate development tasks are those rooted in the concept of Deep Learning (DL). The popularity of such techniques largely stems from their automated feature engineering capabilities, which aid in modeling software artifacts. However, due to the rapid pace at which DL techniques have been adopted, it is difficult to distill the current successes, failures, and opportunities of the current research landscape. In an effort to bring clarity to this cross-cutting area of work, from its modern inception to the present, this article presents a systematic literature review of research at the intersection of SE & DL. The review canvasses work appearing in the most prominent SE and DL conferences and journals and spans 128 papers across 23 unique SE tasks. We center our analysis around the components of learning , a set of principles that governs the application of machine learning techniques (ML) to a given problem domain, discussing several aspects of the surveyed work at a granular level. The end result of our analysis is a research roadmap that both delineates the foundations of DL techniques applied to SE research and highlights likely areas of fertile exploration for the future.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Deep learning models accelerate and enhance the accuracy of complex coding tasks [122, 214, 234].”
From A 2030 Roadmap for Software Engineering (2025) · cites this paper as [214]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Artificial intelligence can improve efficiency and reduce frustration by fixing errors, recommending best practices, and generating documentation [214].”
From A 2030 Roadmap for Software Engineering (2025) · cites this paper as [214]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“rprisingly, there was very little work done between the years of 2009 and 2014. However, even after the popularization of DL techniques brought about by results achieved by approaches such as AlexNet [73], it took the SE community 3 years to begin exploring the use of DL techniques for SE tasks. This also coincides wi…”
MethodAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“It can increase the rate at which the learning rate is optimized [88].”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
(2014) · Cited by 33,419
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
(2014) · Cited by 15,747
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
(2016) · Cited by 11,679
(2010) · Cited by 8,261
Research works printed but not linked: 7
- [2] [n.d.]. Literature Reviews on Applying Artificial Intelligence/Machine Learning to Software Engineering Research Problems: Preliminary. http://ceur-ws.org/Vol-2506/Paper5-seed2019.pdf -
- [3] [2019. ACM Artifact Review Policies https://www.acm.org/publications/policies/artifact-review-badging.](https://www.acm.org/publications/policies/artifact-review-badging) -
- [16] Tal Ben-Nun, Alice Shoshana Jakobovits, and Torsten Hoefler. 2018. Neural Code Comprehension: A Learnable Representation of Code Semantics. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, December 3-8, 2018, Montréal, Canada , Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett (Eds.). 3589-3601. https://proceedings.neurips.cc/paper/2018/hash/17c3433fecc21b57000debdf7ad5c930-Abstract.html -
- [42] Jacob Devlin, Rudy Bunel, Rishabh Singh, Matthew J. Hausknecht, and Pushmeet Kohli. 2017. Neural Program Meta-Induction. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA , Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 2080-2088. https: //proceedings.neurips.cc/paper/2017/hash/3bf55bbad370a8fcad1d09b005e278c2-Abstract.html -
- [45] Pedro Domingos. 1998. Occam's Two Razors: The Sharp and the Blunt. In Proceedings of the Fourth International Conference on Knowledge Discovery and Data Mining (New York, NY) (KDD'98) . AAAI Press, 37-43. -
- [51] F. Falcini, G. Lami, and A. Mitidieri. 2017. Yet Another Challenge for the Automotive Software: Deep Learning. IEEE Software (2017), 1-1. https://doi.org/10.1109/MS.2017.265101102 - · doi:10.1109/ms.2017.265101102
- [142] J. Sayyad Shirabad and T.J. Menzies. 2005. The PROMISE Repository of Software Engineering Databases. School of Information Technology and Engineering, University of Ottawa, Canada. http://promise.site.uottawa.ca/SERepository -
Cited by
Large Language Models for Software Engineering: A Systematic Literature Review
Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., & Wang, H. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3695988 · Cited by 1,029
Software Testing With Large Language Models: Survey, Landscape, and Vision
Wang, J., Huang, Y., Chen, C., Liu, Z., Wang, S., & Wang, Q. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3368208 · Cited by 573
“Yang et al. and Watson et al. respectively carried out surveys about the use of deep learning in software engineering domain [161], [162].”
Automating code review activities by large-scale pre-training
Li, Z., Lu, S., Guo, D., Duan, N., Jannu, S., Jenks, G., Majumder, D., Green, J., Svyatkovskiy, A., Fu, S., et al. (2022) · Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3540250.3549081 · Cited by 196
“Deep Learning techniques have been widely adopted in software engineering research [10, 44].”
MethodRetrieval-Based Prompt Selection for Code-Related Few-Shot Learning
Nashid, N., Sintaha, M., & Mesbah, A. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00205 · Cited by 178
“There is a large body of literature [74]–[78] that applies learning based task-specific models to different source code processing tasks.”
BackgroundRise of the Planet of Serverless Computing: A Systematic Review
Wen, J., Chen, Z., Jin, X., & Liu, X. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3579643 · Cited by 188
“In our study, seven standard online engines or databases [134, 172, 231, 254, 270] are selected as literature sources.”
BackgroundMethodExploring the Potential of ChatGPT in Automated Code Refinement: An Empirical Study
Guo, Q., Cao, J., Xie, X., Liu, S., Li, X., Chen, B., & Peng, X. (2024) · Proceedings of the IEEE/ACM 46th International Conference on Software Engineering · doi:10.1145/3597503.3623306 · Cited by 136
Exploring the Potential of ChatGPT in Automated Code Refinement: An Empirical Study
Guo, Q., Cao, J., Xie, X., Liu, S., Li, X., Chen, B., & Peng, X. (2023) · arXiv · doi:10.48550/arxiv.2309.08221 · Cited by 140
A Survey of Learning-based Automated Program Repair
Zhang, Q., Fang, C., Ma, Y., Sun, W., & Chen, Z. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3631974 · Cited by 132
DeepLineDP: Towards a Deep Learning Approach for Line-Level Defect Prediction
Pornprasit, C., & Tantithamthavorn, C. K. (2022) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3144348 · Cited by 165
“The reason for this is that Watson et al. [72] found that the Dropout is one of the most commonly-used techniques to prevent overfitting and underfitting in software engineering tasks.”
MethodA survey on large language models for software engineering
Zhang, Q., Fang, C., Xie, Y., Zhang, Y., Yu, S., Sun, W., Yang, Y., & Chen, Z. (2026) · Science China Information Sciences · doi:10.1007/s11432-025-4670-0 · Cited by 124
Towards an understanding of large language models in software engineering tasks
Zheng, Z., Ning, K., Zhong, Q., Chen, J., Chen, W., Guo, L., Wang, W., & Wang, Y. (2025) · Empirical Software Engineering · doi:10.1007/s10664-024-10602-0 · Cited by 116
“Watson et al. (2022) provides a systematic literature review of the intersection between SE and deep learning (DL).”
Large Language Models for Software Engineering: A Systematic Literature Review
Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., & Wang, H. (2023) · arXiv · doi:10.48550/arxiv.2308.10620 · Cited by 114
Deep learning-based software engineering: progress, challenges, and opportunities
Chen, X., Hu, X., Huang, Y., Jiang, H., Ji, W., Jiang, Y., Jiang, Y., Liu, B., Liu, H., Li, X., et al. (2024) · Science China Information Sciences · doi:10.1007/s11432-023-4127-5 · Cited by 152
Keeping Deep Learning Models in Check: A History-Based Approach to Mitigate Overfitting
Li, H., Rajbahadur, G. K., Lin, D., Bezemer, C., & Jiang, Z. M. (2024) · IEEE Access · doi:10.1109/access.2024.3402543 · Cited by 129
“Among the overfitting prevention methods, dropout is the most commonly adopted approach [79, 85].”
BackgroundMethodInfluentialUsing Transfer Learning for Code-Related Tasks
Mastropaolo, A., Cooper, N., Palacio, D. N., Scalabrino, S., Poshyvanyk, D., Oliveto, R., & Bavota, G. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2022.3183297 · Cited by 72
“available approaches support a plethora of concrete SE tasks [81], [86], in this section we focus on the ones we target in our study: automated bug-fixing, injection of code mutants, generation of assert statements in test methods,”
BackgroundMethodSelf-citationMulti-source Multimodal Data and Deep Learning for Disaster Response: A Systematic Review
Algiriyage, N., Prasanna, R., Stock, K. M., Doyle, E. E. H., & Johnston, D. M. (2021) · SN Computer Science · doi:10.1007/s42979-021-00971-4 · Cited by 88
“Addressing underfitting while training DL models is a complex task, and these are not well-defined techniques [125].”
BackgroundMethodAI for DevSecOps: A Landscape and Future Opportunities
Fu, M. C., Pasuksmit, J., & Tantithamthavorn, C. K. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3712190 · Cited by 110
GPTSniffer: A CodeBERT-based classifier to detect source code written by ChatGPT
Nguyen, P. T., Rocco, J. D., Sipio, C. D., Rubei, R., Ruscio, D. D., & Penta, M. D. (2024) · Journal of Systems and Software · doi:10.1016/j.jss.2024.112059 · Cited by 112
Mitigating False Positive Static Analysis Warnings: Progress, Challenges, and Opportunities
Guo, Z., Tan, T., Liu, S., Liu, X., Lai, W., Yang, Y., Li, Y., Chen, L., Dong, W., & Zhou, Y. (2023) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2023.3329667 · Cited by 65
“Additionally, with the rapid development of DL techniques, many novel DL models have been leveraged for SE tasks [138], [151], but these models have not been applied to the FPM task.”
BackgroundMethodTrustworthy and Synergistic Artificial Intelligence for Software Engineering: Vision and Roadmaps
Lo, D. (2023) · 2023 IEEE/ACM International Conference on Software Engineering: Future of Software Engineering (ICSE-FoSE) · doi:10.1109/icse-fose59343.2023.00010 · Cited by 40
“Comprehensive surveys on this topic were written by Yang et al. [31] and Watson et al. [32].”
BackgroundMachine Learning for Software Engineering: A Tertiary Study
Kotti, Z., Galanopoulou, R., & Spinellis, D. (2022) · ACM Computing Surveys · doi:10.1145/3572905 · Cited by 45
Abdelaal, Y., Aupetit, M. J. M., Baggag, A., & Al‐Thani, D. (2024) · Journal of Medical Internet Research · doi:10.2196/53863 · Cited by 63
“Prominent DL models include convolutional neural networks (CNNs), residual networks, fully convolutional networks, and long short-term memory (LSTM) [12].”
Towards AI-Assisted Synthesis of Verified Dafny Methods
Misu, M. R. H., Lopes, C. V., Ma, I., & Noble, J. (2024) · Proceedings of the ACM on Software Engineering · doi:10.1145/3643763 · Cited by 38
“…and Ray 2021; Tufano et al. 2019] code review [Li et al. 2022a; Tufano et al. 2022b] — for some tasks, performing as well as or better than the state of the art [Allamanis et al. 2018; Watson et al. 2022; Yang et al. 2022], especially when prompts are carefully constructed [Nashid et al. 2023].”
Deep Learning for Code Intelligence: Survey, Benchmark and Toolkit
Wan, Y., Bi, Z., He, Y., Zhang, J., Zhang, H., Sui, Y., Xu, G., Jin, H., & Yu, P. S. (2024) · ACM Computing Surveys · doi:10.1145/3664597 · Cited by 76
“We propose to summarize existing deep-learning-based approaches to code intelligence from the lens of code representation learning in this paper.”
On the Reliability of the Area Under the ROC Curve in Empirical Software Engineering
Lavazza, L., Morasca, S., & Rotoloni, G. (2023) · International Conference on Evaluation and Assessment in Software Engineering (EASE) · doi:10.1145/3593434.3593456 · Cited by 41
“To this end, Receiver Operating Characteristic (ROC) curves are widely used; specifically, the Area Under the Curve (AUC) is used as an indicator of the extent to which binary classifiers make correct predictions [7, 12, 16, 21].”
MethodAlizadeh, N., & Castor, F. (2024) · IEEE/ACM International Conference on AI Engineering – Software Engineering for AI (CAIN) · doi:10.1145/3644815.3644967 · Cited by 65
On the Effectiveness of LLM-as-a-Judge for Code Generation and Summarization
Crupi, G., Tufano, R., Velasco, A., Mastropaolo, A., Poshyvanyk, D., & Bavota, G. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3586082 · Cited by 88
“Authorized licensed use limited to the terms of the applicable license agreement with IEEE.”
Self-citationAdaptive user interfaces in systems targeting chronic disease: a systematic literature review
Wang, W., Khalajzadeh, H., Grundy, J., Madugalla, A. K., McIntosh, J., & Obie, H. O. (2023) · User Modeling and User-Adapted Interaction · doi:10.1007/s11257-023-09384-9 · Cited by 42
“We followed the SLR guidelines and procedures in [27] and the work of [51] to uphold the integrity of our analysis and provide a reproducible process.”
MethodSurvey of Code Search Based on Deep Learning
Xie, Y., Lin, J., Dong, H., Zhang, L., & Wu, Z. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3628161 · Cited by 56
Augmented Agile: Human-Centered AI-Assisted Software Management
Hoda, R., Dam, H. K., Tantithamthavorn, C. K., Thongtanunam, P., & Storey, M. (2023) · IEEE Software · doi:10.1109/ms.2023.3268725 · Cited by 31
“Feel : Despite advances in AI-assisted programming support for developers, such as GitHub Copilot 6 and tools for debugging and maintenance 7 and theoretical visions of AI-powered APM, 1,8 APM is still performed with little to no intelligent support.”
BackgroundAutomating Code-Related Tasks Through Transformers: The Impact of Pre-training
Tufano, R., Pascarella, L., & Bavota, G. (2023) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1109/icse48619.2023.00203 · Cited by 32
“Watson [84] et al. also presented a SLR aimed at documenting the use of DL in SE.”
BackgroundAccessibility of low-code approaches: A systematic literature review
Khalajzadeh, H., & Grundy, J. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107570 · Cited by 26
Deep Configuration Performance Learning: A Systematic Survey and Taxonomy
Gong, J., & Chen, T. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3702986 · Cited by 39
Toward a Theory of Causation for Interpreting Neural Code Models
Nader Palacio, D., Velasco, A., Cooper, N., Rodriguez, A., Moran, K., & Poshyvanyk, D. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3379943 · Cited by 18
“We employed RNNs and Transformers for all our scenarios as RNNs are widely used in SE [82], while Trans-formers have gained popularity in the SE/NLP domain for their high performance [4].”
MethodSelf-citationTowards Automatically Addressing Self-Admitted Technical Debt: How Far Are We?
Mastropaolo, A., Penta, M. D., & Bavota, G. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00103 · Cited by 33
“For a comprehensive overview of DL-based models applied in software engineering, we point the reader to the systematic literature review by Watson et al. [34].”
MethodRodrigues, W. G., Vieira, G. D. S., Cabacinha, C. D., Bulcão-Neto, R. F., & Soares, F. A. A. D. M. N. (2024) · Computers & Electrical Engineering · doi:10.1016/j.compeleceng.2024.109793 · Cited by 18
Gong, L., Zhang, J., Wei, M., Zhang, H., & Huang, Z. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3569934 · Cited by 27
“Machine learning-based software is a data-driven programming paradigm that relies heavily on training datasets [38, 91].”
BackgroundHerekoğlu, A., & Kabak, Ö. (2023) · Annals of Operations Research · doi:10.1007/s10479-023-05738-z · Cited by 18
“With the increasing amount of computational power and memory, the area called Deep Learning has emerged, which consists of multi-layered neural networks, and performs much better than traditional machine learning approaches based on shallow learning (Watson et al., 2022; Neu et al., 2021).”
An extensive bibliometric analysis of artificial intelligence techniques from 2013 to 2023
Bajpai, A., Yadav, S., & Nagwani, N. K. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-07021-3 · Cited by 24
Zhao, Y. D., Gong, L., Huang, Z., Wang, Y., Wei, M., & Wu, F. Q. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695539 · Cited by 18
“Code embedding techniques aim to acquire compact code vector representations [19, 2, 18], referred to as code embeddings, to encapsulate the essence of source code, which have been instrumental in leveraging deep learning for tasks in software engineering (SE) [41, 23, 52].”
BackgroundBenchmarking Causal Study to Interpret Large Language Models for Source Code
Rodriguez-Cardenas, D., Palacio, D. N., Khati, D., Burke, H., & Poshyvanyk, D. (2023) · 2023 IEEE International Conference on Software Maintenance and Evolution (ICSME) · doi:10.1109/icsme58846.2023.00040 · Cited by 14
“To support actionable downstream SE tasks ( e.g., code completion, code summarization, or commit generation), ample evidence supports that DL4SE approaches in the form of Language Models are able to generate code conditioned on a well-defined prompt [1]–[3].”
BackgroundSelf-citationLog statements generation via deep learning: Widening the support provided to developers
Mastropaolo, A., Ferrari, V., Pascarella, L., & Bavota, G. (2023) · Journal of Systems and Software · doi:10.1016/j.jss.2023.111947 · Cited by 25
Learning to Detect and Localize Multilingual Bugs
Yang, H., Nong, Y., Zhang, T., Luo, X., & Cai, H. (2024) · Proceedings of the ACM on software engineering. · doi:10.1145/3660804 · Cited by 31
Pitfalls in Language Models for Code Intelligence: A Taxonomy and Survey
She, X., Liu, Y., Zhao, Y., He, Y., Li, L., Tantithamthavorn, C. K., Qin, Z., & Wang, H. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3748647 · Cited by 40
Prediction of Vulnerability Characteristics Based on Vulnerability Description and Prompt Learning
Li, X., Ren, X., Xue, Y., Xing, Z., & Sun, J. (2023) · IEEE International Conference on Software Analysis, Evolution and Reengineering (SANER) · doi:10.1109/saner56733.2023.00062 · Cited by 20
“Deep learning models have recently been adopted in vulnerability detection and analysis, due to their ability to solve complex problems even if the source code and software text are very diverse and massive [5], [42].”
Background(2023) · doi:10.1109/ase56229.2023.00157 · Cited by 14
“In this study, we focus on investigating the existence of long-tailed distributions in labels of SE datasets, as learningbased approaches with labeled data have become the mainstream for most SE tasks [16], [17].”
BackgroundMethodA 2030 Roadmap for Software Engineering
Pezzè, M., Abrahão, S., Penzenstadler, B., Poshyvanyk, D., Roychoudhury, A., & Yue, T. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3731559 · Cited by 13
Self-citationAn Exploratory Literature Study on Sharing and Energy Use of Language Models for Source Code
Hort, M., Grishina, A., & Moonen, L. (2023) · Proceedings of the ... ACM-IEEE International Symposium on Empirical Software Engineering and Measurement/International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem56168.2023.10304803 · Cited by 18
“[37]: A literature review of deep learning approaches in SE research.”
BackgroundA Systematic Literature Review on Large Language Models for Automated Program Repair
Zhang, Q., Fang, C., Xie, Y., Ma, Y., Sun, W., Yang, Y. J., & Chen, Z. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3799693 · Cited by 189
From Triumph to Uncertainty: The Journey of Software Engineering in the AI Era
Mastropaolo, A., Escobar‐Velásquez, C., & Linares‐Vásquez, M. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3709360 · Cited by 30
Improving Retrieval-Augmented Deep Assertion Generation via Joint Training
Zhang, Q., Fang, C., Zheng, Y., Qian, R., Yu, S., Zhao, Y., Zhou, J., Yang, Y., Zheng, T., & Chen, Z. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3545970 · Cited by 17
“With the success of DL, researchers have increasingly been utilizing advanced DL techniques to automate a variety of software engineering tasks [24], [25], such as program repair [26], [27], [28] and vulnerability detection [29], [30], [31], [32].”
Improving Deep Assertion Generation via Fine-Tuning Retrieval-Augmented Pre-Trained Language Models
Zhang, Q., Fang, C., Zheng, Y., Zhang, Y., Zhao, Y. F., Huang, R., Zhou, J., Yang, Y. J., Zheng, T., & Chen, Z. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3721128 · Cited by 20
Security of Language Models for Code: A Systematic Literature Review
Chen, Y., Sun, W., Fang, C., Chen, Z., Ge, Y., Han, T., Zhang, Q., Liu, Y., Chen, Z., & Xu, B. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3735554 · Cited by 10
Are They All Good? Studying Practitioners' Expectations on the Readability of Log Messages
Li, Z., Chen, A. R., Hu, X., Xia, X., Chen, T.-H. P., & Shang, W. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00136 · Cited by 33
“In this section, we discuss the results of each RQ.”
BackgroundDeepMig: A transformer-based approach to support coupled library and code migrations
Rocco, J. D., Nguyen, P. T., Sipio, C. D., Rubei, R., Ruscio, D. D., & Penta, M. D. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107588 · Cited by 17
How Propense Are Large Language Models at Producing Code Smells? A Benchmarking Study
Velasco, A., Rodríguez-Cárdenas, D., Alif, L. R., Palacio, D. N., & Poshyvanyk, D. (2025) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1109/icse-nier66352.2025.00025 · Cited by 22
“Although LLMs are largely investigated in SE life-cycle [23], little is known about the likelihood LLMs generate smelly code.”
Self-citationLanguage Models in Software Development Tasks: An Experimental Analysis of Energy and Accuracy
Alizadeh, N., Belchev, B., Saurabh, N., Kelbert, P., & Castor, F. (2025) · Empirical Software Engineering · doi:10.1109/msr66628.2025.00109 · Cited by 32
Weyssow, M., Zhou, X., Kim, K., Lo, D., & Sahraoui, H. A. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3616244 · Cited by 20
“In this perspective, previous works [11, 59] have primarily focused on stationary settings, neglecting the practical need for models to adapt to changing environments and data over time.”
BackgroundA Taxonomy of Self-Admitted Technical Debt in Deep Learning Systems
Pepe, F., Zampetti, F., Mastropaolo, A., Bavota, G., & Penta, M. D. (2024) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme58944.2024.00043 · Cited by 22
“Indeed, DL has been widely used in several domains, such as finance, e.g., fraud detection [3], healthcare to improve diagnosis and treatment of diseases, as well as patient outcomes [2], autonomous driving [10], and, last but not least, software engineering [46].”
A Systematic Mapping Study of the Metrics, Uses and Subjects of Diversity‐Based Testing Techniques
Elgendy, I. T., Hierons, R. M., & McMinn, P. (2025) · Software Testing Verification and Reliability · doi:10.1002/stvr.1914 · Cited by 12
Towards Reliable AI for Source Code Understanding
Suneja, S., Zheng, Y., Zhuang, Y., Laredo, J., & Morari, A. (2021) · ACM Symposium on Cloud Computing (SoCC) · doi:10.1145/3472883.3486995 · Cited by 13
“In the software engineering domain, AI-for-code has been employed in source code understanding tasks such as defect detection [17, 26, 39, 48], code summarization [5, 24, 28, 33, 43], code completion [15, 25, 54], function and variable naming [3, 4, 7, 41], amongst others [23, 37, 58, 60].”
BackgroundMethodHuman-Machine Teaming and Team Effectiveness in AI Tools for Software Engineering
Rauf, I., Sharp, H., López, T., & Wermelinger, M. (2025) · International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE) · doi:10.1109/chase66643.2025.00017 · Cited by 18
“For instance, Watson et al [14] and Yang et al [15] explore the application of deep learning in SE research, while Mohammadkhani et al [16] investigate explainable AI4SE and Hou et al [17] focus on large language models.”
Using Deep Learning to Automatically Improve Code Readability
Vitale, A., Piantadosi, V., Scalabrino, S., & Oliveto, R. (2023) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase56229.2023.00112 · Cited by 18
“A broader overview is available in the surveys by Allamanis et al. [2], Watson et al. [54], and Yang et al. [56].”
BackgroundSoftware Defect Prediction evaluation: New metrics based on the ROC curve
Lavazza, L., Morasca, S., & Rotoloni, G. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107865 · Cited by 7
Nikitin, N. Y., & Stepashkin, A. A. (2025) · Results in Materials · doi:10.1016/j.rinma.2025.100788 · Cited by 12
Automatically Generating Dockerfiles via Deep Learning: Challenges and Promises
Rosa, G., Mastropaolo, A., Scalabrino, S., Bavota, G., & Oliveto, R. (2023) · International Conference on Software and System Process · doi:10.1109/icssp59042.2023.00011 · Cited by 20
“Previous research [12]–[15] show that DL is a viable solution for code generation-related tasks.”
BackgroundA Study on the Battery Usage of Deep Learning Frameworks on iOS Devices
Jacques, V. M. F., Alizadeh, N., & Castor, F. (2024) · IEEE/ACM International Conference on Mobile Software Engineering and Systems (MOBILESoft) · doi:10.1145/3647632.3647990 · Cited by 8
Data-Driven Software Engineering: A Systematic Literature Review
Yalçıner, A., Dikici, A., & Gökalp, E. (2024) · Communications in computer and information science · doi:10.1007/978-3-031-71139-8_2 · Cited by 7
Multi-grained contextual code representation learning for commit message generation
Wang, C., Zhang, L., & Zhang, X. (2023) · Information and Software Technology · doi:10.1016/j.infsof.2023.107393 · Cited by 6
A Systematic Literature Review of Multi-Label Learning in Software Engineering
Hämäläinen, J., Das, T., & Mikkonen, T. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3708532 · Cited by 10
Systematic Literature Review on Software Security Vulnerability Information Extraction
Yitagesu, S., Xing, Z., Zhang, X., Feng, Z., Bi, T., Han, L., & Li, X. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3745026 · Cited by 13
DIAGNOSING DISEASES FROM FINGERNAIL IMAGES
Can, Z., & Işık, Ş. (2022) · Eskişehir Osmangazi Üniversitesi mühendislik ve mimarlık fakültesi dergisi :/Osmangazi Üniversitesi Mühendislik-Mimarlık Fakültesi dergisi · doi:10.31796/ogummf.1111749 · Cited by 7
Measurement of Embedding Choices on Cryptographic API Completion Tasks
Xiao, Y., Song, W., Ahmed, S., Ge, X., Viswanath, B., Meng, N., & Yao, D. D. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3625291 · Cited by 5
Estimating Uncertainty in Line-Level Defect Prediction via Perceptual Borderline Oversampling
Wu, C., Guo, S., Li, H., Li, C., & Chen, R. (2025) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3746225 · Cited by 6
“Dropout (with a dropout rate of 0.2) and Layer Normalization were employed to prevent overfitting, as Watson et al. [65] found Dropout to be one of the most commonly used techniques for preventing overfitting and underfitting in software engineering tasks.”
InfluentialWhich Syntactic Capabilities Are Statistically Learned by Masked Language Models for Code?
Velasco, A., Palacio, D. N., Rodríguez-Cárdenas, D., & Poshyvanyk, D. (2024) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1145/3639476.3639768 · Cited by 11
Self-citationZhao, Y. D., Gong, L., Yu, Y., Huang, Z., & Wei, M. (2025) · Expert Systems with Applications · doi:10.1016/j.eswa.2025.126762 · Cited by 6
Classifying Bug Issue Types for Deep Learning-Oriented Projects with Pre-Trained Model
Zeng, Z., Zhao, Y. D., & Gong, L. (2024) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec65559.2024.00020 · Cited by 5
DeepDiffer: Find Deep Learning Compiler Bugs via Priority-guided Differential Fuzzing
Lin, K., Song, X., Zeng, Y., & Guo, S. (2023) · IEEE International Conference on Software Quality, Reliability and Security (QRS) · doi:10.1109/qrs60937.2023.00066 · Cited by 12
“Deep learning (DL) has emerged as an effective approach to solving problems in various domains, such as autonomous driving cars [16], software engineering [17], [18], and health care [19], [20].”
BackgroundYi, J., & Cui, X. (2022) · Journal of Robotics · doi:10.1155/2022/2425069 · Cited by 4
“Chinese government departments have listed the cultivation of IT talents as a major national strategy [3].”
BackgroundRetractedHydra-Reviewer: A Holistic Multi-Agent System for Automatic Code Review Comment Generation
Ren, X., Dai, C., Huang, Q., Wang, Y., Liu, C., & Jiang, B. (2025) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2025.3621462 · Cited by 14
“Regarding automated code review (ACR) activities, researchers have conducted extensive studies, such as comment location prediction [5], [6] and review comment recommendation [32], [33], emphasizing their importance and potential impact [34].”
Is Quantization a Deal-Breaker? Empirical Insights From Large Code Models
Afrin, S., Xu, B., & Mastropaolo, A. (2025) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme64153.2025.00049 · Cited by 18
“…to as Large Code Models (LCMs), these models have been successfully applied to various aspects of software development automation, including: bug fixing, code summarization, code completion, software testing, and others [1, 2]–showcasing their versatility and impact on modern software engineering.”
Characterizing Multi-Hunk Patches: Divergence, Proximity, and LLM Repair Challenges
Nashid, N., Ding, D., Gallaba, K., Hassan, A. E., & Mesbah, A. (2025) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1109/ase63991.2025.00137 · Cited by 15
Investigating Reproducibility in Deep Learning-Based Software Fault Prediction
Mukhtar, A., Jannach, D., & Wotawa, F. (2024) · International Conference on Software Quality, Reliability and Security · doi:10.1109/qrs62785.2024.00038 · Cited by 4
“Nonetheless, as there is at least some code for a bit less than two-thirds (64.3”
BackgroundInfluentialArtificial Intelligence Methods in Software Refactoring: A Systematic Literature Review
Motogna, S., Berciu, L.-M., & Moldovan, V. (2024) · Euromicro Conference on Software Engineering and Advanced Applications (SEAA) · doi:10.1109/seaa64295.2024.00055 · Cited by 5
“These advancements had a notable impact on software refactoring processes, as discussed in [15].”
A Path Less Traveled: Reimagining Software Engineering Automation via a Neurosymbolic Paradigm
Mastropaolo, A., & Poshyvanyk, D. (2025) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3696630.3728720 · Cited by 5
Self-citationTowards Using Data-Influence Methods to Detect Noisy Samples in Source Code Corpora
Dau, A. T. V., Bui, N. D. Q., Nguyen-Duc, T., & Thanh-Tung, H. (2022) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3551349.3561168 · Cited by 5
A Pilot Study in Surveying Data Challenges of Automatic Software Engineering Tasks
Dong, L., Lu, Q., & Zhu, L. (2024) · doi:10.1145/3663530.3665020 · Cited by 4
Akther, S., & Rahman, M. S. (2025) · Paradigm A Management Research Journal · doi:10.1177/09718907251379055 · Cited by 6
“Search strings were developed based on criteria from Kauppi et al. (2018) and Watson et al. (2022) to ensure relevance to the research questions.”
Why Do Deep Learning Projects Differ in Compatible Framework Versions? An Exploratory Study
Lei, H., Zhang, S., Wang, J., Xiao, G., Liu, Y., & Sui, Y. (2023) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre59848.2023.00076 · Cited by 3
Specialized model initialization and architecture optimization for few-shot code search
Zhang, F., Wu, Q., Peng, M., & Shen, Y. (2024) · Information and Software Technology · doi:10.1016/j.infsof.2024.107571 · Cited by 2
FIPSER: Improving Fairness Testing of DNN by Seed Prioritization
Chen, J., Zhang, Y., Zhang, L., Zhang, M., Wan, C., Su, T., & Pu, G. (2024) · IEEE/ACM International Conference on Automated Software Engineering (ASE) · doi:10.1145/3691620.3695486 · Cited by 2
UniGenCoder: Merging SEQ2SEQ and SEQ2TREE Paradigms for Unified Code Generation
Shao, L., Yan, Y., Poshyvanyk, D., & Su, J. (2025) · IEEE/ACM International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER) · doi:10.1109/icse-nier66352.2025.00020 · Cited by 7
“Given the similarities between NL and programming languages (PLs), it is common practice to adapt these models for PLs (such as CodeBERT [4] and CodeT5 [5]), treating code as a sequence of words to automate various SE tasks [6]–[9].”
Self-citationAccelerQ: Accelerating Quantum Eigensolvers with Machine Learning on Quantum Simulators
Bensoussan, A., Chachkarova, E., Even-Mendoza, K., Fortz, S., & Lenihan, C. (2025) · Proceedings of the ACM on Programming Languages · doi:10.1145/3763132 · Cited by 1
Ruan, C., Xiong, J., & Wang, L. (2025) · Journal of Prosthodontics · doi:10.1111/jopr.70049 · Cited by 1
A Systematic Literature Review of Parameter-Efficient Fine-Tuning for Large Code Models
Afrin, S., Haque, M. Z., & Mastropaolo, A. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3796522 · Cited by 19
Weng, S., Feng, Y., Yin, Y., Zhang, Z., & Xu, B. (2026) · Frontiers of Computer Science · doi:10.1007/s11704-025-41376-3 · Cited by 7
On Using Information Retrieval to Recommend Machine Learning Good Practices for Software Engineers
Cabra-Acela, L., Mojica-Hanke, A. I., Linares‐Vásquez, M., & Herbold, S. (2023) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3611643.3613093 · Cited by 1
“Software engineering (SE) is no exception, and ML is also used daily by developers [44, 47].”
MethodA Systematic Mapping Study of Machine Learning Techniques Applied to Software Traceability
Wang, B., Li, X., Wan, H., & Deng, Y. (2023) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/smc53992.2023.10394446 · Cited by 1
“As far as we know, there are five relevant reviews investigating ML or deep learning (DL) applied for software engineering (SE) and ST. Watson et al. [7] provide a review of the intersection of SE and DL.”
Background
Metadata
- Authors
- Watson, C., Cooper, N., Palacio, D. N., Moran, K., & Poshyvanyk, D.
- Affiliations
- Washington and Lee University (US)
- William & Mary (US)
- George Mason University (US)
- Venue
- ACM Transactions on Software Engineering and Methodology
- Published
- 2022-03-04
- Volume
- 31
- Issue
- 2
- Pages
- 1-58
- ISSN
- 1049-331X (ISSN-L)
- 1557-7392
- OpenAlex
- W3084812981
- Cited by
- 137
- References
- 204
- Citations by year
- 2026: 21 · 2025: 39 · 2024: 37 · 2023: 30 · 2022: 8
- FWCI
- 3.60
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Software Engineering and Methodology (publishedVersion)
- Funders
- NSF (CCF-1927679, CCF-1955853, CCF-2007246, and CCF-1815186)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Software › Advanced Software Engineering Methodologies
- Keywords
- deep learning, software engineering research, software development automation, automated feature engineering, machine learning in software engineering, research roadmap, software engineering tasks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.