A Systematic Literature Review on the Use of Deep Learning in Software Engineering Research
The DOI matches its Crossref record; 207 of the 207 references it cites were found in a registry or an index.
Abstract
An increasingly popular set of techniques adopted by software engineering (SE) researchers to automate development tasks are those rooted in the concept of Deep Learning (DL). The popularity of such techniques largely stems from their automated feature engineering capabilities, which aid in modeling software artifacts. However, due to the rapid pace at which DL techniques have been adopted, it is difficult to distill the current successes, failures, and opportunities of the current research landscape. In an effort to bring clarity to this cross-cutting area of work, from its modern inception to the present, this article presents a systematic literature review of research at the intersection of SE & DL. The review canvasses work appearing in the most prominent SE and DL conferences and journals and spans 128 papers across 23 unique SE tasks. We center our analysis around the components of learning , a set of principles that governs the application of machine learning techniques (ML) to a given problem domain, discussing several aspects of the surveyed work at a granular level. The end result of our analysis is a research roadmap that both delineates the foundations of DL techniques applied to SE research and highlights likely areas of fertile exploration for the future.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Deep learning models accelerate and enhance the accuracy of complex coding tasks [122, 214, 234].”
From A 2030 Roadmap for Software Engineering (2025) · cites this paper as [214]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Artificial intelligence can improve efficiency and reduce frustration by fixing errors, recommending best practices, and generating documentation [214].”
From A 2030 Roadmap for Software Engineering (2025) · cites this paper as [214]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · cited 1×
“rprisingly, there was very little work done between the years of 2009 and 2014. However, even after the popularization of DL techniques brought about by results achieved by approaches such as AlexNet [73], it took the SE community 3 years to begin exploring the use of DL techniques for SE tasks. This also coincides wi…”
MethodAdam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · cited 1×
“It can increase the rate at which the learning rate is optimized [88].”
BackgroundBERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
(2014) · Cited by 33,419
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818
(2014) · Cited by 15,747
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 14,214
(2016) · Cited by 11,679
(2010) · Cited by 8,261
Research works printed but not linked: 13
- Artifact Review and Badging – Version 2.0. Artifact Review and Badging – Version 2.0 Association for Computing Machinery 4 Nov. 2021 https://www.acm.org/publications/policies/artifact-review-badging.
- Tal Ben-Nun, Alice Shoshana Jakobovits, and Torsten Hoefler. 2018. Neural code comprehension: A learnable representation of code semantics. In Proceedings of the Annual Conference on Neural Information Processing Systems, Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett (Eds.). 3589–3601. Retrieved from https://proceedings.neurips.cc/paper/2018/hash/17c3433fecc21b57000debdf7ad5c930-Abstract.html.
- Jacob Devlin, Rudy Bunel, Rishabh Singh, Matthew J. Hausknecht, and Pushmeet Kohli. 2017. Neural program meta-induction. In Proceedings of the Annual Conference on Neural Information Processing Systems, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 2080–2088. Retrieved from https://proceedings.neurips.cc/paper/2017/hash/3bf55bbad370a8fcad1d09b005e278c2-Abstract.html.
- Pedro Domingos. 1998. Occam’s two razors: The sharp and the blunt. In Proceedings of the 4th International Conference on Knowledge Discovery and Data Mining (KDD’98). AAAI Press, 37–43.
- 10.1109/MS.2017.265101102 · doi:10.1109/ms.2017.265101102
- 10.1609/aimag.v17i3.1230 · doi:10.1609/aimag.v17i3.1230
- 10.5555/3155562.3155573 · doi:10.5555/3155562.3155573
- 10.5555/3086952 · doi:10.5555/3086952
- 10.5555/3155562.3155583 · doi:10.5555/3155562.3155583
- B. Kitchenham and S. Charters. 2007. Guidelines for performing Systematic Literature Reviews in Software Engineering (EBSE 2007-001). Keele University and Durham University Joint Report. Retrieved from https://www.elsevier.com/__data/promis_misc/525444systematicreviewsguide.pdf.
- 10.5555/971143 · doi:10.5555/971143
- J. Sayyad Shirabad and T. J. Menzies. 2005. The PROMISE repository of software engineering databases.School of Information Technology and Engineering, University of Ottawa, Canada. Retrieved from http://promise.site.uottawa.ca/SERepository.
- 10.5555/2820518.2820559 · doi:10.5555/2820518.2820559
Cited by
Large Language Models for Software Engineering: A Systematic Literature Review
Hou, X., Zhao, Y., Liu, Y., Yang, Z., Wang, K., Li, L., Luo, X., Lo, D., Grundy, J., & Wang, H. (2024) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3695988 · Cited by 1,029
Software Testing With Large Language Models: Survey, Landscape, and Vision
Wang, J., Huang, Y., Chen, C., Liu, Z., Wang, S., & Wang, Q. (2024) · IEEE Transactions on Software Engineering · doi:10.1109/tse.2024.3368208 · Cited by 573
“Yang et al. and Watson et al. respectively carried out surveys about the use of deep learning in software engineering domain [161], [162].”
Automating code review activities by large-scale pre-training
Li, Z., Lu, S., Guo, D., Duan, N., Jannu, S., Jenks, G., Majumder, D., Green, J., Svyatkovskiy, A., Fu, S., et al. (2022) · Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering · doi:10.1145/3540250.3549081 · Cited by 196
“Deep Learning techniques have been widely adopted in software engineering research [10, 44].”
MethodRetrieval-Based Prompt Selection for Code-Related Few-Shot Learning
Nashid, N., Sintaha, M., & Mesbah, A. (2023) · 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) · doi:10.1109/icse48619.2023.00205 · Cited by 178
“There is a large body of literature [74]–[78] that applies learning based task-specific models to different source code processing tasks.”
BackgroundRise of the Planet of Serverless Computing: A Systematic Review
Wen, J., Chen, Z., Jin, X., & Liu, X. (2023) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3579643 · Cited by 188
“In our study, seven standard online engines or databases [134, 172, 231, 254, 270] are selected as literature sources.”
BackgroundMethod
Metadata
- Authors
- Watson, C., Cooper, N., Palacio, D. N., Moran, K., & Poshyvanyk, D.
- Affiliations
- Washington and Lee University (US)
- William & Mary (US)
- George Mason University (US)
- Venue
- ACM Transactions on Software Engineering and Methodology
- Published
- 2022-03-04
- Volume
- 31
- Issue
- 2
- Pages
- 1-58
- ISSN
- 1049-331X (ISSN-L)
- 1557-7392
- OpenAlex
- W3084812981
- Cited by
- 137
- References
- 204
- Citations by year
- 2026: 21 · 2025: 39 · 2024: 37 · 2023: 30 · 2022: 8
- FWCI
- 3.60
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- ACM Transactions on Software Engineering and Methodology (publishedVersion)
- Funders
- NSF (CCF-1927679, CCF-1955853, CCF-2007246, and CCF-1815186)
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Physical Sciences › Computer Science › Software › Advanced Software Engineering Methodologies
- Keywords
- deep learning, software engineering research, software development automation, automated feature engineering, machine learning in software engineering, research roadmap, software engineering tasks
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: A Systematic Literature Review on the Use of Deep Learning in Software Engineering Research. https://app.cochat.ai/papers/10.1145/3485275
- The paper
- Watson, C., Cooper, N., Palacio, D. N., Moran, K., & Poshyvanyk, D. (2022). A Systematic Literature Review on the Use of Deep Learning in Software Engineering Research. ACM Transactions on Software Engineering and Methodology. https://doi.org/10.1145/3485275
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.