CoChat

A Systematic Literature Review on the Use of Deep Learning in Software Engineering Research

Watson, Cody, Cooper, Nathan, Palacio, David Nader, Moran, Kevin, Poshyvanyk, Denys · 2022 · ACM Transactions on Software Engineering and Methodology

The DOI matches its Crossref record; 207 of the 207 references it cites were found in a registry or an index.

Abstract

An increasingly popular set of techniques adopted by software engineering (SE) researchers to automate development tasks are those rooted in the concept of Deep Learning (DL). The popularity of such techniques largely stems from their automated feature engineering capabilities, which aid in modeling software artifacts. However, due to the rapid pace at which DL techniques have been adopted, it is difficult to distill the current successes, failures, and opportunities of the current research landscape. In an effort to bring clarity to this cross-cutting area of work, from its modern inception to the present, this article presents a systematic literature review of research at the intersection of SE & DL. The review canvasses work appearing in the most prominent SE and DL conferences and journals and spans 128 papers across 23 unique SE tasks. We center our analysis around the components of learning , a set of principles that governs the application of machine learning techniques (ML) to a given problem domain, discussing several aspects of the surveyed work at a granular level. The end result of our analysis is a research roadmap that both delineates the foundations of DL techniques applied to SE research and highlights likely areas of fertile exploration for the future.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1145/3485275

Cites

207 of 207 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 207 references

Research works printed but not linked: 13

  1. Artifact Review and Badging – Version 2.0. Artifact Review and Badging – Version 2.0 Association for Computing Machinery 4 Nov. 2021 https://www.acm.org/publications/policies/artifact-review-badging.
  2. Tal Ben-Nun, Alice Shoshana Jakobovits, and Torsten Hoefler. 2018. Neural code comprehension: A learnable representation of code semantics. In Proceedings of the Annual Conference on Neural Information Processing Systems, Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa-Bianchi, and Roman Garnett (Eds.). 3589–3601. Retrieved from https://proceedings.neurips.cc/paper/2018/hash/17c3433fecc21b57000debdf7ad5c930-Abstract.html.
  3. Jacob Devlin, Rudy Bunel, Rishabh Singh, Matthew J. Hausknecht, and Pushmeet Kohli. 2017. Neural program meta-induction. In Proceedings of the Annual Conference on Neural Information Processing Systems, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 2080–2088. Retrieved from https://proceedings.neurips.cc/paper/2017/hash/3bf55bbad370a8fcad1d09b005e278c2-Abstract.html.
  4. Pedro Domingos. 1998. Occam’s two razors: The sharp and the blunt. In Proceedings of the 4th International Conference on Knowledge Discovery and Data Mining (KDD’98). AAAI Press, 37–43.
  5. 10.1109/MS.2017.265101102 · doi:10.1109/ms.2017.265101102
  6. 10.1609/aimag.v17i3.1230 · doi:10.1609/aimag.v17i3.1230
  7. 10.5555/3155562.3155573 · doi:10.5555/3155562.3155573
  8. 10.5555/3086952 · doi:10.5555/3086952
  9. 10.5555/3155562.3155583 · doi:10.5555/3155562.3155583
  10. B. Kitchenham and S. Charters. 2007. Guidelines for performing Systematic Literature Reviews in Software Engineering (EBSE 2007-001). Keele University and Durham University Joint Report. Retrieved from https://www.elsevier.com/__data/promis_misc/525444systematicreviewsguide.pdf.
  11. 10.5555/971143 · doi:10.5555/971143
  12. J. Sayyad Shirabad and T. J. Menzies. 2005. The PROMISE repository of software engineering databases.School of Information Technology and Engineering, University of Ottawa, Canada. Retrieved from http://promise.site.uottawa.ca/SERepository.
  13. 10.5555/2820518.2820559 · doi:10.5555/2820518.2820559

Cited by

See all 138

Metadata

Authors
Watson, C., Cooper, N., Palacio, D. N., Moran, K., & Poshyvanyk, D.
Affiliations
  • Washington and Lee University (US)
  • William & Mary (US)
  • George Mason University (US)
Venue
ACM Transactions on Software Engineering and Methodology
Published
2022-03-04
Volume
31
Issue
2
Pages
1-58
ISSN
  • 1049-331X (ISSN-L)
  • 1557-7392
OpenAlex
W3084812981
Cited by
137
References
204
Citations by year
2026: 21 · 2025: 39 · 2024: 37 · 2023: 30 · 2022: 8
FWCI
3.60
Type
journal article
Language
en
Open access
closed
Locations
ACM Transactions on Software Engineering and Methodology (publishedVersion)
Funders
NSF (CCF-1927679, CCF-1955853, CCF-2007246, and CCF-1815186)
Topics
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Research
  • Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
  • Physical Sciences › Computer Science › Software › Advanced Software Engineering Methodologies
Keywords
deep learning, software engineering research, software development automation, automated feature engineering, machine learning in software engineering, research roadmap, software engineering tasks

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A Systematic Literature Review on the Use of Deep Learning in Software Engineering Research. https://app.cochat.ai/papers/10.1145/3485275
The paper
Watson, C., Cooper, N., Palacio, D. N., Moran, K., & Poshyvanyk, D. (2022). A Systematic Literature Review on the Use of Deep Learning in Software Engineering Research. ACM Transactions on Software Engineering and Methodology. https://doi.org/10.1145/3485275

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.