Systematic literature reviews in software engineering: Preliminary results from interviews with researchers
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
Systematic Literature Reviews (SLRs) have been gaining significant attention from software engineering researchers since 2004. Several researchers have reported their experiences of and lessons learned from applying systematic reviews to different subject matters in software engineering. However, there has been no attempt at independently exploring experiences and perceptions of the practitioners of systematic reviews in order to gain an in-depth understanding of various aspects of systemic reviews as a new research methodology in software engineering. We assert that there is a need of evidence based body of knowledge about the application of systematic reviews in software engineering. To address this need, we have started an empirical research program that aims to contribute to the growing body of knowledge about systematic reviews in software engineering. This paper reports the design, logistics, and results of the first phase empirical study carried out in this program. The results provide interesting insights into different aspects of systematic reviews based on the analysis of the data gathered from 17 interviewees with varying levels of knowledge of and experiences in systematic reviews. The findings from this study are expected to contribute to the existing knowledge about using systematic reviews and help further improve the state-of-the-practice of this research methodology in software engineering.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Systematic Literature Reviews in Software Engineering—A Systematic Literature Review
Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.09.009 · Cited by 4,537
Empirical studies of agile software development: A systematic review
Dybå, T., & Dingsøyr, T. (2008) · Information and Software Technology · doi:10.1016/j.infsof.2008.01.006 · Cited by 2,668 · cited 3×
“Software Engineering (SE) researchers have been conducting and reporting more and more SLRs on diverse topics such as agile software development [6], regression testing [11], process modeling [26], variability management [5], cost estimation [17], organizational motivators for CMM-based process…”
BackgroundBrereton, P., Kitchenham, B. A., Budgen, D., Turner, M., & Khalil, M. (2007) · Journal of Systems and Software · doi:10.1016/j.jss.2006.07.009 · Cited by 2,394 · cited 3×
“Researchers have also reported best practise and experiences of conducting and reporting systematic reviews [4, 8, 24, 2].”
BackgroundMethod(2004) · doi:10.5555/998675.999432 · Cited by 656
Applying Systematic Reviews to Diverse Study Types: An Experience Report
Dyba, T., Dingsoyr, T., & Hanssen, G. K. (2007) · First International Symposium on Empirical Software Engineering and Measurement (ESEM 2007) · doi:10.1109/esem.2007.59 · Cited by 391
Cited by
Systematic literature studies: database searches vs. backward snowballing
Jalali, S., & Wohlin, C. (2012) · Proceedings of the ACM-IEEE international symposium on Empirical software engineering and measurement · doi:10.1145/2372251.2372257 · Cited by 659
“Given that the number of published secondary studies increases [18], it is perceived as important to understand whether or not the first step in the searches impacts the actual outcomes of the systematic literature study, in particular since many published papers do not use all steps recommended in the guidelines.”
BackgroundResultDeepfake Detection: A Systematic Literature Review
Rana, M. S., Nobi, M. N., Murali, B., & Sung, A. H. (2022) · IEEE Access · doi:10.1109/access.2022.3154404 · Cited by 573
Identifying relevant studies in software engineering
Zhang, H., Babar, M. A., & Tell, P. (2011) · Information and Software Technology · doi:10.1016/j.infsof.2010.12.010 · Cited by 557
Self-citationA Map of Threats to Validity of Systematic Literature Reviews in Software Engineering
Zhou, X., Jin, Y., Zhang, H., Li, S., & Huang, X. (2016) · 2016 23rd Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec.2016.031 · Cited by 260
Self-citationTools to Support Systematic Literature Reviews in Software Engineering: A Mapping Study
Marshall, C., & Brereton, P. (2013) · 2013 ACM / IEEE International Symposium on Empirical Software Engineering and Measurement · doi:10.1109/esem.2013.32 · Cited by 87
“Conclusions: A variety of tools are available to support the SLR process although many are in the early stages of development and usage.”
Background
Metadata
- Authors
- Babar, M. A., & Zhang, H.
- Venue
- 2009 3rd International Symposium on Empirical Software Engineering and Measurement
- Publisher
- IEEE
- Published
- 2009-10
- Pages
- 346-355
- ISSN
- 1938-6451 (ISSN-L)
- OpenAlex
- W1975516699
- Cited by
- 90
- References
- 5
- FWCI
- 2.03
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Systematic literature reviews in software engineering: Preliminary results from interviews with researchers. https://app.cochat.ai/papers/10.1109/esem.2009.5314235
- The paper
- Babar, M. A., & Zhang, H. (2009). Systematic literature reviews in software engineering: Preliminary results from interviews with researchers. 2009 3rd International Symposium on Empirical Software Engineering and Measurement. https://doi.org/10.1109/esem.2009.5314235
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.