Evaluating strategies for study selection in systematic literature studies
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
Context: The study selection process is critical to improve the reliability of secondary studies. Goal: To evaluate the selection strategies commonly employed in secondary studies in software engineering. Method: Building on these strategies, a study selection process was formulated and evaluated in a systematic review. Results: The selection process used a more inclusive strategy than the one typically used in secondary studies, which led to additional relevant articles. Conclusions: The results indicates that a good-enough sample could be obtained by following a less inclusive but more efficient strategy, if the articles identified as relevant for the study are a representative sample of the population, and there is a homogeneity of results and quality of the articles.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Systematic Mapping Studies in Software Engineering
Petersen, K., Feldt, R., Mujtaba, S., & Mattsson, M. (2008) · Electronic Workshops in Computing · doi:10.14236/ewic/ease2008.8 · Cited by 3,239
Self-citationA systematic review of systematic review process research in software engineering
Kitchenham, B., & Brereton, P. (2013) · Information and Software Technology · doi:10.1016/j.infsof.2013.07.010 · Cited by 1,094
A visual analysis approach to validate the selection review of primary studies in systematic reviews
Felizardo, K. R., Andery, G. F., Paulovich, F. V., Minghim, R., & Maldonado, J. C. (2012) · Information and Software Technology · doi:10.1016/j.infsof.2012.04.003 · Cited by 60 · cited 1×
“by visualization of citations and content maps to identify clusters of similar studies [2].”
Background
Cited by
Guidelines for conducting systematic mapping studies in software engineering: An update
Petersen, K., Vakkalanka, S., & Kuzniarz, L. (2015) · Information and Software Technology · doi:10.1016/j.infsof.2015.03.007 · Cited by 2,669
Self-citationTesting machine learning based systems: a systematic mapping
Riccio, V., Jahangirova, G., Stocco, A., Humbatova, N., Weiss, M., & Tonella, P. (2020) · Empirical Software Engineering · doi:10.1007/s10664-020-09881-0 · Cited by 251
“As suggested by Ali and Petersen [71], we used a think-aloud protocol in which each assessor speaks out the thought process of inclusion/exclusion when applying the criteria to a study.”
MethodInfluentialWhat is DevOps?: A Systematic Mapping Study on Definitions and Practices
Jabbari, R., bin Ali, N., Petersen, K., & Tanveer, B. (2016) · Proceedings of the Scientific Workshop Proceedings of XP2016 · doi:10.1145/2962695.2962707 · Cited by 230
Self-citationState of the art of cyber-physical systems security: An automatic control perspective
Zacchia Lun, Y., D’Innocenzo, A., Smarra, F., Malavolta, I., & Di Benedetto, M. D. (2019) · Journal of Systems and Software · doi:10.1016/j.jss.2018.12.006 · Cited by 211
“proposed in [69], two researchers classified each potentially relevant study either as relevant, uncertain, or irrelevant; any study classified as irrelevant has been directly excluded, whereas all the other”
BackgroundMachine learning to optimize literature screening in medical guideline development
Harmsen, W., de Groot, J., Harkema, A., van Dusseldorp, I., de Bruin, J., van den Brand, S., & van de Schoot, R. (2024) · Systematic Reviews · doi:10.1186/s13643-024-02590-5 · Cited by 21
Metadata
- Authors
- Ali, N. B., & Petersen, K.
- Venue
- Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement
- Publisher
- ACM
- Published
- 2014-09-18
- Pages
- 1-4
- ISSN
- 1938-6451 (ISSN-L)
- OpenAlex
- W2137660362
- Cited by
- 64
- References
- 3
- FWCI
- 2.28
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evaluating strategies for study selection in systematic literature studies. https://app.cochat.ai/papers/10.1145/2652524.2652557
- The paper
- Ali, N. B., & Petersen, K. (2014). Evaluating strategies for study selection in systematic literature studies. Proceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement. https://doi.org/10.1145/2652524.2652557
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.