Guidelines for snowballing in systematic literature studies and a replication in software engineering
The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index; 12 of the 23 sources its claims cite were checked for support.
Abstract
Background: Systematic literature studies have become common in software engineering, and hence it is important to understand how to conduct them efficiently and reliably.
Overview
CoChat Summary
TL;DR
The paper proposes a detailed snowballing procedure for systematic literature studies in software engineering and evaluates it by replicating a published systematic literature review. It concludes that snowballing can be a viable first search strategy and, in this case, found the same set of studies as the prior reviews.
Key findings
- The replication began with a tentative start set of 13 candidates; 3 were included in the start set.
- Across the study, 38 papers were evaluated and 11 papers were included in the final set.
- Efficiency varied sharply by step: start set 3/13 = 23%, iteration 1 7/25 = 28%, iteration 2 1/223 = 0.4%, and iteration 3 0%; overall efficiency was 3.7%.
- The paper states that the replication produced the same papers as the prior systematic literature reviews, and that 9 studies were in common with both reviews in [11].
Methods
- The paper develops a snowballing procedure based on experience from several systematic literature reviews and experimenting with different approaches; it specifies backward and forward snowballing, inclusion/exclusion handling, authors, and data extraction.
- For the replication, the author searched Google Scholar using the string “cross-company within-company software effort estimation” with time frame 1995–2005.
- The replication examined candidate papers through reference lists and citations, using title/publication-year screening, then abstracts, then full texts when needed.
Limitations
- The paper states a main validity threat: the researcher had read the original study 6–12 months earlier, which may have affected inclusion/exclusion decisions.
- The author notes that the identified start set was far from perfect, because the three start-set papers had one author in common, which could miss papers not linked to those papers.
- Evidence of limitations evident in the text: the efficiency results are based on a single replication case in one topic area, so generalizability beyond this case is not established (evident from the study design).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
10 Valid · 3 Not valid · 11 Not checked · 131 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“The same papers are identified.”
What the sources say
- Disputedcites [11] [11] MacDonell, S., Shepperd, M., Kitchenham, B. A. and Mendes, E. 2010. How reliable are systematic reviews in empirical software engineering?. IEEE Transactions on Software Engineering 36, 5, 676-687. - sourceExperimental — accuracy not yet measured
The Q1.6 ‘Papers included’ section and the text following Table 1 say the two reviews selected ten primary studies each, but only nine were common to both. The paper describes citation analysis within those reviews, not a snowballing replication that identified exactly the same papers. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Every source the paper cites for this claim was checked and supports it.
In the paper
“MacDonell et al. evaluated the reliability of systematic reviews through comparing the results of two studies with a common research question performed by two independent groups of researchers.”
What the sources say
- Supportedcites [11] [11] MacDonell, S., Shepperd, M., Kitchenham, B. A. and Mendes, E. 2010. How reliable are systematic reviews in empirical software engineering?. IEEE Transactions on Software Engineering 36, 5, 676-687. - source
The Introduction says the paper assesses systematic-review reliability by comparing two independent reviews addressing the same research question. Section 3.1 specifies that two teams undertook those reviews independently, and the Conclusions report that the teams' outcomes were compared. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Nine studies are in common with both previous systematic literature reviews presented in [11].”
What the sources say
- Supportedcites [11] [11] MacDonell, S., Shepperd, M., Kitchenham, B. A. and Mendes, E. 2010. How reliable are systematic reviews in empirical software engineering?. IEEE Transactions on Software Engineering 36, 5, 676-687. - source
The passage following Table 1 states that both systematic literature reviews selected 10 primary studies, with nine in common. This directly supports the citing sentence. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“If using the papers studied in detail as a efficiency measure, then the different approaches requires about the same effort.”
What the sources say
- Supportedcites [11] [11] MacDonell, S., Shepperd, M., Kitchenham, B. A. and Mendes, E. 2010. How reliable are systematic reviews in empirical software engineering?. IEEE Transactions on Software Engineering 36, 5, 676-687. - source
The Table 1 discussion says both reviews selected 10 primary studies, and §4.2 describes their data-extraction work on those studies. Thus, if the number of papers studied in detail is used as the efficiency measure, the two approaches are comparable on that measure; this does not establish equal time spent. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The guidelines for using snowballing as a way to search for relevant literature was successfully applied to a systematic literature review.”
plainstates a causeSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
20 more claims
Create a free account to see all 155 claims and every source verdict.
- Probably not a substantial claim. 111
- It may state two separate claims as one. 60
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“we applied forward and backward snowballing based on the guidelines by Wohlin et al. [26]”
From A Multivocal Review of MLOps Practices, Challenges and Open Issues (2026) · cites this paper as Wohlin et al. [26]
The cited source says what the claim attributes to it.
In the citing paper
“As a guideline, this study was performed following well-known principles of systematic studies: References [193, 196, 207, 220].”
From Quantum Combinatorial Optimization in the NISQ Era: A Systematic Mapping Study (2024) · cites this paper as [220]
The cited source says what the claim attributes to it.
In the citing paper
“To minimize the variables that could compromise our study, we (i) conducted our study following accepted guidelines of systematic studies [181, 193, 207, 220], (ii) carefully designed our study in advance, and (iii) assessed, validated, and discussed the potential threats in each phase of the study.”
From Quantum Combinatorial Optimization in the NISQ Era: A Systematic Mapping Study (2024) · cites this paper as [220]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“HDFS logs describe operations on the storage block pool [22].”
From Practitioners’ expectations on log anomaly detection (2025) · cites this paper as [22]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“only 4 papers [19], [20], [22], [36] evaluate their techniques across different domains, training on a source domain and testing on a different target domain.”
From Practitioners’ expectations on log anomaly detection (2025) · cites this paper as [22]
Experimental — accuracy not yet measured
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Analyzing the Past to Prepare for the Future: Writing a Literature Review
Webster, J., & Watson, R. T. (2002) · MIS Quarterly · doi:10.2307/4132319 · Cited by 7,449 · cited 7×
“Similar ideas have been brought into information systems research, e.g. by Webster and Watson [2].”
BackgroundMethodInfluentialSystematic Literature Reviews in Software Engineering—A Systematic Literature Review
Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.09.009 · Cited by 4,537 · cited 2×
“[13, 14] gives the impression that automation is better than manual search.”
BackgroundSystematic Mapping Studies in Software Engineering
Petersen, K., Feldt, R., Mujtaba, S., & Mattsson, M. (2008) · Electronic Workshops in Computing · doi:10.14236/ewic/ease2008.8 · Cited by 3,239 · cited 1×
“Systematic mapping studies have been highlighted as a complement to systematic literature reviews [8].”
BackgroundGreenhalgh, T., & Peacock, R. (2005) · BMJ · doi:10.1136/bmj.38636.593461.68 · Cited by 2,045 · cited 4×
“Greenhalgh and Peacock [15] applied three different search methods in their research, and concluded that protocol-driven search approaches by themselves are not necessarily the most efficient method regardless of the number of databases used, since some sources may be found through personal knowledge/contacts (e.”
BackgroundResultInfluentialSystematic literature reviews in software engineering – A tertiary study
Kitchenham, B., Pretorius, R., Budgen, D., Pearl Brereton, O., Turner, M., Niazi, M., & Linkman, S. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.03.006 · Cited by 1,230 · cited 2×
“[13, 14] gives the impression that automation is better than manual search.”
BackgroundMethodGLOBAL ECONOMIC PROSPECTS: THE RECESSION MAY BE OVER BUT WHERE NEXT?
(2013) · The Financial Crisis and the Global South · doi:10.2307/j.ctt183pb3w.5 · Cited by 908
Building knowledge through families of experiments
Basili, V. R., Shull, F., & Lanubile, F. (1999) · IEEE Transactions on Software Engineering · doi:10.1109/32.799939 · Cited by 852
Rodrik, D., & Velasco, A. (1999) · National Bureau of Economic Research · doi:10.3386/w7364 · Cited by 711
Leading issues in economic development
(2000) · Choice Reviews Online · doi:10.5860/choice.38-2269 · Cited by 710
Systematic literature studies: database searches vs. backward snowballing
Jalali, S., & Wohlin, C. (2012) · Proceedings of the ACM-IEEE international symposium on Empirical software engineering and measurement · doi:10.1145/2372251.2372257 · Cited by 659 · cited 11×
“Other complementary searches were not done either since the objective is to compare snowballing with a database-firstapproach in a similar way as in [19], although with a better specified procedure for using snowballing.”
BackgroundResultInfluentialSelf-citation
Cited by
Items 3,501–3,600 of 3,934
Ribeiro, P. V. D. S., & Neto, M. G. D. S. (2024) · Revista Brasileira de Informática na Educação · doi:10.5753/rbie.2024.4005 · Cited by 2
Leal, L. G. D. N., Vieira, R. B. F., Marques, I. W. D. A., Filho, Á. M. D. C., Imbiriba, K. G., Pacheco, F. W. F. E. V., & Gomes, S. C. (2025) · Cuadernos de Educación y Desarrollo · doi:10.55905/cuadv17n1-122
The synthesizer in the music education research: A snowball review
Madarnás, P. P. (2023) · Journal of Music Technology and Education · doi:10.1386/jmte_00063_1
Jabłoński, K., & Jarzębowicz, A. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-77493-5_2
REVERSE ENGINEERING FOR SOFTWARE MAINTENANCE: A SYSTEMATIC REVIEW OF LITERATURE
Vera, R. A. A., Pérez, J. D. L. R. B., Mendoza, J. C. D., et al. (2024) · Revista Ingenierías Universidad de Medellín · doi:10.22395/rium.v23n44a2
“Con el propósito de incrementar el número de estudios relevantes, la RSL incorporó la estrategia denominada bola de nieve hacia adelante [9]; dicha estrategia sería aplicada al primer conjunto de Estudios Primarios Seleccionados (EPS).”
Introducing Software Supply Chain
Wang, Y., Cheung, S.-C., Yu, H., & Zhu, Z. (2024) · Springer eBooks · doi:10.1007/978-981-96-1797-5_1
A systematic review on Purpose in Life in Emerging AdulthoodPreprint
Banda, R., Lleó, Á., & García‐Manglano, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5151864
The Progression of Competencies and Dispositions: How do Maturing Software Engineers Compare?
Clear, T., & Clear, A. (2024) · 2022 IEEE Frontiers in Education Conference (FIE) · doi:10.1109/fie61694.2024.10892834
“[6] The review found that research analysing job requirements for software developers of different levels was limited; “experience” as a keyword was seldom mentioned; and a common distinction was made between and "soft" and "hard" skills - the latter being skills that were "technical, such as…”
van Vulpen, P., & Jansen, S. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5160239 · Cited by 1
Navigating Sustainability Paradoxes: A Systematic Literature Review of Paradoxical Cognitive FramePreprint
Gesiot, G., Alosi, A., Rizzi, F., Annunziata, E., & Frey, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5141350
Navigating Sustainability Paradoxes: A Systematic Literature Review of Paradoxical Cognitive FramePreprint
Gesiot, G., Alosi, A., Rizzi, F., Annunziata, E., & Frey, M. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5164476
Value-Based Decision-Making in Software Agents: A Systematic Literature Review
Guerrero, E., Tzeng, S.-T., Păstrăv, C., & Dignum, F. P. M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-85463-7_9 · Cited by 1
Combining Quality and Agile: A Contribution to Hybrid Models in Regulated Environments
da Silva, H. L. S., Amaral, D. C., & Piccirillo, I. N. (2025) · Springer proceedings in business and economics · doi:10.1007/978-3-031-72490-9_52
Enabling Android Application Monitoring by Characterizing Security-Critical Code Fragments
Eikerling, H., & Kampkötter, A. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-82362-6_25
Nabi, F., Maple, C. R., & Attaullah, H. M. (2025) · Research Square · doi:10.21203/rs.3.rs-6372104/v1
The Metaverse and Agritourism Marketing in Africa: Benefits, Challenges, and the Future
Mudzimba, E., & Mashero, K. P. (2025) · Springer eBooks · doi:10.1007/978-3-031-78682-2_2
Irving, A., Lammy, S., & Alakandy, L. (2025) · Case Reports in Oncology · doi:10.1159/000545935
“Table 1 shows the results of literature search using key terms and snowballing techniques [14].”
Vehicle Routing and Scheduling Under Hours of Service Regulations: A ReviewPreprint
Zanella, A. F., Lima, R. H. P., Mulati, M. H., et al. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5227024
IT Project Postmortem Theory: Identifying Root Causes by Process TracingPreprint
Schmidt, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5196936
Uncovering the DevOps Landscape: A Scoping Review and Conceptualization Framework
Zhang, X., & Jaskolka, J. (2024) · Asia-Pacific Software Engineering Conference (APSEC) · doi:10.1109/apsec65559.2024.00067
“To ensure thorough coverage, we perform two rounds of backward and forward snowballing [6] to progressively expand our pool of relevant studies until no new XOps are found.”
Diplock, B. D., Urusov, A., Torchia, A., Turner, G. R., & Desrocher, M. E. (2025) · Brain Impairment · doi:10.1071/ib24092
Evaluation of Software Requirement Patterns for Electronic Health Record Systems
Martins, M. C., Kudo, T. N., & Bulcão-Neto, R. F. (2025) · doi:10.5753/sbsi.2025.246600
Automated Semantic Annotation of Data Management Plans: A Systematic Review
Martínková, J., Suchánek, M., & Kroha, P. (2025) · Data Science Journal · doi:10.5334/dsj-2025-016
A Systematic Literature Review on Multimodal Aspect-Based Sentiment Analysis
Divakarla, U., Krishna, K. V. H., Sharma, H., & Chandrasekarana, K. (2024) · International Conference on Parallel, Distributed and Grid Computing (PDGC) · doi:10.1109/pdgc64653.2024.10984221
“Datasets should cover multiple domains and contexts to ensure model generalizability and minimize potential bias [18].”
MODELOS DE RACIOCÍNIO E ARQUITETURA EXTENSÍVEL PARA PESQUISA INTELIGENTE
Júnior, H. C. P. (2025) · Congresso Internacional IDEA: Inovação, Diálogo e Experiências na Aprendizagem (IDEA) · doi:10.29327/9786527213314.1086647
Moradi, M., Fabarisov, T., Kılınççeker, O., et al. (2026) · Journal of Systems and Software · doi:10.1016/j.jss.2026.112957
Delgado, L. M., Pompeu, B. F., Pompeu, B. F., et al. (2025) · Journal of Laparoendoscopic & Advanced Surgical Techniques · doi:10.1089/lap.2025.0049
Batz, V., Liedtke, V. B., Lameski, P., et al. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-93221-2_1
Circular Economy Indicators for Product Design – Calculation and Applicability
Kuhn, P., Araujo, J., Pruhs, A., Kusch, A., Bertagnolli, F., & Viere, T. (2025) · Wiley eBooks · doi:10.1002/9783527847358.ch07 · Cited by 2
Rong, R.-Z., Wang, Z., & He, C. D. (2025) · Journal of Robotic Surgery · doi:10.1007/s11701-025-02433-y
“In addition, relevant references were manually searched and reviewed to ensure no studies were overlooked [12].”
AI's Role in Collaborative Learning Settings in Higher Education – A Systematic Review
Pernu, H., Paananen, S., Aksovaara, S., & Silvennoinen, M. (2025) · Bled eConference · doi:10.18690/um.fov.4.2025.21
A Systematic Literature Review of Deepfake Face Image Detection with Transfer Learning Techniques
Wimalasena, W., Herath, H., & Hewapathirana, I. U. (2025) · International Research Conference on Smart Computing and Systems Engineering (SCSE) · doi:10.1109/scse65633.2025.11031009 · Cited by 1
“To ensure comprehensive coverage, the study also implemented a snowballing technique, [19]wherein the reference lists of highly cited and particularly relevant papers were systematically examined to identify additional pertinent studies that may not have been captured in the initial database…”
Visualization Tools for Machine Learning Pipelines: A Review
Golendukhina, V., Felderer, M., & Sonnleithner, L. (2025) · IEEE Pacific Visualization Conference (PacificVis) · doi:10.1109/pacificvis64226.2025.00049 · Cited by 2
“Finally, we planned one iteration of backward snowballing, as described by [38] to identify literature that we possibly missed with the search term.”
Rouabhia, D., & Hadjadj, I. (2025) · Research Square · doi:10.21203/rs.3.rs-6793290/v1 · Cited by 14
Kunkcu, H., Besiktepe, D., Koç, K., & Gürgün, A. P. (2025) · CIB Conferences · doi:10.7771/3067-4883.1885
Kaikkonen, P., Carta襻, G. 羖., & Happonen, A. (2025) · Intelligent and sustainable manufacturing · doi:10.70322/ism.2025.10018
Prefabricated biogenic construction and their fire-safety properties – a literary review
Jørgensen, A. J., Ejstrup, H., Beim, A., et al. (2025) · CRC Press eBooks · doi:10.1201/9781003658641-199
Collaboration in Computing Education: Equity and Self-Directed Learning from the Global South
Bispo, E. L., dos Santos, S. C., & Matos, M. V. A. B. D. (2025) · Simpósio Brasileiro de Sistemas Colaborativos (SBSC) · doi:10.5753/sbsc_estendido.2025.8658
dos Santos, A. C. R., & França, J. B. D. S. (2025) · doi:10.5753/sbsc_estendido.2025.6635
Alsanousi, B. (2025) · University of North Texas Digital Library (University of North Texas) · doi:10.12794/metadc2443125
Serter, M., & Ayalp, G. G. (2025) · Resilience · doi:10.32569/resilience.1594157 · Cited by 1
Özyılmaz, A. P., Demirci, F. S., Okudan, O., & Işık, Z. (2025) · Sustainability · doi:10.3390/su17146639
A Research Agenda Advancing Circular Economy Skills
Trevisan, A. H., Acerbi, F., Dukovska‐Popovska, I., et al. (2025) · Proceedings of the I-ESA Conference · doi:10.1007/978-3-031-93995-2_17
Chrzanowski, M., & Zawada, P. (2025) · Journal of Modern Science · doi:10.13166/jms/207309
Constituents of Productivity in Information Systems Work Environments: A Conceptual Model
Maré, A., & Smuts, H. (2025) · Communications in computer and information science · doi:10.1007/978-3-031-95901-1_2
A Conceptual Model for Project Observatories
de Farias, I. H., Vieira, J. K. M., de Moura, H. P., et al. (2025) · IEEE Access · doi:10.1109/access.2025.3589743
“In stage 1, the recommendation 317 presented in [41] was followed, and an exploratory search 318 was conducted on Google Scholar to identify articles that 319 addressed the Observatory theme.”
Femminini, A., Salerni, A., & Stanzione, I. (2025) · EDUCATION SCIENCES AND SOCIETY · doi:10.3280/ess1-2025oa19732
Design Heuristics for Web-Based Severe Weather Risk Communication
Jit, S. S., Leiva, A., Bicksler, R., & Soden, R. (2025) · ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies (COMPASS) · doi:10.1145/3715335.3735482
“Then, using a snowball approach [64], this was supplemented with articles that were commonly cited in the initial set of articles.”
Liu, L., Wei, L., Zhang, W., Li, S., Zhou, Y., Liu, Y., Cheung, S.-C., & Lyu, M. R.-T. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10685-3 · Cited by 1
“We then conduct a snowballing process (Wohlin 2014) to ensure that relevant papers and academic tools that might have been missed during the keyword search process are also included in our survey.”
Generic Design Requirement for Enhanced Remanufacturability of Electric Vehicle Components
L’Allemand, J., Ramström, S., Duberg, J. V., & Sundin, E. (2025) · Procedia CIRP · doi:10.1016/j.procir.2024.12.136 · Cited by 2
Systematic Review of Parent-Youth Discrepancies in Exposures to Community Violence
Hong, K., Morelli, N. M., Tabibian, D. R., Jimenez, M. G., Straub, D., & Villodas, M. T. (2025) · Clinical Child and Family Psychology Review · doi:10.1007/s10567-025-00532-8
“A snowball technique following guidelines from Wohlin (2014) was utilized.”
Mind the Gaps: An Infosec Grc Roadmap Research for Small OrganisationsPreprint
Steglich, C., Poltronieri, I., & Zorzo, A. F. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5384356 · Cited by 1
Lai, J., & Choi, Y. H. (2025) · International Journal of Qualitative Studies in Education · doi:10.1080/09518398.2025.2539370
Delving into data on students’ instructional knowledge: A scoping review of the empirical research
Stevens, M., Depaepe, F., & Elen, J. (2025) · Educational Research Review · doi:10.1016/j.edurev.2025.100717
Digital Transformation Strategies to Strengthen Business Resilience in Economic Crisis Situation
Baehaqi, B., & Amin, M. (2025) · Jurnal Ilmiah Manajemen Kesatuan · doi:10.37641/jimkes.v13i5.3597
Ritchie, C. (2025) · Open Access Victoria University of Wellington | Te Herenga Waka (Figshare) · doi:10.26686/wgtn.29936771
A Review of Estimation in Software Engineering
Tomescu, K. J., Gowran, N., Gomez, L., Delahunty, E., McCarren, A. L., Marks, G., Yılmaz, M., Messnarz, R., & Clarke, P. M. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-04288-0_10
Marquardt, N., Choi, V., Martyn‐Dickens, C., Görgens, M., Mathewlynn, S., Kurth, T., Bouteiller, P., & Wieler, L. H. (2025) · BMJ Open · doi:10.1136/bmjopen-2025-105622 · Cited by 2
Mapping of Maritime Ecosystem Components in the Cybersecurity Landscape
Roponena, E., Lielbārde, S., Citskovska, E., Brilingaitė, A., Bukauskas, L., & Pirta-Dreimane, R. (2025) · IEEE International Conference on Cyber Security and Resilience (CSR) · doi:10.1109/csr64739.2025.11130012 · Cited by 1
“Wohlin [22] also recommends contacting known authors in the reviewed field or authors of selected articles to identify additional articles.”
InfluentialSmart Cities Under Threat: A Systematic Review and Conceptual Risk Model
Roponena, E., Matisons, R., Citskovska, E., Rinkevičs, P. G., Pirta-Dreimane, R., & Priedols, G. (2025) · IEEE International Conference on Cyber Security and Resilience (CSR) · doi:10.1109/csr64739.2025.11130111
“The systematic literature review guidelines for snowballing method in [21] proposes a method for identifying relevant literature in software engineering projects.”
Himmelstoß, H., & Bauernhansl, T. (2025) · IFIP advances in information and communication technology · doi:10.1007/978-3-032-03550-9_1
Energy-Saving Techniques for Raspberry PI
Pyrhönen, J., & Ikonen, J. (2025) · International Conference on Computer Systems and Technologies (CompSysTech) · doi:10.1109/compsystech65493.2025.11136932
“The snowballing [4] and reverse snowball techniques were also utilized.”
The impact of digital literacy on the use of immersive technology: a systematic literature review
Wan, J., Feng, Z., Paes, D., & Lovreglio, R. (2026) · Virtual Reality · doi:10.1007/s10055-026-01449-5
Enhancing A Holistic Approach Towards Computational Thinking Assessment in Education
Lynch, J., Stokke, T., Adgerson, A., & Grant, E. S. (2025) · IEEE Integrated STEM Education Conference (ISEC) · doi:10.1109/isec64801.2025.11147256
“To broaden the scope of the literature and find resources that may be relevant to the research and add insight, forward and backward snowballing techniques were employed [8].”
Measuring Street Built Environments for Children’s Use: A Systematic Review of Measurement Tools
Cao, X., Meng, X., & Zhang, H. (2025) · Journal of Urban Health · doi:10.1007/s11524-025-01003-9 · Cited by 2
“Furthermore, Cao conducted a snowballing approach involving back-ward and forward citation tracking of the included studies [33, 34] to identify additional eligible references.”
Scaling Regulatory Operations (RegOps) in Multi-Team Agile EnvironmentsPreprint
Ochoa-Orihuel, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5472731
Common Threads in Incremental Data Flow Analysis: A Comprehensive Survey
Jana, A., & Khedker, U. P. (2025) · ACM Computing Surveys · doi:10.1145/3768155
Investigating the Use of Snowballing on Q&A Websites
Gomes, F., Mendes, T. S., Freire, S., Spinola, R. O., & Mendonça, M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04200-2_17 · Cited by 4
On Architectural Tactics for Resource-Constrained and Safety-Critical AI-Based Systems
Straub, P., Decker, C., & Kuhrmann, M. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-04200-2_16 · Cited by 1
Reengineering an Adaptive System to Dynamic Software Product Lines: An Experience report
da Cruz, D. T. M., Seixas, N. F. S., Oliveira, M. D. S., et al. (2025) · doi:10.5753/sbcars.2025.14142
“Following this stage, duplicate results were removed, and a “snowballing” technique [40] was employed to identify additional relevant studies by analyzing references from the collected articles.”
Access Granted – Carefully: Securing model information in collaborative modeling
Latifaj, M., Ciccozzi, F., & Cicchetti, A. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112640
Reimagining changeovers as a key value-adding business capability
Gungor, Z. E., Inan, G. G., Marques-McEwan, M., et al. (2025) · Production Planning & Control · doi:10.1080/09537287.2025.2554163
“Having conducted a literature review using the snowballing technique (Wohlin 2014), we identified weaknesses in the literature that assumed changeovers to be non-value-adding, as high-lighted in Section 2.”
Alchokr, R., Sunilkumar, A., Saake, G., Leich, T., & Krüger, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-032-05409-8_23 · Cited by 1
Thoth 2.0: Advancing an RSL Tool for Enhanced Snowballing Support
Comis, D., & Rodrigues, E. (2025) · Simpósio Brasileiro de Engenharia de Software (SBES) · doi:10.5753/sbes.2025.11616 · Cited by 1
“Among the techniques employed in SLRs, snowballing—a method for iteratively identifying relevant studies through backward (references) and forward (citations) chaining—has gained prominence for its ability to uncover literature that traditional database searches might miss [21].”
A Mapping Study About Training in Industry Context in Software Engineering
de Andrade, B. A., Siqueira, R., Gomes, L. C. S., Oliveira, A., & Ribeiro, D. M. (2025) · Simpósio Brasileiro de Engenharia de Software (SBES) · doi:10.5753/sbes.2025.10420 · Cited by 7
Successfully Integrating Early Validation and Verification in Industrial MBSE
Cederbladh, J., & Krems, D. (2025) · INCOSE International Symposium · doi:10.1002/iis2.70100
“Through the addition of a snowballing activity (Wohlin, 2014), i.e., by applying the same criteria on articles referenced or articles referencing the articles meeting the criteria, additional papers are uncovered not caught by the search string (for example not available in the databases).”
Towards Using Personas in Requirements Engineering: What Has Been Changed Recently?
Muzammel, C. S., Spichkova, M., & Harland, J. (2025) · IEEE International Requirements Engineering Conference Workshops (REW) · doi:10.1109/rew66121.2025.00019
“We conducted forward and back-ward snowballing, adapting the process proposed by Wohlin [22].”
Review of autism spectrum disorder databases for the identification of candidate genes
Martínez-Minguet, D., Noël, R., Alberto García, S., Costa, M., & Pastor, O. (2025) · Database · doi:10.1093/database/baaf067 · Cited by 1
“We then conduct forward and backward snowballing as described by Wohlin [ 26 ], to identify other articles providing new databases or referring to updated versions of previously identified databases.”
Modo, T. (2025) · European Journal of Theoretical and Applied Sciences · doi:10.59324/ejtas.2025.3(6).02
Virtual Couture: Exploring the Integration of NFTs in Luxury Fashion Brands
Drašković, N., Šebek, V., & Fraculj, M. (2025) · Palgrave studies in cross-disciplinary business research, in association with EuroMed Academy of Business · doi:10.1007/978-3-031-96680-4_7
Laktuka, K., Luksta, I., & Blumberga, D. (2025) · Environmental and Climate Technologies · doi:10.2478/rtuect-2025-0045
“The snowballing approach [45], [46], was used, whereby references to other policy or regulatory documents found within the text passages were included in analysis.”
Detecting Anomalous Vessel Trajectories: A Collaborative Clustering-Based Approach
Ribeiro, C., Bedo, M. V. N., Mello, R. D. S., Paes, A., & de Oliveira, D. (2025) · Simpósio Brasileiro de Bancos de Dados (SBBD) · doi:10.5753/sbbd.2025.247003
“The snowballing technique was employed as the primary search strategy, following the procedures for backward and forward iterations described by [Wohlin 2014].”
Towards a Framework for Systematic API Migrations
Jíša, N., & Pergl, R. (2025) · Annals of Computer Science and Information Systems · doi:10.15439/2025f1225 · Cited by 1
“In this section, we review challenges relevant for this paper and describe some additional research works discovered by snowballing [10].”
So, A. R. P., Souza, F. C. M., Souza, A. C. C., & Fabri, J. A. (2025) · Simpósio Brasileiro de Jogos e Entretenimento Digital (SBGames) · doi:10.5753/sbgames.2025.10290 · Cited by 2
“First, the search string was applied to the IEEE 1 , ACM 2 and Scopus databases, and then the snowball strategy (snowballing) [Wohlin 2014] was conducted.”
Ilahiyyah, I., Eliyana, A., & Usman, I. (2025) · Competitiveness Review An International Business Journal incorporating Journal of Global Competitiveness · doi:10.1108/cr-08-2024-0158
Limaylla-Lunarejo, M.-I., Condorí-Fernández, N., Luaces, M. R., & Karras, O. (2025) · Empirical Software Engineering · doi:10.1007/s10664-025-10736-9 · Cited by 1
Li, W., Shi, J., & Chai, C. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2573834 · Cited by 1
“The snowball-ing method was used to involve reviewing the reference lists of initially identified papers to find potentially relevant studies not captured in the database search (Wohlin, 2014).”
Williamson, S. D., Aaby, A., Ejersbo, A. O., Vestergaard, M., & Ravn, S. L. (2025) · Spinal Cord · doi:10.1038/s41393-025-01141-6 · Cited by 2
“Additionally, all included studies were subject to forward and backward snowballing [26].”
A systematic literature review on logging smell detection
Madi, N., & Binkhonain, M. (2025) · Information and Software Technology · doi:10.1016/j.infsof.2025.107961 · Cited by 3
“Although Wohlin [10] suggests that the starting set for snowballing should not be too small, this recommendation depends on the breadth of the research area.”
InfluentialAssessing the Usability & Acceptability of PTSD Apps: Insights from Systematic Literature Review
Esener, Y., Kim, H., Kaya, A., & Kaz-Onyeakazi, I. A. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2580545
“Systematic literature reviews enable researchers to develop a shared and thorough understanding of the current state of knowledge in each field (Kitchenham, 2004; Wohlin, 2014).”
bBench: A Comprehensive Performance Benchmark for Blockchain Applications
Vidal, F. R., Ivaki, N., & Laranjeiro, N. (2025) · Proceedings/Proceedings - International Symposium on Software Reliability Engineering · doi:10.1109/issre66568.2025.00046 · Cited by 1
“However, a user may use the benchmark’s output to perform such tests.”
Key Characteristics of Educational Video Games for STEM Education - a Literature Review
Ivanov, S., Nikolova, N., Yordanov, B., & Bontchev, B. (2025) · International Conference on Cybersecurity and AI-Based Systems (Cyber-AI) · doi:10.1109/cyber-ai66431.2025.11233282 · Cited by 1
“The snowballing technique includes the following steps [8]: • Determining the initial set of articles (Seed); • Iteratively adding new articles through forward and backward snowballing; • Select articles based on the inclusion criteria.”
Enhancing STEM Workforce Development via Non-Degree Credentialing: A Systematic Literature Review
Zinnah, F., Minichiello, A., & Asghar, M. (2025) · Community College Journal of Research and Practice · doi:10.1080/10668926.2025.2582707 · Cited by 1
Into the deep: Retracing the liver oil trade of shark, ray, and chimaera fisheries in India
Tyabji, Z., Jabado, R. W., Waghe, R., et al. (2025) · Reviews in Fish Biology and Fisheries · doi:10.1007/s11160-025-10015-2
AI-based emotion recognition in dementia through facial expression: A scoping review
Gerbaudo-González, N., Condorí-Fernández, N., Catalá, A., & Gandoy‐Crego, M. (2025) · Ageing Research Reviews · doi:10.1016/j.arr.2025.102975 · Cited by 10
Sport specialization or diversification? Proposals to move beyond a polarized debate
Allison-Abaunza, A. (2025) · Loisir et Société / Society and Leisure · doi:10.1080/07053436.2025.2586475
“A snowballing strategy was also used to identify the main literature reviews (Wohlin, 2014).”
Cross-technology cloud offloading model
Al‐Ahmad, A., Shrestha, A., Ali, O., & Kahtan, H. (2025) · Journal of Cloud Computing Advances Systems and Applications · doi:10.1186/s13677-025-00819-9 · Cited by 1
“The snowballing process was repeated iteratively until no new relevant studies were found [47, 48].”
The Effect of Face Masks on COVID-19 Transmission: A Meta-Analysis
Lušková, M. (2025) · Economics of Disasters and Climate Change · doi:10.1007/s41885-025-00185-5
“Snowballing refers to the practice of examining the reference lists of relevant studies to identify additional eligible literature (Wohlin 2014).”
Sul, H. H., Gewehr, D. M., Kang, H., Ramos, O., Ungaro, R. C., & Bernstein, C. N. (2025) · Journal of Crohn s and Colitis · doi:10.1093/ecco-jcc/jjaf202 · Cited by 1
Metadata
- Authors
- Wohlin, C.
- Affiliations
- Blekinge Institute of Technology (SE)
- Corresponding author
- Wohlin, Claes
- ORCID
- Wohlin, Claes
- Venue
- Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering
- Publisher
- ACM
- Published
- 2014-05-13
- Pages
- 1-10
- OpenAlex
- W1975675278
- Cited by
- 4,234
- References
- 31
- Citations by year
- 2026: 433 · 2025: 552 · 2024: 651 · 2023: 594 · 2022: 566 · 2021: 477 · 2020: 356 · 2019: 250 · 2018: 175 · 2017: 98 · 2016: 60 · 2015: 17 · 2014: 4
- FWCI
- 50.91
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Evaluation and Assessment in Software Engineering (EASE) (publishedVersion)
- Funders
- Lunds Universitet
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- snowballing, systematic reviews, research guidelines, software engineering literature, replication study, literature search strategies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.