Guidelines for snowballing in systematic literature studies and a replication in software engineering
The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index; 12 of the 23 sources its claims cite were checked for support.
Abstract
Background: Systematic literature studies have become common in software engineering, and hence it is important to understand how to conduct them efficiently and reliably.
Overview
CoChat Summary
TL;DR
The paper proposes a detailed snowballing procedure for systematic literature studies in software engineering and evaluates it by replicating a published systematic literature review. It concludes that snowballing can be a viable first search strategy and, in this case, found the same set of studies as the prior reviews.
Key findings
- The replication began with a tentative start set of 13 candidates; 3 were included in the start set.
- Across the study, 38 papers were evaluated and 11 papers were included in the final set.
- Efficiency varied sharply by step: start set 3/13 = 23%, iteration 1 7/25 = 28%, iteration 2 1/223 = 0.4%, and iteration 3 0%; overall efficiency was 3.7%.
- The paper states that the replication produced the same papers as the prior systematic literature reviews, and that 9 studies were in common with both reviews in [11].
Methods
- The paper develops a snowballing procedure based on experience from several systematic literature reviews and experimenting with different approaches; it specifies backward and forward snowballing, inclusion/exclusion handling, authors, and data extraction.
- For the replication, the author searched Google Scholar using the string “cross-company within-company software effort estimation” with time frame 1995–2005.
- The replication examined candidate papers through reference lists and citations, using title/publication-year screening, then abstracts, then full texts when needed.
Limitations
- The paper states a main validity threat: the researcher had read the original study 6–12 months earlier, which may have affected inclusion/exclusion decisions.
- The author notes that the identified start set was far from perfect, because the three start-set papers had one author in common, which could miss papers not linked to those papers.
- Evidence of limitations evident in the text: the efficiency results are based on a single replication case in one topic area, so generalizability beyond this case is not established (evident from the study design).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
10 Valid · 3 Not valid · 11 Not checked · 131 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“The same papers are identified.”
What the sources say
- Disputedcites [11] [11] MacDonell, S., Shepperd, M., Kitchenham, B. A. and Mendes, E. 2010. How reliable are systematic reviews in empirical software engineering?. IEEE Transactions on Software Engineering 36, 5, 676-687. - sourceExperimental — accuracy not yet measured
The Q1.6 ‘Papers included’ section and the text following Table 1 say the two reviews selected ten primary studies each, but only nine were common to both. The paper describes citation analysis within those reviews, not a snowballing replication that identified exactly the same papers. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Every source the paper cites for this claim was checked and supports it.
In the paper
“MacDonell et al. evaluated the reliability of systematic reviews through comparing the results of two studies with a common research question performed by two independent groups of researchers.”
What the sources say
- Supportedcites [11] [11] MacDonell, S., Shepperd, M., Kitchenham, B. A. and Mendes, E. 2010. How reliable are systematic reviews in empirical software engineering?. IEEE Transactions on Software Engineering 36, 5, 676-687. - source
The Introduction says the paper assesses systematic-review reliability by comparing two independent reviews addressing the same research question. Section 3.1 specifies that two teams undertook those reviews independently, and the Conclusions report that the teams' outcomes were compared. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Nine studies are in common with both previous systematic literature reviews presented in [11].”
What the sources say
- Supportedcites [11] [11] MacDonell, S., Shepperd, M., Kitchenham, B. A. and Mendes, E. 2010. How reliable are systematic reviews in empirical software engineering?. IEEE Transactions on Software Engineering 36, 5, 676-687. - source
The passage following Table 1 states that both systematic literature reviews selected 10 primary studies, with nine in common. This directly supports the citing sentence. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“If using the papers studied in detail as a efficiency measure, then the different approaches requires about the same effort.”
What the sources say
- Supportedcites [11] [11] MacDonell, S., Shepperd, M., Kitchenham, B. A. and Mendes, E. 2010. How reliable are systematic reviews in empirical software engineering?. IEEE Transactions on Software Engineering 36, 5, 676-687. - source
The Table 1 discussion says both reviews selected 10 primary studies, and §4.2 describes their data-extraction work on those studies. Thus, if the number of papers studied in detail is used as the efficiency measure, the two approaches are comparable on that measure; this does not establish equal time spent. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The guidelines for using snowballing as a way to search for relevant literature was successfully applied to a systematic literature review.”
plainstates a causeSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
20 more claims
Create a free account to see all 155 claims and every source verdict.
- Probably not a substantial claim. 111
- It may state two separate claims as one. 60
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“we applied forward and backward snowballing based on the guidelines by Wohlin et al. [26]”
From A Multivocal Review of MLOps Practices, Challenges and Open Issues (2026) · cites this paper as Wohlin et al. [26]
The cited source says what the claim attributes to it.
In the citing paper
“As a guideline, this study was performed following well-known principles of systematic studies: References [193, 196, 207, 220].”
From Quantum Combinatorial Optimization in the NISQ Era: A Systematic Mapping Study (2024) · cites this paper as [220]
The cited source says what the claim attributes to it.
In the citing paper
“To minimize the variables that could compromise our study, we (i) conducted our study following accepted guidelines of systematic studies [181, 193, 207, 220], (ii) carefully designed our study in advance, and (iii) assessed, validated, and discussed the potential threats in each phase of the study.”
From Quantum Combinatorial Optimization in the NISQ Era: A Systematic Mapping Study (2024) · cites this paper as [220]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“HDFS logs describe operations on the storage block pool [22].”
From Practitioners’ expectations on log anomaly detection (2025) · cites this paper as [22]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“only 4 papers [19], [20], [22], [36] evaluate their techniques across different domains, training on a source domain and testing on a different target domain.”
From Practitioners’ expectations on log anomaly detection (2025) · cites this paper as [22]
Experimental — accuracy not yet measured
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Analyzing the Past to Prepare for the Future: Writing a Literature Review
Webster, J., & Watson, R. T. (2002) · MIS Quarterly · doi:10.2307/4132319 · Cited by 7,449 · cited 7×
“Similar ideas have been brought into information systems research, e.g. by Webster and Watson [2].”
BackgroundMethodInfluentialSystematic Literature Reviews in Software Engineering—A Systematic Literature Review
Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.09.009 · Cited by 4,537 · cited 2×
“[13, 14] gives the impression that automation is better than manual search.”
BackgroundSystematic Mapping Studies in Software Engineering
Petersen, K., Feldt, R., Mujtaba, S., & Mattsson, M. (2008) · Electronic Workshops in Computing · doi:10.14236/ewic/ease2008.8 · Cited by 3,239 · cited 1×
“Systematic mapping studies have been highlighted as a complement to systematic literature reviews [8].”
BackgroundGreenhalgh, T., & Peacock, R. (2005) · BMJ · doi:10.1136/bmj.38636.593461.68 · Cited by 2,045 · cited 4×
“Greenhalgh and Peacock [15] applied three different search methods in their research, and concluded that protocol-driven search approaches by themselves are not necessarily the most efficient method regardless of the number of databases used, since some sources may be found through personal knowledge/contacts (e.”
BackgroundResultInfluentialSystematic literature reviews in software engineering – A tertiary study
Kitchenham, B., Pretorius, R., Budgen, D., Pearl Brereton, O., Turner, M., Niazi, M., & Linkman, S. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.03.006 · Cited by 1,230 · cited 2×
“[13, 14] gives the impression that automation is better than manual search.”
BackgroundMethodGLOBAL ECONOMIC PROSPECTS: THE RECESSION MAY BE OVER BUT WHERE NEXT?
(2013) · The Financial Crisis and the Global South · doi:10.2307/j.ctt183pb3w.5 · Cited by 908
Building knowledge through families of experiments
Basili, V. R., Shull, F., & Lanubile, F. (1999) · IEEE Transactions on Software Engineering · doi:10.1109/32.799939 · Cited by 852
Rodrik, D., & Velasco, A. (1999) · National Bureau of Economic Research · doi:10.3386/w7364 · Cited by 711
Leading issues in economic development
(2000) · Choice Reviews Online · doi:10.5860/choice.38-2269 · Cited by 710
Systematic literature studies: database searches vs. backward snowballing
Jalali, S., & Wohlin, C. (2012) · Proceedings of the ACM-IEEE international symposium on Empirical software engineering and measurement · doi:10.1145/2372251.2372257 · Cited by 659 · cited 11×
“Other complementary searches were not done either since the objective is to compare snowballing with a database-firstapproach in a similar way as in [19], although with a better specified procedure for using snowballing.”
BackgroundResultInfluentialSelf-citation
Cited by
Items 3,701–3,800 of 3,934
La medida de la autoeficacia docente: una revisión de alcance
Martín-Martín, M., Muñoz, I. A., & Bueno-Álvarez, J.-A. (2026) · Cadernos de Educação Tecnologia e Sociedade · doi:10.14571/brajets.v19.n1.259-278
Tax Incentives for Small and Medium–Sized Enterprises: A Systematic Literature Review
Lynch, A., Bhaird, C. M. A., & O’Reilly, S. (2026) · Journal of Economic Surveys · doi:10.1111/joes.70101 · Cited by 3
Saliman, S., Abas, H., & Saari, Z. (2026) · Journal of Information System and Technology Management · doi:10.35631/jistm.1142012
Analysis and evolution of dependencies in software ecosystems
Jaime, D. (2025) · theses.fr (ABES) · doi:10.70675/23c6195ezb6f6z4a7dza15ez7c317c646756
Kanaskie, C. R., Hofstetter, R. W., Klepzig, K. D., et al. (2026) · Annals of the Entomological Society of America · doi:10.1093/aesa/saag011
Alluhidan, A., Ma, R., Park, J., Agha, Z., Yin, Z., & Wiśniewski, P. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791092
“We followed the iterative citation search guidelines proposed by Wohlin [166] and screened article titles, publication venues, and author information to assess their alignment with all of our inclusion criteria: for the forward search, we used the “cited by XXX” feature in Google Scholar to…”
An international spatial perspective of upper echelons: A review and research agenda
Carot, I., Andreu, A. B., & García-Granero, A. (2026) · Journal of Management & Organization · doi:10.1017/jmo.2026.10102
Eye-Tracking in Statistical Graphics Research: A Scoping Review
Hvastja, L., Štrumbelj, E., & Žabkar, J. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2649927 · Cited by 1
Salin, H., & Rybarczyk, Y. P. (2026) · Software · doi:10.3390/software5020017 · Cited by 3
“Moreover, from the selected articles that are deemed relevant, we then apply the snowballing technique [45] to further extend the search, following the recommended secondary search strategies.”
Dafonte, K. K., Lutz, P. L., Jansen, C., Strassburg, C. P., Németh, R., Hartmann, G., Jaehde, U., & Coenen, M. (2026) · JMIR Research Protocols · doi:10.2196/89042 · Cited by 1
Liebrand, B., Braunhofer, L., van der Weegen, W., Vissers, K. C. P., Goethem, J. W. V., Zundert, J. H. M. V., van den Boogaart, M., & van der Wal, S. (2026) · European Spine Journal · doi:10.1007/s00586-026-09925-x
Attribute-Based Consent Management System: A Cryptographic Architecture for Data Privacy Compliance
Suntaxi, G., & Galarza, J. (2025) · IEEE CHILEAN Conference on Electrical, Electronics Engineering, Information and Communication Technologies (CHILECON) · doi:10.1109/chilecon66915.2025.11476020
“We use the snowballing approach [21] as a search strategy to identify relevant literature.”
Wright, L., Legarda, I., Lasa, G., & Escallada, O. (2026) · Open Research Europe · doi:10.12688/openreseurope.22034.1 · Cited by 2
Shea of the Savanna Parkland of Northern Ghana
Yussif, K. (2026) · Ghana Studies · doi:10.3368/gs.28.1.136
“…were not left out, the study employed backward snowballing—an approach to identifying sources by using the reference list of already selected literature to identify additional articles that have not already been included but meet the inclusion criteria employed by this research (Wohlin, 2014).”
Anthuvan, T., Prabhuram, S., Sasidharan, S., et al. (2026) · Industry and Higher Education · doi:10.1177/09504222261438614
Computational approaches to controversy detection: a systematic review
Bassi, D., Maggini, M., Vieira, R. L. R., & Pereira-Fariña, M. (2026) · Frontiers in Communication · doi:10.3389/fcomm.2026.1683535
A workload-driven and SLA-aware model for relational to document store schema transformation
Soni, K., & Sachdeva, S. (2026) · Knowledge and Information Systems · doi:10.1007/s10115-026-02759-5
A scoping review of narrative analysis in studies of rural depopulation and land abandonment
Salvia, R., Saulle, M. M. M., & Quaranta, G. (2026) · Discover Sustainability · doi:10.1007/s43621-026-03328-0
“Backwards and forward snowballing were also conducted on the final set of included studies [34].”
User Story Estimation using Natural Language Processing and Deep Learning: A Comparative Study
Silva, D. D. O., Souza, A. C. C., Souza, F. C. M., & Sanches, S. R. R. (2026) · Journal of the Brazilian Computer Society · doi:10.5753/jbcs.2026.5860
How Generative AI is Used in Software Project Management: A Systematic Literature Review
Vito, G. D., Ferrucci, F., & Visentin, L. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-36587-3_19
Θεοδωρόπουλος, Α. (2026) · Virtual Worlds · doi:10.3390/virtualworlds5020020
“Finally, to guarantee exhaustiveness in this rapidly evolving field, backward and forward snowballing techniques were integrated into the search strategy, following the guidelines proposed by Wohlin [25].”
Governing Privacy-Preserving Face Recognition in Transport Infrastructures: A Comprehensive Review
Sanz, E. M. B., Primo, A. G. G., Choudhary, G., & Dragoni, N. (2026) · Sensors · doi:10.3390/s26092832 · Cited by 1
“The search process follows established guidelines for systematic literature studies proposed by Petersen et al. [19], incorporates the snowballing procedure described by Wohlin [20], and is informed by prior systematic surveys in security and privacy research (e.g., [21]).”
Koç, H., & Hynes, J. (2026) · Complex Systems Informatics and Modeling Quarterly · doi:10.7250/csimq.2026-46.02 · Cited by 1
“The procedure was supplemented by forward snowballing to identify additional relevant studies that may not have been captured by database searches alone [41].”
Santos, I. D., Camarotto, J. A., & Tonin, L. A. (2026) · Journal of Air Transportation · doi:10.2514/1.d0523
Ghanmi, H. A., Ahmadjee, S., & Bahsoon, R. (2026) · ACM Transactions on Software Engineering and Methodology · doi:10.1145/3810941 · Cited by 1
“To ensure thoroughness, we conducted a secondary search using snowballing techniques [85] and Google Scholar, adding 6 more studies.”
Malagon, J., & Haelermans, C. (2026) · International Transactions in Operational Research · doi:10.1111/itor.70207
Analysing the Adoption of the Terms-of-Use Field in EBSI Digital Wallets
Bistarelli, S., Luchini, C., & Santini, F. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-16089-8_20
Systematic Literature Review: The Impact Of Football Events On Local Economy And Tourism Sector
Sinurat, R., Louk, M. J. H., Pujiati, A., et al. (2025) · COMPETITOR Jurnal Pendidikan Kepelatihan Olahraga · doi:10.26858/cpjok.v17i2.74
Blended SoS: A New Evolutionary Approach for Complex Information Systems
Lana, C. A., Silva, S. M. D., Affonso, F. J., de Oliveira, J. A., Nakagawa, E. Y., & Vilella, M. L. B. (2026) · Anais do Simpósio Brasileiro de Sistemas de Informação (SBSI) · doi:10.5753/sbsi.2026.248729
“GT concepts [Charmaz 2014, Strauss and Corbin 1998] guided the construction of search strings, while snowballing [Wohlin 2014] ensured comprehensive coverage [Kitchenham et al. 2010].”
Tan, H. W., Arifin, K., & Manaf, A. A. (2026) · Multidisciplinary Reviews · doi:10.31893/multirev.2026521
“From this broad set, 734 articles were removed because they were irrelevant to the specific subject of project management or focused on unrelated fields such as software engineering without a physical component (Wohlin, 2014; Petersen et al., 2008).”
Dubey, C., Singh, A. K., & Dwivedi, V. K. (2026) · Proceedings of the National Academy of Sciences India Section A Physical Sciences · doi:10.1007/s40010-026-01029-x
Koudounas, G., Lomvardeas, O., Kalogirou, T. E., Moulakakis, K. G., Papazoglou, K. O., & Karkos, C. D. (2026) · Angiology · doi:10.1177/00033197261448369 · Cited by 1
Sun, C., Chen, J., Cai, S., Zong, C., Liu, Y., Chen, Z., Chang, X., & Akpaku, E. (2026) · Journal of Software Evolution and Process · doi:10.1002/smr.70119
Balancing Economic Viability and Environmental Impact in Circular Business Transitions
Tleuken, A., Grobben, B., Rogetzer, P. B., & Machado, M. (2026) · Circular Economy and Sustainability · doi:10.1007/s43615-026-00968-2
Pinciroli, F., Antonelli, L., & Bermúdez, Á. A. T. (2026) · Innovations in Systems and Software Engineering · doi:10.1007/s11334-026-00626-8
Glukhikh, M. V., Onishchenko, G. G., Klein, S. D., et al. (2026) · Health Care of the Russian Federation · doi:10.47470/0044-197x-2026-70-2-93-100
Guzmán, Á. M. Z. (2026) · Siembra · doi:10.29166/siembra.v13i1.9707
Identification of Cyber Threats in IoV Communication: A Systematic Literature Review
Ullah, N., Ahmad, I., Almazroi, A. A., & Rahman, M. U. (2026) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-032-23035-5_15
Kuranga, C., & Masenya, T. M. (2026) · Sustainability · doi:10.3390/su18104734
Heifetz, N., Shenhar, G., & Kaim, A. (2026) · Journal of International Humanitarian Action · doi:10.1186/s41018-026-00199-y
“…(reviewing the reference lists of key articles) and forward snowballing (using "Cited by" features) to identify foundational and recent scholarship (Wohlin 2014); and 2) Visual Discovery, using Connected Papers as a visual tool to identify highly influential and interconnected works based on…”
Neves, G. D. S., Moreira, S. B., & Oliveira, S. (2026) · Intelligent systems reference library · doi:10.1007/978-3-032-24395-9_6
Muhammad, N., Farooq, R., Jadoon, K. Z., & Pasha, G. A. (2026) · Water Resources Management · doi:10.1007/s11269-026-04531-4
“This process adhered to the guidelines established by (Wohlin 2014), encompassing both backward (reviewing refer - ence lists) and forward (reviewing citations) screening.”
A Non-Functional Quality Framework for Blockchain Software: ISO/IEC 25010 Revisited
Al-Humaimeedy, A. S., HabibAllah, S., & Alhamadi, R. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-22208-4_31
Braga, C., Schulte, J., & Watz, M. S. (2026) · Journal of Industrial Ecology · doi:10.1007/s44498-026-00099-9
“Due to the breadth of research in sustainable product development and adjacent fields, as well as the multitude of terms and concepts that are used, backward snowballing (Wohlin, 2014) was used to identify additional articles that were not captured by the initial search strings.”
Yang, Z., Zhang, L., Yu, H., Li, J. Q., & Lian, X. (2026) · IEEE Transactions on Parallel and Distributed Systems · doi:10.1109/tpds.2026.3694097 · Cited by 1
“To address this issue, we adopt the snowballing procedure proposed by Claes [20].”
IT Competencies in a Dynamic Job Market: A Review
Sangole, R. C., & Desai, D. (2026) · ANVESHAK-International Journal of Management · doi:10.15410/aijm/2026/v15i1/173283
Joshi, K. K., & Shrestha, S. B. (2026) · Springer eBooks · doi:10.1007/978-981-95-7785-9_6 · Cited by 1
Bojan, A., & Achim, M. V. (2026) · Contributions to finance and accounting · doi:10.1007/978-3-032-20198-0_9
Analyzing Empirical Findings on User Reliance Behaviors in XAI-Assisted Decision-Making
Filho, J. C. D. S., Belloum, R., & de Oliveira, K. M. (2026) · Lecture notes in business information processing · doi:10.1007/978-3-032-26836-5_3
A Systematic Mapping Study on Developer Profile Modeling in Software Engineering
Sarı, A., Alptekin, G. I., & Tosun, A. (2026) · IEEE Access · doi:10.1109/access.2026.3696198
Jakab, M. (2026) · Human Arenas · doi:10.1007/s42087-026-00647-6
“In order to procure the starting keywords for the main review, a reverse snowball research (Otte-Trojel & Wong, 2016; Wohlin, 2014; Wohlin et al., 2022 ) has been con - ducted.”
Perspectives on Routing in LEO/NTN Space Networks: Classical, Agentic, and Quantum
Mattar, C., Abdo, J. B., Makhoul, A., et al. (2026) · doi:10.1109/menacomm69507.2026.11532878
Toma, A. I., Dima, V., Zaharie, G. C., Necula, A. I., Stoiciu, R. P., & Bivoleanu, A. R. (2026) · Medicina · doi:10.3390/medicina62061052
“…include the following: (i) the scoping rather than fully systematic design, with inherent selection bias despite the structured search [11,12]; (ii) the inclusion of one paper from our own group [52], which we have declared transparently; (iii) the exclusion of imaging modalities, whose…”
InfluentialAguilar, P., Iñiguez-Jarrín, C., & Sandobalín, J. (2026) · doi:10.5753/cibse.2026.42442
“This phase applies backward snowballing, whose steps are defined in [Wohlin 2014], to identify additional works aligned with the objective of the review.”
Ye, R., & Yu, C. (2026) · International Journal of Electrical Power & Energy Systems · doi:10.1016/j.ijepes.2026.111954
Animal-assisted treatment in spinal cord injury rehabilitation: a scoping review
Williamson, S. D., Aaby, A., Ejersbo, A. O., Glintborg, C., & Ravn, S. L. (2026) · Spinal Cord · doi:10.1038/s41393-026-01223-z
“Secondly, all included studies in addition to content-relevant studies excluded on account of their format underwent forward and backward snowballing by SDW [24].”
Ghari, S., Fu, P., Abhaionmhen, A. O., Mago, V., & Belle, A. B. (2026) · doi:10.2196/preprints.103004
Ghayoumi, M., Ghazinour, K., Marrero, A., Barmas, D., Cook, C., May, M., Liu, C., Johnson, B., & Fofana, A. (2026) · Electronics · doi:10.3390/electronics15112421 · Cited by 2
Ivanova, R., & Mochkabadi, K. (2026) · Journal of Business Economics · doi:10.1007/s11573-026-01267-w
“Programs: We mapped 54 WEEPs in the DACH region (Germany, Austria, and Switzerland) (2020–2024) via web searches, network mapping, and snowballing (Wohlin 2014 ).”
Design for Community Building in Villages
Zhang, C. (2026) · doi:10.1145/3789492.3796443
Addressing data scarcity with Few-Shot Learning: a systematic literature review
Lamtaai, I., Jilali, B. B., Chairi, I., & Lyhyaoui, A. (2026) · Artificial Intelligence Review · doi:10.1007/s10462-026-11584-9 · Cited by 1
Domingue, M., Lévesque, S., Burgun, A., Éthier, J., & Lemyre, F. C. (2026) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2026.106535
“More specifically, we conducted snowballing keywords search [31] among included articles from this previous review and validated that these keywords could be transferred to the vertically partitioned setting using papers already within our knowledge.”
An Integrated Decision Support Framework for Multi-Phase Flood Event Management
Vidmar, V., & Borštnar, M. K. (2026) · doi:10.18690/um.fov.4.2026.81
Context-Aware Recommendations for the Prevention of Urban Arbovirus Outbreaks
Arouca, M. G., & Dur�ão, F. A. (2026) · Simpósio Brasileiro de Sistemas Colaborativos (SBSC) · doi:10.5753/sbsc.2026.20270
“To ensure the breadth and relevance of the results, the Snowballing literature review technique was adopted [Wohlin 2014].”
How do universities support the business model innovation of small and medium-sized enterprises?
Arzabe, F. A. (2026) · Industry and Higher Education · doi:10.1177/09504222261454528
Goncalves, L., Delgado, L. M., Oliveira, R. E. N. D. N., et al. (2026) · World Journal for Pediatric and Congenital Heart Surgery · doi:10.1177/21501351261449205
A systematic literature review of soil ecosystem services in the East and Southern African regions
Nyamari, N., Kramm, T., Otieno, D. O., Biber‐Freudenberger, L., & Bogner, C. (2026) · Frontiers in Soil Science · doi:10.3389/fsoil.2026.1765141
Group Conversational Agents: A Review of Designs that Support and Shape Group Interaction
Yeo, S., Zhang, T., Bateman, S., Hsieh, G., Kim, Y., Perrault, S. T., Li, J., & Tang, A. (2026) · Designing Interactive Systems Conference · doi:10.1145/3800645.3812934 · Cited by 3
“To expand coverage, we conducted backward and forward snow-balling using Google Scholar, following established guidelines for systematic reviews [97].”
Saifullah, K. M., Rahman, M., López, M. B., et al. (2026) · Communications in computer and information science · doi:10.1007/978-3-032-28812-7_15
Murata, W., Yamamoto, G., Liu, C., Mori, Y., Kishimoto, K., & Kuroda, T. (2026) · Interactive Journal of Medical Research · doi:10.2196/preprints.100914
Task Interdependence in Software Teams: A Systematic Review Anchored in Kiggundu’s Theory
Souza, M. B. D. B., & da Silva, F. Q. B. (2026) · Journal of Software Engineering Research and Development · doi:10.5753/jserd.2026.6507
“Rather than relying on traditional keyword-based database queries, we used forward snowballing (Wohlin, 2014) in order to trace how Kiggundu’s theoretical framework has been taken up, operationalized, and extended across Software Engineering and other team-based knowledge-work domains.”
InfluentialDeep Learning for Operation and Maintenance of Urban Lifeline Systems: A Comprehensive Review
Yin, X., Zhao, J., & Zheng, R. (2026) · Journal of Infrastructure Systems · doi:10.1061/jitse4.iseng-2916
A Qualitative Study on XAI Techniques for Software Defect Prediction
Roy, S., Roy, B., Roy, C. K., & Khomh, F. (2026) · Information and Software Technology · doi:10.1016/j.infsof.2026.108234 · Cited by 4
A multi-actor perspective on barriers to sustainable innovation in the asphalt paving sector
Ruiz, A., Santos, J., Kruijf, J. V., Volker, L., & Dorée, A. G. (2026) · Construction Management and Economics · doi:10.1080/01446193.2026.2662339
A scoping review of interventions to address adverse social behavior in human service organizations
Forssell, R., Neall, A. M., & Yan, Z. (2026) · International Journal of Workplace Health Management · doi:10.1108/ijwhm-01-2024-0012
Reliance-Aware Design of Interactive Explainable User Interfaces
Filho, J. C. D. S., Belloum, R., Monteiro, I. T., & de Oliveira, K. M. (2026) · ACM Conference on Fairness, Accountability, and Transparency (FAccT) · doi:10.1145/3805689.3812231 · Cited by 2
“To broaden the search, the authors applied snowballing procedures [75], including 10 papers from backward snowballing and another 32 from forward snowballing.”
Opportunities and Challenges of Explainable AI (XAI) in Health Care: A Review
Begum, M., & Ferdush, J. (2026) · Journal of Computer and Communications · doi:10.4236/jcc.2026.146006 · Cited by 1
RT: resilient transformation for anomaly detection in heteroscedastic time series
Baroni, L. R., Mello, C. E., Mendes, E., Pacitti, E., Porto, F. A. M., Guedes, G., Coutinho, R. D. C., & Ogasawara, E. (2026) · International Journal of Data Science and Analytics · doi:10.1007/s41060-026-01188-5
Coutinho, M. L., Delgado, L. M., D’Oria, M., et al. (2026) · Annals of Vascular Surgery · doi:10.1016/j.avsg.2026.06.025
Schulte, J. (2026) · Environmental Education Research · doi:10.1080/13504622.2026.2693941 · Cited by 2
“Backward snowballing (Wohlin 2014) was then conducted from the reference lists of eligible studies to identify additional records that met the same criteria, with screening and full-text assessment applied in the same way to these items.”
An assessment of rhodolith bed ecosystem services
Rasmusson, L. M., Eggertsen, M., & Hall-Spencer, J. M. (2026) · Annals of Botany · doi:10.1093/aob/mcag184 · Cited by 1
Human-AI hybrid rationality in decision-making: an integrative literature review
Sbai, A., & Berjaoui, A. (2026) · AI and Ethics · doi:10.1007/s43681-026-01234-9
Quayson, F., & Dadson, I. Y. (2026) · Research Square · doi:10.21203/rs.3.rs-8913236/v2
Dias, A., & Cainé, L. (2026) · Journal of Forensic and Legal Medicine · doi:10.1016/j.jflm.2026.103200 · Cited by 3
A Systematic Mapping Study of Container-Based Virtualization Technologies
Arias-Pinzón, J. A., Correa-Urbano, A. H., Rodríguez, L. E. S., & Uribe, C. A. C. (2026) · Journal of Internet Services and Applications · doi:10.5753/jisa.2026.6589
“…comprised two phases: Baseline Construction , which established a seed set from the database results using the CVI, SCI, and direct inclusion criteria; and Study Selection , which extended coverage through backward (reference) and forward (citation) analysis of each seed article Wohlin (2014).”
Ankle-foot orthosis mechanical testing: a scoping review
Vaiciulis, K., Williston, Z., & Totah, D. (2026) · Journal of NeuroEngineering and Rehabilitation · doi:10.1186/s12984-026-02072-w
“This article presents a scoping review using a data-base search following the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) guidelines [40] and a snowballing search [41].”
Towards Secure Logging: Characterizing and Benchmarking Logging Code Security Issues with LLMs
Yuan, H. Y., Wang, X., Yao, K., Chen, A. R., Ding, Z., & Li, Z. (2026) · Proceedings of the ACM on software engineering. · doi:10.1145/3797129 · Cited by 3
Swipe Gesture-Based Continuous Mobile Device Authentication: a Systematic Literature Review
Naji, Z., & Bouzidi, D. (2026) · SN Computer Science · doi:10.1007/s42979-026-05090-6
“For the secondary approach, we have adopted the snow-balling technique [ 12, 13 ], which comprises two essential stages: ● Backward snowballing: We scrutinized the bibliograph - ic details of the studies already chosen, aiming to identify additional relevant reviews.”
An Operating-System Infrastructure for Embedded BDI-Based Multi-agent Systems
Lazarin, N., Pantoja, C. E., & Viterbo, J. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-30494-0_37
Human–AI teams for decision-making in banking and finance: A systematic scoping review
Aksu, F., Morandini, S., & Pietrantoni, L. (2026) · Human Systems Management · doi:10.1177/01672533261461402
A concept for AI supported knowledge extraction in design for additive manufacturing
Schmitt, P., Zorn, S., & Gericke, K. (2026) · Proceedings of the Design Society · doi:10.1017/pds.2026.10567 · Cited by 3
“” Both backward snowballing and forward snowballing (described in (Wohlin, 2014)) was carried out for relevant articles.”
Project complexity and cost escalations in the early design of railway megaprojects
Schön, P. P., Eckert, C. M., Mallalieu, A., et al. (2026) · Proceedings of the Design Society · doi:10.1017/pds.2026.10642
“To find a relevant project complexity framework, a snowballing method (Wohlin, 2014) was applied, starting with 17 different frameworks identified by Bhangale and Gupta (2025).”
López-Chacón, E., Pérez-Arriaga, J. C., Sánchez-García, Á. J., & Hernández-González, L. A. (2026) · Tecnura · doi:10.14483/22487638.24865
Language poverty alleviation in the Chinese mainland: a language policy and planning perspective
Chen, Y., Ma, R., & Peng, L. (2026) · Current Issues in Language Planning · doi:10.1080/14664208.2026.2696111
Kazemloo, M., Rad, F. N., Goudarzimehr, L., et al. (2026) · International Journal of Emergency Medicine · doi:10.1186/s12245-026-01283-1
“In addition, supplementary searching was performed through Google Scholar and backward snowballing of the reference lists of eligible studies and relevant reviews, according to established snowballing guidance [21].”
Empirical Studies on UML Sequence Diagrams in Education: A Systematic Literature Review
Alhazmi, S. (2026) · ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE) · doi:10.1145/3803400.3809381
“In step 5, we complemented the SLR using a backward snowballing procedure, following the approach described in [37], by examining the reference lists of the 19 studies.”
Lemos, M. J., Visentini, L. M., Queiroz, L. F., & Peloggia, A. (2026) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2026.115287
Testing AIware Systems: A Software Engineering Survey
Gonzalez, K., & Mezouar, M. E. (2026) · ACM International Conference on AI-Powered Software (AIware) · doi:10.1145/3805760.3814894
Metadata
- Authors
- Wohlin, C.
- Affiliations
- Blekinge Institute of Technology (SE)
- Corresponding author
- Wohlin, Claes
- ORCID
- Wohlin, Claes
- Venue
- Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering
- Publisher
- ACM
- Published
- 2014-05-13
- Pages
- 1-10
- OpenAlex
- W1975675278
- Cited by
- 4,234
- References
- 31
- Citations by year
- 2026: 433 · 2025: 552 · 2024: 651 · 2023: 594 · 2022: 566 · 2021: 477 · 2020: 356 · 2019: 250 · 2018: 175 · 2017: 98 · 2016: 60 · 2015: 17 · 2014: 4
- FWCI
- 50.91
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Evaluation and Assessment in Software Engineering (EASE) (publishedVersion)
- Funders
- Lunds Universitet
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- snowballing, systematic reviews, research guidelines, software engineering literature, replication study, literature search strategies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.