Guidelines for snowballing in systematic literature studies and a replication in software engineering
The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index; 12 of the 23 sources its claims cite were checked for support.
Abstract
Background: Systematic literature studies have become common in software engineering, and hence it is important to understand how to conduct them efficiently and reliably.
Overview
CoChat Summary
TL;DR
The paper proposes a detailed snowballing procedure for systematic literature studies in software engineering and evaluates it by replicating a published systematic literature review. It concludes that snowballing can be a viable first search strategy and, in this case, found the same set of studies as the prior reviews.
Key findings
- The replication began with a tentative start set of 13 candidates; 3 were included in the start set.
- Across the study, 38 papers were evaluated and 11 papers were included in the final set.
- Efficiency varied sharply by step: start set 3/13 = 23%, iteration 1 7/25 = 28%, iteration 2 1/223 = 0.4%, and iteration 3 0%; overall efficiency was 3.7%.
- The paper states that the replication produced the same papers as the prior systematic literature reviews, and that 9 studies were in common with both reviews in [11].
Methods
- The paper develops a snowballing procedure based on experience from several systematic literature reviews and experimenting with different approaches; it specifies backward and forward snowballing, inclusion/exclusion handling, authors, and data extraction.
- For the replication, the author searched Google Scholar using the string “cross-company within-company software effort estimation” with time frame 1995–2005.
- The replication examined candidate papers through reference lists and citations, using title/publication-year screening, then abstracts, then full texts when needed.
Limitations
- The paper states a main validity threat: the researcher had read the original study 6–12 months earlier, which may have affected inclusion/exclusion decisions.
- The author notes that the identified start set was far from perfect, because the three start-set papers had one author in common, which could miss papers not linked to those papers.
- Evidence of limitations evident in the text: the efficiency results are based on a single replication case in one topic area, so generalizability beyond this case is not established (evident from the study design).
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
10 Valid · 3 Not valid · 11 Not checked · 131 Own results and interpretations
At least one cited source doesn’t support this claim or points to a different paper, and none confirmed it — see the verdicts below.
In the paper
“The same papers are identified.”
What the sources say
- Disputedcites [11] [11] MacDonell, S., Shepperd, M., Kitchenham, B. A. and Mendes, E. 2010. How reliable are systematic reviews in empirical software engineering?. IEEE Transactions on Software Engineering 36, 5, 676-687. - sourceExperimental — accuracy not yet measured
The Q1.6 ‘Papers included’ section and the text following Table 1 say the two reviews selected ten primary studies each, but only nine were common to both. The paper describes citation analysis within those reviews, not a snowballing replication that identified exactly the same papers. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
Every source the paper cites for this claim was checked and supports it.
In the paper
“MacDonell et al. evaluated the reliability of systematic reviews through comparing the results of two studies with a common research question performed by two independent groups of researchers.”
What the sources say
- Supportedcites [11] [11] MacDonell, S., Shepperd, M., Kitchenham, B. A. and Mendes, E. 2010. How reliable are systematic reviews in empirical software engineering?. IEEE Transactions on Software Engineering 36, 5, 676-687. - source
The Introduction says the paper assesses systematic-review reliability by comparing two independent reviews addressing the same research question. Section 3.1 specifies that two teams undertook those reviews independently, and the Conclusions report that the teams' outcomes were compared. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“Nine studies are in common with both previous systematic literature reviews presented in [11].”
What the sources say
- Supportedcites [11] [11] MacDonell, S., Shepperd, M., Kitchenham, B. A. and Mendes, E. 2010. How reliable are systematic reviews in empirical software engineering?. IEEE Transactions on Software Engineering 36, 5, 676-687. - source
The passage following Table 1 states that both systematic literature reviews selected 10 primary studies, with nine in common. This directly supports the citing sentence. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“If using the papers studied in detail as a efficiency measure, then the different approaches requires about the same effort.”
What the sources say
- Supportedcites [11] [11] MacDonell, S., Shepperd, M., Kitchenham, B. A. and Mendes, E. 2010. How reliable are systematic reviews in empirical software engineering?. IEEE Transactions on Software Engineering 36, 5, 676-687. - source
The Table 1 discussion says both reviews selected 10 primary studies, and §4.2 describes their data-extraction work on those studies. Thus, if the number of papers studied in detail is used as the efficiency measure, the two approaches are comparable on that measure; this does not establish equal time spent. (LLM, conf=0.85)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The guidelines for using snowballing as a way to search for relevant literature was successfully applied to a systematic literature review.”
plainstates a causeSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
20 more claims
Create a free account to see all 155 claims and every source verdict.
- Probably not a substantial claim. 111
- It may state two separate claims as one. 60
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“we applied forward and backward snowballing based on the guidelines by Wohlin et al. [26]”
From A Multivocal Review of MLOps Practices, Challenges and Open Issues (2026) · cites this paper as Wohlin et al. [26]
The cited source says what the claim attributes to it.
In the citing paper
“As a guideline, this study was performed following well-known principles of systematic studies: References [193, 196, 207, 220].”
From Quantum Combinatorial Optimization in the NISQ Era: A Systematic Mapping Study (2024) · cites this paper as [220]
The cited source says what the claim attributes to it.
In the citing paper
“To minimize the variables that could compromise our study, we (i) conducted our study following accepted guidelines of systematic studies [181, 193, 207, 220], (ii) carefully designed our study in advance, and (iii) assessed, validated, and discussed the potential threats in each phase of the study.”
From Quantum Combinatorial Optimization in the NISQ Era: A Systematic Mapping Study (2024) · cites this paper as [220]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“HDFS logs describe operations on the storage block pool [22].”
From Practitioners’ expectations on log anomaly detection (2025) · cites this paper as [22]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“only 4 papers [19], [20], [22], [36] evaluate their techniques across different domains, training on a source domain and testing on a different target domain.”
From Practitioners’ expectations on log anomaly detection (2025) · cites this paper as [22]
Experimental — accuracy not yet measured
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Analyzing the Past to Prepare for the Future: Writing a Literature Review
Webster, J., & Watson, R. T. (2002) · MIS Quarterly · doi:10.2307/4132319 · Cited by 7,449 · cited 7×
“Similar ideas have been brought into information systems research, e.g. by Webster and Watson [2].”
BackgroundMethodInfluentialSystematic Literature Reviews in Software Engineering—A Systematic Literature Review
Kitchenham, B., Pearl Brereton, O., Budgen, D., Turner, M., Bailey, J., & Linkman, S. (2009) · Information and Software Technology · doi:10.1016/j.infsof.2008.09.009 · Cited by 4,537 · cited 2×
“[13, 14] gives the impression that automation is better than manual search.”
BackgroundSystematic Mapping Studies in Software Engineering
Petersen, K., Feldt, R., Mujtaba, S., & Mattsson, M. (2008) · Electronic Workshops in Computing · doi:10.14236/ewic/ease2008.8 · Cited by 3,239 · cited 1×
“Systematic mapping studies have been highlighted as a complement to systematic literature reviews [8].”
BackgroundGreenhalgh, T., & Peacock, R. (2005) · BMJ · doi:10.1136/bmj.38636.593461.68 · Cited by 2,045 · cited 4×
“Greenhalgh and Peacock [15] applied three different search methods in their research, and concluded that protocol-driven search approaches by themselves are not necessarily the most efficient method regardless of the number of databases used, since some sources may be found through personal knowledge/contacts (e.”
BackgroundResultInfluentialSystematic literature reviews in software engineering – A tertiary study
Kitchenham, B., Pretorius, R., Budgen, D., Pearl Brereton, O., Turner, M., Niazi, M., & Linkman, S. (2010) · Information and Software Technology · doi:10.1016/j.infsof.2010.03.006 · Cited by 1,230 · cited 2×
“[13, 14] gives the impression that automation is better than manual search.”
BackgroundMethodGLOBAL ECONOMIC PROSPECTS: THE RECESSION MAY BE OVER BUT WHERE NEXT?
(2013) · The Financial Crisis and the Global South · doi:10.2307/j.ctt183pb3w.5 · Cited by 908
Building knowledge through families of experiments
Basili, V. R., Shull, F., & Lanubile, F. (1999) · IEEE Transactions on Software Engineering · doi:10.1109/32.799939 · Cited by 852
Rodrik, D., & Velasco, A. (1999) · National Bureau of Economic Research · doi:10.3386/w7364 · Cited by 711
Leading issues in economic development
(2000) · Choice Reviews Online · doi:10.5860/choice.38-2269 · Cited by 710
Systematic literature studies: database searches vs. backward snowballing
Jalali, S., & Wohlin, C. (2012) · Proceedings of the ACM-IEEE international symposium on Empirical software engineering and measurement · doi:10.1145/2372251.2372257 · Cited by 659 · cited 11×
“Other complementary searches were not done either since the objective is to compare snowballing with a database-firstapproach in a similar way as in [19], although with a better specified procedure for using snowballing.”
BackgroundResultInfluentialSelf-citation
Cited by
Items 3,201–3,300 of 3,934
de Almeida, L. T., Roque, L. G., Menegassi, C. H. M., & Silva, M. J. (2020) · Editora Poisson eBooks · doi:10.36229/978-85-7042-210-1.cap.19
Gestão da qualidade total na enfermagem em um Hospital Universitário no estado do Rio de Janeiro
Barreto, V. P. M., Braga, F. A. C. D. O., & de Moura, C. F. S. (2020) · Editora Poisson eBooks · doi:10.36229/978-85-7042-210-1.cap.02
Framework Scrum como instrumento de integração de equipes
Bernardo, C. D. P., Silva, R., de Souza, S. B., Barbosa, R. A. P., & Martins, A. V. S. (2020) · Editora Poisson eBooks · doi:10.36229/978-85-7042-210-1.cap.06
Orçamentação paramétrica de poços de petróleo: Uma revisão sobre métodos propostos
Salvalagio, J. S., Accioly, R. M. E. S., Neves, M. M., Lima, G. B. A., & Gavião, L. O. (2020) · Editora Poisson eBooks · doi:10.36229/978-85-7042-210-1.cap.09
Neto, L. E., Pereira, M. S., Teodoro, E. D. C. S., Fuini, L. L., & Tomas, R. N. (2020) · Editora Poisson eBooks · doi:10.36229/978-85-7042-210-1.cap.20
Turnover: Análise no contexto do setor supermercadista
Pazolini, D., Krüger, S. D., Zanella, C., & Bagatini, F. M. (2020) · Editora Poisson eBooks · doi:10.36229/978-85-7042-210-1.cap.03
Liderança positiva: Uma definição polivalente a partir da revisão da literatura
Moura, A. F. A., & de Andrade, E. P. (2020) · Editora Poisson eBooks · doi:10.36229/978-85-7042-210-1.cap.04
Cutovoi, I. T. M. (2020) · Editora Poisson eBooks · doi:10.36229/978-85-7042-210-1.cap.16
Cutovoi, I. T. M. (2020) · Editora Poisson eBooks · doi:10.36229/978-85-7042-210-1.cap.17
Cortez, M. C. A. F., da Silva, D. P., Cortez, M. A. F., & Araújo, A. B. D. S. (2020) · Editora Poisson eBooks · doi:10.36229/978-85-7042-210-1.cap.11
Gusmão, M. A., & Domingos, M. D. L. C. (2020) · Editora Poisson eBooks · doi:10.36229/978-85-7042-210-1.cap.18
Artola, V. (2020) · El Servicio de Difusión de la Creación Intelectual (National University of La Plata) · doi:10.35537/10915/98135
Food in the occupational environment and its benefits in worker’s health
Pereira, P. M., Duarte, J., Pinho, O., Baptista, J. S., Ferraz, J., & de Santana, A. P. (2019) · Symposium on Occupational Safety and Health (SOSH) · doi:10.24840/978-972-752-260-6_0061-0067
Interactive Privacy Preferences Management for Shared Spaces in Internet of Things.Preprint
Muhander, B. A., Wiese, J., RANA, O. F., & Perera, C. (2020) · arXiv (Cornell University)
The Roles of Robots in our Future Society
Zuurbier, Q. L. (2020) · Utrecht University Repository (Utrecht University)
Disciplined or Agile? Two Approaches for Handling Requirement Change Management
Albuquerque, D. W., Guimarães, E. T., Ramos, F., Costa, A., Gomes, A., Dantas, E., Perkusich, M., & de Almeida, H. O. (2020) · Advances in systems analysis, software engineering, and high performance computing book series · doi:10.4018/978-1-7998-4165-4.ch007
Alves, R. C., Frota, Y. A. R., & de Oliveira, D. (2020) · doi:10.5753/bresci.2020.11187
“Seguimos as diretrizes recomendadas 1https://openprovenance.org/store/ por [Wohlin 2014] para definir o conjunto de sementes do processo de snowballing (i.e., usamos o Google Scholar para obter o conjunto de sementes em vez de escolher um repositório especı́fico como IEEEXPlore ou ACM Digital…”
BackgroundOrtega, S. E. A. (2020) · Utrecht University Repository (Utrecht University)
A systematic mapping study on Quality of Service in industrial cloud computing
Latifaj, M. (2020) · DiVA (Mälardalen University College)
Automatic Support for Multi-Domain Model Management
Torres, W., van den Brand, M., & Serebrenik, A. (2020) · IEEE International Conference on Software Maintenance (ICSM) · doi:10.1109/icsme46990.2020.00104
“…in cross-domain model management [6] (an extended version of this paper is currently under review). a) Methodology: In this systematic literature review, we followed strict guidelines proposed by Kitchenham and Charters [28], we used Google Scholar as the search engine [29], [30], [31], [32].”
MethodA Natural Language Interface for Querying Linked Data
Akrin, C., & Tham, S. (2020) · Diva portal (Dalarna University Library)
Detecting Medical Misinformation on Social Media Using Multimodal Deep LearningPreprint
Wang, Z., Yin, Z., & Argyris, Y. A. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2012.13968
“For the data collection, we adopted a snowballing method widely used for systematic literature reviews [51].”
MethodWork Systems in the Context of Digital Work.
Goltz, K. (2020)
Ferramentas Visuais para o Ensino de Machine Learning na Educação Básica
von Wangenheim, C. G., Hauck, J. C. R., Pacheco, F. S., & Bueno, M. F. B. (2021) · RENOTE · doi:10.22456/1679-1916.110291
“A fim de minimizar ainda mais o risco de omissão, também incluímos literatura via backward e forward snowballing e literatura secundária como trabalhos acadêmicos referentes às ferramentas encontradas para obter maiores informações (Wohlin, 2014).”
BackgroundModelo de proceso para proyectos de explotación de información
Martins, S. (2020) · El Servicio de Difusión de la Creación Intelectual (National University of La Plata) · doi:10.35537/10915/111195
Martins, W. D. S. (2020) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/d.55.2020.tde-20012021-121741
Sustainability assessment of maintenance based on concrete – a systematic review
Vogel, M., & Guenther, E. (2019) · Report · doi:10.2749/guimaraes.2019.1600
Definição de modelo de processos para a operação do PSS de uma startup
Souza, C. Q. (2019) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/d.18.2019.tde-23052020-140752
Feature Interaction in Smart EnvironmentsPreprint
de Jesus, L. E. N., Claro, D. B., & Rios, T. N. (2021) · Research Square · doi:10.21203/rs.3.rs-398452/v1
“The snowballing [58], a manual search on the references of relevant studies, was conducted based on 36 pre-filtered publications (33 from the automatic search and 3 from the manual search).”
MethodKopřiva, J., & Tetřevová, L. (2021) · Hradec Economic Days ... · doi:10.36689/uhk/hed/2021-01-037 · Cited by 1
Understanding Sensemaking in Undergraduate Science Classrooms
Fang-fang, Z. (2020) · University of Minnesota Digital Conservancy (University of Minnesota)
Concepts and Models of Environment in Self-Adaptive Systems: A Systematic Literature ReviewPreprint
Shin, Y.-J., Bae, J.-Y., & Bae, D. (2021) · Preprints.org · doi:10.20944/preprints202104.0648.v1
Vanderbeken, Y. (2021) · Advances in information security, privacy, and ethics book series · doi:10.4018/978-1-7998-7367-9.ch001
Shepherd, J., Lourida, I., & Meertens, R. M. (2021) · Research Square · doi:10.21203/rs.3.rs-495358/v1
Tikka, S., Hakanen, J., Saarela, M., & Karvanen, J. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2008.13558
“The remaining studies in Table 5 were identified with a Google Scholar search with the same keywords specified above and by using the snowballing technique (Wohlin, 2014) among all already identified articles.”
Tuncel, D. (2021) · doi:10.1109/icssp-icgse52873.2021.00020
“We have expanded our initial set of results by means of formally described snowballing procedures [21].”
MethodConversational Agents in Software Engineering: Survey, Taxonomy and ChallengesPreprint
Motger, Q., Franch, X., & Marco, J. (2021) · arXiv (Cornell University) · Cited by 1
Malikova, A., & Succi, G. (2021) · International Conference on Software Technologies (ICSOFT) · doi:10.5220/0010576302530260
“…related works and systematize the findings (Siddaway et al., 2019), including • Forward and backward snowballing for the search process of papers (Wohlin, 2014); • Qualitative research approach for collecting the data from the interviews (Bolderston, 2012); As a methodology, we chose…”
BackgroundMethodInfluentialSystematic Literature Review of Validation Methods for AI SystemsPreprint
Myllyaho, L., Raatikainen, M., Männistö, T., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2107.12190
“Exclude search hits already found in previous databases Automatic search was chosen over manual search and snow-balling [20] due to the wide range of domains in which AI can be of use.”
BackgroundThe acceptance of virtual reality exposure therapy by mental healthcare patients: A scoping review
Spelt, N. (2021) · Cited by 1
Towards a Theory on Architecting for Continuous DeploymentPreprint
de França, B. B. N., dos Santos, P. S. M., & Matalonga, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2108.09571 · Cited by 3
“Following the guidelines by [54], we reviewed the suitability of these 12 sources to act as seeds.”
MethodInfluentialDaniakin, K. (2021) · ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) · doi:10.1145/3468264.3473488
“Additionally snowballing was used [44].”
Sustainability certification for deforestation-free supply chains: the cases of palm oil and soy
Hinkes, C. (2021) · eDiss (Georg-August-Universität Göttingen) · doi:10.53846/goediss-8489 · Cited by 4
Short review on occupational noise exposure in the extractive industry and similar works
Duarte, J., Branco, J. C., Rodrigues, F., et al. (2021) · doi:10.24840/978-972-752-279-8_0015-0020
Feizi, F. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3926231
de Lima, L. F., & Peres, L. M. (2021) · doi:10.5753/opensciense.2021.17138
“Esta amostra pode ser complementada com o processo de snowballing [Wohlin 2014], que se trata de uma busca opcional nas referências e/ou citações dos artigos selecionados.”
BackgroundVisualization Methods of 3D Plant Models: A Systematic Mapping Study
Di, S., Cao, L., & Ge, Y. (2021) · Journal of Electrical and Computer Engineering · doi:10.1155/2021/2754343 · Cited by 1
“After the second round of screening, we retained 42 papers and identified 18 papers through reference tracking [3].”
BackgroundMethodda Silveira, A. C. M., da Silva, L. D., & Sobrinho, Á. (2021) · doi:10.5753/sbcas.2021.16098
DevOps Adoption: Eight Emergent PerspectivesPreprint
Pedra, M. L., da Silva, M. F., & Azevedo, L. G. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2109.09601 · Cited by 10
“Initially, it was necessary to conduct preliminary bibliographic research of the literature through snowballing method Wohlin [2014], i.”
MethodDrug intelligence and access to medicine
Muscat, C. (2020)
Oppimisanalytiikka ja eettiset kysymykset: Laadullinen meta-analyysi
Silvola, A., Gedrimiene, E., Pursiainen, J., Rusanen, J., & Muukkonen, H. (2022) · Kasvatus · doi:10.33348/kvt.111448 · Cited by 7
A Taxonomy of Assets for the Development of Software-Intensive Products and ServicesPreprint
Zabardast, E., González‐Huerta, J., Gorschek, T., et al. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4264285
Mooney, V. (2021) · Journal of Investigative Psychology and Offender Profiling · doi:10.1002/jip.1580 · Cited by 7
Comparing Money and Time Donation: What Do Experiments Tell Us?
He, T. (2021) · Marketing of Scientific and Research Organizations · doi:10.2478/minib-2021-0015
“For snowball sampling, it is important to construct a good start set that is neither too small nor too large, and covers different publishers, years, authors and communities (Wohlin, 2014).”
BackgroundMethodInfluentialGalvão, G. D. A. (2021) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/t.3.2021.tde-07102021-160020
Watanabe, C. (1955) · Hokkaido University Collection of Scholarly and Academic Papers (Hokkaido University) · doi:10.5281/zenodo.23943
Using Open Information Extraction to Extract Relations: An Extended Systematic Mapping
dos Santos, V., Silva, P. R., de Souza, É. F., Felizardo, K. R., Watanabe, W. M., Cândido, A., Meinerz, G. V., Aluísio, S. M., & Vijaykumar, N. L. (2021) · Latin American Computer Conference (CLEI) · doi:10.1109/clei53233.2021.9639968
Mourão, É., Dias, M., Pinheiro, E., Viterbo, J., & Maciel, C. D. O. (2021) · Anais do ... Simpósio Brasileiro de Informática na Educação/Anais do Simpósio Brasileiro de Informática na Educação · doi:10.5753/sbie.2021.218388 · Cited by 6
Disciplined or Agile? Two Approaches for Handling Requirement Change Management
Albuquerque, D. W., Guimarães, E. T., Ramos, F., Costa, A., Gomes, A., Dantas, E., Perkusich, M., & de Almeida, H. O. (2021) · IGI Global eBooks · doi:10.4018/978-1-6684-3702-5.ch094
Fiedler, F., Bergweiler, G., & Burggräf, P. (2021) · doi:10.1109/ieem50564.2021.9672883
Passive Social Sensing with Smartphone: A Systematic Review (Preprint)Preprint
Zhang, H., Ibrahim, A., Parsia, B., Poliakoff, E., & Harper, S. (2020) · doi:10.2196/preprints.20539
Peer Review #1 of "Microservice security: a systematic literature review (v0.1)"
(2022) · PeerJ Computer Science · doi:10.7287/peerj-cs.779v0.1/reviews/1
Peer Review #3 of "Microservice security: a systematic literature review (v0.2)"
(2022) · PeerJ Computer Science · doi:10.7287/peerj-cs.779v0.2/reviews/3
Formação Permanente Mediada por Tecnologias na Atenção Primária em Saúde: Revisão Sistemática
Rockenback, L. D. D. S., Barbosa, D. N. F., & Bez, M. R. (2021) · Revista de Atenção à Saúde · doi:10.13037/ras.vol19n69.7577
Smart Detection and Preservation of Privacy Concerns in Iot Systems: A Systematic Literature ReviewPreprint
Giordano, G., Palomba, F., & Ferrucci, F. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3979390
Peer Review #3 of "Microservice security: a systematic literature review (v0.1)"
(2022) · PeerJ Computer Science · doi:10.7287/peerj-cs.779v0.1/reviews/3
Peer Review #2 of "Microservice security: a systematic literature review (v0.2)"
(2022) · PeerJ Computer Science · doi:10.7287/peerj-cs.779v0.2/reviews/2
Peer Review #2 of "Microservice security: a systematic literature review (v0.1)"
(2022) · PeerJ Computer Science · doi:10.7287/peerj-cs.779v0.1/reviews/2
dos Santos, I. N. G., & de Andrade, M. D. (2021) · Revista de Direito Econômico e Socioambiental · doi:10.7213/rev.dir.econ.soc.v12i2.28709
Reis, D. A. (2021) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/t.3.2021.tde-31012022-130430
do Vale, J. W. S. P. (2021) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/t.3.2021.tde-01022022-113615
Religious Engagement within Refugee Resettlement in Canada: Lessons to Learn
Haq, M. I. (2021) · Jurnal Pemberdayaan Masyarakat Media Pemikiran dan Dakwah Pembangunan · doi:10.14421/jpm.2021.052-08
Pérez, Á. S. (2022) · ReiDoCrea Revista electrónica de investigación Docencia Creativa · doi:10.30827/digibug.72818
Evidence and perceptions on GUI test automation - An Exploratory Study
Polepalle, C., Kondoju, R. S., & Badampudi, D. (2022) · Innovations in Software Engineering Conference (ISEC) · doi:10.1145/3511430.3511442 · Cited by 1
“We conducted a literature review using the snowballing approach by following the guidelines proposed by Wohlin [37].”
MethodMaaß, L., Freye, M., Pan, C.-C., Dassow, H.-H., Niess, J., & Jahnel, T. (2022) · doi:10.2196/preprints.37980
Uso de design thinking na elicitação de requisitos em projeto ágil de software
de Oliveira, A. C., & Martins, H. F. (2018) · doi:10.26512/2018.tcc.21555
Gururaj, Y. P., Mittal, R., Karre, S. A., et al. (2022) · IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) · doi:10.1109/vrw55335.2022.00181
Mena, G. O. (2022) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/d.18.2022.tde-07042022-102122
MAMA: A Model-driven Approach Minding Accessibility
Duarte, L. M. C. (2022) · Digital Library of Theses and Dissertations (Universidade de São Paulo) · doi:10.11606/t.55.2022.tde-04052022-084304
GESTÃO DO CONHECIMENTO BENEFICIADA PELA INTEGRAÇÃO DA INDÚSTRIA 4.0: UMA REVISÃO DO ESCOPO
Lista, A. P., Tortorella, G. L., & Bouzon, M. (2022) · Anais ... Congresso Internacional do Conhecimento e Inovação · doi:10.48090/ciki.v1i1.1179
Backsourcing of Information Technology - a Systematic Literature ReviewPreprint
Molléri, J. S., Lassenius, C., & Jørgensen, M. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4049731
Initiating Original Equipment Manufacturer Remanufacturing
Duberg, J. V. (2022) · Linköping studies in science and technology. Thesis · doi:10.3384/9789179293208 · Cited by 2
Diks, M. E., van der Putten, I. M., & Hiligsmann, M. (2020) · Research Square · doi:10.21203/rs.3.rs-123374/v1
IDENTIFYING MICROSERVICES CANDIDATES IN LEGACY CODE
CARVALHO, L. M. D. A. (2020) · doi:10.17771/pucrio.acad.50746
3rd Symposium on Occupational Safety and Health Proceedings Book
Pinho, O., Baptista, J. S., Branco, J. C., Duarte, J., Martins, R., Santiago, R. J., Marques, A. T., Baptista, J. S., Costa, J. T., Rosado, A. S., et al. (2019) · doi:10.24840/978-972-752-260-6
Malaysia, M. (2020) · PLANNING MALAYSIA · doi:10.21837/pm.v18i13.798 · Cited by 1
“• Improvement in the collaboration between academia and industry: Among the studies found in our SMS, only four were proposed and/or validated by industry.”
BackgroundMethodInfluential
Metadata
- Authors
- Wohlin, C.
- Affiliations
- Blekinge Institute of Technology (SE)
- Corresponding author
- Wohlin, Claes
- ORCID
- Wohlin, Claes
- Venue
- Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering
- Publisher
- ACM
- Published
- 2014-05-13
- Pages
- 1-10
- OpenAlex
- W1975675278
- Cited by
- 4,234
- References
- 31
- Citations by year
- 2026: 433 · 2025: 552 · 2024: 651 · 2023: 594 · 2022: 566 · 2021: 477 · 2020: 356 · 2019: 250 · 2018: 175 · 2017: 98 · 2016: 60 · 2015: 17 · 2014: 4
- FWCI
- 50.91
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Evaluation and Assessment in Software Engineering (EASE) (publishedVersion)
- Funders
- Lunds Universitet
- Topics
- Physical Sciences › Computer Science › Information Systems › Software Engineering Techniques and Practices
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Computer Science › Information Systems › Software Engineering Research
- Keywords
- snowballing, systematic reviews, research guidelines, software engineering literature, replication study, literature search strategies
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.