The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources
Open accessAbstract
OBJECTIVE: To describe where papers come from in a systematic review of complex evidence. Method Audit of how the 495 primary sources for the review were originally identified. RESULTS: Only 30% of sources were obtained from the protocol defined at the outset of the study (that is, from the database and hand searches). Fifty one per cent were identified by "snowballing" (such as pursuing references of references), and 24% by personal knowledge or personal contacts. CONCLUSION: Systematic reviews of complex evidence cannot rely solely on protocol-driven search strategies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“and snowballing is the best approach for identifying sources published in obscure journals according to their study.”
From Guidelines for snowballing in systematic literature studies and a replication in software engineering (2014) · cites this paper as [15]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“These additional methods were used because prior work by Greehalgh and Peacock [91] found that less efficient methods like snowballing are likely to identify important sources that would otherwise be missed, since predefined protocol driven search strategies cannot solely be relied on.”
From Challenges and barriers in virtual teams: a literature review (2020) · cites this paper as [91]
The cited source says what the claim attributes to it.
In the citing paper
“We piloted the use of a highly structured search strategy using protocol-driven searches across a range of electronic databases but, like Greenhalgh and Peacock [17] found that was this unsatisfactory.”
From Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups (2006) · cites this paper as [17]
The cited source says what the claim attributes to it.
In the citing paper
“Greenhalgh and Peacock [15] applied three different search methods in their research, and concluded that protocol-driven search approaches by themselves are not necessarily the most efficient method regardless of the number of databases used”
From Guidelines for snowballing in systematic literature studies and a replication in software engineering (2014) · cites this paper as [15]
The cited source says what the claim attributes to it.
In the citing paper
“the included articles’ reference lists also were evaluated (using the snowball method) to ensure that all relevant articles were included in the study [21].”
From Advantages and disadvantages of mobile applications for workplace health promotion: A scoping review (2024) · cites this paper as [21]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“After reading the first author ' s reports on the literature, the full research team decided to exclude another 71 articles from the selection that, based on the full text, proved not to meet the inclusion criteria.”
From The Role of School in Adolescents’ Identity Development. A Literature Review (2019) · cites this paper as Greenhalgh and Peacock (2005)
Experimental — accuracy not yet measuredIn the citing paper
“As many relevant studies may be identified in sources other than electronic databases [29,30], searches that extend beyond the standard electronic databases should be considered.”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Synthesising qualitative and quantitative evidence: A review of possible methods
Dixon-Woods, M., Agarwal, S., Jones, D., Young, B., & Sutton, A. (2005) · Journal of Health Services Research & Policy · doi:10.1177/135581960501000110 · Cited by 2,132
Competing interests: none declared.
Hirsch, L. (2006) · Cited by 1,396
Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review
Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., Kyriakidou, O., & Peacock, R. (2005) · Social Science & Medicine · doi:10.1016/j.socscimed.2004.12.001 · Cited by 913
Self-citationRoyle, P., & Milne, R. (2003) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462303000552 · Cited by 693
Diffusion of Innovations in Health Service Organisations: A Systematic Literature Review
Greenhalgh, T., Robert, G., Bate, P., Macfarlane, F., & Kyriakidou, O. (2007) · Cited by 453
Self-citation
Cited by
Sorted by citations
Items 201–300 of 1,004
Measurement of Physician-Patient Communication—A Systematic Review
Zill, J. M., Christalle, E., Müller, E., Härter, M. C., Dirmaier, J., & Scholl, I. (2014) · PLoS ONE · doi:10.1371/journal.pone.0112637 · Cited by 135
A Systematic Survey of Just-in-Time Software Defect Prediction
Zhao, Y., Damevski, K., & Chen, H. (2022) · ACM Computing Surveys · doi:10.1145/3567550 · Cited by 135
“Empirical evidence suggests that the snowball method should be effective to locate “high quality sources in obscure locations [20].”
Gusenbauer, M. (2024) · Research Synthesis Methods · doi:10.1002/jrsm.1729 · Cited by 174
Effects of meal frequency on weight loss and body composition: a meta-analysis
Jon Schoenfeld, B., Albert Aragon, A., & Krieger, J. W. (2015) · Nutrition Reviews · doi:10.1093/nutrit/nuu017 · Cited by 127
Papoutsi, C., Mattick, K. L., Pearson, M., Brennan, N., Briscoe, S., & Wong, G. (2017) · Journal of Antimicrobial Chemotherapy · doi:10.1093/jac/dkx194 · Cited by 137
Execution of UML models: a systematic review of research and practice
Ciccozzi, F., Malavolta, I., & Selić, B. (2018) · Software & Systems Modeling · doi:10.1007/s10270-018-0675-4 · Cited by 153
“Snowballing To mitigate a potential bias with respect to the construct validity of the study, we complemented the previously described automatic search with a snowballing activity [22].”
MethodImplementation of safety checklists in surgery: a realist synthesis of evidence
Gillespie, B. M., & Marshall, A. (2015) · Implementation Science · doi:10.1186/s13012-015-0319-9 · Cited by 130
“Despite an intervention being deemed to be demonstrably efficacious, its long-term impact is contingent on its effectiveness in the ‘real world’, that is, the clinical environment [67–69].”
BackgroundHarrison, J. K., Fearon, P. A., Noel-Storr, A. H., McShane, R. H., Stott, D. J., & Quinn, T. J. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010772.pub2 · Cited by 135
Tucker, J., Daoud, M., Oates, N., Few, R., Conway, D., Mtisi, S., & Matheson, S. (2014) · Regional Environmental Change · doi:10.1007/s10113-014-0741-6 · Cited by 130
“Citation tracking and expert knowledge have been found to be efficient approaches to identify documents, in comparison with database searches (Greenhalgh and Peacock 2005; Booth 2006), and there is precedent for their use in systematic reviews, broadly defined (Ford et al. 2010a).”
MethodQuality improvement through clinical communities: eight lessons for practice
Aveling, E., Martin, G. P., Armstrong, N., Banerjee, J., & Dixon‐Woods, M. (2012) · Journal of Health Organization and Management · doi:10.1108/14777261211230754 · Cited by 129
“These formal searches were supplemented greatly by personal knowledge and contacts (Greenhalgh & Peacock 2005).”
Backgroundon behalf of the Italian Society of Neonatology, Aceti, A., Gori, D., Barone, G., Callegari, M. L., Mauro, A. D., Fantini, M. P., Indrio, F., Maggio, L., Meneghin, F., et al. (2015) · The Italian Journal of Pediatrics/Italian journal of pediatrics · doi:10.1186/s13052-015-0199-2 · Cited by 130
““Snowballing” technique was also used [20].”
Corden, K., Brewer, R., & Cage, E. (2021) · Review Journal of Autism and Developmental Disorders · doi:10.1007/s40489-021-00263-w · Cited by 137
“However, this limitation is not deemed problematic, as manual searches are intended and recommended to increase sensitivity of the search and minimise retrieval bias (Chapman et al., 2010; Greenhalgh & Peacock, 2005).”
BackgroundParry, R. H., Land, V., & Seymour, J. E. (2014) · BMJ Supportive & Palliative Care · doi:10.1136/bmjspcare-2014-000649 · Cited by 127
“We searched diverse sources 17 20 : existing knowledge within the review team and its contacts; online discussion lists; online specialist bibliographies; and the citations and reference lists of the publications we identified through other means.”
BackgroundPilbeam, C. J., Doherty, N., Davidson, R., & Denyer, D. (2016) · Safety Science · doi:10.1016/j.ssci.2016.02.015 · Cited by 146
“Greenhalgh and Peacock (2005) in a study of search methods discovered less than 30% of relevant articles by protocol driven search strategies (i.e. keywords).”
BackgroundRiopel, M., Nenciovici, L., Potvin, P., Chastenay, P., Charland, P., Sarrasin, J. B., & Masson, S. (2019) · Studies in Science Education · doi:10.1080/03057267.2019.1722420 · Cited by 127
“Using a snowball technique (Greenhalgh & Peacock, 2005), reference lists of these studies were manually search d for more relevant studies, resulting in a new total of 195 potentially relevant studies.”
MethodFads and Fashions in Management Practices: Taking Stock and Looking Forward
Piazza, A., & Abrahamson, E. (2020) · International Journal of Management Reviews · doi:10.1111/ijmr.12225 · Cited by 127
Gates, M., Gates, A., Guitard, S., Pollock, M., & Hartling, L. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01509-0 · Cited by 115
Adopting and assimilating new non-pharmaceutical technologies into health care: A systematic review
Robert, G., Greenhalgh, T., Macfarlane, F., & Peacock, R. (2010) · Journal of Health Services Research & Policy · doi:10.1258/jhsrp.2010.009137 · Cited by 119
Self-citationAt-risk children with asthma (ARC): a systematic review
Buelo, A., McLean, S., Julious, S. A., Kim, J. F., Bush, A., Henderson, J., Paton, J. Y., Sheikh, A., Shields, M. D., & Pinnock, H. (2018) · Thorax · doi:10.1136/thoraxjnl-2017-210939 · Cited by 126
Liedong, T. A., Peprah, A. A., Amartey, A. F. O., & Rajwani, T. S. (2020) · Journal of International Management · doi:10.1016/j.intman.2020.100756 · Cited by 134
Review: Application and Prospective Discussion of Machine Learning for the Management of Dairy Farms
Cockburn, M. (2020) · Animals · doi:10.3390/ani10091690 · Cited by 126
“While analyzing relevant papers, the snowball method was used, and significant cited literature was included [14].”
MethodShahaj, O., Denneny, D., Schwappach, A., Pearce, G., Epiphaniou, E., Parke, H. L., Taylor, S. J. C., & Pinnock, H. (2019) · Journal of Hypertension · doi:10.1097/hjh.0000000000001867 · Cited by 110
“We undertook a prepublication check in April 2018 using the ‘efficient and effective’ approach of forward citation of all included reviews using Google Scholar [20].”
Unmet Medical Need: An Introduction to Definitions and Stakeholder Perceptions
Vreman, R. A., Heikkinen, I., Schuurman, A., Sapède, C., Llinares, J., Hedberg, N., Athanasiou, D., Grueger, J., Leufkens, H. G. M., & Goettsch, W. (2019) · Value in Health · doi:10.1016/j.jval.2019.07.007 · Cited by 124
Process Studies of Organizational Space
Stephenson, K. A., Kuismin, A., Putnam, L. L., & Sivunen, A. E. (2020) · Academy of Management Annals · doi:10.5465/annals.2018.0146 · Cited by 132
Positive Deviance in Theory and Practice: A Conceptual Review
Herington, M. J., & van de Fliert, E. (2017) · Deviant Behavior · doi:10.1080/01639625.2017.1286194 · Cited by 149
“A snowballing process (Greenhalgh and Peacock 2005), whereby the reference lists (backward snowballing) and citations (forward snowballing) of relevant, primary review articles were scanned, identified several (n = 15) critical and frequently cited studies missing from the database search.”
BackgroundAir quality and mental health: evidence, challenges and future directions
Bhui, K. S., Newbury, J. B., Latham, R. M., Ucci, M., Nasir, Z. A., Turner, B., O’Leary, C., Fisher, H. L., Marczylo, E. L., Douglas, P., et al. (2023) · BJPsych Open · doi:10.1192/bjo.2023.507 · Cited by 143
“3,4 Even when systematically analysed, the conclusions of conventional reviews for complex areas often suggest there is inadequate evidence to draw firm conclusions.”
BackgroundPatterns of Access to Community Palliative Care Services: A Literature Review
Walshe, C., Todd, C. J., Caress, A., & Chew-Graham, C. A. (2008) · Journal of Pain and Symptom Management · doi:10.1016/j.jpainsymman.2008.05.004 · Cited by 122
Artificial intelligence maturity model: A systematic literature review
Sadiq, R. B., Safie, N., Abd Rahman, A. H., & Goudarzi, S. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.661 · Cited by 127
“This is attributable to its rigorous approach to provide answers to speci fi c questions or to test particular hypotheses ( Greenhalgh & Peacock, 2005 ; Okoli, 2015 ).”
Smith, I. C., Reichow, B., & Volkmar, F. R. (2015) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-015-2423-8 · Cited by 121
Systematic review on environmental innovativeness: A knowledge-based resource view
Pham, D. D. T., Paillé, P., & Halilem, N. (2019) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2018.11.221 · Cited by 121
Bauer, I. E., Gálvez, J. F., Hamilton, J. E., Balanzá‐Martínez, V., Zunta‐Soares, G. B., Soares, J. C., & Meyer, T. D. (2015) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2015.12.006 · Cited by 134
“Based on existing guidelines for conducting systematic reviews (including The Cochrane Guidelines for Systematic Reviews of Health Promotion and Public Health) (Armstrong et al., 2007; Booth et al., 2011; Greenhalgh and Peacock, 2005) we additionally hand searched content pages of key journals in psychiatry which usua…”
MethodAssociation between sleep bruxism and anxiety symptoms in adults: A systematic review
Polmann, H., Domingos, F. L., Melo, G., Stuginski‐Barbosa, J., Guerra, E. N. S., Porporatti, A. L., Dick, B. D., Flores‐Mir, C., & Canto, G. D. L. (2019) · Journal of Oral Rehabilitation · doi:10.1111/joor.12785 · Cited by 120
A systematic review of published evidence linking wetland plants to water regime components
Webb, J. A., Wallis, E. M., & Stewardson, M. J. (2012) · Aquatic Botany · doi:10.1016/j.aquabot.2012.06.003 · Cited by 119
Hildon, Z. J., Allwood, D., & Black, N. A. (2011) · International Journal for Quality in Health Care · doi:10.1093/intqhc/mzr072 · Cited by 119
Publication Bias in Special Education Meta-Analyses
Gage, N. A., Cook, B. G., & Reichow, B. (2017) · Exceptional Children · doi:10.1177/0014402917691016 · Cited by 125
“Research has shown that reliance on electronic databases is likely to result in missed studies (Greenhalgh & Peacock, 2005).”
Background“Patient participation” and related concepts: A scoping review on their dimensional composition
Halabi, I. O., Scholtes, B., Voz, B., Gillain, N., Durieux, N., Odero, A., Baumann, M., Ziegler, O., Gagnayre, R., Guillaume, M., et al. (2019) · Patient Education and Counseling · doi:10.1016/j.pec.2019.08.001 · Cited by 123
Aligning software engineering education with industrial needs: A meta-analysis
Garousi, V., Giray, G., Tüzün, E., Catal, C., & Felderer, M. (2019) · Journal of Systems and Software · doi:10.1016/j.jss.2019.06.044 · Cited by 119
“This issue has been discussed elsewhere as well, e.g., in the medical sciences ( Greenhalgh and Peacock, 2005 ).”
Beighton, C., & Wills, J. (2019) · Journal of Applied Research in Intellectual Disabilities · doi:10.1111/jar.12617 · Cited by 143
Air Pollution Dispersion Modelling in Urban Environment Using CFD: A Systematic Review
Pantusheva, M., Mitkov, R., Hristov, P. O., & Petrova‐Antonova, D. (2022) · Atmosphere · doi:10.3390/atmos13101640 · Cited by 118
“A method referred to as a ‘bibliographic search’, ‘citation tracking’, ‘snowballing‘, or ‘pearl growing’ was applied as a complimentary technique to identify more papers relevant to the current study [43,44].”
MethodQuinn, T. J., Fearon, P. A., Noel-Storr, A. H., Young, C., McShane, R. H., & Stott, D. J. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010079.pub2 · Cited by 123
Citation searching: a systematic review case study of multiple risk behaviour interventions
Wright, K., Golder, S., & Rodriguez-Lopez, R. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-73 · Cited by 103
“One study [7] audited the sources used in a systematic review of complex evidence i.”
BackgroundDisentangling patient and public involvement in healthcare decisions: why the difference matters
Fredriksson, M., & Tritter, J. Q. (2017) · Sociology of Health & Illness · doi:10.1111/1467-9566.12483 · Cited by 119
“…and including literature and ideal-type characteristics, we have used a snowballing approach (such as pursuing references of references) (Greenhalgh and Peacock 2005) based on previous literature on patient and public involvement in healthcare as well as on literature on citizen and…”
MethodA survey of simulation modeling techniques in production planning and control (PPC)
Jeon, S. M., & Kim, G. (2016) · Production Planning & Control · doi:10.1080/09537287.2015.1128010 · Cited by 116
“…1 illustrates the suitability of different types of simulation modelling technique to the issues in PPC theoretically procedure, with reference to Greenhalgh and Peacock (2005) and Jahangirian et al. (2010): (1) The scientific search engine (Science Direct, IEEE Explore, Scopus) is used to…”
MethodEvidence Gap Maps — A Tool for Promoting Evidence-Informed Policy and Prioritizing Future Research
Snilstveit, B., Vojtkova, M., Bhavsar, A., & Gaarder, M. M. (2013) · World Bank, Washington, DC eBooks · doi:10.1596/1813-9450-6725 · Cited by 115
“19 Snowballing refers to tracking back and pursuing references of relevant papers from both primary studies and reviews to increase the body of evidence - see Greenhalgh and Peacock (2005).”
BackgroundGross, R., Hanna, R. F., Gambhir, A., Heptonstall, P. J., & Speirs, J. F. (2018) · Energy Policy · doi:10.1016/j.enpol.2018.08.061 · Cited by 113
“A snowballing technique (Greenhalgh and Peacock, 2005) was also employed which involved following citation trails of documents identified through the keyword searches and including relevant cited documents in the evidence database, in order to widen the number of potential technology or product case studies.”
MethodPrabhakaran, A., Mahamadu, A., & Mahdjoubi, L. (2022) · Automation in Construction · doi:10.1016/j.autcon.2022.104228 · Cited by 128
“Similarly, Greenhalgh and Peacock [42] also emphasised the importance of using a broader keyword search for inclusiveness.”
Nonprofit Organizational Effectiveness
Liket, K. C., & Maas, K. (2013) · Nonprofit and Voluntary Sector Quarterly · doi:10.1177/0899764013510064 · Cited by 112
“Due to the complexity in defining appropriate search terms to guide a systematic literature review of hypothesized OE determinants, a “snowball” technique was used (Greenhalgh & Peacock, 2005).”
MethodPrip, A., Møller, K. A., Nielsen, D. L., Jarden, M., Olsen, M.-H., & Danielsen, A. K. (2017) · Cancer Nursing · doi:10.1097/ncc.0000000000000533 · Cited by 134
Evidence-based interventions to reduce tuberculosis stigma: a systematic review
Sommerland, N., Wouters, E., Mitchell, E. M. H., Ngicho, M., Redwood, L., Masquillier, C., van Hoorn, R., van den Hof, S., & Rie, A. V. (2017) · The International Journal of Tuberculosis and Lung Disease · doi:10.5588/ijtld.16.0788 · Cited by 112
Xiao, J., & Xiong, K. (2022) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2022.158538 · Cited by 112
Sofaer, N., & Strech, D. (2011) · Public Health Ethics · doi:10.1093/phe/phr013 · Cited by 111
“Last, we searched in-text references, notes and bibliographies of included publications for promising titles (a ‘snow-ball search’) (Greenhalgh and Peacock, 2005).”
MethodPatient involvement in health professionals’ education: a meta-narrative review
Rowland, P., Anderson, M. A., Kumagai, A. K., McMillan, S. E., Sandhu, V. K., & Langlois, S. (2018) · Advances in Health Sciences Education · doi:10.1007/s10459-018-9857-7 · Cited by 116
Cooper, C., Booth, A., Britten, N., & Garside, R. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0625-1 · Cited by 137
Schoenfeld, B. J., Ogborn, D. I., Vigotsky, A. D., Franchi, M. V., & Krieger, J. W. (2017) · Journal of Strength and Conditioning Research · doi:10.1519/jsc.0000000000001983 · Cited by 122
Rodney, A., & Hill, P. (2014) · International Journal for Equity in Health · doi:10.1186/s12939-014-0072-8 · Cited by 110
“Additionally, as similar studies [18,19] have shown that for reviews of complex evidence, formal protocol”
BackgroundCortinovis, C., Haase, D., Zanon, B., & Geneletti, D. (2018) · Landscape and Urban Planning · doi:10.1016/j.landurbplan.2018.09.007 · Cited by 110
Feasible Options for Behavior Change Toward More Effective Ocean Literacy: A Systematic Review
Stoll‐Kleemann, S. (2019) · Frontiers in Marine Science · doi:10.3389/fmars.2019.00273 · Cited by 134
Murray, R. J., Brosch, T., & Sander, D. (2014) · Cortex · doi:10.1016/j.cortex.2014.06.010 · Cited by 114
“…in its superior temporal (Sato et al., 2011b) and spatial (Hashiguchi et al., 2007) resolution of 5 msec and between ~.05 mme1 cm, respectively (Asano et al., 2005; Gray, Maldonado, Wilson, & McNaughton, 1995; Grover & Buchwald, 1970; Legatt, Arezzo, & Vaughan, 1980; Logothetis, 2002, 2003).”
Patients’ experiential knowledge and expertise in health care: A hybrid concept analysis
Castro, E. M., Regenmortel, T. V., Sermeus, W., & Vanhaecht, K. (2018) · Social Theory & Health · doi:10.1057/s41285-018-0081-6 · Cited by 108
McFadden, S. M., Demeke, J., Dada, D., Wilton, L., Wang, M., Vlahov, D., & Nelson, L. E. (2021) · Journal of Urban Health · doi:10.1007/s11524-021-00588-1 · Cited by 108
“To ensure that this search was comprehensive, we used snowball methods for secondary search terms [17].”
MethodMethods for obtaining unpublished data
Young, T., & Hopewell, S. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000027.pub2 · Cited by 107
Wildlife crime: a conceptual integration, literature review, and methodological critique
Kurland, J., Pires, S. F., McFann, S. C., & Moreto, W. D. (2017) · Crime Science · doi:10.1186/s40163-017-0066-0 · Cited by 111
“Snowballing, or reference and citation tracking, was used to find articles based on literature already familiar to the authors through existing knowledge, previous research, and personal contacts (Greenhalgh and Peacock 2005; Wohlin 2014).”
MethodIntergenerational learning in organizations
Ropes, D. (2013) · European journal of training and development · doi:10.1108/ejtd-11-2012-0081 · Cited by 120
Greenhalgh, J., Pawson, R., Wright, J. M., Black, N. A., Valderas, J. M., Meads, D. M., Gibbons, E. J., Wood, L., Wood, C., Mills, C., et al. (2014) · BMJ Open · doi:10.1136/bmjopen-2014-005601 · Cited by 110
IMPRoving Outcomes for children exposed to domestic ViolencE (IMPROVE): an evidence synthesis
Howarth, E., Moore, T. H., Welton, N. J., Lewis, N. V., Stanley, N., MacMillan, H. L., Shaw, A. R. G., Hester, M., Bryden, P. A., & Feder, G. S. (2016) · Public Health Research · doi:10.3310/phr04100 · Cited by 112
An updated review on use of L1 in foreign language classrooms
Shin, J.-Y., Dixon, L. Q., & Choi, Y. (2019) · Journal of Multilingual and Multicultural Development · doi:10.1080/01434632.2019.1684928 · Cited by 105
“We also conducted an additional database search, including the two keywords translanguaging and translation (Greenhalgh and Peacock 2005).”
MethodCitation tracking for systematic literature searching: A scoping review
Hirt, J., Nordhausen, T., Appenzeller‐Herzog, C., & Ewald, H. (2023) · Research Synthesis Methods · doi:10.1002/jrsm.1635 · Cited by 105
Tang, Y., & Hew, K. F. (2019) · International journal of communication · Cited by 104
Gentil, P., Fisher, J., & Steele, J. (2017) · Sports Medicine · doi:10.1007/s40279-016-0627-5 · Cited by 104
Considering rigor and relevance when evaluating test driven development: A systematic review
Munir, H., Moayyed, M., & Petersen, K. (2014) · Information and Software Technology · doi:10.1016/j.infsof.2014.01.002 · Cited by 104
“Apply backward snowball sampling: Through snowball sampling [20, 19] only three additional studies were identified.”
BackgroundPatient-centred care in general dental practice—a systematic review of the literature
Mills, I., Frost, J., Cooper, C., Moles, D. R., & Kay, E. (2014) · BMC Oral Health · doi:10.1186/1472-6831-14-64 · Cited by 100
“It is recognised that protocol driven search strategies do not always manage to identify all the relevant sources of literature, irrespective of the number of databases included within the search [35].”
BackgroundGuise, V., Anderson, J., & Wiig, S. (2014) · BMC Health Services Research · doi:10.1186/s12913-014-0588-z · Cited by 102
“This is a major reason for also conducting hand searches and searches based on reference tracking and citation tracking, to increase the chances of finding further relevant items [52].”
BackgroundMethodEffects of maternal diets on preterm birth and low birth weight: a systematic review
Gete, D. G., Waller, M., & Mishra, G. D. (2019) · British Journal Of Nutrition · doi:10.1017/s0007114519002897 · Cited by 101
“Additional relevant articles were also identified by Snowballing(29) and Pearl growing methods(30).”
BackgroundRobles, M. T. A., Collado‐Mateo, D., Fernández-Espínola, C., Viera, E. C., & Fuentes‐Guerra, F. J. G. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17020505 · Cited by 101
“Subsequently, to find any additional articles that met the inclusion criteria [33], the reference lists of articles retrieved and other sources were screened as part of a complementary search.”
BackgroundSound Externalization: A Review of Recent Research
Best, V., Baumgartner, R., Lavandier, M., Majdak, P., & Kopčo, N. (2020) · Trends in Hearing · doi:10.1177/2331216520948390 · Cited by 101
“…review includes all of the psychoacoustical studies returned by searching the PubMed and Web of Science databases with keywords “sound” and “externalization,” as well as other relevant references identified via snowball sampling and personal knowledge where appropriate (Greenhalgh & Peacock, 2005).”
BackgroundSystematic reviews in medical education: A practical approach: AMEE Guide 94
Sharma, R., Gordon, M. A., Dharamsi, S., & Gibbs, T. J. (2014) · Medical Teacher · doi:10.3109/0142159x.2014.970996 · Cited by 106
A Systematic Review of International Affective Picture System (IAPS) around the World
Branco, D., Gonçalves, Ó. F., & Badia, S. B. I. (2023) · Sensors · doi:10.3390/s23083866 · Cited by 100
“Further articles were found by reviewing the bibliographic references of the articles selected (snowball search method [35]) (See Table 1).”
BackgroundEffect of Adherence to Mediterranean Diet during Pregnancy on Children’s Health: A Systematic Review
Biagi, C., Di Nunzio, M., Bordoni, A., Gori, D., & Lanari, M. (2019) · Nutrients · doi:10.3390/nu11050997 · Cited by 99
“Two researchers (C.B. and M.D.N.) independently and blindly assessed the risk of bias of included studies using the parameters defined by the Cochrane Tool for Quality Assessment [36] and the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) [37].”
MethodEvans, C., Ison, L., Ellis-Smith, C. I., Nicholson, C. J., Costa, A., Oluyase, A. O., Namisango, E., Bone, A. E., Brighton, L. J., Yi, D., et al. (2019) · Milbank Quarterly · doi:10.1111/1468-0009.12373 · Cited by 101
Parry, R. H., Pino, M., Faull, C., & Feathers, L. (2016) · Patient Education and Counseling · doi:10.1016/j.pec.2016.03.013 · Cited by 98
“We searched for publications via multiple sources [18]: the authors’ existing knowledge and collections; electronic databases (ISI Web of Science; Medline; Embase, Google Scholar); and reference and citation tracking of identified publications.”
MethodAndersen, J. H., Malmros, P. Å., Ebbehoej, N. E., Flachs, E. M., Bengtsen, E., & Bonde, J. P. (2018) · Scandinavian Journal of Work Environment & Health · doi:10.5271/sjweh.3775 · Cited by 103
“5%) were included from snowballing, which is not unusual in reviews (14).”
BackgroundAdhesion to high-performance polymers applied in dentistry: A systematic review
Gama, L. T., Duque, T. M., Özcan, M., Philippi, A. G., Mezzomo, L. A., & Gonçalves, T. M. S. V. (2020) · Dental Materials · doi:10.1016/j.dental.2020.01.002 · Cited by 98
Searching for studies: A guide to information retrieval for Campbell systematic reviews
MacDonald, H. R., Comer, C., Foster, M., Labelle, P., Marsalis, S. E., Nyhan, K., Premji, Z., Rogers, M., Splenda, R., Stansfield, C., et al. (2024) · Campbell Systematic Reviews · doi:10.1002/cl2.1433 · Cited by 113
“Along with searching the references cited in existing systematic reviews and meta ‐ analyses, reference lists of identified studies may also be searched for additional studies (Greenhalgh & Peacock, 2005; Horsley et al., 2011).”
BackgroundMethodKarunananthan, S., Wolfson, C. M., Bergman, H., Béland, F., & Hogan, D. B. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-68 · Cited by 98
Secondary schools included: a literature review
Vroey, A. D., Struyf, E., & Petry, K. (2015) · International Journal of Inclusive Education · doi:10.1080/13603116.2015.1075609 · Cited by 98
Khurum, M., Gorschek, T., & Wilson, M. (2012) · Journal of Software Evolution and Process · doi:10.1002/smr.1560 · Cited by 97
“It involves starting with the review of one or two seminal research papers and/or books and pursuing references of references and electronic citation tracking [43].”
BackgroundWu, S. P. W., & Rau, M. A. (2019) · Educational Psychology Review · doi:10.1007/s10648-019-09467-3 · Cited by 115
“To identify these studies, we searched research databases for relevant articles published in journals and books, including ERIC, EBSCO, and PsychINFO, and GoogleScholar using the keywords: (Bdraw*^ Bsketch*^) with (Blearning^ Binstruction^ Bcontent knowledge^) and with (BSTEM^ Bscience^ Bmath*^ Bengineering^ Btechnolo…”
InfluentialNon-medical prescribing in the United Kingdom National Health Service: A systematic policy review
Graham-Clarke, E., Rushton, A. B., Noblet, T., & Marriott, J. F. (2019) · PLoS ONE · doi:10.1371/journal.pone.0214630 · Cited by 97
Engaging students in higher education with educational technology
Godsk, M., & Møller, K. L. (2024) · Education and Information Technologies · doi:10.1007/s10639-024-12901-x · Cited by 171
“The reason for using Google Scholar was to compensate for the low effectiveness associated with protocol-driven searches on standard electronic databases (Greenhalgh & Peacock, 2005).”
Utility of Repeat Head Computed Tomography After Blunt Head Trauma: A Systematic Review
Wang, M. C., Linnau, K. F., Tirschwell, D., & Hollingworth, W. (2006) · The Journal of Trauma: Injury, Infection, and Critical Care · doi:10.1097/01.ta.0000197385.18452.89 · Cited by 97
Man, J. D., Mayega, R. W., Sarkar, N. D. P., Waweru, E., Leys, M., van Olmen, J., & Criel, B. (2016) · The International Journal of Person Centered Medicine · doi:10.5750/ijpcm.v6i3.591 · Cited by 99
Use Cases for Design Personas: A Systematic Review and New Frontiers
Salminen, J., Wenyun Guan, K., Jung, S.-G., & Jansen, B. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517589 · Cited by 140
“At this point, additional articles were also further retrieved by applying snowball sampling based on reading the screened articles and identifying other relevant research articles, as suggested for the implementation of systematic literature reviews [68].”
Mandrik, O. (., Severens, J. L. (., Bardach, A., Ghabri, S., Hamel, C., Mathes, T., Vale, L., Wisløff, T., & Goldhaber-Fiebert, J. D. (2021) · Value in Health · doi:10.1016/j.jval.2021.01.002 · Cited by 110
“27 This means that the process of identifying relevant literature should include supplemental searches 28 using at least 1-step back citation tracking of included studies.”
MethodLand, V., Parry, R. H., & Seymour, J. E. (2017) · Health Expectations · doi:10.1111/hex.12557 · Cited by 96
Bellisario, A., & Pavlov, A. (2018) · Production Planning & Control · doi:10.1080/09537287.2018.1432909 · Cited by 109
Outdoor physical activity bears multiple benefits to health and society
Manferdelli, G., Torre, A. L., & Codella, R. (2019) · The Journal of Sports Medicine and Physical Fitness · doi:10.23736/s0022-4707.18.08771-6 · Cited by 96
Chapter 20: Qualitative research and Cochrane reviews
Noyes, J., Popay, J., Pearson, A., Hannes, K., & Booth, A. (2008) · Cited by 96
How Do I Know Who You Think You Are? A Review of Research Methods on Organizational Identity
Ravasi, D., & Canato, A. (2013) · International Journal of Management Reviews · doi:10.1111/ijmr.12008 · Cited by 112
Costa, D. S. J., Mercieca‐Bebber, R., Tesson, S., Seidler, Z. E., & Lopez, A.-L. (2019) · BMJ Open · doi:10.1136/bmjopen-2018-025166 · Cited by 118
Tools for evaluating educational apps for young children: a systematic review of the literature
Papadakis, S. (2020) · Interactive Technology and Smart Education · doi:10.1108/itse-08-2020-0127 · Cited by 110
Metadata
- Authors
- Greenhalgh, T., & Peacock, R.
- Affiliations
- University College London (GB)
- Corresponding author
- Greenhalgh, Trisha
- ORCID
- Greenhalgh, Trisha
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2005-10-17
- Volume
- 331
- Issue
- 7524
- Pages
- 1064-1065
- ISSN
- 0007-1447 (ISSN-L)
- 0959-8138
- 1468-5833
- PMC
- PMC1283190
- OpenAlex
- W2149364707
- Semantic Scholar
- 45cd85a4c05c0c295e4569e08f20a6f3268e6390
- Cited by
- 2,045
- References
- 5
- Citations by year
- 2026: 162 · 2025: 147 · 2024: 202 · 2023: 188 · 2022: 187 · 2021: 210 · 2020: 157 · 2019: 139 · 2018: 88 · 2017: 92 · 2016: 77 · 2015: 74 · 2014: 83 · 2013: 69 · 2012: 41
- FWCI
- 2.94
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- BMJ (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- HighWire Press Open Archive (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Keywords
- systematic reviews, snowballing, protocol adherence, database searching, hand searching
- MeSH terms
- Data Collection, Review Literature as Topic, Information Storage and Retrieval
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
