The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Effectiveness and efficiency of search methods in systematic reviews of complex evidence: audit of primary sources
Open accessAbstract
OBJECTIVE: To describe where papers come from in a systematic review of complex evidence. Method Audit of how the 495 primary sources for the review were originally identified. RESULTS: Only 30% of sources were obtained from the protocol defined at the outset of the study (that is, from the database and hand searches). Fifty one per cent were identified by "snowballing" (such as pursuing references of references), and 24% by personal knowledge or personal contacts. CONCLUSION: Systematic reviews of complex evidence cannot rely solely on protocol-driven search strategies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Snowballing is the best approach for identifying sources published in obscure journals according to Greenhalgh and Peacock's study.
“and snowballing is the best approach for identifying sources published in obscure journals according to their study.”
From Guidelines for snowballing in systematic literature studies and a replication in software engineering (2014) · cites this paper as [15]
The paper’s “What this study adds” section calls snowball methods especially powerful for finding high-quality sources in obscure locations. The Discussion more specifically says citation tracking helps identify systematic reviews in obscure journals. It does not establish that snowballing is the best approach for finding all sources published in obscure journals. (LLM, conf=0.85)
Experimental — accuracy not yet measuredPrior work by Greenhalgh and Peacock found that snowballing is likely to identify important sources missed by protocol-driven searches.
“These additional methods were used because prior work by Greehalgh and Peacock [91] found that less efficient methods like snowballing are likely to identify important sources that would otherwise be missed, since predefined protocol driven search strategies cannot solely be relied on.”
From Challenges and barriers in virtual teams: a literature review (2020) · cites this paper as [91]
We piloted the use of a highly structured search strategy using protocol-driven searches across a range of electronic databases but found that was unsatisfactory.
“We piloted the use of a highly structured search strategy using protocol-driven searches across a range of electronic databases but, like Greenhalgh and Peacock [17] found that was this unsatisfactory.”
From Conducting a critical interpretive synthesis of the literature on access to healthcare by vulnerable groups (2006) · cites this paper as [17]
Greenhalgh and Peacock concluded that protocol-driven search approaches by themselves are not necessarily the most efficient method regardless of the number of databases used.
“Greenhalgh and Peacock [15] applied three different search methods in their research, and concluded that protocol-driven search approaches by themselves are not necessarily the most efficient method regardless of the number of databases used”
From Guidelines for snowballing in systematic literature studies and a replication in software engineering (2014) · cites this paper as [15]
The included articles' reference lists were evaluated using the snowball method to ensure that all relevant articles were included.
“the included articles’ reference lists also were evaluated (using the snowball method) to ensure that all relevant articles were included in the study [21].”
From Advantages and disadvantages of mobile applications for workplace health promotion: A scoping review (2024) · cites this paper as [21]
After full-text review, another 71 articles were excluded, and citation tracking added 6 articles, resulting in a final selection of 111 articles.
“After reading the first author ' s reports on the literature, the full research team decided to exclude another 71 articles from the selection that, based on the full text, proved not to meet the inclusion criteria.”
From The Role of School in Adolescents’ Identity Development. A Literature Review (2019) · cites this paper as Greenhalgh and Peacock (2005)
The Method section reports that this paper appraised 1,004 full-text sources and cited 495 in its final report; its table lists 34 sources found through citation tracking. Searches for 71 exclusions, 6 additions, and a final selection of 111 found no passage about those figures or the citing team's selection. (LLM, conf=0.76)
Experimental — accuracy not yet measuredMany relevant studies may be identified in sources other than electronic databases.
“As many relevant studies may be identified in sources other than electronic databases [29,30], searches that extend beyond the standard electronic databases should be considered.”
From COSMOS-E: Guidance on conducting systematic reviews and meta-analyses of observational studies of etiology (2019) · cites this paper as [29]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Synthesising qualitative and quantitative evidence: A review of possible methods
Dixon-Woods, M., Agarwal, S., Jones, D., Young, B., & Sutton, A. (2005) · Journal of Health Services Research & Policy · doi:10.1177/135581960501000110 · Cited by 2,132
Competing interests: none declared.
Hirsch, L. (2006) · Cited by 1,396
Storylines of research in diffusion of innovation: a meta-narrative approach to systematic review
Greenhalgh, T., Robert, G., Macfarlane, F., Bate, P., Kyriakidou, O., & Peacock, R. (2005) · Social Science & Medicine · doi:10.1016/j.socscimed.2004.12.001 · Cited by 913
Self-citationRoyle, P., & Milne, R. (2003) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462303000552 · Cited by 693
Diffusion of Innovations in Health Service Organisations: A Systematic Literature Review
Greenhalgh, T., Robert, G., Bate, P., Macfarlane, F., & Kyriakidou, O. (2007) · Cited by 453
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
The Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553
Wohlin, C. (2014) · Proceedings of the 18th International Conference on Evaluation and Assessment in Software Engineering · doi:10.1145/2601248.2601268 · Cited by 4,234
“Greenhalgh and Peacock [15] applied three different search methods in their research, and concluded that protocol-driven search approaches by themselves are not necessarily the most efficient method regardless of the number of databases used, since some sources may be found through personal knowledge/contacts (e.”
BackgroundResultInfluentialGreenhalgh, T., Wherton, J., Papoutsi, C., Lynch, J., Hughes, G., A'Court, C., Hinder, S., Fahy, N., Procter, R., & Shaw, S. (2017) · Journal of Medical Internet Research · doi:10.2196/jmir.8775 · Cited by 2,887
“We chose this “ancestry and snowballing” approach because initial database searching proved neither sensitive nor specific [42].”
Self-citationDoyle, C., Lennox, L., & Bell, D. (2013) · BMJ Open · doi:10.1136/bmjopen-2012-001570 · Cited by 2,666
“Associations refer to cases where one measure of patient experience (typically an overall rating of patient experience for a care provider) has a statistically signi fi cant association with one or more clinical effectiveness or patient safety variable.”
BackgroundDixon-Woods, M., Cavers, D., Agarwal, S., Annandale, E., Arthur, A., Harvey, J., Hsu, R., Katbamna, S., Olsen, R., Smith, L., et al. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-35 · Cited by 2,289
Defining Comorbidity: Implications for Understanding Health and Health Services
Valderas, J. M., Starfield, B. H., Sibbald, B., Salisbury, C., & Roland, M. O. (2009) · The Annals of Family Medicine · doi:10.1370/afm.983 · Cited by 1,730
Making Sense of Assessment Feedback in Higher Education
Evans, C. (2013) · Review of Educational Research · doi:10.3102/0034654312474350 · Cited by 1,143
“The snowball method, frequently employed in systematic reviews, enabled consideration of references that key articles had drawn upon, some of which predated 2000 (Greenhalgh & Peacock, 2005).”
MethodDeveloping design propositions through research synthesis
Denyer, D., Tranfield, D., & van Aken, J. E. (2008) · Organization Studies · doi:10.1177/0170840607088020 · Cited by 1,057
“This search strategy is particularly important for locating complex sources of evidence (Greenhalgh and Peacock 2005) for it explores interesting leads and identifies associated literatures such as books, conference papers and selected internet documents.”
BackgroundGreenhalgh, T., Hinton, L., Finlay, T., Macfarlane, A., Fahy, N., Clyde, B., & Chant, A. (2019) · Health Expectations · doi:10.1111/hex.12888 · Cited by 1,059
Self-citationDissemination and publication of research findings: an updated review of related biases
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I. (2010) · Health Technology Assessment · doi:10.3310/hta14080 · Cited by 1,039
Castro, E. M., Regenmortel, T. V., Vanhaecht, K., Sermeus, W., & Hecke, A. V. (2016) · Patient Education and Counseling · doi:10.1016/j.pec.2016.07.026 · Cited by 955
Life Expectancy and Mortality Rates in the United States, 1959-2017
Woolf, S. H., & Schoomaker, H. (2019) · JAMA · doi:10.1001/jama.2019.16932 · Cited by 963
Establishing a Safe Container for Learning in Simulation
Rudolph, J. W., Raemer, D. B., & Simon, R. (2014) · Simulation in Healthcare The Journal of the Society for Simulation in Healthcare · doi:10.1097/sih.0000000000000047 · Cited by 978
Identities and Identity Work in Organizations
Brown, A. D. (2014) · International Journal of Management Reviews · doi:10.1111/ijmr.12035 · Cited by 963
Schoenfeld, B. J., Grgic, J., Ogborn, D., & Krieger, J. W. (2017) · Journal of Strength and Conditioning Research · doi:10.1519/jsc.0000000000002200 · Cited by 851
“The reference lists of articles retrieved in the search were subsequently perused for any additional articles that had potential applicability to the topic as outlined by Greenhalgh and Peacock (19).”
MethodConducting systematic literature review in operations management
Thomé, A. M. T., Scavarda, L. F., & Scavarda, A. (2016) · Production Planning & Control · doi:10.1080/09537287.2015.1129464 · Cited by 858
“Greenhalgh and Peacock (2005) alert to the importance of extending the search beyond the keywords for inclusiveness, asking expert opinions and doing ‘snowball’ backward and forward criteria for SLR in management: transparency, inclusiveness and explanatory power.”
BackgroundAn Integrative Model of Patient-Centeredness – A Systematic Review and Concept Analysis
Scholl, I., Zill, J. M., Härter, M. C., & Dirmaier, J. (2014) · PLoS ONE · doi:10.1371/journal.pone.0107828 · Cited by 784
“Search strategy The search strategy included an electronic protocol driven search, combined with a comprehensive secondary search, based on the recommendations of Greenhalgh and Peacock [37].”
MethodSchoenfeld, B. J., Ogborn, D., & Krieger, J. W. (2017) · Journal of Sports Sciences · doi:10.1080/02640414.2016.1210197 · Cited by 671
“The reference lists of articles retrieved were then screened for any additional articles that had relevance to the topic, as described by Greenhalgh and Peacock (2005).”
MethodChallenges and barriers in virtual teams: a literature review
Morrison-Smith, S., & Ruiz, J. (2020) · SN Applied Sciences · doi:10.1007/s42452-020-2801-5 · Cited by 633
Sample Selection in Systematic Literature Reviews of Management Research
Hiebl, M. R. W. (2023) · Organizational Research Methods · doi:10.1177/1094428120986851 · Cited by 635
“Others report that researchers only identified 30% of relevant articles through keyword searches in databases while identifying the remaining 70% of articles through “snowballing” techniques, personal knowledge, or personal contacts (Greenhalgh & Peacock, 2005).”
BackgroundSelenium for preventing cancer
Vinceti, M., Filippini, T., Del Giovane, C., Dennert, G., Zwahlen, M., Brinkman, M., Zeegers, M. P., Horneber, M., D'Amico, R., & Crespi, C. M. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005195.pub4 · Cited by 649
Frank, J. R., Mungroo, R., Ahmad, Y., Wang, M., De Rossi, S., & Horsley, T. (2010) · Medical Teacher · doi:10.3109/0142159x.2010.500898 · Cited by 781
“One reviewer searched all sites using a snowballing technique that allowed for examination of both the site itself and of records embedded within each site (Greenhalgh & Peacock 2005).”
MethodUse of simulation-based learning in undergraduate nurse education: An umbrella systematic review
Cant, R., & Cooper, S. J. (2016) · Nurse Education Today · doi:10.1016/j.nedt.2016.11.015 · Cited by 621
Simulation in manufacturing and business: A review
Jahangirian, M., Eldabi, T., Naseer, A., Stergioulas, L. K., & Young, T. P. (2009) · European Journal of Operational Research · doi:10.1016/j.ejor.2009.06.004 · Cited by 599
“Reference chasing was also performed whilst reading the full-text, and the relevant references were added to the list of papers to be analysed [43].”
BackgroundSpecies richness in urban parks and its drivers: A review of empirical evidence
Nielsen, A. B., van den Bosch, M. A., Maruthaveeran, S., & Konijnendijk, C. C. (2013) · Urban Ecosystems · doi:10.1007/s11252-013-0316-1 · Cited by 592
“…and heterogeneous subject serendipity discoveries (such as finding a relevant paper when searching for something else, or by pursuing references of references) often proves important (Greengalgh and Peacock 2005), hence our scope was widened outside the protocol, e.g. by using " snow-balling " .”
BackgroundA Hermeneutic Approach for Conducting Literature Reviews and Literature Searches
Boell, S. K., & Ćećez-Kecmanović, D. (2014) · Communications of the Association for Information Systems · doi:10.17705/1cais.03412 · Cited by 724
“For instance, literature can also be identified through known literature such as using snowballing and citation tracking (see Appendix A), by asking colleagues or through serendipitous encounters when looking for other things [Greenhalgh and Peacock, 2005].”
MethodTaxonomy of approaches to developing interventions to improve health: a systematic methods overview
O’Cathain, A., Croot, L., Sworn, K., Duncan, E., Rousseau, N. S., Turner, K. M., Yardley, L., & Hoddinott, P. (2019) · Pilot and Feasibility Studies · doi:10.1186/s40814-019-0425-6 · Cited by 625
Cooper, C., Booth, A., Varley-Campbell, J., Britten, N., & Garside, R. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0545-3 · Cited by 561
Searching for and selecting studies
Lefebvre, C., Glanville, J., Briscoe, S., Littlewood, A., Marshall, C., Metzendorf, M., Noel‐Storr, A., Rader, T., Shokraneh, F., Thomas, J., et al. (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch4 · Cited by 647
Booth, A. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0249-x · Cited by 542
“Several commentators report that supplementary search strategies are useful in compensating for deficiencies of indexing terms and the limited value of “protocol-driven search strategies” [8, 10, 14, 62].”
BackgroundInfluentialGREENHALGH, T., POTTS, H. W. W., WONG, G., BARK, P., & SWINGLEHURST, D. (2009) · The Milbank Quarterly · doi:10.1111/j.1468-0009.2009.00578.x · Cited by 541
“We recommend that those unfamiliar with this approach access our methodological paper (Greenhalgh, Robert, Macfarlane, Bate, Kyriakidou, and Peacock 2005) and consult the glossary in the appendix to this paper.”
Self-citationEvidence-Based Management: Concept Cleanup Time?
Briner, R. B., Denyer, D., & Rousseau, D. M. (2009) · Academy of Management Perspectives · doi:10.5465/amp.2009.45590138 · Cited by 539
Dekkers, O. M., Vandenbroucke, J. P., Cevallos, M., Renehan, A. G., Altman, D. G., & Egger, M. (2019) · PLOS Medicine · doi:10.1371/journal.pmed.1002742 · Cited by 525
Walter, F. M., Webster, A., Scott, S. E., & Emery, J. D. (2011) · Journal of Health Services Research & Policy · doi:10.1258/jhsrp.2011.010113 · Cited by 517
Systematic literature studies: database searches vs. backward snowballing
Jalali, S., & Wohlin, C. (2012) · Proceedings of the ACM-IEEE international symposium on Empirical software engineering and measurement · doi:10.1145/2372251.2372257 · Cited by 659
“As discussed here, they have either compared different SLRs or mapping studies to check whether the same results are achieved [11] and [14], or investigated more efficient approaches of searching [8], [10] and [12].”
BackgroundResultInfluentialOn being ‘Systematic’ in Literature Reviews in IS
Boell, S. K., & Ćećez-Kecmanović, D. (2014) · Journal of Information Technology · doi:10.1057/jit.2014.26 · Cited by 491
“Knipschild (1995) reports that only 36% of the literature was identified through database searches, and Greenhalgh and Peacock (2005) report only 25%.”
BackgroundSystematic review automation technologies
Tsafnat, G., Glasziou, P., Choong, M. K., Dunn, A., Galgani, F., & Coiera, E. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-74 · Cited by 472
“step, it has been shown to improve retrieval [115], and is a recommended practice [46].”
BackgroundServitization: A contemporary thematic review of four major research streams
Raddats, C., Kowalkowski, C., Benedettini, O., Burton, J., & Gebauer, H. (2019) · Industrial Marketing Management · doi:10.1016/j.indmarman.2019.03.015 · Cited by 547
Stoll‐Kleemann, S., & Schmidt, U. J. (2016) · Regional Environmental Change · doi:10.1007/s10113-016-1057-5 · Cited by 549
“According to Greenhalgh and Peacock (2005), ‘‘in systematic reviews of complex and heterogeneous evidence […] [as is the case for this paper] formal protocoldriven search strategies may fail to identify important evidence’’ (p. 1065).”
BackgroundSystematic review approaches for climate change adaptation research
Berrang-Ford, L., Pearce, T., & Ford, J. D. (2015) · Regional Environmental Change · doi:10.1007/s10113-014-0708-7 · Cited by 474
“…o te se co n d ar y o r p ar ti al re le v an ce Research has shown that systematic review of complex and heterogeneous literature bases cannot rely solely on strict keyword searches (Greenhalgh and Peacock 2005), and this is likely to apply to many complex adaptation policy and practice questions.”
BackgroundComorbidity and dementia: a scoping review of the literature
Bunn, F., Burn, A.-M., Goodman, C., Rait, G., Norton, S., Robinson, L., Schoeman, J., & Brayne, C. (2014) · BMC Medicine · doi:10.1186/s12916-014-0192-4 · Cited by 451
“Such lateral strategies have been shown to be particularly important for identifying non-randomised studies [22].”
BackgroundChecking reference lists to find additional studies for systematic reviews
Horsley, T., Dingwall, O., & Sampson, M. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000026.pub2 · Cited by 446
Systematic Literature Reviews: An Introduction
Lamé, G. (2019) · Proceedings of the ... International Conference on Engineering Design · doi:10.1017/dsi.2019.169 · Cited by 753
Internet-based medical education: a realist review of what works, for whom and in what circumstances
Wong, G., Greenhalgh, T., & Pawson, R. (2010) · BMC Medical Education · doi:10.1186/1472-6920-10-12 · Cited by 457
“pursuing references of references [20].”
Self-citationLim, S. F. W. T., Jin, X., & Srai, J. S. (2018) · International Journal of Physical Distribution & Logistics Management · doi:10.1108/ijpdlm-02-2017-0081 · Cited by 403
Augmented reality in support of intelligent manufacturing – A systematic literature review
Egger, J., & Masood, T. (2019) · Computers & Industrial Engineering · doi:10.1016/j.cie.2019.106195 · Cited by 400
Perfect, M. M., Turley, M. R., Carlson, J. S., Yohanna, J., & Saint Gilles, M. P. (2016) · School Mental Health · doi:10.1007/s12310-016-9175-2 · Cited by 382
Halter, M., Boiko, O., Pelone, F., Beighton, C., Harris, R., Gale, J., Gourlay, S., & Drennan, V. (2017) · BMC Health Services Research · doi:10.1186/s12913-017-2707-0 · Cited by 366
“We identified additional studies by searching on PubMed by using the “related citations” algorithm and screening the reference lists of included studies for other reviews [36].”
MethodBattle, C. E., Hutchings, H. A., & Evans, P. A. (2011) · Injury · doi:10.1016/j.injury.2011.01.004 · Cited by 377
The location choice of foreign direct investments: Empirical evidence and methodological challenges
Nielsen, B. B., Asmussen, C. G., & Weatherall, C. D. (2016) · Journal of World Business · doi:10.1016/j.jwb.2016.10.006 · Cited by 380
“Because formal search techniques entering index terms or keywords in electronic databases may overlook important studies; we also utilized the snowball search technique (Greenhalgh & Peacock, 2005) by searching the bibliographies of our studies for additional work of relevance.”
MethodCurricula for empathy and compassion training in medical education: A systematic review
Patel, S. N., Pelletier‐Bui, A., Smith, S., Roberts, M., Kilgannon, H., Trzeciak, S., & Roberts, B. W. (2019) · PLoS ONE · doi:10.1371/journal.pone.0221412 · Cited by 414
Searching for studies: a guide to information retrieval for Campbell systematic reviews
Kugley, S., Wade, A., Thomas, J., Mahood, Q., Jørgensen, A. K., Hammerstrøm, K., & Sathe, N. (2017) · Campbell Systematic Reviews · doi:10.4073/cmg.2016.1 · Cited by 371
“As well as searching the references cited in existing systematic reviews and meta analyses, reference lists of identified studies may also be searched for additional studies (20).”
InfluentialNature-assisted therapy: Systematic review of controlled and observational studies
Annerstedt, M., & Währborg, P. (2011) · Scandinavian Journal of Public Health · doi:10.1177/1403494810396400 · Cited by 369
“In systematic reviews of complex evidence it has been found that only 30 % of the sources were obtained from the original protocol and the rest from either ‘‘snow-balling’’ (pursuing references of references) or personal knowledge or contacts [30].”
BackgroundGama, F., & Magistretti, S. (2025) · Journal of Product Innovation Management · doi:10.1111/jpim.12698 · Cited by 388
‘Shaken, but not stirred’: Sixty years of defining social innovation
Edwards-Schachter, M., & Wallace, M. L. (2017) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2017.03.012 · Cited by 406
Vulnerability and vulnerable groups from an intersectionality perspective
Kuran, C. H. A., Morsut, C., Kruke, B. I., Krüger, M., Segnestam, L., Orru, K., Nævestad, T. O., Airola, M., Keränen, J., Gabel, F., et al. (2020) · International Journal of Disaster Risk Reduction · doi:10.1016/j.ijdrr.2020.101826 · Cited by 349
A systematic review on Asian's farmers' adaptation practices towards climate change
Shaffril, H. A. M., Krauss, S. E., & Samsuddin, S. F. (2018) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2018.06.349 · Cited by 347
Rethlefsen, M. L., Farrell, A. M., Osterhaus Trzasko, L. C., & Brigham, T. J. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.11.025 · Cited by 331
Does Household Income Affect children’s Outcomes? A Systematic Review of the Evidence
Cooper, K., & Stewart, K. (2020) · Child Indicators Research · doi:10.1007/s12187-020-09782-0 · Cited by 410
“Following Greenhalgh and Peacock (2005), we then added potentially relevant studies referred to us or cited in identified studies (‘snowballed’).”
BackgroundFarina, N., Page, T. E., Daley, S., Brown, A., Bowling, A., Basset, T., Livingston, G., Knapp, M., Murray, J., & Banerjee, S. (2017) · Alzheimer's & Dementia · doi:10.1016/j.jalz.2016.12.010 · Cited by 314
“We did not approach the authors for clarification.”
BackgroundBori, M. S., Ashworth, M. J., Bisquera, A., Dodhia, H., Lynch, R., Wang, Y., & Fox-Rushby, J. A. (2020) · British Journal of General Practice · doi:10.3399/bjgp20x713897 · Cited by 333
“14,15 To identify the initial list of pearls, the authors’ initial knowledge of studies was supplemented by a Boolean logic search on MEDLINE (see Supplementary Appendix S1 for details).”
Evidence & Gap Maps: A tool for promoting evidence informed policy and strategic research agendas
Snilstveit, B., Vojtkova, M., Bhavsar, A., Stevenson, J., & Gaarder, M. M. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.05.015 · Cited by 352
Combination pharmacotherapy for the treatment of neuropathic pain in adults
Chaparro, L. E., Wiffen, P. J., Moore, R. A., & Gilron, I. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd008943.pub2 · Cited by 323
The Role of School in Adolescents’ Identity Development. A Literature Review
Verhoeven, M., Poorthuis, A. M. G., & Volman, M. (2019) · Educational Psychology Review · doi:10.1007/s10648-018-9457-3 · Cited by 318
Achieving the Sustainable Development Goals Requires Transdisciplinary Innovation at the Local Scale
Moallemi, E. A., Malekpour, S., Hadjikakou, M., Raven, R. P. J. M., Szetey, K., Ningrum, D., Dhiaulhaq, A., & Bryan, B. A. (2020) · One Earth · doi:10.1016/j.oneear.2020.08.006 · Cited by 313
Burgi, C., Peters, S., Ardern, C. L., Magill, J. R., Gomez, C. D., Sylvain, J., & Reiman, M. P. (2019) · British Journal of Sports Medicine · doi:10.1136/bjsports-2018-099982 · Cited by 313
Delays and Barriers to Cancer Care in Low- and Middle-Income Countries: A Systematic Review
Brand, N. R., Qu, L. G., Chao, A., & Ilbawi, A. (2019) · The Oncologist · doi:10.1634/theoncologist.2019-0057 · Cited by 316
Schoenfeld, B. J., Ogborn, D., & Krieger, J. W. (2016) · Sports Medicine · doi:10.1007/s40279-016-0543-8 · Cited by 308
Birden, H. H., Glass, N., Wilson, I. G., Harrison, M., Usherwood, T. P., & Nass, D. (2013) · Medical Teacher · doi:10.3109/0142159x.2013.789132 · Cited by 317
“This supports Greenhalgh’s findings that for complex areas, traditional search strings are not enough (Greenhalgh & Peacock 2005).”
BackgroundWoolf, K., POTTS, H. W. W., & McManus, I. C. (2011) · BMJ · doi:10.1136/bmj.d901 · Cited by 303
“and we used her personal knowledge of reports as a starting point.”
MethodValderas, J. M., & Alonso, J. (2008) · Quality of Life Research · doi:10.1007/s11136-008-9396-4 · Cited by 301
“Starting with selected previous classifications [1, 3, 7, 11], we applied a snow ball technique to identify additional references [24] (Table 1).”
MethodCallahan, J. L. (2014) · Human Resource Development Review · doi:10.1177/1534484314536705 · Cited by 301
“Greenhalgh and Peacock (2005) found that literature reviews that are limited to formal, pre-defined search strategies of databases and keywords “may fail to identify important evidence” (p. 2).”
Backgroundde Leeuw, A., Happé, F., & Hoekstra, R. A. (2020) · Autism Research · doi:10.1002/aur.2276 · Cited by 295
“A snowballing method [Greenhalgh & Peacock, 2005] was used, manually examining reference lists and citations of the identified articles.”
MethodIdentity Work and Organizational Identification
Brown, A. D. (2017) · International Journal of Management Reviews · doi:10.1111/ijmr.12152 · Cited by 300
Sierra‐Correa, P. C., & Kintz, J. R. C. (2014) · Marine Policy · doi:10.1016/j.marpol.2014.09.013 · Cited by 294
Systematic meta-review of supported self-management for asthma: a healthcare perspective
groups, F. T. P. A. R., Pinnock, H., Parke, H. L., Panagioti, M., Daines, L., Pearce, G., Epiphaniou, E., Bower, P., Sheikh, A., Griffiths, C. J., et al. (2017) · BMC Medicine · doi:10.1186/s12916-017-0823-7 · Cited by 291
“Forward citation has been shown to be an efficient and effective method of identifying relevant papers in systematic reviews of complex and heterogeneous evidence [22].”
MethodIntimate Partner Violence Among Men Who Have Sex With Men
Finneran, C. A., & Stephenson, R. B. (2012) · Trauma Violence & Abuse · doi:10.1177/1524838012470034 · Cited by 293
“The search strategy used included electronic database searches, hand searches of journals, and snowball searches of citation lists of relevant articles in order to find all eligible articles (Greenhalgh & Peacock, 2005; Wright, Brand, Dunn, & Spindler, 2007).”
Flood, S., Cradock-Henry, N. A., Blackett, P., & Edwards, P. (2018) · Environmental Research Letters · doi:10.1088/1748-9326/aac1c6 · Cited by 300
“Systematic reviews have been widely used in the health sciences (Greenhalgh and Peacock 2005, Heller et al 2008)—to evaluate the effectiveness of medical interventions for example—but more recently have been adapted for use in other fields (Ford and Pearce 2010).”
MethodSustainability trends and gaps in the textile, apparel and fashion industries
Abbate, S., Centobelli, P., Cerchione, R., Nadeem, S., & Riccio, E. (2023) · Environment, Development and Sustainability · doi:10.1007/s10668-022-02887-2 · Cited by 241
“To identify the remaining potentially important studies in our set, we used the ’snowball’ strategy as an inclusion criterion (Greenhalgh & Peacock, 2005).”
Deep reinforcement learning in production systems: a systematic literature review
Panzer, M., & Bender, B. (2021) · International Journal of Production Research · doi:10.1080/00207543.2021.1973138 · Cited by 317
Protocol - realist and meta-narrative evidence synthesis: Evolving Standards (RAMESES)
Greenhalgh, T., Wong, G., Westhorp, G., & Pawson, R. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-115 · Cited by 279
“To that end, and drawing on a previous study which demonstrated the effectiveness and efficiency of the methods proposed [42], and employing the skills of a specialist librarian, we will employ three approaches: 1.”
MethodSelf-citationWong, G., Greenhalgh, T., Westhorp, G., & Pawson, R. (2014) · Health Services and Delivery Research · doi:10.3310/hsdr02300 · Cited by 276
Self-citationRubenstein, M. A., Weiskopf, S. R., Bertrand, R., Carter, S. L., Comte, L., Eaton, M. J., Johnson, C. G., Lenoir, J., Lynch, A. J., Miller, B. W., et al. (2023) · Environmental Evidence · doi:10.1186/s13750-023-00296-0 · Cited by 317
“We used the snowball method [40], whereby we reviewed the cited literature within each article that passed title/ Search Term ((“climate” OR “global warming” OR "temperature" OR "precipita(cid:9)on") AND (“range” OR “distribu(cid:9)on” OR “habitat extent” OR “occupancy”) AND (“species”))…”
Mygind, L., Kjeldsted, E., Hartmeyer, R., Mygind, E., Bølling, M., & Bentsen, P. (2019) · Health & Place · doi:10.1016/j.healthplace.2019.05.014 · Cited by 331
Probiotics for prevention of atopic diseases in infants: systematic review and meta-analysis
Zuccotti, G., Meneghin, F., Aceti, A., Barone, G., Callegari, M. L., Mauro, A. D., Fantini, M. P., Gori, D., Indrio, F., Maggio, L., et al. (2015) · Allergy · doi:10.1111/all.12700 · Cited by 276
Coyle, H., Traynor, V., & Solowij, N. (2015) · The American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2014.04.009 · Cited by 274
“‘snowballing technique’ of retrieving sources from internet searches and recommendations of papers from colleagues, and secondary sourcing of studies cited in papers reviewed (38).”
MethodThe Evidence in Support of Physicians and Health Care Providers as Physical Activity Role Models
Lobelo, F., & de Quevedo, I. G. (2014) · American Journal of Lifestyle Medicine · doi:10.1177/1559827613520120 · Cited by 270
Bunn, F., Goodman, C., Sworn, K., Rait, G., Brayne, C., Robinson, L., McNeilly, E., & Iliffe, S. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001331 · Cited by 284
“search strategies have been shown to be particularly important for identifying non-randomised studies [21].”
BackgroundMuscular adaptations in low- versus high-load resistance training: a meta-analysis.
Schoenfeld, B. J., Wilson, J. M., Lowery, R. P., & Krieger, J. W. (2016) · European Journal of Sport Science · doi:10.1080/17461391.2014.989922 · Cited by 249
“Consistent with methods outlined by Greenhalgh and Peacock (2005), the reference lists of articles retrieved in the search were then screened for any additional articles that had relevance to the topic.”
MethodSynthesizing qualitative research: a review of published reports
Dixon‐Woods, M., Booth, A., & Sutton, A. J. (2007) · Qualitative Research · doi:10.1177/1468794107078517 · Cited by 294
Pollock, M., Fernandes, R. M., Pieper, D., Tricco, A. C., Gates, M., Gates, A., & Hartling, L. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1252-9 · Cited by 228
Pommergaard, H., Gessler, B., Burcharth, J., Angenete, E., Haglind, E., & Rosenberg, J. (2014) · Colorectal Disease · doi:10.1111/codi.12618 · Cited by 253
Hunt, H. A., Stanworth, S. J., Curry, N. S., Woolley, T., Cooper, C., Ukoumunne, O. C., Zhelev, Z., & Hyde, C. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010438.pub2 · Cited by 251
Cultural ecosystem services: Characteristics, challenges and lessons for urban green space research
Dickinson, D. C., & Hobbs, R. J. (2017) · Ecosystem Services · doi:10.1016/j.ecoser.2017.04.014 · Cited by 265
Reinforcement learning in urban network traffic signal control: A systematic literature review
Noaeen, M., Naik, A., Goodman, L., Crebo, J., Abrar, T., Abad, Z. S. H., Bazzan, A. L. C., & Far, B. H. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.116830 · Cited by 284
First-Generation College Students as Academic Learners: A Systematic Review
Ives, J., & Castillo‐Montoya, M. (2020) · Review of Educational Research · doi:10.3102/0034654319899707 · Cited by 336
“Snowball methods are frequently used in systematic reviews to identify important sources that would have otherwise been missed if only using the database search results (Greenhalgh & Peacock, 2005).”
BackgroundMethodHermwille, L. (2015) · Energy Research & Social Science · doi:10.1016/j.erss.2015.11.001 · Cited by 271
“All in all, the research approach chosen did not follow a formal protocol-driven search strategy, but included informal approaches and snowballing techniques that have been shown to be effective for cases of heterogeneous and complex policymaking questions (Greenhalgh & Peacock, 2005, p. 1065).”
MethodTraining load and match-play demands in basketball based on competition level: A systematic review
Petway, A. J., Freitas, T. T., Calleja-González, J., Leal, D. M., & Alcaráz, P. E. (2020) · PLoS ONE · doi:10.1371/journal.pone.0229212 · Cited by 246
Tsesis, I., Rosen, E., Tamse, A., Taschieri, S., & Kfir, A. (2010) · Journal of Endodontics · doi:10.1016/j.joen.2010.05.003 · Cited by 245
Defining professionalism in medical education: A systematic review
Birden, H. H., Glass, N., Wilson, I. G., Harrison, M., Usherwood, T. P., & Nass, D. (2013) · Medical Teacher · doi:10.3109/0142159x.2014.850154 · Cited by 247
“This supports Greenhalgh’s contention that for complex areas, traditional search strings are not enough (Greenhalgh & Peacock 2005).”
Background
Metadata
- Authors
- Greenhalgh, T., & Peacock, R.
- Affiliations
- University College London (GB)
- Corresponding author
- Greenhalgh, Trisha
- ORCID
- Greenhalgh, Trisha
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2005-10-17
- Volume
- 331
- Issue
- 7524
- Pages
- 1064-1065
- ISSN
- 0007-1447 (ISSN-L)
- 0959-8138
- 1468-5833
- PMC
- PMC1283190
- OpenAlex
- W2149364707
- Semantic Scholar
- 45cd85a4c05c0c295e4569e08f20a6f3268e6390
- Cited by
- 2,045
- References
- 5
- Citations by year
- 2026: 162 · 2025: 147 · 2024: 202 · 2023: 188 · 2022: 187 · 2021: 210 · 2020: 157 · 2019: 139 · 2018: 88 · 2017: 92 · 2016: 77 · 2015: 74 · 2014: 83 · 2013: 69 · 2012: 41
- FWCI
- 2.94
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- BMJ (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- UCL Discovery (University College London) (submittedVersion)
- HighWire Press Open Archive (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Keywords
- systematic reviews, snowballing, protocol adherence, database searching, hand searching
- MeSH terms
- Data Collection, Review Literature as Topic, Information Storage and Retrieval
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
