CoChat

Methodological developments in searching for studies for systematic reviews: past, present and future?

Open access
Lefebvre, Carol, Glanville, Julie, Wieland, L Susan, Coles, Bernadette, Weightman, Alison L · 2013 · Systematic Reviews
Cited by 112

The DOI matches its Crossref record; 24 of the 24 references it cites were found in a registry or an index.

Abstract

The Cochrane Collaboration was established in 1993, following the opening of the UK Cochrane Centre in 1992, at a time when searching for studies for inclusion in systematic reviews was not well-developed. Review authors largely conducted their own searches or depended on medical librarians, who often possessed limited awareness and experience of systematic reviews. Guidance on the conduct and reporting of searches was limited. When work began to identify reports of randomized controlled trials (RCTs) for inclusion in Cochrane Reviews in 1992, there were only approximately 20,000 reports indexed as RCTs in MEDLINE and none indexed as RCTs in Embase. No search filters had been developed with the aim of identifying all RCTs in MEDLINE or other major databases. This presented The Cochrane Collaboration with a considerable challenge in identifying relevant studies.Over time, the number of studies indexed as RCTs in the major databases has grown considerably and the Cochrane Central Register of Controlled Trials (CENTRAL) has become the best single source of published controlled trials, with approximately 700,000 records, including records identified by the Collaboration from Embase and MEDLINE. Search filters for various study types, including systematic reviews and the Cochrane Highly Sensitive Search Strategies for RCTs, have been developed. There have been considerable advances in the evidence base for methodological aspects of information retrieval. The Cochrane Handbook for Systematic Reviews of Interventions now provides detailed guidance on the conduct and reporting of searches. Initiatives across The Cochrane Collaboration to improve the quality inter alia of information retrieval include: the recently introduced Methodological Expectations for Cochrane Intervention Reviews (MECIR) programme, which stipulates 'mandatory' and 'highly desirable' standards for various aspects of review conduct and reporting including searching, the development of Standard Training Materials for Cochrane Reviews and work on peer review of electronic search strategies. Almost all Cochrane Review Groups and some Cochrane Centres and Fields now have a Trials Search Co-ordinator responsible for study identification and medical librarians and other information specialists are increasingly experienced in searching for studies for systematic reviews.Prospective registration of clinical trials is increasing and searching trials registers is now mandatory for Cochrane Reviews, where relevant. Portals such as the WHO International Clinical Trials Registry Platform (ICTRP) are likely to become increasingly attractive, given concerns about the number of trials which may not be registered and/or published. The importance of access to information from regulatory and reimbursement agencies is likely to increase. Cross-database searching, gateways or portals and improved access to full-text databases will impact on how searches are conducted and reported, as will services such as Google Scholar, Scopus and Web of Science. Technologies such as textual analysis, semantic analysis, text mining and data linkage will have a major impact on the search process but efficient and effective updating of reviews may remain a challenge.In twenty years' time, we envisage that the impact of universal social networking, as well as national and international legislation, will mean that all trials involving humans will be registered at inception and detailed trial results will be routinely available to all. Challenges will remain, however, to ensure the discoverability of relevant information in diverse and often complex sources and the availability of metadata to provide the most efficient access to information. We envisage an ongoing role for information professionals as experts in identifying new resources, researching efficient ways to link or mine them for relevant data and managing their content for the efficient production of systematic reviews.

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1186/2046-4053-2-78

Cites

24 of 24 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 24 references

Research works printed but not linked: 66

  1. Wilczynski NL, Walker CJ, McKibbon KA, Haynes RB: Assessment of methodologic search filters in MEDLINE. Proc Annu Symp Comput Appl Med Care. 1993, 601-605.
  2. Dickersin K, Scherer R, Lefebvre C: Identifying relevant studies for systematic reviews. BMJ. 1994, 309: 1286-1291. · DOI · Paper page
  3. PROFILE: Excerpta Medica Newsl. 1994, 11: 2-
  4. Lefebvre C: Identification of randomized controlled trials using MEDLINE: the situation in 1993. An Evidence-based Health Care System: the Case for Clinical Trials Registries. Edited by: Armstrong C. 1994, Bethesda: National Insitutes of Health, Office of Medical Applications of Research, 23-28.
  5. Wieland LS, Robinson KA, Dickersin K: Understanding why evidence from randomised clinical trials may not be retrieved from Medline: comparison of indexed and non-indexed records. BMJ. 2012, 344: d7501-doi: 10.1136/bmj.d7501 · DOI · Paper page
  6. Golder S, Loke Y: The performance of adverse effects search filters in MEDLINE and EMBASE. Health Info Libr J. 2012, 29: 141-151. · DOI
  7. Golder S, Loke Y: Failure or success of electronic search strategies to identify adverse effects data. J Med Libr Assoc. 2012, 100: 130-134. · DOI
  8. Golder S, Loke Y, Zorzela L: Some improvements are apparent in identifying adverse effects in systematic reviews from 1994 to 2011. Clin Epidemiol. 2013, 66: 253-260. · DOI
  9. Golder S, Loke Y, McIntosh HM: Poor reporting and inadequate searches were apparent in systematic reviews of adverse effects. J Clin Epidemiol. 2008, 61: 440-448. · DOI
  10. Beynon R, Leeflang MM, McDonald S, Eisinga A, Mitchell RL, Whiting P, Glanville JM: Search strategies to identify diagnostic accuracy studies in MEDLINE and EMBASE. Cochrane Database Syst Rev. 2013, 9: MR000022-
  11. Bachmann LM, Estermann P, Kronenberg C, Ter Riet G: Identifying diagnostic accuracy studies in EMBASE. J Med Libr Assoc. 2003, 91: 341-346.
  12. Ritchie G, Glanville J, Lefebvre C: Do published search filters to identify diagnostic test accuracy studies perform adequately?. Health Info Libr J. 2007, 24: 188-192. · DOI · Paper page
  13. Glanville J, Wilson P, Richardson R: Accessing the online evidence: a guide to key sources of research information on clinical and cost effectiveness. Qual Saf Health Care. 2003, 12: 229-231. · doi:10.1136/qhc.12.3.229
  14. Glanville J, Kaunelis D, Mensinkai S: How well do search filters perform in identifying economic evaluations in MEDLINE and EMBASE?. Int J Technol Assess Health Care. 2009, 25: 522-529. · DOI · Paper page
  15. Glanville J, Paisley S: Identifying economic evaluations for health technology assessment. Int J Technol Assess Health Care. 2010, 26: 436-440. · DOI
  16. McKinlay RJ, Wilczynski NL, Haynes RB: Optimal search strategies for detecting cost and economic studies in EMBASE. BMC Health Serv Res. 2006, 6: 67- · DOI
  17. Wilczynski NL, Haynes RB: Developing optimal search strategies for detecting clinically sound prognostic studies in MEDLINE: an analytic survey. BMC Med. 2004, 2: 23- · DOI · Paper page
  18. Wilczynski NL, Haynes RB: Optimal search strategies for detecting clinically sound prognostic studies in EMBASE: an analytic survey. J Am Med Inform Assoc. 2005, 12: 481-485. · DOI · Paper page
  19. Walker-Dilks C, Wilczynski NL, Haynes RB: Cumulative index to nursing and allied health literature search strategies for identifying methodologically sound causation and prognosis studies. Appl Nurs Res. 2008, 21: 98-103. · DOI
  20. Wilczynski NL, Haynes RB: Developing optimal search strategies for detecting clinically sound causation studies in MEDLINE. AMIA Annu Symp Proc. 2003, 719-723.
  21. Haynes RB, Kastner M, Wilczynski NL: Developing optimal search strategies for detecting clinically sound and relevant causation studies in EMBASE. BMC Med Inform Decis Mak. 2005, 5: 8- · DOI · Paper page
  22. Wieland S, Dickersin K: Selective exposure reporting and Medline indexing limited the search sensitivity for observational studies of the adverse effects of oral contraceptives. J Clin Epidemiol. 2005, 58: 560-567. · DOI
  23. Grant MJ: How does your searching grow? A survey of search preferences and the use of optimal search strategies in the identification of qualitative research. Health Info Libr J. 2004, 21: 21-32. · DOI
  24. Wong SS, Wilczynski NL, Haynes RB: Developing optimal search strategies for detecting clinically relevant qualitative studies in MEDLINE. Stud Health Technol Inform. 2004, 107: 311-316.
  25. McKibbon KA, Wilczynski NL, Haynes RB: Developing optimal search strategies for retrieving qualitative studies in PsycINFO. Eval Health Prof. 2006, 29: 440-454. · DOI · Paper page
  26. Flemming K, Briggs M: Electronic searching to locate qualitative research: evaluation of three strategies. J Adv Nurs. 2007, 57: 95-100. · DOI · Paper page
  27. Boynton J, Glanville J, McDaid D, Lefebvre C: Identifying systematic reviews in MEDLINE: developing an objective approach to search strategy design. J Info Sci. 1998, 24: 137-157. · DOI · Paper page
  28. White V, Glanville J, Lefebvre C, Sheldon T: A statistical approach to designing search filters to find systematic reviews: objectivity enhances accuracy. J Info Sci. 2001, 27: 357-370. · DOI · Paper page
  29. Wong SS, Wilczynski NL, Haynes RB: Comparison of top-performing search strategies for detecting clinically sound treatment studies and systematic reviews in MEDLINE and EMBASE. J Med Libr Assoc. 2006, 94: 451-455.
  30. Wilczynski NL, Haynes RB: EMBASE search strategies achieved high sensitivity and specificity for retrieving methodologically sound systematic reviews. J Clin Epidemiol. 2007, 60: 29-33. · DOI
  31. Glanville JM, Lefebvre C, Miles JN, Camosso-Stefinovic J: How to identify randomized controlled trials in MEDLINE: ten years on. J Med Libr Assoc. 2006, 94: 130-136.
  32. Lefebvre C, Manheimer E, Glanville J: Chapter 6: searching for studies. In Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0. Edited by Higgins JPT, Green S. The Cochrane Collaboration; 2011 http://www.handbook.cochrane.org
  33. Glanville J, Bayliss S, Booth A, Dundar Y, Fernandes H, Fleeman ND, Foster L, Fraser C, Fry-Smith A, Golder S, Lefebvre C, Miller C, Paisley S, Payne L, Price A, Welch K: So many filters, so little time: the development of a search filter appraisal checklist. J Med Libr Assoc. 2008, 96: 356-361. · DOI · Paper page
  34. Sampson M, McGowan J, Lefebvre C, Moher D, Grimshaw J: PRESS: Peer Review of Electronic Search Strategies [technology report number 477]. 2008, Ottawa: Canadian Agency for Drugs and Technologies in Health, http://www.cadth.ca/media/pdf/477_PRESS-Peer-Review-Electronic-Search-Strategies_tr_e.pdf ,
  35. Sampson M, McGowan J, Cogo E, Grimshaw J, Moher D, Lefebvre C: An evidence-based practice guideline for the peer review of electronic search strategies. J Clin Epidemiol. 2009, 62: 944-952. · DOI · Paper page
  36. Hopewell S, Clarke M, Lefebvre C, Scherer R: Handsearching versus electronic searching to identify reports of randomized trials. Cochrane Database Syst Rev. 2007, 2: MR000001-
  37. McDonald S, Lefebvre C, Antes G, Galandi D, Gotzsche P, Hammarquist C, Haugh M, Jensen KL, Kleijnen J, Loep M, Pistotti V, Rüther A: The contribution of handsearching European general health care journals to the Cochrane Controlled Trials Register. Eval Health Prof. 2002, 25: 65-75. · DOI · Paper page
  38. Horsley T, Dingwall O, Sampson M: Checking reference lists to find additional studies for systematic reviews. Cochrane Database Syst Rev. 2011, 8: MR000026-
  39. Schlosser RW, Wendt O, Bhavnani S, Nail-Chiwetalu B: Use of information-seeking strategies for developing systematic reviews and engaging in evidence-based practice: the application of traditional and comprehensive pearl growing. A review. Int J Lang Commun Disord. 2006, 41: 567-582. · DOI · Paper page
  40. Ormstad SS, Isojarvi J: Keeping up to date with information retrieval research: summarized research in information retrieval (SuRe info). Journal of EAHIL. 2013, 9: 17-19.
  41. MECIR: Methodological Standards for the Conduct of New Cochrane Intervention Reviews. 2012, http://www.editorial-unit.cochrane.org/sites/editorial-unit.cochrane.org/files/uploads/MECIR_conduct_standards%202.2%2017122012.pdf ,
  42. House of Commons: Science and technology committee: Clinical Trials: Written Evidence. 2013, http://www.parliament.uk/documents/commons-committees/science-technology/Clinical%20trials%20combined.pdf ,
  43. Goldacre B: AllTrials. 2013, http://www.alltrials.net ,
  44. Bero L:Searching for unpublished trials using trials registers and trials web sites and obtaining unpublished trial data and corresponding trial protocols from regulatory agencies. 2013, http://methods.cochrane.org/projects-developments/searching-unpublished-trials-using-trials-registers-and-trials-web-sites-and-o ,
  45. Ghersi D, Pang T: From Mexico to Mali: four years in the history of clinical trial registration. J Evid Based Med. 2009, 2: 1-7. · DOI · Paper page
  46. Doshi P, Jefferson T, Del Mar C: The imperative to share clinical study reports: recommendations from the Tamiflu experience. PLoS Med. 2012, 9: e1001201- · DOI · Paper page
  47. Jefferson T, Jones MA, Doshi P, Del Mar CB, Heneghan CJ, Hama R, Thompson MJ: Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children. Cochrane Database Syst Rev. 2012, 1: CD008965-
  48. Doshi P: Transparency interrupted: the curtailment of the European Medicines Agency’s policy on access to documents. JAMA Intern Med. 2013, doi:10.1001/jamainternmed.2013.9989 · doi:10.1001/jamainternmed.2013.9989
  49. Doshi P, Jones M, Jefferson T: Rethinking credible evidence synthesis. BMJ. 2012, 344: d7898- · DOI · Paper page
  50. Jefferson T, Doshi P, Thompson M, Heneghan C: Ensuring safe and effective drugs: who can do what it takes?. BMJ. 2011, 342: c7258- · DOI · Paper page
  51. Fitzpatrick RB: Science.gov: gateway to government science information. Med Ref Serv Q. 2010, 29: 58-66. · doi:10.1080/02763860903485191
  52. De Vet H, Eisinga A, Riphagen I, Aertgeerts B, Pewsner D: Chapter 7: searching for studies. In Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy version 04, The Cochrane Collaboration. Edited by Deeks J, Bossuyt P, Gatsonis C; 2008. http://srdta.cochrane.org/sites/srdta.cochrane.org/files/uploads/Chapter07-Searching-%28September-2008%29.pdf
  53. Hausner E, Waffenschmidt S, Kaiser T, Simon M: Routine development of objectively derived search strategies. Syst Rev. 2012, 1: 19- · DOI · Paper page
  54. Spoor P, Airey M, Bennett C, Greensill J, Williams R: Use of the capture-recapture technique to evaluate the completeness of systematic literature searches. BMJ. 1996, 313: 342-343. · DOI · Paper page
  55. Rucker G, Reiser V, Motschall E, Binder H, Meerpohl JJ, Antes G, Schumacher M: Boosting qualifies capture-recapture methods for estimating the comprehensiveness of literature searches for systematic reviews. J Clin Epidemiol. 2011, 64: 1364-1372. · DOI
  56. Sampson M, Zhang L, Morrison A, Barrowman NJ, Clifford TJ, Platt RW, Klassen TP, Moher D: An alternative to the hand searching gold standard: validating methodological search filters using relative recall. BMC Med Res Methodol. 2006, 6: 33- · DOI · Paper page
  57. Kastner M, Straus S, Goldsmith CH: Estimating the horizon of articles to decide when to stop searching in systematic reviews: an example using a systematic review of RCTs evaluating osteoporosis clinical decision support tools. AMIA Annu Symp Proc. 2007, 389-393.
  58. Booth A: How much searching is enough? Comprehensive versus optimal retrieval for technology assessments. Int J Technol Assess Health Care. 2010, 26: 431-435. · DOI · Paper page
  59. Sampson M, Shojania KG, McGowan J, Daniel R, Rader T, Iansavichene AE, Ji J, Ansari MT, Moher D: Surveillance search techniques identified the need to update systematic reviews. J Clin Epidemiol. 2008, 61: 755-762. · DOI
  60. Shekelle PG, Newberry SJ, Wu H, Suttorp M, Motala A, Lim YW, Balk EM, Chung M, Yu WW, Lee J, Gaylor JM, Moher D, Ansari MT, Skidmore R, Garritty C: Identifying Signals for Updating Systematic Reviews: A Comparison of Two Methods [internet]. 2011, Rockville, MD: Agency for Healthcare Research and Quality (US), http://www.ncbi.nlm.nih.gov/books/NBK56774/ ,
  61. CRD: Systematic Reviews: CRD’s Guidance for Undertaking Reviews in Health Care. 2009, York: Centre for Reviews and Dissemination, University of York
  62. IOM: Standards for finding and assessing individual studies. Finding What Works in Health Care: Standards for Systematic Reviews, Committee on Standards for Systematic Reviews of Comparative Effectiveness Research. Edited by: Eden J, Levit L, Berg A, Morton S. 2011, Washington, DC: The National Academies Press, 81-154.
  63. Moher D, Liberati A, Tetzlaff J, Altman DG: Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol. 2009, 62: 1006-1012. · DOI · Paper page
  64. Ellis AK: Teaching and Learning Elementary Social Studies. 1970, Boston, MA: Allyn & Bacon
  65. Cepeda MS, Lobanov V, Berlin JA: From ClinicalTrials.gov trial registry to an analysis-ready database of clinical trial results. Clin Trials. 2013, 10: 347-348. · DOI
  66. Ip S, Hadar N, Keefe S, Parkin C, Iovin R, Balk EM, Lau J: A web-based archive of systematic review data. Syst Rev. 2012, 1: 15- · DOI · Paper page

Cited by

See all 112

Metadata

Authors
Lefebvre, C., Glanville, J., Wieland, L. S., Coles, B., & Weightman, A. L.
Venue
Systematic Reviews
Published
2013-09-25
Volume
2
Issue
1
Pages
78-78
ISSN
2046-4053 (ISSN-L)
OpenAlex
W2130035612
Cited by
112
References
24
FWCI
2.32
Type
journal article
Language
en
Open access
gold
Licence
creativecommons.org/licenses/by/2.0 (tdm)

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Methodological developments in searching for studies for systematic reviews: past, present and future? https://app.cochat.ai/papers/10.1186/2046-4053-2-78
The paper
Lefebvre, C., Glanville, J., Wieland, L. S., Coles, B., & Weightman, A. L. (2013). Methodological developments in searching for studies for systematic reviews: past, present and future? Systematic Reviews. https://doi.org/10.1186/2046-4053-2-78

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.