A review of the reporting of web searching to identify studies for Cochrane systematic reviews
The DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
The literature searches that are used to identify studies for inclusion in a systematic review should be comprehensively reported. This ensures that the literature searches are transparent and reproducible, which is important for assessing the strengths and weaknesses of a systematic review and re‐running the literature searches when conducting an update review. Web searching using search engines and the websites of topically relevant organisations is sometimes used as a supplementary literature search method. Previous research has shown that the reporting of web searching in systematic reviews often lacks important details and is thus not transparent or reproducible. Useful details to report about web searching include the name of the search engine or website, the URL, the date searched, the search strategy, and the number of results. This study reviews the reporting of web searching to identify studies for Cochrane systematic reviews published in the 6‐month period August 2016 to January 2017 (n = 423). Of these reviews, 61 reviews reported using web searching using a search engine or website as a literature search method. In the majority of reviews, the reporting of web searching was found to lack essential detail for ensuring transparency and reproducibility, such as the search terms. Recommendations are made on how to improve the reporting of web searching in Cochrane systematic reviews.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
PRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223
The role of google scholar in evidence reviews and its applicability to grey literature searching
Haddaway, N. R., Collins, A. M., Coughlin, D., & Kirk, S. (2015) · PLOS ONE · doi:10.1371/journal.pone.0138237 · Cited by 2,387
Page, M. J., Shamseer, L., Altman, D. G., Tetzlaff, J., Sampson, M., Tricco, A. C., Catalá-López, F., Li, L., Reid, E. K., Sarkis-Onofre, R., et al. (2016) · PLOS Medicine · doi:10.1371/journal.pmed.1002028 · Cited by 768
Searching for studies: a guide to information retrieval for Campbell systematic reviews
Kugley, S., Wade, A., Thomas, J., Mahood, Q., Jørgensen, A. K., Hammerstrøm, K., & Sathe, N. (2017) · Campbell Systematic Reviews · doi:10.4073/cmg.2016.1 · Cited by 371
Analysis of the reporting of search strategies in Cochrane systematic reviews
Yoshii, A., Plaut, D. A., McGraw, K. A., Anderson, M. J., & Wellik, K. E. (2009) · Journal of the Medical Library Association : JMLA · doi:10.3163/1536-5050.97.1.004 · Cited by 137
A rapid method to increase transparency and efficiency in web-based searches
Haddaway, N. R., Collins, A. M., Coughlin, D., & Kirk, S. (2017) · Environmental Evidence · doi:10.1186/s13750-016-0079-2 · Cited by 129 · cited 1×
“Manual methods of reporting are sufficient to achieve transparency and reproducibility, although automated methods are also available.(64) Examples of how to report web searching are presented in Appendix S4.”
MethodStansfield, C., Dickson, K., & Bangpan, M. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0371-9 · Cited by 122
Koffel, J. B., & Rethlefsen, M. L. (2016) · PLOS ONE · doi:10.1371/journal.pone.0163309 · Cited by 107
Briscoe, S. (2015) · BMC Research Notes · doi:10.1186/s13104-015-1079-y · Cited by 57 · cited 6×
“Relevant data about web searching were collected using an adapted version of the data extraction form used in Briscoe.8 The form was developed by inspecting the Cochrane,3 Centre for Reviews and Dissemination,13 and National Institute for Health and Care Excellence14 systematic review guidance for detail on what to re…”
BackgroundMethodResultInfluentialSelf-citationMullins, M. M., DeLuca, J. B., Crepaz, N., & Lyles, C. M. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1098 · Cited by 39
Quality of search strategies reported in systematic reviews published in stereotactic radiosurgery
Faggion, C. M., Wu, Y.-C., Tu, Y.-K., & Wasiak, J. (2016) · The British Journal of Radiology · doi:10.1259/bjr.20150878 · Cited by 31
Stansfield, C., Brunton, G., & Rees, R. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1100 · Cited by 26 · cited 1×
“This will often consist of unpublished studies and other forms of grey literature(5,6) but can also include published studies.(7) In this study, web searching is defined as searching web‐based resources that are not dedicated literature resources, such as search engines and the websites of topically relevant organisat…”
Background
Research works printed but not linked: 50
- 10.1371/journal.pmed.1000097 · DOI · Paper page
- Higgins JPT, 2011, Cochrane Handbook For Systematic Reviews Of Interventions Version 5.1.0 [Updated March 2011]
- Lefebvre C, 2011, Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 (Updated March 2011)
- Higgins J, 2016, Methodological Expectations of Cochrane Intervention Reviews
- 10.1080/14639230110075459 · DOI · Paper page
- 10.1186/s13643-015-0125-0 · DOI · Paper page
- 10.1017/CBO9780511809071 · DOI · Paper page
- 10.1126/science.1088234 · DOI · Paper page
- Booth A, 2006, “Brimful of Starlite”: toward standards for reporting literature searches, J Med Libr Assoc, 94, 421
- Centre for Reviews and Dissemination, 2008, Systematic Reviews: CRD's Guidance for Undertaking Reviews in Health Care
- 10.1016/S0140-6736(99)04149-5 · DOI · Paper page
- 10.1136/bmj.b2700 · DOI · Paper page
- 10.1016/j.jclinepi.2008.08.003 · DOI
- 10.1371/journal.pmed.0040078 · DOI · Paper page
- 10.1002/14651858.CD010321.pub2 · DOI
- 10.1002/14651858.CD009778.pub2 · DOI · Paper page
- 10.1002/14651858.CD007917.pub2 · DOI
- 10.1002/14651858.CD010342.pub3 · DOI
- 10.1002/14651858.CD006287.pub4 · doi:10.1002/14651858.cd006287.pub4
- 10.1002/14651858.CD010963.pub2 · doi:10.1002/14651858.cd010963.pub2
- 10.1002/14651858.CD011112.pub2 · DOI
- 10.1002/14651858.CD009832.pub2 · DOI
- 10.1002/14651858.CD011684.pub2 · DOI
- 10.1186/s13643-016-0371-9 · DOI · Paper page
- 10.1002/14651858.CD008576.pub4 · DOI · Paper page
- 10.1002/14651858.CD004406.pub4 · DOI
- 10.1002/14651858.CD007871.pub3 · DOI · Paper page
- 10.1002/14651858.CD007095.pub3 · DOI
- 10.1002/14651858.CD007821.pub3 · DOI
- 10.1002/14651858.CD006415.pub3 · doi:10.1002/14651858.cd006415.pub3
- 10.1002/14651858.CD001284.pub2 · doi:10.1002/14651858.cd001284.pub2
- 10.1002/14651858.CD001500.pub3 · DOI · Paper page
- 10.1002/14651858.CD011219.pub3 · DOI
- 10.1002/14651858.CD010246.pub2 · DOI
- 10.1002/14651858.CD008720.pub2 · DOI
- 10.1002/14651858.CD011193.pub2 · DOI
- 10.1002/14651858.CD010034.pub2 · DOI
- 10.1002/14651858.CD010726.pub2 · DOI
- 10.1002/14651858.CD010922.pub2 · DOI
- 10.1002/14651858.CD010669.pub2 · DOI · Paper page
- 10.1002/14651858.CD008856.pub2 · DOI
- 10.1002/14651858.CD010166.pub2 · DOI · Paper page
- 10.1002/14651858.CD009855.pub2 · DOI
- 10.1002/14651858.CD004002.pub3 · DOI · Paper page
- 10.1002/14651858.CD011175.pub2 · DOI
- 10.1002/14651858.CD012029 · DOI
- 10.1002/14651858.CD005598.pub4 · DOI
- 10.1002/14651858.CD011160.pub2 · DOI
- 10.3163/1536-5050.95.3.233 · DOI
- BBC News, 2013, Yahoo to Shut Down Pioneering Altavista Search Site [Online]
Cited by
Searching for and selecting studies
Lefebvre, C., Glanville, J., Briscoe, S., Littlewood, A., Marshall, C., Metzendorf, M., Noel‐Storr, A., Rader, T., Shokraneh, F., Thomas, J., et al. (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch4 · Cited by 647
Self-citationMeeting the review family: exploring review types and associated information retrieval requirements
Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019) · Health Information & Libraries Journal · doi:10.1111/hir.12276 · Cited by 570
Ćurković, M., & Košec, A. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0599-2 · Cited by 62
Collins, R., Silarova, B., & Clare, L. (2019) · Journal of Alzheimer’s Disease · doi:10.3233/jad-180608 · Cited by 37
“Data 299 from web-based resources and grey literature were 300 presented in line with published guidelines [31].”
Metadata
- Authors
- Briscoe, S.
- Venue
- Research Synthesis Methods
- Publisher
- Wiley
- Published
- 2017-11-09
- Volume
- 9
- Issue
- 1
- Pages
- 89-99
- ISSN
- 1759-2879 (ISSN-L)
- 1759-2887
- OpenAlex
- W2765357896
- Cited by
- 27
- References
- 12
- FWCI
- 0.65
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.