An evidence-based practice guideline for the peer review of electronic search strategies
Open accessAbstract
Objective Complex and highly sensitive electronic literature search strategies are required for systematic reviews; however, no guidelines exist for their peer review. Poor searches may fail to identify existing evidence because of inadequate recall (sensitivity) or increase the resource requirements of reviews as a result of inadequate precision. Our objective was to create an annotated checklist for electronic search strategy peer review.Study design A systematic review of the library and information retrieval literature for important elements in electronic search strategies was conducted, along with a survey of individuals experienced in systematic review searching.Results Six elements with a strong consensus as to their importance in peer review were accurate translation of the research question into search concepts, correct choice of Boolean operators and of line numbers, adequate translation of the search strategy for each database, inclusion of relevant subject headings, and absence of spelling errors. Seven additional elements had partial support and are included in this guideline.Conclusion This evidence-based guideline facilitates the improvement of search quality through peer review, and thus the improvement in quality of systematic reviews. It is relevant for librarians/information specialists, journal editors, developers of knowledge translation tools, research organizations, and funding bodies.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The original PRESS guideline was designed for peer reviewing any important search.
“Although the original PRESS guideline [1,3] was informed by research and theory from all fields of library science and was designed for peer reviewing any important search”
From PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement (2016) · cites this paper as [1]
The original PRESS guideline was informed by research and theory from all fields of library science.
“Although the original PRESS guideline [1,3] was informed by research and theory from all fields of library science and was designed for peer reviewing any important search”
From PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement (2016) · cites this paper as [1]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048 · see citing works
The well-built clinical question: A key to evidence-based decisions
(1995) · ACP Journal Club · doi:10.7326/acpjc-1995-123-3-a12 · Cited by 2,931 · see citing works
Lefebvre, C., Manheimer, E. W., & Glanville, J. (2008) · doi:10.1002/9780470712184.ch6 · Cited by 599 · see citing works
Self-citationEditorial peer review for improving the quality of reports of biomedical studies
Jefferson, T., Rudin, M., Brodney Folse, S., & Davidoff, F. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000016.pub3 · Cited by 312 · see citing works
Errors in search strategies were identified by type and frequency
Sampson, M., & McGowan, J. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.01.007 · Cited by 142 · see citing works
Self-citationPeer review for improving the quality of grant applications
Demicheli, V., & Pietrantonj, C. D. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000003.pub2 · Cited by 99 · see citing works
No consensus exists on search reporting methods for systematic reviews
Sampson, M., McGowan, J., Tetzlaff, J., Cogo, E., & Moher, D. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.10.009 · Cited by 63 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 2
- [5]5. Sampson M, McGowan J, Lefebvre C, Moher D, Grimshaw JM. PRESS: peer review of electronic search strategies. CADTH Technical Report 2008; Available at: http://cadth ca/index php/en/publica- tion/781 http://www.cadth ca/media/pdf/477_PRESS-Peer-Review- Electronic-Search-Strategies_tr_Appendices pdf.
- [10]10. McGowan J, Sampson M, Joyce J. Peer review electronic search strategy [oral presentation]. In Fifth Annual Meeting of Health Tech- nology Assessment International (HTAi), Information Retrieval Group (IRG). Montr eal July 7, 2008.
Cited by
Sorted by citations
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · see citing works
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · BMJ · doi:10.1136/bmj.b2700 · Cited by 17,648 · see citing works
Self-citationShamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & the PRISMA-P Group (2015) · BMJ · doi:10.1136/bmj.g7647 · Cited by 13,120 · see citing works
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.006 · Cited by 10,578 · see citing works
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349 · see citing works
Self-citationPRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement
McGowan, J., Sampson, M., Salzwedel, D. M., Cogo, E., Foerster, V., & Lefebvre, C. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.01.021 · Cited by 5,700 · see citing works
“Although the original PRESS guideline [1,3] was informed by research and theory from all fields of library science and was designed for peer reviewing any important search, uptake of the PRESS guideline appears to be largely confined to SR and HTA searches.”
MethodSelf-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Italian Journal of Public Health · doi:10.2427/5768 · Cited by 3,374 · see citing works
Self-citationA scoping review on the conduct and reporting of scoping reviews
Tricco, A. C., Lillie, E., Zarin, W., O’Brien, K., Colquhoun, H., Kastner, M., Levac, D., Ng, C., Sharpe, J. P., Wilson, K., et al. (2016) · BMC Medical Research Methodology · doi:10.1186/s12874-016-0116-4 · Cited by 2,250 · see citing works
Methodology in conducting a systematic review of systematic reviews of healthcare interventions
Smith, V., Devane, D., Begley, C. M., & Clarke, M. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-15 · Cited by 1,227 · see citing works
A systematic approach to searching: an efficient and complete method to develop literature searches
Bramer, W. M., De Jonge, G. B., Rethlefsen, M. L., Mast, F., & Kleijnen, J. (2018) · Journal of the Medical Library Association · doi:10.5195/jmla.2018.283 · Cited by 863 · see citing works
Maher, C. A., Lewis, L. K., Ferrar, K., Marshall, S., De Bourdeaudhuij, I., & Vandelanotte, C. (2014) · Journal of Medical Internet Research · doi:10.2196/jmir.2952 · Cited by 733 · see citing works
Pollock, D., Davies, E. L., Peters, M. D. J., Tricco, A. C., Alexander, L., McInerney, P., Godfrey, C. M., Khalil, H., & Munn, Z. (2021) · Journal of Advanced Nursing · doi:10.1111/jan.14743 · Cited by 769 · see citing works
“To evaluate the search strategy, the Peer Review of Electronic Search Strategies (PRESS) is a checklist developed by librarians is a useful tool (Sampson et al., 2009; Sampson et al., 2008).”
MethodTricco, A. C., Ivers, N. M., Grimshaw, J. M., Moher, D., Turner, L., Galipeau, J., Halperin, I., Vachon, B., Ramsay, T., Manns, B., et al. (2012) · The Lancet · doi:10.1016/s0140-6736(12)60480-2 · Cited by 652 · see citing works
Self-citationCooper, C., Booth, A., Varley-Campbell, J., Britten, N., & Garside, R. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0545-3 · Cited by 561 · see citing works
Anglemyer, A., Horvath, H. T., & Bero, L. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000034.pub2 · Cited by 534 · see citing works
Adams, J., Hillier-Brown, F. C., Moore, H. J., Lake, A. A., Araujo-Soares, V., White, M., & Summerbell, C. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0337-y · Cited by 486 · see citing works
Towards understanding the de-adoption of low-value clinical practices: a scoping review
Niven, D. J., Mrklas, K. J., Holodinsky, J. K., Straus, S. E., Hemmelgarn, B. R., Jeffs, L. P., & Stelfox, H. T. (2015) · BMC Medicine · doi:10.1186/s12916-015-0488-z · Cited by 369 · see citing works
Kastner, M., Tricco, A. C., Soobiah, C., Lillie, E., Perrier, L., Horsley, T., Welch, V., Cogo, E., Antony, J., & Straus, S. E. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-114 · Cited by 345 · see citing works
Self-citationVeroniki, A. A., Cogo, E., Rios, P., Straus, S. E., Finkelstein, Y., Kealey, R., Reynen, E., Soobiah, C., Thavorn, K., Hutton, B., et al. (2017) · BMC Medicine · doi:10.1186/s12916-017-0845-1 · Cited by 339 · see citing works
Self-citationRethlefsen, M. L., Farrell, A. M., Osterhaus Trzasko, L. C., & Brigham, T. J. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.11.025 · Cited by 331 · see citing works
Systematic review adherence to methodological or reporting quality
Pussegoda, K., Turner, L., Garritty, C., Mayhew, A., Skidmore, B., Stevens, A., Boutron, I., Sarkis-Onofre, R., Bjerre, L. M., Hróbjartsson, A., et al. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0527-2 · Cited by 322 · see citing works
Self-citationCosta, E. C., Hay, J. L., Kehler, D. S., Boreskie, K. F., Arora, R. C., Umpierre, D., Szwajcer, A., & Duhamel, T. A. (2018) · Sports Medicine · doi:10.1007/s40279-018-0944-y · Cited by 281 · see citing works
Boeker, M., Vach, W., & Motschall, E. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-131 · Cited by 225 · see citing works
An Evidence Based Checklist for the Peer Review of Electronic Search Strategies (PRESS EBC)
McGowan, J., Sampson, M., & Lefebvre, C. (2010) · Evidence Based Library and Information Practice · doi:10.18438/b8sg8r · Cited by 200 · see citing works
Self-citationErrors in search strategies used in systematic reviews and their effects on information retrieval
Salvador-Oliván, J. A., Marco-Cuenca, G., & Arquero-Avilés, R. (2019) · Journal of the Medical Library Association · doi:10.5195/jmla.2019.567 · Cited by 199 · see citing works
Cooper, H., & Koenka, A. C. (2012) · American Psychologist · doi:10.1037/a0027119 · Cited by 196 · see citing works
Soy, Red Clover, and Isoflavones and Breast Cancer: A Systematic Review
Fritz, H., Seely, D., Flower, G., Skidmore, B., Fernandes, R., Vadeboncoeur, S., Kennedy, D., Cooley, K., Wong, R., Sagar, S., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0081968 · Cited by 188 · see citing works
Kastner, M., Cardoso, R., Lai, Y., Treister, V., Hamid, J. S., Hayden, L., Wong, G., Ivers, N. M., Liu, B., Marr, S., et al. (2018) · Canadian Medical Association Journal · doi:10.1503/cmaj.171391 · Cited by 154 · see citing works
Koffel, J. B. (2015) · PLOS ONE · doi:10.1371/journal.pone.0125931 · Cited by 159 · see citing works
Tricco, A. C., Ashoor, H. M., Antony, J., Beyene, J., Veroniki, A. A., Isaranuwatchai, W., Harrington, A., Wilson, C., Tsouros, S., Soobiah, C., et al. (2014) · BMJ · doi:10.1136/bmj.g5459 · Cited by 117 · see citing works
Self-citationClinton-McHarg, T., Yoong, S. L., Tzelepis, F., Regan, T., Fielding, A., Skelton, E., Kingsland, M., Ooi, J. Y., & Wolfenden, L. (2016) · Implementation Science · doi:10.1186/s13012-016-0512-5 · Cited by 116 · see citing works
Hutton, B., Salanti, G., Chaimani, A., Caldwell, D. M., Schmid, C., Thorlund, K., Mills, E., Catalá-López, F., Turner, L., Altman, D. G., et al. (2014) · PLoS ONE · doi:10.1371/journal.pone.0092508 · Cited by 105 · see citing works
Self-citationPitfalls of Systematic Reviews and Meta-Analyses in Imaging Research
McInnes, M. D. F., & Bossuyt, P. M. M. (2015) · Radiology · doi:10.1148/radiol.2015142779 · Cited by 99 · see citing works
Use of environmental scans in health services delivery research: a scoping review
Charlton, P., Kean, T., Liu, R. H., Nagel, D. A., Azar, R., Doucet, S., Luke, A., Montelpare, W., Mears, K., & Boulos, L. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-050284 · Cited by 96 · see citing works
Rapid Normalization of Vitamin D Levels: A Meta-Analysis
McNally, J. D., Iliriani, K., Pojsupap, S., Sampson, M., O’Hearn, K., McIntyre, L., Fergusson, D., & Menon, K. (2015) · Pediatrics · doi:10.1542/peds.2014-1703 · Cited by 85 · see citing works
Self-citationReproducibility of Literature Search Reporting in Medical Education Reviews
Maggio, L. A., Tannery, N. H., & Kanter, S. L. (2011) · Academic Medicine · doi:10.1097/acm.0b013e31822221e7 · Cited by 77 · see citing works
Rice, D., Corace, K., Wolfe, D., Esmaeilisaraji, L., Michaud, A., Grima, A., Austin, B., Douma, R., Barbeau, P., Butler, C., et al. (2020) · PLOS ONE · doi:10.1371/journal.pone.0244401 · Cited by 64 · see citing works
Self-citationVaratharajan, S., Ferguson, B., Chrobak, K., Shergill, Y., Côté, P., Wong, J. J., Yu, H., Shearer, H. M., Southerst, D., Sutton, D., et al. (2016) · European Spine Journal · doi:10.1007/s00586-016-4376-9 · Cited by 59 · see citing works
Identification of problems in search strategies in <scp>C</scp>ochrane <scp>R</scp>eviews
Franco, J. V. A., Garrote, V. L., Escobar Liquitay, C. M., & Vietto, V. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1302 · Cited by 48 · see citing works
Hjørland, B. (2015) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23250 · Cited by 47 · see citing works
Mullins, M. M., DeLuca, J. B., Crepaz, N., & Lyles, C. M. (2014) · Research Synthesis Methods · doi:10.1002/jrsm.1098 · Cited by 39 · see citing works
The impact of the peer review of literature search strategies in support of rapid review reports
Spry, C., & Mierzwinski‐Urban, M. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1330 · Cited by 37 · see citing works
Tricco, A. C., Cogo, E., Ashoor, H., Perrier, L., McKibbon, K. A., Grimshaw, J. M., & Straus, S. E. (2013) · BMJ Open · doi:10.1136/bmjopen-2013-002970 · Cited by 36 · see citing works
Self-citationWelch, V. A., Ghogomu, E., Hossain, A., Awasthi, S., Bhutta, Z., Cumberbatch, C., Fletcher, R., McGowan, J., Krishnaratne, S., Kristjansson, E., et al. (2016) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.4073/csr.2016.7 · Cited by 35 · see citing works
Self-citationMenta, R., Randhawa, K., Côté, P., Wong, J. J., Yu, H., Sutton, D., Varatharajan, S., Southerst, D., D'Angelo, K., Cox, J., et al. (2015) · Journal of Manipulative and Physiological Therapeutics · doi:10.1016/j.jmpt.2015.06.002 · Cited by 33 · see citing works
Corso, M., Cancelliere, C., Mior, S., Salmi, L. R., Cedraschi, C., Nordin, M., Sci, D. M., Taylor-Vaisey, A., & Côté, P. (2022) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2021.09.007 · Cited by 24 · see citing works
Lagisz, M., Yang, Y., Young, S., & Nakagawa, S. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2024.6 · Cited by 24 · see citing works
Wiles, L., Cafarella, P., & Williams, M. T. (2015) · Respirology · doi:10.1111/resp.12419 · Cited by 21 · see citing works
- Authors
- Sampson, M., McGowan, J., Cogo, E., Grimshaw, J., Moher, D., & Lefebvre, C.
- Affiliations
- Aberystwyth University (GB)
- University of Ottawa (CA)
- National Institute for Health and Care Research (GB)
- Corresponding author
- Sampson, Margaret
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2009-09
- Volume
- 62
- Issue
- 9
- Pages
- 944-952
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W1987609279
- Semantic Scholar
- b6f91522ee14b16f8c997d053c887099691efd55
- Cited by
- 608
- References
- 7
- Citations by year
- 2026: 16 · 2025: 35 · 2024: 26 · 2023: 31 · 2022: 44 · 2021: 50 · 2020: 40 · 2019: 41 · 2018: 46 · 2017: 46 · 2016: 62 · 2015: 49 · 2014: 43 · 2013: 33 · 2012: 20
- FWCI
- 16.57
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Funders
- University of Ottawa
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Keywords
- electronic search strategy, peer review, systematic reviews, search quality, annotated checklist, Boolean operators, subject headings
- MeSH terms
- Peer Review, Review Literature as Topic, Technology Assessment, Biomedical, Bibliometrics, Databases, Bibliographic, Information Storage and Retrieval, Evidence-Based Practice
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
