A practical guide to question formation, systematic searching and study screening for literature reviews in ecology and evolution
Open accessThe DOI matches its Crossref record; 18 of the 18 references it cites were found in a registry or an index.
Abstract
Well‐conducted systematic reviews are invaluable for synthesising research findings. The conclusions of a review depend on how the research question was formulated, how relevant studies were found and how studies were selected for synthesis. Here, we present a practical guide for ecologists and evolutionary biologists on formulating a question for a systematic review, and finding a representative sample of research findings. We explain the steps involved using a worked example and practical training exercises. Throughout this guide we share tricks of the trade, included rules of thumb and software that we have found useful. We hope our paper helps demystify the systematic search process and encourages more researchers to adopt a systematic and reproducible approach when searching the literature.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
PRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223
Rayyan-a web and mobile app for systematic reviews.
Ouzzani, M., Hammady, H., Fedorowicz, Z., & Elmagarmid, A. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0384-4 · Cited by 26,092
Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., et al. (2017) · BMJ · doi:10.1136/bmj.j4008 · Cited by 10,636
Bibliometric Methods in Management and Organization
Zupic, I., & Čater, T. (2015) · Organizational Research Methods · doi:10.1177/1094428114562629 · Cited by 7,495
The journal coverage of Web of Science and Scopus: A comparative analysis
Mongeon, P., & Paul-Hus, A. (2016) · Scientometrics · doi:10.1007/s11192-015-1765-5 · Cited by 5,274
The well-built clinical question: A key to evidence-based decisions
(1995) · ACP Journal Club · doi:10.7326/acpjc-1995-123-3-a12 · Cited by 2,931 · cited 1×
“Once familiar with the literature, you can specify your research question using the PICO/PECO framework (Richardson et al., 1995), which identifies four components in a research question: Population, Intervention/Exposure, Comparator and Outcome.”
MethodGusenbauer, M., & Haddaway, N. R. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1378 · Cited by 2,379 · cited 4×
“Given these shortcomings, researchers should use Google Scholar as a supplementary rather than as the main search platform (Gusenbauer & Haddaway, 2020).”
BackgroundMethodInfluentialThe preregistration revolution
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708274114 · Cited by 1,998 · cited 1×
“As part of the credibility revolution, there is a growing emphasis on registering one's study (Nosek et al., 2018; Parker et al., 2019).”
BackgroundThe nuts and bolts of PROSPERO: an international prospective register of systematic reviews
Booth, A., Clarke, M., Dooley, G., Ghersi, D., Moher, D., Petticrew, M., & Stewart, L. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-2 · Cited by 1,433 · cited 1×
“Second, systematic review methods are iterative (Booth et al., 2012).”
Handbook of Meta-analysis in Ecology and Evolution
(2013) · doi:10.1515/9781400846184 · Cited by 1,183
McAuley, L., Pham, B., Tugwell, P., & Moher, D. (2000) · The Lancet · doi:10.1016/s0140-6736(00)02786-0 · Cited by 812 · cited 1×
“The inclusion of grey literature is necessary if publication bias exists in the published literature (Auger, 1998; McAuley et al., 2000).”
BackgroundThe International Weight Management in Pregnancy (i-WIP) Collaborative Group (2017) · BMJ · doi:10.1136/bmj.j3119 · Cited by 529
Kohl, C., McIntosh, E. J., Unger, S., Haddaway, N. R., Kecke, S., Schiemann, J., & Wilhelm, R. (2018) · Environmental Evidence · doi:10.1186/s13750-018-0115-5 · Cited by 471
Uses and misuses of meta‐analysis in plant ecology
Koricheva, J., & Gurevitch, J. (2014) · Journal of Ecology · doi:10.1111/1365-2745.12224 · Cited by 460
Self-citationAll in the Family: systematic reviews, rapid reviews, scoping reviews, realist reviews, and more
Moher, D., Stewart, L., & Shekelle, P. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0163-7 · Cited by 288
revtools: An R package to support article screening for evidence synthesis
Westgate, M. J. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1374 · Cited by 261
Information Sources in Grey Literature
Auger, P. (1998) · doi:10.1515/9783110977233 · Cited by 56
Hjørland, B. (2015) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23250 · Cited by 47
Cited by
The global human impact on biodiversity
Keck, F., Peller, T., Alther, R., Barouillet, C., Blackman, R., Capo, E., Chonova, T., Couton, M., Fehlinger, L., Kirschner, D., et al. (2025) · Nature · doi:10.1038/s41586-025-08752-2 · Cited by 288
A Guided Tour of Phylogenetic Comparative Methods for Studying Trait Evolution
Cornwallis, C. K., & Griffin, A. S. (2024) · Annual Review of Ecology, Evolution, and Systematics · doi:10.1146/annurev-ecolsys-102221-050754 · Cited by 37
Lagisz, M., Yang, Y., Young, S., & Nakagawa, S. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2024.6 · Cited by 24
Self-citation
Metadata
- Authors
- Foo, Y. Z., O'Dea, R. E., Koricheva, J., Nakagawa, S., & Lagisz, M.
- Venue
- Methods in Ecology and Evolution
- Publisher
- Wiley
- Published
- 2021-06-30
- Volume
- 12
- Issue
- 9
- Pages
- 1705-1720
- ISSN
- 2041-210X (ISSN-L)
- OpenAlex
- W3169648480
- Cited by
- 192
- References
- 18
- FWCI
- 14.83
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.