The DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
An automated approach to identifying search terms for systematic reviews using keyword co‐occurrence networks
Abstract
Systematic review, meta‐analysis and other forms of evidence synthesis are critical to strengthen the evidence base concerning conservation issues and to answer ecological and evolutionary questions. Synthesis lags behind the pace of scientific publishing, however, due to time and resource costs which partial automation of evidence synthesis tasks could reduce. Additionally, current methods of retrieving evidence for synthesis are susceptible to bias towards studies with which researchers are familiar. In fields that lack standardized terminology encoded in an ontology, including ecology and evolution, research teams can unintentionally exclude articles from the review by omitting synonymous phrases in their search terms. To combat these problems, we developed a quick, objective, reproducible method for generating search strategies that uses text mining and keyword co‐occurrence networks to identify the most important terms for a review. The method reduces bias in search strategy development because it does not rely on a predetermined set of articles and can improve search recall by identifying synonymous terms that research teams might otherwise omit. When tested against the search strategies used in published environmental systematic reviews, our method performs as well as the published searches and retrieves gold‐standard hits that replicated versions of the original searches do not. Because the method is quasi‐automated, the amount of time required to develop a search strategy, conduct searches, and assemble results is reduced from approximately 17–34 hr to under 2 hr. To facilitate use of the method for environmental evidence synthesis, we implemented the method in the R package litsearchr, which also contains a suite of functions to improve efficiency of systematic reviews by automatically deduplicating and assembling results from separate databases.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An algorithm for suffix stripping
Porter, M. F. (1980) · Program · doi:10.1108/eb046814 · Cited by 8,096 · cited 1×
“To stem terms and capture additional word forms when searching databases, litsearchr uses a stemming algo - rithm (Porter, 1980) to reduce each word to its root form if its stem is at least four characters long, which is the length suggested by in - formation specialists to balance efficiency with…”
MethodPrimary, Secondary, and Meta-Analysis of Research
GLASS, G. V. (1976) · Educational Researcher · doi:10.3102/0013189x005010003 · Cited by 4,890 · cited 1×
“…to evidence synthesis, whereby a formal strategy is used to study the way in which a topic has been stud - ied, originated in clinical fields (Glass, 1976; Hunt, 1997; Smith & Glass, 1977), but have been co‐opted by ecologists in recognition of the need for more rigorous methods of reviewing…”
BackgroundThe well-built clinical question: A key to evidence-based decisions
(1995) · ACP Journal Club · doi:10.7326/acpjc-1995-123-3-a12 · Cited by 2,931
Meta-analysis of psychotherapy outcome studies.
Smith, M. L., & Glass, G. V. (1977) · American Psychologist · doi:10.1037/0003-066x.32.9.752 · Cited by 2,154 · cited 1×
“…whereby a formal strategy is used to study the way in which a topic has been stud - ied, originated in clinical fields (Glass, 1976; Hunt, 1997; Smith & Glass, 1977), but have been co‐opted by ecologists in recognition of the need for more rigorous methods of reviewing the litera - ture…”
MethodHandbook of Meta-analysis in Ecology and Evolution
(2013) · doi:10.1515/9781400846184 · Cited by 1,183
Guidelines for systematic review in Conservation and environmental management
PULLIN, A. S., & STEWART, G. B. (2006) · Conservation Biology · doi:10.1111/j.1523-1739.2006.00485.x · Cited by 1,137
Automatic Keyword Extraction from Individual Documents
Rose, S., Engel, D., Cramer, N., & Cowley, W. (2010) · Text Mining · doi:10.1002/9780470689646.ch1 · Cited by 1,095
Haddaway, N. R., Macura, B., Whaley, P., & Pullin, A. S. (2018) · Environmental Evidence · doi:10.1186/s13750-018-0121-7 · Cited by 946
Su, H.-N., & Lee, P.-C. (2010) · Scientometrics · doi:10.1007/s11192-010-0259-8 · Cited by 916
A systematic approach to searching: an efficient and complete method to develop literature searches
Bramer, W. M., De Jonge, G. B., Rethlefsen, M. L., Mast, F., & Kleijnen, J. (2018) · Journal of the Medical Library Association · doi:10.5195/jmla.2018.283 · Cited by 863
Cited by
McGuinness, L. A., & Higgins, J. P. T. (2021) · Research Synthesis Methods · doi:10.1002/jrsm.1411 · Cited by 5,579
Conducting systematic literature reviews and bibliometric analyses
Linnenluecke, M. K., Marrone, M., & Singh, A. K. (2020) · Australian Journal of Management · doi:10.1177/0312896219877678 · Cited by 1,649
revtools: An R package to support article screening for evidence synthesis
Westgate, M. J. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1374 · Cited by 261
Resource use during systematic review production varies widely: a scoping review
Nussbaumer-Streit, B., Ellen, M., Klerings, I., Sfetcu, R., Riva, N., Mahmić-Kaknjo, M., Poulentzas, G., Martinez, P., Baladia, E., Ziganshina, L. E., et al. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.05.019 · Cited by 98
Rangel-de Lázaro, G., & Duart, J. M. (2023) · Sustainability · doi:10.3390/su15043507 · Cited by 95
Metadata
- Authors
- Grames, E. M., Stillman, A. N., Tingley, M. W., & Elphick, C. S.
- Venue
- Methods in Ecology and Evolution
- Publisher
- Wiley
- Published
- 2019-08-11
- Volume
- 10
- Issue
- 10
- Pages
- 1645-1654
- ISSN
- 2041-210X (ISSN-L)
- OpenAlex
- W2963483681
- Cited by
- 377
- References
- 44
- FWCI
- 4.36
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: An automated approach to identifying search terms for systematic reviews using keyword co‐occurrence networks. https://app.cochat.ai/papers/10.1111/2041-210x.13268
- The paper
- Grames, E. M., Stillman, A. N., Tingley, M. W., & Elphick, C. S. (2019). An automated approach to identifying search terms for systematic reviews using keyword co‐occurrence networks. Methods in Ecology and Evolution. https://doi.org/10.1111/2041-210x.13268
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.