CoChat
Sign Up
Cited by 604Meta-analysis and systematic reviewsscientometrics and bibliometrics researchInformation Retrieval and Search Behavior

Citationchaser: A tool for transparent and efficient forward and backward citation chasing in systematic searching

Open access
Haddaway, Neal R., Grainger, Matthew J., Gray, Charles T.·2022·Research Synthesis Methods
DOI 10.1002/jrsm.1563
Open this paper in CoChat — free accountOpen PDF

Abstract

Systematic searching aims to find all possibly relevant research from multiple sources, the basis for an unbiased and comprehensive evidence base. Along with bibliographic databases, systematic reviewers use a variety of additional methods to minimise procedural bias. Citation chasing exploits connections between research articles to identify relevant records for a review by making use of explicit mentions of one article within another. Citation chasing is a popular supplementary search method because it helps to build on the work of primary research and review authors. It does so by identifying potentially relevant studies that might otherwise not be retrieved by other search methods; for example, because they did not use the review authors' search terms in the specified combinations in their titles, abstracts, or keywords. Here, we briefly provide an overview of citation chasing as a method for systematic reviews. Furthermore, given the challenges and high resource requirements associated with citation chasing, the limited application of citation chasing in otherwise rigorous systematic reviews, and the potential benefit of identifying terminologically disconnected but semantically linked research studies, we have developed and describe a free and open source tool that allows for rapid forward and backward citation chasing. We introduce citationchaser , an R package and Shiny app for conducting forward and backward citation chasing from a starting set of articles. We describe the sources of data, the backend code functionality, and the user interface provided in the Shiny app.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/jrsm.1563

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 30

  1. Cooper C, Lovell R, Husk K, Booth A, Garside R. Supplemen- tary search methods were more effective and offered better value than bibliographic database searching: a case study from public health and environmental enhancement. Res Synth Methods. 2018;9(2):195-223.
  2. Robinson KA, Dunn AG, Tsafnat G, Glasziou P. Citation networks of related trials are often disconnected: implications for bidirec- tional citation searches. J Clin Epidemiol. 2014;67(7):793-799.
  3. Hinde S, Spackman E. Bidirectional citation searching to com- pletion: an exploration of literature searching methods. PharmacoEconomics. 2015;33(1):5-11.
  4. Janssens ACJ, Gwinn M. Novel citation-based search method for scientific literature: application to meta-analyses. BMC Med Res Methodol. 2015;15(1):84.
  5. Ogilvie D, Hamilton V, Egan M, Petticrew M. Systematic reviews of health effects of social interventions: 1. Finding the evidence: how far should you go? J Epidemiol Community Health. 2005;59(9):804-808.
  6. Eales J, Haddaway NR, Bernes C, et al. What is the effect of prescribed burning in temperate and boreal forest on biodiver- sity, beyond pyrophilous and saproxylic species? A systematic review. Environ Evid. 2018;7(1):1-33.
  7. Cooper C, Booth A, Britten N, Garside R. A comparison of results of empirical studies of supplementary search techniques and recommendations in review methodology handbooks: a methodological review. Syst Rev. 2017;6(1):1-16.
  8. Wang X, Welch V, Li M, et al. The methodological and reporting characteristics of Campbell reviews: a systematic review. Campbell Syst Rev. 2021;17(1):e1134.
  9. Chandler J, Churchill R, Higgins J, Lasserson T, Tovey D. Methodological Standards for the Conduct of New Cochrane Intervention Reviews. Vol 3. Sl: Cochrane Collaboration; 2013: 1-14.
  10. Briscoe S, Bethel A, Rogers M. Conduct and reporting of cita- tion searching in Cochrane systematic reviews: a cross- sectional study. Res Synth Methods. 2020;11(2):169-180.
  11. Wickham H. httr: Tools for Working with URLs and HTTP. 2018. R package version. 1(0):360. http://CRANR-project org/package=httr
  12. Ooms J. The jsonlite package: a practical and consistent map- ping between json data and r objects. arXiv preprint arXiv: 14032805. 2014;
  13. Martín-Martín A, Thelwall M, Orduna-Malea E, L opez- C ozar ED. Google scholar, Microsoft academic, Scopus, dimen- sions, web of science, and OpenCitations' COCI: a multi- disciplinary comparison of coverage via citations. Scientometrics. 2021;126(1):871-906.
  14. Visser M, van Eck NJ, Waltman L. Large-scale comparison of bib- liographic data sources: Scopus, web of science, dimensions, Crossref, and Microsoft academic. Quant Sci Stud. 2021;2(1):20-41.
  15. Tran F, Holland JE, Quesada N, et al. What evidence exists on the effectiveness of the techniques and management approaches used to improve the productivity of field-grown tomatoes under conditions of water-, nitrogen-and/or phosphorus-deficit? A systematic map. Environ Evid. 2021; 10(1):14. doi:10.1186/s13750-021-00229-9 · DOI
  16. O'Leary BC, Copping JP, Mukherjee N, et al. The nature and extent of evidence on methodologies for monitoring and evalu- ating marine spatial management measures in the UKand simi- lar coastal waters: a systematic map. Environ Evid. 2021;10(1): 13. doi:10.1186/s13750-021-00227-x · DOI
  17. Knight T, Price S, Bowler D, et al. How effective is 'greening' of urban areas in reducing human exposure to ground-level ozone concentrations, UV exposure and the 'urban heat Island effect'? An updated systematic review. Environ Evid. 2021; 10(1):12. doi:10.1186/s13750-021-00226-y · DOI
  18. Lugassy L, Amdouni-Boursier L, Alout H, et al. What evidence exists on the impact of specific ecosystem components and functions on infectious diseases? A systematic map. Environ Evid. 2021;10(1):11. doi:10.1186/s13750-021-00220-4 · DOI
  19. Häkkilä M, Johansson A, Sandgren T, et al. Are small protected habitat patches within boreal production forests effective in conserving species richness, abundance and community com- position? A systematic review. Environ Evid. 2021;10(1):2. doi: 10.1186/s13750-020-00216-6 · DOI
  20. Savilaakso S, Johansson A, Häkkilä M, et al. What are the effects of even-aged and uneven-aged forest management on boreal forest biodiversity in Fennoscandia and European Russia? A systematic review. Environ Evid. 2021;10(1):1. doi:10. 1186/s13750-020-00215-7 · DOI
  21. Lorick D, Macura B, Ahlström M, Grimvall A, Harder R. Effec- tiveness of struvite precipitation and ammonia stripping for recovery of phosphorus and nitrogen from anaerobic digestate: a systematic review. Environ Evid. 2020;9(1):27. doi:10.1186/ s13750-020-00211-x · DOI
  22. Dick J, Carruthers-Jones J, Carver S, Dobel AJ, Miller JD. How are nature-based solutions contributing to priority societal challenges surrounding human well-being in the United Kingdom: a systematic map. Environ Evid. 2020;9(1):25. doi:10.1186/s13750-020-00208-6 · DOI
  23. Johannesdottir SL, Macura B, McConville J, et al. What evi- dence exists on ecotechnologies for recycling carbon and nutri- ents from domestic wastewater? A systematic map. Environ Evid. 2020;9(1):24. doi:10.1186/s13750-020-00207-7 · DOI
  24. Sordello R, Ratel O, Flamerie De Lachapelle F, Leger C, Dambry A, Vanpeene S. Evidence of the impact of noise pollu- tion on biodiversity: a systematic map. Environ Evid. 2020;9(1): 20. doi:10.1186/s13750-020-00202-y · DOI
  25. Duporge I, Hodgetts T, Wang T, Macdonald DW. The spatial distribution of illegal hunting of terrestrial mammals in sub- Saharan Africa: a systematic map. Environ Evid. 2020;9(1):15. doi:10.1186/s13750-020-00195-8 · DOI
  26. Ouédraogo D-Y, Villemey A, Vanpeene S, et al. Can linear trans- portation infrastructure verges constitute a habitat and/or a corri- dor for vertebrates in temperate ecosystems? A systematic review. Environ Evid. 2020;9(1):13. doi:10.1186/s13750-020-00196-7 · DOI
  27. Rytwinski T, Harper M, Taylor JJ, et al. What are the effects of flow-regime changes on fish productivity in temperate regions? A systematic map. Environ Evid. 2020;9(1):7. doi:10.1186/ s13750-020-00190-z · DOI
  28. Arton A, Leiman A, Petrokofsky G, Toonen H, Longo CS. What do we know about the impacts of the marine stewardship council seafood ecolabelling program? A systematic map. Envi- ron Evid. 2020;9(1):6. doi:10.1186/s13750-020-0188-9 · DOI
  29. Goulas A, Belhadi D, Descamps A, et al. How effective are strategies to control the dissemination of antibiotic resistance in the environment? A systematic review. Environ Evid. 2020; 9(1):4. doi:10.1186/s13750-020-0187-x · DOI
  30. Algera DA, Rytwinski T, Taylor JJ, et al. What are the relative risks of mortality and injury for fish during downstream passage at hydroelectric dams in temperate regions? A systematic review. Environ Evid. 2020;9(1):3. doi:10.1186/s13750-020-0184-0 · DOI

Cited by

Sorted by citations

Authors
Haddaway, N. R., Grainger, M. J., & Gray, C. T.
Affiliations
  • University of Johannesburg (ZA)
  • Norwegian Institute for Nature Research (NO)
  • Newcastle University (GB)
Corresponding author
Haddaway, Neal R.
ORCID
Venue
Research Synthesis Methods
Publisher
Wiley
Published
2022-05-07
Volume
13
Issue
4
Pages
533-545
ISSN
  • 1759-2879 (ISSN-L)
  • 1759-2887
DOI
10.1002/jrsm.1563
OpenAlex
W4224987102
Cited by
604
References
3
Citations by year
2026: 182 · 2025: 182 · 2024: 133 · 2023: 86 · 2022: 19 · 2021: 1
FWCI
64.59
Type
journal article
Language
en
Open access
Licence
Locations
  • Research Synthesis Methods (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • NORA - Norwegian Open Research Archives (submittedVersion, cc-by)
  • Informationszentrum Lebenswissenschaften (ZB MED) (submittedVersion, cc-by)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
  • Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
Keywords
systematic reviews, bibliographic databases, search strategies, R package, Shiny app, free and open source software
MeSH terms
Systematic Reviews as Topic, Research Design, Databases, Bibliographic, Information Storage and Retrieval

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review