Reference management software for systematic reviews and meta-analyses: an exploration of usage and usability
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Reference management software programs enable researchers to more easily organize and manage large volumes of references typically identified during the production of systematic reviews. The purpose of this study was to determine the extent to which authors are using reference management software to produce systematic reviews; identify which programs are used most frequently and rate their ease of use; and assess the degree to which software usage is documented in published studies. METHODS: We reviewed the full text of systematic reviews published in core clinical journals indexed in ACP Journal Club from 2008 to November 2011 to determine the extent to which reference management software usage is reported in published reviews. We surveyed corresponding authors to verify and supplement information in published reports, and gather frequency and ease-of-use data on individual reference management programs. RESULTS: Of the 78 researchers who responded to our survey, 79.5% reported that they had used a reference management software package to prepare their review. Of these, 4.8% reported this usage in their published studies. EndNote, Reference Manager, and RefWorks were the programs of choice for more than 98% of authors who used this software. Comments with respect to ease-of-use issues focused on the integration of this software with other programs and computer interfaces, and the sharing of reference databases among researchers. CONCLUSIONS: Despite underreporting of use, reference management software is frequently adopted by authors of systematic reviews. The transparency, reproducibility and quality of systematic reviews may be enhanced through increased reporting of reference management software usage.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Royle, P., & Milne, R. (2003) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462303000552 · Cited by 693 · cited 2×
“Previous research has demonstrated that authors typically search multiple electronic databases to identify studies relevant for systematic reviews [11-14].”
BackgroundA systematic comparison of software dedicated to meta-analysis of causal studies
Bax, L., Yu, L.-M., Ikeda, N., & Moons, K. G. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-40 · Cited by 210
Royle, P., & Waugh, N. (2003) · Health Technology Assessment · doi:10.3310/hta7340 · Cited by 194 · cited 2×
“Previous research has demonstrated that authors typically search multiple electronic databases to identify studies relevant for systematic reviews [11-14].”
Background
Cited by
Mendes, K. D. S., Silveira, R. C. D. C. P., & Galvão, C. M. (2019) · Texto & Contexto - Enfermagem · doi:10.1590/1980-265x-tce-2017-0204 · Cited by 548
Reviewing retrieved references for inclusion in systematic reviews using EndNote
Bramer, W. M., Milic, J., & Mast, F. (2017) · Journal of the Medical Library Association · doi:10.5195/jmla.2017.111 · Cited by 264
“A survey in 2013 showed that more than half of all systematic reviewers used EndNote software [1].”
MethodChai, K. E. K., Lines, R. L. J., Gucciardi, D. F., & Ng, L. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01635-3 · Cited by 190
(2023) · doi:10.1186/s12915-023-01686-z · Cited by 103
Hair, K., Bahor, Z., Macleod, M., Liao, J., & Sena, E. S. (2021) · doi:10.1101/2021.05.04.442412 · Cited by 103
Metadata
- Authors
- Lorenzetti, D. L., & Ghali, W. A.
- Venue
- BMC Medical Research Methodology
- Published
- 2013-11-15
- Volume
- 13
- Issue
- 1
- Pages
- 141-141
- ISSN
- 1471-2288 (ISSN-L)
- OpenAlex
- W2146188179
- Cited by
- 102
- References
- 3
- FWCI
- 0.85
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Reference management software for systematic reviews and meta-analyses: an exploration of usage and usability. https://app.cochat.ai/papers/10.1186/1471-2288-13-141
- The paper
- Lorenzetti, D. L., & Ghali, W. A. (2013). Reference management software for systematic reviews and meta-analyses: an exploration of usage and usability. BMC Medical Research Methodology. https://doi.org/10.1186/1471-2288-13-141
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.