BEYOND MEDLINE: Reducing Bias Through Extended Systematic Review Search
The DOI matches its Crossref record.
Abstract
Objective: To evaluate the sensitivity and precision of various extended search methods in identifying randomized controlled trials (RCTs) for systematic reviews. Method: Prospective analysis of extended search methods (specialized databases or trial registries, reference lists, hand-searching, personal communication, and Internet) used in two systematic reviews of RCTs. The gold standard was the total number of RCTs identified by major databases (MEDLINE, EMBASE, etc.) and extended search strategies combined. Sensitivity was the proportion of all known RCTs identified by any extended search method. Precision reflected the proportion of all items uncovered by any extended search method that actually were RCTs. Results: The extended search identified 94 additional RCTs for the systematic reviews beyond those identified with the major databases. Specialized databases and trial registries had the highest sensitivity and precision for the lipid-lowering project (13.6% and 52.7%, respectively; p < .05) followed by scanning of reference lists (7.2% sensitivity and 41.9% precision; p < .05). Hand-searching was more effective than personal communication and Internet searching (1.7% sensitivity and 12.2% precision; p < .05). The acupuncture project had slightly different results, with the specialized databases and trial registries tied with the review of reference lists for highest sensitivity (14.2%). The precision followed the same trend as the lipid-lowering project (17.6% specialized databases; 8.3% reference lists; p < .05). A post-hoc analysis showed that 75 of the 94 RCTs were indexed in the major databases but missed by the major database search. Conclusions: Extended searching identified additional RCTs for the systematic reviews beyond those found in major databases. Specialized databases and trial registries were most effective. An important number of RCTs were missed by the major database search. Timing and accuracy of indexing may explain this finding. The definitive measure, whether there is an association between the method used to uncover RCTs, the quality of the items uncovered and their impact on systematic review results, is yet to be determined.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cited by
Expediting systematic reviews: methods and implications of rapid reviews
Ganann, R., Ciliska, D., & Thomas, H. (2010) · Implementation Science · doi:10.1186/1748-5908-5-56 · Cited by 752
Checking reference lists to find additional studies for systematic reviews
Horsley, T., Dingwall, O., & Sampson, M. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000026.pub2 · Cited by 443
Lahti, M., Hätönen, H., & Välimäki, M. (2014) · International Journal of Nursing Studies · doi:10.1016/j.ijnurstu.2012.12.017 · Cited by 407
Papaioannou, D., Sutton, A., Carroll, C., Booth, A., & Wong, R. (2010) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2009.00863.x · Cited by 318
Kung, J. (2010) · The Open Dentistry Journal · doi:10.2174/1874210601004020084 · Cited by 329
Metadata
- Authors
- Savoie, I., Helmer, D., Green, C. J., & Kazanjian, A.
- Venue
- International Journal of Technology Assessment in Health Care
- Publisher
- Cambridge University Press (CUP)
- Published
- 2003-01
- Volume
- 19
- Issue
- 1
- Pages
- 168-178
- ISSN
- 0266-4623 (ISSN-L)
- 1471-6348
- OpenAlex
- W164210851
- Cited by
- 105
- References
- 0
- FWCI
- 6.78
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: BEYOND MEDLINE: Reducing Bias Through Extended Systematic Review Search. https://app.cochat.ai/papers/10.1017/s0266462303000163
- The paper
- Savoie, I., Helmer, D., Green, C. J., & Kazanjian, A. (2003). BEYOND MEDLINE: Reducing Bias Through Extended Systematic Review Search. International Journal of Technology Assessment in Health Care. https://doi.org/10.1017/s0266462303000163
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.