Cited by 136Meta-analysis and systematic reviewsscientometrics and bibliometrics researchAcademic Publishing and Open Access

Cochrane Centralised Search Service showed high sensitivity identifying randomized controlled trials: A retrospective analysis

Open access
Noel-Storr, A.H., Dooley, G., Wisniewski, S., Glanville, J., Thomas, J., Cox, S., Featherstone, R., Foxlee, R.·2020·Journal of Clinical Epidemiology

Abstract

BACKGROUND AND OBJECTIVES: The Cochrane Central Register of Controlled Trials (CENTRAL) is compiled from a number of sources, including PubMed and Embase. Since 2017, we have increased the number of sources feeding into CENTRAL and improved the efficiency of our processes through the use of application programming interfaces, machine learning, and crowdsourcing.Our objectives were twofold: (1) Assess the effectiveness of Cochrane's centralized search and screening processes to correctly identify references to published reports which are eligible for inclusion in Cochrane systematic reviews of randomized controlled trials (RCTs). (2) Identify opportunities to improve the performance of Cochrane's centralized search and screening processes to identify references to eligible trials. METHODS: We identified all references to RCTs (either published journal articles or trial registration records) with a publication or registration date between 1st January 2017 and 31st December 2018 that had been included in a Cochrane intervention review. We then viewed an audit trail for each included reference to determine if it had been identified by our centralized search process and subsequently added to CENTRAL. RESULTS: We identified 650 references to included studies with a publication year of 2017 or 2018. Of those, 634 (97.5%) had been captured by Cochrane's Centralised Search Service. Sixteen references had been missed by the Cochrane's Centralised Search Service: six had PubMed-not-MEDLINE status, four were missed by the centralized Embase search, three had been misclassified by Cochrane Crowd, one was from a journal not indexed in MEDLINE or Embase, one had only been added to Embase in 2019, and one reference had been rejected by the automated RCT machine learning classifier. Of the sixteen missed references, eight were the main or only publication to the trial in the review in which it had been included. CONCLUSION: This analysis has shown that Cochrane's centralized search and screening processes are highly sensitive. It has also helped us to understand better why some references to eligible RCTs have been missed. The CSS is playing a critical role in helping to populate CENTRAL and is moving us toward making CENTRAL a comprehensive repository of RCTs.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.jclinepi.2020.08.008

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. The Cochrane Library. CENTRAL Creation: how CENTRAL is created. Available at https://www.cochranelibrary.com/central/central-creation. Accessed September 14, 2020.

Cited by

Sorted by citations

Authors
Noel-Storr, A. H., Dooley, G., Wisniewski, S., Glanville, J., Thomas, J., Cox, S., Featherstone, R., & Foxlee, R.
Affiliations
  • University of Oxford (GB)
  • University of York (GB)
  • University College London (GB)
  • Cochrane (GB)
Corresponding author
Noel-Storr, A.H.
Venue
Journal of Clinical Epidemiology
Publisher
Elsevier BV
Published
2020-11
Volume
127
Pages
142-150
ISSN
0895-4356 (ISSN-L)
OpenAlex
W3048480974
Cited by
136
References
18
Citations by year
2026: 10 · 2025: 30 · 2024: 27 · 2023: 21 · 2022: 26 · 2021: 19 · 2020: 3
FWCI
10.40
Type
journal article
Language
en
Open access
Locations
  • Journal of Clinical Epidemiology (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • UCL Discovery (University College London) (submittedVersion)
Funders
Medical Research Council (MR/J005037/1)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
  • Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
Keywords
randomized controlled trials, search sensitivity, MEDLINE, Embase, crowdsourcing, machine learning classifiers, audit trail
MeSH terms
Machine Learning, Systematic Reviews as Topic, Data Aggregation, Humans, Registries, Retrospective Studies, Sensitivity and Specificity, Randomized Controlled Trials as Topic, Databases, Bibliographic, MEDLINE, Information Storage and Retrieval, PubMed, Crowdsourcing

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review