Electronic search strategies to identify reports of cluster randomized trials in MEDLINE: low precision will improve with adherence to reporting standards
Open accessThe DOI matches its Crossref record; 24 of the 25 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Cluster randomized trials (CRTs) present unique methodological and ethical challenges. Researchers conducting systematic reviews of CRTs (e.g., addressing methodological or ethical issues) require efficient electronic search strategies (filters or hedges) to identify trials in electronic databases such as MEDLINE. According to the CONSORT statement extension to CRTs, the clustered design should be clearly identified in titles or abstracts; however, variability in terminology may make electronic identification challenging. Our objectives were to (a) evaluate sensitivity ("recall") and precision of a well-known electronic search strategy ("randomized controlled trial" as publication type) with respect to identifying CRTs, (b) evaluate the feasibility of new search strategies targeted specifically at CRTs, and (c) determine whether CRTs are appropriately identified in titles or abstracts of reports and whether there has been improvement over time. METHODS: We manually examined a wide range of health journals to identify a gold standard set of CRTs. Search strategies were evaluated against the gold standard set, as well as an independent set of CRTs included in previous systematic reviews. RESULTS: The existing strategy (randomized controlled trial.pt) is sensitive (93.8%) for identifying CRTs, but has relatively low precision (9%, number needed to read 11); the number needed to read can be halved to 5 (precision 18.4%) by combining with cluster design-related terms using the Boolean operator AND; combining with the Boolean operator OR maximizes sensitivity (99.4%) but would require 28.6 citations read to identify one CRT. Only about 50% of CRTs are clearly identified as cluster randomized in titles or abstracts; approximately 25% can be identified based on the reported units of randomization but are not amenable to electronic searching; the remaining 25% cannot be identified except through manual inspection of the full-text article. The proportion of trials clearly identified has increased from 28% between the years 2000-2003, to 60% between 2004-2007 (absolute increase 32%, 95% CI 17 to 47%). CONCLUSIONS: CRTs should include the phrase "cluster randomized trial" in titles or abstracts; this will facilitate more accurate indexing of the publication type by reviewers at the National Library of Medicine, and efficient textword retrieval of the subset employing cluster randomization.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Such volume introduces opportunity for human error in the screening process [3–5].”
From An evaluation of DistillerSR’s machine learning-based prioritization tool for title/abstract screening – impact on reviewer-relevant outcomes (2020) · cites this paper as [4]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
Consort 2010 statement: extension to cluster randomised trials
Campbell, M. K., Piaggio, G., Elbourne, D. R., Altman, D. G., & for the CONSORT Group (2012) · BMJ · doi:10.1136/bmj.e5661 · Cited by 3,510
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755
CONSORT statement: extension to cluster randomised trials
Campbell, M. K., Elbourne, D. R., & Altman, D. G. (2004) · BMJ · doi:10.1136/bmj.328.7441.702 · Cited by 2,557
Design and Analysis of Cluster Randomization Trials in Health Research
DONNER, A. P., & Klar, N. S. (2000) · Cited by 1,236
Self-citationDesign and Analysis of Group-Randomized Trials
Sonnad, S. (1999) · Medicine & Science in Sports & Exercise · doi:10.1097/00005768-199902000-00025 · Cited by 1,125
Design and Analysis of Group-Randomized Trials
Murray, D. M. (2007) · doi:10.1093/oso/9780195120363.001.0001 · Cited by 1,116
How to identify randomized controlled trials in MEDLINE: ten years on.
Glanville, J., Lefebvre, C., Miles, J. N. V., & Camosso‐Stefinovic, J. (2006) · Europe PMC (PubMed Central) · Cited by 833
Haynes, R. B., & Wilczynski, N. L. (2004) · BMJ · doi:10.1136/bmj.38068.557998.ee · Cited by 547
Design and Analysis of Cluster Randomization Trials in Health Research
Davis, W. W. (2001) · Journal of the American Statistical Association · doi:10.1198/jasa.2001.s433 · Cited by 442
Cited by
Consort 2010 statement: extension to cluster randomised trials
Campbell, M. K., Piaggio, G., Elbourne, D. R., Altman, D. G., & for the CONSORT Group (2012) · BMJ · doi:10.1136/bmj.e5661 · Cited by 3,510
Ivers, N. M., Taljaard, M., Dixon, S., Bennett, C., McRae, A., Taleban, J., Skea, Z., Brehaut, J. C., Boruch, R. F., Eccles, M. P., et al. (2011) · BMJ · doi:10.1136/bmj.d5886 · Cited by 193
Self-citationHamel, C., Kelly, S. E., Thavorn, K., Rice, D. B., Wells, G. A., & Hutton, B. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01129-1 · Cited by 140
Are missing data adequately handled in cluster randomised trials? A systematic review and guidelines
Diaz‐Ordaz, K., Kenward, M. G., Cohen, A., Coleman, C. L., & Eldridge, S. M. (2014) · Clinical Trials · doi:10.1177/1740774514537136 · Cited by 65
“PubMed was searched for relevant reports in June 2012, using a search strategy previously used by our research team and which compares well with other cluster randomised trial searches.(13) The full electronic search strategy is given in Box A in the Supplemental File.”
MethodA brief history of the cluster randomised trial design
Moberg, J., & Kramer, M. S. (2015) · Journal of the Royal Society of Medicine · doi:10.1177/0141076815582303 · Cited by 77
“Despite extension of the CONSORT statement on reporting guidelines for cluster randomised trials,(62) the titles or abstracts of 50% of cluster randomised trials still fail to indicate this.(73) Conversely, other papers report to be cluster randomised trials in their titles but, on careful inspection, are clearly not…”
Method
Metadata
- Authors
- Taljaard, M., McGowan, J., Grimshaw, J. M., Brehaut, J. C., McRae, A., Eccles, M. P., & Donner, A.
- Affiliations
- Ottawa Hospital Research Institute (CA)
- University of Ottawa (CA)
- Western University (CA)
- Newcastle University (GB)
- Corresponding author
- Taljaard, Monica
- Venue
- BMC Medical Research Methodology
- Published
- 2010-02-16
- Volume
- 10
- Issue
- 1
- Pages
- 15-15
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC2833170
- OpenAlex
- W2051284664
- Cited by
- 62
- References
- 25
- Citations by year
- 2026: 1 · 2025: 2 · 2023: 4 · 2022: 13 · 2021: 4 · 2020: 7 · 2019: 1 · 2018: 3 · 2017: 4 · 2016: 2 · 2015: 3 · 2014: 5 · 2013: 3 · 2012: 5
- FWCI
- 3.98
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/2.0 (tdm)
- Locations
- BMC Medical Research Methodology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Scholarship@Western (Western University) (submittedVersion)
- uO Research (University of Ottawa) (submittedVersion, public-domain)
- Newcastle University ePrints (Newcastle Univesity) (cc-by)
- Funders
- Canadian Institutes of Health Research
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- electronic search strategy, MEDLINE, Boolean operators, CONSORT extension, trial reporting standards
- MeSH terms
- Abstracting and Indexing, Reproducibility of Results, Randomized Controlled Trials as Topic, MEDLINE, Information Storage and Retrieval
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Electronic search strategies to identify reports of cluster randomized trials in MEDLINE: low precision will improve with adherence to reporting standards. https://app.cochat.ai/papers/10.1186/1471-2288-10-15
- The paper
- Taljaard, M., McGowan, J., Grimshaw, J. M., Brehaut, J. C., McRae, A., Eccles, M. P., & Donner, A. (2010). Electronic search strategies to identify reports of cluster randomized trials in MEDLINE: low precision will improve with adherence to reporting standards. BMC Medical Research Methodology. https://doi.org/10.1186/1471-2288-10-15
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.