Development of a highly sensitive search strategy for the retrieval of reports of controlled trials using PubMed

Robinson, Karen A; Dickersin, Kay

doi:10.1093/ije/31.1.150

Cited by 833Meta-analysis and systematic reviewsInformation Retrieval and Search Behaviorscientometrics and bibliometrics research

Development of a highly sensitive search strategy for the retrieval of reports of controlled trials using PubMed

Robinson, Karen A, Dickersin, Kay·2002·International Journal of Epidemiology

DOI 10.1093/ije/31.1.150

Open this paper in CoChat — free account

Abstract

OBJECTIVE: To develop, through revision of the Cochrane Collaboration search strategy for OVID-MEDLINE, a highly sensitive search strategy to retrieve reports of controlled trials using PubMed. METHODS: The original highly sensitive Cochrane strategy was revised to take into account additional Medical Subject Headings (MeSH) and other terminology as well as the current unique features of PubMed. We compared the retrieval of the revised strategy with that of the original Cochrane strategy before and after translation of the strategies into PubMed format. Finally, we used a gold standard database of reports of controlled trials identified by electronic and hand search of selected journals to test the revised strategy in PubMed format. RESULTS: The revised strategy included a search statement modified for increased precision, and added 'Cross-over Studies' as a MeSH term and the term 'latin square' as a text word. Compared to the original Cochrane strategy, the revised strategy identified 53 additional reports of controlled trials accessing MEDLINE through OVID. When the revised strategy and original Cochrane strategy were translated into PubMed format, the revised strategy retrieved 90 reports of controlled trials not identified by the original strategy. Finally, the revised strategy in PubMed format retrieved all of the reports of controlled trials in the gold standard database. Ninety-eight per cent of the gold standard reports of controlled trials were retrieved by Phase 1 of the optimal PubMed search strategy. CONCLUSIONS: Failure to identify all relevant trials for systematic review could result in bias. We developed a highly sensitive search strategy for the retrieval of reports of controlled trials for use with PubMed that retrieves more relevant citations (greater sensitivity) and fewer non-relevant citations (greater precision) than the original Cochrane search strategy.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1093/ije/31.1.150

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

The systematic review was reported according to the PRISMA Statement and followed the Cochrane Handbook for Systematic Reviews of Interventions version 5.1.0.
“This systematic review is reported according to the PRISMA Statement [19]. We also followed the recommendations described in the Cochrane Handbook for Systematic Reviews of Interventions version 5.1.0 [20].”
From Effects of Supervised Multimodal Exercise Interventions on Cancer-Related Fatigue: Systematic Review and Meta-Analysis of Randomized Controlled Trials (2015) · cites this paper as [20]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Systematic Reviews: Identifying relevant studies for systematic reviews
Dickersin, K., Scherer, R. W., & Lefebvre, C. (1994) · BMJ · doi:10.1136/bmj.309.6964.1286 · Cited by 2,255 · see citing works
Self-citation
Identifying Randomised Trials
Lefebvre, C., & Clarke, M. J. (2001) · doi:10.1002/9780470693926.ch4 · Cited by 115 · see citing works
Synthesis of the literature on the effectiveness of regional anesthesia for cataract surgery11This article is based on research conducted by the Johns Hopkins University under contract to the Agency for Healthcare Research and Quality, formerly the Agency for Health Care Policy and Research (contract no.: 290-97-0006). The authors of this article are responsible for its contents, including any clinical or treatment recommendations. No statement in this article should be construed as an official position of the Agency for Healthcare Research and Quality or the United States Department of Health and Human Services.
Friedman, D. S., Bass, E. B., Lubomski, L. H., Fleisher, L. A., Kempen, J. H., Magaziner, J. L., Sprintz, M., Robinson, K. A., & Schein, O. D. (2001) · Ophthalmology · doi:10.1016/s0161-6420(00)00597-2 · Cited by 74 · see citing works
Self-citation
The methodologic quality of clinical trials on regional anesthesia for cataract surgery11This article is based on research conducted by the Johns Hopkins University under contract to the Agency for Healthcare Research and Quality, formerly the Agency for Health Care Policy and Research (contract no.: 290-97-0006). The authors of this article are responsible for its contents, including any clinical or treatment recommendations. No statement in this article should be construed as an official position of the Agency for Healthcare Research and Quality or the United States Department of Health and Human Services.
Friedman, D. S., Bass, E. B., Lubomski, L. H., et al. (2001) · Ophthalmology · doi:10.1016/s0161-6420(00)00596-0 · Cited by 14 · see citing works
Treatment of coexisting cataract and glaucoma
Jampel, H. D., Lubomski, L., Friedman, D., et al. (2003) · Cited by 12 · see citing works

Flagged first, then by citations

Research works printed but not linked: 3

Robinson KA, Hinegardner PG, Lansing P. Development of an optimal search strategy for the retrieval of controlled trials using PubMed [abstract]. Proceedings of the 6th International Cochrane Colloquium; 1998 Oct 22-26. Abstr Workshops Sci Sess Int Cochrane Coll 1998;6:85.
Friedman DS, Bass EB, Lubomski LH et al. Synthesis of the literature on the effectiveness of regional anesthesia for cataract surgery. Ophthalmology 2001;108:519-29.
Friedman DS, Bass EB, Lubomski LH et al. The methodologic quality of clinical trials on regional anesthesia for cataract surgery. Ophthalmology 2001;108:530-41.

Sorted by citations

Updated Method Guidelines for Systematic Reviews in the Cochrane Collaboration Back Review Group
van Tulder, M., Furlan, A., Bombardier, C., & Bouter, L. (2003) · Spine · doi:10.1097/01.brs.0000065484.95996.af · Cited by 2,180 · see citing works
Physical Activity Advice Only or Structured Exercise Training and Association With HbA <sub>1c</sub> Levels in Type 2 Diabetes: A Systematic Review and Meta-analysis
Umpierre, D. (2011) · JAMA · doi:10.1001/jama.2011.576 · Cited by 1,358 · see citing works
Benefits and Risks of Tight Glucose Control in Critically Ill Adults: A Meta-analysis
Wiener, R. S., Wiener, D. C., & Larson, R. J. (2008) · JAMA · doi:10.1001/jama.300.8.933 · Cited by 200 · see citing works
Dissemination and publication of research findings: an updated review of related biases
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I. (2010) · Health Technology Assessment · doi:10.3310/hta14080 · Cited by 1,039 · see citing works
Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes
Boutron, I. (2010) · JAMA · doi:10.1001/jama.2010.651 · Cited by 842 · see citing works
Exercise for improving balance in older people
Howe, T. E., Rochester, L., Neil, F., Skelton, D. A., & Ballinger, C. (2011) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004963.pub3 · Cited by 809 · see citing works
Very-low-carbohydrate ketogenic diet <i>v.</i> low-fat diet for long-term weight loss: a meta-analysis of randomised controlled trials
Bueno, N. B., de Melo, I. S. V., de Oliveira, S. L., & da Rocha Ataide, T. (2013) · British Journal of Nutrition · doi:10.1017/s0007114513000548 · Cited by 778 · see citing works
Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors
Chan, A.-W., & Altman, D. G. (2005) · BMJ · doi:10.1136/bmj.38356.424606.8f · Cited by 613 · see citing works
Applied behavior analytic intervention for autism in early childhood: meta-analysis, meta-regression and dose-response meta-analysis of multiple outcomes
Virués-Ortega, J. (2010) · Clinical Psychology Review · doi:10.1016/j.cpr.2010.01.008 · Cited by 615 · see citing works
Pharmacotherapy for post traumatic stress disorder (PTSD)
Stein, D. J., Ipser, J. C., Seedat, S., Sager, C., & Amos, T. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002795.pub2 · Cited by 519 · see citing works
The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed
Hopewell, S., Dutton, S., Yu, L.-M., Chan, A.-W., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c723 · Cited by 504 · see citing works
Systematic review automation technologies
Tsafnat, G., Glasziou, P., Choong, M. K., Dunn, A., Galgani, F., & Coiera, E. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-74 · Cited by 472 · see citing works
“, MeSH terms [49]) and search strategies [30,34,50] most suitable for the research question.”
Background
Ultrasound guidance versus anatomical landmarks for internal jugular vein catheterization
Brass, P., Hellmich, M., Kolodziej, L., Schick, G., & Smith, A. F. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006962.pub2 · Cited by 476 · see citing works
Efficacy of multicomponent treatment in fibromyalgia syndrome: A meta‐analysis of randomized controlled clinical trials
Häuser, W., Bernardy, K., Arnold, B., Offenbächer, M., & Schiltenwolf, M. (2009) · Arthritis Care & Research · doi:10.1002/art.24276 · Cited by 378 · see citing works
Self-Monitoring of Blood Glucose in Patients With Type 2 Diabetes Who Are Not Using Insulin
Welschen, L. M. C., Bloemendal, E., Nijpels, G., Dekker, J. M., Heine, R. J., Stalman, W. A. B., & Bouter, L. M. (2005) · Diabetes Care · doi:10.2337/diacare.28.6.1510 · Cited by 374 · see citing works
Self-monitoring of blood glucose in patients with type 2 diabetes mellitus who are not using insulin
Malanda, U. L., Welschen, L. M., Riphagen, I. I., Dekker, J. M., Nijpels, G., & Bot, S. D. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005060.pub3 · Cited by 358 · see citing works
Treatment of Fibromyalgia Syndrome With Antidepressants: A Meta-analysis
Häuser, W., Bernardy, K., Üçeyler, N., & Sommer, C. (2009) · JAMA · doi:10.1001/jama.2008.944 · Cited by 338 · see citing works
A Systematic Review and Meta-Analysis of Hypoglycemia and Cardiovascular Events
Gangji, A. S., Cukierman, T., Gerstein, H. C., Goldsmith, C. H., & Clase, C. M. (2007) · Diabetes Care · doi:10.2337/dc06-1789 · Cited by 308 · see citing works
Telephone follow-up, initiated by a hospital-based health professional, for postdischarge problems in patients discharged from hospital to home
Mistiaen, P., & Poot, E. (2006) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004510.pub3 · Cited by 299 · see citing works
Vitamin D treatment for the prevention of falls in older adults: systematic review and meta-analysis
Kalyani, R. R., Stein, B., Valiyil, R., Manno, R., Maynard, J. W., & Crews, D. C. (2010) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.2010.02949.x · Cited by 241 · see citing works
High-intensity interval training versus continuous training on physiological and metabolic variables in prediabetes and type 2 diabetes: A meta-analysis
De Nardi, A. T., Tolves, T., Lenzi, T. L., Signori, L. U., & Silva, A. M. V. D. (2018) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2017.12.017 · Cited by 191 · see citing works
Currently recommended treatments of childhood constipation are not evidence based: a systematic literature review on the effect of laxative treatment and dietary measures
Pijpers, M. A. M., Tabbers, M. M., Benninga, M. A., & Berger, M. Y. (2008) · Archives of Disease in Childhood · doi:10.1136/adc.2007.127233 · Cited by 187 · see citing works
Effects of physical exercise programs on cognitive function in Parkinson’s disease patients: A systematic review of randomized controlled trials of the last 10 years
da Silva, F. C., Iop, R. D. R., de Oliveira, L. C., Boll, A. M., de Alvarenga, J. G. S., Gutierres Filho, P. J. B., de Melo, L. M. A. B., Xavier, A. J., & da Silva, R. (2018) · PLOS ONE · doi:10.1371/journal.pone.0193113 · Cited by 189 · see citing works
Interactive computer-based interventions for sexual health promotion
Bailey, J. V., Murray, E., Rait, G., Mercer, C. H., Morris, R. W., Peacock, R., Cassell, J., & Nazareth, I. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006483.pub2 · Cited by 183 · see citing works
Text Categorization Models for High-Quality Article Retrieval in Internal Medicine
Aphinyanaphongs, Y. (2004) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1641 · Cited by 177 · see citing works
Where now for meta-analysis?
Egger, M., Ebrahim, S., & Smith, G. D. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.1 · Cited by 160 · see citing works
Supervised exercise reduces cancer-related fatigue: a systematic review
Meneses-Echávez, J. F., González-Jiménez, E., & Ramírez-Vélez, R. (2015) · Journal of Physiotherapy · doi:10.1016/j.jphys.2014.08.019 · Cited by 146 · see citing works
Effects of Supervised Multimodal Exercise Interventions on Cancer-Related Fatigue: Systematic Review and Meta-Analysis of Randomized Controlled Trials
Meneses-Echávez, J. F., González-Jiménez, E., & Ramírez-Vélez, R. (2015) · BioMed Research International · doi:10.1155/2015/328636 · Cited by 127 · see citing works
Retrieving randomized controlled trials from <scp>medline</scp> : a comparison of 38 published search filters
McKibbon, K. A., Wilczynski, N. L., Haynes, R. B., & for the Hedges Team (2009) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2008.00827.x · Cited by 126 · see citing works
Self-monitoring of blood glucose in non-insulin treated patients with type 2 diabetes: a systematic review and meta-analysis
Allemann, S., Houriet, C., Diem, P., & Stettler, C. (2009) · Current Medical Research and Opinion · doi:10.1185/03007990903364665 · Cited by 115 · see citing works
Management of multiple myeloma: a systematic review and critical appraisal of published studies
Kumar, A., Loughran, T., Alsina, M., Durie, B. G., & Djulbegovic, B. (2003) · The Lancet Oncology · doi:10.1016/s1470-2045(03)01077-5 · Cited by 113 · see citing works
Continuous positive airway pressure reduces blood pressure in patients with obstructive sleep apnea; a systematic review and meta-analysis with 1000 patients
Schein, A. S. O., Kerkhoff, A. C., Coronel, C. C., Plentz, R. D. M., & Sbruzzi, G. (2014) · Journal of Hypertension · doi:10.1097/hjh.0000000000000250 · Cited by 109 · see citing works
Enhancing access to reports of randomized trials published world-wide – the contribution of EMBASE records to the Cochrane Central Register of Controlled Trials (CENTRAL) in The Cochrane Library
Lefebvre, C., Eisinga, A., McDonald, S., & Paul, N. (2008) · Emerging Themes in Epidemiology · doi:10.1186/1742-7622-5-13 · Cited by 93 · see citing works
Automated confidence ranked classification of randomized controlled trial articles: an aid to evidence-based medicine
Cohen, A. M., Smalheiser, N. R., McDonagh, M. S., Yu, C., Adams, C. E., Davis, J. M., & Yu, P. S. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocu025 · Cited by 78 · see citing works
Effects of Exercise on Inflammatory, Oxidative, and Neurotrophic Biomarkers on Cognitively Impaired Individuals Diagnosed With Dementia or Mild Cognitive Impairment: A Systematic Review and Meta-Analysis
Stigger, F. S., Zago Marcolino, M. A., Portela, K. M., & Plentz, R. D. M. (2019) · The Journals of Gerontology: Series A · doi:10.1093/gerona/gly173 · Cited by 68 · see citing works
The efficacy and safety of insulin-sensitizing drugs in HIV-associated lipodystrophy syndrome: a meta-analysis of randomized trials
Sheth, S. H., & Larson, R. J. (2010) · BMC Infectious Diseases · doi:10.1186/1471-2334-10-183 · Cited by 58 · see citing works
Identifying observational studies of surgical interventions in MEDLINE and EMBASE
Fraser, C., Murray, A., & Burr, J. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-41 · Cited by 58 · see citing works
Association between Physical Activity Advice Only or Structured Exercise Training with Blood Pressure Levels in Patients with Type 2 Diabetes: A Systematic Review and Meta-Analysis
Figueira, F. R., Umpierre, D., Cureau, F. V., Zucatti, A. T. N., Dalzochio, M. B., Leitão, C. B., & Schaan, B. D. (2014) · Sports Medicine · doi:10.1007/s40279-014-0226-2 · Cited by 57 · see citing works
Limited search strategies were effective in finding relevant nonrandomized studies
Furlan, A. D., Irvin, E., & Bombardier, C. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.03.004 · Cited by 54 · see citing works
Interventions to prevent injuries in construction workers
van der Molen, H. F., Basnet, P., Hoonakker, P. L., Lehtola, M. M., Lappalainen, J., Frings-Dresen, M. H., Haslam, R., & Verbeek, J. H. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006251.pub4 · Cited by 55 · see citing works
Effect of Nordic hamstring exercise training on knee flexors eccentric strength and fascicle length: a systematic review and meta-analysis.
Medeiros, D. M., Marchiori, C., & Baroni, B. M. (2021) · Journal of Sport Rehabilitation · doi:10.1123/jsr.2019-0388 · Cited by 52 · see citing works
Self-monitoring of blood glucose in patients with type 2 diabetes mellitus who are not using insulin
Welschen, L. M., Bloemendal, E., Nijpels, G., Dekker, J. M., Heine, R. J., Stalman, W. A., & Bouter, L. M. (2005) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd005060.pub2 · Cited by 51 · see citing works
Randomised controlled trials of psychological & pharmacological treatments for body dysmorphic disorder: A systematic review
Phillipou, A., Rossell, S. L., Wilding, H. E., & Castle, D. J. (2016) · Psychiatry Research · doi:10.1016/j.psychres.2016.05.062 · Cited by 50 · see citing works
Assessing the performance of methodological search filters to improve the efficiency of evidence information retrieval: five literature reviews and a qualitative study
Lefebvre, C., Glanville, J., Beale, S., Boachie, C., Duffy, S., Fraser, C., Harbour, J., McCool, R., & Smith, L. (2017) · Health Technology Assessment · doi:10.3310/hta21690 · Cited by 49 · see citing works
Development of a PubMed based search tool for identifying sex and gender specific health literature
Song, M. M., Simonsen, C. K., Wilson, J. D., & Jenkins, M. R. (2016) · Journal of Women's Health · doi:10.1089/jwh.2015.5217 · Cited by 45 · see citing works
Efficacy of indirect dopamine agonists for psychostimulant dependence: a systematic review and meta-analysis of randomized controlled trials
Pérez-Mañá, C., Castells, X., Vidal, X., Casas, M., & Capellà, D. (2011) · Journal of Substance Abuse Treatment · doi:10.1016/j.jsat.2010.08.012 · Cited by 33 · see citing works
Reporting quality of randomised controlled trial abstracts presented at the SLEEP Annual Meetings: a cross-sectional study
Hua, F., Sun, Q., Zhao, T., Chen, X., & He, H. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-029270 · Cited by 20 · see citing works
Critical Assessment of Search Strategies in Systematic Reviews in Endodontics
Yaylali, I. E., & Alaçam, T. (2016) · Journal of Endodontics · doi:10.1016/j.joen.2016.02.018 · Cited by 18 · see citing works
Exercise in Pregnancy and Children’s Cardiometabolic Risk Factors: A Systematic Review and Meta-Analysis
Guillemette, L., Hay, J. L., Kehler, D. S., Hamm, N. C., Oldfield, C., McGavock, J. M., & Duhamel, T. A. (2018) · Sports Medicine - Open · doi:10.1186/s40798-018-0148-x · Cited by 18 · see citing works
Quality improvement in randomized controlled trial abstracts in prosthodontics since the publication of CONSORT guideline for abstracts: a systematic review
Chen, J., Li, Z., Liu, B., Gan, X., Li, C., & Yu, H. (2018) · Journal of Dentistry · doi:10.1016/j.jdent.2018.04.025 · Cited by 16 · see citing works
Preoperative nutritional counseling versus standard care prior to bariatric surgery: Effects on postoperative weight loss
Antoniou, S. A., Anastasiadou, A., Antoniou, G. A., Granderath, F.-A., & Kafatos, A. (2017) · European Surgery · doi:10.1007/s10353-016-0459-4 · Cited by 6 · see citing works

Authors: Robinson, K. A., & Dickersin, K.
Affiliations: Johns Hopkins University (US)
Brown University (US)
Corresponding author: Robinson, Karen A
ORCID: Robinson, Karen A
Dickersin, Kay
Venue: International Journal of Epidemiology
Publisher: Oxford University Press (OUP)
Published: 2002-02
Volume: 31
Issue: 1
Pages: 150-153
ISSN: 0300-5771 (ISSN-L)
1464-3685
DOI: 10.1093/ije/31.1.150
OpenAlex: W2159865972
Cited by: 833
References: 5
Citations by year: 2026: 10 · 2025: 7 · 2024: 12 · 2023: 14 · 2022: 18 · 2021: 15 · 2020: 20 · 2019: 30 · 2018: 25 · 2017: 34 · 2016: 31 · 2015: 36 · 2014: 33 · 2013: 42 · 2012: 33
FWCI: 75.99
Type: journal article
Language: en
Open access: closed
Locations: International Journal of Epidemiology (publishedVersion)
PubMed (publishedVersion)
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
Keywords: search strategies, PubMed, MEDLINE, mesh, sensitivity, precision, systematic reviews, cross-over study, Latin squares, citation retrieval
MeSH terms: Bias, Randomized Controlled Trials as Topic, MEDLINE, Controlled Clinical Trials as Topic

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Development of a highly sensitive search strategy for the retrieval of reports of controlled trials using PubMed

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Self-check

Cites

Cited by

Related

Record

About these checks