Limited search strategies were effective in finding relevant nonrandomized studies
Abstract
Background and objective Searching for nonrandomized studies in electronic databases is complicated because there is a variety of study designs and lack of standardization in the terminology. The purpose of this study was to develop and evaluate a method to limit search strategies according to study design of comparative nonrandomized studies (cNRSs).Methods Four updated Cochrane systematic reviews that included nonrandomized studies (cohort, case-control, and cross-sectional studies) of the effects of health care interventions were selected. Search strategies limited to study design were devised for each one of these topic areas in two electronic databases (MEDLINE and EMBASE). A progressive method (PM) and a fixed method for selecting the most appropriate search terms associated with study design of nonrandomized studies are suggested.Results The results showed that the sensitivity of search strategies (in two databases combined) limited to study design were between 90% and 100% for the PM using both controlled vocabulary (CV) and textwords (TWs) and between 95% and 100% for a fixed set of controlled vocabulary and TWs.Conclusions It is possible and acceptable to use search strategies limited to study design of cNRSs of health care interventions.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
About one third of the articles acknowledged that searching for observational studies requires additional time and resources because of lack of specific search filters and poorly established or inconsistently used indexing terms.
“About one third of the articles acknowledged that searching for observational studies requires additional time and resources because of lack of specific search filters and poorly established or inconsistently used indexing terms [38, 39, 47, 49, 56, 58, 59, 64, 65].”
From Methods to systematically review and meta-analyse observational studies: a systematic scoping review of recommendations (2018) · cites this paper as [59]
Cites
Robinson, K. A., & Dickersin, K. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.150 · Cited by 833 · see citing works
Haynes, R. B., & Wilczynski, N. L. (2004) · BMJ · doi:10.1136/bmj.38068.557998.ee · Cited by 549 · see citing works
Haynes, R., McKibbon, K., Wilczynski, N., Walter, S. D., & Werre, S. R. (2005) · British medical journal · doi:10.1136/bmj.38446.498542.8f · Cited by 500 · see citing works
Male circumcision for prevention of heterosexual acquisition of HIV in men
Siegfried, N., Muller, M., Deeks, J. J., & Volmink, J. A. (2009) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003362.pub2 · Cited by 492 · see citing works
Optimal search strategies for retrieving systematic reviews from Medline: analytical survey
Montori, V. M., Wilczynski, N. L., Morgan, D., & Haynes, R. B. (2005) · BMJ · doi:10.1136/bmj.38336.804167.47 · Cited by 422 · see citing works
Quality of life after rectal resection for cancer, with or without permanent colostomy.
Pachler, J. H., & Wille-Jørgensen, P. (2012) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004323.pub4 · Cited by 416 · see citing works
Methadone maintenance at different dosages for opioid dependence
Faggiano, F., Vigna‐Taglianti, F., Versino, E., & Lemma, P. (2003) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd002208 · Cited by 409 · see citing works
Local versus general anaesthesia for carotid endarterectomy
Vaniyapong, T., Chongruksut, W., & Rerkasem, K. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000126.pub4 · Cited by 262 · see citing works
Quality of life after rectal resection for cancer, with or without permanent colostomy.
Pachler, J. H., & Wille-Jørgensen, P. (2005) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004323.pub3 · Cited by 204 · see citing works
Taking advantage of the explosion of systematic reviews: an efficient MEDLINE search strategy.
Shojania, K. G., & Bero, L. A. (2001) · PubMed · Cited by 158 · see citing works
Male circumcision for prevention of heterosexual acquisition of HIV in men
Siegfried, N., Muller, M., Volmink, J. A., Deeks, J. J., Egger, M., Low, N. N., Weiss, H. H., Walker, A. S., & Williamson, P. R. (2003) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003362 · Cited by 130 · see citing works
Wilczynski, N. L., & Haynes, R. B. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1752 · Cited by 99 · see citing works
Local versus general anaesthesia for carotid endarterectomy
Rerkasem, K., Bond, R., & Rothwell, P. M. (2004) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd000126.pub2 · Cited by 90 · see citing works
Haynes, R. B., Kastner, M., Wilczynski, N. L., & the Hedges Team (2005) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-5-8 · Cited by 65 · see citing works
Vincent, S., Greenley, S. L., & Beaven, O. (2003) · Health Information & Libraries Journal · doi:10.1046/j.1365-2532.2003.00427.x · Cited by 55 · see citing works
Searching the Literature by Design
Littleton, D. M., Marsalis, S. E., & Bliss, D. Z. (2004) · Western Journal of Nursing Research · doi:10.1177/0193945903258729 · Cited by 7 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
2009 updated method guidelines for systematic reviews in the Cochrane Back Review Group
Furlan, A. D., Pennick, V., Bombardier, C., & van Tulder, M. (2009) · Spine · doi:10.1097/brs.0b013e3181b1c99f · Cited by 1,663 · see citing works
“● The Editorial Board recommends using the search strategy suggested by Golder et al(16) to find reports of adverse events of their interventions, and the search strategy suggested by Furlan et al, if review authors plan to include observational studies.(17) Contact a health sciences librarian or the CBRG Trials Searc…”
MethodSelf-citation2015 Updated Method Guideline for Systematic Reviews in the Cochrane Back and Neck Group
Furlan, A. D., Malmivaara, A., Chou, R., Maher, C. G., Deyo, R. A., Schoene, M., Bronfort, G., & van Tulder, M. W. (2015) · Spine · doi:10.1097/brs.0000000000001061 · Cited by 704 · see citing works
Self-citationMueller, M., D’Addario, M., Egger, M., Cevallos, M., Dekkers, O., Mugglin, C., & Scott, P. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0495-9 · Cited by 400 · see citing works
“Another recommendation was to create two different search terms: one for older dates with no limits on study design and the other for more recent dates, after study design search terms were introduced in databases, with study design restrictions applied [59].”
BackgroundUngprasert, P., Cheungpasitporn, W., Crowson, C. S., & Matteson, E. L. (2015) · European Journal of Internal Medicine · doi:10.1016/j.ejim.2015.03.008 · Cited by 322 · see citing works
“[13].”
Vissers, M. M., Bussmann, J. B. J., Verhaar, J. A. N., Arends, L. R., Furlan, A. D., & Reijman, M. (2011) · Physical Therapy · doi:10.2522/ptj.20100201 · Cited by 182 · see citing works
Self-citationCuello‐García, C. A., Santesso, N. A., Morgan, R. L., Verbeek, J. H. A. M., Thayer, K. A., Ansari, M., Meerpohl, J. J., Schwingshackl, L., Katikireddi, S. V., Brożek, J., et al. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.11.026 · Cited by 178 · see citing works
Including non‐randomized studies on intervention effects
Reeves, B. C., Deeks, J. J., Higgins, J. P. T., Shea, B., Tugwell, P., Wells, G. A., & on behalf of the Cochrane Non‐Randomized Studies of Interventions Methods Group (2019) · Wiley eBooks · doi:10.1002/9781119536604.ch24 · Cited by 167 · see citing works
Chapple, C., Nicholson, H. D., Baxter, D. G., & Abbott, J. H. (2011) · Arthritis Care & Research · doi:10.1002/acr.20492 · Cited by 143 · see citing works
On the use of systematic reviews to inform environmental policies
Bilotta, G. S., Milner, A. M., & Boyd, I. L. (2014) · Environmental Science & Policy · doi:10.1016/j.envsci.2014.05.010 · Cited by 138 · see citing works
“Some review authors have tried to develop, optimise and ‘validate’ search strategies for NRS (Wieland and Dickersin, 2005; Fraser et al., 2006; Furlan et al., 2006; Golder et al., 2006a,b), and there is a dedicated NRS Methods Group who provide guidance to support authors who are considering…”
BackgroundBreast, ovarian, and endometrial malignancies in systemic lupus erythematosus: a meta-analysis
Bernatsky, S., Ramsey‐Goldman, R., Foulkes, W. D., Gordon, C., & Clarke, A. E. (2011) · British Journal of Cancer · doi:10.1038/bjc.2011.115 · Cited by 95 · see citing works
“Progressive search strategies locate essentially 100% of articles as compared with traditional systematic literature searches (Furlan et al, 2006).”
BackgroundInfluentialNi, J., Qiu, L.-J., Hu, L. Q., Cen, H., Zhang, M., Wen, P.-F., Xs, W., Pan, H., & Ye, D. (2014) · Lupus · doi:10.1177/0961203313520060 · Cited by 74 · see citing works
Total knee arthroplasty after high tibial osteotomy. A systematic review
van Raaij, T. M., Reijman, M., Furlan, A. D., & Verhaar, J. A. N. (2009) · BMC Musculoskeletal Disorders · doi:10.1186/1471-2474-10-88 · Cited by 73 · see citing works
“showed that the sensitivity of limited search strategies for a fixed set of controlled vocabulary and text words was between 95 and 100% [9].”
Self-citationSearch strategies to identify observational studies in MEDLINE and Embase
Li, L., Smith, H. E., Atun, R. A., & Car, L. T. (2019) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000041.pub2 · Cited by 61 · see citing works
Sonnappa, S., McQueen, B., Postma, D. S., Martin, R. J., Roche, N., Grigg, J., Guilbert, T. W., Gouder, C., Pizzichini, E., Niimi, A., et al. (2017) · The Journal of Allergy and Clinical Immunology In Practice · doi:10.1016/j.jaip.2017.07.032 · Cited by 61 · see citing works
Ungprasert, P., Srivali, N., & Thongprayoon, C. (2015) · Clinical Cardiology · doi:10.1002/clc.22502 · Cited by 55 · see citing works
Finding evidence for comparing medical interventions: AHRQ and the Effective Health Care Program
Relevo, R., & Balshem, H. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.11.022 · Cited by 52 · see citing works
“[61], Fraser et al.”
Shi, Z.-R., Cao, C.-X., Tan, G.-Z., & Wang, L. (2014) · Lupus · doi:10.1177/0961203314560003 · Cited by 49 · see citing works
Riemsma, R., Al, M., Ramos, I. C., Deshpande, S., Armstrong, N., Lee, Y.-C., Ryder, S., Noake, C., Krol, M., Oppe, M., et al. (2013) · Health Technology Assessment · doi:10.3310/hta17610 · Cited by 47 · see citing works
Ijaz, S., Verbeek, J. H. A. M., Tikka, C., & Ruotsalainen, J. H. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.01.001 · Cited by 41 · see citing works
“For the same reason, it is difficult to define nonrandomized study designs and to be sure what type of studies to include [9].”
BackgroundNonaspirin Nonsteroidal Anti-Inflammatory Drugs and Risk of Hemorrhagic Stroke
Ungprasert, P., Matteson, E. L., & Thongprayoon, C. (2015) · Stroke · doi:10.1161/strokeaha.115.011678 · Cited by 41 · see citing works
“The search terms were compiled from terms for stroke and NSAIDs, including names of individual drugs in conjunction with the terms for nonrandomized studies developed by Furlan et al.12 The detailed search strategy is provided in Supplemental material I in the online-only Data Supplement.”
MethodProstate cancer in systemic lupus erythematosus
Bernatsky, S., Ramsey‐Goldman, R., Gordon, C., & Clarke, A. E. (2011) · International Journal of Cancer · doi:10.1002/ijc.25956 · Cited by 41 · see citing works
Waffenschmidt, S., Navarro, T., Hobson, N., Hausner, E., Sauerland, S., & Haynes, R. B. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1425 · Cited by 39 · see citing works
Post-exposure passive immunisation for preventing rubella and congenital rubella syndrome
Young, M. K., Cripps, A. W., Nimmo, G. R., & van Driel, M. (2015) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010586.pub2 · Cited by 37 · see citing works
Avau, B., Remoortel, H. V., & Buck, E. D. J. D. (2021) · Journal of the Medical Library Association JMLA · doi:10.5195/jmla.2021.1219 · Cited by 33 · see citing works
“describe recall ranging from 48% to 93% using fixed search strategies for observational studies across four gold standards retrieved from the included references of four systematic reviews in Ovid PubMed and Ovid Embase without external validation in an independent reference gold standard [38].”
MethodFuller, T., Pearson, M., Peters, J., & Anderson, R. (2012) · BMJ Open · doi:10.1136/bmjopen-2012-002073 · Cited by 33 · see citing works
Abímbọ́lá, Ṣ., Martiniuk, A. L. C., Hackett, M. L., & Anderson, C. S. (2011) · Neuroepidemiology · doi:10.1159/000327497 · Cited by 31 · see citing works
“was selected to optimise the yield of prognostic studies in Medline [incidence (MeSH) OR explode mortality OR follow-up studies (MeSH) OR prognos* (text word) OR predict* (text word) OR course (text word)]. This was combined with [explode epilepsy (MeSH) OR explode seizure (MeSH) OR seizure disorder (text word)] using…”
MethodUngprasert, P., Srivali, N., & Kittanamongkolchai, W. (2015) · European Journal of Internal Medicine · doi:10.1016/j.ejim.2015.09.012 · Cited by 30 · see citing works
“[10] The detailed search strategy is provided as supplementary material.”
BackgroundLeisure in Children and Adolescents with Juvenile Idiopathic Arthritis: A Systematic Review
Cavallo, S., Toupin‐April, K., Grandpierre, V., Majnemer, A., & Feldman, D. E. (2014) · PLoS ONE · doi:10.1371/journal.pone.0104642 · Cited by 30 · see citing works
“Key term selection was guided by using a PICOS (populations, interventions, comparators, outcomes, study design) framework [15–17].”
MethodGlanville, J., Eyers, J. E., Jones, A. M., Shemilt, I., Wang, G., Johansen, M., Fiander, M., & Rothstein, H. R. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.02.018 · Cited by 24 · see citing works
“AHRQ recommends the use of validated study design search filters and directs researchers to published filters for observational study designs (18, 20-21),”
MethodKhakoo, N. S., Sultan, S., Reynolds, J. M., & Levy, C. (2022) · Digestive Diseases and Sciences · doi:10.1007/s10620-022-07704-4 · Cited by 23 · see citing works
“We ran additional searches for observational studies using the same search strategies in Ovid MEDLINE(R) and Epub Ahead of Print, In-Process, In-Data-Review & Other Non-Indexed Citations and Daily, 1946-present; Embase; and Scopus, using the Furlan Fixed Method B for MEDLINE for Non-randomized Studies and the Scottish…”
MethodBurkowitz, J., Merzenich, C., Grassme, K. S., & Brüggenjürgen, B. (2016) · European Journal of Preventive Cardiology · doi:10.1177/2047487316632628 · Cited by 22 · see citing works
“The search strategy involved each of the indications (syncope and CS), and the intervention ICM (or alternative terms such as ‘‘implantable loop recorder’’) in adult patients aged 18 years.(38,39) Two reviewers independently verified the abstracts with regards to the inclusion and exclusion criteria (Supplemental Tabl…”
BackgroundPerformance of a mixed filter to identify relevant studies for mixed studies reviews
Sherif, R. E., Pluye, P., Gore, G. C., Granikov, V., & Hong, Q. N. (2016) · Journal of the Medical Library Association JMLA · doi:10.3163/1536-5050.104.1.007 · Cited by 21 · see citing works
Changes in the macroscopic morphology of hip muscles in low back pain
Pourahmadi, M., Asadi, M. H., Dommerholt, J., & Yeganeh, A. (2019) · Journal of Anatomy · doi:10.1111/joa.13086 · Cited by 18 · see citing works
“To retrieve only comparative observational studies, a modified version of the syntax developed by Furlan et al. (2006) was used.”
MethodNadal, E., Rifi, N., Kane, S. A., Mbacke, S., Starkman, L., Suero, B., Le, H., & Samjoo, I. A. (2024) · Lung Cancer · doi:10.1016/j.lungcan.2024.107816 · Cited by 16 · see citing works
Hausner, E., Metzendorf, M., Richter, B., Lotz, F., & Waffenschmidt, S. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0625-4 · Cited by 13 · see citing works
“Fixed method A for MEDLINE (Furlan 2006) [21] Cohort, case-control, cross-sectional 5–9 4.”
BackgroundInfluentialSearch strategies to identify observational studies in MEDLINE and EMBASE
Belisario, J. S. M., Car, L. T., Reeves, T., Gunn, L. H., & Car, J. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000041 · Cited by 12 · see citing works
Systemic lupus erythematosus with and without a family history: a meta-analysis
Chen, L. Y., Shi, Z., Tan, G., Han, Y. F., Tang, Z., & Wang, L. (2017) · Lupus · doi:10.1177/0961203317739133 · Cited by 12 · see citing works
Abímbọ́lá, Ṣ., Martiniuk, A. L. C., Hackett, M. L., Glozier, N., Mohamed, A., & Anderson, C. S. (2013) · Neurological Research · doi:10.1179/1743132813y.0000000257 · Cited by 10 · see citing works
Múñoz, Ó., & Morales, Á. J. R. (2018) · Acta Médica Colombiana · doi:10.36104/amc.2018.1095 · Cited by 11 · see citing works
Fifteen Years of the Cochrane Back Review Group
Marin, T. J., Furlan, A. D., Bombardier, C., & van Tulder, M. W. (2013) · Spine · doi:10.1097/brs.0000000000000006 · Cited by 8 · see citing works
Self-citationSGLT2 Inhibitors in COVID-19: Umbrella Review, Meta-Analysis, and Bayesian Sensitivity Assessment
Suresh, V., Shamim, M. A., Ghosh, V., Dave, T., Jayan, M., Verma, A., Sanker, V., Roy, P., & Bardhan, M. (2025) · Diseases · doi:10.3390/diseases13030067 · Cited by 4 · see citing works
“Filters for observational study designs were adopted from the Harvard Countway Library guide [17], which is a modified version of the “fixed method B” proposed by Furlan et al. [18].”
Kognitive Prozesse in Screening-Prozessen – Suchstrategien in der Bildungstechnologieforschung
Buntins, K., Mulders, M., & Schröder, N. (2023) · MedienPädagogik Zeitschrift für Theorie und Praxis der Medienbildung · doi:10.21240/mpaed/54/2023.11.19.x · Cited by 3 · see citing works
Knowles, A., Chadwick, J., & Vancampfort, D. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd010554 · Cited by 3 · see citing works
Broderick, J. M., Vancampfort, D., Mockler, D., Bennett, K., Waugh, A., McDonough, C., Cramer, H., & Walshe, M. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd013213 · Cited by 2 · see citing works
Improving the Reporting of Health and Psychological Research
Fuller, T. E., Pearson, M., & Peters, J. (2016) · Australian Psychologist · doi:10.1111/ap.12135 · Cited by 2 · see citing works
Physiotherapy for osteoarthritis of the knee: predictors of outcome at one year
Chapple, C. (2012) · Otago University Research Archive (University of Otago) · Cited by 2 · see citing works
Toro, M. R., Gómez, M. A. R., Múñoz, Ó., & Rodríguez, J. F. B. (2026) · Journal of Interventional Cardiac Electrophysiology · doi:10.1007/s10840-026-02286-x · Cited by 1 · see citing works
Oh, H., Choi, Y., Lee, J., & Lee, E. (2022) · Annals of Palliative Medicine · doi:10.21037/apm-21-2929 · Cited by 1 · see citing works
Recovery after total hip or knee arthroplasty: physical and mental functioning
Vissers, M. M. (2012) · EUR Research Repository (Erasmus University Rotterdam) · Cited by 1 · see citing works
Abímbọ́lá, Ṣ. (2010) · UPT. Syiah Kuala University Library (Syiah Kuala University)
Fuller, T., Pearson, M., Peters, J., & Anderson, R. (2012)
Cervical And Vaginal/Vulvar Malignancies Risk In Women With Rheumatoid Arthritis (RA)
Diaconescu, M., & Bernatsky, S. (2018) · Journal of Autoimmune Diseases and Rheumatology · doi:10.12970/2310-9874.2018.06.05
Forschungssynthesen in der Mediendidaktik Ansätze und Herausforderungen
Bedenlier, S. M., Buntins, K., Wilmers, A., & Kerres, M. (2023) · MedienPädagogik Zeitschrift für Theorie und Praxis der Medienbildung · doi:10.21240/mpaed/54.x
Yu, W., Wei, A., Yuan, H., & Sun, Z. (2023) · Research Square · doi:10.21203/rs.3.rs-3348599/v1
- Authors
- Furlan, A. D., Irvin, E., & Bombardier, C.
- Affiliations
- University of Toronto (CA)
- Institute for Work & Health (CA)
- Corresponding author
- Furlan, Andrea D.
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2006-12
- Volume
- 59
- Issue
- 12
- Pages
- 1303-1311
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2048634900
- Cited by
- 54
- References
- 16
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 1 · 2023: 3 · 2022: 2 · 2021: 2 · 2020: 1 · 2019: 3 · 2018: 5 · 2017: 3 · 2016: 3 · 2015: 6 · 2014: 5 · 2013: 5 · 2012: 4
- FWCI
- 0.76
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Health Sciences › Health Professions › Health Information Management › Methodologies in Health Research and Practice
- Keywords
- search strategy development, non-randomized studies, electronic database searching, MEDLINE, Embase, controlled vocabulary, cohort studies, case-control study, cross-sectional study, health interventions
- MeSH terms
- Research Design, Epidemiologic Studies, Databases, Bibliographic, MEDLINE, Information Storage and Retrieval, Vocabulary, Controlled, Medical Subject Headings
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
