CoChat

Search strategies to identify diagnostic accuracy studies in MEDLINE and EMBASE

Open access
Beynon, Rebecca, Leeflang, Mariska M.G., McDonald, Steve, Eisinga, Anne, Mitchell, Ruth L, Whiting, Penny, Glanville, Julie M · 2013 · Cochrane Database of Systematic Reviews
Cited by 145

The DOI matches its Crossref record; 2 of the 2 references it cites were found in a registry or an index.

Abstract

BACKGROUND: A systematic and extensive search for as many eligible studies as possible is essential in any systematic review. When searching for diagnostic test accuracy (DTA) studies in bibliographic databases, it is recommended that terms for disease (target condition) are combined with terms for the diagnostic test (index test). Researchers have developed methodological filters to try to increase the precision of these searches. These consist of text words and database indexing terms and would be added to the target condition and index test searches.Efficiently identifying reports of DTA studies presents challenges because the methods are often not well reported in their titles and abstracts, suitable indexing terms may not be available and relevant indexing terms do not seem to be consistently assigned. A consequence of using search filters to identify records for diagnostic reviews is that relevant studies might be missed, while the number of irrelevant studies that need to be assessed may not be reduced. The current guidance for Cochrane DTA reviews recommends against the addition of a methodological search filter to target condition and index test search, as the only search approach. OBJECTIVES: To systematically review empirical studies that report the development or evaluation, or both, of methodological search filters designed to retrieve DTA studies in MEDLINE and EMBASE. SEARCH METHODS: We searched MEDLINE (1950 to week 1 November 2012); EMBASE (1980 to 2012 Week 48); the Cochrane Methodology Register (Issue 3, 2012); ISI Web of Science (11 January 2013); PsycINFO (13 March 2013); Library and Information Science Abstracts (LISA) (31 May 2010); and Library, Information Science & Technology Abstracts (LISTA) (13 March 2013). We undertook citation searches on Web of Science, checked the reference lists of relevant studies, and searched the Search Filters Resource website of the InterTASC Information Specialists' Sub-Group (ISSG). SELECTION CRITERIA: Studies reporting the development or evaluation, or both, of a MEDLINE or EMBASE search filter aimed at retrieving DTA studies, which reported a measure of the filter's performance were eligible. DATA COLLECTION AND ANALYSIS: The main outcome was a measure of filter performance, such as sensitivity or precision. We extracted data on the identification of the reference set (including the gold standard and, if used, the non-gold standard records), how the reference set was used and any limitations, the identification and combination of the search terms in the filters, internal and external validity testing, the number of filters evaluated, the date the study was conducted, the date the searches were completed, and the databases and search interfaces used. Where 2 x 2 data were available on filter performance, we used these to calculate sensitivity, specificity, precision and Number Needed to Read (NNR), and 95% confidence intervals (CIs). We compared the performance of a filter as reported by the original development study and any subsequent studies that evaluated the same filter. MAIN RESULTS: Ninteen studies were included, reporting on 57 MEDLINE filters and 13 EMBASE filters. Thirty MEDLINE and four EMBASE filters were tested in an evaluation study where the performance of one or more filters was tested against one or more gold standards. The reported outcome measures varied. Some studies reported specificity as well as sensitivity if a reference set containing non-gold standard records in addition to gold standard records was used. In some cases, the original development study did not report any performance data on the filters. Original performance from the development study was not available for 17 filters that were subsequently tested in evaluation studies. All 19 studies reported the sensitivity of the filters that they developed or evaluated, nine studies reported the specificities and 14 studies reported the precision.No filter which had original performance data from its development study, and was subsequently tested in an evaluation study, had what we defined a priori as acceptable sensitivity (> 90%) and precision (> 10%). In studies that developed MEDLINE filters that were evaluated in another study (n = 13), the sensitivity ranged from 55% to 100% (median 86%) and specificity from 73% to 98% (median 95%). Estimates of performance were lower in eight studies that evaluated the same 13 MEDLINE filters, with sensitivities ranging from 14% to 100% (median 73%) and specificities ranging from 15% to 96% (median 81%). Precision ranged from 1.1% to 40% (median 9.5%) in studies that developed MEDLINE filters and from 0.2% to 16.7% (median 4%) in studies that evaluated these filters. A similar range of specificities and precision were reported amongst the evaluation studies for MEDLINE filters without an original performance measure. Sensitivities ranged from 31% to 100% (median 71%), specificity ranged from 13% to 90% (median 55.5%) and precision fro

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1002/14651858.mr000022.pub3

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 2 references

Research works printed but not linked: 37

  1. Astin, 2008, Developing a sensitive search strategy in MEDLINE to retrieve studies on assessment of the diagnostic performance of imaging techniques, Radiology, 247, 365, 10.1148/radiol.2472070101 · DOI
  2. Bachmann, 2002, Identifying diagnostic studies in MEDLINE: reducing the number needed to read, Journal of the American Medical Informatics Association, 9, 653, 10.1197/jamia.M1124 · DOI · Paper page
  3. Bachmann, 2003, Identifying diagnostic accuracy studies in EMBASE, Journal of the Medical Library Association, 91, 341
  4. Berg, 2005, Development of two search strategies for literature in MEDLINE-PubMed: nursing diagnoses in the context of evidence-based nursing, International Journal of Nursing Terminologies and Classifications, 16, 26, 10.1111/j.1744-618X.2005.00006.x · DOI
  5. Deville, 2000, Publications on diagnostic test evaluation in family medicine journals: an optimal search strategy, Journal of Clinical Epidemiology, 53, 65, 10.1016/S0895-4356(99)00144-4 · DOI · Paper page
  6. Deville, 2002, Systematic reviews in practice. X. Searching, selecting and the methodological assessment of diagnostic evaluation research, Nederlands Tijdschrift voor Geneeskunde, 146, 2281
  7. Haynes, 1994, Developing optimal search strategies for detecting clinically sound studies in MEDLINE, Journal of the American Medical Informatics Association, 1, 447, 10.1136/jamia.1994.95153434 · DOI · Paper page
  8. Wilczynski, 1993, Assessment of methodologic search filters in MEDLINE, Proceedings of the Annual Symposium on Computer Applications in Medical Care, 601
  9. Wilczynski, 1994, Quantitative comparison of pre-explosions and subheadings with methodologic search terms in MEDLINE, Proceedings of the Annual Symposium on Computer Applications in Medical Care, 905
  10. Haynes, 2004, Optimal search strategies for retrieving scientifically strong studies of diagnosis from Medline: analytical survey, BMJ, 328, 1040, 10.1136/bmj.38068.557998.EE · DOI · Paper page
  11. Kassai, 2006, Literature search parameters marginally improved the pooled estimate accuracy for ultrasound in detecting deep venous thrombosis, Journal of Clinical Epidemiology, 59, 710, 10.1016/j.jclinepi.2005.09.013 · DOI
  12. Kastner, 2009, Diagnostic test systematic reviews: bibliographic search filters ("Clinical Queries") for diagnostic accuracy studies perform well, Journal of Clinical Epidemiology, 62, 974, 10.1016/j.jclinepi.2008.11.006 · DOI
  13. Leeflang, 2006, Use of methodological search filters to identify diagnostic accuracy studies can lead to the omission of relevant studies, Journal of Clinical Epidemiology, 59, 234, 10.1016/j.jclinepi.2005.07.014 · DOI · Paper page
  14. Ritchie, 2007, Do published search filters to identify diagnostic test accuracy studies perform adequately?, Health Information and Libraries Journal, 24, 188, 10.1111/j.1471-1842.2007.00735.x · DOI · Paper page
  15. Weijden, 1997, Identifying relevant diagnostic studies in MEDLINE. The diagnostic value of the erythrocyte sedimentation rate (ESR) and dipstick as an example, Family Practice, 14, 204, 10.1093/fampra/14.3.204 · DOI · Paper page
  16. Vincent, 2003, Clinical Evidence diagnosis: developing a sensitive search strategy to retrieve diagnostic studies on deep vein thrombosis: a pragmatic approach, Health Information and Libraries Journal, 20, 150, 10.1046/j.1365-2532.2003.00427.x · DOI · Paper page
  17. Whiting, 2010, Inclusion of methodological filters in searches for diagnostic test accuracy studies misses relevant studies, Journal of Clinical Epidemiology, 64, 602, 10.1016/j.jclinepi.2010.07.006 · DOI
  18. Wilczynski, 2005, EMBASE search strategies for identifying methodologically sound diagnostic studies for use by clinicians and researchers, BMC Medicine, 3, 7, 10.1186/1741-7015-3-7 · DOI · Paper page
  19. de Vet HCW Eisinga A Riphagen II Aertgeerts B Pewsner D Mitchell R Chapter 7: Searching for studies. In: Deeks JJ, Bossuyt PM, Gatsonis C (editors). Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Version 0.4 [updated September 2008]. The Cochrane Collaboration, 2008 Available from http://srdta.cochrane.org
  20. Deville, 2002, Conducting systematic reviews of diagnostic studies: didactic guidelines, BMC Medical Research Methodology, 2, 9, 10.1186/1471-2288-2-9 · DOI · Paper page
  21. Falck-Ytter YT Motschall E New search filter for diagnostic studies: Ovid and PubMed versions not the same [2004] available at http://www.bmj.com/content/328/7447/1040?tab=responses
  22. Fielding, 2002, Using Medline to achieve an evidence-based approach to diagnostic clinical biochemistry, Annals of Clinical Biochemistry, 39, 345, 10.1258/000456302760042461 · DOI
  23. Glanville, 2008, So many filters, so little time: the development of a search filter appraisal checklist, Journal of the Medical Library Association, 96, 356, 10.3163/1536-5050.96.4.011 · DOI · Paper page
  24. Haynes, 2005, Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytical survey, BMJ, 330, 1179, 10.1136/bmj.38446.498542.8F · DOI · Paper page
  25. Haynes, 2005, Developing optimal search strategies for detecting clinically sound and relevant causation studies in EMBASE, BMC Medical Information and Decision Making, 5, 8, 10.1186/1472-6947-5-8 · DOI · Paper page
  26. Horsley, 2011, Checking reference lists to find additional studies for systematic reviews, Cochrane Database of Systematic Reviews, 10.1002/14651858.MR000026.pub2 · DOI · Paper page
  27. Lefebvre C Manheimer E Glanville J Chapter 6: Searching for studies. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [updated March 2011]. The Cochrane Collaboration, 2011 Available from http://www.cochrane-handbook.org
  28. US National Library of Medicine. Clinical Queries using Research Methodology Filters [updated Jan 2005] Available from http://www.ncbi.nlm.nih.gov/books/NBK3827/#pubmedhelp.Clinical_Queries_Filters
  29. Wolters Kluwer Health MEDLINE® 2013 Database Guide available at http://ovidsp.tx.ovid.com/sp-3.8.1a/ovidweb.cgi?&S=BLIMFPMFDFDDHIFFNCOKFBFBJDCBAA00&Database+Field+Guide=37 [2012]
  30. Wolters Kluwer Health Embase: Excerpta Medica Database Guide http://ovidsp.tx.ovid.com/sp-3.8.1a/ovidweb.cgi?&S=BLIMFPMFDFDDHIFFNCOKFBFBJDCBAA00&Database+Field+Guide=10 [2012]
  31. Whiting P Westwood M Burke M Sterne J Harbord R Glanville J Can diagnostic filters offer similar sensitivity and a reduced number needed to read compared to searches based on index test and target condition? [abstract] Methods for Evaluating Medical Tests. Symposium
  32. Whiting, 2011, Inclusion of methodological filters in searches for diagnostic accuracy studies misses relevant studies, Journal of Clinical Epidemiology, 64, 602, 10.1016/j.jclinepi.2010.07.006 · DOI
  33. Wilczynski, 1995, Reasons for the loss of sensitivity and specificity of methodologic MeSH terms and textwords in MEDLINE, Proceedings - the Annual Symposium on Computer Applications in Medical Care, 436
  34. Wilczynski, 2003, Developing optimal search strategies for detecting clinically sound causation studies in MEDLINE, AMIA - Annual Symposium Proceedings/AMIA Symposium, 719
  35. Wilczynski, 2004, Developing optimal search strategies for detecting clinically sound prognostic studies in MEDLINE: an analytic survey, BMC Medicine, 2, 23, 10.1186/1741-7015-2-23 · DOI · Paper page
  36. Wilczynski, 2005, Optimal search strategies for detecting clinically sound prognostic studies in EMBASE: an analytic survey, Journal of the American Medical Informatics Association, 12, 481, 10.1197/jamia.M1752 · DOI · Paper page
  37. Wilczynski, 2007, Indexing of diagnosis accuracy studies in MEDLINE and EMBASE, AMIA - Annual Symposium Proceedings/AMIA Symposium, 801

Cited by

See all 145

Metadata

Authors
Beynon, R., Leeflang, M. M. G., McDonald, S., Eisinga, A., Mitchell, R. L., Whiting, P., & Glanville, J. M.
Venue
Cochrane Database of Systematic Reviews
Publisher
Wiley
Published
2013-09-11
Volume
2013
Issue
9
Pages
MR000022-MR000022
ISSN
  • 1361-6137 (ISSN-L)
  • 1465-1858
OpenAlex
W1848444059
Cited by
145
References
2
FWCI
8.25
Type
journal article
Language
en
Open access
green

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Search strategies to identify diagnostic accuracy studies in MEDLINE and EMBASE. https://app.cochat.ai/papers/10.1002/14651858.mr000022.pub3
The paper
Beynon, R., Leeflang, M. M. G., McDonald, S., Eisinga, A., Mitchell, R. L., Whiting, P., & Glanville, J. M. (2013). Search strategies to identify diagnostic accuracy studies in MEDLINE and EMBASE. Cochrane Database of Systematic Reviews. https://doi.org/10.1002/14651858.mr000022.pub3

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.