Assessing the performance of methodological search filters to improve the efficiency of evidence information retrieval: five literature reviews and a qualitative study
Open accessThe DOI matches its Crossref record; 129 of the 130 references it cites were found in a registry or an index, and 1 are not checked yet.
Abstract
Background: Effective study identification is essential for conducting health research, developing clinical guidance and health policy and supporting health-care decision-making. Methodological search filters (combinations of search terms to capture a specific study design) can assist in searching to achieve this. Objectives: This project investigated the methods used to assess the performance of methodological search filters, the information that searchers require when choosing search filters and how that information could be better provided. Methods: Five literature reviews were undertaken in 2010/11: search filter development and testing; comparison of search filters; decision-making in choosing search filters; diagnostic test accuracy (DTA) study methods; and decision-making in choosing diagnostic tests. We conducted interviews and a questionnaire with experienced searchers to learn what information assists in the choice of search filters and how filters are used. These investigations informed the development of various approaches to gathering and reporting search filter performance data. We acknowledge that there has been a regrettable delay between carrying out the project, including the searches, and the publication of this report, because of serious illness of the principal investigator. Results: The development of filters most frequently involved using a reference standard derived from hand-searching journals. Most filters were validated internally only. Reporting of methods was generally poor. Sensitivity, precision and specificity were the most commonly reported performance measures and were presented in tables. Aspects of DTA study methods are applicable to search filters, particularly in the development of the reference standard. There is limited evidence on how clinicians choose between diagnostic tests. No published literature was found on how searchers select filters. Interviewing and questioning searchers via a questionnaire found that filters were not appropriate for all tasks but were predominantly used to reduce large numbers of retrieved records and to introduce focus. The Inter Technology Appraisal Support Collaboration (InterTASC) Information Specialists’ Sub-Group (ISSG) Search Filters Resource was most frequently mentioned by both groups as the resource consulted to select a filter. Randomised controlled trial (RCT) and systematic review filters, in particular the Cochrane RCT and the McMaster Hedges filters, were most frequently mentioned. The majority indicated that they used different filters depending on the requirement for sensitivity or precision. Over half of the respondents used the filters available in databases. Interviewees used various approaches when using and adapting search filters. Respondents suggested that the main factors that would make choosing a filter easier were the availability of critical appraisals and more detailed performance information. Provenance and having the filter available in a central storage location were also important. Limitations: The questionnaire could have been shorter and could have included more multiple choice questions, and the reviews of filter performance focused on only four study designs. Conclusions: Search filter studies should use a representative reference standard and explicitly report methods and results. Performance measures should be presented systematically and clearly. Searchers find filters useful in certain circumstances but expressed a need for more user-friendly performance information to aid filter choice. We suggest approaches to use, adapt and report search filter performance. Future work could include research around search filters and performance measures for study designs not addressed here, exploration of alternative methods of displaying performance results and numerical synthesis of performance comparison results. Funding: The National Institute for Health Research (NIHR) Health Technology Assessment programme and Medical Research Council–NIHR Methodology Research Programme (grant number G0901496).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This enables readers to assess the quality of the filter(s) used, as most published search filters are validated and/or peer reviewed [106, 107].”
From PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews (2021) · cites this paper as [106]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The relationship between Precision-Recall and ROC curves
Davis, J., & Goadrich, M. (2006) · Proceedings of the 23rd international conference on Machine learning - ICML '06 · doi:10.1145/1143844.1143874 · Cited by 6,559
PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement
McGowan, J., Sampson, M., Salzwedel, D. M., Cogo, E., Foerster, V., & Lefebvre, C. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.01.021 · Cited by 5,700
Self-citationCochrane Handbook for Systematic Reviews of Interventions
Tarsilla, M. (2010) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v6i14.284 · Cited by 5,178
Whiting, P., Rutjes, A. W., Reitsma, J. B., Bossuyt, P. M., & Kleijnen, J. (2003) · BMC Medical Research Methodology · doi:10.1186/1471-2288-3-25 · Cited by 3,592
Reitsma, J. B., Glas, A. S., Rutjes, A. W. S., Scholten, R. J. P. M., Bossuyt, P. M., & Zwinderman, A. H. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.02.022 · Cited by 3,369
Systematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,245
Towards complete and accurate reporting of studies of diagnostic accuracy: The STARD Initiative.
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., de Vet, H. C. W., et al. (2003) · Annals of Internal Medicine · doi:10.7326/0003-4819-138-1-200301070-00010 · Cited by 3,219
Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L. M., Lijmer, J. G., Moher, D., Rennie, D., & de Vet, H. C. W. (2003) · BMJ · doi:10.1136/bmj.326.7379.41 · Cited by 3,215
Deeks, J. J., Altman, D. G., & Bradburn, M. J. (2001) · Systematic Reviews in Health Care · doi:10.1002/9780470693926.ch15 · Cited by 2,163
Empirical evidence of design-related bias in studies of diagnostic tests.
Lijmer, J. G. (1999) · JAMA · doi:10.1001/jama.282.11.1061 · Cited by 1,861
A hierarchical regression approach to meta‐analysis of diagnostic test accuracy evaluations
Rutter, C. M., & Gatsonis, C. A. (2001) · Statistics in Medicine · doi:10.1002/sim.942 · Cited by 1,505
Systematic reviews of evaluations of diagnostic and screening tests
Deeks, J. J. (2001) · BMJ · doi:10.1136/bmj.323.7305.157 · Cited by 1,340
Systematic Reviews of Diagnostic Test Accuracy
Leeflang, M. M. G., Deeks, J. J., Gatsonis, C., & Bossuyt, P. M. M. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-149-12-200812160-00008 · Cited by 1,124
Conducting systematic reviews of diagnostic studies: didactic guidelines
Devillé, W. L., Buntinx, F., Bouter, L. M., Montori, V. M., de Vet, H. C., van der Windt, D. A., & Bezemer, P. D. (2002) · BMC Medical Research Methodology · doi:10.1186/1471-2288-2-9 · Cited by 960
Systematic reviews of evaluations of diagnostic and screening tests
(2001) · Europe PMC (PubMed Central) · Cited by 850
How to identify randomized controlled trials in MEDLINE: ten years on.
Glanville, J., Lefebvre, C., Miles, J. N. V., & Camosso‐Stefinovic, J. (2006) · Europe PMC (PubMed Central) · Cited by 833
Self-citationRobinson, K. A., & Dickersin, K. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.150 · Cited by 833
Sources of Variation and Bias in Studies of Diagnostic Accuracy: A Systematic Review
Whiting, P., Rutjes, A. W. S., Reitsma, J. B., Glas, A. S., Bossuyt, P. M. M., & Kleijnen, J. (2004) · Annals of Internal Medicine · doi:10.7326/0003-4819-140-3-200402030-00010 · Cited by 798
Developing Optimal Search Strategies for Detecting Clinically Sound Studies in MEDLINE
Haynes, R. B., Wilczynski, N., McKibbon, K. A., Walker, C. J., & Sinclair, J. C. (1994) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1994.95153434 · Cited by 765
Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy
(2023) · doi:10.1002/9781119756194 · Cited by 757
Royle, P., & Milne, R. (2003) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462303000552 · Cited by 693
Evidence of bias and variation in diagnostic accuracy studies
Rutjes, A. W. S., Reitsma, J. B., Di Nisio, M., Smidt, N., van Rijn, J. C., & Bossuyt, P. M. M. (2006) · Canadian Medical Association Journal · doi:10.1503/cmaj.050090 · Cited by 611
Haynes, R. B., & Wilczynski, N. L. (2004) · BMJ · doi:10.1136/bmj.38068.557998.ee · Cited by 547
Evaluation of diagnostic tests when there is no gold standard. A review of methods
Rutjes, A., Reitsma, J., Coomarasamy, A., Khan, K., & Bossuyt, P. (2007) · Health Technology Assessment · doi:10.3310/hta11500 · Cited by 524
Diagnostic test accuracy may vary with prevalence: implications for evidence-based diagnosis
Leeflang, M. M. G., Bossuyt, P. M. M., & Irwig, L. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.04.007 · Cited by 482
Sample size calculation should be performed for design accuracy in diagnostic test studies
Flahault, A., Cadilhac, M., & Thomas, G. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.12.009 · Cited by 468
Haynes, R. B., McKibbon, K. A., Wilczynski, N. L., Walter, S. D., & Werre, S. R. (2005) · BMJ · doi:10.1136/bmj.38446.498542.8f · Cited by 439
Dinnes, J., Deeks, J., Kirby, J., & Roderick, P. (2005) · Health Technology Assessment · doi:10.3310/hta9120 · Cited by 438
Optimal search strategies for retrieving systematic reviews from Medline: analytical survey
Montori, V. M., Wilczynski, N. L., Morgan, D., & Haynes, R. B. (2005) · BMJ · doi:10.1136/bmj.38336.804167.47 · Cited by 422
Reitsma, J. B., Rutjes, A. W. S., Khan, K. S., Coomarasamy, A., & Bossuyt, P. M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.02.005 · Cited by 405
Development and validation of methods for assessing the quality of diagnostic accuracy studies
Whiting, P., Rutjes, A., Dinnes, J., Reitsma, J., Bossuyt, P., & Kleijnen, J. (2004) · Health Technology Assessment · doi:10.3310/hta8250 · Cited by 372
Developing optimal search strategies for detecting clinically sound treatment studies in EMBASE.
Wong, S. S.-L., Wilczynski, N. L., & Haynes, R. B. (2006) · PubMed · Cited by 304
Evaluation of diagnostic tests without gold standards
Hui, S. L., & Zhou, X. H. (1998) · Statistical Methods in Medical Research · doi:10.1177/096228029800700404 · Cited by 252
Handsearching versus electronic searching to identify reports of randomized trials
Hopewell, S., Clarke, M. J., Lefebvre, C., & Scherer, R. W. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000001.pub2 · Cited by 247
Self-citationThe quality of diagnostic accuracy studies since the STARD statement: Has it improved?
Smidt, N., Rutjes, A. W. S., van der Windt, D. A. W. M., Ostelo, R. W. J. G., Bossuyt, P. M., Reitsma, J. B., Bouter, L. M., & de Vet, H. C. W. (2006) · Neurology · doi:10.1212/01.wnl.0000238386.41398.30 · Cited by 235
Evaluation of methodological search filters—a review
Jenkins, M. (2004) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2004.00511.x · Cited by 202
Whiting, P., Westwood, M., Bojke, L., Palmer, S., Richardson, G., Cooper, J., Watt, I., Glanville, J., Sculpher, M., & Kleijnen, J. (2006) · Health Technology Assessment · doi:10.3310/hta10360 · Cited by 201
Self-citationRoyle, P., & Waugh, N. (2003) · Health Technology Assessment · doi:10.3310/hta7340 · Cited by 194
Sample sizes of studies on diagnostic accuracy: literature survey.
Bachmann, L. M., Puhan, M. A., Riet, G. T., & Bossuyt, P. M. (2006) · BMJ · doi:10.1136/bmj.38793.637789.2f · Cited by 184
Taking advantage of the explosion of systematic reviews: an efficient MEDLINE search strategy.
Shojania, K. G., & Bero, L. A. (2001) · PubMed · Cited by 176
Identifying Diagnostic Studies in MEDLINE: Reducing the Number Needed to Read
Bachmann, L. M. (2002) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1124 · Cited by 173
Leeflang, M. M. G., Scholten, R. J. P. M., Rutjes, A. W. S., Reitsma, J. B., & Bossuyt, P. M. M. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.07.014 · Cited by 162
Publications on diagnostic test evaluation in family medicine journals
Devillé, W. L. J. M., Bezemer, P. D., & Bouter, L. M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00144-4 · Cited by 161
Bramwell, R., West, H., & Salmon, P. (2006) · BMJ · doi:10.1136/bmj.38884.663102.ae · Cited by 154
Reporting of precision of estimates for diagnostic accuracy: a review
Harper, R., & Reeves, B. (1999) · BMJ · doi:10.1136/bmj.318.7194.1322 · Cited by 147
Communicating accuracy of tests to general practitioners: a controlled study
Steurer, J., Fischer, J. E., Bachmann, L. M., Koller, M., & ter Riet, G. (2002) · BMJ · doi:10.1136/bmj.324.7341.824 · Cited by 147
Search strategies to identify diagnostic accuracy studies in MEDLINE and EMBASE
Beynon, R., Leeflang, M. M. G., McDonald, S., Eisinga, A., Mitchell, R. L., Whiting, P., & Glanville, J. M. (2013) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000022.pub3 · Cited by 145
Self-citationSampson, M., Zhang, L., Morrison, A., Barrowman, N. J., Clifford, T. J., Platt, R. W., Klassen, T. P., & Moher, D. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-33 · Cited by 131
Wong, S. S.-L., Wilczynski, N. L., & Haynes, R. B. (2006) · PubMed Central · Cited by 127
Retrieving randomized controlled trials from medline : a comparison of 38 published search filters
McKibbon, K. A., Wilczynski, N. L., Haynes, R. B., & for the Hedges Team (2009) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2008.00827.x · Cited by 126
Wilczynski, N. L., & Haynes, R. B. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.04.001 · Cited by 120
Whiting, P. F., Westwood, M. E., Beynon, R., Burke, M., Sterne, J. A. C., & Glanville, J. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.07.006 · Cited by 117
Self-citationReporting of measures of accuracy in systematic reviews of diagnostic literature
Honest, H., & Khan, K. S. (2002) · BMC Health Services Research · doi:10.1186/1472-6963-2-4 · Cited by 112
Eady, A., Wilczynski, N. L., & Haynes, R. B. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.09.016 · Cited by 111
Boynton, J., Glanville, J., McDaid, D., & Lefebvre, C. (1998) · Journal of Information Science · doi:10.1177/016555159802400301 · Cited by 111
Self-citationBoluyt, N., Tjosvold, L., Lefebvre, C., Klassen, T. P., & Offringa, M. (2008) · Archives of Pediatrics & Adolescent Medicine · doi:10.1001/archpediatrics.2007.40 · Cited by 108
Self-citationDoust, J. A., Pietrzak, E., Sanders, S., & Glasziou, P. P. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.09.011 · Cited by 106
Routine development of objectively derived search strategies
Hausner, E., Waffenschmidt, S., Kaiser, T., & Simon, M. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-19 · Cited by 98
Optimal search strategies for detecting health services research studies in MEDLINE
Wilczynski, N. L. (2004) · Canadian Medical Association Journal · doi:10.1503/cmaj.1040512 · Cited by 94
Lefebvre, C., Eisinga, A., McDonald, S., & Paul, N. (2008) · Emerging Themes in Epidemiology · doi:10.1186/1742-7622-5-13 · Cited by 93
Self-citationWhite, V. J., Glanville, J. M., Lefebvre, C., & Sheldon, T. A. (2001) · Journal of Information Science · doi:10.1177/016555150102700601 · Cited by 92
Self-citationHow does study quality affect the results of a diagnostic meta-analysis?
Westwood, M. E., Whiting, P. F., & Kleijnen, J. (2005) · BMC Medical Research Methodology · doi:10.1186/1471-2288-5-20 · Cited by 87
So many filters, so little time: the development of a search filter appraisal checklist
Glanville, J., Bayliss, S., Booth, A., Dundar, Y., Fernandes, H., Fleeman, N. D., Foster, L., Fraser, C., Fry-Smith, A., Golder, S., et al. (2008) · Journal of the Medical Library Association : JMLA · doi:10.3163/1536-5050.96.4.011 · Cited by 87
Self-citationOptimal CINAHL Search Strategies for Identifying Therapy Studies and Review Articles
Wong, S. S. ‐., Wilczynski, N. L., & Haynes, R. B. (2006) · Journal of Nursing Scholarship · doi:10.1111/j.1547-5069.2006.00100.x · Cited by 85
McGinnis, P. Q., Hack, L. M., Nixon-Cave, K., & Michlovitz, S. L. (2009) · Physical Therapy · doi:10.2522/ptj.20080131 · Cited by 83
Precision of healthcare systematic review searches in a cross‐sectional sample
Sampson, M., Tetzlaff, J., & Urquhart, C. (2011) · Research Synthesis Methods · doi:10.1002/jrsm.42 · Cited by 80
Systematic reviews of diagnostic tests in cancer: review of methods and reporting
Mallett, S. (2006) · BMJ · doi:10.1136/bmj.38895.467130.55 · Cited by 80
Whiting, P., Rutjes, A. W. S., Dinnes, J., Reitsma, J. B., Bossuyt, P. M. M., & Kleijnen, J. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.04.008 · Cited by 79
Cahan, A., Gilon, D., Manor, O., & Paltiel, O. (2003) · QJM · doi:10.1093/qjmed/hcg122 · Cited by 74
Sassi, F., Archard, L., & McDaid, D. (2002) · Medical Care · doi:10.1097/00005650-200205000-00004 · Cited by 73
Handsearching still a valuable element of the systematic review
Richards, D. (2008) · Evidence-Based Dentistry · doi:10.1038/sj.ebd.6400602 · Cited by 71
Wilczynski, N. L., Haynes, R. B., & the Hedges Team (2005) · BMC Medicine · doi:10.1186/1741-7015-3-7 · Cited by 71
Cook, C., Cleland, J., & Huijbregts, P. (2007) · Journal of Manual & Manipulative Therapy · doi:10.1179/106698107790819945 · Cited by 69
Graphical presentation of diagnostic information
Whiting, P. F., Sterne, J. A. C., Westwood, M. E., Bachmann, L. M., Harbord, R. M., Egger, M., & Deeks, J. J. (2008) · BMC Medical Research Methodology · doi:10.1186/1471-2288-8-20 · Cited by 68
Smidt, N., Rutjes, A. W., van der Windt, D. A., Ostelo, R. W., Bossuyt, P. M., Reitsma, J. B., Bouter, L. M., & de Vet, H. C. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-12 · Cited by 66
Castro, A. A., Clark, O. A. C., & Atallah, A. N. (1997) · Sao Paulo Medical Journal · doi:10.1590/s1516-31801997000300004 · Cited by 64
Wilczynski, N. L. (2008) · Radiology · doi:10.1148/radiol.2483072067 · Cited by 63
Taljaard, M., McGowan, J., Grimshaw, J. M., Brehaut, J. C., McRae, A., Eccles, M. P., & Donner, A. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-15 · Cited by 62
How well do search filters perform in identifying economic evaluations in MEDLINE and EMBASE
Glanville, J., Kaunelis, D., & Mensinkai, S. (2009) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462309990523 · Cited by 62
Self-citationVincent, S., Greenley, S., & Beaven, O. (2003) · Health Information & Libraries Journal · doi:10.1046/j.1365-2532.2003.00427.x · Cited by 55
Royle, P., & Waugh, N. (2005) · BMC Medical Research Methodology · doi:10.1186/1471-2288-5-23 · Cited by 55
Choosing a control intervention for a randomised clinical trial
Mann, H., & Djulbegović, B. (2003) · BMC Medical Research Methodology · doi:10.1186/1471-2288-3-7 · Cited by 54
Optimizing search strategies to identify randomized controlled trials in MEDLINE
Zhang, L., Ajiferuke, I., & Sampson, M. (2006) · BMC Medical Research Methodology · doi:10.1186/1471-2288-6-23 · Cited by 53
Agoritsas, T., Courvoisier, D. S., Combescure, C., Deom, M., & Perneger, T. (2010) · Journal of General Internal Medicine · doi:10.1007/s11606-010-1540-5 · Cited by 52
Systematic reviews of epidemiology in diabetes: finding the evidence
Royle, P., Bain, L., & Waugh, N. (2005) · BMC Medical Research Methodology · doi:10.1186/1471-2288-5-2 · Cited by 51
Coppus, S. F. P. J., van der Veen, F., Bossuyt, P. M., & Mol, B. W. (2006) · Fertility and Sterility · doi:10.1016/j.fertnstert.2006.03.050 · Cited by 47
Kastner, M., Wilczynski, N. L., McKibbon, A., Garg, A. X., & Haynes, R. B. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.11.006 · Cited by 46
The quality of reporting of diagnostic accuracy studies published in ophthalmic journals
Siddiqui, M. A. R., Azuara-Blanco, A., & Burr, J. (2005) · British Journal of Ophthalmology · doi:10.1136/bjo.2004.051862 · Cited by 44
Willis, B. H., & Quigley, M. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-27 · Cited by 43
Optimal search strategies for detecting cost and economic studies in EMBASE
McKinlay, R. J., Wilczynski, N. L., Haynes, R. B., & the Hedges Team (2006) · BMC Health Services Research · doi:10.1186/1472-6963-6-67 · Cited by 43
Manríquez, J. J. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.06.009 · Cited by 42
Do published search filters to identify diagnostic test accuracy studies perform adequately?
Ritchie, G., Glanville, J., & Lefebvre, C. (2007) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2007.00735.x · Cited by 39
Self-citationIdentifying diagnostic accuracy studies in EMBASE.
Bachmann, L. M., Estermann, P., Kronenberg, C., & ter Riet, G. (2003) · Europe PMC (PubMed Central) · Cited by 37
GPs' and physicians' interpretation of risks, benefits and diagnostic test results
Heller, R. F. (2004) · Family Practice · doi:10.1093/fampra/cmh209 · Cited by 35
Consistency and accuracy of indexing systematic review articles and meta‐analyses in medline
Wilczynski, N. L., & Haynes, R. B. (2009) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2008.00823.x · Cited by 34
Eisinga, A., Siegfried, N., & Clarke, M. J. (2007) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2007.00698.x · Cited by 31
Sample size in studies on diagnostic accuracy in ophthalmology: a literature survey
Bochmann, F., Johnson, Z. K., & Azuara‐Blanco, A. (2007) · British Journal of Ophthalmology · doi:10.1136/bjo.2006.113290 · Cited by 31
The Quality of Reporting in Diagnostic Test Research: Getting Better, Still Not Optimal
Bossuyt, P. M. (2004) · Clinical Chemistry · doi:10.1373/clinchem.2003.029736 · Cited by 30
A pragmatic critical appraisal instrument for search filters: introducing the CADTH CAI
Bak, G., Mierzwinski‐Urban, M., Fitzsimmons, H., Morrison, A., & Maden‐Jenkins, M. (2008) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2008.00830.x · Cited by 30
Controversies in Diagnosis of Pulmonary Embolism
Stein, P. D., Sostman, H. D., Dalen, J. E., Bailey, D. L., Bajc, M., Goldhaber, S. Z., Goodman, L. R., GOTTSCHALK, A. R., Hull, R. D., Matta, F., et al. (2010) · Clinical and Applied Thrombosis/Hemostasis · doi:10.1177/1076029610389027 · Cited by 30
Deurenberg, R., Vlayen, J., Guillo, S., Oliver, T. K., Fervers, B., Burgers, J. S., & on behalf of the SEARCH Group (2007) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2007.00732.x · Cited by 23
Corrao, S., Colomba, D., Arnone, S., Argano, C., Di Chiara, T., Scaglione, R., & Licata, G. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2084 · Cited by 23
Shunmugam, M., & Azuara‐Blanco, A. (2006) · Investigative Ophthalmology & Visual Science · doi:10.1167/iovs.05-1250 · Cited by 22
Royle, P. L., & Waugh, N. R. (2007) · Diabetic Medicine · doi:10.1111/j.1464-5491.2007.02046.x · Cited by 21
Astin, M., Brazzelli, M. G., Fraser, C. M., Counsell, C. E., Needham, G., & Grimshaw, J. (2008) · Radiology · doi:10.1148/radiol.2472070101 · Cited by 21
Self-citationPhysicians' Attitudes toward Health Care Rationing
Perneger, T., Martin, D. P., & BOVIER, P. A. (2002) · Medical Decision Making · doi:10.1177/0272989x0202200106 · Cited by 18
Quality of Reporting of Orthopaedic Diagnostic Accuracy Studies is Suboptimal
Rama, K. R. B. S., Poovali, S., & Apsingi, S. (2006) · Clinical Orthopaedics and Related Research · doi:10.1097/01.blo.0000205906.44103.a3 · Cited by 18
Reporting methodological search filter performance comparisons: a literature review
Harbour, J., Fraser, C., Lefebvre, C., Glanville, J., Beale, S., Boachie, C., Duffy, S., McCool, R., & Smith, L. (2014) · Health Information & Libraries Journal · doi:10.1111/hir.12070 · Cited by 18
Self-citationDay, D., Furlan, A. D., Irvin, E., & Bombardier, C. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.02.005 · Cited by 17
de Freitas, A. E., Herbert, R., Latimer, J., & Ferreira, P. H. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.06.014 · Cited by 16
Berg, A., Fleischer, S., & Behrens, J. (2005) · International Journal of Nursing Terminologies and Classifications · doi:10.1111/j.1744-618x.2005.00006.x · Cited by 16
Choosing and using methodological search filters: searchers' views
Beale, S., Duffy, S., Glanville, J., Lefebvre, C., Wright, D., McCool, R., Varley, D., Boachie, C., Fraser, C. M., Harbour, J., et al. (2014) · Health Information & Libraries Journal · doi:10.1111/hir.12062 · Cited by 16
Self-citationThe influence of types of decision support on physicians' decision making
Sox, C. M., Doctor, J. N., Koepsell, T. D., & Christakis, D. (2009) · Archives of Disease in Childhood · doi:10.1136/adc.2008.141903 · Cited by 16
Jha, S. K., Ho, A., Bhargavan, M., Owen, J. B., & Sunshine, J. H. (2009) · American Journal of Roentgenology · doi:10.2214/ajr.09.2694 · Cited by 16
Search Strategies for Retrieving Complementary and Alternative Medicine Clinical Trials in Oncology
Bardia, A., Wahner‐Roedler, D. L., Erwin, P. L., & Sood, A. (2006) · Integrative Cancer Therapies · doi:10.1177/1534735406292146 · Cited by 15
Examination of the Clinical Queries and Systematic Review “hedges” in EMBASE and MEDLINE
Bradley, S. (2011) · Journal of the Canadian Health Libraries Association / Journal de l Association de bilbiothèques de la santé du Canada · doi:10.5596/c10-022 · Cited by 15
Chow, T. K. F., To, E., Goodchild, C. S., & McNeil, J. J. (2004) · Anesthesia & Analgesia · doi:10.1213/01.ane.0000114071.78448.2d · Cited by 12
Family physicians' choices of and opinions on colorectal cancer screening modalities.
Zettler, M., Mollon, B., da Silva, V., Howe, B., Speechley, M. R., & Vinden, C. (2010) · PubMed · Cited by 12
Jenkins, M. L., & Johnson, F. (2004) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2004.00480.x · Cited by 11
McDonald, S. J. (2002) · Health Information & Libraries Journal · doi:10.1046/j.0265-6647.2001.00359.x · Cited by 11
Pediatricians' Clinical Decision Making
Sox, C. M., Koepsell, T. D., Doctor, J. N., & Christakis, D. (2006) · Archives of Pediatrics and Adolescent Medicine · doi:10.1001/archpedi.160.5.487 · Cited by 11
Searching a biomedical bibliographic database from Hungary—the ‘Magyar Orvosi Bibliográfia’
Kele, I., Bereczki, D., Furtado, V. A., Wright, J. M., & Adams, C. E. (2005) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2005.00577.x · Cited by 8
Searching a biomedical bibliographic database from the Ukraine: the Panteleimon database
Kumar, A., Wright, J. M., & Adams, C. E. (2005) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2005.00578.x · Cited by 7
Morris, R. K., Selman, T. J., Zamora, J., & Khan, K. S. (2011) · BMC Women s Health · doi:10.1186/1472-6874-11-7 · Cited by 7
Searching a biomedical bibliographic database from Bulgaria: the ABS database
Abhijnhan, A., Surcheva, Z., Wright, J. M., & Adams, C. E. (2007) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2007.00723.x · Cited by 6
Searching the Polish Medical Bibliography (Polska Bibliografia Lekarska) for trials
Almerie, M. Q., Matar, H. E., Jones, V., Kumar, A., Wright, J., Włostowska, E., & Adams, C. E. (2007) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2007.00716.x · Cited by 6
Using Decision Tree Models to Depict Primary Care Physicians CRC Screening Decision Heuristics
Wackerbarth, S. B., Tarasenko, Y. N., Curtis, L. A., Joyce, J. M., & Haist, S. A. (2007) · Journal of General Internal Medicine · doi:10.1007/s11606-007-0338-6 · Cited by 4
Evidence based library and information practice
Koufogiannakis, D. (2014) · cIRcle (University of British Columbia) · doi:10.14288/1.0042692 · Cited by 4
Maheswari, U. (2005) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2005.00617.x · Cited by 2
Research works printed but not linked: 16
- Glanville, 2008, ISSG Search Filter Resource
- Glanville, 2009, Development and Testing of Search Filters to Identify Economic Evaluations in MEDLINE and EMBASE
- Bossuyt, 2008, Cochrane Handbook of Systematic Reviews of Diagnostic Test Accuracy Version 0.4
- Food and Drug Administration, 2007, Guidance for Industry and Staff: Statistical Guidance on Reporting Results from Studies Evaluating Diagnostic Tests
- Medical Services Advisory Committee, 2005, Guidelines for the Assessment of Diagnostic Technologies
- Belgian Health Care Knowledge Centre, 2005, HTA: Molecular Diagnostics in Belgium
- Agency for Healthcare Research and Quality, 2011, Comprehensive Overview of the Methods and Reporting of Meta-analyses of Test Accuracy
- Belgian Health Care Knowledge C, 2007, Search for Evidence and Critical Appraisal: Health Technology Assessment
- National Institute for Health and Care Excellence, 2010, Interim Methods Statement: Centre for Health Technology Evaluation, Diagnostics Assessment Programme
- UK National Screening Committee, 2011, Criteria for Appraising the Viability, Effectiveness and Appropriateness of a Screening Programme
- US Preventive Services Task Force, 2008, Procedure Manual
- Australian Population Health Development Principal Committee Screening Subcommittee, 2008, Population Based Screening Framework
- World Health Organization, 2011, Screening for Various Cancers
- Lefebvre, 2011, Cochrane Handbook for Systematic Reviews of Interventions
- Noyes, 2011, Cochrane Handbook for Systematic Reviews of Interventions
- McGowan, 2016, PRESS – Peer Review of Electronic Search Strategies: 2015 Guideline Explanation and Elaboration (PRESS E&E)
Cited by
Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., Koffel, J. B., PRISMA-S Group, Blunt, H., Brigham, T., et al. (2021) · Systematic Reviews · doi:10.1186/s13643-020-01542-z · Cited by 3,714
Self-citationRethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., & Koffel, J. (2019) · doi:10.31219/osf.io/sfc38 · Cited by 3,696
Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., Koffel, J. B., & Group, P.-S. (2021) · Journal of the Medical Library Association · doi:10.5195/jmla.2021.962 · Cited by 409
Point-of-care ultrasonography for diagnosing thoracoabdominal injuries in patients with blunt trauma
Stengel, D., Leisterer, J., Ferrada, P. A., Ekkernkamp, A., Mutze, S., & Hönning, A. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012669.pub2 · Cited by 189
Prady, S. L., Uphoff, E., Power, M., & Golder, S. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0567-x · Cited by 75
Metadata
- Authors
- Lefebvre, C., Glanville, J., Beale, S., Boachie, C., Duffy, S., Fraser, C., Harbour, J., McCool, R., & Smith, L.
- Affiliations
- Cochrane (GB)
- York Health Economics Consortium, York, UK
- University of Aberdeen (GB)
- Healthcare Improvement Scotland (GB)
- Corresponding author
- Lefebvre, Carol
- ORCID
- Venue
- Health Technology Assessment
- Published
- 2017-11
- Volume
- 21
- Issue
- 69
- Pages
- 1-148
- ISSN
- 1366-5278 (ISSN-L)
- 2046-4924
- PMC
- PMC5723931
- OpenAlex
- W2774248142
- Cited by
- 49
- References
- 130
- Citations by year
- 2026: 4 · 2025: 6 · 2024: 3 · 2023: 6 · 2022: 3 · 2021: 12 · 2020: 6 · 2019: 6 · 2018: 3
- FWCI
- 3.67
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Health Technology Assessment (publishedVersion, cc-by)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Health Technology Assessment programme (06/92/58)
- Medical Research Council
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Keywords
- reference materials, diagnostic accuracy, sensitivity and specificity, evidence retrieval, information professionals, critical appraisal
- MeSH terms
- Humans, Surveys and Questionnaires, Review Literature as Topic, Technology Assessment, Biomedical, Databases, Bibliographic, Information Storage and Retrieval, Qualitative Research, Search Engine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.