The DOI matches its Crossref record; 44 of the 45 references it cites were found in a registry or an index.
Search Filters for Finding Prognostic and Diagnostic Prediction Studies in Medline to Enhance Systematic Reviews
Open accessAbstract
BACKGROUND: The interest in prognostic reviews is increasing, but to properly review existing evidence an accurate search filer for finding prediction research is needed. The aim of this paper was to validate and update two previously introduced search filters for finding prediction research in Medline: the Ingui filter and the Haynes Broad filter. METHODOLOGY/PRINCIPAL FINDINGS: Based on a hand search of 6 general journals in 2008 we constructed two sets of papers. Set 1 consisted of prediction research papers (n = 71), and set 2 consisted of the remaining papers (n = 1133). Both search filters were validated in two ways, using diagnostic accuracy measures as performance measures. First, we compared studies in set 1 (reference) with studies retrieved by the search strategies as applied in Medline. Second, we compared studies from 4 published systematic reviews (reference) with studies retrieved by the search filter as applied in Medline. Next--using word frequency methods--we constructed an additional search string for finding prediction research. Both search filters were good in identifying clinical prediction models: sensitivity ranged from 0.94 to 1.0 using our hand search as reference, and 0.78 to 0.89 using the systematic reviews as reference. This latter performance measure even increased to around 0.95 (range 0.90 to 0.97) when either search filter was combined with the additional string that we developed. Retrieval rate of explorative prediction research was poor, both using our hand search or our systematic review as reference, and even combined with our additional search string: sensitivity ranged from 0.44 to 0.85. CONCLUSIONS/SIGNIFICANCE: Explorative prediction research is difficult to find in Medline, using any of the currently available search filters. Yet, application of either the Ingui filter or the Haynes broad filter results in a very low number missed clinical prediction model studies.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Previous guidance for systematic reviews of prediction model studies mainly addressed formulation of the review aim, searching,12 critical appraisal (CHARMS)13 and data extraction of relevant studies. There is, however, little guidance on how to quantitatively synthesize the results of external validation studies.”
From A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes (2019) · cites this paper as [12]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2019) · Statistics for Biology and Health · doi:10.1007/978-3-030-16399-0 · Cited by 3,281
Users' Guides to the Medical Literature
Guyatt, G. H. (1993) · JAMA · doi:10.1001/jama.270.17.2096 · Cited by 3,083
Systematic Reviews: Identifying relevant studies for systematic reviews
Dickersin, K., Scherer, R. W., & Lefebvre, C. (1994) · BMJ · doi:10.1136/bmj.309.6964.1286 · Cited by 2,255
Evaluation of the Quality of Prognosis Studies in Systematic Reviews
Hayden, J. A., Côté, P., & Bombardier, C. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-6-200603210-00010 · Cited by 1,452
Clinical Prediction Rules: Applications and Methodological Standards
Wasson, J. H., Sox, H. C., Neff, R. K., & Goldman, L. (1985) · New England Journal of Medicine · doi:10.1056/nejm198509263131306 · Cited by 1,357
Prognosis and prognostic research: what, why, and how?
Moons, K. G. M., Royston, P., Vergouwe, Y., Grobbee, D. E., & Altman, D. G. (2009) · BMJ · doi:10.1136/bmj.b375 · Cited by 1,337
Self-citationPrognosis and prognostic research: Developing a prognostic model
Royston, P., Moons, K. G. M., Altman, D. G., & Vergouwe, Y. (2009) · BMJ · doi:10.1136/bmj.b604 · Cited by 1,172
Self-citationCriteria for Evaluation of Novel Markers of Cardiovascular Risk
Hlatky, M. A., Greenland, P., Arnett, D. K., Ballantyne, C. M., Criqui, M. H., Elkind, M. S. V., Go, A. S., Harrell, F. E., Hong, Y., Howard, B. V., et al. (2009) · Circulation · doi:10.1161/circulationaha.109.192278 · Cited by 1,146
Users' Guides to the Medical Literature
McGinn, T. G., Guyatt, G., Wyer, P. C., Naylor, C. D., Stiell, I. G., Richardson, W. S., & for the Evidence-Based Medicine Working Group (2000) · JAMA · doi:10.1001/jama.284.1.79 · Cited by 1,009
Prognosis and prognostic research: application and impact of prognostic models in clinical practice
Moons, K. G. M., Altman, D. G., Vergouwe, Y., & Royston, P. (2009) · BMJ · doi:10.1136/bmj.b606 · Cited by 934
Self-citationAutomatic recognition of multi-word terms:. the C-value/NC-value method
Frantzi, K. T., Ananiadou, S., & Mima, H. (2000) · International Journal on Digital Libraries · doi:10.1007/s007999900023 · Cited by 833
Reilly, B. M., & Evans, A. T. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-3-200602070-00009 · Cited by 794
Laupacis, A. (1997) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1997.03540300056034 · Cited by 793
Systematic reviews of evaluations of prognostic variables
Altman, D. G. (2001) · BMJ · doi:10.1136/bmj.323.7306.224 · Cited by 756
Clinical Prediction Models—a Practical Approach to Development, Validation and Updating
Bedogni, G. (2009) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/j.1467-985x.2009.00614_16.x · Cited by 595
Users' Guides to the Medical Literature
Laupacis, A. (1994) · JAMA · doi:10.1001/jama.272.3.234 · Cited by 579
Haynes, R. B., & Wilczynski, N. L. (2004) · BMJ · doi:10.1136/bmj.38068.557998.ee · Cited by 549
Haynes, R., McKibbon, K., Wilczynski, N., Walter, S. D., & Werre, S. R. (2005) · British medical journal · doi:10.1136/bmj.38446.498542.8f · Cited by 500
Clinical prediction rules for pulmonary embolism: a systematic review and meta-analysis
Ceriani, E., Combescure, C., Gal, G. L., NENDAZ, M. R., Perneger, T., Bounameaux, H. M., Perrier, A., & Righini, M. (2010) · Journal of Thrombosis and Haemostasis · doi:10.1111/j.1538-7836.2010.03801.x · Cited by 332
Multicentre trial to introduce the Ottawa ankle rules for use of radiography in acute ankle injuries
Stiell, I. G., Wells, G. A., Laupacis, A., Brison, R., Verbeek, P. R., Vandemheen, K. L., & Naylor, C. D. (1995) · BMJ · doi:10.1136/bmj.311.7005.594 · Cited by 294
Systematic Review of Prognostic Models in Patients with Acute Stroke
Counsell, C. E., & Dennis, M. S. (2001) · Cerebrovascular Diseases · doi:10.1159/000047699 · Cited by 252
Almost all articles on cancer prognostic markers report statistically significant results
Kyzas, P., Denaxa-Kyza, D., & Ioannidis, J. P. A. (2007) · European Journal of Cancer · doi:10.1016/j.ejca.2007.08.030 · Cited by 233
Searching for Clinical Prediction Rules in MEDLINE
Ingui, B. J., & Rogers, M. A. M. (2001) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2001.0080391 · Cited by 183
Identifying Diagnostic Studies in MEDLINE: Reducing the Number Needed to Read
Bachmann, L. M. (2002) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1124 · Cited by 173
Riley, R. D., Sauerbrei, W., & Altman, D. G. (2009) · British Journal of Cancer · doi:10.1038/sj.bjc.6604999 · Cited by 164
Ten steps towards improving prognosis research
Hemingway, H., Riley, R. D., & Altman, D. G. (2009) · BMJ · doi:10.1136/bmj.b4184 · Cited by 160
Predicting Clinical States in Individual Patients
Braitman, L. E., & Davidoff, F. (1996) · Annals of Internal Medicine · doi:10.7326/0003-4819-125-5-199609010-00008 · Cited by 157
Geersing, G., Janssen, K. J. M., Oudega, R., Bax, L., Hoes, A. W., Reitsma, J. B., & Moons, K. G. M. (2009) · BMJ · doi:10.1136/bmj.b2990 · Cited by 155
Self-citationSerrano, L. A., Hess, E. P., Bellolio, M. F., Murad, M. H., Montori, V. M., Erwin, P. J., & Decker, W. W. (2010) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2010.05.013 · Cited by 129
Biomarker studies: a call for a comprehensive biomarker study registry
André, F., McShane, L. M., Michiels, S., Ransohoff, D. F., Altman, D. G., Reis-Filho, J. S., Hayes, D. F., & Pusztai, L. (2011) · Nature Reviews Clinical Oncology · doi:10.1038/nrclinonc.2011.4 · Cited by 118
Developing optimal search strategies for detecting sound clinical prediction studies in MEDLINE.
Wong, S. S.-L., Wilczynski, N. L., Haynes, R. B., & Ramkissoonsingh, R. (2003) · PubMed · Cited by 100
Clinical Prediction Models. A Practical Approach to Development, Validation, and Updating
Steyerberg, E. W. (2009) · EUR Research Repository (Erasmus University Rotterdam) · Cited by 97
Williams, C., Brunskill, S. J., Altman, D. G., Briggs, A. H., Campbell, H., Clarke, M. J., Glanville, J., Gray, A., Harris, A. L., Johnston, K., et al. (2006) · Health Technology Assessment · doi:10.3310/hta10340 · Cited by 92
Developing optimal search strategies for detecting clinically sound causation studies in MEDLINE.
Wilczynski, N. L., & Haynes, R. B. (2003) · PubMed · Cited by 89
How to use an article about prognosis.
Hansebout, R. R., Cornacchi, S. D., Haines, T., & Goldsmith, C. H. (2009) · PubMed Central · Cited by 78
Cost‐effectiveness of ruling out deep venous thrombosis in primary care versus care as usual
ten Cate‐Hoek, A. J., Toll, D. B., Buller, H. R., Hoes, A. W., Moons, K. G. M., Oudega, R., Stoffers, J., van der Velde, E. F., van Weert, H. C. P. M., Prins, M. H., et al. (2009) · Journal of Thrombosis and Haemostasis · doi:10.1111/j.1538-7836.2009.03627.x · Cited by 72
Self-citationCriteria for Scientific Evaluation of Novel Markers: A Perspective
Moons, K. G. M. (2010) · Clinical Chemistry · doi:10.1373/clinchem.2009.134155 · Cited by 70
Self-citationPrognostication and intensive care unit outcome: the evolving role of scoring systems
Herridge, M. S. (2003) · Clinics in Chest Medicine · doi:10.1016/s0272-5231(03)00094-7 · Cited by 66
Strengthening the reporting of genetic risk prediction studies (GRIPS): explanation and elaboration
Janssens, A. C. J. W., Ioannidis, J. P. A., Bedrosian, S. R., Boffetta, P., Dolan, S. M., Dowling, N. F., Fortier, I., Freedman, A. N., Grimshaw, J., Gulcher, J. R., et al. (2011) · European Journal of Clinical Investigation · doi:10.1111/j.1365-2362.2011.02493.x · Cited by 44
Much Ado About (Doing) Nothing
Reilly, B. M., & Evans, A. T. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-150-4-200902170-00008 · Cited by 43
Geersing, G., Janssen, K. J. M., Oudega, R., van Weert, H. C. P. M., Stoffers, J., Hoes, A. W., Moons, K. G. M., & on behalf of the AMUSE Study Group (2010) · British Journal of General Practice · doi:10.3399/bjgp10x532387 · Cited by 36
Self-citationSlot, M. H. E. B., Rutten, F. H., van der Heijden, G. J. M. G., Geersing, G., Glatz, J. F. C., & Hoes, A. W. (2011) · Family Practice · doi:10.1093/fampra/cmq116 · Cited by 35
Self-citationStrengthening the reporting of genetic risk prediction studies (GRIPS): explanation and elaboration
Janssens, A. C. J. W., Ioannidis, J. P. A., Bedrosian, S. R., Boffetta, P., Dolan, S. M., Dowling, N. F., Fortier, I., Freedman, A. N., Grimshaw, J., Gulcher, J. R., et al. (2011) · European Journal of Human Genetics · doi:10.1038/ejhg.2011.27 · Cited by 33
Strengthening the reporting of Genetic RIsk Prediction Studies (GRIPS): explanation and elaboration
Janssens, A. C. J. W., Ioannidis, J. P. A., Bedrosian, S. R., Boffetta, P., Dolan, S. M., Dowling, N. F., Fortier, I., Freedman, A. N., Grimshaw, J., Gulcher, J. R., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.02.003 · Cited by 9
Cited by
Sorted by citations
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Self-citationMoons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001744 · Cited by 2,018
Self-citationMoons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S. (2019) · Annals of Internal Medicine · doi:10.7326/m18-1377 · Cited by 1,669
Self-citationGoldstein, B. A., Navar, A. M., Pencina, M. J., & Ioannidis, J. P. A. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw042 · Cited by 936
Repeatability and Reproducibility of Radiomic Features: A Systematic Review
Traverso, A., Wee, L., Dekker, A. L. A. J., & Gillies, R. J. (2018) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2018.05.053 · Cited by 966
Metadata
- Authors
- Geersing, G.-J., Bouwmeester, W., Zuithoff, P., Spijker, R., Leeflang, M., & Moons, K.
- Affiliations
- University Medical Center Utrecht (NL)
- Amsterdam UMC Location University of Amsterdam (NL)
- Corresponding author
- Geersing, Geert-Jan
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2012-02-29
- Volume
- 7
- Issue
- 2
- Pages
- e32844
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3290602
- OpenAlex
- W2019982728
- Cited by
- 350
- References
- 45
- Citations by year
- 2026: 20 · 2025: 26 · 2024: 39 · 2023: 39 · 2022: 47 · 2021: 47 · 2020: 32 · 2019: 23 · 2018: 24 · 2017: 16 · 2016: 14 · 2015: 11 · 2014: 10 · 2013: 2
- FWCI
- 2.03
- Type
- journal article
- Language
- en
- Update notices
- correction (2012-07-10)
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Keywords
- search filters, MEDLINE, prognostic research, diagnostic research, clinical prediction model, systematic reviews, search strategy validation
- MeSH terms
- Algorithms, Computers, Humans, Prognosis, ROC Curve, Software, Reproducibility of Results, Multivariate Analysis, Databases, Factual, MEDLINE, Computational Biology, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
