When and how to update systematic reviews
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
BACKGROUND: Systematic reviews are most helpful if they are up-to-date. We did a systematic review of strategies and methods describing when and how to update systematic reviews. OBJECTIVES: To identify, describe and assess strategies and methods addressing: 1) when to update systematic reviews and 2) how to update systematic reviews. SEARCH STRATEGY: We searched MEDLINE (1966 to December 2005), PsycINFO, the Cochrane Methodology Register (Issue 1, 2006), and hand searched the 2005 Cochrane Colloquium proceedings. SELECTION CRITERIA: We included methodology reports, updated systematic reviews, commentaries, editorials, or other short reports describing the development, use, or comparison of strategies and methods for determining the need for updating or updating systematic reviews in healthcare. DATA COLLECTION AND ANALYSIS: We abstracted information from each included report using a 15-item questionnaire. The strategies and methods for updating systematic reviews were assessed and compared descriptively with respect to their usefulness, comprehensiveness, advantages, and disadvantages. MAIN RESULTS: Four updating strategies, one technique, and two statistical methods were identified. Three strategies addressed steps for updating and one strategy presented a model for assessing the need to update. One technique discussed the use of the "entry date" field in bibliographic searching. Statistical methods were cumulative meta-analysis and predicting when meta-analyses are outdated. AUTHORS' CONCLUSIONS: Little research has been conducted on when and how to update systematic reviews and the feasibility and efficiency of the identified approaches is uncertain. These shortcomings should be addressed in future research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Systematic Reviews: Synthesis of Best Evidence for Clinical Decisions
Cook, D. J., Mulrow, C. D., & Haynes, R. B. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-126-5-199703010-00006 · Cited by 2,184
Updated Method Guidelines for Systematic Reviews in the Cochrane Collaboration Back Review Group
van Tulder, M., Furlan, A., Bombardier, C., & Bouter, L. (2003) · Spine · doi:10.1097/01.brs.0000065484.95996.af · Cited by 2,180
A Fail-Safe N for Effect Size in Meta-Analysis
Orwin, R. G. (1983) · Journal of Educational Statistics · doi:10.2307/1164923 · Cited by 2,064
Greenhalgh, T., & Peacock, R. (2005) · BMJ · doi:10.1136/bmj.38636.593461.68 · Cited by 2,045
Systematic Reviews in Health Care: Meta‐Analysis in Context
(2001) · doi:10.1002/9780470693926 · Cited by 1,693
Systematic Reviews: Rationale for systematic reviews
Mulrow, C. D. (1994) · BMJ · doi:10.1136/bmj.309.6954.597 · Cited by 1,677
Cumulative Meta-Analysis of Therapeutic Trials for Myocardial Infarction
Lau, J., Antman, E. M., Jimenez-Silva, J., Kupelnick, B., Mosteller, F., & Chalmers, T. C. (1992) · New England Journal of Medicine · doi:10.1056/nejm199207233270406 · Cited by 1,140
Jadad, A. R., Cook, D. J., Jones, A., Klassen, T. P., Tugwell, P., Moher, M., & Moher, D. (1998) · JAMA · doi:10.1001/jama.280.3.278 · Cited by 534
Self-citationPublication and related biases.
Song, Eastwood, Gilbody, Duley, & Sutton (2000) · Health Technology Assessment · doi:10.3310/hta4100 · Cited by 427
Pogue, J. M., & Yusuf, S. (1997) · Controlled Clinical Trials · doi:10.1016/s0197-2456(97)00051-2 · Cited by 415
Time to publication for results of clinical trials
Hopewell, S., Clarke, M. J., Stewart, L., & Tierney, J. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000011.pub2 · Cited by 243
Atkins, D., Fink, K., & Slutsky, J. (2005) · Annals of Internal Medicine · doi:10.7326/0003-4819-142-12_part_2-200506211-00002 · Cited by 129
Systematic reviews: when is an update an update?
Moher, D., & Tsertsvadze, A. (2006) · The Lancet · doi:10.1016/s0140-6736(06)68358-x · Cited by 110
Self-citation
Research works printed but not linked: 49
- Barrowman, 2003, Identifying null meta-analyses that are ripe for updating., BMC Medical Research Methodology, 3, 13, 10.1186/1471-2288-3-13 · DOI
- Baum, 1981, A survey of clinical trials of antibiotic prophylaxis in colon surgery: evidence against further use of no-treatment controls., The New England Journal of Medicine, 305, 795, 10.1056/NEJM198110013051404 · DOI · Paper page
- Berkey, 1996, Uncertainty of the time of first significance in random effects cumulative meta-analysis., Controlled Clinical Trials, 17, 357, 10.1016/S0197-2456(96)00014-1 · DOI
- Chalmers, 1993, Preparing and updating systematic reviews of randomized controlled trials of health care., Milbank Quarterly, 71, 411, 10.2307/3350409 · DOI
- Ioannidis, 1999, Recursive cumulative meta-analysis: a diagnostic for the evolution of total randomized evidence from group and individual patient data., Journal of Clinical Epidemiology, 52, 281, 10.1016/S0895-4356(98)00159-0 · DOI
- Ioannidis JP Lau J Evolution of treatment effects over time: Empirical insight from recursive cumulative meta-analyses. Proceedings of the National Academy of Sciences of the United States of America 2001 831 836 · doi:10.1073/pnas.98.3.831 · Paper page
- Lau, 1995, Cumulative meta-analysis of clinical trials builds evidence for exemplary medical care., Journal of Clinical Epidemiology, 48, 45, 10.1016/0895-4356(94)00106-Z · DOI · Paper page
- Mullen, 2001, Cumulative meta-analysis: a consideration of indicators of sufficiency and stability., Personality and Social Psychology Bulletin, 27, 1450, 10.1177/01461672012711006 · DOI · Paper page
- Sutton AJ Gamble C Donehue S Evaluating two statistical methods to predict when a meta-analysis will gain sufficient power to change the conclusions of a Cochrane review. XIV Cochrane Colloquium, Dublin, Ireland. 2006
- Weller D Managing and updating the evidence. Evidence-based Health Advice Workshop; November 4-5, 1998. Adelaide, Australia 1998 1 8
- Chapman A Middleton P Maddern G Early updates of systematic reviews - a waste of resources? 4th Symposium on Systematic Reviews: Pushing the Boundaries, July 2002 2002
- Counsell, 1994, Archie Cochrane's challenge: can periodically updated reviews of all randomised controlled trials relevant to neurology and neurosurgery be produced?, Journal of Neurology, Neurosurgery, and Psychiatry, 57, 529, 10.1136/jnnp.57.5.529 · DOI
- Eccles, 2002, Updating evidence-based clinical guidelines., Journal of Health Services Research and Policy, 7, 98, 10.1258/1355819021927746 · DOI
- Enkin, 1996, Collecting the evidence systematically. Ensuring that it is complete and up-to-date., International Journal of Technology Assessment in Health Care, 12, 276, 10.1017/S0266462300009624 · DOI
- McCormack, 2004, Value of updating a systematic review in surgery using individual patient data., British Journal of Surgery, 91, 495, 10.1002/bjs.4467 · DOI
- Mowatt, 1998, When is the 'right' time to initiate as assessment of a health technology., International Journal of Health Technology Assessment in Health Care, 14, 372, 10.1017/S0266462300012320 · DOI
- Stead, 2001, Updating a systematic review - what difference did it make? Case study of nicotine replacement therapy., BMC Medical Research Methodology, 1, 10, 10.1186/1471-2288-1-10 · DOI
- Urquhart, 1995, The value of information supplied to clinicians by health libraries: devising an outcomes-based assessment of the contribution of libraries to clinical decision-making., Health Libraries Review, 12, 201, 10.1046/j.1365-2532.1995.1230201.x · doi:10.1046/j.1365-2532.1995.1230201.x
- Alderson, 1997, Corticosteroids in acute traumatic brain injury: a systematic review of randomised trials., British Medical Journal, 314, 1855, 10.1136/bmj.314.7098.1855 · DOI · Paper page
- Chalmers, 1986, Electronic publications for updating controlled trial reviews., The Lancet, 2, 287, 10.1016/S0140-6736(86)92107-0 · DOI
- Chalmers, 1991, Problems induced by meta-analysis., Statistics in Medicine, 10, 971, 10.1002/sim.4780100618 · DOI
- Chalmers, 1994, Systematic reviews: reporting, updating, and correcting systematic reviews of the effects of health care., British Medical Journal, 309, 862, 10.1136/bmj.309.6958.862 · DOI · Paper page
- Chapman A Middleton P Maddern G Early updates of systematic reviews - a waste of resources? Pushing the Boundaries. Proceedings of the 4th Symposium on Systematic Reviews: July, 2002. Oxford, England 2002
- Claxton, 2004, A pilot study on the use of decision theory and value of information analysis as part of the NHS Health Technology Assessment programme., Health Technology Assessment, 8, 31, 10.3310/hta8310 · DOI · Paper page
- Clinton, 1994, Health Care policy and Research. Process for determining need for updates of clinical practice guidelines., Federal Register, 59, 19723
- Cook, 1997, The relation between systematic reviews and practice guidelines., Annals of Internal Medicine, 127, 210, 10.7326/0003-4819-127-3-199708010-00006 · DOI · Paper page
- CRASH trial collaborators, 2004, Effect of intravenous corticosteroids on death within 14 days in 10008 adults with clinically significant head injury (MRC CRASH trial): randomised placebo-controlled trial., The Lancet, 364, 1321, 10.1016/S0140-6736(04)17188-2 · DOI · Paper page
- Eccles, 2001, Using systematic reviews in clinical guideline development., 400, 10.1002/9780470693926.ch21 · DOI
- Fergusson, 2005, Randomized controlled trials of aprotinin in cardiac surgery: could clinical equipoise have stopped the bleeding?, Clinical Trials, 2, 218, 10.1191/1740774505cn085oa · DOI · Paper page
- French, 2005, Investing in updating: how do conclusions change when Cochrane systematic reviews are updated?, BMC Medical Research Methodology, 5, 33, 10.1186/1471-2288-5-33 · DOI
- Friedman, 1998, Fundamentals of Clinical Trials, 10.1007/978-1-4757-2915-3 · DOI · Paper page
- Gartlehner, 2004, Assessing the need to update prevention guidelines: a comparison of two methods., International Journal of Quality in Health Care, 16, 399, 10.1093/intqhc/mzh081 · DOI
- Higgins JPT Prevalence and problems of updated reviews: a survey and discussion. Beyond the Basics. Proceedings of the 2nd Symposium on Systematic Reviews: 1999 Oxford, England 1999
- Ioannidis, 1998, Clinical efficacy of high-dose acyclovir in patients with human immunodeficiency virus infection., Journal of infectious Diseases, 178, 349, 10.1086/515621 · DOI
- Johnston, 2003, Keeping cancer guidelines current: results of a comprehensive prospective literature monitoring strategy for twenty clinical practice guidelines., International Journal of Technology Assessment in Health Care, 19, 646, 10.1017/S0266462303000606 · DOI
- Koch G Are Cochrane Reviews updated regularly or not? Proceedings of the 10th Cochrane Colloquium: 31 July - 3 August. 2002
- Moher D Tetzlaff J Tricco A Sampson M Altman D Epidemiology and reporting characteristics of systematic reviews Proceedings of the 4th Canadian Cochrane Symposium 2005 · DOI · Paper page
- Moher, 2007, Epidemiology and reporting characteristics of systematic reviews., PLoS Medicine, 4, e78, 10.1371/journal.pmed.0040078 · DOI · Paper page
- Montori, 2005, Randomized trials stopped early for benefit: a systematic review., Journal of American Medical Association, 294, 2203, 10.1001/jama.294.17.2203 · DOI · Paper page
- Shekelle, 2001, When should clinical guidelines be updated?, British Medical Journal, 323, 155, 10.1136/bmj.323.7305.155 · DOI · Paper page
- Shekelle, 2001, Validity of the Agency for Healthcare Research and Quality clinical practice guidelines: how quickly do guidelines become outdated?, Journal of American Medical Association, 286, 1461, 10.1001/jama.286.12.1461 · DOI · Paper page
- Stead, 2001, Updating a systematic review - what difference did it make? Case study of nicotine replacement therapy., BMC Medical Research Methodology, 1, 10, 10.1186/1471-2288-1-10 · DOI
- Sutton, 1998, Systematic reviews of trials and other studies., Health Technology Assessment, 2, 1, 10.3310/hta2190 · DOI · Paper page
- Sutton, 2002, Methods for meta-analysis in medical research.
- Swets, 1988, Measuring the accuracy of diagnostic systems., Science, 240, 1285, 10.1126/science.3287615 · DOI · Paper page
- Merriam-Webster. Merriam-Webster's Collegiate Dictionary. 10th edition. Springfield, Massachusetts: Merriam-Webster 1996
- Wiebe, 2006, A systematic review identifies a lack of standardization in methods for handling missing variance data., Journal of Clinical Epidemiology, 59, 342, 10.1016/j.jclinepi.2005.08.017 · DOI · Paper page
- Zarnke, 2000, A novel process for updating recommendations for managing hypertension., Can J Cardiol, 16, 1094
- Moher, 2007, A Systematic Review Identified few Methods and Strategies describing when and how to update systematic reviews, Journal of Clinical Epidemiology, 60, 1095, 10.1016/j.jclinepi.2007.03.008 · DOI
Cited by
Systematic reviews: CRD's guidance for undertaking reviews in health care
Tacconelli, E. (2010) · The Lancet Infectious Diseases · doi:10.1016/s1473-3099(10)70065-7 · Cited by 3,245
Allotey, J., Fernandez, S., Bonet, M., Stallings, E., Yap, M., Kew, T., Zhou, D., Coomar, D., Sheikh, J., Lawson, H., et al. (2020) · BMJ · doi:10.1136/bmj.m3320 · Cited by 2,304
OARSI recommendations for the management of hip and knee osteoarthritis
Zhang, W., Nuki, G., Moskowitz, R. W., Abramson, S., Altman, R. D., Arden, N. K., Bierma-Zeinstra, S., Brandt, K. D., Croft, P., Doherty, M., et al. (2010) · Osteoarthritis and Cartilage · doi:10.1016/j.joca.2010.01.013 · Cited by 1,700
Living Systematic Reviews: An Emerging Opportunity to Narrow the Evidence-Practice Gap
Elliott, J. H., Turner, T., Clavisi, O., Thomas, J., Higgins, J. P. T., Mavergames, C., & Gruen, R. L. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001603 · Cited by 596
The Schizophrenia Patient Outcomes Research Team (PORT): Updated Treatment Recommendations 2009
Kreyenbuhl, J., Buchanan, R. W., Dickerson, F. B., & Dixon, L. B. (2010) · Schizophrenia Bulletin · doi:10.1093/schbul/sbp130 · Cited by 499
Metadata
- Authors
- Moher, D., Tsertsvadze, A., Tricco, A., Eccles, M., Grimshaw, J., Sampson, M., & Barrowman, N.
- Venue
- Cochrane Database of Systematic Reviews
- Publisher
- Wiley
- Published
- 2008-01-23
- Volume
- 2010
- Issue
- 1
- Pages
- MR000023-MR000023
- ISSN
- 1361-6137 (ISSN-L)
- 1465-1858
- OpenAlex
- W1717152970
- Cited by
- 146
- References
- 14
- FWCI
- 3.93
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.