The DOI matches its Crossref record; 32 of the 32 references it cites were found in a registry or an index.
How Quickly Do Systematic Reviews Go Out of Date? A Survival Analysis
Abstract
BACKGROUND: Systematic reviews are often advocated as the best source of evidence to guide clinical decisions and health care policy, yet we know little about the extent to which they require updating. OBJECTIVE: To estimate the average time to changes in evidence that are sufficiently important to warrant updating systematic reviews. DESIGN: Survival analysis of 100 quantitative systematic reviews. SAMPLE: Systematic reviews published from 1995 to 2005 and indexed in ACP Journal Club. Eligible reviews evaluated a specific drug or class of drug, device, or procedure and included only randomized or quasi-randomized, controlled trials. MEASUREMENTS: Quantitative signals for updating were changes in statistical significance or relative changes in effect magnitude of at least 50% involving 1 of the primary outcomes of the original systematic review or any mortality outcome. Qualitative signals included substantial differences in characterizations of effectiveness, new information about harm, and caveats about the previously reported findings that would affect clinical decision making. RESULTS: The cohort of 100 systematic reviews included a median of 13 studies and 2663 participants per review. A qualitative or quantitative signal for updating occurred for 57% of reviews (95% CI, 47% to 67%). Median duration of survival free of a signal for updating was 5.5 years (CI, 4.6 to 7.6 years). However, a signal occurred within 2 years for 23% of reviews and within 1 year for 15%. In 7%, a signal had already occurred at the time of publication. Only 4% of reviews had a signal within 1 year of the end of the reported search period; 11% had a signal within 2 years of the search. Shorter survival was associated with cardiovascular topics (hazard ratio, 2.70 [CI, 1.36 to 5.34]) and heterogeneity in the original review (hazard ratio, 2.15 [CI, 1.12 to 4.11]). LIMITATION: Judgments of the need for updating were made without involving content experts. CONCLUSION: In a cohort of high-quality systematic reviews directly relevant to clinical practice, signals for updating occurred frequently and within a relatively short time.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“SLRs are recognised for being time-consuming and resource-intensive.”
From Artificial intelligence for literature reviews: opportunities and challenges (2024) · cites this paper as Shojania et al. 2007
In the citing paper
“Given that the meta-analysis should be updated within 2 years ( Cumpston and Chandler, 2019 ) and the median time for the update in Cochrane Collaboration is about 5.5 ( Shojania et al., 2007 ), the meta-analysis on the CI effect should have been updated much earlier.”
From The effect of contextual interference on transfer in motor learning - a systematic review and meta-analysis (2024) · cites this paper as Shojania et al., 2007
In the citing paper
“As a consequence, the results of a search may be outdated before the review is published [124 -126].”
From PRISMA-S: an extension to the PRISMA Statement for Reporting Literature Searches in Systematic Reviews (2021) · cites this paper as [125]
In the citing paper
“Long production times can in turn affect the longevity of the findings: an estimated 7% of SLRs are already out of date (defined as availability of new findings that would affect the conclusions of the synthesis) at the time of publication (Shojania et al., 2007).”
From Analysis of article screening and data extraction performance by an AI systematic literature review platform (2025) · cites this paper as Shojania et al., 2007
In the citing paper
“Additionally, 23% of all SLR studies are outdated within 2 years of publication, as reviewers failed to incorporate novel evidence on their subject of interest [8,9].”
From Automation of systematic literature reviews: A systematic literature review (2021) · cites this paper as [9]
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Collaboration, A. T. (2002) · BMJ · doi:10.1136/bmj.324.7329.71 · Cited by 6,943
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563
High-Dose Atorvastatin after Stroke or Transient Ischemic Attack
The Stroke Prevention by Aggressive Reduction in Cholesterol Levels (SPARCL) Investigators (2006) · New England Journal of Medicine · doi:10.1056/nejmoa061894 · Cited by 2,887
A Comparison of Albumin and Saline for Fluid Resuscitation in the Intensive Care Unit
(2004) · New England Journal of Medicine · doi:10.1056/nejmoa040232 · Cited by 2,872
A comparison of observational studies and randomized, controlled trials.
Benson, K., & Hartz, A. J. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422506 · Cited by 2,347
Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
Ioannidis, J. P. A. (2005) · JAMA · doi:10.1001/jama.294.2.218 · Cited by 1,465
Discrepancies between Meta-Analyses and Subsequent Large Randomized, Controlled Trials
LeLorier, J., Grégoire, G., Benhaddad, A., Lapierre, J., & Derderian, F. (1997) · New England Journal of Medicine · doi:10.1056/nejm199708213370806 · Cited by 1,171
Comparison of Evidence of Treatment Effects in Randomized and Nonrandomized Studies
Ioannidis, J. P. A. (2001) · JAMA · doi:10.1001/jama.286.7.821 · Cited by 919
Epidemiology and reporting characteristics of systematic reviews.
Moher, D., Tetzlaff, J., Tricco, A. C., Sampson, M., & Altman, D. G. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040078 · Cited by 891 · cited 1×
“Finally, Cochrane reviews differ in important respects, such as clinical topic coverage, from other peerreviewed systematic reviews (25).”
BackgroundSelf-citation
Research works printed but not linked: 3
- Shojania KG, 2001, Eff Clin Pract, 4, 157
- Alderson P, 2002, Cochrane Database Syst Rev, CD001208
- Abramson MJ, 1995, Am J Respir Crit Care Med, 151, 969
Cited by
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · BMJ · doi:10.1136/bmj.b2700 · Cited by 17,648
Self-citationShamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & the PRISMA-P Group (2015) · BMJ · doi:10.1136/bmj.g7647 · Cited by 13,120
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.06.006 · Cited by 10,578
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Self-citation
Metadata
- Authors
- Shojania, K. G., Sampson, M., Ansari, M. T., Ji, J., Doucette, S., & Moher, D.
- Affiliations
- University of Ottawa (CA)
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2007-08-21
- Volume
- 147
- Issue
- 4
- Pages
- 224-233
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2095623039
- Cited by
- 838
- References
- 32
- Citations by year
- 2026: 45 · 2025: 51 · 2024: 49 · 2023: 59 · 2022: 66 · 2021: 55 · 2020: 73 · 2019: 48 · 2018: 47 · 2017: 42 · 2016: 37 · 2015: 34 · 2014: 38 · 2013: 44 · 2012: 35
- FWCI
- 6.55
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- UCLA - Biblioteca de Ciencias de la Salud (submittedVersion)
- Funders
- U.S. Public Health Service (290-02-0021)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Keywords
- systematic reviews, survival analysis, randomized controlled trials, heterogeneity, cardiovascular interventions, mortality, clinical decision-making
- MeSH terms
- Review Literature as Topic, Time Factors, Meta-Analysis as Topic, Survival Analysis, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How Quickly Do Systematic Reviews Go Out of Date? A Survival Analysis. https://app.cochat.ai/papers/10.7326/0003-4819-147-4-200708210-00179
- The paper
- Shojania, K. G., Sampson, M., Ansari, M. T., Ji, J., Doucette, S., & Moher, D. (2007). How Quickly Do Systematic Reviews Go Out of Date? A Survival Analysis. Annals of Internal Medicine. https://doi.org/10.7326/0003-4819-147-4-200708210-00179
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.