Two methods provide similar signals for the need to update systematic reviews
Open accessThe DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
Objective Apply and compare two methods that identify signals for the need to update systematic reviews, using three Evidence-based Practice Center reports on omega-3 fatty acids as test cases. Study Design and Setting We applied the RAND method, which uses domain (subject matter) expert guidance, and a modified Ottawa method, which uses quantitative and qualitative signals. For both methods, we conducted focused electronic literature searches of recent studies using the key terms from the original reports. We assessed the agreement between the methods and qualitatively assessed the merits of each system. Results Agreement between the two methods was “substantial” or better (kappa > 0.62) in three of the four systematic reviews. Overall agreement between the methods was “substantial” (kappa = 0.64, 95% confidence interval [CI] 0.45–0.83). Conclusion The RAND and modified Ottawa methods appear to provide similar signals for the possible need to update systematic reviews in this pilot study. Future evaluation with a broader range of clinical topics and eventual comparisons between signals to update reports and the results of full evidence review updates will be needed. We propose a hybrid approach combining the best features of both methods, which should allow efficient review and assessment of the need to update.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
How Quickly Do Systematic Reviews Go Out of Date? A Survival Analysis
Shojania, K. G., Sampson, M., Ansari, M. T., Ji, J., Doucette, S., & Moher, D. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-4-200708210-00179 · Cited by 838 · cited 1×
“In parallel, a secondmethodwas devised at theUniversity ofOttawa EPC (UOEPC) through assessment of the predictors of the need to update systematic reviews with meta-analyses [8], and the method was then tested using 100 metaanalyses published from 1995 to 2005 [9].”
MethodSelf-citationEffects of omega-3 fatty acids on serum markers of cardiovascular disease risk: a systematic review
Balk, E. M., Lichtenstein, A. H., Chung, M., Kupelnick, B., Chew, P., & Lau, J. (2006) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2006.02.012 · Cited by 732
Self-citationOikawa, S., Yokoyama, M., Origasa, H., Matsuzaki, M., Matsuzawa, Y., Saito, Y., Ishikawa, Y., Sasaki, J., Hishida, H., Itakura, H., et al. (2009) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2009.03.029 · Cited by 125
Balk, E. M., Lichtenstein, A. H., Chung, M., Kupelnick, B., Chew, P., & Lau, J. (2006) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2005.06.042 · Cited by 63
Self-citation
Research works printed but not linked: 9
- Whitlock, 2010, AHRQ series paper 3: identifying, selecting, and refining topics for comparative effectiveness systematic reviews: AHRQ and the effective health-care program, J Clin Epidemiol, 63, 491, 10.1016/j.jclinepi.2009.03.008 · DOI
- Garritty, 2010, Updating systematic reviews: an international survey, PLoS ONE, 5, e9914, 10.1371/journal.pone.0009914 · DOI · Paper page
- The Cochrane Collaboration. Maintaining your review.Accessed Jan 10, 2011. Available at http://www.cochrane-net.org/openlearning/html/mod19-2.htm. 2002.
- French, 2005, Investing in updating: how do conclusions change when Cochrane systematic reviews are updated?, BMC Med Res Methodol, 5, 33, 10.1186/1471-2288-5-33 · DOI · Paper page
- Shekelle, 2001, When should clinical guidelines be updated?, BMJ, 323, 155, 10.1136/bmj.323.7305.155 · DOI · Paper page
- Shekelle, 2001, Validity of the Agency for Healthcare Research and Quality Clinical Practice Guidelines: how quickly do guidelines become outdated?, JAMA, 286, 1461, 10.1001/jama.286.12.1461 · DOI · Paper page
- Moher D, Tsertsvadze A, Tricco AC, Eccles M, Grimshaw J, Sampson M, et al. When and how to update systematic reviews. Cochrane Database Syst Rev 2008;MR000023. · DOI · Paper page
- Balk, 2007, A collaborative effort to apply the evidence-based review process to the field of nutrition: challenges, benefits, and lessons learned, Am J Clin Nutr, 85, 1448, 10.1093/ajcn/85.6.1448 · DOI
- Landis, 1977, The measurement of observer agreement for categorical data, Biometrics, 33, 159, 10.2307/2529310 · DOI · Paper page
Cited by
Shamseer, L., Moher, D., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., Stewart, L. A., & the PRISMA-P Group (2015) · BMJ · doi:10.1136/bmj.g7647 · Cited by 13,120
Self-citationWhen and how to update systematic reviews: consensus and checklist
Garner, P., Hopewell, S., Chandler, J., MacLehose, H., Schünemann, H. J., Akl, E. A., Beyene, J., Chang, S., Churchill, R., Dearness, K., et al. (2016) · BMJ · doi:10.1136/bmj.i3507 · Cited by 426
Okoth, K., Chandan, J. S., Marshall, T., Thangaratinam, S., Thomas, G. N., Nirantharakumar, K., & Adderley, N. J. (2020) · BMJ · doi:10.1136/bmj.m3502 · Cited by 416
Living systematic reviews: 4. Living guideline recommendations.
Akl, E. A., Meerpohl, J. J., Elliott, J., Kahale, L. A., Schünemann, H. J., Agoritsas, T., Hilton, J., Perron, C., Akl, E., Hodder, R., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.08.009 · Cited by 328
Are systematic reviews up-to-date at the time of publication?
Beller, E. M., Chen, J. K.-H., Wang, U. L.-H., & Glasziou, P. P. (2013) · Systematic Reviews · doi:10.1186/2046-4053-2-36 · Cited by 196
Metadata
- Authors
- Chung, M., Newberry, S. J., Ansari, M. T., Yu, W. W., Wu, H., Lee, J., Suttorp, M., Gaylor, J. M., Motala, A., Moher, D., Balk, E. M., & Shekelle, P. G.
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2012-06
- Volume
- 65
- Issue
- 6
- Pages
- 660-668
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W1987451225
- Cited by
- 42
- References
- 4
- FWCI
- 2.12
- Type
- journal article
- Language
- en
- Open access
- green
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Two methods provide similar signals for the need to update systematic reviews. https://app.cochat.ai/papers/10.1016/j.jclinepi.2011.12.004
- The paper
- Chung, M., Newberry, S. J., Ansari, M. T., Yu, W. W., Wu, H., Lee, J., Suttorp, M., Gaylor, J. M., Motala, A., Moher, D., Balk, E. M., & Shekelle, P. G. (2012). Two methods provide similar signals for the need to update systematic reviews. Journal of Clinical Epidemiology. https://doi.org/10.1016/j.jclinepi.2011.12.004
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.