Proposed triggers for retiring a living systematic review
Open accessThe DOI matches its Crossref record; 31 of the 32 references it cites were found in a registry or an index.
Abstract
Living systematic reviews (LSRs) are systematic reviews that are continually updated, incorporating relevant new evidence as it becomes available. LSRs are critical for decision-making in topics where the evidence continues to evolve. It is not feasible to continue to update LSRs indefinitely; however, guidance on when to retire LSRs from the living mode is not clear. We propose triggers for making such a decision. The first trigger is to retire LSRs when the evidence becomes conclusive for the outcomes that are required for decision-making. Conclusiveness of evidence is best determined based on the GRADE certainty of evidence construct, which is more comprehensive than solely relying on statistical considerations. The second trigger to retire LSRs is when the question becomes less pertinent for decision-making as determined by relevant stakeholders, including people affected by the problem, healthcare professionals, policymakers and researchers. LSRs can also be retired from a living mode when new studies are not anticipated to be published on the topic and when resources become unavailable to continue updating. We describe examples of retired LSRs and apply the proposed approach using one LSR about adjuvant tyrosine kinase inhibitors in high-risk renal cell carcinoma that we retired from a living mode and published its last update.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Murad and colleagues proposed a number of triggers that might lead to an LSR being retired. These include when the evidence becomes conclusive, which can be determined based on certainty of the evidence or statistical methods; when the topic becomes less relevant to stakeholders; when new studies are not expected to be published; when required resources become unavailable 33 ; however, other”
From Extension of the PRISMA 2020 statement for living systematic reviews (PRISMA-LSR): checklist and explanation (2024) · cites this paper as [33]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Grading quality of evidence and strength of recommendations
(2004) · BMJ · doi:10.1136/bmj.328.7454.1490 · Cited by 8,477 · cited 4×
“15 17 While guidance for reporting LSRs is still under development, 31 32 it is important for the LSR to report on triggers for retirement from the living mode.”
BackgroundInfluentialGRADE guidelines 6. Rating the quality of evidence—imprecision
Guyatt, G. H., Oxman, A. D., Kunz, R., Brozek, J., Alonso-Coello, P., Rind, D., Devereaux, P. J., Montori, V. M., Freyschuss, B., Vist, G., et al. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.01.012 · Cited by 2,693
Self-citationTrial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis
Wetterslev, J., Thorlund, K., Brok, J., & Gluud, C. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.03.013 · Cited by 1,881
The GRADE Working Group clarifies the construct of certainty of evidence
Hultcrantz, M., Rind, D., Akl, E. A., Treweek, S., Mustafa, R. A., Iorio, A., Alper, B. S., Meerpohl, J. J., Murad, M. H., Ansari, M. T., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.05.006 · Cited by 853 · cited 1×
“22 23 Thus, future novel statistical methods that incorporate specific thresholds are needed.”
BackgroundSelf-citationDrug treatments for covid-19: living systematic review and network meta-analysis
Siemieniuk, R. A., Bartoszko, J. J., Zeraatkar, D., Kum, E., Qasim, A., Díaz Martinez, J. P., Izcovich, A., Rochwerg, B., Lamontagne, F., Han, M. A., et al. (2020) · BMJ · doi:10.1136/bmj.m2980 · Cited by 846
How Quickly Do Systematic Reviews Go Out of Date? A Survival Analysis
Shojania, K. G., Sampson, M., Ansari, M. T., Ji, J., Doucette, S., & Moher, D. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-4-200708210-00179 · Cited by 838
Living systematic review: 1. Introduction-the why, what, when, and how
Elliott, J. H., Synnot, A., Turner, T., Simmonds, M., Akl, E. A., McDonald, S., Salanti, G., Meerpohl, J., MacLehose, H., Hilton, J., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.08.010 · Cited by 655
Self-citationWhen and how to update systematic reviews: consensus and checklist
Garner, P., Hopewell, S., Chandler, J., MacLehose, H., Schünemann, H. J., Akl, E. A., Beyene, J., Chang, S., Churchill, R., Dearness, K., et al. (2016) · BMJ · doi:10.1136/bmj.i3507 · Cited by 426 · cited 1×
“Finally, it is plausible that moderate certainty evidence may be sufficient in some cases to make the decision that the efforts of keeping a review in the living mode are not warranted, rather, such review can be updated when certain triggers or conditions are met, 14 but it would not be a true LSR.”
BackgroundSelf-citationPogue, J. M., & Yusuf, S. (1997) · Controlled Clinical Trials · doi:10.1016/s0197-2456(97)00051-2 · Cited by 415
Zeng, L., Brignardello-Petersen, R., Hultcrantz, M., Siemieniuk, R. A. C., Santesso, N., Traversy, G., Izcovich, A., Sadeghirad, B., Alexander, P. E., Devji, T., et al. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.03.026 · Cited by 294
Self-citation
Cited by
Systematic reviews of the literature: an introduction to current methods
Brignardello‐Petersen, R., Santesso, N. A., & Guyatt, G. H. (2024) · American Journal of Epidemiology · doi:10.1093/aje/kwae232 · Cited by 246
Akl, E. A., Khabsa, J., Iannizzi, C., Piechotta, V., Kahale, L. A., Barker, J. M., McKenzie, J. E., Page, M. J., & Skoetz, N. (2024) · BMJ · doi:10.1136/bmj-2024-079183 · Cited by 103
“…conclusive, which can be determined based on certainty of the evidence or statistical methods; when the topic becomes less relevant to stakeholders; when new studies are not expected to be published; when required resources become unavailable 33 ; however, other triggers might also be possible.”
Self-citationHasan, B., Saadi, S., Rajjoub, N. S., Hegazi, M., Al-Kordi, M., Fleti, F., Farah, M., Riaz, I. B., Banerjee, I., Wang, Z., et al. (2024) · BMJ Evidence-Based Medicine · doi:10.1136/bmjebm-2023-112597 · Cited by 67
Self-citationLuo, J., Chen, Z., Liu, D., Li, H., He, S., Zeng, L., Yang, M., Liu, Z., Xiao, X., & Zhang, L. (2023) · BMC Medical Research Methodology · doi:10.1186/s12874-023-01980-y · Cited by 27
Zeraatkar, D., Ling, M., Kirsh, S., Jassal, T., Pitre, T., Chakraborty, S. P., Turner, T., Turkstra, L. S., McIntyre, R. S., Izcovich, A., et al. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-086407 · Cited by 18
Metadata
- Authors
- Murad, M. H., Wang, Z., Chu, H., Lin, L., El Mikati, I. K., Khabsa, J., Akl, E. A., Nieuwlaat, R., Schuenemann, H. J., & Riaz, I. B.
- Affiliations
- Mayo Clinic (US)
- University of Minnesota (US)
- Banner - University Medical Center Tucson (US)
- American University of Beirut (LB)
- McMaster University (CA)
- Humanitas University (IT)
- Mayo Clinic Hospital (US)
- Corresponding author
- Murad, Mohammad Hassan
- ORCID
- Venue
- BMJ Evidence-Based Medicine
- Publisher
- BMJ
- Published
- 2023-03-08
- Volume
- 28
- Issue
- 5
- Pages
- 348-352
- ISSN
- 1356-5524 (ISSN-L)
- 2515-446X
- 2515-4478
- PMC
- PMC10579491
- OpenAlex
- W4323542543
- Cited by
- 17
- References
- 32
- Citations by year
- 2026: 4 · 2025: 5 · 2024: 6 · 2023: 2
- FWCI
- 1.21
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- BMJ Evidence-Based Medicine (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- HighWire Press Open Archive (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- living systematic review, GRADE certainty of evidence, stakeholder relevance, resource constraints, high-risk renal cell carcinoma
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Proposed triggers for retiring a living systematic review. https://app.cochat.ai/papers/10.1136/bmjebm-2022-112100
- The paper
- Murad, M. H., Wang, Z., Chu, H., Lin, L., El Mikati, I. K., Khabsa, J., Akl, E. A., Nieuwlaat, R., Schuenemann, H. J., & Riaz, I. B. (2023). Proposed triggers for retiring a living systematic review. BMJ Evidence-Based Medicine. https://doi.org/10.1136/bmjebm-2022-112100
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
