Feasibility and acceptability of living systematic reviews: results from a mixed-methods evaluation
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
Background: Living systematic reviews (LSRs) offer an approach to keeping high-quality evidence synthesis continually up to date, so the most recent, relevant and reliable evidence can be used to inform policy and practice, resulting in improved quality of care and patient health outcomes. However, they require modifications to authoring and editorial processes and pose technical and publishing challenges. Several teams within Cochrane and the international Living Evidence Network have been piloting living systematic reviews. Methods: We conducted a mixed-methods evaluation with participants involved in six LSRs (three Cochrane and three non-Cochrane). Up to three semi-structured interviews were conducted with 27 participants involved with one or more of the pilot LSRs. Interviews explored participants’ experiences contributing to the LSR, barriers and facilitators to their conduct and opportunities for future development. Pilot team members also completed monthly surveys capturing time for key tasks and the number of citations screened for each review. Results: Across the pilot LSRs, search frequency was monthly to three-monthly, with some using tools such as machine learning and Cochrane Crowd to screen searches. Varied approaches were used to communicate updates to readers. The number of citations screened varied widely between the reviews, from three to 300 citations per month. The amount of time spent per month by the author team on each review also varied from 5 min to 32 h. Participants were enthusiastic to be involved in the LSR pilot. They highlighted the importance of a motivated and well-organised team; the value of technology enablers to improve workflow efficiencies; the need to establish reliable and efficient processes to sustain living reviews; and the potential for saving time and effort in the long run. Participants highlighted challenges with the current publication processes, managing ongoing workload and the lack of resources to support LSRs in the long term. Conclusions: Findings to date support feasibility and acceptability of LSR production. There are challenges that need to be addressed for living systematic reviews to be sustainable and have maximum value. The findings from this study will be used in discussions with the Cochrane community, key decision makers and people more broadly concerned with LSRs to identify and develop priorities for scale-up.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
How Quickly Do Systematic Reviews Go Out of Date? A Survival Analysis
Shojania, K. G., Sampson, M., Ansari, M. T., Ji, J., Doucette, S., & Moher, D. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-4-200708210-00179 · Cited by 838
Living Systematic Reviews: An Emerging Opportunity to Narrow the Evidence-Practice Gap
Elliott, J. H., Turner, T., Clavisi, O., Thomas, J., Higgins, J. P. T., Mavergames, C., & Gruen, R. L. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001603 · Cited by 596
Self-citationJadad, A. R., Cook, D. J., Jones, A., Klassen, T. P., Tugwell, P., Moher, M., & Moher, D. (1998) · JAMA · doi:10.1001/jama.280.3.278 · Cited by 534
Living systematic reviews: 2. Combining human and machine effort
Thomas, J., Noel-Storr, A., Marshall, I., Wallace, B., McDonald, S., Mavergames, C., Glasziou, P., Shemilt, I., Synnot, A., Turner, T., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.08.011 · Cited by 355
Self-citationThe Global Evidence Mapping Initiative: Scoping research in broad topic areas
Bragge, P., Clavisi, O., Turner, T., Tavender, E., Collie, A., & Gruen, R. L. (2011) · BMC Medical Research Methodology · doi:10.1186/1471-2288-11-92 · Cited by 310
Self-citationSystematic reviews can be produced and published faster
Sampson, M., Shojania, K. G., Garritty, C., Horsley, T., Ocampo, M., & Moher, D. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.02.004 · Cited by 103
Cited by
Kahale, L. A., Elkhoury, R., El Mikati, I., Pardo-Hernandez, H., Khamis, A. M., Schünemann, H. J., Haddaway, N. R., & Akl, E. A. (2022) · F1000Research · doi:10.12688/f1000research.51723.3 · Cited by 48
Perlman‐Arrow, S., Loo, N., Bobrovitz, N., Yan, T., & Arora, R. K. (2023) · Research Synthesis Methods · doi:10.1002/jrsm.1636 · Cited by 30
Iannizzi, C., Akl, E. A., Anslinger, E., Weibel, S., Kahale, L. A., Aminat, A. M., Piechotta, V., & Skoetz, N. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02396-x · Cited by 31
“Also, we note that some papers provide very broad guidance on several aspects of publishing a LSR [3, 14, 19, 29].”
BackgroundInfluentialExtension of the PRISMA 2020 statement for living systematic reviews (LSRs): protocol
Kahale, L. A., Piechotta, V., McKenzie, J. E., Dorando, E., Iannizzi, C., Barker, J. M., Page, M. J., Skoetz, N., & Akl, E. A. (2022) · F1000Research · doi:10.12688/f1000research.75449.1 · Cited by 24
Living systematic review: new inputs and challenges
Vergara-Merino, L., Verdejo, C., Carrasco, C., & Vargas-Peirano, M. (2020) · Medwave · doi:10.5867/medwave.2020.11.8092 · Cited by 18
Metadata
- Authors
- Millard, T., Synnot, A., Elliott, J., Green, S., McDonald, S., & Turner, T.
- Venue
- Systematic Reviews
- Published
- 2019-12
- Volume
- 8
- Issue
- 1
- Pages
- 325-325
- ISSN
- 2046-4053 (ISSN-L)
- OpenAlex
- W2995323474
- Cited by
- 77
- References
- 6
- FWCI
- 2.03
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Feasibility and acceptability of living systematic reviews: results from a mixed-methods evaluation. https://app.cochat.ai/papers/10.1186/s13643-019-1248-5
- The paper
- Millard, T., Synnot, A., Elliott, J., Green, S., McDonald, S., & Turner, T. (2019). Feasibility and acceptability of living systematic reviews: results from a mixed-methods evaluation. Systematic Reviews. https://doi.org/10.1186/s13643-019-1248-5
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.