Living Systematic Reviews: An Emerging Opportunity to Narrow the Evidence-Practice Gap
Open accessThe DOI matches its Crossref record; 34 of the 35 references it cites were found in a registry or an index.
Abstract
The current difficulties in keeping systematic reviews up to date leads to considerable inaccuracy, hampering the translation of knowledge into action. Incremental advances in conventional review updating are unlikely to lead to substantial improvements in review currency. A new approach is needed. We propose living systematic review as a contribution to evidence synthesis that combines currency with rigour to enhance the accuracy and utility of health evidence. Living systematic reviews are high quality, up-to-date online summaries of health research, updated as new research becomes available, and enabled by improved production efficiency and adherence to the norms of scholarly communication. Together with innovations in primary research reporting and the creation and use of evidence in health systems, living systematic review contributes to an emerging evidence ecosystem.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“for a primary study, it takes 2.5 - 6.5 years before it is incorporated in an SLR study [7,8].”
From Automation of systematic literature reviews: A systematic literature review (2021) · cites this paper as [7]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Between 2014 (when the LSR approach first emerged 3 ) and 2019, the rate of publication of these systematic reviews was low; however, since 2019, there has been a rapid increase.”
From Extension of the PRISMA 2020 statement for living systematic reviews (PRISMA-LSR): checklist and explanation (2024) · cites this paper as [3]
The cited source says what the claim attributes to it.
In the citing paper
“The most efficient use and application of the machine-learning technologies would be in the areas allowing automation of specific SR processes, in particular those involving time-consuming and resource-intensive tasks such as language translation [15], study selection [11, 16–18], data extraction [19] and risk of bias assessment [20].”
From How to conduct systematic reviews more expeditiously? (2015) · cites this paper as [18]
The cited source says what the claim attributes to it.
In the citing paper
“Living systematic reviews are dynamic and constantly changing online-only evidence summaries that demand less intensive work over time compared to static and sporadically more resource-intensive conventional SRs.”
From How to conduct systematic reviews more expeditiously? (2015) · cites this paper as [18]
The cited source says what the claim attributes to it.
In the citing paper
“The authors proposed to initiate living systematic reviews, which represent high-quality online evidence summaries, continuously updated as any new relevant evidence becomes available.”
From How to conduct systematic reviews more expeditiously? (2015) · cites this paper as [18]
6 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Wang, G., Ma, R., Qiao, G., Wada, K., Aizawa, Y., & Satoh, T. (2015) · Integrative Medicine International · doi:10.1159/000368408 · Cited by 18,657
Knowledge translation of research findings
Grimshaw, J. M., Eccles, M. P., Lavis, J. N., Hill, S. J., & Squires, J. E. (2012) · Implementation Science · doi:10.1186/1748-5908-7-50 · Cited by 2,407 · cited 1×
“The reasons for this gap are complex [2], but it is clear that synthesising the complex, incomplete, and at times conflicting findings of biomedical research into forms that can readily inform health decision making is an essential component of the bridge from ‘‘knowing’’ to ‘‘doing.”
BackgroundTrial sequential analysis may establish when firm evidence is reached in cumulative meta-analysis
Wetterslev, J., Thorlund, K., Brok, J., & Gluud, C. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.03.013 · Cited by 1,881 · cited 1×
“Statistical monitoring of metaanalyses using formal sequential methods can control the risk of spurious findings, while achieving pre-specified power to identify a pre-specified clinically relevant magnitude of effect [16,17].”
MethodCentral challenges facing the national clinical research enterprise
Sung, N. S., Crowley, W. F., Genel, M., Salber, P., Sandy, L., Sherwood, L. M., Johnson, S. B., Catanese, V., Tilson, H., Getz, K., et al. (2003) · JAMA · doi:10.1001/jama.289.10.1278 · Cited by 1,235
Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?
Bastian, H., Glasziou, P., & Chalmers, I. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000326 · Cited by 1,188
Finding What Works in Health Care: Standards for Systematic Reviews
Eden, J., Levit, L. A., Berg, A. O., & Morton, S. C. (2013) · Cited by 943
How Quickly Do Systematic Reviews Go Out of Date? A Survival Analysis
Shojania, K. G., Sampson, M., Ansari, M. T., Ji, J., Doucette, S., & Moher, D. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-147-4-200708210-00179 · Cited by 838 · cited 1×
“By 2 years post-publication 23% of SRs that have not been updated will have failed to incorporate new evidence that would substantively change conclusions about the effectiveness or harms of therapies [9].”
Background
Research works printed but not linked: 4
- 22. Elliott J (2012) Exploiting innovations in technology to improve the efficiency of review production. 20th Cochrane Colloquium. Auckland.
- 23. Ciapponi A, Glujovsky D (2012) Survey among Cochrane authors about early stages of systematic reviews. In: Proceedings of the 20th Cochrane Colloquium; 30 September–3 October 2012 Auckland, New Zealand
- 34. McGee R, Willis N, Jones A, Higgins G, Craig J, et al.. (2012) Factors associated with the timely completion of a Cochrane review. In: Proceedings of the 20th Cochrane Colloquium; 30 September–3 October 2012Auckland, New Zealand
- 38. Noel-Storr A, Struthers C, Huckvale C, Dooley G, McShane R (2012) ALOIS, ScreenToGo and the Cochrane Register of Studies: using mobile technology to engage and involve consumers in the work of the Cochrane Collaboration. In: Proceedings of the 20th Cochrane Colloquium; 30 September–3 October 2012Auckland, New Zealand.
Cited by
Traumatic brain injury: integrated approaches to improve prevention, clinical care, and research
Maas, A. I. R., Menon, D. K., Adelson, P. D., Andelic, N., Bell, M. J., Belli, A., Bragge, P., Brazinova, A., Büki, A., Chesnut, R. M., et al. (2017) · The Lancet Neurology · doi:10.1016/s1474-4422(17)30371-x · Cited by 2,609
Self-citationMeta-analysis and the science of research synthesis
Gurevitch, J., Koricheva, J., Nakagawa, S., & Stewart, G. (2018) · Nature · doi:10.1038/nature25753 · Cited by 1,709
Langford, B. J., So, M., Raybardhan, S., Leung, V., Westwood, D., MacFadden, D. R., Soucy, J.-P. R., & Daneman, N. (2020) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2020.07.016 · Cited by 1,569
Buitrago-Garcia, D., Egli-Gany, D., Counotte, M. J., Hossmann, S., Imeri, H., Ipekci, A. M., Salanti, G., & Low, N. (2020) · PLOS Medicine · doi:10.1371/journal.pmed.1003346 · Cited by 1,000
“We conducted a living systematic review, a systematic review that provides an online summary of findings and is updated when relevant new evidence becomes available [4].”
Page, M. J., & Moher, D. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0663-8 · Cited by 769
“enhance reproducibility of meta-analytic results and share data collected [151, 152] and report the methods and results of updated SRs [153] and living SRs [154].”
MethodLiving systematic review: 1. Introduction-the why, what, when, and how
Elliott, J. H., Synnot, A., Turner, T., Simmonds, M., Akl, E. A., McDonald, S., Salanti, G., Meerpohl, J., MacLehose, H., Hilton, J., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.08.010 · Cited by 655
“We have proposed a continual approach to review updating termed ‘‘living systematic review’’ (see Box 1) [10].”
MethodSelf-citationSystematic Reviews in Educational Research: Methodology, Perspectives and Application
Newman, M., & Gough, D. (2020) · Systematic Reviews in Educational Research · doi:10.1007/978-3-658-27602-7_1 · Cited by 591
Pijls, B. G., Jolani, S., Atherley, A., Derckx, R. T., Dijkstra, J. I. R., Franssen, G. H. L., Hendriks, S., Richters, A., Venemans-Jellema, A., Zalpuri, S., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2020-044640 · Cited by 586
Meeting the review family: exploring review types and associated information retrieval requirements
Sutton, A., Clowes, M., Preston, L., & Booth, A. (2019) · Health Information & Libraries Journal · doi:10.1111/hir.12276 · Cited by 570
Epidemiology of Traumatic Brain Injury in Europe: A Living Systematic Review
Brazinova, A., Rehorcikova, V., Taylor, M. S., Buckova, V., Majdan, M., Psota, M., Peeters, W., Feigin, V., Theadom, A., Holkovic, L., et al. (2021) · Journal of Neurotrauma · doi:10.1089/neu.2015.4126 · Cited by 559
“A living systematic review is a high quality, up-to-date online summary of health research that is updated as data from new relevant research that meets study inclusion criteria becomes available.(16) In practice, this means that the searches will be re-run frequently, any new studies incorporated into the review, and…”
BackgroundEmergency remote teaching in higher education: mapping the first global online semester
Bond, M., Bedenlier, S., Marín, V. I., & Händel, M. (2021) · International Journal of Educational Technology in Higher Education · doi:10.1186/s41239-021-00282-x · Cited by 527
Krauer, F., Riesen, M., Reveiz, L., Oladapo, O. T., Martínez-Vega, R., Porgo, T. V., Haefliger, A., Broutet, N. J., Low, N., & WHO Zika Causality Working Group (2017) · PLOS Medicine · doi:10.1371/journal.pmed.1002203 · Cited by 492
“The concept of a “living systematic review” has been proposed as a way to combine rigour with timeliness for intervention research [160].”
BackgroundSystematic review automation technologies
Tsafnat, G., Glasziou, P., Choong, M. K., Dunn, A., Galgani, F., & Coiera, E. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-74 · Cited by 472
“A ‘living’ systematic review is updated at the time it is required to be used in practice [139].”
BackgroundAutomation of systematic literature reviews: A systematic literature review
van Dinter, R., Tekinerdogan, B., & Catal, C. (2021) · Information and Software Technology · doi:10.1016/j.infsof.2021.106589 · Cited by 432
“The time from the last search to publication takes commonly over 1 year for an SLR study, and for a primary study, it takes 2.5 - 6.5 years before it is incorporated in an SLR study [7,8].”
BackgroundInfluentialWhen and how to update systematic reviews: consensus and checklist
Garner, P., Hopewell, S., Chandler, J., MacLehose, H., Schünemann, H. J., Akl, E. A., Beyene, J., Chang, S., Churchill, R., Dearness, K., et al. (2016) · BMJ · doi:10.1136/bmj.i3507 · Cited by 426
Self-citationSimons, D., Shahab, L., Brown, J., & Perski, O. (2021) · Addiction · doi:10.1111/add.15276 · Cited by 371
“This is a living evidence review, which is updated as new evidence becomes available [19].”
BackgroundLiving systematic reviews: 2. Combining human and machine effort
Thomas, J., Noel-Storr, A., Marshall, I., Wallace, B., McDonald, S., Mavergames, C., Glasziou, P., Shemilt, I., Synnot, A., Turner, T., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.08.011 · Cited by 355
“Living systematic reviews represent an opportunity to rethink conventional review processes and take advantage of emerging approaches to reviewing which promise to increase efficiency [3].”
BackgroundSelf-citationMichie, S., Thomas, J., Johnston, M., Aonghusa, P. M., Shawe-Taylor, J., Kelly, M. P., Deleris, L. A., Finnerty, A. N., Marques, M. M., Norris, E., et al. (2017) · Implementation Science · doi:10.1186/s13012-017-0641-5 · Cited by 338
“Insufficient human resources to undertake evidence reviews and syntheses also means that these are often out of date by the time of completion [3].”
BackgroundSelf-citationPrescribable mHealth apps identified from an overview of systematic reviews
Byambasuren, O., Sanders, S., Beller, E., & Glasziou, P. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0021-9 · Cited by 291
“64,65 This will be invaluable in digital health research and evidence base building.”
BackgroundHayden, J. A., Ellis, J., Ogilvie, R., Stewart, S. A., Bagg, M. K., Stanojevic, S., Yamato, T. P., & Saragiotto, B. T. (2021) · Journal of Physiotherapy · doi:10.1016/j.jphys.2021.09.004 · Cited by 321
Haimovich, A. D., Ravindra, N. G., Stoytchev, S., Young, H. P., Wilson, F. P., van Dijk, D., Schulz, W. L., & Taylor, R. A. (2020) · Annals of Emergency Medicine · doi:10.1016/j.annemergmed.2020.07.022 · Cited by 294
Giustini, D. M., Ali, S. M., Fraser, M., & Boulos, M. N. K. (2018) · Online Journal of Public Health Informatics · doi:10.5210/ojphi.v10i2.8270 · Cited by 271
“However, these papers will now be part of a living systematic review [62].”
BackgroundAutomating data extraction in systematic reviews: a systematic review
Jonnalagadda, S. R., Goyal, P., & Huffman, M. D. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0066-7 · Cited by 287
“5 years for a primary study publication to be included and published in a new systematic review [5].”
BackgroundDelayed antibiotic prescriptions for respiratory infections
Spurling, G. K., Del Mar, C. B., Dooley, L., Clark, J., & Askew, D. A. (2017) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004417.pub5 · Cited by 276
Pelch, K., Wignall, J. A., Goldstone, A. E., Ross, P. K., Blain, R. B., Shapiro, A. J., Holmgren, S. D., Hsieh, J.-H., Svoboda, D., Auerbach, S. S., et al. (2019) · Toxicology · doi:10.1016/j.tox.2019.06.006 · Cited by 266
Research Weaving: Visualizing the Future of Research Synthesis
Nakagawa, S., Samarasinghe, G., Haddaway, N. R., Westgate, M. J., O’Dea, R. E., Noble, D. W. A., & Lagisz, M. (2019) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2018.11.007 · Cited by 261
Tleyjeh, I. M., Kashour, Z., Damlaj, M., Riaz, M., Tlayjeh, H., Altannir, M., Altannir, Y., Al-Tannir, M., Tleyjeh, R., Hassett, L., et al. (2021) · Clinical Microbiology and Infection · doi:10.1016/j.cmi.2020.10.036 · Cited by 205
“A living systematic review is a cumulative synthesis that is updated regularly as new evidence becomes available [23].”
van der Vlist, A. C., Winters, M., Weir, A., Ardern, C. L., Welton, N. J., Caldwell, D. M., Verhaar, J. A. N., & de Vos, R.-J. (2021) · British Journal of Sports Medicine · doi:10.1136/bjsports-2019-101872 · Cited by 198
“66 67 The protocol was registered prospectively and patients were involved in selecting outcomes most relevant to them in the design phase.”
Sirois, F. M., & Owens, J. (2021) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2020.589545 · Cited by 191
John, A., Eyles, E., Webb, R. T., Okolie, C., Schmidt, L., Arensman, E., Hawton, K., O'Connor, R. C., Kapur, N., Moran, P., et al. (2021) · F1000Research · doi:10.12688/f1000research.25522.2 · Cited by 189
“Living systematic reviews are high-quality, up-to-date online summaries of research that are regularly updated, using efficient, often semi-automated, systems of production (Elliott et al., 2014).”
Self-citationThe impact of the COVID-19 pandemic on self-harm and suicidal behaviour: a living systematic review
John, A., Okolie, C., Eyles, E., Webb, R. T., Schmidt, L., McGuinness, L. A., Olorisade, B. K., Arensman, E., Hawton, K., Kapur, N., et al. (2020) · F1000Research · doi:10.12688/f1000research.25522.1 · Cited by 178
Self-citationSystematic rapid "living" review on rehabilitation needs due to COVID-19: update to March 31st, 2020
Ceravolo, M. G., de Sire, A., Andrenelli, E., Negrini, F., & Negrini, S. (2020) · European Journal of Physical and Rehabilitation Medicine · doi:10.23736/s1973-9087.20.06329-7 · Cited by 177
O’Connor, A. M., Tsafnat, G., Thomas, J., Glasziou, P., Gilbert, S. B., & Hutton, B. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1062-0 · Cited by 167
Self-citationEgan, K. J., Pinto-Bruno, Á. C., Bighelli, I., Berg-Weger, M., van Straten, A., Albanese, E., & Pot, A.-M. (2018) · Journal of the American Medical Directors Association · doi:10.1016/j.jamda.2017.10.023 · Cited by 166
“37 This approach, if combined with greater reporting transparency 38 could help provide a solid basis for a step change in collaborative research gains in the years ahead.”
BackgroundSexual transmission of Zika virus and other flaviviruses: A living systematic review
Counotte, M. J., Kim, C. R., Wang, J., Bernstein, K., Deal, C. D., Broutet, N. J. N., & Low, N. (2018) · PLOS Medicine · doi:10.1371/journal.pmed.1002611 · Cited by 166
“We performed this review as a living systematic review [15] because research into many aspects of ZIKV is a new and fast-moving field.”
BackgroundMethodUsing machine learning to advance synthesis and use of conservation and environmental evidence
Cheng, S. H., Augustin, C., Bethel, A., Gill, D., Anzaroot, S., Brun, J., DeWilde, B., Minnich, R. C., Garside, R., Masuda, Y. J., et al. (2018) · Conservation Biology · doi:10.1111/cobi.13117 · Cited by 155
Living systematic reviews: 3. Statistical methods for updating meta-analyses
Simmonds, M. C., Salanti, G., McKenzie, J. E., Elliott, J., Agoritsas, T., Hilton, J., Perron, C., Akl, E. A., Hodder, R. K., Pestridge, C., et al. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.08.008 · Cited by 163
Self-citationGiboin, L., & Wolff, W. (2019) · Performance Enhancement & Health · doi:10.1016/j.peh.2019.100150 · Cited by 196
Prioritising references for systematic reviews with RobotAnalyst: A user study
Przybyła, P., Brockmeier, A. J., Kontonatsios, G., Le Pogam, M., McNaught, J., von Elm, E., Nolan, K., & Ananiadou, S. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1311 · Cited by 148
“9 Prioritisation may also enable different screening or review paradigms, such as living systematic reviews 10 , 11 or updates, 12 , 13 since more relevant references are found earlier than with manual screening.”
BackgroundEvidence-Based Research Series-Paper 1: What Evidence-Based Research is and why is it important?
Robinson, K. A., Brunnhuber, K., Ciliska, D., Juhl, C. B., Christensen, R., & Lund, H. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.07.020 · Cited by 146
Decision makers need constantly updated evidence synthesis
Elliott, J., Lawrence, R. N., Minx, J. C., Oladapo, O. T., Ravaud, P., Jeppesen, B. T., Thomas, J., Turner, T., Vandvik, P. O., & Grimshaw, J. (2021) · Nature · doi:10.1038/d41586-021-03690-1 · Cited by 147
Self-citationHealthcareLCA: an open-access living database of health-care environmental impact assessments
Drew, J., Christie, S., Rainham, D. G. C., & Rizan, C. (2022) · The Lancet Planetary Health · doi:10.1016/s2542-5196(22)00257-1 · Cited by 160
The Teaching and Learning Toolkit
Higgins, S. (2018) · Cambridge University Press eBooks · doi:10.1017/9781139519618.004 · Cited by 136
Towards evidence based research
Lund, H., Brunnhuber, K., Juhl, C., Robinson, K., Leenaars, M., Dorch, B. F., Jamtvedt, G., Nortvedt, M. W., Christensen, R., & Chalmers, I. (2016) · BMJ · doi:10.1136/bmj.i5440 · Cited by 131
Thielen, F., Van Mastrigt, G., Burgers, L., Bramer, W., Majoie, H., Evers, S., & Kleijnen, J. (2016) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1080/14737167.2016.1246962 · Cited by 132
Noel-Storr, A., Dooley, G., Elliott, J., Steele, E., Shemilt, I., Mavergames, C., Wisniewski, S., McDonald, S., Murano, M., Glanville, J., et al. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.01.006 · Cited by 132
“This bottleneck in the evidence production process can cause delay and contributes to often lengthy production times for systematic reviews and other evidence syntheses, such as guidelines and technology assessments, leaving important clinical questions unanswered, and possibly resulting in reliance on out-of-date, an…”
BackgroundSelf-citationMichie, S., West, R., Finnerty, A. N., Norris, E., Wright, A. J., Marques, M. M., Johnston, M., Kelly, M. P., Thomas, J., & Hastings, J. (2021) · Wellcome Open Research · doi:10.12688/wellcomeopenres.15902.2 · Cited by 129
Self-citationPatient safety in inpatient mental health settings: a systematic review
Thibaut, B., Dewa, L. H., Ramtale, S. C., D’Lima, D., Adam, S. K., Ashrafian, H., Darzi, A. W., & Archer, S. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-030230 · Cited by 166
Créquit, P., Trinquart, L., Yavchitz, A., & Ravaud, P. (2016) · BMC Medicine · doi:10.1186/s12916-016-0555-0 · Cited by 125
“As an alternative to classical systematic reviews performed at one point in time, a new theoretical framework is “living systematic reviews”, defined as high-quality online summaries of health research updated as new research becomes available [61].”
BackgroundHow to conduct systematic reviews more expeditiously?
Tsertsvadze, A., Chen, Y.-F., Moher, D., Sutcliffe, P., & McCarthy, N. (2015) · Systematic Reviews · doi:10.1186/s13643-015-0147-7 · Cited by 104
“For example, Elliot and colleagues [18], proposed an alternative solution to the problem of keeping SRs and their conclusions up-todate and accurate.”
BackgroundAref-Adib, G., McCloud, T., Ross, J., O'Hanlon, P., Appleton, V., Rowe, S., Murray, E., Johnson, S., & Lobban, F. (2019) · The Lancet Psychiatry · doi:10.1016/s2215-0366(18)30302-x · Cited by 118
Reproducible research practices are underused in systematic reviews of biomedical interventions
Page, M. J., Altman, D. G., Shamseer, L., McKenzie, J. E., Ahmadzai, N., Wolfe, D., Yazdi, F., Catalá-López, F., Tricco, A. C., & Moher, D. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.10.017 · Cited by 115
“In addition, sharing of data sets and statistical analysis code allows other researchers to cumulatively add new data that are published, thus keeping meta-analytic effect estimates up-to-date [6,7].”
BackgroundClarifying differences between reviews within evidence ecosystems
Gough, D., Thomas, J., & Oliver, S. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1089-2 · Cited by 115
Self-citationThe potential for AI to revolutionize conservation: a horizon scan
Reynolds, S. A., Beery, S., Burgess, N. D., Burgman, M. A., Butchart, S. H. M., Cooke, S. J., Coomes, D. A., Danielsen, F., Minin, E. D., Durán, A. P., et al. (2024) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2024.11.013 · Cited by 172
“AI has the potential to be harnessed to identify relevant (new) evidence and integrate it into the existing evidence base and synthesize key messages rapidly (i.e., living evidence syntheses [81]).”
Preventive Treatments for Psychosis: Umbrella Review (Just the Evidence)
Fusar-Poli, P., Davies, C., Solmi, M., Brondino, N., De Micheli, A., Kotlicka-Antczak, M., Shin, J. I., & Radua, J. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00764 · Cited by 108
“reviews” were proposed as a framework for continuously updating meta-analyses[41] (Figure 2).”
BackgroundAdherence to Guidelines in Adult Patients with Traumatic Brain Injury: A Living Systematic Review
Cnossen, M. C., Scholten, A. C., Lingsma, H. F., Synnot, A., Tavender, E., Gantner, D., Lecky, F., Steyerberg, E. W., & Polinder, S. (2021) · Journal of Neurotrauma · doi:10.1089/neu.2015.4121 · Cited by 107
“A living systematic review is a high quality, up-to-date, online summary of health research that is updated as new research becomes available.(14) This means that the searches will be rerun frequently and new studies will be incorporated into the review, with revisions to recommendations as appropriate.”
BackgroundNikolakopoulou, A., Mavridis, D., Furukawa, T. A., Cipriani, A., Tricco, A. C., Straus, S. E., Siontis, G. C. M., Egger, M., & Salanti, G. (2018) · BMJ · doi:10.1136/bmj.k585 · Cited by 107
Elbers, S., Wittink, H. M., Konings, S., Kaiser, U., Kleijnen, J., Pool, J. J. M., Köke, A. J. A., & Smeets, R. (2021) · European Journal of Pain · doi:10.1002/ejp.1875 · Cited by 129
Temporal Instability of Evidence Base: A Threat to Policy Making?
Koricheva, J., & Kulinskaya, E. (2019) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2019.05.006 · Cited by 104
Hair, K., Bahor, Z., Macleod, M., Liao, J., & Sena, E. S. (2021) · doi:10.1101/2021.05.04.442412 · Cited by 103
(2023) · doi:10.1186/s12915-023-01686-z · Cited by 103
“Increasingly, meta-researchers are aspiring to provide automated or “living” systematic reviews [20], producing real-time summaries of a research domain including the most recent research findings.”
BackgroundNeedham, E., McFadyen, C., Newcombe, V., Synnot, A., Czosnyka, M., & Menon, D. K. (2016) · Journal of Neurotrauma · doi:10.1089/neu.2016.4450 · Cited by 110
Twenty years of network meta‐analysis: Continuing controversies and recent developments
Ades, A. E., Welton, N. J., Dias, S., Phillippo, D., & Caldwell, D. M. (2024) · Research Synthesis Methods · doi:10.1002/jrsm.1700 · Cited by 126
“141 Whilst GRADE and CINeMA must, in their current forms, be carried out separately for each outcome, threshold analysis can be applied to treatment recommendations based on decision models that incorporate multiple outcomes, cost-effectiveness, 139 inconsistency, and bias modelling (Section 13).”
Systematic reviews of research in education: aims, myths and multiple methods
Gough, D., & Thomas, J. (2016) · Review of Education · doi:10.1002/rev3.3068 · Cited by 124
Self-citationInformation retrieval in the workplace: A comparison of professional search practices
Russell‐Rose, T., Chamberlain, J., & Azzopardi, L. A. (2018) · Information Processing & Management · doi:10.1016/j.ipm.2018.07.003 · Cited by 118
“ Systematic reviews: serve as the definitive search task, by synthesizing the complex, incomplete and at times conflicting findings of biomedical research into a form that can readily inform healthcare decision making (Elliott et al., 2014).”
BackgroundKorang, S. K., von Rohden, E., Veroniki, A. A., Ong, G., Ngalamika, O., Siddiqui, F., Juul, S., Nielsen, E. E., Feinberg, J. B., Petersen, J. J., et al. (2022) · PLOS ONE · doi:10.1371/journal.pone.0260733 · Cited by 95
“Our living systematic review will be continually updated to incorporate relevant new evidence as it becomes available [8, 89, 90].”
BackgroundBwire, G. M., Njiro, B. J., Mwakawanga, D. L., Sabas, D., & Sunguya, B. F. (2020) · Journal of Medical Virology · doi:10.1002/jmv.26622 · Cited by 102
Vogel, J. P., Dowswell, T., Lewin, S. A., Bonet, M., Hampson, L., Kellie, F. J., Portela, A. G., Bucagu, M., Norris, S. L., Neilson, J. P., et al. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2019-001683 · Cited by 108
Lewin, S., & Glenton, C. (2018) · International Journal for Equity in Health · doi:10.1186/s12939-018-0841-x · Cited by 92
McFadyen, C. A., Zeiler, F. A., Newcombe, V., Synnot, A., Steyerberg, E., Gruen, R. L., Rosand, J., Palotie, A., Maas, A. I. R., & Menon, D. K. (2021) · Journal of Neurotrauma · doi:10.1089/neu.2018.6052 · Cited by 90
“A living systematic review is a high quality up-to-date online summary of health research that is updated as new research becomes available.(13) In practice, this means that the searches will be re-run and any new studies incorporated into the review.”
BackgroundSelf-citationSystematic reviews with published protocols compared to those without: more effort, older search
Allers, K., Hoffmann, F., Mathes, T., & Pieper, D. (2017) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2017.12.005 · Cited by 98
Ramírez‐Campillo, R., Sortwell, A., Moran, J., Afonso, J. R., Clemente, F. M., Lloyd, R. S., Oliver, J. L., Pedley, J. S., & Granacher, U. (2023) · Sports Medicine - Open · doi:10.1186/s40798-023-00568-6 · Cited by 162
Michie, S., & Johnston, M. (2017) · Implementation Science · doi:10.1186/s13012-017-0660-2 · Cited by 83
“Even where good evidence exists and is synthesised, the resulting review and recommendations are inevitably out of date by the time it is complete, due to the laborious work involved [7].”
BackgroundBudrionis, A., Plikynas, D., Daniušis, P., & Indriulionis, A. (2020) · Assistive Technology · doi:10.1080/10400435.2020.1743381 · Cited by 117
“Second, living systematic reviews are a relatively new type of up-to-date semi-automated online summaries of research that are updated as new research becomes available (Elliott et al., 2014).”
BackgroundCurrie, G. L., Angel-Scott, H. N., Colvin, L., Cramond, F., Hair, K., Khandoker, L., Liao, J., Macleod, M. R., McCann, S., Morland, R., et al. (2019) · PLoS Biology · doi:10.1371/journal.pbio.3000243 · Cited by 87
“We plan that the present systematic review form a ‘baseline’ systematic review, which can be updated and developed into a living systematic review, i.e., one that is continually updated as new evidence is published [35].”
BackgroundSelf-citationConservation Research, Policy and Practice
Sutherland, W. J., Brotherton, P. N. M., Ockendon, N., Pettorelli, N., Vickery, J. A., Davies, Z. G., Boyd, I., Wintle, B. C., Kennicutt, M. C., II, Altringham, J. D., et al. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108638210 · Cited by 89
(2022) · doi:10.1371/journal.pmed.1003987 · Cited by 81
Michie, S., West, R., Finnerty, A. N., Norris, E., Wright, A. J., Marques, M. M., Johnston, M., Kelly, M. P., Thomas, J., & Hastings, J. (2020) · Wellcome Open Research · doi:10.12688/wellcomeopenres.15902.1 · Cited by 76
Self-citationFeasibility and acceptability of living systematic reviews: results from a mixed-methods evaluation
Millard, T., Synnot, A., Elliott, J., Green, S., McDonald, S., & Turner, T. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1248-5 · Cited by 77
Self-citationNeale, E. P., & Tapsell, L. C. (2018) · Advances in Nutrition · doi:10.1093/advances/nmy113 · Cited by 83
Epistemonikos: a comprehensive database of systematic reviews for health decision-making
on behalf of Epistemonikos project, Rada, G., Pérez, D., Araya‐Quintanilla, F., Ávila, C., Bravo‐Soto, G., Bravo-Jeria, R., Cánepa, A., Capurro, D., Castro-Gutiérrez, V., et al. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01157-x · Cited by 87
“tion of better and more sustainable review models [33].”
BackgroundBerkhout, W. E. M., van Wijngaarden, J. J., Workum, J. D., van de Sande, D., Hilling, D. E., Jung, C., Meyfroidt, G., Gommers, D., Buijsman, S. N. R., & van Genderen, M. E. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2025.22866 · Cited by 75
Prediction models for diagnosis and prognosis in Covid-19
Sperrin, M., Grant, S. W., & Peek, N. B. (2020) · BMJ · doi:10.1136/bmj.m1464 · Cited by 79
Hussain, S., Riad, A., Singh, A., Klugarová, J., Antony, B., Banna, H., & Klugar, M. (2021) · Journal of Fungi · doi:10.3390/jof7110985 · Cited by 74
“Living Review A living systematic review is a form of evidence synthesis which is updated regularly when sufficient new evidence becomes available [60].”
BackgroundSagaser, K. G., Malinowski, J. R., Westerfield, L. E., Proffitt, J., Hicks, M. A., Toler, T. L., Blakemore, K. J., Stevens, B., & Oakes, L. M. (2023) · Journal of Genetic Counseling · doi:10.1002/jgc4.1676 · Cited by 81
“An alternative would be to update and analyze the evidence base regularly, as part of a “living systematic evidence review” (Elliott et al., 2014) and to reassess the strength and direction of the recommendation statement in accordance with any new evidence.”
BackgroundOerther, B., Nedelcu, A., Engel, H., Schmucker, C. M., Schwarzer, G., Brugger, T., Schoots, I. G., Eisenblätter, M., Sigle, A., Gratzke, C., et al. (2024) · Radiology · doi:10.1148/radiol.233337 · Cited by 116
Marshall, Z., Welch, V. A., Minichiello, A., Swab, M., Brunger, F., & Kaposy, C. (2019) · Transgender Health · doi:10.1089/trgh.2018.0020 · Cited by 86
Akl, E. A., Khabsa, J., Iannizzi, C., Piechotta, V., Kahale, L. A., Barker, J. M., McKenzie, J. E., Page, M. J., & Skoetz, N. (2024) · BMJ · doi:10.1136/bmj-2024-079183 · Cited by 103
Vandvik, P. O., Brignardello-Petersen, R., & Guyatt, G. H. (2016) · BMC Medicine · doi:10.1186/s12916-016-0596-4 · Cited by 69
“Improved methods, strategies, and tools for more efficient authoring, dissemination, and updating of systematic reviews and trustworthy guidelines are emerging [13–15].”
MethodZeiler, F. A., McFadyen, C., Newcombe, V., Synnot, A., Donoghue, E. L., Ripatti, S., Steyerberg, E. W., Gruen, R. L., McAllister, T. W., Rosand, J. M., et al. (2021) · Journal of Neurotrauma · doi:10.1089/neu.2017.5583 · Cited by 73
“A living systematic review is a high-quality, up-to-date online summary of health research that is updated as new research becomes available.(12) In practice, this means that the searches will be rerun frequently and any new studies incorporated into the review.”
BackgroundSelf-citationJuul, S., Nielsen, N., Bentzer, P., Veroniki, A. A., Thabane, L., Linder, A., Klingenberg, S. L., Gluud, C. N., & Jakobsen, J. C. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01371-0 · Cited by 73
“There are four fundamental differences between conventional systematic reviews and living systematic reviews: publication format, work processes, author team management, and statistical methods [19].”
Living systematic review to assess the analgesic undertreatment in cancer patients
Roberto, A., Greco, M. T., Uggeri, S., Cavuto, S., Deandrea, S., Corli, O., & Apolone, G. (2022) · Pain Practice · doi:10.1111/papr.13098 · Cited by 80
“A living systematic review is a recent development in evidencebased medicine that foresees a prospective plan for continual surveillance of evidence with periodic critical appraisal and synthesis of new evidence.(77) Living reviews are usually found in research areas where there is a need of continuous update due to a…”
BackgroundA new ecosystem for evidence synthesis
Nakagawa, S., Dunn, A. G., Lagisz, M., Bannach‐Brown, A., Grames, E. M., Sánchez‐Tójar, A., O’Dea, R. E., Noble, D. W. A., Westgate, M. J., Arnold, P. A., et al. (2020) · Nature Ecology & Evolution · doi:10.1038/s41559-020-1153-2 · Cited by 69
Noel-Storr, A., Dooley, G., Affengruber, L., & Gartlehner, G. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.09.024 · Cited by 61
Cooke, S. J., Cook, C. N., Nguyen, V. M., Walsh, J. C., Young, N., Cvitanovic, C., Grainger, M., Randall, N., Muir, M. J., Kadykalo, A. N., et al. (2023) · Environmental Evidence · doi:10.1186/s13750-023-00302-5 · Cited by 90
“In these cases, “living reviews” have been suggested as a potential solution [37].”
BackgroundVolovici, V., Steyerberg, E. W., Cnossen, M. C., Haitsma, I., Dirven, C. M. F., Maas, A. I. R., & Lingsma, H. F. (2019) · Journal of Neurotrauma · doi:10.1089/neu.2019.6474 · Cited by 59
Riaz, I. B., Khan, M. S., Riaz, H., & Goldberg, R. J. (2015) · The American Journal of Medicine · doi:10.1016/j.amjmed.2015.10.009 · Cited by 61
Cramond, F., O’Mara-Eves, A., Doran-Constant, L., Rice, A. S. C., Macleod, M. R., & Thomas, J. (2018) · Wellcome Open Research · doi:10.12688/wellcomeopenres.14738.3 · Cited by 74
“The ultimate aim for the future would be “living” systematic reviews, which are updated constantly as new research evidence becomes available 13 .”
BackgroundSelf-citationA living systematic review protocol for COVID-19 clinical trial registrations
Maguire, B., & Guerin, P. J. (2020) · Wellcome Open Research · doi:10.12688/wellcomeopenres.15821.1 · Cited by 57
MetaBUS as a vehicle for facilitating meta-analysis
Bosco, F. A., Uggerslev, K. L., & Steel, P. (2016) · Human Resource Management Review · doi:10.1016/j.hrmr.2016.09.013 · Cited by 67
Metadata
- Authors
- Elliott, J. H., Turner, T., Clavisi, O., Thomas, J., Higgins, J. P. T., Mavergames, C., & Gruen, R. L.
- Affiliations
- The Alfred Hospital (AU)
- Vision Australia (AU)
- University of London (GB)
- University of Bristol (GB)
- Corresponding author
- Elliott, Julian H.
- ORCID
- Venue
- PLoS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2014-02-18
- Volume
- 11
- Issue
- 2
- Pages
- e1001603
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC3928029
- OpenAlex
- W2014453502
- Cited by
- 596
- References
- 35
- Citations by year
- 2026: 38 · 2025: 47 · 2024: 48 · 2023: 62 · 2022: 66 · 2021: 80 · 2020: 78 · 2019: 41 · 2018: 56 · 2017: 29 · 2016: 24 · 2015: 19 · 2014: 7
- FWCI
- 7.25
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Monash University Research Portal (Monash University) (submittedVersion, cc-by)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- UCL Discovery (University College London) (submittedVersion)
- ANU Open Research (Australian National University) (submittedVersion)
- Bristol Research (University of Bristol) (submittedVersion)
- Funders
- Medical Research Council (MR/J005037/1)
- National Health and Medical Research Council
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- living systematic review, review updating, knowledge-practice gap, evidence synthesis, health evidence, scholarly communication, evidence ecosystem, health care system
- MeSH terms
- Cooperative Behavior, Humans, Review Literature as Topic, Time Factors, Evidence-Based Medicine, Access to Information, Information Dissemination, Biomedical Research, Workflow
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.