The DOI matches its Crossref record; 48 of the 50 references it cites were found in a registry or an index; 32 of the 89 sources its claims cite were checked for support.
How to conduct systematic reviews more expeditiously?
Open accessAbstract
Healthcare consumers, researchers, patients and policy makers increasingly use systematic reviews (SRs) to aid their decision-making process. However, the conduct of SRs can be a time-consuming and resource-intensive task. Often, clinical practice guideline developers or other decision-makers need to make informed decisions in a timely fashion (e.g. outbreaks of infection, hospital-based health technology assessments). Possible approaches to address the issue of timeliness in the production of SRs are to (a) implement process parallelisation, (b) adapt and apply innovative technologies, and/or (c) modify SR processes (e.g. study eligibility criteria, search sources, data extraction or quality assessment). Highly parallelised systematic reviewing requires substantial resources to support a team of experienced information specialists, reviewers and methodologists working alongside with clinical content experts to minimise the time for completing individual review steps while maximising the parallel progression of multiple steps. Effective coordination and management within the team and across external stakeholders are essential elements of this process. Emerging innovative technologies have a great potential for reducing workload and improving efficiency of SR production. The most promising areas of application would be to allow automation of specific SR tasks, in particular if these tasks are time consuming and resource intensive (e.g. language translation, study selection, data extraction). Modification of SR processes involves restricting, truncating and/or bypassing one or more SR steps, which may risk introducing bias to the review findings. Although the growing experiences in producing various types of rapid reviews (RR) and the accumulation of empirical studies exploring potential bias associated with specific SR tasks have contributed to the methodological development for expediting SR production, there is still a dearth of research examining the actual impact of methodological modifications and comparing the findings between RRs and SRs. This evidence would help to inform as to which SR tasks can be accelerated or truncated and to what degree, while maintaining the validity of review findings. Timely delivered SRs can be of value in informing healthcare decisions and recommendations, especially when there is practical urgency and there is no other relevant synthesised evidence.
Overview
CoChat Summary
TL;DR
This commentary outlines three approaches to expedite systematic reviews (SRs): process parallelisation, innovative technologies/automation, and methodological modifications. It concludes that parallelisation can speed reviews without increasing bias or reducing cost, while automation and methodological modifications may reduce cost but risk bias, and more empirical evidence comparing rapid reviews (RRs) with SRs is needed.
Key findings
- Three approaches are proposed: process parallelisation, application of innovative technologies, and methodological modifications. Table 1 indicates all three increase speed; parallelisation does not reduce cost or increase risk of bias, whereas innovative technologies and methodological modifications reduce cost but possibly increase risk of bias.
- Automation examples include a semi-automated screening strategy that reduced screening workload by 50% without missing any relevant bibliographic citation; RobotReviewer achieved 71.0% overall accuracy versus 78.3% for published review assessments (a 10% difference); and Google Translate accuracy varied by extraction item and language across Chinese, Japanese, Spanish, French, and German.
- Methodological modifications and RRs vary widely: RR production time ranges from 1 to 32 weeks. Adding Embase to Medline changed the pooled effect estimate by only 6%; searches in databases additional to CCTR, Medline, and Embase identified only 2.4% more studies; and in 9 SRs of diagnostic test accuracy studies, 93% of 302 included citations were retrieved by searching Medline, Embase, and reference lists.
- Bypassing quality assessment may introduce substantial bias: pooled estimates from low-quality trials showed 34% greater benefit than high-quality trials. The paper states there is still insufficient empirical evidence comparing RR findings with SR findings and no single accepted definition of RR or formally established methodology guidance for conducting RRs.
Methods
- Design: narrative methodological commentary/overview, not an empirical study or systematic review.
- Data or sample: no primary data; the paper draws on published methodological and empirical literature and examples, including studies of automation, rapid reviews, search strategies, language restriction, and quality assessment.
- Main analysis: conceptual synthesis of three approaches to expediting SRs, with a table summarising expected impacts on speed, cost, and risk of bias, and discussion of illustrative evidence.
Limitations
- Stated: there is a dearth of research examining the actual impact of methodological modifications and comparing findings between RRs and SRs; empirical evidence from such comparisons is insufficient.
- Stated: there is no single accepted definition of what a rapid review constitutes, nor any formally established methodology guidance for how to conduct RRs.
- Stated: much of the evidence focuses on SRs of randomised trials of health interventions, and more empirical evidence is needed as SR methodology expands to other question types. Evident from the text: as a commentary/overview, it does not report a systematic search strategy or formal quality appraisal of the cited evidence.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
15 Valid · 1 Not valid · 21 Not checked · 30 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“RRs are better suited for narrowly defined research questions where one or more SR steps may be reduced or omitted [4, 6, 21, 22, 24, 25].”
§ How to conduct systematic reviews more expeditiously? > Main text > Methodological modifications
What the sources say
- Disputedcites [4] Khangura S, Konnyu K, Cushman R, Grimshaw J, Moher D. Evidence summaries: the evolution of a rapid review approach. Syst Rev. 2012;1:10. doi: 10.1186/2046-4053-1-10 . sourceExperimental — accuracy not yet measured
Khangura et al. (2012) state the opposite on question scope: the Background (p. 1) says systematic reviews 'often focus on a narrow clinical question or set of questions' whereas knowledge users need resources addressing 'a broader scope of scientific evidence quickly', and rapid review 'has evolved to address this need'; Table 1 (Background) codes the rapid review question as 'specified a priori (may include broad PICOS)' against the systematic review's 'focused clinical question (focused PICOS)'. The paper does support the streamlining half of the claim (Table 1: 'Sources may be limited', critical appraisal 'SRs only'; Screening and selection of studies: single-reviewer/'liberal accelerated' screening; Discussion: 'Further research evaluating the relative impact of streamlining other systematic review methods is required'), but it nowhere holds that rapid reviews are better suited to narrowly defined questions — its primary stated difference is timeframe (≤5 weeks vs 6 months–2 years), and it frames rapid review as the vehicle for broader, contextual evidence questions. (LLM, conf=0.70)
Checked by a reasoning model — may be wrong.
- Supportedcites [6] Featherstone RM, Dryden DM, Foisy M, Guise JM, Mitchell MD, Paynter RA, et al. Advancing knowledge of rapid reviews: an analysis of results, conclusions and recommendations from published review articles examining rapid reviews. Syst Rev. 2015;4:50. doi: 10.1186/s13643-015-0040-4 . source
The Results section 'Rapid review scope and question types' (p. n/a) states that RRs have narrower scope than SRs, that appropriate RR questions are 'on focused topics of efficacy or effectiveness', that complex interventions/economics/ethics/safety/social policy questions are inappropriate, and that Cameron cautioned RRs 'should be written in response to specific questions'. The Discussion states that key questions for conducting reviews rapidly are 'what steps are eliminated or reduced in comparison to a standard SR' and that producers must decide the acceptable level of rigour 'given the nature of the question'. Together these support the paraphrase that RRs suit narrowly defined questions in which SR steps are reduced or omitted; the paper reports these as findings/conclusions of the 12 review articles it analysed rather than as its own primary study. (LLM, conf=0.85)
- Not checked yetcites [21] Khangura S, Polisena J, Clifford TJ, Farrah K, Kamel C. Rapid review: an emerging approach to evidence synthesis in health technology assessment. Int J Technol Assess Health Care. 2014;30(1):20–7. doi: 10.1017/S0266462313000664 . source
- Not checked yetcites [22] Harker J, Kleijnen J. What is a rapid review? A methodological exploration of rapid reviews in Health Technology Assessments. Int J Evid Based Healthc. 2012;10(4):397–410. doi: 10.1111/j.1744-1609.2012.00290.x . source
- Not checked yetcites [24] Hartling L, Guise JM, Kato E, Anderson J, Aronson N, Belinson S, et al. EPC methods: an exploration of methods and context for the production of rapid reviews. Rockville (MD). 2015. http://effectivehealthcare.ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/?productid=2047&pageaction=displayproduct . Accessed on 5 November 2015.
- Not checked yetcites [25] Watt A, Cameron A, Sturm L, Lathlean T, Babidge W, Blamey S, et al. Rapid reviews versus full systematic reviews: an inventory of current methods and practice in health technology assessment. Int J Technol Assess Health Care. 2008;24(2):133–9. doi: 10.1017/s0266462308080185 . source
hedgedstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Disputed
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The conduct of SRs can be a time-consuming, cost- and resource-intensive task, which may take on average from 6 months to several years [4–6].”
§ How to conduct systematic reviews more expeditiously? > Background > Problems of timeliness and cost
What the sources say
- Partly supportedcites [4] Khangura S, Konnyu K, Cushman R, Grimshaw J, Moher D. Evidence summaries: the evolution of a rapid review approach. Syst Rev. 2012;1:10. doi: 10.1186/2046-4053-1-10 . sourceExperimental — accuracy not yet measured
The Background states that systematic reviews usually take 6 months to 2 years, supporting the claim that they are time-consuming. It does not give a timeframe of several years. The discussion of minimizing resources in the Systematic literature search section suggests a resource burden, but the paper does not directly establish the claim that systematic reviews are cost-intensive. (LLM, conf=0.88)
- Partly supportedcites [5] Ganann R, Ciliska D, Thomas H. Expediting systematic reviews: methods and implications of rapid reviews. Implement Sci. 2010;5:56. doi: 10.1186/1748-5908-5-56 . sourceExperimental — accuracy not yet measured
The Background states that systematic reviews typically take a minimum of six months to one year, supporting the claim that they take substantial time. The Discussion mentions additional cost and time associated with systematic searching, but the paper does not establish an average duration extending to several years. (LLM, conf=0.85)
- Partly supportedcites [6] Featherstone RM, Dryden DM, Foisy M, Guise JM, Mitchell MD, Paynter RA, et al. Advancing knowledge of rapid reviews: an analysis of results, conclusions and recommendations from published review articles examining rapid reviews. Syst Rev. 2015;4:50. doi: 10.1186/s13643-015-0040-4 . sourceExperimental — accuracy not yet measured
The Background (page unavailable) says systematic reviews typically take many months or even years and contrasts them with rapid reviews that may use fewer resources. It does not give an average duration, specify six months as the lower end, or directly establish that systematic reviews are cost-intensive. The six-month figure elsewhere concerns rapid reviews, not standard systematic reviews. (LLM, conf=0.84)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“The authors observed only a 10 % difference in the overall accuracy between the RoB assessments by the machine-learning system vs. published review (71.0 % vs. 78.3 %) [20].”
§ How to conduct systematic reviews more expeditiously? > Main text > Application of innovative technologies
What the sources say
- Supportedcites [20] Marshall IJ, Kuiper J, Wallace BC. RobotReviewer: evaluation of a system for automatically assessing bias in clinical trials. JAMIA. 2015. doi: 10.1093/jamia/ocv044 . source
The Results (Document-Level Results) report overall accuracy of 71.0% for the machine-learning model versus 78.3% for published reviews, describing the difference as less than 10%. The claim accurately paraphrases those figures. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“The most efficient use and application of the machine-learning technologies would be in the areas allowing automation of specific SR processes, in particular those involving time-consuming and resource-intensive tasks such as language translation [15], study selection [11, 16–18], data extraction [19] and risk of bias assessment [20].”
§ How to conduct systematic reviews more expeditiously? > Main text > Application of innovative technologies
What the sources say
- Not checked yetcites [15] Balk EM, Chung M, Chen ML, Trikalinos TA, Kong Win Chang L. Assessing the accuracy of google translate to allow data extraction from trials published in non-english languages. Rockville (MD). 2013. http://effectivehealthcare.ahrq.gov/index.cfm/search-for-guides-reviews-and-reports/?productid=1386&pageaction=displayproduct . Accessed on 5 November 2015.
- Supportedcites [11] O’Mara-Eves A, Thomas J, McNaught J, Miwa M, Ananiadou S. Using text mining for study identification in systematic reviews: a systematic review of current approaches. Syst Rev. 2015;4:5. doi: 10.1186/2046-4053-4-5 . source
The citation [11, 16–18] is attached to study selection, not to translation, data extraction, or risk-of-bias assessment. In Results > Reducing the number of items that need to be screened, the source reports that the systems tested reduced screening workload; Discussion > Further possibilities describes study-selection automation as possibly the most advanced area of systematic-review automation. (LLM, conf=0.92)
- Supportedcites [16] Wallace BC, Trikalinos TA, Lau J, Brodley C, Schmid CH. Semi-automated screening of biomedical citations for systematic reviews. BMC Bioinformatics. 2010;11:55. doi: 10.1186/1471-2105-11-55 . source
The citation to this paper is attached to study selection, not to the other tasks in the sentence. The Background explains that determining which articles are suitable for inclusion is laborious and time-intensive, and that the paper uses machine learning to semi-automate citation screening. (LLM, conf=0.93)
- Supportedcites [17] Miwa M, Thomas J, O’Mara-Eves A, Ananiadou S. Reducing systematic review workload through certainty-based screening. J Biomed Inform. 2014;51:242–53. doi: 10.1016/j.jbi.2014.06.005 . source
Split into 2 atoms — 2 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
- Supportedcites [18] Elliott JH, Turner T, Clavisi O, Thomas J, Higgins JP, Mavergames C, et al. Living systematic reviews: an emerging opportunity to narrow the evidence-practice gap. PLoS Med. 2014;11(2):e1001603. doi: 10.1371/journal.pmed.1001603 . source
The citation in the sentence is attached to study selection. In “Production of Living Systematic Reviews > Semi-automation,” the paper says machine learning can reduce manual screening of titles and abstracts, a study-selection task. The separately cited claims about translation, data extraction, and risk-of-bias assessment are not necessary to this verdict. (LLM, conf=0.91)
- Supportedcites [19] Jonnalagadda SR, Goyal P, Huffman MD. Automating data extraction in systematic reviews: a systematic review. Syst Rev. 2015;4(1):78. doi: 10.1186/s13643-015-0066-7 . source
Split into 2 atoms — 2 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
hedgedstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Every source the paper cites for this claim was checked and supports it.
In the paper
“The authors illustrated that not all SR tasks are equally amenable to automation.”
§ How to conduct systematic reviews more expeditiously? > Main text > Application of innovative technologies
What the sources say
- Supportedcites [12] Tsafnat G, Glasziou P, Choong MK, Dunn A, Galgani F, Coiera E. Systematic review automation technologies. Syst Rev. 2014;3:74. doi: 10.1186/2046-4053-3-74 . source
The Background states that some systematic review tasks are not amenable to automation or have not yet been examined for it. The Discussion (‘What should and should not be automated’) distinguishes creative tasks requiring human judgment from technical tasks that can be performed automatically. Together, these passages support the paraphrase. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
20 more claims · 7 with their cited sources checked
Create a free account to see all 67 claims and to verify the 57 references still open.
- Probably not a substantial claim. 34
- It may state two separate claims as one. 13
- The quote only partly states this claim. 3
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“A typical SR takes from 6 months to 2 years or more to write [12].”
From Making progress with the automation of systematic reviews: principles of the International Collaboration for the Automation of Systematic Reviews (ICASR) (2018) · cites this paper as [12]
Citations
Cites
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048 · cited 1×
“These approaches have been shown to minimise bias and improve the precision of review findings [3].”
BackgroundA typology of reviews: an analysis of 14 review types and associated methodologies
Grant, M. J., & Booth, A. (2009) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2009.00848.x · Cited by 10,817 · cited 1×
“The evidence regarding the need for quality assessment of studies included in SRs is more consistent in indicating that bypassing this important step may lead to substantial bias in the review estimates [37–39, 49, 50].”
BackgroundMoher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., Tugwell, P., & Klassen, T. P. (1998) · The Lancet · doi:10.1016/s0140-6736(98)01085-x · Cited by 3,560 · cited 1×
“A clear illustration of this phenomenon was shown in the study by Moher and colleagues, where the pooled estimate of low-quality trials, compared to high qualitytrials, demonstrated 34 % greater benefit in the treatment effect [38].”
ResultSelf-citationAssessing the quality of controlled clinical trials
Jüni, P., Altman, D. G., & Egger, M. (2001) · BMJ · doi:10.1136/bmj.323.7303.42 · Cited by 3,309
Systematic Reviews: Synthesis of Best Evidence for Clinical Decisions
Cook, D. J., Mulrow, C. D., & Haynes, R. B. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-126-5-199703010-00006 · Cited by 2,184 · cited 1×
“Systematic reviews (SRs) are transparent and succinct evidence synthesis summaries of empirical results of primary research studies addressing one or more questions regarding any given health problem, intervention(s) or policy decision [1, 2].”
BackgroundSystematic Reviews: Rationale for systematic reviews
Mulrow, C. D. (1994) · BMJ · doi:10.1136/bmj.309.6954.597 · Cited by 1,677
meta-analysis bias in location and selection of studies
Egger, M., & Smith, G. D. (1998) · BMJ · doi:10.1136/bmj.316.7124.61 · Cited by 1,374
Morrison, A., Polisena, J., Husereau, D., Moulton, K., Clark, M., Fiander, M., Mierzwinski-Urban, M., Clifford, T., Hutton, B., & Rabb, D. (2012) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462312000086 · Cited by 1,257 · cited 1×
“However, the evidence whether or not the exclusion of non-English language study publications of conventional healthcare interventions introduces bias has been inconsistent, some authors showing meta-analyses of only English language studies yielding more conservative estimates [29], and others not demonstrating the p…”
ResultEvidence summaries: the evolution of a rapid review approach
Khangura, S., Konnyu, K., Cushman, R., Grimshaw, J., & Moher, D. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-10 · Cited by 1,232 · cited 2×
“Understandably, there is no single accepted definition of what a RR constitutes [22, 26], nor is there any formally established methodology guidance as how to conduct RRs (or any type of RR) [4].”
BackgroundSelf-citationEgger, M., Jüni, P., Bartlett, C., Holenstein, F., & Sterne, J. (2003) · Health Technology Assessment · doi:10.3310/hta7010 · Cited by 1,181
Research works printed but not linked: 7
- Hopman JK Z, Edrees H, Allen T, Allegranzi B. WHO Guideline and systematic review on hand hygiene and the use of chlorine in the context of Ebola. Geneva: World Health Organization; 2015.
- WHO/UNICEF/WFP. Nutritional care of children and adults with Ebola virus disease in treatment centres. Interim guideline. Geneva: World Health Organization; 2014.
- Grama A, Gupta A, Karypis G, Kumar V. Introduction to parallel computing (2nd edition). Essex: Pearson; 2003.
- Scott A, Harstall C. Utilizing diverse HTA products in the Alberta Health Technologies Decision Process: Work in Progress. 2012. http://www.ihe.ca/publications/utilizing-diverse-hta-products-in-the-alberta-health-technologies-decision-process-work-in-progress . Accessed on 5 November 2015.
- Cochrane Innovations. Cochrane response—the new rapid review from Cochrane. 2014. http://innovations.cochrane.org/response . Accessed on 5 November 2015.
- Webber J. Rapid Review Summit: Then Now and in the Future. Vancouver, Biritish Columbia February 3 and 4, 2015. https://www.cadth.ca/sites/default/files/pdf/RR%20Summit_FINAL_Report.pdf . Accessed on 5 November 2015.
- Cochrane Collaboration. The EMBASE screening project: six months old and going strong. 2014. http://community.cochrane.org/news/news-events/current-news/embase-screening-project-six-months-old-and-going-strong . Accessed on 5 November 2015.
Cited by
Grames, E. M., Stillman, A. N., Tingley, M. W., & Elphick, C. S. (2019) · Methods in Ecology and Evolution · doi:10.1111/2041-210x.13268 · Cited by 377
“…for automation of the most time‐intensive tasks while still maintaining the standards of conventional search methods (Delaney & Tams, 2018; O’Mara‐Eves, Thomas, McNaught, Miwa, & Ananiadou, 2015; Paisley & Foster, 2018; Tsafnat et al., 2014; Tsertsvadze, Chen, Moher, Sutcliffe, & McCarthy, 2015).”
MethodOn behalf of the founding members of the ICASR group, Beller, E., Clark, J., Tsafnat, G., Adams, C., Diehl, H., Lund, H., Ouzzani, M., Thayer, K., Thomas, J., et al. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0740-7 · Cited by 191
Langlois, E. V., Straus, S. E., Antony, J., King, V. J., & Tricco, A. C. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2018-001178 · Cited by 160
“12 For instance, some steps, like initial study screening, may be easier to automate than steps requiring more nuanced decisions, such as assessing risk of bias.”
Thielen, F. W. F., van Mastrigt, G. A. P. G., Burgers, L. T., Bramer, W. M., Majoie, M., Evers, S., & Kleijnen, J. (2016) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1080/14737167.2016.1246962 · Cited by 149
Potential Roles of Large Language Models in the Production of Systematic Reviews and Meta-Analyses
Luo, X., Chen, F., Zhu, D., Wang, L., Wang, Z., Liu, H., Lyu, M., Wang, Y., Wang, Q., & Chen, Y. (2024) · Journal of Medical Internet Research · doi:10.2196/56780 · Cited by 109
“The process of conducting systematic reviews demands a substantial investment in terms of time, resources, human effort, and financial capital [8].”
Self-citation
Metadata
- Authors
- Tsertsvadze, A., Chen, Y.-F., Moher, D., Sutcliffe, P., & McCarthy, N.
- Affiliations
- University of Warwick (GB)
- Ottawa Hospital (CA)
- Corresponding author
- Tsertsvadze, Alexander
- Venue
- Systematic Reviews
- Published
- 2015-11-12
- Volume
- 4
- Issue
- 1
- Pages
- 160-160
- ISSN
- 2046-4053 (ISSN-L)
- PMC
- PMC4643500
- OpenAlex
- W2180291732
- Cited by
- 104
- References
- 50
- Citations by year
- 2026: 5 · 2025: 9 · 2024: 11 · 2023: 6 · 2022: 7 · 2021: 25 · 2020: 12 · 2019: 9 · 2018: 7 · 2017: 9 · 2016: 4
- FWCI
- 1.10
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- Systematic Reviews (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Warwick Research Archive Portal (University of Warwick) (acceptedVersion)
- uO Research (University of Ottawa) (submittedVersion, public-domain)
- Funders
- Public Health England (GB)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- timeliness, rapid review, parallelization, data extraction, study eligibility criteria, information professionals, health technology assessment
- MeSH terms
- Administrative Personnel, Cost-Benefit Analysis, Decision Making, Disease Outbreaks, Humans, Research Design, Review Literature as Topic, Technology Assessment, Biomedical, Evidence-Based Medicine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-04, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-04 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash and gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How to conduct systematic reviews more expeditiously? https://app.cochat.ai/papers/10.1186/s13643-015-0147-7
- The paper
- Tsertsvadze, A., Chen, Y.-F., Moher, D., Sutcliffe, P., & McCarthy, N. (2015). How to conduct systematic reviews more expeditiously? Systematic Reviews. https://doi.org/10.1186/s13643-015-0147-7
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.