Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry
Open accessThe DOI matches its Crossref record; 26 of the 29 references it cites were found in a registry or an index; 2 of the 3 sources its claims cite were checked for support.
Abstract
OBJECTIVES: To summarise logistical aspects of recently completed systematic reviews that were registered in the International Prospective Register of Systematic Reviews (PROSPERO) registry to quantify the time and resources required to complete such projects. DESIGN: Meta-analysis. DATA SOURCES AND STUDY SELECTION: All of the 195 registered and completed reviews (status from the PROSPERO registry) with associated publications at the time of our search (1 July 2014). DATA EXTRACTION: All authors extracted data using registry entries and publication information related to the data sources used, the number of initially retrieved citations, the final number of included studies, the time between registration date to publication date and number of authors involved for completion of each publication. Information related to funding and geographical location was also recorded when reported. RESULTS: The mean estimated time to complete the project and publish the review was 67.3 weeks (IQR=42). The number of studies found in the literature searches ranged from 27 to 92 020; the mean yield rate of included studies was 2.94% (IQR=2.5); and the mean number of authors per review was 5, SD=3. Funded reviews took significantly longer to complete and publish (mean=42 vs 26 weeks) and involved more authors and team members (mean=6.8 vs 4.8 people) than those that did not report funding (both p<0.001). CONCLUSIONS: Systematic reviews presently take much time and require large amounts of human resources. In the light of the ever-increasing volume of published studies, application of existing computing and informatics technology should be applied to decrease this time and resource burden. We discuss recently published guidelines that provide a framework to make finding and accessing relevant literature less burdensome.
Overview
CoChat Summary
## TL;DR
Using 195 completed PROSPERO-registered systematic reviews of medical interventions, the authors estimated that reviews took a mean of 67.3 weeks to complete and publish and involved a mean of 5 authors per review. They conclude that systematic reviews are time- and labor-intensive, with funded reviews taking longer and involving more people than unfunded reviews.
## Key findings
- Across 195 completed reviews, the mean estimated time from registered project start to publication was 67.3 weeks (SD=31.0; range 6–186 weeks).
- The mean number of authors/team members was 5 (SD=3), and the mean merged yield rate was 2.94% (SD=6.49; range 0–64.71%).
- Literature searches were highly variable, with the number of studies found ranging from 27 to 92,020.
- Reviews that reported funding took longer and involved more people than those that did not report funding: 42 vs 26 weeks and 6.8 vs 4.8 people, both p<0.001.
## Methods
- Meta-analysis of PROSPERO registry records linked to completed review publications, using a sample of 195 registered and completed reviews with associated publications identified on 1 July 2014.
- Data were extracted from registry entries and publications on timing, authors/team members, funding, search sources, initial citations, and final included studies; the sample comprised 197 literature reviews from the 195 publications.
- Main analyses were descriptive summaries of means, ranges, and yield rates, plus ANOVA to compare funded vs unfunded reviews.
## Limitations
- The paper states that the study is limited by incomplete reporting in both the registry and the published articles, and that registry and publication data were cross-checked only where available.
- The authors state that the true project start date was difficult to capture, so their time estimates may be conservative and may underestimate time from the very first review activities.
- The paper notes that the data are highly skewed, making prediction of timelines statistically unsound; this is an evident limitation of the analysis.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures

Verification
Claims the paper makes
1 Valid · 1 Not valid · 1 Not checked · 23 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the mean number of authors per review was 5, SD=3.”
§ Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The mean estimated time to complete the project and publish the review was 67.3 weeks (IQR=42).”
- Also stated in: § Results · plain
“The mean project length (using the registered project start date to the review's publication date) was 67.3 weeks (SD=31.0; range 6–186 weeks).”
§ Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Also stated in: § Results · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the mean yield rate of included studies was 2.94% (IQR=2.5);”
- Also stated in: § Results · plain
“The mean merged yield rate was 2.94% (SD=6.49; range 0–64.71%).”
§ Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry > Abstract > Results
plainn = 190- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Also stated in: § Results · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Reviews that reported funding took longer to complete and included more authors/team members; this difference was not seen when we considered only whether specific authors were funded.”
§ Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The number of studies found in the literature searches ranged from 27 to 92 020;”
§ Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
9 more claims
Create a free account to see all 26 claims and every source verdict.
- Probably not a substantial claim. 11
- It may state two separate claims as one. 6
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Typically, only a small fraction of the records belong to the relevant class, making title and abstract screening an important bottleneck in systematic reviewing process 25 .”
From An open source machine learning framework for efficient and transparent systematic reviews (2021) · cites this paper as [25]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“requiring a median of five researchers and around 40 weeks of work to reach submission [ 3 , 4 ]”
From Reducing systematic review burden using Deduklick: a novel, automated, reliable, and explainable deduplication algorithm to foster medical research (2022) · cites this paper as [4]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“typically taking over 16 months and costing upwards of $100,000 to complete 1,2 .”
From Automation of Systematic Reviews with Large Language Models (2025) · cites this paper as [2]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs can automate critical tasks for SLRs, including screening abstracts and full texts, assessing risk of bias and extracting data for use in network meta-analyses with high accuracy, all while significantly reducing the time and costs required by using human review [2, 4–7].”
From Automated Mass Extraction of Over 680,000 PICOs from Clinical Study Abstracts Using Generative AI: A Proof-of-Concept Study. (2024) · cites this paper as [4]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The HTA process is very labour and resource intensive, requiring a large team of experts in the field of health economic outcomes research (HEOR), including SLR analysts, statisticians and economic modellers [5, 8].”
From Artificial Intelligence to Automate Network Meta-Analyses: Four Case Studies to Evaluate the Potential Application of Large Language Models (2024) · cites this paper as [8]
Experimental — accuracy not yet measured
18 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · cited 3×
“…clinical trials could be a starting point for testing the use of indexing systems that employ a Population, Intervention, Comparisons, Outcomes, Study Design (PICOS) 15 structure of coding metadata for clinical trials in order to make the scienti fi c literature become a truly searchable database.”
BackgroundInfluentialThe FAIR Guiding Principles for scientific data management and stewardship
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.18 · Cited by 18,423
Shea, B. J., Hamel, C., Wells, G. A., Bouter, L. M., Kristjansson, E., Grimshaw, J., Henry, D. A., & Boers, M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.10.009 · Cited by 1,755
The nuts and bolts of PROSPERO: an international prospective register of systematic reviews
Booth, A., Clarke, M., Dooley, G., Ghersi, D., Moher, D., Petticrew, M., & Stewart, L. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-2 · Cited by 1,433
Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?
Bastian, H., Glasziou, P., & Chalmers, I. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000326 · Cited by 1,188 · cited 1×
“1 Since the body of primary literature has increased, conducting systematic reviews and meta-analyses has become a necessity to more accurately and comprehensively present accumulated knowledge to scienti fi c, clinical and general audiences.”
BackgroundPROSPERO: an international prospective register of systematic reviews
Wright, K. E. (2014) · Cited by 568
(1965) · Nature · doi:10.1038/205236a0 · Cited by 560
Study flow diagrams in Cochrane systematic review updates: an adapted PRISMA flow diagram
Stovold, E., Beecher, D., Foxlee, R., & Noel-Storr, A. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-54 · Cited by 356
Cited by
An open source machine learning framework for efficient and transparent systematic reviews
van de Schoot, R., de Bruin, J., Schram, R., Zahedi, P., de Boer, J., Weijdema, F., Kramer, B., Huijts, M., Hoogerwerf, M., Ferdinands, G., et al. (2021) · Nature Machine Intelligence · doi:10.1038/s42256-020-00287-7 · Cited by 1,157
Munn, Z., Aromataris, E., Tufanaru, C., Stern, C., Porritt, K., Farrow, J., Lockwood, C., Stephenson, M., Moola, S., Lizarondo, L., et al. (2019) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000152 · Cited by 977
Belur, J., Tompson, L., Thornton, A., & Simon, M. (2021) · Sociological Methods & Research · doi:10.1177/0049124118799372 · Cited by 753
“A recent study (Borah et al. 2017) on the resources required to conduct systematic reviews indicated that the mean number of members involved in conducting a systematic review was 5 (SD 1⁄4 3) and the average time period required to conduct and publish a review was 67.”
BackgroundInfluentialA full systematic review was completed in 2 weeks using automation tools: a case study
Clark, J., Glasziou, P., Del Mar, C., Bannach-Brown, A., Stehlik, P., & Scott, A. M. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.01.008 · Cited by 687
Marshall, I. J., & Wallace, B. C. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1074-9 · Cited by 638
Clark, J. M., Sanders, S., Carter, M., Honeyman, D., Cleo, G., Auld, Y., Booth, D., Condron, P., Dalais, C., Bateup, S., et al. (2020) · Journal of the Medical Library Association · doi:10.5195/jmla.2020.834 · Cited by 601
Pollock, A., & Berge, E. (2018) · International Journal of Stroke · doi:10.1177/1747493017743796 · Cited by 530
“High quality up-to-date systematic reviews of primary research studies, addressing questions which are of recognized importance to stroke survivors, carers, and clinicians are therefore essential.(8,9) Systematic reviews are also important for the avoidance of research waste, by ensuring that new primary research is d…”
BackgroundRegistration of systematic reviews in PROSPERO: 30,000 records and counting
Page, M. J., Shamseer, L., & Tricco, A. C. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0699-4 · Cited by 509
“used the start dates recorded in PROSPERO to estimate the time required to complete 195 registered SRs that were published before July 2014 [15].”
MethodBest practice guidelines for abstract screening large‐evidence systematic reviews and meta‐analyses
Polanin, J. R., Pigott, T. D., Espelage, D. L., & Grotpeter, J. K. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1354 · Cited by 401
“After conducting a comprehensive literature search to identify studies, abstract screening is the next stage in the process where bias could threaten the validity of the identified studies: abstract screening actively eliminates studies from the review and therefore the potential for bias must not be overlooked.(7) Ye…”
BackgroundEffects of air pollution on health: A mapping review of systematic reviews and meta-analyses
Dominski, F. H., Lorenzetti Branco, J. H., Buonanno, G., Stabile, L., Gameiro da Silva, M., & Andrade, A. (2021) · Environmental Research · doi:10.1016/j.envres.2021.111487 · Cited by 382
Animal to human translation: a systematic scoping review of reported concordance rates
Leenaars, C. H. C., Kouwenaar, C., Stafleu, F. R., Bleich, A., Ritskes-Hoitinga, M., De Vries, R. B. M., & Meijboom, F. L. B. (2019) · Journal of Translational Medicine · doi:10.1186/s12967-019-1976-2 · Cited by 314
Khraisha, Q., Put, S., Kappenberg, J., Warraitch, A., & Hadfield, K. (2023) · arXiv · doi:10.48550/arxiv.2310.17526 · Cited by 268
“What is already known • Systematic reviews are crucial, but often slow, and the potential of Large Language Models (LLMs) such as Generative Pre-Trained Transformer (GPT)-4 for their automation has not been fully assessed against human performance.”
revtools: An R package to support article screening for evidence synthesis
Westgate, M. J. (2019) · Research Synthesis Methods · doi:10.1002/jrsm.1374 · Cited by 261
Khraisha, Q., Put, S., Kappenberg, J., Warraitch, A., & Hadfield, K. (2024) · Research Synthesis Methods · doi:10.1002/jrsm.1715 · Cited by 269
“What is already known • Systematic reviews are crucial, but often slow, and the potential of Large Language Models (LLMs) such as Generative Pre-Trained Transformer (GPT)-4 for their automation has not been fully assessed against human performance.”
Sutherland, W. J., Taylor, N. G., MacFarlane, D., Amano, T., Christie, A. P., Dicks, L. V., Lemasson, A. J., Littlewood, N. A., Martin, P. A., Ockendon, N., et al. (2019) · Biological Conservation · doi:10.1016/j.biocon.2019.108199 · Cited by 232
Artificial intelligence for literature reviews: opportunities and challenges
Bolaños, F., Salatino, A., Osborne, F., & Motta, E. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10902-3 · Cited by 246
“This is due to several factors, such as the duration of the whole process, which can extend beyond a year [9]; the necessity for a multidisciplinary team of experts, often accompanied by specific training [10]; significant financial implications arising from database subscriptions, specialised…”
Nye, B., Li, J. J., Patel, R., Yang, Y., Marshall, I., Nenkova, A., & Wallace, B. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1019 · Cited by 224
“It is thus practically impossible for physicians to know which is the best medical intervention for a given patient group and condition (Borah et al., 2017; Fraser and Dunstan, 2010; Bastian et al., 2010).”
BackgroundUsing artificial intelligence methods for systematic review in health sciences: A systematic review
Blaizot, A., Veettil, S. K., Saidoung, P., Moreno‐Garcia, C. F., Wiratunga, N., Aceves‐Martins, M., Lai, N. M., & Chaiyakunapruk, N. (2022) · Research Synthesis Methods · doi:10.1002/jrsm.1553 · Cited by 223
“of increasingly long systematic review publication time.(1) It must also be noted that Deng et al.”
BackgroundMethodInfluentialQureshi, R., Shaughnessy, D., Gill, K. A. R., Robinson, K. A., Li, T., & Agai, E. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02243-z · Cited by 201
Chai, K. E. K., Lines, R. L. J., Gucciardi, D. F., & Ng, L. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01635-3 · Cited by 190
Bannach-Brown, A., Przybyła, P., Thomas, J., Rice, A. S. C., Ananiadou, S., Liao, J., & Macleod, M. R. (2019) · Systematic Reviews · doi:10.1186/s13643-019-0942-7 · Cited by 185
“In clinical systematic reviews, Borah and colleagues [6] showed the average clinical systematic review registered on PROSPERO (International Prospective Register of Systematic Reviews) takes an average 67.”
BackgroundAssay of renewable energy transition: A systematic literature review
Bhattarai, U., Maraseni, T., & Apan, A. (2022) · Science of The Total Environment · doi:10.1016/j.scitotenv.2022.155159 · Cited by 173
O’Connor, A. M., Tsafnat, G., Thomas, J., Glasziou, P., Gilbert, S. B., & Hutton, B. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1062-0 · Cited by 167
Howard, B. E., Phillips, J., Tandon, A., Maharana, A., Elmore, R., Mav, D., Sedykh, A., Thayer, K., Merrick, B. A., Walker, V., et al. (2020) · Environment International · doi:10.1016/j.envint.2020.105623 · Cited by 157
“…estimated that more than 4,000 systematic reviews are conducted and published annually (Bastian et al., 2010), and while the precise time commitment can vary depending on the subject matter and protocol, reviews often require a year or more to complete (Ganann et al., 2010, Borah et al., 2016).”
BackgroundGates, A., Johnson, C., & Hartling, L. (2018) · Systematic Reviews · doi:10.1186/s13643-018-0707-8 · Cited by 148
“On average, after registering a protocol, it takes author teams more than 1 year of work before their SR is published [5].”
MethodScherbakov, D., Hubig, N., Jansari, V., Bakumenko, A., & Lenert, L. A. (2024) · arXiv · doi:10.48550/arxiv.2409.04600 · Cited by 156
Scherbakov, D., Hubig, N., Jansari, V., Bakumenko, A., & Lenert, L. A. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf063 · Cited by 154
Michelson, M., & Reuter, K. (2019) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2019.100443 · Cited by 152
“The estimated labor e ff ort to produce a single SLR ranges from a minimum of 6 months when an investigator devotes 10 – 20 h per week [18] to, on average, a total of 16 months involving fi ve co-authors [19], or 1 – 2 years for completing a Cochrane review [20].”
Understanding urban perception with visual data: A systematic review
Ito, K., Kang, Y., Zhang, Y., Zhang, F., & Biljecki, F. (2024) · Cities · doi:10.1016/j.cities.2024.105169 · Cited by 207
Hamel, C., Kelly, S. E., Thavorn, K., Rice, D. B., Wells, G. A., & Hutton, B. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01129-1 · Cited by 140
Scoping Reviews, Systematic Reviews, and Meta-Analysis: Applications in Veterinary Medicine
Sargeant, J. M., & O'Connor, A. M. C. (2020) · Frontiers in Veterinary Science · doi:10.3389/fvets.2020.00011 · Cited by 198
“Systematic reviews not only are time sensitive, but also require a considerable input of time and resources, taking an average of 67 weeks from protocol registration to publication (86).”
BackgroundNoel-Storr, A., Dooley, G., Elliott, J., Steele, E., Shemilt, I., Mavergames, C., Wisniewski, S., McDonald, S., Murano, M., Glanville, J., et al. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.01.006 · Cited by 132
“This information deluge is putting health evidence production systems under strain, as systematic reviewers often need to sift through large numbers of records, identified from sensitive searches performed across these and other databases, in search of eligible studies [3].”
BackgroundGates, A., Guitard, S., Pillay, J., Elliott, S. A., Dyson, M. P., Newton, A. S., & Hartling, L. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1222-2 · Cited by 133
How to optimize the systematic review process using AI tools
Fabiano, N., Gupta, A., Bhambra, N., Luu, B., Wong, S., Maaz, M., Fiedorowicz, J. G., Smith, A. L., & Solmi, M. (2024) · JCPP Advances · doi:10.1002/jcv2.12234 · Cited by 130
“On average, it takes 67 weeks and considerable human resources to take a systematic review from protocol registration to publication, with a range from 6 months to as long as 3 years, depending on the scope, methodology and resources available (Borah et al., 2017; Dicks et al., 2014).”
BackgroundWolffe, T. A. M., Whaley, P., Halsall, C., Rooney, A. A., & Walker, V. R. (2019) · Environment International · doi:10.1016/j.envint.2019.05.065 · Cited by 119
“Evidence from medical systematic reviews indicates it takes on average approximately 70weeks to progress a systematic review from protocol registration in the PROSPERO registry (National Institute for Health Research, 2018) to publication of the final systematic review (Borah et al., 2017).”
BackgroundInformation extraction from scientific articles: a survey
Nasar, Z., Jaffry, S. W., & Malik, M. K. (2018) · Scientometrics · doi:10.1007/s11192-018-2921-5 · Cited by 160
“In the light of another study, systematic literature review can take up to 186 weeks with single/multiple human resources (Borah et al. 2017).”
BackgroundPredicting the time needed for environmental systematic reviews and systematic maps
Haddaway, N. R., & Westgate, M. J. (2019) · Conservation Biology · doi:10.1111/cobi.13231 · Cited by 120
“Our estimate is under half that of the only other evidence-based assessment of which we are aware, which is approximately 337 d (Borah et al. 2017).”
BackgroundInfluentialBorissov, N., Haas, Q., Minder, B., Kopp-Heim, D., von Gernler, M., Janka, H., Teodoro, D., & Amini, P. (2022) · Systematic Reviews · doi:10.1186/s13643-022-02045-9 · Cited by 116
“Such analyses are time and resource intensive, requiring a median of five researchers and around 40 weeks of work to reach submission [3, 4].”
BackgroundTsou, A. Y., Treadwell, J. R., Erinoff, E., & Schoelles, K. (2020) · Systematic Reviews · doi:10.1186/s13643-020-01324-7 · Cited by 122
Mandrik, O. (., Severens, J. L. (., Bardach, A., Ghabri, S., Hamel, C., Mathes, T., Vale, L., Wisløff, T., & Goldhaber-Fiebert, J. D. (2021) · Value in Health · doi:10.1016/j.jval.2021.01.002 · Cited by 110
“19,20 We expect that SR-CCEOs will have similar timelines: adding search words related to costs to the search line used in a clinical SR will result in less hits, but a more complicated complementary search for grey literature will often be needed.”
BackgroundAutomation of duplicate record detection for systematic reviews: Deduplicator
Forbes, C., Greenwood, H., Carter, M., & Clark, J. M. (2024) · Systematic Reviews · doi:10.1186/s13643-024-02619-9 · Cited by 150
An open source machine learning framework for efficient and transparent systematic reviews
van de Schoot, R., Bruin, J. D., Schram, R. D., Zahedi, P., de Boer, J., Weijdema, F., Kramer, B. M. R., Huijts, M., Hoogerwerf, M., Ferdinands, G., et al. (2021) · Utrecht University Repository (Utrecht University) · Cited by 106
Rapid reviews may produce different results to systematic reviews: a meta-epidemiological study
Marshall, I. J., Marshall, R., Wallace, B. C., Brassey, J., & Thomas, J. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.12.015 · Cited by 95
“Systematic reviews registered with PROSPERO take, on average, 67 weeks from protocol registration to publication [1].”
Hennessy, E. A., Johnson, B. T., & Keenan, C. (2019) · Applied Psychology: Health and Well-Being · doi:10.1111/aphw.12169 · Cited by 91
Search strategy formulation for systematic reviews: Issues, challenges and opportunities
MacFarlane, A., Russell‐Rose, T., & Shokraneh, F. (2022) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2022.200091 · Cited by 209
“This entails multiple iterations and can add further costs to the development of a systematic review (Hemmingway and Brereton, 2009; Borah et al., 2017), and it is often hard for the searcher to know when to stop (Booth, 2010) i.”
BackgroundAutomating Biomedical Evidence Synthesis: RobotReviewer
Marshall, I., Kuiper, J., Banner, E., & Wallace, B. C. (2017) · Proceedings of ACL 2017, System Demonstrations · doi:10.18653/v1/p17-4002 · Cited by 96
“One estimate suggests that a single SR requires thousands of person hours (Allen and Olkin, 1999); and a recent analysis suggests it takes an average of nearly 70 weeks to publish a review (Borah et al., 2017).”
BackgroundLund, H., Juhl, C. B., Nørgaard, B., Draborg, E., Henriksen, M., Andreasen, J., Christensen, R., Nasser, M., Ciliska, D. K., Clarke, M. J., et al. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.07.019 · Cited by 108
“Conducting a systematic review from scratch demands specialized knowledge, skills, and a considerable amount of time [38,39] as well as additional expertise from subject matter experts such as specialized librarians and statisticians.”
Ferdinands, G., Schram, R., de Bruin, J., Bagheri, A., Oberski, D. L., Tummers, L., Teijema, J. J., & van de Schoot, R. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02257-7 · Cited by 96
“However, conducting a systematic review is a costly endeavor that often requires over a year of work by a team of researchers [4], and includes the manual screening of thousands of titles and abstracts.”
BackgroundWhaley, P., Aiassa, E., Beausoleil, C., Beronius, A., Bilotta, G., Boobis, A., de Vries, R., Hanberg, A., Hoffmann, S., Hunt, N., et al. (2020) · Environment International · doi:10.1016/j.envint.2020.105926 · Cited by 91
“SRs are complex, multi-disciplinary projects that typically take 12–36 months to conduct (Borah et al., 2017; Haddaway and Westgate, 2019).”
BackgroundEvaluation of the JBI scoping reviews methodology by current users
Khalil, H., Bennett, M., Godfrey, C., McInerney, P., Munn, Z., & Peters, M. (2020) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000202 · Cited by 89
Handling effect size dependency in meta-analysis
Gucciardi, D. F., Lines, R. L. J., & Ntoumanis, N. (2021) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2021.1946835 · Cited by 99
“You’ve likely spent the best part of several months screening articles to identify primary studies using your inclusion and exclusion criteria, and extracting all relevant information (and/or requesting that data from authors) to compute individual effects for your metaanalysis (Borah et al., 2017).”
Scott, A. M., Forbes, C., Clark, J., Carter, M., Glasziou, P., & Munn, Z. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.06.030 · Cited by 81
Author guidelines for conducting systematic reviews and meta‐analyses
Prill, R., Karlsson, J., Ayeni, O. R., & Becker, R. (2021) · Knee Surgery Sports Traumatology Arthroscopy · doi:10.1007/s00167-021-06631-7 · Cited by 115
Cierco Jimenez, R., Lee, T., Rosillo, N., Cordova, R., Cree, I. A., Gonzalez, A., & Indave Ruiz, B. I. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01805-4 · Cited by 85
“However, conducting a SR is still a complex, challenging and time-consuming process [20, 21], and it requires a multi-disciplinary team with at least one experienced reviewer [22].”
BackgroundOfori-Boateng, R., Aceves-Martins, M., Wiratunga, N., & Moreno-Garcia, C. F. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-024-10844-w · Cited by 86
Calculating statistical power for meta-analysis using metapower
Griffin, J. W. (2021) · The Quantitative Methods for Psychology · doi:10.20982/tqmp.17.1.p024 · Cited by 84
Performance of a Large Language Model in Screening Citations
Oami, T., Okada, Y., & Nakada, T.-A. (2024) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.20496 · Cited by 84
Zhang, G., Jin, Q., Zhou, Y., Wang, S., Idnay, B., Luo, Y., Park, E., Nestor, J. G., Spotnitz, M. E., Soroush, A., et al. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01239-w · Cited by 78
SYMBALS: A Systematic Review Methodology Blending Active Learning and Snowballing
van Haastrecht, M., Sarhan, I., Yigit Ozkan, B., Brinkhuis, M., & Spruit, M. (2021) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2021.685591 · Cited by 80
“They are particularly protracted processes (Borah et al., 2017; O ’ Connor et al., 2019), that often require an impractical level of expertise to execute (Zhang and Ali Babar, 2013).”
BackgroundGates, A., Gates, M., Sebastianski, M., Guitard, S., Elliott, S. A., & Hartling, L. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01031-w · Cited by 80
Screening articles for systematic reviews with ChatGPT
Syriani, E., David, I., & Kumar, G. (2024) · Journal of Computer Languages · doi:10.1016/j.cola.2024.101287 · Cited by 82
Potential Roles of Large Language Models in the Production of Systematic Reviews and Meta-Analyses
Luo, X., Chen, F., Zhu, D., Wang, L., Wang, Z., Liu, H., Lyu, M., Wang, Y., Wang, Q., & Chen, Y. (2024) · Journal of Medical Internet Research · doi:10.2196/56780 · Cited by 109
“The average estimated time to complete and publish a systematic review is 67.3 weeks, requiring 5 researchers and costing approximately US $140,000 [18,19].”
InfluentialEvaluation of a new method for librarian‐mediated literature searches for systematic reviews
Bramer, W. M., Rethlefsen, M. L., Mast, F., & Kleijnen, J. (2018) · Research Synthesis Methods · doi:10.1002/jrsm.1279 · Cited by 71
“We further validated our comparison data (DAH) by briefly comparing the outcomes of the DAH and ESM data with another dataset, which was obtained from research by Borah et al.16 They gathered data from finished and published reviews registered in PROSPERO.”
MethodInfluentialReason, T., Benbow, E., Langham, J., Gimblett, A., Klijn, S. L., & Malcolm, B. (2024) · PharmacoEconomics - Open · doi:10.1007/s41669-024-00476-9 · Cited by 71
“In addition, the HTA process is time-consuming, generally taking over a year to complete, with the SLR and NMA part of this process taking several months each [8, 11].”
BackgroundJardim, P. S. J., Rose, C. J., Ames, H. M., Echavez, J. F. M., Van de Velde, S., & Muller, A. E. (2022) · BMC Medical Research Methodology · doi:10.1186/s12874-022-01649-y · Cited by 68
“It has been estimated that the average health or medical-related systematic review takes more than thirteen months to complete [1].”
BackgroundA new ecosystem for evidence synthesis
Nakagawa, S., Dunn, A. G., Lagisz, M., Bannach‐Brown, A., Grames, E. M., Sánchez‐Tójar, A., O’Dea, R. E., Noble, D. W. A., Westgate, M. J., Arnold, P. A., et al. (2020) · Nature Ecology & Evolution · doi:10.1038/s41559-020-1153-2 · Cited by 69
Li, Y., Datta, S., Rastegar-Mojarad, M., Lee, K., Paek, H., Glasgow, J., Liston, C., He, L., Wang, X., & Xu, Y. (2025) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaf030 · Cited by 69
Accelerating clinical evidence synthesis with large language models
Wang, Z., Cao, L., Danek, B., Jin, Q., Lu, Z., & Sun, J. (2025) · npj Digital Medicine · doi:10.1038/s41746-025-01840-7 · Cited by 69
“The results show that TrialMind signi fi cantlyreducedthetimerequiredforstudysearch,citationscreening,and data extraction, while maintaining or improving the quality of the output compared to experts working alone.”
Cowie, K., Rahmatullah, A., Hardy, N., Holub, K., & Kallmes, K. (2022) · JMIR Medical Informatics · doi:10.2196/33219 · Cited by 68
Williamson, P. R., Oliveira, R. D. Á., Clarke, M. J., Gorst, S. L., Hughes, K., Kirkham, J. J., Li, T., Saldanha, I. J., & Schmitt, J. M. (2020) · BMJ Open · doi:10.1136/bmjopen-2019-036562 · Cited by 78
Text mining to support abstract screening for knowledge syntheses: a semi-automated workflow
Pham, B., Jovanovic, J., Bagheri, E., Antony, J., Ashoor, H., Nguyen, T. T., Rios, P., Robson, R., Thomas, S. M., Watt, J., et al. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01700-x · Cited by 63
“However, SRs in healthcare can be time-consuming (e.g., 1 year), [2] labor-intensive (e.g., 1,139 person-hours, 5 reviewers) [3], and expensive (e.g., > $100,000) [4].”
Noel-Storr, A., Dooley, G., Affengruber, L., & Gartlehner, G. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.09.024 · Cited by 61
Study-based registers reduce waste in systematic reviewing: discussion and case report
Shokraneh, F., & Adams, C. E. (2019) · Systematic Reviews · doi:10.1186/s13643-019-1035-3 · Cited by 65
“For systematic reviews, the median time from search to publication has improved from 14months in 2008 [22] to 8 in 2013 [23] (mean time to complete was 17months [24]; median time between first search and appearance of the review in PubMed was nearly 2 years [25]).”
BackgroundApplication ChatGPT in conducting systematic reviews and meta-analyses
Mahuli, S. A., Rai, A., Mahuli, A. V., & Kumar, A. (2023) · British Dental Journal · doi:10.1038/s41415-023-6132-y · Cited by 61
ChatGPT-4o can serve as the second rater for data extraction in systematic reviews
Motzfeldt Jensen, M., Brix Danielsen, M., Riis, J., Assifuah Kristjansen, K., Andersen, S., Okubo, Y., & Jørgensen, M. G. (2025) · PLOS ONE · doi:10.1371/journal.pone.0313401 · Cited by 60
Menon, J. M. L., Nolten, C., Achterberg, E. J. M., Joosten, R. N. J. M. A., Dematteis, M., Feenstra, M. G. P., Drinkenburg, W. H. (., & Leenaars, C. H. C. (2019) · Journal of Circadian Rhythms · doi:10.5334/jcr.174 · Cited by 57
Lund, H., Juhl, C. B., Nørgaard, B., Draborg, E., Henriksen, M., Andreasen, J., Christensen, R., Nasser, M., Ciliska, D. K., Tugwell, P., et al. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.07.021 · Cited by 67
“Preparing a systematic review from scratch is time intensive and effort intensive [16,17].”
BackgroundUMLS-based data augmentation for natural language processing of clinical research literature
Kang, T., Perotte, A., Tang, Y., Ta, C., & Weng, C. (2021) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocaa309 · Cited by 55
Page, M. J., Moher, D., Fidler, F. M., Higgins, J. P. T., Brennan, S., Haddaway, N., Hamilton, D. G., Kanukula, R., Karunananthan, S., Maxwell, L. J., et al. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01670-0 · Cited by 61
“[10–12]), a full systematic review can take anywhere between 6 months and 3 years to complete, depending on the volume of literature to screen and synthesise [65, 66].”
BackgroundThe influence of the team in conducting a systematic review
Uttley, L., & Montgomery, P. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0548-x · Cited by 54
“[15] indicates that funded reviews took significantly longer to complete and”
BackgroundLopinavir/ritonavir for COVID-19: A living systematic review
Verdugo‐Paiva, F., Izcovich, A., Ragusa, M. A., & Gabriel Rada, G. (2020) · Medwave · doi:10.5867/medwave.2020.06.7966 · Cited by 54
Clark, J., McFarlane, C., Cleo, G., Ishikawa Ramos, C., & Marshall, S. (2021) · JMIR Medical Education · doi:10.2196/24418 · Cited by 54
“The time from SR registration to publication has been reported as taking five authors approximately 67 weeks [5], with time frames ranging from 6 months to 2 years [6].”
MethodAffengruber, L., van der Maten, M. M., Spiero, I., Nussbaumer-Streit, B., Mahmić-Kaknjo, M., Ellen, M. E., Goossen, K., Kantorova, L., Hooft, L., Riva, N., et al. (2024) · BMC Medical Research Methodology · doi:10.1186/s12874-024-02320-4 · Cited by 52
Active learning for screening prioritization in systematic reviews - A simulation study
Ferdinands, G., Schram, R., de Bruin, J., Bagheri, A., Oberski, D. L., Tummers, L., & van de Schoot, R. (2020) · doi:10.31219/osf.io/w6qbg · Cited by 51
“Typically, only a fraction of the publications belong to the relevant class (2.94%, [4]).”
InfluentialDevelopment of Prompt Templates for Large Language Model–Driven Screening in Systematic Reviews
Cao, C., Sang, J., Arora, R., Chen, D., Kloosterman, R., Cecere, M., Gorla, J., Saleh, R., Drennan, I., Teja, B., et al. (2025) · Annals of Internal Medicine · doi:10.7326/annals-24-02189 · Cited by 91
“However, SRs are resource intensive, typically requiring 1 year and upwards of $100000 to complete due to the comprehensive processes of conducting detailed searches, screening articles, extracting data, analyzing fi ndings, and report writing (2 – 4).”
Software support for environmental evidence synthesis
Westgate, M. J., Haddaway, N. R., Cheng, S. H., McIntosh, E. J., Marshall, C., & Lindenmayer, D. B. (2018) · Nature Ecology & Evolution · doi:10.1038/s41559-018-0502-x · Cited by 51
Rada, G., Verdugo‐Paiva, F., Ávila, C., Morel-Marambio, M., Bravo-Jeria, R., Pesce, F., Madrid, E., & Izcovich, A. (2020) · Medwave · doi:10.5867/medwave.2020.03.7867 · Cited by 55
Russell-Rose, T., & Chamberlain, J. (2017) · JMIR Medical Informatics · doi:10.2196/medinform.7680 · Cited by 49
“It is clear that respondents commonly work across multiple platforms, in particular MEDLINE, Cochrane, and Embase, and this is in line with findings from previous research [41].”
BackgroundResultNatural language processing to facilitate breast cancer research and management
Hughes, K. S., Zhou, J., Bao, Y., Singh, P., Wang, J., & Yin, K. (2019) · The Breast Journal · doi:10.1111/tbj.13718 · Cited by 53
srBERT: automatic article classification model for systematic review using BERT
Aum, S., & Choe, S. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01763-w · Cited by 48
Odintsova, V. V., Roetman, P. J., Ip, H. F., Pool, R., Van der Laan, C. M., Tona, K.-D., Vermeiren, R. R. J. M., & Boomsma, D. I. (2019) · Psychiatric Genetics · doi:10.1097/ypg.0000000000000239 · Cited by 46
Use of a search summary table to improve systematic review search methods, results, and efficiency
Bethel, A. C., Rogers, M., & Abbott, R. A. (2021) · Journal of the Medical Library Association JMLA · doi:10.5195/jmla.2021.809 · Cited by 58
“The timescales to publication can vary widely [2], and some systematic reviews are regularly updated, particularly if there is new relevant evidence in the field being researched [3].”
Questions and answers about conducting systematic reviews in sport and exercise psychology
Gunnell, K., Poitras, V. J., & Tod, D. (2020) · International Review of Sport and Exercise Psychology · doi:10.1080/1750984x.2019.1695141 · Cited by 45
Methods for using Bing’s AI-powered search engine for data extraction for a systematic review
Hill, J. E., Harris, C., & Clegg, A. (2024) · Research Synthesis Methods · doi:10.1002/jrsm.1689 · Cited by 45
Nama, N., Hennawy, M., Barrowman, N., O’Hearn, K., Sampson, M., & McNally, J. D. (2021) · Systematic Reviews · doi:10.1186/s13643-021-01632-6 · Cited by 53
Text Message Interventions in Adolescent Mental Health and Addiction Services: Scoping Review
MacDougall, S., Jerrott, S., Clark, S. E., Campbell, L. A., Murphy, A. L., & Wozney, L. (2021) · JMIR Mental Health · doi:10.2196/16508 · Cited by 49
Verdugo‐Paiva, F., Vergara, C., Ávila, C., Castro-Guevara, J. A., Cid, J., Contreras, V., Jara, I., Jiménez, V., Lee, M. H., Muñoz, M., et al. (2022) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2022.05.001 · Cited by 41
“Unfortunately, substantial time and resources are needed to comprehensively identify all the evidence, as required by rigorous systematic review methods [21,22].”
Hamel, C., Michaud, A., Thuku, M., Affengruber, L., Skidmore, B., Nussbaumer-Streit, B., Stevens, A., & Garritty, C. (2020) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.06.027 · Cited by 36
Neil‐Sztramko, S., Belita, E., Traynor, R. L., Clark, E., Hagerman, L., & Dobbins, M. (2021) · BMC Medical Research Methodology · doi:10.1186/s12874-021-01436-1 · Cited by 45
Improving the conduct of systematic reviews: a process mining perspective
Pham, B., Bagheri, E., Rios, P., Pourmasoumi, A., Robson, R. C., Hwee, J., Isaranuwatchai, W., Darvesh, N., Page, M. J., & Tricco, A. C. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.06.011 · Cited by 40
Metadata
- Authors
- Borah, R., Brown, A. W., Capers, P. L., & Kaiser, K. A.
- Affiliations
- University of Alabama at Birmingham (US)
- Corresponding author
- Kaiser, Kathryn A
- Venue
- BMJ Open
- Publisher
- BMJ
- Published
- 2017-02
- Volume
- 7
- Issue
- 2
- Pages
- e012545
- ISSN
- 2044-6055 (ISSN-L)
- PMC
- PMC5337708
- OpenAlex
- W2593758073
- Cited by
- 737
- References
- 29
- Citations by year
- 2026: 114 · 2025: 129 · 2024: 105 · 2023: 76 · 2022: 71 · 2021: 80 · 2020: 87 · 2019: 42 · 2018: 24 · 2017: 9
- FWCI
- 10.85
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc
- Locations
- BMJ Open (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- IUScholarWorks Open (Indiana University) (submittedVersion, cc-by-nc)
- HighWire Press Open Archive (submittedVersion)
- Funders
- University of Alabama at Birmingham (P30DK056336)
- University of Alabama (P30DK056336)
- National Institutes of Health
- National Institute of General Medical Sciences (K12GM088010)
- National Institute of Diabetes and Digestive and Kidney Diseases (P30DK056336)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- systematic reviews, time to publication, research workforce, medical interventions, research resource allocation
- MeSH terms
- Workforce, Humans, Publishing, Registries, Review Literature as Topic, Task Performance and Analysis, Databases, Bibliographic, Information Storage and Retrieval, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.