Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry
Open accessThe DOI matches its Crossref record; 26 of the 29 references it cites were found in a registry or an index; 2 of the 3 sources its claims cite were checked for support.
Abstract
OBJECTIVES: To summarise logistical aspects of recently completed systematic reviews that were registered in the International Prospective Register of Systematic Reviews (PROSPERO) registry to quantify the time and resources required to complete such projects. DESIGN: Meta-analysis. DATA SOURCES AND STUDY SELECTION: All of the 195 registered and completed reviews (status from the PROSPERO registry) with associated publications at the time of our search (1 July 2014). DATA EXTRACTION: All authors extracted data using registry entries and publication information related to the data sources used, the number of initially retrieved citations, the final number of included studies, the time between registration date to publication date and number of authors involved for completion of each publication. Information related to funding and geographical location was also recorded when reported. RESULTS: The mean estimated time to complete the project and publish the review was 67.3 weeks (IQR=42). The number of studies found in the literature searches ranged from 27 to 92 020; the mean yield rate of included studies was 2.94% (IQR=2.5); and the mean number of authors per review was 5, SD=3. Funded reviews took significantly longer to complete and publish (mean=42 vs 26 weeks) and involved more authors and team members (mean=6.8 vs 4.8 people) than those that did not report funding (both p<0.001). CONCLUSIONS: Systematic reviews presently take much time and require large amounts of human resources. In the light of the ever-increasing volume of published studies, application of existing computing and informatics technology should be applied to decrease this time and resource burden. We discuss recently published guidelines that provide a framework to make finding and accessing relevant literature less burdensome.
Overview
CoChat Summary
## TL;DR
Using 195 completed PROSPERO-registered systematic reviews of medical interventions, the authors estimated that reviews took a mean of 67.3 weeks to complete and publish and involved a mean of 5 authors per review. They conclude that systematic reviews are time- and labor-intensive, with funded reviews taking longer and involving more people than unfunded reviews.
## Key findings
- Across 195 completed reviews, the mean estimated time from registered project start to publication was 67.3 weeks (SD=31.0; range 6–186 weeks).
- The mean number of authors/team members was 5 (SD=3), and the mean merged yield rate was 2.94% (SD=6.49; range 0–64.71%).
- Literature searches were highly variable, with the number of studies found ranging from 27 to 92,020.
- Reviews that reported funding took longer and involved more people than those that did not report funding: 42 vs 26 weeks and 6.8 vs 4.8 people, both p<0.001.
## Methods
- Meta-analysis of PROSPERO registry records linked to completed review publications, using a sample of 195 registered and completed reviews with associated publications identified on 1 July 2014.
- Data were extracted from registry entries and publications on timing, authors/team members, funding, search sources, initial citations, and final included studies; the sample comprised 197 literature reviews from the 195 publications.
- Main analyses were descriptive summaries of means, ranges, and yield rates, plus ANOVA to compare funded vs unfunded reviews.
## Limitations
- The paper states that the study is limited by incomplete reporting in both the registry and the published articles, and that registry and publication data were cross-checked only where available.
- The authors state that the true project start date was difficult to capture, so their time estimates may be conservative and may underestimate time from the very first review activities.
- The paper notes that the data are highly skewed, making prediction of timelines statistically unsound; this is an evident limitation of the analysis.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures

Verification
Claims the paper makes
1 Valid · 1 Not valid · 1 Not checked · 23 Own results and interpretations
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the mean number of authors per review was 5, SD=3.”
§ Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The mean estimated time to complete the project and publish the review was 67.3 weeks (IQR=42).”
- Also stated in: § Results · plain
“The mean project length (using the registered project start date to the review's publication date) was 67.3 weeks (SD=31.0; range 6–186 weeks).”
§ Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Also stated in: § Results · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the mean yield rate of included studies was 2.94% (IQR=2.5);”
- Also stated in: § Results · plain
“The mean merged yield rate was 2.94% (SD=6.49; range 0–64.71%).”
§ Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry > Abstract > Results
plainn = 190- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Also stated in: § Results · plain
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Reviews that reported funding took longer to complete and included more authors/team members; this difference was not seen when we considered only whether specific authors were funded.”
§ Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The number of studies found in the literature searches ranged from 27 to 92 020;”
§ Analysis of the time and workers needed to conduct systematic reviews of medical interventions using data from the PROSPERO registry > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
9 more claims
Create a free account to see all 26 claims and every source verdict.
- Probably not a substantial claim. 11
- It may state two separate claims as one. 6
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Typically, only a small fraction of the records belong to the relevant class, making title and abstract screening an important bottleneck in systematic reviewing process 25 .”
From An open source machine learning framework for efficient and transparent systematic reviews (2021) · cites this paper as [25]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“requiring a median of five researchers and around 40 weeks of work to reach submission [ 3 , 4 ]”
From Reducing systematic review burden using Deduklick: a novel, automated, reliable, and explainable deduplication algorithm to foster medical research (2022) · cites this paper as [4]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“typically taking over 16 months and costing upwards of $100,000 to complete 1,2 .”
From Automation of Systematic Reviews with Large Language Models (2025) · cites this paper as [2]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“LLMs can automate critical tasks for SLRs, including screening abstracts and full texts, assessing risk of bias and extracting data for use in network meta-analyses with high accuracy, all while significantly reducing the time and costs required by using human review [2, 4–7].”
From Automated Mass Extraction of Over 680,000 PICOs from Clinical Study Abstracts Using Generative AI: A Proof-of-Concept Study. (2024) · cites this paper as [4]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“The HTA process is very labour and resource intensive, requiring a large team of experts in the field of health economic outcomes research (HEOR), including SLR analysts, statisticians and economic modellers [5, 8].”
From Artificial Intelligence to Automate Network Meta-Analyses: Four Case Studies to Evaluate the Potential Application of Large Language Models (2024) · cites this paper as [8]
Experimental — accuracy not yet measured
18 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · cited 3×
“…clinical trials could be a starting point for testing the use of indexing systems that employ a Population, Intervention, Comparisons, Outcomes, Study Design (PICOS) 15 structure of coding metadata for clinical trials in order to make the scienti fi c literature become a truly searchable database.”
BackgroundInfluentialThe FAIR Guiding Principles for scientific data management and stewardship
Wilkinson, M. D., Dumontier, M., Aalbersberg, I. J., Appleton, G., Axton, M., Baak, A., Blomberg, N., Boiten, J.-W., da Silva Santos, L. B., Bourne, P. E., et al. (2016) · Scientific Data · doi:10.1038/sdata.2016.18 · Cited by 18,423
Shea, B. J., Hamel, C., Wells, G. A., Bouter, L. M., Kristjansson, E., Grimshaw, J., Henry, D. A., & Boers, M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.10.009 · Cited by 1,755
The nuts and bolts of PROSPERO: an international prospective register of systematic reviews
Booth, A., Clarke, M., Dooley, G., Ghersi, D., Moher, D., Petticrew, M., & Stewart, L. (2012) · Systematic Reviews · doi:10.1186/2046-4053-1-2 · Cited by 1,433
Seventy-five trials and eleven systematic reviews a day: how will we ever keep up?
Bastian, H., Glasziou, P., & Chalmers, I. (2010) · PLoS Medicine · doi:10.1371/journal.pmed.1000326 · Cited by 1,188 · cited 1×
“1 Since the body of primary literature has increased, conducting systematic reviews and meta-analyses has become a necessity to more accurately and comprehensively present accumulated knowledge to scienti fi c, clinical and general audiences.”
BackgroundPROSPERO: an international prospective register of systematic reviews
Wright, K. E. (2014) · Cited by 568
(1965) · Nature · doi:10.1038/205236a0 · Cited by 560
Study flow diagrams in Cochrane systematic review updates: an adapted PRISMA flow diagram
Stovold, E., Beecher, D., Foxlee, R., & Noel-Storr, A. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-54 · Cited by 356
Cited by
Items 301–400 of 671
The fate of urological systematic reviews registered in PROSPERO
Khaleel, S. S., Cleveland, B. E., Kalapara, A. A., Sathianathen, N. J., Balaji, P., & Dahm, P. (2019) · World Journal of Urology · doi:10.1007/s00345-019-03032-x · Cited by 7
Research Support Services in STEM Libraries: A Scoping Review
Tchangalova, N., Coalter, J., Trost, A., & Pierdinock-Weed, A. (2021) · Issues in Science and Technology Librarianship · doi:10.29173/istl2574 · Cited by 11
Xia, Y., Cai, J., Li, Y., Dou, Z., Zhang, Y., Wu, L., Huang, Z., Xu, S., Sun, J., Liu, Y., et al. (2021) · International Journal of Intelligent Systems · doi:10.1002/int.22748 · Cited by 9
Escaldelai, F. M. D., Escaldelai, L., & Bergamaschi, D. P. (2022) · Revista Brasileira de Epidemiologia · doi:10.1590/1980-549720220030 · Cited by 8
Picot, C., Ajiji, P., Jurek, L., Nourredine, M., Massardier, J., Peron, A., Cucherat, M., & Cottin, J. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02256-8 · Cited by 9
Rapid review: A review of methods and recommendations based on current evidence
Guo, Q., Jiang, G., Zhao, Q., Long, Y., Feng, K., Gu, X., Xu, Y., Li, Z., Huang, J., & Du, L. (2024) · Journal of Evidence-Based Medicine · doi:10.1111/jebm.12594 · Cited by 8
A keyword-based approach to analyzing scientific research trends: ReRAM present and future
Kim, H. C., Kim, S. H., Kim, J., Kim, E. H., Gu, J. H., & Lee, D. (2025) · Scientific Reports · doi:10.1038/s41598-025-93423-5 · Cited by 14
Bonin, F., Gleize, M., Hou, Y., Ganguly, D., Finnerty, A. N., Jochim, C., Pascale, A., Tommasi, P., Aonghusa, P. M., & Michie, S. (2020) · PubMed Central · Cited by 6
Gates, A., Guitard, S., Pillay, J., Elliott, S. A., Dyson, M. P., Newton, A. S., & Hartling, L. (2019) · doi:10.23970/ahrqepcmethmachineperformance · Cited by 8
Toolkit of methodological resources to conduct systematic reviews
Figuls, M. R. I., García, L. M., Solà, I., Alonso‐Coello, P., Bonfill, X., & Zamora, J. (2020) · F1000Research · doi:10.12688/f1000research.22032.1 · Cited by 9
“This paper corroborates that developing a rigorous SR is a complex and resource-intensive task 77,78 .”
Rapid evidence synthesis to enable innovation and adoption in health and social care
Norman, G., Wilson, P. M., Dumville, J. C., Bower, P., & Cullum, N. A. (2022) · Systematic Reviews · doi:10.1186/s13643-022-02106-z · Cited by 10
“Phagenyx is a device which is designed to reduce neurogenic dysphagia (dysfunction of eating) [1].”
BackgroundTrends of Randomized Clinical Trials Citing Prior Systematic Reviews, 2007-2021
Jia, Y., Li, B., Yang, Z., Li, F.-X., Zhao, Z.-Y., Wei, C., Yang, X., Jin, Q., Liu, D., Wei, X., et al. (2023) · JAMA Network Open · doi:10.1001/jamanetworkopen.2023.4219 · Cited by 14
Systematic Reviews in Occupational Health and Safety: where are we and where should we go?
Verbeek, J. H. A. M., Mattioli, S., & Curti, S. (2019) · La Medicina del Lavoro · doi:10.23749/mdl.v110i5.8952 · Cited by 10
“For some review, one has to spend hours and hours going through the results of database searches to weed out the irrelevant references (1).”
BackgroundMaggio, L. A., Thomas, A., & Durning, S. J. (2018) · Wiley eBooks · doi:10.1002/9781119373780.ch31 · Cited by 9
Krag, C. H., Balschmidt, T., Bruun, F. J., Brejnebøl, M. W., Xu, J. J., Boesen, M., Andersen, M. B., & Müller, F. C. (2024) · medRxiv · doi:10.1101/2024.10.01.24314702 · Cited by 17
“A large-scale systematic review can take more than a year to conduct and cost up to a quarter of a million USD 7,8 .”
Posso, M., & Sala, M. (2024) · Cirugía Española · doi:10.1016/j.ciresp.2024.04.001 · Cited by 7
Gitman, V., Maxwell, C. J., & Gamble, J. (2025) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2025.110464 · Cited by 10
Koh, Z. H., Serbetci, D., Skues, J., & Murray, G. (2025) · JMIR Mental Health · doi:10.2196/59351 · Cited by 6
“Considering that a typical review takes, on average, 5 members and approximately 67.3 weeks to conduct and publish [244,245], full-text screening of 8460 records was not feasible with 2 reviewers and limited resources.”
InfluentialCattaneo, A., Vitali, A., Regazzoni, D., & Rizzi, C. (2024) · Studies in health technology and informatics · doi:10.3233/shti240014 · Cited by 11
Gooding, B., Probst, Y., Houston, L., & Neale, E. P. (2019) · Nutrition & Dietetics · doi:10.1111/1747-0080.12598 · Cited by 7
Sartas, M., Cummings, S., Garbero, A., & Акрамханов, А. (2021) · Forests · doi:10.3390/f12081027 · Cited by 8
“A systematic review was estimated to take more than 12 months [19,20] in health research.”
BackgroundAre COVID-19 systematic reviews up to date and can we tell? A cross-sectional study
McDonald, S. J., Turner, S., Nguyen, P., Page, M. J., & Turner, T. (2023) · Systematic Reviews · doi:10.1186/s13643-023-02253-x · Cited by 9
“Reviews are notoriously time-consuming—an analysis of 195 reviews from 2017 estimated the mean time to complete and publish a review was 67 weeks [12].”
BackgroundWelti, R., Jones, B., Moynihan, P. J., & Silva, M. (2023) · Australian Dental Journal · doi:10.1111/adj.12972 · Cited by 8
Scott, A. M., Forbes, C., Clark, J. M., Carter, M., Glasziou, P., & Munn, Z. (2021) · medRxiv · doi:10.1101/2021.04.26.21255833 · Cited by 7
“However, conducting them has historically been both resource- and time-intensive, with systematic reviews on average, requiring approximately 67 weeks to complete,(1) guidelines requiring 18-30 months,(2) and health technology assessments between 6 and 24 months(3).”
BackgroundWang, X., & Luo, G. (2024) · EdArXiv (OSF Preprints) · doi:10.35542/osf.io/wn3cd · Cited by 12
Semi-automated title-abstract screening using natural language processing and machine learning
Pilz, M., Zimmermann, S., Friedrichs, J., Wördehoff, E., Ronellenfitsch, U., Kieser, M., & Vey, J. A. (2024) · Systematic Reviews · doi:10.1186/s13643-024-02688-w · Cited by 13
The Efficacy of Bereavement Interventions: A Systematic Umbrella Review
Avis, K. A., Missler, M. A., van Deursen, D. S., Lenferink, L. I. M., Stroebe, M., & Schut, H. A. W. (2025) · Harvard Review of Psychiatry · doi:10.1097/hrp.0000000000000426 · Cited by 8
Daraio, C. C., Leo, S. D., & Ferazzoli, F. (2025) · International Transactions in Operational Research · doi:10.1111/itor.70054 · Cited by 11
Cheng, Y., & Zhang, H. J. (2025) · ACS ES&T Water · doi:10.1021/acsestwater.5c00551 · Cited by 9
Xu, S., Zhao, Z., Liu, X., & Meng, X.-L. (2025) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-025-03138-w · Cited by 8
Pan, C., Lu, W., Chen, B., Zhang, G., Yang, Z., & Hao, J. (2025) · JMIR Medical Informatics · doi:10.2196/76252 · Cited by 12
“However, meta-analysis and systematic reviews are time-consuming processes, especially the process of collecting and screening literature records [14].”
The hunt for the last relevant paper: blending the best of humans and AI
van de Schoot, R., Messina Coimbra, B., Evenhuis, T., Lombaers, P., Weijdema, F., de Bruin, L., Neeleman, R., Grandfield, E., Sijbrandij, M., Teijema, J. J., et al. (2025) · European Journal of Psychotraumatology · doi:10.1080/20008066.2025.2546214 · Cited by 6
“Systematic literature screening can involve going through tens of thousands of titles, abstracts, and full-text publications, making the process both time-consuming and error-prone (Borah et al., 2017).”
Artificial Intelligence Tools for Automating Evidence Synthesis: Scoping Review
Harasgama, S., Pearce, H., Appel, C., Loftus, L., Painter, H., Kuhn, I. L., Karpusheff, J., Ceesay, A., & Ford, J. A. (2026) · Journal of Medical Internet Research · doi:10.2196/81597 · Cited by 13
“Previous studies have estimated that an average systematic review can take approximately 67 weeks to complete [3], often too long for decision makers.”
Sciurti, A., Migliara, G., Siena, L. M., Isonne, C., Blasiis, M. R. D., Sinopoli, A., Iera, J., Marzuillo, C., Vito, C. D., Villari, P., et al. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2025.10044 · Cited by 14
“2,26 Finally, workload saving is nonstandard metric to evaluate workload reduction by automated screening tools, expressing records correctly excluded by the model, out of the total number of screened records.”
Cassell, K., Ologunowa, A., Rastegar-Mojarad, M., Chun, B., Huang, Y.-L., Wang, D., & Cossrow, N. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1662202 · Cited by 15
Accuracy of LLMs to retrieve numeric data for meta-analysis in dentistry
Caponio, V. C. A., Lorenzo‐Pouso, A. I., Magalhães, M. A. O., Ali, A. A., Adamo, D., Cirillo, N., López‐Pintor, R. M., & Musella, G. (2025) · Journal of Dentistry · doi:10.1016/j.jdent.2025.106245 · Cited by 13
Neurosynth Compose: A web-based platform for flexible and reproducible neuroimaging meta-analysis
Kent, J. D., Lee, N., Laird, A. R., Salo, T., Peraza, J., Bottenhorn, K. L., Oudyk, K., Nichols, T. E., Poline, J., & de la Vega, A. (2026) · Imaging Neuroscience · doi:10.1162/imag.a.1114 · Cited by 14
“This process is extraordinarily time consuming and error prone ( Wang et al., 2020): a single review can demand hundreds of researcher hours ( Borah et al., 2017).”
Dwivedi, A. K. (2026) · PeerJ · doi:10.7717/peerj.20897 · Cited by 6
“The median number of authors included in a meta-analysis study was estimated to be 5 ( Borah et al., 2017 ).”
InfluentialGong, E. J., Bang, C. S., & Shin, Y. S. (2026) · Bioengineering · doi:10.3390/bioengineering13030365 · Cited by 19
“Reviews require an average of 67.3 weeks from protocol registration to publication, with research teams investing hundreds of person-hours in screening thousands of abstracts, extracting data, and synthesizing findings [8].”
InfluentialAccelerating the Early Identification of Relevant Studies in Title and Abstract Screening
Bravo, Á., Bennetts, L. E., & Atanasov, P. (2021) · International Symposium on Computer Science and Intelligent Control (ISCSIC) · doi:10.1109/iscsic54682.2021.00034 · Cited by 6
A guide to using technological applications to facilitate systematic reviews
Gibbs, K. D., Loveless, J., & Crane, S. M. (2022) · Worldviews on Evidence-Based Nursing · doi:10.1111/wvn.12611 · Cited by 14
Byrne, F., Hofstee, L., Teijema, J., De Bruin, J., & van de Schoot, R. (2024) · Systematic Reviews · doi:10.1186/s13643-024-02587-0 · Cited by 9
“Historically, this issue has been noted by studies such as Bastian et al. [3], and Borah et al. [5], highlighting its long-standing nature.”
BackgroundWhitehorn, A., Lockwood, C. S., Hu, Y., Xing, W., Zhu, Z., & Porritt, K. A. (2024) · JBI Evidence Synthesis · doi:10.11124/jbies-23-00557 · Cited by 9
Mapping the Degenerative Cervical Myelopathy Research Landscape: Topic Modeling of the Literature
Karabacak, M., Jagtiani, P., Zipser, C. M., Tetreault, L. A., Davies, B. M., & Margetis, K. D. (2024) · Global Spine Journal · doi:10.1177/21925682241256949 · Cited by 6
“19,20 The slow pace of traditional reviews may impede the application of new research fi ndings and the discovery of overarching patterns within the literature.”
Enhancing PIO Element Detection in Medical Text Using Contextualized Embedding
Mezaoui, H., Gunasekara, I., & Gontcharov, A. (2019) · BioNLP Workshop and Shared Task (BioNLP) · doi:10.18653/v1/w19-5023 · Cited by 12
“The volume of these publications makes it very challenging for physicians to find the best medical intervention for a given patient group and condition (Borah et al., 2017).”
BackgroundGarritty, C., Hersi, M., Hamel, C., Stevens, A., Monfaredi, Z., Butler, C., Tricco, A. C., Hartling, L., Stewart, L. A., Welch, V. A., et al. (2020) · PLoS ONE · doi:10.1371/journal.pone.0238025 · Cited by 7
“A significant barrier is that SRs can be difficult and time-consuming to conduct, usually taking 1 to 2 years to complete [5, 6].”
Metadata: The accelerant we need
Kaiser, K. A., Chodacki, J., Habermann, T., et al. (2020) · Information Services & Use · doi:10.3233/isu-200094 · Cited by 6
“One study reported an average of over five researchers and more than sixty-six weeks from the time a systematic review was planned and registered to when it was published [6].”
Self-citationRapid reviews: A critical perspective
Nordhausen, T., & Hirt, J. (2020) · Zeitschrift für Evidenz Fortbildung und Qualität im Gesundheitswesen · doi:10.1016/j.zefq.2020.09.005 · Cited by 6
Ensuring the rigor in systematic reviews: Part 2, preparation is key: The question and the protocol
Brackett, A. L., & Batten, J. (2020) · Heart & Lung · doi:10.1016/j.hrtlng.2020.07.001 · Cited by 6
Sharp, M. K., Tyner, B., Baki, D. A. B. A., Farrell, C., Devane, D., Mahtani, K. R., Smith, S. M., O’Neill, M., Ryan, M., & Clyne, B. (2021) · HRB Open Research · doi:10.12688/hrbopenres.13325.1 · Cited by 11
Griebler, U., Dobrescu, A., Ledinger, D., Klingenstein, P., Sommer, I., Emprechtinger, R., Persad, E., Gadinger, A., Trivella, M., Klerings, I., et al. (2023) · Research Synthesis Methods · doi:10.1002/jrsm.1656 · Cited by 5
Intra-Template Entity Compatibility based Slot-Filling for Clinical Trial Information Extraction.
Witte, C., & Cimiano, P. (2022) · Proceedings of the 21st Workshop on Biomedical Language Processing · doi:10.18653/v1/2022.bionlp-1.18 · Cited by 5
“Creating a systematic review involves a high effort, involving on average 67.3 weeks and involving 5 authors per review on average (Borah et al., 2017).”
BackgroundInfluentialSugiura, A., Saegusa, S., Jin, Y., Yoshimoto, R., Smith, N. D., Dohi, K., Higuchi, T., & Kozu, T. (2024) · JMIR Formative Research · doi:10.2196/55827 · Cited by 5
Wiwatthanasetthakarn, P., Ponthongmak, W., Looareesuwan, P., Tansawet, A., Numthavaj, P., McKay, G. J., Attia, J. R., & Thakkinstian, A. (2024) · Journal of Medical Internet Research · doi:10.2196/56863 · Cited by 8
“However, conducting an SR requires significant time and human resources, particularly for the screening and selection of potentially eligible studies, data extraction, and bias assessment [4].”
From data extraction to analysis: a comparative study of ELISE capabilities in scientific literature
Gobin, M., Gosnat, M., Touré, S., Faik, L., Belafa, J., de Torcy, A. V., & Armstrong, F. (2025) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2025.1587244 · Cited by 6
Rose, C. J., Meneses-Echavez, J. F., Muller, A. E., Berg, R. C., Borge, T. C., Jardim, P. S. J., & Cooper, C. (2025) · Cochrane Evidence Synthesis and Methods · doi:10.1002/cesm.70030 · Cited by 6
Christie, A. P., Martin, P. A., & Taylor, N. G. (2025) · PLoS ONE · doi:10.1371/journal.pone.0326521 · Cited by 6
“…resampling, a data balancing strategy [60], to decrease the imbalance in the data used to train the classifier, as we expect to find a very high ratio of irrelevant documents to relevant ones (typically 97:3 for medical and environmental topics [61,62] – i.e., relatively few positive data points).”
Spiero, I., Leeuwenberg, A., Moons, K. G. M., Hooft, L., & Damen, J. A. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2025.10025 · Cited by 6
CLEAR: A vision to support clinical evidence lifecycle with continuous learning
Fang, Y., Zhang, G., Chen, F., Hripcsak, G. M., Peng, Y., Ryan, P. B., & Weng, C. (2025) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2025.104884 · Cited by 6
Li, L., Mathrani, A., & Sušnjak, T. (2026) · Research Synthesis Methods · doi:10.1017/rsm.2025.10066 · Cited by 17
“If we follow up the original LLM prompt with a subsequent prompt, asking the LLM to revise and improve upon its initial data extraction, to what degree will the accuracy improve?”
Why can’t epidemiology be automated (yet)?
Bann, D., Lowther, E., Wright, L., & Kovalchuk, Y. (2026) · International Journal of Epidemiology · doi:10.1093/ije/dyaf210 · Cited by 9
“Systematic reviews generally take ≥ 1 year to undertake [12], with much of this time spent on pain-staking and often tedious screening—manually removing the vast majority of irrelevant articles from the selected pool by comparing them against the same inclusion/exclusion criteria.”
Saad, A. M., DULAIMI, M. F., & Gorse, C. A. (2023) · Journal of Architectural Engineering · doi:10.1061/jaeied.aeeng-1547 · Cited by 7
Designing GPT3 prompts to screen articles for systematic reviews of RCTs
Strachan, J. A. (2024) · Health Policy and Technology · doi:10.1016/j.hlpt.2024.100943 · Cited by 5
Ali, F., Tan, A. S.-C., & Wang, S. J.-W. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2024.16 · Cited by 9
“4,5 This challenge is compounded by the rapid recent growth in the scientific literature 6 including in education.”
Development and usage growth of the polyglot search translator
Clark, J. M., Forbes, C., Bateup, S., & Carter, M. (2020) · Journal of Health Information and Libraries Australasia · doi:10.55999/johila.v1i3.42 · Cited by 5
COVID-19 Technology-Enabled Living Systematic Reviews to Enhance Knowledge Translation
Harrington, L. (2021) · AACN Advanced Critical Care · doi:10.4037/aacnacc2021948 · Cited by 5
Sutton, A., O’Keefe, H., Johnson, E. E., & Marshall, C. (2023) · Research Synthesis Methods · doi:10.1002/jrsm.1665 · Cited by 5
Stern, C., Hines, S. J., Leonardi‐Bee, J., Slyer, J. T., Wilson, S., Carrier, J., Wang, N., & Aromataris, E. (2024) · JBI Evidence Synthesis · doi:10.11124/jbies-23-00548 · Cited by 8
Karabacak, M., Jain, A., Jagtiani, P., Hickman, Z. L., Dams-O’Connor, K., & Margetis, K. D. (2024) · Neurotrauma Reports · doi:10.1089/neur.2023.0102 · Cited by 5
“3,4 For instance, Allen and Olkin underscored that even MetaWorks, an efficient firm specializing in meta-analyses, spent * 1139 h completing 37 meta-analyses.”
BackgroundSimplifying synthesis of the expanding glioblastoma literature: a topic modeling approach
Karabacak, M., Jagtiani, P., Carrasquilla, A., Jain, A., Germano, I. M., & Margetis, K. D. (2024) · Journal of Neuro-Oncology · doi:10.1007/s11060-024-04762-8 · Cited by 5
Yu, F., Kincaide, H., & Carlson, R. (2024) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.1039 · Cited by 8
“Future research on ChatGPT can examine potential time savings for other aspects of the review process, as designing and executing the literature search is only the beginning (Borah et al., 2017; Haddaway & Westgate, 2019).”
Aquilina, J., Neves, J. B., & Tran, M. G. B. (2020) · British Journal of Hospital Medicine · doi:10.12968/hmed.2020.0127 · Cited by 6
Introduction to Systematic Reviews
Li, T., Saldanha, I. J., & Robinson, K. A. (2021) · Springer eBooks · doi:10.1007/978-3-319-52677-5_194-1 · Cited by 5
Lubis, M. S. A., Damayanto, A., Bangkara, B. M. A. S. A., Fatoni, M. A., & Suherlan, H. (2022) · AL-TANZIM JURNAL MANAJEMEN PENDIDIKAN ISLAM · doi:10.33650/al-tanzim.v6i2.3366 · Cited by 17
“…problems and hypotheses for data searching and reporting of results, which we adhere to a phenomenological approach, namely an effort to obtain the broadest possible data and examine them into findings that answer the problems of this study with high validity and reliability (Borah et al., 2017).”
BackgroundTesting a filtering strategy for systematic reviews: evaluating work savings and recall.
Proescholdt, R., Hsiao, T., Schneider, J., Cohen, A. M., McDonagh, M. S., & Smalheiser, N. R. (2022) · PubMed · Cited by 8
Systematic review automation methods
Norman, C. R. (2020) · theses.fr (ABES) · doi:10.70675/044ffa34z5c9bz433az99c8z64697dcd28a2 · Cited by 8
Inferring Drug–Gene Relationships in Cancer Using Literature-Augmented Large Language Models
Lai, Y.-J., Wang, L.-J., Yasaka, T. M., Shin, Y., Ning, M., Tan, Y., Shih, C.-H., Guo, Y., Chen, P.-Y., Galloway, H., et al. (2025) · Cancer Research Communications · doi:10.1158/2767-9764.crc-25-0030 · Cited by 4
Eastaugh, C., Still, M., Beyer, F. R., Wallace, S. A., & O’Keefe, H. (2025) · Cochrane Evidence Synthesis and Methods · doi:10.1002/cesm.70049 · Cited by 7
“Automating search strategy development and translation across databases is widely sought after [19], yet it has been historically difficult to develop accurate software to support this, notably due to the differences in ontologies used for subject headings [20, 21].”
Doneva, S. E., de Viragh, S., Hubarava, H., Schandelmaier, S., Briel, M., & Ineichen, B. V. (2025) · Research Synthesis Methods · doi:10.1017/rsm.2025.10031 · Cited by 7
“As a result, the manual screening of a large number of abstracts can take months, 5 a challenge exacerbated by the ever-increasing volume of scientific publications.”
Adam, T. J., Abosabie, S. A., Dittmer, M., Wolf, E., Abosabie, S. A., Behnke, C., Baier, F., Weickmann, A., Köser, L., Correll, C. U., et al. (2026) · Research Synthesis Methods · doi:10.1017/rsm.2026.10093 · Cited by 12
MacPherson, M., & Rourke, S. (2024) · Journal of Medical Internet Research · doi:10.2196/54821 · Cited by 5
“Compared to more traditional review types (eg, systematic or scoping), which can take more than a year to complete [49], rapid reviews provide a practical way of synthesizing available evidence to inform clinical decision-making.”
BackgroundMachine learning enables automated screening for systematic reviews and meta-analysis in urology
Menold, H. S., Wieland, V. L. S., Haney, C. M., Uysal, D., Wessels, F., Cacciamani, G. E., Michel, M. S., Seide, S., & Kowalewski, K. (2024) · World Journal of Urology · doi:10.1007/s00345-024-05078-y · Cited by 4
“In an analysis of 195 SR/ MAs, Borah et al. showed that the average time to pub - lication was more than one year (62 weeks), while some reviews took more than three years to be published [ 4 ].”
Ciurescu, S., Ciupici-Cladovan, M., Buciu, V. B., Ilaș, D. G., Cîtu, C., & Sas, I. (2025) · Medicina · doi:10.3390/medicina61071170 · Cited by 8
“A comprehensive search strategy across multiple major electronic databases, complemented by hand-searching, aimed to minimize publication bias and capture a broad spectrum of relevant literature [27].”
Abukhaled, Y., Allawama, T. M., & Serhan, H. A. A. (2025) · Medical Hypothesis Discovery & Innovation in Ophthalmology · doi:10.51329/mehdiophthal1522 · Cited by 8
Shokraneh, F. (2019) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2019.04.008 · Cited by 6
“Since performing systematic reviews are already time-consuming [2] and resource-rating [3], it is arguable how adding more steps such as reproducibility test that requires more time and human resources could reduce waste and increase value compared to excluding some steps [4].”
BackgroundPap, R., Lockwood, C. S., Stephenson, M., & Simpson, P. (2021) · Emergency Medicine Journal · doi:10.1136/emermed-2020-210455 · Cited by 6
Escaldelai, F. M. D., Escaldelai, L., & Bergamaschi, D. P. (2022) · Revista Brasileira de Epidemiologia · doi:10.1590/1980-549720220030.2 · Cited by 4
Fully Automated Scholarly Search for Biomedical Systematic Literature Reviews
Budau, L., & Ensan, F. (2024) · IEEE Access · doi:10.1109/access.2024.3405529 · Cited by 6
“…that ease some of the burdens in the process of conducting biomedical SLRs [9], [10], [11], [12], SLRs are still exceedingly expensive ( > $100 , 000 per study on health-related topics), often quite lengthy ( > 1 year), and labor-intensive ( > 1 , 100 hours, 5 reviewers) [13], [14], [15].”
BackgroundMachine Learning-Assisted Health Economics and Policy Reviews: A Comparative Assessment
Cavallaro, L., Ardito, V., Drummond, M., & Ciani, O. (2025) · Applied Health Economics and Health Policy · doi:10.1007/s40258-025-00963-y · Cited by 6
“In fact, while the percentage of relevant articles may be as low as 1% of the total search yield, on the other hand, the long conception-to-completion interval may render a systematic or scoping review outdated by the time it is published [4, 5].”
How good does our map of knowledge have to be?: a comment on Berger-Tal et al.
Nakagawa, S., & Lagisz, M. (2018) · Behavioral Ecology · doi:10.1093/beheco/ary137 · Cited by 4
“Given that most systematic reviews take over a year (Borah et al. 2017), a “normal” systematic review or map will fail to deliver evidence on time under such circumstances.”
BackgroundImproving systematic reviews in obstetrics and gynecology
Cusimano, M. C., Fazelzad, R., & Meffe, F. (2020) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2020.09.015 · Cited by 5
A Mixed‐Method Usability Study on User Experience with Systematic Review Software
Wang, M., Sharmin, S., Wang, M., & Yu, F. (2021) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.462 · Cited by 10
“…reproducible and accurate findings, a systematic review usually takes at least six months to a year and three to five team members to complete (Borah et al., 2017; Howard et al., 2016), which covers a range of research activities such as formulating the research question, searching the…”
BackgroundWieland, V. L. S., Uysal, D., Probst, P., Grilli, M., Haney, C. M., Abate, M. A. S., Egen, L., Neuberger, M., Cacciamani, G. E., KRIEGMAIR, M. C., et al. (2023) · International Journal of Surgery Protocols · doi:10.1097/sp9.0000000000000008 · Cited by 4
“However, conducting MAs is a resource and time-consuming process that can cost up to $100 000 for one MA [7,8] and takes an average of 67.3 weeks [9] .”
BackgroundInfluentialTrends in stroke-related journals: Examination of publication patterns using topic modeling
Ozkara, B. B., Karabacak, M., Margetis, K. D., Smith, W. S., Wintermark, M., & Yedavalli, V. S. (2024) · Journal of Stroke and Cerebrovascular Diseases · doi:10.1016/j.jstrokecerebrovasdis.2024.107665 · Cited by 5
Opportunities for Computer Support for Systematic Reviewing - A Gap Analysis
Hoang, L., & Schneider, J. (2018) · Lecture notes in computer science · doi:10.1007/978-3-319-78105-1_40 · Cited by 5
“Despite their importance, systematic reviews require great amount of human effort: a mean of 67 weeks from deposit of a protocol to publication of the review [1], with a mean of 1000 hours of person time [2].”
Sharma, B., Willis, V. C., Huettner, C. S., Beaty, K. A., Snowdon, J. L., Xue, S., South, B. R., Jackson, G. P., Weeraratne, D., & Michelini, V. V. (2020) · JAMIA Open · doi:10.1093/jamiaopen/ooaa028 · Cited by 6
“94% as reported in pure systematic reviews.(35) Early error analysis revealed a number of false positives from review studies, so we modified the process to exclude review papers from classifiers.”
BackgroundGates, A., Gates, M., Sim, S., Elliott, S. A., Pillay, J., & Hartling, L. (2021) · doi:10.23970/ahrqepcmethodscreatingefficiencies · Cited by 3
Oami, T., Okada, Y., Maeda, K., & Nakada, T.-A. (2026) · Frontiers in Digital Health · doi:10.3389/fdgth.2026.1799623 · Cited by 4
“However, conventional systematic review workflows rely heavily on manual screening and data extraction, rendering them labor-intensive, time-consuming, and susceptible to human error (3–5).”
Spiero, I., Leeuwenberg, A. M., Moons, K. G. M., Hooft, L., & Damen, J. A. A. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111987 · Cited by 3
Zhu, S., & Chan, J. (2023) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/3584931.3606985 · Cited by 3
“synthesis work is, by many accounts, extremely arduous [8, 15, 23] and cost-intensive [7, 8, 28, 32, 38].”
Background
Metadata
- Authors
- Borah, R., Brown, A. W., Capers, P. L., & Kaiser, K. A.
- Affiliations
- University of Alabama at Birmingham (US)
- Corresponding author
- Kaiser, Kathryn A
- Venue
- BMJ Open
- Publisher
- BMJ
- Published
- 2017-02
- Volume
- 7
- Issue
- 2
- Pages
- e012545
- ISSN
- 2044-6055 (ISSN-L)
- PMC
- PMC5337708
- OpenAlex
- W2593758073
- Cited by
- 737
- References
- 29
- Citations by year
- 2026: 114 · 2025: 129 · 2024: 105 · 2023: 76 · 2022: 71 · 2021: 80 · 2020: 87 · 2019: 42 · 2018: 24 · 2017: 9
- FWCI
- 10.85
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc
- Locations
- BMJ Open (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- IUScholarWorks Open (Indiana University) (submittedVersion, cc-by-nc)
- HighWire Press Open Archive (submittedVersion)
- Funders
- University of Alabama at Birmingham (P30DK056336)
- University of Alabama (P30DK056336)
- National Institutes of Health
- National Institute of General Medical Sciences (K12GM088010)
- National Institute of Diabetes and Digestive and Kidney Diseases (P30DK056336)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Keywords
- systematic reviews, time to publication, research workforce, medical interventions, research resource allocation
- MeSH terms
- Workforce, Humans, Publishing, Registries, Review Literature as Topic, Task Performance and Analysis, Databases, Bibliographic, Information Storage and Retrieval, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.