Comparison of effect estimates between preprints and peer-reviewed journal articles of COVID-19 trials
Open accessAbstract
Background: Preprints are increasingly used to disseminate research results, providing multiple sources of information for the same study. We assessed the consistency in effect estimates between preprint and subsequent journal article of COVID-19 randomized controlled trials. Methods: The study utilized data from the COVID-NMA living systematic review of pharmacological treatments for COVID-19 (covid-nma.com) up to July 20, 2022. We identified randomized controlled trials (RCTs) evaluating pharmacological treatments vs. standard of care/placebo for patients with COVID-19 that were originally posted as preprints and subsequently published as journal articles. Trials that did not report the same analysis in both documents were excluded. Data were extracted independently by pairs of researchers with consensus to resolve disagreements. Effect estimates extracted from the first preprint were compared to effect estimates from the journal article. Results: The search identified 135 RCTs originally posted as a preprint and subsequently published as a journal article. We excluded 26 RCTs that did not meet the eligibility criteria, of which 13 RCTs reported an interim analysis in the preprint and a final analysis in the journal article. Overall, 109 preprint–article RCTs were included in the analysis. The median (interquartile range) delay between preprint and journal article was 121 (73–187) days, the median sample size was 150 (71–464) participants, 76% of RCTs had been prospectively registered, 60% received industry or mixed funding, 72% were multicentric trials. The overall risk of bias was rated as ‘some concern’ for 80% of RCTs. We found that 81 preprint–article pairs of RCTs were consistent for all outcomes reported. There were nine RCTs with at least one outcome with a discrepancy in the number of participants with outcome events or the number of participants analyzed, which yielded a minor change in the estimate of the effect. Furthermore, six RCTs had at least one outcome missing in the journal article and 14 RCTs had at least one outcome added in the journal article compared to the preprint. There was a change in the direction of effect in one RCT. No changes in statistical significance or conclusions were found. Conclusions: Effect estimates were generally consistent between COVID-19 preprints and subsequent journal articles. The main results and interpretation did not change in any trial. Nevertheless, some outcomes were added and deleted in some journal articles.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Initial data suggest a few differences between trial results on preprint servers and those on final publication.
“Initial data suggest a few differences between trial results on preprint servers and those on final publication.132 133 134”
From SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials (2025) · cites this paper as [134]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,706 · see citing works
RoB 2: a revised tool for assessing risk of bias in randomised trials
Sterne, J. A. C., Savović, J., Page, M. J., Elbers, R. G., Blencowe, N. S., Boutron, I., Cates, C. J., Cheng, H.-Y., Corbett, M. S., Eldridge, S. M., et al. (2019) · BMJ · doi:10.1136/bmj.l4898 · Cited by 33,583 · see citing works
Self-citationConducting Meta-Analyses in <i>R</i> with the <b>metafor</b> Package
Viechtbauer, W. (2010) · Journal of Statistical Software · doi:10.18637/jss.v036.i03 · Cited by 20,464 · see citing works
Drug treatments for covid-19: living systematic review and network meta-analysis
Siemieniuk, R. A., Bartoszko, J. J., Zeraatkar, D., Kum, E., Qasim, A., Díaz Martinez, J. P., Izcovich, A., Rochwerg, B., Lamontagne, F., Han, M. A., et al. (2020) · BMJ · doi:10.1136/bmj.m2980 · Cited by 846 · see citing works
Kyriazopoulou, E., Poulakou, G., Milionis, H. J., Metallidis, S., Adamis, G., Tsiakos, K., Fragkou, A., Rapti, A., Damoulari, C., Fantoni, M., et al. (2021) · Nature Medicine · doi:10.1038/s41591-021-01499-z · Cited by 538 · see citing works
Murai, I. H., Fernandes, A. L., Sales, L. P., Pinto, A. J., Goessler, K. F., Duran, C. S., Silva, C. B. R., Franco, A. S., Macêdo, M. B., Dalmolin, H. H. H., et al. (2021) · JAMA · doi:10.1001/jama.2020.26848 · Cited by 528 · see citing works
Abani, O., Abbas, A. H., Abbas, F., Abbas, M. A., Abbasi, S., Abbass, H., Abbott, A., Abdallah, N., Abdelaziz, A., Abdelfattah, M., et al. (2021) · The Lancet · doi:10.1016/s0140-6736(21)00897-7 · Cited by 493 · see citing works
Abani, O., Abbas, A. H., Abbas, F., Abbas, M. A., Abbasi, S., Abbass, H., Abbott, A., Abdallah, N., Abdelaziz, A., Abdelfattah, M., et al. (2022) · The Lancet · doi:10.1016/s0140-6736(22)00163-5 · Cited by 386 · see citing works
Early Outpatient Treatment for Covid-19 with Convalescent Plasma
Sullivan, D. J., Gebo, K. A., Shoham, S., Bloch, E. M., Lau, B. M., Shenoy, A. G., Mosnaim, G. S., Gniadek, T. J., Fukuta, Y., Patel, B., et al. (2022) · New England Journal of Medicine · doi:10.1056/nejmoa2119657 · Cited by 287 · see citing works
Pilot trial of high-dose vitamin C in critically ill COVID-19 patients
Zhang, J., Rao, X., Li, Y., Zhu, Y., Liu, F., Guo, G., Luo, G., Meng, Z., Backer, D. D., Xiang, H., et al. (2021) · Annals of Intensive Care · doi:10.1186/s13613-020-00792-3 · Cited by 256 · see citing works
How swamped preprint servers are blocking bad coronavirus research
Kwon, D. (2020) · Nature · doi:10.1038/d41586-020-01394-6 · Cited by 174 · see citing works
Carneiro, C. F. D., Queiroz, V. G. S., Moulin, T. C., Carvalho, C. A. M., Haas, C. B., Rayêe, D., Henshall, D. E., De-Souza, E. A., Amorim, F. E., Boos, F. Z., et al. (2020) · Research Integrity and Peer Review · doi:10.1186/s41073-020-00101-3 · Cited by 146 · see citing works
Preprints Involving Medical Research—Do the Benefits Outweigh the Challenges?
Flanagin, A., Fontanarosa, P. B., & Bauchner, H. (2020) · JAMA · doi:10.1001/jama.2020.20674 · Cited by 115 · see citing works
Prophylaxis against covid-19: living systematic review and network meta-analysis
Bartoszko, J. J., Siemieniuk, R. A., Kum, E., Qasim, A., Zeraatkar, D., Martinez, J. P. D., Azab, M., Ibrahim, S., Izcovich, A., Soto, G. B., et al. (2021) · BMJ · doi:10.1136/bmj.n949 · Cited by 113 · see citing works
Systematic examination of preprint platforms for use in the medical and biomedical sciences setting
Kirkham, J. J., Penfold, N. C., Murphy, F., Boutron, I., Ioannidis, J. P., Polka, J., & Moher, D. (2020) · BMJ Open · doi:10.1136/bmjopen-2020-041849 · Cited by 112 · see citing works
Self-citationThe COVID-NMA Project: Building an Evidence Ecosystem for the COVID-19 Pandemic
Boutron, I., Chaimani, A., Meerpohl, J. J., Hróbjartsson, A., Devane, D., Gabriel Rada, G., Tovey, D. I., Grasselli, G., & Ravaud, P. (2020) · Annals of Internal Medicine · doi:10.7326/m20-5261 · Cited by 95 · see citing works
Self-citationTracking changes between preprint posting and journal publication during a pandemic
Brierley, L., Nanni, F., Polka, J., Dey, G., Pálfy, M., Fraser, N., & Coates, J. A. (2022) · PLoS Biology · doi:10.1371/journal.pbio.3001285 · Cited by 94 · see citing works
Temesgen, Z., Burger, C. D., Baker, J., Polk, C. M., Libertin, C. R., Kelley, C. F., Marconi, V. C., Orenstein, R., Catterson, V. M., Aronstein, W. S., et al. (2021) · The Lancet Respiratory Medicine · doi:10.1016/s2213-2600(21)00494-x · Cited by 68 · see citing works
Quinn, T. M., Gaughan, E. E., Bruce, A. M., Antonelli, J., O’Connor, R. A., Li, F., McNamara, S., Koch, O., Mackintosh, C., Dockrell, D. H., et al. (2022) · EBioMedicine · doi:10.1016/j.ebiom.2022.103856 · Cited by 56 · see citing works
the COVID-19 NMA Consortium, Oikonomidi, T., Boutron, I., Pierre, O., Cabanac, G., & Ravaud, P. (2020) · BMC Medicine · doi:10.1186/s12916-020-01880-8 · Cited by 54 · see citing works
Self-citationBero, L., Lawrence, R., Leslie, L., Chiu, K., McDonald, S., Page, M. J., Grundy, Q., Parker, L., Boughton, S., Kirkham, J. J., et al. (2021) · BMJ Open · doi:10.1136/bmjopen-2021-051821 · Cited by 50 · see citing works
Eckmann, P., & Bandrowski, A. E. (2023) · PLoS ONE · doi:10.1371/journal.pone.0281659 · Cited by 49 · see citing works
Zeraatkar, D., Pitre, T., Leung, G., Cusano, E., Agarwal, A., Khalid, F., Escamilla, Z., Cooper, M. A., Ghadimi, M., Wang, Y., et al. (2022) · BMJ Medicine · doi:10.1136/bmjmed-2022-000309 · Cited by 44 · see citing works
van Schalkwyk, M. C. I., Hird, T. R., Maani, N., Petticrew, M., & Gilmore, A. B. (2020) · BMJ · doi:10.1136/bmj.m3111 · Cited by 38 · see citing works
High-dose vitamin C infusion for the treatment of critically ill COVID-19Preprint
Zhang, J., Rao, X., Li, Y., Zhu, Y., Liu, F., Guo, G., Luo, G., Meng, Z., Backer, D. D., Xiang, H., et al. (2020) · Research Square · doi:10.21203/rs.3.rs-52778/v1 · Cited by 32 · see citing works
Transparency and reporting characteristics of COVID-19 randomized controlled trials
Kapp, P., Esmail, L., Ghosn, L., Ravaud, P., & Boutron, I. (2022) · BMC Medicine · doi:10.1186/s12916-022-02567-y · Cited by 25 · see citing works
Self-citationMobarak, S., Salasi, M., Hormati, A., Khodadadi, J., Ziaee, M., Abedi, F., Ebrahimzadeh, A., Azarkar, Z., Mansour‐Ghanaei, F., Joukar, F., et al. (2021) · Journal of Antimicrobial Chemotherapy · doi:10.1093/jac/dkab433 · Cited by 21 · see citing works
Why preprints are good for patients
Horby, P. (2022) · Nature Medicine · doi:10.1038/s41591-022-01812-4 · Cited by 15 · see citing works
Should systematic reviews and meta-analyses include data from preprints?
Brietzke, E. M., Gomes, F. A., Gerchman, F., & Freire, R. C. R. (2021) · Trends in Psychiatry and Psychotherapy · doi:10.47626/2237-6089-2021-0324 · Cited by 14 · see citing works
Davidson, M., Evrenoglou, T., Graña, C., Chaimani, A., & Boutron, I. (2023) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2023.08.011 · Cited by 8 · see citing works
Self-citationMobarak, S., Salasi, M., Hormati, A., Khodadadi, J., Ziaee, M., Abedi, F., Ebrahimzadeh, A., Azarkar, Z., Mansour‐Ghanaei, F., Joukar, F., et al. (2021) · SSRN Electronic Journal · doi:10.2139/ssrn.3792895 · Cited by 5 · see citing works
Flagged first, then by citations
Research works printed but not linked: 7
- [20]Group RC, Horby PW, Mafham M, Peto L, Campbell M, Pessoa-Amorim G et al. Casirivimab and imdevimab in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial [Internet]. medRxiv; 2021 [cited 2022 Dec 7]. p. 2021.06.15.21258542. Available from: https://www.medrxiv.org/content/10.1101/2021.06.15.21258542v1. · DOI
- [24]Group TRC, Horby PW, Estcourt L, Peto L, Emberson JR, Staplin N et al. Convalescent plasma in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial [Internet]. medRxiv; 2021 [cited 2022 Dec 7]. p. 2021.03.09.21252736. Available from: https://www.medrxiv.org/content/10.1101/2021.03.09.21252736v1. · DOI
- [28]Murai IH, Fernandes AL, Sales LP, Pinto AJ, Goessler KF, Duran CSC et al. Effect of Vitamin D3 Supplementation vs Placebo on Hospital Length of Stay in Patients with Severe COVID-19: A Multicenter, Double-blind, Randomized Controlled Trial [Internet]. medRxiv; 2020 [cited 2022 Dec 7]. p. 2020.11.16.20232397. Available from: https://www.medrxiv.org/content/10.1101/2020.11.16.20232397v1. · DOI
- [30]Temesgen Z, Burger CD, Baker J, Polk C, Libertin C, Kelley C et al. Lenzilumab Efficacy and Safety in Newly Hospitalized Covid-19 Subjects: Results from the Live-Air Phase 3 Randomized Double-Blind Placebo-Controlled Trial [Internet]. medRxiv; 2021 [cited 2022 Dec 7]. p. 2021.05.01.21256470. Available from: https://www.medrxiv.org/content/10.1101/2021.05.01.21256470v1. · DOI
- [32]Kyriazopoulou E, Poulakou G, Milionis H, Metallidis S, Adamis G, Tsiakos K et al. Early Anakinra Treatment for COVID-19 Guided by Urokinase Plasminogen Receptor [Internet]. 2021 May [cited 2021 Sep 8] p. 2021.05.16.21257283. Available from: https://www.medrxiv.org/content/10.1101/2021.05.16.21257283v1. · DOI
- [35]Quinn TM, Gaughan EE, Bruce A, Antonelli J, O’Connor R, Li F et al. Randomised controlled trial of intravenous nafamostat mesylate in COVID pneumonitis: phase 1b/2a experimental study to investigate safety. Pharmacokinet Pharmacodynamics eBioMedicine. 2022;76. PMC8831100 35152152
- [36]Sullivan DJ, Gebo KA, Shoham S, Bloch EM, Lau B, Shenoy AG et al. Randomized Controlled Trial of Early Outpatient COVID-19 Treatment with High-Titer Convalescent Plasma [Internet]. medRxiv; 2021 [cited 2022 Dec 7]. p. 2021.12.10.21267485. Available from: https://www.medrxiv.org/content/10.1101/2021.12.10.21267485v1. · DOI
Cited by
Sorted by citations
SPIRIT 2025 explanation and elaboration: updated guideline for protocols of randomised trials
Hróbjartsson, A., Boutron, I., Hopewell, S., Moher, D., Schulz, K. F., Collins, G. S., Tunn, R., Aggarwal, R., Berkwits, M., Berlin, J. A., et al. (2025) · BMJ · doi:10.1136/bmj-2024-081660 · Cited by 171 · see citing works
Self-citationInterventions for the management of long covid (post-covid condition): living systematic review
Zeraatkar, D., Ling, M., Kirsh, S., Jassal, T., Shahab, M., Movahed, H., Talukdar, J. R., Walch, A., Chakraborty, S. P., Turner, T., et al. (2024) · BMJ · doi:10.1136/bmj-2024-081318 · Cited by 149 · see citing works
Zeraatkar, D., Ling, M., Kirsh, S., Jassal, T., Pitre, T., Chakraborty, S. P., Turner, T., Turkstra, L. S., McIntyre, R. S., Izcovich, A., et al. (2025) · BMJ Open · doi:10.1136/bmjopen-2024-086407 · Cited by 14 · see citing works
Yin, H. R., Ahn, W., Forster, P. M., & Rust, R. (2026) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2026.06.30.735556 · Cited by 4 · see citing works
Preprints vs. Journal Articles: Citation Impact in COVID-19 Research
Tsunoda, H., Yuan, S. U. N., Nishizawa, M., Liu, X., Amano, K., & Kominami, R. (2024) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/3677389.3702527 · Cited by 2 · see citing works
Tsunoda, H., Sun, Y., Nishizawa, M., Liu, X., Amano, K., & Kominami, R. (2024) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.1202 · Cited by 2 · see citing works
The Impact of Preprints on the Citations of Journal Articles Related to COVID-19
Tsunoda, H., Sun, Y., Nishizawa, M., Liu, X., Amano, K., & Kominami, R. (2024) · Lecture notes in computer science · doi:10.1007/978-981-96-0868-3_4 · Cited by 2 · see citing works
Prediction of transformative breakthroughs in biomedical researchPreprint
Davis, M. T., Busse, B. L., Arabi, S., Meyer, P., Hoppe, T. A., Meseroll, R. A., Hutchins, B. I., Willis, K. A., & Santangelo, G. M. (2025) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.64898/2025.12.16.694385 · Cited by 3 · see citing works
The World of Academic Preprints: Promises and Pitfalls
Dutta, D., Kumar, M., & Sharma, M. (2025) · International Journal of Advanced Medical and Health Research · doi:10.4103/ijamr.ijamr_60_25
- Authors
- Davidson, M., Evrenoglou, T., Graña, C., Chaimani, A., & Boutron, I.
- Affiliations
- Inserm (FR)
- Corresponding author
- Davidson, Mauricia
- Venue
- BMC Medical Research Methodology
- Published
- 2024-01-11
- Volume
- 24
- Issue
- 1
- Pages
- 9-9
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC10782611
- OpenAlex
- W4390751452
- Cited by
- 9
- References
- 31
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 4
- FWCI
- 3.36
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- BMC Medical Research Methodology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- HAL (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Institut National de la Santé et de la Recherche Médicale
- Topics
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Health Sciences › Medicine › Infectious Diseases › COVID-19 Clinical Research Studies
- Keywords
- preprints, COVID-19 pandemic, randomized controlled trials, effect estimates, pharmacotherapy, reporting discrepancies, outcome reporting, research dissemination, risk of bias, clinical trial registration, funding disclosure, interim analysis, time to publication, research integrity, reproducibility
- MeSH terms
- Preprints as Topic, Systematic Reviews as Topic, COVID-19, Humans, Bias, Randomized Controlled Trials as Topic, Publication Bias, Peer Review, Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
