The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index; 15 of the 32 sources its claims cite were checked for support.
Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature
Open accessPublished version of a preprint
Abstract
Background: Preprint usage is growing rapidly in the life sciences; however, questions remain on the relative quality of preprints when compared to published articles. An objective dimension of quality that is readily measurable is completeness of reporting, as transparency can improve the reader’s ability to independently interpret data and reproduce findings. Methods: In this observational study, we initially compared independent samples of articles published in bioRxiv and in PubMed-indexed journals in 2016 using a quality of reporting questionnaire. After that, we performed paired comparisons between preprints from bioRxiv to their own peer-reviewed versions in journals. Results: Peer-reviewed articles had, on average, higher quality of reporting than preprints, although the difference was small, with absolute differences of 5.0% [95% CI 1.4, 8.6] and 4.7% [95% CI 2.4, 7.0] of reported items in the independent samples and paired sample comparison, respectively. There were larger differences favoring peer-reviewed articles in subjective ratings of how clearly titles and abstracts presented the main findings and how easy it was to locate relevant reporting information. Changes in reporting from preprints to peer-reviewed versions did not correlate with the impact factor of the publication venue or with the time lag from bioRxiv to journal publication. Conclusions: Our results suggest that, on average, publication in a peer-reviewed journal is associated with improvement in quality of reporting. They also show that quality of reporting in preprints in the life sciences is within a similar range as that of peer-reviewed articles, albeit slightly lower on average, supporting the idea that preprints should be considered valid scientific contributions.
Overview
CoChat Summary
TL;DR
In this observational study, peer-reviewed biomedical articles had slightly higher completeness of reporting than bioRxiv preprints, both in independent samples and in paired preprint-to-publication comparisons. The differences were small, and changes from preprint to peer-reviewed version did not correlate with journal impact factor or time to publication.
Key findings
Peer-reviewed articles scored higher than preprints by 5.0% [95% CI 1.4, 8.6] in independent samples (bioRxiv 66.9 ± 12.2 vs. PubMed 71.9 ± 10.1) and by 4.7% [95% CI 2.4, 7.0] in paired comparisons (preprint 67.6 ± 10.8 vs. peer-reviewed 72.3 ± 10.1).
- In the paired sample, 27% of pairs (15/56) showed a decrease in reporting score from preprint to peer-reviewed version.
- The largest paired item difference was conflict of interest statements (46.4% vs. 83.9%, p = 1.9 × 10−5); removing this item reduced the paired difference from 4.7 to 3.3 percentage points.
- Changes in reporting score did not correlate with journal impact factor (ρ = 0.16, p = 0.25) or time from bioRxiv to journal publication (r = 0.03, p = 0.81). Preprints later published had higher reporting scores than unpublished preprints (68.8 ± 11.8 vs. 61.1 ± 11.9, p = 0.02).
- Subjective ratings favored peer-reviewed articles for title/abstract clarity (independent difference 0.4 [95% CI 0.2, 0.6]) and ease of extracting information (independent difference 0.7 [95% CI 0.5, 1.0]; paired difference 0.4 [95% CI 0.2, 0.6]).
Methods
Observational study with preregistered protocols: independent random samples of bioRxiv preprints and PubMed-indexed articles from 2016, matched by biological model, with 76 articles per group; paired comparison of 56 bioRxiv preprints to their own peer-reviewed journal versions.
- Quality of reporting was assessed with an online questionnaire based on existing reporting guidelines; each article was evaluated by three evaluators, with the most prevalent answer used for objective items and the mean used for subjective items. Overall score was the percentage of applicable reporting items reported.
- Analyses used Student’s t-tests, paired t-tests, ANOVA, Fisher’s exact or McNemar’s exact tests, and Spearman or Pearson correlations; primary outcomes used α = 0.05, with Sidak correction for secondary outcomes.
Limitations
The paper states that generalizability is limited by inclusion criteria requiring primary data and statistical comparisons, use of a single preprint platform, analysis of the first table/figure rather than necessarily the main finding, evaluators not blinded to article group origin, and limited power for individual-question and journal-feature analyses.
- The paper notes that independent-sample differences may reflect field, region, and practice differences between bioRxiv and PubMed rather than peer review alone, and that paired changes may stem from other editorial or submission requirements, not only reviewer feedback.
- Evident from the text: the paired sample used only the first preprint version, so changes from other feedback sources cannot be excluded; peer-review model comparisons were not performed because most articles were in single-blind journals.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures

![Fig. 2 Reporting scores by source of the article. a Random samples of bioRxiv and PubMed were evaluated. Mean ± S.D.: bioRxiv = 66.9 ± 12.2, PubMed = 71.9 ± 10.1; n = 76/group. Student’s t-test, t = 2.75, p = 0.007, 95% C.I. [1.40, 8.59]. b A sample of bioRxiv articles was compared against their peer-reviewed versions published by a journal. Mean ± S.D.: Preprint = 67.6 ± 10.8, Peer-Reviewed = 72.3 ± 10.1; n = 56 pairs. Paired t-test, t = 4.15, p = 0.0001, 95% C.I. [2.44, 6.99]. On the right, absolute changes in score from preprint to peer-reviewed versions are plotted for each pair](/paper-figures/fdf3219fb88d4b75b105579ec6e871cf/comparing-quality-of-reporting-between-preprints-and-peer-reviewed-articles-in-figure-2.jpg)
![Fig. 3 Quality of reporting by article size. a Overall scores by number of figure subpanels/tables in the independent samples comparison. Spearman’s correlations: All articles, ρ = − 0.31, 95% C.I. [− 0.46, − 0.15], p = 9.5 × 10− 5; bioRxiv (shown in red), ρ = − 0.35, 95% C.I. [− 0.56, − 0.11], p = 0.002; PubMed (shown in blue), ρ = − 0.22, 95%C.I. [− 0.44, 0.0006], p = 0.05. N = 152 (76/group). b Change in score from preprint to peer-reviewed versions by change in the number of figures subpanels/tables in the paired sample. Spearman’s correlation: ρ = − 0.07, 95% C.I. [− 0.33, 0.17], p = 0.59, N = 56. One article presented a large decrease in number of figures (− 71 figures subpanels/tables), as it was published as a brief communication. c Difference between scores from peer-reviewed to preprint version by mean number of figure subpanels/tables between preprint and peer-reviewed version in the paired sample. Spearman’s correlation: ρ = 0.24, 95% C.I. [− 0.05, 0.49], p = 0.08, N = 56](/paper-figures/5c2de561085340fc82883c402c9da7bf/comparing-quality-of-reporting-between-preprints-and-peer-reviewed-articles-in-figure-3.jpg)
![Fig. 4 Quality of reporting by impact factor of publication venue. a Overall scores by 2016 impact factor of the publication venue in the independent samples comparison. Spearman’s correlation: ρ = − 0.11, 95% C.I. [− 0.35, 0.13], p = 0.35, n = 69. b Change in score from peer-reviewed to preprint version by impact factor of the peer-reviewed publication year in the paired sample. Spearman’s correlation: ρ = 0.16, 95% C.I. [− 0.13, 0.43], p = 0.25, n = 53. Impact factor was unavailable for 7 articles in the independent sample and 3 in the paired one](/paper-figures/48fda122dfba48db87710471f366b737/comparing-quality-of-reporting-between-preprints-and-peer-reviewed-articles-in-figure-4.jpg)
![Fig. 5 Quality of reporting and the peer review process. a Difference between scores from peer-reviewed to preprint version by time to publication (in months) in the paired sample. Pearson’s correlation: r = 0.03, 95% C.I. [− 0.23, 0.29], p = 0.81, n = 56 pairs. b Overall reporting scores by publication status (published or not in a peer-reviewed journal) of preprints assessed in the independent sample. Mean ± S.D.: Unpublished = 61.1 ± 11.9, n = 19; Published = 68.8 ± 11.8, n = 57. 95% C.I. [1.45, 13.93], Student’s t test: t = 2.45, p = 0.02](/paper-figures/b6570df43a9c4ff384aa5c5ffd2207f7/comparing-quality-of-reporting-between-preprints-and-peer-reviewed-articles-in-figure-5.jpg)
![Fig. 6 Subjective assessment by article source. a Clarity of title/abstract for the independent samples comparison. Scores were given as an answer to “Do the title and abstract provide a clear idea of the article’s main findings?”. Mean ± S.D.: bioRxiv = 3.9 ± 0.6, n = 72; PubMed = 4.3 ± 0.7, n = 72. 95% C.I. [0.2–0.6]. Student’s t-test: t = 3.61, p = 0.0004. b Clarity of title/abstract for the paired sample comparison. Mean ± S.D.: Preprint = 4.0 ± 0.6, n = 56; Peer-reviewed = 4.1 ± 0.5. 95% C.I. [− 0.03, 0.4]. Paired t-test: t = 1.66, p = 0.10. Right panel shows the differences between scores in preprint and peer-reviewed versions. c Easiness to extract information for the independent samples comparison. Scores were given as an answer to “Was the required information easy to find and extract from the article?”. Mean ± S.D.: bioRxiv = 3.4 ± 0.8, n = 72; PubMed = 4.2 ± 0.6. 95% C.I. [0.5, 1.0]. Student’s t-test: t = 6.22, p = 5.1 × 10− 9. d Easiness to extract information for the paired sample comparison. Mean ± S.D.: Preprint = 3.5 ± 0.7, n = 56; Peer-reviewed = 4.0 ± 0.6, n = 56. 95% C.I. [0.2, 0.6]. Paired t-test: t = 4.12, p = 0.0001. Right panel shows changes in score from preprint to peer-reviewed versions](/paper-figures/fe8a090deb5049f087558ad787d7f776/comparing-quality-of-reporting-between-preprints-and-peer-reviewed-articles-in-figure-6.jpg)
![Fig. 7 Quality of reporting by subjective scores. a Overall reporting scores by title/abstract clarity in the independent samples. Pearson’s correlation: r = 0.38, 95% C.I. [0.23, 0.51], p = 3.1 × 10− 6, n = 144 (all articles); r = 0.12, 95% C.I. [− 0.11, 0.35], p = 0.29, n = 72 (bioRxiv); r = 0.55, 95% C.I. [0.36, 0.69], p = 6.3 × 10− 7, n = 72 (PubMed). b Changes in overall reporting scores by changes in title/abstract clarity in the paired sample. Pearson’s correlation: r = 0.25, 95% C.I. [− 0.01, 0.68], p = 0.06, n = 56. c Overall reporting scores by easiness to extract information in the independent samples. Pearson’s correlation: r = 0.59, 95% C.I. [0.47, 0.69], p = 8.7 × 10− 15, n = 144 (all articles); r = 0.54, 95% C.I. [0.35, 0.68], p = 1.04 × 10− 6, n = 72 (bioRxiv); r = 0.60, 95% C.I. [0.43, 0.73], p = 2.2 × 10− 8, n = 72 (PubMed). d Changes in overall reporting scores changes in easiness to extract information in the paired sample. Pearson’s correlation: r = 0.20, 95% C.I. [− 0.06, 0.44], p = 0.13, n = 56. In all panels, bioRxiv articles are in red and PubMed ones are in blue, while differences between paired articles are shown in purple](/paper-figures/7503175499c7413d81b737a9c345311d/comparing-quality-of-reporting-between-preprints-and-peer-reviewed-articles-in-figure-7.jpg)
![Fig. 8 Reporting quality by article formatting. a Overall reporting score by embedding of figures in all preprints assessed. Preprints assessed in both stages of the study were included only once for this analysis, with the mean of reporting scores from both assessments. Mean ± S.D.: Non-embedded = 64.6 ± 10.9, n = 59; Embedded = 70.8 ± 11.6, n = 26. 95% C.I. [1.0, 11.4]. Student’s t-test: t = 2.37, p = 0.02. b Difference between scores from peer-reviewed to preprint version by embedding of figures in the preprint version (paired sample). Mean ± S.D.: Non-embedded = 4.3 ± 8.3, n = 38; Embedded = 5.6 ± 9.0, n = 18. 95% C.I. [− 3.6, 6.1]. Student’s t-test, t = 0.55, p = 0.58](/paper-figures/6c976a646dda4ca986d4762154d95e73/comparing-quality-of-reporting-between-preprints-and-peer-reviewed-articles-in-figure-8.jpg)
Verification
Claims the paper makes
3 Valid · 1 Not valid · 12 Not checked · 151 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“Reporting guidelines and checklists provided to authors during the review or manuscript preparation processes have been reported to cause modest improvements limited to a few items in in vivo animal studies [19, 20, 28, 40].”
§ Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature > Discussion
What the sources say
- Supportedcites [19] Hair K, Macleod MR, Sena ES. A randomised controlled trial of an intervention to improve compliance with the ARRIVE guidelines (IICARus). Res Integrity Peer Rev. 2019;4(1). https://doi.org/10.1186/s41073-019-0069-3. source
The Discussion reports that requiring authors to complete an ARRIVE checklist at manuscript submission increased compliance with only one reporting subitem and did not improve full adherence. This supports the claim of a limited improvement in reporting for in vivo animal studies; this trial tested a submission-stage request, not a peer-review intervention. (LLM, conf=0.88)
- Partly supportedcites [20] Han S, et al. A checklist is associated with increased quality of reporting preclinical biomedical research: a systematic review. PLOS ONE. 2017;12(9):e0183591. https://doi.org/10.1371/journal.pone.0183591. sourceExperimental — accuracy not yet measured
The Results (Differences between Cell and Nature; page not available) report improved reporting of three items—randomization, blinding, and sample-size estimation—in in vivo animal studies after a checklist was required at submission. But the gains were large (for example, randomization reporting rose from 15% to 85%), not modest. The Abstract describes the checklist as associated with improvement and says it was likely responsible, so the claim’s unqualified causal wording also exceeds that evidence. (LLM, conf=0.88)
- Partly supportedcites [28] Leung V, et al. ARRIVE has not ARRIVEd: support for the ARRIVE (animal research: reporting of in vivo experiments) guidelines does not improve the reporting quality of papers in animal welfare, analgesia or anesthesia. PLOS ONE. 2018;13(5):e0197882. https://doi.org/10.1371/journal.pone.0197882. sourceExperimental — accuracy not yet measured
The Results (Sub-items; page unavailable) report improvements in a small number of reporting items, and the Discussion describes limited increases without a significant difference between journals supporting ARRIVE and those not supporting it. The Discussion also reports improved reporting after a mandatory manuscript checklist, but attributes that finding to another study and describes a substantial increase for sample-size calculation. This source therefore does not establish that guidelines or checklists provided during review or preparation caused only modest improvements. (LLM, conf=0.85)
- Disputedcites [40] The NPQIP Collaborative group. Did a change in Nature journals’ editorial policy for life sciences research improve reporting? BMJ Open Sci. 2019;3(1):e000035. https://doi.org/10.1136/bmjos-2017-000035. sourceExperimental — accuracy not yet measured
The Background and Intervention passages confirm that Nature journals required a checklist during manuscript revision. But the Results (Reporting of the Landis 4 items; page unavailable) report large increases in reporting of several in vivo items, and the Conclusions describe the improvements as substantial—not modest improvements limited to a few items. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
plainstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There were larger differences favoring peer-reviewed articles in subjective ratings of how clearly titles and abstracts presented the main findings and how easy it was to locate relevant reporting information.”
§ Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature > Abstract > Results
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Though there was no clear evidence that individual evaluators changed their scoring over time”
§ Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature > Methods > Data collection
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Changes in reporting from preprints to peer-reviewed versions did not correlate with the impact factor of the publication venue or with the time lag from bioRxiv to journal publication.”
§ Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature > Abstract > Results
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the equivalence test (with an estimated power of 90% to detect equivalence at ±5% with α = 0.05) failed to provide statistical evidence for equivalence at the ±5% bound”
§ Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature > Methods > Data collection
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
89 more claims
Create a free account to see all 167 claims and to verify the 17 references still open.
- Probably not a substantial claim. 48
- It may state two separate claims as one. 41
- The quote only partly states this claim. 1
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Furthermore, living reviews of evolving topics may be less reliable than traditional SRs.”
From A narrative review of recent tools and innovations toward automating living systematic reviews and evidence syntheses (2023) · cites this paper as [6]
In the citing paper
“Readers of articles from pre-print servers will need to be aware that a pre-print may not follow high standards in terms of clarity, quality of reporting or availability of data, due to a lack of quality-control from an established publisher [6,38].”
From A narrative review of recent tools and innovations toward automating living systematic reviews and evidence syntheses (2023) · cites this paper as [6]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature > Results > Subjective assessment
“For clarity of abstract, we found a difference of 0.4, 95% C.I. [0.2–0.6] (t = 3.61, p = 4.2 × 10− 3, Student’s t-test)”
§ Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature > Results > Subjective assessment
“Student’s t-test: t = 3.61, p = 0.0004.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature > Results > Correlations between formatting and reporting score
“Preprints assessed in both stages of the study were included only once for this analysis, with the mean of reporting scores from both assessments. Mean ± S.D.: Non-embedded = 64.6 ± 10.9, n = 59; Embedded = 70.8 ± 11.6, n = 26.”
§ Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature > Results > Evaluation of articles
“Of the 76 preprints analyzed in the first stage, 49 had been published by Dec. 31st, 2018. Of these, 43 were included in the paired comparison. Additionally, 13 preprints that met inclusion criteria but were not included in the independent samples comparison (due to lack of articles in the same category in the PubMed sample) were included in the paired comparison.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| Paired sample | Title and abstract | 1 ± 0 | 85.7 ± 35.3 | 83.9 ± 37.1 | 1.8 [−8.0, 4.4] | 0.57 | 56 |”
§ Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature > Results > Reporting scores by category of articles
Reported 1.8 · recomputed -1.8
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature > Results > Subjective assessment
“Fig. 6d; 0.4, 95% C.I. [0.2, 0.6]; t = 4.11, p = 1.3 × 10− 3, paired t-test”
§ Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature > Results > Subjective assessment
“Paired t-test: t = 4.12, p = 0.0001.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature > Results > Subjective assessment
“Fig. 7b; r = 0.25, 95% C.I. [− 0.01, 0.48], p = 0.06, Pearson’s correlation”
§ Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature > Results > Subjective assessment
“r = 0.25, 95% C.I. [− 0.01, 0.68], p = 0.06, n = 56.”
Citations
Cites
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020 · cited 2×
“Questions were based on existing reporting guidelines (Moher et al., 2001; von Elm et al., 2007; Kilkenny et al., 2010; Bossuyt et al., 2015), journal checklists (Nature, 2013) and previous studies on quality of reporting (Hair, Macleod and Sena, 2019; The NPQIP Collaborative group, 2019), and are…”
BackgroundMethodImproving Bioscience Research Reporting: The ARRIVE Guidelines for Reporting Animal Research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · PLoS Biology · doi:10.1371/journal.pbio.1000412 · Cited by 7,300
The ASA Statement on p -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306
Improving bioscience research reporting: The ARRIVE guidelines for reporting animal research
Kilkenny, C., Browne, W. J., Cuthill, I. C., Emerson, M., & Altman, D. G. (2010) · Journal of Pharmacology and Pharmacotherapeutics · doi:10.4103/0976-500x.72351 · Cited by 6,277
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
STARD 2015: an updated list of essential items for reporting diagnostic accuracy studies
Bossuyt, P. M., Reitsma, J. B., Bruns, D. E., Gatsonis, C. A., Glasziou, P. P., Irwig, L., Lijmer, J. G., Moher, D., Rennie, D., de Vet, H. C. W., et al. (2015) · BMJ · doi:10.1136/bmj.h5527 · Cited by 3,660
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · BMC Medical Research Methodology · doi:10.1186/1471-2288-1-2 · Cited by 2,755 · cited 3×
“Questions were based on existing reporting guidelines (Moher et al., 2001; von Elm et al., 2007; Kilkenny et al., 2010; Bossuyt et al., 2015), journal checklists (Nature, 2013) and previous studies on quality of reporting (Macleod et al., 2017; Hair et al., 2018), and are presented along with their…”
MethodMoher, D. (2001) · JAMA · doi:10.1001/jama.285.15.1987 · Cited by 2,545
Publication prejudices: An experimental study of confirmatory bias in the peer review system
Mahoney, M. J. (1977) · Cognitive Therapy and Research · doi:10.1007/bf01173636 · Cited by 1,013 · cited 2×
“Additionally, traditional peer review has various drawbacks (Walker and da Silva, 2015), including reviewer bias (Mahoney, 1977; Murray et al., 2018), lack of agreement among reviewers (Rothwell and Martyn, 2000; Pier et al., 2018) and vulnerability to various forms of system gaming such as…”
BackgroundBohannon, J. (2013) · Science · doi:10.1126/science.342.6154.60 · Cited by 934 · cited 2×
“…et al., 2018), lack of agreement among reviewers (Rothwell and Martyn, 2000; Pier et al., 2018) and vulnerability to various forms of system gaming such as ‘lottery behavior’ by authors (Ioannidis et al., 2010), predatory journals (Bohannon, 2013) and self-peer-review scams (Ferguson et al., 2014).”
Research works printed but not linked: 2
- Anaya, J. (2016) ‘bioRxiv vs. PeerJ preprints’, Medium. Available at: https://medium.com/@OmnesRes/biorxiv-vs-peerj-preprints-f7589141c532 (Accessed: 14 Jan 2019).
- Inglis, J. R. and Sever, R. (2016) ‘bioRxiv: a progress report’, ASAPbio blog. Available at: http://asapbio.org/biorxiv (Accessed: 14 Jan 2019).
Cited by
Fraser, N., Brierley, L., Dey, G., Polka, J. K., Pálfy, M., Nanni, F., & Coates, J. A. (2021) · PLOS Biology · doi:10.1371/journal.pbio.3000959 · Cited by 555
“Recent studies have suggested that the quality of reporting in preprints differs little from their later peer-reviewed articles [49], and we ourselves are currently undertaking a more detailed analysis.”
BackgroundThe limitations to our understanding of peer review
Tennant, J. P., & Ross-Hellauer, T. (2020) · Research Integrity and Peer Review · doi:10.1186/s41073-020-00092-1 · Cited by 384
Medical Education During the COVID-19 Pandemic
Kaul, V., Gallo de Moraes, A., Khateeb, D., Greenstein, Y., Winter, G., Chae, J., Stewart, N. H., Qadir, N., & Dangayach, N. S. (2021) · Chest · doi:10.1016/j.chest.2020.12.026 · Cited by 220
Correlates of turnover intention among nursing staff in the COVID-19 pandemic: a systematic review
Tolksdorf, K. H., Tischler, U., & Heinrichs, K. (2022) · BMC Nursing · doi:10.1186/s12912-022-00949-4 · Cited by 170
Sunjaya, A. P., Allida, S. M., Tanna, G. L. D., & Jenkins, C. R. (2021) · Journal of Asthma · doi:10.1080/02770903.2021.1888116 · Cited by 137
Metadata
- Authors
- Carneiro, C. F. D., Queiroz, V. G. S., Moulin, T. C., Carvalho, C. A. M., Haas, C. B., Rayêe, D., Henshall, D. E., De-Souza, E. A., Amorim, F. E., Boos, F. Z., Guercio, G. D., Costa, I. R., Hajdu, K. L., van Egmond, L., Modrák, M., Tan, P. B., Abdill, R. J., Burgess, S. J., Guerra, S. F. S., Bortoluzzi, V. T., & Amaral, O. B.
- Affiliations
- Universidade Federal do Rio de Janeiro (BR)
- Universidade do Estado do Pará (BR)
- Universidade Federal do Rio Grande do Sul (BR)
- University of Edinburgh (GB)
- Universidade Federal de São Paulo (BR)
- University of Minnesota (US)
- Czech Academy of Sciences, Institute of Microbiology (CZ)
- University of Illinois Urbana-Champaign (US)
- Corresponding author
- Carneiro, Clarissa F. D.
- ORCID
- Venue
- Research Integrity and Peer Review
- Published
- 2020-12
- Volume
- 5
- Issue
- 1
- Pages
- 16-16
- ISSN
- 2058-8615 (ISSN-L)
- PMC
- PMC7706207
- OpenAlex
- W3110179196
- Semantic Scholar
- 1d16e6a08e34f7862485bfef99c64b403543736a
- Cited by
- 146
- References
- 47
- Citations by year
- 2026: 7 · 2025: 12 · 2024: 30 · 2023: 31 · 2022: 34 · 2021: 25 · 2020: 4
- FWCI
- 23.31
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0/ (tdm)
- creativecommons.org/licenses/by/4.0/ (vor)
- Locations
- Research Integrity and Peer Review (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Uppsala University Publications (Uppsala University) (acceptedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Fundação Carlos Chagas Filho de Amparo à Pesquisa do Estado do Rio de Janeiro
- Conselho Nacional de Desenvolvimento Científico e Tecnológico
- Programa Institucional de Bolsas de Iniciação Científica - UFRJ
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Keywords
- preprints, bioRxiv, peer-reviewed articles, reporting quality, reporting completeness, transparency, reproducibility, biomedical literature
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-05, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-05 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature. https://app.cochat.ai/papers/10.1186/s41073-020-00101-3
- The paper
- Carneiro, C. F. D., Queiroz, V. G. S., Moulin, T. C., Carvalho, C. A. M., Haas, C. B., Rayêe, D., Henshall, D. E., De-Souza, E. A., Amorim, F. E., Boos, F. Z., Guercio, G. D., Costa, I. R., Hajdu, K. L., van Egmond, L., Modrák, M., Tan, P. B., Abdill, R. J., Burgess, S. J., Guerra, S. F. S., . . . Amaral, O. B. (2020). Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature. Research Integrity and Peer Review. https://doi.org/10.1186/s41073-020-00101-3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.