Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index; 14 of the 32 sources its claims cite were checked for support.

Systematic review and meta-analysis of quotation inaccuracy in medicine

Open access
Baethge, Christopher, Jergas, Hannah · 2025 · Research Integrity and Peer Review

Abstract

BACKGROUND: Quotations are crucial to science but have been shown to be often inaccurate. Quotation errors, that is, a reference not supporting the authors' claim, may still be a significant issue in scientific medical writing. This study aimed to examine the quotation error rate and trends over time in the medical literature. METHODS: A systematic search of PubMed, Web of Science, and reference lists for quotation error studies in medicine and without date or language restrictions identified 46 studies analyzing 32,000 quotations/references. Literature search, data extraction, and risk of bias assessments were performed independently by two raters. Random-effects meta-analyses and meta-regression were used to analyze error rates and trends (protocol pre-registered on OSF). RESULTS: 16.9% (95% CI: 14.1%-20.0%) of quotations were incorrect, with approximately half classified as major errors (8.0% [95% CI: 6.4%-10.0%]). Heterogeneity was high, and Egger's test for small study effects remained negative throughout. Meta-regression showed no significant improvement in quotation accuracy over recent years (slope: -0.002 [95% CI: -0.03 to 0.02], p = 0.85). Neither risk of bias, nor the number of references were statistically significantly associated with total error rate, but journal impact factor was: Spearman's ρ = -0.253 (p = 0.043, binomial test, N = 25). CONCLUSIONS: Quotation errors remain a problem in the medical literature, with no improvement over time. Addressing this issue requires concerted efforts to improve scholarly practices and editorial processes.

Overview

CoChat Summary

## TL;DR

This systematic review and meta-analysis found that quotation errors remain common in medicine: 16.9% of quotations were incorrect, and about half of these were major errors. The error rate showed no significant improvement over time.

## Key findings

  • Across 46 studies and 32,074 quotations, the summary estimate of total quotation error rate was 16.9% (95% CI 14.1%–20.0%).
  • Of the quotation errors, 8.0% (95% CI 6.4%–10.0%) were major errors and 7.8% (95% CI 5.7%–10.5%) were minor errors; secondary quotations were 5.3% (95% CI 3.3%–8.5%) across 18 studies.
  • Meta-regression found no significant improvement over time for total errors (slope −0.002, 95% CI −0.03 to 0.02, p = 0.85), and similarly none for major, minor, or secondary errors.
  • Journal impact factor was the only reported predictor associated with total error rate: Spearman’s ρ = −0.253 (p = 0.043, binomial test, N = 25); risk of bias and number of references were not statistically significant.

## Methods

  • Systematic review and meta-analysis of studies on quotation error rates in medicine, with no date or language restrictions; searches were run in PubMed, Web of Science, and reference lists, and the protocol was pre-registered on OSF.
  • Included 46 studies; the paper reports 32,074 quotations/references analyzed. Two reviewers independently performed screening, data extraction, and risk-of-bias assessment.
  • Pooled estimates were calculated using random-effects meta-analysis; meta-regression assessed time trends, and subgroup analyses examined risk of bias, medical specialty, and journal impact factor.

## Limitations

  • The paper states there is no gold standard for assessing quotation errors, so some judgment is involved in classifying errors.
  • The authors note they may have missed studies (for example, some Chinese papers were not retrievable), though they also say there were no language restrictions and the base of 46 studies was solid.
  • Most included studies focused on quotations within original peer-reviewed research articles; reviews, editorials, grey literature, and quotation errors from single-source papers or non-medicine fields were not systematically analyzed, limiting generalizability.
  • The text makes evident that high heterogeneity was present throughout the meta-analyses, so the exact pooled point estimates should be interpreted cautiously.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s41073-025-00173-z

Cites

63 of 63 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Forest plot of total quotation errors. Legend: Forest plot of total quotation error numbers in all studies included. Horizontal lines indicate 95% confidence intervals, and black boxes denote the mean. The black diamond at the bottom shows the summary effect with 95% confidence intervals
Fig. 1 Forest plot of total quotation errors. Legend: Forest plot of total quotation error numbers in all studies included. Horizontal lines indicate 95% confidence intervals, and black boxes denote the mean. The black diamond at the bottom shows the summary effect with 95% confidence intervalsCC BY · PubMed Central
Fig. 2 Bubble plot: Meta-regression of time of observation on total quotation errors. Legend: Regression of observation period (in years) against the logit-transformed event rate based on the total quotation error rates reported in each study. Each circle represents a study. The y-axis denotes the logit-transformed event rate, while the x-axis represents the year of the observation period (or the mean in case of more than one year). The horizontal trend line indicates no significant association between the observation period and the logit event rate, suggesting no improvement or decline in event rates over time
Fig. 2 Bubble plot: Meta-regression of time of observation on total quotation errors. Legend: Regression of observation period (in years) against the logit-transformed event rate based on the total quotation error rates reported in each study. Each circle represents a study. The y-axis denotes the logit-transformed event rate, while the x-axis represents the year of the observation period (or the mean in case of more than one year). The horizontal trend line indicates no significant association between the observation period and the logit event rate, suggesting no improvement or decline in event rates over timeCC BY · PubMed Central

Verification

Claims the paper makes

11 Valid · 0 Not valid · 4 Not checked · 49 Own results and interpretations

31 more claims · 3 with their cited sources checked

Create a free account to see all 64 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 16
  • It may state two separate claims as one. 16

Create a free account to see these claims.

Citations

Cites

Show all 60 references

Research works printed but not linked: 4

  1. Zhang, Ju (张菊), Zhong, Junxing (钟均行). Analysis of Reference Citation Errors in Medical Journals [医学期刊中参考文献引用错误分析 ]. 2005. Chinese Journal of Scientific and Technical Periodicals;16:845–7.
  2. Zhu, Hongmei, Wang, Jie, Zhong, Yingru, Sun, Yuhang, Hu, Yongguo, Zhang, Dazhi, et al. Quotation accuracy and inappropriate quotation of the references in articles published in medical journals with high impact factors. Chinese Journal of Scientific and Technical Periodicals. 2018;Issue 10:1029–35.
  3. Trost L., Mulhall J.P. Frequency of citation errors among male sexual medicine publications in high-impact urology journals. The Journal of Urology. 4S:e330–1.
  4. Warren KJ, Bhatia N, Teh W, Fleming M, Lange M. Reference and quotation accuracy in the major and minor infectious diseases journals. Third international congress on biomedical peer review and global communications: Prague, Czech Republic; 1997.

Cited by

See all 11

Metadata

Authors
Baethge, C., & Jergas, H.
Affiliations
University of Cologne (DE)
Corresponding author
Baethge, Christopher
Venue
Research Integrity and Peer Review
Published
2025-07-23
Volume
10
Issue
1
Pages
13-13
ISSN
2058-8615 (ISSN-L)
OpenAlex
W4412565252
Cited by
11
References
60
Citations by year
2026: 11
FWCI
1.70
Type
journal article
Language
en
Open access
Licence
Locations
  • Research Integrity and Peer Review (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Kölner Universitäts PublikationsServer (Universität zu Köln) (submittedVersion, other-oa)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • Chancen ergreifen, Forschung und Familie fördern – Programm für chancengerechte Hochschulmedizin in Nordrhein-Westfalen
  • Universitätsklinikum Köln
Topics
  • Social Sciences › Education › Academic integrity and plagiarism
  • Social Sciences › Library and Information Sciences › Academic Writing and Publishing
  • Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
Keywords
quotation errors, citation accuracy, medical literature, scientific writing, error rate, journal impact factor, editorial practice, research integrity, bibliographic references

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Systematic review and meta-analysis of quotation inaccuracy in medicine. https://app.cochat.ai/papers/10.1186/s41073-025-00173-z
The paper
Baethge, C., & Jergas, H. (2025). Systematic review and meta-analysis of quotation inaccuracy in medicine. Research Integrity and Peer Review. https://doi.org/10.1186/s41073-025-00173-z

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.