The DOI matches its Crossref record; 63 of the 63 references it cites were found in a registry or an index; 14 of the 32 sources its claims cite were checked for support.
Systematic review and meta-analysis of quotation inaccuracy in medicine
Open accessAbstract
BACKGROUND: Quotations are crucial to science but have been shown to be often inaccurate. Quotation errors, that is, a reference not supporting the authors' claim, may still be a significant issue in scientific medical writing. This study aimed to examine the quotation error rate and trends over time in the medical literature. METHODS: A systematic search of PubMed, Web of Science, and reference lists for quotation error studies in medicine and without date or language restrictions identified 46 studies analyzing 32,000 quotations/references. Literature search, data extraction, and risk of bias assessments were performed independently by two raters. Random-effects meta-analyses and meta-regression were used to analyze error rates and trends (protocol pre-registered on OSF). RESULTS: 16.9% (95% CI: 14.1%-20.0%) of quotations were incorrect, with approximately half classified as major errors (8.0% [95% CI: 6.4%-10.0%]). Heterogeneity was high, and Egger's test for small study effects remained negative throughout. Meta-regression showed no significant improvement in quotation accuracy over recent years (slope: -0.002 [95% CI: -0.03 to 0.02], p = 0.85). Neither risk of bias, nor the number of references were statistically significantly associated with total error rate, but journal impact factor was: Spearman's ρ = -0.253 (p = 0.043, binomial test, N = 25). CONCLUSIONS: Quotation errors remain a problem in the medical literature, with no improvement over time. Addressing this issue requires concerted efforts to improve scholarly practices and editorial processes.
Overview
CoChat Summary
## TL;DR
This systematic review and meta-analysis found that quotation errors remain common in medicine: 16.9% of quotations were incorrect, and about half of these were major errors. The error rate showed no significant improvement over time.
## Key findings
- Across 46 studies and 32,074 quotations, the summary estimate of total quotation error rate was 16.9% (95% CI 14.1%–20.0%).
- Of the quotation errors, 8.0% (95% CI 6.4%–10.0%) were major errors and 7.8% (95% CI 5.7%–10.5%) were minor errors; secondary quotations were 5.3% (95% CI 3.3%–8.5%) across 18 studies.
- Meta-regression found no significant improvement over time for total errors (slope −0.002, 95% CI −0.03 to 0.02, p = 0.85), and similarly none for major, minor, or secondary errors.
- Journal impact factor was the only reported predictor associated with total error rate: Spearman’s ρ = −0.253 (p = 0.043, binomial test, N = 25); risk of bias and number of references were not statistically significant.
## Methods
- Systematic review and meta-analysis of studies on quotation error rates in medicine, with no date or language restrictions; searches were run in PubMed, Web of Science, and reference lists, and the protocol was pre-registered on OSF.
- Included 46 studies; the paper reports 32,074 quotations/references analyzed. Two reviewers independently performed screening, data extraction, and risk-of-bias assessment.
- Pooled estimates were calculated using random-effects meta-analysis; meta-regression assessed time trends, and subgroup analyses examined risk of bias, medical specialty, and journal impact factor.
## Limitations
- The paper states there is no gold standard for assessing quotation errors, so some judgment is involved in classifying errors.
- The authors note they may have missed studies (for example, some Chinese papers were not retrievable), though they also say there were no language restrictions and the base of 46 studies was solid.
- Most included studies focused on quotations within original peer-reviewed research articles; reviews, editorials, grey literature, and quotation errors from single-source papers or non-medicine fields were not systematically analyzed, limiting generalizability.
- The text makes evident that high heterogeneity was present throughout the meta-analyses, so the exact pooled point estimates should be interpreted cautiously.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures


Verification
Claims the paper makes
11 Valid · 0 Not valid · 4 Not checked · 49 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“When we re-calculated a summary estimate restricted to the 28 studies used in our earlier meta-analysis [8] but with all imputations employed in the current analysis we arrived at a total error rate of 17.0% [13.0–21.9].”
§ Systematic review and meta-analysis of quotation inaccuracy in medicine > Results > Subgroup analyses
What the sources say
- Supportedcites [8] Jergas H, Baethge C. Quotation accuracy in medical journal articles-a systematic review and meta-analysis. PeerJ. 2015;3: e1364. source
The citation identifies the earlier meta-analysis and its set of 28 studies; the citing authors describe the 17.0% estimate as their own recalculation, not a result reported by this source. The source’s Results (Main analysis; page unavailable) confirm that its main analysis included 28 studies and report its original total-error estimate as 25.4%. (LLM, conf=0.93)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption not shown here.
- Supported
Every source the paper cites for this claim was checked and supports it.
In the paper
“De Lacey et al. reported substantial error rates in major medical journals [2].”
§ Systematic review and meta-analysis of quotation inaccuracy in medicine > Introduction
What the sources say
- Supportedcites [2] de Lacey G, Record C, Wade J. How accurate are quotations and references in medical journals? Br Med J (Clin Res Ed). 1985;291:884–6. source
The Results report misquotation rates of 10%–20% across the six medical journals studied. The Discussion gives an overall rate of 15% of references, including 12% with misleading errors, supporting the cited statement about substantial quotation error rates. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“the most comprehensive systematic review and meta-analysis [8] included studies published until 2014.”
§ Systematic review and meta-analysis of quotation inaccuracy in medicine > Introduction
What the sources say
- Partly supportedcites [8] Jergas H, Baethge C. Quotation accuracy in medical journal articles-a systematic review and meta-analysis. PeerJ. 2015;3: e1364. sourceExperimental — accuracy not yet measured
The Methods (Literature search) say the review searched through December 26, 2014. But the Results say the included studies appeared between 1985 and 2013, not that studies published in 2014 were included. The copy also does not establish that this was the most comprehensive review. (LLM, conf=0.88)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“In the meantime, new studies have been published e.g., [3, 4, 10].”
§ Systematic review and meta-analysis of quotation inaccuracy in medicine > Introduction
What the sources say
- Not checked — no full text of the sourcecites [3] Choi J, Gupta A, Kaghazchi A, Htwe TS, Baiocchi M, Spain DA. Citation Inaccuracies in Influential Surgical Journals. JAMA Surg. 2021;156:791–2. source
- Supportedcites [4] Hui A, Rains LS, Todd A, Boaz A, Johnson S. The accuracy and accessibility of cited evidence: a study examining mental health policy documents. Soc Psychiatry Psychiatr Epidemiol. 2020;55:111–21. source
The Discussion reports the present study’s 40.3% accuracy error rate, and the Conclusion describes its investigation of referencing accuracy in mental health policy documents. This paper is therefore an example of a study published since 2014 on quotation or citation errors. (LLM, conf=0.93)
- Not checked — no full text of the sourcecites [10] Baethge C. Importance, Errors, and Patterns of Quotations to Psychiatric Original Articles. Pharmacopsychiatry. 2020;53:247–55. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Every source the paper cites for this claim was checked and supports it.
In the paper
“we included 18 reports on 18 new studies [3, 4, 10, 23–38], for a total of 46 studies – together with 28 studies [2, 3, 5, 6, 16, 17, 17, 39–45, 45–60] from our systematic review published in 2015 [8] – that form the basis of this meta-analysis”
§ Systematic review and meta-analysis of quotation inaccuracy in medicine > Results
What the sources say
- Supportedcites [8] Jergas H, Baethge C. Quotation accuracy in medical journal articles-a systematic review and meta-analysis. PeerJ. 2015;3: e1364. source
The Results section states that the 2015 review included 28 studies in its main analysis. The citing sentence attributes those 28 studies to this source; its total of 46 also includes 18 new studies in the citing authors’ meta-analysis. (LLM, conf=0.94)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
31 more claims · 3 with their cited sources checked
Create a free account to see all 64 claims and every source verdict.
- Probably not a substantial claim. 16
- It may state two separate claims as one. 16
Citations
Cites
The PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Shea, B. J., Reeves, B. C., Wells, G., Thuku, M., Hamel, C., Moran, J., Moher, D., Tugwell, P., Welch, V., Kristjansson, E., et al. (2017) · BMJ · doi:10.1136/bmj.j4008 · Cited by 10,636 · cited 1×
“As recommended by PRISMA 2020 [15], the PRISMA-S extension [18], and the AMSTAR-2 [19] tool for systematic reviews and meta-analyses, we included grey literature (e.g., dissertations, conference abstracts) to minimize publication bias and enhance transparency.”
Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., Koffel, J. B., PRISMA-S Group, Blunt, H., Brigham, T., et al. (2021) · Systematic Reviews · doi:10.1186/s13643-020-01542-z · Cited by 3,714 · cited 1×
“As recommended by PRISMA 2020 [15], the PRISMA-S extension [18], and the AMSTAR-2 [19] tool for systematic reviews and meta-analyses, we included grey literature (e.g., dissertations, conference abstracts) to minimize publication bias and enhance transparency.”
Rethlefsen, M. L., Kirtley, S., Waffenschmidt, S., Ayala, A. P., Moher, D., Page, M. J., & Koffel, J. (2019) · doi:10.31219/osf.io/sfc38 · Cited by 3,696
Munn, Z., Moola, S., Lisy, K., Riitano, D., & Tufanaru, C. (2015) · International Journal of Evidence-Based Healthcare · doi:10.1097/xeb.0000000000000054 · Cited by 3,131 · cited 1×
“Absent a risk of bias tool specifically designed for original research on quotation inaccuracy, we revised the Joanna Briggs Institute (JBI) Critical Appraisal Checklist for Studies Reporting Prevalence Data [20] for the purposes of this study (eTable 3).”
A comparison of heterogeneity variance estimators in combining results of studies
Sidik, K., & Jonkman, J. N. (2007) · Statistics in Medicine · doi:10.1002/sim.2688 · Cited by 380
How accurate are quotations and references in medical journals?
de Lacey, G., Record, C., & Wade, J. (1985) · BMJ · doi:10.1136/bmj.291.6499.884 · Cited by 160 · cited 3×
“And yet, in an early investigation into their accuracy, De Lacey et al. reported substantial error rates in major medical journals [2].”
When Is There a Group that Knows?
(2014) · doi:10.1093/acprof:oso/9780199665792.003.0003 · Cited by 139
Research works printed but not linked: 4
- Zhang, Ju (张菊), Zhong, Junxing (钟均行). Analysis of Reference Citation Errors in Medical Journals [医学期刊中参考文献引用错误分析 ]. 2005. Chinese Journal of Scientific and Technical Periodicals;16:845–7.
- Zhu, Hongmei, Wang, Jie, Zhong, Yingru, Sun, Yuhang, Hu, Yongguo, Zhang, Dazhi, et al. Quotation accuracy and inappropriate quotation of the references in articles published in medical journals with high impact factors. Chinese Journal of Scientific and Technical Periodicals. 2018;Issue 10:1029–35.
- Trost L., Mulhall J.P. Frequency of citation errors among male sexual medicine publications in high-impact urology journals. The Journal of Urology. 4S:e330–1.
- Warren KJ, Bhatia N, Teh W, Fleming M, Lange M. Reference and quotation accuracy in the major and minor infectious diseases journals. Third international congress on biomedical peer review and global communications: Prague, Czech Republic; 1997.
Cited by
Hallucinated citations are polluting the scientific literature. What can be done?
Naddaf, M., & Quill, E. (2026) · Nature · doi:10.1038/d41586-026-00969-z · Cited by 36
Resnik, D. B., & Hosseini, M. (2026) · Accountability in Research · doi:10.1080/08989621.2026.2645390 · Cited by 26
“According to a recent meta-analysis (Baethge and Jergas 2025), almost 17% of all quotations in medical publications have inaccuracies, and roughly half of those inaccuracies involve major errors (i.e., “the claim is not at all supported by the source”).”
de Waal, A. (2026) · Journal of Engineering Management and Competitiveness · doi:10.5937/jemc2600001w · Cited by 2
Тихонова, Е. В. (2026) · Хранение и переработка сельхозсырья · doi:10.36107/spfp.2025.4.699
A scoping review of guides and advice for scholarly citing in health research
Østengaard, L., Tjørnhøj‐Thomsen, T., Paludan‐Müller, A. S., Elkjær, M., Frandsen, T. F., & Hróbjartsson, A. (2026) · Research Integrity and Peer Review · doi:10.1186/s41073-026-00195-1
“However, citations are not always carefully selected, and sometimes, it can be questioned whether the cited papers were adequately read due to the inaccurate representation of information [1, 2].”
Metadata
- Authors
- Baethge, C., & Jergas, H.
- Affiliations
- University of Cologne (DE)
- Corresponding author
- Baethge, Christopher
- Venue
- Research Integrity and Peer Review
- Published
- 2025-07-23
- Volume
- 10
- Issue
- 1
- Pages
- 13-13
- ISSN
- 2058-8615 (ISSN-L)
- PMC
- PMC12285159
- OpenAlex
- W4412565252
- Cited by
- 11
- References
- 60
- Citations by year
- 2026: 11
- FWCI
- 1.70
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- creativecommons.org/licenses/by/4.0 (tdm)
- creativecommons.org/licenses/by/4.0 (vor)
- Locations
- Research Integrity and Peer Review (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Kölner Universitäts PublikationsServer (Universität zu Köln) (submittedVersion, other-oa)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Chancen ergreifen, Forschung und Familie fördern – Programm für chancengerechte Hochschulmedizin in Nordrhein-Westfalen
- Universitätsklinikum Köln
- Topics
- Social Sciences › Education › Academic integrity and plagiarism
- Social Sciences › Library and Information Sciences › Academic Writing and Publishing
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- quotation errors, citation accuracy, medical literature, scientific writing, error rate, journal impact factor, editorial practice, research integrity, bibliographic references
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Systematic review and meta-analysis of quotation inaccuracy in medicine. https://app.cochat.ai/papers/10.1186/s41073-025-00173-z
- The paper
- Baethge, C., & Jergas, H. (2025). Systematic review and meta-analysis of quotation inaccuracy in medicine. Research Integrity and Peer Review. https://doi.org/10.1186/s41073-025-00173-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.