CoChat
Sign Up
Cited by 46scientometrics and bibliometrics researchAcademic Writing and PublishingAcademic integrity and plagiarism

How accurate are citations of frequently cited papers in biomedical literature?

Open access
Pavlovic, Vedrana, Weissgerber, Tracey, Stanisavljevic, Dejana, Pekmezovic, Tatjana, Milicevic, Ognjen, Lazovic, Jelena Milin, Cirkovic, Andja, Savic, Marko +12 more·2021·Clinical Science
DOI 10.1042/cs20201573PMC PMC8048031

Published version of a preprint

Open this paper in CoChat — free account

Abstract

Citations are an important, but often overlooked, part of every scientific paper. They allow the reader to trace the flow of evidence, serving as a gateway to relevant literature. Most scientists are aware of citations’ errors, but few appreciate the prevalence of these problems. The purpose of the present study was to examine how often frequently cited papers in biomedical scientific literature are cited inaccurately. The study included an active participation of the first authors of included papers; to first-hand verify the citations accuracy. Findings from feasibility study, where we reviewed 1540 articles containing 2526 citations of 14 most cited articles in which the authors were affiliated with the Faculty of Medicine University of Belgrade, were further evaluated for external confirmation in an independent verification set of articles. Verification set included 4912 citations identified in 2995 articles that cited 13 most cited articles published by authors affiliated with the Mayo Clinic Division of Nephrology and Hypertension. A citation was defined as being accurate if the cited article supported or was in accordance with the statement by citing authors. At least one inaccurate citation was found in 11 and 15% of articles in the feasibility study and verification set, respectively, suggesting that inaccurate citations are common in biomedical literature. The most common problem was the citation of nonexistent findings (38.4%), followed by an incorrect interpretation of findings (15.4%). One-fifth of inaccurate citations were due to chains of inaccurate citations. Based on these findings, several actions to reduce citation inaccuracies have been proposed.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1042/cs20201573

Cites

36 of 36 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Citation inaccuracies and distortions are common.

    “However, citation inaccuracies and distortions are common (Wager and Middleton 2008, Jergas and Baethge 2015, Pavlovic et al. 2021).”

    From Assessing citation integrity in biomedical publications: corpus annotation and NLP models (2024) · cites this paper as Pavlovic et al. 2021

  • Five studies provided numbers for a mean factor of 1.15 errors per index quotation error.

    “Five studies provided numbers, for a mean factor of 1.15 errors per index quotation error [17, 30, 58, 60], but many reported merely the first error of a quotation.”

    From Systematic review and meta-analysis of quotation inaccuracy in medicine (2025) · cites this paper as [30]

    The Results and Discussion report rates and types of inaccurate citations, but do not give a count of errors per inaccurate quotation. Searches for the 1.15 factor and multiple errors per citation found no passage establishing that figure or this paper’s contribution to the five-study calculation. Page numbers were not provided in our copy. (LLM, conf=0.72)

    Experimental — accuracy not yet measured

  • Traditional peer review often fails to address widely accepted, but suboptimal, research practices and guideline details.

    “Traditional peer review often fails to address widely accepted, but suboptimal, research practices and guideline details.”

    From Is the future of peer review automated? (2022) · cites this paper as [29]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 37 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Pavlovic, V., Weissgerber, T., Stanisavljevic, D., Pekmezovic, T., Milicevic, O., Lazovic, J. M., Cirkovic, A., Savic, M., Rajovic, N., Piperac, P., Djuric, N., Madzarevic, P., Dimitrijevic, A., Randjelovic, S., Nestorovic, E., Akinyombo, R., Pavlovic, A., Ghamrawi, R., Garovic, V., & Milic, N.
Affiliations
  • University of Belgrade (RS)
  • Berlin Institute of Health at Charité - Universitätsmedizin Berlin (DE)
  • Institute for Medical Statistics and Informatics, Faculty of Medicine, University of Belgrade, Belgrade, Serbia
  • Mayo Clinic (US)
Corresponding author
Milic, Natasa
ORCID
Venue
Clinical Science
Publisher
Portland Press Ltd.
Published
2021-03
Volume
135
Issue
5
Pages
671-681
ISSN
  • 0143-5221 (ISSN-L)
  • 1470-8736
DOI
10.1042/cs20201573
PMC
PMC8048031
OpenAlex
W3131199412
Cited by
46
References
37
Citations by year
2026: 5 · 2025: 13 · 2024: 7 · 2023: 11 · 2022: 5 · 2021: 5
FWCI
4.06
Type
journal article
Language
en
Open access
Licence
Locations
  • Clinical Science (publishedVersion, cc-by)
  • bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
  • National Repository of Dissertations in Serbia (submittedVersion, other-oa)
Funders
National Heart, Lung, and Blood Institute (R01HL136348)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Academic Writing and Publishing
  • Social Sciences › Safety Research › Academic integrity and plagiarism
Keywords
citation errors, citation accuracy, biomedical literature, citation verification, scientific integrity
MeSH terms
Data Accuracy, Periodicals as Topic, Bibliometrics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review