The DOI matches its Crossref record; 54 of the 55 references it cites were found in a registry or an index.
Self-correction of science: a comparative study of negative citations and post-publication peer review
Open accessAbstract
This study investigates whether negative citations in articles and comments posted on post-publication peer review platforms are both equally contributing to the correction of science. These 2 types of written evidence of disputes are compared by analyzing their occurrence in relation to articles that have already been retracted or corrected. We identified retracted or corrected articles in a corpus of 72,069 articles coming from the Engineering field, from 3 journals (Science, Tumor Biology, Cancer Research) and from 3 authors with many retractions to their credit (Sarkar, Schön, Voinnet). We used Scite to retrieve contradicting citations and PubPeer to retrieve the number of comments for each article, and then we considered them as traces left by scientists to contest published results. Our study shows that contradicting citations are very uncommon and that retracted or corrected articles are not more contradicted in scholarly articles than those that are neither retracted nor corrected but they do generate more comments on Pubpeer, presumably because of the possibility for contributors to remain anonymous. Moreover, post-publication peer review platforms, although external to the scientific publication process contribute more to the correction of science than negative citations. Consequently, post-publication peer review venues, and more specifically the comments found on it, although not contributing to the scientific literature, are a mechanism for correcting science. Lastly, we introduced the idea of strengthening the role of contradicting citations to rehabilitate the clear expression of judgment in scientific papers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Bordignon’s (2020) finding echoes ours in that retracted papers are not cited differently before and after the retraction.”
From Continued use of retracted papers: Temporal trends in citations and (lack of) awareness of retractions shown in citation contexts in biomedicine (2021) · cites this paper as Bordignon (2020)
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A number of researchers have already made use of scite for quantitative assessments of the literature.”
From scite: a smart citation index that displays the context of citations and classifies their intent using deep learning (2021) · cites this paper as [45]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Interestingly, Bordignon (2020) reported that retracted/corrected papers did not receive more negative citations than nonretracted/corrected papers.”
From Continued use of retracted papers: Temporal trends in citations and (lack of) awareness of retractions shown in citation contexts in biomedicine (2021) · cites this paper as Bordignon (2020)
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
Retracted Science and the Retraction Index
Fang, F. C., & Casadevall, A. (2011) · Infection and Immunity · doi:10.1128/iai.05661-11 · Cited by 405
RetractedWhy Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
The Logic of Scientific Discovery.
Hutchison, T. W., & Popper, K. R. (1959) · Economica · doi:10.2307/2550489 · Cited by 7,499
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“Although (Ioannidis 2005) claims that "most research findings are false for most research designs and most fields", retracted and corrected papers are very scarce.”
BackgroundPrice, D. J. D. S. (1963) · Columbia University Press eBooks · doi:10.7312/pric91844 · Cited by 4,264 · cited 2×
“De Solla Price (1963) discusses the phenomenon of cumulation of papers (“the way in which each paper is built on a foundation of previous papers then, in turn, is one of several points of departure for the next”) and stated that “the most obvious manifestation of this scholarly bricklaying is the…”
BackgroundBibliometrics: The Leiden Manifesto for Research Metrics
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015) · Nature · doi:10.1038/520429a · Cited by 2,828
Fanelli, D. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005738 · Cited by 1,911 · cited 1×
“The frontier is sometimes hard to establish but the intention to deceive is the key aspect (Bar-Ilan and Halevi 2018; Fanelli 2009).”
BackgroundSome Results on the Function and Quality of Citations
Moravcsik, M. J., & Murugesan, P. (1975) · Social Studies of Science · doi:10.1177/030631277500500106 · Cited by 693
Baumgartner, H., & Pieters, R. (2003) · Journal of Marketing · doi:10.1509/jmkg.67.2.123.18610 · Cited by 504
Automatic classification of citation function
Teufel, S., Siddharthan, A., & Tidhar, D. (2006) · Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing - EMNLP '06 · doi:10.3115/1610075.1610091 · Cited by 437 · cited 1×
“(Teufel et al. 2006) Weakness of cited approach; Contrast/Comparison in Goals or Methods (neutral); Author’s work is stated to be superior to cited work; Contrast/Comparison in Results (neutral); Contrast between 2 cited methods 360 conference articles (comp.”
Background
Research works printed but not linked: 1
- Garfield, E. (1964). Can citation indexing be automated? In Statistical association methods for mechanized documentation: Symposium proceedings, Washington 1964. (National Bureau of Standards Mliscellaneous Publication 269) (pp. 189–192). Washington, DC: U.S.G.P.O. http://www.garfield.library.upenn.edu/essays/V1p084y1962-73.pdf. Accessed June 26, 2019.
Cited by
Nicholson, J. M., Mordaunt, M., Lopez, P., Uppala, A., Rosati, D., Rodrigues, N. P., Grabitz, P., & Rife, S. C. (2021) · bioRxiv (published in Quantitative Science Studies) · doi:10.1101/2021.03.15.435418 · Cited by 202
Hsiao, T.-K., & Schneider, J. (2021) · Quantitative Science Studies · doi:10.1162/qss_a_00155 · Cited by 98
“Bordignon ’ s (2020) finding echoes ours in that retracted papers are not cited differently before and after the retraction.”
Backgroundda Silva, J. A. T. (2022) · Journal for General Philosophy of Science · doi:10.1007/s10838-022-09607-4 · Cited by 60
“…but this dynamic can only occur when preprints and TPR co-exist alongside PPPR (Harms and Credé 2020), with a mechanism that allows for studies’ claims to be validated or challenged (Bordignon 2020) in order to, as effectively as possible, prevent predatory publishing practices (Yamada 2021).”
BackgroundClassification and analysis of PubPeer comments: How a web journal club is used
Ortega, J. L. (2021) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24568 · Cited by 52
“More recently, Bordignon (2020) used PubPeer comments to test the influence of negative comments in the retraction/correction of papers, concluding that PubPeer contributes more to the correction of science than negative citations.”
MethodThe right to refuse unwanted citations: rethinking the culture of science around the citation
da Silva, J. A. T., & Vuong, Q. (2021) · Scientometrics · doi:10.1007/s11192-021-03960-9 · Cited by 42
“This is theoretically possible if the editors agree to issue a corrigendum or retract and replace a paper to accommodate this extraordinary request (Bordignon, 2020).”
Background
Metadata
- Authors
- Bordignon, F.
- Venue
- Scientometrics
- Published
- 2020-06-03
- Volume
- 124
- Issue
- 2
- Pages
- 1225-1239
- ISSN
- 0138-9130 (ISSN-L)
- 1588-2861
- OpenAlex
- W3033593507
- Cited by
- 51
- References
- 55
- Citations by year
- 2026: 5 · 2025: 7 · 2024: 6 · 2023: 10 · 2022: 7 · 2021: 12 · 2020: 4
- FWCI
- 5.03
- Type
- journal article
- Language
- en
- Open access
- green
- enpc.hal.science
- Licence
- cc-by
- www.springer.com/tdm (tdm)
- www.springer.com/tdm (vor)
- Locations
- Scientometrics (publishedVersion)
- HAL (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Keywords
- post-publication peer review, retraction, PubPeer, Scite, scientific controversy, engineering literature
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Self-correction of science: a comparative study of negative citations and post-publication peer review. https://app.cochat.ai/papers/10.1007/s11192-020-03536-z
- The paper
- Bordignon, F. (2020). Self-correction of science: a comparative study of negative citations and post-publication peer review. Scientometrics. https://doi.org/10.1007/s11192-020-03536-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.