Characterizing in-text citations in scientific articles: A large-scale analysis
Open accessAbstract
We report characteristics of in-text citations in over five million full text articles from two large databases – the PubMed Central Open Access subset and Elsevier journals – as functions of time, textual progression, and scientific field. The purpose of this study is to understand the characteristics of in-text citations in a detailed way prior to pursuing other studies focused on answering more substantive research questions. As such, we have analyzed in-text citations in several ways and report many findings here. Perhaps most significantly, we find that there are large field-level differences that are reflected in position within the text, citation interval (or reference age), and citation counts of references. In general, the fields of Biomedical and Health Sciences, Life and Earth Sciences, and Physical Sciences and Engineering have similar reference distributions, although they vary in their specifics. The two remaining fields, Mathematics and Computer Science and Social Science and Humanities, have different reference distributions from the other three fields and between themselves. We also show that in all fields the numbers of sentences, references, and in-text mentions per article have increased over time, and that there are field-level and temporal differences in the numbers of in-text mentions per reference. A final finding is that references mentioned only once tend to be much more highly cited than those mentioned multiple times.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The reference distributions vary in different fields of science (e.g. mathematics and computer science as well as social science and humanities).
“Boyack et al. (2018) indicate that the reference distributions vary in different fields of science (e.g. mathematics and computer science as well as social science and humanities).”
From Core Elements in the Process of Citing Publications: Conceptual Overview of the Literature (2018) · cites this paper as Boyack et al., 2018
In general, citations are likely to be concentrated at the beginnings (e.g. introduction) and endings (e.g. discussion) of documents than in the middle.
“In general, citations are likely to be concentrated at the beginnings (e.g. introduction) and endings (e.g. discussion) of documents than in the middle (Boyack et al., 2018).”
From Core Elements in the Process of Citing Publications: Conceptual Overview of the Literature (2018) · cites this paper as Boyack et al., 2018
The number of references has trended upwards after 1983 in all fields.
“The number of references has trended upwards after this period (Swales, 1990) in all fields (Boyack et al., 2018).”
From Core Elements in the Process of Citing Publications: Conceptual Overview of the Literature (2018) · cites this paper as Boyack et al., 2018
There is a rough consensus in the literature that references located outside the introduction tend to be the most valuable.
“Boyack et al. (2018) note that there is a rough consensus in the literature that references located outside the introduction tend to be the most valuable.”
From Core Elements in the Process of Citing Publications: Conceptual Overview of the Literature (2018) · cites this paper as Boyack et al., 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 32 · Cited by 125
Research works printed but not linked: 2
- Bornmann, L., & Daniel, H.-D. (2008). Functional use of frequently and infrequently cited articles in citing publications: A content analysis of citations to articles with low and high citation counts. European Science Editing, 34(2), 35-38.
- Hooten, P. A. (1991). Frequency and functional use of cited documents in information science. Journal of the American Society for Information Science, 42(6), 397-404.
- Authors
- Boyack, K. W., van Eck, N. J., Colavizza, G., & Waltman, L.
- Affiliations
- SciTech Strategies (United States) (US)
- École Polytechnique Fédérale de Lausanne (CH)
- Leiden University (NL)
- Corresponding author
- Boyack, Kevin W.
- Venue
- Journal of Informetrics
- Publisher
- Elsevier BV
- Published
- 2018-02
- Volume
- 12
- Issue
- 1
- Pages
- 59-73
- ISSN
- 1751-1577 (ISSN-L)
- OpenAlex
- W2762538549
- Cited by
- 125
- References
- 32
- Citations by year
- 2026: 5 · 2025: 11 · 2024: 6 · 2023: 13 · 2022: 11 · 2021: 23 · 2020: 20 · 2019: 20 · 2018: 16
- FWCI
- 13.73
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Journal of Informetrics (publishedVersion, other-oa)
- arXiv (Cornell University) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion, other-oa)
- RePEc: Research Papers in Economics (submittedVersion)
- Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion)
- Infoscience (Ecole Polytechnique Fédérale de Lausanne) (submittedVersion, cc-by-nc-nd)
- Leiden Repository (Leiden University) (submittedVersion)
- Leiden Repository (Leiden University) (submittedVersion, cc-by-nc-nd)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- arXiv (Cornell University)
- Funders
- Swiss National Fund (P1ELP2_168489)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
- Social Sciences › Arts and Humanities › History and Philosophy of Science › Academic Writing and Publishing
- Keywords
- in-text citations, citation practices, citation counts, disciplinary differences, scientific articles, full-text articles
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
