Validity of citation criteria for assessing the influence of scientific publications: New evidence with peer assessment
The DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
This article reviews the principal correlational studies employing citation counts as criterion measures for assessing the impact of scientific scholarship. The rationale and limitations of such measures and studies are discussed. New evidence on the validity of citation criteria is presented based on a sample of 870 cancer research papers, divided into three groups (“first‐order” papers, abstracted in the Year Book of Cancer; “second‐order” papers, listed but not abstracted in the yearbook; and “average‐order” papers, a representative cross section of research papers unlisted in the yearbook). Results consistently show that highly rated papers are more highly cited over the ensuing five years after publication, or when controls are introduced for self‐citations, for the influence of listing in the yearbook, and for language and country of authorship. The implications of results are discussed.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Nigel Gilbert, G. (1977) · Social Studies of Science · doi:10.1177/030631277700700112 · Cited by 691
Content Analysis of References: Adjunct or Alternative to Citation Counting?
Chubin, D. E., & Moitra, S. D. (1975) · Social Studies of Science · doi:10.1177/030631277500500403 · Cited by 357
The norms of citation behavior: Prolegomena to the footnote
Kaplan, N. (1965) · American Documentation · doi:10.1002/asi.5090160305 · Cited by 244
Menard, H. W. (1971) · doi:10.4159/harvard.9780674420748 · Cited by 94
Lawani, S. M. (1977) · Journal of Agronomic Education · doi:10.2134/jae.1977.0041 · Cited by 6
Self-citation
Cited by
What do citation counts measure? A review of studies on citing behavior
Bornmann, L., & Daniel, H. (2008) · Journal of Documentation · doi:10.1108/00220410810844150 · Cited by 1,607
Bibliometric Methods: Pitfalls and Possibilities
Wallin, J. A. (2005) · Basic & Clinical Pharmacology & Toxicology · doi:10.1111/j.1742-7843.2005.pto_139.x · Cited by 884
Applying centrality measures to impact analysis: A coauthorship network analysis
Yan, E., & Ding, Y. (2009) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21128 · Cited by 390
A review of bibliometric and other science indicators and their role in research evaluation
King, J. (1987) · Journal of Information Science · doi:10.1177/016555158701300501 · Cited by 334
Some bibliometric correlates of quality in scientific research
Lawani, S. M. (1986) · Scientometrics · doi:10.1007/bf02016604 · Cited by 222
Self-citation
Metadata
- Authors
- Lawani, S. M., & Bayer, A. E.
- Venue
- Journal of the American Society for Information Science
- Publisher
- Wiley
- Published
- 1983-01
- Volume
- 34
- Issue
- 1
- Pages
- 59-66
- ISSN
- 0002-8231 (ISSN-L)
- 1097-4571
- OpenAlex
- W1964104106
- Cited by
- 95
- References
- 5
- FWCI
- 9.36
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.