Citations versus journal impact factor as proxy of quality: could the latter ever be preferable?
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
In recent years bibliometricians have paid increasing attention to research evaluation methodological problems, among these being the choice of the most appropriate indicators for evaluating quality of scientific publications, and thus for evaluating the work of single scientists, research groups and entire organizations. Much literature has been devoted to analyzing the robustness of various indicators, and many works warn against the risks of using easily available and relatively simple proxies, such as journal impact factor. The present work continues this line of research, examining whether it is valid that the use of the impact factor should always be avoided in favour of citations, or whether the use of impact factor could be acceptable, even preferable, in certain circumstances. The evaluation was conducted by observing all scientific publications in the hard sciences by Italian universities, for the period 2004–2007. Performance sensitivity analyses were conducted with changing indicators of quality and years of observation.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Garfield, E. (1972) · Science · doi:10.1126/science.178.4060.471 · Cited by 2,788 · cited 2×
“From an analysis of the distributions of citations, Garfield (1972) observed early on that age distributions of cited references vary significantly among disciplines.”
Why the impact factor of journals should not be used for evaluating research
Seglen, P. O. (1997) · BMJ · doi:10.1136/bmj.314.7079.497 · Cited by 2,447
Journal impact measures in bibliometric research
Glänzel, W., & Moed, H. F. (2002) · Scientometrics · doi:10.1023/a:1014848323806 · Cited by 558
Cited by
A review of the literature on citation impact indicators
Waltman, L. (2016) · Journal of Informetrics · doi:10.1016/j.joi.2016.02.007 · Cited by 1,356
Impact factor: outdated artefact or stepping-stone to journal certification?
Vanclay, J. K. (2012) · Scientometrics · doi:10.1007/s11192-011-0561-0 · Cited by 370
Use of the journal impact factor for assessing individual articles: Statistically flawed or not?
Waltman, L., & Traag, V. A. (2021) · F1000Research · doi:10.12688/f1000research.23418.2 · Cited by 97
Revisiting the Scientometric Conceptualization of Impact and Its Measurement
Abramo, G. (2018) · Journal of Informetrics · doi:10.1016/j.joi.2018.05.001 · Cited by 80
Self-citationEvaluating scientific research in Italy: The 2004–10 research evaluation exercise
Ancaiani, A., Anfossi, A. F., Barbara, A., Benedetto, S., Blasi, B., Carletti, V., Cicero, T., Ciolfi, A., Costa, F., Colizza, G., et al. (2015) · Research Evaluation · doi:10.1093/reseval/rvv008 · Cited by 75
Self-citation
Metadata
- Authors
- Abramo, G., D’Angelo, C. A., & Di Costa, F.
- Venue
- Scientometrics
- Published
- 2010-02-27
- Volume
- 84
- Issue
- 3
- Pages
- 821-833
- ISSN
- 0138-9130 (ISSN-L)
- 1588-2861
- OpenAlex
- W2094649542
- Cited by
- 66
- References
- 3
- FWCI
- 3.97
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Citations versus journal impact factor as proxy of quality: could the latter ever be preferable? https://app.cochat.ai/papers/10.1007/s11192-010-0200-1
- The paper
- Abramo, G., D’Angelo, C. A., & Di Costa, F. (2010). Citations versus journal impact factor as proxy of quality: could the latter ever be preferable? Scientometrics. https://doi.org/10.1007/s11192-010-0200-1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.