Cited by 1,356scientometrics and bibliometrics researchWeb visibility and informetricsPsychology Research and Bibliometrics

A review of the literature on citation impact indicators

Waltman, Ludo·2016·Journal of Informetrics

Abstract

Citation impact indicators nowadays play an important role in research evaluation, and consequently these indicators have received a lot of attention in the bibliometric and scientometric literature. This paper provides an in-depth review of the literature on citation impact indicators. First, an overview is given of the literature on bibliographic databases that can be used to calculate citation impact indicators (Web of Science, Scopus, and Google Scholar). Next, selected topics in the literature on citation impact indicators are reviewed in detail. The first topic is the selection of publications and citations to be included in the calculation of citation impact indicators. The second topic is the normalization of citation impact indicators, in particular normalization for field differences. Counting methods for dealing with co-authored publications are the third topic, and citation impact indicators for journals are the last topic. The paper concludes by offering some recommendations for future research.

Integrity signals

Registry

Crossref · 10.1016/j.joi.2016.02.007

Cites

359 of 359 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • When citations are missing, inaccurate, or fabricated, the resulting break in the evidence chain can obscure logical dependencies, weaken argumentation, and jeopardize academic integrity.

    “When citations are missing, inaccurate, or fabricated, the resulting break in the evidence chain can obscure logical dependencies, weaken argumentation, and jeopardize academic integrity at the level of both individual papers and the broader literature [10, 22, 26].”
  • Having a larger citation window is known to influence citation counts.

    “having a larger citation window (i.e., time period after publication in which an article can be cited) is known to influence citation counts (Waltman, 2016).”
  • Information provided in early studies performed more than several years ago may have become obsolete and could be misleading.

    “Moreover, because DBs are continuously expanding both their contents and capabilities, the information provided in the early studies, performed more than several years ago, may have become obsolete and, thus, could be misleading [26].”
  • The coverage of books and conference proceedings both in WoS and Scopus was generally determined to be insufficient for analyzing or evaluating disciplines where these source types are the most prevalent.

    “Therefore, it is not surprising that the coverage of books and conference proceedings both in WoS and Scopus was generally determined to be insufficient, aiming to use the DBs as the data sources for analyzing or evaluating disciplines where these source types are the most prevalent [26,44,88].”
  • In Computer sciences, research findings tend to be published in conference papers, while books and text-books are more important sources in social sciences and even more so in humanities.

    “For example, it is well known that, in Computer sciences, the research findings tend to be published in conference papers [86,87], while books and text-books are more important sources in social sciences and even more so in humanities [26,46].”

Cites

Flagged first, then by citations

18 of the 377 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Waltman, L.
Affiliations
Leiden University (NL)
Corresponding author
Waltman, Ludo
Venue
Journal of Informetrics
Publisher
Elsevier BV
Published
2016-05
Volume
10
Issue
2
Pages
365-391
ISSN
1751-1577 (ISSN-L)
OpenAlex
W767067438
Cited by
1,356
References
377
Citations by year
2026: 204 · 2025: 135 · 2024: 137 · 2023: 158 · 2022: 142 · 2021: 135 · 2020: 128 · 2019: 104 · 2018: 107 · 2017: 71 · 2016: 32 · 2015: 3
FWCI
97.17
Type
journal article
Language
en
Open access
closed
Locations
  • Journal of Informetrics (publishedVersion)
  • Leiden Repository (Leiden University) (submittedVersion)
  • Data Archiving and Networked Services (DANS) (submittedVersion)
Funders
Higher Education Funding Council for England
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
  • Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Keywords
citation impact indicators, research evaluation, bibliographic databases, Web of Science, Scopus, Google Scholar, indicator normalization, field normalization, journal impact indicators

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review