Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 202 of the 226 references it cites were found in a registry or an index; 1 of the 149 sources its claims cite were checked for support.

What do citation counts measure? A review of studies on citing behavior

Open access
Bornmann, Lutz, Daniel, Hans‐Dieter · 2008 · Journal of Documentation

Abstract

Purpose The purpose of this paper is to present a narrative review of studies on the citing behavior of scientists, covering mainly research published in the last 15 years. Based on the results of these studies, the paper seeks to answer the question of the extent to which scientists are motivated to cite a publication not only to acknowledge intellectual and cognitive influences of scientific peers, but also for other, possibly non‐scientific, reasons. Design/methodology/approach The review covers research published from the early 1960s up to mid‐2005 (approximately 30 studies on citing behavior‐reporting results in about 40 publications). Findings The general tendency of the results of the empirical studies makes it clear that citing behavior is not motivated solely by the wish to acknowledge intellectual and cognitive influences of colleague scientists, since the individual studies reveal also other, in part non‐scientific, factors that play a part in the decision to cite. However, the results of the studies must also be deemed scarcely reliable: the studies vary widely in design, and their results can hardly be replicated. Many of the studies have methodological weaknesses. Furthermore, there is evidence that the different motivations of citers are “not so different or ‘randomly given’ to such an extent that the phenomenon of citation would lose its role as a reliable measure of impact”. Originality/value Given the increasing importance of evaluative bibliometrics in the world of scholarship, the question “What do citation counts measure?” is a particularly relevant and topical issue.

Overview

CoChat Summary

TL;DR

This narrative review of approximately 30 studies on scientists’ citing behavior finds that citations are not motivated solely by intellectual or cognitive influence; non-scientific factors also play a part. It concludes that citation counts remain a complex, multidimensional indicator, with evidence that citer motivations are not so different or random as to destroy their role as a reliable impact measure at high aggregation levels.

Key findings

  • Citing behavior includes non-normative motives. Moravcsik and Murugesan (30 articles, 575 citations) found 69% valuable, 60% organic, and 59% evolutionary citations, but also 41% perfunctory and 31% redundant citations; Chubin and Moitra (43 papers, 443 citations) found 22%/17% perfunctory and 6%/4% negational citations; Spiegel-Rösing (66 articles, 2,309 citations) found 80% of citations substantiated a statement or assumption and only 2% were critical.
  • Across the reviewed studies, citation-type percentages ranged from about 10–90% for affirmational, 5–50% for assumptive, 1–50% for conceptual, 5–40% for contrastive, 5–45% for methodological, 1–15% for negational, 10–50% for perfunctory, and 5–40% for persuasive citations.
  • Empirical tests of the two main theories are mixed: Baldi (1998), Stewart (1983), and White (2004) support a normative interpretation, while Collins (1999) supports the social constructivist view; Cronin (2005b) is quoted as saying the weight of empirical evidence suggests scientists typically cite in a normatively guided manner.
  • Studies on non-citing show authors do not cite all influences: White and Wang (12 agricultural economists; 176 decisions not to cite vs. 413 decisions to cite) found topicality as a motive in 54% of not-cite vs. 35% of cite decisions, content in 13% vs. 23%, and classic/founder in 0% vs. 9%; Kurtz et al. (2005) found that reads and cites measure the same thing, namely usefulness of an article.

Methods

  • Narrative review of research published from the early 1960s to mid-2005, covering approximately 30 studies on citing behavior reporting results in about 40 publications; 48% of included studies were published from the early 1990s to mid-2005, and 52% from the early 1960s to the late 1980s.
  • Studies are organized by research method: context analyses of citations in citing documents, content analyses of citations to characterize cited work, citer motivation surveys or interviews, and studies on motivations for non-citing.
  • Main analysis is a narrative synthesis and a unified typology of citation types, discussed against the normative theory and the social constructivist view of citing behavior.

Limitations

  • Stated: the studies vary widely in design, their results can hardly be replicated, and many have methodological weaknesses; document analyses rely on experts’ subjective judgments or guesses about authors’ motives, while surveys may fail to reveal true reasons because motives are partly unconscious or neglected, memory may be faulty, typologies may be understood inconsistently, or authors may deliberately mislead.
  • Stated: no study has joined document analysis and surveys of scientists to make results more reliable; most published content analyses provide no operational definition for judgment tasks and report no reliability coefficient; development of citation classification schemes has not been cumulative, and findings on explicit textual cues are contradictory.
  • Evident: as a narrative review, it included only the most important and influential pre-1990s studies and excluded unpublished studies, so its coverage is selective.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1108/00220410810844150

Cites

202 of 202 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Verification

Claims the paper makes

0 Valid · 3 Not valid · 113 Not checked · 27 Own results and interpretations

65 more claims

Create a free account to see all 143 claims and to verify the 2 references still open.

See all claims — free account
  • Probably not a substantial claim. 53
  • It may state two separate claims as one. 37

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

5 of 5

Citations

Cites

Show all 226 references

Research works printed but not linked: 16

  1. Broadus, R.N. (1971), “Literature of social sciences – survey of citation studies”, International Social Science Journal, Vol. 23, pp. 236‐43.
  2. Cawkell, A.E. (1968), “Citation practices”, Journal of Documentation, Vol. 24, pp. 299‐305.
  3. Cole, J. and Cole, S. (1971), “Measuring quality of sociological research – problems in use of science citation index”, American Sociologist, Vol. 6, pp. 23‐9.
  4. Crane, D. (1972), Invisible Colleges, University of Chicago Press, Chicago, IL.
  5. Cronin, B. (1981a), “Transatlantic citation patterns in educational psychology”, Social Science Information Studies, Vol. 24, pp. 48‐51.
  6. Cronin, B. (2005b), “A hundred million acts of whimsy?”, Current Science, Vol. 89, pp. 1505‐9.
  7. Finney, B. (1979), “The reference characteristics of scientific texts”, MSc dissertation, Centre for Information Science, City University, London.
  8. Garfield, E. (1962), “Can citation indexing be automated?”, Essays of an Information Scientist, Vol. 1, pp. 84‐90.
  9. Garfield, E. (1978), “High impact science and case of Arthur Jensen”, Current Contents, No. 41, pp. 5‐15.
  10. Garfield, E. (1981), “Citation classics – four years of the human side of science”, Current Contents, No. 22, pp. 5‐16.
  11. Garfield, E. (2002), “Highly cited authors”, Scientist, Vol. 16, p. 10.
  12. Hanney, S., Grant, J., Jones, T. and Buxton, M. (2005b), “Categorising citations to trace research impact”, in Ingwersen, P. and Larsen, B. (Eds), Proceedings of the 10th International Conference of the International Society for Scientometrics and Informetrics, Karolinska University Press, Stockholm.
  13. Hjorland, B. (2000), “Relevance research: the missing perspective(s): ‘non‐relevance’ and ‘epistemological relevance’”, Journal of the American Society for Information Science, Vol. 51, pp. 209‐11.
  14. Johansson, C.M.A. (1976), “Citation behaviour in chemistry”, MSc dissertation, Centre for Information Science, City University, London.
  15. O'Connor, J. (1980), “Citing statements – recognition by computer and use to improve retrieval”, Proceedings of the American Society for Information Science, Vol. 17, pp. 177‐9.
  16. Smith, L.C. (1981), “Citation analysis”, Library Trends, Vol. 30, pp. 83‐106.

Cited by

See all 1,607

Metadata

Authors
Bornmann, L., & Daniel, H.
Affiliations
ETH Zurich (CH)
Venue
Journal of Documentation
Publisher
Emerald
Published
2008-01-18
Volume
64
Issue
1
Pages
45-80
ISSN
0022-0418 (ISSN-L)
OpenAlex
W2019753053
Cited by
1,607
References
226
Citations by year
2026: 173 · 2025: 147 · 2024: 103 · 2023: 99 · 2022: 81 · 2021: 96 · 2020: 116 · 2019: 102 · 2018: 86 · 2017: 84 · 2016: 88 · 2015: 85 · 2014: 71 · 2013: 79 · 2012: 70
FWCI
47.16
Type
journal article
Language
en
Open access
Licence
  • other-oa
  • www.emerald.com/insight/site-policies (tdm)
Locations
  • Journal of Documentation (publishedVersion)
  • MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
Topics
  • Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
  • Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Keywords
citation behavior, citation counts, bibliometric analysis, research impact assessment, research evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: What do citation counts measure? A review of studies on citing behavior. https://app.cochat.ai/papers/10.1108/00220410810844150
The paper
Bornmann, L., & Daniel, H. (2008). What do citation counts measure? A review of studies on citing behavior. Journal of Documentation. https://doi.org/10.1108/00220410810844150

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.