The DOI matches its Crossref record; 202 of the 226 references it cites were found in a registry or an index; 1 of the 149 sources its claims cite were checked for support.
What do citation counts measure? A review of studies on citing behavior
Open accessAbstract
Purpose The purpose of this paper is to present a narrative review of studies on the citing behavior of scientists, covering mainly research published in the last 15 years. Based on the results of these studies, the paper seeks to answer the question of the extent to which scientists are motivated to cite a publication not only to acknowledge intellectual and cognitive influences of scientific peers, but also for other, possibly non‐scientific, reasons. Design/methodology/approach The review covers research published from the early 1960s up to mid‐2005 (approximately 30 studies on citing behavior‐reporting results in about 40 publications). Findings The general tendency of the results of the empirical studies makes it clear that citing behavior is not motivated solely by the wish to acknowledge intellectual and cognitive influences of colleague scientists, since the individual studies reveal also other, in part non‐scientific, factors that play a part in the decision to cite. However, the results of the studies must also be deemed scarcely reliable: the studies vary widely in design, and their results can hardly be replicated. Many of the studies have methodological weaknesses. Furthermore, there is evidence that the different motivations of citers are “not so different or ‘randomly given’ to such an extent that the phenomenon of citation would lose its role as a reliable measure of impact”. Originality/value Given the increasing importance of evaluative bibliometrics in the world of scholarship, the question “What do citation counts measure?” is a particularly relevant and topical issue.
Overview
CoChat Summary
TL;DR
This narrative review of approximately 30 studies on scientists’ citing behavior finds that citations are not motivated solely by intellectual or cognitive influence; non-scientific factors also play a part. It concludes that citation counts remain a complex, multidimensional indicator, with evidence that citer motivations are not so different or random as to destroy their role as a reliable impact measure at high aggregation levels.
Key findings
- Citing behavior includes non-normative motives. Moravcsik and Murugesan (30 articles, 575 citations) found 69% valuable, 60% organic, and 59% evolutionary citations, but also 41% perfunctory and 31% redundant citations; Chubin and Moitra (43 papers, 443 citations) found 22%/17% perfunctory and 6%/4% negational citations; Spiegel-Rösing (66 articles, 2,309 citations) found 80% of citations substantiated a statement or assumption and only 2% were critical.
- Across the reviewed studies, citation-type percentages ranged from about 10–90% for affirmational, 5–50% for assumptive, 1–50% for conceptual, 5–40% for contrastive, 5–45% for methodological, 1–15% for negational, 10–50% for perfunctory, and 5–40% for persuasive citations.
- Empirical tests of the two main theories are mixed: Baldi (1998), Stewart (1983), and White (2004) support a normative interpretation, while Collins (1999) supports the social constructivist view; Cronin (2005b) is quoted as saying the weight of empirical evidence suggests scientists typically cite in a normatively guided manner.
- Studies on non-citing show authors do not cite all influences: White and Wang (12 agricultural economists; 176 decisions not to cite vs. 413 decisions to cite) found topicality as a motive in 54% of not-cite vs. 35% of cite decisions, content in 13% vs. 23%, and classic/founder in 0% vs. 9%; Kurtz et al. (2005) found that reads and cites measure the same thing, namely usefulness of an article.
Methods
- Narrative review of research published from the early 1960s to mid-2005, covering approximately 30 studies on citing behavior reporting results in about 40 publications; 48% of included studies were published from the early 1990s to mid-2005, and 52% from the early 1960s to the late 1980s.
- Studies are organized by research method: context analyses of citations in citing documents, content analyses of citations to characterize cited work, citer motivation surveys or interviews, and studies on motivations for non-citing.
- Main analysis is a narrative synthesis and a unified typology of citation types, discussed against the normative theory and the social constructivist view of citing behavior.
Limitations
- Stated: the studies vary widely in design, their results can hardly be replicated, and many have methodological weaknesses; document analyses rely on experts’ subjective judgments or guesses about authors’ motives, while surveys may fail to reveal true reasons because motives are partly unconscious or neglected, memory may be faulty, typologies may be understood inconsistently, or authors may deliberately mislead.
- Stated: no study has joined document analysis and surveys of scientists to make results more reliable; most published content analyses provide no operational definition for judgment tasks and report no reliability coefficient; development of citation classification schemes has not been cumulative, and findings on explicit textual cues are contradictory.
- Evident: as a narrative review, it included only the most important and influential pre-1990s studies and excluded unpublished studies, so its coverage is selective.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 Valid · 3 Not valid · 113 Not checked · 27 Own results and interpretations
The cited sources that could be checked couldn’t confirm or dispute this claim.
In the paper
“as longer articles have more content that can be cited than shorter articles do, the sheer size of an article influences whether it is cited”
What the sources say
- Could not verifycites Abt, 1993 sourceExperimental — accuracy not yet measured
The Introduction (page unavailable) distinguishes comprehensive studies from brief ones and says citation counts measure usefulness, but it does not link article length to the likelihood of being cited. Searches for longer articles, paper length, and citations found no passage making that connection. (LLM, conf=0.75)
- Not checked — no full text of the sourcecites Baldi, 1998 source
- Not checked — no full text of the sourcecites Stewart, 1990 source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“the studies vary widely in design”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Many of the studies have methodological weaknesses.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“and their results can hardly be replicated”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“since the individual studies reveal also other, in part non-scientific, factors that play a part in the decision to cite”
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
65 more claims
Create a free account to see all 143 claims and to verify the 2 references still open.
- Probably not a substantial claim. 53
- It may state two separate claims as one. 37
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Variability in citation practices across disciplines, for instance, is a well-recognized problem which prevents direct comparability of citation counts across fi elds (Bornmann & Daniel, 2008), unless appropriate corrective methods are applied (Radicchi et al., 2008).”
From Unraveling scientific impact: Citation types in marketing journals (2015) · cites this paper as Bornmann & Daniel, 2008
Experimental — accuracy not yet measuredIn the citing paper
“Bornmann (2008) suggests that variation of citation behaviour across disciplines may also mean that comparisons made via h-index and JIF across scientific domains are invalid [10].”
From Measuring scientific impact beyond academia: An assessment of existing impact metrics and proposed improvements (2017) · cites this paper as Bornmann (2008)
In the citing paper
“Bornmann (2008) suggests that variation of citation behaviour across disciplines may also mean that comparisons made via h-index and JIF across scientific domains are invalid [10].”
From Measuring scientific impact beyond academia: An assessment of existing impact metrics and proposed improvements (2017) · cites this paper as [10]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Notes · p. 26
“(Van Raan, 2004b, p. 461).”
§ JDOC 64,1 · p. 36
“Van Raan, A.F.J. (2004b), 'Sleeping beauties in science', Scientometrics , Vol. 59, pp. 467-72.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ What do citation counts measure? · p. 12
“16 percent used methodology from the cited work and 11 percent used theoretical equations.”
§ What do citation counts measure? · p. 13
“| Use of methodology (i.e. a citing author made use of a practical or theoretical technique given in the cited paper) | 11 | 5 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 1. Theoretical approaches to explaining citing behavior · p. 4
“(Merton, 1968, p. 622, see also Merton, 1957).”
§ What do citation counts measure? · p. 33
“Merton, R.K. (1968), 'The Matthew effect in science', Science , Vol. 159, pp. 56-63.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ What do citation counts measure? · p. 11
“almost one-third of all the citations were just mentions of other works (5 percent) or perfunctory references to works providing an overview (25 percent)”
§ 2.3. Content analyses of citations to characterize the cited works · p. 11
“| Simple mention (of the type 'compare here also,' 'see also,' 'see, for example') without any further more specific reference to the cited document | 25 |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ 3. Discussion · p. 21
“Small (1998) could not be completely convinced by the results of Baldi (1998, p. 143)”
§ References · p. 27
“Baldi, S. (1998), 'Normative versus social constructivist processes in the allocation of citations: a network-analytic model', American Sociological Review , Vol. 63, pp. 829-46.”
Citations
Cites
Molecular Structure of Nucleic Acids: A Structure for Deoxyribose Nucleic Acid
WATSON, J. D., & CRICK, F. H. C. (1953) · Nature · doi:10.1038/171737a0 · Cited by 13,584
The Matthew Effect in Science: The reward and communication systems of science are considered.
Merton, R. K. (1968) · Science · doi:10.1126/science.159.3810.56 · Cited by 6,720
(1968) · PubMed · Cited by 4,803
The Sociology of Science: Theoretical and Empirical Investigations
Kaplan, N., Merton, R. K., & Storer, N. W. (1974) · Technology and Culture · doi:10.2307/3102264 · Cited by 4,005
Laboratory Life: The Social Construction of Scientific Facts
Latour, B., & Woolgar, S. (1986) · Oxford University Research Archive (ORA) (University of Oxford) · Cited by 2,980
How Much Can We Boost IQ and Scholastic Achievement
Jensen, A. (1969) · Harvard Educational Review · doi:10.17763/haer.39.1.l3u15956627424k7 · Cited by 2,907
Garfield, E. (1972) · Science · doi:10.1126/science.178.4060.471 · Cited by 2,788
A general theory of bibliometric and other cumulative advantage processes
Price, D. D. S. (1976) · Journal of the American Society for Information Science · doi:10.1002/asi.4630270505 · Cited by 2,035
(1973) · Cited by 1,638
Priorities in Scientific Discovery: A Chapter in the Sociology of Science
Merton, R. K. (1957) · American Sociological Review · doi:10.2307/2089193 · Cited by 1,593
Research works printed but not linked: 16
- Broadus, R.N. (1971), “Literature of social sciences – survey of citation studies”, International Social Science Journal, Vol. 23, pp. 236‐43.
- Cawkell, A.E. (1968), “Citation practices”, Journal of Documentation, Vol. 24, pp. 299‐305.
- Cole, J. and Cole, S. (1971), “Measuring quality of sociological research – problems in use of science citation index”, American Sociologist, Vol. 6, pp. 23‐9.
- Crane, D. (1972), Invisible Colleges, University of Chicago Press, Chicago, IL.
- Cronin, B. (1981a), “Transatlantic citation patterns in educational psychology”, Social Science Information Studies, Vol. 24, pp. 48‐51.
- Cronin, B. (2005b), “A hundred million acts of whimsy?”, Current Science, Vol. 89, pp. 1505‐9.
- Finney, B. (1979), “The reference characteristics of scientific texts”, MSc dissertation, Centre for Information Science, City University, London.
- Garfield, E. (1962), “Can citation indexing be automated?”, Essays of an Information Scientist, Vol. 1, pp. 84‐90.
- Garfield, E. (1978), “High impact science and case of Arthur Jensen”, Current Contents, No. 41, pp. 5‐15.
- Garfield, E. (1981), “Citation classics – four years of the human side of science”, Current Contents, No. 22, pp. 5‐16.
- Garfield, E. (2002), “Highly cited authors”, Scientist, Vol. 16, p. 10.
- Hanney, S., Grant, J., Jones, T. and Buxton, M. (2005b), “Categorising citations to trace research impact”, in Ingwersen, P. and Larsen, B. (Eds), Proceedings of the 10th International Conference of the International Society for Scientometrics and Informetrics, Karolinska University Press, Stockholm.
- Hjorland, B. (2000), “Relevance research: the missing perspective(s): ‘non‐relevance’ and ‘epistemological relevance’”, Journal of the American Society for Information Science, Vol. 51, pp. 209‐11.
- Johansson, C.M.A. (1976), “Citation behaviour in chemistry”, MSc dissertation, Centre for Information Science, City University, London.
- O'Connor, J. (1980), “Citing statements – recognition by computer and use to improve retrieval”, Proceedings of the American Society for Information Science, Vol. 17, pp. 177‐9.
- Smith, L.C. (1981), “Citation analysis”, Library Trends, Vol. 30, pp. 83‐106.
Cited by
Science Mapping: A Systematic Review of the Literature
Chen, C. (2017) · Journal of Data and Information Science · doi:10.1515/jdis-2017-0006 · Cited by 1,925
“(Bornmann & Daniel, 2008), and a study of the universality of citation distributions (Radicchi, Journal of Data and Information Science Vol. 2 No. 2, 2017 26 Expert Review Journal of Data and Information Science Fortunato, & Castellano, 2008).”
BackgroundA review of the literature on citation impact indicators
Waltman, L. (2016) · Journal of Informetrics · doi:10.1016/j.joi.2016.02.007 · Cited by 1,356
“Furthermore, this review also does not cover literature on the meaning of citations (Bornmann & Daniel, 2008; Nicolaisen, 2007), literature on the practical application of citation impact indicators in the context of research evaluation, and literature on the correlation between citation impact…”
BackgroundCitations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories
Aksnes, D. W., Langfeldt, L., & Wouters, P. (2019) · Sage Open · doi:10.1177/2158244019829575 · Cited by 1,257
“We refer to Bornmann and Daniel (2008) and Nicolaisen (2007) for extensive overviews of this literature.”
BackgroundInfluentialUniversality of citation distributions: Toward an objective measure of scientific impact
Radicchi, F., Fortunato, S., & Castellano, C. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0806977105 · Cited by 828
Analyzing and modeling real-world phenomena with complex networks: a survey of applications
Costa, L. D. F., Oliveira, O. N., Travieso, G., Rodrigues, F. A., Villas Boas, P. R., Antiqueira, L., Viana, M. P., & Correa Rocha, L. E. (2011) · Advances in Physics · doi:10.1080/00018732.2011.572452 · Cited by 800
Metadata
- Authors
- Bornmann, L., & Daniel, H.
- Affiliations
- ETH Zurich (CH)
- Venue
- Journal of Documentation
- Publisher
- Emerald
- Published
- 2008-01-18
- Volume
- 64
- Issue
- 1
- Pages
- 45-80
- ISSN
- 0022-0418 (ISSN-L)
- OpenAlex
- W2019753053
- Semantic Scholar
- fac7e66de0ec1f7283b5b48e55e45263afe55e06
- Cited by
- 1,607
- References
- 226
- Citations by year
- 2026: 173 · 2025: 147 · 2024: 103 · 2023: 99 · 2022: 81 · 2021: 96 · 2020: 116 · 2019: 102 · 2018: 86 · 2017: 84 · 2016: 88 · 2015: 85 · 2014: 71 · 2013: 79 · 2012: 70
- FWCI
- 47.16
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Licence
- other-oa
- www.emerald.com/insight/site-policies (tdm)
- Locations
- Journal of Documentation (publishedVersion)
- MPG.PuRe (Max Planck Society) (submittedVersion, other-oa)
- Topics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- citation behavior, citation counts, bibliometric analysis, research impact assessment, research evaluation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: What do citation counts measure? A review of studies on citing behavior. https://app.cochat.ai/papers/10.1108/00220410810844150
- The paper
- Bornmann, L., & Daniel, H. (2008). What do citation counts measure? A review of studies on citing behavior. Journal of Documentation. https://doi.org/10.1108/00220410810844150
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.