The DOI matches its Crossref record; 143 of the 144 references it cites were found in a registry or an index; none of the 86 sources its claims cite is checked for support yet.
Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories
Open accessAbstract
Citations are increasingly used as performance indicators in research policy and within the research system. Usually, citations are assumed to reflect the impact of the research or its quality. What is the justification for these assumptions and how do citations relate to research quality? These and similar issues have been addressed through several decades of scientometric research. This article provides an overview of some of the main issues at stake, including theories of citation and the interpretation and validity of citations as performance measures. Research quality is a multidimensional concept, where plausibility/soundness, originality, scientific value, and societal value commonly are perceived as key characteristics. The article investigates how citations may relate to these various research quality dimensions. It is argued that citations reflect aspects related to scientific impact and relevance, although with important limitations. On the contrary, there is no evidence that citations reflect other key dimensions of research quality. Hence, an increased use of citation indicators in research evaluation and funding may imply less attention to these other research quality dimensions, such as solidity/plausibility, originality, and societal value.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This review argues that citations can serve as limited indicators of scientific impact and relevance but not of other key research-quality dimensions such as solidity/plausibility, originality, or societal value. It synthesizes citation theories, validation studies, and quality dimensions to caution against using citation indicators as substitutes for peer review.
Key findings
- Citations are argued to reflect aspects of scientific impact and relevance, but with important limitations; there is no evidence that they reflect other key dimensions of research quality—solidity/plausibility, originality, and societal value.
- Comparative validation studies generally find moderately positive correspondence between citation indicators and peer review, but correlations are far from perfect; in REF 2014, field-weighted citation impact had a Spearman correlation of .28 at the overall level, with significant field variation and decreases for more recent outputs, leading to the conclusion that metrics cannot provide a like-for-like replacement for peer review.
- Citation distributions are highly skewed: in many humanities disciplines an average paper receives less than one citation over 10 years, versus more than 40 in some biomedical fields (Web of Science 2005–2015); the Lowry et al. (1951) protein-measurement paper has more than 305,000 Web of Science citations, and the 4S statin trial paper more than 7,700.
- The h-index is popular but not field-normalized and not corrected for career length, disadvantaging younger researchers; field, publication-year, and document-type normalization is standard, but there is no general agreement on the most appropriate normalization method.
Methods
- Design: narrative/conceptual overview and selective literature review, not a systematic review or new empirical study.
- Data or sample: no new dataset; the article draws on prior scientometric literature, citation theories, validation studies, and selected examples.
- Main analysis: conceptual synthesis of citation indicators, referencing behavior, peer-review comparisons, and dimensions of research quality.
Limitations
- Stated: coverage of topics and literature is selective and does not discuss all details; the literature on interaction between citing practices and evaluation processes is only referred to in passing; constructivist and semiotic theories of quality and citation are not discussed.
- Stated/evident: few previous studies have systematically addressed how citations relate to each research-quality dimension, and the final part of the article is described as more explorative and discursive.
- Evident: the paper presents no systematic search strategy or empirical test of its central claims, so its conclusions rely on narrative synthesis and selected examples.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
76 Not checked yet · 44 Own interpretation · 16 No basis given
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“On the contrary, there is no evidence that citations reflect other key dimensions of research quality.”
- Also stated in: § Concluding Remarks · p. 12 · plain
“there is no evidence that citations reflect other key dimensions of research quality”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Research quality is a multidimensional concept comprising plausibility/soundness, originality, scientific value, and societal value as key characteristics.
- Also stated in: § Concluding Remarks · p. 12 · plain
In the paper
“The larger part of all scientific papers are never cited or cited a few times only in the subsequent scientific literature”
What the sources say
- Not checked yetcites Aksnes, 2005
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Citation distributions are very skewed.”
What the sources say
- Not checked yetcites Price (1965)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“There are large variations in average citation rates across different subject areas.”
What the sources say
- Not checked yetcites data from Web of Science 2005-2015
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“but empirical studies have shown that two different methods for normalization, AoR and RoA, did not produce very different results, particular at the level of countries and institutions”
What the sources say
- Not checked yetcites Lariviere & Gingras, 2011
- Not checked yetcites Waltman, van Eck, van Leeuwen, Visser, & van Raan, 2011a
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“On the contrary, some articles have an extremely large number of citations, reaching into the hundreds and even thousands.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“There is no general agreement on what is the most appropriate method”
What the sources say
- Not checked yetcites Ioannidis, Boyack, & Wouters, 2016
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Although important insights on the role of references in the scientific article have been obtained, the accumulation of knowledge at the same time has been hampered by the fact that different classification systems have been applied in previous studies”
What the sources say
- Not checked yetcites Liu, 1993
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“References to highly cited publications are more often present in the introduction than in other parts of the publications”
What the sources say
- Not checked yetcites Voos & Dagaev, 1976
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Because of this, articles describing commonly used methods may receive a very large number of citations.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- It may state two separate claims as one. 42
- Probably not a substantial claim. 58
- The quote only partly states this claim. 1
In the paper
“Most importantly, it is not field-normalized and no corrections are made for career length, which means that the indicator disfavors younger researchers”
What the sources say
- Not checked yetcites Alonso, Cabrerizo, Herrera-Viedma, & Herrera, 2009
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The views of the MacRoberts's previously led to much debate, but their conclusions are generally seen as too sweeping”
What the sources say
- Not checked yetcites Aksnes, 2005
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Even now, in spite of detailed studies of referencing behavior, there is no unified theory.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As indicated above, previous studies of the citation process have not provided any simple answer to the question of what citations stand for.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“While empirical studies have revealed a multitude of factors involved in the citation process, the issue has also been approached from another angle: by comparing citation indicators with the outcome of peer review. During recent decades, many such studies have been carried out.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Nevertheless, some overall findings remain: the references have a multitude of functions in the scientific article, only a small proportion of the relevant literature is cited, and the authors have a multitude of motives for including particular studies as references.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Studies undertaken have revealed that the role of the reference, both in the citing text and with respect to the cited text, is complex.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
In the paper
“Moreover, the citing is biased: some sources are cited essentially every time they are used, while other research is never cited even though it may be used more often than the highly cited work.”
What the sources say
- Not checked yetcites MacRoberts & MacRoberts, 1989, 1996
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“These and later studies have revealed that the references have a multitude of functions in the scientific article.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Moreover, the studies are often based on rather small samples of papers from selected scientific fields, and the results may not have general validity.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Based on empirical case studies, they showed that a very small proportion of the knowledge basis of an article (consisting of hundreds or thousands of former publications) actually are cited.”
What the sources say
- Not checked yetcites MacRoberts & MacRoberts, 1989, 1996
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“Therefore, it will be more persuasive to cite an authoritative paper, and authors tend to select references that will be regarded as authoritative by the intended audience.”
What the sources say
- Not checked yetcites Gilbert, 1977
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“This article has now been cited more than 305,000 times in the Web of Science database”
What the sources say
- Not checked yetcites Van Noorden, Maher, & Nuzzo, 2014
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In a scientific article, some of the references will represent works that are crucial or significant antecedents to the present work; others may represent more general background literature”
What the sources say
- Not checked yetcites Aksnes, 2005
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Hence, the textual functions of citations vary considerably.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
In the paper
“For the citation process, this implies that other incentives shape the citing patterns, like creating visibility for one's work through self-citations or citing a journal editor's work as an attempt to enhance the chances of acceptance for publication.”
What the sources say
- Not checked yetcites Bornmann & Daniel, 2008
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“There is a growing concern about this issue, as well as on potential negative impact of research metrics on the scientific community.”
What the sources say
- Not checked yetcites Hicks et al., 2015
- Not checked yetcites Cagan, 2013
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Citation measures are also core indicators in several university rankings, such as the Leiden ranking and Academic Ranking of World Universities (ARWU)”
What the sources say
- Not checked yetcites Piro & Sivertsen, 2016
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In recent years, bibliometric indicators have increasingly been applied in the context of research evaluation as well as research policy more generally.”
What the sources say
- Not checked yetcites Moed, 2005
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“For example, Levitt and Thelwall (2011) have argued that short citation windows have the problem that articles published earlier in a year have a significant advantage (i.e., are on average more highly cited) compared with publications appearing later in a year.”
What the sources say
- Not checked yetcites Levitt and Thelwall (2011)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Only a small fraction of the scholarly literature in the humanities is represented in the Web of Science, and most of the references and citations will not be captured by the database.”
What the sources say
- Not checked yetcites Marx and Bornmann (2015)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The coverage of the scientific and scholarly literature varies across these databases, and the results of citation studies are thus dependent upon the particular characteristics of the databases and their coverage.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“In the scientific paper, the references have various purposes. Authors are not including references merely because of their scientific quality.”
What the sources say
- Not checked yetcites Bornmann & Daniel, 2008
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is argued that citations reflect aspects related to scientific impact and relevance, although with important limitations.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Research quality is a multidimensional concept comprising plausibility/soundness, originality, scientific value, and societal value as key characteristics.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It has since been the standard to adjust for field, publication year, and publication type when calculating citation indicators.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Moreover, the application of citation indicators has also been criticized more generally, with respect to their validity as performance measures and their potentially negative impact upon the research system”
What the sources say
- Not checked yetcites MacRoberts & MacRoberts, 1989
- Not checked yetcites Osterloh & Frey, 2015
- Not checked yetcites Weingart, 2004
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“When one article is cited by many subsequent publications, even more people become aware of this article. Thus, its visibility, and thereby the chances of getting even more citations, increases.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In the studies, assessments by peers have been typically considered as a kind of standard to which citation indicators can be validated.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Some of these problems have a fundamental character and are inherent in any use of citations as indicators, others may be resolved by the construction of more advanced indicators, while others again may be of less importance in practice.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, panels increasingly are considering citation measures as part of the evaluation procedure, which means that the two cannot be considered as completely independent of each other.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If authors cite the works they find useful, frequently cited publications may be assumed to have been more useful than papers which are hardly cited at all.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, it may be concluded that most of the comparative studies seem to have found a moderately positive correspondence, but the correlations identified have been far from perfect and have varied among the studies.”
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“For the field-weighted citation impact, a Spearman correlation coefficient of .28 was identified at an overall level, albeit with significant variations across fields. Moreover, there were significant decreases in correlation for more recent outputs.”
What the sources say
- Not checked yetcites Higher Education Funding Council for England, 2015
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Finally, a large number of different citation measures exist and the outcome would also depend on which indicators are selected for the comparative analysis.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The quote only partly states this claim.
In the paper
“A significant degree of concordance was found '. . . supporting the choice of using both techniques in order to assess the quality of Italian research institutions' (Ancaiani et al., 2015, p. 254).”
What the sources say
- Not checked yetcites Ancaiani et al., 2015
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Although most citation analysts seem to agree that citing or referencing is biased, it has been argued that this bias is not fatal for the use of citation as performance indicators-to a certain extent, the biases are averaged out at aggregated levels.”
hedgedconditional- Asserted: fails — The authors may not put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The authors may not put it forward as true. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“According to Luukkonen (1990), the presence of different cognitive meanings of citations and motivations for citing does not necessarily invalidate the use of citations as (imperfect) performance measures.”
What the sources say
- Not checked yetcites Luukkonen (1990)
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Problems related to more technical issues, such as discrepancies between target articles and cited references (misspellings of journal names, author names, errors in the reference lists, etc.), and mistakes in the indexing procedures conducted by Clarivate Analytics (previously Thomson Reuters) or Elsevier (Leydesdorff et al., 2016; Moed, 2002) may confuse citation analyses. Such errors affect in”
What the sources say
- Not checked yetcites Leydesdorff et al., 2016
- Not checked yetcites Moed, 2002
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Aksnes and Taxt (2004) analyzed the relationship between bibliometric indicators and the outcomes of a peer review of Norwegian research groups at a mathematical and natural science faculty, reporting positive but weak correlations.”
What the sources say
- Not checked yetcites Aksnes and Taxt (2004)
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This means that the different functions the references may have in a text are much more complex than merely providing documentation and support for particular claims.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
In the paper
“Gilbert (1977) argued that citing ('referencing') is essentially a device for persuasion.”
What the sources say
- Not checked yetcites Gilbert (1977)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Small (1982) identified five distinctions: A cited work may be (a) refuted, (b) noted only, (c) reviewed, (d) applied, or (e) supported by the citing work.”
What the sources say
- Not checked yetcites Small (1982)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Typically, a scientific article is structured as a progression from the general to the particular”
What the sources say
- Not checked yetcites Law, 1986
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“Although the literature coverage of citation databases has improved (Web of Science and Scopus), the coverage of the humanities and several social science disciplines remains limited (Waltman, 2016).”
What the sources say
- Not checked yetcites Waltman, 2016
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As is evident from the overview above, there is no simple answer to the question what citation indicators measure or indicate.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, both the methodological basis for comparing peer assessments and citation indicators and the assumption that the two may be expected to correlate may be questionable.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“At the same time, citation frequencies are determined by other mechanisms and are not a simple reflection of the quality dynamics.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“According to Merton's view, the norms of science oblige researchers to cite the work upon which they draw, and in this way acknowledge or credit contributions by others”
What the sources say
- Not checked yetcites Merton, 1979
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“He found that the h-index and the normalized citation impact indicator both correlated quite well with peer judgments.”
What the sources say
- Not checked yetcites van Raan (2006)
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“According to Bornmann and Daniel (2008), many studies have methodological weaknesses and have provided findings with little reliability.”
What the sources say
- Not checked yetcites Bornmann and Daniel (2008)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“However, the methodological fundament for this conclusion has been contested by Baccini and De Nicolao (2016), who argue that the analysis is flawed and that informed peer review and bibliometrics do not produce similar results.”
What the sources say
- Not checked yetcites Baccini and De Nicolao (2016)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Rinia, van Leeuwen, van Vuren, and van Raan (1998) showed that various citation indicators correlated significantly with peer ratings of research programs in condensed matter physics.”
What the sources say
- Not checked yetcites Rinia, van Leeuwen, van Vuren, and van Raan (1998)
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“According to the founder of the SCI, the fact that authors do not cite all their influences does not invalidate the use of citations as performance measures when enough literature is taken into account”
What the sources say
- Not checked yetcites Garfield, 1997
plain- Asserted: fails — The authors may not put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The authors may not put it forward as true. The paper’s conclusions may not depend on it.
In the paper
“The fact that citation distributions are extremely skewed also raises questions concerning the use of mean as indicator, and Bornmann and Mutz (2011) have proposed to use percentile ranks as a non-parametric alternative to the means of citation distributions for the normalization.”
What the sources say
- Not checked yetcites Bornmann and Mutz (2011)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“According to Welljams-Dorof (1997), this has important implications: In general, the larger the citation data set being used, the higher the confidence level of the results. Analyses involving entire fields of research, nations, regions and large universities are virtually unaffected by the concerns and caveats about citation data . . . The confidence level at these large aggregate levels is quite”
What the sources say
- Not checked yetcites Welljams-Dorof (1997)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, the use of impact as the most appropriate concept has usually been justified by theoretical considerations, and there are few attempts to address the issue empirically or relate it to previous findings on citation behavior.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Still, a basic limitation is that many innovations are not patented and patents are not suitable to assess societal relevance or impact in a broader context.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown above, the question on the relation between citations and research quality is complex and will arise differently depending on the field analyzed, the database used, the timeframe and indicators applied, and so forth.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“According to Waltman, van Eck, and Wouters (2013), citation impact should be distinguished from scientific impact, as an influential researcher sometimes has a lower performance in terms of highly cited publications than some of their less influential colleagues.”
What the sources say
- Not checked yetcites Waltman, van Eck, and Wouters (2013)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, the viewpoint described in the introduction that bibliometric assessment is superior compared to the traditional peer-review method is not justified in our opinion.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, the referee system does not fully ensure the quality dimension related to solidity and plausibility, and there are no indications that high citation counts reflect solidity.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“However, considering the various limitations attached to citations as performance measures, most bibliometricians have argued that a bibliometric analysis should not function as a substitute for a peer review (Moed, 2005).”
What the sources say
- Not checked yetcites Moed, 2005
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Citation indicators may easily be misused or applied in contexts where they lack justification or validity.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
None of the sources cited for this claim have been checked yet.
In the paper
“In addition, research quality is a multidimensional concept, and in this section, we will look further into this issue.”
What the sources say
- Not checked yetcites Polanyi (1962)
- Not checked yetcites Gulbrandsen, 2000
- Not checked yetcites Lamont, 2009
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As an example, even a 'mistaken' publication can have a large impact by stimulating further research.”
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is because of the sociological and aggregational processes involved. In the beginning, an article may be cited for substantive reasons (e.g., its content has been used). Later, when the article is widely known and has received many citations, sociological mechanisms will be of increasing importance (authors citing authoritative papers, the bandwagon effect, etc.). Some papers will benefit”
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The view generally held among experts within bibliometrics seem to be that citations represent a good but not perfect impact measure.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As is evident from the discussion of this paper, this is an important point, as citations are not able to capture all aspects of the quality concept. Hence, an increased use of citation indicators in research evaluation and funding may imply less attention to these other research quality dimensions, such as solidity/plausibility, originality, and societal value.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“As a starting point, we can take the three dimensions distinguished by Polanyi (1962): plausibility, originality, and scientific value.”
What the sources say
- Not checked yetcites Polanyi (1962)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Impact is the most commonly used concept for what citations reflect, although other concepts such as influence, importance, significance, and utility occasionally also are used (Moed, 2005).”
What the sources say
- Not checked yetcites Moed, 2005
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“There seems to be a widespread assumption that this issue cannot be adequately assessed through standard citation indicators, and in recent years, increasing attention has been devoted to developing methodologies for assessing and measuring societal relevance and impact (Bornmann, 2012, 2013).”
What the sources say
- Not checked yetcites Bornmann, 2012, 2013
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Nevertheless, there is a lack of empirical studies confirming that this is actually the case, and possibly some of the biases is of a fundamental nature attached to all citations measures, while the effect of others may tend to level out when aggregated levels are considered.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Societal relevance is often considered to be something which is much harder to measure than scientific relevance or impact (B. R. Martin, 2011).”
What the sources say
- Not checked yetcites B. R. Martin, 2011
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, all assessments of research quality may be context-dependent, in terms of, for example, the time of assessment and the time/field/sector perspectives of the evaluators.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Using these definitions, it is also evident that impact would be a more adequate interpretation of citations than quality.”
What the sources say
- Not checked yetcites B. R. Martin and Irvine, 1983
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Despite these and numerous similar examples, it is not possible to identify societal relevance from citation counts per se, and uncited or little cited publication may have contributed with results of great societal relevance.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Because of this, it has long been emphasized by bibliometricians that more than one indicator should be used in research evaluation contexts (van Raan, 1993).”
What the sources say
- Not checked yetcites van Raan, 1993
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“citations reflect-with important limitations-aspects related to scientific impact and relevance”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The literature contains numerous publications of which the solidity is poor, the results unreliable or even involving misconduct or scientific fraud (Fanelli, 2009).”
What the sources say
- Not checked yetcites Fanelli, 2009
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“For example, the mean normalized citation score is size-independent and does not take into account the number of publications. According to Abramo and D'Angelo (2016), this is a major problem with this indicator because it does not truly represent productivity.”
What the sources say
- Not checked yetcites Abramo and D'Angelo (2016)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Surprisingly, this topic has rarely been addressed specifically in the literature and there are few studies analyzing the issue empirically.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“On this basis, it is often argued that bibliometric analysis can counterbalance shortcomings and mistakes in the peers' judgments (Aksnes, 2005). Thus, a bibliometric study should be considered as complementary to a peer evaluation (Council of Canadian Academies, 2012).”
What the sources say
- Not checked yetcites Aksnes, 2005
- Not checked yetcites Council of Canadian Academies, 2012
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In sum, whereas plausibility/soundness, scientific value, and societal value and originality seem commonly perceived key characteristics of research quality, each of these dimensions include a variety of aspects; they may be contextdependent and may also conflict with each other.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
In the paper
“At the same time, there are also problems with the peer-review system.”
What the sources say
- Not checked yetcites Eyre-Walker and Stoletzki (2013)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
The paper relies on this without support.
Research quality is a multidimensional concept comprising plausibility/soundness, originality, scientific value, and societal value as key characteristics.
“Research quality is a multidimensional concept, where plausibility/soundness, originality, scientific value, and societal value commonly are perceived as key characteristics.”
The authors state this as an assumption.
There should be a correlation between citation indicators and peer assessments if citations legitimately can be used as indicators of scientific performance.
“The basic assumption is that there should be a correlation if citations legitimately can be used as indicators of scientific performance.”
The paper relies on this without support.
Peer reviews are applied in many different contexts, of which peer assessments of manuscripts submitted to journals and publishers is the most fundamental one.
“Peer reviews are applied in many different contexts, of which peer assessments of manuscripts submitted to journals and publishers probably is the most fundamental one.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“It is often argued that the scientific impact of any evaluated entity cannot be determined solely by citations due to the different citing behaviors and resulting incompatible citation counts between disciplines [1-3,5,260,267].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [3]
In the citing paper
“Citations may be appropriate in assessing the scientific impact of the research, but they do not show the impact of the research outside the scientific community [1,3,5,6,260,264,265].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [3]
In the citing paper
“peer-reviewing has its own characteristic biases and limitations, which is one of the reasons why, nowadays, peer-review has been increasingly extensively replaced with bibliometric measures of impact [3,44,276].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [3]
In the citing paper
“this 'publish or perish' attitude, on the one hand, stimulates the productivity of researchers, but, on the other, creates a bias towards quantity, often overlooking quality [2,3,6-8,44,81,163,260,263,284-288].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [3]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Corresponding Author: · p. 2
“Many previous studies have compared citation indicators with the outcome of peer review, and in the 'Understanding Citations' part, this issue is examined.”
§ Validation Studies · p. 6
“## Validation Studies While empirical studies have revealed a multitude of factors involved in the citation process, the issue has also been approached from another angle: by comparing citation indicators with the outcome of peer review.”
Citations
Cites
PROTEIN MEASUREMENT WITH THE FOLIN PHENOL REAGENT
Lowry, O. H., Rosebrough, N., Farr, A. L., & Randall, R. (1951) · Journal of Biological Chemistry · doi:10.1016/s0021-9258(19)52451-6 · Cited by 316,394
An index to quantify an individual’s scientific research output
Hirsch, J. E. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0507655102 · Cited by 11,669
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
The Matthew Effect in Science: The reward and communication systems of science are considered.
Merton, R. K. (1968) · Science · doi:10.1126/science.159.3810.56 · Cited by 6,720
(1968) · PubMed · Cited by 4,803
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“However, the recent attention to the lack of replicable results in biomedical, clinical, and psychological studies (Ioannidis, 2005) may lead to a higher social status of replications studies.”
BackgroundPrice, D. J. D. S. (1965) · Science · doi:10.1126/science.149.3683.510 · Cited by 3,205
Bibliometrics: The Leiden Manifesto for Research Metrics
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015) · Nature · doi:10.1038/520429a · Cited by 2,828
Self-citationFanelli, D. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005738 · Cited by 1,911 · cited 1×
“The literature contains numerous publications of which the solidity is poor, the results unreliable or even involving misconduct or scientific fraud (Fanelli, 2009).”
BackgroundGoogle Scholar, Scopus and the Web of Science: A Longitudinal and Cross-Disciplinary Comparison
Harzing, A.-W., & Alakangas, S. (2016) · Scientometrics · doi:10.1007/s11192-015-1798-9 · Cited by 1,784
Research works printed but not linked: 15
- Biagioli M., Gaming Metrics. Beyond Publish or Perish: Metrics and the new Ecologies of Academic Misconduct
- Carlsson H., 2009, InfoTrend, 64, 82
- Dymond L. H., 2015, A recent history of recognized economic thought—Contributions of the Nobel laureates to economic science
- Garfield E., 1977, Essay of an information scientist, 1, 84
- Gulbrandsen J. M., 2000, Research quality and organisational factors: An investigation of the relationship
- Higher Education Funding Council for England. (2015). The metric tide: Correlation analysis of REF2014 scores and metrics: Supplementary report II to the independent review of the role of metrics in research assessment and management. Retrieved from www.dcscience.net/2015_metrictideS2.pdf
- Langfeldt L., Scordato L. (2015, April). Assessing the broader impacts of research: A review of methods and practices (NIFU Working Paper 8/2015). Oslo.
- le Pair C, 1995, International Forum on Information and Documentation, 20, 16
- Luukkonen T., 1990, Citations in the rhetorical, reward, and communication systems of science
- Merton R. K., 1979, Citation indexing—Its theory and application in science, technology, and humanities
- Smith L. C., 1981, Library Trends, 30, 83
- van Raan A. F. J, 2000, The web of knowledge: A Festschrift in honor of Eugene Garfield, 301
- Welljams-Dorof A., 1997, Evaluating science and scientists: An East-West dialogue on research evaluation in post-communist Europe, 202
- Wouters P. (1999b). The citation culture (Doctoral thesis). University of Amsterdam, The Netherlands. Retrieved from http://garfield.library.upenn.edu/wouters/wouters.pdf
- Wouters P., Handbook of science policy studies
Cited by
Sorted by citations
Items 301–400 of 1,000
Global Insights and Trends in Research on Dermatophytes and Dermatophytosis: A Bibliometric Analysis
Ortíz, B., Ballesteros‐Monrreal, M. G., Rosales‐Tamashiro, J., Bush, M., Salmanton‐García, J., & Fontecha, G. A. (2024) · Mycoses · doi:10.1111/myc.13803 · Cited by 14
Graph Neural Networks: A Bibliometric Mapping of the Research Landscape and Applications
da Silva, A. M. B., Ferreira, N. C. D. S., Braga, L. A. M., Mota, F. B., Maricato, V., & Alves, L. A. (2024) · Information · doi:10.3390/info15100626 · Cited by 14
A bibliometric analysis of ohmic heating on food processing in the last two decades
Maspeke, P. N., Salengke, S., Muhidong, J., & Dirpan, A. (2024) · Heliyon · doi:10.1016/j.heliyon.2024.e39315 · Cited by 14
Holmberg, C. (2024) · Journal of Advanced Nursing · doi:10.1111/jan.16562 · Cited by 14
Decadal evolution of GIS in disaster management and risk assessment
Eminoğlu, Y., & Tarhan, Ç. (2025) · International Journal of Engineering and Geosciences · doi:10.26833/ijeg.1544048 · Cited by 14
Is outsourcing better? Clients' views on contracting in domestic cleaning services
Toit, D. D., & Heinecken, L. (2021) · Employee Relations · doi:10.1108/er-08-2020-0394 · Cited by 14
Smart Health-Enhanced Early Mobilisation in Intensive Care Units
Ferré, M., Batista, E., Solanas, A., & Martínez-Ballesté, A. (2021) · Sensors · doi:10.3390/s21165408 · Cited by 14
Groundhog Day: research without old data and old references
Hong, B. A., Pollio, D. E., Downs, D. L., Coyne, D. W., & North, C. S. (2022) · Psychological Medicine · doi:10.1017/s0033291722000216 · Cited by 14
Губа, К. С., & Tsivinskaya, A. O. (2021) · Journal of Documentation · doi:10.1108/jd-02-2022-0039 · Cited by 14
Kunze, K. N., Vadhera, A. S., Polce, E. M., Higuera, C. A., Siddiqi, A., Chahla, J. A., & Piuzzi, N. S. (2022) · HSS Journal® The Musculoskeletal Journal of Hospital for Special Surgery · doi:10.1177/15563316221115723 · Cited by 14
Pölönen, J., & Auranen, O. (2022) · Scientometrics · doi:10.1007/s11192-022-04385-8 · Cited by 14
Making sense and transparency in finance literature: Evidence from trends in readability
Lahmar, O., & Piras, L. (2023) · Research in International Business and Finance · doi:10.1016/j.ribaf.2023.101900 · Cited by 17
Correlación entre redes sociales y métricas de revistas científicas
Estrada, J. M. C., Nacipucha, N. A. S., Floril, P. R. P., Sánchez‐Bayón, A., & Guerrero, K. C. (2023) · Revista Latina de Comunicación Social · doi:10.4185/rlcs-2023-1897 · Cited by 14
Bibliometric top ten healthcare-related ChatGPT publications in the first ChatGPT anniversary
Sallam, M. (2024) · Narra J · doi:10.52225/narra.v4i2.917 · Cited by 15
The steering effects of citations and metrics
Ma, L. (2020) · Journal of Documentation · doi:10.1108/jd-06-2020-0093 · Cited by 14
Rajeev, J., Kumar, K. C. J., Prasad, D. P., & Utpal, D. (2023) · Journal of Resources and Ecology · doi:10.5814/j.issn.1674-764x.2023.06.011 · Cited by 13
Martikainen, J., Hujala, A., & Laulainen, S. (2021) · Interchange · doi:10.1007/s10780-021-09449-x · Cited by 13
Pourret, O., Irawan, D. E., Shaghaei, N., van Rijsingen, E. M., & Besançon, L. (2022) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2022.850333 · Cited by 13
Singh, P. K. (2022) · Artificial Intelligence Review · doi:10.1007/s10462-022-10249-7 · Cited by 13
Zhang, O. L., Niu, J. Y., Yin, I. X., Yu, O. Y., Mei, M. L., & Chu, C. H. (2022) · Journal of Functional Biomaterials · doi:10.3390/jfb13040210 · Cited by 13
Robert, C., & Wilson, C. S. (2023) · Frontiers in Pain Research · doi:10.3389/fpain.2023.1071453 · Cited by 13
Rank analysis of most cited publications, a new approach for research assessments
Rodríguez‐Navarro, A., & Brito, R. (2024) · Journal of Informetrics · doi:10.1016/j.joi.2024.101503 · Cited by 13
Bibliometric Analysis of Artificial Intelligence in STEM Education
Jantakun, T., Jantakun, K., & Jantakoon, T. (2024) · Higher Education Studies · doi:10.5539/hes.v15n1p69 · Cited by 13
Why are some articles highly cited in applied linguistics? A bibliometric study
Zhang, S., & Aryadoust, V. (2025) · Studies in Second Language Acquisition · doi:10.1017/s0272263124000743 · Cited by 13
Khan, M. O., Keesstra, S. D., Słowik-Opoka, E., Klamerus‐Iwan, A., & Liaqat, W. (2025) · Water · doi:10.3390/w17030322 · Cited by 13
Bibliometrics and altmetrics in the context of traditional, complementary, and integrative medicine
Ng, J. Y., Stephen, D. M., Liu, J., Ostermann, T., Robinson, N., & Cramer, H. (2025) · Integrative Medicine Research · doi:10.1016/j.imr.2025.101181 · Cited by 14
Innovative Ideas and Gender (In)Equality
Koffi, M. (2025) · American Economic Review · doi:10.1257/aer.20211811 · Cited by 17
Polce, E. M., Kunze, K. N., Farivar, D., Fu, M. C., Nwachukwu, B. U., Nho, S. J., & Chahla, J. A. (2020) · Arthroscopy The Journal of Arthroscopic and Related Surgery · doi:10.1016/j.arthro.2020.09.015 · Cited by 13
Social-ecological considerations for the sleep health of rural mothers
Fischer, A. R., Green, S.-R. M., & Gunn, H. E. (2020) · Journal of Behavioral Medicine · doi:10.1007/s10865-020-00189-4 · Cited by 13
Performance implications of combining creative and imitative innovation strategies
Moon, C., & Acquaah, M. (2020) · European Journal of Innovation Management · doi:10.1108/ejim-06-2020-0213 · Cited by 13
Mirnezami, S. R., & Beaudry, C. (2021) · Journal of African Economies · doi:10.1093/jae/ejab008 · Cited by 13
Scientometric engineering: Exploring citation dynamics via arXiv eprints
Okamura, K. (2022) · Quantitative Science Studies · doi:10.1162/qss_a_00174 · Cited by 13
Potter, R. W. K., Szomszor, M., & Adams, J. R. (2022) · Scientometrics · doi:10.1007/s11192-022-04303-y · Cited by 13
Bibliometric analysis of the 100 most‐disruptive articles in ophthalmology
Patel, P. A., Patel, P. N., Becerra, A. Z., & Mehta, M. C. (2022) · Clinical and Experimental Ophthalmology · doi:10.1111/ceo.14109 · Cited by 13
Adeniji, A. O., Okaiyeto, K., Mohammed, J. N., Mabaleha, M., Tanor, E. B., & George, M. J. (2023) · International Journal of Environmental Science and Technology · doi:10.1007/s13762-023-04916-7 · Cited by 16
Hassan, D., Tantawi, M. E., & Hassan, M. G. (2023) · Journal of the World Federation of Orthodontists · doi:10.1016/j.ejwf.2023.05.003 · Cited by 13
A bibliometric analysis of fantasy sports research
Wilkins, L. (2023) · Entertainment Computing · doi:10.1016/j.entcom.2023.100613 · Cited by 13
The relationship of polarity of post-publication peer review to citation count
Zong, Q., Fan, L., Xie, Y., & Huang, J. (2020) · Online Information Review · doi:10.1108/oir-01-2019-0027 · Cited by 13
Determinants of research efficiency in Canadian business schools: evidence from scholar-level data
Rhaiem, M., & Amara, N. (2020) · Scientometrics · doi:10.1007/s11192-020-03633-z · Cited by 13
Automated citation recommendation tools encourage questionable citations
Horbach, S. P. J. M., Maatman, F. O., Halffman, W., & Hepkema, W. M. (2022) · Research Evaluation · doi:10.1093/reseval/rvac016 · Cited by 13
Exploring the effects of strategy-focused instruction in writing skills of 4TH grade students
Rodríguez-Málaga, L., Cueli, M., & Rodríguez, C. (2020) · Metacognition and Learning · doi:10.1007/s11409-020-09247-3 · Cited by 13
Research patterns in sports law and sports governance: a scopus bibliometric study
Kameņecka-Usova, M., Korde, Ž., Žīdens, J., Luika, S., Agapova, E., Palkova, K., & Mārtinsone, K. (2025) · Frontiers in Sports and Active Living · doi:10.3389/fspor.2025.1590858 · Cited by 12
Research in higher education governance: past performance and an agenda for the future
Rajasekharan Pillai, K., Sankaran, K., Ramnarayan, K., & Prabhu, N. K. P. (2021) · Educational Review · doi:10.1080/00131911.2021.1984214 · Cited by 12
Eweje, F. R., Byun, S., Chandra, R., Hu, F., Kamel, I. R., Zhang, P. J., Jiao, Z., & Bai, H. X. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.44742 · Cited by 12
Cueto, R. J., Harris, A. B., Root, K. T., Sabharwal, S., Raad, M., & Oni, J. K. (2023) · The Journal of Arthroplasty · doi:10.1016/j.arth.2023.05.085 · Cited by 12
Social networks as a strategy to improve the visibility of scientific journals
Nacipucha, N. A. S., Sánchez‐Bayón, A., Estrada, J. M. C., & Valencia-Arías, A. (2024) · Cogent Social Sciences · doi:10.1080/23311886.2024.2306715 · Cited by 12
Analyzing Emerging Trends in Wireless Implantable Medical Devices (IMDs): A Bibliometric Study
Fuada, S., Särestöniemi, M., & Katz, M. D. (2024) · International Journal of Online and Biomedical Engineering (iJOE) · doi:10.3991/ijoe.v20i04.46559 · Cited by 12
Cheng, X., Wang, H., Tang, L., Jiang, W., Zhou, M., & Wang, G. (2024) · Journal of Informetrics · doi:10.1016/j.joi.2024.101540 · Cited by 12
The global research landscape and future trends in healthcare Total Quality Management
Hu, Z., Wang, R. S., Qin, X., Huang, Y.-N., Li, L., Chiu, H., Liu, Y., & Wang, B. (2024) · Archives of Public Health · doi:10.1186/s13690-024-01420-3 · Cited by 12
Exploring the Evolution of Nanotechnology Education: Insights from Bibliometric Analysis
Yıldız, M., Kaltakci-Gurel, D., Salmankurt, B., & Gürel, H. H. (2024) · Journal of Chemical Education · doi:10.1021/acs.jchemed.4c01120 · Cited by 12
Academic Impact of Articles by Practitioners in the Field of Library and Information Science
Chang, Y. (2021) · College & Research Libraries · doi:10.5860/crl.82.1.59 · Cited by 12
Smith, M. J., Baxter, A. J., Skivington, K., McCann, M., Hilton, S., & Katikireddi, S. V. (2021) · PLoS ONE · doi:10.1371/journal.pone.0255604 · Cited by 12
Chang, J. Y., Lim, X., & Cheah, J. (2021) · British Food Journal · doi:10.1108/bfj-03-2021-0315 · Cited by 12
Are FT50 journals really leading? A comment on Fassin
Moussa, S. (2021) · Scientometrics · doi:10.1007/s11192-021-04158-9 · Cited by 12
Quantitative and qualitative analysis of higher-order thinking skills in blended learning
He, X., Singh, C. K. S., Idris, U. P. S., & Ebrahim, N. A. (2022) · Perspectives of science and education · doi:10.32744/pse.2022.5.23 · Cited by 12
A systematic review on social currency: a one-decade perspective
Silva, B. N., da Silva, W. V., de Macêdo, Á. F. P., Levino, N. D. A., Dalazen, L. L., Kaczam, F., & da Veiga, C. P. (2023) · Journal of Financial Services Marketing · doi:10.1057/s41264-023-00231-x · Cited by 12
A scientometric analysis of bone cutting tools & methodologies: Mapping the research landscape
Addepalli, P., Sawangsri, W., & Ghani, S. A. C. (2024) · Injury · doi:10.1016/j.injury.2024.111458 · Cited by 12
Alex, F. J., Tan, G., Agyeman, P. K., Ansah, P. O., Olayode, I. O., Fayzullayevich, J. V., & Liang, S. (2022) · Sustainability · doi:10.3390/su142214753 · Cited by 12
Rahaman, S., Govil, P., Khan, D., & Jevremov, T. (2023) · Information Discovery and Delivery · doi:10.1108/idd-11-2022-0110 · Cited by 11
Idea engines: Unifying innovation & obsolescence from markets & genetic evolution to science
Lee, E. D., Kempes, C., & West, G. (2022) · Proceedings of the National Academy of Sciences of the United States of America · doi:10.1073/pnas.2312468120 · Cited by 12
The impact of winning funding on researcher productivity, results from a randomized trial
Barnett, A. G., Blakely, T. A., Liu, M., Garland, L., & Clarke, P. (2024) · Science and Public Policy · doi:10.1093/scipol/scae045 · Cited by 13
Guo, Y., Liu, S., Dong, Y., Wu, G., Liu, J., Wang, W. “., & Wang, J. (2024) · Ecological Frontiers · doi:10.1016/j.ecofro.2024.11.007 · Cited by 11
Argyriou, M., & Tapsis, N. (2025) · European Journal of Open Education and E-learning Studies · doi:10.46827/ejoe.v10i3.6106 · Cited by 11
Film-induced tourism: a bibliometric analysis (1998–2024)
Almendros, E. C., Prados‐Peña, M. B., & Ruiz, M. D. C. P. (2025) · Journal of Hospitality and Tourism Insights · doi:10.1108/jhti-01-2025-0104 · Cited by 11
Mapping the digital landscape: A bibliometric analysis of digital ecosystem research
Jeppe, A., Brée, T., Karger, E., Proff, H., & Ahlemann, F. (2025) · Digital Business · doi:10.1016/j.digbus.2025.100145 · Cited by 11
In Which Fields Do ChatGPT Scores Align More Closely with Research Quality than Do Citation Rates?
Thelwall, M. A. (2026) · Journal of Data and Information Science · doi:10.1515/jdis-2026-0058 · Cited by 17
Interdisciplinary Health Care Evaluation Instruments: A Review of Psychometric Evidence
Kang, H., Flores‐Sandoval, C., Law, B., & Sibbald, S. L. (2021) · Evaluation & the Health Professions · doi:10.1177/01632787211040859 · Cited by 12
The launch of Seismica: a seismic shift in publishing
Rowe, C. D., Agius, M. R., Convers, J. A., Funning, G. J., Galasso, C., Hicks, S., Huynh, T. T., Lange, J., Lecocq, T., Mark, H. F., et al. (2022) · Seismica · doi:10.26443/seismica.v1i1.255 · Cited by 11
Lee, S. L., Then, A. Y., Goh, H. C., Hattam, C., Edwards‐Jones, A., & Austen, M. C. (2022) · Frontiers in Marine Science · doi:10.3389/fmars.2022.1033200 · Cited by 11
Citation culture: Citing authors behaviour vs trust in research results
Тихонова, Е. В., & Kirillova, O. V. (2023) · Science Editor and Publisher · doi:10.24069/sep-22-58 · Cited by 12
Carnino, J. M., Walia, A. S., Lara, F. R., Mwaura, A. M., & Levi, J. R. (2023) · International Journal of Pediatric Otorhinolaryngology · doi:10.1016/j.ijporl.2023.111638 · Cited by 11
Exploring Research Trends and Impact: A Bibliometric Analysis of RESTI Journal from 2018 to 2022
Watrianthos, R., & Yuhefizar, Y. (2023) · Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi) · doi:10.29207/resti.v7i4.5101 · Cited by 11
Knudson, D. V., Cardinal, B. J., & McCullagh, P. D. (2023) · Quest · doi:10.1080/00336297.2023.2237150 · Cited by 11
Do Regulatory Measures Necessarily Affect Oman’s Seafood Export-Supply?
Naabi, A. A., & Bose, S. (2020) · SAGE Open · doi:10.1177/2158244020950658 · Cited by 11
Scientometric Full-Text Analysis of Papers Published in Remote Sensing between 2009 and 2021
Balz, T. (2022) · Remote Sensing · doi:10.3390/rs14174285 · Cited by 11
Revealing the research landscape of Master's degrees via bibliometric analysesPreprint
Chaparro, N., & Rojas–Galeano, S. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2103.09431 · Cited by 11
Jackson, S. R., Chambers, S. K., Leslie, S., & Patel, M. I. (2023) · Patient Education and Counseling · doi:10.1016/j.pec.2023.107887 · Cited by 10
Advancing women to leadership in academia: does personal branding matter?
Heriberta, H., Gaus, N., Paramma, M. A., & Utami, N. (2024) · Qualitative Research Journal · doi:10.1108/qrj-06-2023-0091 · Cited by 10
Shi, W., & Wu, R. (2024) · Scientometrics · doi:10.1007/s11192-024-05089-x · Cited by 11
Padhan, S. R., Saini, S., JAT, S. L., Rathore, S. S., Gathala, M. K., Radheshyam, Padhan, S. R., El-Hendawy, S., & Mattar, M. A. (2024) · Sustainability · doi:10.3390/su16198670 · Cited by 10
Crompton, J. L. (2025) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-08-2024-1201 · Cited by 10
A scientometric analysis of reviews on the Internet of Things
Jain, S., Sukul, P., Groppe, J., Warnke, B., Harde, P., -, R. J., Rehan, W., Sehgelmeble, Y. C., Fischer, S., & Groppe, S. (2025) · The Journal of Supercomputing · doi:10.1007/s11227-025-07230-w · Cited by 10
Tan, Q. L., Hashim, S., & Zheng, Z. (2025) · Sustainable Futures · doi:10.1016/j.sftr.2025.100667 · Cited by 13
Jang, H. (2021) · Science Editing · doi:10.6087/kcse.225 · Cited by 10
Kehyayan, V., Chen, J., & Hirdes, J. P. (2021) · Journal of Long-Term Care · doi:10.31389/jltc.47 · Cited by 10
Pearson, W. S. (2022) · Teaching English as a Second or Foreign Language--TESL-EJ · doi:10.55593/ej.25100a1 · Cited by 10
Schniedermann, A. (2022) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2022.846822 · Cited by 13
Global research trends on diet and nutrition in Crohn’s disease
Shakhshir, M. H., & Zyoud, S. H. (2023) · World Journal of Gastroenterology · doi:10.3748/wjg.v29.i20.3203 · Cited by 10
Conceptual and technical work: Who will disrupt science?
Liu, X., Zhang, C., & Li, J. (2023) · Journal of Informetrics · doi:10.1016/j.joi.2023.101432 · Cited by 10
Reluctance to pursue breakthrough research: A signaling explanation
Besancenot, D., & Radu Vranceanu, B. (2024) · Research Policy · doi:10.1016/j.respol.2024.104974 · Cited by 10
Луговий, В. І., Drach, I., Петроє, О., Zinchenko, V. V., Mielkov, Y., Жиляєв, І., Регейло, І. Ю., Слободянюк, О. М., & БАЗЕЛЮК, Н. В. (2023) · doi:10.31874/978-617-7644-61-2-2023 · Cited by 10
PD-1/PD-L1 inhibitors related adverse events: A bibliometric analysis from 2014 to 2024
Song, Q., Yu, Z., Lü, W., Zhuo, Z., Chang, L., Mei, H., Cui, Y., & Zhang, D. (2025) · Human Vaccines & Immunotherapeutics · doi:10.1080/21645515.2024.2424611 · Cited by 11
Knowledge production and internationalisation of research in Taiwan: a new watershed?
Chan, S., Yang, C.-Y., & Tai, H. (2020) · Journal of Higher Education Policy and Management · doi:10.1080/1360080x.2020.1730536 · Cited by 10
Hanis, T. M., Islam, M. A., & Musa, K. I. (2021) · Current Medicinal Chemistry · doi:10.2174/0929867328666211108110731 · Cited by 10
Santos, D. F., Asif, N., Malavé, G. S., & Izquierdo, N. (2022) · Cureus · doi:10.7759/cureus.31048 · Cited by 10
Support for Cannabis Tourism: A Tale of Two States
Kang, S. K., & McGrady, P. (2020) · Tourism Review International · doi:10.3727/154427220x15990732245691 · Cited by 10
Rushforth, A., & Hammarfelt, B. (2022) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/cdmqz · Cited by 11
Jaksic, C., Gayet‐Ageron, A., & Perneger, T. (2023) · BMC Medical Research Methodology · doi:10.1186/s12874-023-02037-w · Cited by 9
Research performance evaluation model in university: evidence from Indonesia
Hermanu, A. I., Sari, D., Sondari, M. C., & Dimyati, M. (2024) · International Journal of Educational Management · doi:10.1108/ijem-05-2023-0235 · Cited by 9
Metadata
- Authors
- Aksnes, D. W., Langfeldt, L., & Wouters, P.
- Affiliations
- Nordic Institute for Studies in Innovation, Research and Education (NO)
- Leiden University (NL)
- Corresponding author
- Aksnes, Dag W.
- Venue
- Sage Open
- Publisher
- SAGE Publications
- Published
- 2019-01
- Volume
- 9
- Issue
- 1
- ISSN
- 2158-2440 (ISSN-L)
- OpenAlex
- W2913788248
- Semantic Scholar
- 59660bbee2813e0ddc65cd8e7fa35ea51f58b6e7
- Cited by
- 1,257
- References
- 132
- Citations by year
- 2026: 153 · 2025: 207 · 2024: 223 · 2023: 202 · 2022: 181 · 2021: 177 · 2020: 89 · 2019: 23
- FWCI
- 78.79
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- SAGE Open (publishedVersion, cc-by)
- Leiden Repository (Leiden University) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- NORA - Norwegian Open Research Archives (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Funders
- Norges Forskningsråd (256223)
- Topics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- citation, citation metrics, research quality, research evaluation, research funding, scientific impact, originality, social values, plausibility, science policy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-06, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-06 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
