How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations
Open accessThe DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
Although bibliometrics has been a separate research field for many years, there is still no uniformity in the way bibliometric analyses are applied to individual researchers. Therefore, this study aims to set up proposals how to evaluate individual researchers working in the natural and life sciences. 2005 saw the introduction of the h index, which gives information about a researcher’s productivity and the impact of his or her publications in a single number (h is the number of publications with at least h citations); however, it is not possible to cover the multidimensional complexity of research performance and to undertake inter-personal comparisons with this number. This study therefore includes recommendations for a set of indicators to be used for evaluating researchers. Our proposals relate to the selection of data on which an evaluation is based, the analysis of the data and the presentation of the results.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Theory and practise of the g-index
Egghe, L. (2006) · Scientometrics · doi:10.1007/s11192-006-0144-7 · Cited by 2,354
What do citation counts measure? A review of studies on citing behavior
Bornmann, L., & Daniel, H. (2008) · Journal of Documentation · doi:10.1108/00220410810844150 · Cited by 1,607
Self-citationToward a Comparative Sociology of Valuation and Evaluation
Lamont, M. (2012) · Annual Review of Sociology · doi:10.1146/annurev-soc-070308-120022 · Cited by 1,492 · cited 2×
“Since governments have turned to new public management tools to ensure greater efficacy, quantitative measures of performance and benchmarking have been needed (Lamont, 2012).”
BackgroundBornmann, L. (2011) · Annual Review of Information Science and Technology · doi:10.1002/aris.2011.1440450112 · Cited by 567
Self-citationA macro study of self-citation
Aksnes, D. W. (2003) · Scientometrics · doi:10.1023/a:1021919228368 · Cited by 357
García‐Pérez, M. A. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21372 · Cited by 101
Research on research evaluation
Hemlin, S. (1996) · Social Epistemology · doi:10.1080/02691729608578815 · Cited by 48 · cited 1×
“However, it should be considered that many metrics chosen for a study correlate with each to a high degree (Abramo, D'Angelo, & Costa, 2010; Duffy, Jadidian, Webster, & Sandell, 2011; Hemlin, 1996) – even if results differ at a detailed level (Opthof & Wilde, 2011).”
Background
Cited by
Hutchins, B. I., Yuan, X., Anderson, J. M., & Santangelo, G. M. (2016) · PLoS Biology · doi:10.1371/journal.pbio.1002541 · Cited by 500
“Other methods include citation percentiles [14,20], eigenvector normalization [21,22], and source normalization [13,23]; the latter includes both the mean normalized citation score (MNCS) [17] and source-normalized impact per paper metrics [15,17,22–26].”
MethodBornmann, L., & Leydesdorff, L. (2014) · EMBO reports · doi:10.15252/embr.201439608 · Cited by 350
Self-citationLeydesdorff, L., Wouters, P., & Bornmann, L. (2016) · Scientometrics · doi:10.1007/s11192-016-2150-8 · Cited by 173
Self-citationMultidimensional Assessment of Scholarly Research Impact
Moed, H. F., & Halevi, G. (2015) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23314 · Cited by 168
Core Elements in the Process of Citing Publications: Conceptual Overview of the Literature
Tahamtan, I., & Bornmann, L. (2018) · Journal of Informetrics · doi:10.1016/j.joi.2018.01.002 · Cited by 149
Self-citation
Metadata
- Authors
- Bornmann, L., & Marx, W.
- Venue
- Scientometrics
- Published
- 2013-10-23
- Volume
- 98
- Issue
- 1
- Pages
- 487-509
- ISSN
- 0138-9130 (ISSN-L)
- 1588-2861
- OpenAlex
- W1979117318
- Cited by
- 143
- References
- 7
- FWCI
- 11.11
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations. https://app.cochat.ai/papers/10.1007/s11192-013-1161-y
- The paper
- Bornmann, L., & Marx, W. (2014). How to evaluate individual researchers working in the natural and life sciences meaningfully? A proposal of methods based on percentiles of citations. Scientometrics. https://doi.org/10.1007/s11192-013-1161-y
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.