Methods for the Generation of Normalized Citation Impact Scores in Bibliometrics: Which Method Best Reflects the Judgements of Experts?
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Journal impact indicators normalized by subject field have been developed to address these limitations.
“Journal impact indicators that are normalized by subject field have been developed to address these limitations [178,233].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [233]
Performance evaluation of normalization approaches also depends on the study design.
“On the other hand, performance evaluation of normalization approaches also depends on the study design [185,233].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [233]
The underlying idea of citing-side or source normalization is that in high density fields papers would likely have longer reference lists than in low density fields.
“The underlying idea of citing-side or source normalization approach is that in the high density fields papers would likely have longer reference lists than in the low density fields [233].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [233]
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. E., & Cohen, J. (1989) · Technometrics · doi:10.2307/1270020 · Cited by 99,588 · see citing works
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,406 · see citing works
Coefficient Alpha and the Internal Structure of Tests
Cronbach, L. J. (1951) · Psychometrika · doi:10.1007/bf02310555 · Cited by 43,931 · see citing works
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,385 · see citing works
(1990) · Choice Reviews Online · doi:10.5860/choice.27-3338 · Cited by 35,305 · see citing works
Hosmer, D. W., & Lemeshow, S. (2000) · doi:10.1002/0471722146 · Cited by 20,302 · see citing works
An index to quantify an individual’s scientific research output
Hirsch, J. E. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0507655102 · Cited by 11,669 · see citing works
Handbook of Parametric and Nonparametric Statistical Procedures
(2004) · Technometrics · doi:10.1198/tech.2004.s209 · Cited by 4,664 · see citing works
Using the Margins Command to Estimate and Interpret Adjusted Predictions and Marginal Effects
Williams, R. A. (2012) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x1201200209 · Cited by 2,193 · see citing works
Citation Indexing-Its Theory and Application in Science, Technology, and Humanities
Goodwin, J., & Garfield, E. (1980) · Technology and Culture · doi:10.2307/3104125 · Cited by 1,648 · see citing works
Bornmann, L. (2011) · Annual Review of Information Science and Technology · doi:10.1002/aris.2011.1440450112 · Cited by 567 · see citing works
Self-citationThe Leiden ranking 2011/2012: Data collection, indicators, and interpretation
Waltman, L., Calero‐Medina, C., Kosten, J., Noyons, E., Tijssen, R., van Eck, N. J., van Leeuwen, T. N., van Raan, A. F. J., Visser, M. S., & Wouters, P. F. (2012) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22708 · Cited by 452 · see citing works
Schubert, A. P., & Braun, T. (1986) · Scientometrics · doi:10.1007/bf02017249 · Cited by 450 · see citing works
Towards a new crown indicator: Some theoretical considerations
Waltman, L., van Eck, N. J., van Leeuwen, T. N., Visser, M. S., & van Raan, A. F. J. (2010) · Journal of Informetrics · doi:10.1016/j.joi.2010.08.001 · Cited by 438 · see citing works
Psychological relevance and information science
Harter, S. P. (1992) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(199210)43:9 · Cited by 415 · see citing works
Bornmann, L., Mutz, R., Neuhaus, C., & Daniel, H. (2007) · Ethics in Science and Environmental Politics · doi:10.3354/esep00084 · Cited by 351 · see citing works
Self-citationMeasuring Scholarly Impact: Methods and Practice
Ding, Y., Rousseau, R., & Wolfram, D. (2014) · Cited by 335 · see citing works
Power Laws in the Information Production Process: Lotkaian Informetrics
Egghe, L. (2005) · Emerald Group Publishing Limited eBooks · doi:10.1108/s1876-0562(2005)05 · Cited by 297 · see citing works
Storage to be Provided in Impounding Municipal Water Supply
Hazen, A. (1914) · Transactions of the American Society of Civil Engineers · doi:10.1061/taceat.0002563 · Cited by 249 · see citing works
Bornmann, L., Loet Leydesdorff, A., & Mutz, R. (2012) · Journal of Informetrics · doi:10.1016/j.joi.2012.10.001 · Cited by 205 · see citing works
Self-citationUsers, narcissism and control – tracking the impact of scholarly publications in the 21st century
Costas, R., & Wouters, P. F. (2012) · Cited by 195 · see citing works
Modifying the journal impact factor by fractional citation weighting: The audience factor
Zitt, M., & Small, H. G. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20880 · Cited by 184 · see citing works
Zitt, M., Ramanana-Rahary, S., & Bassecoulard, E. (2005) · Scientometrics · doi:10.1007/s11192-005-0218-y · Cited by 182 · see citing works
The new Excellence Indicator in the World Report of the SCImago Institutions Rankings 2011
Bornmann, L., Anegón, F. D. M., & Loet Leydesdorff, A. (2012) · Journal of Informetrics · doi:10.1016/j.joi.2011.11.006 · Cited by 165 · see citing works
Self-citationWaltman, L., & van Eck, N. J. (2013) · Scientometrics · doi:10.1007/s11192-012-0913-4 · Cited by 155 · see citing works
A Systematic Empirical Comparison of Different Approaches for Normalizing Citation Impact Indicators
Waltman, L., & van Eck, N. J. (2013) · Journal of Informetrics · doi:10.1016/j.joi.2013.08.002 · Cited by 153 · see citing works
Waltman, L., & Costas, R. (2013) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23040 · Cited by 132 · see citing works
Bornmann, L., & Leydesdorff, L. (2013) · Journal of Informetrics · doi:10.1016/j.joi.2012.12.003 · Cited by 126 · see citing works
Self-citationLoet Leydesdorff, A., & Bornmann, L. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21450 · Cited by 123 · see citing works
Self-citationRuiz‐Castillo, J., & Waltman, L. (2014) · Journal of Informetrics · doi:10.1016/j.joi.2014.11.010 · Cited by 114 · see citing works
Nine Criteria for a Measure of Scientific Output
Kreiman, G., & Maunsell, J. H. R. (2011) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2011.00048 · Cited by 98 · see citing works
On the causes of subject-specific citation rates in Web of Science
Marx, W., & Bornmann, L. (2014) · Scientometrics · doi:10.1007/s11192-014-1499-9 · Cited by 85 · see citing works
Self-citationBornmann, L., & Williams, R. A. (2013) · Journal of Informetrics · doi:10.1016/j.joi.2013.02.005 · Cited by 76 · see citing works
Self-citationBornmann, L., Loet Leydesdorff, A., & Wang, J. (2013) · Journal of Informetrics · doi:10.1016/j.joi.2013.09.003 · Cited by 55 · see citing works
Self-citationThe wisdom of citing scientists
Bornmann, L., & Marx, W. (2013) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23100 · Cited by 52 · see citing works
Self-citationPost‐publication filtering and evaluation: Faculty of 1000
Wets, K., Weedon, D., & Velterop, J. (2003) · Learned Publishing · doi:10.1087/095315103322421982 · Cited by 47 · see citing works
Calibrating the zoom — a test of Zitt’s hypothesis
Adams, J. R., Gurney, K., & Jackson, L. T. (2008) · Scientometrics · doi:10.1007/s11192-007-1832-7 · Cited by 40 · see citing works
Interrater reliability and convergent validity of F 1000 P rime peer review
Bornmann, L. (2015) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23334 · Cited by 34 · see citing works
Self-citationField‐normalized impact factors ( IFs ): A comparison of rescaling and fractionally counted IFs
Loet Leydesdorff, A., Radicchi, F., Bornmann, L., Castellano, C., & de Nooy, W. (2013) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22911 · Cited by 32 · see citing works
Self-citationWilliams, R. A., & Bornmann, L. (2014) · Springer eBooks · doi:10.1007/978-3-319-10377-8_12 · Cited by 18 · see citing works
Self-citationField-normalized Impact Factors: A Comparison of Rescaling versus Fractionally Counted IFsPreprint
Loet Leydesdorff, A., Radicchi, F., Bornmann, L., Castellano, C., & de Nooy, W. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1211.2571 · Cited by 12 · see citing works
Self-citationFrom P100 to P100': A new citation‐rank approach
Bornmann, L., & Mutz, R. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23152 · Cited by 12 · see citing works
Self-citationSchreiber, M. (2014) · Journal of Informetrics · doi:10.1016/j.joi.2014.06.007 · Cited by 9 · see citing works
From P100 to P100_: Conception and improvement of a new citation-rank approach in bibliometricsPreprint
Bornmann, L., & Mutz, R. (2013) · arXiv (Cornell University) · doi:10.48550/arxiv.1307.0667 · Cited by 6 · see citing works
Self-citationWaltman, L., & Ruiz‐Castillo, J. (2014) · RePEc: Research Papers in Economics · Cited by 4 · see citing works
Is the new citation-rank approach P100′ in bibliometrics really new?
Schreiber, M. (2014) · Journal of Informetrics · doi:10.1016/j.joi.2014.10.001 · Cited by 3 · see citing works
Is the new citation-rank approach P100′ in bibliometrics really new?
Schreiber, M. (n.d.) · RePEc: Research Papers in Economics · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World
Pranckutė, R. (2021) · Publications · doi:10.3390/publications9010012 · Cited by 2,696 · see citing works
Papers and patents are becoming less disruptive over time
Park, M., Leahey, E., & Funk, R. J. (2023) · Nature · doi:10.1038/s41586-022-05543-x · Cited by 899 · see citing works
Bibliometrics: Tracking Research Impact by Selecting the Appropriate Metrics
Agarwal, A., Durairajanayagam, D., Tatagari, S., Esteves, S., Harlev, A., Henkel, R., Roychoudhury, S., Homa, S., Puchalt, N., Ramasamy, R., et al. (2016) · Asian Journal of Andrology · doi:10.4103/1008-682x.171582 · Cited by 639 · see citing works
Climate Change Research in View of Bibliometrics
Haunschild, R., Bornmann, L., & Marx, W. (2016) · PLOS ONE · doi:10.1371/journal.pone.0160393 · Cited by 341 · see citing works
Self-citationMingers, J., & Yang, L. (2017) · European Journal of Operational Research · doi:10.1016/j.ejor.2016.07.058 · Cited by 184 · see citing works
Bornmann, L., & Leydesdorff, L. (2017) · Journal of Informetrics · doi:10.1016/j.joi.2016.12.001 · Cited by 72 · see citing works
Self-citationCritical Rationalism and the Search for Standard (Field-Normalized) Indicators in Bibliometrics
Bornmann, L., & Marx, W. (2018) · Journal of Informetrics · doi:10.1016/j.joi.2018.05.002 · Cited by 41 · see citing works
Self-citation
- Authors
- Bornmann, L., & Marx, W.
- Affiliations
- Max Planck Society (DE)
- Max Planck Institute for Solid State Research (DE)
- Corresponding author
- Bornmann, Lutz
- Venue
- Journal of Informetrics
- Publisher
- Elsevier BV
- Published
- 2015-04
- Volume
- 9
- Issue
- 2
- Pages
- 408-418
- ISSN
- 1751-1577 (ISSN-L)
- OpenAlex
- W1980233487
- Cited by
- 138
- References
- 47
- Citations by year
- 2026: 9 · 2025: 9 · 2024: 14 · 2023: 14 · 2022: 5 · 2021: 14 · 2020: 18 · 2019: 19 · 2018: 7 · 2017: 11 · 2016: 14 · 2015: 4
- FWCI
- 10.95
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Informetrics (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- bibliometric indicators, peer assessment, post-publication peer review, time normalization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
