Evaluating Research and Researchers by the Journal Impact Factor: Is It Better than Coin Flipping?
Open accessThe claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Values of mean-based indicators can be highly distorted by the presence of a few highly cited publications.
“the values of mean-based indicators can be highly distorted by the presence of few highly cited publications [44,45,221,224,277-279].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [277]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Why the impact factor of journals should not be used for evaluating research
Seglen, P. O. (1997) · BMJ · doi:10.1136/bmj.314.7079.497 · Cited by 2,447 · cited 1× · see citing works
“Despite these notable rejections of the use of the JIF in research evaluation, the method is still in use, and opposite positions have been qualified and debated (Archambault & Larivière, 2009; Bornmann & Williams, 2017a; Bornmann & Williams, 2017b; Callaway, 2016; Hammarfelt & Rushforth, 2017; Peters, 2017; Seglen, 1…”
MethodA review of the literature on citation impact indicators
Waltman, L. (2016) · Journal of Informetrics · doi:10.1016/j.joi.2016.02.007 · Cited by 1,356 · see citing works
Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S., Hill, S. A., Jones, R. D., Kain, R. J. P., Kerridge, S., Thelwall, M. A., et al. (2015) · Digital Education Resource Archive (University College London) · doi:10.13140/rg.2.1.4929.1363 · Cited by 738 · see citing works
Bias against novelty in science: A cautionary tale for users of bibliometric indicators
Wang, J., Veugelers, R., & Stephan, P. E. (2017) · Research Policy · doi:10.1016/j.respol.2017.06.006 · Cited by 545 · see citing works
A conceptual and empirical examination of justifications for dichotomization.
Decoster, J., Iselin, A.-M. R., & Gallucci, M. (2009) · Psychological methods · doi:10.1037/a0016956 · Cited by 483 · see citing works
Schubert, A. P., & Braun, T. (1986) · Scientometrics · doi:10.1007/bf02017249 · Cited by 450 · see citing works
Lawrence, P. A. (2003) · Nature · doi:10.1038/422259a · Cited by 354 · see citing works
History of the journal impact factor: Contingencies and consequences
Archambault, É., & Larivière, V. (2009) · Scientometrics · doi:10.1007/s11192-007-2036-x · Cited by 326 · see citing works
Beat it, impact factor! Publishing elite turns against controversial metric
Callaway, E. (2016) · Nature · doi:10.1038/nature.2016.20224 · Cited by 220 · see citing works
A simple proposal for the publication of journal citation distributionsPreprint
Larivière, V., Kiermer, V., MacCallum, C. J., Mcnutt, M. K., Patterson, M., Pulverer, B., Swaminathan, S., Taylor, S., & Curry, S. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/062109 · Cited by 216 · see citing works
Leydesdorff, L., Wouters, P., & Bornmann, L. (2016) · Scientometrics · doi:10.1007/s11192-016-2150-8 · Cited by 173 · see citing works
Cross-field normalization of scientometric indicators
Schubert, A. P., & Braun, T. (1996) · Scientometrics · doi:10.1007/bf02129597 · Cited by 150 · see citing works
Core Elements in the Process of Citing Publications: Conceptual Overview of the Literature
Tahamtan, I., & Bornmann, L. (2018) · Journal of Informetrics · doi:10.1016/j.joi.2018.01.002 · Cited by 149 · see citing works
Bornmann, L., & Marx, W. (2013) · Scientometrics · doi:10.1007/s11192-013-1161-y · Cited by 143 · see citing works
Citations: Indicators of Quality? The Impact Fallacy
Loet Leydesdorff, A., Bornmann, L., Comins, J. A., & Milojević, S. (2016) · Frontiers in Research Metrics and Analytics · doi:10.3389/frma.2016.00001 · Cited by 142 · see citing works
Indicators as judgment devices: An empirical study of citizen bibliometrics in research evaluation
Hammarfelt, B., & Rushforth, A. (2017) · Research Evaluation · doi:10.1093/reseval/rvx018 · Cited by 126 · see citing works
Bibliometrics and Citation Analysis
(2012) · doi:10.5040/9798400675966.ch-011 · Cited by 111 · see citing works
Bornmann, L., & Marx, W. (2012) · EMBO Reports · doi:10.1038/embor.2013.9 · Cited by 92 · see citing works
A combined bibliometric indicator to predict article impact
Levitt, J. M., & Thelwall, M. A. (2010) · Information Processing & Management · doi:10.1016/j.ipm.2010.09.005 · Cited by 88 · see citing works
Zhang, L., Rousseau, R., & Sivertsen, G. (2017) · PLoS ONE · doi:10.1371/journal.pone.0174205 · Cited by 83 · see citing works
Citations versus journal impact factor as proxy of quality: could the latter ever be preferable?
Abramo, G., D’Angelo, C. A., & Costa, F. D. (2010) · Scientometrics · doi:10.1007/s11192-010-0200-1 · Cited by 80 · see citing works
Bornmann, L., & Williams, R. A. (2017) · Journal of Informetrics · doi:10.1016/j.joi.2017.06.001 · Cited by 79 · see citing works
Systematic Analysis of Agreement between Metrics and Peer Review in the UK REF
Traag, V. A., & Waltman, L. (2019) · Palgrave Communications · doi:10.1057/s41599-019-0233-x · Cited by 65 · see citing works
Impact factors, and why they won't go away
Garfield, E. (2001) · Nature · doi:10.1038/35079156 · Cited by 62 · see citing works
How will you judge me if not by impact factor?
Tregoning, J. S. (2018) · Nature · doi:10.1038/d41586-018-05467-5 · Cited by 58 · see citing works
Citation Success Index − An Intuitive Pair-Wise Journal Comparison Metric
Milojević, S., Radicchi, F., & Bar-Ilan, J. (2017) · Journal of Informetrics · doi:10.1016/j.joi.2016.12.006 · Cited by 33 · see citing works
Bias against Novelty in Science: A Cautionary Tale for Users of Bibliometric IndicatorsPreprint
Wang, J., Veugelers, R., & Stephan, P. E. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3051728 · Cited by 32 · see citing works
Use of the journal impact factor for assessing individual articles need not be wrong.Preprint
Waltman, L., & Traag, V. (2017) · arXiv (Cornell University) · Cited by 28 · see citing works
Citation count distributions for large monodisciplinary journals
Thelwall, M. A. (2016) · Journal of Informetrics · doi:10.1016/j.joi.2016.07.006 · Cited by 20 · see citing works
Double rank analysis for research assessment
Rodríguez‐Navarro, A., & Brito, R. (2017) · Journal of Informetrics · doi:10.1016/j.joi.2017.11.004 · Cited by 16 · see citing works
Self-citationPeters, G.-J. (2017) · Journal of Informetrics · doi:10.1016/j.joi.2017.08.001 · Cited by 8 · see citing works
Bornmann, L., & Williams, R. A. (2017) · Journal of Informetrics · doi:10.1016/j.joi.2017.08.005 · Cited by 4 · see citing works
Rodríguez‐Navarro, A., & Brito, R. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.07285 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World
Pranckutė, R. (2021) · Publications · doi:10.3390/publications9010012 · Cited by 2,696 · see citing works
- Authors
- Brito, R., & Rodríguez-Navarro, A.
- Affiliations
- Universidad Complutense de Madrid (ES)
- Corresponding author
- Brito, Ricardo
- ORCID
- Brito, Ricardo
- Venue
- Journal of Informetrics
- Publisher
- Elsevier BV
- Published
- 2019-02
- Volume
- 13
- Issue
- 1
- Pages
- 314-324
- ISSN
- 1751-1577 (ISSN-L)
- OpenAlex
- W2894106314
- Cited by
- 47
- References
- 33
- Citations by year
- 2026: 1 · 2025: 4 · 2024: 5 · 2023: 5 · 2022: 8 · 2021: 10 · 2020: 4 · 2019: 10
- FWCI
- 5.35
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Journal of Informetrics (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- arXiv (Cornell University)
- Funders
- Spanish Ministerio de Economía y Competitividad (FIS2014-52486-R, FIS2017-83709-R)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
- Keywords
- journal impact factor, research evaluation, researcher evaluation, citation analysis, failure probability, bibliometric analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
