The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Use of the Journal Impact Factor in academic review, promotion, and tenure evaluations
Open accessAbstract
We analyzed how often and in what ways the Journal Impact Factor (JIF) is currently used in review, promotion, and tenure (RPT) documents of a representative sample of universities from the United States and Canada. 40% of research-intensive institutions and 18% of master’s institutions mentioned the JIF, or closely related terms. Of the institutions that mentioned the JIF, 87% supported its use in at least one of their RPT documents, 13% expressed caution about its use, and none heavily criticized it or prohibited its use. Furthermore, 63% of institutions that mentioned the JIF associated the metric with quality, 40% with impact, importance, or significance, and 20% with prestige, reputation, or status. We conclude that use of the JIF is encouraged in RPT evaluations, especially at research-intensive universities, and that there is work to be done to avoid the potential misuse of metrics like the JIF.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Bibliometrics: The Leiden Manifesto for Research Metrics
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015) · Nature · doi:10.1038/520429a · Cited by 2,828
The History and Meaning of the Journal Impact Factor
Garfield, E. (2006) · JAMA · doi:10.1001/jama.295.1.90 · Cited by 2,770
Why the impact factor of journals should not be used for evaluating research
Seglen, P. O. (1997) · BMJ · doi:10.1136/bmj.314.7079.497 · Cited by 2,447 · cited 2×
“…the JIF is being used to evaluate the quality and384 significance of research, despite the numerous warnings against such use (Brembs, 2018; Brembs385 et al., 2013; Haustein & Larivière, 2015; Kurmis, 2003; Moustafa, 2015; Seglen, 1997; Sugimoto386 & Larivière, 2018; The Analogue University, 2019).”
Believe it or not: how much can we rely on published data on potential drug targets?
Prinz, F., Schlange, T., & Asadullah, K. (2011) · Nature Reviews Drug Discovery · doi:10.1038/nrd3439-c1 · Cited by 1,888 · cited 1×
“Prinz et al. (2011) found, “that the reproducibility of327 published data did not significantly correlate with journal impact factors” (pg. 2).328 Thus, at least as viewed through the aspects above, there is little to no evidence to justify a329 relationship between the JIF and research quality.”
Szucs, D., & Ioannidis, J. P. A. (2017) · PLOS Biology · doi:10.1371/journal.pbio.2000797 · Cited by 761
Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S., Hill, S. A., Jones, R. D., Kain, R. J. P., Kerridge, S., Thelwall, M. A., et al. (2015) · Digital Education Resource Archive (University College London) · doi:10.13140/rg.2.1.4929.1363 · Cited by 728
Assessing scientists for hiring, promotion, and tenure
Moher, D., Naudet, F., Cristea, I. A., Miedema, F., Ioannidis, J. P. A., & Goodman, S. N. (2018) · PLOS Biology · doi:10.1371/journal.pbio.2004089 · Cited by 462
Deep impact: unintended consequences of journal rank
Brembs, B., Button, K., & Munafò, M. (2013) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2013.00291 · Cited by 441
Fraley, R. C., & Vazire, S. (2014) · PLoS ONE · doi:10.1371/journal.pone.0109019 · Cited by 409
Harley, D., Acord, S. K., Earl‐Novell, S., Lawrence, S., & King, C. J. (2010) · RePEc: Research Papers in Economics · Cited by 384
Editors, T. P. M. (2006) · PLoS Medicine · doi:10.1371/journal.pmed.0030291 · Cited by 373
The evaluation of scholarship in academic promotion and tenure processes: Past, present, and future
Schimanski, L. A., & Alperin, J. P. (2018) · F1000Research · doi:10.12688/f1000research.16493.1 · Cited by 342
Self-citationHistory of the journal impact factor: Contingencies and consequences
Archambault, É., & Larivière, V. (2009) · Scientometrics · doi:10.1007/s11192-007-2036-x · Cited by 325
Quan (2017) · doi:10.1108/ajim-01-2017-0014 · Cited by 306 · cited 1×
“…There are even institutions in some countries that financially reward their faculty for publishing in41 journals with high JIFs (Fuyuno & Cyranoski, 2006; Quan et al., 2017), demonstrating an extreme42 but important example of how reliance on this metric may be distorting academic incentives.”
UNDERSTANDING THE LIMITATIONS OF THE JOURNAL IMPACT FACTOR
KURMIS, A. P. (2003) · The Journal of Bone and Joint Surgery-American Volume · doi:10.2106/00004623-200312000-00028 · Cited by 298
The need for transparency and good practices in the qPCR literature
Bustin, S. A., Benes, V., Garson, J., Hellemans, J., Huggett, J., Kubista, M., Mueller, R., Nolan, T., Pfaffl, M. W., Shipley, G., et al. (2013) · Nature Methods · doi:10.1038/nmeth.2697 · Cited by 281
The Use of Bibliometrics for Assessing Research: Possibilities, Limitations and Adverse Effects
Haustein, S., & Larivière, V. (2015) · Incentives and Performance · doi:10.1007/978-3-319-09785-5_8 · Cited by 276
The National Academies of SCIENCES • ENGINEERING • MEDICINE
Earth, D. O. (2016) · Cited by 231
Prestigious Science Journals Struggle to Reach Even Average Reliability
Brembs, B. (2018) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2018.00037 · Cited by 213 · cited 3×
“Although quality296 is hard to define, and even harder to measure, there are some aspects of methodological rigor297 which could be considered indicative of quality, such as sample sizes, experimental design, and298 reproducibility (Brembs, 2018).”
InfluentialHecht, F., Hecht, B. K., & Sandberg, A. A. (1998) · Cancer Genetics and Cytogenetics · doi:10.1016/s0165-4608(97)00459-7 · Cited by 206
Causes for the Persistence of Impact Factor Mania
Casadevall, A., & Fang, F. C. (2014) · mBio · doi:10.1128/mbio.00064-14 · Cited by 196
Uncovering the Values in Faculty Evaluation of Service as Scholarship
O'Meara, K. (2002) · The Review of Higher Education · doi:10.1353/rhe.2002.0028 · Cited by 194
Measuring Research: What Everyone Needs to Know®
Sugimoto, C. R., & Larivière, V. (2018) · doi:10.1093/wentk/9780190640118.001.0001 · Cited by 184
How significant are the public dimensions of faculty work in review, promotion and tenure documents?
Alperin, J. P., Muñoz Nieves, C., Schimanski, L. A., Fischman, G. E., Niles, M. T., & McKiernan, E. C. (2019) · eLife · doi:10.7554/elife.42254 · Cited by 178
Self-citationCitation indexes in sociological and historical research
Garfield, E. (1963) · American Documentation · doi:10.1002/asi.5090140405 · Cited by 175
Mobley, A., Linder, S. K., Braeuer, R., Ellis, L. M., & Zwelling, L. (2013) · PLoS ONE · doi:10.1371/journal.pone.0063221 · Cited by 166
Perceptions of open access publishing: interviews with journal authors
Schroter, S., Tite, L., & Smith, R. W. (2005) · BMJ · doi:10.1136/bmj.38359.695220.82 · Cited by 134
Wilsdon, J., Bar‐Ilan, J., Frodeman, R., Lex, E., Peters, I., & Wouters, P. F. (2017) · Leiden Repository (Leiden University) · doi:10.2777/337729 · Cited by 119
Tijdink, J. K., Schipper, K., Bouter, L. M., Pont, P. M., de Jonge, J., & Smulders, Y. M. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-008681 · Cited by 109
The Disaster of the Impact Factor
Moustafa, K. (2015) · Science and Engineering Ethics · doi:10.1007/s11948-014-9517-0 · Cited by 90
Authors' opinions on publication in relation to annual performance assessment
Walker, R. L., Sykes, L., Hemmelgarn, B. R., & Quan, H. (2010) · BMC Medical Education · doi:10.1186/1472-6920-10-21 · Cited by 86
Risk of bias of randomized controlled trials published in orthopaedic journals
Chess, L. E., & Gagnier, J. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-76 · Cited by 83
Cash for papers: putting a premium on publication
Fuyuno, I., & Cyranoski, D. (2006) · Nature · doi:10.1038/441792b · Cited by 80
Next-generation altmetrics: responsible metrics and evaluation for open science
Conference, A., & Bar‐Ilan, J. (2016) · Figshare · doi:10.6084/m9.figshare.3978573.v1 · Cited by 72
Bias in genetic association studies and impact factor
Munafò, M. R., Stothart, G., & Flint, J. (2009) · Molecular Psychiatry · doi:10.1038/mp.2008.77 · Cited by 60
ISC/OSI Journal Authors Survey Report
Swan, A. P., & Brown, S. (2004) · ePrints Soton (University of Southampton) · Cited by 46
Validity of the Impact Factor of Journals as a Measure of Randomized Controlled Trial Quality
Barbui, C., Cipriani, A., Malvini, L., & Tansella, M. (2006) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.v67n0106 · Cited by 35
The Leiden Manifesto under review: what libraries can learn from it
Coombs, S. K., & Peters, I. (2017) · Digital Library Perspectives · doi:10.1108/dlp-01-2017-0004 · Cited by 15
Calling all journal editors: Bury the metrics pages!
(2019) · Political Geography · doi:10.1016/j.polgeo.2018.09.002 · Cited by 8
Research works printed but not linked: 12
- Loyola University Chicago. 2015. Chicago: Loyola University. The Institute of Environmental Sustainability: Tenure and Promotion Guidelines.
- Ohio University. 2015. Ohio University. Russ College of Engineering and Technology: Minimal Criteria for Promotion and Tenure.
- Simon Fraser University. 2015. Simon Fraser University. Department of Psychology: Criteria for Promotion.
- University of Alberta. 2012. University of Alberta. Faculty of Science: Criteria for Merit Increments, Tenure and Promotion.
- University of Calgary. 2008. University of Calgary. Faculty of Veterinary Medicine: Guidelines for Appointment, Promotion, and Tenure of Academic Staff.
- University of California, San Diego. 2016. University of California. Where CAP Stood, 2015-16.
- University of Central Florida. 2014. University of Central Florida. School of Social Work: Criteria for Promotion and Tenure.
- University of Central Florida. 2015. Department Criteria for Tenure and Promotion. Department of Sociology.
- University of Guelph. 2008. University of Guelph. Department Human Health and Nutritiona Sciences: Guidelines for Tenure, Promotion and Performance Assessment.
- University of Southern Mississippi, 2016, Department of Political Science, International Development, and International Affairs
- University of Vermont. 2016. University of Vermont. Reappointment, Promotion, and Tenure (RPT) Guidelines and Forms –Green Sheet Form.
- University of Windsor. 2016. University of Windsor. Working Session on Developing Promotion and Tenure Criteria for Research.
Cited by
The limitations to our understanding of peer review
Tennant, J. P., & Ross-Hellauer, T. (2020) · Research Integrity and Peer Review · doi:10.1186/s41073-020-00092-1 · Cited by 384
Rice, D. B., Raffoul, H., Ioannidis, J. P. A., & Moher, D. (2020) · BMJ · doi:10.1136/bmj.m2081 · Cited by 241
A review on blockchain technology and blockchain projects fostering open science
Leible, S., Schlager, S., Schubotz, M., & Gipp, B. (2019) · Frontiers in Blockchain · doi:10.3389/fbloc.2019.00016 · Cited by 134
Chiarelli, A., Johnson, R., Pinfield, S., & Richens, E. (2019) · F1000Research · doi:10.12688/f1000research.19619.2 · Cited by 79
‘Stamp out paper mills’ — science sleuths on how to fight fake research
Abalkina, A., Aquarius, R., Bik, E., Bimler, D., Bishop, D., Byrne, J., Cabanac, G., Day, A., Labbé, C., & Wise, N. (2025) · Nature · doi:10.1038/d41586-025-00212-1 · Cited by 59
Metadata
- Authors
- McKiernan, E. C., Schimanski, L. A., Muñoz Nieves, C., Matthias, L., Niles, M. T., & Alperin, J. P.
- Venue
- eLife
- Publisher
- eLife Sciences Publications, Ltd
- Published
- 2019-07-31
- Volume
- 8
- ISSN
- 2050-084X (ISSN-L)
- OpenAlex
- W2940242975
- Cited by
- 347
- References
- 39
- FWCI
- 25.71
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- creativecommons.org/licenses/by/4.0/ (vor)
- creativecommons.org/licenses/by/4.0/ (am)
- creativecommons.org/licenses/by/4.0/ (tdm)
- Funders
- Open Society Foundations (OR2016-29841)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.