Citation-Based Journal Rankings: Key Questions, Metrics, and Data Sources

Walters, William H.

doi:10.1109/access.2017.2761400

Cited by 56scientometrics and bibliometrics researchWeb visibility and informetricsPsychology Research and Bibliometrics

Citation-Based Journal Rankings: Key Questions, Metrics, and Data Sources

Open access

Walters, William H.·2017·IEEE Access

DOI 10.1109/access.2017.2761400

Open this paper in CoChat — free account

Abstract

This guide presents nine key questions that can help researchers make good use of citation-based journal rankings (metrics) in the natural and social sciences. The nine questions address the characteristics that distinguish one metric from another: the source documents, the citation-counting window, the document types counted, the cited-document window, the impact of highly cited documents, the treatment of self-citations, the distinction between size-dependent and size-independent metrics, the use of normalization to account for disciplinary differences in impact, and the use of weighting to account for the impact or centrality of each citing journal. Next, the guide reviews 19 standard citation metrics, including the h index, g index, impact factor, source normalized impact per paper, eigenfactor, article influence score, and SCImago journal rank. Three underlying data sources (Web of Science, Scopus, and Google Scholar) are described, along with six major data download sites: Journal Citation Reports, Eigenfactor, CWTS Journal Indicators, SCImago, Scopus Journal Metrics, Cabell's International, and Google Scholar Metrics. The paper summarizes the main criticisms of citation metrics and concludes with suggestions for their further development, dissemination, and use.

Integrity signals

Registry

Crossref · 10.1109/access.2017.2761400

Cites

178 of 178 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

JIF shows a certain level of discrimination against less popular publication types that, in certain disciplines, are as important as original research articles.
“In this way, JIF also shows a certain level of discrimination against less popular publication types that, in certain disciplines, are as important as original research articles [2,26,45,46,224,225,229].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [45]
WoS also calculates an Immediacy index, which is basically a one-year JIF.
“WoS also calculates an Immediacy index, which is basically a one-year JIF [26,45].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [45]
In the JIF numerator, citations are counted to all document types, and only specific types of documents, so-called 'citable items' (articles, reviews, and proceedings papers), are included in the denominator.
“In the numerator, citations are counted to all document types, and only specific types of documents, so-called 'citable items', namely articles, reviews, and proceedings papers are included in the denominator, which represents the total amount of cited papers [45,224].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [45]
Journals belonging to A&HCI and ESCI are not included in JCR and therefore do not have JIF or other journal impact metrics.
“since journals belonging to A&HCI and ESCI are not included in JCR, they do not have JIF or other journal impact metrics”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [45]
The five-year window makes Eigenfactor metrics particularly suitable for Social Sciences and Humanities, where citations accrue more slowly.
“the five-year window applied in the calculation of all Eigenfactor metrics (ES and AIS) makes them particularly suitable for application in Social Sciences and Humanities, where citations accrue more slowly [45].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [45]

The JIF version without self-citations, calculated in the same way as classical JIF but excluding journal self-citations from the numerator, is available in JCR.
“On the other hand, the JIF version without self-citations, which is calculated in the same way as the classical JIF, but excluding journal self-citations from the numerator, is also available in JCR [45].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [45]
All Eigenfactor metrics (ES and AIS) use a five-year window in their calculation.
“the five-year window applied in the calculation of all Eigenfactor metrics (ES and AIS)”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [45]
The lack of JIF and other journal impact metrics for A&HCI and ESCI journals makes them inapplicable for Arts and Humanities evaluations.
“they do not have JIF or other journal impact metrics, which makes them inapplicable for Arts and Humanities evaluations [45,224].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [45]
Values of mean-based indicators can be highly distorted by the presence of a few highly cited publications.
“the values of mean-based indicators can be highly distorted by the presence of few highly cited publications [44,45,221,224,277-279].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [45]
Citations can be negative, raising the question about their impact on the quality of the cited document.
“citations can also be negative, raising the question about their impact to the quality of the cited document [45].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [45]
Ability to use DBs as data sources for large-scale bibliographic analyses may be hindered by data export and accessibility limitations.
“the ability to use DBs as a data sources for large-scale bibliographic analyses may also be hindered by certain data export and accessibility limitations [45].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [45]

An index to quantify an individual’s scientific research output
Hirsch, J. E. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0507655102 · Cited by 11,669 · see citing works
The History and Meaning of the Journal Impact Factor
Garfield, E. (2006) · JAMA · doi:10.1001/jama.295.1.90 · Cited by 2,770 · see citing works
Why the impact factor of journals should not be used for evaluating research
Seglen, P. O. (1997) · BMJ · doi:10.1136/bmj.314.7079.497 · Cited by 2,447 · see citing works
Theory and practise of the g-index
Egghe, L. (2006) · Scientometrics · doi:10.1007/s11192-006-0144-7 · Cited by 2,354 · see citing works
Citation Indexes for Science: A New Dimension in Documentation through Association of Ideas
Garfield, E. (1955) · Science · doi:10.1126/science.122.3159.108 · Cited by 2,334 · see citing works
Google Scholar, Scopus and the Web of Science: A Longitudinal and Cross-Disciplinary Comparison
Harzing, A.-W., & Alakangas, S. (2016) · Scientometrics · doi:10.1007/s11192-015-1798-9 · Cited by 1,784 · see citing works
Citation Indexing-Its Theory and Application in Science, Technology, and Humanities
Goodwin, J., & Garfield, E. (1980) · Technology and Culture · doi:10.2307/3104125 · Cited by 1,648 · see citing works
A review of the literature on citation impact indicators
Waltman, L. (2016) · Journal of Informetrics · doi:10.1016/j.joi.2016.02.007 · Cited by 1,356 · see citing works
Mapping the backbone of science
Boyack, K. W., Klavans, R., & Börner, K. (2005) · Scientometrics · doi:10.1007/s11192-005-0255-6 · Cited by 953 · see citing works
A new approach to the metric of journals’ scientific prestige: The SJR indicator
González-Pereira, B., Guerrero‐Bote, V. P., & Anegón, F. D. M. (2010) · Journal of Informetrics · doi:10.1016/j.joi.2010.03.002 · Cited by 867 · see citing works
Comparisons of Citations in Web of Science, Scopus, and Google Scholar for Articles Published in General Medical Journals
Kulkarni, A. V. (2009) · JAMA · doi:10.1001/jama.2009.1307 · Cited by 821 · see citing works
Comparing alternatives to the web of science for coverage of the social sciences’ literature
Norris, M., & Oppenheim, C. (2007) · Journal of Informetrics · doi:10.1016/j.joi.2006.12.001 · Cited by 733 · see citing works
College Libraries and Chemical Education
Gross, P. L. K., & Gross, E. M. (1927) · Science · doi:10.1126/science.66.1713.385 · Cited by 649 · see citing works
Measuring Contextual Citation Impact of Scientific Journals
Moed, H. F. (2010) · Journal of Informetrics · doi:10.1016/j.joi.2010.01.002 · Cited by 628 · see citing works
Citation influence for journal aggregates of scientific publications: Theory, with application to the literature of physics
Pinski, G., & Narin, F. (1976) · Information Processing & Management · doi:10.1016/0306-4573(76)90048-0 · Cited by 601 · see citing works
Scholarly use of social media and altmetrics: A review of the literature
Sugimoto, C. R., Work, S., Larivière, V., & Haustein, S. (2017) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23833 · Cited by 573 · see citing works
Assessing basic research
Martin, B. R., & Irvine, J. M. (1983) · Research Policy · doi:10.1016/0048-7333(83)90005-7 · Cited by 529 · see citing works
Untitled work
(2010) · Cited by 528 · see citing works
A global map of science based on the ISI subject categories
Loet Leydesdorff, A., & Ràfols, I. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20967 · Cited by 509 · see citing works
Eigenfactor: Measuring the value and prestige of scholarly journals
Bergstrom, C. T. (2007) · College & Research Libraries News · doi:10.5860/crln.68.5.7804 · Cited by 479 · see citing works
Assessing the Relative Impacts of Economics Journals
Liebowitz, S. J., & Palmer, J. P. (1983) · Journal of Economic Literature · Cited by 453 · see citing works
The Relative Impacts of Economics Journals: 1970-1990
Laband, D. N., & Piette, M. J. (1994) · Journal of Economic Literature · Cited by 428 · see citing works
A Further Step Forward in Measuring Journals’ Scientific Prestige: The SJR2 Indicator
Guerrero-Bote, V. P., & Moya-Anegón, F. (2012) · Journal of Informetrics · doi:10.1016/j.joi.2012.07.001 · Cited by 406 · see citing works
Journal status
Bollen, J., Rodriquez, M. A., & Herbert Van De Sompel, A. (2006) · Scientometrics · doi:10.1007/s11192-006-0176-z · Cited by 403 · see citing works
How well developed are altmetrics? A cross-disciplinary analysis of the presence of ‘alternative metrics’ in scientific publications
Zahedi, Z., Costas, R., & Wouters, P. F. (2014) · Scientometrics · doi:10.1007/s11192-014-1264-0 · Cited by 386 · see citing works
Citation counting, citation ranking, and h ‐index of human‐computer interaction researchers: A comparison of Scopus and Web of Science
Meho, L. I., & Rogers, Y. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20874 · Cited by 383 · see citing works
Coverage analysis of Scopus: A journal metric approach
Anegón, F. D. M., Chinchilla‐Rodríguez, Z., Vargas‐Quesada, B., Corera‐Álvarez, E., Muñoz-Fernández, F. J., González-Molina, A., & Herrero-Solana, V. (2007) · Scientometrics · doi:10.1007/s11192-007-1681-4 · Cited by 381 · see citing works
Impact factor: outdated artefact or stepping-stone to journal certification?
Vanclay, J. K. (2011) · Scientometrics · doi:10.1007/s11192-011-0561-0 · Cited by 370 · see citing works
The inconsistency of the h‐index
Waltman, L., & van Eck, N. J. (2011) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21678 · Cited by 338 · see citing works
History of the journal impact factor: Contingencies and consequences
Archambault, É., & Larivière, V. (2009) · Scientometrics · doi:10.1007/s11192-007-2036-x · Cited by 326 · see citing works
The Number of Scholarly Documents on the Public Web
Khabsa, M., & Giles, C. L. (2014) · PLoS ONE · doi:10.1371/journal.pone.0093949 · Cited by 325 · see citing works
Popular and/or prestigious? Measures of scholarly esteem
Ding, Y., & Cronin, B. (2010) · Information Processing & Management · doi:10.1016/j.ipm.2010.01.002 · Cited by 317 · see citing works
The weakening relationship between the impact factor and papers' citations in the digital age
Lozano, G. A., Larivière, V., & Gingras, Y. (2012) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22731 · Cited by 293 · see citing works
A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants
Bornmann, L., Mutz, R., Hug, S. E., & Daniel, H. (2011) · Journal of Informetrics · doi:10.1016/j.joi.2011.01.006 · Cited by 291 · see citing works
TRENDS IN RANKINGS OF ECONOMICS DEPARTMENTS IN THE U.S.: AN UPDATE
Scott, L. C., & Mitias, P. M. (1996) · Economic Inquiry · doi:10.1111/j.1465-7295.1996.tb01383.x · Cited by 278 · see citing works
Differences in impact factor across fields and over time
Althouse, B. M., West, J. D., Bergstrom, C. T., & Bergstrom, T. (2009) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20936 · Cited by 270 · see citing works
Coverage and adoption of altmetrics sources in the bibliometric community
Haustein, S., Peters, I., Bar-Ilan, J., Priem, J., Shema, H., & Terliesner, J. (2014) · Scientometrics · doi:10.1007/s11192-013-1221-3 · Cited by 264 · see citing works
The Hirsch index and related impact measures
Egghe, L. (2010) · Annual Review of Information Science and Technology · doi:10.1002/aris.2010.1440440109 · Cited by 252 · see citing works
Scaling the h-index for different scientific ISI fields
Iglesias, J. E., & Pecharromán, C. (2007) · Scientometrics · doi:10.1007/s11192-007-1805-x · Cited by 232 · see citing works
A simple proposal for the publication of journal citation distributionsPreprint
Larivière, V., Kiermer, V., MacCallum, C. J., Mcnutt, M. K., Patterson, M., Pulverer, B., Swaminathan, S., Taylor, S., & Curry, S. (2016) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/062109 · Cited by 216 · see citing works
An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision‐Makers
Oswald, A. J. (2007) · Economica · doi:10.1111/j.1468-0335.2006.00575.x · Cited by 214 · see citing works
Caveats for the use of citation indicators in research and journal evaluations
Loet Leydesdorff, A. (2007) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20743 · Cited by 212 · see citing works
SCImago Journal & Country Rank
Krajna, T. (2008) · Polimeri : časopis za plastiku i gumu · Cited by 210 · see citing works
Using altmetrics for assessing research impact in the humanities
Hammarfelt, B. (2014) · Scientometrics · doi:10.1007/s11192-014-1261-3 · Cited by 209 · see citing works
Trends in the Usage of ISI Bibliometric Data: Uses, Abuses, and Implications
Cameron, B. (2005) · portal Libraries and the Academy · doi:10.1353/pla.2005.0003 · Cited by 199 · see citing works
Altmetrics: an analysis of the state-of-the-art in measuring research impact on social media
Erdt, M., Nagarajan, A., Sin, S. J., & Theng, Y. (2016) · Scientometrics · doi:10.1007/s11192-016-2077-0 · Cited by 193 · see citing works
Some Modifications to the SNIP Journal Impact Indicator
Waltman, L., van Eck, N. J., van Leeuwen, T. N., & Visser, M. S. (2013) · Journal of Informetrics · doi:10.1016/j.joi.2012.11.011 · Cited by 191 · see citing works
Global ranking of knowledge management and intellectual capital academic journals: 2013 update
Serenko, A., & Bontis, N. (2013) · Journal of Knowledge Management · doi:10.1108/13673271311315231 · Cited by 187 · see citing works
Fundamentals of Collection Development and Management
Johnson, P. (2004) · Cited by 181 · see citing works
Effectiveness of Journal Ranking Schemes as a Tool for Locating Information
Stringer, M. J., Sales‐Pardo, M., & Amaral, L. A. N. (2008) · PLoS ONE · doi:10.1371/journal.pone.0001683 · Cited by 179 · see citing works
The Relative Importance of Journals Used in Management Research: An Alternative Ranking
Sharplin, A. D., & Mabry, R. H. (1985) · Human Relations · doi:10.1177/001872678503800204 · Cited by 179 · see citing works
Diversity, value and limitations of the journal impact factor and alternative metrics
Bornmann, L., Marx, W., Gasparyan, A. Y., & Kitas, G. D. (2011) · Rheumatology International · doi:10.1007/s00296-011-2276-1 · Cited by 178 · see citing works
Is g-index better than h-index? An exploratory study at the individual level
Costas, R., & Bordons, M. (2008) · Scientometrics · doi:10.1007/s11192-007-1997-0 · Cited by 174 · see citing works
The Eigenfactor MetricsTM: A Network Approach to Assessing Scholarly Journals
West, J. D., Bergstrom, T. C., & Bergstrom, C. T. (2010) · College & Research Libraries · doi:10.5860/0710236 · Cited by 169 · see citing works
Feedback-seeking behavior within multinational corporations
Gupta, A. K., Govindarajan, V., & Malhotra, A. (1999) · Strategic Management Journal · doi:10.1002/(sici)1097-0266(199903)20:3 · Cited by 164 · see citing works
Scopus’s Source Normalized Impact per Paper (SNIP) versus a Journal Impact Factor Based on Fractional Counting of Citations
Leydesdorff, L., & Opthof, T. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21371 · Cited by 160 · see citing works
The Thomson Reuters Journal Selection Process
Testa, J. (2009) · Transnational Corporation Review · doi:10.1080/19186444.2009.11658213 · Cited by 159 · see citing works
Google Scholar coverage of a multidisciplinary field
Walters, W. H. (2006) · Information Processing & Management · doi:10.1016/j.ipm.2006.08.006 · Cited by 157 · see citing works
Self-citation
Source Normalized Indicators of Citation Impact: An Overview of Different Approaches and an Empirical Comparison
Waltman, L., & van Eck, N. J. (2013) · Scientometrics · doi:10.1007/s11192-012-0913-4 · Cited by 155 · see citing works
Content‐based and algorithmic classifications of journals: Perspectives on the dynamics of scientific communication and indexer effects
Ràfols, I., & Loet Leydesdorff, A. (2009) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21086 · Cited by 150 · see citing works
LIS journals scientific impact and subject categorization: a comparison between Web of Science and Scopus
Abrizah, A., Zainab, A. N., Kaur, K., & Raj, R. G. (2012) · Scientometrics · doi:10.1007/s11192-012-0813-7 · Cited by 149 · see citing works
A Theoretical and Empirical Analysis of Journal Rankings: The Case of Formal Lists
Fleet, D. D. V., McWilliams, A., & Siegel, D. S. (2000) · Journal of Management · doi:10.1016/s0149-2063(00)00060-x · Cited by 149 · see citing works
Methods for estimating the size of Google Scholar
Orduña‐Malea, E., Ayllón, J. M., Martín‐Martín, A., & López‐Cózar, E. D. (2015) · Scientometrics · doi:10.1007/s11192-015-1614-6 · Cited by 147 · see citing works
Are Articles in “Top” Management Journals Necessarily of Higher Quality?
Singh, G., Haddad, K. M., & Chow, C. W. (2007) · Journal of Management Inquiry · doi:10.1177/1056492607305894 · Cited by 145 · see citing works
The Research Assessment Exercise, the state and the dominance of mainstream economics in British universities
Lee, F. S. (2006) · Cambridge Journal of Economics · doi:10.1093/cje/bel021 · Cited by 142 · see citing works
P-Rank: An indicator measuring prestige in heterogeneous scholarly networks
Yan, E., Ding, Y., & Sugimoto, C. R. (2011) · J. Assoc. Inf. Sci. Technol. · doi:10.1002/asi.21461 · Cited by 133 · see citing works
Effect of the Web on Undergraduate Citation Behavior: Guiding Student Scholarship in a Networked Age
Davis, P. M. (2003) · portal Libraries and the Academy · doi:10.1353/pla.2003.0005 · Cited by 133 · see citing works
Ranking accounting journals using dissertation citation analysis: A research note
Chan, K. C., Chan, K. C., Seow, G. S., & Tam, K. (2009) · Accounting Organizations and Society · doi:10.1016/j.aos.2008.12.002 · Cited by 119 · see citing works
The Effect of the Web on Undergraduate Citation Behavior: A 2000 Update
Davis, P. M. (2002) · College & Research Libraries · doi:10.5860/crl.63.1.53 · Cited by 119 · see citing works
A Theoretical and Empirical Analysis of Journal Rankings: The Case of Formal Lists
Fleet, D. D. V., McWilliams, A., & Siegel, D. S. (2000) · Journal of Management · doi:10.1177/014920630002600505 · Cited by 115 · see citing works
What is the best database for computer science journal articles?
Cavacini, A. (2014) · Scientometrics · doi:10.1007/s11192-014-1506-1 · Cited by 112 · see citing works
Measuring the impact of accounting journals using Google Scholar and the g-index
Rosenstreich, D., & Wooliscroft, B. (2009) · The British Accounting Review · doi:10.1016/j.bar.2009.10.002 · Cited by 110 · see citing works
Journal rankings and the ABS Journal Quality Guide
Morris, H., Harvey, C. E., & Kelly, A. (2009) · Management Decision · doi:10.1108/00251740910995648 · Cited by 103 · see citing works
The intellectual core and impact of the knowledge management academic discipline
Serenko, A., & Bontis, N. (2013) · Journal of Knowledge Management · doi:10.1108/13673271311300840 · Cited by 101 · see citing works
An exploratory study of Google Scholar
Mayr, P., & Walter, A. (2007) · Online Information Review · doi:10.1108/14684520710841784 · Cited by 101 · see citing works
Measuring the Quality of Academic Journals: The Case of Economics
Beed, C., & Beed, C. (1996) · Journal of Post Keynesian Economics · doi:10.1080/01603477.1996.11490078 · Cited by 98 · see citing works
SJR and SNIP: two new journal metrics in Elsevier's Scopus
Colledge, L., Anegón, F. D. M., Guerrero‐Bote, V. P., López-Illescas, C., Aisati, M. E., & Moed, H. F. (2010) · Serials The Journal for the Serials Community · doi:10.1629/23215 · Cited by 98 · see citing works
Global ranking of knowledge management and intellectual capital academic journals: 2017 update
Serenko, A., & Bontis, N. (2017) · Journal of Knowledge Management · doi:10.1108/jkm-11-2016-0490 · Cited by 94 · see citing works
Author‐level Eigenfactor metrics: Evaluating the influence of authors, institutions, and countries within the social science research network community
West, J. D., Jensen, M. C., Dandrea, R. J., Gordon, G., & Bergstrom, C. T. (2013) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22790 · Cited by 92 · see citing works
Publish or perish.
van Rijswijk, L. (1999) · PubMed · Cited by 91 · see citing works
Development and application of journal impact measures in the Dutch science system
van Leeuwen, T. N., & Moed, H. F. (2002) · Scientometrics · doi:10.1023/a:1014808709694 · Cited by 86 · see citing works
The source normalized impact per paper is a valid and sophisticated indicator of journal citation impact
Moed, H. F. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21424 · Cited by 85 · see citing works
Beyond usage: measuring library outcomes and value
Tenopir, C. (2011) · Library Management · doi:10.1108/01435121211203275 · Cited by 85 · see citing works
An Additional Rating Method for Journal Articles in the Field of Management
Kacmar, K. M., & Whitfield, J. M. (2000) · Organizational Research Methods · doi:10.1177/109442810034005 · Cited by 85 · see citing works
Core Journals in Economics
Liner, G. H. (2002) · Economic Inquiry · doi:10.1093/ei/40.1.138 · Cited by 85 · see citing works
The difference between popularity and prestige in the sciences and in the social sciences: A bibliometric analysis
Franceschet, M. (2009) · Journal of Informetrics · doi:10.1016/j.joi.2009.08.001 · Cited by 83 · see citing works
JASIS and library and information science journal rankings: A review and analysis of the last half-century
Nisonger, T. E. (1999) · Journal of the American Society for Information Science · doi:10.1002/(sici)1097-4571(1999)50:11 · Cited by 82 · see citing works
Principles of information retrieval
Kochen, M. (1974) · Medical Entomology and Zoology · Cited by 81 · see citing works
The Evaluative Use of Citation Analysis for Communication Journals
HOUSER, E. T. (1996) · Human Communication Research · doi:10.1111/j.1468-2958.1996.tb00379.x · Cited by 79 · see citing works
Weighted citation: An indicator of an article's prestige
Yan, E., & Ding, Y. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21349 · Cited by 79 · see citing works
Bias in the journal impact factor
Vanclay, J. K. (2008) · Scientometrics · doi:10.1007/s11192-008-1778-4 · Cited by 77 · see citing works
The Social Science Citation Index: A Black Box—with an Ideological Bias?
Klein, D. B., Chiang, E. P., & Codes, J. (2004) · Econ journal watch · Cited by 76 · see citing works
Uncertainty Measures for Economics Journal Impact Factors
Stern, D. I. (2013) · Journal of Economic Literature · doi:10.1257/jel.51.1.173 · Cited by 76 · see citing works
A bird's-eye view of scientific trading: Dependency relations among fields of science
Yan, E., Ding, Y., Cronin, B., & Loet Leydesdorff, A. (2013) · Journal of Informetrics · doi:10.1016/j.joi.2012.11.008 · Cited by 71 · see citing works
Reward or persuasion? The battle to define the meaning of a citation
DAVIS, P. M. (2009) · Learned Publishing · doi:10.1087/095315108x378712 · Cited by 70 · see citing works
Citation Statistics as a Measure of Faculty Research Productivity
Hayes, R. M. (1983) · Journal of Education for Librarianship · doi:10.2307/40322880 · Cited by 69 · see citing works
Ranking of library and information science researchers: Comparison of data sources for correlating citation data, and expert judgments
Li, J., Sanderson, M., Willett, P., Norris, M. W., & Oppenheim, C. (2010) · Journal of Informetrics · doi:10.1016/j.joi.2010.06.005 · Cited by 67 · see citing works
Citation analysis and peer ranking of Australian social science journals
Haddow, G., & Genoni, P. (2010) · Scientometrics · doi:10.1007/s11192-010-0198-4 · Cited by 67 · see citing works
Citation Metrics: Serious Drawbacks, Perverse Incentives, and Strategic Options for Heterodox Economics
Kapeller, J. (2010) · American Journal of Economics and Sociology · doi:10.1111/j.1536-7150.2010.00750.x · Cited by 66 · see citing works
The h‐index, h‐core citation rate and the bibliometric profile of the Scopus database
Jacsó, P. (2011) · Online Information Review · doi:10.1108/14684521111151487 · Cited by 66 · see citing works
On Using Journal Rank to Proxy for an Article's Contribution or Value
Chow, C. W., Haddad, K. M., Singh, G., et al. (2007) · Issues in Accounting Education · doi:10.2308/iace.2007.22.3.411 · Cited by 65 · see citing works
Journal impact factors: implications for the nursing profession
Johnstone, M. (2007) · International Nursing Review · doi:10.1111/j.1466-7657.2007.00527.x · Cited by 62 · see citing works
Weighted citation: An indicator of an article's prestigePreprint
Yan, E., & Ding, Y. (2010) · arXiv (Cornell University) · doi:10.48550/arxiv.1012.4876 · Cited by 61 · see citing works
Editorial Essay: Why Do We Still Have Journals?
Davis, G. F. (2014) · Administrative Science Quarterly · doi:10.1177/0001839214534186 · Cited by 61 · see citing works
One in four citations in marine biology papers is inappropriate
Todd, P., Guest, Jr., Lu, J., & Chou, L. (2010) · Marine Ecology Progress Series · doi:10.3354/meps08587 · Cited by 60 · cited 1× · see citing works
“[151], [152] found that 24% of the citations in 51 ecology journals and 18% of the citations in 33 marine biology journals did not directly support the assertions made by the citing authors.”
Background
The Benefits and Drawbacks of Impact Factor for Journal Collection Management in Libraries
Nisonger, T. E. (2004) · The Serials Librarian · doi:10.1300/j123v47n01_04 · Cited by 60 · see citing works
Research Quality Rankings of Heterodox Economic Journals in a Contested Discipline
Lee, F. S., Cronin, B., McConnell, S. L. B., & Dean, E. (2010) · American Journal of Economics and Sociology · doi:10.1111/j.1536-7150.2010.00751.x · Cited by 59 · see citing works
A recursive field-normalized bibliometric performance indicator: an application to the field of library and information science
Waltman, L., Yan, E., & van Eck, N. J. (2011) · Scientometrics · doi:10.1007/s11192-011-0449-z · Cited by 57 · see citing works
A Deficiency in the Algorithm for Calculating the Impact Factor of Scholarly Journals: The Journal Impact Factor
Jacsó, P. (2001) · Cortex · doi:10.1016/s0010-9452(08)70602-6 · Cited by 57 · see citing works
Projects: A Very Short Introduction
Davies, A. C. (2017) · doi:10.1093/actrade/9780198727668.001.0001 · Cited by 57 · see citing works
Citing practices in ecology: can we believe our own words?
Todd, P. A., Yeo, D. C. J., Li, D., & Ladle, R. J. (2007) · Oikos · doi:10.1111/j.0030-1299.2007.15992.x · Cited by 55 · see citing works
The journal impact factor: angel, devil, or scapegoat? A comment on J.K. Vanclay’s article 2011
Zitt, M. (2012) · Scientometrics · doi:10.1007/s11192-012-0697-6 · Cited by 53 · see citing works
THE RELATIVE QUALITY OF ECONOMICS JOURNALS: AN ALTERNATIVE RATING SYSTEM
Skeels, J. W., & Taylor, R. A. (1972) · Economic Inquiry · doi:10.1111/j.1465-7295.1972.tb01569.x · Cited by 52 · see citing works
Use of the Journal Citation Reports for Serials Management in Research Libraries: An Investigation of the Effect of Self-Citation on Journal Rankings in Library and Information Science and Genetics
Nisonger, T. E. (2000) · College & Research Libraries · doi:10.5860/crl.61.3.263 · Cited by 52 · see citing works
Using interactive visualizations of WWW log data to characterize access patterns and inform site design
Hochheiser, H., & Shneiderman, B. (2001) · Journal of the American Society for Information Science and Technology · doi:10.1002/1532-2890(2000)9999:9999 · Cited by 49 · see citing works
Positive and negative aspects of citation indices and journal impact factors
Balaban, A. T. (2012) · Scientometrics · doi:10.1007/s11192-012-0637-5 · Cited by 47 · see citing works
The Interdisciplinary Influence of Library and Information Science 1996–2004: A Journal-to-Journal Citation Analysis
Odell, J. D., & Gabbard, R. (2008) · College & Research Libraries · doi:10.5860/crl.69.6.546 · Cited by 47 · see citing works
Collection Management Basics
Evans, G. E., & Saponaro, M. Z. (2012) · Cited by 46 · see citing works
An Assessment of the Predictive Validity of Impact Factor Scores: Implications for Academic Employment Decisions in Social Work
Holden, G. W., Rosenberg, G., Barker, K., & Onghena, P. (2006) · Research on Social Work Practice · doi:10.1177/1049731506292570 · Cited by 46 · see citing works
Discussing some basic critique on Journal Impact Factors: revision of earlier comments
van Leeuwen, T. N. (2012) · Scientometrics · doi:10.1007/s11192-012-0677-x · Cited by 43 · see citing works
Are elite journals declining?
Larivière, V., Lozano, G. A., & Gingras, Y. (2013) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23005 · Cited by 43 · see citing works
How robust is journal rating in Humanities and Social Sciences? Evidence from a large-scale, multi-method exercise
Ferrara, A., & Bonaccorsi, A. J. (2016) · Research Evaluation · doi:10.1093/reseval/rvv048 · Cited by 43 · see citing works
Should we use the mean citations per paper to summarise a journal’s impact or to rank journals in the same field?
Calver, M., & Bradley, J. S. (2009) · Scientometrics · doi:10.1007/s11192-008-2229-y · Cited by 41 · see citing works
Revisitinghmeasured on UK LIS and IR academics
Sanderson, M. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20771 · Cited by 41 · see citing works
Ten good reasons to use the Eigenfactor™ metrics
Franceschet, M. (2010) · Information Processing & Management · doi:10.1016/j.ipm.2010.01.001 · Cited by 40 · see citing works
Measuring the Quality of Peer-Reviewed Publications in Social Work: Impact Factors—Liberation or Liability?
(2009) · doi:10.1080/02615470902856705 · Cited by 39 · see citing works
The Journal Impact Factor: Moving Toward an Alternative and Combined Scientometric Approach
Gasparyan, A. Y., Nurmashev, B., Yessirkepov, M., Udovik, E. E., Baryshnikov, A. A., & Kitas, G. D. (2017) · Journal of Korean Medical Science · doi:10.3346/jkms.2017.32.2.173 · Cited by 39 · see citing works
Visualizing and comparing four facets of scholarly communication: producers, artifacts, concepts, and gatekeepers
Ni, C., Sugimoto, C. R., & Cronin, B. (2012) · Scientometrics · doi:10.1007/s11192-012-0849-8 · Cited by 38 · see citing works
A discipline-specific journal selection algorithm
He, C., & Pao, M. L. (1986) · Information Processing & Management · doi:10.1016/0306-4573(86)90075-0 · Cited by 35 · see citing works
The development of a ranking tool for refereed journals in which nursing and midwifery researchers publish their work
Crookes, P. A., Reis, S. L., & Jones, S. C. (2009) · Nurse Education Today · doi:10.1016/j.nedt.2009.09.016 · Cited by 35 · see citing works
THE RELATIVE QUALITY OF ECONOMICS JOURNALS: AN ALTERNATIVE RATING SYSTEM
Billings, B. B., & Vīksniņš, G. J. (1972) · Economic Inquiry · doi:10.1111/j.1465-7295.1972.tb01568.x · Cited by 35 · see citing works
Distributional differences of the impact factor in the sciences versus the social sciences: An analysis of the probabilistic structure of the 2005 journal citation reports
Bensman, S. J. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20810 · Cited by 35 · see citing works
Library and Information Science Journal Prestige as Assessed by Library and Information Science Faculty
Manzari, L. (2012) · The Library Quarterly · doi:10.1086/668574 · Cited by 34 · see citing works
Do subjective journal ratings represent whole journals or typical articles? Unweighted or weighted citation impact?
Walters, W. H. (2017) · Journal of Informetrics · doi:10.1016/j.joi.2017.05.001 · Cited by 33 · see citing works
Self-citation
Bibliometric Indicators: Why Do We Need More than One?
Setti, G. (2013) · IEEE Access · doi:10.1109/access.2013.2261115 · Cited by 31 · see citing works
Books as a knowledge translation mechanism: citation analysis and author survey
Serenko, A., Bontis, N., & Moshonsky, M. (2012) · Journal of Knowledge Management · doi:10.1108/13673271211238797 · Cited by 31 · see citing works
Looking across communicative genres: a call for inclusive indicators of interdisciplinarity
Sugimoto, C. R. (2010) · Scientometrics · doi:10.1007/s11192-010-0275-8 · Cited by 30 · see citing works
Don't Get Skewed Over by Journal Rankings
Wall, H. J. (2009) · The B E Journal of Economic Analysis & Policy · doi:10.2202/1935-1682.2280 · Cited by 29 · see citing works
Emerging alternatives to the impact factor
Banks, M. A., & Dellavalle, R. P. (2008) · OCLC Systems & Services · doi:10.1108/10650750810898200 · Cited by 28 · see citing works
A classification of the use of research indicators
Kosten, J. (2016) · Scientometrics · doi:10.1007/s11192-016-1904-7 · Cited by 28 · see citing works
Ranking Economics Journals, Economics Departments, and Economists Using Teaching‐Focused Research Productivity
Lo, M., Wong, M. C. S., & Mixon, F. G. (2008) · Southern Economic Journal · doi:10.1002/j.2325-8012.2008.tb00870.x · Cited by 27 · see citing works
A comparison of 17 article-level bibliometric indicators of institutional research productivity: Evidence from the information management literature of China
Li, J., Wu, D., Li, J., & Li, J. (2017) · Information Processing & Management · doi:10.1016/j.ipm.2017.05.002 · Cited by 26 · see citing works
Are Article Influence Scores Comparable across Scientific Fields?
Arendt, J. (2010) · Issues in Science and Technology Librarianship · doi:10.29173/istl2511 · Cited by 24 · see citing works
Fundamentals of Collection Development and Management
(2015) · Library Management · doi:10.1108/lm-06-2015-0036 · Cited by 24 · see citing works
Beyond use statistics: Recall, precision, and relevance in the assessment and management of academic libraries
Walters, W. H. (2015) · Journal of Librarianship and Information Science · doi:10.1177/0961000615572174 · Cited by 24 · see citing works
Self-citation
Pragmatic issues in calculating and comparing the quantity and quality of research through rating and ranking of researchers based on peer reviews and bibliometric indicators from Web of Science, Scopus and Google Scholar
Jacsó, P. (2010) · Online Information Review · doi:10.1108/14684521011099432 · Cited by 23 · see citing works
Journal Rankings from Citation Studies: A Comparison of National and Local Data from Social Work
Stephen E. Wiberley (1982) · The Library Quarterly · doi:10.1086/601279 · Cited by 22 · see citing works
Eigenfactor and article influence scores in the Journal Citation Reports
Jacsó, P. (2010) · Online Information Review · doi:10.1108/14684521011037034 · Cited by 22 · see citing works
Investing Wisely: Citation Rankings as a Measure of Quality in Library and Information Science Journals
Via, B. J., & Schmidle, D. J. (2007) · portal Libraries and the Academy · doi:10.1353/pla.2007.0038 · Cited by 22 · see citing works
Information Sources and Indicators for the Assessment of Journal Reputation and Impact
Walters, W. H. (2016) · The Reference Librarian · doi:10.1080/02763877.2015.1088426 · Cited by 22 · see citing works
Self-citation
Ranking Economics Departments Using Teaching-Focused Research ProductivityPreprint
Lo, M., Mixon, F. G., & Wong, M. C. S. (2005) · SSRN Electronic Journal · doi:10.2139/ssrn.786244 · Cited by 21 · see citing works
Altmetrics as a means of assessing scholarly output
Khodiyar, V., Rowlett, K., & Lawrence, R. N. (2014) · Learned Publishing · doi:10.1087/20140505 · Cited by 20 · see citing works
Rank normalization of impact factors will resolve Vanclay’s dilemma with TRIF
Pudovkin, A. I., & Garfield, E. (2012) · Scientometrics · doi:10.1007/s11192-012-0634-8 · Cited by 18 · see citing works
What Is Library Use? Facets of Concept and a Typology of Its Application in the Literature of Library and Information Science
Fleming‐May, R. A. (2011) · The Library Quarterly · doi:10.1086/660133 · Cited by 18 · see citing works
Scopus' SNIP indicator: Reply to Moed
Loet Leydesdorff, A., & Tobias Opthof, A. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21405 · Cited by 17 · see citing works
Is an Article in a Top Journal a Top Article?
Horan, S. M. (2005) · CFA Digest · doi:10.2469/dig.v35.n3.1738 · Cited by 17 · see citing works
The influence of effects and phenomena on citations: a comparative analysis of four citation perspectives
Wu, Q., & Wolfram, D. (2011) · Scientometrics · doi:10.1007/s11192-011-0456-0 · Cited by 16 · see citing works
Quality‐structure index: A new metric to measure scientific journal influence
Zhang, C., Liu, X., Xu, Y. C., et al. (2011) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21487 · Cited by 15 · see citing works
Assessing international journal impact: the case of marketing
Rosenstreich, D., & Wooliscroft, B. (2011) · European Business Review · doi:10.1108/09555341211191553 · Cited by 14 · see citing works
The Use and Misuse of Citations as a Measure of Influence in Criminology
Levi, M. (1995) · The British Journal of Criminology · doi:10.1093/oxfordjournals.bjc.a048469 · Cited by 14 · see citing works
Key questions in the development and use of survey-based journal rankings
Walters, W. H. (2017) · The Journal of Academic Librarianship · doi:10.1016/j.acalib.2017.05.002 · Cited by 14 · see citing works
Self-citation
Why usage is useless
Singleton, A. (2010) · Learned Publishing · doi:10.1087/20100301 · Cited by 13 · see citing works
Impact factors, citation distributions and journal stratification
Blanford, C. F. (2016) · Journal of Materials Science · doi:10.1007/s10853-016-0285-x · Cited by 13 · see citing works
Do Article Influence scores overestimate the citation impact of social science journals in subfields that are related to higher-impact natural science disciplines?
Walters, W. H. (2014) · Journal of Informetrics · doi:10.1016/j.joi.2014.02.001 · Cited by 12 · see citing works
Self-citation
Altmetrics
(2016) · Facet eBooks · doi:10.29085/9781783301515 · Cited by 12 · see citing works
Update on Journal Impact Ratings in Communication: 2006–2008
Feeley, T. H., & Moon, S.-I. (2010) · Communication Research Reports · doi:10.1080/08824096.2010.518920 · Cited by 11 · see citing works
Evaluating Online Resources for College and University Libraries: Assessing Value and Cost Based on Academic Needs
Walters, W. H. (2016) · Serials Review · doi:10.1080/00987913.2015.1131519 · Cited by 11 · see citing works
Self-citation
Journal impact and proximity: An assessment using bibliographic features
Ni, C., Shaw, D., Lind, S. M., et al. (2013) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22778 · Cited by 10 · see citing works
The problems with the subject categories schema in the EigenFactor database from the perspective of ranking journals by their prestige and impact
Jacsó, P. (2012) · Online Information Review · doi:10.1108/14684521211276064 · Cited by 10 · see citing works
Effect of Bibliographical Classification on the Impact Factor of Science- and Engineering-Based Journals
Foo, J. Y. A. (2009) · Accountability in Research · doi:10.1080/08989620802689516 · Cited by 7 · see citing works
How Media Scholars’ Attributes Affect Their Ratings of Journalism and Mass Communication Quarterly
Ha, L. S., Yang, C., Fang, L., et al. (2015) · Journalism & Mass Communication Quarterly · doi:10.1177/1077699015573269 · Cited by 6 · see citing works
Altmetrics: A Practical Guide for Librarians, Researchers and Academics
Tattersall, A. (2016) · Cited by 6 · see citing works
Textbook Citations as a Measure of Journal Influence on International Business Education
Urbancic, F. R. (2006) · Journal of Teaching in International Business · doi:10.1300/j066v17n03_03 · Cited by 4 · see citing works
Problems with the Use of Citation Indexes in Communication Research.
Funkhouser, E. (1989) · Journal of International Crisis and Risk Communication Research · Cited by 3 · see citing works
Abstract and Index Databases (Web of Knowledge, Scopus, Google Scholar)
Öchsner, A. Ö. (2013) · SpringerBriefs in applied sciences and technology · doi:10.1007/978-3-642-38646-6_5 · Cited by 3 · see citing works
Subprime Lessons for Academic Librarians
Kohl, D. F. (2008) · The Journal of Academic Librarianship · doi:10.1016/j.acalib.2008.06.001 · Cited by 1 · see citing works
Evaluating Journals
Barusch, A. S. (2011) · Journal of Gerontological Social Work · doi:10.1080/01634372.2011.603987 · Cited by 1 · see citing works
Untitled work
(1994) · doi:10.1002/(sici)1097-4571(199401)45:1

Flagged first, then by citations

Sorted by citations

Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World
Pranckutė, R. (2021) · Publications · doi:10.3390/publications9010012 · Cited by 2,696 · see citing works

Authors: Walters, W. H.
Affiliations: Manhattan University (US)
Corresponding author: Walters, William H.
ORCID: Walters, William H.
Venue: IEEE Access
Publisher: Institute of Electrical and Electronics Engineers (IEEE)
Published: 2017
Volume: 5
Pages: 22036-22053
ISSN: 2169-3536 (ISSN-L)
DOI: 10.1109/access.2017.2761400
OpenAlex: W2761801637
Semantic Scholar: c4aec27d131f1c363667c22604ee81591dc7233d
Cited by: 56
References: 178
Citations by year: 2026: 5 · 2025: 4 · 2024: 7 · 2023: 4 · 2022: 5 · 2021: 15 · 2020: 3 · 2019: 9 · 2018: 3 · 2017: 1
FWCI: 3.92
Type: journal article
Language: en
Open access: gold
doi.org
Licence: ieeexplore.ieee.org/Xplorehelp/downloads/license-information/OAPA.html (vor)
Locations: IEEE Access (publishedVersion)
DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders: Manhattan College
Menlo College
Harris Manchester College
Oxford University
Topics: Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
Keywords: citation metrics, journal impact factor, h-index, g-index, Source Normalized Impact per Paper, Eigenfactor, Article Influence Score, SCImago Journal Rank, self-citation, Web of Science, Scopus, Google Scholar, Journal Citation Reports, Google Scholar Metrics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Citation-Based Journal Rankings: Key Questions, Metrics, and Data Sources

Abstract

Integrity signals

Claims the paper makes

Claims other papers make about this one

Cites

Cited by

Related

Record

About these checks