The DOI matches its Crossref record; 74 of the 74 references it cites were found in a registry or an index.
Bibliometrics: tracking research impact by selecting the appropriate metrics
Open accessAbstract
Traditionally, the success of a researcher is assessed by the number of publications he or she publishes in peer-reviewed, indexed, high impact journals. This essential yardstick, often referred to as the impact of a specific researcher, is assessed through the use of various metrics. While researchers may be acquainted with such matrices, many do not know how to use them to enhance their careers. In addition to these metrics, a number of other factors should be taken into consideration to objectively evaluate a scientist's profile as a researcher and academician. Moreover, each metric has its own limitations that need to be considered when selecting an appropriate metric for evaluation. This paper provides a broad overview of the wide array of metrics currently in use in academia and research. Popular metrics are discussed and defined, including traditional metrics and article-level metrics, some of which are applied to researchers for a greater understanding of a particular concept, including varicocele that is the thematic area of this Special Issue of Asian Journal of Andrology. We recommend the combined use of quantitative and qualitative evaluation using judiciously selected metrics for a more objective assessment of scholarly output and research impact.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“It is used to determine the relationships between topics in the disciplines represented in this literature set, which in turn, can assist in identifying areas (clusters) for current and future research that constitute the theoretical constructs or underlying themes of the research area (Eck and Waltman, 2010; Agarwal et al., 2016).”
From Characteristics and trends in acceptance and commitment therapy research: A bibliometric analysis (2022) · cites this paper as Agarwal et al., 2016
Experimental — accuracy not yet measuredIn the citing paper
“In this way, JIF also shows a certain level of discrimination against less popular publication types that, in certain disciplines, are as important as original research articles [2,26,45,46,224,225,229].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [2]
In the citing paper
“Yet, only a minority of stakeholders employing bibliometric indicators are adequately familiar with their meaning and appropriate use [2,4,5,7].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [2]
In the citing paper
“However, today, JIF is often misused due to a misinterpretation of its meaning, as it has become a standard tool in virtually all evaluation practices at all levels, usually without proper considerations of the rationale for its use [2,44,222,224-227].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [2]
In the citing paper
“The same applies in evaluations of larger entities, such as research or academic institutions and countries, since they can only be compared to those that are similar in size [2,26].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [2]
In the citing paper
“not all of the cited publications could be read at all by the authors [2].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [2]
In the citing paper
“This phenomenon can be well illustrated by the 80-20 rule describing that, usually, the top 20% of papers tend to receive 80% of total citations, while remaining publications are cited very scarcely, if cited at all [2,272].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [2]
In the citing paper
“The simple amount of publications does not reflect the scientific impact, since publication counts measure productivity, rather than quality [1,2].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [2]
In the citing paper
“It is often argued that the scientific impact of any evaluated entity cannot be determined solely by citations due to the different citing behaviors and resulting incompatible citation counts between disciplines [1-3,5,260,267].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [2]
In the citing paper
“this 'publish or perish' attitude, on the one hand, stimulates the productivity of researchers, but, on the other, creates a bias towards quantity, often overlooking quality [2,3,6-8,44,81,163,260,263,284-288].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [2]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
An index to quantify an individual’s scientific research output
Hirsch, J. E. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0507655102 · Cited by 11,669
Bibliometrics: The Leiden Manifesto for Research Metrics
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015) · Nature · doi:10.1038/520429a · Cited by 2,828
Garfield, E. (1972) · Science · doi:10.1126/science.178.4060.471 · Cited by 2,788
The History and Meaning of the Journal Impact Factor
Garfield, E. (2006) · JAMA · doi:10.1001/jama.295.1.90 · Cited by 2,770
Why the impact factor of journals should not be used for evaluating research
Seglen, P. O. (1997) · BMJ · doi:10.1136/bmj.314.7079.497 · Cited by 2,447
Theory and practise of the g-index
Egghe, L. (2006) · Scientometrics · doi:10.1007/s11192-006-0144-7 · Cited by 2,354
Sharing Detailed Research Data Is Associated with Increased Citation Rate
Piwowar, H. A., Day, R. S., & Fridsma, D. B. (2007) · PLoS ONE · doi:10.1371/journal.pone.0000308 · Cited by 881
A new approach to the metric of journals’ scientific prestige: The SJR indicator
González-Pereira, B., Guerrero‐Bote, V. P., & Anegón, F. D. M. (2010) · Journal of Informetrics · doi:10.1016/j.joi.2010.03.002 · Cited by 867
Citation Advantage of Open Access Articles
Eysenbach, G. (2006) · PLoS Biology · doi:10.1371/journal.pbio.0040157 · Cited by 829
Free online availability substantially increases a paper's impact
Lawrence, S. (2001) · Nature · doi:10.1038/35079151 · Cited by 797
Do Open-Access Articles Have a Greater Research Impact?
Antelman, K. (2004) · College & Research Libraries · doi:10.5860/crl.65.5.372 · Cited by 678
Measuring Contextual Citation Impact of Scientific Journals
Moed, H. F. (2010) · Journal of Informetrics · doi:10.1016/j.joi.2010.01.002 · Cited by 628
Author Sequence and Credit for Contributions in Multiauthored Publications
Tscharntke, T., Hochberg, M., Rand, T. A., Resh, V. H., & Krauß, J. (2007) · PLoS Biology · doi:10.1371/journal.pbio.0050018 · Cited by 580
The R- and AR-indices: Complementing the h-index
Jin, B., Liang, L., Rousseau, R., & Egghe, L. (2007) · Chinese Science Bulletin · doi:10.1007/s11434-007-0145-9 · Cited by 563
Bibliometrics and Citation Analysis: From the Science Citation Index to Cybermetrics
Bellis, N. D. (2009) · IRIS UNIMORE (University of Modena and Reggio Emilia) · Cited by 548
New developments in the use of citation analysis in research evaluation
Moed, H. F. (2009) · Archivum Immunologiae et Therapiae Experimentalis · doi:10.1007/s00005-009-0001-5 · Cited by 507
Relative Citation Impact of Various Study Designs in the Health Sciences
Patsopoulos, N. A. (2005) · JAMA · doi:10.1001/jama.293.19.2362 · Cited by 498
Fortnightly Review: How can impact factors be improved?
Garfield, E. (1996) · BMJ · doi:10.1136/bmj.313.7054.411 · Cited by 491
Honorary and ghost authorship in high impact biomedical journals: a cross sectional survey
Wislar, J. S., Flanagin, A., Fontanarosa, P. B., & DeAngelis, C. D. (2011) · BMJ · doi:10.1136/bmj.d6128 · Cited by 478
The e-Index, Complementing the h-Index for Excess Citations
Zhang, C. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005429 · Cited by 455
Schubert, A. P., & Braun, T. (1986) · Scientometrics · doi:10.1007/bf02017249 · Cited by 450
Generalized Hirsch h-index for disclosing latent facts in citation networks
Sidiropoulos, A., Katsaros, D., & Manolopoulos, Y. (2007) · Scientometrics · doi:10.1007/s11192-007-1722-z · Cited by 435
"Science Citation Index"—A New Dimension in Indexing
Garfield, E. (1964) · Science · doi:10.1126/science.144.3619.649 · Cited by 427
Impact factor: outdated artefact or stepping-stone to journal certification?
Vanclay, J. K. (2011) · Scientometrics · doi:10.1007/s11192-011-0561-0 · Cited by 388
Gender-Heterogeneous Working Groups Produce Higher Quality Science
Campbell, L., Mehtani, S., Dozier, M. E., & Rinehart, J. (2013) · PLoS ONE · doi:10.1371/journal.pone.0079147 · Cited by 386
Coverage analysis of Scopus: A journal metric approach
Anegón, F. D. M., Chinchilla‐Rodríguez, Z., Vargas‐Quesada, B., Corera‐Álvarez, E., Muñoz-Fernández, F. J., González-Molina, A., & Herrero-Solana, V. (2007) · Scientometrics · doi:10.1007/s11192-007-1681-4 · Cited by 381
Using publication metrics to highlight academic productivity and research impact
Carpenter, C. R., Cone, D. C., & Sarli, C. C. (2014) · Academic Emergency Medicine · doi:10.1111/acem.12482 · Cited by 377
Do open access articles have greater citation impact?A critical review of the literature
CRAIG, I., PLUME, A., MCVEIGH, M., PRINGLE, J., & AMIN, M. (2007) · Journal of Informetrics · doi:10.1016/j.joi.2007.04.001 · Cited by 373
On the origins of bibliometrics
Godin, B. (2006) · Scientometrics · doi:10.1007/s11192-006-0086-0 · Cited by 290
Citation Analysis: A Comparison of Google Scholar, Scopus, and Web of Science
Yang, K., & Meho, L. I. (2006) · Proceedings of the American Society for Information Science and Technology · doi:10.1002/meet.14504301185 · Cited by 285
Rossner, M., Epps, H. V., & Hill, E. (2007) · The Journal of Experimental Medicine · doi:10.1084/jem.20072544 · Cited by 278
Bibliometrics and beyond: some thoughts on web-based citation analysis
Cronin, B. (2001) · Journal of Information Science · doi:10.1177/016555150102700101 · Cited by 277
The Use of Bibliometrics for Assessing Research: Possibilities, Limitations and Adverse Effects
Haustein, S., & Larivière, V. (2015) · Incentives and Performance · doi:10.1007/978-3-319-09785-5_8 · Cited by 276
Article-Level Metrics and the Evolution of Scientific Impact
Neylon, C., & Wu, S. (2009) · PLoS Biology · doi:10.1371/journal.pbio.1000242 · Cited by 244
Use of the h index in neurosurgery
Lee, J. S., Kraus, K. L., & Couldwell, W. T. (2009) · Journal of neurosurgery · doi:10.3171/2008.10.jns08978 · Cited by 231
Dong, P., Loh, M. C. S., & Mondry, A. (2005) · Biomedical Digital Libraries · doi:10.1186/1742-5581-2-7 · Cited by 229
The use and misuse of journal metrics and other citation indicators
Pendlebury, D. (2009) · Archivum Immunologiae et Therapiae Experimentalis · doi:10.1007/s00005-009-0008-y · Cited by 227
Altmetric: enriching scholarly content with article‐level discussion and metrics
Adie, E. A., & William, R. O. E. (2012) · Learned Publishing · doi:10.1087/20130103 · Cited by 224
Defrosting the Digital Library: Bibliographic Tools for the Next Generation Web
Hull, D., Pettifer, S. R., & Kell, D. B. (2008) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1000204 · Cited by 209
Clickstream Data Yields High-Resolution Maps of Science
Bollen, J., Herbert Van De Sompel, A., Hagberg, A. A., Bettencourt, L. M. A., Chute, R., Rodriguez, M. A., & Balakireva, L. (2009) · PLoS ONE · doi:10.1371/journal.pone.0004803 · Cited by 207
Modifying the journal impact factor by fractional citation weighting: The audience factor
Zitt, M., & Small, H. G. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20880 · Cited by 184
The Eigenfactor MetricsTM: A Network Approach to Assessing Scholarly Journals
West, J. D., Bergstrom, T. C., & Bergstrom, C. T. (2010) · College & Research Libraries · doi:10.5860/0710236 · Cited by 169
Counting on citations: a flawed way to measure quality
Walter, G., Fisher, K. R., Bloch, S., & Hunt, G. E. (2003) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.2003.tb05196.x · Cited by 166
The Meaning of Author Order in Medical Research
Baerlocher, M. O., Newton, M., Gautam, T., TOMLINSON, G. A., & Detsky, A. S. (2007) · Journal of Investigative Medicine · doi:10.2310/6650.2007.06044 · Cited by 165
Cross-field normalization of scientometric indicators
Schubert, A. P., & Braun, T. (1996) · Scientometrics · doi:10.1007/bf02129597 · Cited by 150
Digital libraries and the continuum of scholarly communication
Borgman, C. L. (2000) · Journal of Documentation · doi:10.1108/eum0000000007121 · Cited by 143
On the Predictability of Future Impact in Science
Penner, O., Pan, R. K., Petersen, A. M., Kaski, K., & Fortunato, S. (2013) · Scientific Reports · doi:10.1038/srep03052 · Cited by 140
Bornmann, L., & Marx, W. (2015) · Journal of Informetrics · doi:10.1016/j.joi.2015.01.006 · Cited by 138
Altmetrics – a complement to conventional metrics
Melero, R. (2015) · Biochemia Medica · doi:10.11613/bm.2015.016 · Cited by 130
On the calculation of percentile-based bibliometric indicators
Waltman, L. R., & Schreiber, M. (2012) · Leiden Repository (Leiden University) · Cited by 126
On the Predictability of Future Impact in Science
Penner, O., Pan, R. K., Petersen, A. M., Kaski, K. K., & Fortunato, S. (2013) · IMT Institutional Repository (IMT School for Advanced Studies Lucca) · Cited by 102
Hunt, R. (1991) · Nature · doi:10.1038/352187a0 · Cited by 97
The publishing imperative: the pervasive influence of publication metrics
Steele, C., Butler, L., & Kingsley, D. A. (2006) · Learned Publishing · doi:10.1087/095315106778690751 · Cited by 91
The index is the best measure of a scientist's research productivity
Baldock, C., Ma, R., & Orton, C. G. (2009) · Medical Physics · doi:10.1118/1.3089421 · Cited by 73
Manipulating impact factor: an unethical issue or an Editors choice?
Sevinç, A. (2004) · Swiss Medical Weekly · doi:10.4414/smw.2004.10761 · Cited by 67
Citing-side normalization of journal impact: A robust variant of the Audience Factor
Zitt, M. (2010) · Journal of Informetrics · doi:10.1016/j.joi.2010.03.004 · Cited by 65
Holliday, E. B., Jagsi, R., Thomas, C. R., Wilson, L. D., & Fuller, C. D. (2013) · International Journal of Radiation Oncology*Biology*Physics · doi:10.1016/j.ijrobp.2013.09.035 · Cited by 62
Scarlat, M. M., Mavrogenis, A. F., Pecina, M. M., & Niculescu, M. (2015) · International Orthopaedics · doi:10.1007/s00264-015-2766-y · Cited by 56
Honorary and ghost authorship in high impact biomedical journals: a cross sectional survey
(2011) · BMJ · doi:10.1136/bmj.d7677 · Cited by 55
Journal bibliometrics indicators and citation ethics: A discussion of current issues
Huggett, S. (2013) · Atherosclerosis · doi:10.1016/j.atherosclerosis.2013.07.051 · Cited by 54
Impact beyond the impact factor
Zupanc, G. K. H. (2013) · Journal of Comparative Physiology A · doi:10.1007/s00359-013-0863-1 · Cited by 49
Post‐publication filtering and evaluation: Faculty of 1000
Wets, K., Weedon, D., & Velterop, J. (2003) · Learned Publishing · doi:10.1087/095315103322421982 · Cited by 47
Patel, V., Ashrafian, H., Almoudaris, A. M., Makanjuola, J. K., Bucciarelli‐Ducci, C., Darzi, A. W., & Athanasiou, T. (2012) · Medical Principles and Practice · doi:10.1159/000341756 · Cited by 44
The Open Research Web: A Preview of the Optimal and the Inevitable
Shadbolt, N., Brody, T., Carr, L. A., & Harnad, S. R. (2006) · ePrints Soton (University of Southampton) · Cited by 39
Is it Ethical for Journals to Request Self-citation?
Mahian, O., & Wongwises, S. (2014) · Science and Engineering Ethics · doi:10.1007/s11948-014-9540-1 · Cited by 27
The Use and Abuse of Citations as Indicators of Research Quality
Coryn, C. L. S. (2006) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v3i4.82 · Cited by 26
Citation searching and bibliometric measures: Resources for ranking and tracking
Kear, R. L., & Colbert‐Lewis, D. (2011) · College & Research Libraries News · doi:10.5860/crln.72.8.8620 · Cited by 26
Relative Citation Impact of Various Study Designs in the Health Sciences
(2005) · Obstetrics and Gynecology · doi:10.1097/01.aog.0000173954.21555.b6 · Cited by 26
A Worksheet for Authorship of Scientific Articles
Schmidt, R. H. M. (1987) · Bulletin of the Ecological Society of America · doi:10.2307/20166549 · Cited by 24
Bibliometric indexes, databases and impact factors in cardiology
Bienert, I. R. D. C., de Oliveira, R. C., de Andrade, P. B., & Caramori, C. A. (2015) · Brazilian Journal of Cardiovascular Surgery · doi:10.5935/1678-9741.20150019 · Cited by 20
Industry influenced evidence production in collaborative research communities: A network analysis
Dunn, A. G., Gallego, B., & Coiera, E. W. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.10.010 · Cited by 12
Silva, M. R. E. (2011) · Clinics · doi:10.1590/s1807-59322011001200016 · Cited by 12
Shadbolt, N., Brody, T., Carr, L. A., & Harnad, S. R. (2006) · Elsevier eBooks · doi:10.1016/b978-1-84334-203-8.50020-0 · Cited by 11
Sharing Detailed Research Data Is Associated with Increased Citation RatePreprint
Piwowar, H. (2007) · Nature Precedings · doi:10.1038/npre.2007.361.1 · Cited by 8
Research works printed but not linked: 3
- McClellan, 2003, Specialist Control: The Publications Committee of the Académie Royale Des Sciences (Paris) 1700-1793.Transactions of the American Philosophical Society, i
- Bornmann, 2013, The normalization of citation counts based on classification systems.Publications, 78
- Garfield, 1972, Citation analysis as a tool in journal evaluation, Science, 471
Cited by
Sorted by citations
Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World
Pranckutė, R. (2021) · Publications · doi:10.3390/publications9010012 · Cited by 2,696
Roldán-Valadéz, E., Salazar-Ruiz, S.-Y., Contreras, R. I., & Rı́os, C. (2018) · Irish Journal of Medical Science (1971 -) · doi:10.1007/s11845-018-1936-5 · Cited by 461
Blockchain applications in management: A bibliometric analysis and literature review
Tandon, A., Kaur, P., Mäntymäki, M., & Dhir, A. (2021) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2021.120649 · Cited by 337
Current perspectives and trend of nanomedicine in cancer: A review and bibliometric analysis
Pei, Z., Chen, S., Ding, L., Liu, J., Cui, X., Li, F., & Qiu, F. (2022) · Journal of Controlled Release · doi:10.1016/j.jconrel.2022.10.023 · Cited by 220
Research trends on human trafficking: a bibliometric analysis using Scopus database
Sweileh, W. M. (2018) · Globalization and Health · doi:10.1186/s12992-018-0427-9 · Cited by 206
Metadata
- Authors
- Agarwal, A., Durairajanayagam, D., Tatagari, S., Esteves, S., Harlev, A., Henkel, R., Roychoudhury, S., Homa, S., Puchalt, N., Ramasamy, R., Majzoub, A., Ly, K., Tvrda, E., Assidi, M., Kesari, K., Sharma, R., Banihani, S., Ko, E., Abu-Elmagd, M., Gosalvez, J., & Bashiri, A.
- Affiliations
- Cleveland Clinic (US)
- Ben-Gurion University of the Negev (IL)
- University of the Western Cape (ZA)
- Corresponding author
- Agarwal, Ashok
- ORCID
- Venue
- Asian Journal of Andrology
- Published
- 2016
- Volume
- 18
- Issue
- 2
- Pages
- 296
- ISSN
- 1008-682X (ISSN-L)
- PMC
- PMC4770502
- OpenAlex
- W2263556835
- Cited by
- 639
- References
- 74
- Citations by year
- 2026: 59 · 2025: 98 · 2024: 121 · 2023: 91 · 2022: 79 · 2021: 69 · 2020: 41 · 2019: 36 · 2018: 27 · 2017: 12 · 2016: 4
- FWCI
- 2.47
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc-sa
- Locations
- Asian Journal of Andrology (publishedVersion, cc-by-nc-sa)
- PubMed Central (publishedVersion, cc-by-nc-sa)
- PubMed (publishedVersion)
- Tesis Doctorals en Xarxa (Consorci de Serveis Universitaris de Catalunya) (publishedVersion)
- LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) (acceptedVersion)
- Hispana (acceptedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- OakTrust (Texas A&M University Libraries) (submittedVersion)
- Biblos-e Archivo (Universidad Autónoma de Madrid) (submittedVersion, public-domain)
- Topics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- bibliometric analysis, research impact, research metrics, article-level metrics, varicocele, research career development
- MeSH terms
- Authorship, Humans, Periodicals as Topic, Bibliometrics, Biomedical Research, Journal Impact Factor, Open Access Publishing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
