The DOI matches its Crossref record; 64 of the 71 references it cites were found in a registry or an index.
The operationalization of “fields” as WoS subject categories ( WC s) in evaluative bibliometrics: The cases of “library and information science” and “science & technology studies”
Open accessAbstract
Normalization of citation scores using reference sets based on W eb of S cience subject categories ( WC s) has become an established (“best”) practice in evaluative bibliometrics. For example, the T imes H igher E ducation W orld U niversity R ankings are, among other things, based on this operationalization. However, WC s were developed decades ago for the purpose of information retrieval and evolved incrementally with the database; the classification is machine‐based and partially manually corrected. Using the WC “information science & library science” and the WC s attributed to journals in the field of “science and technology studies,” we show that WC s do not provide sufficient analytical clarity to carry bibliometric normalization in evaluation practices because of “indexer effects.” Can the compliance with “best practices” be replaced with an ambition to develop “best possible practices”? New research questions can then be envisaged.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“different bibliographic DBs sort their indexed content based on their own individual subject classification schemes”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [176]
In the citing paper
“prior attempts to describe a normalized citation metric have resulted in imperfect systems for the comparison of diverse scholarly works (S1 Text and Fig 4E), either because they measure only the average performance of a group of papers [62] or because the article of interest is measured against a control group that includes widely varying areas of science [17,32,48].”
From Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level (2016) · cites this paper as [48]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Fast unfolding of communities in large networks
Blondel, V. D., Guillaume, J.-L., Lambiotte, R., & Lefebvre, E. (2008) · Journal of Statistical Mechanics: Theory and Experiment · doi:10.1088/1742-5468/2008/10/p10008 · Cited by 21,424
Science and Engineering Indicators
Washington, E., & Warren, M. E. (2004) · Cited by 2,822
Garfield, E. (1972) · Science · doi:10.1126/science.178.4060.471 · Cited by 2,788 · cited 2×
“re needed for the comparison. The impact factor (IF) cannot be used for this evaluation because, like other bibliometric indicators, this measure varies itself systematically among fields of science. Garfield (1972) introduced the IF deliberately with a citation window of only the last two years of cited articles in o…”
BackgroundMethodAn algorithm for drawing general undirected graphs
Kamada, T., & Kawai, S. (1989) · Information Processing Letters · doi:10.1016/0020-0190(89)90102-6 · Cited by 2,750
Citation Indexes for Science: A New Dimension in Documentation through Association of Ideas
Garfield, E. (1955) · Science · doi:10.1126/science.122.3159.108 · Cited by 2,334
Handbook of Quantitative Science and Technology Research
Moed, H. F., Glänzel, W., & Schmoch, U. (2005) · Springer eBooks · doi:10.1007/1-4020-2755-9 · Cited by 1,385
Science and Engineering Indicators 1998
Wink, D. J. (1998) · Journal of Chemical Education · doi:10.1021/ed075p1078 · Cited by 1,337
Mapping the backbone of science
Boyack, K. W., Klavans, R., & Börner, K. (2005) · Scientometrics · doi:10.1007/s11192-005-0255-6 · Cited by 992
Sources of information on specific subjects 1934
(1985) · Journal of Information Science · doi:10.1177/016555158501000407 · Cited by 864
Universality of citation distributions: Toward an objective measure of scientific impact
Radicchi, F., Fortunato, S., & Castellano, C. (2008) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0806977105 · Cited by 828
"Sources of information on specific subjects" by S.C. Bradford
Brookes, B. C. (1985) · Journal of Information Science · doi:10.1177/016555158501000406 · Cited by 717
A new methodology for constructing a publication‐level classification system of science
Waltman, L., & van Eck, N. J. (2012) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22748 · Cited by 700
Ràfols, I., Loet Leydesdorff, A., O’Hare, A., Nightingale, P., & Stirling, A. (2012) · Research Policy · doi:10.1016/j.respol.2012.03.015 · Cited by 616
Martin, B. R., & Irvine, J. M. (1983) · Research Policy · doi:10.1016/0048-7333(83)90005-7 · Cited by 529
Moed, H. F., Bruin, R. E. D., & van Leeuwen, T. N. (1995) · Scientometrics · doi:10.1007/bf02017338 · Cited by 502
Communication among scientists and engineers
Nelson, C. E., & Pollock, D. (1970) · Cited by 472
The Leiden ranking 2011/2012: Data collection, indicators, and interpretation
Waltman, L., Calero‐Medina, C., Kosten, J., Noyons, E., Tijssen, R., van Eck, N. J., van Leeuwen, T. N., van Raan, A. F. J., Visser, M. S., & Wouters, P. F. (2012) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22708 · Cited by 452
Schubert, A. P., & Braun, T. (1986) · Scientometrics · doi:10.1007/bf02017249 · Cited by 450
Towards a new crown indicator: Some theoretical considerations
Waltman, L., van Eck, N. J., van Leeuwen, T. N., Visser, M. S., & van Raan, A. F. J. (2010) · Journal of Informetrics · doi:10.1016/j.joi.2010.08.001 · Cited by 438
Glänzel, W., & Schubert, A. P. (2003) · Scientometrics · doi:10.1023/a:1022378804087 · Cited by 333
Global maps of science based on the new Web-of-Science categories
Loet Leydesdorff, A., Carley, S. F., & Ràfols, I. (2012) · Scientometrics · doi:10.1007/s11192-012-0784-8 · Cited by 311
Self-citationNicolaisen, J. (2007) · Annual Review of Information Science and Technology · doi:10.1002/aris.2007.1440410120 · Cited by 309
Schubert, A. P., Glänzel, W., & Braun, T. (1989) · Scientometrics · doi:10.1007/bf02093234 · Cited by 274
Dahler‐Larsen, P. (2011) · Stanford University Press eBooks · doi:10.11126/stanford/9780804776929.001.0001 · Cited by 250
Caveats for the use of citation indicators in research and journal evaluations
Loet Leydesdorff, A. (2007) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20743 · Cited by 212
Algorithmic procedure for finding semantically related journals
Pudovkin, A. I., & Garfield, E. (2002) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.10153 · Cited by 211
Costas, R., van Leeuwen, T. N., & Bordons, M. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21348 · Cited by 185
Modifying the journal impact factor by fractional citation weighting: The audience factor
Zitt, M., & Small, H. G. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20880 · Cited by 184
Tobias Opthof, A., & Loet Leydesdorff, A. (2010) · Journal of Informetrics · doi:10.1016/j.joi.2010.02.003 · Cited by 173
Ràfols, I., & Loet Leydesdorff, A. (2009) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21086 · Cited by 150
Garfield and the impact factor
Bensman, S. J. (2007) · Annual Review of Information Science and Technology · doi:10.1002/aris.2007.1440410110 · Cited by 145
Bornmann, L., & Marx, W. (2013) · Scientometrics · doi:10.1007/s11192-013-1161-y · Cited by 143
Self-citationAn approach to interdisciplinarity through bibliometric indicators
Morillo, F., Bordons, M., & Gómez, I. (2001) · Scientometrics · doi:10.1023/a:1010529114941 · Cited by 137
Leydesdorff, L. (2006) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20322 · Cited by 136
Self-citationLoet Leydesdorff, A., & Bornmann, L. (2010) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21450 · Cited by 123
Self-citationBRICS countries and scientific excellence: A bibliometric analysis of most frequently cited papers
Bornmann, L., Wagner, C. S., & Leydesdorff, L. (2015) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23333 · Cited by 122
Self-citationRuiz‐Castillo, J., & Waltman, L. (2014) · Journal of Informetrics · doi:10.1016/j.joi.2014.11.010 · Cited by 114
Science and technology studies: Exploring the knowledge base
Martin, B. R., Nightingale, P., & Yegros, A. (2012) · Research Policy · doi:10.1016/j.respol.2012.03.010 · Cited by 113
Leydesdorff, L., Wagner, C. S., & Bornmann, L. (2014) · Journal of Informetrics · doi:10.1016/j.joi.2014.05.002 · Cited by 104
Self-citationBornmann, L., Marx, W., Schier, H., Rahm, E., Thor, A., & Daniel, H. (2008) · Journal of Informetrics · doi:10.1016/j.joi.2008.11.001 · Cited by 100
Self-citationLoet Leydesdorff, A. (2007) · Scientometrics · doi:10.1007/s11192-007-1694-z · Cited by 76
Leydesdorff, L., Hammarfelt, B., & Salah, A. A. A. (2011) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21636 · Cited by 75
Self-citationMultidimensional Journal Evaluation
Haustein, S. (2012) · Saur eBooks · doi:10.1515/9783110255553 · Cited by 73
Gingras, Y., & Larivière, V. (2010) · Journal of Informetrics · doi:10.1016/j.joi.2010.10.005 · Cited by 63
Information metrics (iMetrics): a research specialty with a socio-cognitive identity?
Milojević, S., & Leydesdorff, L. (2012) · Scientometrics · doi:10.1007/s11192-012-0861-z · Cited by 62
Self-citationScientometrics and communication theory: Towards theoretically informed indicators
Loet Leydesdorff, A., & van den Besselaar, P. (1997) · Scientometrics · doi:10.1007/bf02461129 · Cited by 62
Wouters, P. F. (1998) · Scientometrics · doi:10.1007/bf02457980 · Cited by 62
Waltman, L., Yan, E., & van Eck, N. J. (2011) · Scientometrics · doi:10.1007/s11192-011-0449-z · Cited by 57
Neuhaus, C., & Daniel, H. (2008) · Scientometrics · doi:10.1007/s11192-007-2007-2 · Cited by 51
van Leeuwen, T. N., & Medina, C. C. (2012) · Research Evaluation · doi:10.1093/reseval/rvr006 · Cited by 50
The cognitive and the social structure of STS
van den Besselaar, P. (2001) · Scientometrics · doi:10.1023/a:1012714020453 · Cited by 49
Bensman, S. J., & Loet Leydesdorff, A. (2009) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21020 · Cited by 49
Glänzel, W. (2010) · Journal of Informetrics · doi:10.1016/j.joi.2010.01.005 · Cited by 38
Bradford's Law and Fuzzy Sets: Statistical Implications for Library Analyses
Bensman, S. J. (2001) · IFLA Journal · doi:10.1177/034003520102700406 · Cited by 32
An evaluation of British scientific journals
Martyn, J. R. C., & Gilchrist, A. (1968) · Cited by 31
Citation analysis with medical subject Headings ( MeSH ) using the W eb of K nowledge: A new routine
Loet Leydesdorff, A., & Tobias Opthof, A. (2013) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.22770 · Cited by 29
Braun, T., Glänzel, W., Maczelka, H., & Schubert, A. P. (1994) · Scientometrics · doi:10.1007/bf02033443 · Cited by 26
A novel approach to citation normalization: A similarity‐based method for creating reference sets
Colliander, C. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23193 · Cited by 20
Is citation normalization realistic?
Kostoff, R. N., & Martinez, W. L. (2004) · Journal of Information Science · doi:10.1177/0165551505049260 · Cited by 18
The role of handbooks in knowledge creation and diffusion: A case of science and technology studies
Milojević, S., Sugimoto, C. R., Larivière, V., Thelwall, M. A., & Ding, Y. (2014) · Journal of Informetrics · doi:10.1016/j.joi.2014.06.003 · Cited by 17
Indices of journal citation relatedness and citation relationships among aquatic biology journals
Pudovkin, A. I., & Fuseler, E. A. (1995) · Scientometrics · doi:10.1007/bf02017642 · Cited by 13
Matching MEDLINE/PubMed Data with Web of Science (WoS): A Routine in R LanguagePreprint
Rotolo, D., & Leydesdorff, L. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2451436 · Cited by 10
Self-citationBRICS countries and scientific excellence: A bibliometric analysis of most frequently-cited papersPreprint
Bornmann, L., Wagner, C. S., & Leydesdorff, L. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1404.3721 · Cited by 9
Self-citationWaltman, L., & Ruiz‐Castillo, J. (2014) · RePEc: Research Papers in Economics · Cited by 4
Research works printed but not linked: 6
- Boyack K.W., 2011, The 13th Conference of the International Society for Scientometrics and Informetrics, 123
- Butler L., 2010, Book of Abstracts of the 11th International Conference on Science and Technology Indicators, September 9–11, 62
- Garfield E., 1971, The mystery of the transposed journal lists—Wherein Bradford's Law of Scattering is generalized according to Garfield's Law of Concentration, Current Contents, 3, 5
- Katz J.S. &Hicks D.(1995 June).The classification of interdisciplinary journals: A new approach.Paper presented at the Proceedings of the Fifth International Conference of the International Society for Scientometrics and Informetrics River Forest IL (pp.105–115).
- 10.1007/s11192‐012‐0783‐8 · DOI
- Rehn C., 2014, Bibliometric handbook for Karolinska Institutet
Cited by
Sorted by citations
Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World
Pranckutė, R. (2021) · Publications · doi:10.3390/publications9010012 · Cited by 2,696
A review of the literature on citation impact indicators
Waltman, L. (2016) · Journal of Informetrics · doi:10.1016/j.joi.2016.02.007 · Cited by 1,356
A Review of Theory and Practice in Scientometrics
Mingers, J., & Leydesdorff, L. (2015) · European Journal of Operational Research · doi:10.1016/j.ejor.2015.04.002 · Cited by 1,002
Hutchins, B. I., Yuan, X., Anderson, J. M., & Santangelo, G. M. (2016) · PLoS Biology · doi:10.1371/journal.pbio.1002541 · Cited by 500
Wang, Q., & Waltman, L. (2016) · Journal of Informetrics · doi:10.1016/j.joi.2016.02.003 · Cited by 417
Metadata
- Authors
- Leydesdorff, L., & Bornmann, L.
- Affiliations
- Amsterdam School of Communication Research (ASCoR) University of Amsterdam P.O. Box 15793, NL‐1001 NG Amsterdam The Netherlands
- Max Planck Society (DE)
- ORCID
- Bornmann, Lutz
- Venue
- Journal of the Association for Information Science and Technology
- Publisher
- Wiley
- Published
- 2015-01-08
- Volume
- 67
- Issue
- 3
- Pages
- 707-714
- ISSN
- 2330-1635 (ISSN-L)
- 2330-1643
- OpenAlex
- W2591659376
- Cited by
- 129
- References
- 71
- Citations by year
- 2026: 3 · 2025: 9 · 2024: 9 · 2023: 6 · 2022: 10 · 2021: 13 · 2020: 14 · 2019: 10 · 2018: 16 · 2017: 11 · 2016: 17 · 2015: 7
- FWCI
- 15.51
- Type
- journal article
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- Journal of the Association for Information Science and Technology (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- UvA-DARE (University of Amsterdam) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
- Social Sciences › Library and Information Sciences › Library Science and Information
- Keywords
- library and information science, science and technology studies, journal classification, Times Higher Education World University Rankings, bibliometric analysis, best practices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
