The DOI matches its Crossref record; 78 of the 83 references it cites were found in a registry or an index.
To what extent is inclusion in the Web of Science an indicator of journal ‘quality’?
Abstract
The assessment of research based on the journal in which it is published is a widely adopted practice. Some research assessments use the Web of Science (WoS) to identify ‘high quality’ journals, which are assumed to publish excellent research. The authority of WoS on journal quality stems from its selection of journals based on editorial standards and scientific impact criteria. These can be considered as universalistic criteria, meaning that they can be applied to any journal regardless of its place of publication, language, or discipline. In this article we examine the coverage by WoS of journals produced in Latin America, Spain, and Portugal. We use a logistic regression to examine the probability of a journal to be covered by WoS given universalistic criteria (editorial standards and scientific impact of the journal) and particularistic criteria (country, language, and discipline of the journal). We find that it is not possible to predict the inclusion of journals in WoS only through the universalistic criteria because particularistic variables such as country of the journal, its discipline, and language are strongly related to inclusion in WoS. We conclude that using WoS as a universalistic tool for research assessment can disadvantage science published in journals with adequate editorial standards and scientific merit. We discuss the implications of these findings within the research evaluation literature, specifically for countries and disciplines not extensively covered by WoS.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“from the perspective of universalistic criteria, it was observed that complying with the editorial standards does not guarantee the inclusion in WoS, while the quality of journal impact may have a greater influence [81].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [81]
In the citing paper
“the majority of journals published in other than English languages and in smaller countries, and journals representing very specific research fields, especially ones that were published by universities, were shown to have a lesser probability to be included in WoS CC.”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [81]
In the citing paper
“this 'publish or perish' attitude, on the one hand, stimulates the productivity of researchers, but, on the other, creates a bias towards quantity, often overlooking quality [2,3,6-8,44,81,163,260,263,284-288].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [81]
In the citing paper
“some countries are explicitly not using WoS for assessments of national research, due to its lack of comprehensive coverage width [81].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [81]
Citations
Cites
Herne, H., Cooley, W. W., & Lohnes, P. R. (1973) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2344428 · Cited by 35,801
(1990) · Choice Reviews Online · doi:10.5860/choice.27-3338 · Cited by 35,305
Discovering Statistics Using SPSS
Field, A. P., & Miles, J. N. V. (2000) · Medical Entomology and Zoology · Cited by 28,269
Hosmer, D. W., Lemeshow, S., & Sturdivant, R. X. (2013) · Wiley Series in Probability and Statistics · doi:10.1002/9781118548387 · Cited by 13,861
An index to quantify an individual’s scientific research output
Hirsch, J. E. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0507655102 · Cited by 11,669
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
O'Fallon, W. M., Cooley, W. W., & Lohnes, P. R. (1973) · Technometrics · doi:10.2307/1266874 · Cited by 6,289
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“Secondly, p-values do not show the extent of uncertainty in the findings, which must be acknowledged and reported especially in exploratory studies such as ours (Cohen, 1995, p. 1002; Ioannidis 2005).”
The New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200
Bibliometrics: The Leiden Manifesto for Research Metrics
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015) · Nature · doi:10.1038/520429a · Cited by 2,828
Self-citationDiscovering statistics using R
(2012) · Choice Reviews Online · doi:10.5860/choice.50-2114 · Cited by 2,548
Why the impact factor of journals should not be used for evaluating research
Seglen, P. O. (1997) · BMJ · doi:10.1136/bmj.314.7079.497 · Cited by 2,447 · cited 3×
“Most of the research evaluation literature warns against this practice (Hicks et al. 2015) given that journals publish different types of articles with highly diverse outcomes in terms of citation impact (Seglen 1997).”
Making Meaningful Inferences About Magnitudes
Batterham, A. M., & Hopkins, W. G. (2006) · International Journal of Sports Physiology and Performance · doi:10.1123/ijspp.1.1.50 · Cited by 2,186
(2011) · Cited by 2,025
(1973) · Cited by 1,638
The oligopoly of academic publishers in the digital era
Larivière, V., Haustein, S., & Mongeon, P. (2015) · PLOS ONE · doi:10.1371/journal.pone.0127502 · Cited by 1,240
Social Stratification in Science
Cole, J. R., Cole, S., & Beaver, D. D. (1974) · American Journal of Physics · doi:10.1119/1.1987897 · Cited by 1,198
Performance-based university research funding systems
Hicks, D. M. (2011) · Research Policy · doi:10.1016/j.respol.2011.09.007 · Cited by 1,076
Zuckerman, H., & Merton, R. (1971) · Minerva · doi:10.1007/bf01553188 · Cited by 789
A Hirsch-type index for journals
Braun, T., Glänzel, W., & Schubert, A. P. (2006) · Scientometrics · doi:10.1007/s11192-006-0147-4 · Cited by 669
Martin, B. R., & Irvine, J. M. (1983) · Research Policy · doi:10.1016/0048-7333(83)90005-7 · Cited by 529
Archambault, É., Vignola-Gagné, É., Côté, G., Larivière, V., & Gingrasb, Y. (2006) · Scientometrics · doi:10.1007/s11192-006-0115-z · Cited by 507
Hopkins, W. G. (2007) · Cited by 497
Academic Writing in a Global Context: The Politics and Practices of Publishing in English
Lillis, T., & Curry, M. J. (2010) · Open Research Online (The Open University) · Cited by 495
Lost Science in the Third World
Gibbs, W. W. (1995) · Scientific American · doi:10.1038/scientificamerican0895-92 · Cited by 472
van Leeuwen, T. N., Moed, H. F., Tijssen, R., Visser, M. S., & van Raan, A. F. J. (2001) · Scientometrics · doi:10.1023/a:1010549719484 · Cited by 433
Bornmann, L., & Daniel, H. (2008) · EMBO Reports · doi:10.1038/embor.2008.233 · Cited by 396
Hicks, D. M. (1999) · Scientometrics · doi:10.1007/bf02457380 · Cited by 391
Web of Science and Scopus: a journal title overlap study
Gavel, Y., & Iselid, L. (2008) · Online Information Review · doi:10.1108/14684520810865958 · Cited by 384
Coverage analysis of Scopus: A journal metric approach
Anegón, F. D. M., Chinchilla‐Rodríguez, Z., Vargas‐Quesada, B., Corera‐Álvarez, E., Muñoz-Fernández, F. J., González-Molina, A., & Herrero-Solana, V. (2007) · Scientometrics · doi:10.1007/s11192-007-1681-4 · Cited by 381
Is there science beyond English?
Meneghini, R., & Packer, A. L. (2007) · EMBO Reports · doi:10.1038/sj.embor.7400906 · Cited by 361
The inconsistency of the h‐index
Waltman, L., & van Eck, N. J. (2011) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21678 · Cited by 338
The Changing Governance of the Sciences
Whitley, R. D., & Gläser, J. (2007) · Sociology of the sciences/Sociology of the sciences yearbook · doi:10.1007/978-1-4020-6746-4 · Cited by 328
Vessuri, H., Guédon, J., & Cetto, A. M. (2013) · Current Sociology · doi:10.1177/0011392113512839 · Cited by 279
Harzing, A.-W. K., & van der Wal, R. (2008) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20953 · Cited by 274
Franceschet, M. (2009) · Scientometrics · doi:10.1007/s11192-009-0021-2 · Cited by 236
Guédon, J. (2009) · E-LIS Repository (University of Naples Federico II) · Cited by 204
The web of knowledge: a festschrift in honor of Eugene Garfield
Garfield, E., Cronin, B., & Atkins, H. (2000) · Medical Entomology and Zoology · Cited by 179
Academic Writing in a Global Context
Lillis, T., & Curry, M. J. (2013) · Routledge eBooks · doi:10.4324/9780203852583 · Cited by 165
Campbell, P. R. (2008) · Ethics in Science and Environmental Politics · doi:10.3354/esep00078 · Cited by 150
Why researchers publish in non-mainstream journals: Training, knowledge bridging, and gap filling
Chavarro, D., Tang, P., & Ràfols, I. (2017) · Research Policy · doi:10.1016/j.respol.2017.08.002 · Cited by 143
Self-citationRegression for citation data: An evaluation of different methods
Thelwall, M. A., & Wilson, P. (2014) · Journal of Informetrics · doi:10.1016/j.joi.2014.09.011 · Cited by 142
Regression for citation data: An evaluation of different methods
Thelwall, M. A., & Wilson, P. (2014) · RePEc: Research Papers in Economics · Cited by 127
Sivertsen, G., & Larsen, B. (2012) · Scientometrics · doi:10.1007/s11192-011-0615-3 · Cited by 114
Schneider, J. W. (2014) · Scientometrics · doi:10.1007/s11192-014-1251-5 · Cited by 112
Hodge, D. R., & Lacasse, J. R. (2011) · Journal of Social Work Education · doi:10.5175/jswe.2011.201000024 · Cited by 105
Collazo‐Reyes, F. (2013) · Scientometrics · doi:10.1007/s11192-013-1036-2 · Cited by 102
The earth is round (p < .05): Rejoinder.
Cohen, J. (1995) · American Psychologist · doi:10.1037/0003-066x.50.12.1103 · Cited by 100
Publication-Based Funding: The Norwegian Model
Sivertsen, G. (2016) · Springer eBooks · doi:10.1007/978-3-319-29016-4_7 · Cited by 90
Social Stratification in Science.
Crane, D., Cole, J. R., & Cole, S. (1974) · Administrative Science Quarterly · doi:10.2307/2393902 · Cited by 74
Academic Evaluation: Universal Instrument? Tool for Development?
Bianco, M., Gras, N., & Sutz, J. (2016) · Minerva · doi:10.1007/s11024-016-9306-9 · Cited by 70
Research Assessment in the Humanities
(2016) · doi:10.1007/978-3-319-29016-4 · Cited by 68
Unseen science? Representation of BRICs in global science
Wagner, C. S., & Wong, S. (2011) · Scientometrics · doi:10.1007/s11192-011-0481-z · Cited by 67
Cuartas, G. V., Lucio‐Arias, D., & Loet Leydesdorff, A. (2016) · El Profesional de la Informacion · doi:10.3145/epi.2016.ene.05 · Cited by 66
Web of Knowledge: A Festschrift in Honor of Eugene Garfield
Duffy, J. C. (2001) · Issues in Science and Technology Librarianship · doi:10.29173/istl1839 · Cited by 63
How relevant are local scholarly journals in global science? A case study of South Africa
Tijssen, R., Mouton, J., van Leeuwen, T. N., & Boshoff, N. (2006) · Research Evaluation · doi:10.3152/147154406781775904 · Cited by 59
Publication and Citation Practices of Brazilian Agricultural Scientists
Velho, L., & Krige, J. (1984) · Social Studies of Science · doi:10.1177/030631284014001004 · Cited by 57
Martin, B. R., & Irvine, J. M. (1993) · Research Policy · doi:10.1016/0048-7333(93)90049-n · Cited by 53
Sancho, R. (1992) · Scientometrics · doi:10.1007/bf02020924 · Cited by 50
Effects of modified-implement training on fast bowling in cricket
Petersen, C. J., Wilson, B. D., & Hopkins, W. G. (2004) · Journal of Sports Sciences · doi:10.1080/02640410410001729973 · Cited by 46
National Scientific Strategies in Tropical Soil Sciences
Arvanitis, R., & Chatelin, Y. (1988) · Social Studies of Science · doi:10.1177/030631288018001005 · Cited by 40
Larivière, V., & Macaluso, B. (2011) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.21632 · Cited by 39
Ranking de revistas científicas en Latinoamérica mediante el índice h: estudio de caso Colombia
Romero‐Torres, M., Acosta, A., & Tejada-Gómez, M. A. (2013) · Revista española de Documentación Científica · doi:10.3989/redc.2013.1.876 · Cited by 39
The function of national journals in disseminating applied science
Sanz, E., Aragon, I., & Méndez, A. (1995) · Journal of Information Science · doi:10.1177/016555159502100408 · Cited by 35
Garfield, E. (1995) · PubMed · Cited by 28
Citation databases omit local journals
Alperín, J. P. (2014) · Nature · doi:10.1038/511155c · Cited by 26
Cetto, A. M., Gamboa, J. O. A., & Córdoba, S. (2010) · Scholarly and Research Communication · doi:10.22230/src.2010v1n1a17 · Cited by 22
Latin American scholarly journal databases: a look back to the way forward
Gamboa, J. O. A., & Russell, J. M. (2012) · Aslib Journal of Information Management · doi:10.1108/00012531211196693 · Cited by 22
Publication subsidies: challenges and dilemmas facing South African researchers.
Woodiwiss, A. J. (2012) · Europe PMC (PubMed Central) · Cited by 21
IBEROAMÉRICA EN LA CIENCIA DE CORRIENTE PRINCIPAL (THOMSON REUTERS / SCOPUS): UNA REGIÓN FRAGMENTADA
Aguado‐López, E., Becerril-García, A., Arriola, M. L., & Martínez-Domínguez, N. D. (2014) · Americanae (AECID Library) · Cited by 18
Romero‐Torres, M., Tejada-Gómez, M. A., & Acosta, A. (2011) · arXiv (Cornell University) · doi:10.48550/arxiv.1101.2711 · Cited by 12
Bohórquez, C., & Andrés, D. (2017) · Cited by 12
Šipka, P., & Kosanović, B. (2012) · doi:10.5937/bioac-111 · Cited by 10
Mario, H. (2014) · Cited by 10
Cuartas, G. V., Lucio‐Arias, D., & Leydesdorff, L. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1510.02453 · Cited by 7
Qualis Periódicos: indicador da política científica no Brasil?
Frigeri, M., & Monteiro, M. S. A. (2014) · Estudos de Sociologia · Cited by 5
[Scientific publications in Latin America].
Fadda, G. (1989) · PubMed · Cited by 4
Characteristics of soil science literature.
McDonald, P. (1994) · Cited by 3
Research works printed but not linked: 12
- Bayaga, 2010, Multinomial Logistic Regression: Usage and Application in Risk Analysis, Journal of Applied Quantitative Methods, 5, 288
- Bonaccorsi, 2015, La Valutazione Possibile: Teoria e Pratica Della Valutazione Della Ricerca. (Studi e ricerche Economia)
- Cetto, 2011, Latindex: Towards a Portal for Ibero-American Journals, 454
- Garfield, 1976, Journal Citation Studies: Latin American Journals, Current Contents, 2, 577
- Garfield, 1980, How ISI Selects Journals for Coverage: Quantitative and Qualitative Considerations, Current Contents, 22, 1
- Garfield, 1985, Editorial Merit vs. Political Pressure, Current Contents, 11, 3
- Garfield, 1997, A Statistically Valid Definition of Bias is Needed to Determine Whether the Science Citation Index Discriminates against Third World Journals, Current Science, 73, 639
- OECD, 2006, Revised Field of Science and Technology (FOS) Classification in the Fracati Manual
- Sivertsen, 2010, A Performance Indicator Based on Complete Data for the Scientific Publication Output at Research Institutions, ISSI Newsletter, 6, 22
- Testa, 2011, Globalization of the Web of Science: 2005-2010
- Velho, 1985, Science in the Periphery: A Study of the Agricultural Scientific Community in Brazilian Universities
- Yaalon, 1962, Has Soil Research National Characteristics?, Soils and Fertilizers, 27, 89
Cited by
Sorted by citations
Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World
Pranckutė, R. (2021) · Publications · doi:10.3390/publications9010012 · Cited by 2,696
Martín-Martín, A., Orduna-Malea, E., Thelwall, M., & Delgado López-Cózar, E. (2018) · Journal of Informetrics · doi:10.1016/j.joi.2018.09.002 · Cited by 1,990
Reviewing Paradox Theory in Corporate Sustainability Toward a Systems Perspective
Carmine, S., & Marchi, V. D. (2022) · Journal of Business Ethics · doi:10.1007/s10551-022-05112-2 · Cited by 256
Martín-Martín, A., Orduña-Malea, E., & López-Cózar, E. D. (2018) · Scientometrics · doi:10.1007/s11192-018-2820-9 · Cited by 228
Multi-Criteria Decision-Making Methods to Address Water Allocation Problems: A Systematic Review
Gebre, S. L., Cattrysse, D., & Orshoven, J. V. (2021) · Water · doi:10.3390/w13020125 · Cited by 114
Metadata
- Authors
- Chavarro, D., Ràfols, I., & Tang, P.
- Affiliations
- University of Sussex (GB)
- Corresponding author
- Chavarro, Diego
- Venue
- Research Evaluation
- Publisher
- Oxford University Press (OUP)
- Published
- 2018-01-29
- Volume
- 27
- Issue
- 2
- Pages
- 106-118
- ISSN
- 0958-2029 (ISSN-L)
- 1471-5449
- OpenAlex
- W2707936849
- Cited by
- 115
- References
- 83
- Citations by year
- 2026: 10 · 2025: 16 · 2024: 18 · 2023: 16 · 2022: 14 · 2021: 17 · 2020: 10 · 2019: 10 · 2018: 4
- FWCI
- 7.82
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Research Evaluation (publishedVersion)
- Sussex Research Online (University of Sussex) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DIGITAL.CSIC (Spanish National Research Council (CSIC)) (submittedVersion)
- Topics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Keywords
- Web of Science, journal inclusion, journal quality, research evaluation, bibliometric databases, editorial standards, scientific impact, Latin American journals, Spanish periodicals, bibliographic coverage
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
