The DOI matches its Crossref record; 78 of the 83 references it cites were found in a registry or an index.
To what extent is inclusion in the Web of Science an indicator of journal ‘quality’?
Abstract
The assessment of research based on the journal in which it is published is a widely adopted practice. Some research assessments use the Web of Science (WoS) to identify ‘high quality’ journals, which are assumed to publish excellent research. The authority of WoS on journal quality stems from its selection of journals based on editorial standards and scientific impact criteria. These can be considered as universalistic criteria, meaning that they can be applied to any journal regardless of its place of publication, language, or discipline. In this article we examine the coverage by WoS of journals produced in Latin America, Spain, and Portugal. We use a logistic regression to examine the probability of a journal to be covered by WoS given universalistic criteria (editorial standards and scientific impact of the journal) and particularistic criteria (country, language, and discipline of the journal). We find that it is not possible to predict the inclusion of journals in WoS only through the universalistic criteria because particularistic variables such as country of the journal, its discipline, and language are strongly related to inclusion in WoS. We conclude that using WoS as a universalistic tool for research assessment can disadvantage science published in journals with adequate editorial standards and scientific merit. We discuss the implications of these findings within the research evaluation literature, specifically for countries and disciplines not extensively covered by WoS.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“from the perspective of universalistic criteria, it was observed that complying with the editorial standards does not guarantee the inclusion in WoS, while the quality of journal impact may have a greater influence [81].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [81]
In the citing paper
“the majority of journals published in other than English languages and in smaller countries, and journals representing very specific research fields, especially ones that were published by universities, were shown to have a lesser probability to be included in WoS CC.”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [81]
In the citing paper
“this 'publish or perish' attitude, on the one hand, stimulates the productivity of researchers, but, on the other, creates a bias towards quantity, often overlooking quality [2,3,6-8,44,81,163,260,263,284-288].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [81]
In the citing paper
“some countries are explicitly not using WoS for assessments of national research, due to its lack of comprehensive coverage width [81].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [81]
Citations
Cites
Herne, H., Cooley, W. W., & Lohnes, P. R. (1973) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2344428 · Cited by 35,801
(1990) · Choice Reviews Online · doi:10.5860/choice.27-3338 · Cited by 35,305
Discovering Statistics Using SPSS
Field, A. P., & Miles, J. N. V. (2000) · Medical Entomology and Zoology · Cited by 28,269
Hosmer, D. W., Lemeshow, S., & Sturdivant, R. X. (2013) · Wiley Series in Probability and Statistics · doi:10.1002/9781118548387 · Cited by 13,861
An index to quantify an individual’s scientific research output
Hirsch, J. E. (2005) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0507655102 · Cited by 11,669
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
O'Fallon, W. M., Cooley, W. W., & Lohnes, P. R. (1973) · Technometrics · doi:10.2307/1266874 · Cited by 6,289
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 1×
“Secondly, p-values do not show the extent of uncertainty in the findings, which must be acknowledged and reported especially in exploratory studies such as ours (Cohen, 1995, p. 1002; Ioannidis 2005).”
The New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200
Bibliometrics: The Leiden Manifesto for Research Metrics
Hicks, D., Wouters, P., Waltman, L., de Rijcke, S., & Rafols, I. (2015) · Nature · doi:10.1038/520429a · Cited by 2,828
Self-citation
Research works printed but not linked: 12
- Bayaga, 2010, Multinomial Logistic Regression: Usage and Application in Risk Analysis, Journal of Applied Quantitative Methods, 5, 288
- Bonaccorsi, 2015, La Valutazione Possibile: Teoria e Pratica Della Valutazione Della Ricerca. (Studi e ricerche Economia)
- Cetto, 2011, Latindex: Towards a Portal for Ibero-American Journals, 454
- Garfield, 1976, Journal Citation Studies: Latin American Journals, Current Contents, 2, 577
- Garfield, 1980, How ISI Selects Journals for Coverage: Quantitative and Qualitative Considerations, Current Contents, 22, 1
- Garfield, 1985, Editorial Merit vs. Political Pressure, Current Contents, 11, 3
- Garfield, 1997, A Statistically Valid Definition of Bias is Needed to Determine Whether the Science Citation Index Discriminates against Third World Journals, Current Science, 73, 639
- OECD, 2006, Revised Field of Science and Technology (FOS) Classification in the Fracati Manual
- Sivertsen, 2010, A Performance Indicator Based on Complete Data for the Scientific Publication Output at Research Institutions, ISSI Newsletter, 6, 22
- Testa, 2011, Globalization of the Web of Science: 2005-2010
- Velho, 1985, Science in the Periphery: A Study of the Agricultural Scientific Community in Brazilian Universities
- Yaalon, 1962, Has Soil Research National Characteristics?, Soils and Fertilizers, 27, 89
Cited by
Sorted by citations
Items 101–115 of 115
Challenges of Open Respiratory Archives: Indexing and Impact Factor
Alonso‐Arroyo, A., de Granda‐Orive, J. I., & Aleixandre‐Benavent, R. (2023) · Open Respiratory Archives · doi:10.1016/j.opresp.2023.100294
Measuring Internationality of Regional Scientific JournalsPreprint
Çolak, O. (2024) · doi:10.31237/osf.io/49pty
University scientific coproduction becomes more social in economic crisis
Azagra‐Caro, J. M., & Yegros, A. (2024) · Research Evaluation · doi:10.1093/reseval/rvaf041
Simulation-based maintenance systems: a systematic review
Liu, P. S., Chin, J. F., Ab-Samat, H., & Muhammad, N. A. (2025) · Journal of Simulation · doi:10.1080/17477778.2025.2579125 · Cited by 1
Giros socio-territoriales en políticas de ciencia y tecnología en Chile
Salas, R. C., Caro, C. O., & Bustos-Velásquez, F. (2025) · Revista Mexicana de Sociología · doi:10.22201/iis.01882503p.2025.e.63178
Artificial Intelligence and Student Engagement in Online Learning : A Literature Review
Zhou, Z. (2025) · American Journal of Distance Education · doi:10.1080/08923647.2025.2594282 · Cited by 5
Cai, H., Wu, B., Shen, Y., Huang, L., & Li, L. (2025) · Combustion Science and Technology · doi:10.1080/00102202.2025.2606192
Metaverse Ethics from The Perspective of Metaverse and Ethics: A Bibliometric Analysis
Erdoğan, Z., & Gürbüz, E. (2025) · İzmir Sosyal Bilimler Dergisi · doi:10.47899/ijss.1818389
Determinants of firms’ acceptability of carbon taxation: a systematic literature review
Mwang’onda, E. S., Steenkamp, L., & Terblanche‐Smit, M. (2026) · Climate Policy · doi:10.1080/14693062.2026.2627746
Liu, Y., & Wan, Y. (2026) · Research Square · doi:10.21203/rs.3.rs-10177526/v1
A systematic literature review of fintech-driven value co-creation in B2B markets
Mohamad, S. N. A., & Ishak, A. H. (2026) · Journal of Business and Industrial Marketing · doi:10.1108/jbim-03-2025-0222
Who Needs National Journals? Dissemination Strategies in Canadian Scholarly Communication
Pradier, C., Céspedes, L., van Bellen, S., Alperín, J. P., & Larivière, V. (2026) · Canadian Journal of Information and Library Science · doi:10.5206/cjils-rcsib.v49i2.24726
Language hierarchies and gender inequalities: the coverage of OpenAlex and the Web of Science
Pradier, C., Kozlowski, D., Mazoni, A., & Larivière, V. (2026) · Research Evaluation · doi:10.1093/reseval/rvag062
Metadata
- Authors
- Chavarro, D., Ràfols, I., & Tang, P.
- Affiliations
- University of Sussex (GB)
- Corresponding author
- Chavarro, Diego
- Venue
- Research Evaluation
- Publisher
- Oxford University Press (OUP)
- Published
- 2018-01-29
- Volume
- 27
- Issue
- 2
- Pages
- 106-118
- ISSN
- 0958-2029 (ISSN-L)
- 1471-5449
- OpenAlex
- W2707936849
- Cited by
- 115
- References
- 83
- Citations by year
- 2026: 10 · 2025: 16 · 2024: 18 · 2023: 16 · 2022: 14 · 2021: 17 · 2020: 10 · 2019: 10 · 2018: 4
- FWCI
- 7.82
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Research Evaluation (publishedVersion)
- Sussex Research Online (University of Sussex) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DIGITAL.CSIC (Spanish National Research Council (CSIC)) (submittedVersion)
- Topics
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Library and Information Sciences › Publishing and Scholarly Communication
- Keywords
- Web of Science, journal inclusion, journal quality, research evaluation, bibliometric databases, editorial standards, scientific impact, Latin American journals, Spanish periodicals, bibliographic coverage
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
