Cited by 115scientometrics and bibliometrics researchPsychology Research and BibliometricsPublishing and Scholarly Communication

To what extent is inclusion in the Web of Science an indicator of journal ‘quality’?

Chavarro, Diego, Ràfols, Ismael, Tang, Puay·2018·Research Evaluation

Abstract

The assessment of research based on the journal in which it is published is a widely adopted practice. Some research assessments use the Web of Science (WoS) to identify ‘high quality’ journals, which are assumed to publish excellent research. The authority of WoS on journal quality stems from its selection of journals based on editorial standards and scientific impact criteria. These can be considered as universalistic criteria, meaning that they can be applied to any journal regardless of its place of publication, language, or discipline. In this article we examine the coverage by WoS of journals produced in Latin America, Spain, and Portugal. We use a logistic regression to examine the probability of a journal to be covered by WoS given universalistic criteria (editorial standards and scientific impact of the journal) and particularistic criteria (country, language, and discipline of the journal). We find that it is not possible to predict the inclusion of journals in WoS only through the universalistic criteria because particularistic variables such as country of the journal, its discipline, and language are strongly related to inclusion in WoS. We conclude that using WoS as a universalistic tool for research assessment can disadvantage science published in journals with adequate editorial standards and scientific merit. We discuss the implications of these findings within the research evaluation literature, specifically for countries and disciplines not extensively covered by WoS.

Integrity signals

Registry

Crossref · 10.1093/reseval/rvy001

Cites

78 of 78 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Complying with editorial standards does not guarantee inclusion in WoS, while the quality of journal impact may have a greater influence.

    “from the perspective of universalistic criteria, it was observed that complying with the editorial standards does not guarantee the inclusion in WoS, while the quality of journal impact may have a greater influence [81].”
  • The majority of journals published in languages other than English, in smaller countries, and journals representing very specific research fields, especially university-published journals, had a lesser probability of being included in WoS CC.

    “the majority of journals published in other than English languages and in smaller countries, and journals representing very specific research fields, especially ones that were published by universities, were shown to have a lesser probability to be included in WoS CC.”
  • The 'publish or perish' attitude stimulates productivity of researchers but creates a bias towards quantity, often overlooking quality.

    “this 'publish or perish' attitude, on the one hand, stimulates the productivity of researchers, but, on the other, creates a bias towards quantity, often overlooking quality [2,3,6-8,44,81,163,260,263,284-288].”
  • Some countries explicitly do not use WoS for assessments of national research due to its lack of comprehensive coverage width.

    “some countries are explicitly not using WoS for assessments of national research, due to its lack of comprehensive coverage width [81].”

Cites

Flagged first, then by citations

5 of the 83 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Authors
Chavarro, D., Ràfols, I., & Tang, P.
Affiliations
University of Sussex (GB)
Corresponding author
Chavarro, Diego
Venue
Research Evaluation
Published
2018-01-29
Volume
27
Issue
2
Pages
106-118
ISSN
  • 0958-2029 (ISSN-L)
  • 1471-5449
OpenAlex
W2707936849
Cited by
115
References
83
Citations by year
2026: 10 · 2025: 16 · 2024: 18 · 2023: 16 · 2022: 14 · 2021: 17 · 2020: 10 · 2019: 10 · 2018: 4
FWCI
7.82
Type
journal article
Language
en
Open access
closed
Locations
  • Research Evaluation (publishedVersion)
  • Sussex Research Online (University of Sussex) (submittedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DIGITAL.CSIC (Spanish National Research Council (CSIC)) (submittedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
  • Social Sciences › Arts and Humanities › History and Philosophy of Science › Publishing and Scholarly Communication
Keywords
Web of Science, journal inclusion, journal quality, research evaluation, bibliometric databases, editorial standards, scientific impact, Latin American journals, Spanish periodicals, bibliographic coverage

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review