CoChat
Sign Up
Cited by 2,696scientometrics and bibliometrics researchLibrary Science and InformationWeb visibility and informetrics

Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World

Open access
Pranckutė, Raminta·2021·Publications
DOI 10.3390/publications9010012
Open this paper in CoChat — free accountOpen PDF

Abstract

Nowadays, the importance of bibliographic databases (DBs) has increased enormously, as they are the main providers of publication metadata and bibliometric indicators universally used both for research assessment practices and for performing daily tasks. Because the reliability of these tasks firstly depends on the data source, all users of the DBs should be able to choose the most suitable one. Web of Science (WoS) and Scopus are the two main bibliographic DBs. The comprehensive evaluation of the DBs’ coverage is practically impossible without extensive bibliometric analyses or literature reviews, but most DBs users do not have bibliometric competence and/or are not willing to invest additional time for such evaluations. Apart from that, the convenience of the DB’s interface, performance, provided impact indicators and additional tools may also influence the users’ choice. The main goal of this work is to provide all of the potential users with an all-inclusive description of the two main bibliographic DBs by gathering the findings that are presented in the most recent literature and information provided by the owners of the DBs at one place. This overview should aid all stakeholders employing publication and citation data in selecting the most suitable DB.

AI snapshot

Written from the paper’s full text.

TL;DR

This review compares Web of Science Core Collection (WoS CC) and Scopus as bibliographic databases, synthesizing recent literature and vendor information on coverage, quality, metrics, search, and data access. It concludes Scopus generally offers wider content and citation coverage and more convenient profiles/metrics, while WoS has deeper citation backfiles and better funding-information coverage, so suitability depends on task and subscription.

Key findings

  • Scopus generally has wider overall coverage: >25,100 active source titles versus WoS CC >21,400 journals only; it indexes more books, conference material, and non-English/regional literature. In a six-source comparison, Scopus retrieved 57% of combined citations to sample documents versus WoS 52%, and Scopus covered 93% of WoS citations while WoS covered 83% of Scopus citations.
  • WoS has deeper citation coverage (cited references since 1900 versus Scopus since 1970), though accessible depth can be shortened by subscription terms. WoS performs better on funding information: in medical journals, WoS funded articles were 29.0% versus Scopus 7.7%; Scopus funding-information recall was about 67%, while WoS recall/precision was often above 90% but with incomplete fields.
  • Both databases share biases: overrepresentation of Natural Sciences, Engineering, and Biomedical Research; underrepresentation of Social Sciences and Humanities; insufficient book and conference coverage (e.g., both covered only 14% of book chapters in a Norwegian output study); and continued English-language bias.
  • Error profiles differ: Scopus generally had fewer author-name/title errors and omitted-reference errors (3.53% versus 4.51% in WoS), but more duplicate entries and missing cited documents; WoS had more phantom citations (0.46% versus 0.10% in Scopus). Scopus author disambiguation precision rose from 98% in 2017 to 99.9% in 2019 with recall >94%; institution disambiguation precision was 0.95 for WoS OE and 0.96 for Scopus AFID, with recall 0.93 for WoS and 0.86 for Scopus; only 20% of investigated institutions were in WoS OE versus 85% in Scopus AFID.

Methods

  • Design: narrative literature review/overview, explicitly not a systematic review, comparing WoS CC and Scopus.
  • Data: findings from studies published in the last five years plus official information from Clarivate and Elsevier, including tables summarizing coverage numbers and impact indicators.
  • Analysis: thematic comparison grouped by content coverage, source/document types, regional/non-English literature, citations, patents, funding information, publication/source/author/institution information, content quality, search/online analysis, data export, impact indicators, and bibliometric concerns.

Limitations

  • The paper states it is not a systematic review; it was impossible to cover all relevant literature, only studies from the last five years were discussed, databases and interfaces constantly change, WoS subscription variations hinder reproducibility/comparison, and Open Access content coverage was not elaborated.
  • Evident from the text: some conclusions rely on the author’s personal opinion (e.g., that Scopus is more convenient or better suited) and on vendor-provided information, which the paper itself notes may be subjective or exaggerated; no primary empirical data collection was performed.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.3390/publications9010012

Cites

245 of 245 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 550 cited sources checked · 0 supported · 0 flagged

  • Findingnone given
    § Content Coverage Evaluation by Source Lists · p. 29

    The accessibility of content might be further restricted by time-frame limitations (e.g., in Lithuanian Universities subscribing WoS CC, available data reach back only to 1990).

    “the accessibility of content might be further restricted by time-frame limitations (for instance, in the case of Lithuanian Universities subscribing WoS CC, the available data reach back only to 1990)”
    Quote in the paperNot a substantial claim
  • Findingown data
    § Content Coverage Evaluation by Source Lists · p. 28

    These chemical indexes are not always accessible with a basic WoS CC subscription.

    “apparently they are not always accessible with a basic WoS CC subscription. For instance, our university subscribes WoS CC, but it has no access to IC and CCR.”
    Quote in the paperNot a substantial claim
  • Premisecitation
    § Main Bibliographic Databases · p. 3

    Google Scholar can be easily manipulated using falsified data.

    “By simple experiment, it was also shown that GS can be easily manipulated using falsified data [34].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [34] 2014, The Google Scholar Experiment: How to Index False Papers and Manipulate Bibliometric Indicators, J. Assoc. Inf. Sci. Technol. source

      Not checked in the free run.

  • Premisecitation
    § Raminta Pranckut˙ e iD · p. 1

    Web of Science (WoS) and Scopus are the two main bibliographic databases.

    “Web of Science (WoS) and Scopus are the two main bibliographic DBs.”
    • Also stated in: § 1. Introduction · p. 1 · boosted
      “Web of Science (WoS) and Scopus are the two bibliographic DBs generally accepted as the most comprehensive data sources for various purposes [10].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [10] Zhu, 2020, A Tale of Two Databases: The Use of Web of Science and Scopus in Academic Papers, Scientometrics source

      Not checked in the free run.

  • Premisecitation
    § 1. Introduction · p. 1

    Institutions are often forced to choose between WoS and Scopus.

    “Consequently, institutions are often forced to choose between these data sources [15].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [15] Dallas, 2018, Variable Bibliographic Database Access Could Limit Reproducibility, Bioscience source

      Not checked in the free run.

  • May combine two claims. 174
  • Classified as probably not a substantial claim. 115
  • The quote only partly states this claim. 1

  • May combine two claims.

    Premisereasoning
    § 2. Literature Overview · p. 4

    Even recent studies published during the last five years were mostly focused on one or few particular aspects, mainly related to content coverage, while other features such as data quality or additional DBs functionalities and their practical use were usually only mentioned briefly.

    “Accordingly, even the recent studies, which were published during the last five years, were mostly focused on one or few particular aspects, mainly related to content coverage. Meanwhile many other features of the DBs, such as data quality or additional DBs functionalities and their practical use, which may also influence the suitability of the DB for a particular task, were usually only mentioned”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § 1. Introduction · p. 2

    Most DBs users do not have the necessary competence to make the most suitable choice and need a trusted source of DB's descriptive information.

    “Yet, most of the DBs users do not have the necessary competence to make the most suitable choice and, therefore, need a trusted source of DB's descriptive information [4,25].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [4] Leydesdorff, 2016, Professional and Citizen Bibliometrics: Complementarities and Ambivalences in the Development and Use of Indicators—a State-of-the-Art Report, Scientometrics source

      Not checked in the free run.

    • cites [25] Ellegaard, 2018, The Application of Bibliometric Analysis: Disciplinary and User Aspects, Scientometrics source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation
    § 2. Literature Overview · p. 6

    The quality of data is very important, especially in performing bibliometric analyses.

    “However, while it is not arguable that comprehensiveness of coverage is essential for any task, the quality of data is also very important, especially in performing bibliometric analyses [4,40].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [4] Leydesdorff, 2016, Professional and Citizen Bibliometrics: Complementarities and Ambivalences in the Development and Use of Indicators—a State-of-the-Art Report, Scientometrics source

      Not checked in the free run.

    • cites [40] Visser, M., Jan Van Eck, N., and Waltman, L. (2020). Large-Scale Comparison of Bibliographic Data Sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. arXiv. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § Main Bibliographic Databases · p. 4

    WoS and Scopus are still considered to be the most reliable sources of bibliographic data, both for most analyses and research evaluations, and for daily tasks.

    “Therefore, the two main specialized DBs-WoS and Scopus-are still considered to be the most reliable sources of bibliographic data, both for the most analyses and research evaluations, and for daily tasks.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § 1. Introduction · p. 2

    Information provided in early studies performed more than several years ago may have become obsolete and could be misleading.

    “Moreover, because DBs are continuously expanding both their contents and capabilities, the information provided in the early studies, performed more than several years ago, may have become obsolete and, thus, could be misleading [26].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [26] Waltman, 2016, A Review of the Literature on Citation Impact Indicators, J. Informetr. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

    Premisenone given
    § 1. Introduction · p. 2

    The literature is overwhelmed with investigations and comparisons of the main bibliographic DBs, but most studies are relatively narrow in context or focused on a particular topic.

    “Although the literature is overwhelmed with various investigations and comparisons of the main bibliographic DBs, most of these studies, even literature reviews, are relatively narrow in context or only focused on a particular topic.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1. Introduction · p. 2

    The convenience and performance of the DB's web interface and provided additional functionalities may greatly influence the preference of the DB.

    “In these cases, the convenience and performance of the DB's web interface and provided additional functionalities may also greatly influence the preference of the DB [14,23,24].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [14] Salehi, 2013, A Comparison between Two Main Academic Literature Collections: Web of Science and Scopus Databases, Asian Soc. Sci.

      Not checked in the free run.

    • cites [23] Gusenbauer, 2019, Google Scholar to Overshadow Them All? Comparing the Sizes of 12 Academic Search Engines and Bibliographic Databases, Scientometrics source

      Not checked in the free run.

    • cites [24] Okhovati, 2017, Novice and Experienced Users’ Search Performance and Satisfaction with Web of Science and Scopus, J. Librariansh. Inf. Sci. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisereasoning
    § 2. Literature Overview · p. 5

    Because WoS and Scopus both include cited references for each publication they cover, better coverage should result in higher citation counts.

    “Because WoS and Scopus both include cited references for each publication they cover, another approach frequently used to evaluate the extent of content coverage is to compare the counts of citations retrieved from the DBs based on the fact that DBs only collect citations from indexed documents. Thus, better coverage should result in higher citation counts.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation
    § 2. Literature Overview · p. 4

    The most accurate evaluation of content coverage and quality of the DBs can only be achieved by performing a thorough large-scale analysis, which requires a huge effort, adequate competencies, and is extremely time-consuming.

    “In all cases, the most accurate evaluation of content coverage and quality of the DBs can only be achieved by performing a thorough large scale analysis, which requires a huge effort, adequate competencies, and is extremely time-consuming [27].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [27] Badia, 2018, Identifying “Best Bets” for Searching in Chemical Engineering: Comparing Database Content and Performance for Information Retrieval, J. Doc. source

      Not checked in the free run.

  • May combine two claims.

    Premisereasoning
    § 1. Introduction · p. 2

    All relevant information regarding the DBs is usually not available in one place, which leads to the possibility that many important features could be left unaccounted for when evaluating DB's suitability.

    “Hence, all relevant information regarding the DBs is usually not available in one place, which leads to the possibility that many important features could be left unaccounted for when evaluating DB's suitability.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § Main Bibliographic Databases · p. 4

    Many other data sources are free and relatively new, and their validity is still questionable.

    “Many of them are also free and relatively new products, such as Microsoft Academic, CrossRef, ResearchGate, OpenCitations, etc. [32,39-41]. Hence, their validity is also still questionable.”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [32] Thelwall, 2021, Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: A Multidisciplinary Comparison of Coverage via Citations, Scientometrics source

      Not checked in the free run.

    • cites [39] Harzing, 2019, Two New Kids on the Block: How Do Crossref and Dimensions Compare with Google Scholar, Microsoft Academic, Scopus and the Web of Science?, Scientometrics source

      Not checked in the free run.

    • cites [40] Visser, M., Jan Van Eck, N., and Waltman, L. (2020). Large-Scale Comparison of Bibliographic Data Sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. arXiv. source

      Not checked in the free run.

    • cites [41] Waltman, 2020, Special Issue on Bibliographic Data Sources, Quant. Sci. Stud. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 1. Introduction · p. 2

    Descriptions provided by DBs owners are spread across numerous resources, making it challenging to gather all relevant information and ensure its accuracy.

    “However, this way of obtaining information rarely is less time consuming, as the descriptions that are provided by DBs owners are spread across numerous resources, which makes it challenging to gather all relevant information and ensure its accuracy [27].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [27] Badia, 2018, Identifying “Best Bets” for Searching in Chemical Engineering: Comparing Database Content and Performance for Information Retrieval, J. Doc. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 2. Literature Overview · p. 6

    WoS and Scopus have specific deficiencies including inconsistencies of journal coverage, inaccuracy of subject classification schemes, missing citation information or citation links, incorrect and missing DOI numbers, duplicate entries, and inconsistent publication dates.

    “Meanwhile, other authors discussed more specific deficiencies of the DBs, such as inconsistencies of journal coverage [63], the accuracy of subject classification schemes [64], missing citation information or citation links [40,65,66], incorrect and missing DOI numbers [53,67-69], duplicate entries [30], and inconsistent publication dates [53].”
    Quote in the paperNot a substantial claim0 of 5 sources checked
    • cites [63] Krauskopf, 2019, Missing Documents in Scopus: The Case of the Journal Enfermeria Nefrologica, Scientometrics source

      Not checked in the free run.

    • cites [64] Wang, 2016, Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus, J. Informetr. source

      Not checked in the free run.

    • cites [40] Visser, M., Jan Van Eck, N., and Waltman, L. (2020). Large-Scale Comparison of Bibliographic Data Sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. arXiv. source

      Not checked in the free run.

    • cites [65] Olensky, 2016, Evaluation of the Citation Matching Algorithms of CWTS and IFQ in Comparison to the Web of Science, J. Assoc. Inf. Sci. Technol. source

      Not checked in the free run.

    • cites [66] Van Eck, N.J., and Waltman, L. (2017, January 16–20). Accuracy of Citation Data in Web of Science and Scopus. Proceedings of the 16th International Conference on Scientometrics and Informetrics (ISSI 2017), Wuhan, China.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 1. Introduction · p. 1

    Scopus has earned an equal place as a comprehensive bibliographic data source and has proven itself reliable and, in some respects, even better than WoS.

    “Over the years, Scopus has earned its equal place as a comprehensive bibliographic data source and it has proven itself to be reliable and, in some respects, even better than WoS [10,13].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [10] Zhu, 2020, A Tale of Two Databases: The Use of Web of Science and Scopus in Academic Papers, Scientometrics source

      Not checked in the free run.

    • cites [13] Harzing, 2016, Google Scholar, Scopus and the Web of Science: A Longitudinal and Cross-Disciplinary Comparison, Scientometrics source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § Main Bibliographic Databases · p. 3

    WoS is a multidisciplinary and selective DB composed of a variety of specialized indexes, grouped according to the type of indexed content or by theme.

    “WoSis a multidisciplinary and selective DB that is composed of a variety of specialized indexes, grouped according to the type of indexed content or by theme.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [28] Carloni, M., Tsenkulovsky, T., and Mangan, R. (2020, August 13). Web of Science Core Collection Descriptive Document. Available online: https://clarivate.libguides.com/ld.php?content_id=45175981.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § Main Bibliographic Databases · p. 3

    The main part of WoS platform is Core Collection (WoS CC), which includes six main citation indexes: SCIE, SSCI, A&HCI, CPCI, BKCI, and ESCI.

    “The main part of WoS platform is Core Collection (WoS CC), which includes six main citation indexes: Science Citation Index Expanded (SCIE); Social Sciences Citation Index (SSCI); Arts & Humanities Citation Index (A&HCI); Conference Proceedings Citation Index (CPCI); Books Citation Index (BKCI); and, established not long ago-Emerging Sources Citation Index (ESCI) [28].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [28] Carloni, M., Tsenkulovsky, T., and Mangan, R. (2020, August 13). Web of Science Core Collection Descriptive Document. Available online: https://clarivate.libguides.com/ld.php?content_id=45175981.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 1. Introduction · p. 1

    Both WoS and Scopus are commercial and subscription-based products, and their high price makes it rarely affordable for an institution to subscribe to both.

    “However, as both WoS and Scopus are commercial and subscription based products, the worldwide recognition and use of these DBs has resulted in their high price, which makes it rarely affordable for an institution to subscribe to both of them [14].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [14] Salehi, 2013, A Comparison between Two Main Academic Literature Collections: Web of Science and Scopus Databases, Asian Soc. Sci.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § Main Bibliographic Databases · p. 3

    Scopus includes content from many specialized databases, such as Embase, Compendex, World Textile Index, Fluidex, Geobase, Biobase, and Medline, and their content is integrated and equally accessible.

    “Thus, although Scopus also includes content from many specialized databases, such as Embase, Compendex, World Textile Index, Fluidex, Geobase, Biobase, and Medline [30], their content is integrated and equally accessible.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [30] 2015, A Systematic Analysis of Duplicate Records in Scopus, J. Informetr. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § Main Bibliographic Databases · p. 3

    Scopus is a multidisciplinary and selective DB, launched by Elsevier in November 2004.

    “Scopus is a similar multidisciplinary and selective DB, which was launched by Elsevier in November 2004 [12].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [12] Baas, 2020, Scopus as a Curated, High-Quality Bibliometric Data Source for Academic Research in Quantitative Science Studies, Quant. Sci. Stud. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.2. Coverage by Source and Document Types · p. 8

    Both WoS and Scopus continuously increase their coverage of all source types, showing stable overall content growth.

    “It was demonstrated that both DBs continuously increase their coverage of all source types, showing stable overall content growth [13,53].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [13] Harzing, 2016, Google Scholar, Scopus and the Web of Science: A Longitudinal and Cross-Disciplinary Comparison, Scientometrics source

      Not checked in the free run.

    • cites [53] Huang, 2020, Comparison of Bibliographic Data Sources: Implications for the Robustness of University Rankings, Quant. Sci. Stud.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.2. Coverage by Source and Document Types · p. 8

    The content of BkCI is from a number of publishers exceeding the book publisher list in Scopus two times, but judging by numbers of indexed items, Scopus provides better coverage of academic books compared to BkCI.

    “Meanwhile, another study has explored the distribution of publishers providing content to WoS and Scopus, and it has shown that the content of BkCI is from a number of publishers, exceeding the book publisher list in Scopus two times [82], but, judging by the numbers of indexed items, Scopus provides better coverage of academic books when compared to BkCI [90].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [82] Sivertsen, 2017, Scholarly Book Publishing: Its Information Sources for Evaluation in the Social Sciences and Humanities, Res. Eval. source

      Not checked in the free run.

    • cites [90] Kousha, 2018, Can Microsoft Academic Help to Assess the Citation Impact of Academic Books?, J. Informetr. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.2. Coverage by Source and Document Types · p. 7

    In Computer sciences, research findings tend to be published in conference papers, while books and text-books are more important sources in social sciences and even more so in humanities.

    “For example, it is well known that, in Computer sciences, the research findings tend to be published in conference papers [86,87], while books and text-books are more important sources in social sciences and even more so in humanities [26,46].”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [86] Meho, 2019, Using Scopus’s CiteScore for Assessing the Quality of Computer Science Conferences, J. Informetr. source

      Not checked in the free run.

    • cites [87] Grégoire, C., Roberge, G., and Archambault, É. Bibliometrics and Patent Indicators for the Science and Engineering Indicators 2016─Comparison of 2016 Bibliometric Indicators to 2014 Indicators. Science & Engineering Indicators 2016 (SEI 2016), Available online: https://science-metrix.com/?q=en/publications/reports&page=2#/?q=en/publications/reports/bibliometrics-and-patent-indicators-for-the-science-and-engineering-indicator-0.

      Not checked in the free run.

    • cites [26] Waltman, 2016, A Review of the Literature on Citation Impact Indicators, J. Informetr. source

      Not checked in the free run.

    • cites [46] Mingers, 2015, A Review of Theory and Practice in Scientometrics, Eur. J. Oper. Res. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.2. Coverage by Source and Document Types · p. 8

    The content coverage of Russian scientific literature is growing even exponentially, especially in Scopus, although higher growth rates in the coverage of conference material were determined in WoS compared to Scopus.

    “Meanwhile, the content coverage of Russian scientific literature was determined to be growing even exponentially, especially in Scopus, although higher growth rates in the coverage of conference material were determined in WoS, when compared to Scopus [52].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [52] Moed, 2018, Trends in Russian Research Output Indexed in Scopus and Web of Science, Scientometrics source

      Not checked in the free run.

  • May combine two claims.

    Premisenone given
    § 3.2. Coverage by Source and Document Types · p. 7

    WoS and Scopus are both concentrated on journal indexing.

    “However, WoS and Scopus are both concentrated on journal indexing.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.1. Overall Content Coverage, Overlap and Variations between Disciplines · p. 7

    The content indexed in WoS and Scopus is highly overlapping, with Scopus indexing a greater amount of unique sources not covered by WoS.

    “Generally, the content indexed in WoS and Scopus was also shown to be highly overlapping, with Scopus indexing a greater amount of unique sources not covered by WoS [44].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [44] Wouters, P., Thelwall, M., Kousha, K., Waltman, L., de Rijcke, S., Rushforth, A., and Franssen, T. (2015). The Metric Tide: Literature Review (Supplementary Report I to the Independent Review of the Role of Metrics in Research Assessment and Management), HEFCE.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.3. Coverage of Regional and Non-English Literature · p. 9

    During 2006-2016, the share of papers in Russian-language journals in Scopus increased from 4.8% to 14.8%, while in WoS it declined from 6.5% to 3.0%.

    “during 2006-2016 period the share of papers in Russian-language journals in Scopus increased from 4.8% to 14.8%, while, in WoS, this percentage declined from 6.5% to 3.0% during the same period [52].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [52] Moed, 2018, Trends in Russian Research Output Indexed in Scopus and Web of Science, Scientometrics source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.1. Overall Content Coverage, Overlap and Variations between Disciplines · p. 7

    Both WoS and Scopus offer the widest coverage of Natural, Medicine, Health Sciences, and Technology, while Social Sciences and Humanities (SSH) are underrepresented in both DBs.

    “These results were confirmed by later comparisons that were performed at the publication level, showing that both DBs offer the widest coverage of Natural, Medicine, Health Sciences, and Technology, while Social Sciences and Humanities (SSH) are underrepresented in both DBs [40,49,53,56].”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [40] Visser, M., Jan Van Eck, N., and Waltman, L. (2020). Large-Scale Comparison of Bibliographic Data Sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. arXiv. source

      Not checked in the free run.

    • cites [49] Thelwall, 2019, Web of Science and Scopus Language Coverage, Scientometrics source

      Not checked in the free run.

    • cites [53] Huang, 2020, Comparison of Bibliographic Data Sources: Implications for the Robustness of University Rankings, Quant. Sci. Stud.

      Not checked in the free run.

    • cites [56] Thelwall, 2018, Google Scholar, Web of Science, and Scopus: A Systematic Comparison of Citations in 252 Subject Categories, J. Informetr. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.2. Coverage by Source and Document Types · p. 8

    In 2011, the Book Citation Index (BkCI) was added to WoS CC, and Scopus expanded book coverage via Book Expansion Project during 2013-2015, adding 210,000 titles from selected publishers.

    “In 2011, the Book Citation Index (BkCI) was added to WoS CC [28]. Meanwhile, Scopus has also expanded the coverage of books by executing Book Expansion Project , which was fulfilled during 2013-2015 period, with 210,000 titles being added from selected publishers to Scopus in addition to the existing book series [89].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [28] Carloni, M., Tsenkulovsky, T., and Mangan, R. (2020, August 13). Web of Science Core Collection Descriptive Document. Available online: https://clarivate.libguides.com/ld.php?content_id=45175981.

      Not checked in the free run.

    • cites [89] Elsevier (2020, August 13). Scopus Content Coverage Guide. Available online: https://www.elsevier.com/solutions/scopus/how-scopus-works/content.

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § 3.2. Coverage by Source and Document Types · p. 8

    Increased growth rates in both DBs were mostly observed within the last decade and were generally higher in Scopus compared to WoS, most likely as a result of content coverage expansion programs executed by Elsevier.

    “Nevertheless, increased growth rates in both DBs were mostly observed within the period of the last decade, and they were were generally determined to be higher in Scopus, as compared to WoS, most likely as a result of aforementioned content coverage expansion programs executed by Elsevier.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.1. Overall Content Coverage, Overlap and Variations between Disciplines · p. 7

    WoS and Scopus are both biased toward Natural Sciences, Engineering, and Biomedical Research, with Scopus offering wider coverage of all investigated broad fields, especially of Biomedical Research.

    “For instance, a large-scale comparison performed at the journal level has shown that WoS and Scopus are both biased toward Natural Sciences, Engineering, and Biomedical Research, with Scopus offering wider coverage of all investigated broad fields, especially of Biomedical Research.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [47] Mongeon, 2016, The Journal Coverage of Web of Science and Scopus: A Comparative Analysis, Scientometrics source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation
    § 2. Literature Overview · p. 6

    The difference in accessible content imposed by WoS subscription terms that differ between institutions makes it very difficult to compare the results from different studies.

    “The difference in accessible content imposed by WoS subscription terms that differ between institutions, which makes it very difficult to compare the results from different studies, is another important limitation of works investigating WoS DB.”
    Quote in the paperSubstantial claim

  • May combine two claims. The quote only partly states this claim.

    Premisecitation
    § 3.3. Coverage of Regional and Non-English Literature · p. 9

    Scopus indexes more documents than WoS in Chinese, Danish, Japanese, Persian, Swedish, Russian, and French, while WoS indexes more documents in Spanish, Portuguese, Catalan, Croatian, Malay, Norwegian, and Turkish.

    “Scopus was shown to be indexing over ten-times more documents in Chinese than WoS.”
    • Also stated in: § 3.3. Coverage of Regional and Non-English Literature · p. 9 · plain
      “When compared to Scopus, WoS indexes more documents in Spanish and Portuguese, also Catalan, Croatian, Malay, Norwegian, and Turkish [49].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [49] Thelwall, 2019, Web of Science and Scopus Language Coverage, Scientometrics source

      Not checked in the free run.

    • cites [87] Grégoire, C., Roberge, G., and Archambault, É. Bibliometrics and Patent Indicators for the Science and Engineering Indicators 2016─Comparison of 2016 Bibliometric Indicators to 2014 Indicators. Science & Engineering Indicators 2016 (SEI 2016), Available online: https://science-metrix.com/?q=en/publications/reports&page=2#/?q=en/publications/reports/bibliometrics-and-patent-indicators-for-the-science-and-engineering-indicator-0.

      Not checked in the free run.

  • May combine two claims.

    Premisenone given
    § 3.3. Coverage of Regional and Non-English Literature · p. 8

    Both Scopus and WoS have been shown to be biased toward English-language sources and to have relatively poor coverage of regional literature.

    “Over the years, both Scopus and WoS were repeatedly shown to be biased toward sources that were published in the English language and providing relatively poor coverage of regional literature.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 3.1. Overall Content Coverage, Overlap and Variations between Disciplines · p. 7

    The majority of studies concluded a better coverage of Humanities in Scopus, but opposite observations indicating slightly better coverage of this discipline in WoS have also been noted.

    “Although the majority of the aforementioned studies concluded a better coverage of Humanities in Scopus, opposite observations indicating slightly better coverage of this discipline in WoS have also been noted [49,53].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [49] Thelwall, 2019, Web of Science and Scopus Language Coverage, Scientometrics source

      Not checked in the free run.

    • cites [53] Huang, 2020, Comparison of Bibliographic Data Sources: Implications for the Robustness of University Rankings, Quant. Sci. Stud.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.2. Coverage by Source and Document Types · p. 8

    In WoS, conference papers are indexed in a separate Conference Citation Index included in WoS CC in 2011; in Scopus, conferences are indexed together with the rest of content and coverage was expanded by Conference Expansion Program from 2011 to 2014.

    “In WoS, conference papers are indexed in a separate Conference Citation Index, which was included in WoS CC in 2011 [28]. In Scopus conferences are indexed together with the rest of content, and not long ago the coverage of conferences in Scopus was expanded by Conference Expansion Program, which ran from 2011 to 2014 [89].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [28] Carloni, M., Tsenkulovsky, T., and Mangan, R. (2020, August 13). Web of Science Core Collection Descriptive Document. Available online: https://clarivate.libguides.com/ld.php?content_id=45175981.

      Not checked in the free run.

    • cites [89] Elsevier (2020, August 13). Scopus Content Coverage Guide. Available online: https://www.elsevier.com/solutions/scopus/how-scopus-works/content.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.2. Coverage by Source and Document Types · p. 8

    Scopus indexes trade publications but does not cover book reviews and meeting abstracts, whereas WoS indexes book reviews and meeting abstracts.

    “Differently than WoS, Scopus indexes trade publications, but do not cover book reviews and meeting abstracts, which are indexed in WoS [28,40,89].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [28] Carloni, M., Tsenkulovsky, T., and Mangan, R. (2020, August 13). Web of Science Core Collection Descriptive Document. Available online: https://clarivate.libguides.com/ld.php?content_id=45175981.

      Not checked in the free run.

    • cites [40] Visser, M., Jan Van Eck, N., and Waltman, L. (2020). Large-Scale Comparison of Bibliographic Data Sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. arXiv. source

      Not checked in the free run.

    • cites [89] Elsevier (2020, August 13). Scopus Content Coverage Guide. Available online: https://www.elsevier.com/solutions/scopus/how-scopus-works/content.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.10.1. Content Indexing and Inclusion Policies · p. 17

    In both WoS and Scopus, the main criteria for source inclusion are the same: the source must be peer-reviewed, published in a timely manner, comply with editorial standards, and meet certain citation thresholds indicating impact.

    “However, the main criteria in both DBs are the same: the source must be peer-reviewed, published in a timely manner, comply with editorial standards and certain citation thresholds indicating its impact.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [12] Baas, 2020, Scopus as a Curated, High-Quality Bibliometric Data Source for Academic Research in Quantitative Science Studies, Quant. Sci. Stud. source

      Not checked in the free run.

    • cites [82] Sivertsen, 2017, Scholarly Book Publishing: Its Information Sources for Evaluation in the Social Sciences and Humanities, Res. Eval. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 3.10.1. Content Indexing and Inclusion Policies · p. 17

    Complying with editorial standards does not guarantee inclusion in WoS, while the quality of journal impact may have a greater influence.

    “from the perspective of universalistic criteria, it was observed that complying with the editorial standards does not guarantee the inclusion in WoS, while the quality of journal impact may have a greater influence [81].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [81] Chavarro, 2018, To What Extent Is Inclusion in TheWeb of Science an Indicator of Journal ‘Quality’?, Res. Eval. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3.10.1. Content Indexing and Inclusion Policies · p. 17

    Other authors have not determined any significant differences in the quality of journals indexed in both WoS and Scopus.

    “Meanwhile, other authors have not determined any significant differences in the quality of journals that were indexed in both DBs [58,85].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [58] 2015, On the Stability of Citation-Based Journal Rankings, J. Informetr. source

      Not checked in the free run.

    • cites [85] Powell, 2017, Coverage and Quality: A Comparison of Web of Science and Scopus Databases for Reporting Faculty Nursing Publication Metrics, Nurs. Outlook source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.6. Coverage of Funding Information · p. 11

    In a sample of the same medical journals, Scopus retrieved the lowest share of funded articles (7.7%), while WoS retrieved 29.0%.

    “Although Scopus indexed the highest number of articles in investigated journals, it retrieved the lowest share of FAs (only 7.7%), while, in WoS, the share of FAs was 29.0% [80].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [80] Kokol, 2018, Discrepancies among Scopus, Web of Science, and PubMed Coverage of Funding Information in Medical Journal Articles, J. Med. Libr. Assoc. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 3.10.1. Content Indexing and Inclusion Policies · p. 17

    The lower citation impact of unique Scopus sources may be attributed to the wider Scopus coverage.

    “but this may be attributed to the wider Scopus coverage.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.9. Institution Information · p. 16

    Both WoS OE and Scopus AFID provide adequate precision (0.95 and 0.96).

    “both of the systems provide adequate levels of precision (0.95 for WoS OE and 0.96 for Scopus AFID)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [72] Donner, 2020, Comparing Institutional-Level Bibliometric Research Performance Indicator Values Based on Different Affiliation Disambiguation Systems, Quant. Sci. Stud. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 3.10.1. Content Indexing and Inclusion Policies · p. 17

    Both WoS and Scopus could be biased against sources published by universities when considering their inclusion.

    “This may suggest that both DBs could be biased against sources published by universities when considering their inclusion.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.10.1. Content Indexing and Inclusion Policies · p. 17

    Both WoS and Scopus require abstracts and titles to be written in English and state the importance of content's international relevance with references listed in Roman script.

    “Additionally, the linguistic criterion exists, requiring abstracts and titles to be written in English language, and the importance of content's international relevance with references being listed in Roman script is clearly stated [12,82].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [12] Baas, 2020, Scopus as a Curated, High-Quality Bibliometric Data Source for Academic Research in Quantitative Science Studies, Quant. Sci. Stud. source

      Not checked in the free run.

    • cites [82] Sivertsen, 2017, Scholarly Book Publishing: Its Information Sources for Evaluation in the Social Sciences and Humanities, Res. Eval. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.10.1. Content Indexing and Inclusion Policies · p. 18

    Sources already indexed are reevaluated annually, and after reevaluation they may be discontinued or suppressed if they fail criteria such as self-citation increase, publishing irregularities, or suspected unethical practices.

    “Sources that are already indexed are being reevaluated annually. After reevaluation, sources may be discontinued (removed) or suppressed. These measures could be applied if the source fails to meet certain indexing criteria, e.g., when journal self-citation patterns drastically increase and exceed the established threshold, the irregularities in publishing are determined or unethical publishing”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [138] Clarivate (2020, November 25). Web of Science Journal Citation Reports: Suppression Policy. Available online: https://clarivate.com/webofsciencegroup/wp-content/uploads/sites/2/2020/06/jcr-suppression-policy-2020.pdf.

      Not checked in the free run.

    • cites [143] Holland, K., Brimblecombe, P., Meester, W., and Steiginga, S. (2020, November 26). The Importance of High-Quality Content: Curation and Re- Evaluation in Scopus. Elsevier. Available online: https://www.elsevier.com/research-intelligence/resource-library/scopus-high-quality-content.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.9. Institution Information · p. 15

    Only 20% of investigated institutions were located in WoS OE, while 85% were located in Scopus AFID.

    “only 20% of investigated institutions (445) were located in WoS OE, which was mainly attributed to the aforementioned fact that WoS OE does not cover non-university research institutes. Meanwhile, in the Scopus AFID system, 85% of institutions were located [72].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [72] Donner, 2020, Comparing Institutional-Level Bibliometric Research Performance Indicator Values Based on Different Affiliation Disambiguation Systems, Quant. Sci. Stud. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3.10.1. Content Indexing and Inclusion Policies · p. 17

    Unique sources indexed by Scopus but not by WoS had lower citation impact values.

    “It was observed that the unique sources that were indexed by Scopus, but not by WoS, had lower citation impact values [57]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [57] 2018, Tale of Three Databases: The Implication of Coverage Demonstrated for a Sample Query, Front. Res. Metrics Anal. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.5. Coverage of Patents · p. 10

    In Scopus, patents are equally accessible with a subscription as the rest of Scopus content, whereas in WoS, DWPI is not part of WoS CC and patent information is not accessible through basic WoS CC subscription.

    “Although the overall number of patents indexed in Scopus is smaller than in WoS, they are equally accessible with a subscription as the rest of the content of Scopus DB. That is not the case in WoS, as DWPI is not a part of WoS CC and, therefore, patent information is not accessible through the basic WoS CC subscription [40].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [40] Visser, M., Jan Van Eck, N., and Waltman, L. (2020). Large-Scale Comparison of Bibliographic Data Sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. arXiv. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.6. Coverage of Funding Information · p. 12

    WoS performs much better than Scopus in both coverage and accuracy of funding information.

    “the most recent comparative studies between WoS and Scopus have shown that WoS performs much better than Scopus in both coverage and accuracy of FI [79,80].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [79] Liu, 2020, Accuracy of Funding Information in Scopus: A Comparative Case Study, Scientometrics source

      Not checked in the free run.

    • cites [80] Kokol, 2018, Discrepancies among Scopus, Web of Science, and PubMed Coverage of Funding Information in Medical Journal Articles, J. Med. Libr. Assoc. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 3.10. Content Quality · p. 16

    OA publishing makes research more visible, transparent, and reproducible, and shortens time to publication, but its possible effects on research and journal quality have been questioned.

    “although OA publishing is very beneficial as free access makes scientific researches more visible, transparent, and reproducible, and also shortens the time required for publication to be available online, the unintended, yet possible, OA effects on research and journal quality have become questioned [133].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [133] 2019, The Effect of Open Access on Research Quality, J. Informetr. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning
    § 3.10.1. Content Indexing and Inclusion Policies · p. 18

    In Scopus, the exact percentage of evaluated journals indexed is not declared, but it is most certainly higher than in WoS and likely contributes to wider Scopus content coverage.

    “while, in Scopus, the exact percentage is not declared, but is most certainly higher and likely contributes to a wider Scopus content coverage.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.4. Coverage of Citations · p. 9

    WoS covers citations up to 1900, while Scopus citation coverage was only extended to 1970.

    “WoS may appear to be more suitable for large-scale citation analyses, as it covers citations up to 1900, while, in Scopus, citation coverage was only extended to 1970 [89].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [89] Elsevier (2020, August 13). Scopus Content Coverage Guide. Available online: https://www.elsevier.com/solutions/scopus/how-scopus-works/content.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.10.1. Content Indexing and Inclusion Policies · p. 18

    In some cases, mostly in Scopus, removal of a journal from the database may result in entire removal of previously indexed papers from that journal, while in other cases, mostly in WoS, previously indexed papers are not necessarily removed.

    “in some cases (mostly in Scopus), the removal of a journal from the DB may result in the entire removal of previously indexed papers from this journal, while, in other cases (mostly on WoS), previously indexed papers are not necessarily removed [62].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [62] Franceschini, 2016, Empirical Analysis and Classification of Database Errors in Scopus and Web of Science, J. Informetr. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation
    § 3.10. Content Quality · p. 16

    Discrepancies in publication metadata, source information, or other inconsistencies in bibliographic databases complicate analysis and can negatively affect the accuracy and reliability of obtained results.

    “all of the discrepancies in the publication metadata, source information, or other inconsistencies occurring in the DBs not only complicate analysis and other tasks, but can also negatively affect the accuracy and reliability of the obtained results [4,61,95].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [4] Leydesdorff, 2016, Professional and Citizen Bibliometrics: Complementarities and Ambivalences in the Development and Use of Indicators—a State-of-the-Art Report, Scientometrics source

      Not checked in the free run.

    • cites [61] Franceschini, 2015, Influence of Omitted Citations on the Bibliometric Statistics of the Major Manufacturing Journals, Scientometrics source

      Not checked in the free run.

    • cites [95] Rafols, I., Molas-Gallart, J., and Woolley, R. (2016, January 1–7). Data Quality and Consistency in Scopus and Web of Science in Their Indexing of Czech Journals. Proceedings of the 21st International Conference on Science and Technology Indicators (STI 2016), València, Spain.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.10.1. Content Indexing and Inclusion Policies · p. 18

    Wider inclusion of journals into Scopus is associated with a higher number of journals removed, but the overall proportion of short-lived journals included and excluded in a relatively short period appears higher in WoS.

    “although the wider inclusion of journals into Scopus is also associated with the higher number of journals being removed, the overall proportion of 'short-lived' journals that have been included and excluded from coverage in a relatively short period appears to be higher in WoS [58].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [58] 2015, On the Stability of Citation-Based Journal Rankings, J. Informetr. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing). May combine two claims.

    Premisecitation
    § 3.10.1. Content Indexing and Inclusion Policies · p. 16

    A high quality of a journal is perceived as its inclusion in WoS and Scopus, because these databases allegedly only index the highest quality sources carefully selected according to strict selection procedures.

    “Generally, a high quality of the journal is perceived as its inclusion in WoS and Scopus, because these DBs allegedly only index the highest quality sources carefully selected according to strict selection procedures [134].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [134] Dorsch, I., Askeridis, J.M., and Stock, W.G. (2018). Truebounded, Overbounded, or Underbounded? Scientists’ Personal Publication Lists versus Lists Generated through Bibliographic Information Services. Publications, 6. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given
    § 3.10.2. Open Access Content and 'Predatory' Journals · p. 19

    OA journals are only published in electronic format, shortening the time delay from submission to publishing.

    “OAjournals are only published in electronic format shortening the time delay from submission to publishing.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.10.2. Open Access Content and 'Predatory' Journals · p. 19

    Predatory journals have evolved and manage to pass the selection process of major bibliographic and OA databases.

    “these artificial journals have evolved and they even manage to pass the selection process of the major bibliographic, as well as OA, DBs [145,150,157,158,166]”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [145] Cortegiani, 2020, Inflated Citations and Metrics of Journals Discontinued from Scopus for Publication Concerns: The GhoS(t)Copus Project, F1000Research

      Not checked in the free run.

    • cites [150] Shen, 2015, “Predatory” Open Access: A Longitudinal Study of Article Volumes and Market Characteristics, BMC Med. source

      Not checked in the free run.

    • cites [157] Fernandez, 2019, Predatory Publishing or a Lack of Peer Review Transparency?—A Contemporary Analysis of Indexed Open and Non-Open Access Articles in Paediatric Urology, J. Pediatr. Urol. source

      Not checked in the free run.

    • cites [158] Demir, 2018, Predatory Journals: Who Publishes in Them and Why?, J. Informetr. source

      Not checked in the free run.

    • cites [166] Björk, B.C., Kanto-Karvonen, S., and Harviainen, J.T. (2020). How Frequently Are Articles in Predatory Open Access Journals Cited. Publications, 8. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationcitation
    § 3.10.2. Open Access Content and 'Predatory' Journals · p. 19

    The validity of measuring the quality of OA journals by citations and impact indicators may seem questionable.

    “Thus, the validity of measuring the quality of OA journals by citations and impact indicators may seem questionable [128].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [128] Wei, 2020, Research on Impact Evaluation of Open Access Journals, Scientometrics source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.10.2. Open Access Content and 'Predatory' Journals · p. 18

    Many studies have demonstrated that OA content attracts more citations, although others did not observe a significant OA advantage in citation impact.

    “Indeed, many studies have demonstrated that OA content attracts more citations [129,133,153-155], although others did not observe significant OA advantage in citation impact [156,157].”
    Quote in the paperNot a substantial claim0 of 7 sources checked
    • cites [129] Asai, 2020, The Effect of Collaboration with Large Publishers on the Internationality and Influence of Open Access Journals for Research Institutions, Scientometrics source

      Not checked in the free run.

    • cites [133] 2019, The Effect of Open Access on Research Quality, J. Informetr. source

      Not checked in the free run.

    • cites [153] Copiello, 2019, The Open Access Citation Premium May Depend on the Openness and Inclusiveness of the Indexing Database, but the Relationship Is Controversial Because It Is Ambiguous Where the Open Access Boundary Lies, Scientometrics source

      Not checked in the free run.

    • cites [154] Piwowar, H., Priem, J., Larivière, V., Alperin, J.P., Matthias, L., Norlander, B., Farley, A., West, J., and Haustein, S. (2018). The State of OA: A Large-Scale Analysis of the Prevalence and Impact of Open Access Articles. PeerJ, 6. source

      Not checked in the free run.

    • cites [155] Wang, 2015, The Open Access Advantage Considering Citation, Article Usage and Social Media Attention, Scientometrics source

      Not checked in the free run.

    • cites [156] Holmberg, 2020, Do Articles in Open Access Journals Have More Frequent Altmetric Activity than Articles in Subscription - Based Journals? An Investigation of the Research Output of Finnish Universities, Scientometrics source

      Not checked in the free run.

    • cites [157] Fernandez, 2019, Predatory Publishing or a Lack of Peer Review Transparency?—A Contemporary Analysis of Indexed Open and Non-Open Access Articles in Paediatric Urology, J. Pediatr. Urol. source

      Not checked in the free run.

  • May combine two claims.

    Premisenone given
    § 3.10.2. Open Access Content and 'Predatory' Journals · p. 19

    In January 2018, Elsevier discontinued indexing of 424 journals in Scopus, indicating 'publication concerns' and, less often, 'metrics' as the reason for discontinuation.

    “in January 2018, Elsevier discontinued indexing of 424 journals in Scopus, indicating 'publication concerns' and, less often, 'metrics' as the reason for discontinuation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 3.10.2. Open Access Content and 'Predatory' Journals · p. 19

    In 2012, Jeffrey Beall composed and published a list of nearly 50 criteria for identifying predatory publishing, and continuously updates these criteria and an index of publishers and individual journals fulfilling such criteria.

    “In 2012, Jeffrey Beall composed and published a list of nearly 50 criteria for identifying predatory publishing, and continuously updates these criteria and an index of publishers as well as individual journals fulfilling such criteria [162].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [162] Beall, J. (2020, November 25). Criteria for Determining Predatory Publishers. Available online: https://beallslist.net/wp-content/uploads/2019/12/criteria-2015.pdf.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationcitation
    § 3.10.2. Open Access Content and 'Predatory' Journals · p. 19

    The OA citation advantage may be more related to free access to content and less to the actual quality of published research.

    “it can be argued that the OA citation advantage may be more related to the free access to content, and less to the actual quality of published research [153]”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [153] Copiello, 2019, The Open Access Citation Premium May Depend on the Openness and Inclusiveness of the Indexing Database, but the Relationship Is Controversial Because It Is Ambiguous Where the Open Access Boundary Lies, Scientometrics source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.10.3. Errors in Publication Metadata · p. 20

    The main consequence of all errors in publication metadata is missing or incorrect links between citing and cited papers in the database, which leads to missing citations.

    “since the main consequence of all errors in publication metadata is missing or incorrect links between citing and cited papers in the DB, which leads to missing citations [61,62,65,66].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [61] Franceschini, 2015, Influence of Omitted Citations on the Bibliometric Statistics of the Major Manufacturing Journals, Scientometrics source

      Not checked in the free run.

    • cites [62] Franceschini, 2016, Empirical Analysis and Classification of Database Errors in Scopus and Web of Science, J. Informetr. source

      Not checked in the free run.

    • cites [65] Olensky, 2016, Evaluation of the Citation Matching Algorithms of CWTS and IFQ in Comparison to the Web of Science, J. Assoc. Inf. Sci. Technol. source

      Not checked in the free run.

    • cites [66] Van Eck, N.J., and Waltman, L. (2017, January 16–20). Accuracy of Citation Data in Web of Science and Scopus. Proceedings of the 16th International Conference on Scientometrics and Informetrics (ISSI 2017), Wuhan, China.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3.10.2. Open Access Content and 'Predatory' Journals · p. 19

    There is no common definition allowing clear distinguishing between predatory and credible OA publishing.

    “However, there is no common definition allowing for clearly distinguishing between predatory and credible OA publishing [164].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [164] Tsigaris, 2019, Predatory and Exploitative Behaviour in Academic Publishing: An Assessment, J. Acad. Librariansh. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 3.10.2. Open Access Content and 'Predatory' Journals · p. 19

    Features that may signal a possible predatory nature of a journal include APC charges usually much lower than credible OA journals, requests to pay APC only after acceptance, absence or questionable location and composition of editorial boards, members lacking academic competencies, and failure to clearly declare manuscript submission, revision, acceptation, and licensing policies.

    “For instance, the APC charges of predatory journals are usually much lower when compared with the APCs applied by credible OA journals. Additionally, often authors are requested to pay APC only after the acceptance of their manuscript. Other features include the absence or a questionable location and composition of journal editorial boards, as their members are often lacking academic competencies.”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [131] Tennant, 2016, The Academic, Economic and Societal Impacts of Open Access: An Evidence-Based Review, F1000Research source

      Not checked in the free run.

    • cites [150] Shen, 2015, “Predatory” Open Access: A Longitudinal Study of Article Volumes and Market Characteristics, BMC Med. source

      Not checked in the free run.

    • cites [157] Fernandez, 2019, Predatory Publishing or a Lack of Peer Review Transparency?—A Contemporary Analysis of Indexed Open and Non-Open Access Articles in Paediatric Urology, J. Pediatr. Urol. source

      Not checked in the free run.

    • cites [158] Demir, 2018, Predatory Journals: Who Publishes in Them and Why?, J. Informetr. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisereasoning
    § 3.10.2. Open Access Content and 'Predatory' Journals · p. 19

    Many publishers and journals exploit the OA model and its attractiveness to collect APCs but are highly questionable regarding peer-review practices and overall trustworthiness.

    “Today, there are many publishers and journals that exploit OA model and its attractiveness for authors in order to collect APCs, but they are highly questionable regarding peer-review practices and overall trustworthiness.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.10.2. Open Access Content and 'Predatory' Journals · p. 19

    Although Beall's criteria are often considered controversial, predatory journals usually fulfilled several of the criteria outlined by Beall.

    “Although these criteria are often considered to be controversial [164], it was shown that predatory journals usually fulfilled several of the criteria outlined by Beall [150,165].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [164] Tsigaris, 2019, Predatory and Exploitative Behaviour in Academic Publishing: An Assessment, J. Acad. Librariansh. source

      Not checked in the free run.

    • cites [150] Shen, 2015, “Predatory” Open Access: A Longitudinal Study of Article Volumes and Market Characteristics, BMC Med. source

      Not checked in the free run.

    • cites [165] Manca, 2017, The Surge of Predatory Open-Access in Neurosciences and Neurology, Neuroscience source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § 3.10.2. Open Access Content and 'Predatory' Journals · p. 19

    Predatory journals usually exploit names and other details, even webpages, of credible journals.

    “Predatory journals usually exploit names and other details, even webpages of credible journals.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given
    § 3.10.3. Errors in Publication Metadata · p. 20

    Errors in databases may be divided into pre-existing errors, made by authors, publishers, or editors, and database errors, introduced by the database due to inaccuracies in data uploading such as transcription errors or omitted data.

    “Errors that occur in the DBs may be divided into two main groups, depending on their origin: pre-existing errors, which are made by authors/publishers/editors in preparation and publication of the manuscripts; and, database errors, which are introduced by the DB due to inaccuracies occurring in data uploading processes, such as transcription errors or omitted data.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.10.3. Errors in Publication Metadata · p. 20

    In a recent large-scale analysis, author name mistakes were present in both databases, but in Scopus frequencies were lower for almost all error types except cases where part of the first name was mistakenly assigned to the last name, which appeared in fewer than 1% of Scopus entries and were not detected at all in WoS.

    “A recent large scale analysis has shown that, although these mistakes are present in both DBs, but, in Scopus, frequencies are lower for almost all error types in author names, except for the cases where part of the first name is being mistakenly assigned to the last name, which appeared in fewer than 1% of Scopus entries, while, in WoS, were not detected at all.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [71] Demetrescu, 2018, Accuracy of Author Names in Bibliographic Data Sources: An Italian Case Study, Scientometrics source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.10.3. Errors in Publication Metadata · p. 24

    Various other mistakes in WoS FI were determined, such as inconsistent registering of co-funded grants or multiple grants being assigned to the same funder.

    “various other mistakes in WoS FI were determined, such as inconsistent registering of co-funded grants or multiple grants being assigned to the same funder”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [78] Morillo, 2017, Funding Acknowledgments in the Web of Science: Completeness and Accuracy of Collected Data, Scientometrics source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.10.3. Errors in Publication Metadata · p. 20

    In both databases, in almost all last names containing diacritics, approximately 95%, diacritics were omitted, but none were incorrectly imported.

    “in both DBs in almost all last names (approx. 95%), containing diacritics (only approx. 1% of investigated names contained this feature), diacritics were omitted, but none of the diacritics were incorrectly imported [71].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [71] Demetrescu, 2018, Accuracy of Author Names in Bibliographic Data Sources: An Italian Case Study, Scientometrics source

      Not checked in the free run.

  • May combine two claims.

    Premiseown data
    § 3.10.4. Inconsistencies in Subject Classification Schemes and Document Types · p. 25

    The situation of journals showing strong relations to categories they are not assigned to was less pronounced in both DBs, and the difference between WoS and Scopus was far less noticeable.

    “the situation of journals showing strong relations to the categories that they are not assigned to was less pronounced in both DBs, and the difference between WoS and Scopus was far less noticeable”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [64] Wang, 2016, Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus, J. Informetr. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.10.4. Inconsistencies in Subject Classification Schemes and Document Types · p. 25

    Classification schemes applied in both WoS and Scopus lack documentation adequately describing the methodology used to construct them, thus making their use for bibliometric analyses problematic.

    “classification schemes applied in both WoS and Scopus lack documentation adequately describing the methodology used to construct them, thus making their use for bibliometric analyses problematic”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [64] Wang, 2016, Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus, J. Informetr. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.10.4. Inconsistencies in Subject Classification Schemes and Document Types · p. 24

    Scopus uses a two-level classification: four major disciplines (Life Sciences, Physical Sciences, Health Sciences, Social Sciences and Humanities), subdivided into 27 categories, further broken down into more than 300 subject fields based on ASJC.

    “Scopus uses classification of two levels. All of the sources are divided into four major disciplines: Life Sciences, Physical Sciences, Health Sciences, and Social Sciences and Humanities. These disciplines are subdivided into 27 categories, which are further broken down into more than 300 subject fields, based on All Science Journal Classification (ASJC)”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [64] Wang, 2016, Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus, J. Informetr. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.10.3. Errors in Publication Metadata · p. 21

    For remaining publications with missing address information in WoS, the information was also not declared in the publications themselves, probably due to different editorial policies of journals.

    “for the remaining publications with missing address information in WoS, the information was also not declared in the publications themselves, which was probably due to the different editorial policies of journals [73].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [73] Liu, 2018, Missing Author Address Information in Web of Science—An Explorative Study, J. Informetr. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.10.3. Errors in Publication Metadata · p. 20

    In both databases, incomplete and mistyped last names were quite rare, rarely exceeding 1%, with approximately two times more such mistakes in WoS than in Scopus.

    “although, in both DBs, incomplete and mistyped last names were quite rare (rarely exceeding 1%), approximately two-times more of these mistakes were identified in WoS when compared to Scopus”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [71] Demetrescu, 2018, Accuracy of Author Names in Bibliographic Data Sources: An Italian Case Study, Scientometrics source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning
    § 3.10.3. Errors in Publication Metadata · p. 20

    Spelling mistakes mostly occur in the publication title, author names, and institutional affiliations.

    “Spelling mistakes mostly occur in the publication title, author names, and institutional affiliations.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisereasoning
    § 3.10.3. Errors in Publication Metadata · p. 20

    Not all mistakes in author names are caused by databases; authors can introduce spelling mistakes or use different valid name variants in different publications.

    “Not all of the mistakes in author names are being caused by the DBs, since the author himself/herself can introduce spelling mistakes or may decide to use different, yet valid, name variants in different publications.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.10.3. Errors in Publication Metadata · p. 21

    More than one-fifth of publications in WoS lacked address information, while full-text analyses revealed that about 40% of articles actually had at least partial address information, and part had full address information but it was not indexed by WoS.

    “The study showed that more than one-fifth of the publications in WoS lacked address information, while full-text analyses have revealed that about 40% of the articles actually had at least a partial address information listed, and a part of investigated publications had full address information, but, for some reason, it was not indexed by WoS.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [73] Liu, 2018, Missing Author Address Information in Web of Science—An Explorative Study, J. Informetr. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.10.3. Errors in Publication Metadata · p. 24

    In WoS, a given funder was almost always included in the FO subfield, while specific grant information appeared in only about half of the entries.

    “a given funder was almost always included in the FO subfield, while specific grant information appeared in only about half of the entries”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [78] Morillo, 2017, Funding Acknowledgments in the Web of Science: Completeness and Accuracy of Collected Data, Scientometrics source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.10.4. Inconsistencies in Subject Classification Schemes and Document Types · p. 25

    The reliability of results obtained from different DBs may be affected not only by incompatible classification schemes, but also by questionable assignation of indexed sources to subject fields within the DBs.

    “the reliability of the results that were obtained from different DBs may be affected not only by the incompatible classification schemes, but also by a questionable assignation of indexed sources to the subject fields within the DBs”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [64] Wang, 2016, Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus, J. Informetr. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § 3.10.4. Inconsistencies in Subject Classification Schemes and Document Types · p. 24

    This becomes problematic when data from both sources are used or compared, since large-scale analyses usually involve aggregation of data from multiple disciplines and subject fields highly differing in publication and citation practices, which requires classifying the data under a common classification scheme.

    “This becomes problematic when data from both sources are used or compared, since the large-scale analyses usually involve the aggregation of data from multiple disciplines and subject fields highly differing in publication and citation practices, which, in turn, requires classifying the data under a common classification scheme.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § 3.10.3. Errors in Publication Metadata · p. 21

    Authors determined that address information may also be missing in Scopus, but the extent of the issue could not be assessed reliably.

    “Although authors have determined that address information may also be missing in Scopus, the extent of the issue could not be assessed reliably [73].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [73] Liu, 2018, Missing Author Address Information in Web of Science—An Explorative Study, J. Informetr. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning
    § 3.10.3. Errors in Publication Metadata · p. 20

    First and last names might be misplaced due to the editorial style of the journal.

    “Additionally, the first and last names might be misplaced due to the editorial style of the journal”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.10.4. Inconsistencies in Subject Classification Schemes and Document Types · p. 25

    Both WoS and Scopus classification schemes are generally regarded as being far from perfect, although their empirical comparisons are very sparse.

    “both WoS and Scopus classification schemes are generally regarded as being far from perfect, although their empirical comparisons are very sparse”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [64] Wang, 2016, Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus, J. Informetr. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § 3.11. Comprehensiveness of Information Provided by the DBs' Owners · p. 26

    Clarivate and Elsevier both provide extensive information describing WoS and Scopus DBs, including content coverage, additional tools, features, and indexing policies.

    “Clarivate and Elsevier both provide extensive information describing WoS and Scopus DBs, including not only content coverage, but also all additional tools, features, and indexing policies of DBs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3.11. Comprehensiveness of Information Provided by the DBs' Owners · p. 26

    Clarivate also provides various fact sheets and reference guides that are available for downloading from LibGuides (and other) sites.

    “Clarivate also provides various fact sheets and reference guides that are available for downloading from LibGuides (and other) sites”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [195] Clarivate (2020, November 26). Web of Science Core Collection: Quick Reference Cards (PDF). Available online: https://clarivate.libguides.com/woscc/guides.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § Content Coverage Evaluation by Source Lists · p. 28

    An accurate evaluation of WoS CC content coverage is highly challenging, as WoS is composed of a variety of specialized indexes and the composition of WoS CC can be modulated by different subscription terms.

    “An accurate evaluation of WoS CC content coverage is highly challenging, as WoS is composed of a variety of specialized indexes and the composition of WoS CC can be modulated by different subscription terms”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [15] Dallas, 2018, Variable Bibliographic Database Access Could Limit Reproducibility, Bioscience source

      Not checked in the free run.

    • cites [42] Gusenbauer, 2020, Which Academic Search Systems Are Suitable for Systematic Reviews or Meta-Analyses? Evaluating Retrieval Qualities of Google Scholar, PubMed, and 26 Other Resources, Res. Synth. Methods source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.10.4. Inconsistencies in Subject Classification Schemes and Document Types · p. 25

    Journals in Scopus are usually assigned with a higher number of categories, and there are significantly more 'multidisciplinary' categories in Scopus classification compared to WoS.

    “journals in Scopus are usually assigned with a higher number of categories, and there are significantly more 'multidisciplinary' categories in Scopus classification when compared to WoS”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [58] 2015, On the Stability of Citation-Based Journal Rankings, J. Informetr. source

      Not checked in the free run.

    • cites [64] Wang, 2016, Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus, J. Informetr. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.10.4. Inconsistencies in Subject Classification Schemes and Document Types · p. 25

    In both DBs, journals are often assigned to multidisciplinary categories or with more than one subject field, but these practices were much more frequent in Scopus than in WoS.

    “In both DBs, journals are often being assigned to multidisciplinary categories or with more than one subject field, but these practices were much more frequent in Scopus than in WoS.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [58] 2015, On the Stability of Citation-Based Journal Rankings, J. Informetr. source

      Not checked in the free run.

    • cites [64] Wang, 2016, Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus, J. Informetr. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3.11. Comprehensiveness of Information Provided by the DBs' Owners · p. 27

    Both documents are available at LibGuides site.

    “Both of the documents are available at LibGuides site”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [191] Matthews, T. (2020, November 25). Web of Science Group: Welcome to our Training Portal. Available online: https://clarivate.libguides.com/home/welcome.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning
    § 3.11. Comprehensiveness of Information Provided by the DBs' Owners · p. 26

    Descriptive information is scattered across a multitude of information resources, which makes it very difficult and time-consuming to gather all relevant facts and characteristics necessary to create an overview of DBs and assess their suitability for a particular task.

    “descriptive information is scattered across a multitude of information resources, which makes it very difficult and time-consuming to gather all of the relevant facts and characteristics that are necessary to create an overview of DBs and assess their suitability for a particular task”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.10.4. Inconsistencies in Subject Classification Schemes and Document Types · p. 26

    Some cases of such double assignment were also noticed in Scopus, but the occurrence of this error was estimated to be very low.

    “Some cases of such double assignment were also noticed in Scopus, but the occurrence of this error was estimated to be very low”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [62] Franceschini, 2016, Empirical Analysis and Classification of Database Errors in Scopus and Web of Science, J. Informetr. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3.11. Comprehensiveness of Information Provided by the DBs' Owners · p. 27

    The most recent version of Scopus Content Coverage Guide available for downloading was last updated in January 2020.

    “The most recent version currently available for downloading at Elsevier webpage [146] and at the Scopus web-interface was last updated in January 2020.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [146] Elsevier (2020, November 25). Content—How Scopus Works. Available online: https://www.elsevier.com/solutions/scopus/how-scopus-works/content.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given
    § 3.10.4. Inconsistencies in Subject Classification Schemes and Document Types · p. 25

    The study only investigated journals with more than 100 citations, which might result in inaccurate evaluation of small subject categories covering less popular and more difficult to assign subject fields.

    “the study only investigated journals with more than 100 citations, which might result in the inaccurate evaluation of small subject categories covering less popular and more difficult to assign subject fields”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning
    § 3.11. Comprehensiveness of Information Provided by the DBs' Owners · p. 27

    Since both WoS and Scopus are commercial DBs and major competitors, their vendors are mostly interested in promoting their product.

    “since both WoS and Scopus are commercial DBs and major competitors, their vendors are mostly interested in promoting their product”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 3.11. Comprehensiveness of Information Provided by the DBs' Owners · p. 27

    Elsevier provides fewer descriptive documents, but more detailed descriptions of newly implemented features, including visual examples, are also provided in Scopus Blog.

    “Although Elsevier provides fewer descriptive documents, but more detailed descriptions of newly implemented features, including visual examples, are also provided in Scopus Blog”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [204] Elsevier (2020, November 25). Product Releases | Elsevier Scopus Blog. Available online: https://blog.scopus.com/product-releases.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationcitation
    § 3.10.4. Inconsistencies in Subject Classification Schemes and Document Types · p. 25

    The higher numbers of inaccuracies determined in Scopus might be related to the higher numbers of Scopus journals included in the study due to wider Scopus coverage, but the differences in DBs' coverage were not accounted for.

    “one can argue that the higher numbers of inaccuracies determined in Scopus might be related to the higher numbers of Scopus journals included in the study due to the wider Scopus coverage, but the differences in DBs' coverage were not accounted for”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [64] Wang, 2016, Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus, J. Informetr. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 3.11. Comprehensiveness of Information Provided by the DBs' Owners · p. 27

    Both DBs' owners also offer visual training materials and organize webinars, explaining the most effective use of their products.

    “both DBs' owners also offer visual training materials and organize webinars, explaining the most effective use of their products”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [205] Elsevier (2020, November 25). Scopus tutorials—Scopus: Access and Use Support Center. Available online: https://service.elsevier.com/app/answers/detail/a_id/14799/kw/export/supporthub/scopus/related/1/.

      Not checked in the free run.

    • cites [206] Clarivate (2020, November 25). Web of Science-Videos. Available online: https://videos.webofsciencegroup.com/.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3.11. Comprehensiveness of Information Provided by the DBs' Owners · p. 27

    The last version of the WoS CC descriptive document available was updated on July 2018.

    “the last version available was updated on July 2018”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [28] Carloni, M., Tsenkulovsky, T., and Mangan, R. (2020, August 13). Web of Science Core Collection Descriptive Document. Available online: https://clarivate.libguides.com/ld.php?content_id=45175981.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.10.4. Inconsistencies in Subject Classification Schemes and Document Types · p. 26

    Inconsistencies in document types may occur because DBs differ by the document types they index and by classification of documents, so some documents from the same source are present in one DB but not the other.

    “These inconsistencies may occur due to the fact that DBs differ by the document types that they index and by the classification of the documents, therefore some documents from the same source are present in one DB, but not in the other”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [59] Hug, 2017, The Coverage of Microsoft Academic: Analyzing the Publication Output of a University, Scientometrics source

      Not checked in the free run.

    • cites [95] Rafols, I., Molas-Gallart, J., and Woolley, R. (2016, January 1–7). Data Quality and Consistency in Scopus and Web of Science in Their Indexing of Czech Journals. Proceedings of the 21st International Conference on Science and Technology Indicators (STI 2016), València, Spain.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3.11. Comprehensiveness of Information Provided by the DBs' Owners · p. 27

    General information regarding Scopus is available at Elsevier website.

    “General information regarding Scopus is available at Elsevier website”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [202] Elsevier (2020, November 25). About Scopus—Abstract and citation database. Available online: https://www.elsevier.com/solutions/scopus.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.10.4. Inconsistencies in Subject Classification Schemes and Document Types · p. 26

    While generally high correlation between journal ranks between WoS and Scopus were observed, considerable variations in rank for individual journals were also noticed.

    “while generally high correlation between journal ranks between WoS and Scopus were observed to be high, considerable variations in rank for individual journals were also noticed”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [48] Leydesdorff, 2016, Aggregated Journal–Journal Citation Relations in Scopus and Web of Science Matched and Compared in Terms of Networks, Maps, and Interactive Overlays, J. Assoc. Inf. Sci. Technol. source

      Not checked in the free run.

    • cites [58] 2015, On the Stability of Citation-Based Journal Rankings, J. Informetr. source

      Not checked in the free run.

    • cites [181] Campanario, 2018, Are Leaders Really Leading? Journals That Are First in Web of Science Subject Categories in the Context of Their Groups, Scientometrics source

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation
    § 3.10.4. Inconsistencies in Subject Classification Schemes and Document Types · p. 26

    In analyses by document types, these publications would be counted twice, while citation counts per publication might decrease.

    “in analyses by document types these publications would be counted twice, while citation counts per publication might decrease”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [26] Waltman, 2016, A Review of the Literature on Citation Impact Indicators, J. Informetr. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.11. Comprehensiveness of Information Provided by the DBs' Owners · p. 27

    Information about Scopus is more concentrated and practically all features are clearly explained at the Scopus Access and Use Support Center.

    “the information about Scopus is more concentrated and practically all of the features are clearly explained at the Scopus Access and Use Support Center”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [203] Elsevier (2020, November 25). Scopus: Access and use Support Center-Home. Available online: https://service.elsevier.com/app/home/supporthub/scopus/.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 3.11. Comprehensiveness of Information Provided by the DBs' Owners · p. 27

    Similar extensive information describing indicators implemented in WoS and additional features is provided in the 'Indicators Handbook'.

    “Similar extensive information describing the indicators implemented in WoS and additional features is also provided in the 'Indicators Handbook'”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [207] Clarivate (2020, November 25). InCites Indicators Handbook. Available online: https://incites.help.clarivate.com/Content/Resources/Docs/indicators-handbook-june-2018.pdf.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisenone given
    § Content Coverage Evaluation by Source Lists · p. 29

    These regional indexes may be accessible with the basic WoS CC subscription, but the terms of their accessibility are not explained.

    “which may be accessible with the basic WoS CC subscription, but the terms of their accessibility are not explained”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning
    § Content Coverage Evaluation by Source Lists · p. 28

    This information is not mentioned in any provided JCR description.

    “this information is not mentioned in any provided JCR description”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given
    § Content Coverage Evaluation by Source Lists · p. 28

    There are no explanations provided regarding such cases.

    “there are no explanations provided regarding such cases”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given
    § Content Coverage Evaluation by Source Lists · p. 28

    Clarivate annually provides a full list of journals included in JCR where full and abbreviated journal titles, country/region, and inclusion to SSCI or SCIE indexes are indicated.

    “Clarivate annually provides a full list of journals included in JCR where full and abbreviated journal titles, country/region, and inclusion to SSCI or SCIE indexes are indicated”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § Content Coverage Evaluation by Source Lists · p. 28

    The downloadable lists at MJL website have only recently been improved from unnumbered journal tables in PDF format to CSV format.

    “The downloadable lists that were provided at the MJL website have only recently been improved from unnumbered journal tables in PFD format [47], which were very inconvenient to analyze and compare and, currently, all of the lists are provided in CSV format.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [47] Mongeon, 2016, The Journal Coverage of Web of Science and Scopus: A Comparative Analysis, Scientometrics source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § Content Coverage Evaluation by Source Lists · p. 29

    Although ESCI is a part of WoS CC, ESCI BackFile covering literature back to 2005 is not included in basic subscription and has to be purchased separately.

    “although ESCI is a part of WoS CC, ESCI BackFile covering literature back to 2005 [211] is not included into basic subscription and it has to be purchased separately”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [211] Clarivate (2020, November 25). Emerging Sources Citation Index Backfile (2005–2014). Available online: https://clarivate.com/wp-content/uploads/2018/05/M255-Crv_SAR_ESCI-Individual-infographic-002.pdf.

      Not checked in the free run.

    • cites [55] 2018, Coverage of Highly-Cited Documents in Google Scholar, Web of Science, and Scopus: A Multidisciplinary Comparison, Scientometrics

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § Content Coverage Evaluation by Source Lists · p. 28

    According to Clarivate, books are integrated within other indexes.

    “according to Clarivate, books are integrated within other indexes”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [196] Ruccolo, M. (2020, November 25). Web of Science Core Collection: Web of Science: Summary of Coverage. Available online: https://clarivate.libguides.com/woscc/coverage.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § Content Coverage Evaluation by Source Lists · p. 29

    In Scopus DB, full lists of source titles (including both currently active and discontinued titles) are provided and can be downloaded in Excel format at Scopus Source page and at Elsevier website.

    “in Scopus DB, the full lists of source titles (including both currently active and discontinued titles) are provided and they can be downloaded in an Excel file format at Scopus Source page (requires user login) [115], and at Elsevier website (registration is not required) [146]”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [115] Scopus (2020, November 26). Scopus preview—Sources. Available online: https://www.scopus.com/sources.uri?zone=TopNavBar&origin=SearchAffiliationLookup.

      Not checked in the free run.

    • cites [146] Elsevier (2020, November 25). Content—How Scopus Works. Available online: https://www.elsevier.com/solutions/scopus/how-scopus-works/content.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.12. Search and Online Analysis Capabilities · p. 33

    In WoS, only publication titles and journal names are linked to publication and journal details pages, and for source titles this applies only to journals included in JCR.

    “Meanwhile, in WoS, only publication titles and journal names are linked to the publication's and journal's details pages, respectively. Yet, in the case of source titles, this only applies for journals that are included in JCR [114].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [114] Clarivate (2020, November 25). Journal Citation Reports—Descriptive Document. Available online: https://clarivate.libguides.com/ld.php?content_id=48842741.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.12. Search and Online Analysis Capabilities · p. 32

    When investigating acknowledgements to libraries in WoS indexed papers, WoS FT search retrieved 56% of articles actually having that information, and only if funding was mentioned.

    “For instance, when investigating acknowledgements to libraries in WoS indexed papers, WoS FT search only retrieved 56% of articles actually having that information, and only if funding was mentioned [104].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [104] Hubbard, 2020, Acknowledgment of Libraries in the Journal Literature: An Exploratory Study, J. Data Inf. Sci.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § Content Coverage Evaluation by Source Lists · p. 29

    A brief description of indexes, including coverage extent and time frame, content providers, and main subject fields covered, can be accessed from WoS website, additional sites, and via WoS 'Help' directory.

    “A brief description of indexes, including coverage extent and time frame, content providers, and the main subject fields covered (for most of the indexes), can be accessed from WoS website [193], and also at additional sites [209,210], as well as via the WoS 'Help' directory.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [193] Clarivate (2020, November 25). Web of Science Platform—Web of Science Group. Available online: https://clarivate.com/webofsciencegroup/solutions/webofscience-platform/.

      Not checked in the free run.

    • cites [209] Matthews, T. (2020, December 04). Regional Citation Indexes. Available online: https://clarivate.libguides.com/webofscienceplatform/rci.

      Not checked in the free run.

    • cites [210] Matthews, T. (2020, November 25). Web of Science: Direct Links: Home. Available online: https://clarivate.libguides.com/c.php?g=648493&p=4547878.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.12. Search and Online Analysis Capabilities · p. 32

    In Scopus there are no such limitations as all content of the database is subscribed as a whole product by a single type subscription.

    “Meanwhile, in Scopus there are no such limitations as all content of the DB is subscribed as a whole product by a single type subscription [53].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [53] Huang, 2020, Comparison of Bibliographic Data Sources: Implications for the Robustness of University Rankings, Quant. Sci. Stud.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 3.12. Search and Online Analysis Capabilities · p. 32

    WoS Cited Reference search does not depend on subscription type, because 'Citing Articles' counts reflect citations from all years and all editions of WoS CC, even unsubscribed ones.

    “The only exception is the case of Cited Reference search, which does not depend on the subscription type, since 'Citing Articles' counts reflects citations from all years and all editions of the WoS CC-even of those that are not subscribed [105].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [105] Clarivate (2020, September 21). Web of Science Core Collection—Quick Reference Guide. Available online: https://clarivate.libguides.com/ld.php?content_id=35888196.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § Content Coverage Evaluation by Source Lists · p. 28

    The book search in the MBL may not be limited only to WoS CC indexes, and it is not clear whether content access restrictions defined by subscription terms are relevant to the search.

    “the book search in the MBL may not be limited only to WoS CC indexes, and it is not clear whether content access restrictions that are defined by subscription terms are relevant to the search”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § Content Coverage Evaluation by Source Lists · p. 28

    The additional information, including citation metrics, peer review details, open access information, and more, can be found at journals' profile pages, but it can only be viewed and collected by searching journals manually.

    “The additional information, including citation metrics, peer review details, open access information, and more, can be found at journals' profile pages, but it can only be viewed and collected by searching journals manually”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [27] Badia, 2018, Identifying “Best Bets” for Searching in Chemical Engineering: Comparing Database Content and Performance for Information Retrieval, J. Doc. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4. Citation Impact Indicators Implemented in WoS and Scopus · p. 33

    The amount of journals indexed in the main bibliographic databases has significantly increased and is still growing.

    “The amount of journals indexed in the main bibliographic DBs has significantly increased and is still growing [220].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [220] Gu, 2016, Recent Trends in Academic Journal Growth, Scientometrics source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § Data Export Limitations · p. 33

    Both WoS and Scopus are commercial, their content is protected by restricting access to raw data, and access to their APIs usually requires additional payment.

    “However, since both WoS and Scopus are commercial, their content is protected by restricting access to raw data [42], and access to their APIs usually requires additional payment [32].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [42] Gusenbauer, 2020, Which Academic Search Systems Are Suitable for Systematic Reviews or Meta-Analyses? Evaluating Retrieval Qualities of Google Scholar, PubMed, and 26 Other Resources, Res. Synth. Methods source

      Not checked in the free run.

    • cites [32] Thelwall, 2021, Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: A Multidisciplinary Comparison of Coverage via Citations, Scientometrics source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4. Citation Impact Indicators Implemented in WoS and Scopus · p. 33

    Over twenty different bibliometric indicators are currently available, and new versions or alternative metrics are still being actively developed.

    “Consequently, over twenty different bibliometric indicators are currently available, and new versions or alternative metrics are still being actively developed [26].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [26] Waltman, 2016, A Review of the Literature on Citation Impact Indicators, J. Informetr. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § Data Export Limitations · p. 33

    Another possibility for analyzing larger data sets is to use additional online tools, such as InCites for WoS data and SciVal for Scopus data.

    “Another possibility for analyzing larger data sets is to use additional online tools, such as InCites (for WoS data) and SciVal (for Scopus data) [26,219].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [26] Waltman, 2016, A Review of the Literature on Citation Impact Indicators, J. Informetr. source

      Not checked in the free run.

    • cites [219] Williams, 2016, Sampling Issues in Bibliometric Analysis, J. Informetr. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 4.1. Basic Journal Impact Indicators · p. 34

    JIF was developed by Eugene Garfield more than half a century ago and was originally intended to define the quality of journals to provide high quality indexed content, and for publication or subscription purposes.

    “JIF was developed by Eugene Garfield more than half a century ago [223] and it was originally intended to define the quality of journals to provide high quality indexed content, and for publication or subscription purposes.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [223] Garfield, 1955, Citation Indexes for Science: A New Dimension in Documentatio through Association of Ideas, Science source

      Not checked in the free run.

  • May combine two claims.

    Premisereasoning
    § 4.1. Basic Journal Impact Indicators · p. 34

    JIF values can be deliberately distorted by publishing more reviews, which are more highly cited than regular articles, or by publishing case reports, letters to editors, and book reviews, which are cited but not included in the total paper count in the JIF denominator.

    “JIF values can be deliberately distorted by publishing more reviews, which are more highly cited than regular articles, or by publishing case reports, letters to editors, and book reviews, which are being cited, but not included in the total paper count in the JIF denominator.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning
    § 4.1. Basic Journal Impact Indicators · p. 34

    Probably the most debated feature of JIF is the inconsistency of document types included in its numerator and denominator.

    “Yet, probably the most debated feature of JIF is the inconsistency of document types included in its numerator and denominator.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § Data Export Limitations · p. 33

    Raw data from both databases can be further analyzed by various bibliometric software tools, but this approach, as well as using APIs for data retrieval, requires some kind of programming skills.

    “Raw data from both DBs can be further analyzed by various bibliometric software tools. However, this approach, as well as using APIs for data retrieval, requires some kind of programming skills [9,42].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [9] Cobo, 2020, Software Tools for Conducting Bibliometric Analysis in Science: An up-to-Date Review, Prof. Inf.

      Not checked in the free run.

    • cites [42] Gusenbauer, 2020, Which Academic Search Systems Are Suitable for Systematic Reviews or Meta-Analyses? Evaluating Retrieval Qualities of Google Scholar, PubMed, and 26 Other Resources, Res. Synth. Methods source

      Not checked in the free run.

  • May combine two claims.

    Premisenone given
    § Data Export Limitations · p. 33

    WoS allows creating data sets of up to 50,000 data rows in one session, and for citation analyses up to 10,000.

    “For example, WoS allows for creating data sets of up to 50,000 data rows in one session (for citation analyses-up to 10,000).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 4.1. Basic Journal Impact Indicators · p. 34

    Today, JIF is often misused due to a misinterpretation of its meaning, as it has become a standard tool in virtually all evaluation practices at all levels, usually without proper considerations of the rationale for its use.

    “However, today, JIF is often misused due to a misinterpretation of its meaning, as it has become a standard tool in virtually all evaluation practices at all levels, usually without proper considerations of the rationale for its use [2,44,222,224-227].”
    Quote in the paperNot a substantial claim0 of 7 sources checked
    • cites [2] Agarwal, 2016, Bibliometrics: Tracking Research Impact by Selecting the Appropriate Metrics, Asian J. Androl. source

      Not checked in the free run.

    • cites [44] Wouters, P., Thelwall, M., Kousha, K., Waltman, L., de Rijcke, S., Rushforth, A., and Franssen, T. (2015). The Metric Tide: Literature Review (Supplementary Report I to the Independent Review of the Role of Metrics in Research Assessment and Management), HEFCE.

      Not checked in the free run.

    • cites [222] Setti, 2013, Bibliometric Indicators: Why Do We Need More than One?, IEEE Access source

      Not checked in the free run.

    • cites [224] Glänzel, W., Moed, H.F., Schmoch, U., and Thelwall, M. (2018). The Journal Impact Factor: A Brief History, Critique, and Discussion of Adverse Effects. Springer Handbook of Science and Technology Indicators, Springer International Publishing. source

      Not checked in the free run.

    • cites [225] Ranjan, 2017, Bibliometric Indices of Scientific Journals: Time to Overcome the Obsession and Think beyond the Impact Factor, Med. J. Armed Forces India source

      Not checked in the free run.

    • cites [226] Callaway, 2016, Publishing Elite Turns against Impact Factor, Nature source

      Not checked in the free run.

    • cites [227] Moustafa, 2014, The Disaster of the Impact Factor, Sci. Eng. Ethics source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4. Citation Impact Indicators Implemented in WoS and Scopus · p. 34

    Only a minority of stakeholders employing bibliometric indicators are adequately familiar with their meaning and appropriate use.

    “Yet, only a minority of stakeholders employing bibliometric indicators are adequately familiar with their meaning and appropriate use [2,4,5,7].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [2] Agarwal, 2016, Bibliometrics: Tracking Research Impact by Selecting the Appropriate Metrics, Asian J. Androl. source

      Not checked in the free run.

    • cites [4] Leydesdorff, 2016, Professional and Citizen Bibliometrics: Complementarities and Ambivalences in the Development and Use of Indicators—a State-of-the-Art Report, Scientometrics source

      Not checked in the free run.

    • cites [5] Moed, 2015, Multidimensional Assessment of Scholarly Research Impact, J. Assoc. Inf. Sci. Technol. source

      Not checked in the free run.

    • cites [7] Wouters, 2016, Evaluation Practices and Effects of Indicator Use—A Literature Review, Res. Eval. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning
    § 4.1. Basic Journal Impact Indicators · p. 34

    JCR also provides five-year JIF, which is basically the same JIF with the time window for cited documents extended to five years.

    “Thus, JCR also provides five-year JIF, which is basically the same JIF with the time window for cited documents extended to five years.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 4. Citation Impact Indicators Implemented in WoS and Scopus · p. 34

    Citation metrics are calculated only using data from the particular database, so they depend on the database's coverage width and depth.

    “Especially, due to the fact that citation metrics are calculated only using data from the particular DB, they are dependent on DBs coverage width and depth.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisenone given
    § Data Export Limitations · p. 33

    Both InCites and SciVal are not part of the databases and are only accessible with separate and costly subscription.

    “However, both of these tools are not a part of the DBs and they are only accessible with separate (and costly) subscription.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 4.1. Basic Journal Impact Indicators · p. 34

    In the JIF numerator, citations are counted to all document types, and only specific types of documents, so-called 'citable items' (articles, reviews, and proceedings papers), are included in the denominator.

    “In the numerator, citations are counted to all document types, and only specific types of documents, so-called 'citable items', namely articles, reviews, and proceedings papers are included in the denominator, which represents the total amount of cited papers [45,224].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [45] Walters, 2017, Citation-Based Journal Rankings: Key Questions, Metrics, and Data Sources, IEEE Access source

      Not checked in the free run.

    • cites [224] Glänzel, W., Moed, H.F., Schmoch, U., and Thelwall, M. (2018). The Journal Impact Factor: A Brief History, Critique, and Discussion of Adverse Effects. Springer Handbook of Science and Technology Indicators, Springer International Publishing. source

      Not checked in the free run.

  • May combine two claims.

    Premisenone given
    § Data Export Limitations · p. 33

    WoS analysis data can be extracted as displayed or all data rows up to 200,000 rows, downloadable as a tab-delimited text file.

    “Analysis data can be extracted as displayed or all data rows (up to 200,000 rows), which can be downloaded as a tab-delimited text file.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § Data Export Limitations · p. 33

    WoS offers several different API types, but access to them has to be acquired separately with additional conditions for use.

    “Meanwhile, WoS offers several different API types, but access to them has to be acquired separately with additional conditions for use [28,218].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [28] Carloni, M., Tsenkulovsky, T., and Mangan, R. (2020, August 13). Web of Science Core Collection Descriptive Document. Available online: https://clarivate.libguides.com/ld.php?content_id=45175981.

      Not checked in the free run.

    • cites [218] Birkle, 2020, Web of Science as a Data Source for Research on Scientific and Scholarly Activity, Quant. Sci. Stud. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4.1. Basic Journal Impact Indicators · p. 34

    Although ESCI journals are not included in the JCR and do not receive JIF, citations from ESCI accrue to all articles in the WoS CC and contribute to the JIF numerator.

    “It should be noted that, although ESCI journals are not included in the JCR and do not receive JIF (and other JCR metrics), the citations from ESCI will accrue to all articles in the WoS CC and contribute to the JIF numerator [28].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [28] Carloni, M., Tsenkulovsky, T., and Mangan, R. (2020, August 13). Web of Science Core Collection Descriptive Document. Available online: https://clarivate.libguides.com/ld.php?content_id=45175981.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.1. Basic Journal Impact Indicators · p. 34

    WoS also calculates an Immediacy index, which is basically a one-year JIF.

    “WoS also calculates an Immediacy index, which is basically a one-year JIF [26,45].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [26] Waltman, 2016, A Review of the Literature on Citation Impact Indicators, J. Informetr. source

      Not checked in the free run.

    • cites [45] Walters, 2017, Citation-Based Journal Rankings: Key Questions, Metrics, and Data Sources, IEEE Access source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4.1. Basic Journal Impact Indicators · p. 34

    The JIF calculation method leads to several biases that limit its application and it is continuously discussed and criticized in scientometric literature.

    “However, the JIF calculation method leads to several biases that limit its application and it is continuously discussed and criticized in scientometric literature concerning bibliometric indicators and their application in research evaluation practices [222,224].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [222] Setti, 2013, Bibliometric Indicators: Why Do We Need More than One?, IEEE Access source

      Not checked in the free run.

    • cites [224] Glänzel, W., Moed, H.F., Schmoch, U., and Thelwall, M. (2018). The Journal Impact Factor: A Brief History, Critique, and Discussion of Adverse Effects. Springer Handbook of Science and Technology Indicators, Springer International Publishing. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § Data Export Limitations · p. 33

    Possibilities to export or download data from the web-interfaces for external analyses are usually limited, and the number of data rows available to export in one batch differs between the databases.

    “Possibilities to export or download data from the web-interfaces for external analyses are also usually limited, and the number of data rows available to export in one batch differs between the DBs [9,26].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [9] Cobo, 2020, Software Tools for Conducting Bibliometric Analysis in Science: An up-to-Date Review, Prof. Inf.

      Not checked in the free run.

    • cites [26] Waltman, 2016, A Review of the Literature on Citation Impact Indicators, J. Informetr. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § Data Export Limitations · p. 33

    According to Elsevier, Scopus provides free access to a basic API version of Scopus data for research purposes without a subscription, allowing limited access to basic metadata for most citation records, while full API access is only granted to subscribed users but does not require additional payments.

    “Although, according to Elsevier, Scopus provides free access to a basic API version of Scopus data for research purposes without a requirement of subscription, allowing limited access to basic metadata for most citation records, while full API access is only granted to subscribed users, but it does not require additional payments.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisenone given
    § 4.1. Basic Journal Impact Indicators · p. 34

    The most obvious limitation of JIF is the short two-year citation time window.

    “The most obvious limitation of the JIF is the short two-year citation time window.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 4.1. Basic Journal Impact Indicators · p. 34

    Scopus provides CiteScore, SJR, and SNIP.

    “Meanwhile, Scopus provides CiteScore (CS) metrics, along with other, more advanced indicators-SCImago Journal Rank (SJR) and Source Normalized Impact per Paper (SNIP) [175].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [175] James, 2019, CiteScore Metrics: Creating Journal Metrics from the Scopus Citation Index, Learn. Publ. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § Data Export Limitations · p. 33

    In Scopus, the export limit reaches 2000 data rows, but Scopus also allows exporting up to 20,000 results in 'Citations only' (CSV) format using e-mail services.

    “In Scopus, the export limit reaches 2000 data rows, but Scopus also allows exporting up to 20,000 results (in 'Citations only' (CSV) format) using e-mail services.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.2.1. Normalized Indicators · p. 35

    The citing-side normalization approach raises the question of validity for comparisons, as journals or other document sets are evaluated against unequal benchmarks.

    “On the other hand, it raises the question of validity for comparisons, as journals or other document sets are being evaluated against unequal benchmarks [221].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [221] Mingers, 2017, Evaluating Journal Quality: A Review of Journal Citation Indicators and Ranking in Business and Management, Eur. J. Oper. Res. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4.2.1. Normalized Indicators · p. 35

    Both WoS and Scopus classifications are not perfect and may introduce certain biases in normalizing impact indicators.

    “However, as was discussed earlier, both WoS and Scopus classifications are not perfect and they may introduce certain biases in normalizing impact indicators [221].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [221] Mingers, 2017, Evaluating Journal Quality: A Review of Journal Citation Indicators and Ranking in Business and Management, Eur. J. Oper. Res. source

      Not checked in the free run.

  • May combine two claims.

    Premisereasoning
    § 4.4. Recommendations for the Correct Choice of Journal Impact Indicators · p. 40

    Journals belonging to A&HCI and ESCI are not included in JCR and therefore do not have JIF or other journal impact metrics.

    “since journals belonging to A&HCI and ESCI are not included in JCR, they do not have JIF or other journal impact metrics”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [45] Walters, 2017, Citation-Based Journal Rankings: Key Questions, Metrics, and Data Sources, IEEE Access source

      Not checked in the free run.

    • cites [224] Glänzel, W., Moed, H.F., Schmoch, U., and Thelwall, M. (2018). The Journal Impact Factor: A Brief History, Critique, and Discussion of Adverse Effects. Springer Handbook of Science and Technology Indicators, Springer International Publishing. source

      Not checked in the free run.

  • May combine two claims.

    Premisereasoning
    § 4.4. Recommendations for the Correct Choice of Journal Impact Indicators · p. 40

    JCR does not include books and conference material, with rare exceptions, so these sources cannot be evaluated by WoS journal metrics.

    “as JCR does not include books and conference material (with rare exceptions), these sources also cannot be evaluated by WoS journal metrics.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § 4.1. Basic Journal Impact Indicators · p. 35

    In CS calculation, the publication types included in numerator and denominator coincide and include all document types indexed in Scopus.

    “The second and most important difference is that, in CS calculation, the publication types included in numerator and denominator coincide and include all document types indexed in Scopus.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 4.4. Recommendations for the Correct Choice of Journal Impact Indicators · p. 41

    Scopus calculates CS for only a minor part of indexed conference material sources; separate conferences and those published as special journal issues or part of book series do not receive CS metrics.

    “Scopus only calculates CS for a minor part of indexed conference material sources. For instance, separate conferences and ones published as a special journal issues or part of book series do not receive CS metrics [86].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [86] Meho, 2019, Using Scopus’s CiteScore for Assessing the Quality of Computer Science Conferences, J. Informetr. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 4.1. Basic Journal Impact Indicators · p. 35

    Scopus offers the CiteScore Tracker, representing trends of the upcoming source's CS value, calculated for the current citation year and updated monthly.

    “Scopus also offers the CiteScore Tracker, representing the trends of upcoming source's CS value (similar to WoS Immediacy Index). It is calculated for the current (citation) year, rather than previous, complete years, and it is updated monthly [175].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [175] James, 2019, CiteScore Metrics: Creating Journal Metrics from the Scopus Citation Index, Learn. Publ. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4.1. Basic Journal Impact Indicators · p. 35

    JIF values can be artificially inflated by pressuring authors to reference articles from a particular journal, as journal self-citations are not accounted for in the calculation of classical and five-year JIF.

    “JIF values can also be artificially inflated by other means, for example, by pressuring authors to reference articles from particular journal, as journal self-citations are not accounted for in the calculation of classical and five-year JIF [224,225,230].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [224] Glänzel, W., Moed, H.F., Schmoch, U., and Thelwall, M. (2018). The Journal Impact Factor: A Brief History, Critique, and Discussion of Adverse Effects. Springer Handbook of Science and Technology Indicators, Springer International Publishing. source

      Not checked in the free run.

    • cites [225] Ranjan, 2017, Bibliometric Indices of Scientific Journals: Time to Overcome the Obsession and Think beyond the Impact Factor, Med. J. Armed Forces India source

      Not checked in the free run.

    • cites [230] Alguliyev, R., Aliguliyev, R., and Ismayilova, N. (2017). Impact Factor Penalized by Self-Citations. Appl. Inf. Commun. Technol. AICT 2016 Conf. Proc., 2–5. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.2.1. Normalized Indicators · p. 36

    The citing-side subject normalization approach was determined as being the most objective and appropriate for field normalization, when compared to normalization based on predefined subject classification schemes, especially in the case of multidisciplinary journals.

    “For instance, the citing-side subject normalization approach was determined as being the most objective and appropriate for field normalization, when compared to normalization that is based on predefined subject classification schemes, especially in the case of multidisciplinary journals [180,185,235].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [180] Waltman, 2013, Source Normalized Indicators of Citation Impact: An Overview of Different Approaches and an Empirical Comparison, Scientometrics source

      Not checked in the free run.

    • cites [185] Waltman, 2013, A Systematic Empirical Comparison of Different Approaches for Normalizing Citation Impact Indicators, J. Informetr. source

      Not checked in the free run.

    • cites [235] Waltman, 2013, Some Modifications to the SNIP Journal Impact Indicator, J. Informetr. source

      Not checked in the free run.

  • May combine two claims.

    Premisenone given
    § 4.4. Recommendations for the Correct Choice of Journal Impact Indicators · p. 41

    CS is calculated for sources after the first full year of indexing in Scopus, whereas JIF is calculated only after three consecutive years of journal indexing in JCR.

    “CS is calculated for sources after first full year of indexing in Scopus, in contrast to JIF, which is only calculated after three consecutive years of journal indexing in JCR.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § 4.4. Recommendations for the Correct Choice of Journal Impact Indicators · p. 40

    All WoS indicators are available only through subscription to the JCR tool, except journal Quartiles (Q), which can be viewed at the WoS webinterface without an additional JCR subscription.

    “all of the indicators provided at WoS are only available through the subscription of the JCR tool. The only exception is journal Quartiles (Q), which can be viewed at WoS webinterface without an additional subscription of JCR [196].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [196] Ruccolo, M. (2020, November 25). Web of Science Core Collection: Web of Science: Summary of Coverage. Available online: https://clarivate.libguides.com/woscc/coverage.

      Not checked in the free run.

  • May combine two claims.

    Premisereasoning
    § 4.1. Basic Journal Impact Indicators · p. 35

    In CS, citations are counted from the four-year period, compared to JIF, where citations are only counted from the current JCR year.

    “Citations are also counted from the four-year period, as compared to JIF, where citations are only counted from the current JCR year.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.1. Basic Journal Impact Indicators · p. 35

    The JIF version without self-citations, calculated in the same way as classical JIF but excluding journal self-citations from the numerator, is available in JCR.

    “On the other hand, the JIF version without self-citations, which is calculated in the same way as the classical JIF, but excluding journal self-citations from the numerator, is also available in JCR [45].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [45] Walters, 2017, Citation-Based Journal Rankings: Key Questions, Metrics, and Data Sources, IEEE Access source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 4.4. Recommendations for the Correct Choice of Journal Impact Indicators · p. 41

    An empirical investigation found that many sources without CS value were discontinued or newly included in the DB, but several hard-to-explain cases were also found.

    “An empirical investigation has shown that many of the sources without CS value were discontinued or newly included in the DB. Although, several hard to explain cases were also found [259].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [259] Krauskopf, 2020, Sources without a CiteScore Value: More Clarity Is Required, Scientometrics source

      Not checked in the free run.

  • May combine two claims.

    Premisereasoning
    § 4.1. Basic Journal Impact Indicators · p. 35

    The time window applied in CS is larger than in classical JIF and has been increased from three to four years.

    “Firstly, the time window applied is larger than in classical JIF and, more recently, it has been increased from three to four years.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisereasoning
    § 4.2.1. Normalized Indicators · p. 35

    The main difference of citing-side normalization from field normalization is that the definition of subject field is not based on WoS or any other predefined category classification schemes, but is determined as the set of all papers or journals citing the target set of papers.

    “Thus, the main difference of this approach from field (or cited-side) normalization is that the definition of subject field is not based on the WoS or any other predefined category classification schemes, but it is determined as the set of all papers or journals citing the target set of papers.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning
    § 4.1. Basic Journal Impact Indicators · p. 35

    Because the basic principle of calculating CS is very similar to that of JIF, CS can be considered the Scopus version of JIF.

    “Because the basic principle of calculating CS is very similar to that of JIF, CS can be considered to be the Scopus version of JIF.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisereasoning
    § 4.4. Recommendations for the Correct Choice of Journal Impact Indicators · p. 40

    WoS indicators are calculated only for journals included in JCR, covering about two-thirds of journals indexed in WoS CC (SCIE and SSCI indexes).

    “indicators are only calculated for journals that are included in JCR, covering only about two-thirds of journals indexed in WoS CC (SCIE and SSCI indexes).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.2.1. Normalized Indicators · p. 35

    The underlying idea of citing-side or source normalization is that in high density fields papers would likely have longer reference lists than in low density fields.

    “The underlying idea of citing-side or source normalization approach is that in the high density fields papers would likely have longer reference lists than in the low density fields [233].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [233] Bornmann, 2015, Methods for the Generation of Normalized Citation Impact Scores in Bibliometrics: Which Method Best Reflects the Judgements of Experts?, J. Informetr. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.1. Basic Journal Impact Indicators · p. 35

    In December 2016, Scopus launched the CiteScore impact indicator.

    “In response to aforementioned JIF drawbacks and limitations, in December 2016 Scopus launched CiteScore (CS) impact indicator.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.2.1. Normalized Indicators · p. 36

    Many authors have determined that the citing-side approach may outperform the traditional method of cited-side normalization.

    “Many authors have determined that the citing-side approach may outperform the traditional method of cited-side normalization.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.2.1. Normalized Indicators · p. 36

    These conclusions about the superiority of citing-side normalization have also been challenged.

    “However, these conclusions have also been challenged [236].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [236] Leydesdorff, 2010, Scopus’s Source Normalized Impact per Paper (SNIP) versus a Journal Impact Factor Based on Fractional Counting of Citations, J. Am. Soc. Inf. Sci. Technol. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 4.2.1. Normalized Indicators · p. 35

    Generally, the WoS subject classification scheme is most commonly used for normalization purposes.

    “Generally, the WoS subject classification scheme is most commonly used for normalization purposes [26].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [26] Waltman, 2016, A Review of the Literature on Citation Impact Indicators, J. Informetr. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisecitation
    § 4.4. Recommendations for the Correct Choice of Journal Impact Indicators · p. 41

    When journals are compared by impact indicators, differences in journal size should be taken into account.

    “Even when journals are being compared by the values of impact indicators, the differences in size of the journals should be also taken into account [261].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [261] Antonoyiannakis, 2018, Impact Factors and the Central Limit Theorem: Why Citation Averages Are Scale Dependent, J. Informetr. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning
    § 4.4. Recommendations for the Correct Choice of Journal Impact Indicators · p. 41

    Comparisons of researchers are only meaningful when comparing authors in the same field and similar age or career stage.

    “comparisons of researchers would only be meaningful when comparing authors in the similar field and of the similar age or stage of their careers.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 5. Fundamental Concerns in Bibliometric Practices · p. 42

    Only part of citations in a publication's reference list are dedicated to the main idea of the paper; other sources are frequently mentioned as simple recognition of similar works or as persuasion on scientific ground.

    “only part of citations, as listed in publication's reference list, are dedicated to the main idea of the paper, while other sources are frequently only being mentioned as a simple recognition of other similar works, or as a persuasion on the scientific ground of the study [1,269].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [1] Abramo, 2018, Revisiting the Scientometric Conceptualization of Impact and Its Measurement, J. Informetr. source

      Not checked in the free run.

    • cites [269] Crothers, 2020, Citation Concept Analysis (CCA) of Robert K. Merton’s Book Social Theory and Social Structure: How Often Are Certain Concepts from the Book Cited in Subsequent Publications?, Quant. Sci. Stud.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisenone given
    § 5. Fundamental Concerns in Bibliometric Practices · p. 42

    Publications themselves are not equal in quality.

    “Moreover, publications themselves are not equal in quality.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Premisecitation
    § 4.4. Recommendations for the Correct Choice of Journal Impact Indicators · p. 41

    Use of publication and citation counts and derived indicators is only valid when applied at similar levels of aggregation and in appropriate context.

    “the use of publication and citation counts and, accordingly, the indicators that are derived from them, is only valid when they are applied at the similar levels of aggregation and in the appropriate context [260].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [260] Bornmann, 2017, Measuring Impact in Research Evaluations: A Thorough Discussion of Methods for, Effects of and Problems with Impact Measurements, High. Educ. source

      Not checked in the free run.

  • May combine two claims.

    Premisenone given
    § 5. Fundamental Concerns in Bibliometric Practices · p. 43

    All most popular journal impact indicators are calculated as ratios of citation and publication counts and are based on arithmetic means.

    “all of the most popular journal impact indicators are calculated as ratios of citation and publication counts and they are based on arithmetic means.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisereasoning
    § 5. Fundamental Concerns in Bibliometric Practices · p. 42

    The extent to which factors influence citations varies, and in many cases the effect may be indirect because most factors are interrelated.

    “The extent to which these factors can influence citations varies. Also, in many cases, the effect may be indirect, since most of these factors are interrelated.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Premisenone given
    § 5. Fundamental Concerns in Bibliometric Practices · p. 42

    Research quality is a multidimensional concept and cannot be assessed only by quantitative measures.

    “However, research quality is a multidimensional concept and, thus, cannot be only assessed by quantitative measures.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 5. Fundamental Concerns in Bibliometric Practices · p. 42

    Citations can be negative, raising the question about their impact on the quality of the cited document.

    “citations can also be negative, raising the question about their impact to the quality of the cited document [45].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [45] Walters, 2017, Citation-Based Journal Rankings: Key Questions, Metrics, and Data Sources, IEEE Access source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § 5. Fundamental Concerns in Bibliometric Practices · p. 42

    Citation-based impact indicators tend to create bias among authors when choosing a journal for publishing and when choosing which literature to cite.

    “Thus, citation-based impact indicators tend to create bias amongst authors not only when choosing journal for publishing, but also when choosing which literature to cite [268,273-275].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [268] Wang, 2018, A Quantitative Exploration on Reasons for Citing Articles from the Perspective of Cited Authors, Scientometrics source

      Not checked in the free run.

    • cites [273] Tahamtan, 2018, Core Elements in the Process of Citing Publications: Conceptual Overview of the Literature, J. Informetr. source

      Not checked in the free run.

    • cites [274] Bornmann, 2017, Skewness of Citation Impact Data and Covariates of Citation Distributions: A Large-Scale Empirical Analysis Based on Web of Science Data, J. Informetr. source

      Not checked in the free run.

    • cites [275] Uddin, 2016, The Impact of Author-Selected Keywords on Citation Counts, J. Informetr. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 5. Fundamental Concerns in Bibliometric Practices · p. 42

    Scientific impact of any evaluated entity cannot be determined solely by citations due to different citing behaviors and incompatible citation counts between disciplines.

    “It is often argued that the scientific impact of any evaluated entity cannot be determined solely by citations due to the different citing behaviors and resulting incompatible citation counts between disciplines [1-3,5,260,267].”
    Quote in the paperNot a substantial claim0 of 6 sources checked
    • cites [1] Abramo, 2018, Revisiting the Scientometric Conceptualization of Impact and Its Measurement, J. Informetr. source

      Not checked in the free run.

    • cites [2] Agarwal, 2016, Bibliometrics: Tracking Research Impact by Selecting the Appropriate Metrics, Asian J. Androl. source

      Not checked in the free run.

    • cites [3] Aksnes, D.W., Langfeldt, L., and Wouters, P. (2019). Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories. SAGE Open, 9. source

      Not checked in the free run.

    • cites [5] Moed, 2015, Multidimensional Assessment of Scholarly Research Impact, J. Assoc. Inf. Sci. Technol. source

      Not checked in the free run.

    • cites [260] Bornmann, 2017, Measuring Impact in Research Evaluations: A Thorough Discussion of Methods for, Effects of and Problems with Impact Measurements, High. Educ. source

      Not checked in the free run.

    • cites [267] MacRoberts, 2018, The Mismeasure of Science: Citation Analysis, J. Assoc. Inf. Sci. Technol. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 5. Fundamental Concerns in Bibliometric Practices · p. 42

    Citations may be appropriate for assessing scientific impact of research, but they do not show impact outside the scientific community.

    “Citations may be appropriate in assessing the scientific impact of the research, but they do not show the impact of the research outside the scientific community [1,3,5,6,260,264,265].”
    Quote in the paperNot a substantial claim0 of 7 sources checked
    • cites [1] Abramo, 2018, Revisiting the Scientometric Conceptualization of Impact and Its Measurement, J. Informetr. source

      Not checked in the free run.

    • cites [3] Aksnes, D.W., Langfeldt, L., and Wouters, P. (2019). Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories. SAGE Open, 9. source

      Not checked in the free run.

    • cites [5] Moed, 2015, Multidimensional Assessment of Scholarly Research Impact, J. Assoc. Inf. Sci. Technol. source

      Not checked in the free run.

    • cites [6] Bianco, 2016, Academic Evaluation: Universal Instrument? Tool for Development?, Minerva source

      Not checked in the free run.

    • cites [260] Bornmann, 2017, Measuring Impact in Research Evaluations: A Thorough Discussion of Methods for, Effects of and Problems with Impact Measurements, High. Educ. source

      Not checked in the free run.

    • cites [264] Ravenscroft, J., Liakata, M., Clare, A., and Duma, D. (2017). Measuring Scientific Impact beyond Academia: An Assessment of Existing Impact Metrics and Proposed Improvements. PLoS ONE, 12. source

      Not checked in the free run.

    • cites [265] Reale, 2018, A Review of Literature on Evaluating the Scientific, Social and Political Impact of Social Sciences and Humanities Research, Res. Eval. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 5. Fundamental Concerns in Bibliometric Practices · p. 42

    The simple amount of publications does not reflect scientific impact, because publication counts measure productivity rather than quality.

    “The simple amount of publications does not reflect the scientific impact, since publication counts measure productivity, rather than quality [1,2].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [1] Abramo, 2018, Revisiting the Scientometric Conceptualization of Impact and Its Measurement, J. Informetr. source

      Not checked in the free run.

    • cites [2] Agarwal, 2016, Bibliometrics: Tracking Research Impact by Selecting the Appropriate Metrics, Asian J. Androl. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 5. Fundamental Concerns in Bibliometric Practices · p. 43

    Peer review has its own characteristic biases and limitations, which is one reason peer review has been increasingly replaced with bibliometric measures of impact.

    “peer-reviewing has its own characteristic biases and limitations, which is one of the reasons why, nowadays, peer-review has been increasingly extensively replaced with bibliometric measures of impact [3,44,276].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [3] Aksnes, D.W., Langfeldt, L., and Wouters, P. (2019). Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories. SAGE Open, 9. source

      Not checked in the free run.

    • cites [44] Wouters, P., Thelwall, M., Kousha, K., Waltman, L., de Rijcke, S., Rushforth, A., and Franssen, T. (2015). The Metric Tide: Literature Review (Supplementary Report I to the Independent Review of the Role of Metrics in Research Assessment and Management), HEFCE.

      Not checked in the free run.

    • cites [276] Abramo, 2019, Peer Review versus Bibliometrics: Which Method Better Predicts the Scholarly Impact of Publications?, Scientometrics source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 5. Fundamental Concerns in Bibliometric Practices · p. 43

    Conclusions stating poor agreement between metric and peer review were also obtained.

    “conclusions stating poor agreement between metric and peer-review were also obtained [283].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [283] Baccini, 2016, Do They Agree? Bibliometric Evaluation versus Informed Peer Review in the Italian Research Assessment Exercise, Scientometrics source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Premisereasoning
    § 5. Fundamental Concerns in Bibliometric Practices · p. 43

    Researchers are highly encouraged to publish their study results in a timely manner in the best possible journals.

    “researchers are highly encouraged to publish their study results in a timely manner in the best possible journals.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 5. Fundamental Concerns in Bibliometric Practices · p. 43

    Metrics are favored over peer review because they are regarded as more trustworthy and objective, believed to avoid biases from personal interest and authority relations.

    “(2) they are regarded as more trustworthy and objective, because it is believed that the use of indicators may help to avoid biases that are introduced by personal interest and authority relations;”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [6] Bianco, 2016, Academic Evaluation: Universal Instrument? Tool for Development?, Minerva source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 5. Fundamental Concerns in Bibliometric Practices · p. 44

    Public recommendations such as European Commission frameworks encourage application of more sustainable evaluation methods.

    “public recommendations were released, such as European Commission frameworks [292,293], encouraging an application of more sustainable evaluation methods.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [292] Wilsdon, J., Bar-ilan, J., Frodeman, R., Lex, E., Peters, I., and Wouters, P. (2017). Next-Generation Metrics: Reponsible Metrics and Evaluation for Open Science, European Commission.

      Not checked in the free run.

    • cites [293] Wouters, P., Ràfols, I., Oancea, A., Kamerlin, L.S.C., Holbrook, B.J., and Jacob, M. (2019). Indicator Frameworks for Fostering Open Knowledge Practices in Science and Scholarship, European Commission.

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation
    § 5. Fundamental Concerns in Bibliometric Practices · p. 44

    WoS and Scopus data are questionable for Social Sciences & Humanities because these disciplines are clearly underrepresented in the DBs.

    “The same can be said about the use of WoS and Scopus data in Social Sciences & Humanities, since these disciplines are clearly underrepresented in the DBs [5,44,47,88,91,163,263].”
    Quote in the paperSubstantial claim0 of 7 sources checked
    • cites [5] Moed, 2015, Multidimensional Assessment of Scholarly Research Impact, J. Assoc. Inf. Sci. Technol. source

      Not checked in the free run.

    • cites [44] Wouters, P., Thelwall, M., Kousha, K., Waltman, L., de Rijcke, S., Rushforth, A., and Franssen, T. (2015). The Metric Tide: Literature Review (Supplementary Report I to the Independent Review of the Role of Metrics in Research Assessment and Management), HEFCE.

      Not checked in the free run.

    • cites [47] Mongeon, 2016, The Journal Coverage of Web of Science and Scopus: A Comparative Analysis, Scientometrics source

      Not checked in the free run.

    • cites [88] Ochsner, M., Hug, S.E., and Daniel, H. (2016). Assessment in the Humanities, Springer Nature. source

      Not checked in the free run.

    • cites [91] Chapman, 2019, An Evaluation of Web of Science, Scopus and Google Scholar Citations in Operations Management, Int. J. Logist. Manag. source

      Not checked in the free run.

    • cites [163] Tennant, J.P., Crane, H., Crick, T., Davila, J., Enkhbayar, A., Havemann, J., Kramer, B., Martin, R., Masuzzo, P., and Nobes, A. (2019). Ten Hot Topics around Scholarly Publishing. Publications, 7. source

      Not checked in the free run.

    • cites [263] Desrochers, 2018, Authorship, Citations, Acknowledgments and Visibility in Social Media: Symbolic Capital in the Multifaceted Reward System of Science, Soc. Sci. Inf. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 5. Fundamental Concerns in Bibliometric Practices · p. 44

    In 2015, Nature published 'The Leiden Manifesto for research metrics', which established ten principles of best practice in metrics-based research assessments.

    “In 2015, Nature published 'The Leiden Manifesto for research metrics' [291], which established the main ten principles of the best practice in metrics-based research assessments.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [291] Hicks, 2015, Bibliometrics: The Leiden Manifesto for Research Metrics, Nature source

      Not checked in the free run.

  • May combine two claims.

    Premisenone given
    § 5. Fundamental Concerns in Bibliometric Practices · p. 44

    Citation data and indicators are only calculated from data covered by the particular DB, and no data source covers all relevant documents.

    “citation data and indicators are only calculated from the data covered by the particular DB, and no data source covers all relevant documents.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: No bibliographic data source covers all relevant documents.

  • May combine two claims.

    Interpretationreasoning
    § 5. Fundamental Concerns in Bibliometric Practices · p. 43

    This approach creates a bias towards publishing only in journals with high impact values and decreases recognition of sciences mainly published in national-oriented journals or less popular sources like books, since impact indicators are usually not calculated for these sources.

    “this approach creates a bias towards only publishing in journals with high impact values and decreases the recognition of sciences mainly published in journals of national orientation, or preferring less popular sources for knowledge dissemination, like books, since impact indicators are usually not calculated for these sources.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationnone given
    § 5. Fundamental Concerns in Bibliometric Practices · p. 43

    Increased imprudent reliance on bibliometric indicators and their widespread use in evaluation practice has led to misinterpretations of scientific impact and what measures can properly assess it.

    “the vastly increased imprudent reliance on the use of bibliometric indicators and their widespread use in evaluation practice has led to misinterpretations of the concept and significance of scientific impact itself and what measures can be used to properly assess.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 5. Fundamental Concerns in Bibliometric Practices · p. 44

    DORA was developed in 2012 and became a worldwide initiative strongly recommending against inappropriate use of impact factor in research assessments.

    “San Francisco Declaration on Research Assessment (DORA) was developed in 2012 and became a worldwide initiative strongly recommending against the inappropriate use of impact factor in the research assessments [290].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [290] ASCB (2020, November 25). Read the declaration—DORA. Available online: https://sfdora.org/read/.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 5. Fundamental Concerns in Bibliometric Practices · p. 44

    These studies are often targeted at general public and published in national language, which narrows potential audience and reduces likelihood of citation.

    “These studies are also often targeted at the general public and, therefore, are often published in the national language, which narrows the size of potential audience and, consequently, reduces the likelihood of citation [87,96,260,265,294].”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [87] Grégoire, C., Roberge, G., and Archambault, É. Bibliometrics and Patent Indicators for the Science and Engineering Indicators 2016─Comparison of 2016 Bibliometric Indicators to 2014 Indicators. Science & Engineering Indicators 2016 (SEI 2016), Available online: https://science-metrix.com/?q=en/publications/reports&page=2#/?q=en/publications/reports/bibliometrics-and-patent-indicators-for-the-science-and-engineering-indicator-0.

      Not checked in the free run.

    • cites [96] Mohammadi, 2018, A Review of Scientific Outputs on Spirituality and Depression Indexed in Important Databases, Int. J. Med. Rev. source

      Not checked in the free run.

    • cites [260] Bornmann, 2017, Measuring Impact in Research Evaluations: A Thorough Discussion of Methods for, Effects of and Problems with Impact Measurements, High. Educ. source

      Not checked in the free run.

    • cites [265] Reale, 2018, A Review of Literature on Evaluating the Scientific, Social and Political Impact of Social Sciences and Humanities Research, Res. Eval. source

      Not checked in the free run.

    • cites [294] Bornmann, 2018, The Negative Effects of Citing with a National Orientation in Terms of Recognition: National and International Citations in Natural-Sciences Papers from Germany, the Netherlands, and the UK, J. Informetr. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation
    § 5. Fundamental Concerns in Bibliometric Practices · p. 43

    The 'publish or perish' attitude stimulates productivity of researchers but creates a bias towards quantity, often overlooking quality.

    “this 'publish or perish' attitude, on the one hand, stimulates the productivity of researchers, but, on the other, creates a bias towards quantity, often overlooking quality [2,3,6-8,44,81,163,260,263,284-288].”
    Quote in the paperNot a substantial claim0 of 10 sources checked
    • cites [2] Agarwal, 2016, Bibliometrics: Tracking Research Impact by Selecting the Appropriate Metrics, Asian J. Androl. source

      Not checked in the free run.

    • cites [3] Aksnes, D.W., Langfeldt, L., and Wouters, P. (2019). Citations, Citation Indicators, and Research Quality: An Overview of Basic Concepts and Theories. SAGE Open, 9. source

      Not checked in the free run.

    • cites [6] Bianco, 2016, Academic Evaluation: Universal Instrument? Tool for Development?, Minerva source

      Not checked in the free run.

    • cites [7] Wouters, 2016, Evaluation Practices and Effects of Indicator Use—A Literature Review, Res. Eval. source

      Not checked in the free run.

    • cites [8] Kun, Á. (2018). Publish and Who Should Perish: You or Science?. Publications, 6. source

      Not checked in the free run.

    • cites [44] Wouters, P., Thelwall, M., Kousha, K., Waltman, L., de Rijcke, S., Rushforth, A., and Franssen, T. (2015). The Metric Tide: Literature Review (Supplementary Report I to the Independent Review of the Role of Metrics in Research Assessment and Management), HEFCE.

      Not checked in the free run.

    • cites [81] Chavarro, 2018, To What Extent Is Inclusion in TheWeb of Science an Indicator of Journal ‘Quality’?, Res. Eval. source

      Not checked in the free run.

    • cites [163] Tennant, J.P., Crane, H., Crick, T., Davila, J., Enkhbayar, A., Havemann, J., Kramer, B., Martin, R., Masuzzo, P., and Nobes, A. (2019). Ten Hot Topics around Scholarly Publishing. Publications, 7. source

      Not checked in the free run.

    • cites [260] Bornmann, 2017, Measuring Impact in Research Evaluations: A Thorough Discussion of Methods for, Effects of and Problems with Impact Measurements, High. Educ. source

      Not checked in the free run.

    • cites [263] Desrochers, 2018, Authorship, Citations, Acknowledgments and Visibility in Social Media: Symbolic Capital in the Multifaceted Reward System of Science, Soc. Sci. Inf. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 5. Fundamental Concerns in Bibliometric Practices · p. 43

    Bibliometric indicators were generally shown to be positively correlated with peer-review evaluations.

    “Although bibliometric indicators were generally shown to be positively correlated with peer-review evaluations”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [44] Wouters, P., Thelwall, M., Kousha, K., Waltman, L., de Rijcke, S., Rushforth, A., and Franssen, T. (2015). The Metric Tide: Literature Review (Supplementary Report I to the Independent Review of the Role of Metrics in Research Assessment and Management), HEFCE.

      Not checked in the free run.

    • cites [276] Abramo, 2019, Peer Review versus Bibliometrics: Which Method Better Predicts the Scholarly Impact of Publications?, Scientometrics source

      Not checked in the free run.

    • cites [280] Haddawy, 2016, A Comprehensive Examination of the Relation of Three Citation-Based Journal Metrics to Expert Judgment of Journal Quality, J. Informetr. source

      Not checked in the free run.

    • cites [282] Traag, 2019, Systematic Analysis of Agreement between Metrics and Peer Review in the UK REF, Palgrave Commun. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § 5. Fundamental Concerns in Bibliometric Practices · p. 44

    Sources publishing papers with a national focus attract much fewer citations, are generally considered inferior quality, and are usually not included in main selective bibliographic DBs.

    “Consequently, sources publishing papers with a national focus attract much fewer citations and they are generally considered to be of inferior quality and are usually not included in the main selective bibliographic DBs.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § 5. Fundamental Concerns in Bibliometric Practices · p. 43

    Publication data and citation metrics continue to play the most important role in research assessments worldwide, so indicator-based research assessments remain at the center of global debate about their validity and influence.

    “because publication data and citation metrics continue to play the most important role in research assessments worldwide, indicator-based research assessments also remain at the center of the global debate, not only about their validity, but also about their influence on research activities, scientific output, and its quality [1,5-7,44,260,265,289].”
    Quote in the paperNot a substantial claim0 of 8 sources checked
    • cites [1] Abramo, 2018, Revisiting the Scientometric Conceptualization of Impact and Its Measurement, J. Informetr. source

      Not checked in the free run.

    • cites [5] Moed, 2015, Multidimensional Assessment of Scholarly Research Impact, J. Assoc. Inf. Sci. Technol. source

      Not checked in the free run.

    • cites [6] Bianco, 2016, Academic Evaluation: Universal Instrument? Tool for Development?, Minerva source

      Not checked in the free run.

    • cites [7] Wouters, 2016, Evaluation Practices and Effects of Indicator Use—A Literature Review, Res. Eval. source

      Not checked in the free run.

    • cites [44] Wouters, P., Thelwall, M., Kousha, K., Waltman, L., de Rijcke, S., Rushforth, A., and Franssen, T. (2015). The Metric Tide: Literature Review (Supplementary Report I to the Independent Review of the Role of Metrics in Research Assessment and Management), HEFCE.

      Not checked in the free run.

    • cites [260] Bornmann, 2017, Measuring Impact in Research Evaluations: A Thorough Discussion of Methods for, Effects of and Problems with Impact Measurements, High. Educ. source

      Not checked in the free run.

    • cites [265] Reale, 2018, A Review of Literature on Evaluating the Scientific, Social and Political Impact of Social Sciences and Humanities Research, Res. Eval. source

      Not checked in the free run.

    • cites [289] Moher, D., Naudet, F., Cristea, I.A., Miedema, F., John, P., Ioannidis, A., and Goodman, S.N. (2018). Assessing Scientists for Hiring, Promotion, and Tenure. PLoS Biol., 16. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 5. Fundamental Concerns in Bibliometric Practices · p. 43

    Metrics are favored over peer review because they are perceived as more accessible and do not require involvement of scientists or subject specialists.

    “(3) they are perceived as more accessible and do not require the involvement of scientists or subject specialists [6].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [6] Bianco, 2016, Academic Evaluation: Universal Instrument? Tool for Development?, Minerva source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 5. Fundamental Concerns in Bibliometric Practices · p. 44

    In 2016, ASM declared that JIFs will no longer be posted to ASM journal websites or used in advertising.

    “in 2016, the American Society of Microbiology (ASM) declared that JIFs will no longer be posted to the ASM journal websites or used in advertising [226].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [226] Callaway, 2016, Publishing Elite Turns against Impact Factor, Nature source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 5. Fundamental Concerns in Bibliometric Practices · p. 44

    'The Metric Tide' report took a deeper look at potential uses and limitations of research metrics and indicators.

    “'The Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management' took a deeper look at the potential uses and limitations of research metrics and indicators [262].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [262] Wilsdon, J., Allen, L., Belfiore, E., Campbell, P., Curry, S., Hill, S., Jones, R., Kain, R., Kerridge, S., and Thelwall, M. (2015). Metric Tide: Report of the Independent Review of the Role of Metrics in Research Assessment and Management, HEFCE. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning
    § 5. Fundamental Concerns in Bibliometric Practices · p. 44

    These recommendations do not entirely neglect the value of bibliometric indicators in evaluating research performance, but highlight the importance of their responsible use regarding their original purpose and intended meaning.

    “It should be noted that these recommendations do not entirely neglect the value of bibliometric indicators in evaluating research performance, but they are highlighting the importance of their responsible use regarding their original purpose and intended meaning.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning
    § 6. Discussion · p. 46

    As both WoS and Scopus index data documents, the comprehensiveness and quality of their coverage in the DBs will gain great importance in the upcoming years.

    “Thus, as both WoS and Scopus indexes data documents, the comprehensiveness and quality of their coverage in the DBs will also gain great importance in the upcoming years.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisenone given
    § 6. Discussion · p. 44

    Two components critical for research evaluation are a bibliometric data source and tools for quality interpretation (metrics), both provided by major bibliographic DBs.

    “For this, two components are critical-a bibliometric data source and tools for quality interpretation-metrics. They are both provided by the major bibliographic DBs.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning
    § 7. Conclusions · p. 47

    WoS and Scopus serve as major tools for a variety of tasks: journal and literature selection, personal career tracking, large-scale bibliometric analyses, and research evaluation practices at all levels.

    “Therefore, they serve as the major tools for a variety of tasks: from journal and literature selection or personal career tracking to large-scale bibliometric analyses and research evaluation practices in all possible levels.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Premisenone given
    § 6. Discussion · p. 45

    Coverage depth, particularly regarding citations, is generally better in WoS.

    “Meanwhile, coverage depth, particularly regarding citations, is generally better in WoS.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § 6. Discussion · p. 45

    WoS and Scopus both offer extensive coverage of Natural, Medicine, Health Sciences, Engineering, and Technology disciplines, and thus could be used in research evaluations of these disciplines.

    “WoS and Scopus both offer an extensive coverage of Natural, Medicine, Health Sciences, Engineering, and Technology disciplines and, thus, could be used in research evaluations of these disciplines.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation
    § 6. Discussion · p. 46

    Scopus is not divided into separate indexes; all searches are performed across all indexed content without differences in search parameters available for separate indexes, unlike WoS CC.

    “Scopus is not divided into separate indexes and, thus, all of the searches are performed in the range of all indexed content without differences in the search parameters available for separate indexes, as in WoS CC case [42].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [42] Gusenbauer, 2020, Which Academic Search Systems Are Suitable for Systematic Reviews or Meta-Analyses? Evaluating Retrieval Qualities of Google Scholar, PubMed, and 26 Other Resources, Res. Synth. Methods source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § References · p. 48

    The availability of individual profiles for all authors, institutions, and serial sources, as well as the interrelated interface of Scopus, makes Scopus more convenient for practical use.

    “Secondly, the availability of individual profiles for all authors, institutions, and serial sources, as well as the interrelated interface of DB, makes Scopus more convenient for practical use.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The majority of information in Scopus is linked, allowing easy transition between different information types.
    • Rests on an assumption: Most lists and other data in Scopus can be opened in separate or emerging windows and tabs.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given
    § 6. Discussion · p. 45

    For a long time, WoS was the only comprehensive bibliographic data source available; with the introduction of Scopus, Scopus rapidly became a major competitor to WoS.

    “For a long time, WoS was the only comprehensive bibliographic data source available. However, the situation has changed with an introduction of Scopus, which rapidly became a major competitor to WoS.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisenone given
    § 6. Discussion · p. 45

    WoS provides more detailed source profiles, but profiles are only available for journals, while Scopus creates profiles for all indexed periodical sources.

    “Regarding source profiles, WoS provides more detailed information, but the profiles are only available for journals, while, in Scopus, they are created for all indexed periodical sources.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning
    § 6. Discussion · p. 47

    The lack of detailed explorations of these tools may be one of the reasons leading to relatively low usage while they might be very helpful.

    “On the other hand, the lack of detailed explorations of these tools may be one of the reasons leading to the relatively low usage while they actually might be very helpful.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 6. Discussion · p. 46

    Published data sets can be cited separately and included in research evaluation practices as additional publications.

    “Published data sets can be cited separately and, accordingly, included in research evaluation practices as additional publications [297].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [297] Mongeon, 2017, DataCite as a Novel Bibliometric Source: Coverage, Strengths and Limitations, J. Informetr. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § 6. Discussion · p. 45

    Main biases towards overrepresentation of English language sources, unequal representation of countries, and underrepresentation of SSH literature remain the main limitations of WoS and Scopus.

    “Therefore, the main biases towards the overrepresentation of English language sources, unequal representation of countries, and underrepresentation of SSH literature still remains the main limitations of these data sources.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 6. Discussion · p. 47

    WoS vendor promises improved search speed and access through mobile gadgets, as well as unified user profiles.

    “WoS vendor also promises an improved search speed and access through mobile gadgets, as well as unified user profiles.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [300] Clarivate (2020, December 29). Release Notes—Web of Science Group. Available online: https://clarivate.com/webofsciencegroup/release-notes/wos/.

      Not checked in the free run.

  • May combine two claims.

    Premisenone given
    § 6. Discussion · p. 45

    Scopus provides searchable and comprehensive profiles, along with unique identifiers for all authors, institutions, and periodical sources.

    “Another advantage of Scopus is that it provides searchable and comprehensive profiles, along with unique identifiers for all authors, institutions, and periodical sources.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 6. Discussion · p. 47

    Most of the changes will be made to the layout of the web page, which should make the interface more modern and user-friendly.

    “Most of the changes will be made to the layout of the web page, which should make the interface more modern and user-friendly.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [300] Clarivate (2020, December 29). Release Notes—Web of Science Group. Available online: https://clarivate.com/webofsciencegroup/release-notes/wos/.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § References · p. 48

    Scopus is subscribed as a single DB, without confusion or additional restrictions regarding content accessibility.

    “However, most importantly, Scopus is subscribed as a one single DB, without the confusion or additional restrictions regarding content accessibility.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Premisenone given
    § References · p. 48

    Scopus is more open to society, as it provides free access to author and source information, including metrics.

    “Moreover, Scopus is more open to the society, as it provides free access to author and source information, including metrics.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § 7. Conclusions · p. 47

    WoS and Scopus remain the two major and most comprehensive sources of publication metadata and impact indicators.

    “Although, during the last decade, there was a significant growth of available bibliographic data sources and metrics, Web of Science (WoS) and Scopus databases (DBs) still remain the two major and most comprehensive sources of publication metadata and impact indicators.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation
    § 6. Discussion · p. 46

    Authors are encouraged (and often required) to open not only their published research results, but also research data.

    “Thus, authors are encouraged (and often, required) to open not only their published research results, but also research data [132].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [132] Ignat, T., and Ayris, P. (2020). Built to Last! Embedding Open Science Principles and Practice into European Universities. Insights UKSG J., 33. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Interpretationreasoning
    § References · p. 48

    Academic institutions will be forced to subscribe to WoS and Scopus, or at least one, as long as their metrics remain core elements in research evaluation and career assessment practices.

    “Nevertheless, academic institutions will be forced to subscribe to WoS and Scopus DBs, or at least to one of them, as long as their provided metrics will remain the core elements in research evaluation and career assessment practices.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 6. Discussion · p. 46

    The distributions of errors between WoS and Scopus are very different, with one error being more frequent in one DB, and others in the other DB.

    “Generally, the distributions of errors between WoS and Scopus are very different, with one error being more frequent in one DB, and others, in the other DB [62,66,79,95].”
    Quote in the paperNot a substantial claim0 of 4 sources checked
    • cites [62] Franceschini, 2016, Empirical Analysis and Classification of Database Errors in Scopus and Web of Science, J. Informetr. source

      Not checked in the free run.

    • cites [66] Van Eck, N.J., and Waltman, L. (2017, January 16–20). Accuracy of Citation Data in Web of Science and Scopus. Proceedings of the 16th International Conference on Scientometrics and Informetrics (ISSI 2017), Wuhan, China.

      Not checked in the free run.

    • cites [79] Liu, 2020, Accuracy of Funding Information in Scopus: A Comparative Case Study, Scientometrics source

      Not checked in the free run.

    • cites [95] Rafols, I., Molas-Gallart, J., and Woolley, R. (2016, January 1–7). Data Quality and Consistency in Scopus and Web of Science in Their Indexing of Czech Journals. Proceedings of the 21st International Conference on Science and Technology Indicators (STI 2016), València, Spain.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § References · p. 48

    The suitability of a DB mainly depends on the objectives and application context of the particular task, including required degree of selectivity and level of aggregation.

    “Generally, the suitability of DB mainly depends on the objectives and application context of the particular task, including consideration of the required degree of the selectivity and the level of aggregation.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationcitation
    § 6. Discussion · p. 46

    Differences in error distributions between WoS and Scopus might be at least partially explained by different data uploading and curation mechanisms; many errors are made by authors/publishers and uploaded with metadata, while other mistakes are introduced by DB algorithms.

    “This might be at least partially explained by the different data uploading and curation mechanisms used in the DBs as many errors are being made by the authors and/or publishers, which are uploaded into DBs with publications' metadata, while other mistakes are being introduced by the algorithms employed in the DBs [62].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [62] Franceschini, 2016, Empirical Analysis and Classification of Database Errors in Scopus and Web of Science, J. Informetr. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation
    § 6. Discussion · p. 45

    Author disambiguation systems in both DBs were shown to be more accurate, but split identities and other discrepancies still occur.

    “although author disambiguation systems in both DBs were shown to be more accurate [121], split identities and other discrepancies still occur.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [121] Ioannidis, J.P.A., Baas, J., Klavans, R., and Boyack, K.W. (2019). A Standardized Citation Metrics Author Database Annotated for Scientific Field. PLoS Biol. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S4 (based).

    Interpretationnone given
    § References · p. 48

    Scopus provides wider and more inclusive content coverage than WoS.

    “First, Scopus provides wider and more inclusive content coverage.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.

    Interpretationreasoning
    § 6. Discussion · p. 45

    Scopus seems to provide better and easier access to the data.

    “However, Scopus seems to provide better and easier access to the data.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning
    § References · p. 48

    Nowadays WoS and Scopus encompass so many features and functionalities that it is impossible to draw a general conclusion which DB is better; one DB may be a better choice for one purpose, but less for another.

    “Consequently, nowadays they encompass so many features and functionalities that it is impossible to draw such a general conclusion, since one DB may be a better choice for one purpose, but less for another.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: WoS and Scopus have been extensively compared for more than 15 years; the scientometric community still has not reached the verdict of which one is better.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 6. Discussion · p. 46

    DBs' vendors are promoting user-orientated changes through cooperation with their customers.

    “In addition, DBs' vendors are promoting user-orientated changes through cooperation with their customers.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 6. Discussion · p. 47

    Clarivate announced major changes planned to be implemented in WoS interface in the near future.

    “In particular, in the case of WoS, since, during the preparation of this article, Clarivate announced major changes that are planned to be implemented in WoS interface in the near future [300].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [300] Clarivate (2020, December 29). Release Notes—Web of Science Group. Available online: https://clarivate.com/webofsciencegroup/release-notes/wos/.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 6. Discussion · p. 46

    Elsevier promotes authors to supervise and maintain their Scopus author profile integrity by providing free access to profiles and correction opportunities; encourages users to report discrepancies.

    “Elsevier promotes authors to supervise and maintain their Scopus author profile integrity by providing a free access to the profiles and their correction opportunities. Additionally, Elsevier encourages Scopus users to report any observed discrepancies [171].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [171] Meester, 2016, A Response to “The Museum of Errors/Horrors in Scopus” by Franceschini et Al, J. Informetr. source

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning
    § 6. Discussion · p. 46

    Error rates in DBs are not very high; they should not significantly affect the results of analyses if properly taken into account.

    “Moreover, because the rates of errors are not very high, they should not significantly affect the results of the analyses, if they are properly taken into account.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

    Premisenone given
    § 6. Discussion · p. 46

    The content coverage and quality of DBs are constantly changing and improving, as well as the convenience of their web-interfaces.

    “The content coverage and quality of DBs are constantly changing and improving, as well as the convenience of their web-interfaces.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationreasoning
    § References · p. 48

    Both DBs are constantly being improved due to intense competition and notable transfer of academic activities into digital internet-based environment.

    “On the other hand, both DBs are constantly being improved due to the intense competition and notable transfer of academic activities into digital internet-based environment.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 6. Discussion · p. 45

    Comparisons of WoS and Scopus have become a major discussion theme in scientometrics because DBs can differ in many aspects determining their suitability as reliable data sources.

    “the comparisons of these data sources have become one of the major discussion themes in scientometrics, as DBs can differ in many aspects, determining their suitability as an appropriate and reliable data sources for various purposes”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [296] Archambault, 2009, Comparing Bibliometric Statistics Obtained From TheWeb of Science and Scopus, J. Am. Soc. Inf. Sci. Technol. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

    Interpretationnone given
    § References · p. 48

    WoS may be more suitable for searching and analyzing Open Access resources at the publication level.

    “For instance, it may be more suitable for searching and analyzing Open Access resources at the publication level.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation
    § 6. Discussion · p. 45

    Multiple studies have shown that Scopus offers wider coverage of publications and citations in all major disciplines and document types, as well as better representation of non-English and regional literature.

    “multiple studies have shown that Scopus offers wider coverage, both of publications and of citations, in all major disciplines and document types, as well as better representation of non-English and regional literature.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [44] Wouters, P., Thelwall, M., Kousha, K., Waltman, L., de Rijcke, S., Rushforth, A., and Franssen, T. (2015). The Metric Tide: Literature Review (Supplementary Report I to the Independent Review of the Role of Metrics in Research Assessment and Management), HEFCE.

      Not checked in the free run.

    • cites [87] Grégoire, C., Roberge, G., and Archambault, É. Bibliometrics and Patent Indicators for the Science and Engineering Indicators 2016─Comparison of 2016 Bibliometric Indicators to 2014 Indicators. Science & Engineering Indicators 2016 (SEI 2016), Available online: https://science-metrix.com/?q=en/publications/reports&page=2#/?q=en/publications/reports/bibliometrics-and-patent-indicators-for-the-science-and-engineering-indicator-0.

      Not checked in the free run.

  • May combine two claims.

    Premisenone given
    § 6. Discussion · p. 45

    Both WoS and Scopus offer wide coverage of highest quality journals, along with additional analysis tools for publications and citations.

    “It is evident that both DBs certainly offer a wide coverage of highest quality journals, along with the additional analysis tools for publications and citations.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisereasoning
    § 6. Discussion · p. 45

    Coverage of certain disciplines or subject fields' literature in selective bibliographic DBs should be evaluated with additional precautions because it depends on indexed source and document types and coverage of non-English language publications.

    “the coverage of certain disciplines or subject fields' literature in selective bibliographic DBs should be evaluated with additional precautions, since it is highly dependent on several other aspects of coverage, such as indexed source and document types and coverage of non-English language publications.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Interpretationcitation
    § 6. Discussion · p. 45

    Scopus might be a better choice for tasks in Arts & Humanities and focused on nationally orientated and novel research.

    “Thus, Scopus might be a better choice for performing tasks within the context of Arts & Humanities and focused on more nationally orientated and novel research [44,87].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [44] Wouters, P., Thelwall, M., Kousha, K., Waltman, L., de Rijcke, S., Rushforth, A., and Franssen, T. (2015). The Metric Tide: Literature Review (Supplementary Report I to the Independent Review of the Role of Metrics in Research Assessment and Management), HEFCE.

      Not checked in the free run.

    • cites [87] Grégoire, C., Roberge, G., and Archambault, É. Bibliometrics and Patent Indicators for the Science and Engineering Indicators 2016─Comparison of 2016 Bibliometric Indicators to 2014 Indicators. Science & Engineering Indicators 2016 (SEI 2016), Available online: https://science-metrix.com/?q=en/publications/reports&page=2#/?q=en/publications/reports/bibliometrics-and-patent-indicators-for-the-science-and-engineering-indicator-0.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Interpretationnone given
    § 6. Discussion · p. 47

    The overall coverage of DBs sources can change significantly due to the global shift towards openness in research and publishing practice.

    “Moreover, the overall coverage of DBs sources can also change significantly due to the global shift towards openness in research and publishing practice.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisenone given
    § 6. Discussion · p. 45

    WoS employs disambiguation systems for authors and institutions, but individual identifiers are not assigned.

    “WoS also employs disambiguation systems of authors and institutions, but individual identifiers are not assigned.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 6. Discussion · p. 47

    The final transition to the new WoS interface is planned for 2021.

    “These changes have just begun to be actively implemented, and the final transition to the new interface is planned for 2021.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [300] Clarivate (2020, December 29). Release Notes—Web of Science Group. Available online: https://clarivate.com/webofsciencegroup/release-notes/wos/.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S4 (based).

    Premisenone given
    § 6. Discussion · p. 45

    The extent and comprehensiveness of covered content is the most important characteristic of a DB for obtaining reliable results.

    “The extent and comprehensiveness of covered content is the most important characteristic of DB for obtaining reliable results.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Premisenone given
    § 6. Discussion · p. 44

    Most valuation methods in today's data-driven society are based on quantitative measures; research evaluations are performed as bibliometric analysis of scientific output.

    “In today's technologically savvy and data-driven society, most of the valuation methods are based on quantitative measures. Research evaluations are also no exception and, today, they are performed as bibliometric analysis of scientific output (publications) produced by the evaluated research unit.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Interpretationnone given
    § References · p. 48

    Scopus impact indicators perform equally well and even better than WoS metrics, are less susceptible to manipulation, and are available for all serial sources in all disciplines.

    “Additionally, thirdly, the implemented impact indicators perform equally well and even better than the metrics that were provided by WoS, are less susceptible to manipulation and are available for all serial sources in all disciplines.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning
    § References · p. 48

    New insights in DBs' suitability for particular assessments may suggest changes for evaluation policies.

    “On the other hand, because publishing and evaluation trends, as well as the DBs themselves, are not constant, new insights in DBs' suitability for particular assessments may, in turn, suggest some changes for these policies.”
    Quote in the paperNot a substantial claim
250 of 250

  • The paper relies on this without support.

    The reliability of any results is highly dependent on the data source used.

    “Thereby, because the reliability of any results is highly dependent on the data source used, all of the potential DBs users, including non-bibliometricians, should be able to objectively evaluate and wisely choose between these two DBs and bibliometric indicators provided by them.”

  • The paper relies on this without support.

    There is no single and adequately recent study describing all aspects relevant to the intended audience.

    “Accordingly, to the author's knowledge, there is no single and adequately recent study describing all aspects, including not only DBs' content coverage and provided metrics, but also their practical use, which are relevant not only to research evaluators, governors, or bibliometricians, but also-for the rest (and probably-the most important) part of academic society.”

  • The paper relies on this without support.

    Findings from studies performed in the last five years adequately reflect the current situation of WoS and Scopus.

    “Additionally, due to the aforementioned obsolescence of information, only findings that were described in studies performed in the last five years will be discussed.”

  • The paper relies on this without support.

    The comprehensiveness of patent coverage in WoS and Scopus has not been extensively compared.

    “However, the comprehensiveness of patent coverage in these two DBs, to the author's knowledge, have not been extensively compared.”

  • The paper relies on this without support.

    The majority of studies on funding information coverage have focused only on WoS or compared WoS with other data sources, not with Scopus.

    “However, the majority of studies addressing this matter has only focused on the coverage and accuracy of funding data in WoS or compared WoS with other data sources, but not with Scopus.”

  • The paper relies on this without support.

    The quality of research is currently judged by the quality of the journal in which it was published.

    “since the quality of research is currently judged by the quality of the journal in which it was published.”

  • The authors state this as an assumption.

    Scopus source inclusion may also be affected by particularistic criteria.

    “as both DBs are selective, it can be assumed that Scopus source inclusion may also be affected by particularistic criteria.”

  • The paper relies on this without support.

    The quality of content indexed in bibliographic databases is defined more by the quality of provided metadata than by the quality of indexed sources, especially for bibliometric analyses.

    “the quality of the content that is indexed in the DBs is defined not only by the quality of indexed sources, but even more by a quality of the provided metadata, especially for bibliometric analyses.”

  • The paper relies on this without support.

    There is no reliable way to identify and analyze publications with missing address information in Scopus online.

    “since there is no reliable way to identify and analyze publications with missing address information in Scopus online.”

  • The paper relies on this without support.

    There is no common rule indicating which date should be documented in publication metadata.

    “Yet, there is no common rule indicating which date should be documented in publication metadata.”

  • The paper relies on this without support.

    Predatory journals aim to become indexed in databases in order to create a reputable image and attract more authors.

    “aiming to become indexed in the DBs in order to create a reputable image and attract more authors”

  • The paper relies on this without support.

    WoS and Scopus are the main bibliographic databases in today's academic world.

    “WoSandScopus are the main sources of the most prevailed scientific impact indicators, therefore the comparison of these DBs will not be comprehensive without mentioning the impact metrics provided at these data sources.”

  • The authors state this as an assumption.

    Journals with higher impact indicators publish papers of higher scientific value.

    “based on an assumption that journals with higher impact indicators publish papers of a higher scientific value.”

  • The authors state this as an assumption.

    Journal quality rank reflects the quality of publications of the researcher and overall research.

    “based on the underlying assumption that journal quality rank reflects the quality of publications of the researcher and, accordingly, overall research.”

  • The paper relies on this without support.

    No bibliographic data source covers all relevant documents.

    “no data source covers all relevant documents.”

  • The paper relies on this without support.

    All research evaluation practices inevitably affect the behavior of evaluated parties.

    “all of the research evaluation practices inevitably affect the behavior of evaluated parties.”

  • The paper relies on this without support.

    In both WoS and Scopus, errors of all types are still present.

    “However, judging by the most recent studies, in both DBs, errors of all types are still present.”

  • The paper relies on this without support.

    Clarivate actively engages in personal communication with WoS customers and addresses concerns expressed in their feedback.

    “Meanwhile, Clarivate actively engages in personal communication with WoS customers and addresses concerns that are expressed in their feedback.”

  • The paper relies on this without support.

    Comparisons of OA content coverage in WoS and Scopus have not received significant attention.

    “comparisons of OAcontent coverage in WoS and Scopus have not received significant attention.”

  • The paper relies on this without support.

    Web-interfaces are being constantly changed.

    “However, as was mentioned before, web-interfaces are being constantly changed.”

  • The paper relies on this without support.

    Scopus also constantly improves its web-interface; layout of profile pages has been updated at least several times.

    “The same can be said for Scopus, because this DB also constantly improves its web-interface. For instance, during the last couple of years, the layout of profile pages has been updated at least several times.”

  • The paper relies on this without support.

    Both DBs owning companies offer additional products and tools, such as InCites, Essential Science Indicators, SciVal, Pure.

    “In addition, both DBs owning companies also offer additional products and tools, in order to assist in online data analyses, such as InCites and Essential Science Indicators powered by WoS data, and SciVal and Pure powered by Scopus data.”

  • The paper relies on this without support.

    These products are only available with an additional subscription, so few WoS and Scopus subscribers have access to them.

    “However, these products are only available with an additional subscription, so few WoS and Scopus subscribers have access to them.”

  • The paper relies on this without support.

    WoS and Scopus have been extensively compared for more than 15 years; the scientometric community still has not reached the verdict of which one is better.

    “Despite the fact that WoS and Scopus DBs have been extensively compared for more than 15 years, the scientometric community still have not reached the verdict of 'which one is better'.”

  • The paper relies on this without support.

    The majority of information in Scopus is linked, allowing easy transition between different information types.

    “Secondly, the majority of information is linked and therefore allows easy transition between different information types.”

  • The paper relies on this without support.

    Most lists and other data in Scopus can be opened in separate or emerging windows and tabs.

    “Besides, most of the lists and other data can be opened in separate or emerging windows and tabs.”
26 of 26

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • WoS h-index without self-citations is both available and unavailable

    Major

    Section 4.3 says the option to view an h-index without self-citations is unavailable in WoS. Section 3.12 says WoS's Citation Report presents the h-index both with and without self-citations.

  • Scopus institutional-profile count differs by roughly 1,000-fold

    Minor

    Table 1 lists approximately 70 million Scopus institutional profiles, while the text says there are more than 70,000. These are incompatible counts for the same type of profile.

2 of 2

Cites

Flagged first, then by citations

1 of the 246 references couldn’t be matched to a known paper.

Research works printed but not linked: 29

  1. 2017, Google Scholar as a Source for Scholarly Evaluation: A Bibliographic Review of Database Errors, Rev. española Doc. Científica, 40, e185
  2. Elsevier (2020, August 13). Scopus Content Coverage Guide. Available online: https://www.elsevier.com/solutions/scopus/how-scopus-works/content.
  3. Rafols, I., Molas-Gallart, J., and Woolley, R. (2016, January 1–7). Data Quality and Consistency in Scopus and Web of Science in Their Indexing of Czech Journals. Proceedings of the 21st International Conference on Science and Technology Indicators (STI 2016), València, Spain.
  4. Elsevier (2020, September 21). Scopus Fact Sheet. Available online: https://www.elsevier.com/__data/assets/pdf_file/0017/114533/Scopus_GlobalResearch_Factsheet2019_FINAL_WEB.pdf.
  5. Elsevier (2020, September 21). Scopus Quick Reference Guide. Available online: https://supportcontent.elsevier.com/RightNow%20Next%20Gen/Scopus/Files/Scopus_User_Guide.pdf.
  6. Clarivate (2020, November 25). Journal Citation Reports: Open Access Data Beta. Available online: https://clarivate.libguides.com/ld.php?content_id=54083756.
  7. Clarivate (2020, November 25). Journal Citation Reports—Descriptive Document. Available online: https://clarivate.libguides.com/ld.php?content_id=48842741.
  8. Clarivate (2020, November 25). Web of Science Master Journal List—WoS MJL. Available online: https://mjl.clarivate.com/home.
  9. Elsevier (2020, November 26). What is Scopus Preview?—Scopus: Access and Use Support Center. Available online: https://service.elsevier.com/app/answers/detail/a_id/15534/c/10543/supporthub/scopus/.
  10. Elsevier (2020, December 03). What Can I Do on an Affiliation Details Page?. Available online: https://service.elsevier.com/app/answers/detail/a_id/11264/supporthub/scopus/.
  11. Elsevier (2020, December 03). About the Institution Profile Wizard. Available online: https://service.elsevier.com/app/answers/detail/a_id/25554/supporthub/scopus/related/1/.
  12. Clarivate (2020, November 25). Web of Science Journal Evaluation Process and Selection Criteria. Available online: https://clarivate.com/webofsciencegroup/journal-evaluation-process-and-selection-criteria/.
  13. Clarivate (2020, November 25). Web of Science Core Collection Journal Selection Process. Available online: https://clarivate.com/webofsciencegroup/wp-content/uploads/sites/2/2019/08/WS369553747_Fact-Sheet_Core-Collection_V4_Updated1.pdf.
  14. Clarivate (2020, November 25). Web of Science Core Collection: Journal Evaluation Criteria. Available online: https://clarivate.com/webofsciencegroup/wp-content/uploads/sites/2/2019/08/WS366313095_EditorialExplanation_Factsheet_A4_RGB_V4.pdf.
  15. Clarivate (2020, November 25). Web of Science Journal Citation Reports: Suppression Policy. Available online: https://clarivate.com/webofsciencegroup/wp-content/uploads/sites/2/2020/06/jcr-suppression-policy-2020.pdf.
  16. Clarivate (2020, November 26). Web of Science Conference Proceedings Selection Process. Available online: https://clarivate.com/webofsciencegroup/essays/web-science-conference-proceedings-selection-process/.
  17. Holland, K., Brimblecombe, P., Meester, W., and Steiginga, S. (2020, November 26). The Importance of High-Quality Content: Curation and Re- Evaluation in Scopus. Elsevier. Available online: https://www.elsevier.com/research-intelligence/resource-library/scopus-high-quality-content.
  18. Elsevier (2020, November 26). Scopus—Content—Content Policy and Selection. Available online: https://www.elsevier.com/solutions/scopus/how-scopus-works/content/content-policy-and-selection.
  19. Elsevier (2020, November 25). Content—How Scopus Works. Available online: https://www.elsevier.com/solutions/scopus/how-scopus-works/content.
  20. Clarivate (2020, November 25). Journal Citation Reports-Inforographic. Available online: https://clarivate.com/webofsciencegroup/web-of-science-journal-citation-reports-2020-infographic/.
  21. Elsevier (2020, November 25). About Scopus—Abstract and citation database. Available online: https://www.elsevier.com/solutions/scopus.
  22. Elsevier (2020, November 25). Scopus: Access and use Support Center-Home. Available online: https://service.elsevier.com/app/home/supporthub/scopus/.
  23. Elsevier (2020, November 25). Scopus tutorials—Scopus: Access and Use Support Center. Available online: https://service.elsevier.com/app/answers/detail/a_id/14799/kw/export/supporthub/scopus/related/1/.
  24. Elsevier (2020, November 25). What is the Scopus Affiliation Identifier?. Available online: https://service.elsevier.com/app/answers/detail/a_id/11215/supporthub/scopus/.
  25. Elsevier (2020, December 04). How Do I Email, Print, or Create a Bibliography, or Save Documents to PDF Format?. Available online: https://service.elsevier.com/app/answers/detail/a_id/12009/supporthub/scopus/kw/bibliography/.
  26. Elsevier (2020, November 25). How Do I Export Documents from Scopus?—Scopus: Access and Use Support Center. Available online: https://service.elsevier.com/app/answers/detail/a_id/11234/kw/export/supporthub/scopus/related/1/.
  27. Clarivate (2020, November 25). Journal Citation Reports—Journal Impact Factor (JIF). Available online: https://clarivate.libguides.com/ld.php?content_id=55911188.
  28. Elsevier (2020, December 05). How Are CiteScore Metrics Used in Scopus?. Available online: https://service.elsevier.com/app/answers/detail/a_id/14880/supporthub/scopus/.
  29. Elsevier (2020, October 09). CiteScore Metrics: The Basics. Available online: https://www.elsevier.com/__data/assets/pdf_file/0007/652552/SC_FS_CiteScore-metrics-The-Basics.pdf.

Cited by

Sorted by citations

Related

Authors
Pranckutė, R.
Affiliations
Vilnius Gediminas Technical University (LT)
Corresponding author
Pranckutė, Raminta
ORCID
Pranckutė, Raminta
Venue
Publications
Publisher
MDPI AG
Published
2021-03-12
Volume
9
Issue
1
Pages
12
ISSN
2304-6775 (ISSN-L)
DOI
10.3390/publications9010012
OpenAlex
W3137875885
Semantic Scholar
8e73dc7cde11952a410feac65a52ad99e27090e3
Cited by
2,696
References
246
Citations by year
2026: 753 · 2025: 771 · 2024: 574 · 2023: 373 · 2022: 187 · 2021: 37
FWCI
169.20
Type
journal article
Language
en
Open access
Licence
Locations
  • Publications (publishedVersion, cc-by)
  • RePEc: Research Papers in Economics (submittedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
  • Physical Sciences › Computer Science › Information Systems › Library Science and Information
  • Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
Keywords
Web of Science, Scopus, bibliographic databases, database comparison, database coverage, bibliometric indicators, research evaluation, bibliographic metadata, citation data, database selection

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review