Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World

In the paper

“Accordingly, even the recent studies, which were published during the last five years, were mostly focused on one or few particular aspects, mainly related to content coverage. Meanwhile many other features of the DBs, such as data quality or additional DBs functionalities and their practical use, which may also influence the suitability of the DB for a particular task, were usually only mentioned”

In the paper

“Yet, most of the DBs users do not have the necessary competence to make the most suitable choice and, therefore, need a trusted source of DB's descriptive information [4,25].”

What the sources say

• Not checked yetcites [4] Leydesdorff, 2016, Professional and Citizen Bibliometrics: Complementarities and Ambivalences in the Development and Use of Indicators—a State-of-the-Art Report, Scientometrics source
• Not checked yetcites [25] Ellegaard, 2018, The Application of Bibliometric Analysis: Disciplinary and User Aspects, Scientometrics source

In the paper

“However, while it is not arguable that comprehensiveness of coverage is essential for any task, the quality of data is also very important, especially in performing bibliometric analyses [4,40].”

What the sources say

• Not checked yetcites [4] Leydesdorff, 2016, Professional and Citizen Bibliometrics: Complementarities and Ambivalences in the Development and Use of Indicators—a State-of-the-Art Report, Scientometrics source
• Not checked yetcites [40] Visser, M., Jan Van Eck, N., and Waltman, L. (2020). Large-Scale Comparison of Bibliographic Data Sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. arXiv. source

In the paper

“Therefore, the two main specialized DBs-WoS and Scopus-are still considered to be the most reliable sources of bibliographic data, both for the most analyses and research evaluations, and for daily tasks.”

In the paper

“Moreover, because DBs are continuously expanding both their contents and capabilities, the information provided in the early studies, performed more than several years ago, may have become obsolete and, thus, could be misleading [26].”

What the sources say

• Not checked yetcites [26] Waltman, 2016, A Review of the Literature on Citation Impact Indicators, J. Informetr. source

In the paper

“Although the literature is overwhelmed with various investigations and comparisons of the main bibliographic DBs, most of these studies, even literature reviews, are relatively narrow in context or only focused on a particular topic.”

In the paper

“In these cases, the convenience and performance of the DB's web interface and provided additional functionalities may also greatly influence the preference of the DB [14,23,24].”

What the sources say

• Not checked yetcites [14] Salehi, 2013, A Comparison between Two Main Academic Literature Collections: Web of Science and Scopus Databases, Asian Soc. Sci.
• Not checked yetcites [23] Gusenbauer, 2019, Google Scholar to Overshadow Them All? Comparing the Sizes of 12 Academic Search Engines and Bibliographic Databases, Scientometrics source
• Not checked yetcites [24] Okhovati, 2017, Novice and Experienced Users’ Search Performance and Satisfaction with Web of Science and Scopus, J. Librariansh. Inf. Sci. source

In the paper

“Because WoS and Scopus both include cited references for each publication they cover, another approach frequently used to evaluate the extent of content coverage is to compare the counts of citations retrieved from the DBs based on the fact that DBs only collect citations from indexed documents. Thus, better coverage should result in higher citation counts.”

In the paper

“In all cases, the most accurate evaluation of content coverage and quality of the DBs can only be achieved by performing a thorough large scale analysis, which requires a huge effort, adequate competencies, and is extremely time-consuming [27].”

What the sources say

• Not checked yetcites [27] Badia, 2018, Identifying “Best Bets” for Searching in Chemical Engineering: Comparing Database Content and Performance for Information Retrieval, J. Doc. source

In the paper

“Hence, all relevant information regarding the DBs is usually not available in one place, which leads to the possibility that many important features could be left unaccounted for when evaluating DB's suitability.”

In the paper

“Many of them are also free and relatively new products, such as Microsoft Academic, CrossRef, ResearchGate, OpenCitations, etc. [32,39-41]. Hence, their validity is also still questionable.”

What the sources say

• Not checked yetcites [32] Thelwall, 2021, Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: A Multidisciplinary Comparison of Coverage via Citations, Scientometrics source
• Not checked yetcites [39] Harzing, 2019, Two New Kids on the Block: How Do Crossref and Dimensions Compare with Google Scholar, Microsoft Academic, Scopus and the Web of Science?, Scientometrics source
• Not checked yetcites [40] Visser, M., Jan Van Eck, N., and Waltman, L. (2020). Large-Scale Comparison of Bibliographic Data Sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. arXiv. source
• Not checked yetcites [41] Waltman, 2020, Special Issue on Bibliographic Data Sources, Quant. Sci. Stud. source

In the paper

“However, this way of obtaining information rarely is less time consuming, as the descriptions that are provided by DBs owners are spread across numerous resources, which makes it challenging to gather all relevant information and ensure its accuracy [27].”

What the sources say

• Not checked yetcites [27] Badia, 2018, Identifying “Best Bets” for Searching in Chemical Engineering: Comparing Database Content and Performance for Information Retrieval, J. Doc. source

In the paper

“Meanwhile, other authors discussed more specific deficiencies of the DBs, such as inconsistencies of journal coverage [63], the accuracy of subject classification schemes [64], missing citation information or citation links [40,65,66], incorrect and missing DOI numbers [53,67-69], duplicate entries [30], and inconsistent publication dates [53].”

What the sources say

• Not checked yetcites [63] Krauskopf, 2019, Missing Documents in Scopus: The Case of the Journal Enfermeria Nefrologica, Scientometrics source
• Not checked yetcites [64] Wang, 2016, Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus, J. Informetr. source
• Not checked yetcites [40] Visser, M., Jan Van Eck, N., and Waltman, L. (2020). Large-Scale Comparison of Bibliographic Data Sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. arXiv. source
• Not checked yetcites [65] Olensky, 2016, Evaluation of the Citation Matching Algorithms of CWTS and IFQ in Comparison to the Web of Science, J. Assoc. Inf. Sci. Technol. source
• Not checked yetcites [66] Van Eck, N.J., and Waltman, L. (2017, January 16–20). Accuracy of Citation Data in Web of Science and Scopus. Proceedings of the 16th International Conference on Scientometrics and Informetrics (ISSI 2017), Wuhan, China.

In the paper

“Over the years, Scopus has earned its equal place as a comprehensive bibliographic data source and it has proven itself to be reliable and, in some respects, even better than WoS [10,13].”

What the sources say

• Not checked yetcites [10] Zhu, 2020, A Tale of Two Databases: The Use of Web of Science and Scopus in Academic Papers, Scientometrics source
• Not checked yetcites [13] Harzing, 2016, Google Scholar, Scopus and the Web of Science: A Longitudinal and Cross-Disciplinary Comparison, Scientometrics source

In the paper

“WoSis a multidisciplinary and selective DB that is composed of a variety of specialized indexes, grouped according to the type of indexed content or by theme.”

What the sources say

• Not checked yetcites [28] Carloni, M., Tsenkulovsky, T., and Mangan, R. (2020, August 13). Web of Science Core Collection Descriptive Document. Available online: https://clarivate.libguides.com/ld.php?content_id=45175981.

In the paper

“The main part of WoS platform is Core Collection (WoS CC), which includes six main citation indexes: Science Citation Index Expanded (SCIE); Social Sciences Citation Index (SSCI); Arts & Humanities Citation Index (A&HCI); Conference Proceedings Citation Index (CPCI); Books Citation Index (BKCI); and, established not long ago-Emerging Sources Citation Index (ESCI) [28].”

What the sources say

• Not checked yetcites [28] Carloni, M., Tsenkulovsky, T., and Mangan, R. (2020, August 13). Web of Science Core Collection Descriptive Document. Available online: https://clarivate.libguides.com/ld.php?content_id=45175981.

In the paper

“However, as both WoS and Scopus are commercial and subscription based products, the worldwide recognition and use of these DBs has resulted in their high price, which makes it rarely affordable for an institution to subscribe to both of them [14].”

What the sources say

• Not checked yetcites [14] Salehi, 2013, A Comparison between Two Main Academic Literature Collections: Web of Science and Scopus Databases, Asian Soc. Sci.

In the paper

“Thus, although Scopus also includes content from many specialized databases, such as Embase, Compendex, World Textile Index, Fluidex, Geobase, Biobase, and Medline [30], their content is integrated and equally accessible.”

What the sources say

• Not checked yetcites [30] 2015, A Systematic Analysis of Duplicate Records in Scopus, J. Informetr. source

In the paper

“Scopus is a similar multidisciplinary and selective DB, which was launched by Elsevier in November 2004 [12].”

What the sources say

• Not checked yetcites [12] Baas, 2020, Scopus as a Curated, High-Quality Bibliometric Data Source for Academic Research in Quantitative Science Studies, Quant. Sci. Stud. source

In the paper

“It was demonstrated that both DBs continuously increase their coverage of all source types, showing stable overall content growth [13,53].”

What the sources say

• Not checked yetcites [13] Harzing, 2016, Google Scholar, Scopus and the Web of Science: A Longitudinal and Cross-Disciplinary Comparison, Scientometrics source
• Not checked yetcites [53] Huang, 2020, Comparison of Bibliographic Data Sources: Implications for the Robustness of University Rankings, Quant. Sci. Stud.

In the paper

“Meanwhile, another study has explored the distribution of publishers providing content to WoS and Scopus, and it has shown that the content of BkCI is from a number of publishers, exceeding the book publisher list in Scopus two times [82], but, judging by the numbers of indexed items, Scopus provides better coverage of academic books when compared to BkCI [90].”

What the sources say

• Not checked yetcites [82] Sivertsen, 2017, Scholarly Book Publishing: Its Information Sources for Evaluation in the Social Sciences and Humanities, Res. Eval. source
• Not checked yetcites [90] Kousha, 2018, Can Microsoft Academic Help to Assess the Citation Impact of Academic Books?, J. Informetr. source

In the paper

“For example, it is well known that, in Computer sciences, the research findings tend to be published in conference papers [86,87], while books and text-books are more important sources in social sciences and even more so in humanities [26,46].”

What the sources say

• Not checked yetcites [86] Meho, 2019, Using Scopus’s CiteScore for Assessing the Quality of Computer Science Conferences, J. Informetr. source
• Not checked yetcites [87] Grégoire, C., Roberge, G., and Archambault, É. Bibliometrics and Patent Indicators for the Science and Engineering Indicators 2016─Comparison of 2016 Bibliometric Indicators to 2014 Indicators. Science & Engineering Indicators 2016 (SEI 2016), Available online: https://science-metrix.com/?q=en/publications/reports&page=2#/?q=en/publications/reports/bibliometrics-and-patent-indicators-for-the-science-and-engineering-indicator-0.
• Not checked yetcites [26] Waltman, 2016, A Review of the Literature on Citation Impact Indicators, J. Informetr. source
• Not checked yetcites [46] Mingers, 2015, A Review of Theory and Practice in Scientometrics, Eur. J. Oper. Res. source

In the paper

“Meanwhile, the content coverage of Russian scientific literature was determined to be growing even exponentially, especially in Scopus, although higher growth rates in the coverage of conference material were determined in WoS, when compared to Scopus [52].”

What the sources say

• Not checked yetcites [52] Moed, 2018, Trends in Russian Research Output Indexed in Scopus and Web of Science, Scientometrics source

In the paper

“However, WoS and Scopus are both concentrated on journal indexing.”

In the paper

“Generally, the content indexed in WoS and Scopus was also shown to be highly overlapping, with Scopus indexing a greater amount of unique sources not covered by WoS [44].”

What the sources say

• Not checked yetcites [44] Wouters, P., Thelwall, M., Kousha, K., Waltman, L., de Rijcke, S., Rushforth, A., and Franssen, T. (2015). The Metric Tide: Literature Review (Supplementary Report I to the Independent Review of the Role of Metrics in Research Assessment and Management), HEFCE.

In the paper

“during 2006-2016 period the share of papers in Russian-language journals in Scopus increased from 4.8% to 14.8%, while, in WoS, this percentage declined from 6.5% to 3.0% during the same period [52].”

What the sources say

• Not checked yetcites [52] Moed, 2018, Trends in Russian Research Output Indexed in Scopus and Web of Science, Scientometrics source

In the paper

“These results were confirmed by later comparisons that were performed at the publication level, showing that both DBs offer the widest coverage of Natural, Medicine, Health Sciences, and Technology, while Social Sciences and Humanities (SSH) are underrepresented in both DBs [40,49,53,56].”

What the sources say

• Not checked yetcites [40] Visser, M., Jan Van Eck, N., and Waltman, L. (2020). Large-Scale Comparison of Bibliographic Data Sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. arXiv. source
• Not checked yetcites [49] Thelwall, 2019, Web of Science and Scopus Language Coverage, Scientometrics source
• Not checked yetcites [53] Huang, 2020, Comparison of Bibliographic Data Sources: Implications for the Robustness of University Rankings, Quant. Sci. Stud.
• Not checked yetcites [56] Thelwall, 2018, Google Scholar, Web of Science, and Scopus: A Systematic Comparison of Citations in 252 Subject Categories, J. Informetr. source

In the paper

“In 2011, the Book Citation Index (BkCI) was added to WoS CC [28]. Meanwhile, Scopus has also expanded the coverage of books by executing Book Expansion Project , which was fulfilled during 2013-2015 period, with 210,000 titles being added from selected publishers to Scopus in addition to the existing book series [89].”

What the sources say

• Not checked yetcites [28] Carloni, M., Tsenkulovsky, T., and Mangan, R. (2020, August 13). Web of Science Core Collection Descriptive Document. Available online: https://clarivate.libguides.com/ld.php?content_id=45175981.
• Not checked yetcites [89] Elsevier (2020, August 13). Scopus Content Coverage Guide. Available online: https://www.elsevier.com/solutions/scopus/how-scopus-works/content.

In the paper

“Nevertheless, increased growth rates in both DBs were mostly observed within the period of the last decade, and they were were generally determined to be higher in Scopus, as compared to WoS, most likely as a result of aforementioned content coverage expansion programs executed by Elsevier.”

In the paper

“For instance, a large-scale comparison performed at the journal level has shown that WoS and Scopus are both biased toward Natural Sciences, Engineering, and Biomedical Research, with Scopus offering wider coverage of all investigated broad fields, especially of Biomedical Research.”

What the sources say

• Not checked yetcites [47] Mongeon, 2016, The Journal Coverage of Web of Science and Scopus: A Comparative Analysis, Scientometrics source

In the paper

“The difference in accessible content imposed by WoS subscription terms that differ between institutions, which makes it very difficult to compare the results from different studies, is another important limitation of works investigating WoS DB.”

In the paper

“Scopus was shown to be indexing over ten-times more documents in Chinese than WoS.”

• Also stated in: § 3.3. Coverage of Regional and Non-English Literature · p. 9 · plain

  “When compared to Scopus, WoS indexes more documents in Spanish and Portuguese, also Catalan, Croatian, Malay, Norwegian, and Turkish [49].”

What the sources say

• Not checked yetcites [49] Thelwall, 2019, Web of Science and Scopus Language Coverage, Scientometrics source
• Not checked yetcites [87] Grégoire, C., Roberge, G., and Archambault, É. Bibliometrics and Patent Indicators for the Science and Engineering Indicators 2016─Comparison of 2016 Bibliometric Indicators to 2014 Indicators. Science & Engineering Indicators 2016 (SEI 2016), Available online: https://science-metrix.com/?q=en/publications/reports&page=2#/?q=en/publications/reports/bibliometrics-and-patent-indicators-for-the-science-and-engineering-indicator-0.

In the paper

“Over the years, both Scopus and WoS were repeatedly shown to be biased toward sources that were published in the English language and providing relatively poor coverage of regional literature.”

In the paper

“Although the majority of the aforementioned studies concluded a better coverage of Humanities in Scopus, opposite observations indicating slightly better coverage of this discipline in WoS have also been noted [49,53].”

What the sources say

• Not checked yetcites [49] Thelwall, 2019, Web of Science and Scopus Language Coverage, Scientometrics source
• Not checked yetcites [53] Huang, 2020, Comparison of Bibliographic Data Sources: Implications for the Robustness of University Rankings, Quant. Sci. Stud.

In the paper

“In WoS, conference papers are indexed in a separate Conference Citation Index, which was included in WoS CC in 2011 [28]. In Scopus conferences are indexed together with the rest of content, and not long ago the coverage of conferences in Scopus was expanded by Conference Expansion Program, which ran from 2011 to 2014 [89].”

What the sources say

• Not checked yetcites [28] Carloni, M., Tsenkulovsky, T., and Mangan, R. (2020, August 13). Web of Science Core Collection Descriptive Document. Available online: https://clarivate.libguides.com/ld.php?content_id=45175981.
• Not checked yetcites [89] Elsevier (2020, August 13). Scopus Content Coverage Guide. Available online: https://www.elsevier.com/solutions/scopus/how-scopus-works/content.

In the paper

“Differently than WoS, Scopus indexes trade publications, but do not cover book reviews and meeting abstracts, which are indexed in WoS [28,40,89].”

What the sources say

• Not checked yetcites [28] Carloni, M., Tsenkulovsky, T., and Mangan, R. (2020, August 13). Web of Science Core Collection Descriptive Document. Available online: https://clarivate.libguides.com/ld.php?content_id=45175981.
• Not checked yetcites [40] Visser, M., Jan Van Eck, N., and Waltman, L. (2020). Large-Scale Comparison of Bibliographic Data Sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. arXiv. source
• Not checked yetcites [89] Elsevier (2020, August 13). Scopus Content Coverage Guide. Available online: https://www.elsevier.com/solutions/scopus/how-scopus-works/content.

In the paper

“However, the main criteria in both DBs are the same: the source must be peer-reviewed, published in a timely manner, comply with editorial standards and certain citation thresholds indicating its impact.”

What the sources say

• Not checked yetcites [12] Baas, 2020, Scopus as a Curated, High-Quality Bibliometric Data Source for Academic Research in Quantitative Science Studies, Quant. Sci. Stud. source
• Not checked yetcites [82] Sivertsen, 2017, Scholarly Book Publishing: Its Information Sources for Evaluation in the Social Sciences and Humanities, Res. Eval. source

In the paper

“from the perspective of universalistic criteria, it was observed that complying with the editorial standards does not guarantee the inclusion in WoS, while the quality of journal impact may have a greater influence [81].”

What the sources say

• Not checked yetcites [81] Chavarro, 2018, To What Extent Is Inclusion in TheWeb of Science an Indicator of Journal ‘Quality’?, Res. Eval. source

In the paper

“Meanwhile, other authors have not determined any significant differences in the quality of journals that were indexed in both DBs [58,85].”

What the sources say

• Not checked yetcites [58] 2015, On the Stability of Citation-Based Journal Rankings, J. Informetr. source
• Not checked yetcites [85] Powell, 2017, Coverage and Quality: A Comparison of Web of Science and Scopus Databases for Reporting Faculty Nursing Publication Metrics, Nurs. Outlook source

In the paper

“Although Scopus indexed the highest number of articles in investigated journals, it retrieved the lowest share of FAs (only 7.7%), while, in WoS, the share of FAs was 29.0% [80].”

What the sources say

• Not checked yetcites [80] Kokol, 2018, Discrepancies among Scopus, Web of Science, and PubMed Coverage of Funding Information in Medical Journal Articles, J. Med. Libr. Assoc. source

In the paper

“but this may be attributed to the wider Scopus coverage.”

In the paper

“both of the systems provide adequate levels of precision (0.95 for WoS OE and 0.96 for Scopus AFID)”

What the sources say

• Not checked yetcites [72] Donner, 2020, Comparing Institutional-Level Bibliometric Research Performance Indicator Values Based on Different Affiliation Disambiguation Systems, Quant. Sci. Stud. source

In the paper

“This may suggest that both DBs could be biased against sources published by universities when considering their inclusion.”

In the paper

“Additionally, the linguistic criterion exists, requiring abstracts and titles to be written in English language, and the importance of content's international relevance with references being listed in Roman script is clearly stated [12,82].”

What the sources say

• Not checked yetcites [12] Baas, 2020, Scopus as a Curated, High-Quality Bibliometric Data Source for Academic Research in Quantitative Science Studies, Quant. Sci. Stud. source
• Not checked yetcites [82] Sivertsen, 2017, Scholarly Book Publishing: Its Information Sources for Evaluation in the Social Sciences and Humanities, Res. Eval. source

In the paper

“Sources that are already indexed are being reevaluated annually. After reevaluation, sources may be discontinued (removed) or suppressed. These measures could be applied if the source fails to meet certain indexing criteria, e.g., when journal self-citation patterns drastically increase and exceed the established threshold, the irregularities in publishing are determined or unethical publishing”

What the sources say

• Not checked yetcites [138] Clarivate (2020, November 25). Web of Science Journal Citation Reports: Suppression Policy. Available online: https://clarivate.com/webofsciencegroup/wp-content/uploads/sites/2/2020/06/jcr-suppression-policy-2020.pdf.
• Not checked yetcites [143] Holland, K., Brimblecombe, P., Meester, W., and Steiginga, S. (2020, November 26). The Importance of High-Quality Content: Curation and Re- Evaluation in Scopus. Elsevier. Available online: https://www.elsevier.com/research-intelligence/resource-library/scopus-high-quality-content.

In the paper

“only 20% of investigated institutions (445) were located in WoS OE, which was mainly attributed to the aforementioned fact that WoS OE does not cover non-university research institutes. Meanwhile, in the Scopus AFID system, 85% of institutions were located [72].”

What the sources say

• Not checked yetcites [72] Donner, 2020, Comparing Institutional-Level Bibliometric Research Performance Indicator Values Based on Different Affiliation Disambiguation Systems, Quant. Sci. Stud. source

In the paper

“It was observed that the unique sources that were indexed by Scopus, but not by WoS, had lower citation impact values [57]”

What the sources say

• Not checked yetcites [57] 2018, Tale of Three Databases: The Implication of Coverage Demonstrated for a Sample Query, Front. Res. Metrics Anal. source

In the paper

“Although the overall number of patents indexed in Scopus is smaller than in WoS, they are equally accessible with a subscription as the rest of the content of Scopus DB. That is not the case in WoS, as DWPI is not a part of WoS CC and, therefore, patent information is not accessible through the basic WoS CC subscription [40].”

What the sources say

• Not checked yetcites [40] Visser, M., Jan Van Eck, N., and Waltman, L. (2020). Large-Scale Comparison of Bibliographic Data Sources: Scopus, Web of Science, Dimensions, Crossref, and Microsoft Academic. arXiv. source

In the paper

“the most recent comparative studies between WoS and Scopus have shown that WoS performs much better than Scopus in both coverage and accuracy of FI [79,80].”

What the sources say

• Not checked yetcites [79] Liu, 2020, Accuracy of Funding Information in Scopus: A Comparative Case Study, Scientometrics source
• Not checked yetcites [80] Kokol, 2018, Discrepancies among Scopus, Web of Science, and PubMed Coverage of Funding Information in Medical Journal Articles, J. Med. Libr. Assoc. source

In the paper

“although OA publishing is very beneficial as free access makes scientific researches more visible, transparent, and reproducible, and also shortens the time required for publication to be available online, the unintended, yet possible, OA effects on research and journal quality have become questioned [133].”

What the sources say

• Not checked yetcites [133] 2019, The Effect of Open Access on Research Quality, J. Informetr. source

In the paper

“while, in Scopus, the exact percentage is not declared, but is most certainly higher and likely contributes to a wider Scopus content coverage.”

In the paper

“WoS may appear to be more suitable for large-scale citation analyses, as it covers citations up to 1900, while, in Scopus, citation coverage was only extended to 1970 [89].”

What the sources say

• Not checked yetcites [89] Elsevier (2020, August 13). Scopus Content Coverage Guide. Available online: https://www.elsevier.com/solutions/scopus/how-scopus-works/content.

In the paper

“in some cases (mostly in Scopus), the removal of a journal from the DB may result in the entire removal of previously indexed papers from this journal, while, in other cases (mostly on WoS), previously indexed papers are not necessarily removed [62].”

What the sources say

• Not checked yetcites [62] Franceschini, 2016, Empirical Analysis and Classification of Database Errors in Scopus and Web of Science, J. Informetr. source

In the paper

“all of the discrepancies in the publication metadata, source information, or other inconsistencies occurring in the DBs not only complicate analysis and other tasks, but can also negatively affect the accuracy and reliability of the obtained results [4,61,95].”

What the sources say

• Not checked yetcites [4] Leydesdorff, 2016, Professional and Citizen Bibliometrics: Complementarities and Ambivalences in the Development and Use of Indicators—a State-of-the-Art Report, Scientometrics source
• Not checked yetcites [61] Franceschini, 2015, Influence of Omitted Citations on the Bibliometric Statistics of the Major Manufacturing Journals, Scientometrics source
• Not checked yetcites [95] Rafols, I., Molas-Gallart, J., and Woolley, R. (2016, January 1–7). Data Quality and Consistency in Scopus and Web of Science in Their Indexing of Czech Journals. Proceedings of the 21st International Conference on Science and Technology Indicators (STI 2016), València, Spain.

In the paper

“although the wider inclusion of journals into Scopus is also associated with the higher number of journals being removed, the overall proportion of 'short-lived' journals that have been included and excluded from coverage in a relatively short period appears to be higher in WoS [58].”

What the sources say

• Not checked yetcites [58] 2015, On the Stability of Citation-Based Journal Rankings, J. Informetr. source

In the paper

“Generally, a high quality of the journal is perceived as its inclusion in WoS and Scopus, because these DBs allegedly only index the highest quality sources carefully selected according to strict selection procedures [134].”

What the sources say

• Not checked yetcites [134] Dorsch, I., Askeridis, J.M., and Stock, W.G. (2018). Truebounded, Overbounded, or Underbounded? Scientists’ Personal Publication Lists versus Lists Generated through Bibliographic Information Services. Publications, 6. source

In the paper

“OAjournals are only published in electronic format shortening the time delay from submission to publishing.”

In the paper

“these artificial journals have evolved and they even manage to pass the selection process of the major bibliographic, as well as OA, DBs [145,150,157,158,166]”

What the sources say

• Not checked yetcites [145] Cortegiani, 2020, Inflated Citations and Metrics of Journals Discontinued from Scopus for Publication Concerns: The GhoS(t)Copus Project, F1000Research
• Not checked yetcites [150] Shen, 2015, “Predatory” Open Access: A Longitudinal Study of Article Volumes and Market Characteristics, BMC Med. source
• Not checked yetcites [157] Fernandez, 2019, Predatory Publishing or a Lack of Peer Review Transparency?—A Contemporary Analysis of Indexed Open and Non-Open Access Articles in Paediatric Urology, J. Pediatr. Urol. source
• Not checked yetcites [158] Demir, 2018, Predatory Journals: Who Publishes in Them and Why?, J. Informetr. source
• Not checked yetcites [166] Björk, B.C., Kanto-Karvonen, S., and Harviainen, J.T. (2020). How Frequently Are Articles in Predatory Open Access Journals Cited. Publications, 8. source

In the paper

“Thus, the validity of measuring the quality of OA journals by citations and impact indicators may seem questionable [128].”

What the sources say

• Not checked yetcites [128] Wei, 2020, Research on Impact Evaluation of Open Access Journals, Scientometrics source

In the paper

“Indeed, many studies have demonstrated that OA content attracts more citations [129,133,153-155], although others did not observe significant OA advantage in citation impact [156,157].”

What the sources say

• Not checked yetcites [129] Asai, 2020, The Effect of Collaboration with Large Publishers on the Internationality and Influence of Open Access Journals for Research Institutions, Scientometrics source
• Not checked yetcites [133] 2019, The Effect of Open Access on Research Quality, J. Informetr. source
• Not checked yetcites [153] Copiello, 2019, The Open Access Citation Premium May Depend on the Openness and Inclusiveness of the Indexing Database, but the Relationship Is Controversial Because It Is Ambiguous Where the Open Access Boundary Lies, Scientometrics source
• Not checked yetcites [154] Piwowar, H., Priem, J., Larivière, V., Alperin, J.P., Matthias, L., Norlander, B., Farley, A., West, J., and Haustein, S. (2018). The State of OA: A Large-Scale Analysis of the Prevalence and Impact of Open Access Articles. PeerJ, 6. source
• Not checked yetcites [155] Wang, 2015, The Open Access Advantage Considering Citation, Article Usage and Social Media Attention, Scientometrics source
• Not checked yetcites [156] Holmberg, 2020, Do Articles in Open Access Journals Have More Frequent Altmetric Activity than Articles in Subscription - Based Journals? An Investigation of the Research Output of Finnish Universities, Scientometrics source
• Not checked yetcites [157] Fernandez, 2019, Predatory Publishing or a Lack of Peer Review Transparency?—A Contemporary Analysis of Indexed Open and Non-Open Access Articles in Paediatric Urology, J. Pediatr. Urol. source

In the paper

“in January 2018, Elsevier discontinued indexing of 424 journals in Scopus, indicating 'publication concerns' and, less often, 'metrics' as the reason for discontinuation.”

In the paper

“In 2012, Jeffrey Beall composed and published a list of nearly 50 criteria for identifying predatory publishing, and continuously updates these criteria and an index of publishers as well as individual journals fulfilling such criteria [162].”

What the sources say

• Not checked yetcites [162] Beall, J. (2020, November 25). Criteria for Determining Predatory Publishers. Available online: https://beallslist.net/wp-content/uploads/2019/12/criteria-2015.pdf.

In the paper

“it can be argued that the OA citation advantage may be more related to the free access to content, and less to the actual quality of published research [153]”

What the sources say

• Not checked yetcites [153] Copiello, 2019, The Open Access Citation Premium May Depend on the Openness and Inclusiveness of the Indexing Database, but the Relationship Is Controversial Because It Is Ambiguous Where the Open Access Boundary Lies, Scientometrics source

In the paper

“since the main consequence of all errors in publication metadata is missing or incorrect links between citing and cited papers in the DB, which leads to missing citations [61,62,65,66].”

What the sources say

• Not checked yetcites [61] Franceschini, 2015, Influence of Omitted Citations on the Bibliometric Statistics of the Major Manufacturing Journals, Scientometrics source
• Not checked yetcites [62] Franceschini, 2016, Empirical Analysis and Classification of Database Errors in Scopus and Web of Science, J. Informetr. source
• Not checked yetcites [65] Olensky, 2016, Evaluation of the Citation Matching Algorithms of CWTS and IFQ in Comparison to the Web of Science, J. Assoc. Inf. Sci. Technol. source
• Not checked yetcites [66] Van Eck, N.J., and Waltman, L. (2017, January 16–20). Accuracy of Citation Data in Web of Science and Scopus. Proceedings of the 16th International Conference on Scientometrics and Informetrics (ISSI 2017), Wuhan, China.

In the paper

“However, there is no common definition allowing for clearly distinguishing between predatory and credible OA publishing [164].”

What the sources say

• Not checked yetcites [164] Tsigaris, 2019, Predatory and Exploitative Behaviour in Academic Publishing: An Assessment, J. Acad. Librariansh. source

In the paper

“For instance, the APC charges of predatory journals are usually much lower when compared with the APCs applied by credible OA journals. Additionally, often authors are requested to pay APC only after the acceptance of their manuscript. Other features include the absence or a questionable location and composition of journal editorial boards, as their members are often lacking academic competencies.”

What the sources say

• Not checked yetcites [131] Tennant, 2016, The Academic, Economic and Societal Impacts of Open Access: An Evidence-Based Review, F1000Research source
• Not checked yetcites [150] Shen, 2015, “Predatory” Open Access: A Longitudinal Study of Article Volumes and Market Characteristics, BMC Med. source
• Not checked yetcites [157] Fernandez, 2019, Predatory Publishing or a Lack of Peer Review Transparency?—A Contemporary Analysis of Indexed Open and Non-Open Access Articles in Paediatric Urology, J. Pediatr. Urol. source
• Not checked yetcites [158] Demir, 2018, Predatory Journals: Who Publishes in Them and Why?, J. Informetr. source

In the paper

“Today, there are many publishers and journals that exploit OA model and its attractiveness for authors in order to collect APCs, but they are highly questionable regarding peer-review practices and overall trustworthiness.”

In the paper

“Although these criteria are often considered to be controversial [164], it was shown that predatory journals usually fulfilled several of the criteria outlined by Beall [150,165].”

What the sources say

• Not checked yetcites [164] Tsigaris, 2019, Predatory and Exploitative Behaviour in Academic Publishing: An Assessment, J. Acad. Librariansh. source
• Not checked yetcites [150] Shen, 2015, “Predatory” Open Access: A Longitudinal Study of Article Volumes and Market Characteristics, BMC Med. source
• Not checked yetcites [165] Manca, 2017, The Surge of Predatory Open-Access in Neurosciences and Neurology, Neuroscience source

In the paper

“Predatory journals usually exploit names and other details, even webpages of credible journals.”

In the paper

“Errors that occur in the DBs may be divided into two main groups, depending on their origin: pre-existing errors, which are made by authors/publishers/editors in preparation and publication of the manuscripts; and, database errors, which are introduced by the DB due to inaccuracies occurring in data uploading processes, such as transcription errors or omitted data.”

In the paper

“A recent large scale analysis has shown that, although these mistakes are present in both DBs, but, in Scopus, frequencies are lower for almost all error types in author names, except for the cases where part of the first name is being mistakenly assigned to the last name, which appeared in fewer than 1% of Scopus entries, while, in WoS, were not detected at all.”

What the sources say

• Not checked yetcites [71] Demetrescu, 2018, Accuracy of Author Names in Bibliographic Data Sources: An Italian Case Study, Scientometrics source

In the paper

“various other mistakes in WoS FI were determined, such as inconsistent registering of co-funded grants or multiple grants being assigned to the same funder”

What the sources say

• Not checked yetcites [78] Morillo, 2017, Funding Acknowledgments in the Web of Science: Completeness and Accuracy of Collected Data, Scientometrics source

In the paper

“in both DBs in almost all last names (approx. 95%), containing diacritics (only approx. 1% of investigated names contained this feature), diacritics were omitted, but none of the diacritics were incorrectly imported [71].”

What the sources say

• Not checked yetcites [71] Demetrescu, 2018, Accuracy of Author Names in Bibliographic Data Sources: An Italian Case Study, Scientometrics source

In the paper

“the situation of journals showing strong relations to the categories that they are not assigned to was less pronounced in both DBs, and the difference between WoS and Scopus was far less noticeable”

What the sources say

• Not checked yetcites [64] Wang, 2016, Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus, J. Informetr. source

In the paper

“classification schemes applied in both WoS and Scopus lack documentation adequately describing the methodology used to construct them, thus making their use for bibliometric analyses problematic”

What the sources say

• Not checked yetcites [64] Wang, 2016, Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus, J. Informetr. source

In the paper

“Scopus uses classification of two levels. All of the sources are divided into four major disciplines: Life Sciences, Physical Sciences, Health Sciences, and Social Sciences and Humanities. These disciplines are subdivided into 27 categories, which are further broken down into more than 300 subject fields, based on All Science Journal Classification (ASJC)”

What the sources say

• Not checked yetcites [64] Wang, 2016, Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus, J. Informetr. source

In the paper

“for the remaining publications with missing address information in WoS, the information was also not declared in the publications themselves, which was probably due to the different editorial policies of journals [73].”

What the sources say

• Not checked yetcites [73] Liu, 2018, Missing Author Address Information in Web of Science—An Explorative Study, J. Informetr. source

In the paper

“although, in both DBs, incomplete and mistyped last names were quite rare (rarely exceeding 1%), approximately two-times more of these mistakes were identified in WoS when compared to Scopus”

What the sources say

• Not checked yetcites [71] Demetrescu, 2018, Accuracy of Author Names in Bibliographic Data Sources: An Italian Case Study, Scientometrics source

In the paper

“Spelling mistakes mostly occur in the publication title, author names, and institutional affiliations.”

In the paper

“Not all of the mistakes in author names are being caused by the DBs, since the author himself/herself can introduce spelling mistakes or may decide to use different, yet valid, name variants in different publications.”

In the paper

“The study showed that more than one-fifth of the publications in WoS lacked address information, while full-text analyses have revealed that about 40% of the articles actually had at least a partial address information listed, and a part of investigated publications had full address information, but, for some reason, it was not indexed by WoS.”

What the sources say

• Not checked yetcites [73] Liu, 2018, Missing Author Address Information in Web of Science—An Explorative Study, J. Informetr. source

In the paper

“a given funder was almost always included in the FO subfield, while specific grant information appeared in only about half of the entries”

What the sources say

• Not checked yetcites [78] Morillo, 2017, Funding Acknowledgments in the Web of Science: Completeness and Accuracy of Collected Data, Scientometrics source

In the paper

“the reliability of the results that were obtained from different DBs may be affected not only by the incompatible classification schemes, but also by a questionable assignation of indexed sources to the subject fields within the DBs”

What the sources say

• Not checked yetcites [64] Wang, 2016, Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus, J. Informetr. source

In the paper

“This becomes problematic when data from both sources are used or compared, since the large-scale analyses usually involve the aggregation of data from multiple disciplines and subject fields highly differing in publication and citation practices, which, in turn, requires classifying the data under a common classification scheme.”

In the paper

“Although authors have determined that address information may also be missing in Scopus, the extent of the issue could not be assessed reliably [73].”

What the sources say

• Not checked yetcites [73] Liu, 2018, Missing Author Address Information in Web of Science—An Explorative Study, J. Informetr. source

In the paper

“Additionally, the first and last names might be misplaced due to the editorial style of the journal”

In the paper

“both WoS and Scopus classification schemes are generally regarded as being far from perfect, although their empirical comparisons are very sparse”

What the sources say

• Not checked yetcites [64] Wang, 2016, Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus, J. Informetr. source

In the paper

“Clarivate and Elsevier both provide extensive information describing WoS and Scopus DBs, including not only content coverage, but also all additional tools, features, and indexing policies of DBs.”

In the paper

“Clarivate also provides various fact sheets and reference guides that are available for downloading from LibGuides (and other) sites”

What the sources say

• Not checked yetcites [195] Clarivate (2020, November 26). Web of Science Core Collection: Quick Reference Cards (PDF). Available online: https://clarivate.libguides.com/woscc/guides.

In the paper

“An accurate evaluation of WoS CC content coverage is highly challenging, as WoS is composed of a variety of specialized indexes and the composition of WoS CC can be modulated by different subscription terms”

What the sources say

• Not checked yetcites [15] Dallas, 2018, Variable Bibliographic Database Access Could Limit Reproducibility, Bioscience source
• Not checked yetcites [42] Gusenbauer, 2020, Which Academic Search Systems Are Suitable for Systematic Reviews or Meta-Analyses? Evaluating Retrieval Qualities of Google Scholar, PubMed, and 26 Other Resources, Res. Synth. Methods source

In the paper

“journals in Scopus are usually assigned with a higher number of categories, and there are significantly more 'multidisciplinary' categories in Scopus classification when compared to WoS”

What the sources say

• Not checked yetcites [58] 2015, On the Stability of Citation-Based Journal Rankings, J. Informetr. source
• Not checked yetcites [64] Wang, 2016, Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus, J. Informetr. source

In the paper

“In both DBs, journals are often being assigned to multidisciplinary categories or with more than one subject field, but these practices were much more frequent in Scopus than in WoS.”

What the sources say

• Not checked yetcites [58] 2015, On the Stability of Citation-Based Journal Rankings, J. Informetr. source
• Not checked yetcites [64] Wang, 2016, Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus, J. Informetr. source

In the paper

“Both of the documents are available at LibGuides site”

What the sources say

• Not checked yetcites [191] Matthews, T. (2020, November 25). Web of Science Group: Welcome to our Training Portal. Available online: https://clarivate.libguides.com/home/welcome.

In the paper

“descriptive information is scattered across a multitude of information resources, which makes it very difficult and time-consuming to gather all of the relevant facts and characteristics that are necessary to create an overview of DBs and assess their suitability for a particular task”

In the paper

“Some cases of such double assignment were also noticed in Scopus, but the occurrence of this error was estimated to be very low”

What the sources say

• Not checked yetcites [62] Franceschini, 2016, Empirical Analysis and Classification of Database Errors in Scopus and Web of Science, J. Informetr. source

In the paper

“The most recent version currently available for downloading at Elsevier webpage [146] and at the Scopus web-interface was last updated in January 2020.”

What the sources say

• Not checked yetcites [146] Elsevier (2020, November 25). Content—How Scopus Works. Available online: https://www.elsevier.com/solutions/scopus/how-scopus-works/content.

In the paper

“the study only investigated journals with more than 100 citations, which might result in the inaccurate evaluation of small subject categories covering less popular and more difficult to assign subject fields”

In the paper

“since both WoS and Scopus are commercial DBs and major competitors, their vendors are mostly interested in promoting their product”

In the paper

“Although Elsevier provides fewer descriptive documents, but more detailed descriptions of newly implemented features, including visual examples, are also provided in Scopus Blog”

What the sources say

• Not checked yetcites [204] Elsevier (2020, November 25). Product Releases | Elsevier Scopus Blog. Available online: https://blog.scopus.com/product-releases.

In the paper

“one can argue that the higher numbers of inaccuracies determined in Scopus might be related to the higher numbers of Scopus journals included in the study due to the wider Scopus coverage, but the differences in DBs' coverage were not accounted for”

What the sources say

• Not checked yetcites [64] Wang, 2016, Large-Scale Analysis of the Accuracy of the Journal Classification Systems of Web of Science and Scopus, J. Informetr. source

In the paper

“both DBs' owners also offer visual training materials and organize webinars, explaining the most effective use of their products”

What the sources say

• Not checked yetcites [205] Elsevier (2020, November 25). Scopus tutorials—Scopus: Access and Use Support Center. Available online: https://service.elsevier.com/app/answers/detail/a_id/14799/kw/export/supporthub/scopus/related/1/.
• Not checked yetcites [206] Clarivate (2020, November 25). Web of Science-Videos. Available online: https://videos.webofsciencegroup.com/.

In the paper

“the last version available was updated on July 2018”

What the sources say

• Not checked yetcites [28] Carloni, M., Tsenkulovsky, T., and Mangan, R. (2020, August 13). Web of Science Core Collection Descriptive Document. Available online: https://clarivate.libguides.com/ld.php?content_id=45175981.

In the paper

“These inconsistencies may occur due to the fact that DBs differ by the document types that they index and by the classification of the documents, therefore some documents from the same source are present in one DB, but not in the other”

What the sources say

• Not checked yetcites [59] Hug, 2017, The Coverage of Microsoft Academic: Analyzing the Publication Output of a University, Scientometrics source
• Not checked yetcites [95] Rafols, I., Molas-Gallart, J., and Woolley, R. (2016, January 1–7). Data Quality and Consistency in Scopus and Web of Science in Their Indexing of Czech Journals. Proceedings of the 21st International Conference on Science and Technology Indicators (STI 2016), València, Spain.

In the paper

“General information regarding Scopus is available at Elsevier website”

What the sources say

• Not checked yetcites [202] Elsevier (2020, November 25). About Scopus—Abstract and citation database. Available online: https://www.elsevier.com/solutions/scopus.

In the paper

“while generally high correlation between journal ranks between WoS and Scopus were observed to be high, considerable variations in rank for individual journals were also noticed”

What the sources say

• Not checked yetcites [48] Leydesdorff, 2016, Aggregated Journal–Journal Citation Relations in Scopus and Web of Science Matched and Compared in Terms of Networks, Maps, and Interactive Overlays, J. Assoc. Inf. Sci. Technol. source
• Not checked yetcites [58] 2015, On the Stability of Citation-Based Journal Rankings, J. Informetr. source
• Not checked yetcites [181] Campanario, 2018, Are Leaders Really Leading? Journals That Are First in Web of Science Subject Categories in the Context of Their Groups, Scientometrics source

In the paper

“in analyses by document types these publications would be counted twice, while citation counts per publication might decrease”

What the sources say

• Not checked yetcites [26] Waltman, 2016, A Review of the Literature on Citation Impact Indicators, J. Informetr. source

In the paper

“the information about Scopus is more concentrated and practically all of the features are clearly explained at the Scopus Access and Use Support Center”

What the sources say

• Not checked yetcites [203] Elsevier (2020, November 25). Scopus: Access and use Support Center-Home. Available online: https://service.elsevier.com/app/home/supporthub/scopus/.

In the paper

“Similar extensive information describing the indicators implemented in WoS and additional features is also provided in the 'Indicators Handbook'”

What the sources say

• Not checked yetcites [207] Clarivate (2020, November 25). InCites Indicators Handbook. Available online: https://incites.help.clarivate.com/Content/Resources/Docs/indicators-handbook-june-2018.pdf.

In the paper

“which may be accessible with the basic WoS CC subscription, but the terms of their accessibility are not explained”

In the paper

“this information is not mentioned in any provided JCR description”

In the paper

“there are no explanations provided regarding such cases”

In the paper

“Clarivate annually provides a full list of journals included in JCR where full and abbreviated journal titles, country/region, and inclusion to SSCI or SCIE indexes are indicated”

In the paper

“The downloadable lists that were provided at the MJL website have only recently been improved from unnumbered journal tables in PFD format [47], which were very inconvenient to analyze and compare and, currently, all of the lists are provided in CSV format.”

What the sources say

• Not checked yetcites [47] Mongeon, 2016, The Journal Coverage of Web of Science and Scopus: A Comparative Analysis, Scientometrics source

In the paper

“although ESCI is a part of WoS CC, ESCI BackFile covering literature back to 2005 [211] is not included into basic subscription and it has to be purchased separately”

What the sources say

• Not checked yetcites [211] Clarivate (2020, November 25). Emerging Sources Citation Index Backfile (2005–2014). Available online: https://clarivate.com/wp-content/uploads/2018/05/M255-Crv_SAR_ESCI-Individual-infographic-002.pdf.
• Not checked yetcites [55] 2018, Coverage of Highly-Cited Documents in Google Scholar, Web of Science, and Scopus: A Multidisciplinary Comparison, Scientometrics

In the paper

“according to Clarivate, books are integrated within other indexes”

What the sources say

• Not checked yetcites [196] Ruccolo, M. (2020, November 25). Web of Science Core Collection: Web of Science: Summary of Coverage. Available online: https://clarivate.libguides.com/woscc/coverage.

In the paper

“in Scopus DB, the full lists of source titles (including both currently active and discontinued titles) are provided and they can be downloaded in an Excel file format at Scopus Source page (requires user login) [115], and at Elsevier website (registration is not required) [146]”

What the sources say

• Not checked yetcites [115] Scopus (2020, November 26). Scopus preview—Sources. Available online: https://www.scopus.com/sources.uri?zone=TopNavBar&origin=SearchAffiliationLookup.
• Not checked yetcites [146] Elsevier (2020, November 25). Content—How Scopus Works. Available online: https://www.elsevier.com/solutions/scopus/how-scopus-works/content.

In the paper

“Meanwhile, in WoS, only publication titles and journal names are linked to the publication's and journal's details pages, respectively. Yet, in the case of source titles, this only applies for journals that are included in JCR [114].”

What the sources say

• Not checked yetcites [114] Clarivate (2020, November 25). Journal Citation Reports—Descriptive Document. Available online: https://clarivate.libguides.com/ld.php?content_id=48842741.

In the paper

“For instance, when investigating acknowledgements to libraries in WoS indexed papers, WoS FT search only retrieved 56% of articles actually having that information, and only if funding was mentioned [104].”

What the sources say

• Not checked yetcites [104] Hubbard, 2020, Acknowledgment of Libraries in the Journal Literature: An Exploratory Study, J. Data Inf. Sci.

In the paper

“A brief description of indexes, including coverage extent and time frame, content providers, and the main subject fields covered (for most of the indexes), can be accessed from WoS website [193], and also at additional sites [209,210], as well as via the WoS 'Help' directory.”

What the sources say

• Not checked yetcites [193] Clarivate (2020, November 25). Web of Science Platform—Web of Science Group. Available online: https://clarivate.com/webofsciencegroup/solutions/webofscience-platform/.
• Not checked yetcites [209] Matthews, T. (2020, December 04). Regional Citation Indexes. Available online: https://clarivate.libguides.com/webofscienceplatform/rci.
• Not checked yetcites [210] Matthews, T. (2020, November 25). Web of Science: Direct Links: Home. Available online: https://clarivate.libguides.com/c.php?g=648493&p=4547878.

In the paper

“Meanwhile, in Scopus there are no such limitations as all content of the DB is subscribed as a whole product by a single type subscription [53].”

What the sources say

• Not checked yetcites [53] Huang, 2020, Comparison of Bibliographic Data Sources: Implications for the Robustness of University Rankings, Quant. Sci. Stud.

In the paper

“The only exception is the case of Cited Reference search, which does not depend on the subscription type, since 'Citing Articles' counts reflects citations from all years and all editions of the WoS CC-even of those that are not subscribed [105].”

What the sources say

• Not checked yetcites [105] Clarivate (2020, September 21). Web of Science Core Collection—Quick Reference Guide. Available online: https://clarivate.libguides.com/ld.php?content_id=35888196.

In the paper

“the book search in the MBL may not be limited only to WoS CC indexes, and it is not clear whether content access restrictions that are defined by subscription terms are relevant to the search”

In the paper

“The additional information, including citation metrics, peer review details, open access information, and more, can be found at journals' profile pages, but it can only be viewed and collected by searching journals manually”

What the sources say

• Not checked yetcites [27] Badia, 2018, Identifying “Best Bets” for Searching in Chemical Engineering: Comparing Database Content and Performance for Information Retrieval, J. Doc. source

In the paper

“The amount of journals indexed in the main bibliographic DBs has significantly increased and is still growing [220].”

What the sources say

• Not checked yetcites [220] Gu, 2016, Recent Trends in Academic Journal Growth, Scientometrics source

In the paper

“However, since both WoS and Scopus are commercial, their content is protected by restricting access to raw data [42], and access to their APIs usually requires additional payment [32].”

What the sources say

• Not checked yetcites [42] Gusenbauer, 2020, Which Academic Search Systems Are Suitable for Systematic Reviews or Meta-Analyses? Evaluating Retrieval Qualities of Google Scholar, PubMed, and 26 Other Resources, Res. Synth. Methods source
• Not checked yetcites [32] Thelwall, 2021, Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: A Multidisciplinary Comparison of Coverage via Citations, Scientometrics source

In the paper

“Consequently, over twenty different bibliometric indicators are currently available, and new versions or alternative metrics are still being actively developed [26].”

What the sources say

• Not checked yetcites [26] Waltman, 2016, A Review of the Literature on Citation Impact Indicators, J. Informetr. source

In the paper

“Another possibility for analyzing larger data sets is to use additional online tools, such as InCites (for WoS data) and SciVal (for Scopus data) [26,219].”

What the sources say

• Not checked yetcites [26] Waltman, 2016, A Review of the Literature on Citation Impact Indicators, J. Informetr. source
• Not checked yetcites [219] Williams, 2016, Sampling Issues in Bibliometric Analysis, J. Informetr. source

In the paper

“JIF was developed by Eugene Garfield more than half a century ago [223] and it was originally intended to define the quality of journals to provide high quality indexed content, and for publication or subscription purposes.”

What the sources say

• Not checked yetcites [223] Garfield, 1955, Citation Indexes for Science: A New Dimension in Documentatio through Association of Ideas, Science source

In the paper

“JIF values can be deliberately distorted by publishing more reviews, which are more highly cited than regular articles, or by publishing case reports, letters to editors, and book reviews, which are being cited, but not included in the total paper count in the JIF denominator.”

In the paper

“Yet, probably the most debated feature of JIF is the inconsistency of document types included in its numerator and denominator.”

In the paper

“Raw data from both DBs can be further analyzed by various bibliometric software tools. However, this approach, as well as using APIs for data retrieval, requires some kind of programming skills [9,42].”

What the sources say

• Not checked yetcites [9] Cobo, 2020, Software Tools for Conducting Bibliometric Analysis in Science: An up-to-Date Review, Prof. Inf.
• Not checked yetcites [42] Gusenbauer, 2020, Which Academic Search Systems Are Suitable for Systematic Reviews or Meta-Analyses? Evaluating Retrieval Qualities of Google Scholar, PubMed, and 26 Other Resources, Res. Synth. Methods source

In the paper

“For example, WoS allows for creating data sets of up to 50,000 data rows in one session (for citation analyses-up to 10,000).”

In the paper

“However, today, JIF is often misused due to a misinterpretation of its meaning, as it has become a standard tool in virtually all evaluation practices at all levels, usually without proper considerations of the rationale for its use [2,44,222,224-227].”

What the sources say

• Not checked yetcites [2] Agarwal, 2016, Bibliometrics: Tracking Research Impact by Selecting the Appropriate Metrics, Asian J. Androl. source
• Not checked yetcites [44] Wouters, P., Thelwall, M., Kousha, K., Waltman, L., de Rijcke, S., Rushforth, A., and Franssen, T. (2015). The Metric Tide: Literature Review (Supplementary Report I to the Independent Review of the Role of Metrics in Research Assessment and Management), HEFCE.
• Not checked yetcites [222] Setti, 2013, Bibliometric Indicators: Why Do We Need More than One?, IEEE Access source
• Not checked yetcites [224] Glänzel, W., Moed, H.F., Schmoch, U., and Thelwall, M. (2018). The Journal Impact Factor: A Brief History, Critique, and Discussion of Adverse Effects. Springer Handbook of Science and Technology Indicators, Springer International Publishing. source
• Not checked yetcites [225] Ranjan, 2017, Bibliometric Indices of Scientific Journals: Time to Overcome the Obsession and Think beyond the Impact Factor, Med. J. Armed Forces India source
• Not checked yetcites [226] Callaway, 2016, Publishing Elite Turns against Impact Factor, Nature source
• Not checked yetcites [227] Moustafa, 2014, The Disaster of the Impact Factor, Sci. Eng. Ethics source

In the paper

“Yet, only a minority of stakeholders employing bibliometric indicators are adequately familiar with their meaning and appropriate use [2,4,5,7].”

What the sources say

• Not checked yetcites [2] Agarwal, 2016, Bibliometrics: Tracking Research Impact by Selecting the Appropriate Metrics, Asian J. Androl. source
• Not checked yetcites [4] Leydesdorff, 2016, Professional and Citizen Bibliometrics: Complementarities and Ambivalences in the Development and Use of Indicators—a State-of-the-Art Report, Scientometrics source
• Not checked yetcites [5] Moed, 2015, Multidimensional Assessment of Scholarly Research Impact, J. Assoc. Inf. Sci. Technol. source
• Not checked yetcites [7] Wouters, 2016, Evaluation Practices and Effects of Indicator Use—A Literature Review, Res. Eval. source

In the paper

“Thus, JCR also provides five-year JIF, which is basically the same JIF with the time window for cited documents extended to five years.”

In the paper

“Especially, due to the fact that citation metrics are calculated only using data from the particular DB, they are dependent on DBs coverage width and depth.”

In the paper

“However, both of these tools are not a part of the DBs and they are only accessible with separate (and costly) subscription.”

In the paper

“In the numerator, citations are counted to all document types, and only specific types of documents, so-called 'citable items', namely articles, reviews, and proceedings papers are included in the denominator, which represents the total amount of cited papers [45,224].”

What the sources say

• Not checked yetcites [45] Walters, 2017, Citation-Based Journal Rankings: Key Questions, Metrics, and Data Sources, IEEE Access source
• Not checked yetcites [224] Glänzel, W., Moed, H.F., Schmoch, U., and Thelwall, M. (2018). The Journal Impact Factor: A Brief History, Critique, and Discussion of Adverse Effects. Springer Handbook of Science and Technology Indicators, Springer International Publishing. source

In the paper

“Analysis data can be extracted as displayed or all data rows (up to 200,000 rows), which can be downloaded as a tab-delimited text file.”

In the paper

“Meanwhile, WoS offers several different API types, but access to them has to be acquired separately with additional conditions for use [28,218].”

What the sources say

• Not checked yetcites [28] Carloni, M., Tsenkulovsky, T., and Mangan, R. (2020, August 13). Web of Science Core Collection Descriptive Document. Available online: https://clarivate.libguides.com/ld.php?content_id=45175981.
• Not checked yetcites [218] Birkle, 2020, Web of Science as a Data Source for Research on Scientific and Scholarly Activity, Quant. Sci. Stud. source

In the paper

“It should be noted that, although ESCI journals are not included in the JCR and do not receive JIF (and other JCR metrics), the citations from ESCI will accrue to all articles in the WoS CC and contribute to the JIF numerator [28].”

What the sources say

• Not checked yetcites [28] Carloni, M., Tsenkulovsky, T., and Mangan, R. (2020, August 13). Web of Science Core Collection Descriptive Document. Available online: https://clarivate.libguides.com/ld.php?content_id=45175981.

In the paper

“WoS also calculates an Immediacy index, which is basically a one-year JIF [26,45].”

What the sources say

• Not checked yetcites [26] Waltman, 2016, A Review of the Literature on Citation Impact Indicators, J. Informetr. source
• Not checked yetcites [45] Walters, 2017, Citation-Based Journal Rankings: Key Questions, Metrics, and Data Sources, IEEE Access source

In the paper

“However, the JIF calculation method leads to several biases that limit its application and it is continuously discussed and criticized in scientometric literature concerning bibliometric indicators and their application in research evaluation practices [222,224].”

What the sources say

• Not checked yetcites [222] Setti, 2013, Bibliometric Indicators: Why Do We Need More than One?, IEEE Access source
• Not checked yetcites [224] Glänzel, W., Moed, H.F., Schmoch, U., and Thelwall, M. (2018). The Journal Impact Factor: A Brief History, Critique, and Discussion of Adverse Effects. Springer Handbook of Science and Technology Indicators, Springer International Publishing. source

In the paper

“Possibilities to export or download data from the web-interfaces for external analyses are also usually limited, and the number of data rows available to export in one batch differs between the DBs [9,26].”

What the sources say

• Not checked yetcites [9] Cobo, 2020, Software Tools for Conducting Bibliometric Analysis in Science: An up-to-Date Review, Prof. Inf.
• Not checked yetcites [26] Waltman, 2016, A Review of the Literature on Citation Impact Indicators, J. Informetr. source

In the paper

“Although, according to Elsevier, Scopus provides free access to a basic API version of Scopus data for research purposes without a requirement of subscription, allowing limited access to basic metadata for most citation records, while full API access is only granted to subscribed users, but it does not require additional payments.”

In the paper

“The most obvious limitation of the JIF is the short two-year citation time window.”

In the paper

“Meanwhile, Scopus provides CiteScore (CS) metrics, along with other, more advanced indicators-SCImago Journal Rank (SJR) and Source Normalized Impact per Paper (SNIP) [175].”

What the sources say

• Not checked yetcites [175] James, 2019, CiteScore Metrics: Creating Journal Metrics from the Scopus Citation Index, Learn. Publ. source

In the paper

“In Scopus, the export limit reaches 2000 data rows, but Scopus also allows exporting up to 20,000 results (in 'Citations only' (CSV) format) using e-mail services.”

In the paper

“On the other hand, it raises the question of validity for comparisons, as journals or other document sets are being evaluated against unequal benchmarks [221].”

What the sources say

• Not checked yetcites [221] Mingers, 2017, Evaluating Journal Quality: A Review of Journal Citation Indicators and Ranking in Business and Management, Eur. J. Oper. Res. source

In the paper

“However, as was discussed earlier, both WoS and Scopus classifications are not perfect and they may introduce certain biases in normalizing impact indicators [221].”

What the sources say

• Not checked yetcites [221] Mingers, 2017, Evaluating Journal Quality: A Review of Journal Citation Indicators and Ranking in Business and Management, Eur. J. Oper. Res. source

In the paper

“since journals belonging to A&HCI and ESCI are not included in JCR, they do not have JIF or other journal impact metrics”

What the sources say

• Not checked yetcites [45] Walters, 2017, Citation-Based Journal Rankings: Key Questions, Metrics, and Data Sources, IEEE Access source
• Not checked yetcites [224] Glänzel, W., Moed, H.F., Schmoch, U., and Thelwall, M. (2018). The Journal Impact Factor: A Brief History, Critique, and Discussion of Adverse Effects. Springer Handbook of Science and Technology Indicators, Springer International Publishing. source

In the paper

“as JCR does not include books and conference material (with rare exceptions), these sources also cannot be evaluated by WoS journal metrics.”

In the paper

“The second and most important difference is that, in CS calculation, the publication types included in numerator and denominator coincide and include all document types indexed in Scopus.”