Variable Bibliographic Database Access Could Limit Reproducibility
Abstract
Bibliographic databases provide access to scientific literature through targeted queries. The most common uses of these services, aside from accessing scientific literature for personal use, are to find relevant citations for formal surveys of scientific literature, such as systematic reviews or meta-analysis, or to estimate the number of publications on a certain topic as a measure of sampling effort. Bibliographic search tools vary in the level of access to the scientific literature they allow. For instance, Google Scholar is a bibliographic search engine which allows users to find (but not necessarily access) scientific literature for no charge, whereas other services, such as Web of Science, are subscription based, allowing access to full texts of academic works at costs that can exceed $100,000 annually for large universities (Goodman 2005). One of the most commonly used bibliographic databases, Clarivate Analytics–produced Web of Science, offers tailored subscriptions to their citation indexing service. This flexibility allows subscriptions and resulting access to be tailored to the needs of researchers at the institution (Goodwin 2014). However, there are issues created by this differential access, which we discuss further below.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Institutions are often forced to choose between WoS and Scopus.
“Consequently, institutions are often forced to choose between these data sources [15].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [15]
The majority of DBs' end users are not usually aware of these limitations and how they can affect the accuracy, reliability, and reproducibility of their task's outcome.
“Unfortunately, the majority of DBs' end users are not usually aware of these limitations and how they can affect the accuracy, reliability, and reproducibility of their task's outcome [15,23].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [15]
The usability of data sources may highly depend on additional content accessibility restrictions and availability to download and export data.
“However, the usability of data sources may also highly depend on the additional content accessibility restrictions and availability to download and export data.”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [15]
An accurate evaluation of WoS CC content coverage is highly challenging, as WoS is composed of a variety of specialized indexes and the composition of WoS CC can be modulated by different subscription terms.
“An accurate evaluation of WoS CC content coverage is highly challenging, as WoS is composed of a variety of specialized indexes and the composition of WoS CC can be modulated by different subscription terms”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [15]
The composition of WoS CC and the extent of actually accessible content for users of a particular institution may differ significantly from the one described by the owner.
“the composition of WoS CC and the extent of actually accessible content for users of particular institution may differ significantly from the one described by the owner”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [15]
Variations in WoS content availability make it very difficult to reassure reproducibility of analyses performed using WoS data.
“these variations in WoS content availability makes it very difficult to reassure a reproducibility of any analyses performed using WoS data [15].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [15]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823 · see citing works
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042 · see citing works
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139 · see citing works
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement.
(2009) · PubMed Central · Cited by 44,859 · see citing works
PRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223 · see citing works
Host and viral traits predict zoonotic spillover from mammals
Olival, K. J., Hosseini, P. R., Zambrana-Torrelio, C., Ross, N., Bogich, T. L., & Daszak, P. (2017) · Nature · doi:10.1038/nature22975 · Cited by 1,130 · see citing works
Google Scholar as a new source for citation analysis
Harzing, A.-W. K., & van der Wal, R. (2007) · Ethics in Science and Environmental Politics · doi:10.3354/esep00076 · Cited by 677 · see citing works
Lindenfors, P., Nunn, C. L., Jones, K. E., Cunningham, A. A., Sechrest, W., & Gittleman, J. L. (2007) · Global Ecology and Biogeography · doi:10.1111/j.1466-8238.2006.00301.x · Cited by 231 · see citing works
Goodwin, C. (2014) · The Charleston Advisor · doi:10.5260/chara.16.2.55 · Cited by 3 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- https://academic.oup.com/bioscience August 2018 / Vol. 68 No. 8 • BioScience 553
Cited by
Sorted by citations
Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World
Pranckutė, R. (2021) · Publications · doi:10.3390/publications9010012 · Cited by 2,696 · see citing works
The Data Source of This Study Is Web of Science Core Collection? Not Enough
Liu, W. (2019) · Scientometrics · doi:10.1007/s11192-019-03238-1 · Cited by 221 · see citing works
- Authors
- Dallas, T., Gehman, A.-L., & Farrell, M. J.
- Affiliations
- University of Helsinki (FI)
- University of British Columbia (CA)
- McGill University (CA)
- Venue
- BioScience
- Publisher
- Oxford University Press (OUP)
- Published
- 2018-07-07
- Volume
- 68
- Issue
- 8
- Pages
- 552-553
- ISSN
- 0006-3568 (ISSN-L)
- 1525-3244
- OpenAlex
- W2847863984
- Cited by
- 15
- References
- 9
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 3 · 2023: 1 · 2022: 1 · 2021: 2 · 2020: 2 · 2019: 1 · 2018: 1
- FWCI
- 1.14
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- BioScience (publishedVersion)
- Funders
- National Science Foundation (131223)
- U.S. Department of Agriculture
- National Institutes of Health
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Social Sciences › Decision Sciences › Information Systems and Management › Academic Publishing and Open Access
- Physical Sciences › Computer Science › Information Systems › Library Science and Information
- Keywords
- bibliographic databases, access to scientific literature, reproducibility, Web of Science, Google Scholar, citation indexing, systematic reviews, meta-analysis, scholarly communication
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
