Google Scholar, Microsoft Academic, Scopus, Dimensions, Web of Science, and OpenCitations’ COCI: A Multidisciplinary Comparison of Coverage via Citations
Open accessAbstract
New sources of citation data have recently become available, such as Microsoft Academic, Dimensions, and the OpenCitations Index of CrossRef open DOI-to-DOI citations (COCI). Although these have been compared to the Web of Science Core Collection (WoS), Scopus, or Google Scholar, there is no systematic evidence of their differences across subject categories. In response, this paper investigates 3,073,351 citations found by these six data sources to 2,515 English-language highly-cited documents published in 2006 from 252 subject categories, expanding and updating the largest previous study. Google Scholar found 88% of all citations, many of which were not found by the other sources, and nearly all citations found by the remaining sources (89-94%). A similar pattern held within most subject categories. Microsoft Academic is the second largest overall (60% of all citations), including 82% of Scopus citations and 86% of WoS citations. In most categories, Microsoft Academic found more citations than Scopus and WoS (182 and 223 subject categories, respectively), but had coverage gaps in some areas, such as Physics and some Humanities categories. After Scopus, Dimensions is fourth largest (54% of all citations), including 84% of Scopus citations and 88% of WoS citations. It found more citations than Scopus in 36 categories, more than WoS in 185, and displays some coverage gaps, especially in the Humanities. Following WoS, COCI is the smallest, with 28% of all citations. Google Scholar is still the most comprehensive source. In many subject categories Microsoft Academic and Dimensions are good alternatives to Scopus and WoS in terms of coverage.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Due to its recent introduction, relatively little is known regarding Dimensions' comprehensiveness and validity as a reliable bibliographic data source.
“However, due to its recent introduction, relatively little is known regarding its comprehensiveness and validity as a reliable bibliographic data source, as the exploring of these features has only just began in the scientometric community [32,37-40].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [32]
Many other data sources are free and relatively new, and their validity is still questionable.
“Many of them are also free and relatively new products, such as Microsoft Academic, CrossRef, ResearchGate, OpenCitations, etc. [32,39-41]. Hence, their validity is also still questionable.”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [32]
Google Scholar's data export is unavailable for large-scale analysis.
“Moreover, one of the main setbacks in trying to use GS for large-scale analysis is the unavailability of its data export [23,31-33].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [32]
In a large-scale comparison of six data sources, Scopus retrieved 57% of all combined citations to sample documents, while WoS retrieved 52%.
“a very recent large scale comparison of six data sources has shown that Scopus retrieved 57% of all combined citations to the sample documents, while WoS retrieved 52% of the total citations.”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [32]
Both WoS and Scopus are commercial, their content is protected by restricting access to raw data, and access to their APIs usually requires additional payment.
“However, since both WoS and Scopus are commercial, their content is protected by restricting access to raw data [42], and access to their APIs usually requires additional payment [32].”
From Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World (2021) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
The citation lists haven’t been fetched yet
References 55 · Cited by 1,018
Research works printed but not linked: 8
- Beel, J., & Gipp, B. (2009c). Google Scholar’s ranking algorithm: An introductory overview. In Proceedings of the 12th International Conference on Scientometrics and Informetrics (ISSI’09) (pp. 230–241). http://www.issi-society.org/proceedings/issi_2009/ISSI2009-proc-vol1_Aug2009_batch2-paper-1.pdf
- Delgado López-Cózar E Orduna-Malea E Martín-Martín A Glaenzel W Moed H Schmoch U Thelwall M Google Scholar as a data source for research assessment Springer handbook of science and technology indicators 2019 Berlin Springer
- Orduña-Malea E Delgado-López-Cózar E Dimensions: Re-discovering the ecosystem of scientific information Profesional de La Informacion 2018 27 2 420 431 10.3145/epi.2018.mar.21 · DOI
- Orduna-Malea E Martín-Martín A Delgado López-Cózar E Google Scholar as a source for scholarly evaluation: A bibliographic review of database errors Revista Española de Documentación Científica 2017 40 4 e185 10.3989/redc.2017.4.1500 · DOI
- Ortega JL Academic search engines: A quantitative outlook 2014 Cambridge Chandos Publishing
- Van Noorden R November 7) Google Scholar pioneer on search engine’s future. Nature. 2014 10.1038/nature.2014.16269 · DOI
- Wickham H ggplot2: Elegant graphics for data analysis 2016 New York Springer
- Wu J Kim K Giles CL CiteSeerX: 20 years of service to scholarly big data Proceedings of the Conference on Artificial Intelligence for Data Discovery and Reuse 2019 10.1145/3359115.3359119 · DOI
- Authors
- Martín-Martín, A., Thelwall, M., Orduna-Malea, E., & Delgado López-Cózar, E.
- Affiliations
- University of Wolverhampton (GB)
- Venue
- Scientometrics
- Published
- 2020-09-21
- Volume
- 126
- Issue
- 1
- Pages
- 871-906
- ISSN
- 0138-9130 (ISSN-L)
- 1588-2861
- PMC
- PMC7505221
- OpenAlex
- W3023758415
- Cited by
- 1,018
- References
- 55
- Citations by year
- 2026: 203 · 2025: 235 · 2024: 230 · 2023: 167 · 2022: 114 · 2021: 62 · 2020: 7
- FWCI
- 53.35
- Type
- journal article
- Language
- en
- Update notices
- correction (2020-12-04)
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Scientometrics (publishedVersion, public-domain)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- Institutional Repository of the University of Granada (University of Granada) (acceptedVersion, other-oa)
- Wolverhampton Intellectual Repository and E-Theses (University of Wolverhampton) (acceptedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- RiuNet (Universitat Politècnica de València) (submittedVersion, cc-by-nc-nd)
- arXiv (Cornell University)
- Funders
- Universidad de Granada
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › scientometrics and bibliometrics research
- Physical Sciences › Computer Science › Information Systems › Web visibility and informetrics
- Physical Sciences › Computer Science › Information Systems › Library Science and Information
- Keywords
- citation indexes, citation coverage, Google Scholar, Microsoft Academic, Scopus, dimensions, Web of Science Core Collection, highly cited papers
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
