An Evidence-Based Review of Academic Web Search Engines, 2014-2016: Implications for Librarians’ Practice and Research Agenda
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Academic web search engines have become central to scholarly research. While the fitness of Google Scholar for research purposes has been examined repeatedly, Microsoft Academic and Google Books have not received much attention. Recent studies have much to tell us about the coverage and utility of Google Scholar, its coverage of the sciences, and its utility for evaluating researcher impact. But other aspects have been woefully understudied, such as coverage of the arts and humanities, books, and non-Western, non-English publications. User research has also tapered off. A small number of articles hint at the opportunity for librarians to become expert advisors concerning opportunities of scholarly communication made possible or enhanced by these platforms. This article seeks to summarize research concerning Google Scholar, Google Books, and Microsoft Academic from the past three years with a mind to informing practice and setting a research agenda. Selected literature from earlier time periods is included to illuminate key findings and to help shape the proposed research agenda, especially in understudied areas.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The role of google scholar in evidence reviews and its applicability to grey literature searching
Haddaway, N. R., Collins, A. M., Coughlin, D., & Kirk, S. (2015) · PLOS ONE · doi:10.1371/journal.pone.0138237 · Cited by 2,387
Google Scholar, Scopus and the Web of Science: A Longitudinal and Cross-Disciplinary Comparison
Harzing, A.-W., & Alakangas, S. (2016) · Scientometrics · doi:10.1007/s11192-015-1798-9 · Cited by 1,784
A review of the literature on citation impact indicators
Waltman, L. (2016) · Journal of Informetrics · doi:10.1016/j.joi.2016.02.007 · Cited by 1,356
Online collaboration: Scientists and the social network
Van Noorden, R. (2014) · Nature · doi:10.1038/512126a · Cited by 654 · cited 6×
“…in the humanities suggests Mendeley and Twitter should be included along with Google Scholar when examining citation impact of humanities research, while a 2014 Nature survey suggests ResearchGate is much less popular in the social sciences and humanities than in the sciences (Van Noorden 2014).”
BackgroundInfluentialIs the coverage of google scholar enough to be used alone for systematic reviews
Gehanno, J.-F., Rollin, L., & Darmoni, S. (2013) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-13-7 · Cited by 357 · cited 1×
“A study by Gehanno et al. (2013) found Google Scholar contained 100% of the references from 29 systematic reviews, and suggested Google Scholar could be the first choice for systematic reviews or meta-analyses.”
BackgroundThe Google scholar experiment: How to index false papers and manipulate bibliometric indicators
Delgado López‐Cózar, E., Robinson‐García, N., & Torres‐Salinas, D. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23056 · Cited by 272
Boeker, M., Vach, W., & Motschall, E. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-131 · Cited by 225
Bramer, W. M., Giustini, D., & Kramer, B. M. R. (2016) · Systematic Reviews · doi:10.1186/s13643-016-0215-7 · Cited by 202
Google Scholar is not enough to be used alone for systematic reviews
Giustini, D., & Kamel Boulos, M. N. (2013) · Online Journal of Public Health Informatics · doi:10.5210/ojphi.v5i2.4623 · Cited by 187
The Depth and Breadth of Google Scholar: An Empirical Study
Neuhaus, C., Neuhaus, E., Asher, A., & Wrede, C. (2006) · portal: Libraries and the Academy · doi:10.1353/pla.2006.0026 · Cited by 156
Cited by
The ABC of systematic literature review: the basic methodological guidance for beginners
Mohamed Shaffril, H. A., Samsuddin, S. F., & Abu Samah, A. (2021) · Quality & Quantity · doi:10.1007/s11135-020-01059-6 · Cited by 656
Wearable sensor-based real-time gait detection: A systematic review
Prasanth, H., Caban, M., Keller, U., Courtine, G., Ijspeert, A., Vallery, H., & von Zitzewitz, J. (2021) · Sensors · doi:10.3390/s21082727 · Cited by 348
Academic Plagiarism Detection: A Systematic Literature Review
Foltýnek, T., Meuschke, N., & Gipp, B. (2020) · ACM Computing Surveys · doi:10.1145/3345317 · Cited by 238
The Coverage of Microsoft Academic: Analyzing the Publication Output of a University
Hug, S. E., & Brändle, M. P. (2017) · Scientometrics · doi:10.1007/s11192-017-2535-3 · Cited by 120
Metadata
- Authors
- Fagan, J. C.
- Venue
- Information Technology and Libraries
- Publisher
- American Library Association
- Published
- 2017-06-28
- Volume
- 36
- Issue
- 2
- Pages
- 7-47
- ISSN
- 0730-9295 (ISSN-L)
- 2163-5226
- OpenAlex
- W2614246374
- Cited by
- 36
- References
- 14
- FWCI
- 4.57
- Type
- journal article
- Language
- en
- Open access
- diamond
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: An Evidence-Based Review of Academic Web Search Engines, 2014-2016: Implications for Librarians’ Practice and Research Agenda. https://app.cochat.ai/papers/10.6017/ital.v36i2.9718
- The paper
- Fagan, J. C. (2017). An Evidence-Based Review of Academic Web Search Engines, 2014-2016: Implications for Librarians’ Practice and Research Agenda. Information Technology and Libraries. https://doi.org/10.6017/ital.v36i2.9718
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.