Medical literature searches: a comparison of PubMed and Google Scholar
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
Background: Medical literature searches provide critical information for clinicians. However, the best strategy for identifying relevant high‐quality literature is unknown. Objectives: We compared search results using P ub M ed and G oogle S cholar on four clinical questions and analysed these results with respect to article relevance and quality. Methods: Abstracts from the first 20 citations for each search were classified into three relevance categories. We used the weighted kappa statistic to analyse reviewer agreement and nonparametric rank tests to compare the number of citations for each article and the corresponding journals' impact factors. Results: Reviewers ranked 67.6% of P ub M ed articles and 80% of G oogle S cholar articles as at least possibly relevant (P = 0.116) with high agreement (all kappa P‐values < 0.01). Google Scholar articles had a higher median number of citations (34 vs. 1.5, P < 0.0001) and came from higher impact factor journals (5.17 vs. 3.55, P = 0.036). Conclusions: P ub M ed searches and G oogle S cholar searches often identify different articles. In this study, G oogle S cholar articles were more likely to be classified as relevant, had higher numbers of citations and were published in higher impact factor journals. The identification of frequently cited articles using G oogle S cholar for searches probably has value for initial literature searches.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Individual Comparisons by Ranking Methods
Wilcoxon, F. (1945) · Biometrics Bulletin · doi:10.2307/3001968 · Cited by 15,354
Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit
Cohen, J. (1968) · Psychological Bulletin · doi:10.1037/h0026256 · Cited by 8,736
Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses
Falagas, M. E., Pitsouni, E. I., Malietzis, G. A., & Pappas, G. (2008) · The FASEB Journal · doi:10.1096/fj.07-9492lsf · Cited by 5,039
Three options for citation tracking: Google Scholar, Scopus and Web of Science
Bakkalbasi, N., Bauer, K., Glover, J., & Wang, L. (2006) · Biomedical Digital Libraries · doi:10.1186/1742-5581-3-7 · Cited by 752
Understanding PubMed(R) user search behavior through log analysis
Islamaj Dogan, R., Murray, G. C., Neveol, A., & Lu, Z. (2009) · Database · doi:10.1093/database/bap018 · Cited by 265
A Day in the Life of PubMed: Analysis of a Typical Day's Query Log
Herskovic, J. R., Tanaka, L. Y., Hersh, W., & Bernstam, E. V. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2191 · Cited by 170
Research works printed but not linked: 18
- 10.1038/432423a · DOI
- Vine R., 2006, Electronic resources reviews‐ Google Scholar, JMIA, 94, 97
- 10.1503/cmaj.050404 · DOI · Paper page
- 10.1056/NEJMp058128 · DOI · Paper page
- 10.1186/1472-6947-8-42 · DOI
- 10.1197/jamia.M1909 · DOI
- 10.1016/S0306-4573(99)00072-2 · DOI · Paper page
- Hollander M., 1973, Nonparametric Statistical Methods, 115
- Beel J.&Gipp B.Google Scholar's ranking algorithm: the impact of citation counts (an empirical study). Proceedings of the 3rd International Conference on Research Challenges in Information Science2009a 439–446. · DOI · Paper page
- Beel J.&Gipp B.Google Scholar's Ranking Algorithm: The impact of articles' age (an empirical study). Proceedings of the 6th International Conference on Information Technology: New Generations2009b 160–164. · DOI
- Shultz M., 2007, Comparing test searches in PubMed and Google Scholar, JMIA, 95, 442
- 10.1186/1756-0500-3-131 · DOI · Paper page
- Anders M. E., 2010, Comparison of PubMed and Google Scholar literature searches, Respiratory Care, 55, 578
- 10.1001/jama.2009.1307 · DOI · Paper page
- 10.1197/jamia.M2215 · DOI
- 10.1016/j.jbi.2009.02.009 · DOI
- Whipple E. C., 2009, The selection of high‐impact health informatics literature: a comparison of results between the content expert and the expert searcher, JMIA., 97, 212
- 10.1016/S1389-1286(00)00031-1 · DOI · Paper page
Cited by
Halevi, G., Moed, H., & Bar-Ilan, J. (2017) · Journal of Informetrics · doi:10.1016/j.joi.2017.06.005 · Cited by 488
Is the coverage of google scholar enough to be used alone for systematic reviews
Gehanno, J.-F., Rollin, L., & Darmoni, S. (2013) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-13-7 · Cited by 357
“116) [28].”
Keating, S. E., Johnson, N. A., Mielke, G. I., & Coombes, J. S. (2017) · Obesity Reviews · doi:10.1111/obr.12536 · Cited by 344
The impact of COVID-19 school closure on child and adolescent health: a rapid systematic review
Chaabane, S., Doraiswamy, S., Chaabna, K., Mamtani, R., & Cheema, S. (2021) · Children · doi:10.3390/children8050415 · Cited by 288
Boeker, M., Vach, W., & Motschall, E. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-131 · Cited by 225
“In the light of this well justified demand, recent results on the quality of Google Scholar as a resource for scientific literature retrieval, even for systematic reviews, gave rise to high expectations on the emergence of such a unifying search interface [7-9].”
Background
Metadata
- Authors
- Nourbakhsh, E., Nugent, R., Wang, H., Cevik, C., & Nugent, K.
- Venue
- Health Information & Libraries Journal
- Publisher
- Wiley
- Published
- 2012-06-19
- Volume
- 29
- Issue
- 3
- Pages
- 214-222
- ISSN
- 1471-1834 (ISSN-L)
- 1471-1842
- OpenAlex
- W2087641610
- Cited by
- 113
- References
- 6
- FWCI
- 15.56
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Medical literature searches: a comparison of PubMed and Google Scholar. https://app.cochat.ai/papers/10.1111/j.1471-1842.2012.00992.x
- The paper
- Nourbakhsh, E., Nugent, R., Wang, H., Cevik, C., & Nugent, K. (2012). Medical literature searches: a comparison of PubMed and Google Scholar. Health Information & Libraries Journal. https://doi.org/10.1111/j.1471-1842.2012.00992.x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.