The DOI matches its Crossref record; 56 of the 58 references it cites were found in a registry or an index, and 1 are not checked yet.
Google Scholar as replacement for systematic literature searches: good relative recall and precision are not enough
Open accessAbstract
BACKGROUND: Recent research indicates a high recall in Google Scholar searches for systematic reviews. These reports raised high expectations of Google Scholar as a unified and easy to use search interface. However, studies on the coverage of Google Scholar rarely used the search interface in a realistic approach but instead merely checked for the existence of gold standard references. In addition, the severe limitations of the Google Search interface must be taken into consideration when comparing with professional literature retrieval tools.The objectives of this work are to measure the relative recall and precision of searches with Google Scholar under conditions which are derived from structured search procedures conventional in scientific literature retrieval; and to provide an overview of current advantages and disadvantages of the Google Scholar search interface in scientific literature retrieval. METHODS: General and MEDLINE-specific search strategies were retrieved from 14 Cochrane systematic reviews. Cochrane systematic review search strategies were translated to Google Scholar search expression as good as possible under consideration of the original search semantics. The references of the included studies from the Cochrane reviews were checked for their inclusion in the result sets of the Google Scholar searches. Relative recall and precision were calculated. RESULTS: We investigated Cochrane reviews with a number of included references between 11 and 70 with a total of 396 references. The Google Scholar searches resulted in sets between 4,320 and 67,800 and a total of 291,190 hits. The relative recall of the Google Scholar searches had a minimum of 76.2% and a maximum of 100% (7 searches). The precision of the Google Scholar searches had a minimum of 0.05% and a maximum of 0.92%. The overall relative recall for all searches was 92.9%, the overall precision was 0.13%. CONCLUSION: The reported relative recall must be interpreted with care. It is a quality indicator of Google Scholar confined to an experimental setting which is unavailable in systematic retrieval due to the severe limitations of the Google Scholar search interface. Currently, Google Scholar does not provide necessary elements for systematic scientific literature retrieval such as tools for incremental query optimization, export of a large number of references, a visual search builder or a history function. Google Scholar is not ready as a professional searching tool for tasks where structured retrieval methodology is necessary.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“This research thus fills a need for support in the choice of search system, currently lacking in evidence‐synthesis methodology, and follows calls for comparative studies on the effectiveness of search systems25 or the “need to develop ‘bias profiles’ for search engines.”( 35 , p1193)”
From Which academic search systems are suitable for systematic reviews or meta‐analyses? Evaluating retrieval qualities of Google Scholar, PubMed, and 26 other resources (2020) · cites this paper as [25]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Giustini and Boulos [14] found that 5% of studies from a systematic review could not be identified using specific searches in Google Scholar, whilst Boeker et al. [11] found that up to 34% of studies from 14 systematic reviews were missed.”
From The role of google scholar in evidence reviews and its applicability to grey literature searching (2015) · cites this paper as [11]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Quality criteria have previously been used to evaluate search systems in terms of recall and precision indicating their suitability of search in general and systematic search in particular.”
The cited source says what the claim attributes to it.
In the citing paper
“Our findings support the criticism of Bramer et al,33 Bramer et al,34 and Boeker et al25 and indicate that Google Scholar's coverage and recall is an inadequate reason to use it as principal search system in systematic searches.53”
In the citing paper
“The ability of a reviewer to manipulate their search query depends on the capabilities of a search system and thus they significantly influence recall and precision.25”
In the citing paper
“Nevertheless, overall, Google Scholar's search precision has been found to be significantly lower than 1% for systematic searches.25”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164 · cited 1×
“In a continuing discussion and agreement process, the information science and the scientific literature retrieval communities have developed methods and standards which aim for a high level of quality in literature retrieval and reporting [14-18].”
BackgroundCochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Cochrane Handbook for Systematic Reviews of Interventions
Tarsilla, M. (2010) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v6i14.284 · Cited by 5,232
Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses
Falagas, M. E., Pitsouni, E. I., Malietzis, G. A., & Pappas, G. (2008) · The FASEB Journal · doi:10.1096/fj.07-9492lsf · Cited by 5,039 · cited 1×
“This technology known as autonomous citation indexing, is also applied to the Web of Knowledge and Scopus, however, with differing results [37,38].”
ResultA catalogue of reporting guidelines for health research
Simera, I., Moher, D., Hoey, J., Schulz, K. F., & Altman, D. G. (2010) · European Journal of Clinical Investigation · doi:10.1111/j.1365-2362.2009.02234.x · Cited by 1,047 · cited 1×
“In a continuing discussion and agreement process, the information science and the scientific literature retrieval communities have developed methods and standards which aim for a high level of quality in literature retrieval and reporting [14-18].”
BackgroundKulkarni, A. V. (2009) · JAMA · doi:10.1001/jama.2009.1307 · Cited by 821 · cited 1×
“This technology known as autonomous citation indexing, is also applied to the Web of Knowledge and Scopus, however, with differing results [37,38].”
ResultDetection of gene pathways with predictive power for breast cancer prognosis
Ma, S., & Kosorok, M. R. (2010) · BMC Bioinformatics · doi:10.1186/1471-2105-11-1 · Cited by 693
An evidence-based practice guideline for the peer review of electronic search strategies
Sampson, M., McGowan, J., Cogo, E., Grimshaw, J., Moher, D., & Lefebvre, C. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.10.012 · Cited by 608 · cited 1×
“In a continuing discussion and agreement process, the information science and the scientific literature retrieval communities have developed methods and standards which aim for a high level of quality in literature retrieval and reporting [14-18].”
BackgroundLefebvre, C., Manheimer, E. W., & Glanville, J. (2008) · doi:10.1002/9780470712184.ch6 · Cited by 599
Selecting Studies and Collecting Data
Higgins, J. P., & Deeks, J. J. (2008) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9780470712184.ch7 · Cited by 467 · cited 2×
“Assuming fast reference checking for exclusion of irrelevant references as a first step of manual study selection [41], an experienced information specialist can check up to 1,000 references a day [53].”
Background
Research works printed but not linked: 6
- Heinzl S: Medizinreport: studien im fokus. Medizinisch-wissenschaftliche recherche: Google scholar ist PubMed möglicherweise überlegen. Dtsch Arztebl. 2012, 109: A–2467-
- Ovaska T, Fiander M, Weightman A: BMC medical informatics and decision making – user comments – is the coverage of Google scholar enough to be used alone for systematic reviews. 2013, http://www.biomedcentral.com/1472-6947/13/7/comments ,
- Cochrane Editorial Unit: Methodological expectations of Cochrane intervention reviews (MECIR). 2013, http://www.editorial-unit.cochrane.org/mecir ,
- Google Inc: About Google scholar. 2013, http://www.google.com/intl/en/scholar/about.html ,
- Jasco P: As we may search – comparison of major features of the Web of science, Scopus, and Google scholar citation-based and citation-enhanced databases. Curr Sci. 2005, 89: 1537-1547.
- The Cochrane Collaboration: The Cochrane Library. Independent high-quality evidence for health care decision making. 2013, http://www.thecochranelibrary.com/view/0/index.html ,
Cited by
Sorted by citations
Items 101–200 of 224
The Google Scholar h-index: useful but burdensome metric
da Silva, J. A. T. (2018) · Scientometrics · doi:10.1007/s11192-018-2859-7 · Cited by 13
“The citation of papers listed on GS may be higher than the citation counts for the same paper on WoS and Scopus (Trapp 2016).”
MethodInfluentialAn analysis of the fate of 917 manuscripts rejected from Clinical Otolaryngology
Earnshaw, C. H., Edwin, C., Bhat, J., Krishnan, M., Mamais, C., Somashekar, S., Sunil, A., Williams, S. P., & Leong, S. C. (2016) · Clinical Otolaryngology · doi:10.1111/coa.12820 · Cited by 13
Trends in Health Quality–Related Publications Over the Past Three Decades: Systematic Review
Mendlovic, J., Mimouni, F. B., Arad, I., & Heiman, E. (2022) · Interactive Journal of Medical Research · doi:10.2196/31055 · Cited by 13
“We repeated the search by each time using one limit according to publication types as classified by PubMed, and we noted the total number of publications per year for the 31 years of the specified period.”
BackgroundInfluentialOn the pitfalls of bibliographic database searching: comparing successful and less successful users
Rosman, T., Mayer, A., & Krampen, G. (2015) · Behaviour and Information Technology · doi:10.1080/0144929x.2015.1066446 · Cited by 12
“Functions like thesauri allow them to rely on controlled vocabulary, and searches can be connected with Boolean operators to ensure maximum search efficiency (Shiri and Revie 2005; Boeker, Vach, and Motschall 2013; Hjørland 2015).”
BackgroundBig Data, Accounting and International Development: Trends and challenges
Esteban, S. A., Grande, E. U., Silva, A. C. C. D. C., & Estébanez, R. P. (2022) · Cuadernos de Gestión · doi:10.5295/cdg.211513sa · Cited by 12
“…Scholar is an important tool, suitable for quick searches, first contact with a subject or limited analyses (Adriaanse & Rensleigh, 2011, 2013; Boeker et al., 2013), standing out for its major alternative sources that enrich and complement academic analysis, especially in Social Sciences…”
Aria, M., Cuccurullo, C., D’Aniello, L., & Spano, M. (2026) · Journal of Informetrics · doi:10.1016/j.joi.2026.101837 · Cited by 16
Tadpatrikar, A., Sharma, M. K., Amudhan, S., & Desai, G. (2024) · Indian Journal of Psychological Medicine · doi:10.1177/02537176241232110 · Cited by 13
“…the study did not include Google Scholar (which is primarily a search engine) due to its limitations in structured scientific literature retrieval, 68 this review was made more systematic and comprehensive by using robust academic databases like EBS-CO-host, PubMed, PsycInfo, PsycArticles, and…”
Digital twins for machine tools: a systematic mapping study
Chen, S., Ellwein, C., Klingel, L., Neumann, R., Zhang, J., Riedel, O., Verl, A. W., & Wortmann, A. (2025) · Digital Twin · doi:10.1080/27525783.2025.2538727 · Cited by 11
IoT-based drinking water quality measurement: systematic literature review
Carriazo-Regino, Y., Baena-Navarro, R., Torres-Hoyos, F., Vergara-Villadiego, J., & Prada, S. R. (2022) · Indonesian Journal of Electrical Engineering and Computer Science · doi:10.11591/ijeecs.v28.i1.pp405-418 · Cited by 11
Clinical Epidemiology and Evidence-Based Medicine
Hinchcliff, K. W. (2017) · Elsevier eBooks · doi:10.1016/b978-0-323-44329-6.00006-1 · Cited by 11
Coloring in the Links: Capturing Social Ties as They are PerceivedPreprint
Deri, S., Rappaz, J., Aiello, L. M., & Quercia, D. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1902.04528 · Cited by 11
“edinandreferencingby the seed papers. Independent from the initial seed set, we queried Google Scholar—which has been shown in prior studies to have comparable recall and precision to similar corpora [16, 18, 41, 96]—to get additional sources. We refined the search strategy iteratively to increase the recall of known…”
MethodAggrawal, A., Das, D., & Kalbar, P. P. (2025) · Journal of Environmental Management · doi:10.1016/j.jenvman.2025.127386 · Cited by 10
Eybye, M. N., Madsen, S. D., Schultz, A. N. Ø., & Nim, C. G. (2022) · Chiropractic & Manual Therapies · doi:10.1186/s12998-022-00468-8 · Cited by 11
Oloyede, A. A., Faruk, N., Noma, N., & Tebep, E. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4106167 · Cited by 10
Manuela, L. F. C., Giallanza, A., Scalia, G. L., & Micale, R. (2025) · The International Journal of Advanced Manufacturing Technology · doi:10.1007/s00170-025-16039-z · Cited by 11
“Other databases, such as Google Scholar, were excluded to reduce the risk of including unvalidated or lower quality sources, so ensuring the scientific robustness and repre-sentativeness of the results [10, 36].”
Klopfenstein, D. V., & Dampier, W. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1456 · Cited by 10
“Citations can not be downloaded in bulk in GS.”
InfluentialNagarajan, S. V., & McAllister, L. (2015) · Journal of Teaching and Learning for Graduate Employability · doi:10.21153/jtlge2015vol6no1art575 · Cited by 9
Literature review in biomedical research: useful search engines beyond PubMed
Cals, J., & Kotz, D. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.10.012 · Cited by 9
Buckton, S. J., Kenter, J. O., Mukherjee, N., Waddock, S. A., Anger-Kraavi, A., Martino, S., Fazey, I., Hejnowicz, A. P., Kabubo‐Mariara, J., Lafayette, J. O., et al. (2024) · Global Social Challenges Journal · doi:10.1332/27523349y2024d000000025 · Cited by 9
Google Scholar as a Resource for Systematic Reviews in Clinical Medicine
Falagas, M. E., Paliogianni, P. M., Kontogiannis, D. S., Ragias, D., & Johnson, E. O. (2025) · Journal of Evaluation in Clinical Practice · doi:10.1111/jep.70206 · Cited by 8
A new innovative method to measure the demographic representation of scientists via Google Scholar
Jozaghi, E. (2019) · Methodological Innovations · doi:10.1177/2059799119884273 · Cited by 8
“Furthermore, T.H.E. combines key factors, such as employment outcomes, teaching quality assessment data, graduation rates, and employment rates, which many other ranking systems do not consider (Boeker et al., 2013).”
BackgroundInfluentialIntracortical Hindlimb Brain–Computer Interface Systems: A Systematic Review
Ghodrati, M. T., Mirfathollahi, A., Shalchyan, V., & Daliri, M. R. (2023) · IEEE Access · doi:10.1109/access.2023.3258969 · Cited by 8
Azevedo, M. L., Giorgi, R., Nedel, F., & Lund, R. G. (2023) · PLoS ONE · doi:10.1371/journal.pone.0281913 · Cited by 8
“In this case, the Google Scholar database was not used due to the fact that it presents low precision in recently established themes, such as miRNAs [25, 26].”
MethodA beginner’s guide to the literature search in medical education
Martin, S., Hussain, Z., & Boyle, J. G. (2017) · Scottish Medical Journal · doi:10.1177/0036933017707163 · Cited by 8
“Scholar is not yet robust enough searching tool for projects where stringent methodologies are required, such as large systematic reviews.(7) Martin et al.”
MethodMartín‐Raugh, M. P., Williams, K. M., & Lentini, J. E. (2020) · ETS Research Report Series · doi:10.1002/ets2.12306 · Cited by 11
Requirements extraction from model-based systems engineering: A systematic literature review
Santos, J., Martins, L. E. G., & Molléri, J. S. (2025) · Journal of Systems and Software · doi:10.1016/j.jss.2025.112407 · Cited by 7
Ng, J. Y., Liu, H., Masood, M., Kochhar, J., Moher, D., Ehrlich, A. M., Iorio, A., & Cobey, K. D. (2024) · Scientific Reports · doi:10.1038/s41598-024-77790-z · Cited by 6
“…have found that 60–90% of clinicians use Google to find information to help them make point-of-care choices 6–8 and many researchers routinely search Google Scholar in conjunction with, and sometimes even as a replacement for other bibliographic databases, when conducting systematic reviews 9,10 .”
BackgroundRanking ‘by Relevance’ in Academic Literature Searches: Prevalence, Definitions, and Implications
Jordan, K., & Tsai, S. P. (2024) · Postdigital Science and Education · doi:10.1007/s42438-024-00530-z · Cited by 8
“Risks have been previously identified in relation to the use of Google Scholar for systematic reviews (Boeker et al. 2013; Gehanno et al. 2013; Giustini et al. 2013; Gusen-bauer and Haddaway 2020).”
Nishi, R., Fukumoto, T., & Asakawa, A. (2023) · Advances in Clinical and Experimental Medicine · doi:10.17219/acem/162537 · Cited by 7
Abusive Comments in Online Media and How to Fight Them
Niemann, M., Welsing, J., Riehle, D. M., Brunk, J., Assenmacher, D., & Becker, J. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61841-4_9 · Cited by 7
How reproducible and reliable is geophysical research?
Ireland, M., Algarabel, G., Steventon, M., & Munafo, M. R. (2023) · Seismica · doi:10.26443/seismica.v2i1.278 · Cited by 9
Oedema as a prognostic factor for seizures in meningioma - a systematic review and meta-analysis
Tanti, M., Nevitt, S. J., Yeo, M., Bolton, W. S., Chumas, P., Mathew, R. K., & Maguire, M. (2025) · Neurosurgical Review · doi:10.1007/s10143-025-03416-1 · Cited by 6
“Google scholar is discouraged in systematic reviews due to issues with storage and reproducibility [100].”
Fernández-Alías, A., Quispe-Becerra, J. I., Conde-Caño, M. R., Marcos, C., & Pérez‐Ruzafa, Á. (2024) · Estuarine Coastal and Shelf Science · doi:10.1016/j.ecss.2024.108699 · Cited by 7
Rothfus, M. A., SKETRIS, I. S., Traynor, R. L., Helwig, M., & Stewart, S. A. (2016) · Science & Technology Libraries · doi:10.1080/0194262x.2016.1192008 · Cited by 6
Google Scholar and the gray literature: A reply to Bonato's review
Orduña‐Malea, E., Martín‐Martín, A., & López‐Cózar, E. D. (2017) · LA Referencia (Red Federada de Repositorios Institucionales de Publicaciones Científicas) · doi:10.13140/rg.2.2.20703.87207 · Cited by 6
Saygitov, R. T. (2018) · bioRxiv (Cold Spring Harbor Laboratory) · doi:10.1101/354662 · Cited by 6
“publications from Google's “database” [40, 41].”
Examining Documentation Tools for Audit and Forensic Accounting Investigations
Smith, K. T., & Smith, L. M. (2024) · Journal of risk and financial management · doi:10.3390/jrfm17110491 · Cited by 13
Fumigation on bulk cargo ships: a chemical threat to seafarers
Djurhuus, R. (2021) · International Maritime Health · doi:10.5603/imh.2021.0039 · Cited by 5
“For a comprehensive evaluation of Google Scholar as a search engine for literature, see [6].”
BackgroundComposition operators for modeling languages: A literature review
Pfeiffer, J., Rumpe⋆, B., Schmalzing, D., & Wortmann, A. (2023) · Journal of Computer Languages · doi:10.1016/j.cola.2023.101226 · Cited by 7
Dittus, M., Aiello, L. M., & Quercia, D. (2017) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3134674 · Cited by 5
“Prior reviews of behavioural and medical studies showed that it has comparable recall and precision to domain-specific corpuses [7, 8, 14, 43].”
BackgroundCortés-Sánchez, J. D., & Andrade, D. A. (2022) · Humanities and Social Sciences Communications · doi:10.1057/s41599-022-01241-1 · Cited by 4
“Search engines such as Google Scholar show limitations in search interface and query optimisation, exporting many cited references, and a visual search builder (Boeker et al., 2013).”
MethodUniversity students' knowledge of potentially predatory journals: A focus group analysis
Schira, R., & Hurst, C. (2024) · The Journal of Academic Librarianship · doi:10.1016/j.acalib.2024.102926 · Cited by 5
Llopis, P., Oviedo-Trespalacios, Ó., Alonso, F., Faus, M., & Useche, S. A. (2025) · Sustainable Futures · doi:10.1016/j.sftr.2025.100982 · Cited by 7
University Research Graph Database For Efficient Multi-Perspective Data Analysis Using Neo4j
Afandi, M. I., & Wahyuni, E. D. (2020) · IEEE Information Technology International Seminar (ITIS) · doi:10.1109/itis50118.2020.9320965 · Cited by 4
“They present that Google Scholar is widely used globally due to its high recall, free access, and broad coverage [12][13].”
BackgroundComparative Analysis of Sentence Transformers for Reference Paper Collection in Five Academic Fields
Fahrudin, T. M., Funabiki, N., Brata, K. C., Noprianto, N., Muhaimin, A., & Hindrayani, K. M. (2025) · International Conference on Computational Intelligence and Intelligent Systems (CIIS) · doi:10.1145/3787256.3787277 · Cited by 3
“The limitations in handling queries containing domain-specific vocabulary, unreliable automatic stemming, and struggles in capturing the topic’s semantic context reduce the relevance of retrieved results [1][8].”
Bibliometric Analysis on ChatGPT in Tourism, Hospitality, and Gastronomy: Papers Published in 2023
Zelený, J., Kalabisová, J., Stupková, L. C., Melas, D., & Turnerová, L. (2024) · Lecture notes in networks and systems · doi:10.1007/978-3-031-74828-8_30 · Cited by 3
Designing ontology-based search systems for research articles
Huettemann, S., Mueller, R. M., & Dinter, B. (2025) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2025.102901 · Cited by 5
State-owned enterprises and corporate scandals: a systematic literature review
Lopes, C. F., Ferreira, A., & Ferreira, C. (2025) · International Journal of Ethics and Systems · doi:10.1108/ijoes-07-2024-0221 · Cited by 6
Borycz, J., Carroll, A. J., & Eskridge, H. N. (2021) · Journal of eScience Librarianship · doi:10.7191/jeslib.2021.1192 · Cited by 4
“…does not offer reliable or stable search results over time and place, does not allow large search results to be exported in other data formats, does not allow for thorough search strategy documentation, and only includes items that have been indexed online (Boeker, Vach, and Motschall 2013).”
BackgroundBhardwaj, R. K., Kumar, R., & Nazim, M. (2024) · Performance Measurement and Metrics · doi:10.1108/pmm-09-2023-0028 · Cited by 4
Kanungo, P., Mohanty, S., & Thamminaina, A. (2024) · Cambridge Journal of Education · doi:10.1080/0305764x.2024.2358812 · Cited by 3
“Google Scholar is a rarely used search interface ue to its limited capabilities and downloading restrictions, despite its broad coverage of articles (Boeker et al., 2013; Falagas et al., 2008; Jacsó, 2008).”
BackgroundJanssen, J., Ineich, I., Shacham, B., Vasconcelos, R., & Kaiser, H. (2024) · Journal of Ethnobiology · doi:10.1177/02780771241261224 · Cited by 6
“” Although the use of Google Scholar as the sole source for systematic reviews has faced criticism (Boeker, Vach, and Motschall 2013), its effectiveness in searching grey literature is well-documented (Haddaway, Collins, and Coughlin 2015).”
Investigating school ground vegetation research: A systematic mapping review
Ignell, S., Wiström, B., Levinsson, A., & Jansson, M. (2024) · Urban forestry & urban greening · doi:10.1016/j.ufug.2024.128494 · Cited by 3
Briscoe, S. (2016) · Evidence Based Library and Information Practice · doi:10.18438/b8f049 · Cited by 3
Eaton, C., Banks, A., Weger, K., Mesmer, B. L., & Moreland, R. (2022) · Systems Research and Behavioral Science · doi:10.1002/sres.2883 · Cited by 3
Keller, C., Heck, T., & Rittberger, M. (2022) · ACM/IEEE Joint Conference on Digital Libraries · doi:10.1145/3529372.3530933 · Cited by 5
“With regard to the coverage of reflected in the databases, GS ranks first, but might not be useful for systematic review searches in practice as the precision rate is too low [17].”
BackgroundInfluentialSelf-Tracking Technology for mHealth: A Systematic Review and the PAST SELF Framework.Preprint
Yfantidou, S., Sermpezis, P., & Vakali, A. I. (2021) · arXiv (Cornell University) · Cited by 3
Examining Trends in Homelessness Research: A Literature Review
Dagsoy, R., & Abbas, A. E. (2025) · Sociology Compass · doi:10.1111/soc4.70098 · Cited by 2
Employee Delight: Conceptualization, Antecedents, and Consequences
Rivera, D. E., Manresa, A., & Gilabert, E. R. (2025) · Human Resource Development Quarterly · doi:10.1002/hrdq.70001 · Cited by 2
Rebuttal to Winn et al. (2024) ’s “Redefining searching in non-medical sciences systematic reviews”
Lipke, L., Monnin, J., & Fonseca, C. (2025) · Journal of Librarianship and Information Science · doi:10.1177/09610006251387675 · Cited by 2
Ellwein, C., Dietrich, D. M., Maisch, N., Neumann, R., Ajdinović, S., Lechler, A., & Wortmann, A. (2025) · Production Engineering · doi:10.1007/s11740-025-01378-3 · Cited by 2
“We decided to omit Google Scholar due to its limitations in retrieving structured literature, [30], and to ensure the quality of the included sources.”
Finding a place for Kindness within higher education: A systematic literature review
Fox, J. L., & Aspland, T. (2024) · Journal of Applied Learning & Teaching · doi:10.37074/jalt.2024.7.s1.6 · Cited by 3
A avifauna brasileira na educação ambiental: um recorte a partir de uma revisão sistemática
Dourado, R. D. C. S., da Cruz, E. P., Lins, L. D., Leão, G. D. J., dos Santos, J. P. V., de Oliveira, I. F., de Souza, M. R. R., & Reis, Q. T. D. S. (2024) · Caderno Pedagógico · doi:10.54033/cadpedv21n7-208 · Cited by 2
Multi-criteria Decision-Making methods to evaluate urban environmental quality: a scoping review
Pimenta, L. B., Beltrão, N. E. S., de Oliveira, R. M. E. S., & Gomes, D. J. C. (2024) · Concilium · doi:10.53660/clm-3931-24q44 · Cited by 2
Literature Searches in Medical Informatics Systematic Reviews: Suggested Approaches
Afraz, A., Chashmyazdan, M., Khajouei, R., & Bagherinezhad, Z. (2024) · Medical Reference Services Quarterly · doi:10.1080/02763869.2024.2429066 · Cited by 2
“30 Due to these factors, we think Scopus is a suitable alternative to Google Scholar.”
A Replication of Failure, Not a Failure to Replicate
Holden, G. W., Barker, K., Kuppens, S., Rosenberg, G., & LeBreton, J. (2014) · Research on Social Work Practice · doi:10.1177/1049731514530000 · Cited by 2
O'Connor, S. R., Dempster, M., & McCorry, N. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0489-4 · Cited by 2
“The search strategy will also use previous recommendations, which are based on a feasible and robust method of applying a systematic search to identify grey literature [15].”
MethodSturm, B., & Sunyaev, A. (2017) · Arrow@dit (Dublin Institute of Technology) · Cited by 2
Huang, M. (2022) · The Science of The Total Environment · doi:10.1016/j.scitotenv.2022.155624 · Cited by 2
A Study on China’s TESOL Pre-Service Teachers’ Knowledge and Attitudes on Gamification
Zhu, J. (2022) · Advances in Social Science, Education and Humanities Research/Advances in social science, education and humanities research · doi:10.2991/assehr.k.220504.527 · Cited by 1
Conflicting Perceptions of Project Success and Failure Relationships: A Systematic Literature Review
Eaton, C., Glandon, K. L., Teper, T., Mesmer, B. L., & Ferreira, S. (2025) · Engineering Management Journal · doi:10.1080/10429247.2025.2492921 · Cited by 1
“While there has been some historical debate on Google Scholar’s coverage (Boeker et al., 2013; De Winter et al., 2014; Giustini & Boulos, 2013), subsequent studies have concluded it is a powerful search tool for systematic reviews with superior coverage (Gusenbauer, 2019; Haddaway et al., 2015;…”
Evaluating AI toolchains for systematic literature reviews using a rubric-based workflow approach
Biju, N., Sinha, A., Saini, S., Beniwal, G., & Das, S. (2025) · Discover Artificial Intelligence · doi:10.1007/s44163-025-00628-8 · Cited by 1
The Development and Drivers of the Impact Advantages of Open Access Research
Taylor, M. (2026) · Goodfellow Publishers Ltd eBooks · doi:10.23912/open_access_implications · Cited by 1
Kulp, C. M. (2014) · Science & Technology Libraries · doi:10.1080/0194262x.2014.912183 · Cited by 1
Stephani, V., Quentin, W., & Opoku, D. (2015) · DepositOnce · doi:10.14279/depositonce-4373 · Cited by 1
“These and other limitations of Google-scholar as a replacement for systematic literature searches were also mentioned by (Boeker et al. 2013).”
Finding the Evidence for Practice in Social Work
Cargill, J. (2014) · Advances in human services and public health (AHSPH) book series · doi:10.4018/978-1-4666-6563-7.ch002 · Cited by 1
Managing volume in discovery systems
Tay, A. (2018) · Facet eBooks · doi:10.29085/9781783301546.005 · Cited by 1
“Boeker, Vach, and Motschall (2013) suggested that Google scholar queries inherently had low precision compared to databases, due to the lack of advanced search features.”
Species synonyms: How important are they for the retrieval of ethnobotanical information?
Perdomo, O., & Singer, R. B. (2020) · Ethnobotany Research and Applications · doi:10.32859/era.20.28.1-18 · Cited by 1
Epstein, R. H., Mueller, D. A., Walco, J. P., Manresa, C. D., Banks, S. E., & Freundlich, R. E. (2023) · Cureus · doi:10.7759/cureus.47976 · Cited by 1
“We did not use Google Scholar (Alphabet, Mountain View, CA) because of search string length limitations (256 characters), nonsupport of complex search expressions, absence of search export functions, and potential blocking issues related to multiple searches from a single IP address [3].”
MethodGoogle Scholar in Systematic Reviews
Winn, J. G., Hao, T., Hardaway, J. W., & Oh, H. Y. (2026) · Journal of Librarianship and Information Science · doi:10.1177/09610006261445270 · Cited by 1
Tips and tricks for using the internet for professional purposes
Ceylan, H. H., Güngören, N., & Küçükdurmaz, F. (2017) · EFORT Open Reviews · doi:10.1302/2058-5241.2.160066 · Cited by 1
Comparing source coverage, citation counts and speed of indexing in Google Scholar and Scopus.Preprint
Moed, H. F., Bar‐Ilan, J., & Halevi, G. (2015) · arXiv (Cornell University) · Cited by 1
Focused Search of the Literature
Marcos, F. Á. (2019) · European Journal of Vascular and Endovascular Surgery · doi:10.1016/j.ejvs.2019.03.038 · Cited by 1
Ireland, M., Algarabel, G., Steventon, M., & Munafo, M. R. (2022) · doi:10.31223/x5w943 · Cited by 1
Family Business Research in Bangladesh: A Scoping Review and Direction for Research OpportunitiesPreprint
Khaled, M. C., Sultana, T., & Tabassum, T. (2023) · Preprints.org · doi:10.20944/preprints202311.0572.v1 · Cited by 1
A dataset of pre-pandemic African protected area visitation
Buschke, F., Capitani, C., Schägner, J. P., Nsengiyumva, C., Okelo, H., Cissé, O., & Snyman, S. (2025) · Scientific Data · doi:10.1038/s41597-025-04998-7 · Cited by 1
Efenakpo, O. D., Nneji, L. M., Jacob, D. E., Ringim, A. S., Eniang, E. A., Petrozzi, F., Ikegwu, C. M., & Luiselli, L. M. (2026) · Biodiversity · doi:10.1080/14888386.2026.2646915 · Cited by 1
“Under such circumstances – as in the present study – comprehensive manual screening of all retrieved records is impractical (Boeker, Vach, and Motschall 2013).”
Buis, J. (2015)
Fraser, M. (2016) · UWE Research Repository (UWE Bristol)
How to carry out an efficient literature search
Tichter, A. M., Falzon, L., & Wyer, P. C. (2015) · doi:10.1002/9781118643440.ch5
Many Paths to Discovery: The Increasingly Complex Literature Maze
Ramsay, L., & Williamson, M. J. (2016) · Springer eBooks · doi:10.1007/978-3-319-39853-2_11
Barros‐Justo, J. L., Sepúlveda, S., Martínez-Araujo, N., & González-García, A. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1704.00822
“We ran these searches in the most often used Electronic Data Sources (EDS): ACM Digital Library 1 , IEEE Xplore 2 , ISI Web of Science 3 and SCOPUS 4 .”
MethodInfluentialMawdsley, H. (2018) · Scholarship@Western (Western University)
Trouilloud, K. T., Sanlaville, N., Yvars, S., & Savey, A. (2019) · Evidence Based Library and Information Practice · doi:10.18438/eblip29448
Murillo, D., & Fernandez, M. (2019) · International Engineering, Sciences and Technology Conference (IESTEC) · doi:10.1109/iestec46403.2019.00127
“…diseminación y uso de la información registrada, desarrollando modelos y medidas matemáticas; es una herramienta útil para determinar patrones de publicación y como puede ayudar al diseño de estrategias que redunden en la visibilidad internacional de las investigaciones científicas [7], [8].”
BackgroundSTIs among YMSM in Canada: Implications for Public Health Policies, Programs and Activities
Foley, S. N. (2014) · uO Research (University of Ottawa)
Detection of Malicious Web Pages Using Machine Learning Technique
(2020) · International Journal of Advanced Trends in Computer Science and Engineering · doi:10.30534/ijatcse/2020/243952020
Following the Trail of eSports
Vera, J. A. C., Terrón, J. M. A., & García, S. G. (2021) · IGI Global eBooks · doi:10.4018/978-1-7998-7707-3.ch025
Abat, M. E. M., Mantaring, J. B. V., & Dans, A. L. (2016) · Wiley eBooks · doi:10.1002/9781119196150.ch9
Metadata
- Authors
- Boeker, M., Vach, W., & Motschall, E.
- Affiliations
- University Medical Center Freiburg (DE)
- Corresponding author
- Boeker, Martin
- Venue
- BMC Medical Research Methodology
- Published
- 2013-10-26
- Volume
- 13
- Issue
- 1
- Pages
- 131-131
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC3840556
- OpenAlex
- W2073728848
- Cited by
- 225
- References
- 58
- Citations by year
- 2026: 17 · 2025: 22 · 2024: 27 · 2023: 22 · 2022: 22 · 2021: 10 · 2020: 16 · 2019: 19 · 2018: 16 · 2017: 16 · 2016: 15 · 2015: 13 · 2014: 8 · 2013: 1
- FWCI
- 13.59
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Medical Research Methodology (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- FreiDok plus (Universitätsbibliothek Freiburg) (submittedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Computer Science › Information Systems › Information Retrieval and Search Behavior
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Keywords
- Google Scholar, systematic search, search recall, search precision, search strategies, MEDLINE
- MeSH terms
- Abstracting and Indexing, Algorithms, Humans, Review Literature as Topic, MEDLINE, Internet, Data Mining, Search Engine
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
