Where you search determines what you find: the effects of bibliographic databases on systematic reviews
Open accessThe DOI matches its Crossref record; 14 of the 14 references it cites were found in a registry or an index.
Abstract
Systematic literature reviews are common in social research for integrating and synthesising existing research. This paper argues that the outcomes of such reviews are affected by the choice of bibliographic databases. It presents evidence of substantial variation across three large electronic databases (Scopus, Web of Science and EBSCO) in a study on employee retention and staff turnover. It considers the specific articles, numbers returned, numbers shared across databases and perceived quality of journals hosting the retrieved articles. Results show that only 130 articles (5.7% of 2267 retrieved) were found common to all three databases, suggesting that decisions on how and where literature is retrieved can substantially affect the results of systematic reviews and meta-analyses. The findings caution against the use of single databases and claiming comprehensiveness. The paper reflects on how additional literature search methods (e.g., contacting experts, citation indices) and their sequence of use can affect systematic review quality.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
PRISMA-P Group, Moher, D., Shamseer, L., Clarke, M., Ghersi, D., Liberati, A., Petticrew, M., Shekelle, P., & Stewart, L. A. (2015) · Systematic Reviews · doi:10.1186/2046-4053-4-1 · Cited by 26,223
Saturation in qualitative research: exploring its conceptualization and operationalization
Saunders, B., Sim, J., Kingstone, T., Baker, S., Waterfield, J., Bartlam, B., Burroughs, H., & Jinks, C. (2018) · Quality & Quantity · doi:10.1007/s11135-017-0574-8 · Cited by 12,033 · cited 2×
“when further search ceases to yield anything substantially different (Saunders et al., 2018; Webster & Watson, 2002).”
Literature review as a research methodology: An overview and guidelines
Snyder, H. (2019) · Journal of Business Research · doi:10.1016/j.jbusres.2019.07.039 · Cited by 11,524
A three-component conceptualization of organizational commitment
Meyer, J. P., & Allen, N. J. (1991) · Human Resource Management Review · doi:10.1016/1053-4822(91)90011-z · Cited by 11,049 · cited 1×
“This multi-method approach to the literature search revealed that some seminal works (e.g., Mathieu and Zajac 1990; Meyer and Allen 1991; Schaufeli et al. 2002) relate to the thematic scope of our study, exemplified by organizational commitment and employee retention being omitted from all three…”
The measurement of engagement and burnout: a two sample confirmatory factor analytic approach.
Schaufeli, W. B., Salanova, M., González-romá, V., & Bakker, A. B. (2002) · Journal of Happiness Studies · doi:10.1023/a:1015630930326 · Cited by 10,014 · cited 2×
“This multi-method approach to the literature search revealed that some seminal works (e.g., Mathieu and Zajac 1990; Meyer and Allen 1991; Schaufeli et al. 2002) relate to the thematic scope of our study, exemplified by organizational commitment and employee retention being omitted from all three…”
Mathieu, J. E., & Zajac, D. M. (1990) · Psychological Bulletin · doi:10.1037//0033-2909.108.2.171 · Cited by 6,449
The journal coverage of Web of Science and Scopus: A comparative analysis
Mongeon, P., & Paul-Hus, A. (2016) · Scientometrics · doi:10.1007/s11192-015-1765-5 · Cited by 5,274
Comparison of PubMed, Scopus, Web of Science, and Google Scholar: strengths and weaknesses
Falagas, M. E., Pitsouni, E. I., Malietzis, G. A., & Pappas, G. (2008) · The FASEB Journal · doi:10.1096/fj.07-9492lsf · Cited by 5,039
Systematic Reviews in the Social Sciences: A Practical Guide
Petticrew, M., & Roberts, H. (2006) · doi:10.1002/9780470754887 · Cited by 3,586
Siddaway, A. P., Wood, A. M., & Hedges, L. V. (2019) · Annual Review of Psychology · doi:10.1146/annurev-psych-010418-102803 · Cited by 2,550 · cited 2×
“Whereas there are many literature search sources including physical libraries, contacting subject experts and reference checks, databases are often used in preference to other search methods (Green et al., 2006; Siddaway et al., 2019) due to their ability to access literature from a wide range of”
Cited by
A systematic literature review on network marketing: What do we know and where should we be heading?
Chopra, I. P., Jebarajakirthy, C., Acharyya, M., Saha, R., Maseeh, H. I., & Nahar, S. (2023) · Industrial Marketing Management · doi:10.1016/j.indmarman.2023.05.010 · Cited by 37
Bui, H. T., & Filimonau, V. (2021) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-03-2021-0330 · Cited by 28
Yousef, K. (2026) · Journal of Global Mobility · doi:10.1108/jgm-04-2025-0044 · Cited by 2
Metadata
- Authors
- Wanyama, S. B., McQuaid, R. W., & Kittler, M.
- Venue
- International Journal of Social Research Methodology
- Publisher
- Informa UK Limited
- Published
- 2021-03-05
- Volume
- 25
- Issue
- 3
- Pages
- 409-422
- ISSN
- 1364-5579 (ISSN-L)
- 1464-5300
- OpenAlex
- W3135876726
- Cited by
- 101
- References
- 14
- FWCI
- 0.43
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Where you search determines what you find: the effects of bibliographic databases on systematic reviews. https://app.cochat.ai/papers/10.1080/13645579.2021.1892378
- The paper
- Wanyama, S. B., McQuaid, R. W., & Kittler, M. (2022). Where you search determines what you find: the effects of bibliographic databases on systematic reviews. International Journal of Social Research Methodology. https://doi.org/10.1080/13645579.2021.1892378
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.