The DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
The contribution of databases to the results of systematic reviews: a cross-sectional study
Open accessAbstract
BACKGROUND: One of the best sources for high quality information about healthcare interventions is a systematic review. A well-conducted systematic review includes a comprehensive literature search. There is limited empiric evidence to guide the extent of searching, in particular the number of electronic databases that should be searched. We conducted a cross-sectional quantitative analysis to examine the potential impact of selective database searching on results of meta-analyses. METHODS: Our sample included systematic reviews (SRs) with at least one meta-analysis from three Cochrane Review Groups: Acute Respiratory Infections (ARI), Infectious Diseases (ID), Developmental Psychosocial and Learning Problems (DPLP) (n = 129). Outcomes included: 1) proportion of relevant studies indexed in each of 10 databases; and 2) changes in results and statistical significance of primary meta-analysis for studies identified in Medline only and in Medline plus each of the other databases. RESULTS: Due to variation across topics, we present results by group (ARI n = 57, ID n = 38, DPLP n = 34). For ARI, identification of relevant studies was highest for Medline (85 %) and Embase (80 %). Restricting meta-analyses to trials that appeared in Medline + Embase yielded fewest changes in statistical significance: 53/55 meta-analyses showed no change. Point estimates changed in 12 cases; in 7 the change was less than 20 %. For ID, yield was highest for Medline (92 %), Embase (81 %), and BIOSIS (67 %). Restricting meta-analyses to trials that appeared in Medline + BIOSIS yielded fewest changes with 1 meta-analysis changing in statistical significance. Point estimates changed in 8 of 31 meta-analyses; change less than 20 % in all cases. For DPLP, identification of relevant studies was highest for Medline (75 %) and Embase (62 %). Restricting meta-analyses to trials that appeared in Medline + PsycINFO resulted in only one change in significance. Point estimates changed for 13 of 33 meta-analyses; less than 20 % in 9 cases. CONCLUSIONS: Majority of relevant studies can be found within a limited number of databases. Results of meta-analyses based on the majority of studies did not differ in most cases. There were very few cases of changes in statistical significance. Effect estimates changed in a minority of meta-analyses but in most the change was small. Results did not change in a systematic manner (i.e., regularly over- or underestimating treatment effects), suggesting that selective searching may not introduce bias in terms of effect estimates.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Pooled effect estimates of both extraction methods varied only slightly and were not statistically significant different from the reference standard in any of the meta-analyses ( N = 6).”
From Frequency of data extraction errors and methods to increase data extraction quality: a methodological review (2017) · cites this paper as [18]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“For example, a study on the influence of searching multiple databases found only a weak impact of reducing the number of searched databases (because of not searching additional databases) on the results of meta-analysis and review conclusions [ 18 , 25 ].”
From Frequency of data extraction errors and methods to increase data extraction quality: a methodological review (2017) · cites this paper as [18]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“However, in average double data extraction took 49 min longer.”
From Frequency of data extraction errors and methods to increase data extraction quality: a methodological review (2017) · cites this paper as [18]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“The risk difference of the total error rate was statistically significant (risk difference 21.7%, p = 0.019) in favor of double data extraction.”
From Frequency of data extraction errors and methods to increase data extraction quality: a methodological review (2017) · cites this paper as [18]
Experimental — accuracy not yet measuredThe check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“The agreement rate of the two extraction methods was 28%.”
From Frequency of data extraction errors and methods to increase data extraction quality: a methodological review (2017) · cites this paper as [18]
Experimental — accuracy not yet measured
2 more claims about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A scoping review of rapid review methods
Tricco, A. C., Antony, J., Zarin, W., Strifler, L., Ghassemi, M., Ivory, J., Perrier, L., Hutton, B., Moher, D., & Straus, S. E. (2015) · BMC Medicine · doi:10.1186/s12916-015-0465-6 · Cited by 1,146 · cited 1×
“Searching is one dimension of SRs that is typically altered to streamline processes and produce reviews more quickly [37, 38].”
BackgroundEpidemiology and reporting characteristics of systematic reviews.
Moher, D., Tetzlaff, J., Tricco, A. C., Sampson, M., & Altman, D. G. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040078 · Cited by 891 · cited 1×
“The CDSR was chosen for this analysis as Cochrane reviews: 1) provide tabulated data from the component trials allowing for re-analysis; 2) provide a detailed list of references for all relevant trials; and, 3) have been reported to be of higher methodological quality [26, 27], which may translate into more comprehens…”
MethodPage, M. J., Shamseer, L., Altman, D. G., Tetzlaff, J., Sampson, M., Tricco, A. C., Catalá-López, F., Li, L., Reid, E. K., Sarkis-Onofre, R., et al. (2016) · PLOS Medicine · doi:10.1371/journal.pmed.1002028 · Cited by 768 · cited 1×
“The CDSR was chosen for this analysis as Cochrane reviews: 1) provide tabulated data from the component trials allowing for re-analysis; 2) provide a detailed list of references for all relevant trials; and, 3) have been reported to be of higher methodological quality [26, 27], which may translate into more comprehens…”
MethodExpediting systematic reviews: methods and implications of rapid reviews
Ganann, R., Ciliska, D., & Thomas, H. (2010) · Implementation Science · doi:10.1186/1748-5908-5-56 · Cited by 752
Royle, P., & Milne, R. (2003) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462303000552 · Cited by 693 · cited 1×
“, cited reference searching) may yield more relevant studies than more database searching [6, 10, 13, 19].”
BackgroundMusic therapy for people with autism spectrum disorder
Geretsegger, M., Elefant, C., Mössler, K. A., & Gold, C. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004381.pub3 · Cited by 422 · cited 1×
“In two cases the original meta-analyses had 2 included studies and Medline + PsycInfo only identified 1 (50 %); topics were music therapy for autism spectrum disorder and financial benefits for child health and well-being in low income or socially disadvantaged families in developed world countries [32, 33].”
BackgroundKnowledge translation is the use of knowledge in health care decision making
Straus, S. E., Tetroe, J. M., & Graham, I. D. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.08.016 · Cited by 383 · cited 1×
“They have been described as the cornerstone of knowledge translation and are the foundation for key knowledge tools such as clinical practice guidelines and patient decision aids [34].”
BackgroundRoyle, P., & Waugh, N. (2003) · Health Technology Assessment · doi:10.3310/hta7340 · Cited by 194 · cited 2×
“While methodological guidance for SRs encourages comprehensive searching, there are diminishing returns with each additional database searched [7] and the impact of searching each additional database in terms of the final results and conclusions is not known.”
BackgroundResultIdentifying Clinical Trials in the Medical Literature with Electronic Databases
Suarez-Almazor, M. E., Belseck, E., Homik, J., Dorgan, M., & Ramos-Remus, C. (2000) · Controlled Clinical Trials · doi:10.1016/s0197-2456(00)00067-2 · Cited by 180
Halladay, C. W., Trikalinos, T. A., Schmid, I. T., Schmid, C. H., & Dahabreh, I. J. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.12.017 · Cited by 166 · cited 2×
“This study confirms previous methods studies demonstrating that the majority of SR trials are found in Medline [7, 25], and that Embase is more likely than other databases to find additional relevant trials not retrieved by Medline [23, 24].”
Result
Research works printed but not linked: 2
- Lefebvre C, Manheimer E, Glanville J. Chapter 6: Searching for studies. In: Higgins JP, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [Internet]. The Cochrane Collaboration; 2011 [cited 2016 Jun 7]. Available from: http://handbook.cochrane.org/ .
- Institute of Medicine. Finding what works in health care: standards for systematic reviews [Internet]. Washington, DC: The National Academies Press; 2011 [cited 2016 Jun 7]. Available from: http://www.nationalacademies.org/hmd/Reports/2011/Finding-What-Works-in-Health-Care-Standards-for-Systematic-Reviews.aspx .
Cited by
Bramer, W. M., Rethlefsen, M. L., Kleijnen, J., & Franco, O. H. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0644-y · Cited by 2,114
“Most articles on this topic draw their conclusions based on the coverage of databases [14].”
BackgroundCooper, C., Booth, A., Varley-Campbell, J., Britten, N., & Garside, R. (2018) · BMC Medical Research Methodology · doi:10.1186/s12874-018-0545-3 · Cited by 561
Searching for and selecting studies
Lefebvre, C., Glanville, J., Briscoe, S., Littlewood, A., Marshall, C., Metzendorf, M., Noel‐Storr, A., Rader, T., Shokraneh, F., Thomas, J., et al. (2019) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9781119536604.ch4 · Cited by 647
Hartling, L., Featherstone, R., Nuspl, M., Shave, K., Dryden, D. M., & Vandermeer, B. (2017) · BMC Medical Research Methodology · doi:10.1186/s12874-017-0347-z · Cited by 324
“The rationale for choosing reviews from the CDSR for this analysis is provided in our previous publication [23].”
BackgroundMethodResultInfluentialSelf-citationOptimal literature search for systematic reviews in surgery
Goossen, K., Tenckhoff, S., Probst, P., Grummich, K., Mihaljevic, A. L., Büchler, M. W., & Diener, M. K. (2018) · Langenbeck's Archives of Surgery · doi:10.1007/s00423-017-1646-x · Cited by 248
Metadata
- Authors
- Hartling, L., Featherstone, R., Nuspl, M., Shave, K., Dryden, D. M., & Vandermeer, B.
- Affiliations
- University of Alberta (CA)
- Corresponding author
- Hartling, Lisa
- Venue
- BMC Medical Research Methodology
- Published
- 2016-09-26
- Volume
- 16
- Issue
- 1
- Pages
- 127-127
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC5037618
- OpenAlex
- W2524295027
- Cited by
- 118
- References
- 39
- Citations by year
- 2026: 13 · 2025: 9 · 2024: 8 · 2023: 16 · 2022: 15 · 2021: 20 · 2020: 14 · 2019: 6 · 2018: 6 · 2017: 10
- FWCI
- 2.52
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Medical Research Methodology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Canadian Institutes of Health Research (New Investigator, CON-105529)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Health Sciences › Health Professions › General Health Professions › Health Sciences Research and Education
- Keywords
- systematic reviews, electronic database searching, literature search comprehensiveness, MEDLINE, Embase, BIOSIS, PsycINFO, statistical significance, acute respiratory infections, infectious diseases, Cochrane Review Groups, study identification, effect estimates, search strategy optimization
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The contribution of databases to the results of systematic reviews: a cross-sectional study. https://app.cochat.ai/papers/10.1186/s12874-016-0232-1
- The paper
- Hartling, L., Featherstone, R., Nuspl, M., Shave, K., Dryden, D. M., & Vandermeer, B. (2016). The contribution of databases to the results of systematic reviews: a cross-sectional study. BMC Medical Research Methodology. https://doi.org/10.1186/s12874-016-0232-1
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.