Evaluation of the usefulness of Internet searches to identify unpublished clinical trials for systematic reviews
Open accessThe DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Abstract
PRIMARY OBJECTIVE: To avoid selection and publication bias, systematic reviewers should employ a broad range of search techniques and make efforts to locate unpublished studies. We tried to establish whether searches on the World Wide Web (WWW) are useful to identify additional unpublished and ongoing clinical trials. RESEARCH DESIGN: Search strategies seven Cochrane systematic reviews were retrospectively adapted for the WWW in an attempt to find additional randomized controlled trials. METHODS AND PROCEDURES: A search strategy with the general pattern 'study methodology NEAR intervention NEAR condition' for the Internet search engine AltaVista was evaluated by measuring search time, recall of Internet searches for published studies; precision (proportion of webpages containing hints to relevant published and unpublished randomized clinical trials); number of additional unpublished or ongoing studies found on the Internet. MAIN OUTCOMES AND RESULTS: We reviewed 429 webpages in 21 hours and found hints to 14 unpublished, ongoing or recently finished trials, at least 9 were considered relevant for 4 systematic reviews. The recall of Internet searches to find references to published studies ranged between 0% and 43.6%, the precision for hints to published or unpublished studies range between 0% and 20.2%. CONCLUSIONS: Information on unpublished and particularly ongoing trials can be found on the Internet. A potential problem is the appraisal of non-peer reviewed electronic publications with questionable quality. More powerful search tools are needed. An 'Open Trial Initiative' is proposed to define a syntax for publishing trials on the web and to ensure interoperability of trial registers, so that special search engines can harvest information on ongoing and complete clinical trials.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The role of google scholar in evidence reviews and its applicability to grey literature searching
Haddaway, N. R., Collins, A. M., Coughlin, D., & Kirk, S. (2015) · PLOS ONE · doi:10.1371/journal.pone.0138237 · Cited by 2,387
Empirical assessment of effect of publication bias on meta-analyses
Sutton, A. J. (2000) · BMJ · doi:10.1136/bmj.320.7249.1574 · Cited by 1,398
meta-analysis bias in location and selection of studies
Egger, M., & Smith, G. D. (1998) · BMJ · doi:10.1136/bmj.316.7124.61 · Cited by 1,374
Full publication of results initially presented in abstracts.
Scherer, R. W., Langenberg, P., & von Elm, E. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000005.pub3 · Cited by 997
Jenkinson, C., Davies, R. J., Mullins, R., & Stradling, J. R. (1999) · The Lancet · doi:10.1016/s0140-6736(98)10532-9 · Cited by 826
Publication bias: the case for an international registry of clinical trials.
Simes, R. J. (1986) · Journal of Clinical Oncology · doi:10.1200/jco.1986.4.10.1529 · Cited by 747
CHALMERS, I. (1993) · Annals of the New York Academy of Sciences · doi:10.1111/j.1749-6632.1993.tb26345.x · Cited by 561
Eysenbach, G., Gray, J. A. M., Bonati, M., Arunachalam, S., Diepgen, T. L., Impicciatore, P., & Pandolfini, C. (1998) · BMJ · doi:10.1136/bmj.317.7171.1496 · Cited by 461 · cited 1×
“Experiences from searching the Internet to answer clinical questions [16] and considerations regarding quality of Internet information [17, 18] raise the question whether the signal-to-noise ratio of information on the World Wide Web (WWW) is good enough to use it as a serious research tool.”
BackgroundIs the coverage of google scholar enough to be used alone for systematic reviews
Gehanno, J.-F., Rollin, L., & Darmoni, S. (2013) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-13-7 · Cited by 357
Rethlefsen, M. L., Farrell, A. M., Osterhaus Trzasko, L. C., & Brigham, T. J. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.11.025 · Cited by 331
Cited by
The Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553
Dissemination and publication of research findings: an updated review of related biases
Song, F., Parekh, Hooper, L., Loke, Y., Ryder, J., Sutton, A., Hing, C., Kwok, C., Pang, C., & Harvey, I. (2010) · Health Technology Assessment · doi:10.3310/hta14080 · Cited by 1,039
Howe, C., Suich, H., Vira, B., & Mace, G. M. (2014) · Global Environmental Change · doi:10.1016/j.gloenvcha.2014.07.005 · Cited by 842
Using the Internet for Surveys and Health Research
Eysenbach, G., & Wyatt, J. (2002) · Journal of Medical Internet Research · doi:10.2196/jmir.4.2.e13 · Cited by 790
Systematic review automation technologies
Tsafnat, G., Glasziou, P., Choong, M. K., Dunn, A., Galgani, F., & Coiera, E. (2014) · Systematic Reviews · doi:10.1186/2046-4053-3-74 · Cited by 472
“Task description The biomedical literature is the main source of evidence for systematic reviews; however, some research studies can only be found outside of literature databases - the so called grey literature - and sometimes not at all [51,52].”
Background
Metadata
- Authors
- Eysenbach, J. T.
- Venue
- Medical Informatics and the Internet in Medicine
- Publisher
- Informa UK Limited
- Published
- 2001-01
- Volume
- 26
- Issue
- 3
- Pages
- 203-218
- ISSN
- 1463-9238 (ISSN-L)
- OpenAlex
- W2169201348
- Cited by
- 53
- References
- 13
- FWCI
- 1.26
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evaluation of the usefulness of Internet searches to identify unpublished clinical trials for systematic reviews. https://app.cochat.ai/papers/10.1080/14639230110075459
- The paper
- Eysenbach, J. T. (2001). Evaluation of the usefulness of Internet searches to identify unpublished clinical trials for systematic reviews. Medical Informatics and the Internet in Medicine. https://doi.org/10.1080/14639230110075459
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.