Reference bias in reports of drug trials.
Open accessThe DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.
Abstract
Articles published before 1985 describing double blind trials of two or more non-steroidal anti-inflammatory drugs in rheumatoid arthritis were examined to see whether there was any bias in the references they cited. Althogether 244 articles meeting the criteria were found through a Medline search and through examining the reference lists of the articles retrieved. The drugs compared in the studies were classified as new or as control drugs and the outcome of the trial as positive or not positive. The reference lists of all papers with references to other trials on the new drug were then examined for reference bias. Positive bias was judged to have occurred if the reference list contained a higher proportion of references with a positive outcome for that drug than among all the articles assumed to have been available to the authors (those published more than two years earlier than the index article). Altogether 133 of the 244 articles were excluded for various reasons--for example, 44 because of multiple publication and 19 because they had no references. Among the 111 articles analysed bias was not possible in the references of 35 (because all the references gave the same outcome); 10 had a neutral selection of references, 22 a negative selection, and 44 a positive selection--a significant positive bias. This bias was not caused by better scientific standing of the cited articles over the uncited ones. Thus retrieving literature by scanning reference lists may produce a biased sample of articles, and reference bias may also render the conclusions of an article less reliable.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Sackett, D. L. (1979) · Journal of Chronic Diseases · doi:10.1016/0021-9681(79)90012-2 · Cited by 2,075
Research works printed but not linked: 3
- Poynard T, Conn HO. The retrieval of randomized clinical trials in liver disease from the medical literature: a comparison of MEDLARS and manual methods. Controlled Clin Tnals 1985;6: 271-9. · DOI · Paper page
- Dickersin K, Hewitt P, Mutch L, Chalmers I, Chalmers TC. Perusing the literature: comparison of MEDLINE searching with a perinatal trials database. ControUed Clin Trials 1985;6:306-17. · DOI · Paper page
- Institute for Scientific Information. Science citation index. Journal citation reports. Philadelphia: ISI, 1986.
Cited by
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gotzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c869 · Cited by 8,883
Self-citationThe Handbook of Research Synthesis and Meta-Analysis
(2019) · doi:10.7758/9781610448864 · Cited by 4,553
The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
Altman, D. G., Schulz, K. F., Moher, D., Egger, M., Davidoff, F., Elbourne, D., Gøtzsche, P. C., Lang, T., & for the CONSORT Group (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00012 · Cited by 3,835
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2012) · International Journal of Surgery · doi:10.1016/j.ijsu.2011.10.001 · Cited by 2,345
Self-citationPublication and related bias in meta-analysis
Sterne, J. A. C., Gavaghan, D., & Egger, M. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(00)00242-0 · Cited by 2,282
Investigating and dealing with publication and other biases in meta-analysis
Sterne, J. A. C., Egger, M., & Smith, G. D. (2001) · BMJ · doi:10.1136/bmj.323.7304.101 · Cited by 2,225
The Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
Jüni, P. (1999) · JAMA · doi:10.1001/jama.282.11.1054 · Cited by 1,981
“The fallibility of meta-analysis is not surprising, considering the various biases that may be introduced by the process of locating and selecting studies, including publication bias,(52) language bias,(53) and citation bias.(54) Low methodological quality of component studies is another potential source of systematic…”
Moher, D., Hopewell, S., Schulz, K. F., Montori, V., Gøtzsche, P. C., Devereaux, P. J., Elbourne, D., Egger, M., & Altman, D. G. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.004 · Cited by 1,858
Self-citationQuantifying publication bias in meta-analysis
Lin, L., & Chu, H. (2018) · Biometrics · doi:10.1111/biom.12817 · Cited by 1,674
Increasing value and reducing waste in research design, conduct, and analysis
Ioannidis, J. P. A., Greenland, S., Hlatky, M. A., Khoury, M. J., Macleod, M. R., Moher, D., Schulz, K. F., & Tibshirani, R. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62227-8 · Cited by 1,597
How to increase value and reduce waste when research priorities are set
Chalmers, I., Bracken, M. B., Djulbegovic, B., Garattini, S., Grant, J., Gülmezoglu, A. M., Howells, D. W., Ioannidis, J. P. A., & Oliver, S. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62229-1 · Cited by 1,436
Reducing waste from incomplete or unusable reports of biomedical research
Glasziou, P., Altman, D. G., Bossuyt, P., Boutron, I., Clarke, M., Julious, S., Michie, S., Moher, D., & Wager, E. (2014) · The Lancet · doi:10.1016/s0140-6736(13)62228-x · Cited by 1,319
Publication bias in meta-analysis its causes and consequences
Thornton, A. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00161-4 · Cited by 1,291
Guidelines for systematic review in Conservation and environmental management
PULLIN, A. S., & STEWART, G. B. (2006) · Conservation Biology · doi:10.1111/j.1523-1739.2006.00485.x · Cited by 1,137
Language bias in randomised controlled trials published in English and German
Egger, M., Zellweger-Zähner, T., Schneider, M., Junker, C., Lengeler, C., & Antes, G. (1997) · The Lancet · doi:10.1016/s0140-6736(97)02419-7 · Cited by 1,028
Direction and impact of language bias in meta-analyses of controlled trials: empirical study
Jüni, P., Holenstein, F., Sterne, J., Bartlett, C., & Egger, M. (2002) · International Journal of Epidemiology · doi:10.1093/ije/31.1.115 · Cited by 892
Publication bias: evidence of delayed publication in a cohort study of clinical research projects
Stern, J. M., & Simes, R. J. (1997) · BMJ · doi:10.1136/bmj.315.7109.640 · Cited by 818
Publication bias—a problem in interpreting medical data
Begg, C. B., & Berlin, J. A. (1988) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2982993 · Cited by 814
Users' Guides to the Medical Literature: VI. How to Use an Overview
Oxman, A. D. (1994) · JAMA · doi:10.1001/jama.1994.03520170077040 · Cited by 775
Meta-analysis: Potentials and promise
Egger, M., & Smith, G. D. (1997) · BMJ · doi:10.1136/bmj.315.7119.1371 · Cited by 646
Bias in clinical intervention research.
Gluud, L. L. (2006) · American Journal of Epidemiology · doi:10.1093/aje/kwj069 · Cited by 636
Effects of Alcohol Tax and Price Policies on Morbidity and Mortality: A Systematic Review
Wagenaar, A. C., Tobler, A. L., & Komro, K. A. (2010) · American Journal of Public Health · doi:10.2105/ajph.2009.186007 · Cited by 576
Egger, M., & Smith, G. D. (1995) · BMJ · doi:10.1136/bmj.310.6982.752 · Cited by 508
Formulating Questions and Locating Primary Studies for Inclusion in Systematic Reviews
Counsell, C. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-5-199709010-00008 · Cited by 505
Practical methodology of meta‐analyses (overviews) using updated individual patient data
Stewart, L. A., & Cochrane Working Group On Meta‐Analysis Using Individual Patient Data (1995) · Statistics in Medicine · doi:10.1002/sim.4780141902 · Cited by 492
Reporting bias in medical research - a narrative review
McGauran, N., Wieseler, B., Kreis, J., Schüler, Y.-B., Kölsch, H., & Kaiser, T. (2010) · Trials · doi:10.1186/1745-6215-11-37 · Cited by 452
Investigating and Dealing with Publication and Other Biases
Sterne, J. A. C., Egger, M., & Smith, G. D. (2001) · Systematic Reviews in Health Care · doi:10.1002/9780470693926.ch11 · Cited by 449
Publication and related biases.
Song, Eastwood, Gilbody, Duley, & Sutton (2000) · Health Technology Assessment · doi:10.3310/hta4100 · Cited by 427
Exercise therapy for multiple sclerosis
Rietberg, M. B., Brooks, D., Uitdehaag, B. M., & Kwakkel, G. (2005) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd003980.pub2 · Cited by 422
Duncan, M. J., Spence, J. C., & Mummery, W. K. (2005) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/1479-5868-2-11 · Cited by 413
Assembling Comparison Groups to Assess the Effects of Health care
Chalmers, L. (1997) · Journal of the Royal Society of Medicine · doi:10.1177/014107689709000706 · Cited by 409
Factors that limit the quality, number and progress of randomised controlled trials
Prescott, Counsell, Gillespie, Grant, Russell, Kiauka, Colthart, Ross, Shepherd, & Russell (1999) · Health Technology Assessment · doi:10.3310/hta3200 · Cited by 390
Systematic Evaluation and Comparison of Statistical Tests for Publication Bias
Hayashino, Y., Noguchi, Y., & Fukui, T. (2005) · Journal of Epidemiology · doi:10.2188/jea.15.235 · Cited by 379
What determines the citation frequency of ecological papers?
LEIMU, R., & KORICHEVA, J. (2005) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2004.10.010 · Cited by 367
Gøtzsche, P. C. (1989) · Controlled Clinical Trials · doi:10.1016/0197-2456(89)90017-2 · Cited by 339
Self-citationBias caused by sampling error in meta-analysis with small sample sizes
Lin, L. (2018) · PLOS ONE · doi:10.1371/journal.pone.0204056 · Cited by 335
Multidisciplinary rehabilitation for adults with multiple sclerosis
Khan, F., Turner-Stokes, L., Ng, L., Kilpatrick, T., & Amatya, B. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006036.pub2 · Cited by 286
Muscle relaxants for non-specific low-back pain
van Tulder, M. W., Touray, T., Furlan, A. D., Solway, S., & Bouter, L. M. (2003) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd004252 · Cited by 285
Association of Journal Quality Indicators With Methodological Quality of Clinical Research Articles
Lee, K. P. (2002) · JAMA · doi:10.1001/jama.287.21.2805 · Cited by 284
Agreement among reviewers of review articles
Oxman, A. D., Guyatt, G. H., Singer, J., Goldsmith, C. H., Hutchison, B. G., Milner, R. A., & Streiner, D. L. (1991) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(91)90205-n · Cited by 282
Uses and abuses of meta-analysis
Egger, M., Smith, G. D., & Sterne, J. A. (2001) · Clinical Medicine · doi:10.7861/clinmedicine.1-6-478 · Cited by 255
van Tulder, M. W., Touray, T., Furlan, A. D., Solway, S., & Bouter, L. M. (2003) · Spine · doi:10.1097/01.brs.0000090503.38830.ad · Cited by 252
Evidence-Based Research Series-Paper 1: What Evidence-Based Research is and why is it important?
Robinson, K. A., Brunnhuber, K., Ciliska, D., Juhl, C. B., Christensen, R., & Lund, H. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2020.07.020 · Cited by 146
Multiple publication of reports of drug trials.
G�tzsche, P. C. (1989) · European Journal of Clinical Pharmacology · doi:10.1007/bf00558064 · Cited by 113
“Authors must be aware that publication of conference proceedings, unless solely as abstracts, may render later publication in a reputable, peer reviewed journal difficult [ 1 , 11, 12], at least in the same language.”
BackgroundMethodInfluentialSelf-citationNon-pharmacological interventions for chronic pain in multiple sclerosis
Amatya, B., Young, J., & Khan, F. (2018) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd012622.pub2 · Cited by 78
Toward prospective registration of clinical trials
Meinert, C. L. (1988) · Controlled Clinical Trials · doi:10.1016/0197-2456(88)90002-5 · Cited by 57
Finding evidence for comparing medical interventions: AHRQ and the Effective Health Care Program
Relevo, R., & Balshem, H. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.11.022 · Cited by 52
Norris, S. L., Moher, D., Reeves, B. C., Shea, B., Loke, Y., Garner, S., Anderson, L., Tugwell, P., & Wells, G. (2013) · Research Synthesis Methods · doi:10.1002/jrsm.1062 · Cited by 50
Information retrieval for ecological syntheses
Bayliss, H. R., & Beyer, F. R. (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1120 · Cited by 45
Examining the predictive validity of NIH peer review scores
Lindner, M. D., & Nakamura, R. K. (2015) · PLOS ONE · doi:10.1371/journal.pone.0126938 · Cited by 24
Metadata
- Authors
- Gotzsche, P. C.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 1987-09-12
- Volume
- 295
- Issue
- 6599
- Pages
- 654-656
- ISSN
- 0007-1447 (ISSN-L)
- 0959-8138
- 1468-5833
- OpenAlex
- W1982885757
- Cited by
- 222
- References
- 1
- FWCI
- 4.83
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.