Towards a reduction in publication bias.
Open accessThe DOI matches its Crossref record; 3 of the 3 references it cites were found in a registry or an index.
Abstract
Current practice results in the publication of many research studies in medical and related disciplines which may be criticised on the grounds of inadequate sample size and statistical power. Small studies continue to be carried out with little more than a blind hope of showing the desired effect. Nevertheless, papers based on such work are submitted for publication, especially if the results turn out to be statistically significant. There is confusion about what makes a result suitable for publication. Often there is a preference for statistically significant results at the peer review stage. Consequently published reports of small studies tend to contain too many false positive results and to exaggerate the true effects. The use of a criterion of a posteriori power does not eliminate the bias; a priori power is the criterion of choice. This could be implemented by peer review of study protocols at the planning stage by funding bodies and journals.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Freiman, J. A., Chalmers, T. C., Smith, H., & Kuebler, R. R. (1978) · New England Journal of Medicine · doi:10.1056/nejm197809282991304 · Cited by 1,628
Publication prejudices: An experimental study of confirmatory bias in the peer review system
Mahoney, M. J. (1977) · Cognitive Therapy and Research · doi:10.1007/bf01173636 · Cited by 1,013
Statistical Evaluation of Medical Journal Manuscripts
Schor, S. (1966) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1966.03100130097026 · Cited by 207
Cited by
Publication bias in meta-analysis its causes and consequences
Thornton, A. (2000) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(99)00161-4 · Cited by 1,291
Publication bias—a problem in interpreting medical data
Begg, C. B., & Berlin, J. A. (1988) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2982993 · Cited by 814
Effect of lipid reduction on the progression of renal disease: A meta-analysis
Fried, L. F., Orchard, T. J., & Kasiske, B. L. (2001) · Kidney International · doi:10.1046/j.1523-1755.2001.00487.x · Cited by 672
Formulating Questions and Locating Primary Studies for Inclusion in Systematic Reviews
Counsell, C. (1997) · Annals of Internal Medicine · doi:10.7326/0003-4819-127-5-199709010-00008 · Cited by 505
The past, present and future of Registered Reports
Chambers, C. D., & Tzavella, L. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01193-7 · Cited by 499
Publication Bias in Research Synthesis: Sensitivity Analysis Using A Priori Weight Functions.
Vevea, J. L., & Woods, C. M. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.4.428 · Cited by 461
Sterling, T. D., Rosenbaum, W. L., & Weinkam, J. J. (1995) · The American Statistician · doi:10.1080/00031305.1995.10476125 · Cited by 453
on behalf of all surgeons contributing towards The Royal College of Ophthalmologists’ National Ophthalmology Database, Day, A. C., Donachie, P. H. J., Sparrow, J. M., & Johnston, R. L. (2015) · Eye · doi:10.1038/eye.2015.3 · Cited by 364
Callaham, M. L. (1998) · JAMA · doi:10.1001/jama.280.3.254 · Cited by 305
(2014) · Ophthalmology · doi:10.1016/j.ophtha.2013.11.031 · Cited by 283
Publication bias and the canonization of false facts
Nissen, S. B., Magidson, T., Gross, K., & Bergstrom, C. T. (2016) · eLife · doi:10.7554/elife.21451 · Cited by 266
Statistics in medical journals: Developments in the 1980s
Altman, D. G. (1991) · Statistics in Medicine · doi:10.1002/sim.4780101206 · Cited by 268
Peer review for journals: Evidence on quality control, fairness, and innovation
Armstrong, J. S. (1997) · Science and Engineering Ethics · doi:10.1007/s11948-997-0017-3 · Cited by 262
Mendell, M. J., & Smith, A. H. (1990) · American Journal of Public Health · doi:10.2105/ajph.80.10.1193 · Cited by 258
Improving Doctors' Understanding of Statistics
Altman, D. G., & Bland, J. M. (1991) · Journal of the Royal Statistical Society. Series A (Statistics in Society) · doi:10.2307/2983040 · Cited by 195
Bishop, D. V. M., & Thompson, P. A. (2016) · PeerJ · doi:10.7717/peerj.1715 · Cited by 90
“With their extremely large set of Abstracts, Head et al. (2015) found evidence of p-hacking in only two of the ten subject areas they investigated, but in six areas there were less than ten p-values between .”
BackgroundToward prospective registration of clinical trials
Meinert, C. L. (1988) · Controlled Clinical Trials · doi:10.1016/0197-2456(88)90002-5 · Cited by 57
A Systems-Based Approach to Fostering Robust Science in Industrial-Organizational Psychology
Grand, J. A., Rogelberg, S. G., Allen, T. D., Landis, R. S., Reynolds, D. H., Scott, J. C., Tonidandel, S., & Truxillo, D. M. (2018) · Industrial and Organizational Psychology · doi:10.1017/iop.2017.55 · Cited by 52
Results Blind Science Publishing
Locascio, J. J. (2017) · Basic and Applied Social Psychology · doi:10.1080/01973533.2017.1336093 · Cited by 45
Metadata
- Authors
- Newcombe, R. G.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 1987-09-12
- Volume
- 295
- Issue
- 6599
- Pages
- 656-659
- ISSN
- 0007-1447 (ISSN-L)
- 0959-8138
- 1468-5833
- OpenAlex
- W2088158147
- Cited by
- 113
- References
- 3
- FWCI
- 9.67
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.