Cited by 1,466Statistical Methods in EpidemiologyMeta-analysis and systematic reviewsStatistical Methods in Clinical Trials

The DOI matches its Crossref record; 46 of the 48 references it cites were found in a registry or an index.

Sifting the evidence---what's wrong with significance tests? Another comment on the role of statistical methods

Open access

Sterne, J. A C·2001·BMJ

DOI 10.1136/bmj.322.7280.226PMC PMC1119478

Open this paper in CoChat — free account

Abstract

The findings of medical research are often met with considerable scepticism, even when they have apparently come from studies with sound methodologies that have been subjected to appropriate statistical analysis. This is perhaps particularly the case with respect to epidemiological findings that suggest that some aspect of everyday life is bad for people. Indeed, one recent popular history, the medical journalist James Le Fanu's The Rise and Fall of Modern Medicine , went so far as to suggest that the solution to medicine's ills would be the closure of all departments of epidemiology.1 One contributory factor is that the medical literature shows a strong tendency to accentuate the positive; positive outcomes are more likely to be reported than null results.2–4 By this means alone a host of purely chance findings will be published, as by conventional reasoning examining 20 associations will produce one result that is “significant at P=0.05” by chance alone. If only positive findings are published then they may be mistakenly considered to be of importance rather than being the necessary chance results produced by the application of criteria for meaningfulness based on statistical significance. As many studies contain long questionnaires collecting information on hundreds of variables, and measure a wide range of potential outcomes, several false positive findings are virtually guaranteed. The high volume and often contradictory nature5 of medical research findings, however, is not only because of publication bias. A more fundamental problem is the widespread misunderstanding of the nature of statistical significance. #### Summary points P values, or significance levels, measure the strength of the evidence against the null hypothesis; the smaller the P value, the stronger the evidence against the null hypothesis An arbitrary division of results, into “significant” or “non-significant” according to the P value, was not the intention of the …

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.1136/bmj.322.7280.226

Cites

46 of 46 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Statistical Methods for Research Workers
Fisher, R. A. (1945) · American Journal of Clinical Pathology · doi:10.1093/ajcp/15.11.552b · Cited by 10,883
Association between Water Intake, Chronic Kidney Disease, and Cardiovascular Disease: A Cross-Sectional Analysis of NHANES Data
Sontrop, J. M., Dixon, S. N., Garg, A. X., Jiménez, I. B., Dohein, O., Huang, S. S., & Clark, W. F. (2013) · American Journal of Nephrology · doi:10.1159/000350377 · Cited by 5,903
Statistical Inference
Casella, G., & Berger, R. L. (1993) · Chapman & Hall/CRC eBooks · doi:10.1201/9781003456285 · Cited by 5,835
Publication bias in clinical research
Easterbrook, P. J., Gopalan, R., Berlin, J. A., & Matthews, D. R. (1991) · The Lancet · doi:10.1016/0140-6736(91)90201-y · Cited by 3,109
IX. On the problem of the most efficient tests of statistical hypotheses
Neyman, J., & Pearson, E. S. (1933) · Philosophical Transactions of the Royal Society of London Series A Containing Papers of a Mathematical or Physical Character · doi:10.1098/rsta.1933.0009 · Cited by 2,370
Confidence intervals rather than P values: estimation rather than hypothesis testing.
Gardner, M. J., & Altman, D. G. (1986) · BMJ · doi:10.1136/bmj.292.6522.746 · Cited by 2,014
Statistics with Confidence
Smithson, M. J. (2000) · SAGE Publications Ltd eBooks · doi:10.4135/9781446218525 · Cited by 1,871
Bayesian statistical inference for psychological research.
Edwards, W., Lindman, H., & Savage, L. J. (1963) · Psychological Review · doi:10.1037/h0044139 · Cited by 1,734
The Importance of Beta, the Type II Error and Sample Size in the Design and Interpretation of the Randomized Control Trial: Survey of 71 Negative Trials
Freiman, J. A., Chalmers, T. C., Smith, H., & Kuebler, R. R. (1978) · New England Journal of Medicine · doi:10.1056/nejm197809282991304 · Cited by 1,628
ON THE PROBLEM OF THE MOST EFFICIENT TESTS OF STATISTICAL HYPOTHESES
Neyman, J., & Pearson, E. S. (1967) · University of California Press eBooks · doi:10.1525/9780520339897-008 · Cited by 1,545

Show all 48 references

Research works printed but not linked: 5

[38]Davey Smith G. Reflections on the limits to epidemiology. J Clin Epidemiol (in press).
BMJ. 2001 Jan 27;322(7280):226–231.
1-1.Fisher RA. Statistical methods for research workers. 1st ed. Edinburgh: Oliver and Boyd; 1925. . Reprinted by Oxford University Press. [Google Scholar]
1-2.Fisher RA. Design of experiments. 1st ed. Edinburgh: Oliver and Boyd; 1935. . Reprinted by Oxford University Press. [Google Scholar]
1-4.Cox DR. Statistical significance tests. Br J Clin Pharmacol. 1982;14:325–331. doi: 10.1111/j.1365-2125.1982.tb01987.x. [DOI] [PMC free article] [PubMed] [Google Scholar] · DOI

Sorted by citations

Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
Progressive Statistics for Studies in Sports Medicine and Exercise Science
HOPKINS, W. G., MARSHALL, S. W., BATTERHAM, A. M., & HANIN, J. (2009) · Medicine & Science in Sports & Exercise · doi:10.1249/mss.0b013e31818cb278 · Cited by 9,128
Power failure: why small sample size undermines the reliability of neuroscience
Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013) · Nature Reviews Neuroscience · doi:10.1038/nrn3475 · Cited by 7,641
The ASA Statement on p -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306
Sensitivity analysis in observational research: Introducing the E-value
VanderWeele, T. J., & Ding, P. (2017) · Annals of Internal Medicine · doi:10.7326/m16-2607 · Cited by 6,350
A manifesto for reproducible science
Munafò, M. R., Nosek, B. A., Bishop, D. V. M., Button, K. S., Chambers, C. D., Percie du Sert, N., Simonsohn, U., Wagenmakers, E.-J., Ware, J. J., & Ioannidis, J. P. A. (2017) · Nature Human Behaviour · doi:10.1038/s41562-016-0021 · Cited by 3,397
Statistical tests, P values, confidence intervals, and power: a guide to misinterpretations
Greenland, S., Senn, S. J., Rothman, K. J., Carlin, J. B., Poole, C., Goodman, S. N., & Altman, D. G. (2016) · European Journal of Epidemiology · doi:10.1007/s10654-016-0149-3 · Cited by 3,200
Orienting the causal relationship between imprecisely measured traits using GWAS summary data.
Hemani, G., Tilling, K., & Davey Smith, G. (2017) · PLoS Genetics · doi:10.1371/journal.pgen.1007081 · Cited by 2,607
Making Meaningful Inferences About Magnitudes
Batterham, A. M., & Hopkins, W. G. (2006) · International Journal of Sports Physiology and Performance · doi:10.1123/ijspp.1.1.50 · Cited by 2,186
Pre-exposure prophylaxis to prevent the acquisition of HIV-1 infection (PROUD): effectiveness results from the pilot phase of a pragmatic open-label randomised trial.
McCormack, S., Dunn, D. T., Desai, M., Dolling, D. I., Gafos, M., Gilson, R., Sullivan, A. K., Clarke, A., Reeves, I., Schembri, G., et al. (2016) · The Lancet · doi:10.1016/s0140-6736(15)00056-2 · Cited by 1,878
Assessing the Probability That a Positive Report is False: An Approach for Molecular Epidemiology Studies
Wacholder, S., Chanock, S. J., García‐Closas, M., ghormli, L. K. E., & Rothman, N. (2004) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djh075 · Cited by 1,753
Comparative efficacy and acceptability of 12 new-generation antidepressants: a multiple-treatments meta-analysis
Cipriani, A., Furukawa, T. A., Salanti, G., Geddes, J. R., Higgins, J. P., Churchill, R., Watanabe, N., Nakagawa, A., Omori, I. M., McGuire, H., et al. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60046-5 · Cited by 1,654
Strengthening the Reporting of Observational Studies in Epidemiology using Mendelian Randomisation (STROBE-MR): explanation and elaboration.
Skrivankova, V. W., Richmond, R. C., Woolf, B. A. R., Davies, N. M., Swanson, S. A., VanderWeele, T. J., Timpson, N. J., Higgins, J. P. T., Dimou, N., Langenberg, C., et al. (2021) · BMJ · doi:10.1136/bmj.n2233 · Cited by 1,648
Bias and causal associations in observational research
Grimes, D. A., & Schulz, K. F. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07451-2 · Cited by 1,553
The Extent and Consequences of P-Hacking in Science
Head, M. L., Holman, L., Lanfear, R., Kahn, A. T., & Jennions, M. D. (2015) · PLOS Biology · doi:10.1371/journal.pbio.1002106 · Cited by 1,388
“NHST is the most widely used data analysis method in most scientific disciplines [22,23].”
Method
PRACTICE GUIDELINE FOR THE Treatment of Patients With Major Depressive Disorder
Gelenberg, A. J., Freeman, P. B., Markowitz, J. C., Rosenbaum, J. F., Thase, M. E., Trivedi, M. H., & Rhoads, R. S. (2010) · Cited by 1,372
Beyond Power Calculations: Assessing Type S (Sign) and Type M (Magnitude) Errors
Gelman, A., & Carlin, J. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614551642 · Cited by 1,343
Reporting of Patient-Reported Outcomes in Randomized Trials: The CONSORT PRO Extension
Calvert, M., Blazeby, J., Altman, D. G., Revicki, D. A., Moher, D., Brundage, M. D., & CONSORT PRO Group, F. T. (2013) · JAMA · doi:10.1001/jama.2013.879 · Cited by 1,386
Problems of reporting genetic associations with complex outcomes
Colhoun, H. M., McKeigue, P., & Smith, G. D. (2003) · The Lancet · doi:10.1016/s0140-6736(03)12715-8 · Cited by 1,203
Vitamin D and multiple health outcomes: umbrella review of systematic reviews and meta-analyses of observational studies and randomised trials
Theodoratou, E., Tzoulaki, I., Zgaga, L., & Ioannidis, J. P. A. (2014) · BMJ · doi:10.1136/bmj.g2035 · Cited by 956
Meta-regression in Stata
Harbord, R. M., & Higgins, J. P. T. (2008) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0800800403 · Cited by 897
Type 2 diabetes and cancer: umbrella review of meta-analyses of observational studies
Tsilidis, K. K., Kasimis, J. C., López, D. S., Ntzani, E., & Ioannidis, J. P. A. (2015) · BMJ · doi:10.1136/bmj.g7607 · Cited by 885
Evaluating the relationship between circulating lipoprotein lipids and apolipoproteins with risk of coronary heart disease: A multivariable Mendelian randomisation analysis
Richardson, T. G., Sanderson, E., Palmer, T. M., Ala-Korpela, M., Ference, B. A., Davey Smith, G., & Holmes, M. V. (2020) · PLOS Medicine · doi:10.1371/journal.pmed.1003062 · Cited by 843
Can trial sequential monitoring boundaries reduce spurious inferences from meta-analyses?
Thorlund, K., Devereaux, P. J., Wetterslev, J., Guyatt, G. H., Ioannidis, J. P. A., Thabane, L., Gluud, L. L., Als-Nielsen, B. E. T., & Gluud, C. N. (2008) · International Journal of Epidemiology · doi:10.1093/ije/dyn179 · Cited by 834
Effects of Different Modes of Exercise Training on Glucose Control and Risk Factors for Complications in Type 2 Diabetic Patients
Snowling, N. J., & Hopkins, W. G. (2006) · Diabetes Care · doi:10.2337/dc06-1317 · Cited by 813
Adiposity and cancer at major anatomical sites: umbrella review of the literature
Kyrgiou, M., Kalliala, I., Markozannes, G., Gunter, M. J., Paraskevaidis, E., Gabra, H., Martin-Hirsch, P., & Tsilidis, K. K. (2017) · BMJ · doi:10.1136/bmj.j477 · Cited by 803
The statistical significance of randomized controlled trial results is frequently fragile: a case for a Fragility Index
Walsh, M. W., Srinathan, S., McAuley, D. F., Mrkobrada, M., Levine, O. H., Ribic, C. M., Molnar, A. O., Dattani, N. D., Burke, A., Guyatt, G. H., et al. (2014) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.10.019 · Cited by 792
An overview of clinical research: the lay of the land
Grimes, D. A., & Schulz, K. F. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07283-5 · Cited by 734
The Strategic Use of Decentralized Institutions: Exploring Certification With the ISO 14001 Management Standard
King, A. A., Lenox, M. J., & Terlaak, A. (2005) · Academy of Management Journal · doi:10.5465/amj.2005.19573111 · Cited by 875
Serum BDNF concentrations as peripheral manifestations of depression: evidence from a systematic review and meta-analyses on 179 associations (N=9484)
Molendijk, M. L., Spinhoven, P., Polak, M., Bus, B. A. A., Penninx, B. W. J. H., & Elzinga, B. M. (2014) · Molecular Psychiatry · doi:10.1038/mp.2013.105 · Cited by 702
Subgroup analyses in randomized trials: risks of subgroup-specific analyses;
Brookes, S. T., Whitely, E., Egger, M., Smith, G. D., Mulheran, P. A., & Peters, T. J. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2003.08.009 · Cited by 775
Sample Size Guidelines for Logistic Regression from Observational Studies with Large Population: Emphasis on the Accuracy Between Statistics and Parameters Based on Real Life Clinical Data
Bujang, M., Sa'at, N., Sidik, T. M. I. T. A. B., & Joo, L. C. (2018) · Malaysian Journal of Medical Sciences · doi:10.21315/mjms2018.25.4.12 · Cited by 661
Risk factors for endometrial cancer: An umbrella review of the literature
Raglan, O., Kalliala, I. E. J., Markozannes, G., Cividini, S., Gunter, M. J., Nautiyal, J., Gabra, H., Paraskevaidis, E. A., Martin‐Hirsch, P. L., Tsilidis, K. K., et al. (2018) · International Journal of Cancer · doi:10.1002/ijc.31961 · Cited by 664
An investigation of the false discovery rate and the misinterpretation of p -values
Colquhoun, D. (2014) · Royal Society Open Science · doi:10.1098/rsos.140216 · Cited by 792
Subgroup analyses in randomised controlled trials: quantifying the risks of false-positives and false-negatives
Brookes, S. T., Whitley, E., Peters, T. J., Mulheran, P. A., Egger, M., & Davey Smith, G. (2001) · Health Technology Assessment · doi:10.3310/hta5330 · Cited by 580
Clustered environments and randomized genes: a fundamental distinction between conventional and genetic epidemiology
Smith, G. D., Lawlor, D. A., Harbord, R., Timpson, N., Day, I., & Ebrahim, S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040352 · Cited by 569
Escaping the Bonferroni iron claw in ecological studies
García, L. V. (2004) · Oikos · doi:10.1111/j.0030-1299.2004.13046.x · Cited by 556
Design, power, and interpretation of studies in the standard murine model of ALS
Scott, S., Kranz, J. E., Cole, J., Lincecum, J. M., Thompson, K., Kelly, N., Bostrom, A., Theodoss, J., Al‐Nakhala, B. M., Vieira, F. G., et al. (2008) · Amyotrophic Lateral Sclerosis · doi:10.1080/17482960701856300 · Cited by 504
Effectiveness of home based support for older people: systematic review and meta-analysisCommentary: When, where, and why do preventive home visits work?
Elkan, R., Egger, M., Kendrick, D., Dewey, M., Hewitt, M., Robinson, J., Blair, M., Williams, D., & Brummell, K. (2001) · BMJ · doi:10.1136/bmj.323.7315.719 · Cited by 489
Why, When and How to Adjust Your P Values?
Jafari, M., & Ansari‐Pour, N. (2019) · Europe PMC (PubMed Central) · doi:10.22074/cellj.2019.5992 · Cited by 542
The Healthy Activity Program (HAP), a lay counsellor-delivered brief psychological treatment for severe depression, in primary care in India: a randomised controlled trial
Patel, V., Weobong, B., Weiss, H. A., Anand, A., Bhat, B., Katti, B., Dimidjian, S., Araya, R., Hollon, S. D., King, M., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(16)31589-6 · Cited by 448
Bayesian statistical methods for genetic association studies
Stephens, M., & Balding, D. J. (2009) · Nature Reviews Genetics · doi:10.1038/nrg2615 · Cited by 480
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2019) · CHANCE · doi:10.1080/09332480.2019.1579573 · Cited by 474
Longer‐term primary prevention for alcohol misuse in young people: a systematic review
Foxcroft, D. R., Ireland, D., Lister‐Sharp, D. J., Lowe, G., & Breen, R. (2003) · Addiction · doi:10.1046/j.1360-0443.2003.00355.x · Cited by 433
WHO systematic review of randomised controlled trials of routine antenatal care
Carroli, G., Villar, J., Piaggio, G., Khan-Neelofur, D., Gülmezoglu, M., Mugford, M., Lumbiganon, P., Farnot, U., & Bersgjø, P. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04723-1 · Cited by 435
Accuracy and reproducibility of transient elastography for the diagnosis of fibrosis in pediatric nonalcoholic steatohepatitis†
Nobili, V., Vizzutti, F., Arena, U., Abraldeṣ, J. G., Marra, F., Pietrobattista, A., Fruhwirth, R., Marcellini, M., & Pinzani, M. (2008) · Hepatology · doi:10.1002/hep.22376 · Cited by 433
Statistics in Brief: The Importance of Sample Size in the Planning and Interpretation of Medical Research
Biau, D. J., Kernéis, S., & Porcher, R. (2008) · Clinical Orthopaedics and Related Research · doi:10.1007/s11999-008-0346-9 · Cited by 428
Effect of a Low-Intensity PSA-Based Screening Intervention on Prostate Cancer Mortality
Martin, R. M., Donovan, J., Turner, E. L., Metcalfe, C. D., Young, G., Walsh, E. I., Lane, J. A., Noble, S., Oliver, S. E., Evans, S., et al. (2018) · JAMA · doi:10.1001/jama.2018.0154 · Cited by 415
Self-citation
Insulin resistance in polycystic ovary syndrome: a systematic review and meta-analysis of euglycaemic–hyperinsulinaemic clamp studies
Cassar, S., Misso, M. L., Hopkins, W. G., Shaw, C. S., Teede, H. J., & Stepto, N. K. (2016) · Human Reproduction · doi:10.1093/humrep/dew243 · Cited by 415
Lack of benefit of dietary advice to men with angina: results of a controlled trial
Burr, M. L., Ashfield-Watt, P. A. L., Dunstan, F. D. J., Fehily, A. M., Breay, P., Ashton, T., Zotos, P. C., Haboubi, N. A. A., & Elwood, P. C. (2003) · European Journal of Clinical Nutrition · doi:10.1038/sj.ejcn.1601539 · Cited by 409
Bayes factors for genome‐wide association studies: comparison with P ‐values
Wakefield, J. (2008) · Genetic Epidemiology · doi:10.1002/gepi.20359 · Cited by 404
P-Value, a true test of statistical significance? a cautionary note
Dahiru, T. (2011) · Annals of Ibadan Postgraduate Medicine · doi:10.4314/aipm.v6i1.64038 · Cited by 390
Evolution of Reporting P Values in the Biomedical Literature, 1990-2015
Chavalarias, D., Wallach, J. D., Li, A. H. T., & Ioannidis, J. P. A. (2016) · JAMA · doi:10.1001/jama.2016.1952 · Cited by 370
Changes in weight, physical activity, sedentary behaviour and dietary intake during the transition to higher education: a prospective study
Deforche, B. I., Dyck, D. V., Deliens, T., & Bourdeaudhuij, I. D. (2015) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-015-0173-9 · Cited by 387
The Association Between Childhood Depression and Adulthood Body Mass Index
Pine, D. S., Goldstein, R. B., Wolk, S. I., & Weissman, M. M. (2001) · PEDIATRICS · doi:10.1542/peds.107.5.1049 · Cited by 371
Multiplicity in randomised trials I: endpoints and treatments
Schulz, K. F., & Grimes, D. A. (2005) · The Lancet · doi:10.1016/s0140-6736(05)66461-6 · Cited by 366
A Comparison of Hand-Sewn Versus Stapled Ileal Pouch Anal Anastomosis (IPAA) Following Proctocolectomy
Lovegrove, R. E., Constantinides, V. A., Heriot, A. G., Athanasiou, T., Darzi, A. W., Remzi, F. H., Nicholls, R. J., Fazio, V. W., & Tekkis, P. P. (2006) · Annals of Surgery · doi:10.1097/01.sla.0000225031.15405.a3 · Cited by 364
Could Fisher, Jeffreys and Neyman Have Agreed on Testing?
Berger, J. O. (2003) · Statistical Science · doi:10.1214/ss/1056397485 · Cited by 348
Bench to Bedside: The Quest for Quality in Experimental Stroke Research
Dirnagl, U. (2006) · Journal of Cerebral Blood Flow & Metabolism · doi:10.1038/sj.jcbfm.9600298 · Cited by 348
C-reactive protein and its role in metabolic syndrome: mendelian randomisation study
Timpson, N. J., Lawlor, D. A., Harbord, R. M., Gaunt, T. R., Day, I. N. M., Palmer, L. J., Hattersley, A. T., Ebrahim, S. B. J., Lowe, G. D. O., Rumley, A., et al. (2005) · The Lancet · doi:10.1016/s0140-6736(05)67786-0 · Cited by 345
Hepatic venous pressure gradient and prognosis in patients with acute variceal bleeding treated with pharmacologic and endoscopic therapy
Abraldeṣ, J. G., Villanueva, C., Bañares, R., Aracil, C., Catalina, M., García‐Pagán, J. C., & Bosch, J. (2007) · Journal of Hepatology · doi:10.1016/j.jhep.2007.10.008 · Cited by 343
Synthesising quantitative evidence in systematic reviews of complex health interventions
Higgins, J. P. T., López-López, J. A., Becker, B. J., Davies, S. R., Dawson, S., Grimshaw, J. M., McGuinness, L. A., Moore, T. H. M., Rehfuess, E. A., Thomas, J., et al. (2019) · BMJ Global Health · doi:10.1136/bmjgh-2018-000858 · Cited by 322
Epidemiology, epigenetics and the ‘Gloomy Prospect’: embracing randomness in population health research and practice
Smith, G. D. (2011) · International Journal of Epidemiology · doi:10.1093/ije/dyr117 · Cited by 339
A Comprehensive Analysis of Deep Regression
Lathuilière, S., Mesejo, P., Alameda-Pineda, X., & Horaud, R. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2910523 · Cited by 358
A tutorial on support vector machine-based methods for classification problems in chemometrics
Luts, J., Ojeda, F., de Plas, R. V., Moor, B. D., Huffel, S. V., & Suykens, J. A. K. (2010) · Analytica Chimica Acta · doi:10.1016/j.aca.2010.03.030 · Cited by 336
The False-positive to False-negative Ratio in Epidemiologic Studies
Ioannidis, J. P. A., Tarone, R., & McLaughlin, J. K. (2011) · Epidemiology · doi:10.1097/ede.0b013e31821b506e · Cited by 321
Issues in the reporting of epidemiological studies: a survey of recent practice.
Pocock, S. J., Collier, T. J., Dandreo, K. J., de Stavola, B. L., Goldman, M. B., Kalish, L. A., Kasten, L. E., & McCormack, V. A. (2004) · BMJ · doi:10.1136/bmj.38250.571088.55 · Cited by 313
The earth is flat ( p > 0.05): significance thresholds and the crisis of unreplicable research
Amrhein, V., Korner‐Nievergelt, F., & Roth, T. (2017) · PeerJ · doi:10.7717/peerj.3544 · Cited by 331
Mining Electronic Health Records (EHRs): A Survey
Yadav, P., Steinbach, M., Kumar, V., & Simon, G. (2018) · ACM Computing Surveys · doi:10.1145/3127881 · Cited by 299
Expected Value of Sample Information Calculations in Medical Decision Modeling
Ades, A. E., Lu, G., & Claxton, K. (2004) · Medical Decision Making · doi:10.1177/0272989x04263162 · Cited by 364
Randomised trials of human albumin for adults with sepsis: systematic review and meta-analysis with trial sequential analysis of all-cause mortality
Patel, A., Laffan, M., Waheed, U., & Brett, S. J. (2014) · BMJ · doi:10.1136/bmj.g4561 · Cited by 292
The effect of smoking after surgery for Crohn’s disease: a meta-analysis of observational studies
Reese, G., Nanidis, T. G., Borysiewicz, C., Yamamoto, T., Orchard, T. R., & Tekkis, P. P. (2008) · International Journal of Colorectal Disease · doi:10.1007/s00384-008-0542-9 · Cited by 287
When Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment
Szucs, D., & Ioannidis, J. P. A. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00390 · Cited by 298
Reliability of transient elastography for the diagnosis of advanced fibrosis in chronic hepatitis C
Arena, U., Vizzutti, F., Abraldeṣ, J. G., Corti, G., Stasi, C., Moscarella, S., Milani, S., Lorefice, E., Petrarca, A., Romanelli, R. G., et al. (2008) · Gut · doi:10.1136/gut.2008.149708 · Cited by 279
Statistical Foundations for Model-Based Adjustments
Greenland, S., & Pearce, N. E. (2015) · Annual Review of Public Health · doi:10.1146/annurev-publhealth-031914-122559 · Cited by 274
Environmental risk factors, protective factors, and peripheral biomarkers for ADHD: an umbrella review
Kim, J. H., Kim, J. Y., Lee, J., Jeong, G. H., Lee, E., Lee, S., Lee, K. H., Kronbichler, A., Stubbs, B., Solmi, M., et al. (2020) · The Lancet Psychiatry · doi:10.1016/s2215-0366(20)30312-6 · Cited by 258
The Primary Outcome Fails — What Next?
Pocock, S., & Stone, G. W. (2016) · New England Journal of Medicine · doi:10.1056/nejmra1510064 · Cited by 275
Calibration of an accelerometer during free-living activities in children
Mattocks, C., Leary, S. D., Ness, A., Deere, K. C., Saunders, J., Tilling, K. M., Kirkby, J., Blair, S. N., & Riddoch, C. (2007) · International Journal of Pediatric Obesity · doi:10.1080/17477160701408809 · Cited by 268
Type 2 Diabetes and Cancer: An Umbrella Review of Observational and Mendelian Randomization Studies
Pearson‐Stuttard, J., Papadimitriou, N., Markozannes, G., Cividini, S., Κακούρου, Α., Gill, D., Rizos, E. C., Monori, G., Ward, H. A., Kyrgiou, M., et al. (2021) · Cancer Epidemiology Biomarkers & Prevention · doi:10.1158/1055-9965.epi-20-1245 · Cited by 281
Environmental risk factors and biomarkers for autism spectrum disorder: an umbrella review of the evidence
Kim, J. Y., Son, M. J., Son, C. Y., Radua, J., Eisenhut, M., Gressier, F., Koyanagi, A., Carvalho, A. F., Stubbs, B., Solmi, M., et al. (2019) · The Lancet Psychiatry · doi:10.1016/s2215-0366(19)30181-6 · Cited by 271
The International Fitness Scale (IFIS): usefulness of self-reported fitness in youth
Ortega, F. B., Ruiz, J. R., España‐Romero, V., Vicente‐Rodríguez, G., Martínez‐Gómez, D., Μanios, Y., Béghin, L., Molnár, D., Widhalm, K., Moreno, L. A., et al. (2011) · International Journal of Epidemiology · doi:10.1093/ije/dyr039 · Cited by 300
Trial and Error
Kaul, S., & Diamond, G. A. (2010) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2009.06.065 · Cited by 270
Comparison of colonic stenting and open surgery for malignant large bowel obstruction
Tilney, H. S., Lovegrove, R. E., Purkayastha, S., Sains, P., Weston-Petrides, G. K., Darzi, A. W., Tekkis, P. P., & Heriot, A. G. (2006) · Surgical Endoscopy · doi:10.1007/s00464-005-0644-1 · Cited by 264
Impact of progressive resistance training on lipids and lipoproteins in adults: A meta-analysis of randomized controlled trials
Kelley, G. A., & Kelley, K. S. (2009) · Preventive Medicine · doi:10.1016/j.ypmed.2008.10.010 · Cited by 263
An umbrella review of the evidence linking oral health and systemic noncommunicable diseases
Botelho, J., Mascarenhas, P., Viana, J., Proença, L., Orlandi, M., Leira, Y., Chambrone, L., Mendes, J. J., & Machado, V. (2022) · Nature Communications · doi:10.1038/s41467-022-35337-8 · Cited by 308
Empirical Evaluation of Very Large Treatment Effects of Medical Interventions
Pereira, T. V., Horwitz, R. I., & Ioannidis, J. P. A. (2012) · JAMA · doi:10.1001/jama.2012.13444 · Cited by 257
The Importance of Replication in Wildlife Research
Johnson, D. H. (2002) · Journal of Wildlife Management · doi:10.2307/3802926 · Cited by 254
Metabolic equivalent of task (METs) thresholds as an indicator of physical activity intensity
Mendes, M. D. A., Crochemore‐Silva, I., Ramires, V. V., Reichert, F. F., Martins, R. C., Ferreira, R., & Tomasi, E. (2018) · PLoS ONE · doi:10.1371/journal.pone.0200701 · Cited by 279
False Alarms and Pseudo-Epidemics: The Limitations of Observational Epidemiology
Grimes, D. A., & Schulz, K. F. (2012) · Obstetrics & Gynecology · doi:10.1097/aog.0b013e31826af61a · Cited by 254
A dosimetric uncertainty analysis for photon‐emitting brachytherapy sources: Report of AAPM Task Group No. 138 and GEC‐ESTRO
DeWerd, L. A., Ibbott, G. S., Meigooni, A. S., Mitch, M. G., Rivard, M. J., Stump, K. E., Thomadsen, B., & Venselaar, J. L. M. (2011) · Medical Physics · doi:10.1118/1.3533720 · Cited by 247
Improving the odds of drug development success through human genomics: modelling study
Hingorani, A. D., Kuan, V., Finan, C., Krüger, F. A., Gaulton, A., Chopade, S., Sofat, R., MacAllister, R. J., Overington, J. P., Hemingway, H., et al. (2019) · Scientific Reports · doi:10.1038/s41598-019-54849-w · Cited by 252
The effect direction plot revisited: Application of the 2019 Cochrane Handbook guidance on alternative synthesis methods
Boon, M. H., & Thomson, H. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1458 · Cited by 247
Non-Replication and Inconsistency in the Genome-Wide Association Setting
Ioannidis, J. P. A. (2007) · Human Heredity · doi:10.1159/000103512 · Cited by 246
An umbrella review of the evidence associating diet and cancer risk at 11 anatomical sites
Papadimitriou, N., Markozannes, G., Kanellopoulou, A., Critselis, E., Alhardan, S., Karafousia, V., Kasimis, J. C., Katsaraki, C., Papadopoulou, A., Zografou, M. T., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-24861-8 · Cited by 254
Systematic review of isolation policies in the hospital management of methicillin-resistant Staphylococcus aureus: a review of the literature with epidemiological and economic modelling
Cooper, B. S., Stone, S. P., Kibbler, C. C., Cookson, B., Roberts, J., Medley, G. F., Duckworth, G., Lai, R., & Ebrahim, S. B. J. (2003) · Health Technology Assessment · doi:10.3310/hta7390 · Cited by 230
P Value and the Theory of Hypothesis Testing: An Explanation for New Researchers
Biau, D. J., Jolles, B. M., & Porcher, R. (2009) · Clinical Orthopaedics and Related Research · doi:10.1007/s11999-009-1164-4 · Cited by 225
The Fragility Index in Multicenter Randomized Controlled Critical Care Trials*
Ridgeon, E. E., Young, P. J., Bellomo, R., Mucchetti, M., Lembo, R., & Landoni, G. (2016) · Critical Care Medicine · doi:10.1097/ccm.0000000000001670 · Cited by 224
Tackle Injuries in Professional Rugby Union
Quarrie, K. L., & Hopkins, W. G. (2008) · The American Journal of Sports Medicine · doi:10.1177/0363546508316768 · Cited by 223
Low statistical power in biomedical science: a review of three human research domains
Dumas-Mallet, E., Button, K. S., Boraud, T., Gonon, F., & Munafò, M. R. (2017) · Royal Society Open Science · doi:10.1098/rsos.160254 · Cited by 208
Epigenome-wide Association Studies and the Interpretation of Disease -Omics
Birney, E., Smith, G. D., & Greally, J. M. (2016) · PLoS Genetics · doi:10.1371/journal.pgen.1006105 · Cited by 221

Authors: Sterne, J. A. C.
Affiliations: University of Bristol (GB)
Corresponding author: Sterne, J. A C
ORCID: Sterne, J. A C
Venue: BMJ
Publisher: BMJ
Published: 2001-01-27
Volume: 322
Issue: 7280
Pages: 226-231
ISSN: 0959-8138 (ISSN-L)
DOI: 10.1136/bmj.322.7280.226
PMC: PMC1119478
OpenAlex: W2025651303
Cited by: 1,466
References: 48
Citations by year: 2026: 48 · 2025: 64 · 2024: 75 · 2023: 63 · 2022: 63 · 2021: 82 · 2020: 72 · 2019: 82 · 2018: 75 · 2017: 79 · 2016: 74 · 2015: 48 · 2014: 56 · 2013: 50 · 2012: 53
FWCI: 45.33
Type: journal article
Language: en
Open access: green
www.ncbi.nlm.nih.gov
Locations: BMJ (publishedVersion)
Europe PMC (PubMed Central) (publishedVersion)
PubMed Central (publishedVersion)
PubMed (publishedVersion)
HighWire Press Open Archive (submittedVersion)
Topics: Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
Keywords: statistical significance, p-values, publication bias, false positives, null hypothesis, evidence evaluation
MeSH terms: Bayes Theorem, Data Interpretation, Statistical, Humans, Research Design, Statistics as Topic, History, 20th Century

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Sifting the evidence---what's wrong with significance tests? Another comment on the role of statistical methods

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks