Moving to a World Beyond “ p < 0.05”
Open accessAbstract
EDITORIAL: The editorial was written by the three editors acting as individuals and reflects their scientific views not an endorsed position of the American Statistical Association.Some of you expl...
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In hypothesis-testing studies using inferential statistics, investigators frequently confuse statistical significance and small p-values with biological or clinical importance.
“In hypothesis-testing studies using inferential statistics, investigators frequently confuse statistical significance and small p-values with biological or clinical importance [149].”
From Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0 (2020) · cites this paper as [149]
The differences among the means in KWH and CTM per season according to ANOVA results had p < 0.05.
“The differences among the means in KWH and CTM per season according to ANOVA results had p < 0.05, which means that there is high compatibility between the statistical model and the data [52].”
From Seasonality Effect Analysis and Recognition of Charging Behaviors of Electric Vehicles: A Data Science Approach (2020) · cites this paper as [52]
Cites
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068 · cited 1× · see citing works
“Furthermore, this false split into “worthy” and “unworthy” results leads to the selective reporting and publishing of results based on their statistical significance—the so-called “file drawer problem” (Rosenthal 1979).”
The ASA Statement on <i>p</i> -Values: Context, Process, and Purpose
Wasserstein, R. L., & Lazar, N. A. (2016) · The American Statistician · doi:10.1080/00031305.2016.1154108 · Cited by 6,306 · cited 3× · see citing works
“This concern is also addressed in the literature on equivalence testing (Wellek 2017).”
Self-citationThe New Statistics: Why and How
Cumming, G. (2014) · Psychological Science · doi:10.1177/0956797613504966 · Cited by 3,200 · see citing works
The Difference Between “Significant” and “Not Significant” is not Itself Statistically Significant
Gelman, A., & Stern, H. (2006) · The American Statistician · doi:10.1198/000313006x152649 · Cited by 1,316 · cited 1× · see citing works
“As Gelman and Stern (2006) famously observed, the difference between “significant” and “not significant” is not itself statistically significant.”
Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars
Mayo, D. G. (2018) · doi:10.1017/9781107286184 · Cited by 376 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Reporting animal research: Explanation and elaboration for the ARRIVE guidelines 2.0
Percie du Sert, N., Ahluwalia, A., Alam, S., Avey, M. T., Baker, M., Browne, W. J., Clark, A., Cuthill, I. C., Dirnagl, U., Emerson, M., et al. (2020) · PLOS Biology · doi:10.1371/journal.pbio.3000411 · Cited by 2,677 · see citing works
Meta-analysis of tumor- and T cell-intrinsic mechanisms of sensitization to checkpoint inhibition
Litchfield, K., Reading, J. L., Puttick, C., Thakkar, K., Abbosh, C., Bentham, R., Watkins, T. B. K., Rosenthal, R., Biswas, D., Rowan, A., et al. (2021) · Cell · doi:10.1016/j.cell.2021.01.002 · Cited by 893 · see citing works
Low-dose metformin targets the lysosomal AMPK pathway through PEN2
Ma, T., Tian, X., Zhang, B., Li, M., Wang, Y., Yang, C., Wu, J., Wei, X., Qu, Q., Yu, Y., et al. (2022) · Nature · doi:10.1038/s41586-022-04431-8 · Cited by 644 · see citing works
Rafi, Z., & Greenland, S. (2020) · BMC Medical Research Methodology · doi:10.1186/s12874-020-01105-9 · Cited by 243 · see citing works
Ten common statistical mistakes to watch out for when writing or reviewing a manuscript
Makin, T. R., & Orban de Xivry, J.-J. (2019) · eLife · doi:10.7554/elife.48175 · Cited by 238 · see citing works
Systems informed positive psychology
Kern, M. L., Williams, P., Spong, C., Colla, R., Sharma, K., Downie, A., Taylor, J. A., Sharp, S., Siokou, C., & Oades, L. G. (2020) · The Journal of Positive Psychology · doi:10.1080/17439760.2019.1639799 · Cited by 230 · see citing works
Interpretation of chronic pain clinical trial outcomes: IMMPACT recommended considerations
Smith, S. M., Dworkin, R. H., Turk, D. C., McDermott, M. P., Eccleston, C., Farrar, J. T., Rowbotham, M. C., Bhagwagar, Z., Burke, L. B., Cowan, P., et al. (2020) · Pain · doi:10.1097/j.pain.0000000000001952 · Cited by 194 · see citing works
Micronized vaginal progesterone to prevent miscarriage: a critical evaluation of randomized evidence
Coomarasamy, A., Devall, A. J., Brosens, J. J., Quenby, S., Stephenson, M. D., Sierra, S., Christiansen, O. B., Small, R., Brewin, J., Roberts, T. E., et al. (2020) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2019.12.006 · Cited by 163 · see citing works
The Community Resiliency Model® to promote nurse well-being
Grabbe, L., Higgins, M. K., Baird, M., Craven, P. A., & San Fratello, S. (2020) · Nursing Outlook · doi:10.1016/j.outlook.2019.11.002 · Cited by 136 · see citing works
Aspiring to greater intellectual humility in science
Hoekstra, R., & Vazire, S. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01203-8 · Cited by 117 · see citing works
Reproducibility and Rigor in Animal-Based Research
Macleod, M., & Mohan, S. (2019) · ILAR Journal · doi:10.1093/ilar/ilz015 · Cited by 104 · see citing works
Frontline physician burnout during the COVID-19 pandemic: national survey findings
Melnikow, J., Padovani, A., & Miller, M. (2022) · BMC Health Services Research · doi:10.1186/s12913-022-07728-6 · Cited by 101 · see citing works
Engel, R. S., Corsaro, N., Isaza, G. T., & McManus, H. D. (2022) · Criminology & Public Policy · doi:10.1111/1745-9133.12574 · Cited by 107 · see citing works
“15 Following Wasserstein et al. (2019), we detail the findings from Table 6 as marginally significant, and highlight all relevant parameters from each of the model specifications.”
To adjust, or not to adjust, for multiple comparisons
Hooper, R. (2025) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2025.111688 · Cited by 97 · see citing works
The Past, Present, and Future of Experimental Methods in the Social Sciences
Mize, T. D., & Manago, B. (2022) · Social Science Research · doi:10.1016/j.ssresearch.2022.102799 · Cited by 76 · see citing works
Scientific Credibility of Machine Translation Research: A Meta-Evaluation of 769 Papers
Marie, B., Fujita, A., & Rubino, R. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.566 · Cited by 75 · see citing works
Fyfe, E. R., de Leeuw, J. R., Carvalho, P. F., Goldstone, R. L., Sherman, J., Admiraal, D., Alford, L. K., Bonner, A., Brassil, C. E., Brooks, C. A., et al. (2021) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459211027575 · Cited by 59 · see citing works
Montessori education's impact on academic and nonacademic outcomes: A systematic review
Randolph, J. J., Bryson, A., Menon, L., Henderson, D. K., Kureethara Manuel, A., Michaels, S., rosenstein, D. L. W., McPherson, W., O'Grady, R., & Lillard, A. S. (2023) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.1002/cl2.1330 · Cited by 44 · see citing works
Cullen, B., Eichel, K., Lindahl, J. R., Rahrig, H., Kini, N., Flahive, J., & Britton, W. B. (2021) · PLOS ONE · doi:10.1371/journal.pone.0244838 · Cited by 43 · see citing works
Wolf, B., & Harbatkin, E. (2023) · Journal of Research on Educational Effectiveness · doi:10.1080/19345747.2022.2071364 · Cited by 45 · see citing works
Cassidy, S. A., Dimova, R., Giguère, B., Spence, J. R., & Stanley, D. J. (2019) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245919858072 · Cited by 39 · see citing works
Jha, S., Turcu, A. F., Sinaii, N., Brookner, B., Auchus, R. J., & Merke, D. P. (2021) · Journal of the Endocrine Society · doi:10.1210/jendso/bvaa192 · Cited by 39 · see citing works
Kahale, L. A., Khamis, A. M., Diab, B., Chang, Y., Lopes, L. C., Agarwal, A., Li, L., Mustafa, R. A., Koujanian, S., Waziry, R., et al. (2020) · BMJ · doi:10.1136/bmj.m2898 · Cited by 33 · see citing works
Dominguez-Jimenez, J. A., Campillo, J. E., Montoya, O. D., Delahoz, E., & Hernández, J. C. (2020) · Sustainability · doi:10.3390/su12187769 · Cited by 28 · see citing works
“05, which means that there is high compatibility between the statistical model and the data [52].”
BackgroundBrand, S., Ebner, K., Mikoteit, T., Lejri, I., Gerber, M., Beck, J., Holsboer-Trachsler, E., & Eckert, A. (2020) · Journal of Clinical Medicine · doi:10.3390/jcm9030667 · Cited by 27 · see citing works
King, W. C., Hinerman, A. S., & White, G. E. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.31593 · Cited by 19 · see citing works
Weiss de Souza, I. C., Kozasa, E. H., Bowen, S., Richter, K. P., Sartes, L. M. A., Colugnati, F. A. B., & Noto, A. R. (2020) · Nicotine and Tobacco Research · doi:10.1093/ntr/ntaa057 · Cited by 18 · see citing works
Prospective association of psychological pain and hopelessness with suicidal thoughts
Ballard, E. D., Farmer, C. A., Gerner, J., Bloomfield-Clagett, B., Park, L. T., & Zarate, C. A. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2022.04.033 · Cited by 16 · see citing works
Prefrontal tDCS modulates autonomic responses in COVID-19 inpatients
Pinto, T. P., Inácio, J. C., de Aguiar, E., Ferreira, A. S., Sudo, F. K., Tovar-Moll, F., & Rodrigues, E. C. (2023) · Brain Stimulation · doi:10.1016/j.brs.2023.03.001 · Cited by 11 · see citing works
The Medical Outreach Team of the Drug Information Association Bayesian Scientific Working Group, Bray, R., Hartley, A., Wenkert, D., Muehlemann, N., Natanegara, F., Harrell, F. E., Wang, F., & Clark, J. (2023) · Therapeutic Innovation & Regulatory Science · doi:10.1007/s43441-022-00482-1 · Cited by 9 · see citing works
Clinical indicators of the suicide crisis and response to ketamine
Ballard, E. D., Neely, L., Waldman, L., Greenstein, D., & Zarate, C. A. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.11.080 · Cited by 5 · see citing works
Manolov, R., & Onghena, P. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02230-1 · Cited by 5 · see citing works
Best, A. M., Lang, T. A., Greenberg, B. L., Gunsolley, J. C., Ioannidou, E., & _, _. (2024) · The Angle Orthodontist · doi:10.2319/ohstat_guidelines.1 · Cited by 2 · see citing works
Zhan, X., & Liu, X. (2026) · Educational Administration Quarterly · doi:10.1177/0013161x261418738
- Authors
- Wasserstein, R. L., Schirm, A. L., & Lazar, N. A.
- Affiliations
- American Statistical Association, Alexandria
- Mathematica Policy Research (US)
- University of Georgia (US)
- ORCID
- Lazar, Nicole A.
- Venue
- The American Statistician
- Publisher
- Informa UK Limited
- Published
- 2019-03-20
- Volume
- 73
- Issue
- sup1
- Pages
- 1-19
- ISSN
- 0003-1305 (ISSN-L)
- 1537-2731
- OpenAlex
- W2922853138
- Cited by
- 2,976
- References
- 5
- Citations by year
- 2026: 323 · 2025: 331 · 2024: 353 · 2023: 325 · 2022: 446 · 2021: 457 · 2020: 494 · 2019: 241 · 2018: 3 · 2017: 2
- FWCI
- 319.76
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- The American Statistician (publishedVersion, cc-by-nc-nd)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
- Keywords
- p-values, statistical significance thresholds, statistical inference, research reporting practices
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review