The ASA Statement on p -Values: Context, Process, and Purpose
Open accessAbstract
Emeritus of Mathematics and Statistics at
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 1 cited sources checked · 0 supported · 0 flagged
- Interpretationreasoning
Using statistical significance, usually p ≤ 0.05, as a permit to claim a scientific finding distorts the scientific process considerably.
“The widespread use of”
Quote in the paperSubstantial claim - Interpretationreasoning
Statistical significance is not the same as scientific, human, or economic importance.
“Statistical significance is not equivalent to scientific, human, or economic significance.”
Quote in the paperSubstantial claim - Interpretationnone given
A p-value close to 0.05, considered alone, provides only weak evidence against the null hypothesis.
“For example, a p-value near 0.05 taken by itself offers only weak evidence against the null hypothesis.”
Quote in the paperSubstantial claim - Interpretationreasoning
A p-value alone is not a good measure of evidence about a model or hypothesis.
“By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis.”
Quote in the paperSubstantial claim - Interpretationreasoning
Two identical estimated effects will have different p-values when the precision of their estimates differs.
“Similarly, identical estimated effects will have different p-values if the precision of the estimates differs.”
Quote in the paperNot a substantial claim
- Interpretationnone given
A p-value offers a way to summarize how incompatible a given dataset is with a proposed model.
“A p-value provides one approach to summarizing the incompatibility between a particular set of data and a proposed model for the data.”
Quote in the paperSubstantial claim - Interpretationnone given
Running several analyses and reporting only those with particular p-values makes the reported p-values essentially uninterpretable.
“Conducting multiple analyses of the data and reporting only those with certain p-values (typically those passing a significance threshold) renders the reported p-values essentially uninterpretable.”
Quote in the paperNot a substantial claim - Interpretationreasoning
When a researcher decides what to present based on statistical results, the results' valid interpretation is badly compromised unless the reader is told of that choice and its reason.
“Whenever a researcher chooses what to present based on statistical results, valid interpretation of those results is severely compromised if the reader is not informed of the choice and its basis.”
Quote in the paperSubstantial claim - Interpretationnone given
When the assumptions used to compute a p-value hold, a smaller p-value indicates greater statistical incompatibility between the data and the null hypothesis.
“The smaller the p-value, the greater the statistical incompatibility of the data with the null hypothesis, if the underlying assumptions used to calculate the p-value hold.”
Quote in the paperSubstantial claim - Interpretationreasoning
Valid scientific conclusions from p-values and related statistics require knowing how many and which analyses were done and how they were chosen for reporting.
“Valid scientific conclusions based on p-values and related statistics cannot be drawn without at least knowing how many and which analyses were conducted, and how those analyses (including p-values) were selected for reporting.”
Quote in the paperSubstantial claim - Interpretationreasoning
Reducing analysis or inference to mechanical cutoff rules like p < 0.05 can produce false beliefs and bad decisions.
“Practices that reduce data analysis or scientific inference to mechanical”
Quote in the paperSubstantial claim - Interpretationreasoning
Selecting promising findings, also called data dredging, significance chasing, selective inference, or p-hacking, creates a spurious surplus of significant results in the literature.
“Cherrypicking promising findings, also known by such terms as data dredging, significance chasing, significance questing, selective inference, and”
Quote in the paperSubstantial claim
- Classified as probably not a substantial claim. 10
- May combine two claims. 10
Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.
FindingreasoningThis situation has led some journals to discourage p-values and some scientists and statisticians to recommend abandoning them, with arguments that have barely changed since p-values were introduced.
“This has led to some scientific journals discouraging the use of p-values, and some scientists and statisticians recommending their abandonment, with some arguments essentially unchanged since p-values were first introduced.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Findingnone givenThe statistical community has been deeply worried about the reproducibility and replicability of scientific conclusions.
“The statistical community has been deeply concerned about issues of reproducibility and replicability of scientific conclusions.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone givenThe p-value can be a useful statistical measure, but it is frequently misused and misinterpreted.
“While the p-value can be a useful statistical measure, it is commonly misused and misinterpreted.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
InterpretationreasoningA conclusion does not switch from false to true at a threshold.
“A conclusion does not immediately become”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Interpretationnone givenThose other methods also depend on assumptions, but they may address effect size, its uncertainty, or hypothesis correctness more directly.
“All these measures and approaches rely on further assumptions, but they may more directly address the size of an effect (and its associated uncertainty) or whether the hypothesis is correct.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone givenThe validity and reproducibility of scientific conclusions depend on factors beyond the statistical methods used.
“The validity of scientific conclusions, including their reproducibility, depends on more than the statistical methods themselves.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Interpretationnone givenA p-value describes data relative to a specified hypothetical explanation; it does not describe that explanation itself.
“It is a statement about data in relation to a specified hypothetical explanation, and is not a statement about the explanation itself.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
InterpretationreasoningA p-value without context or other evidence gives limited information.
“Researchers should recognize that a p-value without context or other evidence provides limited information.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningA relatively large p-value does not mean the null hypothesis is supported; other hypotheses may fit the observed data equally well or better.
“Likewise, a relatively large p-value does not imply evidence in favor of the null hypothesis; many other hypotheses may be equally or more consistent with the observed data.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningThat incompatibility can be taken as evidence against the null hypothesis or against the assumptions behind it.
“This incompatibility can be interpreted as casting doubt on or providing evidence against the null hypothesis or the underlying assumptions.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.
Interpretationnone givenWell-chosen techniques, properly done analyses, and correct interpretation of statistical results are also crucial for sound conclusions and proper representation of uncertainty.
“Appropriately chosen techniques, properly conducted analyses and correct interpretation of statistical results also play a key role in ensuring that conclusions are sound and that uncertainty surrounding them is represented properly.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationcitationMisunderstanding or misuse of statistical inference is only one contributor to the reproducibility crisis, but the statistical community considers it an important one.
“Misunderstanding or misuse of statistical inference is only one cause of the”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites Peng 2015 source
May combine two claims.
InterpretationreasoningSmaller p-values do not necessarily mean larger or more important effects, and larger p-values do not necessarily mean no effect or no importance.
“Smaller p-values do not necessarily imply the presence of larger or more important effects, and larger p-values do not imply a lack of importance or even lack of effect.”
Quote in the paperSubstantial claimMay combine two claims.
InterpretationreasoningA p-value does not give the probability that the hypothesis under study is true, nor the probability that chance alone generated the data.
“P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
InterpretationreasoningBecause p-values are often misused and misunderstood, some statisticians choose to supplement or replace them with other methods.
“In view of the prevalent misuses of and misconceptions concerning p-values, some statisticians prefer to supplement or even replace p-values with other approaches.”
Quote in the paperNot a substantial claim
The paper relies on this without support.
There is widespread consensus in the statistical community on the principles articulated in the ASA statement.
“according to widespread consensus in the statistical community”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The large scale of the Tabula Muris Senis dataset provides substantial power to predict subtle changes in LR interactions, but the biological complexity, dependencies among communication pairs, and multiple testing burden require that p-values be interpreted within a broader biological and statistical context.
“While the large scale of the Tabula Muris Senis dataset provides substantial power to predict subtle changes in LR interactions, the biological complexity, dependencies among communication pairs, and multiple testing burden require that these p-values be interpreted within a broader biological and statistical context [33].”
From Quantitative profiling of lifespan-dependent cell-cell communication potential reveals dynamic ligand-receptor network shifts across mouse tissues (2026) · cites this paper as [33]
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
Experimental — accuracy not yet measuredThese types of incorrect descriptions contribute to confirmation bias, overinterpretation of results, and reporting correlation as implied causation.
“These types of incorrect descriptions contribute to confirmation bias, overinterpretation of results, and reporting correlation as implied causation (Wasserstein and Lazar 2016).”
From Unethical Numbers? A Meta-analysis of Library Learning Analytics Studies (2019) · cites this paper as Wasserstein and Lazar 2016
In principle 3 of the ASA statement (page not available in our copy), the paper says treating statistical significance as proof of a finding can distort scientific conclusions; principle 4 warns against cherry-picking results. These passages support the concern about overinterpretation, but they do not say that incorrect descriptions contribute to confirmation bias or to reporting correlation as implied causation. (LLM, conf=0.84)
Experimental — accuracy not yet measuredRelatively large-sample-size studies are likely to obtain statistically significant findings in relatively small variations in measures such as GPA.
“This is especially true of relatively large-sample-size studies (as most learning analytics studies are), which are likely to obtain statistically significant findings in relatively small variations in measures such as GPA (Field 2017; Salkind 2016 Wasserstein and Lazar 2016).”
From Unethical Numbers? A Meta-analysis of Library Learning Analytics Studies (2019) · cites this paper as Wasserstein and Lazar 2016
In 2016, the American Statistical Association released a statement on statistical significance and the P value with six principles underlying the proper use and interpretation of this statistical tool.
“In 2016, the American Statistical Association released a statement on statistical significance and the P value with six principles underlying the proper use and interpretation of this statistical tool (77).”
From Misrepresentation and distortion of research in biomedical literature (2018) · cites this paper as [77]
The authors refrained from labeling exploratory analyses as statistically significant, following best-practice recommendations.
“Although we also present p values for exploratory analyses, we refrain from labeling any of them as statistically significant, following best-practice recommendations for statistical analysis [43].”
From Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature (2020) · cites this paper as [43]
Contrary to IJE author recommendations and recent guidance from the American Statistics Society, Gao et al. largely base their conclusions on findings with a P-value equivalent to <0.05 after multiple testing.
“Contrary to IJE author recommendations and recent guidance from the American Statistics Society, 16 Gao et al. largely base their conclusions on findings with a P -value equivalent to < 0.05 after multiple testing.”
From Commentary: Two-sample Mendelian randomization: opportunities and challenges (2016) · cites this paper as [16]
We focused on 95% CIs rather than statistical significance, interpreting effects as 'suggestive' if the 95% CI bound was slightly above/below the null value.
“Following guidance,50 we focused on 95% CIs, rather than statistical significance. For example, unlike conventional interpretations, we did not outright interpret an effect estimate “non‐significant” if the lower or upper bound of the 95% CI was slightly above/below the null value; we interpreted them as “suggestive” of an effect.”
From Third‐wave cognitive behaviour therapies for weight management: A systematic review and network meta‐analysis (2020) · cites this paper as [50]
Cites
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567 · see citing works
(1947) · The American Statistician · doi:10.1080/00031305.1947.10501534 · Cited by 3,745 · see citing works
The Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis
Hoenig, J. M., & Heisey, D. M. (2001) · The American Statistician · doi:10.1198/000313001300339897 · Cited by 2,046 · see citing works
Statistics notes: Absence of evidence is not evidence of absence
Altman, D. G., & Bland, J. M. (1995) · BMJ · doi:10.1136/bmj.311.7003.485 · Cited by 1,946 · see citing works
Smithson, M. J. (2000) · SAGE Publications Ltd eBooks · doi:10.4135/9781446218525 · Cited by 1,871 · see citing works
Bayesian statistical inference for psychological research.
Edwards, W., Lindman, H., & Savage, L. J. (1963) · Psychological Review · doi:10.1037/h0044139 · Cited by 1,734 · see citing works
Scientific method: Statistical errors
Nuzzo, R. (2014) · Nature · doi:10.1038/506150a · Cited by 1,536 · see citing works
Why Most Discovered True Associations Are Inflated
Ioannidis, J. P. A. (2008) · Epidemiology · doi:10.1097/ede.0b013e31818131e7 · Cited by 1,495 · see citing works
Sterne, J. A. C. (2001) · BMJ · doi:10.1136/bmj.322.7280.226 · Cited by 1,466 · see citing works
Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
Ioannidis, J. P. A. (2005) · JAMA · doi:10.1001/jama.294.2.218 · Cited by 1,465 · see citing works
The Difference Between “Significant” and “Not Significant” is not Itself Statistically Significant
Gelman, A., & Stern, H. (2006) · The American Statistician · doi:10.1198/000313006x152649 · Cited by 1,316 · see citing works
Toward Evidence-Based Medical Statistics. 1: The P Value Fallacy
Goodman, S. N. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-12-199906150-00008 · Cited by 1,188 · see citing works
The Statistical Crisis in Science
Gelman, A., & Loken, E. (2014) · American Scientist · doi:10.1511/2014.111.460 · Cited by 963 · see citing works
Toward Evidence-Based Medical Statistics. 2: The Bayes Factor
Goodman, S. N. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-12-199906150-00019 · Cited by 913 · see citing works
Gigerenzer, G. (2004) · The Journal of Socio-Economics · doi:10.1016/j.socec.2004.09.033 · Cited by 890 · see citing works
Revised standards for statistical evidence
Johnson, V. E. (2013) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1313476110 · Cited by 808 · see citing works
The fallacy of the null-hypothesis significance test.
Rozeboom, W. W. (1960) · Psychological Bulletin · doi:10.1037/h0042040 · Cited by 738 · see citing works
The cult of statistical significance: how the standard error costs us jobs, justice, and lives
(2008) · Choice Reviews Online · doi:10.5860/choice.46-0372 · Cited by 716 · see citing works
A dirty dozen: twelve p-value misconceptions.
Goodman, S. (2008) · Seminars in hematology (Print) · doi:10.1053/j.seminhematol.2008.04.003 · Cited by 694 · see citing works
Berger, J. O., & Delampady, M. (1987) · Statistical Science · doi:10.1214/ss/1177013238 · Cited by 664 · see citing works
Trafimow, D., & Marks, M. (2015) · Basic and Applied Social Psychology · doi:10.1080/01973533.2015.1012991 · Cited by 612 · see citing works
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · PsycEXTRA Dataset · doi:10.1037/e519702015-014 · Cited by 462 · see citing works
Low P-Values or Narrow Confidence Intervals: Which Are More Durable?
Poole, C. (2001) · Epidemiology · doi:10.1097/00001648-200105000-00005 · Cited by 363 · see citing works
Drinking Water Source and Chlorination Byproducts II. Risk of Colon and Rectal Cancers
Hildesheim, M. E., Cantor, K. P., Lynch, C. F., Dosemeci, M., Lubin, J. H., Alavanja, M. C. R., & Craun, G. F. (1998) · Epidemiology · doi:10.1097/00001648-199801000-00008 · Cited by 357 · see citing works
Statistical significance tests.
Cox, D. (1982) · British Journal of Clinical Pharmacology · doi:10.1111/j.1365-2125.1982.tb01987.x · Cited by 335 · see citing works
Sifting the evidence—what's wrong with significance tests?
Sterne, J. A. C., & Smith, G. D. (2001) · Physical Therapy · doi:10.1093/ptj/81.8.1464 · Cited by 301 · see citing works
Rothman, K. J. (1978) · New England Journal of Medicine · doi:10.1056/nejm197812142992410 · Cited by 281 · see citing works
PValues: What They are and What They are Not
Schervish, M. J. (1996) · The American Statistician · doi:10.1080/00031305.1996.10474380 · Cited by 248 · see citing works
Beyond the confidence interval.
Poole, C. (1987) · American Journal of Public Health · doi:10.2105/ajph.77.2.195 · Cited by 229 · see citing works
The Reproducibility Crisis in Science: A Statistical Counterattack
Peng, R. (2015) · Significance · doi:10.1111/j.1740-9713.2015.00827.x · Cited by 224 · see citing works
The ongoing tyranny of statistical significance testing in biomedical research
Stang, A., Poole, C., & Kuß, O. (2010) · European Journal of Epidemiology · doi:10.1007/s10654-010-9440-x · Cited by 221 · see citing works
P Values: What They Are and What They Are Not
Schervish, M. J. (1996) · The American Statistician · doi:10.2307/2684655 · Cited by 200 · see citing works
Rothman, K. J. (1986) · Annals of Internal Medicine · doi:10.7326/0003-4819-105-3-445 · Cited by 144 · see citing works
Testing Fisher, Neyman, Pearson, and Bayes
Christensen, R. (2005) · The American Statistician · doi:10.1198/000313005x20871 · Cited by 132 · see citing works
Absence of evidence is not evidence of absence
Altman, D. G., & Bland, J. M. (1996) · Australian Veterinary Journal · doi:10.1111/j.1751-0813.1996.tb13786.x · Cited by 120 · see citing works
Null misinterpretation in statistical testing and its impact on health risk assessment
Greenland, S. (2011) · Preventive Medicine · doi:10.1016/j.ypmed.2011.08.010 · Cited by 99 · see citing works
Nonsignificance Plus High Power Does Not Imply Support for the Null Over the Alternative
Greenland, S. (2012) · Annals of Epidemiology · doi:10.1016/j.annepidem.2012.02.007 · Cited by 97 · see citing works
Phillips, C. V. (2004) · BMC Medical Research Methodology · doi:10.1186/1471-2288-4-20 · Cited by 94 · see citing works
Uniformly most powerful Bayesian tests
Johnson, V. E. (2013) · The Annals of Statistics · doi:10.1214/13-aos1123 · Cited by 87 · see citing works
Odds are, it's wrong: Science fails to face the shortcomings of statistics
Siegfried, T. (2010) · Science News · doi:10.1002/scin.5591770721 · Cited by 84 · see citing works
Curbing type I and type II errors
Rothman, K. J. (2010) · European Journal of Epidemiology · doi:10.1007/s10654-010-9437-5 · Cited by 80 · see citing works
UNIFORMLY MOST POWERFUL BAYESIAN TESTS
Johnson, V. E. (2013) · OakTrust (Texas A&M University Libraries) · Cited by 60 · see citing works
Lew, M. J. (2012) · British Journal of Pharmacology · doi:10.1111/j.1476-5381.2012.01931.x · Cited by 56 · see citing works
Multiplicities in Cancer Research: Ubiquitous and Necessary Evils
Berry, D. A. (2012) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/djs301 · Cited by 50 · see citing works
Teaching hypothesis tests – time for significant change?
Sterne, J. A. C. (2002) · Statistics in Medicine · doi:10.1002/sim.1129 · Cited by 43 · see citing works
Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
Ioannidis, J. P. A. (2005) · ACC Current Journal Review · doi:10.1016/j.accreview.2005.09.021 · Cited by 41 · see citing works
ASA Statement on Value-Added Models
Morganstein, D. R., & Wasserstein, R. (2014) · Statistics and Public Policy · doi:10.1080/2330443x.2014.956906 · Cited by 36 · see citing works
Self-citationThe Validus Medicus and a new gold standard
Ziliak, S. T. (2010) · The Lancet · doi:10.1016/s0140-6736(10)61174-9 · Cited by 15 · see citing works
That confounded P-value revisited
Stang, A., & Rothman, K. J. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.03.004 · Cited by 11 · see citing works
&NA; (1998) · Epidemiology · doi:10.1097/00001648-199801000-00004 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- Leek, J. (2014), “On the Scalability of Statistical Procedures: Why the p-Value Bashers Just Don't Get It,” Simply Statistics Blog, Available at http://simplystatistics.org/2014/02/14/on-the-scalability-of-statistical-procedures-why-the-p-value-bashers-just-dont-get-it/.
- Gelman A. and Loken E. (2014) “The Statistical Crisis in Science [online] ” American Scientist 102. Available at http://www.americanscientist.org/issues/feature/2014/6/the-statistical-crisis-in-science · DOI · Paper page
- Greenland, S., and Poole, C. (2011), “Problems in Common Interpretations of Statistics in Scientific Articles, Expert Reports, and Testimony,” Jurimetrics, 51, 113–129.
- 10.1214/13-AOS1123 · DOI
- Lavine, M. (1999), “What is Bayesian Statistics and Why Everything Else is Wrong,” UMAP Journal, 20, 2.
Cited by
Sorted by citations
Sensitivity analysis in observational research: Introducing the E-value
VanderWeele, T. J., & Ding, P. (2017) · Annals of Internal Medicine · doi:10.7326/m16-2607 · Cited by 6,350 · see citing works
“While potentially subject to mis-use and misinterpretation (1,5-12), the p-value can be informative as a continuous measure of evidence as to whether an association is present.”
BackgroundMoving to a World Beyond “ <i>p</i> < 0.05”
Wasserstein, R. L., Schirm, A. L., & Lazar, N. A. (2019) · The American Statistician · doi:10.1080/00031305.2019.1583913 · Cited by 2,976 · see citing works
“This concern is also addressed in the literature on equivalence testing (Wellek 2017).”
Self-citationRedefine statistical significance
Benjamin, D. J., Berger, J. O., Johannesson, M., Nosek, B. A., Wagenmakers, E.-J., Berk, R., Bollen, K. A., Brembs, B., Brown, L., Camerer, C., et al. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0189-z · Cited by 2,772 · see citing works
The preregistration revolution
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708274114 · Cited by 1,998 · see citing works
“05 are not actually very unlikely in typical research practices (31), leading some researchers to recommend 0.”
BackgroundContemporary Guidance for Stated Preference Studies
Johnston, R. J., Boyle, K., Adamowicz, W., Bennett, J., Brouwer, R., Cameron, T. A., Hanemann, W. M., Hanley, N. D., Ryan, M., Riccardo Scarpa, A., et al. (2017) · Journal of the Association of Environmental and Resource Economists · doi:10.1086/691697 · Cited by 1,458 · see citing works
“One comment is particularly relevant to econometric analyses of SP data: “P-values can indicate how incompatible the data are with a specified statistical model” (Wasserstein and Lazar 2016, 131).”
BackgroundKruschke, J. K., & Liddell, T. M. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-016-1221-4 · Cited by 1,288 · see citing works
“Among the most recent and prominent cautions against NHST is a statement from the American Statistical Association (ASA; Wasserstein & Lazar, 2016), which featured six principles for properly interpreting p values.”
Do “brain-training” programs work?
Simons, D. J., Boot, W. R., Charness, N., Gathercole, S. E., Chabris, C. F., Hambrick, D. Z., & Stine-Morrow, E. A. L. (2016) · Psychological Science in the Public Interest · doi:10.1177/1529100616661983 · Cited by 1,183 · see citing works
Indices of Effect Existence and Significance in the Bayesian Framework
Makowski, D., Ben-Shachar, M. S., Chen, S. H. A., & Lüdecke, D. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02767 · Cited by 983 · see citing works
“The p-value represents the probability that for a given statistical model, when the null hypothesis is true, the effect would be greater than or equal to the observed coefficient (Wasserstein and Lazar, 2016).”
BackgroundThe natural selection of bad science
Smaldino, P. E., & McElreath, R. (2016) · Royal Society Open Science · doi:10.1098/rsos.160384 · Cited by 860 · see citing works
“In March 2016, the American Statistical Association published a set of corrective guidelines about the use and misuse of p-values [2].”
BackgroundForecasting: theory and practice
Petropoulos, F., Apiletti, D., Assimakopoulos, V., Babai, M. Z., Barrow, D. K., Taieb, S. B., Bergmeir, C., Bessa, R., Bijak, J., Boylan, J., et al. (2020) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2021.11.001 · Cited by 630 · see citing works
van Laar, E., van Deursen, A. J. A. M., van Dijk, J. A. G. M., & de Haan, J. (2020) · Sage Open · doi:10.1177/2158244019900176 · Cited by 927 · see citing works
“Critics call for a broader approach because p -values are commonly misused and misinterpreted (Wasserstein & Lazar, 2016).”
BackgroundThe role of multiple global change factors in driving soil functions and microbial biodiversity
Rillig, M. C., Ryo, M., Lehmann, A., Aguilar‐Trigueros, C. A., Buchert, S., Wulf, A., Iwasaki, A., Roy, J., & Yang, G. (2019) · Science · doi:10.1126/science.aay2832 · Cited by 899 · see citing works
Saltelli, A., Aleksankina, K., Becker, W., Fennell, P. J., Ferretti, F., Holst, N., Li, S., & Wu, Q. (2019) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2019.01.012 · Cited by 828 · see citing works
Commentary: Two-sample Mendelian randomization: opportunities and challenges
Lawlor, D. A. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyw127 · Cited by 810 · see citing works
Machine Learning for Precision Psychiatry: Opportunities and Challenges
Bzdok, D., & Meyer‐Lindenberg, A. (2017) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2017.11.007 · Cited by 833 · see citing works
“Classical significance testing is probably most often used to decide between two possible outcomes, expressed in the null and alternative hypothesis, by considering the probability of obtaining an equal or more extreme effect in the data under the null hypothesis (24).”
BackgroundMethodResultAbandon Statistical Significance
McShane, B. B., Gal, D., Gelman, A., Robert, C. P., & Tackett, J. L. (2019) · The American Statistician · doi:10.1080/00031305.2018.1527253 · Cited by 852 · see citing works
“s, the external evidence for the phenomenon under study, and the validity ofassumptions that underlie the data analysis” and cautions that “no single index should substitute for scientific reasoning;” Wasserstein and Lazar (2016)). Second, as discussed above, while we believe treating the p-value continuously rather th…”
BackgroundInfluentialRejecting or Accepting Parameter Values in Bayesian Estimation
Kruschke, J. K. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918771304 · Cited by 734 · see citing works
“Discrete decisions should be avoided if possible, because such decisions encourage people to ignore the magnitude of the parameter value and its uncertainty (e.g., Cumming, 2014; Kruschke & Liddell, 2018b; Wasserstein & Lazar, 2016, and many references cited therein).”
BackgroundLederer, D. J., Bell, S. C., Branson, R. D., Chalmers, J. D., Marshall, R., Maslove, D. M., Ost, D. E., Punjabi, N. M., Schatz, M., Smyth, A. R., et al. (2019) · Annals of the American Thoracic Society · doi:10.1513/annalsats.201808-564ps · Cited by 715 · see citing works
Best practices in data analysis and sharing in neuroimaging using MRI
Nichols, T. E., Das, S., Eickhoff, S. B., Evans, A. C., Glatard, T., Hanke, M., Kriegeskorte, N., Milham, M. P., Poldrack, R. A., Poline, J.-B., et al. (2017) · Nature Neuroscience · doi:10.1038/nn.4500 · Cited by 714 · see citing works
Best practice guidance for linear mixed-effects models in psychological science
Meteyard, L., & Davies, R. (2019) · Journal of Memory and Language · doi:10.1016/j.jml.2020.104092 · Cited by 664 · see citing works
A Statistical Significance Test for Necessary Condition Analysis
Dul, J., van der Laan, E., & Kuik, R. (2018) · Organizational Research Methods · doi:10.1177/1094428118795272 · Cited by 718 · see citing works
“Just like any other statistical method that uses the p value for statistical inference, the proposed approximate permutation test has all the limitations of the p value (Wasserstein & Lazar, 2016).”
MethodGaudel, A., Cooper, O. R., Ancellet, G. M., Barret, B., Boynard, A., Burrows, J. P., Clerbaux, C., Coheur, P.-F., Cuesta, J., Cuevas, E., et al. (2018) · Elementa Science of the Anthropocene · doi:10.1525/elementa.291 · Cited by 668 · see citing works
A solution to minimum sample size for regressions
Jenkins, D. G., & Quintana-Ascencio, P. F. (2020) · PLOS ONE · doi:10.1371/journal.pone.0229345 · Cited by 665 · see citing works
“Power analyses carry forward fundamental problems with null hypothesis inference, which is the long-standing basis for statistical analyses but that has been recently and widely criticized for several reasons [25,26,27,28,29].”
BackgroundThe Proposal to Lower P Value Thresholds to .005
Ioannidis, J. P. A. (2018) · JAMA · doi:10.1001/jama.2018.1536 · Cited by 655 · see citing works
Rigdon, E. E. (2016) · European Management Journal · doi:10.1016/j.emj.2016.05.006 · Cited by 676 · see citing works
Rewriting results sections in the language of evidence
Muff, S., Nilsen, E. B., O’Hara, R. B., & Nater, C. R. (2021) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2021.10.009 · Cited by 639 · see citing works
Detecting RNA modifications in the epitranscriptome: predict and validate
Helm, M., & Motorin, Y. (2017) · Nature Reviews Genetics · doi:10.1038/nrg.2016.169 · Cited by 641 · see citing works
The Future of Sensitivity Analysis: An essential discipline for systems modeling and policy support
Razavi, S., Jakeman, A. J., Saltelli, A., Prieur, C., Iooss, B., Borgonovo, E., Plischke, E., Piano, S. L., Iwanaga, T., Becker, W., et al. (2020) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2020.104954 · Cited by 651 · see citing works
“…’ (Saltelli and Funtowicz, 2017; Saltelli, 2018) where misuse or abuse of statistics (Stark and Saltelli, 2018; Gigerenzer and Marewski, 2015; Leek et al., 2017; Singh Chawla, 2017; Wasserstein and Lazar, 2016; Gigerenzer, 2018) is often cited as the root cause of the crisis (Ioannidis, 2005).”
Yu, Z., Guindani, M., Grieco, S. F., Chen, L., Holmes, T. C., & Xu, X. (2022) · Neuron · doi:10.1016/j.neuron.2021.10.030 · Cited by 563 · see citing works
“…research articles 696 (Aickin and Gensler, 1996; Altman and Bland, 1995; Benjamin and Berger, 2019; Benjamini and 697 Hochberg, 1995; Gelman and Stern, 2006; Goodman, 2008; Holm, 1979; McHugh, 2011; Storey, 698 2002; Wasserstein and Lazar, 2016) or consult with experienced statisticians.”
Douglas, E. J., Shepherd, D. A., & Prentice, C. (2019) · Journal of Business Venturing · doi:10.1016/j.jbusvent.2019.105970 · Cited by 545 · see citing works
Instagram photos reveal predictive markers of depression
Reece, A. G., & Danforth, C. M. (2017) · EPJ Data Science · doi:10.1140/epjds/s13688-017-0110-z · Cited by 519 · see citing works
Carrasco, J., García, S., Rueda, M. D. M., Das, S., & Herrera, F. (2020) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2020.100665 · Cited by 533 · see citing works
Presidential Address: The Scientific Outlook in Financial Economics
Harvey, C. R. (2017) · The Journal of Finance · doi:10.1111/jofi.12530 · Cited by 454 · see citing works
Liengaard, B. D., Sharma, P. N., Hult, G. T. M., Jensen, M. B., Sarstedt, M., Hair, J. F., & Ringle, C. M. (2020) · Decision Sciences · doi:10.1111/deci.12445 · Cited by 580 · see citing works
Contract cheating: a survey of australian university students
Bretag, T., Harper, R., Burton, M., Ellis, C., Newton, P., Rozenberg, P., Saddiqui, S., & van Haeringen, K. (2019) · Studies in Higher Education · doi:10.1080/03075079.2018.1462788 · Cited by 476 · see citing works
“Wasserstein and Lazar (2016) state that often the null hypothesis postulates the absence of an effect, such as no difference between two groups, or the absence of a relationship between a factor and an outcome.”
BackgroundCross validation for model selection: A review with examples from ecology
Yates, L. A., Aandahl, Z., Richards, S. A., & Brook, B. W. (2022) · Ecological Monographs · doi:10.1002/ecm.1557 · Cited by 512 · see citing works
Pre-registration in social psychology—A discussion and suggested template
Veer, A. V. '., & Giner‐Sorolla, R. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2016.03.004 · Cited by 436 · see citing works
Chromosome errors in human eggs shape natural fertility over reproductive life span
Gruhn, J. R., Zielinska, A. P., Shukla, V., Blanshard, R., Capalbo, A., Cimadomo, D., Nikiforov, D., Chan, A. C.-H., Newnham, L. J., Vogel, I., et al. (2019) · Science · doi:10.1126/science.aav7321 · Cited by 466 · see citing works
Treatment selection in depression.
Cohen, Z. D., & DeRubeis, R. J. (2018) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050817-084746 · Cited by 435 · see citing works
“Reliance on tests of significance can result in misleading impressions about the importance of predictive variables (Nuzzo 2014, Wasserstein & Lazar 2016).”
BackgroundLeo, G. D., & Sardanelli, F. (2020) · European Radiology Experimental · doi:10.1186/s41747-020-0145-y · Cited by 489 · see citing works
“A special issue in The American Statistician [5] presented 37 papers on “Statistical inference in the 21st century: a world beyond p < 0.”
BackgroundInfluentialLichtenberg, E. M., Kennedy, C. M., Kremen, C., Batáry, P., Berendse, F., Bommarco, R., Bosque‐Pérez, N. A., Carvalheiro, L. G., Snyder, W. E., Williams, N. M., et al. (2017) · Global Change Biology · doi:10.1111/gcb.13714 · Cited by 448 · see citing works
“This accords with a recent policy statement by the American Statistical Association (Wasserstein & Lazar, 2016), which notes that reliance on arbitrary alpha values can lead to erroneous conclusions.”
ResultBayesian Analysis Reporting Guidelines
Kruschke, J. K. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01177-7 · Cited by 462 · see citing works
Abandon Statistical SignificancePreprint
McShane, B. B., Gal, D., Gelman, A., Robert, C. P., & Tackett, J. L. (2019) · HAL · Cited by 449 · see citing works
OECD (2009) · Programme for international student assessment/Internationale Schulleistungsstudie · doi:10.1787/19963777 · Cited by 446 · see citing works
Comparing multiple comparisons: practical guidance for choosing the best multiple comparisons test
Midway, S. R., Robertson, M. D., Flinn, S., & Kaller, M. D. (2020) · PeerJ · doi:10.7717/peerj.10387 · Cited by 452 · see citing works
“…and concerns over the use of these criteria in this manner are well documented in the literature (Stephens et al., 2005; Midway et al. (2020), PeerJ, DOI 10.7717/peerj.10387 16/26 Stephens, Buskirk & Del Rio, 2007;Murtaugh, 2014;Wasserstein & Lazar, 2016;Wasserstein, Schrim & Lazar, 2019).”
BackgroundInfluentialBayesian data analysis for newcomers
Kruschke, J. K., & Liddell, T. M. (2017) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1272-1 · Cited by 409 · see citing works
Misinformation in and about science
West, J. D., & Bergstrom, C. T. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1912444117 · Cited by 391 · see citing works
Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars
Mayo, D. G. (2018) · doi:10.1017/9781107286184 · Cited by 376 · see citing works
Validation practices for satellite soil moisture retrievals: What are (the) errors?
Gruber, A., Lannoy, G. J. M. D., Albergel, C., Al‐Yaari, A., Brocca, L., Calvet, J., Colliander, A., Cosh, M. H., Crow, W. T., Dorigo, W. A., et al. (2020) · Remote Sensing of Environment · doi:10.1016/j.rse.2020.111806 · Cited by 372 · see citing works
Halsey, L. (2019) · Biology Letters · doi:10.1098/rsbl.2019.0174 · Cited by 327 · see citing works
Kers, J. G., & Saccenti, E. (2022) · Frontiers in Microbiology · doi:10.3389/fmicb.2021.796025 · Cited by 359 · see citing works
“…(p-hacking) (Simmons et al., 2011). p-Hacking (also called data dredging, significance chasing, significance questing, or selective inference (Wasserstein and Lazar, 2016)) is the improper use of data (like adding or removing observations) or statistical procedures (like applying many…”
MethodNew Guidelines for Statistical Reporting in the Journal
Harrington, D. P., D’Agostino, R. B., Gatsonis, C., Hogan, J. W., Hunter, D. J., Normand, S. T., Drazen, J. M., & Hamel, M. B. (2019) · New England Journal of Medicine · doi:10.1056/nejme1906559 · Cited by 337 · see citing works
Quality of life under the COVID-19 quarantine
Ferreira, L. N., Pereira, L. N., da Fé Brás, M., & Ilchuk, K. (2021) · Quality of Life Research · doi:10.1007/s11136-020-02724-x · Cited by 318 · see citing works
Kline, R. B. (2018) · Canadian Studies in Population · doi:10.25336/csp29418 · Cited by 336 · see citing works
Guidelines for Reporting of Statistics for Clinical Research in Urology
Assel, M., Sjoberg, D., Elders, A., Wang, X., Huo, D., Botchway, A., Delfino, K., Fan, Y., Zhao, Z., Koyama, T., et al. (2019) · European Urology · doi:10.1016/j.eururo.2018.12.014 · Cited by 335 · see citing works
Reporting effect sizes in original psychological research: A discussion and tutorial.
Pek, J. C. X., & Flora, D. B. (2017) · Psychological Methods · doi:10.1037/met0000126 · Cited by 321 · see citing works
“For instance, an overemphasis on statistical significance and p values is addressed by the recent American Statistical Association statement which advocates that “[n]o single index should substitute for scientific reasoning” (Wasserstein & Lazar, 2016, p. 12).”
BackgroundWhat’s in a p? Reassessing best practices for conducting and reporting hypothesis-testing research
Meyer, K., Witteloostuijn, A., & Beugelsdijk, S. (2017) · Journal of International Business Studies · doi:10.1057/s41267-017-0078-8 · Cited by 337 · see citing works
Coens, C., Pe, M. L., Dueck, A. C., Sloan, J. A., Basch, E., Calvert, M., Campbell, A., Cleeland, C. S., Cocks, K., Collette, L., et al. (2020) · The Lancet Oncology · doi:10.1016/s1470-2045(19)30790-9 · Cited by 327 · see citing works
Greenland, S. (2019) · The American Statistician · doi:10.1080/00031305.2018.1529625 · Cited by 313 · see citing works
“…supplies against the hypothesis; and most of all 4. avoiding dichotomization of p or comparison of p to α-levels (see, e.g., Poole 1987a; Hoekstra et al. 2006; Hurlbert and Lombardi 2009; Greenland et al. 2016, 2017; Amrhein et al. 2017, 2018; McShane et al. 2018, 2017; Wasserstein and Lazar 2016).”
BackgroundJack, C. R., Therneau, T. M., Weigand, S. D., Wiste, H. J., Knopman, D. S., Vemuri, P., Lowe, V. J., Mielke, M. M., Roberts, R. O., Machulda, M. M., et al. (2019) · JAMA Neurology · doi:10.1001/jamaneurol.2019.1971 · Cited by 300 · see citing works
The earth is flat (p > 0.05): significance thresholds and the crisis of unreplicable research
Amrhein, V., Korner-Nievergelt, F., & Roth, T. (2017) · PeerJ · doi:10.7717/peerj.3544 · Cited by 293 · see citing works
“Whoever is interested in describing the average state of a system, however, should not "choose what to present based on statistical results", because "valid interpretation of those results is severely compromised" unless all tests that were done are disclosed (ASA statement, Wasserstein & Lazar 2016).”
BackgroundMethodInfluentialCheong, E. V., Sinnott, C., Dahly, D. L., & Kearney, P. M. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-013228 · Cited by 311 · see citing works
“This is consistent with current practice in major epidemiological journals, particularly with observational study designs, and recent guidance from the American Statistical Association.”
ResultLane, M. M., Gamage, E., Travica, N., Dissanayaka, T. D., Ashtree, D. N., Gauci, S., Lotfaliany, M., O'Neil, A. E., Jacka, F. N., & Marx, W. (2022) · Nutrients · doi:10.3390/nu14132568 · Cited by 345 · see citing works
Beyond small, medium, or large: points of consideration when interpreting effect sizes
Bakker, A., Cai, J., English, L., Kaiser, G., Mesa, V., & Van Dooren, W. (2019) · Educational Studies in Mathematics · doi:10.1007/s10649-019-09908-4 · Cited by 302 · see citing works
Endometriosis and cancer: a systematic review and meta-analysis
Kvaskoff, M., Mahamat-Saleh, Y., Farland, L. V., Shigesi, N., Terry, K. L., Harris, H. R., Roman, H., Becker, C. M., As-Sanie, S., Zondervan, K. T., et al. (2021) · Human Reproduction Update · doi:10.1093/humupd/dmaa045 · Cited by 303 · see citing works
“…cancer is very rare, extremely large cohort sample sizes or multi-site case-control studies are needed to yield power to quantify statistical significance that is too often the focus (Farland et al., 2016b; Greenland et al., 2016; Rothman, 2016; Wasserstein and Lazar, 2016; Amrhein et al., 2019).”
BackgroundNambisan, S., & Baron, R. A. (2019) · Journal of Business Research · doi:10.1016/j.jbusres.2019.06.037 · Cited by 307 · see citing works
Bayesian Random-Effects Meta-Analysis Using the <b>bayesmeta</b> <i>R</i> Package
Röver, C. (2020) · Journal of Statistical Software · doi:10.18637/jss.v093.i06 · Cited by 310 · see citing works
Interpreting and reporting 40Ar/39Ar geochronologic data
Schaen, A. J., Jicha, B. R., Hodges, K. V., Vermeesch, P., Stelten, M. E., Mercer, C. M., Phillips, D., Rivera, T., Jourdan, F., Matchan, E. L., et al. (2020) · Geological Society of America Bulletin · doi:10.1130/b35560.1 · Cited by 283 · see citing works
Coad, A., Holm, J. R., Krafft, J., et al. (2017) · Journal of Evolutionary Economics · doi:10.1007/s00191-017-0532-6 · Cited by 285 · see citing works
“Alongside the disappointment with fishing for significant p-values, also known as p-hacking or asterisk-hunting (Bettis, 2012; Bettis et al., 2014; Wasserstein and Lazar, 2016) and problems with regression estimation of U-shapes (Haans et al.”
BackgroundMisrepresentation and distortion of research in biomedical literature
Boutron, I., & Ravaud, P. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1710755115 · Cited by 273 · see citing works
“In 2016, the American Statistical Association released a statement on statistical significance and the P value with six principles underlying the proper use and interpretation of this statistical tool (77).”
BackgroundWhen Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment
Szucs, D., & Ioannidis, J. P. A. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00390 · Cited by 298 · see citing works
“, the recent American Statistical Association statement on p-values (Wasserstein and Lazar, 2016) was accompanied by 21 editorials from the statisticians and methodologists who participated in crafting it and who disagreed in different aspects among themselves.”
BackgroundInfluentialHeld, L., & Ott, M. (2017) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-031017-100307 · Cited by 273 · see citing works
“P -values for point null hypotheses still dominate most of the applied literature (Greenland & Poole 2013), despite the fact that P -values are commonly misused (Wasserstein & Lazar 2016; Matthews et al. 2017).”
BackgroundCarriedo, A., Estrada, J. A. C., Fernández‐Río, J., & Mendez‐Giménez, A. (2020) · American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2020.08.007 · Cited by 282 · see citing works
Hahn, E. D., & Ang, S. H. (2017) · Journal of World Business · doi:10.1016/j.jwb.2016.12.003 · Cited by 275 · see citing works
“Lengthy discussions ensued (see Wasserstein & Lazar, 2016) but the general consensus of the committee was that “the current culture of statistical significance testing, interpretation, and reporting has to go” (Ziliak, 2016, p. 2).”
BackgroundLeisman, D. E., Harhay, M. O., Lederer, D. J., Abramson, M., Adjei, A. A., Bakker, J., Ballas, Z. K., Barreiro, E., Bell, S. C., Bellomo, R., et al. (2020) · Critical Care Medicine · doi:10.1097/ccm.0000000000004246 · Cited by 271 · see citing works
Markov Chain Monte Carlo Methods for Bayesian Data Analysis in Astronomy
Sharma, S. (2017) · Annual Review of Astronomy and Astrophysics · doi:10.1146/annurev-astro-082214-122339 · Cited by 298 · see citing works
“Recent statement by the American Statistical Association, (Wasserstein & Lazar 2016) warning on the misuse of P values is another example of the superiority of the Bayesian methods of hypothesis testing.”
BackgroundThe reproducibility of research and the misinterpretation of p -values
Colquhoun, D. (2017) · Royal Society Open Science · doi:10.1098/rsos.171085 · Cited by 258 · see citing works
Three Recommendations for Improving the Use of p -Values
Benjamin, D. J., & Berger, J. O. (2019) · The American Statistician · doi:10.1080/00031305.2018.1543135 · Cited by 258 · see citing works
“Once our recommendations are adopted, we will advocate for further steps that move us yet closer to the ideal, holistic approach to assessing statistical evidence summarized in the ASA statement (Wasserstein and Lazar 2016).”
BackgroundMethodInfluentialBeyond Subjective and Objective in Statistics
Gelman, A., & Hennig, C. M. (2017) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/rssa.12276 · Cited by 257 · see citing works
Kamo, M. (2023) · Theoretical biology · doi:10.1007/978-981-99-0309-2_8 · Cited by 253 · see citing works
Humidity’s Role in Heat-Related Health Outcomes: A Heated Debate
Baldwin, J. W., Benmarhnia, T., Ebi, K. L., Jay, O., Lutsko, N. J., & Vanos, J. (2023) · Environmental Health Perspectives · doi:10.1289/ehp11807 · Cited by 275 · see citing works
“76,77 In summary, including humidity as a confounder can omit important mechanisms linking temperature and humidity in producing negative health outcomes, especially when these variables are highly correlated.”
Wang, H.-T., Smallwood, J., Mourão-Miranda, J., Xia, C. H., Satterthwaite, T. D., Bassett, D. S., & Bzdok, D. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116745 · Cited by 238 · see citing works
“Finally, drawing inferences on model fits has frequently been based on statistical null hypothesis testing and accompanying methodology (Wasserstein and Lazar, 2016).”
MethodHarmonizing Genetic Ancestry and Self-identified Race/Ethnicity in Genome-wide Association Studies
Fang, H., Hui, Q., Lynch, J. A., Honerlaw, J. P., Assimes, T. L., Huang, J., Vujković, M., Damrauer, S. M., Pyarajan, S., Gaziano, J. M., et al. (2019) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2019.08.012 · Cited by 246 · see citing works
Meaningful associations in the adolescent brain cognitive development study
Dick, A. S., Lopez, D. A., Watts, A. L., Heeringa, S. G., Reuter, C., Bartsch, H., Fan, C. C., Kennedy, D. N., Palmer, C. E., Marshall, A. T., et al. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2021.118262 · Cited by 234 · see citing works
Bayesian data analysis in the phonetic sciences: A tutorial introduction
Vasishth, S., Nicenboim, B., Beckman, M. E., Li, F., & Kong, E. J. (2018) · Journal of Phonetics · doi:10.1016/j.wocn.2018.07.008 · Cited by 246 · see citing works
“As statisticians have repeatedly pointed out (Wasserstein and Lazar, 2016), these kinds of binary decisions (based on p-values or any other statistic such as confidence intervals) are common but highly misleading.”
BackgroundKhazaei, B., Khatami, S., Alemohammad, S. H., Rashidi, L., Wu, C., Madani, K., Kalantari, Z., Destouni, G., & Aghakouchak, A. (2019) · Journal of Hydrology · doi:10.1016/j.jhydrol.2018.12.004 · Cited by 245 · see citing works
“Negative low values near −1 normally refer to water, and small values close to zero (between −0.1 and 0.1) represent urban areas or areas covered by rock, soil, etc. (Weier and Herring, 2000).”
BackgroundInfluentialA Practical Guide for Transparency in Psychological Science
Klein, O., Hardwicke, T. E., Aust, F., Breuer, J., Danielsson, H., Mohr, A. H., IJzerman, H., Nilsonne, G., Vanpaemel, W., & Frank, M. C. (2018) · Collabra Psychology · doi:10.1525/collabra.158 · Cited by 233 · see citing works
“Across statistical frameworks, conducting multiple tests and then selectively reporting only a subset may lead to improper and ungeneralisable conclusions (Goodman et al., 2016; Wasserstein & Lazar, 2016).”
MethodGuidelines for reporting of statistics for clinical research in urology
Assel, M. J., Sjoberg, D. D., Elders, A., Wang, X., Huo, D., Botchway, A., Delfino, K. R., Fan, Y., Zhao, Z., Koyama, T., et al. (2018) · British Journal of Urology · doi:10.1111/bju.14640 · Cited by 244 · see citing works
“We follow the American Statistical Association statement on p values and encourage all researchers to read either the full statement [5] or the summary [6].”
Power, precision, and sample size estimation in sport and exercise science research
Abt, G., Boreham, C. A., Davison, G. W., Jackson, R. C., Nevill, A. M., Wallace, E. S., & Williams, M. R. F. (2020) · Journal of Sports Sciences · doi:10.1080/02640414.2020.1776002 · Cited by 260 · see citing works
“…power to AIPE for sample size estimation (Cumming & Calin-Jageman, 2017; Kelley et al., 2003), the approach still suffers from using a frequentist confidence interval, which is inherently tied to the p value and all of its problems (Cohen, 1994; McShane et al., 2019; Wasserstein & Lazar, 2016).”
BackgroundDeep learning for credit scoring: Do or don’t?
Gunnarsson, B. R., Broucke, S. V., Baesens, B., Óskarsdóttir, M., & Lemahieu, W. (2021) · European Journal of Operational Research · doi:10.1016/j.ejor.2021.03.006 · Cited by 256 · see citing works
A tutorial on testing hypotheses using the Bayes factor.
Hoijtink, H. J. A., Mulder, J., Lissa, C. J. V., & Gu, X. (2019) · Psychological Methods · doi:10.1037/met0000201 · Cited by 237 · see citing works
Practical guide to the meta-analysis of rare events
Efthimiou, O. (2018) · Evidence Based Mental Health · doi:10.1136/eb-2018-102911 · Cited by 231 · see citing works
On the use of the coefficient of variation to quantify and compare trait variation
Pélabon, C., Hilde, C. H., Einum, S., & Gamelon, M. (2020) · Evolution Letters · doi:10.1002/evl3.171 · Cited by 238 · see citing works
“This issue affects several aspects of the scientific process, from the measurement procedures to the interpretation of the statistical analyses where biological significance is often confounded with statistical significance (Yoccoz 1991; Tarka et al. 2015; Wasserstein and Lazar 2016).”
BackgroundEvaluation of USPSTF Lung Cancer Screening Guidelines Among African American Adult Smokers
Aldrich, M. C., Mercaldo, S., Sandler, K. L., Blot, W. J., Grogan, E. L., & Blume, J. D. (2019) · JAMA Oncology · doi:10.1001/jamaoncol.2019.1402 · Cited by 234 · see citing works
Statistical primer: multivariable regression considerations and pitfalls†
Grant, S. W., Hickey, G. L., & Head, S. J. (2018) · European Journal of Cardio-Thoracic Surgery · doi:10.1093/ejcts/ezy403 · Cited by 227 · see citing works
“In addition, the corresponding P-value may be reported; however, the P-value alone is useless without the effect size [21].”
BackgroundTest–retest: Agreement or reliability?
Berchtold, A. (2016) · Methodological Innovations · doi:10.1177/2059799116672875 · Cited by 235 · see citing works
“We use here the concept of p-value as a means of measuring the robustness of the correlation coefficient value, but there are many discussions about the merits of this approach and many are advocating alternative approaches, such as confidence intervals (Wasserstein and Lazar, 2016).”
MethodRusthoven, C. G., Yamamoto, M., Bernhardt, D., Smith, D. E., Gao, D., Serizawa, T., Yomo, S., Aiyama, H., Higuchi, Y., Shuto, T., et al. (2020) · JAMA Oncology · doi:10.1001/jamaoncol.2020.1271 · Cited by 225 · see citing works
Power to Detect What? Considerations for Planning and Evaluating Sample Size
Giner‐Sorolla, R., Montoya, A. K., Reifman, A. S., Carpenter, T., Lewis, N. A., Aberson, C. L., Bostyn, D. H., Conrique, B. G., Ng, B., Schoemann, A. M., et al. (2024) · Personality and Social Psychology Review · doi:10.1177/10888683241228328 · Cited by 292 · see citing works
“The approximate nature of effect size criteria should be a caution against applying overly rigid “bright lines” to power statistics, and against repeating the statistical mistake that has treated p values as rigid, live-or-die criteria of evidence (Wasserstein & Lazar, 2016).”
BackgroundUntenable nonstationarity: An assessment of the fitness for purpose of trend tests in hydrology
Serinaldi, F., Kilsby, C. G., & Lombardo, F. (2017) · Advances in Water Resources · doi:10.1016/j.advwatres.2017.10.015 · Cited by 215 · see citing works
Gaudel, A., Cooper, O. R., Chang, K., Bourgeois, I., Ziemke, J. R., Strode, S. A., Oman, L. D., Sellitto, P., Nédélec, P., Blot, R., et al. (2020) · Science Advances · doi:10.1126/sciadv.aba8272 · Cited by 221 · see citing works
- Authors
- Wasserstein, R. L., & Lazar, N. A.
- Affiliations
- American Statistical Association, 732 North Washington Street, Alexandria, VA 22314-1943
- American Statistical Association (US)
- ORCID
- Lazar, Nicole A.
- Venue
- The American Statistician
- Publisher
- Informa UK Limited
- Published
- 2016-04-02
- Volume
- 70
- Issue
- 2
- Pages
- 129-133
- ISSN
- 0003-1305 (ISSN-L)
- 1537-2731
- OpenAlex
- W2293040502
- Semantic Scholar
- 81a65f23474ffceb2481836328608102950da979
- Cited by
- 6,306
- References
- 45
- Citations by year
- 2026: 625 · 2025: 629 · 2024: 575 · 2023: 543 · 2022: 642 · 2021: 676 · 2020: 678 · 2019: 666 · 2018: 596 · 2017: 438 · 2016: 226 · 2015: 5 · 2014: 1 · 2013: 1 · 2012: 1
- FWCI
- 649.74
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- The American Statistician (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Keywords
- p-values, statistical evidence, p-value interpretation, statistical inference, hypothesis testing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read under definition version 3 (extract-v3) on 2026-09-28.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-09-28.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
