The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index; none of the 1 sources its claims cite is checked for support yet.
The ASA Statement on p -Values: Context, Process, and Purpose
Open accessAbstract
Emeritus of Mathematics and Statistics at
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
24 Own interpretation · 2 No basis given · 1 No full text of the sources
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The widespread use of”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Statistical significance is not equivalent to scientific, human, or economic significance.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, a p-value near 0.05 taken by itself offers only weak evidence against the null hypothesis.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By itself, a p-value does not provide a good measure of evidence regarding a model or hypothesis.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly, identical estimated effects will have different p-values if the precision of the estimates differs.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A p-value provides one approach to summarizing the incompatibility between a particular set of data and a proposed model for the data.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 1. P-values can indicate how incompatible the data are with a specified statistical model.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Conducting multiple analyses of the data and reporting only those with certain p-values (typically those passing a significance threshold) renders the reported p-values essentially uninterpretable.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Whenever a researcher chooses what to present based on statistical results, valid interpretation of those results is severely compromised if the reader is not informed of the choice and its basis.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The smaller the p-value, the greater the statistical incompatibility of the data with the null hypothesis, if the underlying assumptions used to calculate the p-value hold.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 1. P-values can indicate how incompatible the data are with a specified statistical model.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Valid scientific conclusions based on p-values and related statistics cannot be drawn without at least knowing how many and which analyses were conducted, and how those analyses (including p-values) were selected for reporting.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
plainconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 10
- It may state two separate claims as one. 10
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This has led to some scientific journals discouraging the use of p-values, and some scientists and statisticians recommending their abandonment, with some arguments essentially unchanged since p-values were first introduced.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 1. Introduction
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The statistical community has been deeply concerned about issues of reproducibility and replicability of scientific conclusions.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > Abstract
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“While the p-value can be a useful statistical measure, it is commonly misused and misinterpreted.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 1. Introduction
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A conclusion does not immediately become”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All these measures and approaches rely on further assumptions, but they may more directly address the size of an effect (and its associated uncertainty) or whether the hypothesis is correct.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 4. Other Approaches
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The validity of scientific conclusions, including their reproducibility, depends on more than the statistical methods themselves.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 1. Introduction
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“It is a statement about data in relation to a specified hypothetical explanation, and is not a statement about the explanation itself.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 2. P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Researchers should recognize that a p-value without context or other evidence provides limited information.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Likewise, a relatively large p-value does not imply evidence in favor of the null hypothesis; many other hypotheses may be equally or more consistent with the observed data.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This incompatibility can be interpreted as casting doubt on or providing evidence against the null hypothesis or the underlying assumptions.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 1. P-values can indicate how incompatible the data are with a specified statistical model.
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Appropriately chosen techniques, properly conducted analyses and correct interpretation of statistical results also play a key role in ensuring that conclusions are sound and that uncertainty surrounding them is represented properly.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 1. Introduction
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
In the paper
“Misunderstanding or misuse of statistical inference is only one cause of the”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > Abstract
What the sources say
- Not checked — no full text of the sourcecites Peng 2015 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Smaller p-values do not necessarily imply the presence of larger or more important effects, and larger p-values do not imply a lack of importance or even lack of effect.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 3. Scientific conclusions and business or policy decisions should not be based only on whether a p-value passes a specific threshold.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 2. P-values do not measure the probability that the studied hypothesis is true, or the probability that the data were produced by random chance alone.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In view of the prevalent misuses of and misconceptions concerning p-values, some statisticians prefer to supplement or even replace p-values with other approaches.”
§ The ASA's Statement on p-Values: Context, Process, and Purpose In > 4. Other Approaches
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper relies on this without support.
There is widespread consensus in the statistical community on the principles articulated in the ASA statement.
“according to widespread consensus in the statistical community”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While the large scale of the Tabula Muris Senis dataset provides substantial power to predict subtle changes in LR interactions, the biological complexity, dependencies among communication pairs, and multiple testing burden require that these p-values be interpreted within a broader biological and statistical context [33].”
From Quantitative profiling of lifespan-dependent cell-cell communication potential reveals dynamic ligand-receptor network shifts across mouse tissues (2026) · cites this paper as [33]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“These types of incorrect descriptions contribute to confirmation bias, overinterpretation of results, and reporting correlation as implied causation (Wasserstein and Lazar 2016).”
From Unethical Numbers? A Meta-analysis of Library Learning Analytics Studies (2019) · cites this paper as Wasserstein and Lazar 2016
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“This is especially true of relatively large-sample-size studies (as most learning analytics studies are), which are likely to obtain statistically significant findings in relatively small variations in measures such as GPA (Field 2017; Salkind 2016 Wasserstein and Lazar 2016).”
From Unethical Numbers? A Meta-analysis of Library Learning Analytics Studies (2019) · cites this paper as Wasserstein and Lazar 2016
In the citing paper
“In 2016, the American Statistical Association released a statement on statistical significance and the P value with six principles underlying the proper use and interpretation of this statistical tool (77).”
From Misrepresentation and distortion of research in biomedical literature (2018) · cites this paper as [77]
In the citing paper
“Although we also present p values for exploratory analyses, we refrain from labeling any of them as statistically significant, following best-practice recommendations for statistical analysis [43].”
From Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature (2020) · cites this paper as [43]
In the citing paper
“Contrary to IJE author recommendations and recent guidance from the American Statistics Society, 16 Gao et al. largely base their conclusions on findings with a P -value equivalent to < 0.05 after multiple testing.”
From Commentary: Two-sample Mendelian randomization: opportunities and challenges (2016) · cites this paper as [16]
In the citing paper
“Following guidance,50 we focused on 95% CIs, rather than statistical significance. For example, unlike conventional interpretations, we did not outright interpret an effect estimate “non‐significant” if the lower or upper bound of the 95% CI was slightly above/below the null value; we interpreted them as “suggestive” of an effect.”
From Third‐wave cognitive behaviour therapies for weight management: A systematic review and network meta‐analysis (2020) · cites this paper as [50]
Citations
Cites
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
(1947) · The American Statistician · doi:10.1080/00031305.1947.10501534 · Cited by 3,745
The Abuse of Power: The Pervasive Fallacy of Power Calculations for Data Analysis
Hoenig, J. M., & Heisey, D. M. (2001) · The American Statistician · doi:10.1198/000313001300339897 · Cited by 2,046
Statistics notes: Absence of evidence is not evidence of absence
Altman, D. G., & Bland, J. M. (1995) · BMJ · doi:10.1136/bmj.311.7003.485 · Cited by 1,946
Smithson, M. J. (2000) · SAGE Publications Ltd eBooks · doi:10.4135/9781446218525 · Cited by 1,871
Bayesian statistical inference for psychological research.
Edwards, W., Lindman, H., & Savage, L. J. (1963) · Psychological Review · doi:10.1037/h0044139 · Cited by 1,734
Scientific method: Statistical errors
Nuzzo, R. (2014) · Nature · doi:10.1038/506150a · Cited by 1,536
Why Most Discovered True Associations Are Inflated
Ioannidis, J. P. A. (2008) · Epidemiology · doi:10.1097/ede.0b013e31818131e7 · Cited by 1,495
Sterne, J. A. C. (2001) · BMJ · doi:10.1136/bmj.322.7280.226 · Cited by 1,466
Contradicted and Initially Stronger Effects in Highly Cited Clinical Research
Ioannidis, J. P. A. (2005) · JAMA · doi:10.1001/jama.294.2.218 · Cited by 1,465
Research works printed but not linked: 5
- Leek, J. (2014), “On the Scalability of Statistical Procedures: Why the p-Value Bashers Just Don't Get It,” Simply Statistics Blog, Available at http://simplystatistics.org/2014/02/14/on-the-scalability-of-statistical-procedures-why-the-p-value-bashers-just-dont-get-it/.
- Gelman A. and Loken E. (2014) “The Statistical Crisis in Science [online] ” American Scientist 102. Available at http://www.americanscientist.org/issues/feature/2014/6/the-statistical-crisis-in-science · DOI · Paper page
- Greenland, S., and Poole, C. (2011), “Problems in Common Interpretations of Statistics in Scientific Articles, Expert Reports, and Testimony,” Jurimetrics, 51, 113–129.
- 10.1214/13-AOS1123 · DOI
- Lavine, M. (1999), “What is Bayesian Statistics and Why Everything Else is Wrong,” UMAP Journal, 20, 2.
Cited by
Sorted by citations
Sensitivity analysis in observational research: Introducing the E-value
VanderWeele, T. J., & Ding, P. (2017) · Annals of Internal Medicine · doi:10.7326/m16-2607 · Cited by 6,350
“While potentially subject to mis-use and misinterpretation (1,5-12), the p-value can be informative as a continuous measure of evidence as to whether an association is present.”
BackgroundMoving to a World Beyond “ p < 0.05”
Wasserstein, R. L., Schirm, A. L., & Lazar, N. A. (2019) · The American Statistician · doi:10.1080/00031305.2019.1583913 · Cited by 2,976
“This concern is also addressed in the literature on equivalence testing (Wellek 2017).”
Self-citationRedefine statistical significance
Benjamin, D. J., Berger, J. O., Johannesson, M., Nosek, B. A., Wagenmakers, E.-J., Berk, R., Bollen, K. A., Brembs, B., Brown, L., Camerer, C., et al. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0189-z · Cited by 2,772
The preregistration revolution
Nosek, B. A., Ebersole, C. R., DeHaven, A. C., & Mellor, D. T. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1708274114 · Cited by 1,998
“05 are not actually very unlikely in typical research practices (31), leading some researchers to recommend 0.”
BackgroundContemporary Guidance for Stated Preference Studies
Johnston, R. J., Boyle, K., Adamowicz, W., Bennett, J., Brouwer, R., Cameron, T. A., Hanemann, W. M., Hanley, N. D., Ryan, M., Scarpa, R., et al. (2017) · Journal of the Association of Environmental and Resource Economists · doi:10.1086/691697 · Cited by 1,566
“One comment is particularly relevant to econometric analyses of SP data: “P-values can indicate how incompatible the data are with a specified statistical model” (Wasserstein and Lazar 2016, 131).”
BackgroundKruschke, J. K., & Liddell, T. M. (2018) · Psychonomic Bulletin & Review · doi:10.3758/s13423-016-1221-4 · Cited by 1,288
“Among the most recent and prominent cautions against NHST is a statement from the American Statistical Association (ASA; Wasserstein & Lazar, 2016), which featured six principles for properly interpreting p values.”
Do “brain-training” programs work?
Simons, D. J., Boot, W. R., Charness, N., Gathercole, S. E., Chabris, C. F., Hambrick, D. Z., & Stine-Morrow, E. A. L. (2016) · Psychological Science in the Public Interest · doi:10.1177/1529100616661983 · Cited by 1,183
Indices of Effect Existence and Significance in the Bayesian Framework
Makowski, D., Ben-Shachar, M. S., Chen, S. H. A., & Lüdecke, D. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.02767 · Cited by 983
“The p-value represents the probability that for a given statistical model, when the null hypothesis is true, the effect would be greater than or equal to the observed coefficient (Wasserstein and Lazar, 2016).”
BackgroundThe natural selection of bad science
Smaldino, P. E., & McElreath, R. (2016) · Royal Society Open Science · doi:10.1098/rsos.160384 · Cited by 860
“In March 2016, the American Statistical Association published a set of corrective guidelines about the use and misuse of p-values [2].”
BackgroundForecasting: theory and practice
Petropoulos, F., Apiletti, D., Assimakopoulos, V., Babaï, M. Z., Barrow, D. K., Taieb, S. B., Bergmeir, C., Bessa, R. J., Bijak, J., Boylan, J. E., et al. (2022) · International Journal of Forecasting · doi:10.1016/j.ijforecast.2021.11.001 · Cited by 1,187
van Laar, E., van Deursen, A. J. A. M., van Dijk, J. A. G. M., & de Haan, J. (2020) · Sage Open · doi:10.1177/2158244019900176 · Cited by 927
“Critics call for a broader approach because p -values are commonly misused and misinterpreted (Wasserstein & Lazar, 2016).”
BackgroundThe role of multiple global change factors in driving soil functions and microbial biodiversity
Rillig, M. C., Ryo, M., Lehmann, A., Aguilar‐Trigueros, C. A., Buchert, S., Wulf, A., Iwasaki, A., Roy, J., & Yang, G. (2019) · Science · doi:10.1126/science.aay2832 · Cited by 932
Saltelli, A., Aleksankina, K., Becker, W., Fennell, P., Ferretti, F., Holst, N., Li, S., & Wu, Q.-L. (2017) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2019.01.012 · Cited by 740
Commentary: Two-sample Mendelian randomization: opportunities and challenges
Lawlor, D. A. (2016) · International Journal of Epidemiology · doi:10.1093/ije/dyw127 · Cited by 810
Machine Learning for Precision Psychiatry: Opportunities and Challenges
Bzdok, D., & Meyer‐Lindenberg, A. (2017) · Biological Psychiatry Cognitive Neuroscience and Neuroimaging · doi:10.1016/j.bpsc.2017.11.007 · Cited by 888
“Classical significance testing is probably most often used to decide between two possible outcomes, expressed in the null and alternative hypothesis, by considering the probability of obtaining an equal or more extreme effect in the data under the null hypothesis (24).”
BackgroundMethodResultAbandon Statistical Significance
McShane, B. B., Gal, D., Gelman, A., Robert, C. P., & Tackett, J. L. (2019) · The American Statistician · doi:10.1080/00031305.2018.1527253 · Cited by 921
“s, the external evidence for the phenomenon under study, and the validity ofassumptions that underlie the data analysis” and cautions that “no single index should substitute for scientific reasoning;” Wasserstein and Lazar (2016)). Second, as discussed above, while we believe treating the p-value continuously rather th…”
BackgroundInfluentialRejecting or Accepting Parameter Values in Bayesian Estimation
Kruschke, J. K. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918771304 · Cited by 734
“Discrete decisions should be avoided if possible, because such decisions encourage people to ignore the magnitude of the parameter value and its uncertainty (e.g., Cumming, 2014; Kruschke & Liddell, 2018b; Wasserstein & Lazar, 2016, and many references cited therein).”
BackgroundLederer, D. J., Bell, S. C., Branson, R. D., Chalmers, J. D., Marshall, R., Maslove, D. M., Ost, D. E., Punjabi, N. M., Schatz, M., Smyth, A. R., et al. (2019) · Annals of the American Thoracic Society · doi:10.1513/annalsats.201808-564ps · Cited by 715
Best practices in data analysis and sharing in neuroimaging using MRI
Nichols, T. E., Das, S., Eickhoff, S. B., Evans, A. C., Glatard, T., Hanke, M., Kriegeskorte, N., Milham, M. P., Poldrack, R. A., Poline, J.-B., et al. (2017) · Nature Neuroscience · doi:10.1038/nn.4500 · Cited by 714
Best practice guidance for linear mixed-effects models in psychological science
Meteyard, L., & Davies, R. A. I. (2020) · Journal of Memory and Language · doi:10.1016/j.jml.2020.104092 · Cited by 817
A Statistical Significance Test for Necessary Condition Analysis
Dul, J., van der Laan, E., & Kuik, R. (2018) · Organizational Research Methods · doi:10.1177/1094428118795272 · Cited by 726
“Just like any other statistical method that uses the p value for statistical inference, the proposed approximate permutation test has all the limitations of the p value (Wasserstein & Lazar, 2016).”
MethodGaudel, A., Cooper, O. R., Ancellet, G. M., Barret, B., Boynard, A., Burrows, J. P., Clerbaux, C., Coheur, P.-F., Cuesta, J., Cuevas, E., et al. (2018) · Elementa Science of the Anthropocene · doi:10.1525/elementa.291 · Cited by 668
A solution to minimum sample size for regressions
Jenkins, D. G., & Quintana-Ascencio, P. F. (2020) · PLOS ONE · doi:10.1371/journal.pone.0229345 · Cited by 665
“Power analyses carry forward fundamental problems with null hypothesis inference, which is the long-standing basis for statistical analyses but that has been recently and widely criticized for several reasons [25,26,27,28,29].”
BackgroundThe Proposal to Lower P Value Thresholds to .005
Ioannidis, J. P. A. (2018) · JAMA · doi:10.1001/jama.2018.1536 · Cited by 660
Rigdon, E. E. (2016) · European Management Journal · doi:10.1016/j.emj.2016.05.006 · Cited by 653
Rewriting results sections in the language of evidence
Muff, S., Nilsen, E. B., O’Hara, R. B., & Nater, C. R. (2021) · Trends in Ecology & Evolution · doi:10.1016/j.tree.2021.10.009 · Cited by 675
Detecting RNA modifications in the epitranscriptome: predict and validate
Helm, M., & Motorin, Y. (2017) · Nature Reviews Genetics · doi:10.1038/nrg.2016.169 · Cited by 647
The Future of Sensitivity Analysis: An essential discipline for systems modeling and policy support
Razavi, S., Jakeman, A. J., Saltelli, A., Prieur, C., Iooss, B., Borgonovo, E., Plischke, E., Piano, S. L., Iwanaga, T., Becker, W., et al. (2020) · Environmental Modelling & Software · doi:10.1016/j.envsoft.2020.104954 · Cited by 703
“…’ (Saltelli and Funtowicz, 2017; Saltelli, 2018) where misuse or abuse of statistics (Stark and Saltelli, 2018; Gigerenzer and Marewski, 2015; Leek et al., 2017; Singh Chawla, 2017; Wasserstein and Lazar, 2016; Gigerenzer, 2018) is often cited as the root cause of the crisis (Ioannidis, 2005).”
Yu, Z., Guindani, M., Grieco, S. F., Chen, L., Holmes, T. C., & Xu, X. (2022) · Neuron · doi:10.1016/j.neuron.2021.10.030 · Cited by 563
“…research articles 696 (Aickin and Gensler, 1996; Altman and Bland, 1995; Benjamin and Berger, 2019; Benjamini and 697 Hochberg, 1995; Gelman and Stern, 2006; Goodman, 2008; Holm, 1979; McHugh, 2011; Storey, 698 2002; Wasserstein and Lazar, 2016) or consult with experienced statisticians.”
Douglas, E. J., Shepherd, D. A., & Prentice, C. (2019) · Journal of Business Venturing · doi:10.1016/j.jbusvent.2019.105970 · Cited by 570
Instagram photos reveal predictive markers of depression
Reece, A. G., & Danforth, C. M. (2017) · EPJ Data Science · doi:10.1140/epjds/s13688-017-0110-z · Cited by 519
Carrasco, J., García, S., Rueda, M. D. M., Das, S., & Herrera, F. (2020) · Swarm and Evolutionary Computation · doi:10.1016/j.swevo.2020.100665 · Cited by 533
Presidential Address: The Scientific Outlook in Financial Economics
Harvey, C. R. (2017) · The Journal of Finance · doi:10.1111/jofi.12530 · Cited by 466
Liengaard, B. D., Sharma, P. N., Hult, G. T. M., Jensen, M. B., Sarstedt, M., Hair, J. F., & Ringle, C. M. (2020) · Decision Sciences · doi:10.1111/deci.12445 · Cited by 634
Contract cheating: a survey of australian university students
Bretag, T., Harper, R., Burton, M., Ellis, C., Newton, P., Rozenberg, P., Saddiqui, S., & van Haeringen, K. (2019) · Studies in Higher Education · doi:10.1080/03075079.2018.1462788 · Cited by 476
“Wasserstein and Lazar (2016) state that often the null hypothesis postulates the absence of an effect, such as no difference between two groups, or the absence of a relationship between a factor and an outcome.”
BackgroundCross validation for model selection: A review with examples from ecology
Yates, L. A., Aandahl, Z., Richards, S. A., & Brook, B. W. (2022) · Ecological Monographs · doi:10.1002/ecm.1557 · Cited by 565
Pre-registration in social psychology—A discussion and suggested template
Veer, A. V. '., & Giner‐Sorolla, R. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2016.03.004 · Cited by 464
Chromosome errors in human eggs shape natural fertility over reproductive life span
Gruhn, J. R., Zielinska, A. P., Shukla, V., Blanshard, R., Capalbo, A., Cimadomo, D., Nikiforov, D., Chan, A. C.-H., Newnham, L. J., Vogel, I., et al. (2019) · Science · doi:10.1126/science.aav7321 · Cited by 466
Treatment selection in depression.
Cohen, Z. D., & DeRubeis, R. J. (2018) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-050817-084746 · Cited by 435
“Reliance on tests of significance can result in misleading impressions about the importance of predictive variables (Nuzzo 2014, Wasserstein & Lazar 2016).”
BackgroundLeo, G. D., & Sardanelli, F. (2020) · European Radiology Experimental · doi:10.1186/s41747-020-0145-y · Cited by 555
“A special issue in The American Statistician [5] presented 37 papers on “Statistical inference in the 21st century: a world beyond p < 0.”
BackgroundInfluentialLichtenberg, E. M., Kennedy, C. M., Kremen, C., Batáry, P., Berendse, F., Bommarco, R., Bosque‐Pérez, N. A., Carvalheiro, L. G., Snyder, W. E., Williams, N. M., et al. (2017) · Global Change Biology · doi:10.1111/gcb.13714 · Cited by 483
“This accords with a recent policy statement by the American Statistical Association (Wasserstein & Lazar, 2016), which notes that reliance on arbitrary alpha values can lead to erroneous conclusions.”
ResultBayesian Analysis Reporting Guidelines
Kruschke, J. K. (2021) · Nature Human Behaviour · doi:10.1038/s41562-021-01177-7 · Cited by 525
Abandon Statistical SignificancePreprint
McShane, B. B., Gal, D., Gelman, A., Robert, C. P., & Tackett, J. L. (2019) · HAL · Cited by 449
OECD (2009) · Programme for international student assessment/Internationale Schulleistungsstudie · doi:10.1787/19963777 · Cited by 447
Comparing multiple comparisons: practical guidance for choosing the best multiple comparisons test
Midway, S. R., Robertson, M. D., Flinn, S., & Kaller, M. D. (2020) · PeerJ · doi:10.7717/peerj.10387 · Cited by 452
“…and concerns over the use of these criteria in this manner are well documented in the literature (Stephens et al., 2005; Midway et al. (2020), PeerJ, DOI 10.7717/peerj.10387 16/26 Stephens, Buskirk & Del Rio, 2007;Murtaugh, 2014;Wasserstein & Lazar, 2016;Wasserstein, Schrim & Lazar, 2019).”
BackgroundInfluentialBayesian data analysis for newcomers
Kruschke, J. K., & Liddell, T. M. (2017) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1272-1 · Cited by 439
Misinformation in and about science
West, J. D., & Bergstrom, C. T. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1912444117 · Cited by 391
Statistical Inference as Severe Testing: How to Get Beyond the Statistics Wars
Mayo, D. G. (2018) · doi:10.1017/9781107286184 · Cited by 376
Validation practices for satellite soil moisture retrievals: What are (the) errors?
Gruber, A., Lannoy, G. J. M. D., Albergel, C., Al‐Yaari, A., Brocca, L., Calvet, J., Colliander, A., Cosh, M. H., Crow, W. T., Dorigo, W. A., et al. (2020) · Remote Sensing of Environment · doi:10.1016/j.rse.2020.111806 · Cited by 372
Halsey, L. G. (2019) · Biology Letters · doi:10.1098/rsbl.2019.0174 · Cited by 382
Kers, J. G., & Saccenti, E. (2022) · Frontiers in Microbiology · doi:10.3389/fmicb.2021.796025 · Cited by 359
“…(p-hacking) (Simmons et al., 2011). p-Hacking (also called data dredging, significance chasing, significance questing, or selective inference (Wasserstein and Lazar, 2016)) is the improper use of data (like adding or removing observations) or statistical procedures (like applying many…”
MethodNew Guidelines for Statistical Reporting in the Journal
Harrington, D. P., D’Agostino, R. B., Gatsonis, C., Hogan, J. W., Hunter, D. J., Normand, S. T., Drazen, J. M., & Hamel, M. B. (2019) · New England Journal of Medicine · doi:10.1056/nejme1906559 · Cited by 337
Kline, R. B. (2018) · Canadian Studies in Population · doi:10.25336/csp29418 · Cited by 336
Quality of life under the COVID-19 quarantine
Ferreira, L. N., Pereira, L. N., da Fé Brás, M., & Ilchuk, K. (2021) · Quality of Life Research · doi:10.1007/s11136-020-02724-x · Cited by 318
Guidelines for Reporting of Statistics for Clinical Research in Urology
Assel, M., Sjoberg, D., Elders, A., Wang, X., Huo, D., Botchway, A., Delfino, K., Fan, Y., Zhao, Z., Koyama, T., et al. (2019) · European Urology · doi:10.1016/j.eururo.2018.12.014 · Cited by 335
Reporting effect sizes in original psychological research: A discussion and tutorial.
Pek, J. C. X., & Flora, D. B. (2017) · Psychological Methods · doi:10.1037/met0000126 · Cited by 356
“For instance, an overemphasis on statistical significance and p values is addressed by the recent American Statistical Association statement which advocates that “[n]o single index should substitute for scientific reasoning” (Wasserstein & Lazar, 2016, p. 12).”
BackgroundWhat’s in a p? Reassessing best practices for conducting and reporting hypothesis-testing research
Meyer, K. E., van Witteloostuijn, A. C., & Beugelsdijk, S. (2017) · Journal of International Business Studies · doi:10.1057/s41267-017-0078-8 · Cited by 323
Coens, C., Pe, M. L., Dueck, A. C., Sloan, J. A., Basch, E., Calvert, M., Campbell, A., Cleeland, C. S., Cocks, K., Collette, L., et al. (2020) · The Lancet Oncology · doi:10.1016/s1470-2045(19)30790-9 · Cited by 337
Greenland, S. (2019) · The American Statistician · doi:10.1080/00031305.2018.1529625 · Cited by 313
“…supplies against the hypothesis; and most of all 4. avoiding dichotomization of p or comparison of p to α-levels (see, e.g., Poole 1987a; Hoekstra et al. 2006; Hurlbert and Lombardi 2009; Greenland et al. 2016, 2017; Amrhein et al. 2017, 2018; McShane et al. 2018, 2017; Wasserstein and Lazar 2016).”
BackgroundJack, C. R., Therneau, T. M., Weigand, S. D., Wiste, H. J., Knopman, D. S., Vemuri, P., Lowe, V. J., Mielke, M. M., Roberts, R. O., Machulda, M. M., et al. (2019) · JAMA Neurology · doi:10.1001/jamaneurol.2019.1971 · Cited by 300
The earth is flat ( p > 0.05): significance thresholds and the crisis of unreplicable research
Amrhein, V., Korner‐Nievergelt, F., & Roth, T. (2017) · PeerJ · doi:10.7717/peerj.3544 · Cited by 331
“Whoever is interested in describing the average state of a system, however, should not "choose what to present based on statistical results", because "valid interpretation of those results is severely compromised" unless all tests that were done are disclosed (ASA statement, Wasserstein & Lazar 2016).”
BackgroundMethodInfluentialBayesian Random-Effects Meta-Analysis Using the bayesmeta R Package
Röver, C. (2020) · Journal of Statistical Software · doi:10.18637/jss.v093.i06 · Cited by 310
Cheong, E. V., Sinnott, C., Dahly, D. L., & Kearney, P. M. (2017) · BMJ Open · doi:10.1136/bmjopen-2016-013228 · Cited by 311
“This is consistent with current practice in major epidemiological journals, particularly with observational study designs, and recent guidance from the American Statistical Association.”
ResultLane, M. M., Gamage, E., Travica, N., Dissanayaka, T., Ashtree, D. N., Gauci, S., Lotfaliany, M., O’Neil, A., Jacka, F. N., & Marx, W. (2022) · Nutrients · doi:10.3390/nu14132568 · Cited by 320
Beyond small, medium, or large: points of consideration when interpreting effect sizes
Bakker, A., Cai, J., English, L., Kaiser, G., Mesa, V., & Van Dooren, W. (2019) · Educational Studies in Mathematics · doi:10.1007/s10649-019-09908-4 · Cited by 302
Endometriosis and cancer: a systematic review and meta-analysis
Kvaskoff, M., Mahamat-Saleh, Y., Farland, L. V., Shigesi, N., Terry, K. L., Harris, H. R., Roman, H., Becker, C. M., As-Sanie, S., Zondervan, K. T., et al. (2021) · Human Reproduction Update · doi:10.1093/humupd/dmaa045 · Cited by 303
“…cancer is very rare, extremely large cohort sample sizes or multi-site case-control studies are needed to yield power to quantify statistical significance that is too often the focus (Farland et al., 2016b; Greenland et al., 2016; Rothman, 2016; Wasserstein and Lazar, 2016; Amrhein et al., 2019).”
BackgroundNambisan, S., & Baron, R. A. (2019) · Journal of Business Research · doi:10.1016/j.jbusres.2019.06.037 · Cited by 328
Interpreting and reporting 40Ar/39Ar geochronologic data
Schaen, A. J., Jicha, B. R., Hodges, K. V., Vermeesch, P., Stelten, M. E., Mercer, C. M., Phillips, D., Rivera, T., Jourdan, F., Matchan, E. L., et al. (2020) · Geological Society of America Bulletin · doi:10.1130/b35560.1 · Cited by 283
Coad, A., Holm, J. R., Krafft, J., et al. (2017) · Journal of Evolutionary Economics · doi:10.1007/s00191-017-0532-6 · Cited by 285
“Alongside the disappointment with fishing for significant p-values, also known as p-hacking or asterisk-hunting (Bettis, 2012; Bettis et al., 2014; Wasserstein and Lazar, 2016) and problems with regression estimation of U-shapes (Haans et al.”
BackgroundMisrepresentation and distortion of research in biomedical literature
Boutron, I., & Ravaud, P. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1710755115 · Cited by 273
“In 2016, the American Statistical Association released a statement on statistical significance and the P value with six principles underlying the proper use and interpretation of this statistical tool (77).”
BackgroundWhen Null Hypothesis Significance Testing Is Unsuitable for Research: A Reassessment
Szucs, D., & Ioannidis, J. P. A. (2017) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2017.00390 · Cited by 298
“, the recent American Statistical Association statement on p-values (Wasserstein and Lazar, 2016) was accompanied by 21 editorials from the statisticians and methodologists who participated in crafting it and who disagreed in different aspects among themselves.”
BackgroundInfluentialHeld, L., & Ott, M. (2017) · Annual Review of Statistics and Its Application · doi:10.1146/annurev-statistics-031017-100307 · Cited by 283
“P -values for point null hypotheses still dominate most of the applied literature (Greenland & Poole 2013), despite the fact that P -values are commonly misused (Wasserstein & Lazar 2016; Matthews et al. 2017).”
BackgroundCarriedo, A., Estrada, J. A. C., Fernández‐Río, J., & Mendez‐Giménez, A. (2020) · American Journal of Geriatric Psychiatry · doi:10.1016/j.jagp.2020.08.007 · Cited by 302
Hahn, E. D., & Ang, S. H. (2017) · Journal of World Business · doi:10.1016/j.jwb.2016.12.003 · Cited by 275
“Lengthy discussions ensued (see Wasserstein & Lazar, 2016) but the general consensus of the committee was that “the current culture of statistical significance testing, interpretation, and reporting has to go” (Ziliak, 2016, p. 2).”
BackgroundLeisman, D. E., Harhay, M. O., Lederer, D. J., Abramson, M., Adjei, A. A., Bakker, J., Ballas, Z. K., Barreiro, E., Bell, S. C., Bellomo, R., et al. (2020) · Critical Care Medicine · doi:10.1097/ccm.0000000000004246 · Cited by 271
Markov Chain Monte Carlo Methods for Bayesian Data Analysis in Astronomy
Sharma, S. (2017) · Annual Review of Astronomy and Astrophysics · doi:10.1146/annurev-astro-082214-122339 · Cited by 298
“Recent statement by the American Statistical Association, (Wasserstein & Lazar 2016) warning on the misuse of P values is another example of the superiority of the Bayesian methods of hypothesis testing.”
BackgroundThe reproducibility of research and the misinterpretation of p -values
Colquhoun, D. (2017) · Royal Society Open Science · doi:10.1098/rsos.171085 · Cited by 277
Three Recommendations for Improving the Use of p -Values
Benjamin, D. J., & Berger, J. O. (2019) · The American Statistician · doi:10.1080/00031305.2018.1543135 · Cited by 273
“Once our recommendations are adopted, we will advocate for further steps that move us yet closer to the ideal, holistic approach to assessing statistical evidence summarized in the ASA statement (Wasserstein and Lazar 2016).”
BackgroundMethodInfluentialBeyond Subjective and Objective in Statistics
Gelman, A., & Hennig, C. M. (2017) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.1111/rssa.12276 · Cited by 275
Kamo, M. (2023) · Theoretical biology · doi:10.1007/978-981-99-0309-2_8 · Cited by 253
Humidity’s Role in Heat-Related Health Outcomes: A Heated Debate
Baldwin, J. W., Benmarhnia, T., Ebi, K. L., Jay, O., Lutsko, N. J., & Vanos, J. (2023) · Environmental Health Perspectives · doi:10.1289/ehp11807 · Cited by 275
“76,77 In summary, including humidity as a confounder can omit important mechanisms linking temperature and humidity in producing negative health outcomes, especially when these variables are highly correlated.”
Wang, H.-T., Smallwood, J., Mourão-Miranda, J., Xia, C. H., Satterthwaite, T. D., Bassett, D. S., & Bzdok, D. (2020) · NeuroImage · doi:10.1016/j.neuroimage.2020.116745 · Cited by 252
“Finally, drawing inferences on model fits has frequently been based on statistical null hypothesis testing and accompanying methodology (Wasserstein and Lazar, 2016).”
MethodHarmonizing Genetic Ancestry and Self-identified Race/Ethnicity in Genome-wide Association Studies
Fang, H., Hui, Q., Lynch, J. A., Honerlaw, J. P., Assimes, T. L., Huang, J., Vujković, M., Damrauer, S. M., Pyarajan, S., Gaziano, J. M., et al. (2019) · The American Journal of Human Genetics · doi:10.1016/j.ajhg.2019.08.012 · Cited by 226
Meaningful associations in the adolescent brain cognitive development study
Dick, A. S., Lopez, D. A., Watts, A. L., Heeringa, S. G., Reuter, C., Bartsch, H., Fan, C. C., Kennedy, D. N., Palmer, C. E., Marshall, A. T., et al. (2021) · NeuroImage · doi:10.1016/j.neuroimage.2021.118262 · Cited by 240
Bayesian data analysis in the phonetic sciences: A tutorial introduction
Vasishth, S., Nicenboim, B., Beckman, M. E., Li, F., & Kong, E. J. (2018) · Journal of Phonetics · doi:10.1016/j.wocn.2018.07.008 · Cited by 246
“As statisticians have repeatedly pointed out (Wasserstein and Lazar, 2016), these kinds of binary decisions (based on p-values or any other statistic such as confidence intervals) are common but highly misleading.”
BackgroundKhazaei, B., Khatami, S., Alemohammad, S. H., Rashidi, L., Wu, C., Madani, K., Kalantari, Z., Destouni, G., & Aghakouchak, A. (2019) · Journal of Hydrology · doi:10.1016/j.jhydrol.2018.12.004 · Cited by 245
“Negative low values near −1 normally refer to water, and small values close to zero (between −0.1 and 0.1) represent urban areas or areas covered by rock, soil, etc. (Weier and Herring, 2000).”
BackgroundInfluentialA Practical Guide for Transparency in Psychological Science
Klein, O., Hardwicke, T. E., Aust, F., Breuer, J., Danielsson, H., Mohr, A. H., IJzerman, H., Nilsonne, G., Vanpaemel, W., & Frank, M. C. (2018) · Collabra Psychology · doi:10.1525/collabra.158 · Cited by 251
“Across statistical frameworks, conducting multiple tests and then selectively reporting only a subset may lead to improper and ungeneralisable conclusions (Goodman et al., 2016; Wasserstein & Lazar, 2016).”
MethodGuidelines for reporting of statistics for clinical research in urology
Assel, M. J., Sjoberg, D. D., Elders, A., Wang, X., Huo, D., Botchway, A., Delfino, K. R., Fan, Y., Zhao, Z., Koyama, T., et al. (2018) · British Journal of Urology · doi:10.1111/bju.14640 · Cited by 244
“We follow the American Statistical Association statement on p values and encourage all researchers to read either the full statement [5] or the summary [6].”
Power, precision, and sample size estimation in sport and exercise science research
Abt, G., Boreham, C. A., Davison, G. W., Jackson, R. C., Nevill, A. M., Wallace, E. S., & Williams, M. R. F. (2020) · Journal of Sports Sciences · doi:10.1080/02640414.2020.1776002 · Cited by 277
“…power to AIPE for sample size estimation (Cumming & Calin-Jageman, 2017; Kelley et al., 2003), the approach still suffers from using a frequentist confidence interval, which is inherently tied to the p value and all of its problems (Cohen, 1994; McShane et al., 2019; Wasserstein & Lazar, 2016).”
BackgroundDeep learning for credit scoring: Do or don’t?
Gunnarsson, B. R., Broucke, S. V., Baesens, B., Óskarsdóttir, M., & Lemahieu, W. (2021) · European Journal of Operational Research · doi:10.1016/j.ejor.2021.03.006 · Cited by 272
A tutorial on testing hypotheses using the Bayes factor.
Hoijtink, H., Mulder, J., Lissa, C. J. V., & Gu, X. (2019) · Psychological Methods · doi:10.1037/met0000201 · Cited by 256
Practical guide to the meta-analysis of rare events
Efthimiou, O. (2018) · Evidence Based Mental Health · doi:10.1136/eb-2018-102911 · Cited by 231
On the use of the coefficient of variation to quantify and compare trait variation
Pélabon, C., Hilde, C. H., Einum, S., & Gamelon, M. (2020) · Evolution Letters · doi:10.1002/evl3.171 · Cited by 238
“This issue affects several aspects of the scientific process, from the measurement procedures to the interpretation of the statistical analyses where biological significance is often confounded with statistical significance (Yoccoz 1991; Tarka et al. 2015; Wasserstein and Lazar 2016).”
BackgroundEvaluation of USPSTF Lung Cancer Screening Guidelines Among African American Adult Smokers
Aldrich, M. C., Mercaldo, S., Sandler, K. L., Blot, W. J., Grogan, E. L., & Blume, J. D. (2019) · JAMA Oncology · doi:10.1001/jamaoncol.2019.1402 · Cited by 234
Statistical primer: multivariable regression considerations and pitfalls†
Grant, S. W., Hickey, G. L., & Head, S. J. (2018) · European Journal of Cardio-Thoracic Surgery · doi:10.1093/ejcts/ezy403 · Cited by 227
“In addition, the corresponding P-value may be reported; however, the P-value alone is useless without the effect size [21].”
BackgroundTest–retest: Agreement or reliability?
Berchtold, A. (2016) · Methodological Innovations · doi:10.1177/2059799116672875 · Cited by 235
“We use here the concept of p-value as a means of measuring the robustness of the correlation coefficient value, but there are many discussions about the merits of this approach and many are advocating alternative approaches, such as confidence intervals (Wasserstein and Lazar, 2016).”
MethodRusthoven, C. G., Yamamoto, M., Bernhardt, D., Smith, D. E., Gao, D., Serizawa, T., Yomo, S., Aiyama, H., Higuchi, Y., Shuto, T., et al. (2020) · JAMA Oncology · doi:10.1001/jamaoncol.2020.1271 · Cited by 226
Power to Detect What? Considerations for Planning and Evaluating Sample Size
Giner‐Sorolla, R., Montoya, A. K., Reifman, A. S., Carpenter, T., Lewis, N. A., Aberson, C. L., Bostyn, D. H., Conrique, B. G., Ng, B., Schoemann, A. M., et al. (2024) · Personality and Social Psychology Review · doi:10.1177/10888683241228328 · Cited by 324
“The approximate nature of effect size criteria should be a caution against applying overly rigid “bright lines” to power statistics, and against repeating the statistical mistake that has treated p values as rigid, live-or-die criteria of evidence (Wasserstein & Lazar, 2016).”
BackgroundUntenable nonstationarity: An assessment of the fitness for purpose of trend tests in hydrology
Serinaldi, F., Kilsby, C. G., & Lombardo, F. (2017) · Advances in Water Resources · doi:10.1016/j.advwatres.2017.10.015 · Cited by 222
Gaudel, A., Cooper, O. R., Chang, K., Bourgeois, I., Ziemke, J. R., Strode, S. A., Oman, L. D., Sellitto, P., Nédélec, P., Blot, R., et al. (2020) · Science Advances · doi:10.1126/sciadv.aba8272 · Cited by 221
Metadata
- Authors
- Wasserstein, R. L., & Lazar, N. A.
- Affiliations
- American Statistical Association, 732 North Washington Street, Alexandria, VA 22314-1943
- American Statistical Association (US)
- ORCID
- Lazar, Nicole A.
- Venue
- The American Statistician
- Publisher
- Informa UK Limited
- Published
- 2016-04-02
- Volume
- 70
- Issue
- 2
- Pages
- 129-133
- ISSN
- 0003-1305 (ISSN-L)
- 1537-2731
- OpenAlex
- W2293040502
- Semantic Scholar
- 81a65f23474ffceb2481836328608102950da979
- Cited by
- 6,306
- References
- 45
- Citations by year
- 2026: 625 · 2025: 629 · 2024: 575 · 2023: 543 · 2022: 642 · 2021: 676 · 2020: 678 · 2019: 666 · 2018: 596 · 2017: 438 · 2016: 226
- FWCI
- 649.74
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- The American Statistician (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Physical Sciences › Mathematics › Statistics and Probability › Statistics Education and Methodologies
- Physical Sciences › Mathematics › Statistics and Probability › Probability and Statistical Research
- Keywords
- p-values, statistical evidence, p-value interpretation, statistical inference, hypothesis testing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-09-28, under version 3 of the claim definition (prompt extract-v3).
A second model labelled each claim on 2026-09-28 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
