Malignant side effects of null-hypothesis significance testing
The DOI matches its Crossref record; 88 of the 89 references it cites were found in a registry or an index.
Abstract
Six decades-worth of published information has shown irrefutably that null-hypothesis significance tests (NHSTs) provide no information about the reliability of research outcomes. Nevertheless, they are still the core of editorial decision-making in Psychology. Two reasons appear to contribute to the continuing practice. One, survey information suggests that a majority of psychological researchers incorrectly believe that p values provide information about reliability of results. Two, a position sometimes taken is that using them to make decisions has been essentially benign. The mistaken belief has been pointed out many times, so it is briefly covered because of the apparent persistence of the misunderstanding. The idea that NHSTs have been benign is challenged by seven “side-effects” that continue to retard effective development of psychological science. The article concludes with both a few suggestions about possible alternatives and a challenge to psychological researchers to develop new methods that actually assess the reliability of research findings.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“We also checked whether the relationships we found were artifacts of between-person differences rather than within-person associations.”
From Context and craving during stressful events in the daily lives of drug-dependent patients (2017) · cites this paper as Branch 2014
In the citing paper
“Even so, when we examined the raw data person by person (Branch 2014), we found that the relationship was not perfect: of the 118 pts who used more than one categorical label for severity across occasions, only 49% gave numerical ratings that increased as would be expected with severity category, for 39%, numerical ratings were flat or fluctuated across categories, and for 12%, numerical ratings”
From Context and craving during stressful events in the daily lives of drug-dependent patients (2017) · cites this paper as Branch 2014
Citations
Cites
Gelman, A., Carlin, J. B., Stern, H. S., & Rubin, D. B. (2003) · doi:10.1201/9780429258480 · Cited by 27,276
David, F. N., & Tukey, J. W. (1977) · Biometrics · doi:10.2307/2529486 · Cited by 12,708
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · PLoS Medicine · doi:10.1371/journal.pmed.0020124 · Cited by 10,505
The Logic of Scientific Discovery
(2013) · A Realist Theory of Science · doi:10.4324/9780203090732-9 · Cited by 7,731
The Logic of Scientific Discovery.
Hutchison, T. W., & Popper, K. R. (1959) · Economica · doi:10.2307/2550489 · Cited by 7,499
Journal of The American Statistical Association
Fienberg, S. E. (2005) · Encyclopedia of Statistical Sciences · doi:10.1002/0471667196.ess1317 · Cited by 6,125
Why Most Published Research Findings Are False
Ioannidis, J. P. A. (2005) · CHANCE · doi:10.1080/09332480.2005.10722754 · Cited by 4,702 · cited 2×
“The lack of replicability of statistically significant effects is increasingly being recognized as a clear negative effect of NHSTs in medical research (Ioannidis, 2005).”
BackgroundThe Behavior of Organisms: An Experimental Analysis
Jones, F. N., & Skinner, B. F. (1939) · The American Journal of Psychology · doi:10.2307/1416495 · Cited by 4,398
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
Personality: A Psychological Interpretation
Jenkins, J. G., & Allport, G. W. (1938) · The American Journal of Psychology · doi:10.2307/1415717 · Cited by 3,214
Research works printed but not linked: 12
- Beaudette C. G., 2002, Excess heat: Why cold fusion research prevailed
- Berger J. O., 1988, American Scientist, 76, 159
- 10.1080/01621459.1942.10501760 · doi:10.1080/01621459.1942.10501760 · Paper page
- Cohen J., 1988, Statistical power analysis for the behavior sciences
- Committee on CMMP 2010, Solid State Sciences Committee, National Research Council, 2007, Condensed-matter and materials physics: The science of the world around us
- Ebbinghaus H., 1885, Über das Gedächtnis
- 10.1201/9780429258411 · doi:10.1201/9780429258411 · Paper page
- Haller S., 2002, Methods of Psychological Research, 7, 1
- Hays W. L., 1981, Statistics
- 10.1037/0003-066X.56.2.119 · doi:10.1037/0003-066x.56.2.119 · Paper page
- Oakes M., 1986, Statistical inference: A commentary for the social and behavioral sciences
- Piaget J., 1928, The child’s conception of the world
Cited by
What’s in a p? Reassessing best practices for conducting and reporting hypothesis-testing research
Meyer, K. E., van Witteloostuijn, A., & Beugelsdijk, S. (2017) · Journal of International Business Studies · doi:10.1057/s41267-017-0078-8 · Cited by 318
“The regression result does not prove or disprove a hypothesis, it does not provide evidence regarding the reliability of the research (Branch, 2014: 257), and it does not make statements about a population other than the sample.”
ResultThe earth is flat ( p > 0.05): significance thresholds and the crisis of unreplicable research
Amrhein, V., Korner-Nievergelt, F., & Roth, T. (2017) · PeerJ · doi:10.7717/peerj.3544 · Cited by 308
“This is one of the reasons why reliance on significance testing has been accused of systematically retarding the development of cumulative knowledge (Schmidt 1996; Branch 2014).”
BackgroundBettis, R. A., Ethiraj, S., Gambardella, A., Helfat, C., & Mitchell, W. (2016) · Strategic Management Journal · doi:10.1002/smj.2477 · Cited by 225
“P-values arising from NHTSs provide no information “regarding the reliability of the research” (Branch, 2014: 257, citing 13 other papers).”
Replicability Crisis in Social Psychology: Looking at the Past to Find New Pathways for the Future
Świątkowski, W., & Dompnier, B. (2017) · International Review of Social Psychology · doi:10.5334/irsp.66 · Cited by 114
“2 We hasten to make clear that this statistical tool is not bad per se, contrary to what might be thought based on the recent debates on the difficulties associated with p-values (e.g., Branch, 2014; Trafimow & Marks, 2015).”
Result60th Anniversary Essay: How Journals Could Improve Research Practices in Social Science
Starbuck, W. H. (2016) · Administrative Science Quarterly · doi:10.1177/0001839216629644 · Cited by 122
“Unfortunately, p-values based on sample data are unreliable evidence about the reproducibility of research findings because they vary quite a bit from sample to sample (Bedeian, Sturman, and Streiner, 2009; Cumming, 2013; Branch, 2014).”
Exacerbated craving in the presence of stress and drug cues in drug-dependent patients
Preston, K. L., Kowalczyk, W. J., Phillips, K. A., Jobes, M. L., Vahabzadeh, M., Lin, J.-L., Mezghanni, M., & Epstein, D. H. (2018) · Neuropsychopharmacology · doi:10.1038/npp.2017.275 · Cited by 84
Sex differences in daily life stress and craving in opioid-dependent patients
Moran, L. M., Kowalczyk, W. J., Phillips, K. A., Vahabzadeh, M., Lin, J.-L., Mezghanni, M., Epstein, D. H., & Preston, K. L. (2018) · The American Journal of Drug and Alcohol Abuse · doi:10.1080/00952990.2018.1454934 · Cited by 61
“We also examined how many participants conformed to the overall patterns of group-level sex differences to assess personlevel heterogeneity in the data (25).”
BackgroundContext and craving during stressful events in the daily lives of drug-dependent patients
Preston, K. L., Kowalczyk, W. J., Phillips, K. A., Jobes, M. L., Vahabzadeh, M., Lin, J.-L., Mezghanni, M., & Epstein, D. H. (2017) · Psychopharmacology · doi:10.1007/s00213-017-4663-0 · Cited by 61
“Even so, when we examined the raw data person by person (Branch 2014), we found that the relationship was not perfect: of the 118 pts who used more than one categorical label for 2634 Psychopharmacology (2017) 234:2631–2642”
BackgroundResultInfluentialPrediction of stress and drug craving ninety minutes in the future with passively collected GPS data
Epstein, D. H., Tyburski, M., Kowalczyk, W. J., Burgess-Hull, A. J., Phillips, K. A., Curtis, B. L., & Preston, K. L. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0234-6 · Cited by 61
Improving Psychological Science through Transparency and Openness: An Overview
Hales, A. H., Wesselmann, E. D., & Hilgard, J. (2018) · Perspectives on Behavior Science · doi:10.1007/s40614-018-00186-8 · Cited by 70
“Readers may be familiar with criticisms levied by Branch (2014), who correctly argued that a p value does not represent “an indication of the probability of replication nor of the probability that the results are due to chance” (p. 257).”
BackgroundAlligood, C. A., Dorey, N. R., Mehrkam, L. R., & Leighty, K. A. (2017) · Zoo Biology · doi:10.1002/zoo.21368 · Cited by 60
Randomization tests as alternative analysis methods for behavior‐analytic data
Craig, A. R., & Fisher, W. W. (2019) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.500 · Cited by 56
“…embracing statistical methods is a good thing or a bad thing for the science is up to debate (for contrasting views, interested readers may see, e.g., Branch, 1999; 2014; Hopkins, Cole, & Mason, 1998; Huitema, 1986; Kratochwill & Levin, 2014; Michael, 1974; Shadish, Hedges, & Pustejovsky, 2014).”
Medical aromatherapy revisited—Basic mechanisms, critique, and a new development
Schneider, R., Singer, N., & Singer, T. (2018) · Human Psychopharmacology Clinical and Experimental · doi:10.1002/hup.2683 · Cited by 59
“However, for more than 70 years, methodologists have warned that the use of significance tests is in many cases not only statistically and logically wrong, but potentially detrimental to the advancement of knowledge (for recent overviews, cf. Branch, 2014; Lambdin, 2012).”
BackgroundGilroy, S. P., & Kaplan, B. A. (2019) · Perspectives on Behavior Science · doi:10.1007/s40614-019-00202-5 · Cited by 52
“In addition, preregistering study parameters and methods may provide a further safeguard against QRPs more specific to SCRDs (e.g., criteria for dropping/excluding participants, criteria for advancing in treatment).”
BackgroundInfluentialThe “Reproducibility Crisis:” Might the Methods Used Frequently in Behavior-Analysis Research Help?
Branch, M. N. (2018) · Perspectives on Behavior Science · doi:10.1007/s40614-018-0158-5 · Cited by 45
“Examples of additional ways to direct the focus onto individuals and to emphasize replication can be found in Branch (2014), Loftus (1996), and Thompson (1993, 1994).”
BackgroundResultInfluentialSelf-citationTutorial: Small-N Power Analysis
Kyonka, E. G. E. (2018) · Perspectives on Behavior Science · doi:10.1007/s40614-018-0167-4 · Cited by 54
“Branch (1999, 2014) outlined the logical fallacy and malignant consequences of NHST for behavior analysts in at least two publications.”
BackgroundEquivalent statistics for a one-sample t-test
Francis, G., & Jakicic, V. (2022) · Behavior Research Methods · doi:10.3758/s13428-021-01775-3 · Cited by 55
“Recognition of these problems has highlighted long-running concerns about null hypothesis significance testing (Branch, 2014; Craig & Abramson, 2018; Gelman, 2017; Szucs & Ioannidis, 2017).”
BackgroundBehavioral Research with Planaria
Deochand, N., Costello, M. S., & Deochand, M. E. (2018) · Perspectives on Behavior Science · doi:10.1007/s40614-018-00176-w · Cited by 37
“Halas et al. (1962) had used null-hypothesis significance testing (NHST) to conclude differences between the experimental group and controls were not significant, despite the clear visual analysis of data (It should be note that NHST in psychology is deeply problematic [Branch, 2014]).”
MethodSchneider, S. M., & Sanguinetti, A. (2021) · Energy Research & Social Science · doi:10.1016/j.erss.2021.101958 · Cited by 38
New Guidelines for Null Hypothesis Significance Testing in Hypothetico-Deductive IS Research
Mertens, W., & Recker, J. (2023) · Technology, work and globalization · doi:10.1007/978-3-031-38719-7_13 · Cited by 36
Thou Shalt Not Bear False Witness Against Null Hypothesis Significance Testing
García-Pérez, M. Á. (2016) · Educational and Psychological Measurement · doi:10.1177/0013164416668232 · Cited by 33
“…itself, a methodology that is now purported to suffer from ‘‘problems’’ or ‘‘fatal flaws’’ and criticized for not allowing the type of inferences that researchers seek (e.g., Branch, 2014; Cumming, 2014; Gill, 1999; Goodman, 1999a; Krueger, 2001; Lambdin, 2012; Schneider, 2015; Wagenmakers, 2007).”
BackgroundTransplant Renal Vein Thrombosis.
Elzorkany, K. M. A., Bridson, J.-M., Sharma, A. K., & Halawa, A. M. (2017) · Experimental and Clinical Transplantation · doi:10.6002/ect.2016.0060 · Cited by 33
Killeen, P. R. (2018) · Perspectives on Behavior Science · doi:10.1007/s40614-018-0171-8 · Cited by 32
“It is not clear why traditional statistical inference continues when it is so inept and its “warts” so conspicuous (Krueger, 2001; Nickerson, 2000, 2015; Wagenmakers, 2007; Trafimow, 2003; Branch, 2014).”
BackgroundMoreira, J. G. D. V., & Naghettini, M. (2016) · Revista Brasileira de Meteorologia · doi:10.1590/0102-778631231420140155 · Cited by 33
Bayesian data analysis as a tool for behavior analysts
Young, M. E. (2019) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.512 · Cited by 30
Assessing Nonoverlap in Single-Case Data: Strengths, Challenges, and Recommendations
Manolov, R., & Tanious, R. (2024) · Journal of Behavioral Education · doi:10.1007/s10864-024-09552-w · Cited by 30
“…noted that relying on the judgment of the applied researchers, on the basis of the graphed data, the experimental manipulation, and their knowledge of the client, the issue dealt with and the target behavior, is inherent to SCED research and data analysis (Branch, 2014; Imam, 2021; Perone, 1999).”
InfluentialStatistical errors in software engineering experiments: a preliminary literature review
Ch., R. P. R., Dieste, Ó., Fonseca, E. R., & Juristo, N. (2018) · ACM/IEEE International Conference on Software Engineering: Companion Proceedings (ICSE-Companion) · doi:10.1145/3180155.3180161 · Cited by 26
“2 may look outdated due to the increasing criticisms of the null-hypothesis and significance testing (NHST), and the recommendations to adopt other statistical approaches such as confidence intervals and effect size indices [10, 71, 77].”
BackgroundZimmermann, Z. J., Watkins, E. E., & Poling, A. D. (2015) · The Behavior Analyst · doi:10.1007/s40614-015-0034-5 · Cited by 21
“As Branch (2014) has noted, however, it is crucial to recognize that inferential statistics do not provide a measure of reliability or replicability, as they are sometimes mistaken to do.”
BackgroundSchneider, R. (2015) · Stress and Health · doi:10.1002/smi.2636 · Cited by 23
“2Despite the widespread use of so called null hypothesis signifi- cance testing (NHST), methodologists have harshly criticized and protested its use since the 1930s (for a recent and thorough over- view, cf. Branch, 2014; Lambdin, 2012).”
Wilcox, R. R., & Serang, S. (2016) · Educational and Psychological Measurement · doi:10.1177/0013164416667983 · Cited by 28
Schneider, R. (2017) · Journal of Back and Musculoskeletal Rehabilitation · doi:10.3233/bmr-169662 · Cited by 20
“1 Specifically, mean comparisons were ana-1 Despite the widespread use of so called null hypothesis sig-nificance testing (NHST), methodologists have harshly criticized and protested its use since the 1930s (for a recent and thorough overview, cf. [25,26]).”
Bradley, M. T., & Brand, A. (2016) · Psychological Reports · doi:10.1177/0033294116662659 · Cited by 22
“Several authors (see a review Perezgonzales, 2014) state that researchers follow a “hybridization” of the approaches especially regarding significance levels with some aspects of inferential procedures being of limited value and other aspects being impediments to progress in science (Branch, 2014).”
BackgroundBeyond inference by eye: Statistical and graphing practices in JEAB, 1992‐2017
Kyonka, E. G. E., Mitchell, S. H., & Bizo, L. A. (2019) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.509 · Cited by 20
Elliffe, D. M., & Elliffe, M. (2019) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.502 · Cited by 19
Single‐case experimental designs for behavioral neuroscience
Soto, P. L. (2020) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.633 · Cited by 20
“A second benefit of SCEDS for behavioral neuroscientists is that use of SCEDs generates a body of knowledge regarding processes that apply to the individual organism rather than a population parameter (Branch, 2014; Branch & Pennypacker, 2013; Sidman, 1960).”
BackgroundQuantification of behavioral data with effect sizes and statistical significance tests
Costello, M. S., Bagley, R. F., Bustamante, L. F., & Deochand, N. (2022) · Journal of Applied Behavior Analysis · doi:10.1002/jaba.938 · Cited by 17
What’s in a p? Reassessing Best Practices for Conducting and Reporting Hypothesis-Testing Research
Meyer, K. E., van Witteloostuijn, A. C., & Beugelsdijk, S. (2019) · JIBS special collections · doi:10.1007/978-3-030-22113-3_4 · Cited by 24
Defining and assessing immediacy in single‐case experimental designs
Manolov, R., & Onghena, P. (2022) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.799 · Cited by 21
“In relation to any potential criticism directed towards p-values (e.g., Branch, 2014; J. Cohen, 1994), it should be noted that a p-value could be understood simply as the degree to which the observed difference is extreme, considering all possible differences (all potential moments of change in…”
BackgroundInterpreting functional analysis outcomes using automated nonparametric statistical analysis
Hall, S. S., Pollard, J. S., Monlux, K. D., & Baker, J. M. (2020) · Journal of Applied Behavior Analysis · doi:10.1002/jaba.689 · Cited by 17
Manolov, R., Tanious, R., & Onghena, P. (2021) · Perspectives on Behavior Science · doi:10.1007/s40614-021-00289-9 · Cited by 20
“…fact that p -values in the classical null hypothesis significance testing approach do not provide information about the replicability of an effect (Branch, 2014; Killeen, 2005), we consider that it is important to emphasize quantifications that emphasize the consistency of effects across…”
BackgroundTheory construction and data analysis
Tryon, W. W., Patelis, T., Chajewski, M., & Lewis, C. (2017) · Theory & Psychology · doi:10.1177/0959354316684043 · Cited by 17
“Considerable concern has recently been expressed about the replicability of psychological research (Branch, 2014; Ioannidis et al., 2014; Lambdin, 2012; Maxwell, Lau, & Howard, 2015).”
BackgroundImam, A. A. (2020) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.661 · Cited by 15
A proposal for the assessment of replication of effects in single‐case experimental designs
Manolov, R., Tanious, R., & Fernández‐Castilla, B. (2022) · Journal of Applied Behavior Analysis · doi:10.1002/jaba.923 · Cited by 15
“The need for expert judgment is not a limitation, given that expert judgment is necessary when using a benchmark outcome or mastery criteria (Branch, 2014; Hagopian, 2020; Imam, 2021; Kazdin, 1977; McDougale et al., 2020; Perone, 1999; Shepley et al., 2020).”
BackgroundStatistics Using Neural Networks in the Context of Sustainable Development Goal 9.5
Okulich-Kazarin, V. (2024) · Sustainability · doi:10.3390/su16198395 · Cited by 14
“…[26], higher education (Eastern Europe) [27], the banking sector (Indonesia and China) [28,29], psychology (Canada, Germany, and the USA) [30–32], the field of health care and sports (Great Britain, Spain, Kazakhstan, Indonesia, and Malaysia) [33–37], engineering sciences (China) [38],…”
Friedel, J. E., Cox, A. D., Galizio, A., Swisher, M., Small, M. L., & Perez, S. (2021) · Perspectives on Behavior Science · doi:10.1007/s40614-021-00318-7 · Cited by 10
Fox, A. E. (2018) · Behavior Analysis Research and Practice · doi:10.1037/bar0000106 · Cited by 11
Becker, A. M. (2023) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2023.1190412 · Cited by 14
“…methods carry limitations that have been treated thoroughly elsewhere [e.g., the generic problems with null-hypothesis statistical testing NHST (Branch, 2014), the inappropriate use of frequentist statistics contrary to their best use and design (Moen et al., 2016; Wasserstein and Lazar,…”
MethodCarlin, M. T., & Costello, M. S. (2021) · Perspectives on Behavior Science · doi:10.1007/s40614-021-00317-8 · Cited by 14
“, Branch, 1999, 2014). Skinner (1938) argued against the use of statistics in favor of experimental control with clearly replicable methods and results.”
BackgroundHow to assess and take into account trend in single-case experimental design data
Manolov, R., Lebrault, H., & Krasny‐Pacini, A. (2023) · Neuropsychological Rehabilitation · doi:10.1080/09602011.2023.2190129 · Cited by 11
“It also avoids relying on null hypothesis testing (e.g., the Mann-Kendall test), with its issues of statistical power (Tarlow, 2017), and with its conceptual drawbacks (Branch, 2014; Greenland et al., 2016; Perone, 1999).”
MethodSchneider, R. (2016) · Applied Cognitive Psychology · doi:10.1002/acp.3237 · Cited by 10
Exploring Physiological Linkage in Same-Sex Male Couples
Li, X., Kuelz, A., Boyd, S. M., August, K. J., Markey, C. N., & Butler, E. A. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.619255 · Cited by 10
“We preferred Bayesian to traditional Null-Hypothesis Significance Testing (NHST) for the following reasons: First, Bayesian analyses are less sensitive than NHST to sample size and will, therefore, generate more robust estimation for smallto-modest sized sample (as is the case with the present study in which n = 133 a…”
MethodLi, X., Curran, M. A., Butler, E. A., Toomey, R. B., Cao, H., & Fang, X. (2021) · Journal of Social and Personal Relationships · doi:10.1177/02654075211055236 · Cited by 11
“First, Bayesian analyses are less sensitive than NHST to sample size and will therefore generate more robust estimation (Branch, 2014).”
Lessons worth repeating: Sidman's Tactics of Scientific Research
Branch, M. N. (2020) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.643 · Cited by 9
Self-citationVisual Analysis of Single-Case Experimental Designs Data: Beyond Time-Series Graphs
Tanious, R., & Manolov, R. (2025) · Single Case in the Social Sciences · doi:10.15763/issn.1936-9298.2025.2.1.43-64 · Cited by 7
“These early proposals for statistical inference were often met with skepticism and the disregard for p -values has also been expressed more recently (Branch, 2014, 2019; Imam, 2021; Perone, 1999).”
Gillespie‐Lynch, K., Brooks, P. J., Shane‐Simpson, C., Gaggi, N. L., Sturm, D., & Ploog, B. O. (2015) · Advances in medical technologies and clinical practice book series · doi:10.4018/978-1-4666-8395-2.ch007 · Cited by 6
Multilevel Model Selection Applied to Single-Case Experimental Design Data
Manolov, R., & Moeyaert, M. (2025) · Journal of Behavioral Education · doi:10.1007/s10864-025-09593-9 · Cited by 6
“Finally, it is an example of the application of informed judgment, which is considered to be an essential part of behavioral research (Branch, 2014; Imam, 2021).”
Merlo, G., Chifari, A., Chiazzese, G., Denaro, P., Firrera, N., Savio, N. L., Patti, S., Palmegiano, L., Taibi, D., & Seta, L. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2022.943370 · Cited by 5
“Recently, a growing number of works have criticized the general usefulness of these clinical trials to discern the impact on the individual health of some specific treatments (Jacobson and Christensen, 1996; Westen and Bradley, 2005; Wachtel, 2010; Branch, 2014; Perone, 2019).”
BackgroundImam, A. A. (2018) · European Journal of Behavior Analysis · doi:10.1080/15021149.2018.1463123 · Cited by 5
“…psychologists because of the heavy reliance on dichotomous thinking inherent in NHST. Alternative ways of thinking about data and analyses (e.g., Branch, 2014; Fiedler, 2017; Killeen, 2005a, 2005b,; Sidman, 1960) may be ripe for consideration or reconsideration by the wider psychological…”
InfluentialManolov, R., & Onghena, P. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02230-1 · Cited by 5
“P values and null hypothesis statistical testing have been objects of controversy and criticism in the context of social and behavioral sciences (Cohen, 1990, 1994; Gigerenzer, 2004; Nickerson, 2000) and more specifically in the SCED context (Branch, 2014; Perone, 1999).”
BackgroundGillespie‐Lynch, K., Brooks, P. J., Shane‐Simpson, C., Gaggi, N. L., Sturm, D., & Ploog, B. O. (2016) · IGI Global eBooks · doi:10.4018/978-1-5225-0034-6.ch003 · Cited by 4
Recker, J. (2021) · Progress in IS · doi:10.1007/978-3-030-85436-2_5 · Cited by 5
Qin, Y. F., Sheremet, A., Cooper, T. L., Burke, S. N., & Maurer, A. P. (2023) · eNeuro · doi:10.1523/eneuro.0470-21.2023 · Cited by 5
“…this action in itself becomes an exercise in significance chasing by seeking the “N” that will confirm an a priori idea (Szucs, 2016), and may more generally speaks to the shortcomings of traditional null hypothesis significance testing (Krantz, 1999; Branch, 2014; Szucs and Ioannidis, 2017).”
BackgroundABkPowerCalculator: An App to Compute Power for Balanced (AB) k Single Case Experimental Designs
Batley, P. N., Thamaran, M., & Hedges, L. V. (2023) · Multivariate Behavioral Research · doi:10.1080/00273171.2023.2261229 · Cited by 4
“Sometimes finding statistical significance in SCEDs is purposely avoided as well (Branch, 2014, 2019; Perone, 1999).”
BackgroundImam, A. A. (2022) · BMC Psychology · doi:10.1186/s40359-022-00982-7 · Cited by 3
“The issues and problems introduced by the wholesale embrace of NHST in psychology seem not to have been necessary for a productive scientific endeavor to create a cumulative science (see [39, 107]) prior to the coupling of inferential statistics and research in psychology.”
BackgroundHow to Start Analyzing, Test Assumptions and Deal with that Pesky p-Value
Mertens, W., Pugliese, A., & Recker, J. (2016) · Springer eBooks · doi:10.1007/978-3-319-42700-3_8 · Cited by 2
Comparing masked and traditional visual analysis of multiple‐baseline designs
Wolfe, K., Dowdy, A., Ferron, J. M., & Li, R. (2025) · Journal of Applied Behavior Analysis · doi:10.1002/jaba.70043 · Cited by 1
“Although historically many single-case researchers have been opposed to null hypothesis significance testing for several valid reasons (e.g., Branch, 2014; Michael, 1974), 75.00% of respondents agreed that the goal of MVA — to control the probability of Type I error and produce a p value — was…”
InfluentialManolov, R. (2026) · Behavior Research Methods · doi:10.3758/s13428-026-02951-z · Cited by 1
“Instead, it is likely to require the researchers’ judgment (Branch, 2014; Hagopian, 2020; Imam, 2021) based on their knowledge of the participant and the issue studied.”
El continuo de la psicosis: Evidencia psicométrica
Tamayo-Agudelo, W., Cruz, R. Z., Bell, V., & Holguín-Lew, J. C. (2016) · Revista de Psicopatología y Psicología Clínica · doi:10.5944/rppc.vol.21.num.3.2016.15711 · Cited by 2
Schneider, R. (2015) · Neuroscience and Biomedical Engineering · doi:10.2174/2213385203666150930001154 · Cited by 1
Schneider, R. (2018) · The Natural Products Journal · doi:10.2174/2210315508666180919144725 · Cited by 2
Statistical Controversies in Psychological Science
Hales, A. H., & Wood, N. R. (2022) · Springer eBooks · doi:10.1007/978-3-031-04968-2_9 · Cited by 2
Replication study design: confidence intervals and commentary
Kupper, L. L., & Martin, S. L. (2022) · Statistical Papers · doi:10.1007/s00362-022-01291-2
“Currently, the large majority of published research studies involve the use of confidence intervals, and not just P-values, for making statistical inferences [e.g., see Branch (2014), Cumming (2008), Lash (2017), and Wasserman and Lazar (2016) for justifications].”
BackgroundThe requisite boring stuff part I: Defining a statistic and the benefit of numbers
Cox, D. J., & Vladescu, J. C. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-99885-7.00007-6 · Cited by 1
The requisite boring stuff part II: Data types and data distributions
Cox, D. J., & Vladescu, J. C. (2023) · Elsevier eBooks · doi:10.1016/b978-0-323-99885-7.00010-6
Okulicz-Kozaryn, W. (2026) · Sci · doi:10.3390/sci8010024 · Cited by 3
“At the turn of 2023–2025, Generative AI gradually transformed from an individual automation tool into a cognitive assistant capable of participating in the creation of texts [6], data collection and visualization [5,11], hypothesis development and analysis of statistical information [12–15].”
Work-to-rest ratios in repeated sprint assessments: A replication study
Jutlah, L. B. L., Stevens, K. A., Meelu, A., Cohen, J. N., Murphy, J., Warne, J. P., & Au, J. S. (2026) · Sports Medicine International Open · doi:10.1055/a-2940-7047
Metadata
- Authors
- Branch, M.
- Affiliations
- University of Florida (US)
- Corresponding author
- Branch, Marc
- ORCID
- Branch, Marc
- Venue
- Theory & Psychology
- Publisher
- SAGE Publications
- Published
- 2014-04
- Volume
- 24
- Issue
- 2
- Pages
- 256-277
- ISSN
- 0959-3543 (ISSN-L)
- 1461-7447
- OpenAlex
- W2335213174
- Cited by
- 81
- References
- 89
- Citations by year
- 2026: 3 · 2025: 3 · 2024: 2 · 2023: 9 · 2022: 7 · 2021: 7 · 2020: 5 · 2019: 7 · 2018: 12 · 2017: 12 · 2016: 9 · 2015: 5
- FWCI
- 12.05
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Theory & Psychology (publishedVersion)
- Funders
- U.S. Public Health Service
- National Institute on Drug Abuse
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods and Bayesian Inference
- Keywords
- null hypothesis significance testing, p-values, editorial decision-making, research methodology critique
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.